Jul 05 00:07:13.417876 ip-10-0-142-202 systemd[1]: kubelet.service: Failed to load environment files: No such file or directory
Jul 05 00:07:13.418078 ip-10-0-142-202 systemd[1]: kubelet.service: Failed to run 'start-pre' task: No such file or directory
Jul 05 00:07:13.418283 ip-10-0-142-202 systemd[1]: kubelet.service: Failed with result 'resources'.
Jul 05 00:07:13.418869 ip-10-0-142-202 systemd[1]: Failed to start Kubernetes Kubelet.
Jul 05 00:07:23.638225 ip-10-0-142-202 systemd[1]: kubelet.service: Failed to schedule restart job: Unit crio.service not found.
Jul 05 00:07:23.638252 ip-10-0-142-202 systemd[1]: kubelet.service: Failed with result 'resources'.
-- Boot 300820185d664c3e869d25b387dc3e53 --
Jul 05 00:09:11.543491 ip-10-0-142-202 systemd[1]: Starting Kubernetes Kubelet...
Jul 05 00:09:11.929940 ip-10-0-142-202 kubenswrapper[2563]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Jul 05 00:09:11.929940 ip-10-0-142-202 kubenswrapper[2563]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version.
Jul 05 00:09:11.929940 ip-10-0-142-202 kubenswrapper[2563]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Jul 05 00:09:11.929940 ip-10-0-142-202 kubenswrapper[2563]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI.
Jul 05 00:09:11.929940 ip-10-0-142-202 kubenswrapper[2563]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Jul 05 00:09:11.930602 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:09:11.929998    2563 server.go:213] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime"
Jul 05 00:09:11.935658 ip-10-0-142-202 kubenswrapper[2563]: W0705 00:09:11.935641    2563 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 05 00:09:11.935658 ip-10-0-142-202 kubenswrapper[2563]: W0705 00:09:11.935656    2563 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 05 00:09:11.935658 ip-10-0-142-202 kubenswrapper[2563]: W0705 00:09:11.935660    2563 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 05 00:09:11.935751 ip-10-0-142-202 kubenswrapper[2563]: W0705 00:09:11.935663    2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 05 00:09:11.935751 ip-10-0-142-202 kubenswrapper[2563]: W0705 00:09:11.935666    2563 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 05 00:09:11.935751 ip-10-0-142-202 kubenswrapper[2563]: W0705 00:09:11.935669    2563 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 05 00:09:11.935751 ip-10-0-142-202 kubenswrapper[2563]: W0705 00:09:11.935672    2563 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 05 00:09:11.935751 ip-10-0-142-202 kubenswrapper[2563]: W0705 00:09:11.935675    2563 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 05 00:09:11.935751 ip-10-0-142-202 kubenswrapper[2563]: W0705 00:09:11.935678    2563 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 05 00:09:11.935751 ip-10-0-142-202 kubenswrapper[2563]: W0705 00:09:11.935681    2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 05 00:09:11.935751 ip-10-0-142-202 kubenswrapper[2563]: W0705 00:09:11.935683    2563 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 05 00:09:11.935751 ip-10-0-142-202 kubenswrapper[2563]: W0705 00:09:11.935686    2563 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 05 00:09:11.935751 ip-10-0-142-202 kubenswrapper[2563]: W0705 00:09:11.935689    2563 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 05 00:09:11.935751 ip-10-0-142-202 kubenswrapper[2563]: W0705 00:09:11.935691    2563 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 05 00:09:11.935751 ip-10-0-142-202 kubenswrapper[2563]: W0705 00:09:11.935697    2563 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 05 00:09:11.935751 ip-10-0-142-202 kubenswrapper[2563]: W0705 00:09:11.935701    2563 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 05 00:09:11.935751 ip-10-0-142-202 kubenswrapper[2563]: W0705 00:09:11.935704    2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 05 00:09:11.935751 ip-10-0-142-202 kubenswrapper[2563]: W0705 00:09:11.935711    2563 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 05 00:09:11.935751 ip-10-0-142-202 kubenswrapper[2563]: W0705 00:09:11.935714    2563 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 05 00:09:11.935751 ip-10-0-142-202 kubenswrapper[2563]: W0705 00:09:11.935717    2563 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 05 00:09:11.935751 ip-10-0-142-202 kubenswrapper[2563]: W0705 00:09:11.935719    2563 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 05 00:09:11.935751 ip-10-0-142-202 kubenswrapper[2563]: W0705 00:09:11.935722    2563 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 05 00:09:11.936204 ip-10-0-142-202 kubenswrapper[2563]: W0705 00:09:11.935725    2563 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 05 00:09:11.936204 ip-10-0-142-202 kubenswrapper[2563]: W0705 00:09:11.935727    2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 05 00:09:11.936204 ip-10-0-142-202 kubenswrapper[2563]: W0705 00:09:11.935730    2563 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 05 00:09:11.936204 ip-10-0-142-202 kubenswrapper[2563]: W0705 00:09:11.935732    2563 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 05 00:09:11.936204 ip-10-0-142-202 kubenswrapper[2563]: W0705 00:09:11.935735    2563 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 05 00:09:11.936204 ip-10-0-142-202 kubenswrapper[2563]: W0705 00:09:11.935745    2563 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 05 00:09:11.936204 ip-10-0-142-202 kubenswrapper[2563]: W0705 00:09:11.935748    2563 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 05 00:09:11.936204 ip-10-0-142-202 kubenswrapper[2563]: W0705 00:09:11.935751    2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 05 00:09:11.936204 ip-10-0-142-202 kubenswrapper[2563]: W0705 00:09:11.935754    2563 feature_gate.go:328] unrecognized feature gate: Example2
Jul 05 00:09:11.936204 ip-10-0-142-202 kubenswrapper[2563]: W0705 00:09:11.935757    2563 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 05 00:09:11.936204 ip-10-0-142-202 kubenswrapper[2563]: W0705 00:09:11.935760    2563 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 05 00:09:11.936204 ip-10-0-142-202 kubenswrapper[2563]: W0705 00:09:11.935763    2563 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 05 00:09:11.936204 ip-10-0-142-202 kubenswrapper[2563]: W0705 00:09:11.935766    2563 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 05 00:09:11.936204 ip-10-0-142-202 kubenswrapper[2563]: W0705 00:09:11.935769    2563 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 05 00:09:11.936204 ip-10-0-142-202 kubenswrapper[2563]: W0705 00:09:11.935772    2563 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 05 00:09:11.936204 ip-10-0-142-202 kubenswrapper[2563]: W0705 00:09:11.935774    2563 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 05 00:09:11.936204 ip-10-0-142-202 kubenswrapper[2563]: W0705 00:09:11.935777    2563 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 05 00:09:11.936204 ip-10-0-142-202 kubenswrapper[2563]: W0705 00:09:11.935779    2563 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 05 00:09:11.936204 ip-10-0-142-202 kubenswrapper[2563]: W0705 00:09:11.935782    2563 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 05 00:09:11.936204 ip-10-0-142-202 kubenswrapper[2563]: W0705 00:09:11.935784    2563 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 05 00:09:11.936688 ip-10-0-142-202 kubenswrapper[2563]: W0705 00:09:11.935787    2563 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 05 00:09:11.936688 ip-10-0-142-202 kubenswrapper[2563]: W0705 00:09:11.935790    2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 05 00:09:11.936688 ip-10-0-142-202 kubenswrapper[2563]: W0705 00:09:11.935793    2563 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 05 00:09:11.936688 ip-10-0-142-202 kubenswrapper[2563]: W0705 00:09:11.935797    2563 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 05 00:09:11.936688 ip-10-0-142-202 kubenswrapper[2563]: W0705 00:09:11.935800    2563 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 05 00:09:11.936688 ip-10-0-142-202 kubenswrapper[2563]: W0705 00:09:11.935804    2563 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 05 00:09:11.936688 ip-10-0-142-202 kubenswrapper[2563]: W0705 00:09:11.935807    2563 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 05 00:09:11.936688 ip-10-0-142-202 kubenswrapper[2563]: W0705 00:09:11.935809    2563 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 05 00:09:11.936688 ip-10-0-142-202 kubenswrapper[2563]: W0705 00:09:11.935812    2563 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 05 00:09:11.936688 ip-10-0-142-202 kubenswrapper[2563]: W0705 00:09:11.935814    2563 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 05 00:09:11.936688 ip-10-0-142-202 kubenswrapper[2563]: W0705 00:09:11.935817    2563 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 05 00:09:11.936688 ip-10-0-142-202 kubenswrapper[2563]: W0705 00:09:11.935819    2563 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 05 00:09:11.936688 ip-10-0-142-202 kubenswrapper[2563]: W0705 00:09:11.935822    2563 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 05 00:09:11.936688 ip-10-0-142-202 kubenswrapper[2563]: W0705 00:09:11.935825    2563 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 05 00:09:11.936688 ip-10-0-142-202 kubenswrapper[2563]: W0705 00:09:11.935828    2563 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 05 00:09:11.936688 ip-10-0-142-202 kubenswrapper[2563]: W0705 00:09:11.935830    2563 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 05 00:09:11.936688 ip-10-0-142-202 kubenswrapper[2563]: W0705 00:09:11.935833    2563 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 05 00:09:11.936688 ip-10-0-142-202 kubenswrapper[2563]: W0705 00:09:11.935836    2563 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 05 00:09:11.936688 ip-10-0-142-202 kubenswrapper[2563]: W0705 00:09:11.935838    2563 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 05 00:09:11.937157 ip-10-0-142-202 kubenswrapper[2563]: W0705 00:09:11.935862    2563 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 05 00:09:11.937157 ip-10-0-142-202 kubenswrapper[2563]: W0705 00:09:11.935867    2563 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 05 00:09:11.937157 ip-10-0-142-202 kubenswrapper[2563]: W0705 00:09:11.935870    2563 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 05 00:09:11.937157 ip-10-0-142-202 kubenswrapper[2563]: W0705 00:09:11.935873    2563 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 05 00:09:11.937157 ip-10-0-142-202 kubenswrapper[2563]: W0705 00:09:11.935876    2563 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 05 00:09:11.937157 ip-10-0-142-202 kubenswrapper[2563]: W0705 00:09:11.935879    2563 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 05 00:09:11.937157 ip-10-0-142-202 kubenswrapper[2563]: W0705 00:09:11.935881    2563 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 05 00:09:11.937157 ip-10-0-142-202 kubenswrapper[2563]: W0705 00:09:11.935884    2563 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 05 00:09:11.937157 ip-10-0-142-202 kubenswrapper[2563]: W0705 00:09:11.935886    2563 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 05 00:09:11.937157 ip-10-0-142-202 kubenswrapper[2563]: W0705 00:09:11.935889    2563 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 05 00:09:11.937157 ip-10-0-142-202 kubenswrapper[2563]: W0705 00:09:11.935891    2563 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 05 00:09:11.937157 ip-10-0-142-202 kubenswrapper[2563]: W0705 00:09:11.935894    2563 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 05 00:09:11.937157 ip-10-0-142-202 kubenswrapper[2563]: W0705 00:09:11.935897    2563 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 05 00:09:11.937157 ip-10-0-142-202 kubenswrapper[2563]: W0705 00:09:11.935899    2563 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 05 00:09:11.937157 ip-10-0-142-202 kubenswrapper[2563]: W0705 00:09:11.935902    2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 05 00:09:11.937157 ip-10-0-142-202 kubenswrapper[2563]: W0705 00:09:11.935906    2563 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 05 00:09:11.937157 ip-10-0-142-202 kubenswrapper[2563]: W0705 00:09:11.935909    2563 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 05 00:09:11.937157 ip-10-0-142-202 kubenswrapper[2563]: W0705 00:09:11.935912    2563 feature_gate.go:328] unrecognized feature gate: Example
Jul 05 00:09:11.937157 ip-10-0-142-202 kubenswrapper[2563]: W0705 00:09:11.935914    2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 05 00:09:11.937157 ip-10-0-142-202 kubenswrapper[2563]: W0705 00:09:11.935918    2563 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 05 00:09:11.937638 ip-10-0-142-202 kubenswrapper[2563]: W0705 00:09:11.935920    2563 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 05 00:09:11.937638 ip-10-0-142-202 kubenswrapper[2563]: W0705 00:09:11.935923    2563 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 05 00:09:11.937638 ip-10-0-142-202 kubenswrapper[2563]: W0705 00:09:11.935925    2563 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 05 00:09:11.937638 ip-10-0-142-202 kubenswrapper[2563]: W0705 00:09:11.935928    2563 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 05 00:09:11.937638 ip-10-0-142-202 kubenswrapper[2563]: W0705 00:09:11.935930    2563 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 05 00:09:11.937638 ip-10-0-142-202 kubenswrapper[2563]: W0705 00:09:11.935933    2563 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 05 00:09:11.937638 ip-10-0-142-202 kubenswrapper[2563]: W0705 00:09:11.935936    2563 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 05 00:09:11.937638 ip-10-0-142-202 kubenswrapper[2563]: W0705 00:09:11.935939    2563 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 05 00:09:11.937638 ip-10-0-142-202 kubenswrapper[2563]: W0705 00:09:11.935944    2563 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 05 00:09:11.937638 ip-10-0-142-202 kubenswrapper[2563]: W0705 00:09:11.935946    2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 05 00:09:11.937638 ip-10-0-142-202 kubenswrapper[2563]: W0705 00:09:11.935949    2563 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 05 00:09:11.937638 ip-10-0-142-202 kubenswrapper[2563]: W0705 00:09:11.935952    2563 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 05 00:09:11.937638 ip-10-0-142-202 kubenswrapper[2563]: W0705 00:09:11.935955    2563 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 05 00:09:11.937638 ip-10-0-142-202 kubenswrapper[2563]: W0705 00:09:11.935958    2563 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 05 00:09:11.937638 ip-10-0-142-202 kubenswrapper[2563]: W0705 00:09:11.935960    2563 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 05 00:09:11.937638 ip-10-0-142-202 kubenswrapper[2563]: W0705 00:09:11.935963    2563 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 05 00:09:11.937638 ip-10-0-142-202 kubenswrapper[2563]: W0705 00:09:11.935965    2563 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 05 00:09:11.937638 ip-10-0-142-202 kubenswrapper[2563]: W0705 00:09:11.935968    2563 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 05 00:09:11.937638 ip-10-0-142-202 kubenswrapper[2563]: W0705 00:09:11.936671    2563 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 05 00:09:11.937638 ip-10-0-142-202 kubenswrapper[2563]: W0705 00:09:11.936680    2563 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 05 00:09:11.938158 ip-10-0-142-202 kubenswrapper[2563]: W0705 00:09:11.936685    2563 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 05 00:09:11.938158 ip-10-0-142-202 kubenswrapper[2563]: W0705 00:09:11.936688    2563 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 05 00:09:11.938158 ip-10-0-142-202 kubenswrapper[2563]: W0705 00:09:11.936691    2563 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 05 00:09:11.938158 ip-10-0-142-202 kubenswrapper[2563]: W0705 00:09:11.936694    2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 05 00:09:11.938158 ip-10-0-142-202 kubenswrapper[2563]: W0705 00:09:11.936697    2563 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 05 00:09:11.938158 ip-10-0-142-202 kubenswrapper[2563]: W0705 00:09:11.936700    2563 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 05 00:09:11.938158 ip-10-0-142-202 kubenswrapper[2563]: W0705 00:09:11.936703    2563 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 05 00:09:11.938158 ip-10-0-142-202 kubenswrapper[2563]: W0705 00:09:11.936706    2563 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 05 00:09:11.938158 ip-10-0-142-202 kubenswrapper[2563]: W0705 00:09:11.936708    2563 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 05 00:09:11.938158 ip-10-0-142-202 kubenswrapper[2563]: W0705 00:09:11.936711    2563 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 05 00:09:11.938158 ip-10-0-142-202 kubenswrapper[2563]: W0705 00:09:11.936713    2563 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 05 00:09:11.938158 ip-10-0-142-202 kubenswrapper[2563]: W0705 00:09:11.936716    2563 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 05 00:09:11.938158 ip-10-0-142-202 kubenswrapper[2563]: W0705 00:09:11.936719    2563 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 05 00:09:11.938158 ip-10-0-142-202 kubenswrapper[2563]: W0705 00:09:11.936722    2563 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 05 00:09:11.938158 ip-10-0-142-202 kubenswrapper[2563]: W0705 00:09:11.936724    2563 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 05 00:09:11.938158 ip-10-0-142-202 kubenswrapper[2563]: W0705 00:09:11.936727    2563 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 05 00:09:11.938158 ip-10-0-142-202 kubenswrapper[2563]: W0705 00:09:11.936729    2563 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 05 00:09:11.938158 ip-10-0-142-202 kubenswrapper[2563]: W0705 00:09:11.936732    2563 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 05 00:09:11.938158 ip-10-0-142-202 kubenswrapper[2563]: W0705 00:09:11.936737    2563 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 05 00:09:11.938673 ip-10-0-142-202 kubenswrapper[2563]: W0705 00:09:11.936740    2563 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 05 00:09:11.938673 ip-10-0-142-202 kubenswrapper[2563]: W0705 00:09:11.936742    2563 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 05 00:09:11.938673 ip-10-0-142-202 kubenswrapper[2563]: W0705 00:09:11.936745    2563 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 05 00:09:11.938673 ip-10-0-142-202 kubenswrapper[2563]: W0705 00:09:11.936748    2563 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 05 00:09:11.938673 ip-10-0-142-202 kubenswrapper[2563]: W0705 00:09:11.936750    2563 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 05 00:09:11.938673 ip-10-0-142-202 kubenswrapper[2563]: W0705 00:09:11.936753    2563 feature_gate.go:328] unrecognized feature gate: Example2
Jul 05 00:09:11.938673 ip-10-0-142-202 kubenswrapper[2563]: W0705 00:09:11.936758    2563 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 05 00:09:11.938673 ip-10-0-142-202 kubenswrapper[2563]: W0705 00:09:11.936761    2563 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 05 00:09:11.938673 ip-10-0-142-202 kubenswrapper[2563]: W0705 00:09:11.936763    2563 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 05 00:09:11.938673 ip-10-0-142-202 kubenswrapper[2563]: W0705 00:09:11.936766    2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 05 00:09:11.938673 ip-10-0-142-202 kubenswrapper[2563]: W0705 00:09:11.936769    2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 05 00:09:11.938673 ip-10-0-142-202 kubenswrapper[2563]: W0705 00:09:11.936771    2563 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 05 00:09:11.938673 ip-10-0-142-202 kubenswrapper[2563]: W0705 00:09:11.936774    2563 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 05 00:09:11.938673 ip-10-0-142-202 kubenswrapper[2563]: W0705 00:09:11.936777    2563 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 05 00:09:11.938673 ip-10-0-142-202 kubenswrapper[2563]: W0705 00:09:11.936779    2563 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 05 00:09:11.938673 ip-10-0-142-202 kubenswrapper[2563]: W0705 00:09:11.936782    2563 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 05 00:09:11.938673 ip-10-0-142-202 kubenswrapper[2563]: W0705 00:09:11.936784    2563 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 05 00:09:11.938673 ip-10-0-142-202 kubenswrapper[2563]: W0705 00:09:11.936787    2563 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 05 00:09:11.938673 ip-10-0-142-202 kubenswrapper[2563]: W0705 00:09:11.936790    2563 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 05 00:09:11.939125 ip-10-0-142-202 kubenswrapper[2563]: W0705 00:09:11.936792    2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 05 00:09:11.939125 ip-10-0-142-202 kubenswrapper[2563]: W0705 00:09:11.936795    2563 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 05 00:09:11.939125 ip-10-0-142-202 kubenswrapper[2563]: W0705 00:09:11.936797    2563 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 05 00:09:11.939125 ip-10-0-142-202 kubenswrapper[2563]: W0705 00:09:11.936799    2563 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 05 00:09:11.939125 ip-10-0-142-202 kubenswrapper[2563]: W0705 00:09:11.936802    2563 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 05 00:09:11.939125 ip-10-0-142-202 kubenswrapper[2563]: W0705 00:09:11.936805    2563 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 05 00:09:11.939125 ip-10-0-142-202 kubenswrapper[2563]: W0705 00:09:11.936808    2563 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 05 00:09:11.939125 ip-10-0-142-202 kubenswrapper[2563]: W0705 00:09:11.936811    2563 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 05 00:09:11.939125 ip-10-0-142-202 kubenswrapper[2563]: W0705 00:09:11.936814    2563 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 05 00:09:11.939125 ip-10-0-142-202 kubenswrapper[2563]: W0705 00:09:11.936817    2563 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 05 00:09:11.939125 ip-10-0-142-202 kubenswrapper[2563]: W0705 00:09:11.936819    2563 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 05 00:09:11.939125 ip-10-0-142-202 kubenswrapper[2563]: W0705 00:09:11.936822    2563 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 05 00:09:11.939125 ip-10-0-142-202 kubenswrapper[2563]: W0705 00:09:11.936824    2563 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 05 00:09:11.939125 ip-10-0-142-202 kubenswrapper[2563]: W0705 00:09:11.936827    2563 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 05 00:09:11.939125 ip-10-0-142-202 kubenswrapper[2563]: W0705 00:09:11.936830    2563 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 05 00:09:11.939125 ip-10-0-142-202 kubenswrapper[2563]: W0705 00:09:11.936832    2563 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 05 00:09:11.939125 ip-10-0-142-202 kubenswrapper[2563]: W0705 00:09:11.936835    2563 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 05 00:09:11.939125 ip-10-0-142-202 kubenswrapper[2563]: W0705 00:09:11.936837    2563 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 05 00:09:11.939125 ip-10-0-142-202 kubenswrapper[2563]: W0705 00:09:11.936840    2563 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 05 00:09:11.939125 ip-10-0-142-202 kubenswrapper[2563]: W0705 00:09:11.936842    2563 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 05 00:09:11.939624 ip-10-0-142-202 kubenswrapper[2563]: W0705 00:09:11.936845    2563 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 05 00:09:11.939624 ip-10-0-142-202 kubenswrapper[2563]: W0705 00:09:11.936847    2563 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 05 00:09:11.939624 ip-10-0-142-202 kubenswrapper[2563]: W0705 00:09:11.936850    2563 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 05 00:09:11.939624 ip-10-0-142-202 kubenswrapper[2563]: W0705 00:09:11.936852    2563 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 05 00:09:11.939624 ip-10-0-142-202 kubenswrapper[2563]: W0705 00:09:11.936855    2563 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 05 00:09:11.939624 ip-10-0-142-202 kubenswrapper[2563]: W0705 00:09:11.936857    2563 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 05 00:09:11.939624 ip-10-0-142-202 kubenswrapper[2563]: W0705 00:09:11.936860    2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 05 00:09:11.939624 ip-10-0-142-202 kubenswrapper[2563]: W0705 00:09:11.936863    2563 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 05 00:09:11.939624 ip-10-0-142-202 kubenswrapper[2563]: W0705 00:09:11.936865    2563 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 05 00:09:11.939624 ip-10-0-142-202 kubenswrapper[2563]: W0705 00:09:11.936868    2563 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 05 00:09:11.939624 ip-10-0-142-202 kubenswrapper[2563]: W0705 00:09:11.936870    2563 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 05 00:09:11.939624 ip-10-0-142-202 kubenswrapper[2563]: W0705 00:09:11.936872    2563 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 05 00:09:11.939624 ip-10-0-142-202 kubenswrapper[2563]: W0705 00:09:11.936875    2563 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 05 00:09:11.939624 ip-10-0-142-202 kubenswrapper[2563]: W0705 00:09:11.936877    2563 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 05 00:09:11.939624 ip-10-0-142-202 kubenswrapper[2563]: W0705 00:09:11.936880    2563 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 05 00:09:11.939624 ip-10-0-142-202 kubenswrapper[2563]: W0705 00:09:11.936882    2563 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 05 00:09:11.939624 ip-10-0-142-202 kubenswrapper[2563]: W0705 00:09:11.936885    2563 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 05 00:09:11.939624 ip-10-0-142-202 kubenswrapper[2563]: W0705 00:09:11.936887    2563 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 05 00:09:11.939624 ip-10-0-142-202 kubenswrapper[2563]: W0705 00:09:11.936890    2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 05 00:09:11.939624 ip-10-0-142-202 kubenswrapper[2563]: W0705 00:09:11.936892    2563 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 05 00:09:11.940112 ip-10-0-142-202 kubenswrapper[2563]: W0705 00:09:11.936896    2563 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 05 00:09:11.940112 ip-10-0-142-202 kubenswrapper[2563]: W0705 00:09:11.936898    2563 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 05 00:09:11.940112 ip-10-0-142-202 kubenswrapper[2563]: W0705 00:09:11.936901    2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 05 00:09:11.940112 ip-10-0-142-202 kubenswrapper[2563]: W0705 00:09:11.936904    2563 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 05 00:09:11.940112 ip-10-0-142-202 kubenswrapper[2563]: W0705 00:09:11.936907    2563 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 05 00:09:11.940112 ip-10-0-142-202 kubenswrapper[2563]: W0705 00:09:11.936910    2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 05 00:09:11.940112 ip-10-0-142-202 kubenswrapper[2563]: W0705 00:09:11.936912    2563 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 05 00:09:11.940112 ip-10-0-142-202 kubenswrapper[2563]: W0705 00:09:11.936915    2563 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 05 00:09:11.940112 ip-10-0-142-202 kubenswrapper[2563]: W0705 00:09:11.936917    2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 05 00:09:11.940112 ip-10-0-142-202 kubenswrapper[2563]: W0705 00:09:11.936919    2563 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 05 00:09:11.940112 ip-10-0-142-202 kubenswrapper[2563]: W0705 00:09:11.936922    2563 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 05 00:09:11.940112 ip-10-0-142-202 kubenswrapper[2563]: W0705 00:09:11.936925    2563 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 05 00:09:11.940112 ip-10-0-142-202 kubenswrapper[2563]: W0705 00:09:11.936927    2563 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 05 00:09:11.940112 ip-10-0-142-202 kubenswrapper[2563]: W0705 00:09:11.936929    2563 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 05 00:09:11.940112 ip-10-0-142-202 kubenswrapper[2563]: W0705 00:09:11.936932    2563 feature_gate.go:328] unrecognized feature gate: Example
Jul 05 00:09:11.940112 ip-10-0-142-202 kubenswrapper[2563]: W0705 00:09:11.936935    2563 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 05 00:09:11.940112 ip-10-0-142-202 kubenswrapper[2563]: W0705 00:09:11.936937    2563 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 05 00:09:11.940112 ip-10-0-142-202 kubenswrapper[2563]: W0705 00:09:11.936940    2563 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 05 00:09:11.940112 ip-10-0-142-202 kubenswrapper[2563]: W0705 00:09:11.936943    2563 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 05 00:09:11.940587 ip-10-0-142-202 kubenswrapper[2563]: W0705 00:09:11.937413    2563 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 05 00:09:11.940587 ip-10-0-142-202 kubenswrapper[2563]: W0705 00:09:11.937419    2563 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 05 00:09:11.940587 ip-10-0-142-202 kubenswrapper[2563]: W0705 00:09:11.937422    2563 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 05 00:09:11.940587 ip-10-0-142-202 kubenswrapper[2563]: W0705 00:09:11.937425    2563 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 05 00:09:11.940587 ip-10-0-142-202 kubenswrapper[2563]: W0705 00:09:11.937428    2563 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 05 00:09:11.940587 ip-10-0-142-202 kubenswrapper[2563]: W0705 00:09:11.937431    2563 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 05 00:09:11.940587 ip-10-0-142-202 kubenswrapper[2563]: W0705 00:09:11.937433    2563 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 05 00:09:11.940587 ip-10-0-142-202 kubenswrapper[2563]: W0705 00:09:11.937437    2563 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 05 00:09:11.940587 ip-10-0-142-202 kubenswrapper[2563]: W0705 00:09:11.937440    2563 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 05 00:09:11.940587 ip-10-0-142-202 kubenswrapper[2563]: W0705 00:09:11.937443    2563 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 05 00:09:11.940587 ip-10-0-142-202 kubenswrapper[2563]: W0705 00:09:11.937445    2563 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 05 00:09:11.940587 ip-10-0-142-202 kubenswrapper[2563]: W0705 00:09:11.937448    2563 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 05 00:09:11.940587 ip-10-0-142-202 kubenswrapper[2563]: W0705 00:09:11.937450    2563 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 05 00:09:11.940587 ip-10-0-142-202 kubenswrapper[2563]: W0705 00:09:11.937459    2563 feature_gate.go:328] unrecognized feature gate: Example
Jul 05 00:09:11.940587 ip-10-0-142-202 kubenswrapper[2563]: W0705 00:09:11.937462    2563 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 05 00:09:11.940587 ip-10-0-142-202 kubenswrapper[2563]: W0705 00:09:11.937464    2563 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 05 00:09:11.940587 ip-10-0-142-202 kubenswrapper[2563]: W0705 00:09:11.937467    2563 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 05 00:09:11.940587 ip-10-0-142-202 kubenswrapper[2563]: W0705 00:09:11.937470    2563 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 05 00:09:11.940587 ip-10-0-142-202 kubenswrapper[2563]: W0705 00:09:11.937473    2563 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 05 00:09:11.940587 ip-10-0-142-202 kubenswrapper[2563]: W0705 00:09:11.937475    2563 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 05 00:09:11.941054 ip-10-0-142-202 kubenswrapper[2563]: W0705 00:09:11.937478    2563 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 05 00:09:11.941054 ip-10-0-142-202 kubenswrapper[2563]: W0705 00:09:11.937480    2563 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 05 00:09:11.941054 ip-10-0-142-202 kubenswrapper[2563]: W0705 00:09:11.937483    2563 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 05 00:09:11.941054 ip-10-0-142-202 kubenswrapper[2563]: W0705 00:09:11.937486    2563 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 05 00:09:11.941054 ip-10-0-142-202 kubenswrapper[2563]: W0705 00:09:11.937488    2563 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 05 00:09:11.941054 ip-10-0-142-202 kubenswrapper[2563]: W0705 00:09:11.937491    2563 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 05 00:09:11.941054 ip-10-0-142-202 kubenswrapper[2563]: W0705 00:09:11.937495    2563 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 05 00:09:11.941054 ip-10-0-142-202 kubenswrapper[2563]: W0705 00:09:11.937498    2563 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 05 00:09:11.941054 ip-10-0-142-202 kubenswrapper[2563]: W0705 00:09:11.937501    2563 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 05 00:09:11.941054 ip-10-0-142-202 kubenswrapper[2563]: W0705 00:09:11.937503    2563 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 05 00:09:11.941054 ip-10-0-142-202 kubenswrapper[2563]: W0705 00:09:11.937506    2563 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 05 00:09:11.941054 ip-10-0-142-202 kubenswrapper[2563]: W0705 00:09:11.937509    2563 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 05 00:09:11.941054 ip-10-0-142-202 kubenswrapper[2563]: W0705 00:09:11.937511    2563 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 05 00:09:11.941054 ip-10-0-142-202 kubenswrapper[2563]: W0705 00:09:11.937514    2563 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 05 00:09:11.941054 ip-10-0-142-202 kubenswrapper[2563]: W0705 00:09:11.937516    2563 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 05 00:09:11.941054 ip-10-0-142-202 kubenswrapper[2563]: W0705 00:09:11.937519    2563 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 05 00:09:11.941054 ip-10-0-142-202 kubenswrapper[2563]: W0705 00:09:11.937521    2563 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 05 00:09:11.941054 ip-10-0-142-202 kubenswrapper[2563]: W0705 00:09:11.937524    2563 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 05 00:09:11.941054 ip-10-0-142-202 kubenswrapper[2563]: W0705 00:09:11.937526    2563 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 05 00:09:11.941054 ip-10-0-142-202 kubenswrapper[2563]: W0705 00:09:11.937532    2563 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 05 00:09:11.941539 ip-10-0-142-202 kubenswrapper[2563]: W0705 00:09:11.937534    2563 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 05 00:09:11.941539 ip-10-0-142-202 kubenswrapper[2563]: W0705 00:09:11.937538    2563 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 05 00:09:11.941539 ip-10-0-142-202 kubenswrapper[2563]: W0705 00:09:11.937540    2563 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 05 00:09:11.941539 ip-10-0-142-202 kubenswrapper[2563]: W0705 00:09:11.937543    2563 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 05 00:09:11.941539 ip-10-0-142-202 kubenswrapper[2563]: W0705 00:09:11.937545    2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 05 00:09:11.941539 ip-10-0-142-202 kubenswrapper[2563]: W0705 00:09:11.937548    2563 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 05 00:09:11.941539 ip-10-0-142-202 kubenswrapper[2563]: W0705 00:09:11.937557    2563 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 05 00:09:11.941539 ip-10-0-142-202 kubenswrapper[2563]: W0705 00:09:11.937559    2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 05 00:09:11.941539 ip-10-0-142-202 kubenswrapper[2563]: W0705 00:09:11.937562    2563 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 05 00:09:11.941539 ip-10-0-142-202 kubenswrapper[2563]: W0705 00:09:11.937565    2563 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 05 00:09:11.941539 ip-10-0-142-202 kubenswrapper[2563]: W0705 00:09:11.937567    2563 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 05 00:09:11.941539 ip-10-0-142-202 kubenswrapper[2563]: W0705 00:09:11.937570    2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 05 00:09:11.941539 ip-10-0-142-202 kubenswrapper[2563]: W0705 00:09:11.937572    2563 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 05 00:09:11.941539 ip-10-0-142-202 kubenswrapper[2563]: W0705 00:09:11.937574    2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 05 00:09:11.941539 ip-10-0-142-202 kubenswrapper[2563]: W0705 00:09:11.937577    2563 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 05 00:09:11.941539 ip-10-0-142-202 kubenswrapper[2563]: W0705 00:09:11.937580    2563 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 05 00:09:11.941539 ip-10-0-142-202 kubenswrapper[2563]: W0705 00:09:11.937582    2563 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 05 00:09:11.941539 ip-10-0-142-202 kubenswrapper[2563]: W0705 00:09:11.937585    2563 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 05 00:09:11.941539 ip-10-0-142-202 kubenswrapper[2563]: W0705 00:09:11.937587    2563 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 05 00:09:11.941539 ip-10-0-142-202 kubenswrapper[2563]: W0705 00:09:11.937590    2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 05 00:09:11.942005 ip-10-0-142-202 kubenswrapper[2563]: W0705 00:09:11.937592    2563 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 05 00:09:11.942005 ip-10-0-142-202 kubenswrapper[2563]: W0705 00:09:11.937594    2563 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 05 00:09:11.942005 ip-10-0-142-202 kubenswrapper[2563]: W0705 00:09:11.937597    2563 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 05 00:09:11.942005 ip-10-0-142-202 kubenswrapper[2563]: W0705 00:09:11.937599    2563 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 05 00:09:11.942005 ip-10-0-142-202 kubenswrapper[2563]: W0705 00:09:11.937602    2563 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 05 00:09:11.942005 ip-10-0-142-202 kubenswrapper[2563]: W0705 00:09:11.937604    2563 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 05 00:09:11.942005 ip-10-0-142-202 kubenswrapper[2563]: W0705 00:09:11.937606    2563 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 05 00:09:11.942005 ip-10-0-142-202 kubenswrapper[2563]: W0705 00:09:11.937609    2563 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 05 00:09:11.942005 ip-10-0-142-202 kubenswrapper[2563]: W0705 00:09:11.937613    2563 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 05 00:09:11.942005 ip-10-0-142-202 kubenswrapper[2563]: W0705 00:09:11.937616    2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 05 00:09:11.942005 ip-10-0-142-202 kubenswrapper[2563]: W0705 00:09:11.937619    2563 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 05 00:09:11.942005 ip-10-0-142-202 kubenswrapper[2563]: W0705 00:09:11.937623    2563 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 05 00:09:11.942005 ip-10-0-142-202 kubenswrapper[2563]: W0705 00:09:11.937626    2563 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 05 00:09:11.942005 ip-10-0-142-202 kubenswrapper[2563]: W0705 00:09:11.937628    2563 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 05 00:09:11.942005 ip-10-0-142-202 kubenswrapper[2563]: W0705 00:09:11.937631    2563 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 05 00:09:11.942005 ip-10-0-142-202 kubenswrapper[2563]: W0705 00:09:11.937636    2563 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 05 00:09:11.942005 ip-10-0-142-202 kubenswrapper[2563]: W0705 00:09:11.937639    2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 05 00:09:11.942005 ip-10-0-142-202 kubenswrapper[2563]: W0705 00:09:11.937642    2563 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 05 00:09:11.942005 ip-10-0-142-202 kubenswrapper[2563]: W0705 00:09:11.937645    2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 05 00:09:11.942520 ip-10-0-142-202 kubenswrapper[2563]: W0705 00:09:11.937654    2563 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 05 00:09:11.942520 ip-10-0-142-202 kubenswrapper[2563]: W0705 00:09:11.937657    2563 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 05 00:09:11.942520 ip-10-0-142-202 kubenswrapper[2563]: W0705 00:09:11.937660    2563 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 05 00:09:11.942520 ip-10-0-142-202 kubenswrapper[2563]: W0705 00:09:11.937662    2563 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 05 00:09:11.942520 ip-10-0-142-202 kubenswrapper[2563]: W0705 00:09:11.937665    2563 feature_gate.go:328] unrecognized feature gate: Example2
Jul 05 00:09:11.942520 ip-10-0-142-202 kubenswrapper[2563]: W0705 00:09:11.937668    2563 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 05 00:09:11.942520 ip-10-0-142-202 kubenswrapper[2563]: W0705 00:09:11.937670    2563 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 05 00:09:11.942520 ip-10-0-142-202 kubenswrapper[2563]: W0705 00:09:11.937673    2563 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 05 00:09:11.942520 ip-10-0-142-202 kubenswrapper[2563]: W0705 00:09:11.937675    2563 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 05 00:09:11.942520 ip-10-0-142-202 kubenswrapper[2563]: W0705 00:09:11.937678    2563 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 05 00:09:11.942520 ip-10-0-142-202 kubenswrapper[2563]: W0705 00:09:11.937680    2563 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 05 00:09:11.942520 ip-10-0-142-202 kubenswrapper[2563]: W0705 00:09:11.937682    2563 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 05 00:09:11.942520 ip-10-0-142-202 kubenswrapper[2563]: W0705 00:09:11.937685    2563 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 05 00:09:11.942520 ip-10-0-142-202 kubenswrapper[2563]: W0705 00:09:11.937687    2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 05 00:09:11.942520 ip-10-0-142-202 kubenswrapper[2563]: W0705 00:09:11.937690    2563 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 05 00:09:11.942520 ip-10-0-142-202 kubenswrapper[2563]: W0705 00:09:11.937692    2563 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 05 00:09:11.942520 ip-10-0-142-202 kubenswrapper[2563]: W0705 00:09:11.937695    2563 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 05 00:09:11.942520 ip-10-0-142-202 kubenswrapper[2563]: W0705 00:09:11.937697    2563 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 05 00:09:11.942520 ip-10-0-142-202 kubenswrapper[2563]: W0705 00:09:11.937700    2563 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 05 00:09:11.942520 ip-10-0-142-202 kubenswrapper[2563]: W0705 00:09:11.937705    2563 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 05 00:09:11.942997 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:09:11.937776    2563 flags.go:64] FLAG: --address="0.0.0.0"
Jul 05 00:09:11.942997 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:09:11.937783    2563 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]"
Jul 05 00:09:11.942997 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:09:11.937797    2563 flags.go:64] FLAG: --anonymous-auth="true"
Jul 05 00:09:11.942997 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:09:11.937801    2563 flags.go:64] FLAG: --application-metrics-count-limit="100"
Jul 05 00:09:11.942997 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:09:11.937807    2563 flags.go:64] FLAG: --authentication-token-webhook="false"
Jul 05 00:09:11.942997 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:09:11.937810    2563 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s"
Jul 05 00:09:11.942997 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:09:11.937814    2563 flags.go:64] FLAG: --authorization-mode="AlwaysAllow"
Jul 05 00:09:11.942997 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:09:11.937819    2563 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s"
Jul 05 00:09:11.942997 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:09:11.937822    2563 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s"
Jul 05 00:09:11.942997 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:09:11.937825    2563 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id"
Jul 05 00:09:11.942997 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:09:11.937828    2563 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig"
Jul 05 00:09:11.942997 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:09:11.937831    2563 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki"
Jul 05 00:09:11.942997 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:09:11.937835    2563 flags.go:64] FLAG: --cgroup-driver="cgroupfs"
Jul 05 00:09:11.942997 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:09:11.937852    2563 flags.go:64] FLAG: --cgroup-root=""
Jul 05 00:09:11.942997 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:09:11.937855    2563 flags.go:64] FLAG: --cgroups-per-qos="true"
Jul 05 00:09:11.942997 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:09:11.937858    2563 flags.go:64] FLAG: --client-ca-file=""
Jul 05 00:09:11.942997 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:09:11.937861    2563 flags.go:64] FLAG: --cloud-provider="external"
Jul 05 00:09:11.942997 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:09:11.937864    2563 flags.go:64] FLAG: --cluster-dns="[]"
Jul 05 00:09:11.942997 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:09:11.937871    2563 flags.go:64] FLAG: --cluster-domain=""
Jul 05 00:09:11.942997 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:09:11.937873    2563 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf"
Jul 05 00:09:11.942997 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:09:11.937876    2563 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d"
Jul 05 00:09:11.942997 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:09:11.937880    2563 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json"
Jul 05 00:09:11.942997 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:09:11.937883    2563 flags.go:64] FLAG: --container-log-max-files="5"
Jul 05 00:09:11.942997 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:09:11.937886    2563 flags.go:64] FLAG: --container-log-max-size="10Mi"
Jul 05 00:09:11.943580 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:09:11.937889    2563 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock"
Jul 05 00:09:11.943580 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:09:11.937892    2563 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock"
Jul 05 00:09:11.943580 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:09:11.937895    2563 flags.go:64] FLAG: --containerd-namespace="k8s.io"
Jul 05 00:09:11.943580 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:09:11.937898    2563 flags.go:64] FLAG: --contention-profiling="false"
Jul 05 00:09:11.943580 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:09:11.937901    2563 flags.go:64] FLAG: --cpu-cfs-quota="true"
Jul 05 00:09:11.943580 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:09:11.937904    2563 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms"
Jul 05 00:09:11.943580 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:09:11.937907    2563 flags.go:64] FLAG: --cpu-manager-policy="none"
Jul 05 00:09:11.943580 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:09:11.937911    2563 flags.go:64] FLAG: --cpu-manager-policy-options=""
Jul 05 00:09:11.943580 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:09:11.937915    2563 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s"
Jul 05 00:09:11.943580 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:09:11.937918    2563 flags.go:64] FLAG: --enable-controller-attach-detach="true"
Jul 05 00:09:11.943580 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:09:11.937921    2563 flags.go:64] FLAG: --enable-debugging-handlers="true"
Jul 05 00:09:11.943580 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:09:11.937924    2563 flags.go:64] FLAG: --enable-load-reader="false"
Jul 05 00:09:11.943580 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:09:11.937928    2563 flags.go:64] FLAG: --enable-server="true"
Jul 05 00:09:11.943580 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:09:11.937931    2563 flags.go:64] FLAG: --enforce-node-allocatable="[pods]"
Jul 05 00:09:11.943580 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:09:11.937939    2563 flags.go:64] FLAG: --event-burst="100"
Jul 05 00:09:11.943580 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:09:11.937942    2563 flags.go:64] FLAG: --event-qps="50"
Jul 05 00:09:11.943580 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:09:11.937944    2563 flags.go:64] FLAG: --event-storage-age-limit="default=0"
Jul 05 00:09:11.943580 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:09:11.937947    2563 flags.go:64] FLAG: --event-storage-event-limit="default=0"
Jul 05 00:09:11.943580 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:09:11.937950    2563 flags.go:64] FLAG: --eviction-hard=""
Jul 05 00:09:11.943580 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:09:11.937954    2563 flags.go:64] FLAG: --eviction-max-pod-grace-period="0"
Jul 05 00:09:11.943580 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:09:11.937957    2563 flags.go:64] FLAG: --eviction-minimum-reclaim=""
Jul 05 00:09:11.943580 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:09:11.937960    2563 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s"
Jul 05 00:09:11.943580 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:09:11.937969    2563 flags.go:64] FLAG: --eviction-soft=""
Jul 05 00:09:11.943580 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:09:11.937972    2563 flags.go:64] FLAG: --eviction-soft-grace-period=""
Jul 05 00:09:11.943580 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:09:11.937975    2563 flags.go:64] FLAG: --exit-on-lock-contention="false"
Jul 05 00:09:11.944231 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:09:11.937978    2563 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false"
Jul 05 00:09:11.944231 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:09:11.937981    2563 flags.go:64] FLAG: --experimental-mounter-path=""
Jul 05 00:09:11.944231 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:09:11.937983    2563 flags.go:64] FLAG: --fail-cgroupv1="false"
Jul 05 00:09:11.944231 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:09:11.937986    2563 flags.go:64] FLAG: --fail-swap-on="true"
Jul 05 00:09:11.944231 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:09:11.937989    2563 flags.go:64] FLAG: --feature-gates=""
Jul 05 00:09:11.944231 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:09:11.937993    2563 flags.go:64] FLAG: --file-check-frequency="20s"
Jul 05 00:09:11.944231 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:09:11.937996    2563 flags.go:64] FLAG: --global-housekeeping-interval="1m0s"
Jul 05 00:09:11.944231 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:09:11.937999    2563 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge"
Jul 05 00:09:11.944231 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:09:11.938002    2563 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1"
Jul 05 00:09:11.944231 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:09:11.938005    2563 flags.go:64] FLAG: --healthz-port="10248"
Jul 05 00:09:11.944231 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:09:11.938007    2563 flags.go:64] FLAG: --help="false"
Jul 05 00:09:11.944231 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:09:11.938010    2563 flags.go:64] FLAG: --hostname-override="ip-10-0-142-202.ec2.internal"
Jul 05 00:09:11.944231 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:09:11.938013    2563 flags.go:64] FLAG: --housekeeping-interval="10s"
Jul 05 00:09:11.944231 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:09:11.938016    2563 flags.go:64] FLAG: --http-check-frequency="20s"
Jul 05 00:09:11.944231 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:09:11.938020    2563 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins"
Jul 05 00:09:11.944231 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:09:11.938023    2563 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml"
Jul 05 00:09:11.944231 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:09:11.938027    2563 flags.go:64] FLAG: --image-gc-high-threshold="85"
Jul 05 00:09:11.944231 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:09:11.938029    2563 flags.go:64] FLAG: --image-gc-low-threshold="80"
Jul 05 00:09:11.944231 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:09:11.938032    2563 flags.go:64] FLAG: --image-service-endpoint=""
Jul 05 00:09:11.944231 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:09:11.938037    2563 flags.go:64] FLAG: --kernel-memcg-notification="false"
Jul 05 00:09:11.944231 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:09:11.938040    2563 flags.go:64] FLAG: --kube-api-burst="100"
Jul 05 00:09:11.944231 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:09:11.938043    2563 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf"
Jul 05 00:09:11.944231 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:09:11.938046    2563 flags.go:64] FLAG: --kube-api-qps="50"
Jul 05 00:09:11.944231 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:09:11.938049    2563 flags.go:64] FLAG: --kube-reserved=""
Jul 05 00:09:11.944859 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:09:11.938052    2563 flags.go:64] FLAG: --kube-reserved-cgroup=""
Jul 05 00:09:11.944859 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:09:11.938055    2563 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig"
Jul 05 00:09:11.944859 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:09:11.938058    2563 flags.go:64] FLAG: --kubelet-cgroups=""
Jul 05 00:09:11.944859 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:09:11.938061    2563 flags.go:64] FLAG: --local-storage-capacity-isolation="true"
Jul 05 00:09:11.944859 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:09:11.938063    2563 flags.go:64] FLAG: --lock-file=""
Jul 05 00:09:11.944859 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:09:11.938066    2563 flags.go:64] FLAG: --log-cadvisor-usage="false"
Jul 05 00:09:11.944859 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:09:11.938075    2563 flags.go:64] FLAG: --log-flush-frequency="5s"
Jul 05 00:09:11.944859 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:09:11.938078    2563 flags.go:64] FLAG: --log-json-info-buffer-size="0"
Jul 05 00:09:11.944859 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:09:11.938084    2563 flags.go:64] FLAG: --log-json-split-stream="false"
Jul 05 00:09:11.944859 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:09:11.938087    2563 flags.go:64] FLAG: --log-text-info-buffer-size="0"
Jul 05 00:09:11.944859 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:09:11.938089    2563 flags.go:64] FLAG: --log-text-split-stream="false"
Jul 05 00:09:11.944859 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:09:11.938092    2563 flags.go:64] FLAG: --logging-format="text"
Jul 05 00:09:11.944859 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:09:11.938095    2563 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id"
Jul 05 00:09:11.944859 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:09:11.938098    2563 flags.go:64] FLAG: --make-iptables-util-chains="true"
Jul 05 00:09:11.944859 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:09:11.938101    2563 flags.go:64] FLAG: --manifest-url=""
Jul 05 00:09:11.944859 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:09:11.938104    2563 flags.go:64] FLAG: --manifest-url-header=""
Jul 05 00:09:11.944859 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:09:11.938108    2563 flags.go:64] FLAG: --max-housekeeping-interval="15s"
Jul 05 00:09:11.944859 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:09:11.938111    2563 flags.go:64] FLAG: --max-open-files="1000000"
Jul 05 00:09:11.944859 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:09:11.938115    2563 flags.go:64] FLAG: --max-pods="110"
Jul 05 00:09:11.944859 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:09:11.938118    2563 flags.go:64] FLAG: --maximum-dead-containers="-1"
Jul 05 00:09:11.944859 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:09:11.938120    2563 flags.go:64] FLAG: --maximum-dead-containers-per-container="1"
Jul 05 00:09:11.944859 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:09:11.938123    2563 flags.go:64] FLAG: --memory-manager-policy="None"
Jul 05 00:09:11.944859 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:09:11.938127    2563 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s"
Jul 05 00:09:11.944859 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:09:11.938131    2563 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s"
Jul 05 00:09:11.944859 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:09:11.938133    2563 flags.go:64] FLAG: --node-ip="0.0.0.0"
Jul 05 00:09:11.945474 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:09:11.938136    2563 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel"
Jul 05 00:09:11.945474 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:09:11.938144    2563 flags.go:64] FLAG: --node-status-max-images="50"
Jul 05 00:09:11.945474 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:09:11.938149    2563 flags.go:64] FLAG: --node-status-update-frequency="10s"
Jul 05 00:09:11.945474 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:09:11.938152    2563 flags.go:64] FLAG: --oom-score-adj="-999"
Jul 05 00:09:11.945474 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:09:11.938155    2563 flags.go:64] FLAG: --pod-cidr=""
Jul 05 00:09:11.945474 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:09:11.938158    2563 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:ce93f6b4197b9c62a26a3131f8057ffe9c9379c5079a4f62e5919b75151394e5"
Jul 05 00:09:11.945474 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:09:11.938163    2563 flags.go:64] FLAG: --pod-manifest-path=""
Jul 05 00:09:11.945474 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:09:11.938166    2563 flags.go:64] FLAG: --pod-max-pids="-1"
Jul 05 00:09:11.945474 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:09:11.938169    2563 flags.go:64] FLAG: --pods-per-core="0"
Jul 05 00:09:11.945474 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:09:11.938172    2563 flags.go:64] FLAG: --port="10250"
Jul 05 00:09:11.945474 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:09:11.938175    2563 flags.go:64] FLAG: --protect-kernel-defaults="false"
Jul 05 00:09:11.945474 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:09:11.938178    2563 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-09d7324c3be832884"
Jul 05 00:09:11.945474 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:09:11.938182    2563 flags.go:64] FLAG: --qos-reserved=""
Jul 05 00:09:11.945474 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:09:11.938184    2563 flags.go:64] FLAG: --read-only-port="10255"
Jul 05 00:09:11.945474 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:09:11.938193    2563 flags.go:64] FLAG: --register-node="true"
Jul 05 00:09:11.945474 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:09:11.938197    2563 flags.go:64] FLAG: --register-with-taints=""
Jul 05 00:09:11.945474 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:09:11.938201    2563 flags.go:64] FLAG: --registry-burst="10"
Jul 05 00:09:11.945474 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:09:11.938204    2563 flags.go:64] FLAG: --registry-qps="5"
Jul 05 00:09:11.945474 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:09:11.938206    2563 flags.go:64] FLAG: --reserved-cpus=""
Jul 05 00:09:11.945474 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:09:11.938209    2563 flags.go:64] FLAG: --reserved-memory=""
Jul 05 00:09:11.945474 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:09:11.938213    2563 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf"
Jul 05 00:09:11.945474 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:09:11.938230    2563 flags.go:64] FLAG: --root-dir="/var/lib/kubelet"
Jul 05 00:09:11.945474 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:09:11.938236    2563 flags.go:64] FLAG: --rotate-certificates="false"
Jul 05 00:09:11.945474 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:09:11.938240    2563 flags.go:64] FLAG: --rotate-server-certificates="false"
Jul 05 00:09:11.945474 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:09:11.938245    2563 flags.go:64] FLAG: --runonce="false"
Jul 05 00:09:11.946116 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:09:11.938249    2563 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service"
Jul 05 00:09:11.946116 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:09:11.938252    2563 flags.go:64] FLAG: --runtime-request-timeout="2m0s"
Jul 05 00:09:11.946116 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:09:11.938255    2563 flags.go:64] FLAG: --seccomp-default="false"
Jul 05 00:09:11.946116 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:09:11.938258    2563 flags.go:64] FLAG: --serialize-image-pulls="true"
Jul 05 00:09:11.946116 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:09:11.938261    2563 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s"
Jul 05 00:09:11.946116 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:09:11.938265    2563 flags.go:64] FLAG: --storage-driver-db="cadvisor"
Jul 05 00:09:11.946116 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:09:11.938268    2563 flags.go:64] FLAG: --storage-driver-host="localhost:8086"
Jul 05 00:09:11.946116 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:09:11.938271    2563 flags.go:64] FLAG: --storage-driver-password="root"
Jul 05 00:09:11.946116 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:09:11.938274    2563 flags.go:64] FLAG: --storage-driver-secure="false"
Jul 05 00:09:11.946116 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:09:11.938279    2563 flags.go:64] FLAG: --storage-driver-table="stats"
Jul 05 00:09:11.946116 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:09:11.938282    2563 flags.go:64] FLAG: --storage-driver-user="root"
Jul 05 00:09:11.946116 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:09:11.938285    2563 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s"
Jul 05 00:09:11.946116 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:09:11.938288    2563 flags.go:64] FLAG: --sync-frequency="1m0s"
Jul 05 00:09:11.946116 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:09:11.938291    2563 flags.go:64] FLAG: --system-cgroups=""
Jul 05 00:09:11.946116 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:09:11.938294    2563 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi"
Jul 05 00:09:11.946116 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:09:11.938299    2563 flags.go:64] FLAG: --system-reserved-cgroup=""
Jul 05 00:09:11.946116 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:09:11.938302    2563 flags.go:64] FLAG: --tls-cert-file=""
Jul 05 00:09:11.946116 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:09:11.938305    2563 flags.go:64] FLAG: --tls-cipher-suites="[]"
Jul 05 00:09:11.946116 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:09:11.938312    2563 flags.go:64] FLAG: --tls-min-version=""
Jul 05 00:09:11.946116 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:09:11.938315    2563 flags.go:64] FLAG: --tls-private-key-file=""
Jul 05 00:09:11.946116 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:09:11.938318    2563 flags.go:64] FLAG: --topology-manager-policy="none"
Jul 05 00:09:11.946116 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:09:11.938321    2563 flags.go:64] FLAG: --topology-manager-policy-options=""
Jul 05 00:09:11.946116 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:09:11.938325    2563 flags.go:64] FLAG: --topology-manager-scope="container"
Jul 05 00:09:11.946116 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:09:11.938328    2563 flags.go:64] FLAG: --v="2"
Jul 05 00:09:11.946116 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:09:11.938332    2563 flags.go:64] FLAG: --version="false"
Jul 05 00:09:11.946730 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:09:11.938335    2563 flags.go:64] FLAG: --vmodule=""
Jul 05 00:09:11.946730 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:09:11.938339    2563 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec"
Jul 05 00:09:11.946730 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:09:11.938342    2563 flags.go:64] FLAG: --volume-stats-agg-period="1m0s"
Jul 05 00:09:11.946730 ip-10-0-142-202 kubenswrapper[2563]: W0705 00:09:11.938481    2563 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 05 00:09:11.946730 ip-10-0-142-202 kubenswrapper[2563]: W0705 00:09:11.938485    2563 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 05 00:09:11.946730 ip-10-0-142-202 kubenswrapper[2563]: W0705 00:09:11.938488    2563 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 05 00:09:11.946730 ip-10-0-142-202 kubenswrapper[2563]: W0705 00:09:11.938490    2563 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 05 00:09:11.946730 ip-10-0-142-202 kubenswrapper[2563]: W0705 00:09:11.938493    2563 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 05 00:09:11.946730 ip-10-0-142-202 kubenswrapper[2563]: W0705 00:09:11.938496    2563 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 05 00:09:11.946730 ip-10-0-142-202 kubenswrapper[2563]: W0705 00:09:11.938499    2563 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 05 00:09:11.946730 ip-10-0-142-202 kubenswrapper[2563]: W0705 00:09:11.938501    2563 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 05 00:09:11.946730 ip-10-0-142-202 kubenswrapper[2563]: W0705 00:09:11.938504    2563 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 05 00:09:11.946730 ip-10-0-142-202 kubenswrapper[2563]: W0705 00:09:11.938508    2563 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 05 00:09:11.946730 ip-10-0-142-202 kubenswrapper[2563]: W0705 00:09:11.938510    2563 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 05 00:09:11.946730 ip-10-0-142-202 kubenswrapper[2563]: W0705 00:09:11.938513    2563 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 05 00:09:11.946730 ip-10-0-142-202 kubenswrapper[2563]: W0705 00:09:11.938515    2563 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 05 00:09:11.946730 ip-10-0-142-202 kubenswrapper[2563]: W0705 00:09:11.938520    2563 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 05 00:09:11.946730 ip-10-0-142-202 kubenswrapper[2563]: W0705 00:09:11.938522    2563 feature_gate.go:328] unrecognized feature gate: Example
Jul 05 00:09:11.946730 ip-10-0-142-202 kubenswrapper[2563]: W0705 00:09:11.938525    2563 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 05 00:09:11.946730 ip-10-0-142-202 kubenswrapper[2563]: W0705 00:09:11.938527    2563 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 05 00:09:11.947215 ip-10-0-142-202 kubenswrapper[2563]: W0705 00:09:11.938530    2563 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 05 00:09:11.947215 ip-10-0-142-202 kubenswrapper[2563]: W0705 00:09:11.938532    2563 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 05 00:09:11.947215 ip-10-0-142-202 kubenswrapper[2563]: W0705 00:09:11.938535    2563 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 05 00:09:11.947215 ip-10-0-142-202 kubenswrapper[2563]: W0705 00:09:11.938538    2563 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 05 00:09:11.947215 ip-10-0-142-202 kubenswrapper[2563]: W0705 00:09:11.938540    2563 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 05 00:09:11.947215 ip-10-0-142-202 kubenswrapper[2563]: W0705 00:09:11.938543    2563 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 05 00:09:11.947215 ip-10-0-142-202 kubenswrapper[2563]: W0705 00:09:11.938545    2563 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 05 00:09:11.947215 ip-10-0-142-202 kubenswrapper[2563]: W0705 00:09:11.938548    2563 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 05 00:09:11.947215 ip-10-0-142-202 kubenswrapper[2563]: W0705 00:09:11.938550    2563 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 05 00:09:11.947215 ip-10-0-142-202 kubenswrapper[2563]: W0705 00:09:11.938553    2563 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 05 00:09:11.947215 ip-10-0-142-202 kubenswrapper[2563]: W0705 00:09:11.938556    2563 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 05 00:09:11.947215 ip-10-0-142-202 kubenswrapper[2563]: W0705 00:09:11.938558    2563 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 05 00:09:11.947215 ip-10-0-142-202 kubenswrapper[2563]: W0705 00:09:11.938561    2563 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 05 00:09:11.947215 ip-10-0-142-202 kubenswrapper[2563]: W0705 00:09:11.938564    2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 05 00:09:11.947215 ip-10-0-142-202 kubenswrapper[2563]: W0705 00:09:11.938566    2563 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 05 00:09:11.947215 ip-10-0-142-202 kubenswrapper[2563]: W0705 00:09:11.938569    2563 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 05 00:09:11.947215 ip-10-0-142-202 kubenswrapper[2563]: W0705 00:09:11.938573    2563 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 05 00:09:11.947215 ip-10-0-142-202 kubenswrapper[2563]: W0705 00:09:11.938576    2563 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 05 00:09:11.947215 ip-10-0-142-202 kubenswrapper[2563]: W0705 00:09:11.938579    2563 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 05 00:09:11.947215 ip-10-0-142-202 kubenswrapper[2563]: W0705 00:09:11.938581    2563 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 05 00:09:11.947754 ip-10-0-142-202 kubenswrapper[2563]: W0705 00:09:11.938584    2563 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 05 00:09:11.947754 ip-10-0-142-202 kubenswrapper[2563]: W0705 00:09:11.938586    2563 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 05 00:09:11.947754 ip-10-0-142-202 kubenswrapper[2563]: W0705 00:09:11.938589    2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 05 00:09:11.947754 ip-10-0-142-202 kubenswrapper[2563]: W0705 00:09:11.938591    2563 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 05 00:09:11.947754 ip-10-0-142-202 kubenswrapper[2563]: W0705 00:09:11.938595    2563 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 05 00:09:11.947754 ip-10-0-142-202 kubenswrapper[2563]: W0705 00:09:11.938598    2563 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 05 00:09:11.947754 ip-10-0-142-202 kubenswrapper[2563]: W0705 00:09:11.938601    2563 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 05 00:09:11.947754 ip-10-0-142-202 kubenswrapper[2563]: W0705 00:09:11.938604    2563 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 05 00:09:11.947754 ip-10-0-142-202 kubenswrapper[2563]: W0705 00:09:11.938608    2563 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 05 00:09:11.947754 ip-10-0-142-202 kubenswrapper[2563]: W0705 00:09:11.938610    2563 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 05 00:09:11.947754 ip-10-0-142-202 kubenswrapper[2563]: W0705 00:09:11.938613    2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 05 00:09:11.947754 ip-10-0-142-202 kubenswrapper[2563]: W0705 00:09:11.938615    2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 05 00:09:11.947754 ip-10-0-142-202 kubenswrapper[2563]: W0705 00:09:11.938618    2563 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 05 00:09:11.947754 ip-10-0-142-202 kubenswrapper[2563]: W0705 00:09:11.938620    2563 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 05 00:09:11.947754 ip-10-0-142-202 kubenswrapper[2563]: W0705 00:09:11.938623    2563 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 05 00:09:11.947754 ip-10-0-142-202 kubenswrapper[2563]: W0705 00:09:11.938625    2563 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 05 00:09:11.947754 ip-10-0-142-202 kubenswrapper[2563]: W0705 00:09:11.938628    2563 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 05 00:09:11.947754 ip-10-0-142-202 kubenswrapper[2563]: W0705 00:09:11.938631    2563 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 05 00:09:11.947754 ip-10-0-142-202 kubenswrapper[2563]: W0705 00:09:11.938635    2563 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 05 00:09:11.948211 ip-10-0-142-202 kubenswrapper[2563]: W0705 00:09:11.938638    2563 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 05 00:09:11.948211 ip-10-0-142-202 kubenswrapper[2563]: W0705 00:09:11.938641    2563 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 05 00:09:11.948211 ip-10-0-142-202 kubenswrapper[2563]: W0705 00:09:11.938644    2563 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 05 00:09:11.948211 ip-10-0-142-202 kubenswrapper[2563]: W0705 00:09:11.938646    2563 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 05 00:09:11.948211 ip-10-0-142-202 kubenswrapper[2563]: W0705 00:09:11.938649    2563 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 05 00:09:11.948211 ip-10-0-142-202 kubenswrapper[2563]: W0705 00:09:11.938652    2563 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 05 00:09:11.948211 ip-10-0-142-202 kubenswrapper[2563]: W0705 00:09:11.938654    2563 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 05 00:09:11.948211 ip-10-0-142-202 kubenswrapper[2563]: W0705 00:09:11.938656    2563 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 05 00:09:11.948211 ip-10-0-142-202 kubenswrapper[2563]: W0705 00:09:11.938659    2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 05 00:09:11.948211 ip-10-0-142-202 kubenswrapper[2563]: W0705 00:09:11.938662    2563 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 05 00:09:11.948211 ip-10-0-142-202 kubenswrapper[2563]: W0705 00:09:11.938664    2563 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 05 00:09:11.948211 ip-10-0-142-202 kubenswrapper[2563]: W0705 00:09:11.938666    2563 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 05 00:09:11.948211 ip-10-0-142-202 kubenswrapper[2563]: W0705 00:09:11.938669    2563 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 05 00:09:11.948211 ip-10-0-142-202 kubenswrapper[2563]: W0705 00:09:11.938671    2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 05 00:09:11.948211 ip-10-0-142-202 kubenswrapper[2563]: W0705 00:09:11.938674    2563 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 05 00:09:11.948211 ip-10-0-142-202 kubenswrapper[2563]: W0705 00:09:11.938677    2563 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 05 00:09:11.948211 ip-10-0-142-202 kubenswrapper[2563]: W0705 00:09:11.938679    2563 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 05 00:09:11.948211 ip-10-0-142-202 kubenswrapper[2563]: W0705 00:09:11.938683    2563 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 05 00:09:11.948211 ip-10-0-142-202 kubenswrapper[2563]: W0705 00:09:11.938686    2563 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 05 00:09:11.948211 ip-10-0-142-202 kubenswrapper[2563]: W0705 00:09:11.938689    2563 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 05 00:09:11.948812 ip-10-0-142-202 kubenswrapper[2563]: W0705 00:09:11.938691    2563 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 05 00:09:11.948812 ip-10-0-142-202 kubenswrapper[2563]: W0705 00:09:11.938695    2563 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 05 00:09:11.948812 ip-10-0-142-202 kubenswrapper[2563]: W0705 00:09:11.938698    2563 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 05 00:09:11.948812 ip-10-0-142-202 kubenswrapper[2563]: W0705 00:09:11.938701    2563 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 05 00:09:11.948812 ip-10-0-142-202 kubenswrapper[2563]: W0705 00:09:11.938703    2563 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 05 00:09:11.948812 ip-10-0-142-202 kubenswrapper[2563]: W0705 00:09:11.938706    2563 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 05 00:09:11.948812 ip-10-0-142-202 kubenswrapper[2563]: W0705 00:09:11.938708    2563 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 05 00:09:11.948812 ip-10-0-142-202 kubenswrapper[2563]: W0705 00:09:11.938711    2563 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 05 00:09:11.948812 ip-10-0-142-202 kubenswrapper[2563]: W0705 00:09:11.938715    2563 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 05 00:09:11.948812 ip-10-0-142-202 kubenswrapper[2563]: W0705 00:09:11.938718    2563 feature_gate.go:328] unrecognized feature gate: Example2
Jul 05 00:09:11.948812 ip-10-0-142-202 kubenswrapper[2563]: W0705 00:09:11.938721    2563 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 05 00:09:11.948812 ip-10-0-142-202 kubenswrapper[2563]: W0705 00:09:11.938724    2563 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 05 00:09:11.948812 ip-10-0-142-202 kubenswrapper[2563]: W0705 00:09:11.938727    2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 05 00:09:11.948812 ip-10-0-142-202 kubenswrapper[2563]: W0705 00:09:11.938730    2563 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 05 00:09:11.948812 ip-10-0-142-202 kubenswrapper[2563]: W0705 00:09:11.938732    2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 05 00:09:11.948812 ip-10-0-142-202 kubenswrapper[2563]: W0705 00:09:11.938735    2563 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 05 00:09:11.948812 ip-10-0-142-202 kubenswrapper[2563]: W0705 00:09:11.938738    2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 05 00:09:11.948812 ip-10-0-142-202 kubenswrapper[2563]: W0705 00:09:11.938741    2563 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 05 00:09:11.948812 ip-10-0-142-202 kubenswrapper[2563]: W0705 00:09:11.938744    2563 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 05 00:09:11.948812 ip-10-0-142-202 kubenswrapper[2563]: W0705 00:09:11.938746    2563 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 05 00:09:11.949345 ip-10-0-142-202 kubenswrapper[2563]: W0705 00:09:11.938749    2563 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 05 00:09:11.949345 ip-10-0-142-202 kubenswrapper[2563]: W0705 00:09:11.938751    2563 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 05 00:09:11.949345 ip-10-0-142-202 kubenswrapper[2563]: W0705 00:09:11.938754    2563 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 05 00:09:11.949345 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:09:11.939383    2563 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Jul 05 00:09:11.949345 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:09:11.946207    2563 server.go:529] "Kubelet version" kubeletVersion="v1.34.8"
Jul 05 00:09:11.949345 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:09:11.946239    2563 server.go:531] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK=""
Jul 05 00:09:11.949345 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:09:11.946253    2563 watchdog_linux.go:95] "Systemd watchdog is not enabled"
Jul 05 00:09:11.949345 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:09:11.946259    2563 watchdog_linux.go:137] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started."
Jul 05 00:09:11.949345 ip-10-0-142-202 kubenswrapper[2563]: W0705 00:09:11.946315    2563 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 05 00:09:11.949345 ip-10-0-142-202 kubenswrapper[2563]: W0705 00:09:11.946319    2563 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 05 00:09:11.949345 ip-10-0-142-202 kubenswrapper[2563]: W0705 00:09:11.946322    2563 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 05 00:09:11.949345 ip-10-0-142-202 kubenswrapper[2563]: W0705 00:09:11.946325    2563 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 05 00:09:11.949345 ip-10-0-142-202 kubenswrapper[2563]: W0705 00:09:11.946328    2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 05 00:09:11.949345 ip-10-0-142-202 kubenswrapper[2563]: W0705 00:09:11.946330    2563 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 05 00:09:11.949345 ip-10-0-142-202 kubenswrapper[2563]: W0705 00:09:11.946333    2563 feature_gate.go:328] unrecognized feature gate: Example2
Jul 05 00:09:11.949726 ip-10-0-142-202 kubenswrapper[2563]: W0705 00:09:11.946336    2563 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 05 00:09:11.949726 ip-10-0-142-202 kubenswrapper[2563]: W0705 00:09:11.946338    2563 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 05 00:09:11.949726 ip-10-0-142-202 kubenswrapper[2563]: W0705 00:09:11.946342    2563 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 05 00:09:11.949726 ip-10-0-142-202 kubenswrapper[2563]: W0705 00:09:11.946344    2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 05 00:09:11.949726 ip-10-0-142-202 kubenswrapper[2563]: W0705 00:09:11.946349    2563 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 05 00:09:11.949726 ip-10-0-142-202 kubenswrapper[2563]: W0705 00:09:11.946352    2563 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 05 00:09:11.949726 ip-10-0-142-202 kubenswrapper[2563]: W0705 00:09:11.946356    2563 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 05 00:09:11.949726 ip-10-0-142-202 kubenswrapper[2563]: W0705 00:09:11.946359    2563 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 05 00:09:11.949726 ip-10-0-142-202 kubenswrapper[2563]: W0705 00:09:11.946362    2563 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 05 00:09:11.949726 ip-10-0-142-202 kubenswrapper[2563]: W0705 00:09:11.946366    2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 05 00:09:11.949726 ip-10-0-142-202 kubenswrapper[2563]: W0705 00:09:11.946369    2563 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 05 00:09:11.949726 ip-10-0-142-202 kubenswrapper[2563]: W0705 00:09:11.946372    2563 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 05 00:09:11.949726 ip-10-0-142-202 kubenswrapper[2563]: W0705 00:09:11.946374    2563 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 05 00:09:11.949726 ip-10-0-142-202 kubenswrapper[2563]: W0705 00:09:11.946377    2563 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 05 00:09:11.949726 ip-10-0-142-202 kubenswrapper[2563]: W0705 00:09:11.946380    2563 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 05 00:09:11.949726 ip-10-0-142-202 kubenswrapper[2563]: W0705 00:09:11.946383    2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 05 00:09:11.949726 ip-10-0-142-202 kubenswrapper[2563]: W0705 00:09:11.946385    2563 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 05 00:09:11.949726 ip-10-0-142-202 kubenswrapper[2563]: W0705 00:09:11.946388    2563 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 05 00:09:11.949726 ip-10-0-142-202 kubenswrapper[2563]: W0705 00:09:11.946390    2563 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 05 00:09:11.950180 ip-10-0-142-202 kubenswrapper[2563]: W0705 00:09:11.946393    2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 05 00:09:11.950180 ip-10-0-142-202 kubenswrapper[2563]: W0705 00:09:11.946395    2563 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 05 00:09:11.950180 ip-10-0-142-202 kubenswrapper[2563]: W0705 00:09:11.946398    2563 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 05 00:09:11.950180 ip-10-0-142-202 kubenswrapper[2563]: W0705 00:09:11.946402    2563 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 05 00:09:11.950180 ip-10-0-142-202 kubenswrapper[2563]: W0705 00:09:11.946405    2563 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 05 00:09:11.950180 ip-10-0-142-202 kubenswrapper[2563]: W0705 00:09:11.946408    2563 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 05 00:09:11.950180 ip-10-0-142-202 kubenswrapper[2563]: W0705 00:09:11.946411    2563 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 05 00:09:11.950180 ip-10-0-142-202 kubenswrapper[2563]: W0705 00:09:11.946413    2563 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 05 00:09:11.950180 ip-10-0-142-202 kubenswrapper[2563]: W0705 00:09:11.946416    2563 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 05 00:09:11.950180 ip-10-0-142-202 kubenswrapper[2563]: W0705 00:09:11.946419    2563 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 05 00:09:11.950180 ip-10-0-142-202 kubenswrapper[2563]: W0705 00:09:11.946421    2563 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 05 00:09:11.950180 ip-10-0-142-202 kubenswrapper[2563]: W0705 00:09:11.946424    2563 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 05 00:09:11.950180 ip-10-0-142-202 kubenswrapper[2563]: W0705 00:09:11.946426    2563 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 05 00:09:11.950180 ip-10-0-142-202 kubenswrapper[2563]: W0705 00:09:11.946429    2563 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 05 00:09:11.950180 ip-10-0-142-202 kubenswrapper[2563]: W0705 00:09:11.946432    2563 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 05 00:09:11.950180 ip-10-0-142-202 kubenswrapper[2563]: W0705 00:09:11.946434    2563 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 05 00:09:11.950180 ip-10-0-142-202 kubenswrapper[2563]: W0705 00:09:11.946437    2563 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 05 00:09:11.950180 ip-10-0-142-202 kubenswrapper[2563]: W0705 00:09:11.946440    2563 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 05 00:09:11.950180 ip-10-0-142-202 kubenswrapper[2563]: W0705 00:09:11.946442    2563 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 05 00:09:11.950654 ip-10-0-142-202 kubenswrapper[2563]: W0705 00:09:11.946445    2563 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 05 00:09:11.950654 ip-10-0-142-202 kubenswrapper[2563]: W0705 00:09:11.946447    2563 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 05 00:09:11.950654 ip-10-0-142-202 kubenswrapper[2563]: W0705 00:09:11.946450    2563 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 05 00:09:11.950654 ip-10-0-142-202 kubenswrapper[2563]: W0705 00:09:11.946453    2563 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 05 00:09:11.950654 ip-10-0-142-202 kubenswrapper[2563]: W0705 00:09:11.946455    2563 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 05 00:09:11.950654 ip-10-0-142-202 kubenswrapper[2563]: W0705 00:09:11.946458    2563 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 05 00:09:11.950654 ip-10-0-142-202 kubenswrapper[2563]: W0705 00:09:11.946460    2563 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 05 00:09:11.950654 ip-10-0-142-202 kubenswrapper[2563]: W0705 00:09:11.946463    2563 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 05 00:09:11.950654 ip-10-0-142-202 kubenswrapper[2563]: W0705 00:09:11.946465    2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 05 00:09:11.950654 ip-10-0-142-202 kubenswrapper[2563]: W0705 00:09:11.946468    2563 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 05 00:09:11.950654 ip-10-0-142-202 kubenswrapper[2563]: W0705 00:09:11.946471    2563 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 05 00:09:11.950654 ip-10-0-142-202 kubenswrapper[2563]: W0705 00:09:11.946474    2563 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 05 00:09:11.950654 ip-10-0-142-202 kubenswrapper[2563]: W0705 00:09:11.946476    2563 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 05 00:09:11.950654 ip-10-0-142-202 kubenswrapper[2563]: W0705 00:09:11.946479    2563 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 05 00:09:11.950654 ip-10-0-142-202 kubenswrapper[2563]: W0705 00:09:11.946481    2563 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 05 00:09:11.950654 ip-10-0-142-202 kubenswrapper[2563]: W0705 00:09:11.946484    2563 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 05 00:09:11.950654 ip-10-0-142-202 kubenswrapper[2563]: W0705 00:09:11.946487    2563 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 05 00:09:11.950654 ip-10-0-142-202 kubenswrapper[2563]: W0705 00:09:11.946489    2563 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 05 00:09:11.950654 ip-10-0-142-202 kubenswrapper[2563]: W0705 00:09:11.946492    2563 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 05 00:09:11.950654 ip-10-0-142-202 kubenswrapper[2563]: W0705 00:09:11.946494    2563 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 05 00:09:11.951121 ip-10-0-142-202 kubenswrapper[2563]: W0705 00:09:11.946497    2563 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 05 00:09:11.951121 ip-10-0-142-202 kubenswrapper[2563]: W0705 00:09:11.946499    2563 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 05 00:09:11.951121 ip-10-0-142-202 kubenswrapper[2563]: W0705 00:09:11.946502    2563 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 05 00:09:11.951121 ip-10-0-142-202 kubenswrapper[2563]: W0705 00:09:11.946504    2563 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 05 00:09:11.951121 ip-10-0-142-202 kubenswrapper[2563]: W0705 00:09:11.946507    2563 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 05 00:09:11.951121 ip-10-0-142-202 kubenswrapper[2563]: W0705 00:09:11.946509    2563 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 05 00:09:11.951121 ip-10-0-142-202 kubenswrapper[2563]: W0705 00:09:11.946512    2563 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 05 00:09:11.951121 ip-10-0-142-202 kubenswrapper[2563]: W0705 00:09:11.946514    2563 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 05 00:09:11.951121 ip-10-0-142-202 kubenswrapper[2563]: W0705 00:09:11.946517    2563 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 05 00:09:11.951121 ip-10-0-142-202 kubenswrapper[2563]: W0705 00:09:11.946519    2563 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 05 00:09:11.951121 ip-10-0-142-202 kubenswrapper[2563]: W0705 00:09:11.946522    2563 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 05 00:09:11.951121 ip-10-0-142-202 kubenswrapper[2563]: W0705 00:09:11.946524    2563 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 05 00:09:11.951121 ip-10-0-142-202 kubenswrapper[2563]: W0705 00:09:11.946527    2563 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 05 00:09:11.951121 ip-10-0-142-202 kubenswrapper[2563]: W0705 00:09:11.946530    2563 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 05 00:09:11.951121 ip-10-0-142-202 kubenswrapper[2563]: W0705 00:09:11.946533    2563 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 05 00:09:11.951121 ip-10-0-142-202 kubenswrapper[2563]: W0705 00:09:11.946536    2563 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 05 00:09:11.951121 ip-10-0-142-202 kubenswrapper[2563]: W0705 00:09:11.946539    2563 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 05 00:09:11.951121 ip-10-0-142-202 kubenswrapper[2563]: W0705 00:09:11.946541    2563 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 05 00:09:11.951121 ip-10-0-142-202 kubenswrapper[2563]: W0705 00:09:11.946544    2563 feature_gate.go:328] unrecognized feature gate: Example
Jul 05 00:09:11.951121 ip-10-0-142-202 kubenswrapper[2563]: W0705 00:09:11.946547    2563 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 05 00:09:11.951610 ip-10-0-142-202 kubenswrapper[2563]: W0705 00:09:11.946549    2563 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 05 00:09:11.951610 ip-10-0-142-202 kubenswrapper[2563]: W0705 00:09:11.946552    2563 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 05 00:09:11.951610 ip-10-0-142-202 kubenswrapper[2563]: W0705 00:09:11.946555    2563 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 05 00:09:11.951610 ip-10-0-142-202 kubenswrapper[2563]: W0705 00:09:11.946558    2563 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 05 00:09:11.951610 ip-10-0-142-202 kubenswrapper[2563]: W0705 00:09:11.946562    2563 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 05 00:09:11.951610 ip-10-0-142-202 kubenswrapper[2563]: W0705 00:09:11.946566    2563 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 05 00:09:11.951610 ip-10-0-142-202 kubenswrapper[2563]: W0705 00:09:11.946569    2563 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 05 00:09:11.951610 ip-10-0-142-202 kubenswrapper[2563]: W0705 00:09:11.946571    2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 05 00:09:11.951610 ip-10-0-142-202 kubenswrapper[2563]: W0705 00:09:11.946575    2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 05 00:09:11.951610 ip-10-0-142-202 kubenswrapper[2563]: W0705 00:09:11.946578    2563 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 05 00:09:11.951610 ip-10-0-142-202 kubenswrapper[2563]: W0705 00:09:11.946580    2563 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 05 00:09:11.951610 ip-10-0-142-202 kubenswrapper[2563]: W0705 00:09:11.946583    2563 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 05 00:09:11.951610 ip-10-0-142-202 kubenswrapper[2563]: W0705 00:09:11.946586    2563 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 05 00:09:11.951610 ip-10-0-142-202 kubenswrapper[2563]: W0705 00:09:11.946589    2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 05 00:09:11.951610 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:09:11.946593    2563 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Jul 05 00:09:11.951974 ip-10-0-142-202 kubenswrapper[2563]: W0705 00:09:11.946699    2563 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 05 00:09:11.951974 ip-10-0-142-202 kubenswrapper[2563]: W0705 00:09:11.946703    2563 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 05 00:09:11.951974 ip-10-0-142-202 kubenswrapper[2563]: W0705 00:09:11.946706    2563 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 05 00:09:11.951974 ip-10-0-142-202 kubenswrapper[2563]: W0705 00:09:11.946710    2563 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 05 00:09:11.951974 ip-10-0-142-202 kubenswrapper[2563]: W0705 00:09:11.946713    2563 feature_gate.go:328] unrecognized feature gate: Example
Jul 05 00:09:11.951974 ip-10-0-142-202 kubenswrapper[2563]: W0705 00:09:11.946715    2563 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 05 00:09:11.951974 ip-10-0-142-202 kubenswrapper[2563]: W0705 00:09:11.946718    2563 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 05 00:09:11.951974 ip-10-0-142-202 kubenswrapper[2563]: W0705 00:09:11.946721    2563 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 05 00:09:11.951974 ip-10-0-142-202 kubenswrapper[2563]: W0705 00:09:11.946723    2563 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 05 00:09:11.951974 ip-10-0-142-202 kubenswrapper[2563]: W0705 00:09:11.946726    2563 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 05 00:09:11.951974 ip-10-0-142-202 kubenswrapper[2563]: W0705 00:09:11.946729    2563 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 05 00:09:11.951974 ip-10-0-142-202 kubenswrapper[2563]: W0705 00:09:11.946731    2563 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 05 00:09:11.951974 ip-10-0-142-202 kubenswrapper[2563]: W0705 00:09:11.946734    2563 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 05 00:09:11.951974 ip-10-0-142-202 kubenswrapper[2563]: W0705 00:09:11.946736    2563 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 05 00:09:11.951974 ip-10-0-142-202 kubenswrapper[2563]: W0705 00:09:11.946739    2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 05 00:09:11.951974 ip-10-0-142-202 kubenswrapper[2563]: W0705 00:09:11.946742    2563 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 05 00:09:11.951974 ip-10-0-142-202 kubenswrapper[2563]: W0705 00:09:11.946744    2563 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 05 00:09:11.951974 ip-10-0-142-202 kubenswrapper[2563]: W0705 00:09:11.946747    2563 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 05 00:09:11.951974 ip-10-0-142-202 kubenswrapper[2563]: W0705 00:09:11.946750    2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 05 00:09:11.951974 ip-10-0-142-202 kubenswrapper[2563]: W0705 00:09:11.946752    2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 05 00:09:11.952552 ip-10-0-142-202 kubenswrapper[2563]: W0705 00:09:11.946755    2563 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 05 00:09:11.952552 ip-10-0-142-202 kubenswrapper[2563]: W0705 00:09:11.946757    2563 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 05 00:09:11.952552 ip-10-0-142-202 kubenswrapper[2563]: W0705 00:09:11.946760    2563 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 05 00:09:11.952552 ip-10-0-142-202 kubenswrapper[2563]: W0705 00:09:11.946762    2563 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 05 00:09:11.952552 ip-10-0-142-202 kubenswrapper[2563]: W0705 00:09:11.946765    2563 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 05 00:09:11.952552 ip-10-0-142-202 kubenswrapper[2563]: W0705 00:09:11.946767    2563 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 05 00:09:11.952552 ip-10-0-142-202 kubenswrapper[2563]: W0705 00:09:11.946770    2563 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 05 00:09:11.952552 ip-10-0-142-202 kubenswrapper[2563]: W0705 00:09:11.946772    2563 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 05 00:09:11.952552 ip-10-0-142-202 kubenswrapper[2563]: W0705 00:09:11.946775    2563 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 05 00:09:11.952552 ip-10-0-142-202 kubenswrapper[2563]: W0705 00:09:11.946778    2563 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 05 00:09:11.952552 ip-10-0-142-202 kubenswrapper[2563]: W0705 00:09:11.946781    2563 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 05 00:09:11.952552 ip-10-0-142-202 kubenswrapper[2563]: W0705 00:09:11.946784    2563 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 05 00:09:11.952552 ip-10-0-142-202 kubenswrapper[2563]: W0705 00:09:11.946786    2563 feature_gate.go:328] unrecognized feature gate: Example2
Jul 05 00:09:11.952552 ip-10-0-142-202 kubenswrapper[2563]: W0705 00:09:11.946788    2563 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 05 00:09:11.952552 ip-10-0-142-202 kubenswrapper[2563]: W0705 00:09:11.946791    2563 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 05 00:09:11.952552 ip-10-0-142-202 kubenswrapper[2563]: W0705 00:09:11.946793    2563 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 05 00:09:11.952552 ip-10-0-142-202 kubenswrapper[2563]: W0705 00:09:11.946796    2563 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 05 00:09:11.952552 ip-10-0-142-202 kubenswrapper[2563]: W0705 00:09:11.946798    2563 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 05 00:09:11.952552 ip-10-0-142-202 kubenswrapper[2563]: W0705 00:09:11.946801    2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 05 00:09:11.952552 ip-10-0-142-202 kubenswrapper[2563]: W0705 00:09:11.946803    2563 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 05 00:09:11.953020 ip-10-0-142-202 kubenswrapper[2563]: W0705 00:09:11.946806    2563 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 05 00:09:11.953020 ip-10-0-142-202 kubenswrapper[2563]: W0705 00:09:11.946808    2563 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 05 00:09:11.953020 ip-10-0-142-202 kubenswrapper[2563]: W0705 00:09:11.946811    2563 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 05 00:09:11.953020 ip-10-0-142-202 kubenswrapper[2563]: W0705 00:09:11.946813    2563 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 05 00:09:11.953020 ip-10-0-142-202 kubenswrapper[2563]: W0705 00:09:11.946816    2563 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 05 00:09:11.953020 ip-10-0-142-202 kubenswrapper[2563]: W0705 00:09:11.946818    2563 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 05 00:09:11.953020 ip-10-0-142-202 kubenswrapper[2563]: W0705 00:09:11.946821    2563 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 05 00:09:11.953020 ip-10-0-142-202 kubenswrapper[2563]: W0705 00:09:11.946823    2563 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 05 00:09:11.953020 ip-10-0-142-202 kubenswrapper[2563]: W0705 00:09:11.946825    2563 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 05 00:09:11.953020 ip-10-0-142-202 kubenswrapper[2563]: W0705 00:09:11.946828    2563 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 05 00:09:11.953020 ip-10-0-142-202 kubenswrapper[2563]: W0705 00:09:11.946831    2563 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 05 00:09:11.953020 ip-10-0-142-202 kubenswrapper[2563]: W0705 00:09:11.946833    2563 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 05 00:09:11.953020 ip-10-0-142-202 kubenswrapper[2563]: W0705 00:09:11.946836    2563 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 05 00:09:11.953020 ip-10-0-142-202 kubenswrapper[2563]: W0705 00:09:11.946839    2563 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 05 00:09:11.953020 ip-10-0-142-202 kubenswrapper[2563]: W0705 00:09:11.946843    2563 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 05 00:09:11.953020 ip-10-0-142-202 kubenswrapper[2563]: W0705 00:09:11.946846    2563 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 05 00:09:11.953020 ip-10-0-142-202 kubenswrapper[2563]: W0705 00:09:11.946848    2563 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 05 00:09:11.953020 ip-10-0-142-202 kubenswrapper[2563]: W0705 00:09:11.946851    2563 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 05 00:09:11.953020 ip-10-0-142-202 kubenswrapper[2563]: W0705 00:09:11.946854    2563 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 05 00:09:11.953020 ip-10-0-142-202 kubenswrapper[2563]: W0705 00:09:11.946857    2563 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 05 00:09:11.953535 ip-10-0-142-202 kubenswrapper[2563]: W0705 00:09:11.946859    2563 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 05 00:09:11.953535 ip-10-0-142-202 kubenswrapper[2563]: W0705 00:09:11.946862    2563 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 05 00:09:11.953535 ip-10-0-142-202 kubenswrapper[2563]: W0705 00:09:11.946864    2563 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 05 00:09:11.953535 ip-10-0-142-202 kubenswrapper[2563]: W0705 00:09:11.946866    2563 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 05 00:09:11.953535 ip-10-0-142-202 kubenswrapper[2563]: W0705 00:09:11.946869    2563 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 05 00:09:11.953535 ip-10-0-142-202 kubenswrapper[2563]: W0705 00:09:11.946871    2563 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 05 00:09:11.953535 ip-10-0-142-202 kubenswrapper[2563]: W0705 00:09:11.946874    2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 05 00:09:11.953535 ip-10-0-142-202 kubenswrapper[2563]: W0705 00:09:11.946876    2563 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 05 00:09:11.953535 ip-10-0-142-202 kubenswrapper[2563]: W0705 00:09:11.946879    2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 05 00:09:11.953535 ip-10-0-142-202 kubenswrapper[2563]: W0705 00:09:11.946881    2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 05 00:09:11.953535 ip-10-0-142-202 kubenswrapper[2563]: W0705 00:09:11.946883    2563 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 05 00:09:11.953535 ip-10-0-142-202 kubenswrapper[2563]: W0705 00:09:11.946886    2563 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 05 00:09:11.953535 ip-10-0-142-202 kubenswrapper[2563]: W0705 00:09:11.946888    2563 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 05 00:09:11.953535 ip-10-0-142-202 kubenswrapper[2563]: W0705 00:09:11.946891    2563 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 05 00:09:11.953535 ip-10-0-142-202 kubenswrapper[2563]: W0705 00:09:11.946893    2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 05 00:09:11.953535 ip-10-0-142-202 kubenswrapper[2563]: W0705 00:09:11.946895    2563 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 05 00:09:11.953535 ip-10-0-142-202 kubenswrapper[2563]: W0705 00:09:11.946898    2563 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 05 00:09:11.953535 ip-10-0-142-202 kubenswrapper[2563]: W0705 00:09:11.946901    2563 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 05 00:09:11.953535 ip-10-0-142-202 kubenswrapper[2563]: W0705 00:09:11.946903    2563 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 05 00:09:11.953982 ip-10-0-142-202 kubenswrapper[2563]: W0705 00:09:11.946906    2563 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 05 00:09:11.953982 ip-10-0-142-202 kubenswrapper[2563]: W0705 00:09:11.946908    2563 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 05 00:09:11.953982 ip-10-0-142-202 kubenswrapper[2563]: W0705 00:09:11.946911    2563 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 05 00:09:11.953982 ip-10-0-142-202 kubenswrapper[2563]: W0705 00:09:11.946914    2563 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 05 00:09:11.953982 ip-10-0-142-202 kubenswrapper[2563]: W0705 00:09:11.946916    2563 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 05 00:09:11.953982 ip-10-0-142-202 kubenswrapper[2563]: W0705 00:09:11.946918    2563 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 05 00:09:11.953982 ip-10-0-142-202 kubenswrapper[2563]: W0705 00:09:11.946921    2563 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 05 00:09:11.953982 ip-10-0-142-202 kubenswrapper[2563]: W0705 00:09:11.946923    2563 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 05 00:09:11.953982 ip-10-0-142-202 kubenswrapper[2563]: W0705 00:09:11.946926    2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 05 00:09:11.953982 ip-10-0-142-202 kubenswrapper[2563]: W0705 00:09:11.946929    2563 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 05 00:09:11.953982 ip-10-0-142-202 kubenswrapper[2563]: W0705 00:09:11.946932    2563 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 05 00:09:11.953982 ip-10-0-142-202 kubenswrapper[2563]: W0705 00:09:11.946934    2563 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 05 00:09:11.953982 ip-10-0-142-202 kubenswrapper[2563]: W0705 00:09:11.946936    2563 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 05 00:09:11.953982 ip-10-0-142-202 kubenswrapper[2563]: W0705 00:09:11.946939    2563 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 05 00:09:11.953982 ip-10-0-142-202 kubenswrapper[2563]: W0705 00:09:11.946941    2563 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 05 00:09:11.953982 ip-10-0-142-202 kubenswrapper[2563]: W0705 00:09:11.946944    2563 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 05 00:09:11.953982 ip-10-0-142-202 kubenswrapper[2563]: W0705 00:09:11.946946    2563 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 05 00:09:11.953982 ip-10-0-142-202 kubenswrapper[2563]: W0705 00:09:11.946950    2563 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 05 00:09:11.953982 ip-10-0-142-202 kubenswrapper[2563]: W0705 00:09:11.946953    2563 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 05 00:09:11.954436 ip-10-0-142-202 kubenswrapper[2563]: W0705 00:09:11.946956    2563 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 05 00:09:11.954436 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:09:11.946960    2563 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Jul 05 00:09:11.954436 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:09:11.947053    2563 server.go:962] "Client rotation is on, will bootstrap in background"
Jul 05 00:09:11.954436 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:09:11.949001    2563 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir"
Jul 05 00:09:11.954436 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:09:11.950297    2563 server.go:1019] "Starting client certificate rotation"
Jul 05 00:09:11.954436 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:09:11.950392    2563 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet"
Jul 05 00:09:11.954436 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:09:11.950423    2563 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet"
Jul 05 00:09:11.954436 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:09:11.953297    2563 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Jul 05 00:09:11.954436 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:09:11.954403    2563 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Jul 05 00:09:11.966824 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:09:11.966804    2563 log.go:25] "Validated CRI v1 runtime API"
Jul 05 00:09:11.968694 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:09:11.968681    2563 log.go:25] "Validated CRI v1 image API"
Jul 05 00:09:11.971722 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:09:11.971704    2563 server.go:1429] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd"
Jul 05 00:09:11.973840 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:09:11.973823    2563 fs.go:135] Filesystem UUIDs: map[7B77-95E7:/dev/nvme0n1p2 b2942e4c-d553-4bc6-8b11-b46d2150b879:/dev/nvme0n1p4 fb66b458-f679-438c-aa6d-3105b76fa73b:/dev/nvme0n1p3]
Jul 05 00:09:11.973896 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:09:11.973840    2563 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}]
Jul 05 00:09:11.979581 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:09:11.979471    2563 manager.go:216] Machine: {Timestamp:2026-07-05 00:09:11.977394817 +0000 UTC m=+0.319723867 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3098507 MemoryCapacity:33164500992 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec2950c45efac41b2795ff142fa41a2e SystemUUID:ec2950c4-5efa-c41b-2795-ff142fa41a2e BootID:30082018-5d66-4c3e-869d-25b387dc3e53 Filesystems:[{Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16582250496 Type:vfs Inodes:4048401 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6632902656 Type:vfs Inodes:819200 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6107136 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16582250496 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:9e:54:86:0e:bb Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:9e:54:86:0e:bb Speed:0 Mtu:9001} {Name:ovs-system MacAddress:f6:b0:2a:18:66:9d Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:33164500992 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:37486592 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None}
Jul 05 00:09:11.979581 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:09:11.979575    2563 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available.
Jul 05 00:09:11.979714 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:09:11.979645    2563 manager.go:232] Version: {KernelVersion:5.14.0-570.125.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260630-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:}
Jul 05 00:09:11.982592 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:09:11.982562    2563 container_manager_linux.go:273] "Container manager verified user specified cgroup-root exists" cgroupRoot=[]
Jul 05 00:09:11.982726 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:09:11.982597    2563 container_manager_linux.go:278] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-142-202.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2}
Jul 05 00:09:11.983269 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:09:11.983259    2563 topology_manager.go:138] "Creating topology manager with none policy"
Jul 05 00:09:11.983300 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:09:11.983272    2563 container_manager_linux.go:309] "Creating device plugin manager"
Jul 05 00:09:11.983300 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:09:11.983280    2563 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock"
Jul 05 00:09:11.984694 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:09:11.984680    2563 reflector.go:439] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Jul 05 00:09:11.984766 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:09:11.984752    2563 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock"
Jul 05 00:09:11.986032 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:09:11.986021    2563 container_manager_linux.go:318] "Creating Dynamic Resource Allocation (DRA) manager"
Jul 05 00:09:11.988853 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:09:11.988841    2563 state_mem.go:36] "Initialized new in-memory state store"
Jul 05 00:09:11.988970 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:09:11.988960    2563 server.go:1240] "Using root directory" path="/var/lib/kubelet"
Jul 05 00:09:11.989009 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:09:11.988990    2563 kubelet.go:486] "Attempting to sync node with API server"
Jul 05 00:09:11.989009 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:09:11.989006    2563 kubelet.go:387] "Adding static pod path" path="/etc/kubernetes/manifests"
Jul 05 00:09:11.989080 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:09:11.989020    2563 file.go:69] "Watching path" path="/etc/kubernetes/manifests"
Jul 05 00:09:11.989080 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:09:11.989029    2563 kubelet.go:398] "Adding apiserver pod source"
Jul 05 00:09:11.989080 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:09:11.989038    2563 apiserver.go:42] "Waiting for node sync before watching apiserver pods"
Jul 05 00:09:11.989395 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:09:11.989383    2563 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Jul 05 00:09:11.989438 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:09:11.989406    2563 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Jul 05 00:09:11.991884 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:09:11.991860    2563 kuberuntime_manager.go:301] "Container runtime initialized" containerRuntime="cri-o" version="1.34.8-3.rhaos4.21.git91b4b0b.el9" apiVersion="v1"
Jul 05 00:09:11.993778 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:09:11.993761    2563 kubelet.go:960] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled"
Jul 05 00:09:11.993831 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:09:11.993794    2563 kubelet.go:984] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled"
Jul 05 00:09:11.994809 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:09:11.994796    2563 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume"
Jul 05 00:09:11.994853 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:09:11.994815    2563 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir"
Jul 05 00:09:11.994853 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:09:11.994826    2563 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo"
Jul 05 00:09:11.994853 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:09:11.994839    2563 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path"
Jul 05 00:09:11.994853 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:09:11.994850    2563 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs"
Jul 05 00:09:11.994956 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:09:11.994864    2563 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret"
Jul 05 00:09:11.994956 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:09:11.994872    2563 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi"
Jul 05 00:09:11.994956 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:09:11.994877    2563 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api"
Jul 05 00:09:11.994956 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:09:11.994890    2563 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc"
Jul 05 00:09:11.994956 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:09:11.994896    2563 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap"
Jul 05 00:09:11.994956 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:09:11.994908    2563 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected"
Jul 05 00:09:11.994956 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:09:11.994916    2563 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume"
Jul 05 00:09:11.995662 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:09:11.995651    2563 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi"
Jul 05 00:09:11.995662 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:09:11.995661    2563 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image"
Jul 05 00:09:11.998693 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:09:11.998678    2563 server.go:1268] "Started kubelet"
Jul 05 00:09:11.998950 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:09:11.998783    2563 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10
Jul 05 00:09:11.998950 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:09:11.998908    2563 server_v1.go:49] "podresources" method="list" useActivePods=true
Jul 05 00:09:11.999071 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:09:11.999045    2563 server.go:180] "Starting to listen" address="0.0.0.0" port=10250
Jul 05 00:09:12.002508 ip-10-0-142-202 systemd[1]: Started Kubernetes Kubelet.
Jul 05 00:09:12.002725 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:09:12.002595    2563 server.go:249] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock"
Jul 05 00:09:12.005453 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:09:12.005426    2563 server.go:310] "Adding debug handlers to kubelet server"
Jul 05 00:09:12.006344 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:09:12.006323    2563 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-4kjd8"
Jul 05 00:09:12.006466 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:09:12.006437    2563 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-142-202.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope
Jul 05 00:09:12.006869 ip-10-0-142-202 kubenswrapper[2563]: E0705 00:09:12.006852    2563 kubelet.go:1638] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache"
Jul 05 00:09:12.007165 ip-10-0-142-202 kubenswrapper[2563]: E0705 00:09:12.007145    2563 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service"
Jul 05 00:09:12.007212 ip-10-0-142-202 kubenswrapper[2563]: E0705 00:09:12.007144    2563 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-142-202.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node"
Jul 05 00:09:12.009913 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:09:12.009894    2563 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving"
Jul 05 00:09:12.009988 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:09:12.009898    2563 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer"
Jul 05 00:09:12.010340 ip-10-0-142-202 kubenswrapper[2563]: E0705 00:09:12.010010    2563 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-142-202.ec2.internal\" not found"
Jul 05 00:09:12.010469 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:09:12.010454    2563 volume_manager.go:311] "The desired_state_of_world populator starts"
Jul 05 00:09:12.010588 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:09:12.010577    2563 volume_manager.go:313] "Starting Kubelet Volume Manager"
Jul 05 00:09:12.010748 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:09:12.010734    2563 desired_state_of_world_populator.go:146] "Desired state populator starts to run"
Jul 05 00:09:12.010820 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:09:12.010814    2563 reconstruct.go:97] "Volume reconstruction finished"
Jul 05 00:09:12.010870 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:09:12.010821    2563 reconciler.go:29] "Reconciler: start to sync state"
Jul 05 00:09:12.010957 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:09:12.010939    2563 factory.go:55] Registering systemd factory
Jul 05 00:09:12.010957 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:09:12.010959    2563 factory.go:223] Registration of the systemd container factory successfully
Jul 05 00:09:12.011158 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:09:12.011146    2563 factory.go:153] Registering CRI-O factory
Jul 05 00:09:12.011158 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:09:12.011159    2563 factory.go:223] Registration of the crio container factory successfully
Jul 05 00:09:12.011287 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:09:12.011213    2563 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory
Jul 05 00:09:12.011287 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:09:12.011247    2563 factory.go:103] Registering Raw factory
Jul 05 00:09:12.011287 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:09:12.011257    2563 manager.go:1196] Started watching for new ooms in manager
Jul 05 00:09:12.011692 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:09:12.011677    2563 manager.go:349] Starting recovery of all containers
Jul 05 00:09:12.013262 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:09:12.013244    2563 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-4kjd8"
Jul 05 00:09:12.019334 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:09:12.019313    2563 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 05 00:09:12.022663 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:09:12.022647    2563 manager.go:354] Recovery completed
Jul 05 00:09:12.026578 ip-10-0-142-202 kubenswrapper[2563]: E0705 00:09:12.026560    2563 nodelease.go:49] "Failed to get node when trying to set owner ref to the node lease" err="nodes \"ip-10-0-142-202.ec2.internal\" not found" node="ip-10-0-142-202.ec2.internal"
Jul 05 00:09:12.027204 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:09:12.027192    2563 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 05 00:09:12.028928 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:09:12.028912    2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-202.ec2.internal" event="NodeHasSufficientMemory"
Jul 05 00:09:12.029015 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:09:12.028938    2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-202.ec2.internal" event="NodeHasNoDiskPressure"
Jul 05 00:09:12.029015 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:09:12.028952    2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-202.ec2.internal" event="NodeHasSufficientPID"
Jul 05 00:09:12.029861 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:09:12.029848    2563 cpu_manager.go:222] "Starting CPU manager" policy="none"
Jul 05 00:09:12.029861 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:09:12.029856    2563 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 05 00:09:12.029861 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:09:12.029860    2563 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s"
Jul 05 00:09:12.029987 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:09:12.029875    2563 state_mem.go:36] "Initialized new in-memory state store"
Jul 05 00:09:12.030503 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:09:12.030485    2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-202.ec2.internal" event="NodeHasSufficientMemory"
Jul 05 00:09:12.030585 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:09:12.030513    2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-202.ec2.internal" event="NodeHasNoDiskPressure"
Jul 05 00:09:12.030585 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:09:12.030527    2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-202.ec2.internal" event="NodeHasSufficientPID"
Jul 05 00:09:12.030926 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:09:12.030912    2563 policy_none.go:49] "None policy: Start"
Jul 05 00:09:12.030969 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:09:12.030932    2563 memory_manager.go:187] "Starting memorymanager" policy="None"
Jul 05 00:09:12.030969 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:09:12.030947    2563 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint"
Jul 05 00:09:12.031890 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:09:12.031877    2563 policy_none.go:47] "Start"
Jul 05 00:09:12.070901 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:09:12.070781    2563 manager.go:337] "Starting Device Plugin manager"
Jul 05 00:09:12.070996 ip-10-0-142-202 kubenswrapper[2563]: E0705 00:09:12.070917    2563 manager.go:513] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint"
Jul 05 00:09:12.070996 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:09:12.070929    2563 server.go:87] "Starting device plugin registration server"
Jul 05 00:09:12.071174 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:09:12.071161    2563 eviction_manager.go:189] "Eviction manager: starting control loop"
Jul 05 00:09:12.071248 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:09:12.071176    2563 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s"
Jul 05 00:09:12.071303 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:09:12.071288    2563 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry"
Jul 05 00:09:12.071404 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:09:12.071366    2563 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts"
Jul 05 00:09:12.071404 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:09:12.071375    2563 plugin_manager.go:118] "Starting Kubelet Plugin Manager"
Jul 05 00:09:12.071795 ip-10-0-142-202 kubenswrapper[2563]: E0705 00:09:12.071775    2563 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\""
Jul 05 00:09:12.071862 ip-10-0-142-202 kubenswrapper[2563]: E0705 00:09:12.071827    2563 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-142-202.ec2.internal\" not found"
Jul 05 00:09:12.118944 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:09:12.118916    2563 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4"
Jul 05 00:09:12.120600 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:09:12.120062    2563 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6"
Jul 05 00:09:12.120600 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:09:12.120094    2563 status_manager.go:244] "Starting to sync pod status with apiserver"
Jul 05 00:09:12.120600 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:09:12.120112    2563 kubelet.go:2448] "Starting kubelet main sync loop"
Jul 05 00:09:12.120600 ip-10-0-142-202 kubenswrapper[2563]: E0705 00:09:12.120140    2563 kubelet.go:2472] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful"
Jul 05 00:09:12.122503 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:09:12.122486    2563 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 05 00:09:12.171611 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:09:12.171580    2563 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 05 00:09:12.172408 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:09:12.172392    2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-202.ec2.internal" event="NodeHasSufficientMemory"
Jul 05 00:09:12.172484 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:09:12.172421    2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-202.ec2.internal" event="NodeHasNoDiskPressure"
Jul 05 00:09:12.172484 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:09:12.172434    2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-202.ec2.internal" event="NodeHasSufficientPID"
Jul 05 00:09:12.172484 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:09:12.172457    2563 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-142-202.ec2.internal"
Jul 05 00:09:12.181712 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:09:12.181673    2563 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-142-202.ec2.internal"
Jul 05 00:09:12.181712 ip-10-0-142-202 kubenswrapper[2563]: E0705 00:09:12.181693    2563 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-142-202.ec2.internal\": node \"ip-10-0-142-202.ec2.internal\" not found"
Jul 05 00:09:12.194660 ip-10-0-142-202 kubenswrapper[2563]: E0705 00:09:12.194643    2563 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-142-202.ec2.internal\" not found"
Jul 05 00:09:12.221261 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:09:12.221229    2563 kubelet.go:2535] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-202.ec2.internal","kube-system/kube-apiserver-proxy-ip-10-0-142-202.ec2.internal"]
Jul 05 00:09:12.221357 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:09:12.221315    2563 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 05 00:09:12.222124 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:09:12.222111    2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-202.ec2.internal" event="NodeHasSufficientMemory"
Jul 05 00:09:12.222177 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:09:12.222136    2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-202.ec2.internal" event="NodeHasNoDiskPressure"
Jul 05 00:09:12.222177 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:09:12.222147    2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-202.ec2.internal" event="NodeHasSufficientPID"
Jul 05 00:09:12.223301 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:09:12.223289    2563 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 05 00:09:12.223409 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:09:12.223397    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-202.ec2.internal"
Jul 05 00:09:12.223457 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:09:12.223422    2563 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 05 00:09:12.223952 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:09:12.223938    2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-202.ec2.internal" event="NodeHasSufficientMemory"
Jul 05 00:09:12.223952 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:09:12.223947    2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-202.ec2.internal" event="NodeHasSufficientMemory"
Jul 05 00:09:12.224106 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:09:12.223968    2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-202.ec2.internal" event="NodeHasNoDiskPressure"
Jul 05 00:09:12.224106 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:09:12.223979    2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-202.ec2.internal" event="NodeHasSufficientPID"
Jul 05 00:09:12.224106 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:09:12.223969    2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-202.ec2.internal" event="NodeHasNoDiskPressure"
Jul 05 00:09:12.224106 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:09:12.224016    2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-202.ec2.internal" event="NodeHasSufficientPID"
Jul 05 00:09:12.225326 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:09:12.225310    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-142-202.ec2.internal"
Jul 05 00:09:12.225408 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:09:12.225342    2563 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 05 00:09:12.225951 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:09:12.225934    2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-202.ec2.internal" event="NodeHasSufficientMemory"
Jul 05 00:09:12.226035 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:09:12.225965    2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-202.ec2.internal" event="NodeHasNoDiskPressure"
Jul 05 00:09:12.226035 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:09:12.225981    2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-202.ec2.internal" event="NodeHasSufficientPID"
Jul 05 00:09:12.248775 ip-10-0-142-202 kubenswrapper[2563]: E0705 00:09:12.248754    2563 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-142-202.ec2.internal\" not found" node="ip-10-0-142-202.ec2.internal"
Jul 05 00:09:12.252694 ip-10-0-142-202 kubenswrapper[2563]: E0705 00:09:12.252680    2563 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-142-202.ec2.internal\" not found" node="ip-10-0-142-202.ec2.internal"
Jul 05 00:09:12.295277 ip-10-0-142-202 kubenswrapper[2563]: E0705 00:09:12.295259    2563 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-142-202.ec2.internal\" not found"
Jul 05 00:09:12.312823 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:09:12.312800    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/34a0b5b76f9b2c1d92188a84f4deb896-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-142-202.ec2.internal\" (UID: \"34a0b5b76f9b2c1d92188a84f4deb896\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-202.ec2.internal"
Jul 05 00:09:12.312889 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:09:12.312828    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/34a0b5b76f9b2c1d92188a84f4deb896-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-142-202.ec2.internal\" (UID: \"34a0b5b76f9b2c1d92188a84f4deb896\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-202.ec2.internal"
Jul 05 00:09:12.312889 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:09:12.312846    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/ec580e48b6696694176fa0643f6420e5-config\") pod \"kube-apiserver-proxy-ip-10-0-142-202.ec2.internal\" (UID: \"ec580e48b6696694176fa0643f6420e5\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-142-202.ec2.internal"
Jul 05 00:09:12.396031 ip-10-0-142-202 kubenswrapper[2563]: E0705 00:09:12.396001    2563 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-142-202.ec2.internal\" not found"
Jul 05 00:09:12.413564 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:09:12.413516    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/ec580e48b6696694176fa0643f6420e5-config\") pod \"kube-apiserver-proxy-ip-10-0-142-202.ec2.internal\" (UID: \"ec580e48b6696694176fa0643f6420e5\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-142-202.ec2.internal"
Jul 05 00:09:12.413564 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:09:12.413569    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/34a0b5b76f9b2c1d92188a84f4deb896-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-142-202.ec2.internal\" (UID: \"34a0b5b76f9b2c1d92188a84f4deb896\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-202.ec2.internal"
Jul 05 00:09:12.413734 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:09:12.413589    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/34a0b5b76f9b2c1d92188a84f4deb896-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-142-202.ec2.internal\" (UID: \"34a0b5b76f9b2c1d92188a84f4deb896\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-202.ec2.internal"
Jul 05 00:09:12.413734 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:09:12.413626    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/34a0b5b76f9b2c1d92188a84f4deb896-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-142-202.ec2.internal\" (UID: \"34a0b5b76f9b2c1d92188a84f4deb896\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-202.ec2.internal"
Jul 05 00:09:12.413734 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:09:12.413631    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/34a0b5b76f9b2c1d92188a84f4deb896-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-142-202.ec2.internal\" (UID: \"34a0b5b76f9b2c1d92188a84f4deb896\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-202.ec2.internal"
Jul 05 00:09:12.413734 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:09:12.413634    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/ec580e48b6696694176fa0643f6420e5-config\") pod \"kube-apiserver-proxy-ip-10-0-142-202.ec2.internal\" (UID: \"ec580e48b6696694176fa0643f6420e5\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-142-202.ec2.internal"
Jul 05 00:09:12.496818 ip-10-0-142-202 kubenswrapper[2563]: E0705 00:09:12.496747    2563 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-142-202.ec2.internal\" not found"
Jul 05 00:09:12.553464 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:09:12.553420    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-202.ec2.internal"
Jul 05 00:09:12.555772 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:09:12.555757    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-142-202.ec2.internal"
Jul 05 00:09:12.597491 ip-10-0-142-202 kubenswrapper[2563]: E0705 00:09:12.597456    2563 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-142-202.ec2.internal\" not found"
Jul 05 00:09:12.697591 ip-10-0-142-202 kubenswrapper[2563]: E0705 00:09:12.697556    2563 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-142-202.ec2.internal\" not found"
Jul 05 00:09:12.799196 ip-10-0-142-202 kubenswrapper[2563]: E0705 00:09:12.799120    2563 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-142-202.ec2.internal\" not found"
Jul 05 00:09:12.830131 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:09:12.830107    2563 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 05 00:09:12.910610 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:09:12.910583    2563 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-142-202.ec2.internal"
Jul 05 00:09:12.920881 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:09:12.920856    2563 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Jul 05 00:09:12.922486 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:09:12.922473    2563 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-202.ec2.internal"
Jul 05 00:09:12.932210 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:09:12.932193    2563 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Jul 05 00:09:12.950706 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:09:12.950685    2563 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials"
Jul 05 00:09:12.950823 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:09:12.950804    2563 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received"
Jul 05 00:09:12.950881 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:09:12.950850    2563 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received"
Jul 05 00:09:12.950881 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:09:12.950847    2563 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received"
Jul 05 00:09:12.992152 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:09:12.992127    2563 apiserver.go:52] "Watching apiserver"
Jul 05 00:09:13.008227 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:09:13.008201    2563 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66"
Jul 05 00:09:13.011100 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:09:13.011001    2563 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving"
Jul 05 00:09:13.011494 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:09:13.011471    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-bj4b9","openshift-cluster-node-tuning-operator/tuned-r2vw8","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-202.ec2.internal","openshift-network-diagnostics/network-check-target-4pm8c","openshift-network-operator/iptables-alerter-8tjwr","openshift-ovn-kubernetes/ovnkube-node-7sxq4","kube-system/global-pull-secret-syncer-dkwx5","kube-system/konnectivity-agent-rwbw4","kube-system/kube-apiserver-proxy-ip-10-0-142-202.ec2.internal","openshift-dns/node-resolver-kjz9w","openshift-image-registry/node-ca-s547v","openshift-multus/multus-additional-cni-plugins-g568x","openshift-multus/multus-kwgzc","openshift-multus/network-metrics-daemon-5fn94"]
Jul 05 00:09:13.013068 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:09:13.013050    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-bj4b9"
Jul 05 00:09:13.014122 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:09:13.014106    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-r2vw8"
Jul 05 00:09:13.015042 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:09:13.015024    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\""
Jul 05 00:09:13.015240 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:09:13.015207    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-4pm8c"
Jul 05 00:09:13.015305 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:09:13.015271    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\""
Jul 05 00:09:13.015305 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:09:13.015277    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\""
Jul 05 00:09:13.015412 ip-10-0-142-202 kubenswrapper[2563]: E0705 00:09:13.015295    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-4pm8c" podUID="e84aff2e-272a-42d8-8bd9-e37a98cfc7a7"
Jul 05 00:09:13.015412 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:09:13.015309    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-r6c8s\""
Jul 05 00:09:13.015848 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:09:13.015832    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\""
Jul 05 00:09:13.015928 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:09:13.015859    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-7l7bt\""
Jul 05 00:09:13.016054 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:09:13.016021    2563 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-07-04 00:04:12 +0000 UTC" deadline="2027-12-17 00:45:20.288464195 +0000 UTC"
Jul 05 00:09:13.016112 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:09:13.016054    2563 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="12720h36m7.272412413s"
Jul 05 00:09:13.016223 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:09:13.016207    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\""
Jul 05 00:09:13.016360 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:09:13.016345    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-8tjwr"
Jul 05 00:09:13.018065 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:09:13.018049    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-7sxq4"
Jul 05 00:09:13.018282 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:09:13.018259    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\""
Jul 05 00:09:13.018633 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:09:13.018607    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\""
Jul 05 00:09:13.018747 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:09:13.018712    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\""
Jul 05 00:09:13.018849 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:09:13.018832    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-czjkq\""
Jul 05 00:09:13.019177 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:09:13.019165    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-dkwx5"
Jul 05 00:09:13.019280 ip-10-0-142-202 kubenswrapper[2563]: E0705 00:09:13.019252    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-dkwx5" podUID="00bb9c2a-9bed-4d69-9ae4-e4f1fad50578"
Jul 05 00:09:13.020134 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:09:13.020113    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-zkpmw\""
Jul 05 00:09:13.020364 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:09:13.020344    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\""
Jul 05 00:09:13.020364 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:09:13.020357    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\""
Jul 05 00:09:13.020501 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:09:13.020382    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\""
Jul 05 00:09:13.020553 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:09:13.020520    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\""
Jul 05 00:09:13.020600 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:09:13.020589    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\""
Jul 05 00:09:13.020644 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:09:13.020613    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\""
Jul 05 00:09:13.020644 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:09:13.020636    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-rwbw4"
Jul 05 00:09:13.021815 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:09:13.021798    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-kjz9w"
Jul 05 00:09:13.022218 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:09:13.022195    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\""
Jul 05 00:09:13.022343 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:09:13.022325    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-ht57q\""
Jul 05 00:09:13.022524 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:09:13.022510    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\""
Jul 05 00:09:13.023273 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:09:13.023257    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-mlzqh\""
Jul 05 00:09:13.023342 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:09:13.023262    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\""
Jul 05 00:09:13.023403 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:09:13.023389    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\""
Jul 05 00:09:13.023534 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:09:13.023520    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-s547v"
Jul 05 00:09:13.024898 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:09:13.024862    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-g568x"
Jul 05 00:09:13.025693 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:09:13.025678    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\""
Jul 05 00:09:13.025775 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:09:13.025746    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\""
Jul 05 00:09:13.025775 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:09:13.025766    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\""
Jul 05 00:09:13.025864 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:09:13.025812    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-z47bv\""
Jul 05 00:09:13.026328 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:09:13.026283    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-kwgzc"
Jul 05 00:09:13.026533 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:09:13.026514    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\""
Jul 05 00:09:13.026643 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:09:13.026574    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-m9nfv\""
Jul 05 00:09:13.026724 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:09:13.026709    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\""
Jul 05 00:09:13.026724 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:09:13.026718    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\""
Jul 05 00:09:13.026811 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:09:13.026771    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\""
Jul 05 00:09:13.027087 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:09:13.027070    2563 reflector.go:439] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Jul 05 00:09:13.027137 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:09:13.027106    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\""
Jul 05 00:09:13.027582 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:09:13.027564    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-5fn94"
Jul 05 00:09:13.027704 ip-10-0-142-202 kubenswrapper[2563]: E0705 00:09:13.027632    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-5fn94" podUID="c085d4be-fcda-4931-a018-02cbaca17c59"
Jul 05 00:09:13.027837 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:09:13.027808    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\""
Jul 05 00:09:13.028033 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:09:13.028018    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-c7flq\""
Jul 05 00:09:13.042598 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:09:13.042579    2563 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 05 00:09:13.046110 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:09:13.046093    2563 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-r8sqs"
Jul 05 00:09:13.055378 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:09:13.055304    2563 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-r8sqs"
Jul 05 00:09:13.112125 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:09:13.112104    2563 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world"
Jul 05 00:09:13.116884 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:09:13.116863    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/5b9038ef-5e8c-4ee8-9bb7-e9c249715b79-var-lib-kubelet\") pod \"tuned-r2vw8\" (UID: \"5b9038ef-5e8c-4ee8-9bb7-e9c249715b79\") " pod="openshift-cluster-node-tuning-operator/tuned-r2vw8"
Jul 05 00:09:13.117121 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:09:13.116889    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vntcz\" (UniqueName: \"kubernetes.io/projected/5b9038ef-5e8c-4ee8-9bb7-e9c249715b79-kube-api-access-vntcz\") pod \"tuned-r2vw8\" (UID: \"5b9038ef-5e8c-4ee8-9bb7-e9c249715b79\") " pod="openshift-cluster-node-tuning-operator/tuned-r2vw8"
Jul 05 00:09:13.117121 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:09:13.116908    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/8a2c0b38-b2ce-4570-8963-2f75863cc6b2-iptables-alerter-script\") pod \"iptables-alerter-8tjwr\" (UID: \"8a2c0b38-b2ce-4570-8963-2f75863cc6b2\") " pod="openshift-network-operator/iptables-alerter-8tjwr"
Jul 05 00:09:13.117121 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:09:13.116932    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/b96bfb4d-2fd3-4be3-9064-e24efeb904f4-systemd-units\") pod \"ovnkube-node-7sxq4\" (UID: \"b96bfb4d-2fd3-4be3-9064-e24efeb904f4\") " pod="openshift-ovn-kubernetes/ovnkube-node-7sxq4"
Jul 05 00:09:13.117121 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:09:13.116953    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/b96bfb4d-2fd3-4be3-9064-e24efeb904f4-node-log\") pod \"ovnkube-node-7sxq4\" (UID: \"b96bfb4d-2fd3-4be3-9064-e24efeb904f4\") " pod="openshift-ovn-kubernetes/ovnkube-node-7sxq4"
Jul 05 00:09:13.117121 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:09:13.116971    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/b96bfb4d-2fd3-4be3-9064-e24efeb904f4-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-7sxq4\" (UID: \"b96bfb4d-2fd3-4be3-9064-e24efeb904f4\") " pod="openshift-ovn-kubernetes/ovnkube-node-7sxq4"
Jul 05 00:09:13.117121 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:09:13.116998    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/b96bfb4d-2fd3-4be3-9064-e24efeb904f4-ovn-node-metrics-cert\") pod \"ovnkube-node-7sxq4\" (UID: \"b96bfb4d-2fd3-4be3-9064-e24efeb904f4\") " pod="openshift-ovn-kubernetes/ovnkube-node-7sxq4"
Jul 05 00:09:13.117121 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:09:13.117041    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/aeaa1e13-243b-4c7f-8045-85ba45081c42-system-cni-dir\") pod \"multus-additional-cni-plugins-g568x\" (UID: \"aeaa1e13-243b-4c7f-8045-85ba45081c42\") " pod="openshift-multus/multus-additional-cni-plugins-g568x"
Jul 05 00:09:13.117121 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:09:13.117073    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/aa4848f6-1480-4acd-af73-e8e055306078-os-release\") pod \"multus-kwgzc\" (UID: \"aa4848f6-1480-4acd-af73-e8e055306078\") " pod="openshift-multus/multus-kwgzc"
Jul 05 00:09:13.117121 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:09:13.117097    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/aeaa1e13-243b-4c7f-8045-85ba45081c42-cnibin\") pod \"multus-additional-cni-plugins-g568x\" (UID: \"aeaa1e13-243b-4c7f-8045-85ba45081c42\") " pod="openshift-multus/multus-additional-cni-plugins-g568x"
Jul 05 00:09:13.117647 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:09:13.117143    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/aeaa1e13-243b-4c7f-8045-85ba45081c42-cni-binary-copy\") pod \"multus-additional-cni-plugins-g568x\" (UID: \"aeaa1e13-243b-4c7f-8045-85ba45081c42\") " pod="openshift-multus/multus-additional-cni-plugins-g568x"
Jul 05 00:09:13.117647 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:09:13.117190    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hncnr\" (UniqueName: \"kubernetes.io/projected/aeaa1e13-243b-4c7f-8045-85ba45081c42-kube-api-access-hncnr\") pod \"multus-additional-cni-plugins-g568x\" (UID: \"aeaa1e13-243b-4c7f-8045-85ba45081c42\") " pod="openshift-multus/multus-additional-cni-plugins-g568x"
Jul 05 00:09:13.117647 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:09:13.117220    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/aa4848f6-1480-4acd-af73-e8e055306078-host-var-lib-cni-multus\") pod \"multus-kwgzc\" (UID: \"aa4848f6-1480-4acd-af73-e8e055306078\") " pod="openshift-multus/multus-kwgzc"
Jul 05 00:09:13.117647 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:09:13.117245    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/91e13b36-f504-4b21-b8ae-3b66858bddf2-sys-fs\") pod \"aws-ebs-csi-driver-node-bj4b9\" (UID: \"91e13b36-f504-4b21-b8ae-3b66858bddf2\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-bj4b9"
Jul 05 00:09:13.117647 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:09:13.117288    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/5b9038ef-5e8c-4ee8-9bb7-e9c249715b79-etc-kubernetes\") pod \"tuned-r2vw8\" (UID: \"5b9038ef-5e8c-4ee8-9bb7-e9c249715b79\") " pod="openshift-cluster-node-tuning-operator/tuned-r2vw8"
Jul 05 00:09:13.117647 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:09:13.117312    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/5b9038ef-5e8c-4ee8-9bb7-e9c249715b79-tmp\") pod \"tuned-r2vw8\" (UID: \"5b9038ef-5e8c-4ee8-9bb7-e9c249715b79\") " pod="openshift-cluster-node-tuning-operator/tuned-r2vw8"
Jul 05 00:09:13.117647 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:09:13.117336    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-z4cn8\" (UniqueName: \"kubernetes.io/projected/e84aff2e-272a-42d8-8bd9-e37a98cfc7a7-kube-api-access-z4cn8\") pod \"network-check-target-4pm8c\" (UID: \"e84aff2e-272a-42d8-8bd9-e37a98cfc7a7\") " pod="openshift-network-diagnostics/network-check-target-4pm8c"
Jul 05 00:09:13.117647 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:09:13.117377    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/b96bfb4d-2fd3-4be3-9064-e24efeb904f4-host-run-ovn-kubernetes\") pod \"ovnkube-node-7sxq4\" (UID: \"b96bfb4d-2fd3-4be3-9064-e24efeb904f4\") " pod="openshift-ovn-kubernetes/ovnkube-node-7sxq4"
Jul 05 00:09:13.117647 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:09:13.117412    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gz6zl\" (UniqueName: \"kubernetes.io/projected/b96bfb4d-2fd3-4be3-9064-e24efeb904f4-kube-api-access-gz6zl\") pod \"ovnkube-node-7sxq4\" (UID: \"b96bfb4d-2fd3-4be3-9064-e24efeb904f4\") " pod="openshift-ovn-kubernetes/ovnkube-node-7sxq4"
Jul 05 00:09:13.117647 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:09:13.117433    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/d3a9b78d-cf60-4acc-a3c5-016ee34903f8-host\") pod \"node-ca-s547v\" (UID: \"d3a9b78d-cf60-4acc-a3c5-016ee34903f8\") " pod="openshift-image-registry/node-ca-s547v"
Jul 05 00:09:13.117647 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:09:13.117451    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/d3a9b78d-cf60-4acc-a3c5-016ee34903f8-serviceca\") pod \"node-ca-s547v\" (UID: \"d3a9b78d-cf60-4acc-a3c5-016ee34903f8\") " pod="openshift-image-registry/node-ca-s547v"
Jul 05 00:09:13.117647 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:09:13.117466    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/aeaa1e13-243b-4c7f-8045-85ba45081c42-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-g568x\" (UID: \"aeaa1e13-243b-4c7f-8045-85ba45081c42\") " pod="openshift-multus/multus-additional-cni-plugins-g568x"
Jul 05 00:09:13.117647 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:09:13.117484    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/5b9038ef-5e8c-4ee8-9bb7-e9c249715b79-etc-systemd\") pod \"tuned-r2vw8\" (UID: \"5b9038ef-5e8c-4ee8-9bb7-e9c249715b79\") " pod="openshift-cluster-node-tuning-operator/tuned-r2vw8"
Jul 05 00:09:13.117647 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:09:13.117502    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/aeaa1e13-243b-4c7f-8045-85ba45081c42-os-release\") pod \"multus-additional-cni-plugins-g568x\" (UID: \"aeaa1e13-243b-4c7f-8045-85ba45081c42\") " pod="openshift-multus/multus-additional-cni-plugins-g568x"
Jul 05 00:09:13.117647 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:09:13.117519    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/aeaa1e13-243b-4c7f-8045-85ba45081c42-tuning-conf-dir\") pod \"multus-additional-cni-plugins-g568x\" (UID: \"aeaa1e13-243b-4c7f-8045-85ba45081c42\") " pod="openshift-multus/multus-additional-cni-plugins-g568x"
Jul 05 00:09:13.117647 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:09:13.117541    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/aa4848f6-1480-4acd-af73-e8e055306078-host-run-k8s-cni-cncf-io\") pod \"multus-kwgzc\" (UID: \"aa4848f6-1480-4acd-af73-e8e055306078\") " pod="openshift-multus/multus-kwgzc"
Jul 05 00:09:13.118482 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:09:13.117561    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/aa4848f6-1480-4acd-af73-e8e055306078-host-var-lib-cni-bin\") pod \"multus-kwgzc\" (UID: \"aa4848f6-1480-4acd-af73-e8e055306078\") " pod="openshift-multus/multus-kwgzc"
Jul 05 00:09:13.118482 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:09:13.117580    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/aa4848f6-1480-4acd-af73-e8e055306078-multus-daemon-config\") pod \"multus-kwgzc\" (UID: \"aa4848f6-1480-4acd-af73-e8e055306078\") " pod="openshift-multus/multus-kwgzc"
Jul 05 00:09:13.118482 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:09:13.117598    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7kxsj\" (UniqueName: \"kubernetes.io/projected/91e13b36-f504-4b21-b8ae-3b66858bddf2-kube-api-access-7kxsj\") pod \"aws-ebs-csi-driver-node-bj4b9\" (UID: \"91e13b36-f504-4b21-b8ae-3b66858bddf2\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-bj4b9"
Jul 05 00:09:13.118482 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:09:13.117624    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/5b9038ef-5e8c-4ee8-9bb7-e9c249715b79-etc-modprobe-d\") pod \"tuned-r2vw8\" (UID: \"5b9038ef-5e8c-4ee8-9bb7-e9c249715b79\") " pod="openshift-cluster-node-tuning-operator/tuned-r2vw8"
Jul 05 00:09:13.118482 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:09:13.117655    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/5b9038ef-5e8c-4ee8-9bb7-e9c249715b79-etc-sysctl-conf\") pod \"tuned-r2vw8\" (UID: \"5b9038ef-5e8c-4ee8-9bb7-e9c249715b79\") " pod="openshift-cluster-node-tuning-operator/tuned-r2vw8"
Jul 05 00:09:13.118482 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:09:13.117683    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/5b9038ef-5e8c-4ee8-9bb7-e9c249715b79-sys\") pod \"tuned-r2vw8\" (UID: \"5b9038ef-5e8c-4ee8-9bb7-e9c249715b79\") " pod="openshift-cluster-node-tuning-operator/tuned-r2vw8"
Jul 05 00:09:13.118482 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:09:13.117707    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/b96bfb4d-2fd3-4be3-9064-e24efeb904f4-host-slash\") pod \"ovnkube-node-7sxq4\" (UID: \"b96bfb4d-2fd3-4be3-9064-e24efeb904f4\") " pod="openshift-ovn-kubernetes/ovnkube-node-7sxq4"
Jul 05 00:09:13.118482 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:09:13.117732    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/91e13b36-f504-4b21-b8ae-3b66858bddf2-registration-dir\") pod \"aws-ebs-csi-driver-node-bj4b9\" (UID: \"91e13b36-f504-4b21-b8ae-3b66858bddf2\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-bj4b9"
Jul 05 00:09:13.118482 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:09:13.117760    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/00bb9c2a-9bed-4d69-9ae4-e4f1fad50578-dbus\") pod \"global-pull-secret-syncer-dkwx5\" (UID: \"00bb9c2a-9bed-4d69-9ae4-e4f1fad50578\") " pod="kube-system/global-pull-secret-syncer-dkwx5"
Jul 05 00:09:13.118482 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:09:13.117786    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/00bb9c2a-9bed-4d69-9ae4-e4f1fad50578-original-pull-secret\") pod \"global-pull-secret-syncer-dkwx5\" (UID: \"00bb9c2a-9bed-4d69-9ae4-e4f1fad50578\") " pod="kube-system/global-pull-secret-syncer-dkwx5"
Jul 05 00:09:13.118482 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:09:13.117810    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/cecee17b-8d41-4a16-a3b2-53919db096bb-tmp-dir\") pod \"node-resolver-kjz9w\" (UID: \"cecee17b-8d41-4a16-a3b2-53919db096bb\") " pod="openshift-dns/node-resolver-kjz9w"
Jul 05 00:09:13.118482 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:09:13.117833    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/5b9038ef-5e8c-4ee8-9bb7-e9c249715b79-etc-sysctl-d\") pod \"tuned-r2vw8\" (UID: \"5b9038ef-5e8c-4ee8-9bb7-e9c249715b79\") " pod="openshift-cluster-node-tuning-operator/tuned-r2vw8"
Jul 05 00:09:13.118482 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:09:13.117857    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/b96bfb4d-2fd3-4be3-9064-e24efeb904f4-log-socket\") pod \"ovnkube-node-7sxq4\" (UID: \"b96bfb4d-2fd3-4be3-9064-e24efeb904f4\") " pod="openshift-ovn-kubernetes/ovnkube-node-7sxq4"
Jul 05 00:09:13.118482 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:09:13.117906    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/aa4848f6-1480-4acd-af73-e8e055306078-hostroot\") pod \"multus-kwgzc\" (UID: \"aa4848f6-1480-4acd-af73-e8e055306078\") " pod="openshift-multus/multus-kwgzc"
Jul 05 00:09:13.118482 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:09:13.117937    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hxt4x\" (UniqueName: \"kubernetes.io/projected/8a2c0b38-b2ce-4570-8963-2f75863cc6b2-kube-api-access-hxt4x\") pod \"iptables-alerter-8tjwr\" (UID: \"8a2c0b38-b2ce-4570-8963-2f75863cc6b2\") " pod="openshift-network-operator/iptables-alerter-8tjwr"
Jul 05 00:09:13.118482 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:09:13.117965    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/5b9038ef-5e8c-4ee8-9bb7-e9c249715b79-etc-sysconfig\") pod \"tuned-r2vw8\" (UID: \"5b9038ef-5e8c-4ee8-9bb7-e9c249715b79\") " pod="openshift-cluster-node-tuning-operator/tuned-r2vw8"
Jul 05 00:09:13.119048 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:09:13.118004    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/b96bfb4d-2fd3-4be3-9064-e24efeb904f4-etc-openvswitch\") pod \"ovnkube-node-7sxq4\" (UID: \"b96bfb4d-2fd3-4be3-9064-e24efeb904f4\") " pod="openshift-ovn-kubernetes/ovnkube-node-7sxq4"
Jul 05 00:09:13.119048 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:09:13.118043    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/91e13b36-f504-4b21-b8ae-3b66858bddf2-device-dir\") pod \"aws-ebs-csi-driver-node-bj4b9\" (UID: \"91e13b36-f504-4b21-b8ae-3b66858bddf2\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-bj4b9"
Jul 05 00:09:13.119048 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:09:13.118068    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/1f3c24e8-591d-4e53-9b63-091f6f1dec51-agent-certs\") pod \"konnectivity-agent-rwbw4\" (UID: \"1f3c24e8-591d-4e53-9b63-091f6f1dec51\") " pod="kube-system/konnectivity-agent-rwbw4"
Jul 05 00:09:13.119048 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:09:13.118091    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/5b9038ef-5e8c-4ee8-9bb7-e9c249715b79-etc-tuned\") pod \"tuned-r2vw8\" (UID: \"5b9038ef-5e8c-4ee8-9bb7-e9c249715b79\") " pod="openshift-cluster-node-tuning-operator/tuned-r2vw8"
Jul 05 00:09:13.119048 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:09:13.118116    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/b96bfb4d-2fd3-4be3-9064-e24efeb904f4-run-systemd\") pod \"ovnkube-node-7sxq4\" (UID: \"b96bfb4d-2fd3-4be3-9064-e24efeb904f4\") " pod="openshift-ovn-kubernetes/ovnkube-node-7sxq4"
Jul 05 00:09:13.119048 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:09:13.118159    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/b96bfb4d-2fd3-4be3-9064-e24efeb904f4-run-ovn\") pod \"ovnkube-node-7sxq4\" (UID: \"b96bfb4d-2fd3-4be3-9064-e24efeb904f4\") " pod="openshift-ovn-kubernetes/ovnkube-node-7sxq4"
Jul 05 00:09:13.119048 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:09:13.118185    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/b96bfb4d-2fd3-4be3-9064-e24efeb904f4-host-cni-netd\") pod \"ovnkube-node-7sxq4\" (UID: \"b96bfb4d-2fd3-4be3-9064-e24efeb904f4\") " pod="openshift-ovn-kubernetes/ovnkube-node-7sxq4"
Jul 05 00:09:13.119048 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:09:13.118209    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/b96bfb4d-2fd3-4be3-9064-e24efeb904f4-ovnkube-script-lib\") pod \"ovnkube-node-7sxq4\" (UID: \"b96bfb4d-2fd3-4be3-9064-e24efeb904f4\") " pod="openshift-ovn-kubernetes/ovnkube-node-7sxq4"
Jul 05 00:09:13.119048 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:09:13.118232    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/aa4848f6-1480-4acd-af73-e8e055306078-host-run-netns\") pod \"multus-kwgzc\" (UID: \"aa4848f6-1480-4acd-af73-e8e055306078\") " pod="openshift-multus/multus-kwgzc"
Jul 05 00:09:13.119048 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:09:13.118279    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/aa4848f6-1480-4acd-af73-e8e055306078-host-var-lib-kubelet\") pod \"multus-kwgzc\" (UID: \"aa4848f6-1480-4acd-af73-e8e055306078\") " pod="openshift-multus/multus-kwgzc"
Jul 05 00:09:13.119048 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:09:13.118303    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rpxd6\" (UniqueName: \"kubernetes.io/projected/aa4848f6-1480-4acd-af73-e8e055306078-kube-api-access-rpxd6\") pod \"multus-kwgzc\" (UID: \"aa4848f6-1480-4acd-af73-e8e055306078\") " pod="openshift-multus/multus-kwgzc"
Jul 05 00:09:13.119048 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:09:13.118359    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/5b9038ef-5e8c-4ee8-9bb7-e9c249715b79-host\") pod \"tuned-r2vw8\" (UID: \"5b9038ef-5e8c-4ee8-9bb7-e9c249715b79\") " pod="openshift-cluster-node-tuning-operator/tuned-r2vw8"
Jul 05 00:09:13.119048 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:09:13.118402    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/b96bfb4d-2fd3-4be3-9064-e24efeb904f4-env-overrides\") pod \"ovnkube-node-7sxq4\" (UID: \"b96bfb4d-2fd3-4be3-9064-e24efeb904f4\") " pod="openshift-ovn-kubernetes/ovnkube-node-7sxq4"
Jul 05 00:09:13.119048 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:09:13.118430    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/aeaa1e13-243b-4c7f-8045-85ba45081c42-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-g568x\" (UID: \"aeaa1e13-243b-4c7f-8045-85ba45081c42\") " pod="openshift-multus/multus-additional-cni-plugins-g568x"
Jul 05 00:09:13.119048 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:09:13.118457    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/aa4848f6-1480-4acd-af73-e8e055306078-system-cni-dir\") pod \"multus-kwgzc\" (UID: \"aa4848f6-1480-4acd-af73-e8e055306078\") " pod="openshift-multus/multus-kwgzc"
Jul 05 00:09:13.119048 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:09:13.118480    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/aa4848f6-1480-4acd-af73-e8e055306078-cnibin\") pod \"multus-kwgzc\" (UID: \"aa4848f6-1480-4acd-af73-e8e055306078\") " pod="openshift-multus/multus-kwgzc"
Jul 05 00:09:13.119048 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:09:13.118554    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/aa4848f6-1480-4acd-af73-e8e055306078-host-run-multus-certs\") pod \"multus-kwgzc\" (UID: \"aa4848f6-1480-4acd-af73-e8e055306078\") " pod="openshift-multus/multus-kwgzc"
Jul 05 00:09:13.119709 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:09:13.118592    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/b96bfb4d-2fd3-4be3-9064-e24efeb904f4-host-run-netns\") pod \"ovnkube-node-7sxq4\" (UID: \"b96bfb4d-2fd3-4be3-9064-e24efeb904f4\") " pod="openshift-ovn-kubernetes/ovnkube-node-7sxq4"
Jul 05 00:09:13.119709 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:09:13.118614    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/b96bfb4d-2fd3-4be3-9064-e24efeb904f4-ovnkube-config\") pod \"ovnkube-node-7sxq4\" (UID: \"b96bfb4d-2fd3-4be3-9064-e24efeb904f4\") " pod="openshift-ovn-kubernetes/ovnkube-node-7sxq4"
Jul 05 00:09:13.119709 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:09:13.118634    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/aa4848f6-1480-4acd-af73-e8e055306078-etc-kubernetes\") pod \"multus-kwgzc\" (UID: \"aa4848f6-1480-4acd-af73-e8e055306078\") " pod="openshift-multus/multus-kwgzc"
Jul 05 00:09:13.119709 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:09:13.118717    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/91e13b36-f504-4b21-b8ae-3b66858bddf2-socket-dir\") pod \"aws-ebs-csi-driver-node-bj4b9\" (UID: \"91e13b36-f504-4b21-b8ae-3b66858bddf2\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-bj4b9"
Jul 05 00:09:13.119709 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:09:13.118737    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/cecee17b-8d41-4a16-a3b2-53919db096bb-hosts-file\") pod \"node-resolver-kjz9w\" (UID: \"cecee17b-8d41-4a16-a3b2-53919db096bb\") " pod="openshift-dns/node-resolver-kjz9w"
Jul 05 00:09:13.119709 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:09:13.118753    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-89gv4\" (UniqueName: \"kubernetes.io/projected/d3a9b78d-cf60-4acc-a3c5-016ee34903f8-kube-api-access-89gv4\") pod \"node-ca-s547v\" (UID: \"d3a9b78d-cf60-4acc-a3c5-016ee34903f8\") " pod="openshift-image-registry/node-ca-s547v"
Jul 05 00:09:13.119709 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:09:13.118776    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/91e13b36-f504-4b21-b8ae-3b66858bddf2-etc-selinux\") pod \"aws-ebs-csi-driver-node-bj4b9\" (UID: \"91e13b36-f504-4b21-b8ae-3b66858bddf2\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-bj4b9"
Jul 05 00:09:13.119709 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:09:13.118800    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/1f3c24e8-591d-4e53-9b63-091f6f1dec51-konnectivity-ca\") pod \"konnectivity-agent-rwbw4\" (UID: \"1f3c24e8-591d-4e53-9b63-091f6f1dec51\") " pod="kube-system/konnectivity-agent-rwbw4"
Jul 05 00:09:13.119709 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:09:13.118824    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/5b9038ef-5e8c-4ee8-9bb7-e9c249715b79-run\") pod \"tuned-r2vw8\" (UID: \"5b9038ef-5e8c-4ee8-9bb7-e9c249715b79\") " pod="openshift-cluster-node-tuning-operator/tuned-r2vw8"
Jul 05 00:09:13.119709 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:09:13.118840    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/5b9038ef-5e8c-4ee8-9bb7-e9c249715b79-lib-modules\") pod \"tuned-r2vw8\" (UID: \"5b9038ef-5e8c-4ee8-9bb7-e9c249715b79\") " pod="openshift-cluster-node-tuning-operator/tuned-r2vw8"
Jul 05 00:09:13.119709 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:09:13.118854    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/8a2c0b38-b2ce-4570-8963-2f75863cc6b2-host-slash\") pod \"iptables-alerter-8tjwr\" (UID: \"8a2c0b38-b2ce-4570-8963-2f75863cc6b2\") " pod="openshift-network-operator/iptables-alerter-8tjwr"
Jul 05 00:09:13.119709 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:09:13.118870    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/b96bfb4d-2fd3-4be3-9064-e24efeb904f4-var-lib-openvswitch\") pod \"ovnkube-node-7sxq4\" (UID: \"b96bfb4d-2fd3-4be3-9064-e24efeb904f4\") " pod="openshift-ovn-kubernetes/ovnkube-node-7sxq4"
Jul 05 00:09:13.119709 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:09:13.118894    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/b96bfb4d-2fd3-4be3-9064-e24efeb904f4-run-openvswitch\") pod \"ovnkube-node-7sxq4\" (UID: \"b96bfb4d-2fd3-4be3-9064-e24efeb904f4\") " pod="openshift-ovn-kubernetes/ovnkube-node-7sxq4"
Jul 05 00:09:13.119709 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:09:13.118918    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/aa4848f6-1480-4acd-af73-e8e055306078-multus-cni-dir\") pod \"multus-kwgzc\" (UID: \"aa4848f6-1480-4acd-af73-e8e055306078\") " pod="openshift-multus/multus-kwgzc"
Jul 05 00:09:13.119709 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:09:13.118939    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/aa4848f6-1480-4acd-af73-e8e055306078-cni-binary-copy\") pod \"multus-kwgzc\" (UID: \"aa4848f6-1480-4acd-af73-e8e055306078\") " pod="openshift-multus/multus-kwgzc"
Jul 05 00:09:13.119709 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:09:13.118953    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/aa4848f6-1480-4acd-af73-e8e055306078-multus-conf-dir\") pod \"multus-kwgzc\" (UID: \"aa4848f6-1480-4acd-af73-e8e055306078\") " pod="openshift-multus/multus-kwgzc"
Jul 05 00:09:13.119709 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:09:13.118968    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/b96bfb4d-2fd3-4be3-9064-e24efeb904f4-host-cni-bin\") pod \"ovnkube-node-7sxq4\" (UID: \"b96bfb4d-2fd3-4be3-9064-e24efeb904f4\") " pod="openshift-ovn-kubernetes/ovnkube-node-7sxq4"
Jul 05 00:09:13.120338 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:09:13.118985    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/aa4848f6-1480-4acd-af73-e8e055306078-multus-socket-dir-parent\") pod \"multus-kwgzc\" (UID: \"aa4848f6-1480-4acd-af73-e8e055306078\") " pod="openshift-multus/multus-kwgzc"
Jul 05 00:09:13.120338 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:09:13.119026    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/91e13b36-f504-4b21-b8ae-3b66858bddf2-kubelet-dir\") pod \"aws-ebs-csi-driver-node-bj4b9\" (UID: \"91e13b36-f504-4b21-b8ae-3b66858bddf2\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-bj4b9"
Jul 05 00:09:13.120338 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:09:13.119061    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mqrck\" (UniqueName: \"kubernetes.io/projected/c085d4be-fcda-4931-a018-02cbaca17c59-kube-api-access-mqrck\") pod \"network-metrics-daemon-5fn94\" (UID: \"c085d4be-fcda-4931-a018-02cbaca17c59\") " pod="openshift-multus/network-metrics-daemon-5fn94"
Jul 05 00:09:13.120338 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:09:13.119086    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/b96bfb4d-2fd3-4be3-9064-e24efeb904f4-host-kubelet\") pod \"ovnkube-node-7sxq4\" (UID: \"b96bfb4d-2fd3-4be3-9064-e24efeb904f4\") " pod="openshift-ovn-kubernetes/ovnkube-node-7sxq4"
Jul 05 00:09:13.120338 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:09:13.119101    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c085d4be-fcda-4931-a018-02cbaca17c59-metrics-certs\") pod \"network-metrics-daemon-5fn94\" (UID: \"c085d4be-fcda-4931-a018-02cbaca17c59\") " pod="openshift-multus/network-metrics-daemon-5fn94"
Jul 05 00:09:13.120338 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:09:13.119120    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/00bb9c2a-9bed-4d69-9ae4-e4f1fad50578-kubelet-config\") pod \"global-pull-secret-syncer-dkwx5\" (UID: \"00bb9c2a-9bed-4d69-9ae4-e4f1fad50578\") " pod="kube-system/global-pull-secret-syncer-dkwx5"
Jul 05 00:09:13.120338 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:09:13.119160    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2rtk9\" (UniqueName: \"kubernetes.io/projected/cecee17b-8d41-4a16-a3b2-53919db096bb-kube-api-access-2rtk9\") pod \"node-resolver-kjz9w\" (UID: \"cecee17b-8d41-4a16-a3b2-53919db096bb\") " pod="openshift-dns/node-resolver-kjz9w"
Jul 05 00:09:13.220489 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:09:13.220458    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/aa4848f6-1480-4acd-af73-e8e055306078-cnibin\") pod \"multus-kwgzc\" (UID: \"aa4848f6-1480-4acd-af73-e8e055306078\") " pod="openshift-multus/multus-kwgzc"
Jul 05 00:09:13.220644 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:09:13.220494    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/aa4848f6-1480-4acd-af73-e8e055306078-host-run-multus-certs\") pod \"multus-kwgzc\" (UID: \"aa4848f6-1480-4acd-af73-e8e055306078\") " pod="openshift-multus/multus-kwgzc"
Jul 05 00:09:13.220644 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:09:13.220514    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/b96bfb4d-2fd3-4be3-9064-e24efeb904f4-host-run-netns\") pod \"ovnkube-node-7sxq4\" (UID: \"b96bfb4d-2fd3-4be3-9064-e24efeb904f4\") " pod="openshift-ovn-kubernetes/ovnkube-node-7sxq4"
Jul 05 00:09:13.220644 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:09:13.220532    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/b96bfb4d-2fd3-4be3-9064-e24efeb904f4-ovnkube-config\") pod \"ovnkube-node-7sxq4\" (UID: \"b96bfb4d-2fd3-4be3-9064-e24efeb904f4\") " pod="openshift-ovn-kubernetes/ovnkube-node-7sxq4"
Jul 05 00:09:13.220644 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:09:13.220550    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/aa4848f6-1480-4acd-af73-e8e055306078-etc-kubernetes\") pod \"multus-kwgzc\" (UID: \"aa4848f6-1480-4acd-af73-e8e055306078\") " pod="openshift-multus/multus-kwgzc"
Jul 05 00:09:13.220644 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:09:13.220567    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/91e13b36-f504-4b21-b8ae-3b66858bddf2-socket-dir\") pod \"aws-ebs-csi-driver-node-bj4b9\" (UID: \"91e13b36-f504-4b21-b8ae-3b66858bddf2\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-bj4b9"
Jul 05 00:09:13.220644 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:09:13.220585    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/cecee17b-8d41-4a16-a3b2-53919db096bb-hosts-file\") pod \"node-resolver-kjz9w\" (UID: \"cecee17b-8d41-4a16-a3b2-53919db096bb\") " pod="openshift-dns/node-resolver-kjz9w"
Jul 05 00:09:13.220644 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:09:13.220600    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/b96bfb4d-2fd3-4be3-9064-e24efeb904f4-host-run-netns\") pod \"ovnkube-node-7sxq4\" (UID: \"b96bfb4d-2fd3-4be3-9064-e24efeb904f4\") " pod="openshift-ovn-kubernetes/ovnkube-node-7sxq4"
Jul 05 00:09:13.220644 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:09:13.220600    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/aa4848f6-1480-4acd-af73-e8e055306078-cnibin\") pod \"multus-kwgzc\" (UID: \"aa4848f6-1480-4acd-af73-e8e055306078\") " pod="openshift-multus/multus-kwgzc"
Jul 05 00:09:13.220644 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:09:13.220600    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/aa4848f6-1480-4acd-af73-e8e055306078-host-run-multus-certs\") pod \"multus-kwgzc\" (UID: \"aa4848f6-1480-4acd-af73-e8e055306078\") " pod="openshift-multus/multus-kwgzc"
Jul 05 00:09:13.220644 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:09:13.220611    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-89gv4\" (UniqueName: \"kubernetes.io/projected/d3a9b78d-cf60-4acc-a3c5-016ee34903f8-kube-api-access-89gv4\") pod \"node-ca-s547v\" (UID: \"d3a9b78d-cf60-4acc-a3c5-016ee34903f8\") " pod="openshift-image-registry/node-ca-s547v"
Jul 05 00:09:13.221195 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:09:13.220653    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/cecee17b-8d41-4a16-a3b2-53919db096bb-hosts-file\") pod \"node-resolver-kjz9w\" (UID: \"cecee17b-8d41-4a16-a3b2-53919db096bb\") " pod="openshift-dns/node-resolver-kjz9w"
Jul 05 00:09:13.221195 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:09:13.220654    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/aa4848f6-1480-4acd-af73-e8e055306078-etc-kubernetes\") pod \"multus-kwgzc\" (UID: \"aa4848f6-1480-4acd-af73-e8e055306078\") " pod="openshift-multus/multus-kwgzc"
Jul 05 00:09:13.221195 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:09:13.220676    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/91e13b36-f504-4b21-b8ae-3b66858bddf2-etc-selinux\") pod \"aws-ebs-csi-driver-node-bj4b9\" (UID: \"91e13b36-f504-4b21-b8ae-3b66858bddf2\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-bj4b9"
Jul 05 00:09:13.221195 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:09:13.220701    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/1f3c24e8-591d-4e53-9b63-091f6f1dec51-konnectivity-ca\") pod \"konnectivity-agent-rwbw4\" (UID: \"1f3c24e8-591d-4e53-9b63-091f6f1dec51\") " pod="kube-system/konnectivity-agent-rwbw4"
Jul 05 00:09:13.221195 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:09:13.220723    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/5b9038ef-5e8c-4ee8-9bb7-e9c249715b79-run\") pod \"tuned-r2vw8\" (UID: \"5b9038ef-5e8c-4ee8-9bb7-e9c249715b79\") " pod="openshift-cluster-node-tuning-operator/tuned-r2vw8"
Jul 05 00:09:13.221195 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:09:13.220746    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/5b9038ef-5e8c-4ee8-9bb7-e9c249715b79-lib-modules\") pod \"tuned-r2vw8\" (UID: \"5b9038ef-5e8c-4ee8-9bb7-e9c249715b79\") " pod="openshift-cluster-node-tuning-operator/tuned-r2vw8"
Jul 05 00:09:13.221195 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:09:13.220771    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/8a2c0b38-b2ce-4570-8963-2f75863cc6b2-host-slash\") pod \"iptables-alerter-8tjwr\" (UID: \"8a2c0b38-b2ce-4570-8963-2f75863cc6b2\") " pod="openshift-network-operator/iptables-alerter-8tjwr"
Jul 05 00:09:13.221195 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:09:13.220763    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/91e13b36-f504-4b21-b8ae-3b66858bddf2-socket-dir\") pod \"aws-ebs-csi-driver-node-bj4b9\" (UID: \"91e13b36-f504-4b21-b8ae-3b66858bddf2\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-bj4b9"
Jul 05 00:09:13.221195 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:09:13.220821    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/8a2c0b38-b2ce-4570-8963-2f75863cc6b2-host-slash\") pod \"iptables-alerter-8tjwr\" (UID: \"8a2c0b38-b2ce-4570-8963-2f75863cc6b2\") " pod="openshift-network-operator/iptables-alerter-8tjwr"
Jul 05 00:09:13.221195 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:09:13.220851    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/5b9038ef-5e8c-4ee8-9bb7-e9c249715b79-run\") pod \"tuned-r2vw8\" (UID: \"5b9038ef-5e8c-4ee8-9bb7-e9c249715b79\") " pod="openshift-cluster-node-tuning-operator/tuned-r2vw8"
Jul 05 00:09:13.221195 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:09:13.220869    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/5b9038ef-5e8c-4ee8-9bb7-e9c249715b79-lib-modules\") pod \"tuned-r2vw8\" (UID: \"5b9038ef-5e8c-4ee8-9bb7-e9c249715b79\") " pod="openshift-cluster-node-tuning-operator/tuned-r2vw8"
Jul 05 00:09:13.221195 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:09:13.220874    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/b96bfb4d-2fd3-4be3-9064-e24efeb904f4-var-lib-openvswitch\") pod \"ovnkube-node-7sxq4\" (UID: \"b96bfb4d-2fd3-4be3-9064-e24efeb904f4\") " pod="openshift-ovn-kubernetes/ovnkube-node-7sxq4"
Jul 05 00:09:13.221195 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:09:13.220915    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/b96bfb4d-2fd3-4be3-9064-e24efeb904f4-var-lib-openvswitch\") pod \"ovnkube-node-7sxq4\" (UID: \"b96bfb4d-2fd3-4be3-9064-e24efeb904f4\") " pod="openshift-ovn-kubernetes/ovnkube-node-7sxq4"
Jul 05 00:09:13.221195 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:09:13.220907    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/b96bfb4d-2fd3-4be3-9064-e24efeb904f4-run-openvswitch\") pod \"ovnkube-node-7sxq4\" (UID: \"b96bfb4d-2fd3-4be3-9064-e24efeb904f4\") " pod="openshift-ovn-kubernetes/ovnkube-node-7sxq4"
Jul 05 00:09:13.221195 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:09:13.220950    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/b96bfb4d-2fd3-4be3-9064-e24efeb904f4-run-openvswitch\") pod \"ovnkube-node-7sxq4\" (UID: \"b96bfb4d-2fd3-4be3-9064-e24efeb904f4\") " pod="openshift-ovn-kubernetes/ovnkube-node-7sxq4"
Jul 05 00:09:13.221195 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:09:13.220954    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/aa4848f6-1480-4acd-af73-e8e055306078-multus-cni-dir\") pod \"multus-kwgzc\" (UID: \"aa4848f6-1480-4acd-af73-e8e055306078\") " pod="openshift-multus/multus-kwgzc"
Jul 05 00:09:13.221195 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:09:13.220989    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/aa4848f6-1480-4acd-af73-e8e055306078-cni-binary-copy\") pod \"multus-kwgzc\" (UID: \"aa4848f6-1480-4acd-af73-e8e055306078\") " pod="openshift-multus/multus-kwgzc"
Jul 05 00:09:13.221195 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:09:13.221015    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/aa4848f6-1480-4acd-af73-e8e055306078-multus-conf-dir\") pod \"multus-kwgzc\" (UID: \"aa4848f6-1480-4acd-af73-e8e055306078\") " pod="openshift-multus/multus-kwgzc"
Jul 05 00:09:13.222111 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:09:13.220940    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/91e13b36-f504-4b21-b8ae-3b66858bddf2-etc-selinux\") pod \"aws-ebs-csi-driver-node-bj4b9\" (UID: \"91e13b36-f504-4b21-b8ae-3b66858bddf2\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-bj4b9"
Jul 05 00:09:13.222111 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:09:13.221039    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/b96bfb4d-2fd3-4be3-9064-e24efeb904f4-host-cni-bin\") pod \"ovnkube-node-7sxq4\" (UID: \"b96bfb4d-2fd3-4be3-9064-e24efeb904f4\") " pod="openshift-ovn-kubernetes/ovnkube-node-7sxq4"
Jul 05 00:09:13.222111 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:09:13.221067    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/aa4848f6-1480-4acd-af73-e8e055306078-multus-socket-dir-parent\") pod \"multus-kwgzc\" (UID: \"aa4848f6-1480-4acd-af73-e8e055306078\") " pod="openshift-multus/multus-kwgzc"
Jul 05 00:09:13.222111 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:09:13.221074    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/aa4848f6-1480-4acd-af73-e8e055306078-multus-conf-dir\") pod \"multus-kwgzc\" (UID: \"aa4848f6-1480-4acd-af73-e8e055306078\") " pod="openshift-multus/multus-kwgzc"
Jul 05 00:09:13.222111 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:09:13.221096    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/91e13b36-f504-4b21-b8ae-3b66858bddf2-kubelet-dir\") pod \"aws-ebs-csi-driver-node-bj4b9\" (UID: \"91e13b36-f504-4b21-b8ae-3b66858bddf2\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-bj4b9"
Jul 05 00:09:13.222111 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:09:13.221116    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/b96bfb4d-2fd3-4be3-9064-e24efeb904f4-host-cni-bin\") pod \"ovnkube-node-7sxq4\" (UID: \"b96bfb4d-2fd3-4be3-9064-e24efeb904f4\") " pod="openshift-ovn-kubernetes/ovnkube-node-7sxq4"
Jul 05 00:09:13.222111 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:09:13.221150    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mqrck\" (UniqueName: \"kubernetes.io/projected/c085d4be-fcda-4931-a018-02cbaca17c59-kube-api-access-mqrck\") pod \"network-metrics-daemon-5fn94\" (UID: \"c085d4be-fcda-4931-a018-02cbaca17c59\") " pod="openshift-multus/network-metrics-daemon-5fn94"
Jul 05 00:09:13.222111 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:09:13.221167    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/aa4848f6-1480-4acd-af73-e8e055306078-multus-cni-dir\") pod \"multus-kwgzc\" (UID: \"aa4848f6-1480-4acd-af73-e8e055306078\") " pod="openshift-multus/multus-kwgzc"
Jul 05 00:09:13.222111 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:09:13.221170    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/aa4848f6-1480-4acd-af73-e8e055306078-multus-socket-dir-parent\") pod \"multus-kwgzc\" (UID: \"aa4848f6-1480-4acd-af73-e8e055306078\") " pod="openshift-multus/multus-kwgzc"
Jul 05 00:09:13.222111 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:09:13.221180    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/b96bfb4d-2fd3-4be3-9064-e24efeb904f4-host-kubelet\") pod \"ovnkube-node-7sxq4\" (UID: \"b96bfb4d-2fd3-4be3-9064-e24efeb904f4\") " pod="openshift-ovn-kubernetes/ovnkube-node-7sxq4"
Jul 05 00:09:13.222111 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:09:13.221194    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/91e13b36-f504-4b21-b8ae-3b66858bddf2-kubelet-dir\") pod \"aws-ebs-csi-driver-node-bj4b9\" (UID: \"91e13b36-f504-4b21-b8ae-3b66858bddf2\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-bj4b9"
Jul 05 00:09:13.222111 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:09:13.221206    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/b96bfb4d-2fd3-4be3-9064-e24efeb904f4-ovnkube-config\") pod \"ovnkube-node-7sxq4\" (UID: \"b96bfb4d-2fd3-4be3-9064-e24efeb904f4\") " pod="openshift-ovn-kubernetes/ovnkube-node-7sxq4"
Jul 05 00:09:13.222111 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:09:13.221216    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c085d4be-fcda-4931-a018-02cbaca17c59-metrics-certs\") pod \"network-metrics-daemon-5fn94\" (UID: \"c085d4be-fcda-4931-a018-02cbaca17c59\") " pod="openshift-multus/network-metrics-daemon-5fn94"
Jul 05 00:09:13.222111 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:09:13.221267    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/00bb9c2a-9bed-4d69-9ae4-e4f1fad50578-kubelet-config\") pod \"global-pull-secret-syncer-dkwx5\" (UID: \"00bb9c2a-9bed-4d69-9ae4-e4f1fad50578\") " pod="kube-system/global-pull-secret-syncer-dkwx5"
Jul 05 00:09:13.222111 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:09:13.221287    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2rtk9\" (UniqueName: \"kubernetes.io/projected/cecee17b-8d41-4a16-a3b2-53919db096bb-kube-api-access-2rtk9\") pod \"node-resolver-kjz9w\" (UID: \"cecee17b-8d41-4a16-a3b2-53919db096bb\") " pod="openshift-dns/node-resolver-kjz9w"
Jul 05 00:09:13.222111 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:09:13.221291    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/1f3c24e8-591d-4e53-9b63-091f6f1dec51-konnectivity-ca\") pod \"konnectivity-agent-rwbw4\" (UID: \"1f3c24e8-591d-4e53-9b63-091f6f1dec51\") " pod="kube-system/konnectivity-agent-rwbw4"
Jul 05 00:09:13.222111 ip-10-0-142-202 kubenswrapper[2563]: E0705 00:09:13.221307    2563 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 05 00:09:13.222111 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:09:13.221313    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/5b9038ef-5e8c-4ee8-9bb7-e9c249715b79-var-lib-kubelet\") pod \"tuned-r2vw8\" (UID: \"5b9038ef-5e8c-4ee8-9bb7-e9c249715b79\") " pod="openshift-cluster-node-tuning-operator/tuned-r2vw8"
Jul 05 00:09:13.223046 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:09:13.221222    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/b96bfb4d-2fd3-4be3-9064-e24efeb904f4-host-kubelet\") pod \"ovnkube-node-7sxq4\" (UID: \"b96bfb4d-2fd3-4be3-9064-e24efeb904f4\") " pod="openshift-ovn-kubernetes/ovnkube-node-7sxq4"
Jul 05 00:09:13.223046 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:09:13.221340    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vntcz\" (UniqueName: \"kubernetes.io/projected/5b9038ef-5e8c-4ee8-9bb7-e9c249715b79-kube-api-access-vntcz\") pod \"tuned-r2vw8\" (UID: \"5b9038ef-5e8c-4ee8-9bb7-e9c249715b79\") " pod="openshift-cluster-node-tuning-operator/tuned-r2vw8"
Jul 05 00:09:13.223046 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:09:13.221365    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/8a2c0b38-b2ce-4570-8963-2f75863cc6b2-iptables-alerter-script\") pod \"iptables-alerter-8tjwr\" (UID: \"8a2c0b38-b2ce-4570-8963-2f75863cc6b2\") " pod="openshift-network-operator/iptables-alerter-8tjwr"
Jul 05 00:09:13.223046 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:09:13.221370    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/5b9038ef-5e8c-4ee8-9bb7-e9c249715b79-var-lib-kubelet\") pod \"tuned-r2vw8\" (UID: \"5b9038ef-5e8c-4ee8-9bb7-e9c249715b79\") " pod="openshift-cluster-node-tuning-operator/tuned-r2vw8"
Jul 05 00:09:13.223046 ip-10-0-142-202 kubenswrapper[2563]: E0705 00:09:13.221380    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c085d4be-fcda-4931-a018-02cbaca17c59-metrics-certs podName:c085d4be-fcda-4931-a018-02cbaca17c59 nodeName:}" failed. No retries permitted until 2026-07-05 00:09:13.721360121 +0000 UTC m=+2.063689166 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/c085d4be-fcda-4931-a018-02cbaca17c59-metrics-certs") pod "network-metrics-daemon-5fn94" (UID: "c085d4be-fcda-4931-a018-02cbaca17c59") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 05 00:09:13.223046 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:09:13.221415    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/b96bfb4d-2fd3-4be3-9064-e24efeb904f4-systemd-units\") pod \"ovnkube-node-7sxq4\" (UID: \"b96bfb4d-2fd3-4be3-9064-e24efeb904f4\") " pod="openshift-ovn-kubernetes/ovnkube-node-7sxq4"
Jul 05 00:09:13.223046 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:09:13.221430    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/00bb9c2a-9bed-4d69-9ae4-e4f1fad50578-kubelet-config\") pod \"global-pull-secret-syncer-dkwx5\" (UID: \"00bb9c2a-9bed-4d69-9ae4-e4f1fad50578\") " pod="kube-system/global-pull-secret-syncer-dkwx5"
Jul 05 00:09:13.223046 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:09:13.221442    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/b96bfb4d-2fd3-4be3-9064-e24efeb904f4-node-log\") pod \"ovnkube-node-7sxq4\" (UID: \"b96bfb4d-2fd3-4be3-9064-e24efeb904f4\") " pod="openshift-ovn-kubernetes/ovnkube-node-7sxq4"
Jul 05 00:09:13.223046 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:09:13.221470    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/b96bfb4d-2fd3-4be3-9064-e24efeb904f4-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-7sxq4\" (UID: \"b96bfb4d-2fd3-4be3-9064-e24efeb904f4\") " pod="openshift-ovn-kubernetes/ovnkube-node-7sxq4"
Jul 05 00:09:13.223046 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:09:13.221481    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/b96bfb4d-2fd3-4be3-9064-e24efeb904f4-systemd-units\") pod \"ovnkube-node-7sxq4\" (UID: \"b96bfb4d-2fd3-4be3-9064-e24efeb904f4\") " pod="openshift-ovn-kubernetes/ovnkube-node-7sxq4"
Jul 05 00:09:13.223046 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:09:13.221497    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/b96bfb4d-2fd3-4be3-9064-e24efeb904f4-ovn-node-metrics-cert\") pod \"ovnkube-node-7sxq4\" (UID: \"b96bfb4d-2fd3-4be3-9064-e24efeb904f4\") " pod="openshift-ovn-kubernetes/ovnkube-node-7sxq4"
Jul 05 00:09:13.223046 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:09:13.221519    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/b96bfb4d-2fd3-4be3-9064-e24efeb904f4-node-log\") pod \"ovnkube-node-7sxq4\" (UID: \"b96bfb4d-2fd3-4be3-9064-e24efeb904f4\") " pod="openshift-ovn-kubernetes/ovnkube-node-7sxq4"
Jul 05 00:09:13.223046 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:09:13.221527    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/aeaa1e13-243b-4c7f-8045-85ba45081c42-system-cni-dir\") pod \"multus-additional-cni-plugins-g568x\" (UID: \"aeaa1e13-243b-4c7f-8045-85ba45081c42\") " pod="openshift-multus/multus-additional-cni-plugins-g568x"
Jul 05 00:09:13.223046 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:09:13.221553    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/aa4848f6-1480-4acd-af73-e8e055306078-os-release\") pod \"multus-kwgzc\" (UID: \"aa4848f6-1480-4acd-af73-e8e055306078\") " pod="openshift-multus/multus-kwgzc"
Jul 05 00:09:13.223046 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:09:13.221558    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/b96bfb4d-2fd3-4be3-9064-e24efeb904f4-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-7sxq4\" (UID: \"b96bfb4d-2fd3-4be3-9064-e24efeb904f4\") " pod="openshift-ovn-kubernetes/ovnkube-node-7sxq4"
Jul 05 00:09:13.223046 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:09:13.221578    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/aeaa1e13-243b-4c7f-8045-85ba45081c42-cnibin\") pod \"multus-additional-cni-plugins-g568x\" (UID: \"aeaa1e13-243b-4c7f-8045-85ba45081c42\") " pod="openshift-multus/multus-additional-cni-plugins-g568x"
Jul 05 00:09:13.223963 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:09:13.221591    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/aeaa1e13-243b-4c7f-8045-85ba45081c42-system-cni-dir\") pod \"multus-additional-cni-plugins-g568x\" (UID: \"aeaa1e13-243b-4c7f-8045-85ba45081c42\") " pod="openshift-multus/multus-additional-cni-plugins-g568x"
Jul 05 00:09:13.223963 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:09:13.221604    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/aeaa1e13-243b-4c7f-8045-85ba45081c42-cni-binary-copy\") pod \"multus-additional-cni-plugins-g568x\" (UID: \"aeaa1e13-243b-4c7f-8045-85ba45081c42\") " pod="openshift-multus/multus-additional-cni-plugins-g568x"
Jul 05 00:09:13.223963 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:09:13.221685    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/aa4848f6-1480-4acd-af73-e8e055306078-os-release\") pod \"multus-kwgzc\" (UID: \"aa4848f6-1480-4acd-af73-e8e055306078\") " pod="openshift-multus/multus-kwgzc"
Jul 05 00:09:13.223963 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:09:13.221647    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/aeaa1e13-243b-4c7f-8045-85ba45081c42-cnibin\") pod \"multus-additional-cni-plugins-g568x\" (UID: \"aeaa1e13-243b-4c7f-8045-85ba45081c42\") " pod="openshift-multus/multus-additional-cni-plugins-g568x"
Jul 05 00:09:13.223963 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:09:13.221726    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hncnr\" (UniqueName: \"kubernetes.io/projected/aeaa1e13-243b-4c7f-8045-85ba45081c42-kube-api-access-hncnr\") pod \"multus-additional-cni-plugins-g568x\" (UID: \"aeaa1e13-243b-4c7f-8045-85ba45081c42\") " pod="openshift-multus/multus-additional-cni-plugins-g568x"
Jul 05 00:09:13.223963 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:09:13.221751    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/aa4848f6-1480-4acd-af73-e8e055306078-host-var-lib-cni-multus\") pod \"multus-kwgzc\" (UID: \"aa4848f6-1480-4acd-af73-e8e055306078\") " pod="openshift-multus/multus-kwgzc"
Jul 05 00:09:13.223963 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:09:13.221755    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/aa4848f6-1480-4acd-af73-e8e055306078-cni-binary-copy\") pod \"multus-kwgzc\" (UID: \"aa4848f6-1480-4acd-af73-e8e055306078\") " pod="openshift-multus/multus-kwgzc"
Jul 05 00:09:13.223963 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:09:13.221795    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/aa4848f6-1480-4acd-af73-e8e055306078-host-var-lib-cni-multus\") pod \"multus-kwgzc\" (UID: \"aa4848f6-1480-4acd-af73-e8e055306078\") " pod="openshift-multus/multus-kwgzc"
Jul 05 00:09:13.223963 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:09:13.221821    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/91e13b36-f504-4b21-b8ae-3b66858bddf2-sys-fs\") pod \"aws-ebs-csi-driver-node-bj4b9\" (UID: \"91e13b36-f504-4b21-b8ae-3b66858bddf2\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-bj4b9"
Jul 05 00:09:13.223963 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:09:13.221841    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/5b9038ef-5e8c-4ee8-9bb7-e9c249715b79-etc-kubernetes\") pod \"tuned-r2vw8\" (UID: \"5b9038ef-5e8c-4ee8-9bb7-e9c249715b79\") " pod="openshift-cluster-node-tuning-operator/tuned-r2vw8"
Jul 05 00:09:13.223963 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:09:13.221861    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/5b9038ef-5e8c-4ee8-9bb7-e9c249715b79-tmp\") pod \"tuned-r2vw8\" (UID: \"5b9038ef-5e8c-4ee8-9bb7-e9c249715b79\") " pod="openshift-cluster-node-tuning-operator/tuned-r2vw8"
Jul 05 00:09:13.223963 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:09:13.221871    2563 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory"
Jul 05 00:09:13.223963 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:09:13.221888    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-z4cn8\" (UniqueName: \"kubernetes.io/projected/e84aff2e-272a-42d8-8bd9-e37a98cfc7a7-kube-api-access-z4cn8\") pod \"network-check-target-4pm8c\" (UID: \"e84aff2e-272a-42d8-8bd9-e37a98cfc7a7\") " pod="openshift-network-diagnostics/network-check-target-4pm8c"
Jul 05 00:09:13.223963 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:09:13.221902    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/5b9038ef-5e8c-4ee8-9bb7-e9c249715b79-etc-kubernetes\") pod \"tuned-r2vw8\" (UID: \"5b9038ef-5e8c-4ee8-9bb7-e9c249715b79\") " pod="openshift-cluster-node-tuning-operator/tuned-r2vw8"
Jul 05 00:09:13.223963 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:09:13.221906    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/91e13b36-f504-4b21-b8ae-3b66858bddf2-sys-fs\") pod \"aws-ebs-csi-driver-node-bj4b9\" (UID: \"91e13b36-f504-4b21-b8ae-3b66858bddf2\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-bj4b9"
Jul 05 00:09:13.223963 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:09:13.221915    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/b96bfb4d-2fd3-4be3-9064-e24efeb904f4-host-run-ovn-kubernetes\") pod \"ovnkube-node-7sxq4\" (UID: \"b96bfb4d-2fd3-4be3-9064-e24efeb904f4\") " pod="openshift-ovn-kubernetes/ovnkube-node-7sxq4"
Jul 05 00:09:13.223963 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:09:13.221962    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/b96bfb4d-2fd3-4be3-9064-e24efeb904f4-host-run-ovn-kubernetes\") pod \"ovnkube-node-7sxq4\" (UID: \"b96bfb4d-2fd3-4be3-9064-e24efeb904f4\") " pod="openshift-ovn-kubernetes/ovnkube-node-7sxq4"
Jul 05 00:09:13.224935 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:09:13.221987    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/8a2c0b38-b2ce-4570-8963-2f75863cc6b2-iptables-alerter-script\") pod \"iptables-alerter-8tjwr\" (UID: \"8a2c0b38-b2ce-4570-8963-2f75863cc6b2\") " pod="openshift-network-operator/iptables-alerter-8tjwr"
Jul 05 00:09:13.224935 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:09:13.222048    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gz6zl\" (UniqueName: \"kubernetes.io/projected/b96bfb4d-2fd3-4be3-9064-e24efeb904f4-kube-api-access-gz6zl\") pod \"ovnkube-node-7sxq4\" (UID: \"b96bfb4d-2fd3-4be3-9064-e24efeb904f4\") " pod="openshift-ovn-kubernetes/ovnkube-node-7sxq4"
Jul 05 00:09:13.224935 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:09:13.222083    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/d3a9b78d-cf60-4acc-a3c5-016ee34903f8-host\") pod \"node-ca-s547v\" (UID: \"d3a9b78d-cf60-4acc-a3c5-016ee34903f8\") " pod="openshift-image-registry/node-ca-s547v"
Jul 05 00:09:13.224935 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:09:13.222111    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/d3a9b78d-cf60-4acc-a3c5-016ee34903f8-serviceca\") pod \"node-ca-s547v\" (UID: \"d3a9b78d-cf60-4acc-a3c5-016ee34903f8\") " pod="openshift-image-registry/node-ca-s547v"
Jul 05 00:09:13.224935 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:09:13.222137    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/aeaa1e13-243b-4c7f-8045-85ba45081c42-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-g568x\" (UID: \"aeaa1e13-243b-4c7f-8045-85ba45081c42\") " pod="openshift-multus/multus-additional-cni-plugins-g568x"
Jul 05 00:09:13.224935 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:09:13.222198    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/aeaa1e13-243b-4c7f-8045-85ba45081c42-cni-binary-copy\") pod \"multus-additional-cni-plugins-g568x\" (UID: \"aeaa1e13-243b-4c7f-8045-85ba45081c42\") " pod="openshift-multus/multus-additional-cni-plugins-g568x"
Jul 05 00:09:13.224935 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:09:13.222190    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/d3a9b78d-cf60-4acc-a3c5-016ee34903f8-host\") pod \"node-ca-s547v\" (UID: \"d3a9b78d-cf60-4acc-a3c5-016ee34903f8\") " pod="openshift-image-registry/node-ca-s547v"
Jul 05 00:09:13.224935 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:09:13.222273    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/5b9038ef-5e8c-4ee8-9bb7-e9c249715b79-etc-systemd\") pod \"tuned-r2vw8\" (UID: \"5b9038ef-5e8c-4ee8-9bb7-e9c249715b79\") " pod="openshift-cluster-node-tuning-operator/tuned-r2vw8"
Jul 05 00:09:13.224935 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:09:13.222304    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/aeaa1e13-243b-4c7f-8045-85ba45081c42-os-release\") pod \"multus-additional-cni-plugins-g568x\" (UID: \"aeaa1e13-243b-4c7f-8045-85ba45081c42\") " pod="openshift-multus/multus-additional-cni-plugins-g568x"
Jul 05 00:09:13.224935 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:09:13.222341    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/5b9038ef-5e8c-4ee8-9bb7-e9c249715b79-etc-systemd\") pod \"tuned-r2vw8\" (UID: \"5b9038ef-5e8c-4ee8-9bb7-e9c249715b79\") " pod="openshift-cluster-node-tuning-operator/tuned-r2vw8"
Jul 05 00:09:13.224935 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:09:13.222349    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/aeaa1e13-243b-4c7f-8045-85ba45081c42-tuning-conf-dir\") pod \"multus-additional-cni-plugins-g568x\" (UID: \"aeaa1e13-243b-4c7f-8045-85ba45081c42\") " pod="openshift-multus/multus-additional-cni-plugins-g568x"
Jul 05 00:09:13.224935 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:09:13.222367    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/aa4848f6-1480-4acd-af73-e8e055306078-host-run-k8s-cni-cncf-io\") pod \"multus-kwgzc\" (UID: \"aa4848f6-1480-4acd-af73-e8e055306078\") " pod="openshift-multus/multus-kwgzc"
Jul 05 00:09:13.224935 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:09:13.222372    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/aeaa1e13-243b-4c7f-8045-85ba45081c42-os-release\") pod \"multus-additional-cni-plugins-g568x\" (UID: \"aeaa1e13-243b-4c7f-8045-85ba45081c42\") " pod="openshift-multus/multus-additional-cni-plugins-g568x"
Jul 05 00:09:13.224935 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:09:13.222406    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/aa4848f6-1480-4acd-af73-e8e055306078-host-var-lib-cni-bin\") pod \"multus-kwgzc\" (UID: \"aa4848f6-1480-4acd-af73-e8e055306078\") " pod="openshift-multus/multus-kwgzc"
Jul 05 00:09:13.224935 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:09:13.222412    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/aa4848f6-1480-4acd-af73-e8e055306078-host-run-k8s-cni-cncf-io\") pod \"multus-kwgzc\" (UID: \"aa4848f6-1480-4acd-af73-e8e055306078\") " pod="openshift-multus/multus-kwgzc"
Jul 05 00:09:13.224935 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:09:13.222430    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/aa4848f6-1480-4acd-af73-e8e055306078-multus-daemon-config\") pod \"multus-kwgzc\" (UID: \"aa4848f6-1480-4acd-af73-e8e055306078\") " pod="openshift-multus/multus-kwgzc"
Jul 05 00:09:13.224935 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:09:13.222447    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/aa4848f6-1480-4acd-af73-e8e055306078-host-var-lib-cni-bin\") pod \"multus-kwgzc\" (UID: \"aa4848f6-1480-4acd-af73-e8e055306078\") " pod="openshift-multus/multus-kwgzc"
Jul 05 00:09:13.225812 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:09:13.222456    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7kxsj\" (UniqueName: \"kubernetes.io/projected/91e13b36-f504-4b21-b8ae-3b66858bddf2-kube-api-access-7kxsj\") pod \"aws-ebs-csi-driver-node-bj4b9\" (UID: \"91e13b36-f504-4b21-b8ae-3b66858bddf2\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-bj4b9"
Jul 05 00:09:13.225812 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:09:13.222478    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/5b9038ef-5e8c-4ee8-9bb7-e9c249715b79-etc-modprobe-d\") pod \"tuned-r2vw8\" (UID: \"5b9038ef-5e8c-4ee8-9bb7-e9c249715b79\") " pod="openshift-cluster-node-tuning-operator/tuned-r2vw8"
Jul 05 00:09:13.225812 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:09:13.222498    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/5b9038ef-5e8c-4ee8-9bb7-e9c249715b79-etc-sysctl-conf\") pod \"tuned-r2vw8\" (UID: \"5b9038ef-5e8c-4ee8-9bb7-e9c249715b79\") " pod="openshift-cluster-node-tuning-operator/tuned-r2vw8"
Jul 05 00:09:13.225812 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:09:13.222497    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/aeaa1e13-243b-4c7f-8045-85ba45081c42-tuning-conf-dir\") pod \"multus-additional-cni-plugins-g568x\" (UID: \"aeaa1e13-243b-4c7f-8045-85ba45081c42\") " pod="openshift-multus/multus-additional-cni-plugins-g568x"
Jul 05 00:09:13.225812 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:09:13.222535    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/5b9038ef-5e8c-4ee8-9bb7-e9c249715b79-sys\") pod \"tuned-r2vw8\" (UID: \"5b9038ef-5e8c-4ee8-9bb7-e9c249715b79\") " pod="openshift-cluster-node-tuning-operator/tuned-r2vw8"
Jul 05 00:09:13.225812 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:09:13.222564    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/b96bfb4d-2fd3-4be3-9064-e24efeb904f4-host-slash\") pod \"ovnkube-node-7sxq4\" (UID: \"b96bfb4d-2fd3-4be3-9064-e24efeb904f4\") " pod="openshift-ovn-kubernetes/ovnkube-node-7sxq4"
Jul 05 00:09:13.225812 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:09:13.222591    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/91e13b36-f504-4b21-b8ae-3b66858bddf2-registration-dir\") pod \"aws-ebs-csi-driver-node-bj4b9\" (UID: \"91e13b36-f504-4b21-b8ae-3b66858bddf2\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-bj4b9"
Jul 05 00:09:13.225812 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:09:13.222600    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/5b9038ef-5e8c-4ee8-9bb7-e9c249715b79-etc-sysctl-conf\") pod \"tuned-r2vw8\" (UID: \"5b9038ef-5e8c-4ee8-9bb7-e9c249715b79\") " pod="openshift-cluster-node-tuning-operator/tuned-r2vw8"
Jul 05 00:09:13.225812 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:09:13.222616    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/00bb9c2a-9bed-4d69-9ae4-e4f1fad50578-dbus\") pod \"global-pull-secret-syncer-dkwx5\" (UID: \"00bb9c2a-9bed-4d69-9ae4-e4f1fad50578\") " pod="kube-system/global-pull-secret-syncer-dkwx5"
Jul 05 00:09:13.225812 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:09:13.222642    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/aeaa1e13-243b-4c7f-8045-85ba45081c42-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-g568x\" (UID: \"aeaa1e13-243b-4c7f-8045-85ba45081c42\") " pod="openshift-multus/multus-additional-cni-plugins-g568x"
Jul 05 00:09:13.225812 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:09:13.222651    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/d3a9b78d-cf60-4acc-a3c5-016ee34903f8-serviceca\") pod \"node-ca-s547v\" (UID: \"d3a9b78d-cf60-4acc-a3c5-016ee34903f8\") " pod="openshift-image-registry/node-ca-s547v"
Jul 05 00:09:13.225812 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:09:13.222666    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/00bb9c2a-9bed-4d69-9ae4-e4f1fad50578-original-pull-secret\") pod \"global-pull-secret-syncer-dkwx5\" (UID: \"00bb9c2a-9bed-4d69-9ae4-e4f1fad50578\") " pod="kube-system/global-pull-secret-syncer-dkwx5"
Jul 05 00:09:13.225812 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:09:13.222710    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/cecee17b-8d41-4a16-a3b2-53919db096bb-tmp-dir\") pod \"node-resolver-kjz9w\" (UID: \"cecee17b-8d41-4a16-a3b2-53919db096bb\") " pod="openshift-dns/node-resolver-kjz9w"
Jul 05 00:09:13.225812 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:09:13.222716    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/91e13b36-f504-4b21-b8ae-3b66858bddf2-registration-dir\") pod \"aws-ebs-csi-driver-node-bj4b9\" (UID: \"91e13b36-f504-4b21-b8ae-3b66858bddf2\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-bj4b9"
Jul 05 00:09:13.225812 ip-10-0-142-202 kubenswrapper[2563]: E0705 00:09:13.222738    2563 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 05 00:09:13.225812 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:09:13.222739    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/5b9038ef-5e8c-4ee8-9bb7-e9c249715b79-etc-sysctl-d\") pod \"tuned-r2vw8\" (UID: \"5b9038ef-5e8c-4ee8-9bb7-e9c249715b79\") " pod="openshift-cluster-node-tuning-operator/tuned-r2vw8"
Jul 05 00:09:13.225812 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:09:13.222759    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/5b9038ef-5e8c-4ee8-9bb7-e9c249715b79-sys\") pod \"tuned-r2vw8\" (UID: \"5b9038ef-5e8c-4ee8-9bb7-e9c249715b79\") " pod="openshift-cluster-node-tuning-operator/tuned-r2vw8"
Jul 05 00:09:13.225812 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:09:13.222786    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/5b9038ef-5e8c-4ee8-9bb7-e9c249715b79-etc-modprobe-d\") pod \"tuned-r2vw8\" (UID: \"5b9038ef-5e8c-4ee8-9bb7-e9c249715b79\") " pod="openshift-cluster-node-tuning-operator/tuned-r2vw8"
Jul 05 00:09:13.226766 ip-10-0-142-202 kubenswrapper[2563]: E0705 00:09:13.222796    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/00bb9c2a-9bed-4d69-9ae4-e4f1fad50578-original-pull-secret podName:00bb9c2a-9bed-4d69-9ae4-e4f1fad50578 nodeName:}" failed. No retries permitted until 2026-07-05 00:09:13.722781452 +0000 UTC m=+2.065110501 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/00bb9c2a-9bed-4d69-9ae4-e4f1fad50578-original-pull-secret") pod "global-pull-secret-syncer-dkwx5" (UID: "00bb9c2a-9bed-4d69-9ae4-e4f1fad50578") : object "kube-system"/"original-pull-secret" not registered
Jul 05 00:09:13.226766 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:09:13.222856    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/00bb9c2a-9bed-4d69-9ae4-e4f1fad50578-dbus\") pod \"global-pull-secret-syncer-dkwx5\" (UID: \"00bb9c2a-9bed-4d69-9ae4-e4f1fad50578\") " pod="kube-system/global-pull-secret-syncer-dkwx5"
Jul 05 00:09:13.226766 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:09:13.222864    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/b96bfb4d-2fd3-4be3-9064-e24efeb904f4-host-slash\") pod \"ovnkube-node-7sxq4\" (UID: \"b96bfb4d-2fd3-4be3-9064-e24efeb904f4\") " pod="openshift-ovn-kubernetes/ovnkube-node-7sxq4"
Jul 05 00:09:13.226766 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:09:13.222867    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/aa4848f6-1480-4acd-af73-e8e055306078-multus-daemon-config\") pod \"multus-kwgzc\" (UID: \"aa4848f6-1480-4acd-af73-e8e055306078\") " pod="openshift-multus/multus-kwgzc"
Jul 05 00:09:13.226766 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:09:13.222882    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/5b9038ef-5e8c-4ee8-9bb7-e9c249715b79-etc-sysctl-d\") pod \"tuned-r2vw8\" (UID: \"5b9038ef-5e8c-4ee8-9bb7-e9c249715b79\") " pod="openshift-cluster-node-tuning-operator/tuned-r2vw8"
Jul 05 00:09:13.226766 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:09:13.222867    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/b96bfb4d-2fd3-4be3-9064-e24efeb904f4-log-socket\") pod \"ovnkube-node-7sxq4\" (UID: \"b96bfb4d-2fd3-4be3-9064-e24efeb904f4\") " pod="openshift-ovn-kubernetes/ovnkube-node-7sxq4"
Jul 05 00:09:13.226766 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:09:13.222924    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/b96bfb4d-2fd3-4be3-9064-e24efeb904f4-log-socket\") pod \"ovnkube-node-7sxq4\" (UID: \"b96bfb4d-2fd3-4be3-9064-e24efeb904f4\") " pod="openshift-ovn-kubernetes/ovnkube-node-7sxq4"
Jul 05 00:09:13.226766 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:09:13.222926    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/aa4848f6-1480-4acd-af73-e8e055306078-hostroot\") pod \"multus-kwgzc\" (UID: \"aa4848f6-1480-4acd-af73-e8e055306078\") " pod="openshift-multus/multus-kwgzc"
Jul 05 00:09:13.226766 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:09:13.222958    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/aa4848f6-1480-4acd-af73-e8e055306078-hostroot\") pod \"multus-kwgzc\" (UID: \"aa4848f6-1480-4acd-af73-e8e055306078\") " pod="openshift-multus/multus-kwgzc"
Jul 05 00:09:13.226766 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:09:13.222966    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hxt4x\" (UniqueName: \"kubernetes.io/projected/8a2c0b38-b2ce-4570-8963-2f75863cc6b2-kube-api-access-hxt4x\") pod \"iptables-alerter-8tjwr\" (UID: \"8a2c0b38-b2ce-4570-8963-2f75863cc6b2\") " pod="openshift-network-operator/iptables-alerter-8tjwr"
Jul 05 00:09:13.226766 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:09:13.223008    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/5b9038ef-5e8c-4ee8-9bb7-e9c249715b79-etc-sysconfig\") pod \"tuned-r2vw8\" (UID: \"5b9038ef-5e8c-4ee8-9bb7-e9c249715b79\") " pod="openshift-cluster-node-tuning-operator/tuned-r2vw8"
Jul 05 00:09:13.226766 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:09:13.223049    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/b96bfb4d-2fd3-4be3-9064-e24efeb904f4-etc-openvswitch\") pod \"ovnkube-node-7sxq4\" (UID: \"b96bfb4d-2fd3-4be3-9064-e24efeb904f4\") " pod="openshift-ovn-kubernetes/ovnkube-node-7sxq4"
Jul 05 00:09:13.226766 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:09:13.223041    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/cecee17b-8d41-4a16-a3b2-53919db096bb-tmp-dir\") pod \"node-resolver-kjz9w\" (UID: \"cecee17b-8d41-4a16-a3b2-53919db096bb\") " pod="openshift-dns/node-resolver-kjz9w"
Jul 05 00:09:13.226766 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:09:13.223078    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/91e13b36-f504-4b21-b8ae-3b66858bddf2-device-dir\") pod \"aws-ebs-csi-driver-node-bj4b9\" (UID: \"91e13b36-f504-4b21-b8ae-3b66858bddf2\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-bj4b9"
Jul 05 00:09:13.226766 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:09:13.223088    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/5b9038ef-5e8c-4ee8-9bb7-e9c249715b79-etc-sysconfig\") pod \"tuned-r2vw8\" (UID: \"5b9038ef-5e8c-4ee8-9bb7-e9c249715b79\") " pod="openshift-cluster-node-tuning-operator/tuned-r2vw8"
Jul 05 00:09:13.226766 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:09:13.223108    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/1f3c24e8-591d-4e53-9b63-091f6f1dec51-agent-certs\") pod \"konnectivity-agent-rwbw4\" (UID: \"1f3c24e8-591d-4e53-9b63-091f6f1dec51\") " pod="kube-system/konnectivity-agent-rwbw4"
Jul 05 00:09:13.226766 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:09:13.223133    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/5b9038ef-5e8c-4ee8-9bb7-e9c249715b79-etc-tuned\") pod \"tuned-r2vw8\" (UID: \"5b9038ef-5e8c-4ee8-9bb7-e9c249715b79\") " pod="openshift-cluster-node-tuning-operator/tuned-r2vw8"
Jul 05 00:09:13.227719 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:09:13.223136    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/b96bfb4d-2fd3-4be3-9064-e24efeb904f4-etc-openvswitch\") pod \"ovnkube-node-7sxq4\" (UID: \"b96bfb4d-2fd3-4be3-9064-e24efeb904f4\") " pod="openshift-ovn-kubernetes/ovnkube-node-7sxq4"
Jul 05 00:09:13.227719 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:09:13.223150    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/91e13b36-f504-4b21-b8ae-3b66858bddf2-device-dir\") pod \"aws-ebs-csi-driver-node-bj4b9\" (UID: \"91e13b36-f504-4b21-b8ae-3b66858bddf2\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-bj4b9"
Jul 05 00:09:13.227719 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:09:13.223203    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/b96bfb4d-2fd3-4be3-9064-e24efeb904f4-run-systemd\") pod \"ovnkube-node-7sxq4\" (UID: \"b96bfb4d-2fd3-4be3-9064-e24efeb904f4\") " pod="openshift-ovn-kubernetes/ovnkube-node-7sxq4"
Jul 05 00:09:13.227719 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:09:13.223233    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/b96bfb4d-2fd3-4be3-9064-e24efeb904f4-run-ovn\") pod \"ovnkube-node-7sxq4\" (UID: \"b96bfb4d-2fd3-4be3-9064-e24efeb904f4\") " pod="openshift-ovn-kubernetes/ovnkube-node-7sxq4"
Jul 05 00:09:13.227719 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:09:13.225742    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/5b9038ef-5e8c-4ee8-9bb7-e9c249715b79-tmp\") pod \"tuned-r2vw8\" (UID: \"5b9038ef-5e8c-4ee8-9bb7-e9c249715b79\") " pod="openshift-cluster-node-tuning-operator/tuned-r2vw8"
Jul 05 00:09:13.227719 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:09:13.225868    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/5b9038ef-5e8c-4ee8-9bb7-e9c249715b79-etc-tuned\") pod \"tuned-r2vw8\" (UID: \"5b9038ef-5e8c-4ee8-9bb7-e9c249715b79\") " pod="openshift-cluster-node-tuning-operator/tuned-r2vw8"
Jul 05 00:09:13.227719 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:09:13.225893    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/b96bfb4d-2fd3-4be3-9064-e24efeb904f4-ovn-node-metrics-cert\") pod \"ovnkube-node-7sxq4\" (UID: \"b96bfb4d-2fd3-4be3-9064-e24efeb904f4\") " pod="openshift-ovn-kubernetes/ovnkube-node-7sxq4"
Jul 05 00:09:13.227719 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:09:13.226110    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/1f3c24e8-591d-4e53-9b63-091f6f1dec51-agent-certs\") pod \"konnectivity-agent-rwbw4\" (UID: \"1f3c24e8-591d-4e53-9b63-091f6f1dec51\") " pod="kube-system/konnectivity-agent-rwbw4"
Jul 05 00:09:13.227719 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:09:13.226177    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/b96bfb4d-2fd3-4be3-9064-e24efeb904f4-run-ovn\") pod \"ovnkube-node-7sxq4\" (UID: \"b96bfb4d-2fd3-4be3-9064-e24efeb904f4\") " pod="openshift-ovn-kubernetes/ovnkube-node-7sxq4"
Jul 05 00:09:13.227719 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:09:13.226223    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/b96bfb4d-2fd3-4be3-9064-e24efeb904f4-run-systemd\") pod \"ovnkube-node-7sxq4\" (UID: \"b96bfb4d-2fd3-4be3-9064-e24efeb904f4\") " pod="openshift-ovn-kubernetes/ovnkube-node-7sxq4"
Jul 05 00:09:13.227719 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:09:13.226262    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/b96bfb4d-2fd3-4be3-9064-e24efeb904f4-host-cni-netd\") pod \"ovnkube-node-7sxq4\" (UID: \"b96bfb4d-2fd3-4be3-9064-e24efeb904f4\") " pod="openshift-ovn-kubernetes/ovnkube-node-7sxq4"
Jul 05 00:09:13.227719 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:09:13.226340    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/b96bfb4d-2fd3-4be3-9064-e24efeb904f4-ovnkube-script-lib\") pod \"ovnkube-node-7sxq4\" (UID: \"b96bfb4d-2fd3-4be3-9064-e24efeb904f4\") " pod="openshift-ovn-kubernetes/ovnkube-node-7sxq4"
Jul 05 00:09:13.227719 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:09:13.226368    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/aa4848f6-1480-4acd-af73-e8e055306078-host-run-netns\") pod \"multus-kwgzc\" (UID: \"aa4848f6-1480-4acd-af73-e8e055306078\") " pod="openshift-multus/multus-kwgzc"
Jul 05 00:09:13.227719 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:09:13.226393    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/aa4848f6-1480-4acd-af73-e8e055306078-host-var-lib-kubelet\") pod \"multus-kwgzc\" (UID: \"aa4848f6-1480-4acd-af73-e8e055306078\") " pod="openshift-multus/multus-kwgzc"
Jul 05 00:09:13.227719 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:09:13.226417    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/b96bfb4d-2fd3-4be3-9064-e24efeb904f4-host-cni-netd\") pod \"ovnkube-node-7sxq4\" (UID: \"b96bfb4d-2fd3-4be3-9064-e24efeb904f4\") " pod="openshift-ovn-kubernetes/ovnkube-node-7sxq4"
Jul 05 00:09:13.227719 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:09:13.226420    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rpxd6\" (UniqueName: \"kubernetes.io/projected/aa4848f6-1480-4acd-af73-e8e055306078-kube-api-access-rpxd6\") pod \"multus-kwgzc\" (UID: \"aa4848f6-1480-4acd-af73-e8e055306078\") " pod="openshift-multus/multus-kwgzc"
Jul 05 00:09:13.227719 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:09:13.226470    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/5b9038ef-5e8c-4ee8-9bb7-e9c249715b79-host\") pod \"tuned-r2vw8\" (UID: \"5b9038ef-5e8c-4ee8-9bb7-e9c249715b79\") " pod="openshift-cluster-node-tuning-operator/tuned-r2vw8"
Jul 05 00:09:13.227719 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:09:13.226500    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/b96bfb4d-2fd3-4be3-9064-e24efeb904f4-env-overrides\") pod \"ovnkube-node-7sxq4\" (UID: \"b96bfb4d-2fd3-4be3-9064-e24efeb904f4\") " pod="openshift-ovn-kubernetes/ovnkube-node-7sxq4"
Jul 05 00:09:13.228691 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:09:13.226524    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/aeaa1e13-243b-4c7f-8045-85ba45081c42-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-g568x\" (UID: \"aeaa1e13-243b-4c7f-8045-85ba45081c42\") " pod="openshift-multus/multus-additional-cni-plugins-g568x"
Jul 05 00:09:13.228691 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:09:13.226552    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/aa4848f6-1480-4acd-af73-e8e055306078-system-cni-dir\") pod \"multus-kwgzc\" (UID: \"aa4848f6-1480-4acd-af73-e8e055306078\") " pod="openshift-multus/multus-kwgzc"
Jul 05 00:09:13.228691 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:09:13.226632    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/aa4848f6-1480-4acd-af73-e8e055306078-system-cni-dir\") pod \"multus-kwgzc\" (UID: \"aa4848f6-1480-4acd-af73-e8e055306078\") " pod="openshift-multus/multus-kwgzc"
Jul 05 00:09:13.228691 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:09:13.226683    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/5b9038ef-5e8c-4ee8-9bb7-e9c249715b79-host\") pod \"tuned-r2vw8\" (UID: \"5b9038ef-5e8c-4ee8-9bb7-e9c249715b79\") " pod="openshift-cluster-node-tuning-operator/tuned-r2vw8"
Jul 05 00:09:13.228691 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:09:13.227103    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/b96bfb4d-2fd3-4be3-9064-e24efeb904f4-ovnkube-script-lib\") pod \"ovnkube-node-7sxq4\" (UID: \"b96bfb4d-2fd3-4be3-9064-e24efeb904f4\") " pod="openshift-ovn-kubernetes/ovnkube-node-7sxq4"
Jul 05 00:09:13.228691 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:09:13.227167    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/aa4848f6-1480-4acd-af73-e8e055306078-host-run-netns\") pod \"multus-kwgzc\" (UID: \"aa4848f6-1480-4acd-af73-e8e055306078\") " pod="openshift-multus/multus-kwgzc"
Jul 05 00:09:13.228691 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:09:13.227215    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/aa4848f6-1480-4acd-af73-e8e055306078-host-var-lib-kubelet\") pod \"multus-kwgzc\" (UID: \"aa4848f6-1480-4acd-af73-e8e055306078\") " pod="openshift-multus/multus-kwgzc"
Jul 05 00:09:13.228691 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:09:13.227579    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/b96bfb4d-2fd3-4be3-9064-e24efeb904f4-env-overrides\") pod \"ovnkube-node-7sxq4\" (UID: \"b96bfb4d-2fd3-4be3-9064-e24efeb904f4\") " pod="openshift-ovn-kubernetes/ovnkube-node-7sxq4"
Jul 05 00:09:13.228691 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:09:13.227708    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/aeaa1e13-243b-4c7f-8045-85ba45081c42-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-g568x\" (UID: \"aeaa1e13-243b-4c7f-8045-85ba45081c42\") " pod="openshift-multus/multus-additional-cni-plugins-g568x"
Jul 05 00:09:13.228691 ip-10-0-142-202 kubenswrapper[2563]: W0705 00:09:13.228137    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podec580e48b6696694176fa0643f6420e5.slice/crio-44a4b7f9b8da55b24865a44f21986f152a1c7ed03c1556de9b38e06e502c33ac WatchSource:0}: Error finding container 44a4b7f9b8da55b24865a44f21986f152a1c7ed03c1556de9b38e06e502c33ac: Status 404 returned error can't find the container with id 44a4b7f9b8da55b24865a44f21986f152a1c7ed03c1556de9b38e06e502c33ac
Jul 05 00:09:13.228691 ip-10-0-142-202 kubenswrapper[2563]: W0705 00:09:13.228530    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod34a0b5b76f9b2c1d92188a84f4deb896.slice/crio-9dcfd62b145a46d4b51da376ed623db6d499fa415f89f3a24d675349637b37ee WatchSource:0}: Error finding container 9dcfd62b145a46d4b51da376ed623db6d499fa415f89f3a24d675349637b37ee: Status 404 returned error can't find the container with id 9dcfd62b145a46d4b51da376ed623db6d499fa415f89f3a24d675349637b37ee
Jul 05 00:09:13.229259 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:09:13.228994    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-89gv4\" (UniqueName: \"kubernetes.io/projected/d3a9b78d-cf60-4acc-a3c5-016ee34903f8-kube-api-access-89gv4\") pod \"node-ca-s547v\" (UID: \"d3a9b78d-cf60-4acc-a3c5-016ee34903f8\") " pod="openshift-image-registry/node-ca-s547v"
Jul 05 00:09:13.230469 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:09:13.230441    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-2rtk9\" (UniqueName: \"kubernetes.io/projected/cecee17b-8d41-4a16-a3b2-53919db096bb-kube-api-access-2rtk9\") pod \"node-resolver-kjz9w\" (UID: \"cecee17b-8d41-4a16-a3b2-53919db096bb\") " pod="openshift-dns/node-resolver-kjz9w"
Jul 05 00:09:13.230583 ip-10-0-142-202 kubenswrapper[2563]: E0705 00:09:13.230536    2563 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 05 00:09:13.230583 ip-10-0-142-202 kubenswrapper[2563]: E0705 00:09:13.230555    2563 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 05 00:09:13.230583 ip-10-0-142-202 kubenswrapper[2563]: E0705 00:09:13.230568    2563 projected.go:196] Error preparing data for projected volume kube-api-access-z4cn8 for pod openshift-network-diagnostics/network-check-target-4pm8c: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 05 00:09:13.230756 ip-10-0-142-202 kubenswrapper[2563]: E0705 00:09:13.230638    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/e84aff2e-272a-42d8-8bd9-e37a98cfc7a7-kube-api-access-z4cn8 podName:e84aff2e-272a-42d8-8bd9-e37a98cfc7a7 nodeName:}" failed. No retries permitted until 2026-07-05 00:09:13.730619883 +0000 UTC m=+2.072948945 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-z4cn8" (UniqueName: "kubernetes.io/projected/e84aff2e-272a-42d8-8bd9-e37a98cfc7a7-kube-api-access-z4cn8") pod "network-check-target-4pm8c" (UID: "e84aff2e-272a-42d8-8bd9-e37a98cfc7a7") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 05 00:09:13.231366 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:09:13.231340    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-gz6zl\" (UniqueName: \"kubernetes.io/projected/b96bfb4d-2fd3-4be3-9064-e24efeb904f4-kube-api-access-gz6zl\") pod \"ovnkube-node-7sxq4\" (UID: \"b96bfb4d-2fd3-4be3-9064-e24efeb904f4\") " pod="openshift-ovn-kubernetes/ovnkube-node-7sxq4"
Jul 05 00:09:13.231366 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:09:13.231358    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-vntcz\" (UniqueName: \"kubernetes.io/projected/5b9038ef-5e8c-4ee8-9bb7-e9c249715b79-kube-api-access-vntcz\") pod \"tuned-r2vw8\" (UID: \"5b9038ef-5e8c-4ee8-9bb7-e9c249715b79\") " pod="openshift-cluster-node-tuning-operator/tuned-r2vw8"
Jul 05 00:09:13.231603 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:09:13.231581    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-mqrck\" (UniqueName: \"kubernetes.io/projected/c085d4be-fcda-4931-a018-02cbaca17c59-kube-api-access-mqrck\") pod \"network-metrics-daemon-5fn94\" (UID: \"c085d4be-fcda-4931-a018-02cbaca17c59\") " pod="openshift-multus/network-metrics-daemon-5fn94"
Jul 05 00:09:13.231732 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:09:13.231710    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-hxt4x\" (UniqueName: \"kubernetes.io/projected/8a2c0b38-b2ce-4570-8963-2f75863cc6b2-kube-api-access-hxt4x\") pod \"iptables-alerter-8tjwr\" (UID: \"8a2c0b38-b2ce-4570-8963-2f75863cc6b2\") " pod="openshift-network-operator/iptables-alerter-8tjwr"
Jul 05 00:09:13.232430 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:09:13.232411    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-hncnr\" (UniqueName: \"kubernetes.io/projected/aeaa1e13-243b-4c7f-8045-85ba45081c42-kube-api-access-hncnr\") pod \"multus-additional-cni-plugins-g568x\" (UID: \"aeaa1e13-243b-4c7f-8045-85ba45081c42\") " pod="openshift-multus/multus-additional-cni-plugins-g568x"
Jul 05 00:09:13.232867 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:09:13.232849    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-7kxsj\" (UniqueName: \"kubernetes.io/projected/91e13b36-f504-4b21-b8ae-3b66858bddf2-kube-api-access-7kxsj\") pod \"aws-ebs-csi-driver-node-bj4b9\" (UID: \"91e13b36-f504-4b21-b8ae-3b66858bddf2\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-bj4b9"
Jul 05 00:09:13.233787 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:09:13.233770    2563 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 05 00:09:13.235550 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:09:13.235532    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-rpxd6\" (UniqueName: \"kubernetes.io/projected/aa4848f6-1480-4acd-af73-e8e055306078-kube-api-access-rpxd6\") pod \"multus-kwgzc\" (UID: \"aa4848f6-1480-4acd-af73-e8e055306078\") " pod="openshift-multus/multus-kwgzc"
Jul 05 00:09:13.361736 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:09:13.361631    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-bj4b9"
Jul 05 00:09:13.371937 ip-10-0-142-202 kubenswrapper[2563]: W0705 00:09:13.371911    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod91e13b36_f504_4b21_b8ae_3b66858bddf2.slice/crio-5e00d48e26d5f2efec7e231a458afc3aa76579ca2562bb5e86cf4862b9d2bcd2 WatchSource:0}: Error finding container 5e00d48e26d5f2efec7e231a458afc3aa76579ca2562bb5e86cf4862b9d2bcd2: Status 404 returned error can't find the container with id 5e00d48e26d5f2efec7e231a458afc3aa76579ca2562bb5e86cf4862b9d2bcd2
Jul 05 00:09:13.384845 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:09:13.384823    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-r2vw8"
Jul 05 00:09:13.391714 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:09:13.391691    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-8tjwr"
Jul 05 00:09:13.393701 ip-10-0-142-202 kubenswrapper[2563]: W0705 00:09:13.393675    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5b9038ef_5e8c_4ee8_9bb7_e9c249715b79.slice/crio-e5f4aca93408143f8e7de3f444457221dd062a71ce777c6052ae32ff311c8f76 WatchSource:0}: Error finding container e5f4aca93408143f8e7de3f444457221dd062a71ce777c6052ae32ff311c8f76: Status 404 returned error can't find the container with id e5f4aca93408143f8e7de3f444457221dd062a71ce777c6052ae32ff311c8f76
Jul 05 00:09:13.400795 ip-10-0-142-202 kubenswrapper[2563]: W0705 00:09:13.400771    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8a2c0b38_b2ce_4570_8963_2f75863cc6b2.slice/crio-02e4866d200be09af493e69a1f7cc3145c65854333d1d81ce4208baf5bedb451 WatchSource:0}: Error finding container 02e4866d200be09af493e69a1f7cc3145c65854333d1d81ce4208baf5bedb451: Status 404 returned error can't find the container with id 02e4866d200be09af493e69a1f7cc3145c65854333d1d81ce4208baf5bedb451
Jul 05 00:09:13.419620 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:09:13.419601    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-7sxq4"
Jul 05 00:09:13.428141 ip-10-0-142-202 kubenswrapper[2563]: W0705 00:09:13.428120    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb96bfb4d_2fd3_4be3_9064_e24efeb904f4.slice/crio-addb015676c73a1f4e4e33ac3bab1b301d538bb8394a42b0f9abc6e94d8c553a WatchSource:0}: Error finding container addb015676c73a1f4e4e33ac3bab1b301d538bb8394a42b0f9abc6e94d8c553a: Status 404 returned error can't find the container with id addb015676c73a1f4e4e33ac3bab1b301d538bb8394a42b0f9abc6e94d8c553a
Jul 05 00:09:13.430264 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:09:13.430248    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-rwbw4"
Jul 05 00:09:13.437245 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:09:13.437211    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-kjz9w"
Jul 05 00:09:13.438407 ip-10-0-142-202 kubenswrapper[2563]: W0705 00:09:13.438385    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1f3c24e8_591d_4e53_9b63_091f6f1dec51.slice/crio-9682fd170d427c013e161d2802e8466a5387c52107d1ae9b62120f20720a598b WatchSource:0}: Error finding container 9682fd170d427c013e161d2802e8466a5387c52107d1ae9b62120f20720a598b: Status 404 returned error can't find the container with id 9682fd170d427c013e161d2802e8466a5387c52107d1ae9b62120f20720a598b
Jul 05 00:09:13.445419 ip-10-0-142-202 kubenswrapper[2563]: W0705 00:09:13.445394    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcecee17b_8d41_4a16_a3b2_53919db096bb.slice/crio-19f4682aa0c6c0051e09d78d226928762937c4c48cdcf0c5b99a997658be1ce0 WatchSource:0}: Error finding container 19f4682aa0c6c0051e09d78d226928762937c4c48cdcf0c5b99a997658be1ce0: Status 404 returned error can't find the container with id 19f4682aa0c6c0051e09d78d226928762937c4c48cdcf0c5b99a997658be1ce0
Jul 05 00:09:13.451452 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:09:13.451432    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-s547v"
Jul 05 00:09:13.459152 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:09:13.459131    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-g568x"
Jul 05 00:09:13.459413 ip-10-0-142-202 kubenswrapper[2563]: W0705 00:09:13.459395    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd3a9b78d_cf60_4acc_a3c5_016ee34903f8.slice/crio-f2fcef79e8356b3ce39739ec6eb47d33448ef162b06e5d9f9a1c36fb03f07eff WatchSource:0}: Error finding container f2fcef79e8356b3ce39739ec6eb47d33448ef162b06e5d9f9a1c36fb03f07eff: Status 404 returned error can't find the container with id f2fcef79e8356b3ce39739ec6eb47d33448ef162b06e5d9f9a1c36fb03f07eff
Jul 05 00:09:13.465059 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:09:13.465042    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-kwgzc"
Jul 05 00:09:13.467002 ip-10-0-142-202 kubenswrapper[2563]: W0705 00:09:13.466983    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podaeaa1e13_243b_4c7f_8045_85ba45081c42.slice/crio-05eaa3ba9d66b8a6c8720d0a8a09db88f4d63709cd8094343cafd7c999700a8b WatchSource:0}: Error finding container 05eaa3ba9d66b8a6c8720d0a8a09db88f4d63709cd8094343cafd7c999700a8b: Status 404 returned error can't find the container with id 05eaa3ba9d66b8a6c8720d0a8a09db88f4d63709cd8094343cafd7c999700a8b
Jul 05 00:09:13.476956 ip-10-0-142-202 kubenswrapper[2563]: W0705 00:09:13.476937    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podaa4848f6_1480_4acd_af73_e8e055306078.slice/crio-f28a3fc6a8cbb3e11602c44943c535536e1511435b103a0732cbcd310464fc9a WatchSource:0}: Error finding container f28a3fc6a8cbb3e11602c44943c535536e1511435b103a0732cbcd310464fc9a: Status 404 returned error can't find the container with id f28a3fc6a8cbb3e11602c44943c535536e1511435b103a0732cbcd310464fc9a
Jul 05 00:09:13.729120 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:09:13.729026    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/00bb9c2a-9bed-4d69-9ae4-e4f1fad50578-original-pull-secret\") pod \"global-pull-secret-syncer-dkwx5\" (UID: \"00bb9c2a-9bed-4d69-9ae4-e4f1fad50578\") " pod="kube-system/global-pull-secret-syncer-dkwx5"
Jul 05 00:09:13.729287 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:09:13.729176    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c085d4be-fcda-4931-a018-02cbaca17c59-metrics-certs\") pod \"network-metrics-daemon-5fn94\" (UID: \"c085d4be-fcda-4931-a018-02cbaca17c59\") " pod="openshift-multus/network-metrics-daemon-5fn94"
Jul 05 00:09:13.729287 ip-10-0-142-202 kubenswrapper[2563]: E0705 00:09:13.729188    2563 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 05 00:09:13.729287 ip-10-0-142-202 kubenswrapper[2563]: E0705 00:09:13.729282    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/00bb9c2a-9bed-4d69-9ae4-e4f1fad50578-original-pull-secret podName:00bb9c2a-9bed-4d69-9ae4-e4f1fad50578 nodeName:}" failed. No retries permitted until 2026-07-05 00:09:14.729260243 +0000 UTC m=+3.071589297 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/00bb9c2a-9bed-4d69-9ae4-e4f1fad50578-original-pull-secret") pod "global-pull-secret-syncer-dkwx5" (UID: "00bb9c2a-9bed-4d69-9ae4-e4f1fad50578") : object "kube-system"/"original-pull-secret" not registered
Jul 05 00:09:13.729468 ip-10-0-142-202 kubenswrapper[2563]: E0705 00:09:13.729302    2563 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 05 00:09:13.729468 ip-10-0-142-202 kubenswrapper[2563]: E0705 00:09:13.729354    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c085d4be-fcda-4931-a018-02cbaca17c59-metrics-certs podName:c085d4be-fcda-4931-a018-02cbaca17c59 nodeName:}" failed. No retries permitted until 2026-07-05 00:09:14.729336894 +0000 UTC m=+3.071665938 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/c085d4be-fcda-4931-a018-02cbaca17c59-metrics-certs") pod "network-metrics-daemon-5fn94" (UID: "c085d4be-fcda-4931-a018-02cbaca17c59") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 05 00:09:13.830319 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:09:13.830187    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-z4cn8\" (UniqueName: \"kubernetes.io/projected/e84aff2e-272a-42d8-8bd9-e37a98cfc7a7-kube-api-access-z4cn8\") pod \"network-check-target-4pm8c\" (UID: \"e84aff2e-272a-42d8-8bd9-e37a98cfc7a7\") " pod="openshift-network-diagnostics/network-check-target-4pm8c"
Jul 05 00:09:13.830502 ip-10-0-142-202 kubenswrapper[2563]: E0705 00:09:13.830381    2563 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 05 00:09:13.830502 ip-10-0-142-202 kubenswrapper[2563]: E0705 00:09:13.830403    2563 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 05 00:09:13.830502 ip-10-0-142-202 kubenswrapper[2563]: E0705 00:09:13.830415    2563 projected.go:196] Error preparing data for projected volume kube-api-access-z4cn8 for pod openshift-network-diagnostics/network-check-target-4pm8c: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 05 00:09:13.830502 ip-10-0-142-202 kubenswrapper[2563]: E0705 00:09:13.830469    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/e84aff2e-272a-42d8-8bd9-e37a98cfc7a7-kube-api-access-z4cn8 podName:e84aff2e-272a-42d8-8bd9-e37a98cfc7a7 nodeName:}" failed. No retries permitted until 2026-07-05 00:09:14.830450476 +0000 UTC m=+3.172779529 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-z4cn8" (UniqueName: "kubernetes.io/projected/e84aff2e-272a-42d8-8bd9-e37a98cfc7a7-kube-api-access-z4cn8") pod "network-check-target-4pm8c" (UID: "e84aff2e-272a-42d8-8bd9-e37a98cfc7a7") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 05 00:09:14.057963 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:09:14.057778    2563 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-04 00:04:13 +0000 UTC" deadline="2028-02-24 00:24:14.623802557 +0000 UTC"
Jul 05 00:09:14.057963 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:09:14.057846    2563 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="14376h15m0.565961012s"
Jul 05 00:09:14.148775 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:09:14.148653    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-142-202.ec2.internal" event={"ID":"ec580e48b6696694176fa0643f6420e5","Type":"ContainerStarted","Data":"44a4b7f9b8da55b24865a44f21986f152a1c7ed03c1556de9b38e06e502c33ac"}
Jul 05 00:09:14.155233 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:09:14.155188    2563 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 05 00:09:14.168997 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:09:14.168960    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-s547v" event={"ID":"d3a9b78d-cf60-4acc-a3c5-016ee34903f8","Type":"ContainerStarted","Data":"f2fcef79e8356b3ce39739ec6eb47d33448ef162b06e5d9f9a1c36fb03f07eff"}
Jul 05 00:09:14.178480 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:09:14.178394    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-kjz9w" event={"ID":"cecee17b-8d41-4a16-a3b2-53919db096bb","Type":"ContainerStarted","Data":"19f4682aa0c6c0051e09d78d226928762937c4c48cdcf0c5b99a997658be1ce0"}
Jul 05 00:09:14.181822 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:09:14.181738    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-8tjwr" event={"ID":"8a2c0b38-b2ce-4570-8963-2f75863cc6b2","Type":"ContainerStarted","Data":"02e4866d200be09af493e69a1f7cc3145c65854333d1d81ce4208baf5bedb451"}
Jul 05 00:09:14.197837 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:09:14.197806    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-bj4b9" event={"ID":"91e13b36-f504-4b21-b8ae-3b66858bddf2","Type":"ContainerStarted","Data":"5e00d48e26d5f2efec7e231a458afc3aa76579ca2562bb5e86cf4862b9d2bcd2"}
Jul 05 00:09:14.215222 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:09:14.215190    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-202.ec2.internal" event={"ID":"34a0b5b76f9b2c1d92188a84f4deb896","Type":"ContainerStarted","Data":"9dcfd62b145a46d4b51da376ed623db6d499fa415f89f3a24d675349637b37ee"}
Jul 05 00:09:14.230535 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:09:14.230506    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-kwgzc" event={"ID":"aa4848f6-1480-4acd-af73-e8e055306078","Type":"ContainerStarted","Data":"f28a3fc6a8cbb3e11602c44943c535536e1511435b103a0732cbcd310464fc9a"}
Jul 05 00:09:14.249285 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:09:14.249218    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-g568x" event={"ID":"aeaa1e13-243b-4c7f-8045-85ba45081c42","Type":"ContainerStarted","Data":"05eaa3ba9d66b8a6c8720d0a8a09db88f4d63709cd8094343cafd7c999700a8b"}
Jul 05 00:09:14.269689 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:09:14.269616    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-rwbw4" event={"ID":"1f3c24e8-591d-4e53-9b63-091f6f1dec51","Type":"ContainerStarted","Data":"9682fd170d427c013e161d2802e8466a5387c52107d1ae9b62120f20720a598b"}
Jul 05 00:09:14.299852 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:09:14.299820    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-7sxq4" event={"ID":"b96bfb4d-2fd3-4be3-9064-e24efeb904f4","Type":"ContainerStarted","Data":"addb015676c73a1f4e4e33ac3bab1b301d538bb8394a42b0f9abc6e94d8c553a"}
Jul 05 00:09:14.316489 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:09:14.316408    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-r2vw8" event={"ID":"5b9038ef-5e8c-4ee8-9bb7-e9c249715b79","Type":"ContainerStarted","Data":"e5f4aca93408143f8e7de3f444457221dd062a71ce777c6052ae32ff311c8f76"}
Jul 05 00:09:14.330327 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:09:14.330280    2563 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 05 00:09:14.737958 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:09:14.737868    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/00bb9c2a-9bed-4d69-9ae4-e4f1fad50578-original-pull-secret\") pod \"global-pull-secret-syncer-dkwx5\" (UID: \"00bb9c2a-9bed-4d69-9ae4-e4f1fad50578\") " pod="kube-system/global-pull-secret-syncer-dkwx5"
Jul 05 00:09:14.737958 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:09:14.737943    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c085d4be-fcda-4931-a018-02cbaca17c59-metrics-certs\") pod \"network-metrics-daemon-5fn94\" (UID: \"c085d4be-fcda-4931-a018-02cbaca17c59\") " pod="openshift-multus/network-metrics-daemon-5fn94"
Jul 05 00:09:14.738203 ip-10-0-142-202 kubenswrapper[2563]: E0705 00:09:14.738068    2563 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 05 00:09:14.738203 ip-10-0-142-202 kubenswrapper[2563]: E0705 00:09:14.738130    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c085d4be-fcda-4931-a018-02cbaca17c59-metrics-certs podName:c085d4be-fcda-4931-a018-02cbaca17c59 nodeName:}" failed. No retries permitted until 2026-07-05 00:09:16.738108617 +0000 UTC m=+5.080437657 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/c085d4be-fcda-4931-a018-02cbaca17c59-metrics-certs") pod "network-metrics-daemon-5fn94" (UID: "c085d4be-fcda-4931-a018-02cbaca17c59") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 05 00:09:14.738596 ip-10-0-142-202 kubenswrapper[2563]: E0705 00:09:14.738572    2563 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 05 00:09:14.738705 ip-10-0-142-202 kubenswrapper[2563]: E0705 00:09:14.738636    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/00bb9c2a-9bed-4d69-9ae4-e4f1fad50578-original-pull-secret podName:00bb9c2a-9bed-4d69-9ae4-e4f1fad50578 nodeName:}" failed. No retries permitted until 2026-07-05 00:09:16.738618267 +0000 UTC m=+5.080947312 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/00bb9c2a-9bed-4d69-9ae4-e4f1fad50578-original-pull-secret") pod "global-pull-secret-syncer-dkwx5" (UID: "00bb9c2a-9bed-4d69-9ae4-e4f1fad50578") : object "kube-system"/"original-pull-secret" not registered
Jul 05 00:09:14.839316 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:09:14.838679    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-z4cn8\" (UniqueName: \"kubernetes.io/projected/e84aff2e-272a-42d8-8bd9-e37a98cfc7a7-kube-api-access-z4cn8\") pod \"network-check-target-4pm8c\" (UID: \"e84aff2e-272a-42d8-8bd9-e37a98cfc7a7\") " pod="openshift-network-diagnostics/network-check-target-4pm8c"
Jul 05 00:09:14.839316 ip-10-0-142-202 kubenswrapper[2563]: E0705 00:09:14.838855    2563 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 05 00:09:14.839316 ip-10-0-142-202 kubenswrapper[2563]: E0705 00:09:14.838878    2563 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 05 00:09:14.839316 ip-10-0-142-202 kubenswrapper[2563]: E0705 00:09:14.838892    2563 projected.go:196] Error preparing data for projected volume kube-api-access-z4cn8 for pod openshift-network-diagnostics/network-check-target-4pm8c: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 05 00:09:14.839316 ip-10-0-142-202 kubenswrapper[2563]: E0705 00:09:14.838950    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/e84aff2e-272a-42d8-8bd9-e37a98cfc7a7-kube-api-access-z4cn8 podName:e84aff2e-272a-42d8-8bd9-e37a98cfc7a7 nodeName:}" failed. No retries permitted until 2026-07-05 00:09:16.83893063 +0000 UTC m=+5.181259695 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-z4cn8" (UniqueName: "kubernetes.io/projected/e84aff2e-272a-42d8-8bd9-e37a98cfc7a7-kube-api-access-z4cn8") pod "network-check-target-4pm8c" (UID: "e84aff2e-272a-42d8-8bd9-e37a98cfc7a7") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 05 00:09:15.058872 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:09:15.058788    2563 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-04 00:04:13 +0000 UTC" deadline="2028-04-21 13:12:31.198672194 +0000 UTC"
Jul 05 00:09:15.058872 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:09:15.058827    2563 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="15757h3m16.139849822s"
Jul 05 00:09:15.120402 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:09:15.120368    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-5fn94"
Jul 05 00:09:15.120565 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:09:15.120369    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-4pm8c"
Jul 05 00:09:15.120653 ip-10-0-142-202 kubenswrapper[2563]: E0705 00:09:15.120564    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-5fn94" podUID="c085d4be-fcda-4931-a018-02cbaca17c59"
Jul 05 00:09:15.121223 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:09:15.120938    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-dkwx5"
Jul 05 00:09:15.121223 ip-10-0-142-202 kubenswrapper[2563]: E0705 00:09:15.121054    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-dkwx5" podUID="00bb9c2a-9bed-4d69-9ae4-e4f1fad50578"
Jul 05 00:09:15.121223 ip-10-0-142-202 kubenswrapper[2563]: E0705 00:09:15.121141    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-4pm8c" podUID="e84aff2e-272a-42d8-8bd9-e37a98cfc7a7"
Jul 05 00:09:15.894594 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:09:15.894559    2563 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 05 00:09:16.761511 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:09:16.760941    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c085d4be-fcda-4931-a018-02cbaca17c59-metrics-certs\") pod \"network-metrics-daemon-5fn94\" (UID: \"c085d4be-fcda-4931-a018-02cbaca17c59\") " pod="openshift-multus/network-metrics-daemon-5fn94"
Jul 05 00:09:16.761511 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:09:16.761024    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/00bb9c2a-9bed-4d69-9ae4-e4f1fad50578-original-pull-secret\") pod \"global-pull-secret-syncer-dkwx5\" (UID: \"00bb9c2a-9bed-4d69-9ae4-e4f1fad50578\") " pod="kube-system/global-pull-secret-syncer-dkwx5"
Jul 05 00:09:16.761511 ip-10-0-142-202 kubenswrapper[2563]: E0705 00:09:16.761148    2563 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 05 00:09:16.761511 ip-10-0-142-202 kubenswrapper[2563]: E0705 00:09:16.761208    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/00bb9c2a-9bed-4d69-9ae4-e4f1fad50578-original-pull-secret podName:00bb9c2a-9bed-4d69-9ae4-e4f1fad50578 nodeName:}" failed. No retries permitted until 2026-07-05 00:09:20.761188946 +0000 UTC m=+9.103517988 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/00bb9c2a-9bed-4d69-9ae4-e4f1fad50578-original-pull-secret") pod "global-pull-secret-syncer-dkwx5" (UID: "00bb9c2a-9bed-4d69-9ae4-e4f1fad50578") : object "kube-system"/"original-pull-secret" not registered
Jul 05 00:09:16.761511 ip-10-0-142-202 kubenswrapper[2563]: E0705 00:09:16.761323    2563 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 05 00:09:16.761511 ip-10-0-142-202 kubenswrapper[2563]: E0705 00:09:16.761391    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c085d4be-fcda-4931-a018-02cbaca17c59-metrics-certs podName:c085d4be-fcda-4931-a018-02cbaca17c59 nodeName:}" failed. No retries permitted until 2026-07-05 00:09:20.761371731 +0000 UTC m=+9.103700774 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/c085d4be-fcda-4931-a018-02cbaca17c59-metrics-certs") pod "network-metrics-daemon-5fn94" (UID: "c085d4be-fcda-4931-a018-02cbaca17c59") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 05 00:09:16.862588 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:09:16.861883    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-z4cn8\" (UniqueName: \"kubernetes.io/projected/e84aff2e-272a-42d8-8bd9-e37a98cfc7a7-kube-api-access-z4cn8\") pod \"network-check-target-4pm8c\" (UID: \"e84aff2e-272a-42d8-8bd9-e37a98cfc7a7\") " pod="openshift-network-diagnostics/network-check-target-4pm8c"
Jul 05 00:09:16.862588 ip-10-0-142-202 kubenswrapper[2563]: E0705 00:09:16.862114    2563 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 05 00:09:16.862588 ip-10-0-142-202 kubenswrapper[2563]: E0705 00:09:16.862137    2563 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 05 00:09:16.862588 ip-10-0-142-202 kubenswrapper[2563]: E0705 00:09:16.862150    2563 projected.go:196] Error preparing data for projected volume kube-api-access-z4cn8 for pod openshift-network-diagnostics/network-check-target-4pm8c: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 05 00:09:16.862588 ip-10-0-142-202 kubenswrapper[2563]: E0705 00:09:16.862209    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/e84aff2e-272a-42d8-8bd9-e37a98cfc7a7-kube-api-access-z4cn8 podName:e84aff2e-272a-42d8-8bd9-e37a98cfc7a7 nodeName:}" failed. No retries permitted until 2026-07-05 00:09:20.862191192 +0000 UTC m=+9.204520238 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-z4cn8" (UniqueName: "kubernetes.io/projected/e84aff2e-272a-42d8-8bd9-e37a98cfc7a7-kube-api-access-z4cn8") pod "network-check-target-4pm8c" (UID: "e84aff2e-272a-42d8-8bd9-e37a98cfc7a7") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 05 00:09:17.121184 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:09:17.121052    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-4pm8c"
Jul 05 00:09:17.121384 ip-10-0-142-202 kubenswrapper[2563]: E0705 00:09:17.121186    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-4pm8c" podUID="e84aff2e-272a-42d8-8bd9-e37a98cfc7a7"
Jul 05 00:09:17.121603 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:09:17.121585    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-dkwx5"
Jul 05 00:09:17.121719 ip-10-0-142-202 kubenswrapper[2563]: E0705 00:09:17.121697    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-dkwx5" podUID="00bb9c2a-9bed-4d69-9ae4-e4f1fad50578"
Jul 05 00:09:17.121778 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:09:17.121756    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-5fn94"
Jul 05 00:09:17.121861 ip-10-0-142-202 kubenswrapper[2563]: E0705 00:09:17.121839    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-5fn94" podUID="c085d4be-fcda-4931-a018-02cbaca17c59"
Jul 05 00:09:19.121124 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:09:19.121083    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-4pm8c"
Jul 05 00:09:19.121644 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:09:19.121212    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-dkwx5"
Jul 05 00:09:19.121644 ip-10-0-142-202 kubenswrapper[2563]: E0705 00:09:19.121228    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-4pm8c" podUID="e84aff2e-272a-42d8-8bd9-e37a98cfc7a7"
Jul 05 00:09:19.121644 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:09:19.121250    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-5fn94"
Jul 05 00:09:19.121644 ip-10-0-142-202 kubenswrapper[2563]: E0705 00:09:19.121353    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-dkwx5" podUID="00bb9c2a-9bed-4d69-9ae4-e4f1fad50578"
Jul 05 00:09:19.121644 ip-10-0-142-202 kubenswrapper[2563]: E0705 00:09:19.121521    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-5fn94" podUID="c085d4be-fcda-4931-a018-02cbaca17c59"
Jul 05 00:09:20.796500 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:09:20.796459    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/00bb9c2a-9bed-4d69-9ae4-e4f1fad50578-original-pull-secret\") pod \"global-pull-secret-syncer-dkwx5\" (UID: \"00bb9c2a-9bed-4d69-9ae4-e4f1fad50578\") " pod="kube-system/global-pull-secret-syncer-dkwx5"
Jul 05 00:09:20.797005 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:09:20.796623    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c085d4be-fcda-4931-a018-02cbaca17c59-metrics-certs\") pod \"network-metrics-daemon-5fn94\" (UID: \"c085d4be-fcda-4931-a018-02cbaca17c59\") " pod="openshift-multus/network-metrics-daemon-5fn94"
Jul 05 00:09:20.797005 ip-10-0-142-202 kubenswrapper[2563]: E0705 00:09:20.796641    2563 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 05 00:09:20.797005 ip-10-0-142-202 kubenswrapper[2563]: E0705 00:09:20.796726    2563 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 05 00:09:20.797005 ip-10-0-142-202 kubenswrapper[2563]: E0705 00:09:20.796734    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/00bb9c2a-9bed-4d69-9ae4-e4f1fad50578-original-pull-secret podName:00bb9c2a-9bed-4d69-9ae4-e4f1fad50578 nodeName:}" failed. No retries permitted until 2026-07-05 00:09:28.796710186 +0000 UTC m=+17.139039228 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/00bb9c2a-9bed-4d69-9ae4-e4f1fad50578-original-pull-secret") pod "global-pull-secret-syncer-dkwx5" (UID: "00bb9c2a-9bed-4d69-9ae4-e4f1fad50578") : object "kube-system"/"original-pull-secret" not registered
Jul 05 00:09:20.797005 ip-10-0-142-202 kubenswrapper[2563]: E0705 00:09:20.796828    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c085d4be-fcda-4931-a018-02cbaca17c59-metrics-certs podName:c085d4be-fcda-4931-a018-02cbaca17c59 nodeName:}" failed. No retries permitted until 2026-07-05 00:09:28.796811048 +0000 UTC m=+17.139140094 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/c085d4be-fcda-4931-a018-02cbaca17c59-metrics-certs") pod "network-metrics-daemon-5fn94" (UID: "c085d4be-fcda-4931-a018-02cbaca17c59") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 05 00:09:20.897703 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:09:20.897627    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-z4cn8\" (UniqueName: \"kubernetes.io/projected/e84aff2e-272a-42d8-8bd9-e37a98cfc7a7-kube-api-access-z4cn8\") pod \"network-check-target-4pm8c\" (UID: \"e84aff2e-272a-42d8-8bd9-e37a98cfc7a7\") " pod="openshift-network-diagnostics/network-check-target-4pm8c"
Jul 05 00:09:20.897893 ip-10-0-142-202 kubenswrapper[2563]: E0705 00:09:20.897835    2563 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 05 00:09:20.897893 ip-10-0-142-202 kubenswrapper[2563]: E0705 00:09:20.897856    2563 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 05 00:09:20.897893 ip-10-0-142-202 kubenswrapper[2563]: E0705 00:09:20.897865    2563 projected.go:196] Error preparing data for projected volume kube-api-access-z4cn8 for pod openshift-network-diagnostics/network-check-target-4pm8c: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 05 00:09:20.898069 ip-10-0-142-202 kubenswrapper[2563]: E0705 00:09:20.897928    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/e84aff2e-272a-42d8-8bd9-e37a98cfc7a7-kube-api-access-z4cn8 podName:e84aff2e-272a-42d8-8bd9-e37a98cfc7a7 nodeName:}" failed. No retries permitted until 2026-07-05 00:09:28.897908616 +0000 UTC m=+17.240237656 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-z4cn8" (UniqueName: "kubernetes.io/projected/e84aff2e-272a-42d8-8bd9-e37a98cfc7a7-kube-api-access-z4cn8") pod "network-check-target-4pm8c" (UID: "e84aff2e-272a-42d8-8bd9-e37a98cfc7a7") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 05 00:09:21.121336 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:09:21.121253    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-5fn94"
Jul 05 00:09:21.121505 ip-10-0-142-202 kubenswrapper[2563]: E0705 00:09:21.121403    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-5fn94" podUID="c085d4be-fcda-4931-a018-02cbaca17c59"
Jul 05 00:09:21.121894 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:09:21.121843    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-4pm8c"
Jul 05 00:09:21.122001 ip-10-0-142-202 kubenswrapper[2563]: E0705 00:09:21.121941    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-4pm8c" podUID="e84aff2e-272a-42d8-8bd9-e37a98cfc7a7"
Jul 05 00:09:21.122001 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:09:21.121991    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-dkwx5"
Jul 05 00:09:21.122121 ip-10-0-142-202 kubenswrapper[2563]: E0705 00:09:21.122063    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-dkwx5" podUID="00bb9c2a-9bed-4d69-9ae4-e4f1fad50578"
Jul 05 00:09:23.120699 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:09:23.120650    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-4pm8c"
Jul 05 00:09:23.121208 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:09:23.120808    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-dkwx5"
Jul 05 00:09:23.121208 ip-10-0-142-202 kubenswrapper[2563]: E0705 00:09:23.120809    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-4pm8c" podUID="e84aff2e-272a-42d8-8bd9-e37a98cfc7a7"
Jul 05 00:09:23.121208 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:09:23.120854    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-5fn94"
Jul 05 00:09:23.121208 ip-10-0-142-202 kubenswrapper[2563]: E0705 00:09:23.120948    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-dkwx5" podUID="00bb9c2a-9bed-4d69-9ae4-e4f1fad50578"
Jul 05 00:09:23.121208 ip-10-0-142-202 kubenswrapper[2563]: E0705 00:09:23.121052    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-5fn94" podUID="c085d4be-fcda-4931-a018-02cbaca17c59"
Jul 05 00:09:25.120526 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:09:25.120458    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-4pm8c"
Jul 05 00:09:25.120526 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:09:25.120494    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-dkwx5"
Jul 05 00:09:25.120526 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:09:25.120496    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-5fn94"
Jul 05 00:09:25.121153 ip-10-0-142-202 kubenswrapper[2563]: E0705 00:09:25.120589    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-4pm8c" podUID="e84aff2e-272a-42d8-8bd9-e37a98cfc7a7"
Jul 05 00:09:25.121153 ip-10-0-142-202 kubenswrapper[2563]: E0705 00:09:25.120698    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-dkwx5" podUID="00bb9c2a-9bed-4d69-9ae4-e4f1fad50578"
Jul 05 00:09:25.121153 ip-10-0-142-202 kubenswrapper[2563]: E0705 00:09:25.120799    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-5fn94" podUID="c085d4be-fcda-4931-a018-02cbaca17c59"
Jul 05 00:09:27.121045 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:09:27.121005    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-5fn94"
Jul 05 00:09:27.121532 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:09:27.121012    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-4pm8c"
Jul 05 00:09:27.121532 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:09:27.121137    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-dkwx5"
Jul 05 00:09:27.121532 ip-10-0-142-202 kubenswrapper[2563]: E0705 00:09:27.121138    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-5fn94" podUID="c085d4be-fcda-4931-a018-02cbaca17c59"
Jul 05 00:09:27.121532 ip-10-0-142-202 kubenswrapper[2563]: E0705 00:09:27.121249    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-4pm8c" podUID="e84aff2e-272a-42d8-8bd9-e37a98cfc7a7"
Jul 05 00:09:27.121532 ip-10-0-142-202 kubenswrapper[2563]: E0705 00:09:27.121337    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-dkwx5" podUID="00bb9c2a-9bed-4d69-9ae4-e4f1fad50578"
Jul 05 00:09:28.864761 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:09:28.864724    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c085d4be-fcda-4931-a018-02cbaca17c59-metrics-certs\") pod \"network-metrics-daemon-5fn94\" (UID: \"c085d4be-fcda-4931-a018-02cbaca17c59\") " pod="openshift-multus/network-metrics-daemon-5fn94"
Jul 05 00:09:28.865270 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:09:28.864787    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/00bb9c2a-9bed-4d69-9ae4-e4f1fad50578-original-pull-secret\") pod \"global-pull-secret-syncer-dkwx5\" (UID: \"00bb9c2a-9bed-4d69-9ae4-e4f1fad50578\") " pod="kube-system/global-pull-secret-syncer-dkwx5"
Jul 05 00:09:28.865270 ip-10-0-142-202 kubenswrapper[2563]: E0705 00:09:28.864886    2563 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 05 00:09:28.865270 ip-10-0-142-202 kubenswrapper[2563]: E0705 00:09:28.864889    2563 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 05 00:09:28.865270 ip-10-0-142-202 kubenswrapper[2563]: E0705 00:09:28.864956    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/00bb9c2a-9bed-4d69-9ae4-e4f1fad50578-original-pull-secret podName:00bb9c2a-9bed-4d69-9ae4-e4f1fad50578 nodeName:}" failed. No retries permitted until 2026-07-05 00:09:44.864937611 +0000 UTC m=+33.207266666 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/00bb9c2a-9bed-4d69-9ae4-e4f1fad50578-original-pull-secret") pod "global-pull-secret-syncer-dkwx5" (UID: "00bb9c2a-9bed-4d69-9ae4-e4f1fad50578") : object "kube-system"/"original-pull-secret" not registered
Jul 05 00:09:28.865270 ip-10-0-142-202 kubenswrapper[2563]: E0705 00:09:28.864972    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c085d4be-fcda-4931-a018-02cbaca17c59-metrics-certs podName:c085d4be-fcda-4931-a018-02cbaca17c59 nodeName:}" failed. No retries permitted until 2026-07-05 00:09:44.864964905 +0000 UTC m=+33.207293943 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/c085d4be-fcda-4931-a018-02cbaca17c59-metrics-certs") pod "network-metrics-daemon-5fn94" (UID: "c085d4be-fcda-4931-a018-02cbaca17c59") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 05 00:09:28.965947 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:09:28.965915    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-z4cn8\" (UniqueName: \"kubernetes.io/projected/e84aff2e-272a-42d8-8bd9-e37a98cfc7a7-kube-api-access-z4cn8\") pod \"network-check-target-4pm8c\" (UID: \"e84aff2e-272a-42d8-8bd9-e37a98cfc7a7\") " pod="openshift-network-diagnostics/network-check-target-4pm8c"
Jul 05 00:09:28.966118 ip-10-0-142-202 kubenswrapper[2563]: E0705 00:09:28.966073    2563 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 05 00:09:28.966118 ip-10-0-142-202 kubenswrapper[2563]: E0705 00:09:28.966091    2563 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 05 00:09:28.966118 ip-10-0-142-202 kubenswrapper[2563]: E0705 00:09:28.966103    2563 projected.go:196] Error preparing data for projected volume kube-api-access-z4cn8 for pod openshift-network-diagnostics/network-check-target-4pm8c: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 05 00:09:28.966309 ip-10-0-142-202 kubenswrapper[2563]: E0705 00:09:28.966163    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/e84aff2e-272a-42d8-8bd9-e37a98cfc7a7-kube-api-access-z4cn8 podName:e84aff2e-272a-42d8-8bd9-e37a98cfc7a7 nodeName:}" failed. No retries permitted until 2026-07-05 00:09:44.966145585 +0000 UTC m=+33.308474627 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-z4cn8" (UniqueName: "kubernetes.io/projected/e84aff2e-272a-42d8-8bd9-e37a98cfc7a7-kube-api-access-z4cn8") pod "network-check-target-4pm8c" (UID: "e84aff2e-272a-42d8-8bd9-e37a98cfc7a7") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 05 00:09:29.120936 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:09:29.120863    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-dkwx5"
Jul 05 00:09:29.121083 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:09:29.120867    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-4pm8c"
Jul 05 00:09:29.121083 ip-10-0-142-202 kubenswrapper[2563]: E0705 00:09:29.120985    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-dkwx5" podUID="00bb9c2a-9bed-4d69-9ae4-e4f1fad50578"
Jul 05 00:09:29.121083 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:09:29.120864    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-5fn94"
Jul 05 00:09:29.121083 ip-10-0-142-202 kubenswrapper[2563]: E0705 00:09:29.121050    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-4pm8c" podUID="e84aff2e-272a-42d8-8bd9-e37a98cfc7a7"
Jul 05 00:09:29.121283 ip-10-0-142-202 kubenswrapper[2563]: E0705 00:09:29.121133    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-5fn94" podUID="c085d4be-fcda-4931-a018-02cbaca17c59"
Jul 05 00:09:31.120637 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:09:31.120605    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-4pm8c"
Jul 05 00:09:31.121192 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:09:31.120727    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-dkwx5"
Jul 05 00:09:31.121192 ip-10-0-142-202 kubenswrapper[2563]: E0705 00:09:31.120732    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-4pm8c" podUID="e84aff2e-272a-42d8-8bd9-e37a98cfc7a7"
Jul 05 00:09:31.121192 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:09:31.120747    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-5fn94"
Jul 05 00:09:31.121192 ip-10-0-142-202 kubenswrapper[2563]: E0705 00:09:31.120816    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-dkwx5" podUID="00bb9c2a-9bed-4d69-9ae4-e4f1fad50578"
Jul 05 00:09:31.121192 ip-10-0-142-202 kubenswrapper[2563]: E0705 00:09:31.120880    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-5fn94" podUID="c085d4be-fcda-4931-a018-02cbaca17c59"
Jul 05 00:09:33.121328 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:09:33.121291    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-4pm8c"
Jul 05 00:09:33.121774 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:09:33.121341    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-5fn94"
Jul 05 00:09:33.121774 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:09:33.121341    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-dkwx5"
Jul 05 00:09:33.121774 ip-10-0-142-202 kubenswrapper[2563]: E0705 00:09:33.121405    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-4pm8c" podUID="e84aff2e-272a-42d8-8bd9-e37a98cfc7a7"
Jul 05 00:09:33.121774 ip-10-0-142-202 kubenswrapper[2563]: E0705 00:09:33.121522    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-dkwx5" podUID="00bb9c2a-9bed-4d69-9ae4-e4f1fad50578"
Jul 05 00:09:33.121774 ip-10-0-142-202 kubenswrapper[2563]: E0705 00:09:33.121626    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-5fn94" podUID="c085d4be-fcda-4931-a018-02cbaca17c59"
Jul 05 00:09:34.361898 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:09:34.361855    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-kwgzc" event={"ID":"aa4848f6-1480-4acd-af73-e8e055306078","Type":"ContainerStarted","Data":"7bebf86758e17a051142366da5ee2bd323ab236b7d72623b84a06d4c512f9c61"}
Jul 05 00:09:34.363730 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:09:34.363712    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-7sxq4_b96bfb4d-2fd3-4be3-9064-e24efeb904f4/ovn-acl-logging/0.log"
Jul 05 00:09:34.364053 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:09:34.364035    2563 generic.go:356] "Generic (PLEG): container finished" podID="b96bfb4d-2fd3-4be3-9064-e24efeb904f4" containerID="c1eaac11e919a315f2de43e689c6ea3f5d08869968e30ce408991c96515a624d" exitCode=1
Jul 05 00:09:34.364122 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:09:34.364095    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-7sxq4" event={"ID":"b96bfb4d-2fd3-4be3-9064-e24efeb904f4","Type":"ContainerStarted","Data":"c9b261a7d4be9482e521eb6d5123971dcce92bb4d7fb547c751a755f8c43c7f7"}
Jul 05 00:09:34.364190 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:09:34.364175    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-7sxq4" event={"ID":"b96bfb4d-2fd3-4be3-9064-e24efeb904f4","Type":"ContainerDied","Data":"c1eaac11e919a315f2de43e689c6ea3f5d08869968e30ce408991c96515a624d"}
Jul 05 00:09:34.364265 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:09:34.364252    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-7sxq4" event={"ID":"b96bfb4d-2fd3-4be3-9064-e24efeb904f4","Type":"ContainerStarted","Data":"86b238e3d280f9a0dce00321a3533262a73a0e436dd34686206219836a3fed2a"}
Jul 05 00:09:34.371472 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:09:34.371441    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-r2vw8" event={"ID":"5b9038ef-5e8c-4ee8-9bb7-e9c249715b79","Type":"ContainerStarted","Data":"79fff20b66b7ae5a1b490bb7f038ceaf84b470fb5df2026640fd7ad6abfc19b1"}
Jul 05 00:09:34.373043 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:09:34.373020    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-142-202.ec2.internal" event={"ID":"ec580e48b6696694176fa0643f6420e5","Type":"ContainerStarted","Data":"2510a222caf28f3e79defe3240f121bc80819abf2aca1b96a9c01fc67324547f"}
Jul 05 00:09:34.398031 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:09:34.397978    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-kwgzc" podStartSLOduration=0.933488733 podStartE2EDuration="21.397963015s" totalImagesPullingTime="20.464474282s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-05 00:09:13 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-05 00:09:34.384983408 +0000 UTC m=+22.727312467" watchObservedRunningTime="2026-07-05 00:09:34.397963015 +0000 UTC m=+22.740292075"
Jul 05 00:09:34.414206 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:09:34.414165    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-142-202.ec2.internal" podStartSLOduration=21.414135063 podStartE2EDuration="21.414135063s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-05 00:09:13 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-05 00:09:34.39779139 +0000 UTC m=+22.740120451" watchObservedRunningTime="2026-07-05 00:09:34.414135063 +0000 UTC m=+22.756464123"
Jul 05 00:09:35.120829 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:09:35.120671    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-5fn94"
Jul 05 00:09:35.121080 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:09:35.120673    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-dkwx5"
Jul 05 00:09:35.121080 ip-10-0-142-202 kubenswrapper[2563]: E0705 00:09:35.120926    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-5fn94" podUID="c085d4be-fcda-4931-a018-02cbaca17c59"
Jul 05 00:09:35.121080 ip-10-0-142-202 kubenswrapper[2563]: E0705 00:09:35.121010    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-dkwx5" podUID="00bb9c2a-9bed-4d69-9ae4-e4f1fad50578"
Jul 05 00:09:35.121080 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:09:35.120674    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-4pm8c"
Jul 05 00:09:35.121247 ip-10-0-142-202 kubenswrapper[2563]: E0705 00:09:35.121091    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-4pm8c" podUID="e84aff2e-272a-42d8-8bd9-e37a98cfc7a7"
Jul 05 00:09:35.378007 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:09:35.377973    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-bj4b9" event={"ID":"91e13b36-f504-4b21-b8ae-3b66858bddf2","Type":"ContainerStarted","Data":"9710180d306e1f11114ac4ed2418e95e47f38ad7c6cc9a430460681e719fa17b"}
Jul 05 00:09:35.379493 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:09:35.379467    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-202.ec2.internal" event={"ID":"34a0b5b76f9b2c1d92188a84f4deb896","Type":"ContainerDied","Data":"8d2e8bd4b452f607009507c5e563df985b22d3cb3b56c600dc2d1e860ba57501"}
Jul 05 00:09:35.379664 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:09:35.379404    2563 generic.go:356] "Generic (PLEG): container finished" podID="34a0b5b76f9b2c1d92188a84f4deb896" containerID="8d2e8bd4b452f607009507c5e563df985b22d3cb3b56c600dc2d1e860ba57501" exitCode=0
Jul 05 00:09:35.384025 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:09:35.384001    2563 generic.go:356] "Generic (PLEG): container finished" podID="aeaa1e13-243b-4c7f-8045-85ba45081c42" containerID="520b4a5ae1bceaa1c31249bf9f10e4384189dfdc654b168c9edf951d28a4934d" exitCode=0
Jul 05 00:09:35.384124 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:09:35.384065    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-g568x" event={"ID":"aeaa1e13-243b-4c7f-8045-85ba45081c42","Type":"ContainerDied","Data":"520b4a5ae1bceaa1c31249bf9f10e4384189dfdc654b168c9edf951d28a4934d"}
Jul 05 00:09:35.385533 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:09:35.385499    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-rwbw4" event={"ID":"1f3c24e8-591d-4e53-9b63-091f6f1dec51","Type":"ContainerStarted","Data":"806dccf94c5bfa514b6fde42c1fcdc46deaaebfbaa98c1b02fbf9a7095f40d5d"}
Jul 05 00:09:35.388446 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:09:35.388428    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-7sxq4_b96bfb4d-2fd3-4be3-9064-e24efeb904f4/ovn-acl-logging/0.log"
Jul 05 00:09:35.388845 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:09:35.388825    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-7sxq4" event={"ID":"b96bfb4d-2fd3-4be3-9064-e24efeb904f4","Type":"ContainerStarted","Data":"0d4336c63924606c7ef8b6e05804c0606d58f24cf406615d331e578dc223c651"}
Jul 05 00:09:35.388928 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:09:35.388882    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-7sxq4" event={"ID":"b96bfb4d-2fd3-4be3-9064-e24efeb904f4","Type":"ContainerStarted","Data":"d12adb12153e5ae5d6c917fb16aad72991745fcc4518d8601a3647e23bd4db89"}
Jul 05 00:09:35.388982 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:09:35.388929    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-7sxq4" event={"ID":"b96bfb4d-2fd3-4be3-9064-e24efeb904f4","Type":"ContainerStarted","Data":"f9fc8b8036ad1e597d5946b29defb5e5e306a0bb118a70f2f0a1d49846f256ba"}
Jul 05 00:09:35.390145 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:09:35.390126    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-s547v" event={"ID":"d3a9b78d-cf60-4acc-a3c5-016ee34903f8","Type":"ContainerStarted","Data":"6abe9d54a8e7c7ad4641f413dcaaec32913bc6317e0ccc98d3f1d2fc281ab53a"}
Jul 05 00:09:35.391419 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:09:35.391400    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-kjz9w" event={"ID":"cecee17b-8d41-4a16-a3b2-53919db096bb","Type":"ContainerStarted","Data":"3b92e2bd700ddd2c256983cf135ae798c29f439a16c3a9cb69ac891b47af183e"}
Jul 05 00:09:35.392610 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:09:35.392587    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-8tjwr" event={"ID":"8a2c0b38-b2ce-4570-8963-2f75863cc6b2","Type":"ContainerStarted","Data":"7a84b911ebe542dc155b6288af2ea2d01a61453514b4c10d4b274750609641a9"}
Jul 05 00:09:35.402263 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:09:35.402206    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-r2vw8" podStartSLOduration=1.9151764070000001 podStartE2EDuration="22.402194587s" totalImagesPullingTime="20.48701818s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-05 00:09:13 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-05 00:09:34.413916446 +0000 UTC m=+22.756245527" watchObservedRunningTime="2026-07-05 00:09:35.402194587 +0000 UTC m=+23.744523647"
Jul 05 00:09:35.413359 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:09:35.413322    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/konnectivity-agent-rwbw4" podStartSLOduration=1.969800534 podStartE2EDuration="22.413301353s" totalImagesPullingTime="20.443500819s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-05 00:09:13 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-05 00:09:35.412888631 +0000 UTC m=+23.755217690" watchObservedRunningTime="2026-07-05 00:09:35.413301353 +0000 UTC m=+23.755630412"
Jul 05 00:09:35.423941 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:09:35.423904    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-8tjwr" podStartSLOduration=1.941126193 podStartE2EDuration="22.423893173s" totalImagesPullingTime="20.48276698s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-05 00:09:13 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-05 00:09:35.423511389 +0000 UTC m=+23.765840448" watchObservedRunningTime="2026-07-05 00:09:35.423893173 +0000 UTC m=+23.766222248"
Jul 05 00:09:35.434956 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:09:35.434921    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/node-resolver-kjz9w" podStartSLOduration=1.997725049 podStartE2EDuration="22.434909938s" totalImagesPullingTime="20.437184889s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-05 00:09:13 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-05 00:09:35.434668784 +0000 UTC m=+23.776997841" watchObservedRunningTime="2026-07-05 00:09:35.434909938 +0000 UTC m=+23.777238998"
Jul 05 00:09:35.450417 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:09:35.450382    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/node-ca-s547v" podStartSLOduration=2.027806257 podStartE2EDuration="22.450370405s" totalImagesPullingTime="20.422564148s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-05 00:09:13 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-05 00:09:35.45007886 +0000 UTC m=+23.792407939" watchObservedRunningTime="2026-07-05 00:09:35.450370405 +0000 UTC m=+23.792699465"
Jul 05 00:09:36.378962 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:09:36.378811    2563 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock"
Jul 05 00:09:36.395720 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:09:36.395687    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-bj4b9" event={"ID":"91e13b36-f504-4b21-b8ae-3b66858bddf2","Type":"ContainerStarted","Data":"355c4961ba13203039400d30c7b288808bdbd5ac8f0d8b7cb7748c087dbefe3d"}
Jul 05 00:09:37.088919 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:09:37.088776    2563 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-07-05T00:09:36.378958494Z","UUID":"21bae6ec-ab9f-4076-882e-12a4d5192bf9","Handler":null,"Name":"","Endpoint":""}
Jul 05 00:09:37.090593 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:09:37.090566    2563 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0
Jul 05 00:09:37.090730 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:09:37.090607    2563 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock
Jul 05 00:09:37.120875 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:09:37.120849    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-5fn94"
Jul 05 00:09:37.120875 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:09:37.120873    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-dkwx5"
Jul 05 00:09:37.121062 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:09:37.120979    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-4pm8c"
Jul 05 00:09:37.121062 ip-10-0-142-202 kubenswrapper[2563]: E0705 00:09:37.120988    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-5fn94" podUID="c085d4be-fcda-4931-a018-02cbaca17c59"
Jul 05 00:09:37.121142 ip-10-0-142-202 kubenswrapper[2563]: E0705 00:09:37.121076    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-4pm8c" podUID="e84aff2e-272a-42d8-8bd9-e37a98cfc7a7"
Jul 05 00:09:37.121192 ip-10-0-142-202 kubenswrapper[2563]: E0705 00:09:37.121153    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-dkwx5" podUID="00bb9c2a-9bed-4d69-9ae4-e4f1fad50578"
Jul 05 00:09:37.401584 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:09:37.401517    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-7sxq4_b96bfb4d-2fd3-4be3-9064-e24efeb904f4/ovn-acl-logging/0.log"
Jul 05 00:09:37.402075 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:09:37.401966    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-7sxq4" event={"ID":"b96bfb4d-2fd3-4be3-9064-e24efeb904f4","Type":"ContainerStarted","Data":"d5928076bbeedd767a8cbd081ef60cdf958b9c522001f7bdcbbbba43419ef370"}
Jul 05 00:09:37.403796 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:09:37.403770    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-202.ec2.internal" event={"ID":"34a0b5b76f9b2c1d92188a84f4deb896","Type":"ContainerStarted","Data":"b5856e0fa26c8fcdc4c54db3367618a90a7c1525d8bf59a264309d7763b2621f"}
Jul 05 00:09:37.417081 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:09:37.417021    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-202.ec2.internal" podStartSLOduration=24.41700345 podStartE2EDuration="24.41700345s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-05 00:09:13 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-05 00:09:37.416063259 +0000 UTC m=+25.758392342" watchObservedRunningTime="2026-07-05 00:09:37.41700345 +0000 UTC m=+25.759332511"
Jul 05 00:09:38.202940 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:09:38.202902    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-rwbw4"
Jul 05 00:09:38.203556 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:09:38.203536    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-rwbw4"
Jul 05 00:09:38.408235 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:09:38.408193    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-bj4b9" event={"ID":"91e13b36-f504-4b21-b8ae-3b66858bddf2","Type":"ContainerStarted","Data":"ebf20250e6ea8ac6f0141092735ce1c7cbbd7ca281b201884e5a3dae92b40e78"}
Jul 05 00:09:38.408645 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:09:38.408621    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-rwbw4"
Jul 05 00:09:38.409105 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:09:38.409086    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-rwbw4"
Jul 05 00:09:38.422349 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:09:38.422295    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-bj4b9" podStartSLOduration=2.5797183759999998 podStartE2EDuration="25.422277794s" totalImagesPullingTime="22.842559418s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-05 00:09:13 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-05 00:09:38.421964463 +0000 UTC m=+26.764293548" watchObservedRunningTime="2026-07-05 00:09:38.422277794 +0000 UTC m=+26.764606856"
Jul 05 00:09:39.121347 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:09:39.121258    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-4pm8c"
Jul 05 00:09:39.121347 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:09:39.121292    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-dkwx5"
Jul 05 00:09:39.121553 ip-10-0-142-202 kubenswrapper[2563]: E0705 00:09:39.121383    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-4pm8c" podUID="e84aff2e-272a-42d8-8bd9-e37a98cfc7a7"
Jul 05 00:09:39.121553 ip-10-0-142-202 kubenswrapper[2563]: E0705 00:09:39.121453    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-dkwx5" podUID="00bb9c2a-9bed-4d69-9ae4-e4f1fad50578"
Jul 05 00:09:39.121553 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:09:39.121479    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-5fn94"
Jul 05 00:09:39.121670 ip-10-0-142-202 kubenswrapper[2563]: E0705 00:09:39.121596    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-5fn94" podUID="c085d4be-fcda-4931-a018-02cbaca17c59"
Jul 05 00:09:40.416387 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:09:40.416362    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-7sxq4_b96bfb4d-2fd3-4be3-9064-e24efeb904f4/ovn-acl-logging/0.log"
Jul 05 00:09:40.416813 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:09:40.416751    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-7sxq4" event={"ID":"b96bfb4d-2fd3-4be3-9064-e24efeb904f4","Type":"ContainerStarted","Data":"4a81bf18785cc0851d4ea0d7a2ce24aaa0c800007d6dbd7dbf7d216263b43e73"}
Jul 05 00:09:40.417206 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:09:40.417183    2563 scope.go:117] "RemoveContainer" containerID="c1eaac11e919a315f2de43e689c6ea3f5d08869968e30ce408991c96515a624d"
Jul 05 00:09:41.120469 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:09:41.120438    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-4pm8c"
Jul 05 00:09:41.120638 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:09:41.120521    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-dkwx5"
Jul 05 00:09:41.120638 ip-10-0-142-202 kubenswrapper[2563]: E0705 00:09:41.120526    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-4pm8c" podUID="e84aff2e-272a-42d8-8bd9-e37a98cfc7a7"
Jul 05 00:09:41.120638 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:09:41.120545    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-5fn94"
Jul 05 00:09:41.120638 ip-10-0-142-202 kubenswrapper[2563]: E0705 00:09:41.120622    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-dkwx5" podUID="00bb9c2a-9bed-4d69-9ae4-e4f1fad50578"
Jul 05 00:09:41.120794 ip-10-0-142-202 kubenswrapper[2563]: E0705 00:09:41.120690    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-5fn94" podUID="c085d4be-fcda-4931-a018-02cbaca17c59"
Jul 05 00:09:41.420167 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:09:41.420091    2563 generic.go:356] "Generic (PLEG): container finished" podID="aeaa1e13-243b-4c7f-8045-85ba45081c42" containerID="5ab2a32254293c57653ac99519f0cd30a07663c3982358878a377f9bf1481a5b" exitCode=0
Jul 05 00:09:41.420874 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:09:41.420179    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-g568x" event={"ID":"aeaa1e13-243b-4c7f-8045-85ba45081c42","Type":"ContainerDied","Data":"5ab2a32254293c57653ac99519f0cd30a07663c3982358878a377f9bf1481a5b"}
Jul 05 00:09:41.424128 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:09:41.424113    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-7sxq4_b96bfb4d-2fd3-4be3-9064-e24efeb904f4/ovn-acl-logging/0.log"
Jul 05 00:09:41.424473 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:09:41.424457    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-7sxq4" event={"ID":"b96bfb4d-2fd3-4be3-9064-e24efeb904f4","Type":"ContainerStarted","Data":"0e9fabe96f363932db10935259c078de28b8ab8b16f5d3bd0f3495755b3694eb"}
Jul 05 00:09:41.424633 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:09:41.424621    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-7sxq4"
Jul 05 00:09:41.424671 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:09:41.424666    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-7sxq4"
Jul 05 00:09:41.424721 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:09:41.424711    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-7sxq4"
Jul 05 00:09:41.440000 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:09:41.439981    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-7sxq4"
Jul 05 00:09:41.440191 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:09:41.440179    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-7sxq4"
Jul 05 00:09:41.458639 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:09:41.458605    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-7sxq4" podStartSLOduration=7.942584261 podStartE2EDuration="28.458592003s" totalImagesPullingTime="20.516007742s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-05 00:09:13 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-05 00:09:41.45827487 +0000 UTC m=+29.800603951" watchObservedRunningTime="2026-07-05 00:09:41.458592003 +0000 UTC m=+29.800921060"
Jul 05 00:09:42.509131 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:09:42.508958    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-4pm8c"]
Jul 05 00:09:42.509538 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:09:42.509254    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-4pm8c"
Jul 05 00:09:42.509538 ip-10-0-142-202 kubenswrapper[2563]: E0705 00:09:42.509361    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-4pm8c" podUID="e84aff2e-272a-42d8-8bd9-e37a98cfc7a7"
Jul 05 00:09:42.512370 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:09:42.512344    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-5fn94"]
Jul 05 00:09:42.512479 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:09:42.512454    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-5fn94"
Jul 05 00:09:42.512591 ip-10-0-142-202 kubenswrapper[2563]: E0705 00:09:42.512568    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-5fn94" podUID="c085d4be-fcda-4931-a018-02cbaca17c59"
Jul 05 00:09:42.512994 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:09:42.512970    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-dkwx5"]
Jul 05 00:09:42.513081 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:09:42.513060    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-dkwx5"
Jul 05 00:09:42.513187 ip-10-0-142-202 kubenswrapper[2563]: E0705 00:09:42.513167    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-dkwx5" podUID="00bb9c2a-9bed-4d69-9ae4-e4f1fad50578"
Jul 05 00:09:43.431103 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:09:43.431075    2563 generic.go:356] "Generic (PLEG): container finished" podID="aeaa1e13-243b-4c7f-8045-85ba45081c42" containerID="645ed5944046a755ee82d5ee30cb9b5be7cbd591b4d797023d54d475492f538e" exitCode=0
Jul 05 00:09:43.431266 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:09:43.431152    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-g568x" event={"ID":"aeaa1e13-243b-4c7f-8045-85ba45081c42","Type":"ContainerDied","Data":"645ed5944046a755ee82d5ee30cb9b5be7cbd591b4d797023d54d475492f538e"}
Jul 05 00:09:44.120665 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:09:44.120636    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-5fn94"
Jul 05 00:09:44.121107 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:09:44.120636    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-4pm8c"
Jul 05 00:09:44.121107 ip-10-0-142-202 kubenswrapper[2563]: E0705 00:09:44.120765    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-5fn94" podUID="c085d4be-fcda-4931-a018-02cbaca17c59"
Jul 05 00:09:44.121107 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:09:44.120636    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-dkwx5"
Jul 05 00:09:44.121107 ip-10-0-142-202 kubenswrapper[2563]: E0705 00:09:44.120848    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-4pm8c" podUID="e84aff2e-272a-42d8-8bd9-e37a98cfc7a7"
Jul 05 00:09:44.121107 ip-10-0-142-202 kubenswrapper[2563]: E0705 00:09:44.120908    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-dkwx5" podUID="00bb9c2a-9bed-4d69-9ae4-e4f1fad50578"
Jul 05 00:09:44.891082 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:09:44.890998    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c085d4be-fcda-4931-a018-02cbaca17c59-metrics-certs\") pod \"network-metrics-daemon-5fn94\" (UID: \"c085d4be-fcda-4931-a018-02cbaca17c59\") " pod="openshift-multus/network-metrics-daemon-5fn94"
Jul 05 00:09:44.891082 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:09:44.891059    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/00bb9c2a-9bed-4d69-9ae4-e4f1fad50578-original-pull-secret\") pod \"global-pull-secret-syncer-dkwx5\" (UID: \"00bb9c2a-9bed-4d69-9ae4-e4f1fad50578\") " pod="kube-system/global-pull-secret-syncer-dkwx5"
Jul 05 00:09:44.891317 ip-10-0-142-202 kubenswrapper[2563]: E0705 00:09:44.891139    2563 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 05 00:09:44.891317 ip-10-0-142-202 kubenswrapper[2563]: E0705 00:09:44.891151    2563 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 05 00:09:44.891317 ip-10-0-142-202 kubenswrapper[2563]: E0705 00:09:44.891199    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c085d4be-fcda-4931-a018-02cbaca17c59-metrics-certs podName:c085d4be-fcda-4931-a018-02cbaca17c59 nodeName:}" failed. No retries permitted until 2026-07-05 00:10:16.891185685 +0000 UTC m=+65.233514723 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/c085d4be-fcda-4931-a018-02cbaca17c59-metrics-certs") pod "network-metrics-daemon-5fn94" (UID: "c085d4be-fcda-4931-a018-02cbaca17c59") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 05 00:09:44.891317 ip-10-0-142-202 kubenswrapper[2563]: E0705 00:09:44.891233    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/00bb9c2a-9bed-4d69-9ae4-e4f1fad50578-original-pull-secret podName:00bb9c2a-9bed-4d69-9ae4-e4f1fad50578 nodeName:}" failed. No retries permitted until 2026-07-05 00:10:16.891210243 +0000 UTC m=+65.233539281 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/00bb9c2a-9bed-4d69-9ae4-e4f1fad50578-original-pull-secret") pod "global-pull-secret-syncer-dkwx5" (UID: "00bb9c2a-9bed-4d69-9ae4-e4f1fad50578") : object "kube-system"/"original-pull-secret" not registered
Jul 05 00:09:44.991616 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:09:44.991584    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-z4cn8\" (UniqueName: \"kubernetes.io/projected/e84aff2e-272a-42d8-8bd9-e37a98cfc7a7-kube-api-access-z4cn8\") pod \"network-check-target-4pm8c\" (UID: \"e84aff2e-272a-42d8-8bd9-e37a98cfc7a7\") " pod="openshift-network-diagnostics/network-check-target-4pm8c"
Jul 05 00:09:44.991769 ip-10-0-142-202 kubenswrapper[2563]: E0705 00:09:44.991729    2563 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 05 00:09:44.991769 ip-10-0-142-202 kubenswrapper[2563]: E0705 00:09:44.991747    2563 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 05 00:09:44.991769 ip-10-0-142-202 kubenswrapper[2563]: E0705 00:09:44.991756    2563 projected.go:196] Error preparing data for projected volume kube-api-access-z4cn8 for pod openshift-network-diagnostics/network-check-target-4pm8c: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 05 00:09:44.991885 ip-10-0-142-202 kubenswrapper[2563]: E0705 00:09:44.991804    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/e84aff2e-272a-42d8-8bd9-e37a98cfc7a7-kube-api-access-z4cn8 podName:e84aff2e-272a-42d8-8bd9-e37a98cfc7a7 nodeName:}" failed. No retries permitted until 2026-07-05 00:10:16.991788831 +0000 UTC m=+65.334117869 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-z4cn8" (UniqueName: "kubernetes.io/projected/e84aff2e-272a-42d8-8bd9-e37a98cfc7a7-kube-api-access-z4cn8") pod "network-check-target-4pm8c" (UID: "e84aff2e-272a-42d8-8bd9-e37a98cfc7a7") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 05 00:09:45.437663 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:09:45.437632    2563 generic.go:356] "Generic (PLEG): container finished" podID="aeaa1e13-243b-4c7f-8045-85ba45081c42" containerID="7d3adf1f8ff1d3354b2d96daf665688ad1431f5eda216876ae5521e6a77279fa" exitCode=0
Jul 05 00:09:45.438010 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:09:45.437677    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-g568x" event={"ID":"aeaa1e13-243b-4c7f-8045-85ba45081c42","Type":"ContainerDied","Data":"7d3adf1f8ff1d3354b2d96daf665688ad1431f5eda216876ae5521e6a77279fa"}
Jul 05 00:09:46.120689 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:09:46.120651    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-5fn94"
Jul 05 00:09:46.120860 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:09:46.120698    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-4pm8c"
Jul 05 00:09:46.120860 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:09:46.120811    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-dkwx5"
Jul 05 00:09:46.120860 ip-10-0-142-202 kubenswrapper[2563]: E0705 00:09:46.120824    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-5fn94" podUID="c085d4be-fcda-4931-a018-02cbaca17c59"
Jul 05 00:09:46.121015 ip-10-0-142-202 kubenswrapper[2563]: E0705 00:09:46.120909    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-4pm8c" podUID="e84aff2e-272a-42d8-8bd9-e37a98cfc7a7"
Jul 05 00:09:46.121067 ip-10-0-142-202 kubenswrapper[2563]: E0705 00:09:46.121013    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-dkwx5" podUID="00bb9c2a-9bed-4d69-9ae4-e4f1fad50578"
Jul 05 00:09:46.953776 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:09:46.953703    2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-202.ec2.internal" event="NodeReady"
Jul 05 00:09:46.954173 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:09:46.953834    2563 kubelet_node_status.go:489] "Fast updating node status as it just became ready"
Jul 05 00:09:46.994769 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:09:46.994736    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-snsng"]
Jul 05 00:09:46.997530 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:09:46.997511    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-snsng"
Jul 05 00:09:46.997667 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:09:46.997586    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-xqpm2"]
Jul 05 00:09:46.999423 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:09:46.999402    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\""
Jul 05 00:09:46.999514 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:09:46.999408    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-z9vd9\""
Jul 05 00:09:46.999514 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:09:46.999452    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\""
Jul 05 00:09:47.000620 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:09:47.000434    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-xqpm2"
Jul 05 00:09:47.002342 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:09:47.002323    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-ch6vj\""
Jul 05 00:09:47.002342 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:09:47.002332    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\""
Jul 05 00:09:47.002533 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:09:47.002433    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\""
Jul 05 00:09:47.002587 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:09:47.002567    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\""
Jul 05 00:09:47.007833 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:09:47.007296    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-snsng"]
Jul 05 00:09:47.010321 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:09:47.010295    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-xqpm2"]
Jul 05 00:09:47.108278 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:09:47.108253    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/0d75ce89-37fb-4d36-ae84-aee968c71dfa-tmp-dir\") pod \"dns-default-snsng\" (UID: \"0d75ce89-37fb-4d36-ae84-aee968c71dfa\") " pod="openshift-dns/dns-default-snsng"
Jul 05 00:09:47.108278 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:09:47.108279    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/63e680ec-84d5-4c0a-b259-84c64914aab7-cert\") pod \"ingress-canary-xqpm2\" (UID: \"63e680ec-84d5-4c0a-b259-84c64914aab7\") " pod="openshift-ingress-canary/ingress-canary-xqpm2"
Jul 05 00:09:47.108517 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:09:47.108305    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jb52h\" (UniqueName: \"kubernetes.io/projected/0d75ce89-37fb-4d36-ae84-aee968c71dfa-kube-api-access-jb52h\") pod \"dns-default-snsng\" (UID: \"0d75ce89-37fb-4d36-ae84-aee968c71dfa\") " pod="openshift-dns/dns-default-snsng"
Jul 05 00:09:47.108517 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:09:47.108337    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/0d75ce89-37fb-4d36-ae84-aee968c71dfa-metrics-tls\") pod \"dns-default-snsng\" (UID: \"0d75ce89-37fb-4d36-ae84-aee968c71dfa\") " pod="openshift-dns/dns-default-snsng"
Jul 05 00:09:47.108517 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:09:47.108438    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-m5bkm\" (UniqueName: \"kubernetes.io/projected/63e680ec-84d5-4c0a-b259-84c64914aab7-kube-api-access-m5bkm\") pod \"ingress-canary-xqpm2\" (UID: \"63e680ec-84d5-4c0a-b259-84c64914aab7\") " pod="openshift-ingress-canary/ingress-canary-xqpm2"
Jul 05 00:09:47.108517 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:09:47.108473    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/0d75ce89-37fb-4d36-ae84-aee968c71dfa-config-volume\") pod \"dns-default-snsng\" (UID: \"0d75ce89-37fb-4d36-ae84-aee968c71dfa\") " pod="openshift-dns/dns-default-snsng"
Jul 05 00:09:47.208879 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:09:47.208790    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-m5bkm\" (UniqueName: \"kubernetes.io/projected/63e680ec-84d5-4c0a-b259-84c64914aab7-kube-api-access-m5bkm\") pod \"ingress-canary-xqpm2\" (UID: \"63e680ec-84d5-4c0a-b259-84c64914aab7\") " pod="openshift-ingress-canary/ingress-canary-xqpm2"
Jul 05 00:09:47.208879 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:09:47.208836    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/0d75ce89-37fb-4d36-ae84-aee968c71dfa-config-volume\") pod \"dns-default-snsng\" (UID: \"0d75ce89-37fb-4d36-ae84-aee968c71dfa\") " pod="openshift-dns/dns-default-snsng"
Jul 05 00:09:47.208879 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:09:47.208870    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/0d75ce89-37fb-4d36-ae84-aee968c71dfa-tmp-dir\") pod \"dns-default-snsng\" (UID: \"0d75ce89-37fb-4d36-ae84-aee968c71dfa\") " pod="openshift-dns/dns-default-snsng"
Jul 05 00:09:47.209157 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:09:47.208894    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/63e680ec-84d5-4c0a-b259-84c64914aab7-cert\") pod \"ingress-canary-xqpm2\" (UID: \"63e680ec-84d5-4c0a-b259-84c64914aab7\") " pod="openshift-ingress-canary/ingress-canary-xqpm2"
Jul 05 00:09:47.209157 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:09:47.208936    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jb52h\" (UniqueName: \"kubernetes.io/projected/0d75ce89-37fb-4d36-ae84-aee968c71dfa-kube-api-access-jb52h\") pod \"dns-default-snsng\" (UID: \"0d75ce89-37fb-4d36-ae84-aee968c71dfa\") " pod="openshift-dns/dns-default-snsng"
Jul 05 00:09:47.209157 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:09:47.208968    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/0d75ce89-37fb-4d36-ae84-aee968c71dfa-metrics-tls\") pod \"dns-default-snsng\" (UID: \"0d75ce89-37fb-4d36-ae84-aee968c71dfa\") " pod="openshift-dns/dns-default-snsng"
Jul 05 00:09:47.209157 ip-10-0-142-202 kubenswrapper[2563]: E0705 00:09:47.209070    2563 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 05 00:09:47.209157 ip-10-0-142-202 kubenswrapper[2563]: E0705 00:09:47.209123    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0d75ce89-37fb-4d36-ae84-aee968c71dfa-metrics-tls podName:0d75ce89-37fb-4d36-ae84-aee968c71dfa nodeName:}" failed. No retries permitted until 2026-07-05 00:09:47.709105267 +0000 UTC m=+36.051434308 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/0d75ce89-37fb-4d36-ae84-aee968c71dfa-metrics-tls") pod "dns-default-snsng" (UID: "0d75ce89-37fb-4d36-ae84-aee968c71dfa") : secret "dns-default-metrics-tls" not found
Jul 05 00:09:47.209407 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:09:47.209247    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/0d75ce89-37fb-4d36-ae84-aee968c71dfa-tmp-dir\") pod \"dns-default-snsng\" (UID: \"0d75ce89-37fb-4d36-ae84-aee968c71dfa\") " pod="openshift-dns/dns-default-snsng"
Jul 05 00:09:47.209571 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:09:47.209546    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/0d75ce89-37fb-4d36-ae84-aee968c71dfa-config-volume\") pod \"dns-default-snsng\" (UID: \"0d75ce89-37fb-4d36-ae84-aee968c71dfa\") " pod="openshift-dns/dns-default-snsng"
Jul 05 00:09:47.213585 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:09:47.213563    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/63e680ec-84d5-4c0a-b259-84c64914aab7-cert\") pod \"ingress-canary-xqpm2\" (UID: \"63e680ec-84d5-4c0a-b259-84c64914aab7\") " pod="openshift-ingress-canary/ingress-canary-xqpm2"
Jul 05 00:09:47.216755 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:09:47.216710    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-jb52h\" (UniqueName: \"kubernetes.io/projected/0d75ce89-37fb-4d36-ae84-aee968c71dfa-kube-api-access-jb52h\") pod \"dns-default-snsng\" (UID: \"0d75ce89-37fb-4d36-ae84-aee968c71dfa\") " pod="openshift-dns/dns-default-snsng"
Jul 05 00:09:47.217551 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:09:47.217510    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-m5bkm\" (UniqueName: \"kubernetes.io/projected/63e680ec-84d5-4c0a-b259-84c64914aab7-kube-api-access-m5bkm\") pod \"ingress-canary-xqpm2\" (UID: \"63e680ec-84d5-4c0a-b259-84c64914aab7\") " pod="openshift-ingress-canary/ingress-canary-xqpm2"
Jul 05 00:09:47.316270 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:09:47.316230    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-xqpm2"
Jul 05 00:09:47.465040 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:09:47.464968    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-xqpm2"]
Jul 05 00:09:47.471150 ip-10-0-142-202 kubenswrapper[2563]: W0705 00:09:47.471120    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod63e680ec_84d5_4c0a_b259_84c64914aab7.slice/crio-66cfc4a13b7f9bdfdd27d568d173abf8dddd1fd4d07f9477c759a9eafc082713 WatchSource:0}: Error finding container 66cfc4a13b7f9bdfdd27d568d173abf8dddd1fd4d07f9477c759a9eafc082713: Status 404 returned error can't find the container with id 66cfc4a13b7f9bdfdd27d568d173abf8dddd1fd4d07f9477c759a9eafc082713
Jul 05 00:09:47.712235 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:09:47.712184    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/0d75ce89-37fb-4d36-ae84-aee968c71dfa-metrics-tls\") pod \"dns-default-snsng\" (UID: \"0d75ce89-37fb-4d36-ae84-aee968c71dfa\") " pod="openshift-dns/dns-default-snsng"
Jul 05 00:09:47.712410 ip-10-0-142-202 kubenswrapper[2563]: E0705 00:09:47.712352    2563 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 05 00:09:47.712467 ip-10-0-142-202 kubenswrapper[2563]: E0705 00:09:47.712429    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0d75ce89-37fb-4d36-ae84-aee968c71dfa-metrics-tls podName:0d75ce89-37fb-4d36-ae84-aee968c71dfa nodeName:}" failed. No retries permitted until 2026-07-05 00:09:48.712405036 +0000 UTC m=+37.054734075 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/0d75ce89-37fb-4d36-ae84-aee968c71dfa-metrics-tls") pod "dns-default-snsng" (UID: "0d75ce89-37fb-4d36-ae84-aee968c71dfa") : secret "dns-default-metrics-tls" not found
Jul 05 00:09:48.121269 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:09:48.121228    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-5fn94"
Jul 05 00:09:48.121269 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:09:48.121249    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-4pm8c"
Jul 05 00:09:48.121269 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:09:48.121249    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-dkwx5"
Jul 05 00:09:48.124480 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:09:48.123911    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Jul 05 00:09:48.124480 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:09:48.123927    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-fmjx4\""
Jul 05 00:09:48.124480 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:09:48.123990    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-m9r6w\""
Jul 05 00:09:48.124480 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:09:48.123915    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Jul 05 00:09:48.124480 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:09:48.123915    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Jul 05 00:09:48.124480 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:09:48.124328    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Jul 05 00:09:48.446731 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:09:48.446637    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-xqpm2" event={"ID":"63e680ec-84d5-4c0a-b259-84c64914aab7","Type":"ContainerStarted","Data":"66cfc4a13b7f9bdfdd27d568d173abf8dddd1fd4d07f9477c759a9eafc082713"}
Jul 05 00:09:48.721031 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:09:48.720946    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/0d75ce89-37fb-4d36-ae84-aee968c71dfa-metrics-tls\") pod \"dns-default-snsng\" (UID: \"0d75ce89-37fb-4d36-ae84-aee968c71dfa\") " pod="openshift-dns/dns-default-snsng"
Jul 05 00:09:48.721198 ip-10-0-142-202 kubenswrapper[2563]: E0705 00:09:48.721118    2563 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 05 00:09:48.721283 ip-10-0-142-202 kubenswrapper[2563]: E0705 00:09:48.721201    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0d75ce89-37fb-4d36-ae84-aee968c71dfa-metrics-tls podName:0d75ce89-37fb-4d36-ae84-aee968c71dfa nodeName:}" failed. No retries permitted until 2026-07-05 00:09:50.721178923 +0000 UTC m=+39.063507964 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/0d75ce89-37fb-4d36-ae84-aee968c71dfa-metrics-tls") pod "dns-default-snsng" (UID: "0d75ce89-37fb-4d36-ae84-aee968c71dfa") : secret "dns-default-metrics-tls" not found
Jul 05 00:09:50.736437 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:09:50.736398    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/0d75ce89-37fb-4d36-ae84-aee968c71dfa-metrics-tls\") pod \"dns-default-snsng\" (UID: \"0d75ce89-37fb-4d36-ae84-aee968c71dfa\") " pod="openshift-dns/dns-default-snsng"
Jul 05 00:09:50.737067 ip-10-0-142-202 kubenswrapper[2563]: E0705 00:09:50.736546    2563 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 05 00:09:50.737067 ip-10-0-142-202 kubenswrapper[2563]: E0705 00:09:50.736611    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0d75ce89-37fb-4d36-ae84-aee968c71dfa-metrics-tls podName:0d75ce89-37fb-4d36-ae84-aee968c71dfa nodeName:}" failed. No retries permitted until 2026-07-05 00:09:54.736592365 +0000 UTC m=+43.078921410 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/0d75ce89-37fb-4d36-ae84-aee968c71dfa-metrics-tls") pod "dns-default-snsng" (UID: "0d75ce89-37fb-4d36-ae84-aee968c71dfa") : secret "dns-default-metrics-tls" not found
Jul 05 00:09:51.455793 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:09:51.455160    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-xqpm2" event={"ID":"63e680ec-84d5-4c0a-b259-84c64914aab7","Type":"ContainerStarted","Data":"ba3f71527e96a138c4b73aae4dbf405ec9eb638662a1b3a8cb9ce4f3aac5022c"}
Jul 05 00:09:51.458395 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:09:51.458371    2563 generic.go:356] "Generic (PLEG): container finished" podID="aeaa1e13-243b-4c7f-8045-85ba45081c42" containerID="d3b4de23b5313ab8dcf2896f11d70463e0305057d320df8a80cf1d3f1848d787" exitCode=0
Jul 05 00:09:51.458492 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:09:51.458416    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-g568x" event={"ID":"aeaa1e13-243b-4c7f-8045-85ba45081c42","Type":"ContainerDied","Data":"d3b4de23b5313ab8dcf2896f11d70463e0305057d320df8a80cf1d3f1848d787"}
Jul 05 00:09:51.472095 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:09:51.472055    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-xqpm2" podStartSLOduration=1.684860149 podStartE2EDuration="5.472040851s" totalImagesPullingTime="3.787180702s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-05 00:09:46 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-05 00:09:51.471399092 +0000 UTC m=+39.813728153" watchObservedRunningTime="2026-07-05 00:09:51.472040851 +0000 UTC m=+39.814369910"
Jul 05 00:09:52.462979 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:09:52.462941    2563 generic.go:356] "Generic (PLEG): container finished" podID="aeaa1e13-243b-4c7f-8045-85ba45081c42" containerID="543a72fe7f169a897789755259b53fa3d465a8dca7656af0f016e7d7fb1d8fa1" exitCode=0
Jul 05 00:09:52.463451 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:09:52.463024    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-g568x" event={"ID":"aeaa1e13-243b-4c7f-8045-85ba45081c42","Type":"ContainerDied","Data":"543a72fe7f169a897789755259b53fa3d465a8dca7656af0f016e7d7fb1d8fa1"}
Jul 05 00:09:53.470497 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:09:53.470466    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-g568x" event={"ID":"aeaa1e13-243b-4c7f-8045-85ba45081c42","Type":"ContainerStarted","Data":"1db9c5c071cfddbfcc23f06aecbe081d7a3468cc231b0d05011f164d34bf3097"}
Jul 05 00:09:53.490170 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:09:53.490125    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-g568x" podStartSLOduration=6.973104998 podStartE2EDuration="40.490108691s" totalImagesPullingTime="33.517003693s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-05 00:09:13 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-05 00:09:53.489064624 +0000 UTC m=+41.831393684" watchObservedRunningTime="2026-07-05 00:09:53.490108691 +0000 UTC m=+41.832437773"
Jul 05 00:09:54.765530 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:09:54.765492    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/0d75ce89-37fb-4d36-ae84-aee968c71dfa-metrics-tls\") pod \"dns-default-snsng\" (UID: \"0d75ce89-37fb-4d36-ae84-aee968c71dfa\") " pod="openshift-dns/dns-default-snsng"
Jul 05 00:09:54.765978 ip-10-0-142-202 kubenswrapper[2563]: E0705 00:09:54.765676    2563 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 05 00:09:54.765978 ip-10-0-142-202 kubenswrapper[2563]: E0705 00:09:54.765761    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0d75ce89-37fb-4d36-ae84-aee968c71dfa-metrics-tls podName:0d75ce89-37fb-4d36-ae84-aee968c71dfa nodeName:}" failed. No retries permitted until 2026-07-05 00:10:02.765738504 +0000 UTC m=+51.108067558 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/0d75ce89-37fb-4d36-ae84-aee968c71dfa-metrics-tls") pod "dns-default-snsng" (UID: "0d75ce89-37fb-4d36-ae84-aee968c71dfa") : secret "dns-default-metrics-tls" not found
Jul 05 00:10:02.818473 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:10:02.818438    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/0d75ce89-37fb-4d36-ae84-aee968c71dfa-metrics-tls\") pod \"dns-default-snsng\" (UID: \"0d75ce89-37fb-4d36-ae84-aee968c71dfa\") " pod="openshift-dns/dns-default-snsng"
Jul 05 00:10:02.818916 ip-10-0-142-202 kubenswrapper[2563]: E0705 00:10:02.818589    2563 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 05 00:10:02.818916 ip-10-0-142-202 kubenswrapper[2563]: E0705 00:10:02.818654    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0d75ce89-37fb-4d36-ae84-aee968c71dfa-metrics-tls podName:0d75ce89-37fb-4d36-ae84-aee968c71dfa nodeName:}" failed. No retries permitted until 2026-07-05 00:10:18.818635019 +0000 UTC m=+67.160964076 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/0d75ce89-37fb-4d36-ae84-aee968c71dfa-metrics-tls") pod "dns-default-snsng" (UID: "0d75ce89-37fb-4d36-ae84-aee968c71dfa") : secret "dns-default-metrics-tls" not found
Jul 05 00:10:13.446336 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:10:13.446309    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-7sxq4"
Jul 05 00:10:16.907046 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:10:16.907004    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c085d4be-fcda-4931-a018-02cbaca17c59-metrics-certs\") pod \"network-metrics-daemon-5fn94\" (UID: \"c085d4be-fcda-4931-a018-02cbaca17c59\") " pod="openshift-multus/network-metrics-daemon-5fn94"
Jul 05 00:10:16.907463 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:10:16.907069    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/00bb9c2a-9bed-4d69-9ae4-e4f1fad50578-original-pull-secret\") pod \"global-pull-secret-syncer-dkwx5\" (UID: \"00bb9c2a-9bed-4d69-9ae4-e4f1fad50578\") " pod="kube-system/global-pull-secret-syncer-dkwx5"
Jul 05 00:10:16.909307 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:10:16.909287    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Jul 05 00:10:16.909419 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:10:16.909289    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Jul 05 00:10:16.917929 ip-10-0-142-202 kubenswrapper[2563]: E0705 00:10:16.917909    2563 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found
Jul 05 00:10:16.917999 ip-10-0-142-202 kubenswrapper[2563]: E0705 00:10:16.917987    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c085d4be-fcda-4931-a018-02cbaca17c59-metrics-certs podName:c085d4be-fcda-4931-a018-02cbaca17c59 nodeName:}" failed. No retries permitted until 2026-07-05 00:11:20.917968101 +0000 UTC m=+129.260297153 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/c085d4be-fcda-4931-a018-02cbaca17c59-metrics-certs") pod "network-metrics-daemon-5fn94" (UID: "c085d4be-fcda-4931-a018-02cbaca17c59") : secret "metrics-daemon-secret" not found
Jul 05 00:10:16.920316 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:10:16.920289    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/00bb9c2a-9bed-4d69-9ae4-e4f1fad50578-original-pull-secret\") pod \"global-pull-secret-syncer-dkwx5\" (UID: \"00bb9c2a-9bed-4d69-9ae4-e4f1fad50578\") " pod="kube-system/global-pull-secret-syncer-dkwx5"
Jul 05 00:10:16.949700 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:10:16.949684    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-dkwx5"
Jul 05 00:10:17.008090 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:10:17.008048    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-z4cn8\" (UniqueName: \"kubernetes.io/projected/e84aff2e-272a-42d8-8bd9-e37a98cfc7a7-kube-api-access-z4cn8\") pod \"network-check-target-4pm8c\" (UID: \"e84aff2e-272a-42d8-8bd9-e37a98cfc7a7\") " pod="openshift-network-diagnostics/network-check-target-4pm8c"
Jul 05 00:10:17.010684 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:10:17.010654    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Jul 05 00:10:17.020734 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:10:17.020715    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Jul 05 00:10:17.031822 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:10:17.031789    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-z4cn8\" (UniqueName: \"kubernetes.io/projected/e84aff2e-272a-42d8-8bd9-e37a98cfc7a7-kube-api-access-z4cn8\") pod \"network-check-target-4pm8c\" (UID: \"e84aff2e-272a-42d8-8bd9-e37a98cfc7a7\") " pod="openshift-network-diagnostics/network-check-target-4pm8c"
Jul 05 00:10:17.063659 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:10:17.063630    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-dkwx5"]
Jul 05 00:10:17.067529 ip-10-0-142-202 kubenswrapper[2563]: W0705 00:10:17.067503    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod00bb9c2a_9bed_4d69_9ae4_e4f1fad50578.slice/crio-44e1c5d35e58cdc7f9df2022c9650818105f4456eda7353c6f87f030b293b165 WatchSource:0}: Error finding container 44e1c5d35e58cdc7f9df2022c9650818105f4456eda7353c6f87f030b293b165: Status 404 returned error can't find the container with id 44e1c5d35e58cdc7f9df2022c9650818105f4456eda7353c6f87f030b293b165
Jul 05 00:10:17.244643 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:10:17.244571    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-fmjx4\""
Jul 05 00:10:17.252924 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:10:17.252902    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-4pm8c"
Jul 05 00:10:17.369343 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:10:17.369312    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-4pm8c"]
Jul 05 00:10:17.373593 ip-10-0-142-202 kubenswrapper[2563]: W0705 00:10:17.373569    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode84aff2e_272a_42d8_8bd9_e37a98cfc7a7.slice/crio-386ba65939f4bb5b1c14b7f78521a7c9d2340117b7b766869ccdbd2898882ef7 WatchSource:0}: Error finding container 386ba65939f4bb5b1c14b7f78521a7c9d2340117b7b766869ccdbd2898882ef7: Status 404 returned error can't find the container with id 386ba65939f4bb5b1c14b7f78521a7c9d2340117b7b766869ccdbd2898882ef7
Jul 05 00:10:17.526085 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:10:17.526035    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-4pm8c" event={"ID":"e84aff2e-272a-42d8-8bd9-e37a98cfc7a7","Type":"ContainerStarted","Data":"386ba65939f4bb5b1c14b7f78521a7c9d2340117b7b766869ccdbd2898882ef7"}
Jul 05 00:10:17.527120 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:10:17.527082    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-dkwx5" event={"ID":"00bb9c2a-9bed-4d69-9ae4-e4f1fad50578","Type":"ContainerStarted","Data":"44e1c5d35e58cdc7f9df2022c9650818105f4456eda7353c6f87f030b293b165"}
Jul 05 00:10:18.822904 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:10:18.822858    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/0d75ce89-37fb-4d36-ae84-aee968c71dfa-metrics-tls\") pod \"dns-default-snsng\" (UID: \"0d75ce89-37fb-4d36-ae84-aee968c71dfa\") " pod="openshift-dns/dns-default-snsng"
Jul 05 00:10:18.823395 ip-10-0-142-202 kubenswrapper[2563]: E0705 00:10:18.823048    2563 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 05 00:10:18.823395 ip-10-0-142-202 kubenswrapper[2563]: E0705 00:10:18.823148    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0d75ce89-37fb-4d36-ae84-aee968c71dfa-metrics-tls podName:0d75ce89-37fb-4d36-ae84-aee968c71dfa nodeName:}" failed. No retries permitted until 2026-07-05 00:10:50.823122597 +0000 UTC m=+99.165451638 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/0d75ce89-37fb-4d36-ae84-aee968c71dfa-metrics-tls") pod "dns-default-snsng" (UID: "0d75ce89-37fb-4d36-ae84-aee968c71dfa") : secret "dns-default-metrics-tls" not found
Jul 05 00:10:22.540926 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:10:22.540886    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-4pm8c" event={"ID":"e84aff2e-272a-42d8-8bd9-e37a98cfc7a7","Type":"ContainerStarted","Data":"5a3f2561f7c24c29115cac4bb645f015149d90b4f573af7d78e0ad1fb44847a0"}
Jul 05 00:10:22.542373 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:10:22.542353    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-dkwx5" event={"ID":"00bb9c2a-9bed-4d69-9ae4-e4f1fad50578","Type":"ContainerStarted","Data":"0942da703528f0eece3681ef777c095cda7c29450c3d7b8c3aabb5566db93720"}
Jul 05 00:10:23.563233 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:10:23.563154    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-dkwx5" podStartSLOduration=65.229203741 podStartE2EDuration="1m10.5631354s" totalImagesPullingTime="5.333931659s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-05 00:09:13 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-05 00:10:23.559946067 +0000 UTC m=+71.902275147" watchObservedRunningTime="2026-07-05 00:10:23.5631354 +0000 UTC m=+71.905464456"
Jul 05 00:10:24.451558 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:10:24.451504    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-4pm8c" podStartSLOduration=66.415161004 podStartE2EDuration="1m11.451486076s" totalImagesPullingTime="5.036325072s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-05 00:09:13 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-05 00:10:23.572344577 +0000 UTC m=+71.914673637" watchObservedRunningTime="2026-07-05 00:10:24.451486076 +0000 UTC m=+72.793815114"
Jul 05 00:10:24.451840 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:10:24.451827    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-6b958749d4-wh9tw"]
Jul 05 00:10:24.453605 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:10:24.453584    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-6b958749d4-wh9tw"
Jul 05 00:10:24.455455 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:10:24.455434    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"managed-serviceaccount-dockercfg-rz7t5\""
Jul 05 00:10:24.455601 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:10:24.455534    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"openshift-service-ca.crt\""
Jul 05 00:10:24.455601 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:10:24.455542    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"open-cluster-management-image-pull-credentials\""
Jul 05 00:10:24.456242 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:10:24.456196    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"kube-root-ca.crt\""
Jul 05 00:10:24.456242 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:10:24.456234    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"managed-serviceaccount-hub-kubeconfig\""
Jul 05 00:10:24.462196 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:10:24.462177    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-6b958749d4-wh9tw"]
Jul 05 00:10:24.465166 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:10:24.465147    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-6657fbc78f-qr4kz"]
Jul 05 00:10:24.467084 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:10:24.467069    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-6657fbc78f-qr4kz"
Jul 05 00:10:24.469083 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:10:24.469067    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"work-manager-hub-kubeconfig\""
Jul 05 00:10:24.476948 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:10:24.476929    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-6657fbc78f-qr4kz"]
Jul 05 00:10:24.558232 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:10:24.558194    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/5ca7870b-a946-4574-8c66-e9d4bfda1406-tmp\") pod \"klusterlet-addon-workmgr-6657fbc78f-qr4kz\" (UID: \"5ca7870b-a946-4574-8c66-e9d4bfda1406\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-6657fbc78f-qr4kz"
Jul 05 00:10:24.558393 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:10:24.558240    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pkfbc\" (UniqueName: \"kubernetes.io/projected/45610df8-3547-4c76-ac88-146c5c32ae8a-kube-api-access-pkfbc\") pod \"managed-serviceaccount-addon-agent-6b958749d4-wh9tw\" (UID: \"45610df8-3547-4c76-ac88-146c5c32ae8a\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-6b958749d4-wh9tw"
Jul 05 00:10:24.558393 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:10:24.558303    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hdsrr\" (UniqueName: \"kubernetes.io/projected/5ca7870b-a946-4574-8c66-e9d4bfda1406-kube-api-access-hdsrr\") pod \"klusterlet-addon-workmgr-6657fbc78f-qr4kz\" (UID: \"5ca7870b-a946-4574-8c66-e9d4bfda1406\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-6657fbc78f-qr4kz"
Jul 05 00:10:24.558393 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:10:24.558323    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/45610df8-3547-4c76-ac88-146c5c32ae8a-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-6b958749d4-wh9tw\" (UID: \"45610df8-3547-4c76-ac88-146c5c32ae8a\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-6b958749d4-wh9tw"
Jul 05 00:10:24.558393 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:10:24.558380    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/5ca7870b-a946-4574-8c66-e9d4bfda1406-klusterlet-config\") pod \"klusterlet-addon-workmgr-6657fbc78f-qr4kz\" (UID: \"5ca7870b-a946-4574-8c66-e9d4bfda1406\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-6657fbc78f-qr4kz"
Jul 05 00:10:24.659519 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:10:24.659490    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/5ca7870b-a946-4574-8c66-e9d4bfda1406-tmp\") pod \"klusterlet-addon-workmgr-6657fbc78f-qr4kz\" (UID: \"5ca7870b-a946-4574-8c66-e9d4bfda1406\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-6657fbc78f-qr4kz"
Jul 05 00:10:24.659976 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:10:24.659524    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pkfbc\" (UniqueName: \"kubernetes.io/projected/45610df8-3547-4c76-ac88-146c5c32ae8a-kube-api-access-pkfbc\") pod \"managed-serviceaccount-addon-agent-6b958749d4-wh9tw\" (UID: \"45610df8-3547-4c76-ac88-146c5c32ae8a\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-6b958749d4-wh9tw"
Jul 05 00:10:24.659976 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:10:24.659557    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hdsrr\" (UniqueName: \"kubernetes.io/projected/5ca7870b-a946-4574-8c66-e9d4bfda1406-kube-api-access-hdsrr\") pod \"klusterlet-addon-workmgr-6657fbc78f-qr4kz\" (UID: \"5ca7870b-a946-4574-8c66-e9d4bfda1406\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-6657fbc78f-qr4kz"
Jul 05 00:10:24.659976 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:10:24.659577    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/45610df8-3547-4c76-ac88-146c5c32ae8a-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-6b958749d4-wh9tw\" (UID: \"45610df8-3547-4c76-ac88-146c5c32ae8a\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-6b958749d4-wh9tw"
Jul 05 00:10:24.659976 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:10:24.659641    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/5ca7870b-a946-4574-8c66-e9d4bfda1406-klusterlet-config\") pod \"klusterlet-addon-workmgr-6657fbc78f-qr4kz\" (UID: \"5ca7870b-a946-4574-8c66-e9d4bfda1406\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-6657fbc78f-qr4kz"
Jul 05 00:10:24.659976 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:10:24.659954    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/5ca7870b-a946-4574-8c66-e9d4bfda1406-tmp\") pod \"klusterlet-addon-workmgr-6657fbc78f-qr4kz\" (UID: \"5ca7870b-a946-4574-8c66-e9d4bfda1406\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-6657fbc78f-qr4kz"
Jul 05 00:10:24.662001 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:10:24.661981    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/5ca7870b-a946-4574-8c66-e9d4bfda1406-klusterlet-config\") pod \"klusterlet-addon-workmgr-6657fbc78f-qr4kz\" (UID: \"5ca7870b-a946-4574-8c66-e9d4bfda1406\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-6657fbc78f-qr4kz"
Jul 05 00:10:24.662092 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:10:24.662043    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/45610df8-3547-4c76-ac88-146c5c32ae8a-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-6b958749d4-wh9tw\" (UID: \"45610df8-3547-4c76-ac88-146c5c32ae8a\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-6b958749d4-wh9tw"
Jul 05 00:10:24.666652 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:10:24.666628    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-hdsrr\" (UniqueName: \"kubernetes.io/projected/5ca7870b-a946-4574-8c66-e9d4bfda1406-kube-api-access-hdsrr\") pod \"klusterlet-addon-workmgr-6657fbc78f-qr4kz\" (UID: \"5ca7870b-a946-4574-8c66-e9d4bfda1406\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-6657fbc78f-qr4kz"
Jul 05 00:10:24.666764 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:10:24.666671    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-pkfbc\" (UniqueName: \"kubernetes.io/projected/45610df8-3547-4c76-ac88-146c5c32ae8a-kube-api-access-pkfbc\") pod \"managed-serviceaccount-addon-agent-6b958749d4-wh9tw\" (UID: \"45610df8-3547-4c76-ac88-146c5c32ae8a\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-6b958749d4-wh9tw"
Jul 05 00:10:24.770529 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:10:24.770450    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-6b958749d4-wh9tw"
Jul 05 00:10:24.778336 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:10:24.778314    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-6657fbc78f-qr4kz"
Jul 05 00:10:24.894269 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:10:24.894239    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-6b958749d4-wh9tw"]
Jul 05 00:10:24.897241 ip-10-0-142-202 kubenswrapper[2563]: W0705 00:10:24.897201    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod45610df8_3547_4c76_ac88_146c5c32ae8a.slice/crio-81c32a4cbd9b2d23f7442b08a8114591343df6e3a27423827aa146768477bbd5 WatchSource:0}: Error finding container 81c32a4cbd9b2d23f7442b08a8114591343df6e3a27423827aa146768477bbd5: Status 404 returned error can't find the container with id 81c32a4cbd9b2d23f7442b08a8114591343df6e3a27423827aa146768477bbd5
Jul 05 00:10:24.912544 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:10:24.912522    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-6657fbc78f-qr4kz"]
Jul 05 00:10:24.915790 ip-10-0-142-202 kubenswrapper[2563]: W0705 00:10:24.915766    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5ca7870b_a946_4574_8c66_e9d4bfda1406.slice/crio-00f672259298d03ceadd0721df4160d1f8509e295dfcdbcde0ce006e1486c486 WatchSource:0}: Error finding container 00f672259298d03ceadd0721df4160d1f8509e295dfcdbcde0ce006e1486c486: Status 404 returned error can't find the container with id 00f672259298d03ceadd0721df4160d1f8509e295dfcdbcde0ce006e1486c486
Jul 05 00:10:25.550249 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:10:25.550199    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-6657fbc78f-qr4kz" event={"ID":"5ca7870b-a946-4574-8c66-e9d4bfda1406","Type":"ContainerStarted","Data":"00f672259298d03ceadd0721df4160d1f8509e295dfcdbcde0ce006e1486c486"}
Jul 05 00:10:25.551163 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:10:25.551140    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-6b958749d4-wh9tw" event={"ID":"45610df8-3547-4c76-ac88-146c5c32ae8a","Type":"ContainerStarted","Data":"81c32a4cbd9b2d23f7442b08a8114591343df6e3a27423827aa146768477bbd5"}
Jul 05 00:10:28.562537 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:10:28.562493    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-6b958749d4-wh9tw" event={"ID":"45610df8-3547-4c76-ac88-146c5c32ae8a","Type":"ContainerStarted","Data":"137c03ac702e675fa0cc8b20a93bb01920de859b377c84458d762166aa70e0ea"}
Jul 05 00:10:28.576666 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:10:28.576615    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-6b958749d4-wh9tw" podStartSLOduration=1.7916767409999999 podStartE2EDuration="4.576598432s" totalImagesPullingTime="2.784921691s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-05 00:10:24 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-05 00:10:28.574911958 +0000 UTC m=+76.917241020" watchObservedRunningTime="2026-07-05 00:10:28.576598432 +0000 UTC m=+76.918927491"
Jul 05 00:10:29.567429 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:10:29.567391    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-6657fbc78f-qr4kz" event={"ID":"5ca7870b-a946-4574-8c66-e9d4bfda1406","Type":"ContainerStarted","Data":"81ff1cd13f428ea2ff009a88c0715a803660210e976e08dfc9fbaaf3e4b5e289"}
Jul 05 00:10:30.571109 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:10:30.571082    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-6657fbc78f-qr4kz"
Jul 05 00:10:30.572720 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:10:30.572701    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-6657fbc78f-qr4kz"
Jul 05 00:10:30.586975 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:10:30.586931    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-6657fbc78f-qr4kz" podStartSLOduration=2.613189117 podStartE2EDuration="6.58691649s" totalImagesPullingTime="3.973727373s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-05 00:10:24 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-05 00:10:29.58207539 +0000 UTC m=+77.924404452" watchObservedRunningTime="2026-07-05 00:10:30.58691649 +0000 UTC m=+78.929245568"
Jul 05 00:10:33.546083 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:10:33.546044    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-4pm8c"
Jul 05 00:10:50.837567 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:10:50.837507    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/0d75ce89-37fb-4d36-ae84-aee968c71dfa-metrics-tls\") pod \"dns-default-snsng\" (UID: \"0d75ce89-37fb-4d36-ae84-aee968c71dfa\") " pod="openshift-dns/dns-default-snsng"
Jul 05 00:10:50.837996 ip-10-0-142-202 kubenswrapper[2563]: E0705 00:10:50.837644    2563 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 05 00:10:50.837996 ip-10-0-142-202 kubenswrapper[2563]: E0705 00:10:50.837703    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0d75ce89-37fb-4d36-ae84-aee968c71dfa-metrics-tls podName:0d75ce89-37fb-4d36-ae84-aee968c71dfa nodeName:}" failed. No retries permitted until 2026-07-05 00:11:54.837689805 +0000 UTC m=+163.180018843 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/0d75ce89-37fb-4d36-ae84-aee968c71dfa-metrics-tls") pod "dns-default-snsng" (UID: "0d75ce89-37fb-4d36-ae84-aee968c71dfa") : secret "dns-default-metrics-tls" not found
Jul 05 00:10:53.548913 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:10:53.548887    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-4pm8c"
Jul 05 00:11:20.931771 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:11:20.931734    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c085d4be-fcda-4931-a018-02cbaca17c59-metrics-certs\") pod \"network-metrics-daemon-5fn94\" (UID: \"c085d4be-fcda-4931-a018-02cbaca17c59\") " pod="openshift-multus/network-metrics-daemon-5fn94"
Jul 05 00:11:20.932263 ip-10-0-142-202 kubenswrapper[2563]: E0705 00:11:20.931848    2563 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found
Jul 05 00:11:20.932263 ip-10-0-142-202 kubenswrapper[2563]: E0705 00:11:20.931905    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c085d4be-fcda-4931-a018-02cbaca17c59-metrics-certs podName:c085d4be-fcda-4931-a018-02cbaca17c59 nodeName:}" failed. No retries permitted until 2026-07-05 00:13:22.931891481 +0000 UTC m=+251.274220519 (durationBeforeRetry 2m2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/c085d4be-fcda-4931-a018-02cbaca17c59-metrics-certs") pod "network-metrics-daemon-5fn94" (UID: "c085d4be-fcda-4931-a018-02cbaca17c59") : secret "metrics-daemon-secret" not found
Jul 05 00:11:50.011860 ip-10-0-142-202 kubenswrapper[2563]: E0705 00:11:50.011805    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[metrics-tls], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-dns/dns-default-snsng" podUID="0d75ce89-37fb-4d36-ae84-aee968c71dfa"
Jul 05 00:11:50.786433 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:11:50.786404    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-snsng"
Jul 05 00:11:51.134686 ip-10-0-142-202 kubenswrapper[2563]: E0705 00:11:51.134579    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[metrics-certs], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-multus/network-metrics-daemon-5fn94" podUID="c085d4be-fcda-4931-a018-02cbaca17c59"
Jul 05 00:11:54.856235 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:11:54.856188    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/0d75ce89-37fb-4d36-ae84-aee968c71dfa-metrics-tls\") pod \"dns-default-snsng\" (UID: \"0d75ce89-37fb-4d36-ae84-aee968c71dfa\") " pod="openshift-dns/dns-default-snsng"
Jul 05 00:11:54.856592 ip-10-0-142-202 kubenswrapper[2563]: E0705 00:11:54.856318    2563 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 05 00:11:54.856592 ip-10-0-142-202 kubenswrapper[2563]: E0705 00:11:54.856382    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0d75ce89-37fb-4d36-ae84-aee968c71dfa-metrics-tls podName:0d75ce89-37fb-4d36-ae84-aee968c71dfa nodeName:}" failed. No retries permitted until 2026-07-05 00:13:56.856366013 +0000 UTC m=+285.198695051 (durationBeforeRetry 2m2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/0d75ce89-37fb-4d36-ae84-aee968c71dfa-metrics-tls") pod "dns-default-snsng" (UID: "0d75ce89-37fb-4d36-ae84-aee968c71dfa") : secret "dns-default-metrics-tls" not found
Jul 05 00:12:06.124552 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:12:06.124484    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-5fn94"
Jul 05 00:12:27.888694 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:12:27.888658    2563 generic.go:356] "Generic (PLEG): container finished" podID="45610df8-3547-4c76-ac88-146c5c32ae8a" containerID="137c03ac702e675fa0cc8b20a93bb01920de859b377c84458d762166aa70e0ea" exitCode=255
Jul 05 00:12:27.889138 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:12:27.888699    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-6b958749d4-wh9tw" event={"ID":"45610df8-3547-4c76-ac88-146c5c32ae8a","Type":"ContainerDied","Data":"137c03ac702e675fa0cc8b20a93bb01920de859b377c84458d762166aa70e0ea"}
Jul 05 00:12:27.899406 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:12:27.899388    2563 scope.go:117] "RemoveContainer" containerID="137c03ac702e675fa0cc8b20a93bb01920de859b377c84458d762166aa70e0ea"
Jul 05 00:12:28.893410 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:12:28.893379    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-6b958749d4-wh9tw" event={"ID":"45610df8-3547-4c76-ac88-146c5c32ae8a","Type":"ContainerStarted","Data":"494862c5859bb63b4885ba33970f122800953c1b00aa40715569d862b615b288"}
Jul 05 00:12:29.897734 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:12:29.897702    2563 generic.go:356] "Generic (PLEG): container finished" podID="5ca7870b-a946-4574-8c66-e9d4bfda1406" containerID="81ff1cd13f428ea2ff009a88c0715a803660210e976e08dfc9fbaaf3e4b5e289" exitCode=1
Jul 05 00:12:29.898104 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:12:29.897778    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-6657fbc78f-qr4kz" event={"ID":"5ca7870b-a946-4574-8c66-e9d4bfda1406","Type":"ContainerDied","Data":"81ff1cd13f428ea2ff009a88c0715a803660210e976e08dfc9fbaaf3e4b5e289"}
Jul 05 00:12:29.898104 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:12:29.898098    2563 scope.go:117] "RemoveContainer" containerID="81ff1cd13f428ea2ff009a88c0715a803660210e976e08dfc9fbaaf3e4b5e289"
Jul 05 00:12:30.571443 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:12:30.571414    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-6657fbc78f-qr4kz"
Jul 05 00:12:30.902551 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:12:30.902464    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-6657fbc78f-qr4kz" event={"ID":"5ca7870b-a946-4574-8c66-e9d4bfda1406","Type":"ContainerStarted","Data":"f4ef0cb8cd588c3f271b8ca93ed71ff67cf05378ddacd5b4adfeeb2b6b8c714d"}
Jul 05 00:12:30.902993 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:12:30.902668    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-6657fbc78f-qr4kz"
Jul 05 00:12:30.903284 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:12:30.903270    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-6657fbc78f-qr4kz"
Jul 05 00:13:16.214622 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:13:16.214590    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-5884d96665-mcl4l"]
Jul 05 00:13:16.216765 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:13:16.216739    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-5884d96665-mcl4l"
Jul 05 00:13:16.218789 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:13:16.218769    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-private-configuration\""
Jul 05 00:13:16.218902 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:13:16.218802    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-tls\""
Jul 05 00:13:16.219077 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:13:16.219063    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"installation-pull-secrets\""
Jul 05 00:13:16.219278 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:13:16.219259    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-qmcqv\""
Jul 05 00:13:16.224521 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:13:16.224504    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"trusted-ca\""
Jul 05 00:13:16.232717 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:13:16.232698    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-5884d96665-mcl4l"]
Jul 05 00:13:16.283124 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:13:16.283095    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-58g7w\" (UniqueName: \"kubernetes.io/projected/63850164-7354-4352-a5f8-c3f68d980dfa-kube-api-access-58g7w\") pod \"image-registry-5884d96665-mcl4l\" (UID: \"63850164-7354-4352-a5f8-c3f68d980dfa\") " pod="openshift-image-registry/image-registry-5884d96665-mcl4l"
Jul 05 00:13:16.283283 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:13:16.283152    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/63850164-7354-4352-a5f8-c3f68d980dfa-ca-trust-extracted\") pod \"image-registry-5884d96665-mcl4l\" (UID: \"63850164-7354-4352-a5f8-c3f68d980dfa\") " pod="openshift-image-registry/image-registry-5884d96665-mcl4l"
Jul 05 00:13:16.283283 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:13:16.283180    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/63850164-7354-4352-a5f8-c3f68d980dfa-registry-certificates\") pod \"image-registry-5884d96665-mcl4l\" (UID: \"63850164-7354-4352-a5f8-c3f68d980dfa\") " pod="openshift-image-registry/image-registry-5884d96665-mcl4l"
Jul 05 00:13:16.283283 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:13:16.283203    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/63850164-7354-4352-a5f8-c3f68d980dfa-trusted-ca\") pod \"image-registry-5884d96665-mcl4l\" (UID: \"63850164-7354-4352-a5f8-c3f68d980dfa\") " pod="openshift-image-registry/image-registry-5884d96665-mcl4l"
Jul 05 00:13:16.283283 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:13:16.283260    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/63850164-7354-4352-a5f8-c3f68d980dfa-installation-pull-secrets\") pod \"image-registry-5884d96665-mcl4l\" (UID: \"63850164-7354-4352-a5f8-c3f68d980dfa\") " pod="openshift-image-registry/image-registry-5884d96665-mcl4l"
Jul 05 00:13:16.283489 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:13:16.283318    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/63850164-7354-4352-a5f8-c3f68d980dfa-image-registry-private-configuration\") pod \"image-registry-5884d96665-mcl4l\" (UID: \"63850164-7354-4352-a5f8-c3f68d980dfa\") " pod="openshift-image-registry/image-registry-5884d96665-mcl4l"
Jul 05 00:13:16.283489 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:13:16.283358    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/63850164-7354-4352-a5f8-c3f68d980dfa-registry-tls\") pod \"image-registry-5884d96665-mcl4l\" (UID: \"63850164-7354-4352-a5f8-c3f68d980dfa\") " pod="openshift-image-registry/image-registry-5884d96665-mcl4l"
Jul 05 00:13:16.283489 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:13:16.283407    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/63850164-7354-4352-a5f8-c3f68d980dfa-bound-sa-token\") pod \"image-registry-5884d96665-mcl4l\" (UID: \"63850164-7354-4352-a5f8-c3f68d980dfa\") " pod="openshift-image-registry/image-registry-5884d96665-mcl4l"
Jul 05 00:13:16.305525 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:13:16.305501    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-7rv4m"]
Jul 05 00:13:16.307504 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:13:16.307490    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-7rv4m"
Jul 05 00:13:16.309280 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:13:16.309261    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca-operator\"/\"serving-cert\""
Jul 05 00:13:16.309389 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:13:16.309352    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"service-ca-operator-config\""
Jul 05 00:13:16.309389 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:13:16.309383    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca-operator\"/\"service-ca-operator-dockercfg-hj6b7\""
Jul 05 00:13:16.309508 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:13:16.309494    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"openshift-service-ca.crt\""
Jul 05 00:13:16.309803 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:13:16.309790    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"kube-root-ca.crt\""
Jul 05 00:13:16.319444 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:13:16.319422    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-7rv4m"]
Jul 05 00:13:16.384151 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:13:16.384118    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/63850164-7354-4352-a5f8-c3f68d980dfa-image-registry-private-configuration\") pod \"image-registry-5884d96665-mcl4l\" (UID: \"63850164-7354-4352-a5f8-c3f68d980dfa\") " pod="openshift-image-registry/image-registry-5884d96665-mcl4l"
Jul 05 00:13:16.384151 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:13:16.384152    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/213b2b3a-180f-4560-b670-6bcf420899e8-config\") pod \"service-ca-operator-5d46d9ffd4-7rv4m\" (UID: \"213b2b3a-180f-4560-b670-6bcf420899e8\") " pod="openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-7rv4m"
Jul 05 00:13:16.384375 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:13:16.384177    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/63850164-7354-4352-a5f8-c3f68d980dfa-registry-tls\") pod \"image-registry-5884d96665-mcl4l\" (UID: \"63850164-7354-4352-a5f8-c3f68d980dfa\") " pod="openshift-image-registry/image-registry-5884d96665-mcl4l"
Jul 05 00:13:16.384375 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:13:16.384200    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/63850164-7354-4352-a5f8-c3f68d980dfa-bound-sa-token\") pod \"image-registry-5884d96665-mcl4l\" (UID: \"63850164-7354-4352-a5f8-c3f68d980dfa\") " pod="openshift-image-registry/image-registry-5884d96665-mcl4l"
Jul 05 00:13:16.384375 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:13:16.384295    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-58g7w\" (UniqueName: \"kubernetes.io/projected/63850164-7354-4352-a5f8-c3f68d980dfa-kube-api-access-58g7w\") pod \"image-registry-5884d96665-mcl4l\" (UID: \"63850164-7354-4352-a5f8-c3f68d980dfa\") " pod="openshift-image-registry/image-registry-5884d96665-mcl4l"
Jul 05 00:13:16.384375 ip-10-0-142-202 kubenswrapper[2563]: E0705 00:13:16.384303    2563 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 05 00:13:16.384375 ip-10-0-142-202 kubenswrapper[2563]: E0705 00:13:16.384331    2563 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-5884d96665-mcl4l: secret "image-registry-tls" not found
Jul 05 00:13:16.384549 ip-10-0-142-202 kubenswrapper[2563]: E0705 00:13:16.384387    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/63850164-7354-4352-a5f8-c3f68d980dfa-registry-tls podName:63850164-7354-4352-a5f8-c3f68d980dfa nodeName:}" failed. No retries permitted until 2026-07-05 00:13:16.884370486 +0000 UTC m=+245.226699524 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/63850164-7354-4352-a5f8-c3f68d980dfa-registry-tls") pod "image-registry-5884d96665-mcl4l" (UID: "63850164-7354-4352-a5f8-c3f68d980dfa") : secret "image-registry-tls" not found
Jul 05 00:13:16.384549 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:13:16.384410    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/213b2b3a-180f-4560-b670-6bcf420899e8-serving-cert\") pod \"service-ca-operator-5d46d9ffd4-7rv4m\" (UID: \"213b2b3a-180f-4560-b670-6bcf420899e8\") " pod="openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-7rv4m"
Jul 05 00:13:16.384549 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:13:16.384459    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/63850164-7354-4352-a5f8-c3f68d980dfa-ca-trust-extracted\") pod \"image-registry-5884d96665-mcl4l\" (UID: \"63850164-7354-4352-a5f8-c3f68d980dfa\") " pod="openshift-image-registry/image-registry-5884d96665-mcl4l"
Jul 05 00:13:16.384549 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:13:16.384476    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-m89hn\" (UniqueName: \"kubernetes.io/projected/213b2b3a-180f-4560-b670-6bcf420899e8-kube-api-access-m89hn\") pod \"service-ca-operator-5d46d9ffd4-7rv4m\" (UID: \"213b2b3a-180f-4560-b670-6bcf420899e8\") " pod="openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-7rv4m"
Jul 05 00:13:16.384549 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:13:16.384498    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/63850164-7354-4352-a5f8-c3f68d980dfa-registry-certificates\") pod \"image-registry-5884d96665-mcl4l\" (UID: \"63850164-7354-4352-a5f8-c3f68d980dfa\") " pod="openshift-image-registry/image-registry-5884d96665-mcl4l"
Jul 05 00:13:16.384859 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:13:16.384613    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/63850164-7354-4352-a5f8-c3f68d980dfa-trusted-ca\") pod \"image-registry-5884d96665-mcl4l\" (UID: \"63850164-7354-4352-a5f8-c3f68d980dfa\") " pod="openshift-image-registry/image-registry-5884d96665-mcl4l"
Jul 05 00:13:16.384859 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:13:16.384645    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/63850164-7354-4352-a5f8-c3f68d980dfa-installation-pull-secrets\") pod \"image-registry-5884d96665-mcl4l\" (UID: \"63850164-7354-4352-a5f8-c3f68d980dfa\") " pod="openshift-image-registry/image-registry-5884d96665-mcl4l"
Jul 05 00:13:16.384859 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:13:16.384818    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/63850164-7354-4352-a5f8-c3f68d980dfa-ca-trust-extracted\") pod \"image-registry-5884d96665-mcl4l\" (UID: \"63850164-7354-4352-a5f8-c3f68d980dfa\") " pod="openshift-image-registry/image-registry-5884d96665-mcl4l"
Jul 05 00:13:16.385135 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:13:16.385112    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/63850164-7354-4352-a5f8-c3f68d980dfa-registry-certificates\") pod \"image-registry-5884d96665-mcl4l\" (UID: \"63850164-7354-4352-a5f8-c3f68d980dfa\") " pod="openshift-image-registry/image-registry-5884d96665-mcl4l"
Jul 05 00:13:16.385558 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:13:16.385536    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/63850164-7354-4352-a5f8-c3f68d980dfa-trusted-ca\") pod \"image-registry-5884d96665-mcl4l\" (UID: \"63850164-7354-4352-a5f8-c3f68d980dfa\") " pod="openshift-image-registry/image-registry-5884d96665-mcl4l"
Jul 05 00:13:16.386806 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:13:16.386786    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/63850164-7354-4352-a5f8-c3f68d980dfa-image-registry-private-configuration\") pod \"image-registry-5884d96665-mcl4l\" (UID: \"63850164-7354-4352-a5f8-c3f68d980dfa\") " pod="openshift-image-registry/image-registry-5884d96665-mcl4l"
Jul 05 00:13:16.386898 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:13:16.386884    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/63850164-7354-4352-a5f8-c3f68d980dfa-installation-pull-secrets\") pod \"image-registry-5884d96665-mcl4l\" (UID: \"63850164-7354-4352-a5f8-c3f68d980dfa\") " pod="openshift-image-registry/image-registry-5884d96665-mcl4l"
Jul 05 00:13:16.392730 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:13:16.392711    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/63850164-7354-4352-a5f8-c3f68d980dfa-bound-sa-token\") pod \"image-registry-5884d96665-mcl4l\" (UID: \"63850164-7354-4352-a5f8-c3f68d980dfa\") " pod="openshift-image-registry/image-registry-5884d96665-mcl4l"
Jul 05 00:13:16.393810 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:13:16.393790    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-58g7w\" (UniqueName: \"kubernetes.io/projected/63850164-7354-4352-a5f8-c3f68d980dfa-kube-api-access-58g7w\") pod \"image-registry-5884d96665-mcl4l\" (UID: \"63850164-7354-4352-a5f8-c3f68d980dfa\") " pod="openshift-image-registry/image-registry-5884d96665-mcl4l"
Jul 05 00:13:16.485679 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:13:16.485591    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/213b2b3a-180f-4560-b670-6bcf420899e8-serving-cert\") pod \"service-ca-operator-5d46d9ffd4-7rv4m\" (UID: \"213b2b3a-180f-4560-b670-6bcf420899e8\") " pod="openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-7rv4m"
Jul 05 00:13:16.485679 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:13:16.485644    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-m89hn\" (UniqueName: \"kubernetes.io/projected/213b2b3a-180f-4560-b670-6bcf420899e8-kube-api-access-m89hn\") pod \"service-ca-operator-5d46d9ffd4-7rv4m\" (UID: \"213b2b3a-180f-4560-b670-6bcf420899e8\") " pod="openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-7rv4m"
Jul 05 00:13:16.485679 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:13:16.485680    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/213b2b3a-180f-4560-b670-6bcf420899e8-config\") pod \"service-ca-operator-5d46d9ffd4-7rv4m\" (UID: \"213b2b3a-180f-4560-b670-6bcf420899e8\") " pod="openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-7rv4m"
Jul 05 00:13:16.486148 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:13:16.486129    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/213b2b3a-180f-4560-b670-6bcf420899e8-config\") pod \"service-ca-operator-5d46d9ffd4-7rv4m\" (UID: \"213b2b3a-180f-4560-b670-6bcf420899e8\") " pod="openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-7rv4m"
Jul 05 00:13:16.487880 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:13:16.487855    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/213b2b3a-180f-4560-b670-6bcf420899e8-serving-cert\") pod \"service-ca-operator-5d46d9ffd4-7rv4m\" (UID: \"213b2b3a-180f-4560-b670-6bcf420899e8\") " pod="openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-7rv4m"
Jul 05 00:13:16.492653 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:13:16.492633    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-m89hn\" (UniqueName: \"kubernetes.io/projected/213b2b3a-180f-4560-b670-6bcf420899e8-kube-api-access-m89hn\") pod \"service-ca-operator-5d46d9ffd4-7rv4m\" (UID: \"213b2b3a-180f-4560-b670-6bcf420899e8\") " pod="openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-7rv4m"
Jul 05 00:13:16.616025 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:13:16.615979    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-7rv4m"
Jul 05 00:13:16.730262 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:13:16.730169    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-7rv4m"]
Jul 05 00:13:16.733951 ip-10-0-142-202 kubenswrapper[2563]: W0705 00:13:16.733925    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod213b2b3a_180f_4560_b670_6bcf420899e8.slice/crio-d0f5c043804c9c2c2b12593fbffb643d0b59e556635e9f0073eefe4370d65394 WatchSource:0}: Error finding container d0f5c043804c9c2c2b12593fbffb643d0b59e556635e9f0073eefe4370d65394: Status 404 returned error can't find the container with id d0f5c043804c9c2c2b12593fbffb643d0b59e556635e9f0073eefe4370d65394
Jul 05 00:13:16.890007 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:13:16.889968    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/63850164-7354-4352-a5f8-c3f68d980dfa-registry-tls\") pod \"image-registry-5884d96665-mcl4l\" (UID: \"63850164-7354-4352-a5f8-c3f68d980dfa\") " pod="openshift-image-registry/image-registry-5884d96665-mcl4l"
Jul 05 00:13:16.890168 ip-10-0-142-202 kubenswrapper[2563]: E0705 00:13:16.890078    2563 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 05 00:13:16.890168 ip-10-0-142-202 kubenswrapper[2563]: E0705 00:13:16.890089    2563 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-5884d96665-mcl4l: secret "image-registry-tls" not found
Jul 05 00:13:16.890168 ip-10-0-142-202 kubenswrapper[2563]: E0705 00:13:16.890136    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/63850164-7354-4352-a5f8-c3f68d980dfa-registry-tls podName:63850164-7354-4352-a5f8-c3f68d980dfa nodeName:}" failed. No retries permitted until 2026-07-05 00:13:17.890123387 +0000 UTC m=+246.232452426 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/63850164-7354-4352-a5f8-c3f68d980dfa-registry-tls") pod "image-registry-5884d96665-mcl4l" (UID: "63850164-7354-4352-a5f8-c3f68d980dfa") : secret "image-registry-tls" not found
Jul 05 00:13:17.032418 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:13:17.032388    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-7rv4m" event={"ID":"213b2b3a-180f-4560-b670-6bcf420899e8","Type":"ContainerStarted","Data":"d0f5c043804c9c2c2b12593fbffb643d0b59e556635e9f0073eefe4370d65394"}
Jul 05 00:13:17.898735 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:13:17.898696    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/63850164-7354-4352-a5f8-c3f68d980dfa-registry-tls\") pod \"image-registry-5884d96665-mcl4l\" (UID: \"63850164-7354-4352-a5f8-c3f68d980dfa\") " pod="openshift-image-registry/image-registry-5884d96665-mcl4l"
Jul 05 00:13:17.899138 ip-10-0-142-202 kubenswrapper[2563]: E0705 00:13:17.898829    2563 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 05 00:13:17.899138 ip-10-0-142-202 kubenswrapper[2563]: E0705 00:13:17.898853    2563 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-5884d96665-mcl4l: secret "image-registry-tls" not found
Jul 05 00:13:17.899138 ip-10-0-142-202 kubenswrapper[2563]: E0705 00:13:17.898922    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/63850164-7354-4352-a5f8-c3f68d980dfa-registry-tls podName:63850164-7354-4352-a5f8-c3f68d980dfa nodeName:}" failed. No retries permitted until 2026-07-05 00:13:19.898901023 +0000 UTC m=+248.241230065 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/63850164-7354-4352-a5f8-c3f68d980dfa-registry-tls") pod "image-registry-5884d96665-mcl4l" (UID: "63850164-7354-4352-a5f8-c3f68d980dfa") : secret "image-registry-tls" not found
Jul 05 00:13:19.039860 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:13:19.039822    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-7rv4m" event={"ID":"213b2b3a-180f-4560-b670-6bcf420899e8","Type":"ContainerStarted","Data":"caeb50a659c9f23083667f2fef77b8fac2bf7280afdeb05081f891a912c8a447"}
Jul 05 00:13:19.053382 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:13:19.053336    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-7rv4m" podStartSLOduration=0.973362747 podStartE2EDuration="3.053320702s" totalImagesPullingTime="2.079957955s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-05 00:13:16 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-05 00:13:19.052612989 +0000 UTC m=+247.394942044" watchObservedRunningTime="2026-07-05 00:13:19.053320702 +0000 UTC m=+247.395649798"
Jul 05 00:13:19.911357 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:13:19.911323    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/63850164-7354-4352-a5f8-c3f68d980dfa-registry-tls\") pod \"image-registry-5884d96665-mcl4l\" (UID: \"63850164-7354-4352-a5f8-c3f68d980dfa\") " pod="openshift-image-registry/image-registry-5884d96665-mcl4l"
Jul 05 00:13:19.911532 ip-10-0-142-202 kubenswrapper[2563]: E0705 00:13:19.911470    2563 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 05 00:13:19.911532 ip-10-0-142-202 kubenswrapper[2563]: E0705 00:13:19.911491    2563 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-5884d96665-mcl4l: secret "image-registry-tls" not found
Jul 05 00:13:19.911601 ip-10-0-142-202 kubenswrapper[2563]: E0705 00:13:19.911546    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/63850164-7354-4352-a5f8-c3f68d980dfa-registry-tls podName:63850164-7354-4352-a5f8-c3f68d980dfa nodeName:}" failed. No retries permitted until 2026-07-05 00:13:23.9115309 +0000 UTC m=+252.253859938 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/63850164-7354-4352-a5f8-c3f68d980dfa-registry-tls") pod "image-registry-5884d96665-mcl4l" (UID: "63850164-7354-4352-a5f8-c3f68d980dfa") : secret "image-registry-tls" not found
Jul 05 00:13:20.920155 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:13:20.920126    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-snrdh"]
Jul 05 00:13:20.922118 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:13:20.922102    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-snrdh"
Jul 05 00:13:20.925626 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:13:20.925599    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\""
Jul 05 00:13:20.925758 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:13:20.925599    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\""
Jul 05 00:13:20.925993 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:13:20.925979    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\""
Jul 05 00:13:20.926035 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:13:20.925990    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-m5c25\""
Jul 05 00:13:20.926317 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:13:20.926286    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\""
Jul 05 00:13:20.949840 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:13:20.949819    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-snrdh"]
Jul 05 00:13:21.019976 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:13:21.019949    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/7f6da6e5-bbfb-48e4-bb8a-b471c187e3ee-crio-socket\") pod \"insights-runtime-extractor-snrdh\" (UID: \"7f6da6e5-bbfb-48e4-bb8a-b471c187e3ee\") " pod="openshift-insights/insights-runtime-extractor-snrdh"
Jul 05 00:13:21.020121 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:13:21.019994    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/7f6da6e5-bbfb-48e4-bb8a-b471c187e3ee-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-snrdh\" (UID: \"7f6da6e5-bbfb-48e4-bb8a-b471c187e3ee\") " pod="openshift-insights/insights-runtime-extractor-snrdh"
Jul 05 00:13:21.020121 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:13:21.020025    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/7f6da6e5-bbfb-48e4-bb8a-b471c187e3ee-data-volume\") pod \"insights-runtime-extractor-snrdh\" (UID: \"7f6da6e5-bbfb-48e4-bb8a-b471c187e3ee\") " pod="openshift-insights/insights-runtime-extractor-snrdh"
Jul 05 00:13:21.020121 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:13:21.020042    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/7f6da6e5-bbfb-48e4-bb8a-b471c187e3ee-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-snrdh\" (UID: \"7f6da6e5-bbfb-48e4-bb8a-b471c187e3ee\") " pod="openshift-insights/insights-runtime-extractor-snrdh"
Jul 05 00:13:21.020121 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:13:21.020105    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tsvk6\" (UniqueName: \"kubernetes.io/projected/7f6da6e5-bbfb-48e4-bb8a-b471c187e3ee-kube-api-access-tsvk6\") pod \"insights-runtime-extractor-snrdh\" (UID: \"7f6da6e5-bbfb-48e4-bb8a-b471c187e3ee\") " pod="openshift-insights/insights-runtime-extractor-snrdh"
Jul 05 00:13:21.121374 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:13:21.121339    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tsvk6\" (UniqueName: \"kubernetes.io/projected/7f6da6e5-bbfb-48e4-bb8a-b471c187e3ee-kube-api-access-tsvk6\") pod \"insights-runtime-extractor-snrdh\" (UID: \"7f6da6e5-bbfb-48e4-bb8a-b471c187e3ee\") " pod="openshift-insights/insights-runtime-extractor-snrdh"
Jul 05 00:13:21.121542 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:13:21.121385    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/7f6da6e5-bbfb-48e4-bb8a-b471c187e3ee-crio-socket\") pod \"insights-runtime-extractor-snrdh\" (UID: \"7f6da6e5-bbfb-48e4-bb8a-b471c187e3ee\") " pod="openshift-insights/insights-runtime-extractor-snrdh"
Jul 05 00:13:21.121542 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:13:21.121455    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/7f6da6e5-bbfb-48e4-bb8a-b471c187e3ee-crio-socket\") pod \"insights-runtime-extractor-snrdh\" (UID: \"7f6da6e5-bbfb-48e4-bb8a-b471c187e3ee\") " pod="openshift-insights/insights-runtime-extractor-snrdh"
Jul 05 00:13:21.121542 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:13:21.121498    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/7f6da6e5-bbfb-48e4-bb8a-b471c187e3ee-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-snrdh\" (UID: \"7f6da6e5-bbfb-48e4-bb8a-b471c187e3ee\") " pod="openshift-insights/insights-runtime-extractor-snrdh"
Jul 05 00:13:21.121700 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:13:21.121547    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/7f6da6e5-bbfb-48e4-bb8a-b471c187e3ee-data-volume\") pod \"insights-runtime-extractor-snrdh\" (UID: \"7f6da6e5-bbfb-48e4-bb8a-b471c187e3ee\") " pod="openshift-insights/insights-runtime-extractor-snrdh"
Jul 05 00:13:21.121700 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:13:21.121566    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/7f6da6e5-bbfb-48e4-bb8a-b471c187e3ee-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-snrdh\" (UID: \"7f6da6e5-bbfb-48e4-bb8a-b471c187e3ee\") " pod="openshift-insights/insights-runtime-extractor-snrdh"
Jul 05 00:13:21.121700 ip-10-0-142-202 kubenswrapper[2563]: E0705 00:13:21.121637    2563 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 05 00:13:21.121700 ip-10-0-142-202 kubenswrapper[2563]: E0705 00:13:21.121675    2563 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 05 00:13:21.121877 ip-10-0-142-202 kubenswrapper[2563]: E0705 00:13:21.121704    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/7f6da6e5-bbfb-48e4-bb8a-b471c187e3ee-kube-rbac-proxy-cm podName:7f6da6e5-bbfb-48e4-bb8a-b471c187e3ee nodeName:}" failed. No retries permitted until 2026-07-05 00:13:21.621689943 +0000 UTC m=+249.964018981 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/7f6da6e5-bbfb-48e4-bb8a-b471c187e3ee-kube-rbac-proxy-cm") pod "insights-runtime-extractor-snrdh" (UID: "7f6da6e5-bbfb-48e4-bb8a-b471c187e3ee") : configmap "kube-rbac-proxy" not found
Jul 05 00:13:21.121877 ip-10-0-142-202 kubenswrapper[2563]: E0705 00:13:21.121735    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7f6da6e5-bbfb-48e4-bb8a-b471c187e3ee-insights-runtime-extractor-tls podName:7f6da6e5-bbfb-48e4-bb8a-b471c187e3ee nodeName:}" failed. No retries permitted until 2026-07-05 00:13:21.621720948 +0000 UTC m=+249.964049986 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/7f6da6e5-bbfb-48e4-bb8a-b471c187e3ee-insights-runtime-extractor-tls") pod "insights-runtime-extractor-snrdh" (UID: "7f6da6e5-bbfb-48e4-bb8a-b471c187e3ee") : secret "insights-runtime-extractor-tls" not found
Jul 05 00:13:21.121976 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:13:21.121910    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/7f6da6e5-bbfb-48e4-bb8a-b471c187e3ee-data-volume\") pod \"insights-runtime-extractor-snrdh\" (UID: \"7f6da6e5-bbfb-48e4-bb8a-b471c187e3ee\") " pod="openshift-insights/insights-runtime-extractor-snrdh"
Jul 05 00:13:21.133697 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:13:21.133670    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-tsvk6\" (UniqueName: \"kubernetes.io/projected/7f6da6e5-bbfb-48e4-bb8a-b471c187e3ee-kube-api-access-tsvk6\") pod \"insights-runtime-extractor-snrdh\" (UID: \"7f6da6e5-bbfb-48e4-bb8a-b471c187e3ee\") " pod="openshift-insights/insights-runtime-extractor-snrdh"
Jul 05 00:13:21.173987 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:13:21.173929    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-kube-storage-version-migrator/migrator-d57d58cc9-9qgkt"]
Jul 05 00:13:21.175864 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:13:21.175851    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator/migrator-d57d58cc9-9qgkt"
Jul 05 00:13:21.177767 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:13:21.177746    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator\"/\"openshift-service-ca.crt\""
Jul 05 00:13:21.177853 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:13:21.177746    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator\"/\"kube-storage-version-migrator-sa-dockercfg-bkrl8\""
Jul 05 00:13:21.177853 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:13:21.177749    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator\"/\"kube-root-ca.crt\""
Jul 05 00:13:21.183835 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:13:21.183816    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator/migrator-d57d58cc9-9qgkt"]
Jul 05 00:13:21.322645 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:13:21.322615    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hgg2t\" (UniqueName: \"kubernetes.io/projected/b0d434b5-78a3-4a5e-bf15-082da796dfdf-kube-api-access-hgg2t\") pod \"migrator-d57d58cc9-9qgkt\" (UID: \"b0d434b5-78a3-4a5e-bf15-082da796dfdf\") " pod="openshift-kube-storage-version-migrator/migrator-d57d58cc9-9qgkt"
Jul 05 00:13:21.423135 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:13:21.423109    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hgg2t\" (UniqueName: \"kubernetes.io/projected/b0d434b5-78a3-4a5e-bf15-082da796dfdf-kube-api-access-hgg2t\") pod \"migrator-d57d58cc9-9qgkt\" (UID: \"b0d434b5-78a3-4a5e-bf15-082da796dfdf\") " pod="openshift-kube-storage-version-migrator/migrator-d57d58cc9-9qgkt"
Jul 05 00:13:21.429840 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:13:21.429777    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-hgg2t\" (UniqueName: \"kubernetes.io/projected/b0d434b5-78a3-4a5e-bf15-082da796dfdf-kube-api-access-hgg2t\") pod \"migrator-d57d58cc9-9qgkt\" (UID: \"b0d434b5-78a3-4a5e-bf15-082da796dfdf\") " pod="openshift-kube-storage-version-migrator/migrator-d57d58cc9-9qgkt"
Jul 05 00:13:21.484742 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:13:21.484711    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator/migrator-d57d58cc9-9qgkt"
Jul 05 00:13:21.607979 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:13:21.607951    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator/migrator-d57d58cc9-9qgkt"]
Jul 05 00:13:21.612111 ip-10-0-142-202 kubenswrapper[2563]: W0705 00:13:21.612083    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb0d434b5_78a3_4a5e_bf15_082da796dfdf.slice/crio-114234736c127bc0c368ccca7c713ee01d1bea255557162e84106586d8bce13e WatchSource:0}: Error finding container 114234736c127bc0c368ccca7c713ee01d1bea255557162e84106586d8bce13e: Status 404 returned error can't find the container with id 114234736c127bc0c368ccca7c713ee01d1bea255557162e84106586d8bce13e
Jul 05 00:13:21.624988 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:13:21.624966    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/7f6da6e5-bbfb-48e4-bb8a-b471c187e3ee-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-snrdh\" (UID: \"7f6da6e5-bbfb-48e4-bb8a-b471c187e3ee\") " pod="openshift-insights/insights-runtime-extractor-snrdh"
Jul 05 00:13:21.625085 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:13:21.625022    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/7f6da6e5-bbfb-48e4-bb8a-b471c187e3ee-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-snrdh\" (UID: \"7f6da6e5-bbfb-48e4-bb8a-b471c187e3ee\") " pod="openshift-insights/insights-runtime-extractor-snrdh"
Jul 05 00:13:21.625148 ip-10-0-142-202 kubenswrapper[2563]: E0705 00:13:21.625082    2563 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 05 00:13:21.625148 ip-10-0-142-202 kubenswrapper[2563]: E0705 00:13:21.625146    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/7f6da6e5-bbfb-48e4-bb8a-b471c187e3ee-kube-rbac-proxy-cm podName:7f6da6e5-bbfb-48e4-bb8a-b471c187e3ee nodeName:}" failed. No retries permitted until 2026-07-05 00:13:22.625130663 +0000 UTC m=+250.967459703 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/7f6da6e5-bbfb-48e4-bb8a-b471c187e3ee-kube-rbac-proxy-cm") pod "insights-runtime-extractor-snrdh" (UID: "7f6da6e5-bbfb-48e4-bb8a-b471c187e3ee") : configmap "kube-rbac-proxy" not found
Jul 05 00:13:21.625272 ip-10-0-142-202 kubenswrapper[2563]: E0705 00:13:21.625147    2563 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 05 00:13:21.625272 ip-10-0-142-202 kubenswrapper[2563]: E0705 00:13:21.625197    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7f6da6e5-bbfb-48e4-bb8a-b471c187e3ee-insights-runtime-extractor-tls podName:7f6da6e5-bbfb-48e4-bb8a-b471c187e3ee nodeName:}" failed. No retries permitted until 2026-07-05 00:13:22.625181203 +0000 UTC m=+250.967510264 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/7f6da6e5-bbfb-48e4-bb8a-b471c187e3ee-insights-runtime-extractor-tls") pod "insights-runtime-extractor-snrdh" (UID: "7f6da6e5-bbfb-48e4-bb8a-b471c187e3ee") : secret "insights-runtime-extractor-tls" not found
Jul 05 00:13:22.050257 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:13:22.050205    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-d57d58cc9-9qgkt" event={"ID":"b0d434b5-78a3-4a5e-bf15-082da796dfdf","Type":"ContainerStarted","Data":"114234736c127bc0c368ccca7c713ee01d1bea255557162e84106586d8bce13e"}
Jul 05 00:13:22.632711 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:13:22.632680    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/7f6da6e5-bbfb-48e4-bb8a-b471c187e3ee-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-snrdh\" (UID: \"7f6da6e5-bbfb-48e4-bb8a-b471c187e3ee\") " pod="openshift-insights/insights-runtime-extractor-snrdh"
Jul 05 00:13:22.632887 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:13:22.632751    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/7f6da6e5-bbfb-48e4-bb8a-b471c187e3ee-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-snrdh\" (UID: \"7f6da6e5-bbfb-48e4-bb8a-b471c187e3ee\") " pod="openshift-insights/insights-runtime-extractor-snrdh"
Jul 05 00:13:22.632887 ip-10-0-142-202 kubenswrapper[2563]: E0705 00:13:22.632836    2563 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 05 00:13:22.632887 ip-10-0-142-202 kubenswrapper[2563]: E0705 00:13:22.632844    2563 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 05 00:13:22.632988 ip-10-0-142-202 kubenswrapper[2563]: E0705 00:13:22.632890    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/7f6da6e5-bbfb-48e4-bb8a-b471c187e3ee-kube-rbac-proxy-cm podName:7f6da6e5-bbfb-48e4-bb8a-b471c187e3ee nodeName:}" failed. No retries permitted until 2026-07-05 00:13:24.632876266 +0000 UTC m=+252.975205303 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/7f6da6e5-bbfb-48e4-bb8a-b471c187e3ee-kube-rbac-proxy-cm") pod "insights-runtime-extractor-snrdh" (UID: "7f6da6e5-bbfb-48e4-bb8a-b471c187e3ee") : configmap "kube-rbac-proxy" not found
Jul 05 00:13:22.632988 ip-10-0-142-202 kubenswrapper[2563]: E0705 00:13:22.632903    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7f6da6e5-bbfb-48e4-bb8a-b471c187e3ee-insights-runtime-extractor-tls podName:7f6da6e5-bbfb-48e4-bb8a-b471c187e3ee nodeName:}" failed. No retries permitted until 2026-07-05 00:13:24.632897289 +0000 UTC m=+252.975226327 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/7f6da6e5-bbfb-48e4-bb8a-b471c187e3ee-insights-runtime-extractor-tls") pod "insights-runtime-extractor-snrdh" (UID: "7f6da6e5-bbfb-48e4-bb8a-b471c187e3ee") : secret "insights-runtime-extractor-tls" not found
Jul 05 00:13:22.893584 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:13:22.893564    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-kjz9w_cecee17b-8d41-4a16-a3b2-53919db096bb/dns-node-resolver/0.log"
Jul 05 00:13:22.936897 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:13:22.936864    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c085d4be-fcda-4931-a018-02cbaca17c59-metrics-certs\") pod \"network-metrics-daemon-5fn94\" (UID: \"c085d4be-fcda-4931-a018-02cbaca17c59\") " pod="openshift-multus/network-metrics-daemon-5fn94"
Jul 05 00:13:22.937024 ip-10-0-142-202 kubenswrapper[2563]: E0705 00:13:22.937007    2563 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found
Jul 05 00:13:22.937070 ip-10-0-142-202 kubenswrapper[2563]: E0705 00:13:22.937061    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c085d4be-fcda-4931-a018-02cbaca17c59-metrics-certs podName:c085d4be-fcda-4931-a018-02cbaca17c59 nodeName:}" failed. No retries permitted until 2026-07-05 00:15:24.937046378 +0000 UTC m=+373.279375416 (durationBeforeRetry 2m2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/c085d4be-fcda-4931-a018-02cbaca17c59-metrics-certs") pod "network-metrics-daemon-5fn94" (UID: "c085d4be-fcda-4931-a018-02cbaca17c59") : secret "metrics-daemon-secret" not found
Jul 05 00:13:23.054353 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:13:23.054323    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-d57d58cc9-9qgkt" event={"ID":"b0d434b5-78a3-4a5e-bf15-082da796dfdf","Type":"ContainerStarted","Data":"a5b87dcce52b33bbe45cad7c853f37c4119c7981ffeea1e426e7019b720146a3"}
Jul 05 00:13:23.054691 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:13:23.054374    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-d57d58cc9-9qgkt" event={"ID":"b0d434b5-78a3-4a5e-bf15-082da796dfdf","Type":"ContainerStarted","Data":"c26d3d05d17215eb2b908e7c67e3ebbe65d66bf7f4fdf9c3abfaeb9e986e24b2"}
Jul 05 00:13:23.073783 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:13:23.073736    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-kube-storage-version-migrator/migrator-d57d58cc9-9qgkt" podStartSLOduration=0.809116286 podStartE2EDuration="2.07372048s" totalImagesPullingTime="1.264604194s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-05 00:13:21 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-05 00:13:23.072233398 +0000 UTC m=+251.414562467" watchObservedRunningTime="2026-07-05 00:13:23.07372048 +0000 UTC m=+251.416049582"
Jul 05 00:13:23.944211 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:13:23.944174    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/63850164-7354-4352-a5f8-c3f68d980dfa-registry-tls\") pod \"image-registry-5884d96665-mcl4l\" (UID: \"63850164-7354-4352-a5f8-c3f68d980dfa\") " pod="openshift-image-registry/image-registry-5884d96665-mcl4l"
Jul 05 00:13:23.944392 ip-10-0-142-202 kubenswrapper[2563]: E0705 00:13:23.944320    2563 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 05 00:13:23.944392 ip-10-0-142-202 kubenswrapper[2563]: E0705 00:13:23.944336    2563 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-5884d96665-mcl4l: secret "image-registry-tls" not found
Jul 05 00:13:23.944392 ip-10-0-142-202 kubenswrapper[2563]: E0705 00:13:23.944389    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/63850164-7354-4352-a5f8-c3f68d980dfa-registry-tls podName:63850164-7354-4352-a5f8-c3f68d980dfa nodeName:}" failed. No retries permitted until 2026-07-05 00:13:31.944375227 +0000 UTC m=+260.286704265 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/63850164-7354-4352-a5f8-c3f68d980dfa-registry-tls") pod "image-registry-5884d96665-mcl4l" (UID: "63850164-7354-4352-a5f8-c3f68d980dfa") : secret "image-registry-tls" not found
Jul 05 00:13:24.093986 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:13:24.093957    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-s547v_d3a9b78d-cf60-4acc-a3c5-016ee34903f8/node-ca/0.log"
Jul 05 00:13:24.649120 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:13:24.649082    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/7f6da6e5-bbfb-48e4-bb8a-b471c187e3ee-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-snrdh\" (UID: \"7f6da6e5-bbfb-48e4-bb8a-b471c187e3ee\") " pod="openshift-insights/insights-runtime-extractor-snrdh"
Jul 05 00:13:24.649335 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:13:24.649149    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/7f6da6e5-bbfb-48e4-bb8a-b471c187e3ee-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-snrdh\" (UID: \"7f6da6e5-bbfb-48e4-bb8a-b471c187e3ee\") " pod="openshift-insights/insights-runtime-extractor-snrdh"
Jul 05 00:13:24.649335 ip-10-0-142-202 kubenswrapper[2563]: E0705 00:13:24.649214    2563 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 05 00:13:24.649335 ip-10-0-142-202 kubenswrapper[2563]: E0705 00:13:24.649275    2563 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 05 00:13:24.649335 ip-10-0-142-202 kubenswrapper[2563]: E0705 00:13:24.649315    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/7f6da6e5-bbfb-48e4-bb8a-b471c187e3ee-kube-rbac-proxy-cm podName:7f6da6e5-bbfb-48e4-bb8a-b471c187e3ee nodeName:}" failed. No retries permitted until 2026-07-05 00:13:28.649298619 +0000 UTC m=+256.991627657 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/7f6da6e5-bbfb-48e4-bb8a-b471c187e3ee-kube-rbac-proxy-cm") pod "insights-runtime-extractor-snrdh" (UID: "7f6da6e5-bbfb-48e4-bb8a-b471c187e3ee") : configmap "kube-rbac-proxy" not found
Jul 05 00:13:24.649335 ip-10-0-142-202 kubenswrapper[2563]: E0705 00:13:24.649335    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7f6da6e5-bbfb-48e4-bb8a-b471c187e3ee-insights-runtime-extractor-tls podName:7f6da6e5-bbfb-48e4-bb8a-b471c187e3ee nodeName:}" failed. No retries permitted until 2026-07-05 00:13:28.649321037 +0000 UTC m=+256.991650076 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/7f6da6e5-bbfb-48e4-bb8a-b471c187e3ee-insights-runtime-extractor-tls") pod "insights-runtime-extractor-snrdh" (UID: "7f6da6e5-bbfb-48e4-bb8a-b471c187e3ee") : secret "insights-runtime-extractor-tls" not found
Jul 05 00:13:24.894644 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:13:24.894614    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-xqpm2_63e680ec-84d5-4c0a-b259-84c64914aab7/serve-healthcheck-canary/0.log"
Jul 05 00:13:28.679445 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:13:28.679408    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/7f6da6e5-bbfb-48e4-bb8a-b471c187e3ee-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-snrdh\" (UID: \"7f6da6e5-bbfb-48e4-bb8a-b471c187e3ee\") " pod="openshift-insights/insights-runtime-extractor-snrdh"
Jul 05 00:13:28.679985 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:13:28.679462    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/7f6da6e5-bbfb-48e4-bb8a-b471c187e3ee-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-snrdh\" (UID: \"7f6da6e5-bbfb-48e4-bb8a-b471c187e3ee\") " pod="openshift-insights/insights-runtime-extractor-snrdh"
Jul 05 00:13:28.679985 ip-10-0-142-202 kubenswrapper[2563]: E0705 00:13:28.679577    2563 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 05 00:13:28.679985 ip-10-0-142-202 kubenswrapper[2563]: E0705 00:13:28.679630    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7f6da6e5-bbfb-48e4-bb8a-b471c187e3ee-insights-runtime-extractor-tls podName:7f6da6e5-bbfb-48e4-bb8a-b471c187e3ee nodeName:}" failed. No retries permitted until 2026-07-05 00:13:36.679616424 +0000 UTC m=+265.021945463 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/7f6da6e5-bbfb-48e4-bb8a-b471c187e3ee-insights-runtime-extractor-tls") pod "insights-runtime-extractor-snrdh" (UID: "7f6da6e5-bbfb-48e4-bb8a-b471c187e3ee") : secret "insights-runtime-extractor-tls" not found
Jul 05 00:13:28.679985 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:13:28.679902    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/7f6da6e5-bbfb-48e4-bb8a-b471c187e3ee-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-snrdh\" (UID: \"7f6da6e5-bbfb-48e4-bb8a-b471c187e3ee\") " pod="openshift-insights/insights-runtime-extractor-snrdh"
Jul 05 00:13:32.005351 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:13:32.005308    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/63850164-7354-4352-a5f8-c3f68d980dfa-registry-tls\") pod \"image-registry-5884d96665-mcl4l\" (UID: \"63850164-7354-4352-a5f8-c3f68d980dfa\") " pod="openshift-image-registry/image-registry-5884d96665-mcl4l"
Jul 05 00:13:32.007669 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:13:32.007646    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/63850164-7354-4352-a5f8-c3f68d980dfa-registry-tls\") pod \"image-registry-5884d96665-mcl4l\" (UID: \"63850164-7354-4352-a5f8-c3f68d980dfa\") " pod="openshift-image-registry/image-registry-5884d96665-mcl4l"
Jul 05 00:13:32.126548 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:13:32.126515    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-5884d96665-mcl4l"
Jul 05 00:13:32.252735 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:13:32.252708    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-5884d96665-mcl4l"]
Jul 05 00:13:32.256883 ip-10-0-142-202 kubenswrapper[2563]: W0705 00:13:32.256830    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod63850164_7354_4352_a5f8_c3f68d980dfa.slice/crio-a825ba259aa97b37e60045cad4b6ffb242ea2c7b604a0867bc70bcbf6e760b6b WatchSource:0}: Error finding container a825ba259aa97b37e60045cad4b6ffb242ea2c7b604a0867bc70bcbf6e760b6b: Status 404 returned error can't find the container with id a825ba259aa97b37e60045cad4b6ffb242ea2c7b604a0867bc70bcbf6e760b6b
Jul 05 00:13:33.085106 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:13:33.085073    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-5884d96665-mcl4l" event={"ID":"63850164-7354-4352-a5f8-c3f68d980dfa","Type":"ContainerStarted","Data":"434072f1ea5bdf1b156b820dd87baaa59f55f97554a463d7846aa7d325ffb080"}
Jul 05 00:13:33.085580 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:13:33.085121    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-5884d96665-mcl4l" event={"ID":"63850164-7354-4352-a5f8-c3f68d980dfa","Type":"ContainerStarted","Data":"a825ba259aa97b37e60045cad4b6ffb242ea2c7b604a0867bc70bcbf6e760b6b"}
Jul 05 00:13:33.085580 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:13:33.085189    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-5884d96665-mcl4l"
Jul 05 00:13:33.101564 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:13:33.101524    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/image-registry-5884d96665-mcl4l" podStartSLOduration=17.101510971 podStartE2EDuration="17.101510971s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-05 00:13:16 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-05 00:13:33.099951665 +0000 UTC m=+261.442280746" watchObservedRunningTime="2026-07-05 00:13:33.101510971 +0000 UTC m=+261.443840031"
Jul 05 00:13:36.739313 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:13:36.739213    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/7f6da6e5-bbfb-48e4-bb8a-b471c187e3ee-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-snrdh\" (UID: \"7f6da6e5-bbfb-48e4-bb8a-b471c187e3ee\") " pod="openshift-insights/insights-runtime-extractor-snrdh"
Jul 05 00:13:36.741598 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:13:36.741574    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/7f6da6e5-bbfb-48e4-bb8a-b471c187e3ee-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-snrdh\" (UID: \"7f6da6e5-bbfb-48e4-bb8a-b471c187e3ee\") " pod="openshift-insights/insights-runtime-extractor-snrdh"
Jul 05 00:13:36.830638 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:13:36.830595    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-snrdh"
Jul 05 00:13:36.954829 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:13:36.951160    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-snrdh"]
Jul 05 00:13:37.097996 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:13:37.097959    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-snrdh" event={"ID":"7f6da6e5-bbfb-48e4-bb8a-b471c187e3ee","Type":"ContainerStarted","Data":"031cfe5467cbc5712953bc5e2f8b9e7b16b0431c149c6f24d52ae3d8b2131e84"}
Jul 05 00:13:41.112740 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:13:41.112709    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-snrdh" event={"ID":"7f6da6e5-bbfb-48e4-bb8a-b471c187e3ee","Type":"ContainerStarted","Data":"5eaa704cb602764e67aac90daea53249350f95421da733a431402575da801025"}
Jul 05 00:13:42.119156 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:13:42.119118    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-5884d96665-mcl4l"]
Jul 05 00:13:42.125135 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:13:42.125105    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-snrdh" event={"ID":"7f6da6e5-bbfb-48e4-bb8a-b471c187e3ee","Type":"ContainerStarted","Data":"81084a0c6b638757d4bd7f2f61d191a4de98a27d29e129b11056aaac692cbd2a"}
Jul 05 00:13:44.130254 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:13:44.130201    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-snrdh" event={"ID":"7f6da6e5-bbfb-48e4-bb8a-b471c187e3ee","Type":"ContainerStarted","Data":"03b6b63ef1c6762c6d4b7701a43a025ae89fd6bc0e29dbc132f2144302605b99"}
Jul 05 00:13:44.144899 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:13:44.144858    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-snrdh" podStartSLOduration=17.65743454 podStartE2EDuration="24.144845979s" totalImagesPullingTime="6.487411439s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-05 00:13:20 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-05 00:13:44.143967383 +0000 UTC m=+272.486296467" watchObservedRunningTime="2026-07-05 00:13:44.144845979 +0000 UTC m=+272.487175040"
Jul 05 00:13:51.004520 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:13:51.004490    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-d559d66d5-x2tdh"]
Jul 05 00:13:51.007491 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:13:51.007476    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-admission-webhook-d559d66d5-x2tdh"
Jul 05 00:13:51.009374 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:13:51.009353    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-admission-webhook-tls\""
Jul 05 00:13:51.009492 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:13:51.009373    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-admission-webhook-dockercfg-vj8fs\""
Jul 05 00:13:51.013757 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:13:51.013733    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-d559d66d5-x2tdh"]
Jul 05 00:13:51.046686 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:13:51.046663    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/bd0ac8a5-2f74-481a-824a-db88b85f3078-tls-certificates\") pod \"prometheus-operator-admission-webhook-d559d66d5-x2tdh\" (UID: \"bd0ac8a5-2f74-481a-824a-db88b85f3078\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-d559d66d5-x2tdh"
Jul 05 00:13:51.147705 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:13:51.147675    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/bd0ac8a5-2f74-481a-824a-db88b85f3078-tls-certificates\") pod \"prometheus-operator-admission-webhook-d559d66d5-x2tdh\" (UID: \"bd0ac8a5-2f74-481a-824a-db88b85f3078\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-d559d66d5-x2tdh"
Jul 05 00:13:51.147824 ip-10-0-142-202 kubenswrapper[2563]: E0705 00:13:51.147806    2563 secret.go:189] Couldn't get secret openshift-monitoring/prometheus-operator-admission-webhook-tls: secret "prometheus-operator-admission-webhook-tls" not found
Jul 05 00:13:51.147872 ip-10-0-142-202 kubenswrapper[2563]: E0705 00:13:51.147865    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/bd0ac8a5-2f74-481a-824a-db88b85f3078-tls-certificates podName:bd0ac8a5-2f74-481a-824a-db88b85f3078 nodeName:}" failed. No retries permitted until 2026-07-05 00:13:51.647849334 +0000 UTC m=+279.990178371 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "tls-certificates" (UniqueName: "kubernetes.io/secret/bd0ac8a5-2f74-481a-824a-db88b85f3078-tls-certificates") pod "prometheus-operator-admission-webhook-d559d66d5-x2tdh" (UID: "bd0ac8a5-2f74-481a-824a-db88b85f3078") : secret "prometheus-operator-admission-webhook-tls" not found
Jul 05 00:13:51.650058 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:13:51.650024    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/bd0ac8a5-2f74-481a-824a-db88b85f3078-tls-certificates\") pod \"prometheus-operator-admission-webhook-d559d66d5-x2tdh\" (UID: \"bd0ac8a5-2f74-481a-824a-db88b85f3078\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-d559d66d5-x2tdh"
Jul 05 00:13:51.652470 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:13:51.652444    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/bd0ac8a5-2f74-481a-824a-db88b85f3078-tls-certificates\") pod \"prometheus-operator-admission-webhook-d559d66d5-x2tdh\" (UID: \"bd0ac8a5-2f74-481a-824a-db88b85f3078\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-d559d66d5-x2tdh"
Jul 05 00:13:51.916543 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:13:51.916451    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-admission-webhook-d559d66d5-x2tdh"
Jul 05 00:13:52.034267 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:13:52.034238    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-d559d66d5-x2tdh"]
Jul 05 00:13:52.037378 ip-10-0-142-202 kubenswrapper[2563]: W0705 00:13:52.037348    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbd0ac8a5_2f74_481a_824a_db88b85f3078.slice/crio-38af993c8b7e178d8d3f9e7100a96435a04c6a3e35147ab5e0f4896278518e58 WatchSource:0}: Error finding container 38af993c8b7e178d8d3f9e7100a96435a04c6a3e35147ab5e0f4896278518e58: Status 404 returned error can't find the container with id 38af993c8b7e178d8d3f9e7100a96435a04c6a3e35147ab5e0f4896278518e58
Jul 05 00:13:52.126041 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:13:52.126004    2563 patch_prober.go:28] interesting pod/image-registry-5884d96665-mcl4l container/registry namespace/openshift-image-registry: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body={"errors":[{"code":"UNAVAILABLE","message":"service unavailable","detail":"health check failed: please see /debug/health"}]}
Jul 05 00:13:52.126157 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:13:52.126063    2563 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-image-registry/image-registry-5884d96665-mcl4l" podUID="63850164-7354-4352-a5f8-c3f68d980dfa" containerName="registry" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 05 00:13:52.153737 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:13:52.153714    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-admission-webhook-d559d66d5-x2tdh" event={"ID":"bd0ac8a5-2f74-481a-824a-db88b85f3078","Type":"ContainerStarted","Data":"38af993c8b7e178d8d3f9e7100a96435a04c6a3e35147ab5e0f4896278518e58"}
Jul 05 00:13:53.158461 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:13:53.158424    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-admission-webhook-d559d66d5-x2tdh" event={"ID":"bd0ac8a5-2f74-481a-824a-db88b85f3078","Type":"ContainerStarted","Data":"ad7a9ac28d4cd6d0bba96ed069163d11505bb755d84450b42ef87bfe17b7980c"}
Jul 05 00:13:53.158782 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:13:53.158615    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-operator-admission-webhook-d559d66d5-x2tdh"
Jul 05 00:13:53.160164 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:13:53.160145    2563 patch_prober.go:28] interesting pod/prometheus-operator-admission-webhook-d559d66d5-x2tdh container/prometheus-operator-admission-webhook namespace/openshift-monitoring: Readiness probe status=failure output="Get \"https://10.133.0.14:8443/healthz\": dial tcp 10.133.0.14:8443: connect: connection refused" start-of-body=
Jul 05 00:13:53.160250 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:13:53.160182    2563 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-monitoring/prometheus-operator-admission-webhook-d559d66d5-x2tdh" podUID="bd0ac8a5-2f74-481a-824a-db88b85f3078" containerName="prometheus-operator-admission-webhook" probeResult="failure" output="Get \"https://10.133.0.14:8443/healthz\": dial tcp 10.133.0.14:8443: connect: connection refused"
Jul 05 00:13:53.172387 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:13:53.172350    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-operator-admission-webhook-d559d66d5-x2tdh" podStartSLOduration=2.111403627 podStartE2EDuration="3.172337124s" totalImagesPullingTime="1.060933497s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-05 00:13:50 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-05 00:13:53.171320205 +0000 UTC m=+281.513649265" watchObservedRunningTime="2026-07-05 00:13:53.172337124 +0000 UTC m=+281.514666183"
Jul 05 00:13:53.787610 ip-10-0-142-202 kubenswrapper[2563]: E0705 00:13:53.787566    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[metrics-tls], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-dns/dns-default-snsng" podUID="0d75ce89-37fb-4d36-ae84-aee968c71dfa"
Jul 05 00:13:54.160676 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:13:54.160600    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-snsng"
Jul 05 00:13:54.164499 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:13:54.164477    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-operator-admission-webhook-d559d66d5-x2tdh"
Jul 05 00:13:55.057458 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:13:55.057422    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-operator-567f7b8b7c-qch55"]
Jul 05 00:13:55.060515 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:13:55.060500    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-567f7b8b7c-qch55"
Jul 05 00:13:55.062557 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:13:55.062536    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\""
Jul 05 00:13:55.062671 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:13:55.062536    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-dockercfg-jfhkl\""
Jul 05 00:13:55.062671 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:13:55.062535    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-tls\""
Jul 05 00:13:55.063070 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:13:55.063052    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\""
Jul 05 00:13:55.063158 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:13:55.063065    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-kube-rbac-proxy-config\""
Jul 05 00:13:55.063158 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:13:55.063059    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\""
Jul 05 00:13:55.069819 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:13:55.069796    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-567f7b8b7c-qch55"]
Jul 05 00:13:55.071045 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:13:55.071027    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/483ae7bd-5811-4563-8bdb-c3a3c31e6d27-metrics-client-ca\") pod \"prometheus-operator-567f7b8b7c-qch55\" (UID: \"483ae7bd-5811-4563-8bdb-c3a3c31e6d27\") " pod="openshift-monitoring/prometheus-operator-567f7b8b7c-qch55"
Jul 05 00:13:55.071111 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:13:55.071058    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/483ae7bd-5811-4563-8bdb-c3a3c31e6d27-prometheus-operator-tls\") pod \"prometheus-operator-567f7b8b7c-qch55\" (UID: \"483ae7bd-5811-4563-8bdb-c3a3c31e6d27\") " pod="openshift-monitoring/prometheus-operator-567f7b8b7c-qch55"
Jul 05 00:13:55.071111 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:13:55.071104    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cgvrn\" (UniqueName: \"kubernetes.io/projected/483ae7bd-5811-4563-8bdb-c3a3c31e6d27-kube-api-access-cgvrn\") pod \"prometheus-operator-567f7b8b7c-qch55\" (UID: \"483ae7bd-5811-4563-8bdb-c3a3c31e6d27\") " pod="openshift-monitoring/prometheus-operator-567f7b8b7c-qch55"
Jul 05 00:13:55.071176 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:13:55.071137    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/483ae7bd-5811-4563-8bdb-c3a3c31e6d27-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-567f7b8b7c-qch55\" (UID: \"483ae7bd-5811-4563-8bdb-c3a3c31e6d27\") " pod="openshift-monitoring/prometheus-operator-567f7b8b7c-qch55"
Jul 05 00:13:55.172015 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:13:55.171988    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cgvrn\" (UniqueName: \"kubernetes.io/projected/483ae7bd-5811-4563-8bdb-c3a3c31e6d27-kube-api-access-cgvrn\") pod \"prometheus-operator-567f7b8b7c-qch55\" (UID: \"483ae7bd-5811-4563-8bdb-c3a3c31e6d27\") " pod="openshift-monitoring/prometheus-operator-567f7b8b7c-qch55"
Jul 05 00:13:55.172411 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:13:55.172031    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/483ae7bd-5811-4563-8bdb-c3a3c31e6d27-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-567f7b8b7c-qch55\" (UID: \"483ae7bd-5811-4563-8bdb-c3a3c31e6d27\") " pod="openshift-monitoring/prometheus-operator-567f7b8b7c-qch55"
Jul 05 00:13:55.172411 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:13:55.172090    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/483ae7bd-5811-4563-8bdb-c3a3c31e6d27-metrics-client-ca\") pod \"prometheus-operator-567f7b8b7c-qch55\" (UID: \"483ae7bd-5811-4563-8bdb-c3a3c31e6d27\") " pod="openshift-monitoring/prometheus-operator-567f7b8b7c-qch55"
Jul 05 00:13:55.172411 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:13:55.172122    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/483ae7bd-5811-4563-8bdb-c3a3c31e6d27-prometheus-operator-tls\") pod \"prometheus-operator-567f7b8b7c-qch55\" (UID: \"483ae7bd-5811-4563-8bdb-c3a3c31e6d27\") " pod="openshift-monitoring/prometheus-operator-567f7b8b7c-qch55"
Jul 05 00:13:55.172411 ip-10-0-142-202 kubenswrapper[2563]: E0705 00:13:55.172243    2563 secret.go:189] Couldn't get secret openshift-monitoring/prometheus-operator-tls: secret "prometheus-operator-tls" not found
Jul 05 00:13:55.172411 ip-10-0-142-202 kubenswrapper[2563]: E0705 00:13:55.172296    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/483ae7bd-5811-4563-8bdb-c3a3c31e6d27-prometheus-operator-tls podName:483ae7bd-5811-4563-8bdb-c3a3c31e6d27 nodeName:}" failed. No retries permitted until 2026-07-05 00:13:55.672279721 +0000 UTC m=+284.014608760 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "prometheus-operator-tls" (UniqueName: "kubernetes.io/secret/483ae7bd-5811-4563-8bdb-c3a3c31e6d27-prometheus-operator-tls") pod "prometheus-operator-567f7b8b7c-qch55" (UID: "483ae7bd-5811-4563-8bdb-c3a3c31e6d27") : secret "prometheus-operator-tls" not found
Jul 05 00:13:55.172748 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:13:55.172729    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/483ae7bd-5811-4563-8bdb-c3a3c31e6d27-metrics-client-ca\") pod \"prometheus-operator-567f7b8b7c-qch55\" (UID: \"483ae7bd-5811-4563-8bdb-c3a3c31e6d27\") " pod="openshift-monitoring/prometheus-operator-567f7b8b7c-qch55"
Jul 05 00:13:55.174501 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:13:55.174483    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/483ae7bd-5811-4563-8bdb-c3a3c31e6d27-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-567f7b8b7c-qch55\" (UID: \"483ae7bd-5811-4563-8bdb-c3a3c31e6d27\") " pod="openshift-monitoring/prometheus-operator-567f7b8b7c-qch55"
Jul 05 00:13:55.179605 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:13:55.179578    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-cgvrn\" (UniqueName: \"kubernetes.io/projected/483ae7bd-5811-4563-8bdb-c3a3c31e6d27-kube-api-access-cgvrn\") pod \"prometheus-operator-567f7b8b7c-qch55\" (UID: \"483ae7bd-5811-4563-8bdb-c3a3c31e6d27\") " pod="openshift-monitoring/prometheus-operator-567f7b8b7c-qch55"
Jul 05 00:13:55.675935 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:13:55.675905    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/483ae7bd-5811-4563-8bdb-c3a3c31e6d27-prometheus-operator-tls\") pod \"prometheus-operator-567f7b8b7c-qch55\" (UID: \"483ae7bd-5811-4563-8bdb-c3a3c31e6d27\") " pod="openshift-monitoring/prometheus-operator-567f7b8b7c-qch55"
Jul 05 00:13:55.678176 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:13:55.678147    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/483ae7bd-5811-4563-8bdb-c3a3c31e6d27-prometheus-operator-tls\") pod \"prometheus-operator-567f7b8b7c-qch55\" (UID: \"483ae7bd-5811-4563-8bdb-c3a3c31e6d27\") " pod="openshift-monitoring/prometheus-operator-567f7b8b7c-qch55"
Jul 05 00:13:55.969644 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:13:55.969567    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-567f7b8b7c-qch55"
Jul 05 00:13:56.089318 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:13:56.089281    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-567f7b8b7c-qch55"]
Jul 05 00:13:56.092946 ip-10-0-142-202 kubenswrapper[2563]: W0705 00:13:56.092919    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod483ae7bd_5811_4563_8bdb_c3a3c31e6d27.slice/crio-56d79d9368514902b25dd68f380a5f952d0601a15d5b631d21ae0719f9621da2 WatchSource:0}: Error finding container 56d79d9368514902b25dd68f380a5f952d0601a15d5b631d21ae0719f9621da2: Status 404 returned error can't find the container with id 56d79d9368514902b25dd68f380a5f952d0601a15d5b631d21ae0719f9621da2
Jul 05 00:13:56.167476 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:13:56.167447    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-567f7b8b7c-qch55" event={"ID":"483ae7bd-5811-4563-8bdb-c3a3c31e6d27","Type":"ContainerStarted","Data":"56d79d9368514902b25dd68f380a5f952d0601a15d5b631d21ae0719f9621da2"}
Jul 05 00:13:56.884209 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:13:56.884178    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/0d75ce89-37fb-4d36-ae84-aee968c71dfa-metrics-tls\") pod \"dns-default-snsng\" (UID: \"0d75ce89-37fb-4d36-ae84-aee968c71dfa\") " pod="openshift-dns/dns-default-snsng"
Jul 05 00:13:56.886366 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:13:56.886341    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/0d75ce89-37fb-4d36-ae84-aee968c71dfa-metrics-tls\") pod \"dns-default-snsng\" (UID: \"0d75ce89-37fb-4d36-ae84-aee968c71dfa\") " pod="openshift-dns/dns-default-snsng"
Jul 05 00:13:57.163347 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:13:57.163285    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-z9vd9\""
Jul 05 00:13:57.171587 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:13:57.171558    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-snsng"
Jul 05 00:13:57.586887 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:13:57.586851    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-snsng"]
Jul 05 00:13:57.590679 ip-10-0-142-202 kubenswrapper[2563]: W0705 00:13:57.590657    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0d75ce89_37fb_4d36_ae84_aee968c71dfa.slice/crio-6e264dc49c04389e5928422fdc172ebf8a1085cda5fb44bcadf223ce81715736 WatchSource:0}: Error finding container 6e264dc49c04389e5928422fdc172ebf8a1085cda5fb44bcadf223ce81715736: Status 404 returned error can't find the container with id 6e264dc49c04389e5928422fdc172ebf8a1085cda5fb44bcadf223ce81715736
Jul 05 00:13:58.175517 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:13:58.175476    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-567f7b8b7c-qch55" event={"ID":"483ae7bd-5811-4563-8bdb-c3a3c31e6d27","Type":"ContainerStarted","Data":"54d67c5266f1ebfcbfcb46bbca4678d38339d97306f8496aca6dd5f0e7969d4b"}
Jul 05 00:13:58.175994 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:13:58.175547    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-567f7b8b7c-qch55" event={"ID":"483ae7bd-5811-4563-8bdb-c3a3c31e6d27","Type":"ContainerStarted","Data":"06bf14f81bcc136abdf67240a0c4370b68816b1b36a34bbaba43a34efb9777d6"}
Jul 05 00:13:58.176707 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:13:58.176684    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-snsng" event={"ID":"0d75ce89-37fb-4d36-ae84-aee968c71dfa","Type":"ContainerStarted","Data":"6e264dc49c04389e5928422fdc172ebf8a1085cda5fb44bcadf223ce81715736"}
Jul 05 00:13:58.190474 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:13:58.190424    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-operator-567f7b8b7c-qch55" podStartSLOduration=1.765217499 podStartE2EDuration="3.190410732s" totalImagesPullingTime="1.425193233s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-05 00:13:55 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-05 00:13:58.189370554 +0000 UTC m=+286.531699647" watchObservedRunningTime="2026-07-05 00:13:58.190410732 +0000 UTC m=+286.532739793"
Jul 05 00:14:00.185415 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:14:00.185381    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-snsng" event={"ID":"0d75ce89-37fb-4d36-ae84-aee968c71dfa","Type":"ContainerStarted","Data":"54b4a159038261e9d7aedbc1cb8edbda5cc447f664a113e7c7305c420a2e9708"}
Jul 05 00:14:00.185972 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:14:00.185430    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-snsng" event={"ID":"0d75ce89-37fb-4d36-ae84-aee968c71dfa","Type":"ContainerStarted","Data":"3a35fa44f40590bacee6d6ede3bbbd007be21dbd567635fbfe1edd98a94934bb"}
Jul 05 00:14:00.185972 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:14:00.185521    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-snsng"
Jul 05 00:14:00.199991 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:14:00.199950    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/dns-default-snsng" podStartSLOduration=252.436200789 podStartE2EDuration="4m14.199930963s" totalImagesPullingTime="1.763730174s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-05 00:09:46 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-05 00:14:00.199117168 +0000 UTC m=+288.541446228" watchObservedRunningTime="2026-07-05 00:14:00.199930963 +0000 UTC m=+288.542260020"
Jul 05 00:14:00.412300 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:14:00.412269    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-zfrjf"]
Jul 05 00:14:00.415344 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:14:00.415328    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/kube-state-metrics-7c9dbb8b7d-2pcmn"]
Jul 05 00:14:00.415463 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:14:00.415446    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-zfrjf"
Jul 05 00:14:00.417077 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:14:00.417051    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\""
Jul 05 00:14:00.417077 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:14:00.417064    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\""
Jul 05 00:14:00.417309 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:14:00.417144    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\""
Jul 05 00:14:00.417309 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:14:00.417151    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-qgkjb\""
Jul 05 00:14:00.418529 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:14:00.418513    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-2pcmn"
Jul 05 00:14:00.420470 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:14:00.420450    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-dockercfg-hksms\""
Jul 05 00:14:00.420602 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:14:00.420586    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-tls\""
Jul 05 00:14:00.420680 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:14:00.420590    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-kube-rbac-proxy-config\""
Jul 05 00:14:00.420882 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:14:00.420868    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-custom-resource-state-configmap\""
Jul 05 00:14:00.424827 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:14:00.424798    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/kube-state-metrics-7c9dbb8b7d-2pcmn"]
Jul 05 00:14:00.514579 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:14:00.514508    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/b323372b-85f9-406b-ae16-1f71d04a90f2-metrics-client-ca\") pod \"kube-state-metrics-7c9dbb8b7d-2pcmn\" (UID: \"b323372b-85f9-406b-ae16-1f71d04a90f2\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-2pcmn"
Jul 05 00:14:00.514579 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:14:00.514546    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/b323372b-85f9-406b-ae16-1f71d04a90f2-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-7c9dbb8b7d-2pcmn\" (UID: \"b323372b-85f9-406b-ae16-1f71d04a90f2\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-2pcmn"
Jul 05 00:14:00.514579 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:14:00.514570    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/2a8bcefb-0550-490f-b3a0-6421287ee9d9-root\") pod \"node-exporter-zfrjf\" (UID: \"2a8bcefb-0550-490f-b3a0-6421287ee9d9\") " pod="openshift-monitoring/node-exporter-zfrjf"
Jul 05 00:14:00.514771 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:14:00.514592    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/b323372b-85f9-406b-ae16-1f71d04a90f2-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-7c9dbb8b7d-2pcmn\" (UID: \"b323372b-85f9-406b-ae16-1f71d04a90f2\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-2pcmn"
Jul 05 00:14:00.514771 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:14:00.514626    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8dmrn\" (UniqueName: \"kubernetes.io/projected/b323372b-85f9-406b-ae16-1f71d04a90f2-kube-api-access-8dmrn\") pod \"kube-state-metrics-7c9dbb8b7d-2pcmn\" (UID: \"b323372b-85f9-406b-ae16-1f71d04a90f2\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-2pcmn"
Jul 05 00:14:00.514771 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:14:00.514667    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/b323372b-85f9-406b-ae16-1f71d04a90f2-volume-directive-shadow\") pod \"kube-state-metrics-7c9dbb8b7d-2pcmn\" (UID: \"b323372b-85f9-406b-ae16-1f71d04a90f2\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-2pcmn"
Jul 05 00:14:00.514771 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:14:00.514691    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/2a8bcefb-0550-490f-b3a0-6421287ee9d9-node-exporter-wtmp\") pod \"node-exporter-zfrjf\" (UID: \"2a8bcefb-0550-490f-b3a0-6421287ee9d9\") " pod="openshift-monitoring/node-exporter-zfrjf"
Jul 05 00:14:00.514771 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:14:00.514705    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fjhgr\" (UniqueName: \"kubernetes.io/projected/2a8bcefb-0550-490f-b3a0-6421287ee9d9-kube-api-access-fjhgr\") pod \"node-exporter-zfrjf\" (UID: \"2a8bcefb-0550-490f-b3a0-6421287ee9d9\") " pod="openshift-monitoring/node-exporter-zfrjf"
Jul 05 00:14:00.514771 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:14:00.514722    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/b323372b-85f9-406b-ae16-1f71d04a90f2-kube-state-metrics-tls\") pod \"kube-state-metrics-7c9dbb8b7d-2pcmn\" (UID: \"b323372b-85f9-406b-ae16-1f71d04a90f2\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-2pcmn"
Jul 05 00:14:00.514771 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:14:00.514770    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/2a8bcefb-0550-490f-b3a0-6421287ee9d9-metrics-client-ca\") pod \"node-exporter-zfrjf\" (UID: \"2a8bcefb-0550-490f-b3a0-6421287ee9d9\") " pod="openshift-monitoring/node-exporter-zfrjf"
Jul 05 00:14:00.514993 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:14:00.514788    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/2a8bcefb-0550-490f-b3a0-6421287ee9d9-node-exporter-tls\") pod \"node-exporter-zfrjf\" (UID: \"2a8bcefb-0550-490f-b3a0-6421287ee9d9\") " pod="openshift-monitoring/node-exporter-zfrjf"
Jul 05 00:14:00.514993 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:14:00.514835    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/2a8bcefb-0550-490f-b3a0-6421287ee9d9-sys\") pod \"node-exporter-zfrjf\" (UID: \"2a8bcefb-0550-490f-b3a0-6421287ee9d9\") " pod="openshift-monitoring/node-exporter-zfrjf"
Jul 05 00:14:00.514993 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:14:00.514850    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/2a8bcefb-0550-490f-b3a0-6421287ee9d9-node-exporter-textfile\") pod \"node-exporter-zfrjf\" (UID: \"2a8bcefb-0550-490f-b3a0-6421287ee9d9\") " pod="openshift-monitoring/node-exporter-zfrjf"
Jul 05 00:14:00.514993 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:14:00.514866    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/2a8bcefb-0550-490f-b3a0-6421287ee9d9-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-zfrjf\" (UID: \"2a8bcefb-0550-490f-b3a0-6421287ee9d9\") " pod="openshift-monitoring/node-exporter-zfrjf"
Jul 05 00:14:00.514993 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:14:00.514915    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/2a8bcefb-0550-490f-b3a0-6421287ee9d9-node-exporter-accelerators-collector-config\") pod \"node-exporter-zfrjf\" (UID: \"2a8bcefb-0550-490f-b3a0-6421287ee9d9\") " pod="openshift-monitoring/node-exporter-zfrjf"
Jul 05 00:14:00.615885 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:14:00.615854    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/b323372b-85f9-406b-ae16-1f71d04a90f2-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-7c9dbb8b7d-2pcmn\" (UID: \"b323372b-85f9-406b-ae16-1f71d04a90f2\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-2pcmn"
Jul 05 00:14:00.615885 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:14:00.615886    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/2a8bcefb-0550-490f-b3a0-6421287ee9d9-root\") pod \"node-exporter-zfrjf\" (UID: \"2a8bcefb-0550-490f-b3a0-6421287ee9d9\") " pod="openshift-monitoring/node-exporter-zfrjf"
Jul 05 00:14:00.616062 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:14:00.615905    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/b323372b-85f9-406b-ae16-1f71d04a90f2-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-7c9dbb8b7d-2pcmn\" (UID: \"b323372b-85f9-406b-ae16-1f71d04a90f2\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-2pcmn"
Jul 05 00:14:00.616062 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:14:00.615925    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8dmrn\" (UniqueName: \"kubernetes.io/projected/b323372b-85f9-406b-ae16-1f71d04a90f2-kube-api-access-8dmrn\") pod \"kube-state-metrics-7c9dbb8b7d-2pcmn\" (UID: \"b323372b-85f9-406b-ae16-1f71d04a90f2\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-2pcmn"
Jul 05 00:14:00.616062 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:14:00.615982    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/2a8bcefb-0550-490f-b3a0-6421287ee9d9-root\") pod \"node-exporter-zfrjf\" (UID: \"2a8bcefb-0550-490f-b3a0-6421287ee9d9\") " pod="openshift-monitoring/node-exporter-zfrjf"
Jul 05 00:14:00.616062 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:14:00.616021    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/b323372b-85f9-406b-ae16-1f71d04a90f2-volume-directive-shadow\") pod \"kube-state-metrics-7c9dbb8b7d-2pcmn\" (UID: \"b323372b-85f9-406b-ae16-1f71d04a90f2\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-2pcmn"
Jul 05 00:14:00.616062 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:14:00.616048    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/2a8bcefb-0550-490f-b3a0-6421287ee9d9-node-exporter-wtmp\") pod \"node-exporter-zfrjf\" (UID: \"2a8bcefb-0550-490f-b3a0-6421287ee9d9\") " pod="openshift-monitoring/node-exporter-zfrjf"
Jul 05 00:14:00.616339 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:14:00.616072    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fjhgr\" (UniqueName: \"kubernetes.io/projected/2a8bcefb-0550-490f-b3a0-6421287ee9d9-kube-api-access-fjhgr\") pod \"node-exporter-zfrjf\" (UID: \"2a8bcefb-0550-490f-b3a0-6421287ee9d9\") " pod="openshift-monitoring/node-exporter-zfrjf"
Jul 05 00:14:00.616339 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:14:00.616097    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/b323372b-85f9-406b-ae16-1f71d04a90f2-kube-state-metrics-tls\") pod \"kube-state-metrics-7c9dbb8b7d-2pcmn\" (UID: \"b323372b-85f9-406b-ae16-1f71d04a90f2\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-2pcmn"
Jul 05 00:14:00.616339 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:14:00.616129    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/2a8bcefb-0550-490f-b3a0-6421287ee9d9-metrics-client-ca\") pod \"node-exporter-zfrjf\" (UID: \"2a8bcefb-0550-490f-b3a0-6421287ee9d9\") " pod="openshift-monitoring/node-exporter-zfrjf"
Jul 05 00:14:00.616339 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:14:00.616157    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/2a8bcefb-0550-490f-b3a0-6421287ee9d9-node-exporter-tls\") pod \"node-exporter-zfrjf\" (UID: \"2a8bcefb-0550-490f-b3a0-6421287ee9d9\") " pod="openshift-monitoring/node-exporter-zfrjf"
Jul 05 00:14:00.616339 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:14:00.616199    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/2a8bcefb-0550-490f-b3a0-6421287ee9d9-sys\") pod \"node-exporter-zfrjf\" (UID: \"2a8bcefb-0550-490f-b3a0-6421287ee9d9\") " pod="openshift-monitoring/node-exporter-zfrjf"
Jul 05 00:14:00.616339 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:14:00.616243    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/2a8bcefb-0550-490f-b3a0-6421287ee9d9-node-exporter-textfile\") pod \"node-exporter-zfrjf\" (UID: \"2a8bcefb-0550-490f-b3a0-6421287ee9d9\") " pod="openshift-monitoring/node-exporter-zfrjf"
Jul 05 00:14:00.616339 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:14:00.616259    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/2a8bcefb-0550-490f-b3a0-6421287ee9d9-node-exporter-wtmp\") pod \"node-exporter-zfrjf\" (UID: \"2a8bcefb-0550-490f-b3a0-6421287ee9d9\") " pod="openshift-monitoring/node-exporter-zfrjf"
Jul 05 00:14:00.616339 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:14:00.616271    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/2a8bcefb-0550-490f-b3a0-6421287ee9d9-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-zfrjf\" (UID: \"2a8bcefb-0550-490f-b3a0-6421287ee9d9\") " pod="openshift-monitoring/node-exporter-zfrjf"
Jul 05 00:14:00.616339 ip-10-0-142-202 kubenswrapper[2563]: E0705 00:14:00.616261    2563 secret.go:189] Couldn't get secret openshift-monitoring/kube-state-metrics-tls: secret "kube-state-metrics-tls" not found
Jul 05 00:14:00.616761 ip-10-0-142-202 kubenswrapper[2563]: E0705 00:14:00.616419    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b323372b-85f9-406b-ae16-1f71d04a90f2-kube-state-metrics-tls podName:b323372b-85f9-406b-ae16-1f71d04a90f2 nodeName:}" failed. No retries permitted until 2026-07-05 00:14:01.116394746 +0000 UTC m=+289.458723784 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-state-metrics-tls" (UniqueName: "kubernetes.io/secret/b323372b-85f9-406b-ae16-1f71d04a90f2-kube-state-metrics-tls") pod "kube-state-metrics-7c9dbb8b7d-2pcmn" (UID: "b323372b-85f9-406b-ae16-1f71d04a90f2") : secret "kube-state-metrics-tls" not found
Jul 05 00:14:00.616761 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:14:00.616326    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/2a8bcefb-0550-490f-b3a0-6421287ee9d9-node-exporter-accelerators-collector-config\") pod \"node-exporter-zfrjf\" (UID: \"2a8bcefb-0550-490f-b3a0-6421287ee9d9\") " pod="openshift-monitoring/node-exporter-zfrjf"
Jul 05 00:14:00.616761 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:14:00.616494    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/b323372b-85f9-406b-ae16-1f71d04a90f2-metrics-client-ca\") pod \"kube-state-metrics-7c9dbb8b7d-2pcmn\" (UID: \"b323372b-85f9-406b-ae16-1f71d04a90f2\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-2pcmn"
Jul 05 00:14:00.616761 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:14:00.616327    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/2a8bcefb-0550-490f-b3a0-6421287ee9d9-sys\") pod \"node-exporter-zfrjf\" (UID: \"2a8bcefb-0550-490f-b3a0-6421287ee9d9\") " pod="openshift-monitoring/node-exporter-zfrjf"
Jul 05 00:14:00.616761 ip-10-0-142-202 kubenswrapper[2563]: E0705 00:14:00.616324    2563 secret.go:189] Couldn't get secret openshift-monitoring/node-exporter-tls: secret "node-exporter-tls" not found
Jul 05 00:14:00.616761 ip-10-0-142-202 kubenswrapper[2563]: E0705 00:14:00.616607    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2a8bcefb-0550-490f-b3a0-6421287ee9d9-node-exporter-tls podName:2a8bcefb-0550-490f-b3a0-6421287ee9d9 nodeName:}" failed. No retries permitted until 2026-07-05 00:14:01.116593025 +0000 UTC m=+289.458922063 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "node-exporter-tls" (UniqueName: "kubernetes.io/secret/2a8bcefb-0550-490f-b3a0-6421287ee9d9-node-exporter-tls") pod "node-exporter-zfrjf" (UID: "2a8bcefb-0550-490f-b3a0-6421287ee9d9") : secret "node-exporter-tls" not found
Jul 05 00:14:00.616761 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:14:00.616629    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/2a8bcefb-0550-490f-b3a0-6421287ee9d9-node-exporter-textfile\") pod \"node-exporter-zfrjf\" (UID: \"2a8bcefb-0550-490f-b3a0-6421287ee9d9\") " pod="openshift-monitoring/node-exporter-zfrjf"
Jul 05 00:14:00.616761 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:14:00.616665    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/b323372b-85f9-406b-ae16-1f71d04a90f2-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-7c9dbb8b7d-2pcmn\" (UID: \"b323372b-85f9-406b-ae16-1f71d04a90f2\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-2pcmn"
Jul 05 00:14:00.617151 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:14:00.616908    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/2a8bcefb-0550-490f-b3a0-6421287ee9d9-metrics-client-ca\") pod \"node-exporter-zfrjf\" (UID: \"2a8bcefb-0550-490f-b3a0-6421287ee9d9\") " pod="openshift-monitoring/node-exporter-zfrjf"
Jul 05 00:14:00.617151 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:14:00.616940    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/b323372b-85f9-406b-ae16-1f71d04a90f2-volume-directive-shadow\") pod \"kube-state-metrics-7c9dbb8b7d-2pcmn\" (UID: \"b323372b-85f9-406b-ae16-1f71d04a90f2\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-2pcmn"
Jul 05 00:14:00.617151 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:14:00.617038    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/2a8bcefb-0550-490f-b3a0-6421287ee9d9-node-exporter-accelerators-collector-config\") pod \"node-exporter-zfrjf\" (UID: \"2a8bcefb-0550-490f-b3a0-6421287ee9d9\") " pod="openshift-monitoring/node-exporter-zfrjf"
Jul 05 00:14:00.617151 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:14:00.617077    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/b323372b-85f9-406b-ae16-1f71d04a90f2-metrics-client-ca\") pod \"kube-state-metrics-7c9dbb8b7d-2pcmn\" (UID: \"b323372b-85f9-406b-ae16-1f71d04a90f2\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-2pcmn"
Jul 05 00:14:00.618518 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:14:00.618497    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/b323372b-85f9-406b-ae16-1f71d04a90f2-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-7c9dbb8b7d-2pcmn\" (UID: \"b323372b-85f9-406b-ae16-1f71d04a90f2\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-2pcmn"
Jul 05 00:14:00.618716 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:14:00.618699    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/2a8bcefb-0550-490f-b3a0-6421287ee9d9-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-zfrjf\" (UID: \"2a8bcefb-0550-490f-b3a0-6421287ee9d9\") " pod="openshift-monitoring/node-exporter-zfrjf"
Jul 05 00:14:00.624979 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:14:00.624956    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-8dmrn\" (UniqueName: \"kubernetes.io/projected/b323372b-85f9-406b-ae16-1f71d04a90f2-kube-api-access-8dmrn\") pod \"kube-state-metrics-7c9dbb8b7d-2pcmn\" (UID: \"b323372b-85f9-406b-ae16-1f71d04a90f2\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-2pcmn"
Jul 05 00:14:00.625117 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:14:00.625101    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-fjhgr\" (UniqueName: \"kubernetes.io/projected/2a8bcefb-0550-490f-b3a0-6421287ee9d9-kube-api-access-fjhgr\") pod \"node-exporter-zfrjf\" (UID: \"2a8bcefb-0550-490f-b3a0-6421287ee9d9\") " pod="openshift-monitoring/node-exporter-zfrjf"
Jul 05 00:14:01.120538 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:14:01.120504    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/b323372b-85f9-406b-ae16-1f71d04a90f2-kube-state-metrics-tls\") pod \"kube-state-metrics-7c9dbb8b7d-2pcmn\" (UID: \"b323372b-85f9-406b-ae16-1f71d04a90f2\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-2pcmn"
Jul 05 00:14:01.120700 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:14:01.120552    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/2a8bcefb-0550-490f-b3a0-6421287ee9d9-node-exporter-tls\") pod \"node-exporter-zfrjf\" (UID: \"2a8bcefb-0550-490f-b3a0-6421287ee9d9\") " pod="openshift-monitoring/node-exporter-zfrjf"
Jul 05 00:14:01.122845 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:14:01.122813    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/2a8bcefb-0550-490f-b3a0-6421287ee9d9-node-exporter-tls\") pod \"node-exporter-zfrjf\" (UID: \"2a8bcefb-0550-490f-b3a0-6421287ee9d9\") " pod="openshift-monitoring/node-exporter-zfrjf"
Jul 05 00:14:01.123005 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:14:01.122984    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/b323372b-85f9-406b-ae16-1f71d04a90f2-kube-state-metrics-tls\") pod \"kube-state-metrics-7c9dbb8b7d-2pcmn\" (UID: \"b323372b-85f9-406b-ae16-1f71d04a90f2\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-2pcmn"
Jul 05 00:14:01.325682 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:14:01.325648    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-zfrjf"
Jul 05 00:14:01.330766 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:14:01.330753    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-2pcmn"
Jul 05 00:14:01.338771 ip-10-0-142-202 kubenswrapper[2563]: W0705 00:14:01.338742    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2a8bcefb_0550_490f_b3a0_6421287ee9d9.slice/crio-2b5b130499be2e15b2ce3e99954a56a9e37c77cb831ef055f76b8a95697c1f29 WatchSource:0}: Error finding container 2b5b130499be2e15b2ce3e99954a56a9e37c77cb831ef055f76b8a95697c1f29: Status 404 returned error can't find the container with id 2b5b130499be2e15b2ce3e99954a56a9e37c77cb831ef055f76b8a95697c1f29
Jul 05 00:14:01.466259 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:14:01.466003    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/kube-state-metrics-7c9dbb8b7d-2pcmn"]
Jul 05 00:14:01.470627 ip-10-0-142-202 kubenswrapper[2563]: W0705 00:14:01.470600    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb323372b_85f9_406b_ae16_1f71d04a90f2.slice/crio-adce34ac4168ee3cf8d003ef17ca8c08dbf76565bdaea97b4a4e05ca9d8de8d3 WatchSource:0}: Error finding container adce34ac4168ee3cf8d003ef17ca8c08dbf76565bdaea97b4a4e05ca9d8de8d3: Status 404 returned error can't find the container with id adce34ac4168ee3cf8d003ef17ca8c08dbf76565bdaea97b4a4e05ca9d8de8d3
Jul 05 00:14:02.127558 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:14:02.127528    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-image-registry/image-registry-5884d96665-mcl4l"
Jul 05 00:14:02.193458 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:14:02.193425    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-2pcmn" event={"ID":"b323372b-85f9-406b-ae16-1f71d04a90f2","Type":"ContainerStarted","Data":"adce34ac4168ee3cf8d003ef17ca8c08dbf76565bdaea97b4a4e05ca9d8de8d3"}
Jul 05 00:14:02.194818 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:14:02.194782    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-zfrjf" event={"ID":"2a8bcefb-0550-490f-b3a0-6421287ee9d9","Type":"ContainerStarted","Data":"2b5b130499be2e15b2ce3e99954a56a9e37c77cb831ef055f76b8a95697c1f29"}
Jul 05 00:14:03.201630 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:14:03.201533    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-2pcmn" event={"ID":"b323372b-85f9-406b-ae16-1f71d04a90f2","Type":"ContainerStarted","Data":"3999d9df0aeedd07bd85361aa8b11625b775a7b7c350f44bebedf5aa5ce40af1"}
Jul 05 00:14:03.201630 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:14:03.201604    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-2pcmn" event={"ID":"b323372b-85f9-406b-ae16-1f71d04a90f2","Type":"ContainerStarted","Data":"2bc1c621879181a5726de89d3d16c26b8247bad24bd50fe4f98013dad0ec471a"}
Jul 05 00:14:03.202130 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:14:03.201631    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-2pcmn" event={"ID":"b323372b-85f9-406b-ae16-1f71d04a90f2","Type":"ContainerStarted","Data":"86307b30131e3f1a4914d109de231d7d88350890013763c6650c4faee66fac6f"}
Jul 05 00:14:03.203361 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:14:03.203329    2563 generic.go:356] "Generic (PLEG): container finished" podID="2a8bcefb-0550-490f-b3a0-6421287ee9d9" containerID="c9c2cc9ff7072984ca7c157037eb694d88be97650ce17793a775a25712b7ddf0" exitCode=0
Jul 05 00:14:03.203484 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:14:03.203411    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-zfrjf" event={"ID":"2a8bcefb-0550-490f-b3a0-6421287ee9d9","Type":"ContainerDied","Data":"c9c2cc9ff7072984ca7c157037eb694d88be97650ce17793a775a25712b7ddf0"}
Jul 05 00:14:03.224293 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:14:03.224245    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-2pcmn" podStartSLOduration=1.813105644 podStartE2EDuration="3.224211047s" totalImagesPullingTime="1.411105403s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-05 00:14:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-05 00:14:03.222707901 +0000 UTC m=+291.565036983" watchObservedRunningTime="2026-07-05 00:14:03.224211047 +0000 UTC m=+291.566540106"
Jul 05 00:14:04.212295 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:14:04.212254    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-zfrjf" event={"ID":"2a8bcefb-0550-490f-b3a0-6421287ee9d9","Type":"ContainerStarted","Data":"ee1b8c1868585f9ccc9ae516761093dc50e29d18ad551977d7e71b387b3969f4"}
Jul 05 00:14:04.212756 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:14:04.212334    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-zfrjf" event={"ID":"2a8bcefb-0550-490f-b3a0-6421287ee9d9","Type":"ContainerStarted","Data":"a24ceee62b4ed693ecfb2160f70ae515158471977dd1f9f725ee4e18656049e8"}
Jul 05 00:14:04.229449 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:14:04.229407    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-zfrjf" podStartSLOduration=3.010322477 podStartE2EDuration="4.229393027s" totalImagesPullingTime="1.146709117s" totalInitContainerRuntime="72.361433ms" isStatefulPod=true podCreationTimestamp="2026-07-05 00:14:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-05 00:14:04.227650378 +0000 UTC m=+292.569979470" watchObservedRunningTime="2026-07-05 00:14:04.229393027 +0000 UTC m=+292.571722087"
Jul 05 00:14:06.626025 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:14:06.625988    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Jul 05 00:14:06.629893 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:14:06.629866    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Jul 05 00:14:06.632324 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:14:06.632300    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-trusted-ca-bundle\""
Jul 05 00:14:06.632451 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:14:06.632396    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-client-certs\""
Jul 05 00:14:06.632451 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:14:06.632331    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-rbac-proxy\""
Jul 05 00:14:06.632563 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:14:06.632300    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-kube-rbac-proxy-web\""
Jul 05 00:14:06.632563 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:14:06.632356    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-grpc-tls-dc17ieuvn19j0\""
Jul 05 00:14:06.632756 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:14:06.632737    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-1\""
Jul 05 00:14:06.633060 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:14:06.633043    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-2\""
Jul 05 00:14:06.633171 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:14:06.633117    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls\""
Jul 05 00:14:06.633281 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:14:06.633258    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"serving-certs-ca-bundle\""
Jul 05 00:14:06.633354 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:14:06.633123    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kubelet-serving-ca-bundle\""
Jul 05 00:14:06.633568 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:14:06.633549    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-prometheus-http-client-file\""
Jul 05 00:14:06.633785 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:14:06.633768    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-web-config\""
Jul 05 00:14:06.633908 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:14:06.633783    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-sidecar-tls\""
Jul 05 00:14:06.634033 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:14:06.634010    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-dockercfg-j9nl5\""
Jul 05 00:14:06.634147 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:14:06.633918    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls-assets-0\""
Jul 05 00:14:06.634470 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:14:06.634452    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s\""
Jul 05 00:14:06.635857 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:14:06.635835    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-0\""
Jul 05 00:14:06.645458 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:14:06.645431    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Jul 05 00:14:06.666014 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:14:06.665979    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/d4aa743c-f71b-4023-97bd-05452c4e4e15-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"d4aa743c-f71b-4023-97bd-05452c4e4e15\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 05 00:14:06.666201 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:14:06.666021    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/d4aa743c-f71b-4023-97bd-05452c4e4e15-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"d4aa743c-f71b-4023-97bd-05452c4e4e15\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 05 00:14:06.666201 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:14:06.666107    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/d4aa743c-f71b-4023-97bd-05452c4e4e15-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"d4aa743c-f71b-4023-97bd-05452c4e4e15\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 05 00:14:06.666201 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:14:06.666168    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8lc7t\" (UniqueName: \"kubernetes.io/projected/d4aa743c-f71b-4023-97bd-05452c4e4e15-kube-api-access-8lc7t\") pod \"prometheus-k8s-0\" (UID: \"d4aa743c-f71b-4023-97bd-05452c4e4e15\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 05 00:14:06.666391 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:14:06.666204    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/d4aa743c-f71b-4023-97bd-05452c4e4e15-web-config\") pod \"prometheus-k8s-0\" (UID: \"d4aa743c-f71b-4023-97bd-05452c4e4e15\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 05 00:14:06.666391 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:14:06.666274    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/d4aa743c-f71b-4023-97bd-05452c4e4e15-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"d4aa743c-f71b-4023-97bd-05452c4e4e15\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 05 00:14:06.666391 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:14:06.666310    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/d4aa743c-f71b-4023-97bd-05452c4e4e15-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"d4aa743c-f71b-4023-97bd-05452c4e4e15\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 05 00:14:06.666391 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:14:06.666347    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/d4aa743c-f71b-4023-97bd-05452c4e4e15-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"d4aa743c-f71b-4023-97bd-05452c4e4e15\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 05 00:14:06.666391 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:14:06.666374    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/d4aa743c-f71b-4023-97bd-05452c4e4e15-config\") pod \"prometheus-k8s-0\" (UID: \"d4aa743c-f71b-4023-97bd-05452c4e4e15\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 05 00:14:06.666642 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:14:06.666415    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/d4aa743c-f71b-4023-97bd-05452c4e4e15-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"d4aa743c-f71b-4023-97bd-05452c4e4e15\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 05 00:14:06.666642 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:14:06.666443    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/d4aa743c-f71b-4023-97bd-05452c4e4e15-config-out\") pod \"prometheus-k8s-0\" (UID: \"d4aa743c-f71b-4023-97bd-05452c4e4e15\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 05 00:14:06.666642 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:14:06.666474    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d4aa743c-f71b-4023-97bd-05452c4e4e15-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"d4aa743c-f71b-4023-97bd-05452c4e4e15\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 05 00:14:06.666642 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:14:06.666498    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/d4aa743c-f71b-4023-97bd-05452c4e4e15-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"d4aa743c-f71b-4023-97bd-05452c4e4e15\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 05 00:14:06.666642 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:14:06.666521    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/d4aa743c-f71b-4023-97bd-05452c4e4e15-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"d4aa743c-f71b-4023-97bd-05452c4e4e15\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 05 00:14:06.666642 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:14:06.666562    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/d4aa743c-f71b-4023-97bd-05452c4e4e15-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"d4aa743c-f71b-4023-97bd-05452c4e4e15\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 05 00:14:06.666642 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:14:06.666583    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d4aa743c-f71b-4023-97bd-05452c4e4e15-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"d4aa743c-f71b-4023-97bd-05452c4e4e15\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 05 00:14:06.666642 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:14:06.666609    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/d4aa743c-f71b-4023-97bd-05452c4e4e15-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"d4aa743c-f71b-4023-97bd-05452c4e4e15\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 05 00:14:06.667013 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:14:06.666649    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/d4aa743c-f71b-4023-97bd-05452c4e4e15-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"d4aa743c-f71b-4023-97bd-05452c4e4e15\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 05 00:14:06.667013 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:14:06.666678    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/d4aa743c-f71b-4023-97bd-05452c4e4e15-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"d4aa743c-f71b-4023-97bd-05452c4e4e15\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 05 00:14:06.667013 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:14:06.666708    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d4aa743c-f71b-4023-97bd-05452c4e4e15-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"d4aa743c-f71b-4023-97bd-05452c4e4e15\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 05 00:14:06.768134 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:14:06.768102    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/d4aa743c-f71b-4023-97bd-05452c4e4e15-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"d4aa743c-f71b-4023-97bd-05452c4e4e15\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 05 00:14:06.768343 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:14:06.768141    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/d4aa743c-f71b-4023-97bd-05452c4e4e15-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"d4aa743c-f71b-4023-97bd-05452c4e4e15\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 05 00:14:06.768343 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:14:06.768160    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/d4aa743c-f71b-4023-97bd-05452c4e4e15-config\") pod \"prometheus-k8s-0\" (UID: \"d4aa743c-f71b-4023-97bd-05452c4e4e15\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 05 00:14:06.768343 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:14:06.768177    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/d4aa743c-f71b-4023-97bd-05452c4e4e15-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"d4aa743c-f71b-4023-97bd-05452c4e4e15\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 05 00:14:06.768343 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:14:06.768193    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/d4aa743c-f71b-4023-97bd-05452c4e4e15-config-out\") pod \"prometheus-k8s-0\" (UID: \"d4aa743c-f71b-4023-97bd-05452c4e4e15\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 05 00:14:06.768343 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:14:06.768268    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d4aa743c-f71b-4023-97bd-05452c4e4e15-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"d4aa743c-f71b-4023-97bd-05452c4e4e15\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 05 00:14:06.768343 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:14:06.768292    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/d4aa743c-f71b-4023-97bd-05452c4e4e15-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"d4aa743c-f71b-4023-97bd-05452c4e4e15\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 05 00:14:06.768343 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:14:06.768317    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/d4aa743c-f71b-4023-97bd-05452c4e4e15-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"d4aa743c-f71b-4023-97bd-05452c4e4e15\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 05 00:14:06.768705 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:14:06.768376    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/d4aa743c-f71b-4023-97bd-05452c4e4e15-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"d4aa743c-f71b-4023-97bd-05452c4e4e15\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 05 00:14:06.768705 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:14:06.768400    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d4aa743c-f71b-4023-97bd-05452c4e4e15-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"d4aa743c-f71b-4023-97bd-05452c4e4e15\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 05 00:14:06.768705 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:14:06.768430    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/d4aa743c-f71b-4023-97bd-05452c4e4e15-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"d4aa743c-f71b-4023-97bd-05452c4e4e15\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 05 00:14:06.768705 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:14:06.768471    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/d4aa743c-f71b-4023-97bd-05452c4e4e15-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"d4aa743c-f71b-4023-97bd-05452c4e4e15\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 05 00:14:06.768705 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:14:06.768493    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/d4aa743c-f71b-4023-97bd-05452c4e4e15-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"d4aa743c-f71b-4023-97bd-05452c4e4e15\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 05 00:14:06.768705 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:14:06.768522    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d4aa743c-f71b-4023-97bd-05452c4e4e15-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"d4aa743c-f71b-4023-97bd-05452c4e4e15\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 05 00:14:06.768705 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:14:06.768546    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/d4aa743c-f71b-4023-97bd-05452c4e4e15-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"d4aa743c-f71b-4023-97bd-05452c4e4e15\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 05 00:14:06.768705 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:14:06.768568    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/d4aa743c-f71b-4023-97bd-05452c4e4e15-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"d4aa743c-f71b-4023-97bd-05452c4e4e15\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 05 00:14:06.768705 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:14:06.768621    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/d4aa743c-f71b-4023-97bd-05452c4e4e15-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"d4aa743c-f71b-4023-97bd-05452c4e4e15\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 05 00:14:06.768705 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:14:06.768653    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8lc7t\" (UniqueName: \"kubernetes.io/projected/d4aa743c-f71b-4023-97bd-05452c4e4e15-kube-api-access-8lc7t\") pod \"prometheus-k8s-0\" (UID: \"d4aa743c-f71b-4023-97bd-05452c4e4e15\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 05 00:14:06.768705 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:14:06.768708    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/d4aa743c-f71b-4023-97bd-05452c4e4e15-web-config\") pod \"prometheus-k8s-0\" (UID: \"d4aa743c-f71b-4023-97bd-05452c4e4e15\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 05 00:14:06.769348 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:14:06.768761    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/d4aa743c-f71b-4023-97bd-05452c4e4e15-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"d4aa743c-f71b-4023-97bd-05452c4e4e15\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 05 00:14:06.769348 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:14:06.768977    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/d4aa743c-f71b-4023-97bd-05452c4e4e15-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"d4aa743c-f71b-4023-97bd-05452c4e4e15\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 05 00:14:06.769348 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:14:06.769184    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d4aa743c-f71b-4023-97bd-05452c4e4e15-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"d4aa743c-f71b-4023-97bd-05452c4e4e15\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 05 00:14:06.769966 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:14:06.769603    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/d4aa743c-f71b-4023-97bd-05452c4e4e15-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"d4aa743c-f71b-4023-97bd-05452c4e4e15\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 05 00:14:06.773173 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:14:06.771501    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d4aa743c-f71b-4023-97bd-05452c4e4e15-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"d4aa743c-f71b-4023-97bd-05452c4e4e15\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 05 00:14:06.773173 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:14:06.771790    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/d4aa743c-f71b-4023-97bd-05452c4e4e15-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"d4aa743c-f71b-4023-97bd-05452c4e4e15\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 05 00:14:06.773173 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:14:06.771866    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/d4aa743c-f71b-4023-97bd-05452c4e4e15-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"d4aa743c-f71b-4023-97bd-05452c4e4e15\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 05 00:14:06.773173 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:14:06.772078    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/d4aa743c-f71b-4023-97bd-05452c4e4e15-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"d4aa743c-f71b-4023-97bd-05452c4e4e15\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 05 00:14:06.773173 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:14:06.772165    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/d4aa743c-f71b-4023-97bd-05452c4e4e15-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"d4aa743c-f71b-4023-97bd-05452c4e4e15\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 05 00:14:06.773173 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:14:06.772308    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/d4aa743c-f71b-4023-97bd-05452c4e4e15-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"d4aa743c-f71b-4023-97bd-05452c4e4e15\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 05 00:14:06.773173 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:14:06.772635    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/d4aa743c-f71b-4023-97bd-05452c4e4e15-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"d4aa743c-f71b-4023-97bd-05452c4e4e15\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 05 00:14:06.773173 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:14:06.772991    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/d4aa743c-f71b-4023-97bd-05452c4e4e15-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"d4aa743c-f71b-4023-97bd-05452c4e4e15\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 05 00:14:06.773173 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:14:06.773034    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d4aa743c-f71b-4023-97bd-05452c4e4e15-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"d4aa743c-f71b-4023-97bd-05452c4e4e15\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 05 00:14:06.773671 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:14:06.773180    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/d4aa743c-f71b-4023-97bd-05452c4e4e15-config-out\") pod \"prometheus-k8s-0\" (UID: \"d4aa743c-f71b-4023-97bd-05452c4e4e15\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 05 00:14:06.773759 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:14:06.773735    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/d4aa743c-f71b-4023-97bd-05452c4e4e15-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"d4aa743c-f71b-4023-97bd-05452c4e4e15\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 05 00:14:06.774177 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:14:06.774143    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/d4aa743c-f71b-4023-97bd-05452c4e4e15-web-config\") pod \"prometheus-k8s-0\" (UID: \"d4aa743c-f71b-4023-97bd-05452c4e4e15\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 05 00:14:06.774971 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:14:06.774903    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/secret/d4aa743c-f71b-4023-97bd-05452c4e4e15-config\") pod \"prometheus-k8s-0\" (UID: \"d4aa743c-f71b-4023-97bd-05452c4e4e15\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 05 00:14:06.775291 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:14:06.775272    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/d4aa743c-f71b-4023-97bd-05452c4e4e15-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"d4aa743c-f71b-4023-97bd-05452c4e4e15\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 05 00:14:06.775500 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:14:06.775480    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/d4aa743c-f71b-4023-97bd-05452c4e4e15-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"d4aa743c-f71b-4023-97bd-05452c4e4e15\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 05 00:14:06.775833 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:14:06.775812    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/d4aa743c-f71b-4023-97bd-05452c4e4e15-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"d4aa743c-f71b-4023-97bd-05452c4e4e15\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 05 00:14:06.778051 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:14:06.778028    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-8lc7t\" (UniqueName: \"kubernetes.io/projected/d4aa743c-f71b-4023-97bd-05452c4e4e15-kube-api-access-8lc7t\") pod \"prometheus-k8s-0\" (UID: \"d4aa743c-f71b-4023-97bd-05452c4e4e15\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 05 00:14:06.944237 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:14:06.944126    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Jul 05 00:14:07.099116 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:14:07.099078    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Jul 05 00:14:07.103556 ip-10-0-142-202 kubenswrapper[2563]: W0705 00:14:07.103530    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd4aa743c_f71b_4023_97bd_05452c4e4e15.slice/crio-e68893d1afb92d251014ac8788c3b1c4e0fd4e62279901b92cde4096ed7b2bf1 WatchSource:0}: Error finding container e68893d1afb92d251014ac8788c3b1c4e0fd4e62279901b92cde4096ed7b2bf1: Status 404 returned error can't find the container with id e68893d1afb92d251014ac8788c3b1c4e0fd4e62279901b92cde4096ed7b2bf1
Jul 05 00:14:07.149442 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:14:07.149390    2563 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-image-registry/image-registry-5884d96665-mcl4l" podUID="63850164-7354-4352-a5f8-c3f68d980dfa" containerName="registry" containerID="cri-o://434072f1ea5bdf1b156b820dd87baaa59f55f97554a463d7846aa7d325ffb080" gracePeriod=30
Jul 05 00:14:07.224135 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:14:07.224054    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"d4aa743c-f71b-4023-97bd-05452c4e4e15","Type":"ContainerStarted","Data":"e68893d1afb92d251014ac8788c3b1c4e0fd4e62279901b92cde4096ed7b2bf1"}
Jul 05 00:14:07.375532 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:14:07.375506    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-5884d96665-mcl4l"
Jul 05 00:14:07.476275 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:14:07.476178    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/63850164-7354-4352-a5f8-c3f68d980dfa-installation-pull-secrets\") pod \"63850164-7354-4352-a5f8-c3f68d980dfa\" (UID: \"63850164-7354-4352-a5f8-c3f68d980dfa\") "
Jul 05 00:14:07.476275 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:14:07.476215    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/63850164-7354-4352-a5f8-c3f68d980dfa-bound-sa-token\") pod \"63850164-7354-4352-a5f8-c3f68d980dfa\" (UID: \"63850164-7354-4352-a5f8-c3f68d980dfa\") "
Jul 05 00:14:07.476275 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:14:07.476251    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-58g7w\" (UniqueName: \"kubernetes.io/projected/63850164-7354-4352-a5f8-c3f68d980dfa-kube-api-access-58g7w\") pod \"63850164-7354-4352-a5f8-c3f68d980dfa\" (UID: \"63850164-7354-4352-a5f8-c3f68d980dfa\") "
Jul 05 00:14:07.476491 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:14:07.476350    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/63850164-7354-4352-a5f8-c3f68d980dfa-trusted-ca\") pod \"63850164-7354-4352-a5f8-c3f68d980dfa\" (UID: \"63850164-7354-4352-a5f8-c3f68d980dfa\") "
Jul 05 00:14:07.476491 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:14:07.476410    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/63850164-7354-4352-a5f8-c3f68d980dfa-registry-tls\") pod \"63850164-7354-4352-a5f8-c3f68d980dfa\" (UID: \"63850164-7354-4352-a5f8-c3f68d980dfa\") "
Jul 05 00:14:07.476491 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:14:07.476458    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/63850164-7354-4352-a5f8-c3f68d980dfa-ca-trust-extracted\") pod \"63850164-7354-4352-a5f8-c3f68d980dfa\" (UID: \"63850164-7354-4352-a5f8-c3f68d980dfa\") "
Jul 05 00:14:07.476618 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:14:07.476497    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/63850164-7354-4352-a5f8-c3f68d980dfa-image-registry-private-configuration\") pod \"63850164-7354-4352-a5f8-c3f68d980dfa\" (UID: \"63850164-7354-4352-a5f8-c3f68d980dfa\") "
Jul 05 00:14:07.476618 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:14:07.476534    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/63850164-7354-4352-a5f8-c3f68d980dfa-registry-certificates\") pod \"63850164-7354-4352-a5f8-c3f68d980dfa\" (UID: \"63850164-7354-4352-a5f8-c3f68d980dfa\") "
Jul 05 00:14:07.476900 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:14:07.476843    2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/63850164-7354-4352-a5f8-c3f68d980dfa-trusted-ca" (OuterVolumeSpecName: "trusted-ca") pod "63850164-7354-4352-a5f8-c3f68d980dfa" (UID: "63850164-7354-4352-a5f8-c3f68d980dfa"). InnerVolumeSpecName "trusted-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 05 00:14:07.477172 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:14:07.477147    2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/63850164-7354-4352-a5f8-c3f68d980dfa-registry-certificates" (OuterVolumeSpecName: "registry-certificates") pod "63850164-7354-4352-a5f8-c3f68d980dfa" (UID: "63850164-7354-4352-a5f8-c3f68d980dfa"). InnerVolumeSpecName "registry-certificates". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 05 00:14:07.478787 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:14:07.478758    2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/63850164-7354-4352-a5f8-c3f68d980dfa-kube-api-access-58g7w" (OuterVolumeSpecName: "kube-api-access-58g7w") pod "63850164-7354-4352-a5f8-c3f68d980dfa" (UID: "63850164-7354-4352-a5f8-c3f68d980dfa"). InnerVolumeSpecName "kube-api-access-58g7w". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 05 00:14:07.478787 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:14:07.478773    2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/63850164-7354-4352-a5f8-c3f68d980dfa-installation-pull-secrets" (OuterVolumeSpecName: "installation-pull-secrets") pod "63850164-7354-4352-a5f8-c3f68d980dfa" (UID: "63850164-7354-4352-a5f8-c3f68d980dfa"). InnerVolumeSpecName "installation-pull-secrets". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 05 00:14:07.478931 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:14:07.478791    2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/63850164-7354-4352-a5f8-c3f68d980dfa-bound-sa-token" (OuterVolumeSpecName: "bound-sa-token") pod "63850164-7354-4352-a5f8-c3f68d980dfa" (UID: "63850164-7354-4352-a5f8-c3f68d980dfa"). InnerVolumeSpecName "bound-sa-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 05 00:14:07.478931 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:14:07.478921    2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/63850164-7354-4352-a5f8-c3f68d980dfa-image-registry-private-configuration" (OuterVolumeSpecName: "image-registry-private-configuration") pod "63850164-7354-4352-a5f8-c3f68d980dfa" (UID: "63850164-7354-4352-a5f8-c3f68d980dfa"). InnerVolumeSpecName "image-registry-private-configuration". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 05 00:14:07.479029 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:14:07.479004    2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/63850164-7354-4352-a5f8-c3f68d980dfa-registry-tls" (OuterVolumeSpecName: "registry-tls") pod "63850164-7354-4352-a5f8-c3f68d980dfa" (UID: "63850164-7354-4352-a5f8-c3f68d980dfa"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 05 00:14:07.484670 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:14:07.484651    2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/63850164-7354-4352-a5f8-c3f68d980dfa-ca-trust-extracted" (OuterVolumeSpecName: "ca-trust-extracted") pod "63850164-7354-4352-a5f8-c3f68d980dfa" (UID: "63850164-7354-4352-a5f8-c3f68d980dfa"). InnerVolumeSpecName "ca-trust-extracted". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 05 00:14:07.577646 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:14:07.577612    2563 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/63850164-7354-4352-a5f8-c3f68d980dfa-registry-tls\") on node \"ip-10-0-142-202.ec2.internal\" DevicePath \"\""
Jul 05 00:14:07.577646 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:14:07.577640    2563 reconciler_common.go:299] "Volume detached for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/63850164-7354-4352-a5f8-c3f68d980dfa-ca-trust-extracted\") on node \"ip-10-0-142-202.ec2.internal\" DevicePath \"\""
Jul 05 00:14:07.577646 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:14:07.577651    2563 reconciler_common.go:299] "Volume detached for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/63850164-7354-4352-a5f8-c3f68d980dfa-image-registry-private-configuration\") on node \"ip-10-0-142-202.ec2.internal\" DevicePath \"\""
Jul 05 00:14:07.577871 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:14:07.577660    2563 reconciler_common.go:299] "Volume detached for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/63850164-7354-4352-a5f8-c3f68d980dfa-registry-certificates\") on node \"ip-10-0-142-202.ec2.internal\" DevicePath \"\""
Jul 05 00:14:07.577871 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:14:07.577668    2563 reconciler_common.go:299] "Volume detached for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/63850164-7354-4352-a5f8-c3f68d980dfa-installation-pull-secrets\") on node \"ip-10-0-142-202.ec2.internal\" DevicePath \"\""
Jul 05 00:14:07.577871 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:14:07.577677    2563 reconciler_common.go:299] "Volume detached for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/63850164-7354-4352-a5f8-c3f68d980dfa-bound-sa-token\") on node \"ip-10-0-142-202.ec2.internal\" DevicePath \"\""
Jul 05 00:14:07.577871 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:14:07.577686    2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-58g7w\" (UniqueName: \"kubernetes.io/projected/63850164-7354-4352-a5f8-c3f68d980dfa-kube-api-access-58g7w\") on node \"ip-10-0-142-202.ec2.internal\" DevicePath \"\""
Jul 05 00:14:07.577871 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:14:07.577694    2563 reconciler_common.go:299] "Volume detached for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/63850164-7354-4352-a5f8-c3f68d980dfa-trusted-ca\") on node \"ip-10-0-142-202.ec2.internal\" DevicePath \"\""
Jul 05 00:14:08.229251 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:14:08.229200    2563 generic.go:356] "Generic (PLEG): container finished" podID="63850164-7354-4352-a5f8-c3f68d980dfa" containerID="434072f1ea5bdf1b156b820dd87baaa59f55f97554a463d7846aa7d325ffb080" exitCode=0
Jul 05 00:14:08.229857 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:14:08.229251    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-5884d96665-mcl4l" event={"ID":"63850164-7354-4352-a5f8-c3f68d980dfa","Type":"ContainerDied","Data":"434072f1ea5bdf1b156b820dd87baaa59f55f97554a463d7846aa7d325ffb080"}
Jul 05 00:14:08.229857 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:14:08.229285    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-5884d96665-mcl4l"
Jul 05 00:14:08.229857 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:14:08.229329    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-5884d96665-mcl4l" event={"ID":"63850164-7354-4352-a5f8-c3f68d980dfa","Type":"ContainerDied","Data":"a825ba259aa97b37e60045cad4b6ffb242ea2c7b604a0867bc70bcbf6e760b6b"}
Jul 05 00:14:08.229857 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:14:08.229351    2563 scope.go:117] "RemoveContainer" containerID="434072f1ea5bdf1b156b820dd87baaa59f55f97554a463d7846aa7d325ffb080"
Jul 05 00:14:08.231301 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:14:08.231271    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"d4aa743c-f71b-4023-97bd-05452c4e4e15","Type":"ContainerStarted","Data":"9909d1a59ce1b194b709d33f2cf20019fe9eeb3bc3a91f851f4a915f5d3f848e"}
Jul 05 00:14:08.285043 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:14:08.285023    2563 scope.go:117] "RemoveContainer" containerID="434072f1ea5bdf1b156b820dd87baaa59f55f97554a463d7846aa7d325ffb080"
Jul 05 00:14:08.285591 ip-10-0-142-202 kubenswrapper[2563]: E0705 00:14:08.285569    2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"434072f1ea5bdf1b156b820dd87baaa59f55f97554a463d7846aa7d325ffb080\": container with ID starting with 434072f1ea5bdf1b156b820dd87baaa59f55f97554a463d7846aa7d325ffb080 not found: ID does not exist" containerID="434072f1ea5bdf1b156b820dd87baaa59f55f97554a463d7846aa7d325ffb080"
Jul 05 00:14:08.285658 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:14:08.285599    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"434072f1ea5bdf1b156b820dd87baaa59f55f97554a463d7846aa7d325ffb080"} err="failed to get container status \"434072f1ea5bdf1b156b820dd87baaa59f55f97554a463d7846aa7d325ffb080\": rpc error: code = NotFound desc = could not find container \"434072f1ea5bdf1b156b820dd87baaa59f55f97554a463d7846aa7d325ffb080\": container with ID starting with 434072f1ea5bdf1b156b820dd87baaa59f55f97554a463d7846aa7d325ffb080 not found: ID does not exist"
Jul 05 00:14:08.296144 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:14:08.296118    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-5884d96665-mcl4l"]
Jul 05 00:14:08.304109 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:14:08.304092    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-image-registry/image-registry-5884d96665-mcl4l"]
Jul 05 00:14:09.125512 ip-10-0-142-202 kubenswrapper[2563]: E0705 00:14:09.125463    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[metrics-certs], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-multus/network-metrics-daemon-5fn94" podUID="c085d4be-fcda-4931-a018-02cbaca17c59"
Jul 05 00:14:09.237013 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:14:09.236979    2563 generic.go:356] "Generic (PLEG): container finished" podID="d4aa743c-f71b-4023-97bd-05452c4e4e15" containerID="9909d1a59ce1b194b709d33f2cf20019fe9eeb3bc3a91f851f4a915f5d3f848e" exitCode=0
Jul 05 00:14:09.237391 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:14:09.237031    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"d4aa743c-f71b-4023-97bd-05452c4e4e15","Type":"ContainerDied","Data":"9909d1a59ce1b194b709d33f2cf20019fe9eeb3bc3a91f851f4a915f5d3f848e"}
Jul 05 00:14:10.127492 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:14:10.127456    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="63850164-7354-4352-a5f8-c3f68d980dfa" path="/var/lib/kubelet/pods/63850164-7354-4352-a5f8-c3f68d980dfa/volumes"
Jul 05 00:14:10.190658 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:14:10.190628    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-snsng"
Jul 05 00:14:12.892988 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:14:12.892941    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-7sxq4_b96bfb4d-2fd3-4be3-9064-e24efeb904f4/ovn-acl-logging/0.log"
Jul 05 00:14:12.894057 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:14:12.894034    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-7sxq4_b96bfb4d-2fd3-4be3-9064-e24efeb904f4/ovn-acl-logging/0.log"
Jul 05 00:14:12.897529 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:14:12.897503    2563 kubelet.go:1648] "Image garbage collection succeeded"
Jul 05 00:14:13.254643 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:14:13.254565    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"d4aa743c-f71b-4023-97bd-05452c4e4e15","Type":"ContainerStarted","Data":"858bc5fc5defff06bc449feb6dd4a13d605c3b3894c5b10872e9e0a04e35797e"}
Jul 05 00:14:13.254643 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:14:13.254625    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"d4aa743c-f71b-4023-97bd-05452c4e4e15","Type":"ContainerStarted","Data":"f0562f69f45c99d07fbe7209ec6f38e0492cba081e0347006ee82fdcb2815150"}
Jul 05 00:14:16.270783 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:14:16.270740    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"d4aa743c-f71b-4023-97bd-05452c4e4e15","Type":"ContainerStarted","Data":"cd68a68a8365e2629e1d0d4a913cc7a6f84086d3ef721633f50aecb14f1efbb5"}
Jul 05 00:14:16.271212 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:14:16.270831    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"d4aa743c-f71b-4023-97bd-05452c4e4e15","Type":"ContainerStarted","Data":"4d8d45938b90e095514a24e1d33d9cab39ef3f5046220f82392a1db2c4f03631"}
Jul 05 00:14:16.271212 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:14:16.270874    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"d4aa743c-f71b-4023-97bd-05452c4e4e15","Type":"ContainerStarted","Data":"67cf95f5aedd2985e563ab59baf0f8b73abcd9ea1af1ef643fd552bcd779ce2b"}
Jul 05 00:14:16.271212 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:14:16.270919    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"d4aa743c-f71b-4023-97bd-05452c4e4e15","Type":"ContainerStarted","Data":"49332755a2eced22ca04fc0f0f11d21844c941fc4a069591a5c6ba87e6041389"}
Jul 05 00:14:16.295501 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:14:16.295450    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-k8s-0" podStartSLOduration=3.2271634909999998 podStartE2EDuration="10.295429347s" totalImagesPullingTime="7.052023286s" totalInitContainerRuntime="16.24257ms" isStatefulPod=false podCreationTimestamp="2026-07-05 00:14:06 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-05 00:14:16.294565718 +0000 UTC m=+304.636894781" watchObservedRunningTime="2026-07-05 00:14:16.295429347 +0000 UTC m=+304.637758406"
Jul 05 00:14:16.945197 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:14:16.945161    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-k8s-0"
Jul 05 00:14:21.121037 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:14:21.121001    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-5fn94"
Jul 05 00:14:44.373926 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:14:44.373123    2563 generic.go:356] "Generic (PLEG): container finished" podID="213b2b3a-180f-4560-b670-6bcf420899e8" containerID="caeb50a659c9f23083667f2fef77b8fac2bf7280afdeb05081f891a912c8a447" exitCode=0
Jul 05 00:14:44.373926 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:14:44.373253    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-7rv4m" event={"ID":"213b2b3a-180f-4560-b670-6bcf420899e8","Type":"ContainerDied","Data":"caeb50a659c9f23083667f2fef77b8fac2bf7280afdeb05081f891a912c8a447"}
Jul 05 00:14:44.373926 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:14:44.373583    2563 scope.go:117] "RemoveContainer" containerID="caeb50a659c9f23083667f2fef77b8fac2bf7280afdeb05081f891a912c8a447"
Jul 05 00:14:45.378207 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:14:45.378175    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-7rv4m" event={"ID":"213b2b3a-180f-4560-b670-6bcf420899e8","Type":"ContainerStarted","Data":"230825fe801d115cbbe18bd0d29e1e9506b928adb00a3cd08d6d929c691e6f8f"}
Jul 05 00:15:06.945117 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:15:06.945027    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/prometheus-k8s-0"
Jul 05 00:15:06.964935 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:15:06.964912    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/prometheus-k8s-0"
Jul 05 00:15:07.473446 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:15:07.473422    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-k8s-0"
Jul 05 00:15:25.006502 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:15:25.006466    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c085d4be-fcda-4931-a018-02cbaca17c59-metrics-certs\") pod \"network-metrics-daemon-5fn94\" (UID: \"c085d4be-fcda-4931-a018-02cbaca17c59\") " pod="openshift-multus/network-metrics-daemon-5fn94"
Jul 05 00:15:25.008776 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:15:25.008750    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c085d4be-fcda-4931-a018-02cbaca17c59-metrics-certs\") pod \"network-metrics-daemon-5fn94\" (UID: \"c085d4be-fcda-4931-a018-02cbaca17c59\") " pod="openshift-multus/network-metrics-daemon-5fn94"
Jul 05 00:15:25.023399 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:15:25.023374    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-m9r6w\""
Jul 05 00:15:25.032726 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:15:25.032709    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-5fn94"
Jul 05 00:15:25.155468 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:15:25.155434    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-5fn94"]
Jul 05 00:15:25.158514 ip-10-0-142-202 kubenswrapper[2563]: W0705 00:15:25.158485    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc085d4be_fcda_4931_a018_02cbaca17c59.slice/crio-ae07f508e9f9d4d5fc8b6f66de5042a1c0563eff1b63627b92191a1119c4c489 WatchSource:0}: Error finding container ae07f508e9f9d4d5fc8b6f66de5042a1c0563eff1b63627b92191a1119c4c489: Status 404 returned error can't find the container with id ae07f508e9f9d4d5fc8b6f66de5042a1c0563eff1b63627b92191a1119c4c489
Jul 05 00:15:25.160662 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:15:25.160642    2563 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 05 00:15:25.522762 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:15:25.522723    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-5fn94" event={"ID":"c085d4be-fcda-4931-a018-02cbaca17c59","Type":"ContainerStarted","Data":"ae07f508e9f9d4d5fc8b6f66de5042a1c0563eff1b63627b92191a1119c4c489"}
Jul 05 00:15:26.528374 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:15:26.528341    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-5fn94" event={"ID":"c085d4be-fcda-4931-a018-02cbaca17c59","Type":"ContainerStarted","Data":"ece5c86ae270df289161881c1b8914a2678cdadd35f3a5f2aaec55b3088c2f4b"}
Jul 05 00:15:26.528742 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:15:26.528392    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-5fn94" event={"ID":"c085d4be-fcda-4931-a018-02cbaca17c59","Type":"ContainerStarted","Data":"ba6bd39be389ffca433779d3cb9b5e9197b6557f2fa00c0c42bfbb9696711506"}
Jul 05 00:15:26.542281 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:15:26.542232    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-5fn94" podStartSLOduration=372.601915427 podStartE2EDuration="6m13.542196338s" totalImagesPullingTime="940.280911ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-05 00:09:13 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-05 00:15:26.541024464 +0000 UTC m=+374.883353550" watchObservedRunningTime="2026-07-05 00:15:26.542196338 +0000 UTC m=+374.884525399"
Jul 05 00:18:41.860864 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:18:41.860823    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-lws-operator/lws-controller-manager-7dcb9fcb66-btmss"]
Jul 05 00:18:41.861290 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:18:41.861086    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="63850164-7354-4352-a5f8-c3f68d980dfa" containerName="registry"
Jul 05 00:18:41.861290 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:18:41.861098    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="63850164-7354-4352-a5f8-c3f68d980dfa" containerName="registry"
Jul 05 00:18:41.861290 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:18:41.861154    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="63850164-7354-4352-a5f8-c3f68d980dfa" containerName="registry"
Jul 05 00:18:41.863732 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:18:41.863715    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-7dcb9fcb66-btmss"
Jul 05 00:18:41.866265 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:18:41.866187    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"kube-root-ca.crt\""
Jul 05 00:18:41.866265 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:18:41.866194    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"metrics-server-cert\""
Jul 05 00:18:41.866486 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:18:41.866278    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"openshift-service-ca.crt\""
Jul 05 00:18:41.866486 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:18:41.866293    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"lws-controller-manager-dockercfg-hctcz\""
Jul 05 00:18:41.866486 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:18:41.866279    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"lws-manager-config\""
Jul 05 00:18:41.866486 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:18:41.866338    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"webhook-server-cert\""
Jul 05 00:18:41.871641 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:18:41.871622    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-7dcb9fcb66-btmss"]
Jul 05 00:18:41.996028 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:18:41.995993    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6rpts\" (UniqueName: \"kubernetes.io/projected/4daa43eb-7628-421d-96e8-fbb8e3051e5b-kube-api-access-6rpts\") pod \"lws-controller-manager-7dcb9fcb66-btmss\" (UID: \"4daa43eb-7628-421d-96e8-fbb8e3051e5b\") " pod="openshift-lws-operator/lws-controller-manager-7dcb9fcb66-btmss"
Jul 05 00:18:41.996182 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:18:41.996060    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/4daa43eb-7628-421d-96e8-fbb8e3051e5b-manager-config\") pod \"lws-controller-manager-7dcb9fcb66-btmss\" (UID: \"4daa43eb-7628-421d-96e8-fbb8e3051e5b\") " pod="openshift-lws-operator/lws-controller-manager-7dcb9fcb66-btmss"
Jul 05 00:18:41.996255 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:18:41.996211    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/4daa43eb-7628-421d-96e8-fbb8e3051e5b-cert\") pod \"lws-controller-manager-7dcb9fcb66-btmss\" (UID: \"4daa43eb-7628-421d-96e8-fbb8e3051e5b\") " pod="openshift-lws-operator/lws-controller-manager-7dcb9fcb66-btmss"
Jul 05 00:18:41.996300 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:18:41.996286    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/4daa43eb-7628-421d-96e8-fbb8e3051e5b-metrics-cert\") pod \"lws-controller-manager-7dcb9fcb66-btmss\" (UID: \"4daa43eb-7628-421d-96e8-fbb8e3051e5b\") " pod="openshift-lws-operator/lws-controller-manager-7dcb9fcb66-btmss"
Jul 05 00:18:42.097290 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:18:42.097248    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6rpts\" (UniqueName: \"kubernetes.io/projected/4daa43eb-7628-421d-96e8-fbb8e3051e5b-kube-api-access-6rpts\") pod \"lws-controller-manager-7dcb9fcb66-btmss\" (UID: \"4daa43eb-7628-421d-96e8-fbb8e3051e5b\") " pod="openshift-lws-operator/lws-controller-manager-7dcb9fcb66-btmss"
Jul 05 00:18:42.097496 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:18:42.097307    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/4daa43eb-7628-421d-96e8-fbb8e3051e5b-manager-config\") pod \"lws-controller-manager-7dcb9fcb66-btmss\" (UID: \"4daa43eb-7628-421d-96e8-fbb8e3051e5b\") " pod="openshift-lws-operator/lws-controller-manager-7dcb9fcb66-btmss"
Jul 05 00:18:42.097496 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:18:42.097375    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/4daa43eb-7628-421d-96e8-fbb8e3051e5b-cert\") pod \"lws-controller-manager-7dcb9fcb66-btmss\" (UID: \"4daa43eb-7628-421d-96e8-fbb8e3051e5b\") " pod="openshift-lws-operator/lws-controller-manager-7dcb9fcb66-btmss"
Jul 05 00:18:42.097496 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:18:42.097408    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/4daa43eb-7628-421d-96e8-fbb8e3051e5b-metrics-cert\") pod \"lws-controller-manager-7dcb9fcb66-btmss\" (UID: \"4daa43eb-7628-421d-96e8-fbb8e3051e5b\") " pod="openshift-lws-operator/lws-controller-manager-7dcb9fcb66-btmss"
Jul 05 00:18:42.097905 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:18:42.097875    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/4daa43eb-7628-421d-96e8-fbb8e3051e5b-manager-config\") pod \"lws-controller-manager-7dcb9fcb66-btmss\" (UID: \"4daa43eb-7628-421d-96e8-fbb8e3051e5b\") " pod="openshift-lws-operator/lws-controller-manager-7dcb9fcb66-btmss"
Jul 05 00:18:42.099786 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:18:42.099769    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/4daa43eb-7628-421d-96e8-fbb8e3051e5b-metrics-cert\") pod \"lws-controller-manager-7dcb9fcb66-btmss\" (UID: \"4daa43eb-7628-421d-96e8-fbb8e3051e5b\") " pod="openshift-lws-operator/lws-controller-manager-7dcb9fcb66-btmss"
Jul 05 00:18:42.099856 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:18:42.099825    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/4daa43eb-7628-421d-96e8-fbb8e3051e5b-cert\") pod \"lws-controller-manager-7dcb9fcb66-btmss\" (UID: \"4daa43eb-7628-421d-96e8-fbb8e3051e5b\") " pod="openshift-lws-operator/lws-controller-manager-7dcb9fcb66-btmss"
Jul 05 00:18:42.110383 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:18:42.110355    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-6rpts\" (UniqueName: \"kubernetes.io/projected/4daa43eb-7628-421d-96e8-fbb8e3051e5b-kube-api-access-6rpts\") pod \"lws-controller-manager-7dcb9fcb66-btmss\" (UID: \"4daa43eb-7628-421d-96e8-fbb8e3051e5b\") " pod="openshift-lws-operator/lws-controller-manager-7dcb9fcb66-btmss"
Jul 05 00:18:42.173403 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:18:42.173338    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-7dcb9fcb66-btmss"
Jul 05 00:18:42.301888 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:18:42.301862    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-7dcb9fcb66-btmss"]
Jul 05 00:18:43.228397 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:18:43.228321    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-7dcb9fcb66-btmss" event={"ID":"4daa43eb-7628-421d-96e8-fbb8e3051e5b","Type":"ContainerStarted","Data":"f785b844763cf4cde0c56f469993f8af68a395429834de9033182441ce816c1c"}
Jul 05 00:18:45.237582 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:18:45.237498    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-7dcb9fcb66-btmss" event={"ID":"4daa43eb-7628-421d-96e8-fbb8e3051e5b","Type":"ContainerStarted","Data":"764a42c272109c22ea2da19e9440b92f4b160151b0dc74fa9bdad60b59a444a4"}
Jul 05 00:18:45.237990 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:18:45.237633    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-lws-operator/lws-controller-manager-7dcb9fcb66-btmss"
Jul 05 00:18:45.251019 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:18:45.250886    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-lws-operator/lws-controller-manager-7dcb9fcb66-btmss" podStartSLOduration=1.688844056 podStartE2EDuration="4.25086668s" totalImagesPullingTime="2.562022624s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-05 00:18:41 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-05 00:18:45.250599981 +0000 UTC m=+573.592929055" watchObservedRunningTime="2026-07-05 00:18:45.25086668 +0000 UTC m=+573.593195740"
Jul 05 00:18:56.243099 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:18:56.243067    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-lws-operator/lws-controller-manager-7dcb9fcb66-btmss"
Jul 05 00:19:02.563710 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:19:02.563652    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-674c4bd948-9brxw"]
Jul 05 00:19:02.567350 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:19:02.567305    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-674c4bd948-9brxw"
Jul 05 00:19:02.570335 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:19:02.570315    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-dockercfg-5n2nm\""
Jul 05 00:19:02.570335 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:19:02.570325    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-service-cert\""
Jul 05 00:19:02.570946 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:19:02.570926    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\""
Jul 05 00:19:02.571284 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:19:02.571266    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\""
Jul 05 00:19:02.571471 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:19:02.571456    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-webhook-cert\""
Jul 05 00:19:02.581709 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:19:02.581688    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-674c4bd948-9brxw"]
Jul 05 00:19:02.648283 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:19:02.648259    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/07bb806b-cabf-4e75-8758-19c65226e9ad-webhook-cert\") pod \"opendatahub-operator-controller-manager-674c4bd948-9brxw\" (UID: \"07bb806b-cabf-4e75-8758-19c65226e9ad\") " pod="opendatahub/opendatahub-operator-controller-manager-674c4bd948-9brxw"
Jul 05 00:19:02.648417 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:19:02.648292    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mq8bq\" (UniqueName: \"kubernetes.io/projected/07bb806b-cabf-4e75-8758-19c65226e9ad-kube-api-access-mq8bq\") pod \"opendatahub-operator-controller-manager-674c4bd948-9brxw\" (UID: \"07bb806b-cabf-4e75-8758-19c65226e9ad\") " pod="opendatahub/opendatahub-operator-controller-manager-674c4bd948-9brxw"
Jul 05 00:19:02.648417 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:19:02.648315    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/07bb806b-cabf-4e75-8758-19c65226e9ad-tmp\") pod \"opendatahub-operator-controller-manager-674c4bd948-9brxw\" (UID: \"07bb806b-cabf-4e75-8758-19c65226e9ad\") " pod="opendatahub/opendatahub-operator-controller-manager-674c4bd948-9brxw"
Jul 05 00:19:02.648417 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:19:02.648387    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/07bb806b-cabf-4e75-8758-19c65226e9ad-manifests\") pod \"opendatahub-operator-controller-manager-674c4bd948-9brxw\" (UID: \"07bb806b-cabf-4e75-8758-19c65226e9ad\") " pod="opendatahub/opendatahub-operator-controller-manager-674c4bd948-9brxw"
Jul 05 00:19:02.648539 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:19:02.648432    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/07bb806b-cabf-4e75-8758-19c65226e9ad-apiservice-cert\") pod \"opendatahub-operator-controller-manager-674c4bd948-9brxw\" (UID: \"07bb806b-cabf-4e75-8758-19c65226e9ad\") " pod="opendatahub/opendatahub-operator-controller-manager-674c4bd948-9brxw"
Jul 05 00:19:02.749296 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:19:02.749263    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/07bb806b-cabf-4e75-8758-19c65226e9ad-manifests\") pod \"opendatahub-operator-controller-manager-674c4bd948-9brxw\" (UID: \"07bb806b-cabf-4e75-8758-19c65226e9ad\") " pod="opendatahub/opendatahub-operator-controller-manager-674c4bd948-9brxw"
Jul 05 00:19:02.749454 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:19:02.749305    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/07bb806b-cabf-4e75-8758-19c65226e9ad-apiservice-cert\") pod \"opendatahub-operator-controller-manager-674c4bd948-9brxw\" (UID: \"07bb806b-cabf-4e75-8758-19c65226e9ad\") " pod="opendatahub/opendatahub-operator-controller-manager-674c4bd948-9brxw"
Jul 05 00:19:02.749454 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:19:02.749421    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/07bb806b-cabf-4e75-8758-19c65226e9ad-webhook-cert\") pod \"opendatahub-operator-controller-manager-674c4bd948-9brxw\" (UID: \"07bb806b-cabf-4e75-8758-19c65226e9ad\") " pod="opendatahub/opendatahub-operator-controller-manager-674c4bd948-9brxw"
Jul 05 00:19:02.749530 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:19:02.749476    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mq8bq\" (UniqueName: \"kubernetes.io/projected/07bb806b-cabf-4e75-8758-19c65226e9ad-kube-api-access-mq8bq\") pod \"opendatahub-operator-controller-manager-674c4bd948-9brxw\" (UID: \"07bb806b-cabf-4e75-8758-19c65226e9ad\") " pod="opendatahub/opendatahub-operator-controller-manager-674c4bd948-9brxw"
Jul 05 00:19:02.749530 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:19:02.749516    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/07bb806b-cabf-4e75-8758-19c65226e9ad-tmp\") pod \"opendatahub-operator-controller-manager-674c4bd948-9brxw\" (UID: \"07bb806b-cabf-4e75-8758-19c65226e9ad\") " pod="opendatahub/opendatahub-operator-controller-manager-674c4bd948-9brxw"
Jul 05 00:19:02.749633 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:19:02.749616    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/07bb806b-cabf-4e75-8758-19c65226e9ad-manifests\") pod \"opendatahub-operator-controller-manager-674c4bd948-9brxw\" (UID: \"07bb806b-cabf-4e75-8758-19c65226e9ad\") " pod="opendatahub/opendatahub-operator-controller-manager-674c4bd948-9brxw"
Jul 05 00:19:02.751652 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:19:02.751624    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/07bb806b-cabf-4e75-8758-19c65226e9ad-tmp\") pod \"opendatahub-operator-controller-manager-674c4bd948-9brxw\" (UID: \"07bb806b-cabf-4e75-8758-19c65226e9ad\") " pod="opendatahub/opendatahub-operator-controller-manager-674c4bd948-9brxw"
Jul 05 00:19:02.751794 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:19:02.751697    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/07bb806b-cabf-4e75-8758-19c65226e9ad-apiservice-cert\") pod \"opendatahub-operator-controller-manager-674c4bd948-9brxw\" (UID: \"07bb806b-cabf-4e75-8758-19c65226e9ad\") " pod="opendatahub/opendatahub-operator-controller-manager-674c4bd948-9brxw"
Jul 05 00:19:02.751864 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:19:02.751822    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/07bb806b-cabf-4e75-8758-19c65226e9ad-webhook-cert\") pod \"opendatahub-operator-controller-manager-674c4bd948-9brxw\" (UID: \"07bb806b-cabf-4e75-8758-19c65226e9ad\") " pod="opendatahub/opendatahub-operator-controller-manager-674c4bd948-9brxw"
Jul 05 00:19:02.756134 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:19:02.756112    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-mq8bq\" (UniqueName: \"kubernetes.io/projected/07bb806b-cabf-4e75-8758-19c65226e9ad-kube-api-access-mq8bq\") pod \"opendatahub-operator-controller-manager-674c4bd948-9brxw\" (UID: \"07bb806b-cabf-4e75-8758-19c65226e9ad\") " pod="opendatahub/opendatahub-operator-controller-manager-674c4bd948-9brxw"
Jul 05 00:19:02.877035 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:19:02.876960    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-674c4bd948-9brxw"
Jul 05 00:19:03.017944 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:19:03.017860    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-674c4bd948-9brxw"]
Jul 05 00:19:03.022742 ip-10-0-142-202 kubenswrapper[2563]: W0705 00:19:03.022702    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod07bb806b_cabf_4e75_8758_19c65226e9ad.slice/crio-101db6d09f220602d0ee570db7fe9ed03202e61b1e1ae3007809d0f5a3f78d8d WatchSource:0}: Error finding container 101db6d09f220602d0ee570db7fe9ed03202e61b1e1ae3007809d0f5a3f78d8d: Status 404 returned error can't find the container with id 101db6d09f220602d0ee570db7fe9ed03202e61b1e1ae3007809d0f5a3f78d8d
Jul 05 00:19:03.303510 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:19:03.303479    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-674c4bd948-9brxw" event={"ID":"07bb806b-cabf-4e75-8758-19c65226e9ad","Type":"ContainerStarted","Data":"101db6d09f220602d0ee570db7fe9ed03202e61b1e1ae3007809d0f5a3f78d8d"}
Jul 05 00:19:06.317624 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:19:06.317586    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-674c4bd948-9brxw" event={"ID":"07bb806b-cabf-4e75-8758-19c65226e9ad","Type":"ContainerStarted","Data":"43f955cb89a6584210fc71b202d996b0d2bf00d8d6b12ebddeccc3d5d4fc980c"}
Jul 05 00:19:07.322314 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:19:07.322213    2563 generic.go:356] "Generic (PLEG): container finished" podID="07bb806b-cabf-4e75-8758-19c65226e9ad" containerID="43f955cb89a6584210fc71b202d996b0d2bf00d8d6b12ebddeccc3d5d4fc980c" exitCode=0
Jul 05 00:19:07.322314 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:19:07.322258    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-674c4bd948-9brxw" event={"ID":"07bb806b-cabf-4e75-8758-19c65226e9ad","Type":"ContainerDied","Data":"43f955cb89a6584210fc71b202d996b0d2bf00d8d6b12ebddeccc3d5d4fc980c"}
Jul 05 00:19:08.327341 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:19:08.327307    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-674c4bd948-9brxw" event={"ID":"07bb806b-cabf-4e75-8758-19c65226e9ad","Type":"ContainerStarted","Data":"438f44cf22750875b0d9eb57de7c8a4fe332c3850c34db9c832f6bcdea7297d8"}
Jul 05 00:19:08.327715 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:19:08.327394    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/opendatahub-operator-controller-manager-674c4bd948-9brxw"
Jul 05 00:19:08.347554 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:19:08.347507    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/opendatahub-operator-controller-manager-674c4bd948-9brxw" podStartSLOduration=1.9562454 podStartE2EDuration="6.347493402s" totalImagesPullingTime="3.330327876s" totalInitContainerRuntime="1.060920126s" isStatefulPod=false podCreationTimestamp="2026-07-05 00:19:02 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-05 00:19:08.345360209 +0000 UTC m=+596.687689290" watchObservedRunningTime="2026-07-05 00:19:08.347493402 +0000 UTC m=+596.689822462"
Jul 05 00:19:12.918945 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:19:12.918832    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-7sxq4_b96bfb4d-2fd3-4be3-9064-e24efeb904f4/ovn-acl-logging/0.log"
Jul 05 00:19:12.943456 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:19:12.919376    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-7sxq4_b96bfb4d-2fd3-4be3-9064-e24efeb904f4/ovn-acl-logging/0.log"
Jul 05 00:19:19.333550 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:19:19.333522    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/opendatahub-operator-controller-manager-674c4bd948-9brxw"
Jul 05 00:19:31.939406 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:19:31.939374    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/kube-auth-proxy-6bddf4ddb8-mb7vs"]
Jul 05 00:19:31.948642 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:19:31.948622    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-6bddf4ddb8-mb7vs"
Jul 05 00:19:31.950838 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:19:31.950812    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-tls\""
Jul 05 00:19:31.951035 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:19:31.950979    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-dockercfg-nw2x6\""
Jul 05 00:19:31.951981 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:19:31.951650    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\""
Jul 05 00:19:31.951981 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:19:31.951687    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-creds\""
Jul 05 00:19:31.951981 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:19:31.951828    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\""
Jul 05 00:19:31.954107 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:19:31.954086    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-6bddf4ddb8-mb7vs"]
Jul 05 00:19:32.089054 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:19:32.089024    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tkw9l\" (UniqueName: \"kubernetes.io/projected/6c8d44d6-8738-4ab3-8950-f2b687d3026b-kube-api-access-tkw9l\") pod \"kube-auth-proxy-6bddf4ddb8-mb7vs\" (UID: \"6c8d44d6-8738-4ab3-8950-f2b687d3026b\") " pod="openshift-ingress/kube-auth-proxy-6bddf4ddb8-mb7vs"
Jul 05 00:19:32.089258 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:19:32.089062    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/6c8d44d6-8738-4ab3-8950-f2b687d3026b-tls-certs\") pod \"kube-auth-proxy-6bddf4ddb8-mb7vs\" (UID: \"6c8d44d6-8738-4ab3-8950-f2b687d3026b\") " pod="openshift-ingress/kube-auth-proxy-6bddf4ddb8-mb7vs"
Jul 05 00:19:32.089258 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:19:32.089086    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/6c8d44d6-8738-4ab3-8950-f2b687d3026b-tmp\") pod \"kube-auth-proxy-6bddf4ddb8-mb7vs\" (UID: \"6c8d44d6-8738-4ab3-8950-f2b687d3026b\") " pod="openshift-ingress/kube-auth-proxy-6bddf4ddb8-mb7vs"
Jul 05 00:19:32.189595 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:19:32.189516    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tkw9l\" (UniqueName: \"kubernetes.io/projected/6c8d44d6-8738-4ab3-8950-f2b687d3026b-kube-api-access-tkw9l\") pod \"kube-auth-proxy-6bddf4ddb8-mb7vs\" (UID: \"6c8d44d6-8738-4ab3-8950-f2b687d3026b\") " pod="openshift-ingress/kube-auth-proxy-6bddf4ddb8-mb7vs"
Jul 05 00:19:32.189595 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:19:32.189553    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/6c8d44d6-8738-4ab3-8950-f2b687d3026b-tls-certs\") pod \"kube-auth-proxy-6bddf4ddb8-mb7vs\" (UID: \"6c8d44d6-8738-4ab3-8950-f2b687d3026b\") " pod="openshift-ingress/kube-auth-proxy-6bddf4ddb8-mb7vs"
Jul 05 00:19:32.189595 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:19:32.189573    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/6c8d44d6-8738-4ab3-8950-f2b687d3026b-tmp\") pod \"kube-auth-proxy-6bddf4ddb8-mb7vs\" (UID: \"6c8d44d6-8738-4ab3-8950-f2b687d3026b\") " pod="openshift-ingress/kube-auth-proxy-6bddf4ddb8-mb7vs"
Jul 05 00:19:32.189864 ip-10-0-142-202 kubenswrapper[2563]: E0705 00:19:32.189680    2563 secret.go:189] Couldn't get secret openshift-ingress/kube-auth-proxy-tls: secret "kube-auth-proxy-tls" not found
Jul 05 00:19:32.189864 ip-10-0-142-202 kubenswrapper[2563]: E0705 00:19:32.189758    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6c8d44d6-8738-4ab3-8950-f2b687d3026b-tls-certs podName:6c8d44d6-8738-4ab3-8950-f2b687d3026b nodeName:}" failed. No retries permitted until 2026-07-05 00:19:32.689738211 +0000 UTC m=+621.032067254 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "tls-certs" (UniqueName: "kubernetes.io/secret/6c8d44d6-8738-4ab3-8950-f2b687d3026b-tls-certs") pod "kube-auth-proxy-6bddf4ddb8-mb7vs" (UID: "6c8d44d6-8738-4ab3-8950-f2b687d3026b") : secret "kube-auth-proxy-tls" not found
Jul 05 00:19:32.191804 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:19:32.191777    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/6c8d44d6-8738-4ab3-8950-f2b687d3026b-tmp\") pod \"kube-auth-proxy-6bddf4ddb8-mb7vs\" (UID: \"6c8d44d6-8738-4ab3-8950-f2b687d3026b\") " pod="openshift-ingress/kube-auth-proxy-6bddf4ddb8-mb7vs"
Jul 05 00:19:32.199163 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:19:32.199137    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-tkw9l\" (UniqueName: \"kubernetes.io/projected/6c8d44d6-8738-4ab3-8950-f2b687d3026b-kube-api-access-tkw9l\") pod \"kube-auth-proxy-6bddf4ddb8-mb7vs\" (UID: \"6c8d44d6-8738-4ab3-8950-f2b687d3026b\") " pod="openshift-ingress/kube-auth-proxy-6bddf4ddb8-mb7vs"
Jul 05 00:19:32.693266 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:19:32.693234    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/6c8d44d6-8738-4ab3-8950-f2b687d3026b-tls-certs\") pod \"kube-auth-proxy-6bddf4ddb8-mb7vs\" (UID: \"6c8d44d6-8738-4ab3-8950-f2b687d3026b\") " pod="openshift-ingress/kube-auth-proxy-6bddf4ddb8-mb7vs"
Jul 05 00:19:32.695588 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:19:32.695562    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/6c8d44d6-8738-4ab3-8950-f2b687d3026b-tls-certs\") pod \"kube-auth-proxy-6bddf4ddb8-mb7vs\" (UID: \"6c8d44d6-8738-4ab3-8950-f2b687d3026b\") " pod="openshift-ingress/kube-auth-proxy-6bddf4ddb8-mb7vs"
Jul 05 00:19:32.860058 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:19:32.860026    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-6bddf4ddb8-mb7vs"
Jul 05 00:19:32.997643 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:19:32.997568    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-6bddf4ddb8-mb7vs"]
Jul 05 00:19:33.001752 ip-10-0-142-202 kubenswrapper[2563]: W0705 00:19:33.001719    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6c8d44d6_8738_4ab3_8950_f2b687d3026b.slice/crio-4109e35229e6aadf3beb69cf0cc0ced6b314d42f8e637de40df2e142b2f0c8e7 WatchSource:0}: Error finding container 4109e35229e6aadf3beb69cf0cc0ced6b314d42f8e637de40df2e142b2f0c8e7: Status 404 returned error can't find the container with id 4109e35229e6aadf3beb69cf0cc0ced6b314d42f8e637de40df2e142b2f0c8e7
Jul 05 00:19:33.424528 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:19:33.424496    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-6bddf4ddb8-mb7vs" event={"ID":"6c8d44d6-8738-4ab3-8950-f2b687d3026b","Type":"ContainerStarted","Data":"4109e35229e6aadf3beb69cf0cc0ced6b314d42f8e637de40df2e142b2f0c8e7"}
Jul 05 00:19:36.445108 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:19:36.445071    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-6bddf4ddb8-mb7vs" event={"ID":"6c8d44d6-8738-4ab3-8950-f2b687d3026b","Type":"ContainerStarted","Data":"19c4f421e4900cd21fdc024c0e7628e8f6263be95b95ccbb3b3a11c555c5985f"}
Jul 05 00:19:36.460065 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:19:36.459792    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/kube-auth-proxy-6bddf4ddb8-mb7vs" podStartSLOduration=2.530382697 podStartE2EDuration="5.459775257s" totalImagesPullingTime="2.92939256s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-05 00:19:31 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-05 00:19:36.459283117 +0000 UTC m=+624.801612197" watchObservedRunningTime="2026-07-05 00:19:36.459775257 +0000 UTC m=+624.802104320"
Jul 05 00:21:07.072187 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:21:07.072088    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-nvgkm"]
Jul 05 00:21:07.074551 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:21:07.074527    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-nvgkm"
Jul 05 00:21:07.076571 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:21:07.076551    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\""
Jul 05 00:21:07.076571 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:21:07.076561    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\""
Jul 05 00:21:07.076738 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:21:07.076552    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-controller-manager-dockercfg-r8hkz\""
Jul 05 00:21:07.087717 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:21:07.087690    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-nvgkm"]
Jul 05 00:21:07.147458 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:21:07.147432    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fqj5r\" (UniqueName: \"kubernetes.io/projected/cf6baec2-ef0e-4938-af29-dbd0a02074bb-kube-api-access-fqj5r\") pod \"kuadrant-operator-controller-manager-8684c9ff49-nvgkm\" (UID: \"cf6baec2-ef0e-4938-af29-dbd0a02074bb\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-nvgkm"
Jul 05 00:21:07.147582 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:21:07.147479    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/cf6baec2-ef0e-4938-af29-dbd0a02074bb-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-nvgkm\" (UID: \"cf6baec2-ef0e-4938-af29-dbd0a02074bb\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-nvgkm"
Jul 05 00:21:07.248253 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:21:07.248205    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/cf6baec2-ef0e-4938-af29-dbd0a02074bb-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-nvgkm\" (UID: \"cf6baec2-ef0e-4938-af29-dbd0a02074bb\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-nvgkm"
Jul 05 00:21:07.248389 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:21:07.248296    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fqj5r\" (UniqueName: \"kubernetes.io/projected/cf6baec2-ef0e-4938-af29-dbd0a02074bb-kube-api-access-fqj5r\") pod \"kuadrant-operator-controller-manager-8684c9ff49-nvgkm\" (UID: \"cf6baec2-ef0e-4938-af29-dbd0a02074bb\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-nvgkm"
Jul 05 00:21:07.248595 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:21:07.248577    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/cf6baec2-ef0e-4938-af29-dbd0a02074bb-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-nvgkm\" (UID: \"cf6baec2-ef0e-4938-af29-dbd0a02074bb\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-nvgkm"
Jul 05 00:21:07.257070 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:21:07.257043    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-fqj5r\" (UniqueName: \"kubernetes.io/projected/cf6baec2-ef0e-4938-af29-dbd0a02074bb-kube-api-access-fqj5r\") pod \"kuadrant-operator-controller-manager-8684c9ff49-nvgkm\" (UID: \"cf6baec2-ef0e-4938-af29-dbd0a02074bb\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-nvgkm"
Jul 05 00:21:07.383979 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:21:07.383902    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-nvgkm"
Jul 05 00:21:07.510431 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:21:07.510204    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-nvgkm"]
Jul 05 00:21:07.513349 ip-10-0-142-202 kubenswrapper[2563]: W0705 00:21:07.513320    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcf6baec2_ef0e_4938_af29_dbd0a02074bb.slice/crio-2d060baee7295d55ab7509bbdd36608e3617db3e4030824b24fd7266da82e00f WatchSource:0}: Error finding container 2d060baee7295d55ab7509bbdd36608e3617db3e4030824b24fd7266da82e00f: Status 404 returned error can't find the container with id 2d060baee7295d55ab7509bbdd36608e3617db3e4030824b24fd7266da82e00f
Jul 05 00:21:07.515716 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:21:07.515700    2563 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 05 00:21:07.822506 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:21:07.822468    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-nvgkm" event={"ID":"cf6baec2-ef0e-4938-af29-dbd0a02074bb","Type":"ContainerStarted","Data":"2d060baee7295d55ab7509bbdd36608e3617db3e4030824b24fd7266da82e00f"}
Jul 05 00:21:14.856007 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:21:14.855965    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-nvgkm" event={"ID":"cf6baec2-ef0e-4938-af29-dbd0a02074bb","Type":"ContainerStarted","Data":"1be8ea0310633ea8f855bb4a17496714e6e7a5e3b89c32dd720b7692f944b6c8"}
Jul 05 00:21:14.856404 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:21:14.856105    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-nvgkm"
Jul 05 00:21:14.875201 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:21:14.875147    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-nvgkm" podStartSLOduration=1.481462238 podStartE2EDuration="7.87512957s" totalImagesPullingTime="6.393667332s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-05 00:21:07 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-05 00:21:14.872901002 +0000 UTC m=+723.215230083" watchObservedRunningTime="2026-07-05 00:21:14.87512957 +0000 UTC m=+723.217458630"
Jul 05 00:21:25.862918 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:21:25.862888    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-nvgkm"
Jul 05 00:21:27.547655 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:21:27.547625    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-nvgkm"]
Jul 05 00:21:27.548105 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:21:27.547818    2563 kuberuntime_container.go:872] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-nvgkm" podUID="cf6baec2-ef0e-4938-af29-dbd0a02074bb" containerName="manager" containerID="cri-o://1be8ea0310633ea8f855bb4a17496714e6e7a5e3b89c32dd720b7692f944b6c8" gracePeriod=2
Jul 05 00:21:27.558113 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:21:27.558088    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-nvgkm"]
Jul 05 00:21:27.560355 ip-10-0-142-202 kubenswrapper[2563]: E0705 00:21:27.560317    2563 status_manager.go:1041] "Failed to update status for pod" err="failed to patch status \"{\\\"metadata\\\":{\\\"uid\\\":\\\"cf6baec2-ef0e-4938-af29-dbd0a02074bb\\\"},\\\"status\\\":{\\\"$setElementOrder/conditions\\\":[{\\\"type\\\":\\\"PodReadyToStartContainers\\\"},{\\\"type\\\":\\\"Initialized\\\"},{\\\"type\\\":\\\"Ready\\\"},{\\\"type\\\":\\\"ContainersReady\\\"},{\\\"type\\\":\\\"PodScheduled\\\"}],\\\"conditions\\\":[{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"PodReadyToStartContainers\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"Initialized\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"Ready\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"ContainersReady\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"PodScheduled\\\"}],\\\"observedGeneration\\\":2}}\" for pod \"kuadrant-system\"/\"kuadrant-operator-controller-manager-8684c9ff49-nvgkm\": pods \"kuadrant-operator-controller-manager-8684c9ff49-nvgkm\" not found" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-nvgkm"
Jul 05 00:21:27.567540 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:21:27.567516    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-jcx5n"]
Jul 05 00:21:27.567814 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:21:27.567802    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="cf6baec2-ef0e-4938-af29-dbd0a02074bb" containerName="manager"
Jul 05 00:21:27.567814 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:21:27.567814    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="cf6baec2-ef0e-4938-af29-dbd0a02074bb" containerName="manager"
Jul 05 00:21:27.567929 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:21:27.567865    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="cf6baec2-ef0e-4938-af29-dbd0a02074bb" containerName="manager"
Jul 05 00:21:27.569818 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:21:27.569799    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-jcx5n"
Jul 05 00:21:27.589950 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:21:27.589920    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-jcx5n"]
Jul 05 00:21:27.602978 ip-10-0-142-202 kubenswrapper[2563]: E0705 00:21:27.602935    2563 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-nvgkm\" is forbidden: User \"system:node:ip-10-0-142-202.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-142-202.ec2.internal' and this object" podUID="cf6baec2-ef0e-4938-af29-dbd0a02074bb" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-nvgkm"
Jul 05 00:21:27.723869 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:21:27.723833    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/7a1c52c9-4ea1-4bea-b4d4-d5f51ac89acc-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-jcx5n\" (UID: \"7a1c52c9-4ea1-4bea-b4d4-d5f51ac89acc\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-jcx5n"
Jul 05 00:21:27.723988 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:21:27.723945    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7dpkl\" (UniqueName: \"kubernetes.io/projected/7a1c52c9-4ea1-4bea-b4d4-d5f51ac89acc-kube-api-access-7dpkl\") pod \"kuadrant-operator-controller-manager-8684c9ff49-jcx5n\" (UID: \"7a1c52c9-4ea1-4bea-b4d4-d5f51ac89acc\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-jcx5n"
Jul 05 00:21:27.776834 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:21:27.776798    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-nvgkm"
Jul 05 00:21:27.778791 ip-10-0-142-202 kubenswrapper[2563]: E0705 00:21:27.778765    2563 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-nvgkm\" is forbidden: User \"system:node:ip-10-0-142-202.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-142-202.ec2.internal' and this object" podUID="cf6baec2-ef0e-4938-af29-dbd0a02074bb" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-nvgkm"
Jul 05 00:21:27.825180 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:21:27.825122    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7dpkl\" (UniqueName: \"kubernetes.io/projected/7a1c52c9-4ea1-4bea-b4d4-d5f51ac89acc-kube-api-access-7dpkl\") pod \"kuadrant-operator-controller-manager-8684c9ff49-jcx5n\" (UID: \"7a1c52c9-4ea1-4bea-b4d4-d5f51ac89acc\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-jcx5n"
Jul 05 00:21:27.825305 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:21:27.825180    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/7a1c52c9-4ea1-4bea-b4d4-d5f51ac89acc-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-jcx5n\" (UID: \"7a1c52c9-4ea1-4bea-b4d4-d5f51ac89acc\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-jcx5n"
Jul 05 00:21:27.825507 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:21:27.825490    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/7a1c52c9-4ea1-4bea-b4d4-d5f51ac89acc-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-jcx5n\" (UID: \"7a1c52c9-4ea1-4bea-b4d4-d5f51ac89acc\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-jcx5n"
Jul 05 00:21:27.832650 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:21:27.832620    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-7dpkl\" (UniqueName: \"kubernetes.io/projected/7a1c52c9-4ea1-4bea-b4d4-d5f51ac89acc-kube-api-access-7dpkl\") pod \"kuadrant-operator-controller-manager-8684c9ff49-jcx5n\" (UID: \"7a1c52c9-4ea1-4bea-b4d4-d5f51ac89acc\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-jcx5n"
Jul 05 00:21:27.908875 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:21:27.908841    2563 generic.go:356] "Generic (PLEG): container finished" podID="cf6baec2-ef0e-4938-af29-dbd0a02074bb" containerID="1be8ea0310633ea8f855bb4a17496714e6e7a5e3b89c32dd720b7692f944b6c8" exitCode=0
Jul 05 00:21:27.908995 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:21:27.908887    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-nvgkm"
Jul 05 00:21:27.908995 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:21:27.908933    2563 scope.go:117] "RemoveContainer" containerID="1be8ea0310633ea8f855bb4a17496714e6e7a5e3b89c32dd720b7692f944b6c8"
Jul 05 00:21:27.910882 ip-10-0-142-202 kubenswrapper[2563]: E0705 00:21:27.910857    2563 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-nvgkm\" is forbidden: User \"system:node:ip-10-0-142-202.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-142-202.ec2.internal' and this object" podUID="cf6baec2-ef0e-4938-af29-dbd0a02074bb" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-nvgkm"
Jul 05 00:21:27.916842 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:21:27.916828    2563 scope.go:117] "RemoveContainer" containerID="1be8ea0310633ea8f855bb4a17496714e6e7a5e3b89c32dd720b7692f944b6c8"
Jul 05 00:21:27.917089 ip-10-0-142-202 kubenswrapper[2563]: E0705 00:21:27.917071    2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1be8ea0310633ea8f855bb4a17496714e6e7a5e3b89c32dd720b7692f944b6c8\": container with ID starting with 1be8ea0310633ea8f855bb4a17496714e6e7a5e3b89c32dd720b7692f944b6c8 not found: ID does not exist" containerID="1be8ea0310633ea8f855bb4a17496714e6e7a5e3b89c32dd720b7692f944b6c8"
Jul 05 00:21:27.917133 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:21:27.917097    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1be8ea0310633ea8f855bb4a17496714e6e7a5e3b89c32dd720b7692f944b6c8"} err="failed to get container status \"1be8ea0310633ea8f855bb4a17496714e6e7a5e3b89c32dd720b7692f944b6c8\": rpc error: code = NotFound desc = could not find container \"1be8ea0310633ea8f855bb4a17496714e6e7a5e3b89c32dd720b7692f944b6c8\": container with ID starting with 1be8ea0310633ea8f855bb4a17496714e6e7a5e3b89c32dd720b7692f944b6c8 not found: ID does not exist"
Jul 05 00:21:27.920279 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:21:27.920265    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-jcx5n"
Jul 05 00:21:27.925805 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:21:27.925788    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-fqj5r\" (UniqueName: \"kubernetes.io/projected/cf6baec2-ef0e-4938-af29-dbd0a02074bb-kube-api-access-fqj5r\") pod \"cf6baec2-ef0e-4938-af29-dbd0a02074bb\" (UID: \"cf6baec2-ef0e-4938-af29-dbd0a02074bb\") "
Jul 05 00:21:27.925877 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:21:27.925818    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/cf6baec2-ef0e-4938-af29-dbd0a02074bb-extensions-socket-volume\") pod \"cf6baec2-ef0e-4938-af29-dbd0a02074bb\" (UID: \"cf6baec2-ef0e-4938-af29-dbd0a02074bb\") "
Jul 05 00:21:27.926383 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:21:27.926359    2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/cf6baec2-ef0e-4938-af29-dbd0a02074bb-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "cf6baec2-ef0e-4938-af29-dbd0a02074bb" (UID: "cf6baec2-ef0e-4938-af29-dbd0a02074bb"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 05 00:21:27.927852 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:21:27.927827    2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/cf6baec2-ef0e-4938-af29-dbd0a02074bb-kube-api-access-fqj5r" (OuterVolumeSpecName: "kube-api-access-fqj5r") pod "cf6baec2-ef0e-4938-af29-dbd0a02074bb" (UID: "cf6baec2-ef0e-4938-af29-dbd0a02074bb"). InnerVolumeSpecName "kube-api-access-fqj5r". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 05 00:21:28.026678 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:21:28.026646    2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-fqj5r\" (UniqueName: \"kubernetes.io/projected/cf6baec2-ef0e-4938-af29-dbd0a02074bb-kube-api-access-fqj5r\") on node \"ip-10-0-142-202.ec2.internal\" DevicePath \"\""
Jul 05 00:21:28.026678 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:21:28.026681    2563 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/cf6baec2-ef0e-4938-af29-dbd0a02074bb-extensions-socket-volume\") on node \"ip-10-0-142-202.ec2.internal\" DevicePath \"\""
Jul 05 00:21:28.047799 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:21:28.047774    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-jcx5n"]
Jul 05 00:21:28.051151 ip-10-0-142-202 kubenswrapper[2563]: W0705 00:21:28.051129    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7a1c52c9_4ea1_4bea_b4d4_d5f51ac89acc.slice/crio-a058b25e56290fadb3b58830db0e7d82b9fd8bf75db6dab200dc0c78167bd875 WatchSource:0}: Error finding container a058b25e56290fadb3b58830db0e7d82b9fd8bf75db6dab200dc0c78167bd875: Status 404 returned error can't find the container with id a058b25e56290fadb3b58830db0e7d82b9fd8bf75db6dab200dc0c78167bd875
Jul 05 00:21:28.126753 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:21:28.126723    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="cf6baec2-ef0e-4938-af29-dbd0a02074bb" path="/var/lib/kubelet/pods/cf6baec2-ef0e-4938-af29-dbd0a02074bb/volumes"
Jul 05 00:21:28.915189 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:21:28.915155    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-jcx5n" event={"ID":"7a1c52c9-4ea1-4bea-b4d4-d5f51ac89acc","Type":"ContainerStarted","Data":"1d8ac6c259a9f379ad1400c06ef20c419d8c2f2e55cd8357301a9cbc5747152b"}
Jul 05 00:21:28.915664 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:21:28.915201    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-jcx5n" event={"ID":"7a1c52c9-4ea1-4bea-b4d4-d5f51ac89acc","Type":"ContainerStarted","Data":"a058b25e56290fadb3b58830db0e7d82b9fd8bf75db6dab200dc0c78167bd875"}
Jul 05 00:21:28.915664 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:21:28.915248    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-jcx5n"
Jul 05 00:21:28.933197 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:21:28.933152    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-jcx5n" podStartSLOduration=1.933136868 podStartE2EDuration="1.933136868s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-05 00:21:27 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-05 00:21:28.932990744 +0000 UTC m=+737.275319809" watchObservedRunningTime="2026-07-05 00:21:28.933136868 +0000 UTC m=+737.275465928"
Jul 05 00:21:39.922503 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:21:39.922473    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-jcx5n"
Jul 05 00:21:43.474848 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:21:43.474808    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-jcx5n"]
Jul 05 00:21:43.475319 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:21:43.475197    2563 kuberuntime_container.go:872] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-jcx5n" podUID="7a1c52c9-4ea1-4bea-b4d4-d5f51ac89acc" containerName="manager" containerID="cri-o://1d8ac6c259a9f379ad1400c06ef20c419d8c2f2e55cd8357301a9cbc5747152b" gracePeriod=10
Jul 05 00:21:43.724886 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:21:43.724833    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-jcx5n"
Jul 05 00:21:43.864986 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:21:43.864952    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-7dpkl\" (UniqueName: \"kubernetes.io/projected/7a1c52c9-4ea1-4bea-b4d4-d5f51ac89acc-kube-api-access-7dpkl\") pod \"7a1c52c9-4ea1-4bea-b4d4-d5f51ac89acc\" (UID: \"7a1c52c9-4ea1-4bea-b4d4-d5f51ac89acc\") "
Jul 05 00:21:43.865149 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:21:43.864995    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/7a1c52c9-4ea1-4bea-b4d4-d5f51ac89acc-extensions-socket-volume\") pod \"7a1c52c9-4ea1-4bea-b4d4-d5f51ac89acc\" (UID: \"7a1c52c9-4ea1-4bea-b4d4-d5f51ac89acc\") "
Jul 05 00:21:43.865431 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:21:43.865408    2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7a1c52c9-4ea1-4bea-b4d4-d5f51ac89acc-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "7a1c52c9-4ea1-4bea-b4d4-d5f51ac89acc" (UID: "7a1c52c9-4ea1-4bea-b4d4-d5f51ac89acc"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 05 00:21:43.866985 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:21:43.866965    2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7a1c52c9-4ea1-4bea-b4d4-d5f51ac89acc-kube-api-access-7dpkl" (OuterVolumeSpecName: "kube-api-access-7dpkl") pod "7a1c52c9-4ea1-4bea-b4d4-d5f51ac89acc" (UID: "7a1c52c9-4ea1-4bea-b4d4-d5f51ac89acc"). InnerVolumeSpecName "kube-api-access-7dpkl". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 05 00:21:43.965738 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:21:43.965710    2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-7dpkl\" (UniqueName: \"kubernetes.io/projected/7a1c52c9-4ea1-4bea-b4d4-d5f51ac89acc-kube-api-access-7dpkl\") on node \"ip-10-0-142-202.ec2.internal\" DevicePath \"\""
Jul 05 00:21:43.965738 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:21:43.965733    2563 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/7a1c52c9-4ea1-4bea-b4d4-d5f51ac89acc-extensions-socket-volume\") on node \"ip-10-0-142-202.ec2.internal\" DevicePath \"\""
Jul 05 00:21:43.979379 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:21:43.979312    2563 generic.go:356] "Generic (PLEG): container finished" podID="7a1c52c9-4ea1-4bea-b4d4-d5f51ac89acc" containerID="1d8ac6c259a9f379ad1400c06ef20c419d8c2f2e55cd8357301a9cbc5747152b" exitCode=0
Jul 05 00:21:43.979379 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:21:43.979352    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-jcx5n"
Jul 05 00:21:43.979516 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:21:43.979397    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-jcx5n" event={"ID":"7a1c52c9-4ea1-4bea-b4d4-d5f51ac89acc","Type":"ContainerDied","Data":"1d8ac6c259a9f379ad1400c06ef20c419d8c2f2e55cd8357301a9cbc5747152b"}
Jul 05 00:21:43.979516 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:21:43.979447    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-jcx5n" event={"ID":"7a1c52c9-4ea1-4bea-b4d4-d5f51ac89acc","Type":"ContainerDied","Data":"a058b25e56290fadb3b58830db0e7d82b9fd8bf75db6dab200dc0c78167bd875"}
Jul 05 00:21:43.979516 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:21:43.979469    2563 scope.go:117] "RemoveContainer" containerID="1d8ac6c259a9f379ad1400c06ef20c419d8c2f2e55cd8357301a9cbc5747152b"
Jul 05 00:21:43.989696 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:21:43.989636    2563 scope.go:117] "RemoveContainer" containerID="1d8ac6c259a9f379ad1400c06ef20c419d8c2f2e55cd8357301a9cbc5747152b"
Jul 05 00:21:43.990320 ip-10-0-142-202 kubenswrapper[2563]: E0705 00:21:43.990299    2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1d8ac6c259a9f379ad1400c06ef20c419d8c2f2e55cd8357301a9cbc5747152b\": container with ID starting with 1d8ac6c259a9f379ad1400c06ef20c419d8c2f2e55cd8357301a9cbc5747152b not found: ID does not exist" containerID="1d8ac6c259a9f379ad1400c06ef20c419d8c2f2e55cd8357301a9cbc5747152b"
Jul 05 00:21:43.990385 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:21:43.990334    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1d8ac6c259a9f379ad1400c06ef20c419d8c2f2e55cd8357301a9cbc5747152b"} err="failed to get container status \"1d8ac6c259a9f379ad1400c06ef20c419d8c2f2e55cd8357301a9cbc5747152b\": rpc error: code = NotFound desc = could not find container \"1d8ac6c259a9f379ad1400c06ef20c419d8c2f2e55cd8357301a9cbc5747152b\": container with ID starting with 1d8ac6c259a9f379ad1400c06ef20c419d8c2f2e55cd8357301a9cbc5747152b not found: ID does not exist"
Jul 05 00:21:44.001565 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:21:44.001546    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-jcx5n"]
Jul 05 00:21:44.007507 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:21:44.007488    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-jcx5n"]
Jul 05 00:21:44.126096 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:21:44.126067    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="7a1c52c9-4ea1-4bea-b4d4-d5f51ac89acc" path="/var/lib/kubelet/pods/7a1c52c9-4ea1-4bea-b4d4-d5f51ac89acc/volumes"
Jul 05 00:21:59.760138 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:21:59.760100    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-wzqkx"]
Jul 05 00:21:59.760612 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:21:59.760596    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7a1c52c9-4ea1-4bea-b4d4-d5f51ac89acc" containerName="manager"
Jul 05 00:21:59.760659 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:21:59.760616    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="7a1c52c9-4ea1-4bea-b4d4-d5f51ac89acc" containerName="manager"
Jul 05 00:21:59.760722 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:21:59.760711    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="7a1c52c9-4ea1-4bea-b4d4-d5f51ac89acc" containerName="manager"
Jul 05 00:21:59.763652 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:21:59.763633    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-wzqkx"
Jul 05 00:21:59.765606 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:21:59.765584    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"maas-default-gateway-openshift-default-dockercfg-8zqrq\""
Jul 05 00:21:59.765726 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:21:59.765588    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\""
Jul 05 00:21:59.772181 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:21:59.772165    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-wzqkx"]
Jul 05 00:21:59.786877 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:21:59.786849    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a95fc6e8-4791-4960-b7f5-2d8e09342c4a-workload-socket\") pod \"maas-default-gateway-openshift-default-56654dcf7-wzqkx\" (UID: \"a95fc6e8-4791-4960-b7f5-2d8e09342c4a\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-wzqkx"
Jul 05 00:21:59.786966 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:21:59.786878    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a95fc6e8-4791-4960-b7f5-2d8e09342c4a-istio-data\") pod \"maas-default-gateway-openshift-default-56654dcf7-wzqkx\" (UID: \"a95fc6e8-4791-4960-b7f5-2d8e09342c4a\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-wzqkx"
Jul 05 00:21:59.786966 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:21:59.786901    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a95fc6e8-4791-4960-b7f5-2d8e09342c4a-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-56654dcf7-wzqkx\" (UID: \"a95fc6e8-4791-4960-b7f5-2d8e09342c4a\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-wzqkx"
Jul 05 00:21:59.786966 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:21:59.786918    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a95fc6e8-4791-4960-b7f5-2d8e09342c4a-istio-envoy\") pod \"maas-default-gateway-openshift-default-56654dcf7-wzqkx\" (UID: \"a95fc6e8-4791-4960-b7f5-2d8e09342c4a\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-wzqkx"
Jul 05 00:21:59.787141 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:21:59.787019    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a95fc6e8-4791-4960-b7f5-2d8e09342c4a-istio-token\") pod \"maas-default-gateway-openshift-default-56654dcf7-wzqkx\" (UID: \"a95fc6e8-4791-4960-b7f5-2d8e09342c4a\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-wzqkx"
Jul 05 00:21:59.787141 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:21:59.787051    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cgdz4\" (UniqueName: \"kubernetes.io/projected/a95fc6e8-4791-4960-b7f5-2d8e09342c4a-kube-api-access-cgdz4\") pod \"maas-default-gateway-openshift-default-56654dcf7-wzqkx\" (UID: \"a95fc6e8-4791-4960-b7f5-2d8e09342c4a\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-wzqkx"
Jul 05 00:21:59.787141 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:21:59.787086    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a95fc6e8-4791-4960-b7f5-2d8e09342c4a-credential-socket\") pod \"maas-default-gateway-openshift-default-56654dcf7-wzqkx\" (UID: \"a95fc6e8-4791-4960-b7f5-2d8e09342c4a\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-wzqkx"
Jul 05 00:21:59.787141 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:21:59.787133    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a95fc6e8-4791-4960-b7f5-2d8e09342c4a-workload-certs\") pod \"maas-default-gateway-openshift-default-56654dcf7-wzqkx\" (UID: \"a95fc6e8-4791-4960-b7f5-2d8e09342c4a\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-wzqkx"
Jul 05 00:21:59.787329 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:21:59.787176    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a95fc6e8-4791-4960-b7f5-2d8e09342c4a-istio-podinfo\") pod \"maas-default-gateway-openshift-default-56654dcf7-wzqkx\" (UID: \"a95fc6e8-4791-4960-b7f5-2d8e09342c4a\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-wzqkx"
Jul 05 00:21:59.888398 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:21:59.888360    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a95fc6e8-4791-4960-b7f5-2d8e09342c4a-workload-certs\") pod \"maas-default-gateway-openshift-default-56654dcf7-wzqkx\" (UID: \"a95fc6e8-4791-4960-b7f5-2d8e09342c4a\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-wzqkx"
Jul 05 00:21:59.888398 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:21:59.888397    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a95fc6e8-4791-4960-b7f5-2d8e09342c4a-istio-podinfo\") pod \"maas-default-gateway-openshift-default-56654dcf7-wzqkx\" (UID: \"a95fc6e8-4791-4960-b7f5-2d8e09342c4a\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-wzqkx"
Jul 05 00:21:59.888654 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:21:59.888439    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a95fc6e8-4791-4960-b7f5-2d8e09342c4a-workload-socket\") pod \"maas-default-gateway-openshift-default-56654dcf7-wzqkx\" (UID: \"a95fc6e8-4791-4960-b7f5-2d8e09342c4a\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-wzqkx"
Jul 05 00:21:59.888654 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:21:59.888461    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a95fc6e8-4791-4960-b7f5-2d8e09342c4a-istio-data\") pod \"maas-default-gateway-openshift-default-56654dcf7-wzqkx\" (UID: \"a95fc6e8-4791-4960-b7f5-2d8e09342c4a\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-wzqkx"
Jul 05 00:21:59.888654 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:21:59.888497    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a95fc6e8-4791-4960-b7f5-2d8e09342c4a-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-56654dcf7-wzqkx\" (UID: \"a95fc6e8-4791-4960-b7f5-2d8e09342c4a\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-wzqkx"
Jul 05 00:21:59.888654 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:21:59.888522    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a95fc6e8-4791-4960-b7f5-2d8e09342c4a-istio-envoy\") pod \"maas-default-gateway-openshift-default-56654dcf7-wzqkx\" (UID: \"a95fc6e8-4791-4960-b7f5-2d8e09342c4a\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-wzqkx"
Jul 05 00:21:59.888654 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:21:59.888551    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a95fc6e8-4791-4960-b7f5-2d8e09342c4a-istio-token\") pod \"maas-default-gateway-openshift-default-56654dcf7-wzqkx\" (UID: \"a95fc6e8-4791-4960-b7f5-2d8e09342c4a\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-wzqkx"
Jul 05 00:21:59.888654 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:21:59.888578    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cgdz4\" (UniqueName: \"kubernetes.io/projected/a95fc6e8-4791-4960-b7f5-2d8e09342c4a-kube-api-access-cgdz4\") pod \"maas-default-gateway-openshift-default-56654dcf7-wzqkx\" (UID: \"a95fc6e8-4791-4960-b7f5-2d8e09342c4a\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-wzqkx"
Jul 05 00:21:59.888962 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:21:59.888665    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a95fc6e8-4791-4960-b7f5-2d8e09342c4a-credential-socket\") pod \"maas-default-gateway-openshift-default-56654dcf7-wzqkx\" (UID: \"a95fc6e8-4791-4960-b7f5-2d8e09342c4a\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-wzqkx"
Jul 05 00:21:59.888962 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:21:59.888812    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a95fc6e8-4791-4960-b7f5-2d8e09342c4a-workload-socket\") pod \"maas-default-gateway-openshift-default-56654dcf7-wzqkx\" (UID: \"a95fc6e8-4791-4960-b7f5-2d8e09342c4a\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-wzqkx"
Jul 05 00:21:59.888962 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:21:59.888854    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a95fc6e8-4791-4960-b7f5-2d8e09342c4a-istio-data\") pod \"maas-default-gateway-openshift-default-56654dcf7-wzqkx\" (UID: \"a95fc6e8-4791-4960-b7f5-2d8e09342c4a\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-wzqkx"
Jul 05 00:21:59.889109 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:21:59.889011    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a95fc6e8-4791-4960-b7f5-2d8e09342c4a-workload-certs\") pod \"maas-default-gateway-openshift-default-56654dcf7-wzqkx\" (UID: \"a95fc6e8-4791-4960-b7f5-2d8e09342c4a\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-wzqkx"
Jul 05 00:21:59.889109 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:21:59.889079    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a95fc6e8-4791-4960-b7f5-2d8e09342c4a-credential-socket\") pod \"maas-default-gateway-openshift-default-56654dcf7-wzqkx\" (UID: \"a95fc6e8-4791-4960-b7f5-2d8e09342c4a\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-wzqkx"
Jul 05 00:21:59.889328 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:21:59.889308    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a95fc6e8-4791-4960-b7f5-2d8e09342c4a-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-56654dcf7-wzqkx\" (UID: \"a95fc6e8-4791-4960-b7f5-2d8e09342c4a\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-wzqkx"
Jul 05 00:21:59.890719 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:21:59.890691    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a95fc6e8-4791-4960-b7f5-2d8e09342c4a-istio-envoy\") pod \"maas-default-gateway-openshift-default-56654dcf7-wzqkx\" (UID: \"a95fc6e8-4791-4960-b7f5-2d8e09342c4a\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-wzqkx"
Jul 05 00:21:59.890849 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:21:59.890833    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a95fc6e8-4791-4960-b7f5-2d8e09342c4a-istio-podinfo\") pod \"maas-default-gateway-openshift-default-56654dcf7-wzqkx\" (UID: \"a95fc6e8-4791-4960-b7f5-2d8e09342c4a\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-wzqkx"
Jul 05 00:21:59.895489 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:21:59.895466    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-cgdz4\" (UniqueName: \"kubernetes.io/projected/a95fc6e8-4791-4960-b7f5-2d8e09342c4a-kube-api-access-cgdz4\") pod \"maas-default-gateway-openshift-default-56654dcf7-wzqkx\" (UID: \"a95fc6e8-4791-4960-b7f5-2d8e09342c4a\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-wzqkx"
Jul 05 00:21:59.895489 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:21:59.895483    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a95fc6e8-4791-4960-b7f5-2d8e09342c4a-istio-token\") pod \"maas-default-gateway-openshift-default-56654dcf7-wzqkx\" (UID: \"a95fc6e8-4791-4960-b7f5-2d8e09342c4a\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-wzqkx"
Jul 05 00:22:00.075298 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:22:00.075266    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-wzqkx"
Jul 05 00:22:00.204118 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:22:00.204090    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-wzqkx"]
Jul 05 00:22:00.207525 ip-10-0-142-202 kubenswrapper[2563]: W0705 00:22:00.207496    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda95fc6e8_4791_4960_b7f5_2d8e09342c4a.slice/crio-30639a28b624274e524e1bfa30016d1a8a4f02c64aa91ee013709fd50a53bb12 WatchSource:0}: Error finding container 30639a28b624274e524e1bfa30016d1a8a4f02c64aa91ee013709fd50a53bb12: Status 404 returned error can't find the container with id 30639a28b624274e524e1bfa30016d1a8a4f02c64aa91ee013709fd50a53bb12
Jul 05 00:22:01.055922 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:22:01.055883    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-wzqkx" event={"ID":"a95fc6e8-4791-4960-b7f5-2d8e09342c4a","Type":"ContainerStarted","Data":"30639a28b624274e524e1bfa30016d1a8a4f02c64aa91ee013709fd50a53bb12"}
Jul 05 00:22:02.765266 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:22:02.765201    2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"}
Jul 05 00:22:02.765514 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:22:02.765302    2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"}
Jul 05 00:22:02.765514 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:22:02.765332    2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"}
Jul 05 00:22:03.073063 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:22:03.073010    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-wzqkx" event={"ID":"a95fc6e8-4791-4960-b7f5-2d8e09342c4a","Type":"ContainerStarted","Data":"f7e8f57700e7d7d729d4c873a8bf679a15faa605eb74bd76ec10a7633c0d246b"}
Jul 05 00:22:03.075394 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:22:03.075371    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-wzqkx"
Jul 05 00:22:03.080169 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:22:03.080150    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-wzqkx"
Jul 05 00:22:03.091177 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:22:03.091131    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-wzqkx" podStartSLOduration=1.535656317 podStartE2EDuration="4.091116669s" totalImagesPullingTime="2.555460352s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-05 00:21:59 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-05 00:22:03.089074041 +0000 UTC m=+771.431403100" watchObservedRunningTime="2026-07-05 00:22:03.091116669 +0000 UTC m=+771.433445729"
Jul 05 00:22:04.077990 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:22:04.077954    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-wzqkx"
Jul 05 00:22:04.079079 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:22:04.079061    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-wzqkx"
Jul 05 00:22:17.988681 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:22:17.988648    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-x5szm"]
Jul 05 00:22:17.993113 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:22:17.993093    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-54d6d988d5-x5szm"
Jul 05 00:22:17.994885 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:22:17.994857    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\""
Jul 05 00:22:17.995001 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:22:17.994857    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"default-dockercfg-j8vgb\""
Jul 05 00:22:17.995341 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:22:17.995324    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\""
Jul 05 00:22:17.995431 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:22:17.995324    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"limitador-limits-config-limitador\""
Jul 05 00:22:17.998933 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:22:17.998910    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-x5szm"]
Jul 05 00:22:18.023370 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:22:18.023346    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-x5szm"]
Jul 05 00:22:18.149556 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:22:18.149528    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/cacf0cb6-3f72-44ee-a6d9-77d3a0d89616-config-file\") pod \"limitador-limitador-54d6d988d5-x5szm\" (UID: \"cacf0cb6-3f72-44ee-a6d9-77d3a0d89616\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-x5szm"
Jul 05 00:22:18.149707 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:22:18.149575    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jj2q2\" (UniqueName: \"kubernetes.io/projected/cacf0cb6-3f72-44ee-a6d9-77d3a0d89616-kube-api-access-jj2q2\") pod \"limitador-limitador-54d6d988d5-x5szm\" (UID: \"cacf0cb6-3f72-44ee-a6d9-77d3a0d89616\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-x5szm"
Jul 05 00:22:18.250515 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:22:18.250433    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/cacf0cb6-3f72-44ee-a6d9-77d3a0d89616-config-file\") pod \"limitador-limitador-54d6d988d5-x5szm\" (UID: \"cacf0cb6-3f72-44ee-a6d9-77d3a0d89616\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-x5szm"
Jul 05 00:22:18.250515 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:22:18.250480    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jj2q2\" (UniqueName: \"kubernetes.io/projected/cacf0cb6-3f72-44ee-a6d9-77d3a0d89616-kube-api-access-jj2q2\") pod \"limitador-limitador-54d6d988d5-x5szm\" (UID: \"cacf0cb6-3f72-44ee-a6d9-77d3a0d89616\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-x5szm"
Jul 05 00:22:18.251056 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:22:18.251036    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/cacf0cb6-3f72-44ee-a6d9-77d3a0d89616-config-file\") pod \"limitador-limitador-54d6d988d5-x5szm\" (UID: \"cacf0cb6-3f72-44ee-a6d9-77d3a0d89616\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-x5szm"
Jul 05 00:22:18.257804 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:22:18.257776    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-jj2q2\" (UniqueName: \"kubernetes.io/projected/cacf0cb6-3f72-44ee-a6d9-77d3a0d89616-kube-api-access-jj2q2\") pod \"limitador-limitador-54d6d988d5-x5szm\" (UID: \"cacf0cb6-3f72-44ee-a6d9-77d3a0d89616\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-x5szm"
Jul 05 00:22:18.304889 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:22:18.304862    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-54d6d988d5-x5szm"
Jul 05 00:22:18.400794 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:22:18.400766    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-58cd955675-mc7r8"]
Jul 05 00:22:18.404179 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:22:18.404155    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-58cd955675-mc7r8"
Jul 05 00:22:18.405910 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:22:18.405890    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-authorino-dockercfg-6m47r\""
Jul 05 00:22:18.410571 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:22:18.410545    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-58cd955675-mc7r8"]
Jul 05 00:22:18.430200 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:22:18.430178    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-x5szm"]
Jul 05 00:22:18.433347 ip-10-0-142-202 kubenswrapper[2563]: W0705 00:22:18.433323    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcacf0cb6_3f72_44ee_a6d9_77d3a0d89616.slice/crio-c954d3e77b0f1069aa96474b8472bdb9d09014331b23717db55c553b5aec461c WatchSource:0}: Error finding container c954d3e77b0f1069aa96474b8472bdb9d09014331b23717db55c553b5aec461c: Status 404 returned error can't find the container with id c954d3e77b0f1069aa96474b8472bdb9d09014331b23717db55c553b5aec461c
Jul 05 00:22:18.542771 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:22:18.542745    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-75d556bd77-m97jz"]
Jul 05 00:22:18.545812 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:22:18.545795    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-75d556bd77-m97jz"
Jul 05 00:22:18.551423 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:22:18.551400    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-75d556bd77-m97jz"]
Jul 05 00:22:18.553421 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:22:18.553402    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-z9bbt\" (UniqueName: \"kubernetes.io/projected/00e3ff7a-2abe-4a46-9bfc-3e8c1f518c93-kube-api-access-z9bbt\") pod \"authorino-58cd955675-mc7r8\" (UID: \"00e3ff7a-2abe-4a46-9bfc-3e8c1f518c93\") " pod="kuadrant-system/authorino-58cd955675-mc7r8"
Jul 05 00:22:18.654338 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:22:18.654307    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8n4mz\" (UniqueName: \"kubernetes.io/projected/2114e1cf-4ecf-41ec-a1aa-71394e96204e-kube-api-access-8n4mz\") pod \"authorino-75d556bd77-m97jz\" (UID: \"2114e1cf-4ecf-41ec-a1aa-71394e96204e\") " pod="kuadrant-system/authorino-75d556bd77-m97jz"
Jul 05 00:22:18.654492 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:22:18.654349    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-z9bbt\" (UniqueName: \"kubernetes.io/projected/00e3ff7a-2abe-4a46-9bfc-3e8c1f518c93-kube-api-access-z9bbt\") pod \"authorino-58cd955675-mc7r8\" (UID: \"00e3ff7a-2abe-4a46-9bfc-3e8c1f518c93\") " pod="kuadrant-system/authorino-58cd955675-mc7r8"
Jul 05 00:22:18.661209 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:22:18.661189    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-z9bbt\" (UniqueName: \"kubernetes.io/projected/00e3ff7a-2abe-4a46-9bfc-3e8c1f518c93-kube-api-access-z9bbt\") pod \"authorino-58cd955675-mc7r8\" (UID: \"00e3ff7a-2abe-4a46-9bfc-3e8c1f518c93\") " pod="kuadrant-system/authorino-58cd955675-mc7r8"
Jul 05 00:22:18.716114 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:22:18.716090    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-58cd955675-mc7r8"
Jul 05 00:22:18.755306 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:22:18.755278    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8n4mz\" (UniqueName: \"kubernetes.io/projected/2114e1cf-4ecf-41ec-a1aa-71394e96204e-kube-api-access-8n4mz\") pod \"authorino-75d556bd77-m97jz\" (UID: \"2114e1cf-4ecf-41ec-a1aa-71394e96204e\") " pod="kuadrant-system/authorino-75d556bd77-m97jz"
Jul 05 00:22:18.763417 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:22:18.763012    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-8n4mz\" (UniqueName: \"kubernetes.io/projected/2114e1cf-4ecf-41ec-a1aa-71394e96204e-kube-api-access-8n4mz\") pod \"authorino-75d556bd77-m97jz\" (UID: \"2114e1cf-4ecf-41ec-a1aa-71394e96204e\") " pod="kuadrant-system/authorino-75d556bd77-m97jz"
Jul 05 00:22:18.839936 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:22:18.839804    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-58cd955675-mc7r8"]
Jul 05 00:22:18.842494 ip-10-0-142-202 kubenswrapper[2563]: W0705 00:22:18.842467    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod00e3ff7a_2abe_4a46_9bfc_3e8c1f518c93.slice/crio-45958e62ab2fa2e06a862a578a1df45da81f384074a447c8d8554163acaa567b WatchSource:0}: Error finding container 45958e62ab2fa2e06a862a578a1df45da81f384074a447c8d8554163acaa567b: Status 404 returned error can't find the container with id 45958e62ab2fa2e06a862a578a1df45da81f384074a447c8d8554163acaa567b
Jul 05 00:22:18.855245 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:22:18.855205    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-75d556bd77-m97jz"
Jul 05 00:22:19.015556 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:22:19.015512    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-75d556bd77-m97jz"]
Jul 05 00:22:19.019446 ip-10-0-142-202 kubenswrapper[2563]: W0705 00:22:19.019418    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod2114e1cf_4ecf_41ec_a1aa_71394e96204e.slice/crio-448f0ebbd194161b211cb9081448fb53673a230169030cdd89b922783397374d WatchSource:0}: Error finding container 448f0ebbd194161b211cb9081448fb53673a230169030cdd89b922783397374d: Status 404 returned error can't find the container with id 448f0ebbd194161b211cb9081448fb53673a230169030cdd89b922783397374d
Jul 05 00:22:19.141831 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:22:19.141743    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-54d6d988d5-x5szm" event={"ID":"cacf0cb6-3f72-44ee-a6d9-77d3a0d89616","Type":"ContainerStarted","Data":"c954d3e77b0f1069aa96474b8472bdb9d09014331b23717db55c553b5aec461c"}
Jul 05 00:22:19.143871 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:22:19.143838    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-75d556bd77-m97jz" event={"ID":"2114e1cf-4ecf-41ec-a1aa-71394e96204e","Type":"ContainerStarted","Data":"448f0ebbd194161b211cb9081448fb53673a230169030cdd89b922783397374d"}
Jul 05 00:22:19.148362 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:22:19.148304    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-58cd955675-mc7r8" event={"ID":"00e3ff7a-2abe-4a46-9bfc-3e8c1f518c93","Type":"ContainerStarted","Data":"45958e62ab2fa2e06a862a578a1df45da81f384074a447c8d8554163acaa567b"}
Jul 05 00:22:24.176159 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:22:24.176112    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-54d6d988d5-x5szm" event={"ID":"cacf0cb6-3f72-44ee-a6d9-77d3a0d89616","Type":"ContainerStarted","Data":"6d9a994c5a3b2bfda121f668014b3ad35ed1c3a21c4846986f3415666956442e"}
Jul 05 00:22:24.176624 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:22:24.176210    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-limitador-54d6d988d5-x5szm"
Jul 05 00:22:24.177733 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:22:24.177687    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-75d556bd77-m97jz" event={"ID":"2114e1cf-4ecf-41ec-a1aa-71394e96204e","Type":"ContainerStarted","Data":"8853bc32aaa4b556d946b4474f8f908eae0247e867c3157e0db0e9026c877555"}
Jul 05 00:22:24.179122 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:22:24.179098    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-58cd955675-mc7r8" event={"ID":"00e3ff7a-2abe-4a46-9bfc-3e8c1f518c93","Type":"ContainerStarted","Data":"da4a0ddbd74fc1b6556e523e30719fb9a57f1a604a505702b5bd6498c397afab"}
Jul 05 00:22:24.190697 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:22:24.190655    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-limitador-54d6d988d5-x5szm" podStartSLOduration=2.34844817 podStartE2EDuration="7.190642327s" totalImagesPullingTime="4.842194157s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-05 00:22:17 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-05 00:22:24.188899831 +0000 UTC m=+792.531228941" watchObservedRunningTime="2026-07-05 00:22:24.190642327 +0000 UTC m=+792.532971388"
Jul 05 00:22:24.201790 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:22:24.201743    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-58cd955675-mc7r8" podStartSLOduration=1.7760604039999999 podStartE2EDuration="6.201731239s" totalImagesPullingTime="4.425670835s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-05 00:22:18 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-05 00:22:24.20039759 +0000 UTC m=+792.542726678" watchObservedRunningTime="2026-07-05 00:22:24.201731239 +0000 UTC m=+792.544060298"
Jul 05 00:22:24.214150 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:22:24.214109    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-75d556bd77-m97jz" podStartSLOduration=1.995758361 podStartE2EDuration="6.214092808s" totalImagesPullingTime="4.218334447s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-05 00:22:18 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-05 00:22:24.212885957 +0000 UTC m=+792.555215047" watchObservedRunningTime="2026-07-05 00:22:24.214092808 +0000 UTC m=+792.556421869"
Jul 05 00:22:24.236294 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:22:24.236266    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-58cd955675-mc7r8"]
Jul 05 00:22:26.188434 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:22:26.188382    2563 kuberuntime_container.go:872] "Killing container with a grace period" pod="kuadrant-system/authorino-58cd955675-mc7r8" podUID="00e3ff7a-2abe-4a46-9bfc-3e8c1f518c93" containerName="authorino" containerID="cri-o://da4a0ddbd74fc1b6556e523e30719fb9a57f1a604a505702b5bd6498c397afab" gracePeriod=30
Jul 05 00:22:26.431286 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:22:26.431263    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-58cd955675-mc7r8"
Jul 05 00:22:26.528364 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:22:26.528340    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-z9bbt\" (UniqueName: \"kubernetes.io/projected/00e3ff7a-2abe-4a46-9bfc-3e8c1f518c93-kube-api-access-z9bbt\") pod \"00e3ff7a-2abe-4a46-9bfc-3e8c1f518c93\" (UID: \"00e3ff7a-2abe-4a46-9bfc-3e8c1f518c93\") "
Jul 05 00:22:26.530350 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:22:26.530318    2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/00e3ff7a-2abe-4a46-9bfc-3e8c1f518c93-kube-api-access-z9bbt" (OuterVolumeSpecName: "kube-api-access-z9bbt") pod "00e3ff7a-2abe-4a46-9bfc-3e8c1f518c93" (UID: "00e3ff7a-2abe-4a46-9bfc-3e8c1f518c93"). InnerVolumeSpecName "kube-api-access-z9bbt". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 05 00:22:26.628956 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:22:26.628920    2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-z9bbt\" (UniqueName: \"kubernetes.io/projected/00e3ff7a-2abe-4a46-9bfc-3e8c1f518c93-kube-api-access-z9bbt\") on node \"ip-10-0-142-202.ec2.internal\" DevicePath \"\""
Jul 05 00:22:27.194017 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:22:27.193989    2563 generic.go:356] "Generic (PLEG): container finished" podID="00e3ff7a-2abe-4a46-9bfc-3e8c1f518c93" containerID="da4a0ddbd74fc1b6556e523e30719fb9a57f1a604a505702b5bd6498c397afab" exitCode=0
Jul 05 00:22:27.194017 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:22:27.194025    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-58cd955675-mc7r8"
Jul 05 00:22:27.194474 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:22:27.194068    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-58cd955675-mc7r8" event={"ID":"00e3ff7a-2abe-4a46-9bfc-3e8c1f518c93","Type":"ContainerDied","Data":"da4a0ddbd74fc1b6556e523e30719fb9a57f1a604a505702b5bd6498c397afab"}
Jul 05 00:22:27.194474 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:22:27.194106    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-58cd955675-mc7r8" event={"ID":"00e3ff7a-2abe-4a46-9bfc-3e8c1f518c93","Type":"ContainerDied","Data":"45958e62ab2fa2e06a862a578a1df45da81f384074a447c8d8554163acaa567b"}
Jul 05 00:22:27.194474 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:22:27.194131    2563 scope.go:117] "RemoveContainer" containerID="da4a0ddbd74fc1b6556e523e30719fb9a57f1a604a505702b5bd6498c397afab"
Jul 05 00:22:27.202387 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:22:27.202372    2563 scope.go:117] "RemoveContainer" containerID="da4a0ddbd74fc1b6556e523e30719fb9a57f1a604a505702b5bd6498c397afab"
Jul 05 00:22:27.202653 ip-10-0-142-202 kubenswrapper[2563]: E0705 00:22:27.202633    2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"da4a0ddbd74fc1b6556e523e30719fb9a57f1a604a505702b5bd6498c397afab\": container with ID starting with da4a0ddbd74fc1b6556e523e30719fb9a57f1a604a505702b5bd6498c397afab not found: ID does not exist" containerID="da4a0ddbd74fc1b6556e523e30719fb9a57f1a604a505702b5bd6498c397afab"
Jul 05 00:22:27.202744 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:22:27.202657    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"da4a0ddbd74fc1b6556e523e30719fb9a57f1a604a505702b5bd6498c397afab"} err="failed to get container status \"da4a0ddbd74fc1b6556e523e30719fb9a57f1a604a505702b5bd6498c397afab\": rpc error: code = NotFound desc = could not find container \"da4a0ddbd74fc1b6556e523e30719fb9a57f1a604a505702b5bd6498c397afab\": container with ID starting with da4a0ddbd74fc1b6556e523e30719fb9a57f1a604a505702b5bd6498c397afab not found: ID does not exist"
Jul 05 00:22:27.211835 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:22:27.211813    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-58cd955675-mc7r8"]
Jul 05 00:22:27.213538 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:22:27.213520    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-58cd955675-mc7r8"]
Jul 05 00:22:28.125907 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:22:28.125875    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="00e3ff7a-2abe-4a46-9bfc-3e8c1f518c93" path="/var/lib/kubelet/pods/00e3ff7a-2abe-4a46-9bfc-3e8c1f518c93/volumes"
Jul 05 00:22:35.186930 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:22:35.186857    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-limitador-54d6d988d5-x5szm"
Jul 05 00:22:54.363932 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:22:54.363900    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-5d4b67c446-cqr29"]
Jul 05 00:22:54.364412 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:22:54.364195    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="00e3ff7a-2abe-4a46-9bfc-3e8c1f518c93" containerName="authorino"
Jul 05 00:22:54.364412 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:22:54.364207    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="00e3ff7a-2abe-4a46-9bfc-3e8c1f518c93" containerName="authorino"
Jul 05 00:22:54.364412 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:22:54.364283    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="00e3ff7a-2abe-4a46-9bfc-3e8c1f518c93" containerName="authorino"
Jul 05 00:22:54.423790 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:22:54.423755    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-5d4b67c446-cqr29"]
Jul 05 00:22:54.423945 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:22:54.423840    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5d4b67c446-cqr29"
Jul 05 00:22:54.427749 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:22:54.427724    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gvrvg\" (UniqueName: \"kubernetes.io/projected/b821dfde-cd45-4fda-87d1-0c70365ecfd0-kube-api-access-gvrvg\") pod \"authorino-5d4b67c446-cqr29\" (UID: \"b821dfde-cd45-4fda-87d1-0c70365ecfd0\") " pod="kuadrant-system/authorino-5d4b67c446-cqr29"
Jul 05 00:22:54.528346 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:22:54.528318    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gvrvg\" (UniqueName: \"kubernetes.io/projected/b821dfde-cd45-4fda-87d1-0c70365ecfd0-kube-api-access-gvrvg\") pod \"authorino-5d4b67c446-cqr29\" (UID: \"b821dfde-cd45-4fda-87d1-0c70365ecfd0\") " pod="kuadrant-system/authorino-5d4b67c446-cqr29"
Jul 05 00:22:54.535278 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:22:54.535257    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-gvrvg\" (UniqueName: \"kubernetes.io/projected/b821dfde-cd45-4fda-87d1-0c70365ecfd0-kube-api-access-gvrvg\") pod \"authorino-5d4b67c446-cqr29\" (UID: \"b821dfde-cd45-4fda-87d1-0c70365ecfd0\") " pod="kuadrant-system/authorino-5d4b67c446-cqr29"
Jul 05 00:22:54.590249 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:22:54.590206    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-5d4b67c446-cqr29"]
Jul 05 00:22:54.590472 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:22:54.590460    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5d4b67c446-cqr29"
Jul 05 00:22:54.610282 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:22:54.610254    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-7c876ffffb-92gdc"]
Jul 05 00:22:54.642801 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:22:54.642771    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-7c876ffffb-92gdc"
Jul 05 00:22:54.642937 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:22:54.642773    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-7c876ffffb-92gdc"]
Jul 05 00:22:54.693211 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:22:54.693186    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-7c876ffffb-92gdc"]
Jul 05 00:22:54.693542 ip-10-0-142-202 kubenswrapper[2563]: E0705 00:22:54.693516    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[kube-api-access-ffnld], unattached volumes=[], failed to process volumes=[kube-api-access-ffnld]: context canceled" pod="kuadrant-system/authorino-7c876ffffb-92gdc" podUID="fb32d8dc-c25b-4d48-8f10-1e3fdfd5ab1a"
Jul 05 00:22:54.718924 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:22:54.718897    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-6dd86584-c8wws"]
Jul 05 00:22:54.729669 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:22:54.729642    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ffnld\" (UniqueName: \"kubernetes.io/projected/fb32d8dc-c25b-4d48-8f10-1e3fdfd5ab1a-kube-api-access-ffnld\") pod \"authorino-7c876ffffb-92gdc\" (UID: \"fb32d8dc-c25b-4d48-8f10-1e3fdfd5ab1a\") " pod="kuadrant-system/authorino-7c876ffffb-92gdc"
Jul 05 00:22:54.733486 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:22:54.733462    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-6dd86584-c8wws"]
Jul 05 00:22:54.733620 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:22:54.733547    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-6dd86584-c8wws"
Jul 05 00:22:54.735468 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:22:54.735449    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-server-cert\""
Jul 05 00:22:54.738001 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:22:54.737978    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-5d4b67c446-cqr29"]
Jul 05 00:22:54.740942 ip-10-0-142-202 kubenswrapper[2563]: W0705 00:22:54.740902    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podb821dfde_cd45_4fda_87d1_0c70365ecfd0.slice/crio-5f14a65532617d74705428237d6aeecf496a2b08838facc40701995796a5b865 WatchSource:0}: Error finding container 5f14a65532617d74705428237d6aeecf496a2b08838facc40701995796a5b865: Status 404 returned error can't find the container with id 5f14a65532617d74705428237d6aeecf496a2b08838facc40701995796a5b865
Jul 05 00:22:54.830050 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:22:54.830021    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ldg76\" (UniqueName: \"kubernetes.io/projected/dea7f9d0-0e6a-4be9-bb32-fc570dbcadab-kube-api-access-ldg76\") pod \"authorino-6dd86584-c8wws\" (UID: \"dea7f9d0-0e6a-4be9-bb32-fc570dbcadab\") " pod="kuadrant-system/authorino-6dd86584-c8wws"
Jul 05 00:22:54.830050 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:22:54.830052    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/dea7f9d0-0e6a-4be9-bb32-fc570dbcadab-tls-cert\") pod \"authorino-6dd86584-c8wws\" (UID: \"dea7f9d0-0e6a-4be9-bb32-fc570dbcadab\") " pod="kuadrant-system/authorino-6dd86584-c8wws"
Jul 05 00:22:54.830244 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:22:54.830075    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ffnld\" (UniqueName: \"kubernetes.io/projected/fb32d8dc-c25b-4d48-8f10-1e3fdfd5ab1a-kube-api-access-ffnld\") pod \"authorino-7c876ffffb-92gdc\" (UID: \"fb32d8dc-c25b-4d48-8f10-1e3fdfd5ab1a\") " pod="kuadrant-system/authorino-7c876ffffb-92gdc"
Jul 05 00:22:54.839163 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:22:54.839131    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-ffnld\" (UniqueName: \"kubernetes.io/projected/fb32d8dc-c25b-4d48-8f10-1e3fdfd5ab1a-kube-api-access-ffnld\") pod \"authorino-7c876ffffb-92gdc\" (UID: \"fb32d8dc-c25b-4d48-8f10-1e3fdfd5ab1a\") " pod="kuadrant-system/authorino-7c876ffffb-92gdc"
Jul 05 00:22:54.930793 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:22:54.930714    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ldg76\" (UniqueName: \"kubernetes.io/projected/dea7f9d0-0e6a-4be9-bb32-fc570dbcadab-kube-api-access-ldg76\") pod \"authorino-6dd86584-c8wws\" (UID: \"dea7f9d0-0e6a-4be9-bb32-fc570dbcadab\") " pod="kuadrant-system/authorino-6dd86584-c8wws"
Jul 05 00:22:54.930793 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:22:54.930753    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/dea7f9d0-0e6a-4be9-bb32-fc570dbcadab-tls-cert\") pod \"authorino-6dd86584-c8wws\" (UID: \"dea7f9d0-0e6a-4be9-bb32-fc570dbcadab\") " pod="kuadrant-system/authorino-6dd86584-c8wws"
Jul 05 00:22:54.933074 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:22:54.933048    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/dea7f9d0-0e6a-4be9-bb32-fc570dbcadab-tls-cert\") pod \"authorino-6dd86584-c8wws\" (UID: \"dea7f9d0-0e6a-4be9-bb32-fc570dbcadab\") " pod="kuadrant-system/authorino-6dd86584-c8wws"
Jul 05 00:22:54.937597 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:22:54.937579    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-ldg76\" (UniqueName: \"kubernetes.io/projected/dea7f9d0-0e6a-4be9-bb32-fc570dbcadab-kube-api-access-ldg76\") pod \"authorino-6dd86584-c8wws\" (UID: \"dea7f9d0-0e6a-4be9-bb32-fc570dbcadab\") " pod="kuadrant-system/authorino-6dd86584-c8wws"
Jul 05 00:22:55.051911 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:22:55.051879    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-6dd86584-c8wws"
Jul 05 00:22:55.173960 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:22:55.173887    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-6dd86584-c8wws"]
Jul 05 00:22:55.176153 ip-10-0-142-202 kubenswrapper[2563]: W0705 00:22:55.176130    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-poddea7f9d0_0e6a_4be9_bb32_fc570dbcadab.slice/crio-2d6038b3a55028b286ae046ac0d1989eb8dd681be8aff44faccff10ede6062ab WatchSource:0}: Error finding container 2d6038b3a55028b286ae046ac0d1989eb8dd681be8aff44faccff10ede6062ab: Status 404 returned error can't find the container with id 2d6038b3a55028b286ae046ac0d1989eb8dd681be8aff44faccff10ede6062ab
Jul 05 00:22:55.313439 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:22:55.313402    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-5d4b67c446-cqr29" event={"ID":"b821dfde-cd45-4fda-87d1-0c70365ecfd0","Type":"ContainerStarted","Data":"3356d8d9b0a6bfe5b4afa92706fb2818e33d6b31927bc6d3d57d5739e44d692d"}
Jul 05 00:22:55.313640 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:22:55.313450    2563 kuberuntime_container.go:872] "Killing container with a grace period" pod="kuadrant-system/authorino-5d4b67c446-cqr29" podUID="b821dfde-cd45-4fda-87d1-0c70365ecfd0" containerName="authorino" containerID="cri-o://3356d8d9b0a6bfe5b4afa92706fb2818e33d6b31927bc6d3d57d5739e44d692d" gracePeriod=30
Jul 05 00:22:55.313640 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:22:55.313456    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-5d4b67c446-cqr29" event={"ID":"b821dfde-cd45-4fda-87d1-0c70365ecfd0","Type":"ContainerStarted","Data":"5f14a65532617d74705428237d6aeecf496a2b08838facc40701995796a5b865"}
Jul 05 00:22:55.314817 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:22:55.314792    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-7c876ffffb-92gdc"
Jul 05 00:22:55.314817 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:22:55.314811    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-6dd86584-c8wws" event={"ID":"dea7f9d0-0e6a-4be9-bb32-fc570dbcadab","Type":"ContainerStarted","Data":"2d6038b3a55028b286ae046ac0d1989eb8dd681be8aff44faccff10ede6062ab"}
Jul 05 00:22:55.321633 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:22:55.321587    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-7c876ffffb-92gdc"
Jul 05 00:22:55.325620 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:22:55.325574    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-5d4b67c446-cqr29" podStartSLOduration=0.88251123 podStartE2EDuration="1.32555776s" totalImagesPullingTime="443.04653ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-05 00:22:54 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-05 00:22:55.325261022 +0000 UTC m=+823.667590102" watchObservedRunningTime="2026-07-05 00:22:55.32555776 +0000 UTC m=+823.667886821"
Jul 05 00:22:55.334802 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:22:55.334782    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-ffnld\" (UniqueName: \"kubernetes.io/projected/fb32d8dc-c25b-4d48-8f10-1e3fdfd5ab1a-kube-api-access-ffnld\") pod \"fb32d8dc-c25b-4d48-8f10-1e3fdfd5ab1a\" (UID: \"fb32d8dc-c25b-4d48-8f10-1e3fdfd5ab1a\") "
Jul 05 00:22:55.336956 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:22:55.336929    2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/fb32d8dc-c25b-4d48-8f10-1e3fdfd5ab1a-kube-api-access-ffnld" (OuterVolumeSpecName: "kube-api-access-ffnld") pod "fb32d8dc-c25b-4d48-8f10-1e3fdfd5ab1a" (UID: "fb32d8dc-c25b-4d48-8f10-1e3fdfd5ab1a"). InnerVolumeSpecName "kube-api-access-ffnld". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 05 00:22:55.436347 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:22:55.436287    2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-ffnld\" (UniqueName: \"kubernetes.io/projected/fb32d8dc-c25b-4d48-8f10-1e3fdfd5ab1a-kube-api-access-ffnld\") on node \"ip-10-0-142-202.ec2.internal\" DevicePath \"\""
Jul 05 00:22:55.603562 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:22:55.603544    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5d4b67c446-cqr29"
Jul 05 00:22:55.637424 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:22:55.637388    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-gvrvg\" (UniqueName: \"kubernetes.io/projected/b821dfde-cd45-4fda-87d1-0c70365ecfd0-kube-api-access-gvrvg\") pod \"b821dfde-cd45-4fda-87d1-0c70365ecfd0\" (UID: \"b821dfde-cd45-4fda-87d1-0c70365ecfd0\") "
Jul 05 00:22:55.639605 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:22:55.639580    2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b821dfde-cd45-4fda-87d1-0c70365ecfd0-kube-api-access-gvrvg" (OuterVolumeSpecName: "kube-api-access-gvrvg") pod "b821dfde-cd45-4fda-87d1-0c70365ecfd0" (UID: "b821dfde-cd45-4fda-87d1-0c70365ecfd0"). InnerVolumeSpecName "kube-api-access-gvrvg". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 05 00:22:55.738381 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:22:55.738300    2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-gvrvg\" (UniqueName: \"kubernetes.io/projected/b821dfde-cd45-4fda-87d1-0c70365ecfd0-kube-api-access-gvrvg\") on node \"ip-10-0-142-202.ec2.internal\" DevicePath \"\""
Jul 05 00:22:56.320591 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:22:56.320560    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-6dd86584-c8wws" event={"ID":"dea7f9d0-0e6a-4be9-bb32-fc570dbcadab","Type":"ContainerStarted","Data":"8bedac2d6edd36f1bbb92bb93daa2f94da2a1c0c775eeb2117002f98c02d2087"}
Jul 05 00:22:56.321771 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:22:56.321747    2563 generic.go:356] "Generic (PLEG): container finished" podID="b821dfde-cd45-4fda-87d1-0c70365ecfd0" containerID="3356d8d9b0a6bfe5b4afa92706fb2818e33d6b31927bc6d3d57d5739e44d692d" exitCode=0
Jul 05 00:22:56.321855 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:22:56.321776    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5d4b67c446-cqr29"
Jul 05 00:22:56.321855 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:22:56.321814    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-5d4b67c446-cqr29" event={"ID":"b821dfde-cd45-4fda-87d1-0c70365ecfd0","Type":"ContainerDied","Data":"3356d8d9b0a6bfe5b4afa92706fb2818e33d6b31927bc6d3d57d5739e44d692d"}
Jul 05 00:22:56.321855 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:22:56.321844    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-5d4b67c446-cqr29" event={"ID":"b821dfde-cd45-4fda-87d1-0c70365ecfd0","Type":"ContainerDied","Data":"5f14a65532617d74705428237d6aeecf496a2b08838facc40701995796a5b865"}
Jul 05 00:22:56.321968 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:22:56.321863    2563 scope.go:117] "RemoveContainer" containerID="3356d8d9b0a6bfe5b4afa92706fb2818e33d6b31927bc6d3d57d5739e44d692d"
Jul 05 00:22:56.321968 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:22:56.321918    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-7c876ffffb-92gdc"
Jul 05 00:22:56.329862 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:22:56.329835    2563 scope.go:117] "RemoveContainer" containerID="3356d8d9b0a6bfe5b4afa92706fb2818e33d6b31927bc6d3d57d5739e44d692d"
Jul 05 00:22:56.330100 ip-10-0-142-202 kubenswrapper[2563]: E0705 00:22:56.330085    2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3356d8d9b0a6bfe5b4afa92706fb2818e33d6b31927bc6d3d57d5739e44d692d\": container with ID starting with 3356d8d9b0a6bfe5b4afa92706fb2818e33d6b31927bc6d3d57d5739e44d692d not found: ID does not exist" containerID="3356d8d9b0a6bfe5b4afa92706fb2818e33d6b31927bc6d3d57d5739e44d692d"
Jul 05 00:22:56.330161 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:22:56.330110    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3356d8d9b0a6bfe5b4afa92706fb2818e33d6b31927bc6d3d57d5739e44d692d"} err="failed to get container status \"3356d8d9b0a6bfe5b4afa92706fb2818e33d6b31927bc6d3d57d5739e44d692d\": rpc error: code = NotFound desc = could not find container \"3356d8d9b0a6bfe5b4afa92706fb2818e33d6b31927bc6d3d57d5739e44d692d\": container with ID starting with 3356d8d9b0a6bfe5b4afa92706fb2818e33d6b31927bc6d3d57d5739e44d692d not found: ID does not exist"
Jul 05 00:22:56.334424 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:22:56.334386    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-6dd86584-c8wws" podStartSLOduration=1.998823827 podStartE2EDuration="2.33437345s" totalImagesPullingTime="335.549623ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-05 00:22:54 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-05 00:22:56.333266786 +0000 UTC m=+824.675595867" watchObservedRunningTime="2026-07-05 00:22:56.33437345 +0000 UTC m=+824.676702509"
Jul 05 00:22:56.347533 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:22:56.347509    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-5d4b67c446-cqr29"]
Jul 05 00:22:56.352495 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:22:56.352457    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-5d4b67c446-cqr29"]
Jul 05 00:22:56.358644 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:22:56.358623    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-75d556bd77-m97jz"]
Jul 05 00:22:56.358837 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:22:56.358814    2563 kuberuntime_container.go:872] "Killing container with a grace period" pod="kuadrant-system/authorino-75d556bd77-m97jz" podUID="2114e1cf-4ecf-41ec-a1aa-71394e96204e" containerName="authorino" containerID="cri-o://8853bc32aaa4b556d946b4474f8f908eae0247e867c3157e0db0e9026c877555" gracePeriod=30
Jul 05 00:22:56.375583 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:22:56.375563    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-7c876ffffb-92gdc"]
Jul 05 00:22:56.380489 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:22:56.380468    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-7c876ffffb-92gdc"]
Jul 05 00:22:56.613029 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:22:56.613007    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-75d556bd77-m97jz"
Jul 05 00:22:56.647061 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:22:56.647030    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-8n4mz\" (UniqueName: \"kubernetes.io/projected/2114e1cf-4ecf-41ec-a1aa-71394e96204e-kube-api-access-8n4mz\") pod \"2114e1cf-4ecf-41ec-a1aa-71394e96204e\" (UID: \"2114e1cf-4ecf-41ec-a1aa-71394e96204e\") "
Jul 05 00:22:56.649064 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:22:56.649041    2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2114e1cf-4ecf-41ec-a1aa-71394e96204e-kube-api-access-8n4mz" (OuterVolumeSpecName: "kube-api-access-8n4mz") pod "2114e1cf-4ecf-41ec-a1aa-71394e96204e" (UID: "2114e1cf-4ecf-41ec-a1aa-71394e96204e"). InnerVolumeSpecName "kube-api-access-8n4mz". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 05 00:22:56.747652 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:22:56.747581    2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-8n4mz\" (UniqueName: \"kubernetes.io/projected/2114e1cf-4ecf-41ec-a1aa-71394e96204e-kube-api-access-8n4mz\") on node \"ip-10-0-142-202.ec2.internal\" DevicePath \"\""
Jul 05 00:22:57.328331 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:22:57.328298    2563 generic.go:356] "Generic (PLEG): container finished" podID="2114e1cf-4ecf-41ec-a1aa-71394e96204e" containerID="8853bc32aaa4b556d946b4474f8f908eae0247e867c3157e0db0e9026c877555" exitCode=0
Jul 05 00:22:57.328512 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:22:57.328361    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-75d556bd77-m97jz"
Jul 05 00:22:57.328512 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:22:57.328404    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-75d556bd77-m97jz" event={"ID":"2114e1cf-4ecf-41ec-a1aa-71394e96204e","Type":"ContainerDied","Data":"8853bc32aaa4b556d946b4474f8f908eae0247e867c3157e0db0e9026c877555"}
Jul 05 00:22:57.328512 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:22:57.328450    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-75d556bd77-m97jz" event={"ID":"2114e1cf-4ecf-41ec-a1aa-71394e96204e","Type":"ContainerDied","Data":"448f0ebbd194161b211cb9081448fb53673a230169030cdd89b922783397374d"}
Jul 05 00:22:57.328512 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:22:57.328469    2563 scope.go:117] "RemoveContainer" containerID="8853bc32aaa4b556d946b4474f8f908eae0247e867c3157e0db0e9026c877555"
Jul 05 00:22:57.336666 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:22:57.336649    2563 scope.go:117] "RemoveContainer" containerID="8853bc32aaa4b556d946b4474f8f908eae0247e867c3157e0db0e9026c877555"
Jul 05 00:22:57.336927 ip-10-0-142-202 kubenswrapper[2563]: E0705 00:22:57.336896    2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8853bc32aaa4b556d946b4474f8f908eae0247e867c3157e0db0e9026c877555\": container with ID starting with 8853bc32aaa4b556d946b4474f8f908eae0247e867c3157e0db0e9026c877555 not found: ID does not exist" containerID="8853bc32aaa4b556d946b4474f8f908eae0247e867c3157e0db0e9026c877555"
Jul 05 00:22:57.337017 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:22:57.336932    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8853bc32aaa4b556d946b4474f8f908eae0247e867c3157e0db0e9026c877555"} err="failed to get container status \"8853bc32aaa4b556d946b4474f8f908eae0247e867c3157e0db0e9026c877555\": rpc error: code = NotFound desc = could not find container \"8853bc32aaa4b556d946b4474f8f908eae0247e867c3157e0db0e9026c877555\": container with ID starting with 8853bc32aaa4b556d946b4474f8f908eae0247e867c3157e0db0e9026c877555 not found: ID does not exist"
Jul 05 00:22:57.346871 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:22:57.346851    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-75d556bd77-m97jz"]
Jul 05 00:22:57.350139 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:22:57.350121    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-75d556bd77-m97jz"]
Jul 05 00:22:58.125579 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:22:58.125543    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="2114e1cf-4ecf-41ec-a1aa-71394e96204e" path="/var/lib/kubelet/pods/2114e1cf-4ecf-41ec-a1aa-71394e96204e/volumes"
Jul 05 00:22:58.126054 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:22:58.126031    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b821dfde-cd45-4fda-87d1-0c70365ecfd0" path="/var/lib/kubelet/pods/b821dfde-cd45-4fda-87d1-0c70365ecfd0/volumes"
Jul 05 00:22:58.126397 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:22:58.126377    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="fb32d8dc-c25b-4d48-8f10-1e3fdfd5ab1a" path="/var/lib/kubelet/pods/fb32d8dc-c25b-4d48-8f10-1e3fdfd5ab1a/volumes"
Jul 05 00:23:35.386461 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:23:35.386425    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-x5szm"]
Jul 05 00:23:43.529627 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:23:43.529592    2563 generic.go:356] "Generic (PLEG): container finished" podID="a95fc6e8-4791-4960-b7f5-2d8e09342c4a" containerID="f7e8f57700e7d7d729d4c873a8bf679a15faa605eb74bd76ec10a7633c0d246b" exitCode=0
Jul 05 00:23:43.530043 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:23:43.529665    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-wzqkx" event={"ID":"a95fc6e8-4791-4960-b7f5-2d8e09342c4a","Type":"ContainerDied","Data":"f7e8f57700e7d7d729d4c873a8bf679a15faa605eb74bd76ec10a7633c0d246b"}
Jul 05 00:23:43.530043 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:23:43.530023    2563 scope.go:117] "RemoveContainer" containerID="f7e8f57700e7d7d729d4c873a8bf679a15faa605eb74bd76ec10a7633c0d246b"
Jul 05 00:23:43.530981 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:23:43.530953    2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"}
Jul 05 00:23:43.531054 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:23:43.531041    2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"}
Jul 05 00:23:43.531098 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:23:43.531076    2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"}
Jul 05 00:23:44.535862 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:23:44.535832    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-wzqkx" event={"ID":"a95fc6e8-4791-4960-b7f5-2d8e09342c4a","Type":"ContainerStarted","Data":"8d9f60f1c5470a566b12948e7f027c935319488013ce2b5f3f8e941671a3bda8"}
Jul 05 00:23:45.075865 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:23:45.075770    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-wzqkx"
Jul 05 00:23:45.077279 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:23:45.077254    2563 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-56654dcf7-wzqkx container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.23:15021/healthz/ready\": dial tcp 10.133.0.23:15021: connect: connection refused" start-of-body=
Jul 05 00:23:45.077485 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:23:45.077458    2563 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-wzqkx" podUID="a95fc6e8-4791-4960-b7f5-2d8e09342c4a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.23:15021/healthz/ready\": dial tcp 10.133.0.23:15021: connect: connection refused"
Jul 05 00:23:46.076362 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:23:46.076326    2563 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-56654dcf7-wzqkx container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.23:15021/healthz/ready\": dial tcp 10.133.0.23:15021: connect: connection refused" start-of-body=
Jul 05 00:23:46.076914 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:23:46.076883    2563 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-wzqkx" podUID="a95fc6e8-4791-4960-b7f5-2d8e09342c4a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.23:15021/healthz/ready\": dial tcp 10.133.0.23:15021: connect: connection refused"
Jul 05 00:23:47.075681 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:23:47.075639    2563 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-56654dcf7-wzqkx container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.23:15021/healthz/ready\": dial tcp 10.133.0.23:15021: connect: connection refused" start-of-body=
Jul 05 00:23:47.075888 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:23:47.075707    2563 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-wzqkx" podUID="a95fc6e8-4791-4960-b7f5-2d8e09342c4a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.23:15021/healthz/ready\": dial tcp 10.133.0.23:15021: connect: connection refused"
Jul 05 00:23:48.075695 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:23:48.075656    2563 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-56654dcf7-wzqkx container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.23:15021/healthz/ready\": dial tcp 10.133.0.23:15021: connect: connection refused" start-of-body=
Jul 05 00:23:48.076284 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:23:48.076254    2563 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-wzqkx" podUID="a95fc6e8-4791-4960-b7f5-2d8e09342c4a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.23:15021/healthz/ready\": dial tcp 10.133.0.23:15021: connect: connection refused"
Jul 05 00:23:49.078411 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:23:49.078368    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-wzqkx"
Jul 05 00:23:49.080131 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:23:49.080111    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-wzqkx"
Jul 05 00:23:49.559128 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:23:49.559103    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-wzqkx"
Jul 05 00:23:52.536738 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:23:52.536704    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-x5szm"]
Jul 05 00:24:03.127299 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:24:03.127267    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-x5szm"]
Jul 05 00:24:12.945421 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:24:12.945395    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-7sxq4_b96bfb4d-2fd3-4be3-9064-e24efeb904f4/ovn-acl-logging/0.log"
Jul 05 00:24:12.945855 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:24:12.945647    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-7sxq4_b96bfb4d-2fd3-4be3-9064-e24efeb904f4/ovn-acl-logging/0.log"
Jul 05 00:24:14.331710 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:24:14.331677    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-x5szm"]
Jul 05 00:24:22.320843 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:24:22.320813    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-x5szm"]
Jul 05 00:24:30.205076 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:24:30.205040    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-74b68447df-2p6f8"]
Jul 05 00:24:30.205461 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:24:30.205402    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2114e1cf-4ecf-41ec-a1aa-71394e96204e" containerName="authorino"
Jul 05 00:24:30.205461 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:24:30.205416    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="2114e1cf-4ecf-41ec-a1aa-71394e96204e" containerName="authorino"
Jul 05 00:24:30.205461 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:24:30.205429    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b821dfde-cd45-4fda-87d1-0c70365ecfd0" containerName="authorino"
Jul 05 00:24:30.205461 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:24:30.205435    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="b821dfde-cd45-4fda-87d1-0c70365ecfd0" containerName="authorino"
Jul 05 00:24:30.205592 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:24:30.205488    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="2114e1cf-4ecf-41ec-a1aa-71394e96204e" containerName="authorino"
Jul 05 00:24:30.205592 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:24:30.205497    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="b821dfde-cd45-4fda-87d1-0c70365ecfd0" containerName="authorino"
Jul 05 00:24:30.207549 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:24:30.207534    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-2p6f8"
Jul 05 00:24:30.209658 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:24:30.209634    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"openshift-service-ca.crt\""
Jul 05 00:24:30.209780 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:24:30.209634    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"kube-root-ca.crt\""
Jul 05 00:24:30.210163 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:24:30.210145    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"facebook-opt-125m-simulated-kserve-self-signed-certs\""
Jul 05 00:24:30.210279 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:24:30.210194    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"default-dockercfg-vz78f\""
Jul 05 00:24:30.217242 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:24:30.217204    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-74b68447df-2p6f8"]
Jul 05 00:24:30.256952 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:24:30.256926    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/5669b7cd-6e37-4231-bce1-83061f981f12-home\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-2p6f8\" (UID: \"5669b7cd-6e37-4231-bce1-83061f981f12\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-2p6f8"
Jul 05 00:24:30.256952 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:24:30.256953    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/5669b7cd-6e37-4231-bce1-83061f981f12-model-cache\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-2p6f8\" (UID: \"5669b7cd-6e37-4231-bce1-83061f981f12\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-2p6f8"
Jul 05 00:24:30.257093 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:24:30.256970    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/5669b7cd-6e37-4231-bce1-83061f981f12-tmp-dir\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-2p6f8\" (UID: \"5669b7cd-6e37-4231-bce1-83061f981f12\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-2p6f8"
Jul 05 00:24:30.257093 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:24:30.256988    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/5669b7cd-6e37-4231-bce1-83061f981f12-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-2p6f8\" (UID: \"5669b7cd-6e37-4231-bce1-83061f981f12\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-2p6f8"
Jul 05 00:24:30.257093 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:24:30.257047    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/5669b7cd-6e37-4231-bce1-83061f981f12-dshm\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-2p6f8\" (UID: \"5669b7cd-6e37-4231-bce1-83061f981f12\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-2p6f8"
Jul 05 00:24:30.257093 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:24:30.257063    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lbvdg\" (UniqueName: \"kubernetes.io/projected/5669b7cd-6e37-4231-bce1-83061f981f12-kube-api-access-lbvdg\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-2p6f8\" (UID: \"5669b7cd-6e37-4231-bce1-83061f981f12\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-2p6f8"
Jul 05 00:24:30.357895 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:24:30.357865    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/5669b7cd-6e37-4231-bce1-83061f981f12-home\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-2p6f8\" (UID: \"5669b7cd-6e37-4231-bce1-83061f981f12\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-2p6f8"
Jul 05 00:24:30.357895 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:24:30.357895    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/5669b7cd-6e37-4231-bce1-83061f981f12-model-cache\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-2p6f8\" (UID: \"5669b7cd-6e37-4231-bce1-83061f981f12\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-2p6f8"
Jul 05 00:24:30.358067 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:24:30.357914    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/5669b7cd-6e37-4231-bce1-83061f981f12-tmp-dir\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-2p6f8\" (UID: \"5669b7cd-6e37-4231-bce1-83061f981f12\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-2p6f8"
Jul 05 00:24:30.358067 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:24:30.357938    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/5669b7cd-6e37-4231-bce1-83061f981f12-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-2p6f8\" (UID: \"5669b7cd-6e37-4231-bce1-83061f981f12\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-2p6f8"
Jul 05 00:24:30.358067 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:24:30.357985    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/5669b7cd-6e37-4231-bce1-83061f981f12-dshm\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-2p6f8\" (UID: \"5669b7cd-6e37-4231-bce1-83061f981f12\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-2p6f8"
Jul 05 00:24:30.358067 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:24:30.358011    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lbvdg\" (UniqueName: \"kubernetes.io/projected/5669b7cd-6e37-4231-bce1-83061f981f12-kube-api-access-lbvdg\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-2p6f8\" (UID: \"5669b7cd-6e37-4231-bce1-83061f981f12\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-2p6f8"
Jul 05 00:24:30.358355 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:24:30.358337    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/5669b7cd-6e37-4231-bce1-83061f981f12-model-cache\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-2p6f8\" (UID: \"5669b7cd-6e37-4231-bce1-83061f981f12\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-2p6f8"
Jul 05 00:24:30.358426 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:24:30.358364    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/5669b7cd-6e37-4231-bce1-83061f981f12-home\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-2p6f8\" (UID: \"5669b7cd-6e37-4231-bce1-83061f981f12\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-2p6f8"
Jul 05 00:24:30.358426 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:24:30.358377    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/5669b7cd-6e37-4231-bce1-83061f981f12-tmp-dir\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-2p6f8\" (UID: \"5669b7cd-6e37-4231-bce1-83061f981f12\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-2p6f8"
Jul 05 00:24:30.360254 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:24:30.360205    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/5669b7cd-6e37-4231-bce1-83061f981f12-dshm\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-2p6f8\" (UID: \"5669b7cd-6e37-4231-bce1-83061f981f12\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-2p6f8"
Jul 05 00:24:30.360444 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:24:30.360429    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/5669b7cd-6e37-4231-bce1-83061f981f12-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-2p6f8\" (UID: \"5669b7cd-6e37-4231-bce1-83061f981f12\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-2p6f8"
Jul 05 00:24:30.364784 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:24:30.364763    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-lbvdg\" (UniqueName: \"kubernetes.io/projected/5669b7cd-6e37-4231-bce1-83061f981f12-kube-api-access-lbvdg\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-2p6f8\" (UID: \"5669b7cd-6e37-4231-bce1-83061f981f12\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-2p6f8"
Jul 05 00:24:30.519209 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:24:30.519127    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-2p6f8"
Jul 05 00:24:30.645552 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:24:30.645506    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-74b68447df-2p6f8"]
Jul 05 00:24:30.647940 ip-10-0-142-202 kubenswrapper[2563]: W0705 00:24:30.647914    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5669b7cd_6e37_4231_bce1_83061f981f12.slice/crio-a130e85fee124b3ae7b313a37eae3eb276a05e8dcbc2455035068fd884d6e065 WatchSource:0}: Error finding container a130e85fee124b3ae7b313a37eae3eb276a05e8dcbc2455035068fd884d6e065: Status 404 returned error can't find the container with id a130e85fee124b3ae7b313a37eae3eb276a05e8dcbc2455035068fd884d6e065
Jul 05 00:24:30.732149 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:24:30.732117    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-2p6f8" event={"ID":"5669b7cd-6e37-4231-bce1-83061f981f12","Type":"ContainerStarted","Data":"a130e85fee124b3ae7b313a37eae3eb276a05e8dcbc2455035068fd884d6e065"}
Jul 05 00:24:31.737483 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:24:31.737444    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-2p6f8" event={"ID":"5669b7cd-6e37-4231-bce1-83061f981f12","Type":"ContainerStarted","Data":"3031d32cf3c172c2608966b7d01d6c067bcff02c9d37564b4e5abce38ab82f88"}
Jul 05 00:24:31.753488 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:24:31.753440    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-2p6f8" podStartSLOduration=0.801016319 podStartE2EDuration="1.753424406s" totalImagesPullingTime="952.408087ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-05 00:24:30 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-05 00:24:31.75133943 +0000 UTC m=+920.093668489" watchObservedRunningTime="2026-07-05 00:24:31.753424406 +0000 UTC m=+920.095753466"
Jul 05 00:24:32.328434 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:24:32.328404    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-x5szm"]
Jul 05 00:24:34.796203 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:24:34.796171    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-ql2w2"]
Jul 05 00:24:34.798983 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:24:34.798965    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-ql2w2"
Jul 05 00:24:34.801354 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:24:34.801331    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"premium-simulated-simulated-premium-kserve-self-signed-certs\""
Jul 05 00:24:34.809194 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:24:34.809159    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-ql2w2"]
Jul 05 00:24:34.897352 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:24:34.897301    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/731bd138-d4b7-4193-9f52-4bf2c656c123-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-ql2w2\" (UID: \"731bd138-d4b7-4193-9f52-4bf2c656c123\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-ql2w2"
Jul 05 00:24:34.897532 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:24:34.897405    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/731bd138-d4b7-4193-9f52-4bf2c656c123-dshm\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-ql2w2\" (UID: \"731bd138-d4b7-4193-9f52-4bf2c656c123\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-ql2w2"
Jul 05 00:24:34.897532 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:24:34.897428    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/731bd138-d4b7-4193-9f52-4bf2c656c123-tmp-dir\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-ql2w2\" (UID: \"731bd138-d4b7-4193-9f52-4bf2c656c123\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-ql2w2"
Jul 05 00:24:34.897532 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:24:34.897460    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/731bd138-d4b7-4193-9f52-4bf2c656c123-model-cache\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-ql2w2\" (UID: \"731bd138-d4b7-4193-9f52-4bf2c656c123\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-ql2w2"
Jul 05 00:24:34.897532 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:24:34.897487    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/731bd138-d4b7-4193-9f52-4bf2c656c123-home\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-ql2w2\" (UID: \"731bd138-d4b7-4193-9f52-4bf2c656c123\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-ql2w2"
Jul 05 00:24:34.897532 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:24:34.897505    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xcqdm\" (UniqueName: \"kubernetes.io/projected/731bd138-d4b7-4193-9f52-4bf2c656c123-kube-api-access-xcqdm\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-ql2w2\" (UID: \"731bd138-d4b7-4193-9f52-4bf2c656c123\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-ql2w2"
Jul 05 00:24:34.998587 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:24:34.998554    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/731bd138-d4b7-4193-9f52-4bf2c656c123-dshm\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-ql2w2\" (UID: \"731bd138-d4b7-4193-9f52-4bf2c656c123\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-ql2w2"
Jul 05 00:24:34.998587 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:24:34.998589    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/731bd138-d4b7-4193-9f52-4bf2c656c123-tmp-dir\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-ql2w2\" (UID: \"731bd138-d4b7-4193-9f52-4bf2c656c123\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-ql2w2"
Jul 05 00:24:34.998787 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:24:34.998609    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/731bd138-d4b7-4193-9f52-4bf2c656c123-model-cache\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-ql2w2\" (UID: \"731bd138-d4b7-4193-9f52-4bf2c656c123\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-ql2w2"
Jul 05 00:24:34.998787 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:24:34.998631    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/731bd138-d4b7-4193-9f52-4bf2c656c123-home\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-ql2w2\" (UID: \"731bd138-d4b7-4193-9f52-4bf2c656c123\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-ql2w2"
Jul 05 00:24:34.998787 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:24:34.998747    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xcqdm\" (UniqueName: \"kubernetes.io/projected/731bd138-d4b7-4193-9f52-4bf2c656c123-kube-api-access-xcqdm\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-ql2w2\" (UID: \"731bd138-d4b7-4193-9f52-4bf2c656c123\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-ql2w2"
Jul 05 00:24:34.998907 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:24:34.998811    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/731bd138-d4b7-4193-9f52-4bf2c656c123-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-ql2w2\" (UID: \"731bd138-d4b7-4193-9f52-4bf2c656c123\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-ql2w2"
Jul 05 00:24:34.999010 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:24:34.998994    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/731bd138-d4b7-4193-9f52-4bf2c656c123-home\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-ql2w2\" (UID: \"731bd138-d4b7-4193-9f52-4bf2c656c123\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-ql2w2"
Jul 05 00:24:34.999083 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:24:34.999062    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/731bd138-d4b7-4193-9f52-4bf2c656c123-tmp-dir\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-ql2w2\" (UID: \"731bd138-d4b7-4193-9f52-4bf2c656c123\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-ql2w2"
Jul 05 00:24:34.999140 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:24:34.999096    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/731bd138-d4b7-4193-9f52-4bf2c656c123-model-cache\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-ql2w2\" (UID: \"731bd138-d4b7-4193-9f52-4bf2c656c123\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-ql2w2"
Jul 05 00:24:35.000890 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:24:35.000863    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/731bd138-d4b7-4193-9f52-4bf2c656c123-dshm\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-ql2w2\" (UID: \"731bd138-d4b7-4193-9f52-4bf2c656c123\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-ql2w2"
Jul 05 00:24:35.001199 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:24:35.001181    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/731bd138-d4b7-4193-9f52-4bf2c656c123-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-ql2w2\" (UID: \"731bd138-d4b7-4193-9f52-4bf2c656c123\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-ql2w2"
Jul 05 00:24:35.005456 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:24:35.005435    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-xcqdm\" (UniqueName: \"kubernetes.io/projected/731bd138-d4b7-4193-9f52-4bf2c656c123-kube-api-access-xcqdm\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-ql2w2\" (UID: \"731bd138-d4b7-4193-9f52-4bf2c656c123\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-ql2w2"
Jul 05 00:24:35.108614 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:24:35.108532    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-ql2w2"
Jul 05 00:24:35.233535 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:24:35.233503    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-ql2w2"]
Jul 05 00:24:35.236473 ip-10-0-142-202 kubenswrapper[2563]: W0705 00:24:35.236450    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod731bd138_d4b7_4193_9f52_4bf2c656c123.slice/crio-0f7dcd15fdccf9fcb3e6e6576ed44566e4ae8abd535cf8427dfdc5b2055e2f41 WatchSource:0}: Error finding container 0f7dcd15fdccf9fcb3e6e6576ed44566e4ae8abd535cf8427dfdc5b2055e2f41: Status 404 returned error can't find the container with id 0f7dcd15fdccf9fcb3e6e6576ed44566e4ae8abd535cf8427dfdc5b2055e2f41
Jul 05 00:24:35.755031 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:24:35.754934    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-ql2w2" event={"ID":"731bd138-d4b7-4193-9f52-4bf2c656c123","Type":"ContainerStarted","Data":"8415053fb04fb16757f287dfacb3e96d757e85303de995fb31e5eeb072e70111"}
Jul 05 00:24:35.755031 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:24:35.754984    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-ql2w2" event={"ID":"731bd138-d4b7-4193-9f52-4bf2c656c123","Type":"ContainerStarted","Data":"0f7dcd15fdccf9fcb3e6e6576ed44566e4ae8abd535cf8427dfdc5b2055e2f41"}
Jul 05 00:24:35.769361 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:24:35.769311    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-ql2w2" podStartSLOduration=1.598855581 podStartE2EDuration="1.769296436s" totalImagesPullingTime="170.440855ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-05 00:24:34 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-05 00:24:35.768716075 +0000 UTC m=+924.111045136" watchObservedRunningTime="2026-07-05 00:24:35.769296436 +0000 UTC m=+924.111625496"
Jul 05 00:24:38.728909 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:24:38.728874    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-x5szm"]
Jul 05 00:24:40.519627 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:24:40.519592    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-2p6f8"
Jul 05 00:24:40.519997 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:24:40.519658    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-2p6f8"
Jul 05 00:24:40.531917 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:24:40.531890    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-2p6f8"
Jul 05 00:24:40.791251 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:24:40.791202    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-2p6f8"
Jul 05 00:24:45.109467 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:24:45.109431    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-ql2w2"
Jul 05 00:24:45.109978 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:24:45.109510    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-ql2w2"
Jul 05 00:24:45.122536 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:24:45.122514    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-ql2w2"
Jul 05 00:24:45.817804 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:24:45.817776    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-ql2w2"
Jul 05 00:25:30.757951 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:25:30.757919    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-7694cb889c-r9vqc"]
Jul 05 00:25:30.760938 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:25:30.760922    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-7694cb889c-r9vqc"
Jul 05 00:25:30.769726 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:25:30.769701    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-7694cb889c-r9vqc"]
Jul 05 00:25:30.855442 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:25:30.855412    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/aa1985d1-619d-40d7-8734-5ae6e64d24ee-tls-cert\") pod \"authorino-7694cb889c-r9vqc\" (UID: \"aa1985d1-619d-40d7-8734-5ae6e64d24ee\") " pod="kuadrant-system/authorino-7694cb889c-r9vqc"
Jul 05 00:25:30.855603 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:25:30.855471    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2g68n\" (UniqueName: \"kubernetes.io/projected/aa1985d1-619d-40d7-8734-5ae6e64d24ee-kube-api-access-2g68n\") pod \"authorino-7694cb889c-r9vqc\" (UID: \"aa1985d1-619d-40d7-8734-5ae6e64d24ee\") " pod="kuadrant-system/authorino-7694cb889c-r9vqc"
Jul 05 00:25:30.956002 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:25:30.955970    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2g68n\" (UniqueName: \"kubernetes.io/projected/aa1985d1-619d-40d7-8734-5ae6e64d24ee-kube-api-access-2g68n\") pod \"authorino-7694cb889c-r9vqc\" (UID: \"aa1985d1-619d-40d7-8734-5ae6e64d24ee\") " pod="kuadrant-system/authorino-7694cb889c-r9vqc"
Jul 05 00:25:30.956148 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:25:30.956056    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/aa1985d1-619d-40d7-8734-5ae6e64d24ee-tls-cert\") pod \"authorino-7694cb889c-r9vqc\" (UID: \"aa1985d1-619d-40d7-8734-5ae6e64d24ee\") " pod="kuadrant-system/authorino-7694cb889c-r9vqc"
Jul 05 00:25:30.958486 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:25:30.958459    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/aa1985d1-619d-40d7-8734-5ae6e64d24ee-tls-cert\") pod \"authorino-7694cb889c-r9vqc\" (UID: \"aa1985d1-619d-40d7-8734-5ae6e64d24ee\") " pod="kuadrant-system/authorino-7694cb889c-r9vqc"
Jul 05 00:25:30.962732 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:25:30.962708    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-2g68n\" (UniqueName: \"kubernetes.io/projected/aa1985d1-619d-40d7-8734-5ae6e64d24ee-kube-api-access-2g68n\") pod \"authorino-7694cb889c-r9vqc\" (UID: \"aa1985d1-619d-40d7-8734-5ae6e64d24ee\") " pod="kuadrant-system/authorino-7694cb889c-r9vqc"
Jul 05 00:25:31.069713 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:25:31.069685    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-7694cb889c-r9vqc"
Jul 05 00:25:31.199077 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:25:31.199040    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-7694cb889c-r9vqc"]
Jul 05 00:25:31.202371 ip-10-0-142-202 kubenswrapper[2563]: W0705 00:25:31.202341    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podaa1985d1_619d_40d7_8734_5ae6e64d24ee.slice/crio-c2174bd237f927756899cbc768e1d231a522a0b86cd589e0fb01b3694ff2b24f WatchSource:0}: Error finding container c2174bd237f927756899cbc768e1d231a522a0b86cd589e0fb01b3694ff2b24f: Status 404 returned error can't find the container with id c2174bd237f927756899cbc768e1d231a522a0b86cd589e0fb01b3694ff2b24f
Jul 05 00:25:32.009271 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:25:32.009158    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-7694cb889c-r9vqc" event={"ID":"aa1985d1-619d-40d7-8734-5ae6e64d24ee","Type":"ContainerStarted","Data":"9647646c9e3b3da0d4eeaa4d28bb188631fa0ac5027c9044c86e54f84723efe6"}
Jul 05 00:25:32.009271 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:25:32.009205    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-7694cb889c-r9vqc" event={"ID":"aa1985d1-619d-40d7-8734-5ae6e64d24ee","Type":"ContainerStarted","Data":"c2174bd237f927756899cbc768e1d231a522a0b86cd589e0fb01b3694ff2b24f"}
Jul 05 00:25:32.027320 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:25:32.027260    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-7694cb889c-r9vqc" podStartSLOduration=1.674569915 podStartE2EDuration="2.027212397s" totalImagesPullingTime="352.642482ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-05 00:25:30 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-05 00:25:32.024026174 +0000 UTC m=+980.366355247" watchObservedRunningTime="2026-07-05 00:25:32.027212397 +0000 UTC m=+980.369541458"
Jul 05 00:25:32.056779 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:25:32.056740    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-6dd86584-c8wws"]
Jul 05 00:25:32.056988 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:25:32.056962    2563 kuberuntime_container.go:872] "Killing container with a grace period" pod="kuadrant-system/authorino-6dd86584-c8wws" podUID="dea7f9d0-0e6a-4be9-bb32-fc570dbcadab" containerName="authorino" containerID="cri-o://8bedac2d6edd36f1bbb92bb93daa2f94da2a1c0c775eeb2117002f98c02d2087" gracePeriod=30
Jul 05 00:25:32.306067 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:25:32.306041    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-6dd86584-c8wws"
Jul 05 00:25:32.371432 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:25:32.371402    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-ldg76\" (UniqueName: \"kubernetes.io/projected/dea7f9d0-0e6a-4be9-bb32-fc570dbcadab-kube-api-access-ldg76\") pod \"dea7f9d0-0e6a-4be9-bb32-fc570dbcadab\" (UID: \"dea7f9d0-0e6a-4be9-bb32-fc570dbcadab\") "
Jul 05 00:25:32.371614 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:25:32.371503    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/dea7f9d0-0e6a-4be9-bb32-fc570dbcadab-tls-cert\") pod \"dea7f9d0-0e6a-4be9-bb32-fc570dbcadab\" (UID: \"dea7f9d0-0e6a-4be9-bb32-fc570dbcadab\") "
Jul 05 00:25:32.374112 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:25:32.374076    2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/dea7f9d0-0e6a-4be9-bb32-fc570dbcadab-kube-api-access-ldg76" (OuterVolumeSpecName: "kube-api-access-ldg76") pod "dea7f9d0-0e6a-4be9-bb32-fc570dbcadab" (UID: "dea7f9d0-0e6a-4be9-bb32-fc570dbcadab"). InnerVolumeSpecName "kube-api-access-ldg76". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 05 00:25:32.381569 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:25:32.381545    2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/dea7f9d0-0e6a-4be9-bb32-fc570dbcadab-tls-cert" (OuterVolumeSpecName: "tls-cert") pod "dea7f9d0-0e6a-4be9-bb32-fc570dbcadab" (UID: "dea7f9d0-0e6a-4be9-bb32-fc570dbcadab"). InnerVolumeSpecName "tls-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 05 00:25:32.472628 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:25:32.472594    2563 reconciler_common.go:299] "Volume detached for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/dea7f9d0-0e6a-4be9-bb32-fc570dbcadab-tls-cert\") on node \"ip-10-0-142-202.ec2.internal\" DevicePath \"\""
Jul 05 00:25:32.472628 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:25:32.472627    2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-ldg76\" (UniqueName: \"kubernetes.io/projected/dea7f9d0-0e6a-4be9-bb32-fc570dbcadab-kube-api-access-ldg76\") on node \"ip-10-0-142-202.ec2.internal\" DevicePath \"\""
Jul 05 00:25:33.015624 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:25:33.015588    2563 generic.go:356] "Generic (PLEG): container finished" podID="dea7f9d0-0e6a-4be9-bb32-fc570dbcadab" containerID="8bedac2d6edd36f1bbb92bb93daa2f94da2a1c0c775eeb2117002f98c02d2087" exitCode=0
Jul 05 00:25:33.016096 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:25:33.015637    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-6dd86584-c8wws"
Jul 05 00:25:33.016096 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:25:33.015676    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-6dd86584-c8wws" event={"ID":"dea7f9d0-0e6a-4be9-bb32-fc570dbcadab","Type":"ContainerDied","Data":"8bedac2d6edd36f1bbb92bb93daa2f94da2a1c0c775eeb2117002f98c02d2087"}
Jul 05 00:25:33.016096 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:25:33.015733    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-6dd86584-c8wws" event={"ID":"dea7f9d0-0e6a-4be9-bb32-fc570dbcadab","Type":"ContainerDied","Data":"2d6038b3a55028b286ae046ac0d1989eb8dd681be8aff44faccff10ede6062ab"}
Jul 05 00:25:33.016096 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:25:33.015760    2563 scope.go:117] "RemoveContainer" containerID="8bedac2d6edd36f1bbb92bb93daa2f94da2a1c0c775eeb2117002f98c02d2087"
Jul 05 00:25:33.025102 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:25:33.025083    2563 scope.go:117] "RemoveContainer" containerID="8bedac2d6edd36f1bbb92bb93daa2f94da2a1c0c775eeb2117002f98c02d2087"
Jul 05 00:25:33.025455 ip-10-0-142-202 kubenswrapper[2563]: E0705 00:25:33.025427    2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8bedac2d6edd36f1bbb92bb93daa2f94da2a1c0c775eeb2117002f98c02d2087\": container with ID starting with 8bedac2d6edd36f1bbb92bb93daa2f94da2a1c0c775eeb2117002f98c02d2087 not found: ID does not exist" containerID="8bedac2d6edd36f1bbb92bb93daa2f94da2a1c0c775eeb2117002f98c02d2087"
Jul 05 00:25:33.025579 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:25:33.025466    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8bedac2d6edd36f1bbb92bb93daa2f94da2a1c0c775eeb2117002f98c02d2087"} err="failed to get container status \"8bedac2d6edd36f1bbb92bb93daa2f94da2a1c0c775eeb2117002f98c02d2087\": rpc error: code = NotFound desc = could not find container \"8bedac2d6edd36f1bbb92bb93daa2f94da2a1c0c775eeb2117002f98c02d2087\": container with ID starting with 8bedac2d6edd36f1bbb92bb93daa2f94da2a1c0c775eeb2117002f98c02d2087 not found: ID does not exist"
Jul 05 00:25:33.037433 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:25:33.037409    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-6dd86584-c8wws"]
Jul 05 00:25:33.039080 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:25:33.039062    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-6dd86584-c8wws"]
Jul 05 00:25:34.131820 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:25:34.131743    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="dea7f9d0-0e6a-4be9-bb32-fc570dbcadab" path="/var/lib/kubelet/pods/dea7f9d0-0e6a-4be9-bb32-fc570dbcadab/volumes"
Jul 05 00:26:02.331009 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:26:02.330972    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-x5szm"]
Jul 05 00:26:12.126128 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:26:12.126096    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-x5szm"]
Jul 05 00:26:22.419125 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:26:22.419092    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-x5szm"]
Jul 05 00:26:32.847468 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:26:32.847429    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-x5szm"]
Jul 05 00:26:42.623328 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:26:42.623288    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-x5szm"]
Jul 05 00:27:51.223099 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:27:51.223062    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-x5szm"]
Jul 05 00:28:00.226987 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:28:00.226954    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-x5szm"]
Jul 05 00:28:08.136885 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:28:08.136853    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-x5szm"]
Jul 05 00:28:18.525707 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:28:18.525671    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-x5szm"]
Jul 05 00:28:28.129183 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:28:28.129147    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-x5szm"]
Jul 05 00:28:37.226689 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:28:37.226597    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-x5szm"]
Jul 05 00:28:49.124590 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:28:49.124560    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-x5szm"]
Jul 05 00:28:56.126452 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:28:56.126421    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-x5szm"]
Jul 05 00:29:03.822975 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:29:03.822939    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-x5szm"]
Jul 05 00:29:12.328268 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:29:12.328237    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-x5szm"]
Jul 05 00:29:12.970819 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:29:12.970787    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-7sxq4_b96bfb4d-2fd3-4be3-9064-e24efeb904f4/ovn-acl-logging/0.log"
Jul 05 00:29:12.972843 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:29:12.972823    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-7sxq4_b96bfb4d-2fd3-4be3-9064-e24efeb904f4/ovn-acl-logging/0.log"
Jul 05 00:29:24.431287 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:29:24.431255    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-x5szm"]
Jul 05 00:29:32.932436 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:29:32.932398    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-x5szm"]
Jul 05 00:29:54.035089 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:29:54.035045    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-x5szm"]
Jul 05 00:30:00.145168 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:30:00.145133    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-api-key-cleanup-29720190-jnlkj"]
Jul 05 00:30:00.145789 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:30:00.145471    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="dea7f9d0-0e6a-4be9-bb32-fc570dbcadab" containerName="authorino"
Jul 05 00:30:00.145789 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:30:00.145481    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="dea7f9d0-0e6a-4be9-bb32-fc570dbcadab" containerName="authorino"
Jul 05 00:30:00.145789 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:30:00.145529    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="dea7f9d0-0e6a-4be9-bb32-fc570dbcadab" containerName="authorino"
Jul 05 00:30:00.148725 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:30:00.148696    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-key-cleanup-29720190-jnlkj"
Jul 05 00:30:00.150577 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:30:00.150555    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-api-dockercfg-2p4nn\""
Jul 05 00:30:00.163748 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:30:00.163475    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-key-cleanup-29720190-jnlkj"]
Jul 05 00:30:00.217015 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:30:00.216967    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fvvxv\" (UniqueName: \"kubernetes.io/projected/1837769c-81b2-45b4-9a21-cd351361f660-kube-api-access-fvvxv\") pod \"maas-api-key-cleanup-29720190-jnlkj\" (UID: \"1837769c-81b2-45b4-9a21-cd351361f660\") " pod="opendatahub/maas-api-key-cleanup-29720190-jnlkj"
Jul 05 00:30:00.318294 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:30:00.318258    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fvvxv\" (UniqueName: \"kubernetes.io/projected/1837769c-81b2-45b4-9a21-cd351361f660-kube-api-access-fvvxv\") pod \"maas-api-key-cleanup-29720190-jnlkj\" (UID: \"1837769c-81b2-45b4-9a21-cd351361f660\") " pod="opendatahub/maas-api-key-cleanup-29720190-jnlkj"
Jul 05 00:30:00.325761 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:30:00.325739    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-fvvxv\" (UniqueName: \"kubernetes.io/projected/1837769c-81b2-45b4-9a21-cd351361f660-kube-api-access-fvvxv\") pod \"maas-api-key-cleanup-29720190-jnlkj\" (UID: \"1837769c-81b2-45b4-9a21-cd351361f660\") " pod="opendatahub/maas-api-key-cleanup-29720190-jnlkj"
Jul 05 00:30:00.458963 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:30:00.458883    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-key-cleanup-29720190-jnlkj"
Jul 05 00:30:00.597264 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:30:00.597187    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-key-cleanup-29720190-jnlkj"]
Jul 05 00:30:00.600132 ip-10-0-142-202 kubenswrapper[2563]: W0705 00:30:00.600104    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1837769c_81b2_45b4_9a21_cd351361f660.slice/crio-4c6dcfd54da612f1ca4f4269cda48fe79e7ed7ffd0dedcfc78069d21694a820f WatchSource:0}: Error finding container 4c6dcfd54da612f1ca4f4269cda48fe79e7ed7ffd0dedcfc78069d21694a820f: Status 404 returned error can't find the container with id 4c6dcfd54da612f1ca4f4269cda48fe79e7ed7ffd0dedcfc78069d21694a820f
Jul 05 00:30:00.602901 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:30:00.602879    2563 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 05 00:30:01.176279 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:30:01.176239    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-key-cleanup-29720190-jnlkj" event={"ID":"1837769c-81b2-45b4-9a21-cd351361f660","Type":"ContainerStarted","Data":"4c6dcfd54da612f1ca4f4269cda48fe79e7ed7ffd0dedcfc78069d21694a820f"}
Jul 05 00:30:04.029804 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:30:04.029773    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-x5szm"]
Jul 05 00:30:04.192345 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:30:04.192265    2563 generic.go:356] "Generic (PLEG): container finished" podID="1837769c-81b2-45b4-9a21-cd351361f660" containerID="6ec7d0937aa2fcaeadd31698945eab2f4481936851d7edd3eeb7dc6656b913f8" exitCode=0
Jul 05 00:30:04.192491 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:30:04.192353    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-key-cleanup-29720190-jnlkj" event={"ID":"1837769c-81b2-45b4-9a21-cd351361f660","Type":"ContainerDied","Data":"6ec7d0937aa2fcaeadd31698945eab2f4481936851d7edd3eeb7dc6656b913f8"}
Jul 05 00:30:05.333471 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:30:05.333440    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-key-cleanup-29720190-jnlkj"
Jul 05 00:30:05.357740 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:30:05.357712    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-fvvxv\" (UniqueName: \"kubernetes.io/projected/1837769c-81b2-45b4-9a21-cd351361f660-kube-api-access-fvvxv\") pod \"1837769c-81b2-45b4-9a21-cd351361f660\" (UID: \"1837769c-81b2-45b4-9a21-cd351361f660\") "
Jul 05 00:30:05.359956 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:30:05.359929    2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1837769c-81b2-45b4-9a21-cd351361f660-kube-api-access-fvvxv" (OuterVolumeSpecName: "kube-api-access-fvvxv") pod "1837769c-81b2-45b4-9a21-cd351361f660" (UID: "1837769c-81b2-45b4-9a21-cd351361f660"). InnerVolumeSpecName "kube-api-access-fvvxv". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 05 00:30:05.458644 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:30:05.458563    2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-fvvxv\" (UniqueName: \"kubernetes.io/projected/1837769c-81b2-45b4-9a21-cd351361f660-kube-api-access-fvvxv\") on node \"ip-10-0-142-202.ec2.internal\" DevicePath \"\""
Jul 05 00:30:06.201994 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:30:06.201970    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-key-cleanup-29720190-jnlkj"
Jul 05 00:30:06.201994 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:30:06.201987    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-key-cleanup-29720190-jnlkj" event={"ID":"1837769c-81b2-45b4-9a21-cd351361f660","Type":"ContainerDied","Data":"4c6dcfd54da612f1ca4f4269cda48fe79e7ed7ffd0dedcfc78069d21694a820f"}
Jul 05 00:30:06.202182 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:30:06.202042    2563 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="4c6dcfd54da612f1ca4f4269cda48fe79e7ed7ffd0dedcfc78069d21694a820f"
Jul 05 00:30:18.235387 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:30:18.235353    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-x5szm"]
Jul 05 00:30:28.127860 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:30:28.127822    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-x5szm"]
Jul 05 00:30:37.130286 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:30:37.130251    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-x5szm"]
Jul 05 00:30:46.824471 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:30:46.824437    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-x5szm"]
Jul 05 00:30:55.825631 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:30:55.825597    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-x5szm"]
Jul 05 00:32:47.331923 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:32:47.331887    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-x5szm"]
Jul 05 00:32:55.928844 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:32:55.928795    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-x5szm"]
Jul 05 00:33:08.933775 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:33:08.933674    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-x5szm"]
Jul 05 00:33:18.128660 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:33:18.128628    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-x5szm"]
Jul 05 00:33:28.628706 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:33:28.628676    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-x5szm"]
Jul 05 00:33:38.938396 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:33:38.938359    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-x5szm"]
Jul 05 00:33:46.724467 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:33:46.724423    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-x5szm"]
Jul 05 00:33:55.417437 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:33:55.417397    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-x5szm"]
Jul 05 00:34:04.829065 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:34:04.829032    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-x5szm"]
Jul 05 00:34:12.996212 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:34:12.996184    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-7sxq4_b96bfb4d-2fd3-4be3-9064-e24efeb904f4/ovn-acl-logging/0.log"
Jul 05 00:34:12.999783 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:34:12.999762    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-7sxq4_b96bfb4d-2fd3-4be3-9064-e24efeb904f4/ovn-acl-logging/0.log"
Jul 05 00:34:14.825514 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:34:14.825481    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-x5szm"]
Jul 05 00:34:21.927075 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:34:21.927046    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-x5szm"]
Jul 05 00:34:33.025769 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:34:33.025734    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-x5szm"]
Jul 05 00:34:54.322711 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:34:54.322636    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-x5szm"]
Jul 05 00:35:03.729131 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:35:03.729100    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-x5szm"]
Jul 05 00:35:14.722122 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:35:14.722090    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-x5szm"]
Jul 05 00:35:23.822561 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:35:23.822525    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-x5szm"]
Jul 05 00:35:39.426289 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:35:39.426252    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-x5szm"]
Jul 05 00:35:48.132045 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:35:48.132015    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-x5szm"]
Jul 05 00:35:58.226276 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:35:58.226241    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-x5szm"]
Jul 05 00:36:08.321080 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:36:08.320989    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-x5szm"]
Jul 05 00:36:16.327498 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:36:16.327463    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-x5szm"]
Jul 05 00:36:25.424393 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:36:25.424359    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-x5szm"]
Jul 05 00:36:54.729723 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:36:54.729687    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-x5szm"]
Jul 05 00:37:03.227462 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:37:03.227427    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-x5szm"]
Jul 05 00:37:13.631408 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:37:13.631378    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-x5szm"]
Jul 05 00:37:22.828424 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:37:22.828386    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-x5szm"]
Jul 05 00:37:34.126189 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:37:34.126115    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-x5szm"]
Jul 05 00:37:45.622314 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:37:45.622276    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-x5szm"]
Jul 05 00:37:52.826881 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:37:52.826847    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-x5szm"]
Jul 05 00:38:03.829577 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:38:03.829544    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-x5szm"]
Jul 05 00:38:15.832831 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:38:15.832792    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-x5szm"]
Jul 05 00:38:28.730661 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:38:28.730620    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-x5szm"]
Jul 05 00:38:36.729021 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:38:36.728987    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-x5szm"]
Jul 05 00:38:46.328890 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:38:46.328858    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-x5szm"]
Jul 05 00:38:54.234921 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:38:54.234882    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-x5szm"]
Jul 05 00:39:03.332555 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:39:03.332524    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-x5szm"]
Jul 05 00:39:13.023339 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:39:13.023311    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-7sxq4_b96bfb4d-2fd3-4be3-9064-e24efeb904f4/ovn-acl-logging/0.log"
Jul 05 00:39:13.027091 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:39:13.027072    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-7sxq4_b96bfb4d-2fd3-4be3-9064-e24efeb904f4/ovn-acl-logging/0.log"
Jul 05 00:39:23.036910 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:39:23.036877    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-x5szm"]
Jul 05 00:39:37.429156 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:39:37.429127    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-x5szm"]
Jul 05 00:41:11.521321 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:41:11.521237    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-x5szm"]
Jul 05 00:41:18.429052 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:41:18.429015    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-x5szm"]
Jul 05 00:41:26.825813 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:41:26.825782    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-x5szm"]
Jul 05 00:41:37.733064 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:41:37.733029    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-x5szm"]
Jul 05 00:41:47.934804 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:41:47.934764    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-x5szm"]
Jul 05 00:42:12.934760 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:42:12.934668    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-x5szm"]
Jul 05 00:42:44.000802 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:42:44.000767    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-mt-e214bcba/e2e-lifecycle-model-e214bcba-kserve-dbc8c4cff-vbw87"]
Jul 05 00:42:44.001257 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:42:44.001091    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1837769c-81b2-45b4-9a21-cd351361f660" containerName="cleanup"
Jul 05 00:42:44.001257 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:42:44.001101    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="1837769c-81b2-45b4-9a21-cd351361f660" containerName="cleanup"
Jul 05 00:42:44.001257 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:42:44.001163    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="1837769c-81b2-45b4-9a21-cd351361f660" containerName="cleanup"
Jul 05 00:42:44.003898 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:42:44.003883    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-e214bcba/e2e-lifecycle-model-e214bcba-kserve-dbc8c4cff-vbw87"
Jul 05 00:42:44.006214 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:42:44.006190    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-mt-e214bcba\"/\"default-dockercfg-smdt4\""
Jul 05 00:42:44.006214 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:42:44.006190    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-mt-e214bcba\"/\"openshift-service-ca.crt\""
Jul 05 00:42:44.006425 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:42:44.006195    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-mt-e214bcba\"/\"e2e-lifecycle-model-e214bcba-kserve-self-signed-certs\""
Jul 05 00:42:44.006425 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:42:44.006198    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-mt-e214bcba\"/\"kube-root-ca.crt\""
Jul 05 00:42:44.010563 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:42:44.010541    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-mt-e214bcba/e2e-lifecycle-model-e214bcba-kserve-dbc8c4cff-vbw87"]
Jul 05 00:42:44.182281 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:42:44.182247    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-f6dqv\" (UniqueName: \"kubernetes.io/projected/605e95b6-4e9f-4e98-af45-fa779f723312-kube-api-access-f6dqv\") pod \"e2e-lifecycle-model-e214bcba-kserve-dbc8c4cff-vbw87\" (UID: \"605e95b6-4e9f-4e98-af45-fa779f723312\") " pod="ai-tenant-e2e-mt-e214bcba/e2e-lifecycle-model-e214bcba-kserve-dbc8c4cff-vbw87"
Jul 05 00:42:44.182281 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:42:44.182285    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/605e95b6-4e9f-4e98-af45-fa779f723312-dshm\") pod \"e2e-lifecycle-model-e214bcba-kserve-dbc8c4cff-vbw87\" (UID: \"605e95b6-4e9f-4e98-af45-fa779f723312\") " pod="ai-tenant-e2e-mt-e214bcba/e2e-lifecycle-model-e214bcba-kserve-dbc8c4cff-vbw87"
Jul 05 00:42:44.182516 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:42:44.182371    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/605e95b6-4e9f-4e98-af45-fa779f723312-model-cache\") pod \"e2e-lifecycle-model-e214bcba-kserve-dbc8c4cff-vbw87\" (UID: \"605e95b6-4e9f-4e98-af45-fa779f723312\") " pod="ai-tenant-e2e-mt-e214bcba/e2e-lifecycle-model-e214bcba-kserve-dbc8c4cff-vbw87"
Jul 05 00:42:44.182516 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:42:44.182412    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/605e95b6-4e9f-4e98-af45-fa779f723312-tmp-dir\") pod \"e2e-lifecycle-model-e214bcba-kserve-dbc8c4cff-vbw87\" (UID: \"605e95b6-4e9f-4e98-af45-fa779f723312\") " pod="ai-tenant-e2e-mt-e214bcba/e2e-lifecycle-model-e214bcba-kserve-dbc8c4cff-vbw87"
Jul 05 00:42:44.182516 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:42:44.182429    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/605e95b6-4e9f-4e98-af45-fa779f723312-tls-certs\") pod \"e2e-lifecycle-model-e214bcba-kserve-dbc8c4cff-vbw87\" (UID: \"605e95b6-4e9f-4e98-af45-fa779f723312\") " pod="ai-tenant-e2e-mt-e214bcba/e2e-lifecycle-model-e214bcba-kserve-dbc8c4cff-vbw87"
Jul 05 00:42:44.182516 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:42:44.182503    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/605e95b6-4e9f-4e98-af45-fa779f723312-home\") pod \"e2e-lifecycle-model-e214bcba-kserve-dbc8c4cff-vbw87\" (UID: \"605e95b6-4e9f-4e98-af45-fa779f723312\") " pod="ai-tenant-e2e-mt-e214bcba/e2e-lifecycle-model-e214bcba-kserve-dbc8c4cff-vbw87"
Jul 05 00:42:44.283416 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:42:44.283385    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/605e95b6-4e9f-4e98-af45-fa779f723312-model-cache\") pod \"e2e-lifecycle-model-e214bcba-kserve-dbc8c4cff-vbw87\" (UID: \"605e95b6-4e9f-4e98-af45-fa779f723312\") " pod="ai-tenant-e2e-mt-e214bcba/e2e-lifecycle-model-e214bcba-kserve-dbc8c4cff-vbw87"
Jul 05 00:42:44.283569 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:42:44.283425    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/605e95b6-4e9f-4e98-af45-fa779f723312-tmp-dir\") pod \"e2e-lifecycle-model-e214bcba-kserve-dbc8c4cff-vbw87\" (UID: \"605e95b6-4e9f-4e98-af45-fa779f723312\") " pod="ai-tenant-e2e-mt-e214bcba/e2e-lifecycle-model-e214bcba-kserve-dbc8c4cff-vbw87"
Jul 05 00:42:44.283569 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:42:44.283441    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/605e95b6-4e9f-4e98-af45-fa779f723312-tls-certs\") pod \"e2e-lifecycle-model-e214bcba-kserve-dbc8c4cff-vbw87\" (UID: \"605e95b6-4e9f-4e98-af45-fa779f723312\") " pod="ai-tenant-e2e-mt-e214bcba/e2e-lifecycle-model-e214bcba-kserve-dbc8c4cff-vbw87"
Jul 05 00:42:44.283569 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:42:44.283472    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/605e95b6-4e9f-4e98-af45-fa779f723312-home\") pod \"e2e-lifecycle-model-e214bcba-kserve-dbc8c4cff-vbw87\" (UID: \"605e95b6-4e9f-4e98-af45-fa779f723312\") " pod="ai-tenant-e2e-mt-e214bcba/e2e-lifecycle-model-e214bcba-kserve-dbc8c4cff-vbw87"
Jul 05 00:42:44.283569 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:42:44.283519    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-f6dqv\" (UniqueName: \"kubernetes.io/projected/605e95b6-4e9f-4e98-af45-fa779f723312-kube-api-access-f6dqv\") pod \"e2e-lifecycle-model-e214bcba-kserve-dbc8c4cff-vbw87\" (UID: \"605e95b6-4e9f-4e98-af45-fa779f723312\") " pod="ai-tenant-e2e-mt-e214bcba/e2e-lifecycle-model-e214bcba-kserve-dbc8c4cff-vbw87"
Jul 05 00:42:44.283569 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:42:44.283545    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/605e95b6-4e9f-4e98-af45-fa779f723312-dshm\") pod \"e2e-lifecycle-model-e214bcba-kserve-dbc8c4cff-vbw87\" (UID: \"605e95b6-4e9f-4e98-af45-fa779f723312\") " pod="ai-tenant-e2e-mt-e214bcba/e2e-lifecycle-model-e214bcba-kserve-dbc8c4cff-vbw87"
Jul 05 00:42:44.283798 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:42:44.283760    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/605e95b6-4e9f-4e98-af45-fa779f723312-model-cache\") pod \"e2e-lifecycle-model-e214bcba-kserve-dbc8c4cff-vbw87\" (UID: \"605e95b6-4e9f-4e98-af45-fa779f723312\") " pod="ai-tenant-e2e-mt-e214bcba/e2e-lifecycle-model-e214bcba-kserve-dbc8c4cff-vbw87"
Jul 05 00:42:44.283856 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:42:44.283827    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/605e95b6-4e9f-4e98-af45-fa779f723312-tmp-dir\") pod \"e2e-lifecycle-model-e214bcba-kserve-dbc8c4cff-vbw87\" (UID: \"605e95b6-4e9f-4e98-af45-fa779f723312\") " pod="ai-tenant-e2e-mt-e214bcba/e2e-lifecycle-model-e214bcba-kserve-dbc8c4cff-vbw87"
Jul 05 00:42:44.283916 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:42:44.283892    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/605e95b6-4e9f-4e98-af45-fa779f723312-home\") pod \"e2e-lifecycle-model-e214bcba-kserve-dbc8c4cff-vbw87\" (UID: \"605e95b6-4e9f-4e98-af45-fa779f723312\") " pod="ai-tenant-e2e-mt-e214bcba/e2e-lifecycle-model-e214bcba-kserve-dbc8c4cff-vbw87"
Jul 05 00:42:44.285841 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:42:44.285813    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/605e95b6-4e9f-4e98-af45-fa779f723312-dshm\") pod \"e2e-lifecycle-model-e214bcba-kserve-dbc8c4cff-vbw87\" (UID: \"605e95b6-4e9f-4e98-af45-fa779f723312\") " pod="ai-tenant-e2e-mt-e214bcba/e2e-lifecycle-model-e214bcba-kserve-dbc8c4cff-vbw87"
Jul 05 00:42:44.286007 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:42:44.285993    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/605e95b6-4e9f-4e98-af45-fa779f723312-tls-certs\") pod \"e2e-lifecycle-model-e214bcba-kserve-dbc8c4cff-vbw87\" (UID: \"605e95b6-4e9f-4e98-af45-fa779f723312\") " pod="ai-tenant-e2e-mt-e214bcba/e2e-lifecycle-model-e214bcba-kserve-dbc8c4cff-vbw87"
Jul 05 00:42:44.290602 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:42:44.290580    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-f6dqv\" (UniqueName: \"kubernetes.io/projected/605e95b6-4e9f-4e98-af45-fa779f723312-kube-api-access-f6dqv\") pod \"e2e-lifecycle-model-e214bcba-kserve-dbc8c4cff-vbw87\" (UID: \"605e95b6-4e9f-4e98-af45-fa779f723312\") " pod="ai-tenant-e2e-mt-e214bcba/e2e-lifecycle-model-e214bcba-kserve-dbc8c4cff-vbw87"
Jul 05 00:42:44.314303 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:42:44.314268    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-e214bcba/e2e-lifecycle-model-e214bcba-kserve-dbc8c4cff-vbw87"
Jul 05 00:42:44.439141 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:42:44.439112    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-mt-e214bcba/e2e-lifecycle-model-e214bcba-kserve-dbc8c4cff-vbw87"]
Jul 05 00:42:44.441269 ip-10-0-142-202 kubenswrapper[2563]: W0705 00:42:44.441239    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod605e95b6_4e9f_4e98_af45_fa779f723312.slice/crio-ff299020cc0ee3b0f06675165487b68d8a4b5b21677c71356b5758d5c48414b1 WatchSource:0}: Error finding container ff299020cc0ee3b0f06675165487b68d8a4b5b21677c71356b5758d5c48414b1: Status 404 returned error can't find the container with id ff299020cc0ee3b0f06675165487b68d8a4b5b21677c71356b5758d5c48414b1
Jul 05 00:42:44.480196 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:42:44.480164    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-e214bcba/e2e-lifecycle-model-e214bcba-kserve-dbc8c4cff-vbw87" event={"ID":"605e95b6-4e9f-4e98-af45-fa779f723312","Type":"ContainerStarted","Data":"ff299020cc0ee3b0f06675165487b68d8a4b5b21677c71356b5758d5c48414b1"}
Jul 05 00:42:45.486326 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:42:45.486291    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-e214bcba/e2e-lifecycle-model-e214bcba-kserve-dbc8c4cff-vbw87" event={"ID":"605e95b6-4e9f-4e98-af45-fa779f723312","Type":"ContainerStarted","Data":"d464e82f56c2a6346b1711a49cafe37acc3875cf17a3aae7d88eafbf96d3c550"}
Jul 05 00:42:45.502540 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:42:45.502491    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-mt-e214bcba/e2e-lifecycle-model-e214bcba-kserve-dbc8c4cff-vbw87" podStartSLOduration=2.502476711 podStartE2EDuration="2.502476711s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-05 00:42:43 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-05 00:42:45.500206764 +0000 UTC m=+2013.842535824" watchObservedRunningTime="2026-07-05 00:42:45.502476711 +0000 UTC m=+2013.844805770"
Jul 05 00:42:47.027818 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:42:47.027764    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-x5szm"]
Jul 05 00:42:54.315322 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:42:54.315288    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-mt-e214bcba/e2e-lifecycle-model-e214bcba-kserve-dbc8c4cff-vbw87"
Jul 05 00:42:54.315705 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:42:54.315438    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-mt-e214bcba/e2e-lifecycle-model-e214bcba-kserve-dbc8c4cff-vbw87"
Jul 05 00:42:54.327593 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:42:54.327568    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-mt-e214bcba/e2e-lifecycle-model-e214bcba-kserve-dbc8c4cff-vbw87"
Jul 05 00:42:54.535885 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:42:54.535859    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-mt-e214bcba/e2e-lifecycle-model-e214bcba-kserve-dbc8c4cff-vbw87"
Jul 05 00:43:07.421750 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:43:07.421719    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-x5szm"]
Jul 05 00:43:20.924174 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:43:20.924141    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-x5szm"]
Jul 05 00:43:21.129886 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:43:21.129850    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-mt-e214bcba/e2e-lifecycle-model-e214bcba-kserve-dbc8c4cff-vbw87"]
Jul 05 00:43:36.150461 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:43:36.150395    2563 kuberuntime_container.go:872] "Killing container with a grace period" pod="ai-tenant-e2e-mt-e214bcba/e2e-lifecycle-model-e214bcba-kserve-dbc8c4cff-vbw87" podUID="605e95b6-4e9f-4e98-af45-fa779f723312" containerName="main" containerID="cri-o://d464e82f56c2a6346b1711a49cafe37acc3875cf17a3aae7d88eafbf96d3c550" gracePeriod=45
Jul 05 00:43:36.397689 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:43:36.397667    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-e214bcba/e2e-lifecycle-model-e214bcba-kserve-dbc8c4cff-vbw87"
Jul 05 00:43:36.512325 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:43:36.512236    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/605e95b6-4e9f-4e98-af45-fa779f723312-home\") pod \"605e95b6-4e9f-4e98-af45-fa779f723312\" (UID: \"605e95b6-4e9f-4e98-af45-fa779f723312\") "
Jul 05 00:43:36.512485 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:43:36.512338    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-f6dqv\" (UniqueName: \"kubernetes.io/projected/605e95b6-4e9f-4e98-af45-fa779f723312-kube-api-access-f6dqv\") pod \"605e95b6-4e9f-4e98-af45-fa779f723312\" (UID: \"605e95b6-4e9f-4e98-af45-fa779f723312\") "
Jul 05 00:43:36.512485 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:43:36.512357    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/605e95b6-4e9f-4e98-af45-fa779f723312-model-cache\") pod \"605e95b6-4e9f-4e98-af45-fa779f723312\" (UID: \"605e95b6-4e9f-4e98-af45-fa779f723312\") "
Jul 05 00:43:36.512485 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:43:36.512401    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/605e95b6-4e9f-4e98-af45-fa779f723312-tmp-dir\") pod \"605e95b6-4e9f-4e98-af45-fa779f723312\" (UID: \"605e95b6-4e9f-4e98-af45-fa779f723312\") "
Jul 05 00:43:36.512485 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:43:36.512428    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/605e95b6-4e9f-4e98-af45-fa779f723312-dshm\") pod \"605e95b6-4e9f-4e98-af45-fa779f723312\" (UID: \"605e95b6-4e9f-4e98-af45-fa779f723312\") "
Jul 05 00:43:36.512485 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:43:36.512462    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/605e95b6-4e9f-4e98-af45-fa779f723312-tls-certs\") pod \"605e95b6-4e9f-4e98-af45-fa779f723312\" (UID: \"605e95b6-4e9f-4e98-af45-fa779f723312\") "
Jul 05 00:43:36.512736 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:43:36.512544    2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/605e95b6-4e9f-4e98-af45-fa779f723312-home" (OuterVolumeSpecName: "home") pod "605e95b6-4e9f-4e98-af45-fa779f723312" (UID: "605e95b6-4e9f-4e98-af45-fa779f723312"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 05 00:43:36.512736 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:43:36.512634    2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/605e95b6-4e9f-4e98-af45-fa779f723312-model-cache" (OuterVolumeSpecName: "model-cache") pod "605e95b6-4e9f-4e98-af45-fa779f723312" (UID: "605e95b6-4e9f-4e98-af45-fa779f723312"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 05 00:43:36.512736 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:43:36.512657    2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/605e95b6-4e9f-4e98-af45-fa779f723312-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "605e95b6-4e9f-4e98-af45-fa779f723312" (UID: "605e95b6-4e9f-4e98-af45-fa779f723312"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 05 00:43:36.512872 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:43:36.512779    2563 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/605e95b6-4e9f-4e98-af45-fa779f723312-model-cache\") on node \"ip-10-0-142-202.ec2.internal\" DevicePath \"\""
Jul 05 00:43:36.512872 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:43:36.512800    2563 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/605e95b6-4e9f-4e98-af45-fa779f723312-tmp-dir\") on node \"ip-10-0-142-202.ec2.internal\" DevicePath \"\""
Jul 05 00:43:36.512872 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:43:36.512813    2563 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/605e95b6-4e9f-4e98-af45-fa779f723312-home\") on node \"ip-10-0-142-202.ec2.internal\" DevicePath \"\""
Jul 05 00:43:36.514604 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:43:36.514574    2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/605e95b6-4e9f-4e98-af45-fa779f723312-kube-api-access-f6dqv" (OuterVolumeSpecName: "kube-api-access-f6dqv") pod "605e95b6-4e9f-4e98-af45-fa779f723312" (UID: "605e95b6-4e9f-4e98-af45-fa779f723312"). InnerVolumeSpecName "kube-api-access-f6dqv". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 05 00:43:36.514717 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:43:36.514604    2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/605e95b6-4e9f-4e98-af45-fa779f723312-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "605e95b6-4e9f-4e98-af45-fa779f723312" (UID: "605e95b6-4e9f-4e98-af45-fa779f723312"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 05 00:43:36.514717 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:43:36.514619    2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/605e95b6-4e9f-4e98-af45-fa779f723312-dshm" (OuterVolumeSpecName: "dshm") pod "605e95b6-4e9f-4e98-af45-fa779f723312" (UID: "605e95b6-4e9f-4e98-af45-fa779f723312"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 05 00:43:36.613536 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:43:36.613506    2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-f6dqv\" (UniqueName: \"kubernetes.io/projected/605e95b6-4e9f-4e98-af45-fa779f723312-kube-api-access-f6dqv\") on node \"ip-10-0-142-202.ec2.internal\" DevicePath \"\""
Jul 05 00:43:36.613536 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:43:36.613531    2563 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/605e95b6-4e9f-4e98-af45-fa779f723312-dshm\") on node \"ip-10-0-142-202.ec2.internal\" DevicePath \"\""
Jul 05 00:43:36.613536 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:43:36.613542    2563 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/605e95b6-4e9f-4e98-af45-fa779f723312-tls-certs\") on node \"ip-10-0-142-202.ec2.internal\" DevicePath \"\""
Jul 05 00:43:36.716593 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:43:36.716564    2563 generic.go:356] "Generic (PLEG): container finished" podID="605e95b6-4e9f-4e98-af45-fa779f723312" containerID="d464e82f56c2a6346b1711a49cafe37acc3875cf17a3aae7d88eafbf96d3c550" exitCode=0
Jul 05 00:43:36.716767 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:43:36.716607    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-e214bcba/e2e-lifecycle-model-e214bcba-kserve-dbc8c4cff-vbw87"
Jul 05 00:43:36.716767 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:43:36.716647    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-e214bcba/e2e-lifecycle-model-e214bcba-kserve-dbc8c4cff-vbw87" event={"ID":"605e95b6-4e9f-4e98-af45-fa779f723312","Type":"ContainerDied","Data":"d464e82f56c2a6346b1711a49cafe37acc3875cf17a3aae7d88eafbf96d3c550"}
Jul 05 00:43:36.716767 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:43:36.716694    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-e214bcba/e2e-lifecycle-model-e214bcba-kserve-dbc8c4cff-vbw87" event={"ID":"605e95b6-4e9f-4e98-af45-fa779f723312","Type":"ContainerDied","Data":"ff299020cc0ee3b0f06675165487b68d8a4b5b21677c71356b5758d5c48414b1"}
Jul 05 00:43:36.716767 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:43:36.716717    2563 scope.go:117] "RemoveContainer" containerID="d464e82f56c2a6346b1711a49cafe37acc3875cf17a3aae7d88eafbf96d3c550"
Jul 05 00:43:36.725847 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:43:36.725834    2563 scope.go:117] "RemoveContainer" containerID="d464e82f56c2a6346b1711a49cafe37acc3875cf17a3aae7d88eafbf96d3c550"
Jul 05 00:43:36.726082 ip-10-0-142-202 kubenswrapper[2563]: E0705 00:43:36.726067    2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d464e82f56c2a6346b1711a49cafe37acc3875cf17a3aae7d88eafbf96d3c550\": container with ID starting with d464e82f56c2a6346b1711a49cafe37acc3875cf17a3aae7d88eafbf96d3c550 not found: ID does not exist" containerID="d464e82f56c2a6346b1711a49cafe37acc3875cf17a3aae7d88eafbf96d3c550"
Jul 05 00:43:36.726128 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:43:36.726088    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d464e82f56c2a6346b1711a49cafe37acc3875cf17a3aae7d88eafbf96d3c550"} err="failed to get container status \"d464e82f56c2a6346b1711a49cafe37acc3875cf17a3aae7d88eafbf96d3c550\": rpc error: code = NotFound desc = could not find container \"d464e82f56c2a6346b1711a49cafe37acc3875cf17a3aae7d88eafbf96d3c550\": container with ID starting with d464e82f56c2a6346b1711a49cafe37acc3875cf17a3aae7d88eafbf96d3c550 not found: ID does not exist"
Jul 05 00:43:36.736577 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:43:36.736556    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-mt-e214bcba/e2e-lifecycle-model-e214bcba-kserve-dbc8c4cff-vbw87"]
Jul 05 00:43:36.742780 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:43:36.742762    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-mt-e214bcba/e2e-lifecycle-model-e214bcba-kserve-dbc8c4cff-vbw87"]
Jul 05 00:43:38.126145 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:43:38.126114    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="605e95b6-4e9f-4e98-af45-fa779f723312" path="/var/lib/kubelet/pods/605e95b6-4e9f-4e98-af45-fa779f723312/volumes"
Jul 05 00:43:50.423824 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:43:50.423787    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-x5szm"]
Jul 05 00:43:59.133964 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:43:59.133920    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-x5szm"]
Jul 05 00:44:09.523934 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:44:09.523899    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-x5szm"]
Jul 05 00:44:13.049547 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:44:13.049514    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-7sxq4_b96bfb4d-2fd3-4be3-9064-e24efeb904f4/ovn-acl-logging/0.log"
Jul 05 00:44:13.055333 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:44:13.055312    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-7sxq4_b96bfb4d-2fd3-4be3-9064-e24efeb904f4/ovn-acl-logging/0.log"
Jul 05 00:44:54.325273 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:44:54.325237    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-x5szm"]
Jul 05 00:45:00.129595 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:45:00.129555    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-api-key-cleanup-e2e-inf-b-11db4a-29720205-dkrkf"]
Jul 05 00:45:00.130191 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:45:00.129902    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="605e95b6-4e9f-4e98-af45-fa779f723312" containerName="main"
Jul 05 00:45:00.130191 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:45:00.129915    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="605e95b6-4e9f-4e98-af45-fa779f723312" containerName="main"
Jul 05 00:45:00.130191 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:45:00.129980    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="605e95b6-4e9f-4e98-af45-fa779f723312" containerName="main"
Jul 05 00:45:00.132874 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:45:00.132852    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-api-key-cleanup-29720205-jdftl"]
Jul 05 00:45:00.133011 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:45:00.132978    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-key-cleanup-e2e-inf-b-11db4a-29720205-dkrkf"
Jul 05 00:45:00.134916 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:45:00.134888    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-api-dockercfg-2p4nn\""
Jul 05 00:45:00.136162 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:45:00.136144    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-api-key-cleanup-e2e-inf-a-4e7441-29720205-hqggk"]
Jul 05 00:45:00.136305 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:45:00.136288    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-key-cleanup-29720205-jdftl"
Jul 05 00:45:00.139265 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:45:00.139244    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-key-cleanup-e2e-inf-a-4e7441-29720205-hqggk"
Jul 05 00:45:00.140053 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:45:00.140035    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-key-cleanup-29720205-jdftl"]
Jul 05 00:45:00.142127 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:45:00.142104    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-key-cleanup-e2e-inf-a-4e7441-29720205-hqggk"]
Jul 05 00:45:00.144181 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:45:00.144162    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-key-cleanup-e2e-inf-b-11db4a-29720205-dkrkf"]
Jul 05 00:45:00.243130 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:45:00.243089    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-h5jk5\" (UniqueName: \"kubernetes.io/projected/e72b157a-2968-4b65-9709-d077e8293eaa-kube-api-access-h5jk5\") pod \"maas-api-key-cleanup-e2e-inf-a-4e7441-29720205-hqggk\" (UID: \"e72b157a-2968-4b65-9709-d077e8293eaa\") " pod="opendatahub/maas-api-key-cleanup-e2e-inf-a-4e7441-29720205-hqggk"
Jul 05 00:45:00.243360 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:45:00.243137    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qbwpz\" (UniqueName: \"kubernetes.io/projected/0e349b60-e213-445d-8e58-0244a5dda437-kube-api-access-qbwpz\") pod \"maas-api-key-cleanup-e2e-inf-b-11db4a-29720205-dkrkf\" (UID: \"0e349b60-e213-445d-8e58-0244a5dda437\") " pod="opendatahub/maas-api-key-cleanup-e2e-inf-b-11db4a-29720205-dkrkf"
Jul 05 00:45:00.243360 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:45:00.243253    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sxdzg\" (UniqueName: \"kubernetes.io/projected/33370136-cdfa-45c3-9f8f-a10e21bb37e4-kube-api-access-sxdzg\") pod \"maas-api-key-cleanup-29720205-jdftl\" (UID: \"33370136-cdfa-45c3-9f8f-a10e21bb37e4\") " pod="opendatahub/maas-api-key-cleanup-29720205-jdftl"
Jul 05 00:45:00.344825 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:45:00.344773    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-h5jk5\" (UniqueName: \"kubernetes.io/projected/e72b157a-2968-4b65-9709-d077e8293eaa-kube-api-access-h5jk5\") pod \"maas-api-key-cleanup-e2e-inf-a-4e7441-29720205-hqggk\" (UID: \"e72b157a-2968-4b65-9709-d077e8293eaa\") " pod="opendatahub/maas-api-key-cleanup-e2e-inf-a-4e7441-29720205-hqggk"
Jul 05 00:45:00.345037 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:45:00.344845    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qbwpz\" (UniqueName: \"kubernetes.io/projected/0e349b60-e213-445d-8e58-0244a5dda437-kube-api-access-qbwpz\") pod \"maas-api-key-cleanup-e2e-inf-b-11db4a-29720205-dkrkf\" (UID: \"0e349b60-e213-445d-8e58-0244a5dda437\") " pod="opendatahub/maas-api-key-cleanup-e2e-inf-b-11db4a-29720205-dkrkf"
Jul 05 00:45:00.345037 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:45:00.344894    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sxdzg\" (UniqueName: \"kubernetes.io/projected/33370136-cdfa-45c3-9f8f-a10e21bb37e4-kube-api-access-sxdzg\") pod \"maas-api-key-cleanup-29720205-jdftl\" (UID: \"33370136-cdfa-45c3-9f8f-a10e21bb37e4\") " pod="opendatahub/maas-api-key-cleanup-29720205-jdftl"
Jul 05 00:45:00.353097 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:45:00.353068    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-h5jk5\" (UniqueName: \"kubernetes.io/projected/e72b157a-2968-4b65-9709-d077e8293eaa-kube-api-access-h5jk5\") pod \"maas-api-key-cleanup-e2e-inf-a-4e7441-29720205-hqggk\" (UID: \"e72b157a-2968-4b65-9709-d077e8293eaa\") " pod="opendatahub/maas-api-key-cleanup-e2e-inf-a-4e7441-29720205-hqggk"
Jul 05 00:45:00.353361 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:45:00.353340    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-sxdzg\" (UniqueName: \"kubernetes.io/projected/33370136-cdfa-45c3-9f8f-a10e21bb37e4-kube-api-access-sxdzg\") pod \"maas-api-key-cleanup-29720205-jdftl\" (UID: \"33370136-cdfa-45c3-9f8f-a10e21bb37e4\") " pod="opendatahub/maas-api-key-cleanup-29720205-jdftl"
Jul 05 00:45:00.353411 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:45:00.353340    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-qbwpz\" (UniqueName: \"kubernetes.io/projected/0e349b60-e213-445d-8e58-0244a5dda437-kube-api-access-qbwpz\") pod \"maas-api-key-cleanup-e2e-inf-b-11db4a-29720205-dkrkf\" (UID: \"0e349b60-e213-445d-8e58-0244a5dda437\") " pod="opendatahub/maas-api-key-cleanup-e2e-inf-b-11db4a-29720205-dkrkf"
Jul 05 00:45:00.448293 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:45:00.448174    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-key-cleanup-e2e-inf-b-11db4a-29720205-dkrkf"
Jul 05 00:45:00.455504 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:45:00.455481    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-key-cleanup-29720205-jdftl"
Jul 05 00:45:00.462735 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:45:00.462712    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-key-cleanup-e2e-inf-a-4e7441-29720205-hqggk"
Jul 05 00:45:00.806655 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:45:00.806627    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-key-cleanup-e2e-inf-b-11db4a-29720205-dkrkf"]
Jul 05 00:45:00.808197 ip-10-0-142-202 kubenswrapper[2563]: W0705 00:45:00.808169    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0e349b60_e213_445d_8e58_0244a5dda437.slice/crio-c7840d5941f360f7790722bf798904c76556bac39405c2102b20522e6d24d0e8 WatchSource:0}: Error finding container c7840d5941f360f7790722bf798904c76556bac39405c2102b20522e6d24d0e8: Status 404 returned error can't find the container with id c7840d5941f360f7790722bf798904c76556bac39405c2102b20522e6d24d0e8
Jul 05 00:45:00.828363 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:45:00.828318    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-key-cleanup-29720205-jdftl"]
Jul 05 00:45:00.830987 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:45:00.830961    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-key-cleanup-e2e-inf-a-4e7441-29720205-hqggk"]
Jul 05 00:45:00.841447 ip-10-0-142-202 kubenswrapper[2563]: W0705 00:45:00.841424    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod33370136_cdfa_45c3_9f8f_a10e21bb37e4.slice/crio-498223d1eb0f6212e66a5e7abd357d17366cf21e5c0a00291653d6c1561abab1 WatchSource:0}: Error finding container 498223d1eb0f6212e66a5e7abd357d17366cf21e5c0a00291653d6c1561abab1: Status 404 returned error can't find the container with id 498223d1eb0f6212e66a5e7abd357d17366cf21e5c0a00291653d6c1561abab1
Jul 05 00:45:00.841622 ip-10-0-142-202 kubenswrapper[2563]: W0705 00:45:00.841602    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode72b157a_2968_4b65_9709_d077e8293eaa.slice/crio-4d468a9e6c8e5981c111afc51fa6408086a326dc2678742146da83ef2cb3c84a WatchSource:0}: Error finding container 4d468a9e6c8e5981c111afc51fa6408086a326dc2678742146da83ef2cb3c84a: Status 404 returned error can't find the container with id 4d468a9e6c8e5981c111afc51fa6408086a326dc2678742146da83ef2cb3c84a
Jul 05 00:45:01.083966 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:45:01.083936    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-key-cleanup-e2e-inf-a-4e7441-29720205-hqggk" event={"ID":"e72b157a-2968-4b65-9709-d077e8293eaa","Type":"ContainerStarted","Data":"4d468a9e6c8e5981c111afc51fa6408086a326dc2678742146da83ef2cb3c84a"}
Jul 05 00:45:01.085389 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:45:01.085364    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-key-cleanup-e2e-inf-b-11db4a-29720205-dkrkf" event={"ID":"0e349b60-e213-445d-8e58-0244a5dda437","Type":"ContainerStarted","Data":"02e9f29b596097cd35d48ab8b5827c89d7e5ccd3070627b5f2ed7135fc257bff"}
Jul 05 00:45:01.085498 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:45:01.085426    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-key-cleanup-e2e-inf-b-11db4a-29720205-dkrkf" event={"ID":"0e349b60-e213-445d-8e58-0244a5dda437","Type":"ContainerStarted","Data":"c7840d5941f360f7790722bf798904c76556bac39405c2102b20522e6d24d0e8"}
Jul 05 00:45:01.086482 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:45:01.086463    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-key-cleanup-29720205-jdftl" event={"ID":"33370136-cdfa-45c3-9f8f-a10e21bb37e4","Type":"ContainerStarted","Data":"498223d1eb0f6212e66a5e7abd357d17366cf21e5c0a00291653d6c1561abab1"}
Jul 05 00:45:01.099160 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:45:01.099117    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-api-key-cleanup-e2e-inf-b-11db4a-29720205-dkrkf" podStartSLOduration=1.099104217 podStartE2EDuration="1.099104217s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-05 00:45:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-05 00:45:01.098584823 +0000 UTC m=+2149.440913886" watchObservedRunningTime="2026-07-05 00:45:01.099104217 +0000 UTC m=+2149.441433276"
Jul 05 00:45:02.093347 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:45:02.093311    2563 generic.go:356] "Generic (PLEG): container finished" podID="33370136-cdfa-45c3-9f8f-a10e21bb37e4" containerID="81beed628e90285d305683a074e460d6c37efe7875a849c9e31d1da3a2772704" exitCode=0
Jul 05 00:45:02.093746 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:45:02.093394    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-key-cleanup-29720205-jdftl" event={"ID":"33370136-cdfa-45c3-9f8f-a10e21bb37e4","Type":"ContainerDied","Data":"81beed628e90285d305683a074e460d6c37efe7875a849c9e31d1da3a2772704"}
Jul 05 00:45:02.094772 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:45:02.094748    2563 generic.go:356] "Generic (PLEG): container finished" podID="e72b157a-2968-4b65-9709-d077e8293eaa" containerID="bc96216c730160bb60f64730a08979ff3c7b14b8d378ba2a59d7098e594f5fca" exitCode=0
Jul 05 00:45:02.094882 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:45:02.094815    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-key-cleanup-e2e-inf-a-4e7441-29720205-hqggk" event={"ID":"e72b157a-2968-4b65-9709-d077e8293eaa","Type":"ContainerDied","Data":"bc96216c730160bb60f64730a08979ff3c7b14b8d378ba2a59d7098e594f5fca"}
Jul 05 00:45:02.096237 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:45:02.096196    2563 generic.go:356] "Generic (PLEG): container finished" podID="0e349b60-e213-445d-8e58-0244a5dda437" containerID="02e9f29b596097cd35d48ab8b5827c89d7e5ccd3070627b5f2ed7135fc257bff" exitCode=7
Jul 05 00:45:02.096335 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:45:02.096251    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-key-cleanup-e2e-inf-b-11db4a-29720205-dkrkf" event={"ID":"0e349b60-e213-445d-8e58-0244a5dda437","Type":"ContainerDied","Data":"02e9f29b596097cd35d48ab8b5827c89d7e5ccd3070627b5f2ed7135fc257bff"}
Jul 05 00:45:02.096494 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:45:02.096482    2563 scope.go:117] "RemoveContainer" containerID="02e9f29b596097cd35d48ab8b5827c89d7e5ccd3070627b5f2ed7135fc257bff"
Jul 05 00:45:03.102857 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:45:03.102821    2563 generic.go:356] "Generic (PLEG): container finished" podID="0e349b60-e213-445d-8e58-0244a5dda437" containerID="93632daedb6d62e06a0395e17ebce77eaa64e79d018b818b7013f7195f48f71c" exitCode=7
Jul 05 00:45:03.103281 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:45:03.102909    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-key-cleanup-e2e-inf-b-11db4a-29720205-dkrkf" event={"ID":"0e349b60-e213-445d-8e58-0244a5dda437","Type":"ContainerDied","Data":"93632daedb6d62e06a0395e17ebce77eaa64e79d018b818b7013f7195f48f71c"}
Jul 05 00:45:03.104013 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:45:03.102980    2563 scope.go:117] "RemoveContainer" containerID="02e9f29b596097cd35d48ab8b5827c89d7e5ccd3070627b5f2ed7135fc257bff"
Jul 05 00:45:03.104100 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:45:03.104088    2563 scope.go:117] "RemoveContainer" containerID="93632daedb6d62e06a0395e17ebce77eaa64e79d018b818b7013f7195f48f71c"
Jul 05 00:45:03.104360 ip-10-0-142-202 kubenswrapper[2563]: E0705 00:45:03.104336    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 10s restarting failed container=cleanup pod=maas-api-key-cleanup-e2e-inf-b-11db4a-29720205-dkrkf_opendatahub(0e349b60-e213-445d-8e58-0244a5dda437)\"" pod="opendatahub/maas-api-key-cleanup-e2e-inf-b-11db4a-29720205-dkrkf" podUID="0e349b60-e213-445d-8e58-0244a5dda437"
Jul 05 00:45:03.239369 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:45:03.239349    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-key-cleanup-e2e-inf-a-4e7441-29720205-hqggk"
Jul 05 00:45:03.254412 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:45:03.254387    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-key-cleanup-29720205-jdftl"
Jul 05 00:45:03.375298 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:45:03.375189    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-h5jk5\" (UniqueName: \"kubernetes.io/projected/e72b157a-2968-4b65-9709-d077e8293eaa-kube-api-access-h5jk5\") pod \"e72b157a-2968-4b65-9709-d077e8293eaa\" (UID: \"e72b157a-2968-4b65-9709-d077e8293eaa\") "
Jul 05 00:45:03.375298 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:45:03.375259    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-sxdzg\" (UniqueName: \"kubernetes.io/projected/33370136-cdfa-45c3-9f8f-a10e21bb37e4-kube-api-access-sxdzg\") pod \"33370136-cdfa-45c3-9f8f-a10e21bb37e4\" (UID: \"33370136-cdfa-45c3-9f8f-a10e21bb37e4\") "
Jul 05 00:45:03.377265 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:45:03.377241    2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e72b157a-2968-4b65-9709-d077e8293eaa-kube-api-access-h5jk5" (OuterVolumeSpecName: "kube-api-access-h5jk5") pod "e72b157a-2968-4b65-9709-d077e8293eaa" (UID: "e72b157a-2968-4b65-9709-d077e8293eaa"). InnerVolumeSpecName "kube-api-access-h5jk5". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 05 00:45:03.377365 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:45:03.377284    2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/33370136-cdfa-45c3-9f8f-a10e21bb37e4-kube-api-access-sxdzg" (OuterVolumeSpecName: "kube-api-access-sxdzg") pod "33370136-cdfa-45c3-9f8f-a10e21bb37e4" (UID: "33370136-cdfa-45c3-9f8f-a10e21bb37e4"). InnerVolumeSpecName "kube-api-access-sxdzg". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 05 00:45:03.476831 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:45:03.476804    2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-sxdzg\" (UniqueName: \"kubernetes.io/projected/33370136-cdfa-45c3-9f8f-a10e21bb37e4-kube-api-access-sxdzg\") on node \"ip-10-0-142-202.ec2.internal\" DevicePath \"\""
Jul 05 00:45:03.476831 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:45:03.476828    2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-h5jk5\" (UniqueName: \"kubernetes.io/projected/e72b157a-2968-4b65-9709-d077e8293eaa-kube-api-access-h5jk5\") on node \"ip-10-0-142-202.ec2.internal\" DevicePath \"\""
Jul 05 00:45:04.108212 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:45:04.108184    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-key-cleanup-29720205-jdftl"
Jul 05 00:45:04.108212 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:45:04.108192    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-key-cleanup-29720205-jdftl" event={"ID":"33370136-cdfa-45c3-9f8f-a10e21bb37e4","Type":"ContainerDied","Data":"498223d1eb0f6212e66a5e7abd357d17366cf21e5c0a00291653d6c1561abab1"}
Jul 05 00:45:04.108674 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:45:04.108259    2563 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="498223d1eb0f6212e66a5e7abd357d17366cf21e5c0a00291653d6c1561abab1"
Jul 05 00:45:04.109442 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:45:04.109424    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-key-cleanup-e2e-inf-a-4e7441-29720205-hqggk"
Jul 05 00:45:04.109442 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:45:04.109431    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-key-cleanup-e2e-inf-a-4e7441-29720205-hqggk" event={"ID":"e72b157a-2968-4b65-9709-d077e8293eaa","Type":"ContainerDied","Data":"4d468a9e6c8e5981c111afc51fa6408086a326dc2678742146da83ef2cb3c84a"}
Jul 05 00:45:04.109619 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:45:04.109465    2563 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="4d468a9e6c8e5981c111afc51fa6408086a326dc2678742146da83ef2cb3c84a"
Jul 05 00:45:04.117690 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:45:04.117667    2563 scope.go:117] "RemoveContainer" containerID="93632daedb6d62e06a0395e17ebce77eaa64e79d018b818b7013f7195f48f71c"
Jul 05 00:45:04.117882 ip-10-0-142-202 kubenswrapper[2563]: E0705 00:45:04.117863    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 10s restarting failed container=cleanup pod=maas-api-key-cleanup-e2e-inf-b-11db4a-29720205-dkrkf_opendatahub(0e349b60-e213-445d-8e58-0244a5dda437)\"" pod="opendatahub/maas-api-key-cleanup-e2e-inf-b-11db4a-29720205-dkrkf" podUID="0e349b60-e213-445d-8e58-0244a5dda437"
Jul 05 00:45:04.633750 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:45:04.633716    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-x5szm"]
Jul 05 00:45:18.195843 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:45:18.195808    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-inf-a-4e7441/test-model-4e7441-kserve-695f7d9fff-gv7fq"]
Jul 05 00:45:18.196300 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:45:18.196192    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e72b157a-2968-4b65-9709-d077e8293eaa" containerName="cleanup"
Jul 05 00:45:18.196300 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:45:18.196207    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="e72b157a-2968-4b65-9709-d077e8293eaa" containerName="cleanup"
Jul 05 00:45:18.196300 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:45:18.196244    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="33370136-cdfa-45c3-9f8f-a10e21bb37e4" containerName="cleanup"
Jul 05 00:45:18.196300 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:45:18.196253    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="33370136-cdfa-45c3-9f8f-a10e21bb37e4" containerName="cleanup"
Jul 05 00:45:18.196463 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:45:18.196324    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="e72b157a-2968-4b65-9709-d077e8293eaa" containerName="cleanup"
Jul 05 00:45:18.196463 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:45:18.196332    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="33370136-cdfa-45c3-9f8f-a10e21bb37e4" containerName="cleanup"
Jul 05 00:45:18.199430 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:45:18.199409    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-4e7441/test-model-4e7441-kserve-695f7d9fff-gv7fq"
Jul 05 00:45:18.201446 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:45:18.201426    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-a-4e7441\"/\"kube-root-ca.crt\""
Jul 05 00:45:18.201824 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:45:18.201805    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-a-4e7441\"/\"openshift-service-ca.crt\""
Jul 05 00:45:18.201920 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:45:18.201825    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-a-4e7441\"/\"test-model-4e7441-kserve-self-signed-certs\""
Jul 05 00:45:18.201920 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:45:18.201842    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-a-4e7441\"/\"default-dockercfg-98spd\""
Jul 05 00:45:18.208839 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:45:18.208819    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-a-4e7441/test-model-4e7441-kserve-695f7d9fff-gv7fq"]
Jul 05 00:45:18.300897 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:45:18.300867    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/bf8ed536-a450-45a6-9758-df18c69a4de1-model-cache\") pod \"test-model-4e7441-kserve-695f7d9fff-gv7fq\" (UID: \"bf8ed536-a450-45a6-9758-df18c69a4de1\") " pod="ai-tenant-e2e-inf-a-4e7441/test-model-4e7441-kserve-695f7d9fff-gv7fq"
Jul 05 00:45:18.301073 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:45:18.300909    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2nf6j\" (UniqueName: \"kubernetes.io/projected/bf8ed536-a450-45a6-9758-df18c69a4de1-kube-api-access-2nf6j\") pod \"test-model-4e7441-kserve-695f7d9fff-gv7fq\" (UID: \"bf8ed536-a450-45a6-9758-df18c69a4de1\") " pod="ai-tenant-e2e-inf-a-4e7441/test-model-4e7441-kserve-695f7d9fff-gv7fq"
Jul 05 00:45:18.301073 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:45:18.300941    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/bf8ed536-a450-45a6-9758-df18c69a4de1-tmp-dir\") pod \"test-model-4e7441-kserve-695f7d9fff-gv7fq\" (UID: \"bf8ed536-a450-45a6-9758-df18c69a4de1\") " pod="ai-tenant-e2e-inf-a-4e7441/test-model-4e7441-kserve-695f7d9fff-gv7fq"
Jul 05 00:45:18.301073 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:45:18.301028    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/bf8ed536-a450-45a6-9758-df18c69a4de1-tls-certs\") pod \"test-model-4e7441-kserve-695f7d9fff-gv7fq\" (UID: \"bf8ed536-a450-45a6-9758-df18c69a4de1\") " pod="ai-tenant-e2e-inf-a-4e7441/test-model-4e7441-kserve-695f7d9fff-gv7fq"
Jul 05 00:45:18.301073 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:45:18.301070    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/bf8ed536-a450-45a6-9758-df18c69a4de1-dshm\") pod \"test-model-4e7441-kserve-695f7d9fff-gv7fq\" (UID: \"bf8ed536-a450-45a6-9758-df18c69a4de1\") " pod="ai-tenant-e2e-inf-a-4e7441/test-model-4e7441-kserve-695f7d9fff-gv7fq"
Jul 05 00:45:18.301260 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:45:18.301120    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/bf8ed536-a450-45a6-9758-df18c69a4de1-home\") pod \"test-model-4e7441-kserve-695f7d9fff-gv7fq\" (UID: \"bf8ed536-a450-45a6-9758-df18c69a4de1\") " pod="ai-tenant-e2e-inf-a-4e7441/test-model-4e7441-kserve-695f7d9fff-gv7fq"
Jul 05 00:45:18.402040 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:45:18.402006    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2nf6j\" (UniqueName: \"kubernetes.io/projected/bf8ed536-a450-45a6-9758-df18c69a4de1-kube-api-access-2nf6j\") pod \"test-model-4e7441-kserve-695f7d9fff-gv7fq\" (UID: \"bf8ed536-a450-45a6-9758-df18c69a4de1\") " pod="ai-tenant-e2e-inf-a-4e7441/test-model-4e7441-kserve-695f7d9fff-gv7fq"
Jul 05 00:45:18.402261 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:45:18.402053    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/bf8ed536-a450-45a6-9758-df18c69a4de1-tmp-dir\") pod \"test-model-4e7441-kserve-695f7d9fff-gv7fq\" (UID: \"bf8ed536-a450-45a6-9758-df18c69a4de1\") " pod="ai-tenant-e2e-inf-a-4e7441/test-model-4e7441-kserve-695f7d9fff-gv7fq"
Jul 05 00:45:18.402261 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:45:18.402081    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/bf8ed536-a450-45a6-9758-df18c69a4de1-tls-certs\") pod \"test-model-4e7441-kserve-695f7d9fff-gv7fq\" (UID: \"bf8ed536-a450-45a6-9758-df18c69a4de1\") " pod="ai-tenant-e2e-inf-a-4e7441/test-model-4e7441-kserve-695f7d9fff-gv7fq"
Jul 05 00:45:18.402261 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:45:18.402110    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/bf8ed536-a450-45a6-9758-df18c69a4de1-dshm\") pod \"test-model-4e7441-kserve-695f7d9fff-gv7fq\" (UID: \"bf8ed536-a450-45a6-9758-df18c69a4de1\") " pod="ai-tenant-e2e-inf-a-4e7441/test-model-4e7441-kserve-695f7d9fff-gv7fq"
Jul 05 00:45:18.402261 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:45:18.402139    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/bf8ed536-a450-45a6-9758-df18c69a4de1-home\") pod \"test-model-4e7441-kserve-695f7d9fff-gv7fq\" (UID: \"bf8ed536-a450-45a6-9758-df18c69a4de1\") " pod="ai-tenant-e2e-inf-a-4e7441/test-model-4e7441-kserve-695f7d9fff-gv7fq"
Jul 05 00:45:18.402261 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:45:18.402199    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/bf8ed536-a450-45a6-9758-df18c69a4de1-model-cache\") pod \"test-model-4e7441-kserve-695f7d9fff-gv7fq\" (UID: \"bf8ed536-a450-45a6-9758-df18c69a4de1\") " pod="ai-tenant-e2e-inf-a-4e7441/test-model-4e7441-kserve-695f7d9fff-gv7fq"
Jul 05 00:45:18.402538 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:45:18.402500    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/bf8ed536-a450-45a6-9758-df18c69a4de1-tmp-dir\") pod \"test-model-4e7441-kserve-695f7d9fff-gv7fq\" (UID: \"bf8ed536-a450-45a6-9758-df18c69a4de1\") " pod="ai-tenant-e2e-inf-a-4e7441/test-model-4e7441-kserve-695f7d9fff-gv7fq"
Jul 05 00:45:18.402600 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:45:18.402567    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/bf8ed536-a450-45a6-9758-df18c69a4de1-home\") pod \"test-model-4e7441-kserve-695f7d9fff-gv7fq\" (UID: \"bf8ed536-a450-45a6-9758-df18c69a4de1\") " pod="ai-tenant-e2e-inf-a-4e7441/test-model-4e7441-kserve-695f7d9fff-gv7fq"
Jul 05 00:45:18.402658 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:45:18.402616    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/bf8ed536-a450-45a6-9758-df18c69a4de1-model-cache\") pod \"test-model-4e7441-kserve-695f7d9fff-gv7fq\" (UID: \"bf8ed536-a450-45a6-9758-df18c69a4de1\") " pod="ai-tenant-e2e-inf-a-4e7441/test-model-4e7441-kserve-695f7d9fff-gv7fq"
Jul 05 00:45:18.404522 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:45:18.404489    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/bf8ed536-a450-45a6-9758-df18c69a4de1-dshm\") pod \"test-model-4e7441-kserve-695f7d9fff-gv7fq\" (UID: \"bf8ed536-a450-45a6-9758-df18c69a4de1\") " pod="ai-tenant-e2e-inf-a-4e7441/test-model-4e7441-kserve-695f7d9fff-gv7fq"
Jul 05 00:45:18.404629 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:45:18.404557    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/bf8ed536-a450-45a6-9758-df18c69a4de1-tls-certs\") pod \"test-model-4e7441-kserve-695f7d9fff-gv7fq\" (UID: \"bf8ed536-a450-45a6-9758-df18c69a4de1\") " pod="ai-tenant-e2e-inf-a-4e7441/test-model-4e7441-kserve-695f7d9fff-gv7fq"
Jul 05 00:45:18.408888 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:45:18.408866    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-2nf6j\" (UniqueName: \"kubernetes.io/projected/bf8ed536-a450-45a6-9758-df18c69a4de1-kube-api-access-2nf6j\") pod \"test-model-4e7441-kserve-695f7d9fff-gv7fq\" (UID: \"bf8ed536-a450-45a6-9758-df18c69a4de1\") " pod="ai-tenant-e2e-inf-a-4e7441/test-model-4e7441-kserve-695f7d9fff-gv7fq"
Jul 05 00:45:18.509100 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:45:18.509010    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-4e7441/test-model-4e7441-kserve-695f7d9fff-gv7fq"
Jul 05 00:45:18.639780 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:45:18.639755    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-a-4e7441/test-model-4e7441-kserve-695f7d9fff-gv7fq"]
Jul 05 00:45:18.642397 ip-10-0-142-202 kubenswrapper[2563]: W0705 00:45:18.642353    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podbf8ed536_a450_45a6_9758_df18c69a4de1.slice/crio-1dd5806229129f36d435b9fbf776b48b9fae784d1c5e653a4add9f159115d758 WatchSource:0}: Error finding container 1dd5806229129f36d435b9fbf776b48b9fae784d1c5e653a4add9f159115d758: Status 404 returned error can't find the container with id 1dd5806229129f36d435b9fbf776b48b9fae784d1c5e653a4add9f159115d758
Jul 05 00:45:19.120873 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:45:19.120846    2563 scope.go:117] "RemoveContainer" containerID="93632daedb6d62e06a0395e17ebce77eaa64e79d018b818b7013f7195f48f71c"
Jul 05 00:45:19.186189 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:45:19.186159    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-4e7441/test-model-4e7441-kserve-695f7d9fff-gv7fq" event={"ID":"bf8ed536-a450-45a6-9758-df18c69a4de1","Type":"ContainerStarted","Data":"63eccb2a1baf95b3a0d294cf3a4f1c045185617a8aec168a171d9f64c441816b"}
Jul 05 00:45:19.186316 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:45:19.186206    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-4e7441/test-model-4e7441-kserve-695f7d9fff-gv7fq" event={"ID":"bf8ed536-a450-45a6-9758-df18c69a4de1","Type":"ContainerStarted","Data":"1dd5806229129f36d435b9fbf776b48b9fae784d1c5e653a4add9f159115d758"}
Jul 05 00:45:19.200003 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:45:19.199957    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-inf-a-4e7441/test-model-4e7441-kserve-695f7d9fff-gv7fq" podStartSLOduration=1.199937807 podStartE2EDuration="1.199937807s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-05 00:45:18 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-05 00:45:19.199183885 +0000 UTC m=+2167.541512944" watchObservedRunningTime="2026-07-05 00:45:19.199937807 +0000 UTC m=+2167.542266871"
Jul 05 00:45:20.192573 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:45:20.192543    2563 generic.go:356] "Generic (PLEG): container finished" podID="0e349b60-e213-445d-8e58-0244a5dda437" containerID="ce34358c677044defac3f93deeb3800c81974f4de28a955eebd08604b1075e92" exitCode=0
Jul 05 00:45:20.192718 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:45:20.192616    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-key-cleanup-e2e-inf-b-11db4a-29720205-dkrkf" event={"ID":"0e349b60-e213-445d-8e58-0244a5dda437","Type":"ContainerDied","Data":"ce34358c677044defac3f93deeb3800c81974f4de28a955eebd08604b1075e92"}
Jul 05 00:45:20.192718 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:45:20.192675    2563 scope.go:117] "RemoveContainer" containerID="93632daedb6d62e06a0395e17ebce77eaa64e79d018b818b7013f7195f48f71c"
Jul 05 00:45:21.220893 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:45:21.220863    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-api-key-cleanup-e2e-inf-b-11db4a-29720205-dkrkf"]
Jul 05 00:45:21.330660 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:45:21.330639    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-key-cleanup-e2e-inf-b-11db4a-29720205-dkrkf"
Jul 05 00:45:21.429063 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:45:21.429031    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-qbwpz\" (UniqueName: \"kubernetes.io/projected/0e349b60-e213-445d-8e58-0244a5dda437-kube-api-access-qbwpz\") pod \"0e349b60-e213-445d-8e58-0244a5dda437\" (UID: \"0e349b60-e213-445d-8e58-0244a5dda437\") "
Jul 05 00:45:21.430954 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:45:21.430929    2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0e349b60-e213-445d-8e58-0244a5dda437-kube-api-access-qbwpz" (OuterVolumeSpecName: "kube-api-access-qbwpz") pod "0e349b60-e213-445d-8e58-0244a5dda437" (UID: "0e349b60-e213-445d-8e58-0244a5dda437"). InnerVolumeSpecName "kube-api-access-qbwpz". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 05 00:45:21.530579 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:45:21.530555    2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-qbwpz\" (UniqueName: \"kubernetes.io/projected/0e349b60-e213-445d-8e58-0244a5dda437-kube-api-access-qbwpz\") on node \"ip-10-0-142-202.ec2.internal\" DevicePath \"\""
Jul 05 00:45:22.210783 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:45:22.210606    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-key-cleanup-e2e-inf-b-11db4a-29720205-dkrkf"
Jul 05 00:45:22.210783 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:45:22.210658    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-key-cleanup-e2e-inf-b-11db4a-29720205-dkrkf" event={"ID":"0e349b60-e213-445d-8e58-0244a5dda437","Type":"ContainerDied","Data":"c7840d5941f360f7790722bf798904c76556bac39405c2102b20522e6d24d0e8"}
Jul 05 00:45:22.210783 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:45:22.210704    2563 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="c7840d5941f360f7790722bf798904c76556bac39405c2102b20522e6d24d0e8"
Jul 05 00:45:22.230972 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:45:22.230942    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-x5szm"]
Jul 05 00:45:22.240196 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:45:22.240171    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-api-key-cleanup-e2e-inf-b-11db4a-29720205-dkrkf"]
Jul 05 00:45:22.243636 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:45:22.243612    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-api-key-cleanup-e2e-inf-b-11db4a-29720205-dkrkf"]
Jul 05 00:45:24.126215 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:45:24.126181    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="0e349b60-e213-445d-8e58-0244a5dda437" path="/var/lib/kubelet/pods/0e349b60-e213-445d-8e58-0244a5dda437/volumes"
Jul 05 00:45:28.509465 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:45:28.509434    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-inf-a-4e7441/test-model-4e7441-kserve-695f7d9fff-gv7fq"
Jul 05 00:45:28.509861 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:45:28.509483    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-inf-a-4e7441/test-model-4e7441-kserve-695f7d9fff-gv7fq"
Jul 05 00:45:28.521715 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:45:28.521690    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-inf-a-4e7441/test-model-4e7441-kserve-695f7d9fff-gv7fq"
Jul 05 00:45:29.254122 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:45:29.254095    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-inf-a-4e7441/test-model-4e7441-kserve-695f7d9fff-gv7fq"
Jul 05 00:45:37.101505 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:45:37.101473    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-inf-b-11db4a/test-model-11db4a-kserve-5d6c4f5fd7-t827w"]
Jul 05 00:45:37.101925 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:45:37.101804    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0e349b60-e213-445d-8e58-0244a5dda437" containerName="cleanup"
Jul 05 00:45:37.101925 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:45:37.101815    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="0e349b60-e213-445d-8e58-0244a5dda437" containerName="cleanup"
Jul 05 00:45:37.101925 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:45:37.101835    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0e349b60-e213-445d-8e58-0244a5dda437" containerName="cleanup"
Jul 05 00:45:37.101925 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:45:37.101841    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="0e349b60-e213-445d-8e58-0244a5dda437" containerName="cleanup"
Jul 05 00:45:37.101925 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:45:37.101891    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="0e349b60-e213-445d-8e58-0244a5dda437" containerName="cleanup"
Jul 05 00:45:37.101925 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:45:37.101900    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="0e349b60-e213-445d-8e58-0244a5dda437" containerName="cleanup"
Jul 05 00:45:37.101925 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:45:37.101908    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="0e349b60-e213-445d-8e58-0244a5dda437" containerName="cleanup"
Jul 05 00:45:37.106136 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:45:37.106119    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-11db4a/test-model-11db4a-kserve-5d6c4f5fd7-t827w"
Jul 05 00:45:37.112346 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:45:37.108957    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-b-11db4a\"/\"kube-root-ca.crt\""
Jul 05 00:45:37.112346 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:45:37.109031    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-b-11db4a\"/\"openshift-service-ca.crt\""
Jul 05 00:45:37.112346 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:45:37.108957    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-b-11db4a\"/\"default-dockercfg-w98mb\""
Jul 05 00:45:37.112346 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:45:37.108959    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-b-11db4a\"/\"test-model-11db4a-kserve-self-signed-certs\""
Jul 05 00:45:37.114516 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:45:37.114474    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-b-11db4a/test-model-11db4a-kserve-5d6c4f5fd7-t827w"]
Jul 05 00:45:37.171915 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:45:37.171889    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/6715015b-de5b-456d-bf58-e07940749636-tmp-dir\") pod \"test-model-11db4a-kserve-5d6c4f5fd7-t827w\" (UID: \"6715015b-de5b-456d-bf58-e07940749636\") " pod="ai-tenant-e2e-inf-b-11db4a/test-model-11db4a-kserve-5d6c4f5fd7-t827w"
Jul 05 00:45:37.172035 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:45:37.171933    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/6715015b-de5b-456d-bf58-e07940749636-home\") pod \"test-model-11db4a-kserve-5d6c4f5fd7-t827w\" (UID: \"6715015b-de5b-456d-bf58-e07940749636\") " pod="ai-tenant-e2e-inf-b-11db4a/test-model-11db4a-kserve-5d6c4f5fd7-t827w"
Jul 05 00:45:37.172035 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:45:37.171954    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/6715015b-de5b-456d-bf58-e07940749636-tls-certs\") pod \"test-model-11db4a-kserve-5d6c4f5fd7-t827w\" (UID: \"6715015b-de5b-456d-bf58-e07940749636\") " pod="ai-tenant-e2e-inf-b-11db4a/test-model-11db4a-kserve-5d6c4f5fd7-t827w"
Jul 05 00:45:37.172111 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:45:37.172050    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/6715015b-de5b-456d-bf58-e07940749636-dshm\") pod \"test-model-11db4a-kserve-5d6c4f5fd7-t827w\" (UID: \"6715015b-de5b-456d-bf58-e07940749636\") " pod="ai-tenant-e2e-inf-b-11db4a/test-model-11db4a-kserve-5d6c4f5fd7-t827w"
Jul 05 00:45:37.172111 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:45:37.172079    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ghn6p\" (UniqueName: \"kubernetes.io/projected/6715015b-de5b-456d-bf58-e07940749636-kube-api-access-ghn6p\") pod \"test-model-11db4a-kserve-5d6c4f5fd7-t827w\" (UID: \"6715015b-de5b-456d-bf58-e07940749636\") " pod="ai-tenant-e2e-inf-b-11db4a/test-model-11db4a-kserve-5d6c4f5fd7-t827w"
Jul 05 00:45:37.172174 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:45:37.172112    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/6715015b-de5b-456d-bf58-e07940749636-model-cache\") pod \"test-model-11db4a-kserve-5d6c4f5fd7-t827w\" (UID: \"6715015b-de5b-456d-bf58-e07940749636\") " pod="ai-tenant-e2e-inf-b-11db4a/test-model-11db4a-kserve-5d6c4f5fd7-t827w"
Jul 05 00:45:37.273436 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:45:37.273396    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/6715015b-de5b-456d-bf58-e07940749636-dshm\") pod \"test-model-11db4a-kserve-5d6c4f5fd7-t827w\" (UID: \"6715015b-de5b-456d-bf58-e07940749636\") " pod="ai-tenant-e2e-inf-b-11db4a/test-model-11db4a-kserve-5d6c4f5fd7-t827w"
Jul 05 00:45:37.273681 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:45:37.273448    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ghn6p\" (UniqueName: \"kubernetes.io/projected/6715015b-de5b-456d-bf58-e07940749636-kube-api-access-ghn6p\") pod \"test-model-11db4a-kserve-5d6c4f5fd7-t827w\" (UID: \"6715015b-de5b-456d-bf58-e07940749636\") " pod="ai-tenant-e2e-inf-b-11db4a/test-model-11db4a-kserve-5d6c4f5fd7-t827w"
Jul 05 00:45:37.273681 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:45:37.273480    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/6715015b-de5b-456d-bf58-e07940749636-model-cache\") pod \"test-model-11db4a-kserve-5d6c4f5fd7-t827w\" (UID: \"6715015b-de5b-456d-bf58-e07940749636\") " pod="ai-tenant-e2e-inf-b-11db4a/test-model-11db4a-kserve-5d6c4f5fd7-t827w"
Jul 05 00:45:37.273681 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:45:37.273516    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/6715015b-de5b-456d-bf58-e07940749636-tmp-dir\") pod \"test-model-11db4a-kserve-5d6c4f5fd7-t827w\" (UID: \"6715015b-de5b-456d-bf58-e07940749636\") " pod="ai-tenant-e2e-inf-b-11db4a/test-model-11db4a-kserve-5d6c4f5fd7-t827w"
Jul 05 00:45:37.273681 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:45:37.273569    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/6715015b-de5b-456d-bf58-e07940749636-home\") pod \"test-model-11db4a-kserve-5d6c4f5fd7-t827w\" (UID: \"6715015b-de5b-456d-bf58-e07940749636\") " pod="ai-tenant-e2e-inf-b-11db4a/test-model-11db4a-kserve-5d6c4f5fd7-t827w"
Jul 05 00:45:37.273681 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:45:37.273595    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/6715015b-de5b-456d-bf58-e07940749636-tls-certs\") pod \"test-model-11db4a-kserve-5d6c4f5fd7-t827w\" (UID: \"6715015b-de5b-456d-bf58-e07940749636\") " pod="ai-tenant-e2e-inf-b-11db4a/test-model-11db4a-kserve-5d6c4f5fd7-t827w"
Jul 05 00:45:37.274351 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:45:37.274322    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/6715015b-de5b-456d-bf58-e07940749636-model-cache\") pod \"test-model-11db4a-kserve-5d6c4f5fd7-t827w\" (UID: \"6715015b-de5b-456d-bf58-e07940749636\") " pod="ai-tenant-e2e-inf-b-11db4a/test-model-11db4a-kserve-5d6c4f5fd7-t827w"
Jul 05 00:45:37.274483 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:45:37.274430    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/6715015b-de5b-456d-bf58-e07940749636-home\") pod \"test-model-11db4a-kserve-5d6c4f5fd7-t827w\" (UID: \"6715015b-de5b-456d-bf58-e07940749636\") " pod="ai-tenant-e2e-inf-b-11db4a/test-model-11db4a-kserve-5d6c4f5fd7-t827w"
Jul 05 00:45:37.274753 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:45:37.274720    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/6715015b-de5b-456d-bf58-e07940749636-tmp-dir\") pod \"test-model-11db4a-kserve-5d6c4f5fd7-t827w\" (UID: \"6715015b-de5b-456d-bf58-e07940749636\") " pod="ai-tenant-e2e-inf-b-11db4a/test-model-11db4a-kserve-5d6c4f5fd7-t827w"
Jul 05 00:45:37.276562 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:45:37.276543    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/6715015b-de5b-456d-bf58-e07940749636-dshm\") pod \"test-model-11db4a-kserve-5d6c4f5fd7-t827w\" (UID: \"6715015b-de5b-456d-bf58-e07940749636\") " pod="ai-tenant-e2e-inf-b-11db4a/test-model-11db4a-kserve-5d6c4f5fd7-t827w"
Jul 05 00:45:37.276846 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:45:37.276826    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/6715015b-de5b-456d-bf58-e07940749636-tls-certs\") pod \"test-model-11db4a-kserve-5d6c4f5fd7-t827w\" (UID: \"6715015b-de5b-456d-bf58-e07940749636\") " pod="ai-tenant-e2e-inf-b-11db4a/test-model-11db4a-kserve-5d6c4f5fd7-t827w"
Jul 05 00:45:37.282900 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:45:37.282880    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-ghn6p\" (UniqueName: \"kubernetes.io/projected/6715015b-de5b-456d-bf58-e07940749636-kube-api-access-ghn6p\") pod \"test-model-11db4a-kserve-5d6c4f5fd7-t827w\" (UID: \"6715015b-de5b-456d-bf58-e07940749636\") " pod="ai-tenant-e2e-inf-b-11db4a/test-model-11db4a-kserve-5d6c4f5fd7-t827w"
Jul 05 00:45:37.421515 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:45:37.421440    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-11db4a/test-model-11db4a-kserve-5d6c4f5fd7-t827w"
Jul 05 00:45:37.552607 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:45:37.552575    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-b-11db4a/test-model-11db4a-kserve-5d6c4f5fd7-t827w"]
Jul 05 00:45:37.555589 ip-10-0-142-202 kubenswrapper[2563]: W0705 00:45:37.555561    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod6715015b_de5b_456d_bf58_e07940749636.slice/crio-28ce677af8a3383c30eaf5ef62c84487add1349967cb20c37f288068ca0d744a WatchSource:0}: Error finding container 28ce677af8a3383c30eaf5ef62c84487add1349967cb20c37f288068ca0d744a: Status 404 returned error can't find the container with id 28ce677af8a3383c30eaf5ef62c84487add1349967cb20c37f288068ca0d744a
Jul 05 00:45:38.285535 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:45:38.285504    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-11db4a/test-model-11db4a-kserve-5d6c4f5fd7-t827w" event={"ID":"6715015b-de5b-456d-bf58-e07940749636","Type":"ContainerStarted","Data":"12b00f182bd80e24918c7070ba3d3e37d8366acf66429e4435378d9be42af6d5"}
Jul 05 00:45:38.285947 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:45:38.285543    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-11db4a/test-model-11db4a-kserve-5d6c4f5fd7-t827w" event={"ID":"6715015b-de5b-456d-bf58-e07940749636","Type":"ContainerStarted","Data":"28ce677af8a3383c30eaf5ef62c84487add1349967cb20c37f288068ca0d744a"}
Jul 05 00:45:38.299553 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:45:38.299510    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-inf-b-11db4a/test-model-11db4a-kserve-5d6c4f5fd7-t827w" podStartSLOduration=1.299495235 podStartE2EDuration="1.299495235s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-05 00:45:37 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-05 00:45:38.29869477 +0000 UTC m=+2186.641023830" watchObservedRunningTime="2026-07-05 00:45:38.299495235 +0000 UTC m=+2186.641824305"
Jul 05 00:45:41.122850 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:45:41.122808    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-x5szm"]
Jul 05 00:45:47.422414 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:45:47.421700    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-inf-b-11db4a/test-model-11db4a-kserve-5d6c4f5fd7-t827w"
Jul 05 00:45:47.422414 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:45:47.421755    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-inf-b-11db4a/test-model-11db4a-kserve-5d6c4f5fd7-t827w"
Jul 05 00:45:47.454485 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:45:47.454460    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-inf-b-11db4a/test-model-11db4a-kserve-5d6c4f5fd7-t827w"
Jul 05 00:45:48.342620 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:45:48.342587    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-inf-b-11db4a/test-model-11db4a-kserve-5d6c4f5fd7-t827w"
Jul 05 00:45:53.322440 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:45:53.322409    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-x5szm"]
Jul 05 00:45:56.084359 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:45:56.084329    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-reuse-5b7734-openshift-default-5d48bfbc59-hl86n"]
Jul 05 00:45:56.084712 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:45:56.084659    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0e349b60-e213-445d-8e58-0244a5dda437" containerName="cleanup"
Jul 05 00:45:56.084712 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:45:56.084669    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="0e349b60-e213-445d-8e58-0244a5dda437" containerName="cleanup"
Jul 05 00:45:56.087893 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:45:56.087875    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-reuse-5b7734-openshift-default-5d48bfbc59-hl86n"
Jul 05 00:45:56.089801 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:45:56.089781    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-reuse-5b7734-openshift-default-dockercfg-lz58w\""
Jul 05 00:45:56.097347 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:45:56.097324    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-reuse-5b7734-openshift-default-5d48bfbc59-hl86n"]
Jul 05 00:45:56.235025 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:45:56.234990    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/decaaef4-8799-4958-ac74-1c0562838d9c-istio-envoy\") pod \"e2e-reuse-5b7734-openshift-default-5d48bfbc59-hl86n\" (UID: \"decaaef4-8799-4958-ac74-1c0562838d9c\") " pod="openshift-ingress/e2e-reuse-5b7734-openshift-default-5d48bfbc59-hl86n"
Jul 05 00:45:56.235248 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:45:56.235056    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/decaaef4-8799-4958-ac74-1c0562838d9c-istio-token\") pod \"e2e-reuse-5b7734-openshift-default-5d48bfbc59-hl86n\" (UID: \"decaaef4-8799-4958-ac74-1c0562838d9c\") " pod="openshift-ingress/e2e-reuse-5b7734-openshift-default-5d48bfbc59-hl86n"
Jul 05 00:45:56.235248 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:45:56.235096    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/decaaef4-8799-4958-ac74-1c0562838d9c-istiod-ca-cert\") pod \"e2e-reuse-5b7734-openshift-default-5d48bfbc59-hl86n\" (UID: \"decaaef4-8799-4958-ac74-1c0562838d9c\") " pod="openshift-ingress/e2e-reuse-5b7734-openshift-default-5d48bfbc59-hl86n"
Jul 05 00:45:56.235248 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:45:56.235140    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/decaaef4-8799-4958-ac74-1c0562838d9c-credential-socket\") pod \"e2e-reuse-5b7734-openshift-default-5d48bfbc59-hl86n\" (UID: \"decaaef4-8799-4958-ac74-1c0562838d9c\") " pod="openshift-ingress/e2e-reuse-5b7734-openshift-default-5d48bfbc59-hl86n"
Jul 05 00:45:56.235248 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:45:56.235173    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/decaaef4-8799-4958-ac74-1c0562838d9c-istio-podinfo\") pod \"e2e-reuse-5b7734-openshift-default-5d48bfbc59-hl86n\" (UID: \"decaaef4-8799-4958-ac74-1c0562838d9c\") " pod="openshift-ingress/e2e-reuse-5b7734-openshift-default-5d48bfbc59-hl86n"
Jul 05 00:45:56.235421 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:45:56.235270    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/decaaef4-8799-4958-ac74-1c0562838d9c-istio-data\") pod \"e2e-reuse-5b7734-openshift-default-5d48bfbc59-hl86n\" (UID: \"decaaef4-8799-4958-ac74-1c0562838d9c\") " pod="openshift-ingress/e2e-reuse-5b7734-openshift-default-5d48bfbc59-hl86n"
Jul 05 00:45:56.235421 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:45:56.235295    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rhprg\" (UniqueName: \"kubernetes.io/projected/decaaef4-8799-4958-ac74-1c0562838d9c-kube-api-access-rhprg\") pod \"e2e-reuse-5b7734-openshift-default-5d48bfbc59-hl86n\" (UID: \"decaaef4-8799-4958-ac74-1c0562838d9c\") " pod="openshift-ingress/e2e-reuse-5b7734-openshift-default-5d48bfbc59-hl86n"
Jul 05 00:45:56.235421 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:45:56.235340    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/decaaef4-8799-4958-ac74-1c0562838d9c-workload-socket\") pod \"e2e-reuse-5b7734-openshift-default-5d48bfbc59-hl86n\" (UID: \"decaaef4-8799-4958-ac74-1c0562838d9c\") " pod="openshift-ingress/e2e-reuse-5b7734-openshift-default-5d48bfbc59-hl86n"
Jul 05 00:45:56.235421 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:45:56.235376    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/decaaef4-8799-4958-ac74-1c0562838d9c-workload-certs\") pod \"e2e-reuse-5b7734-openshift-default-5d48bfbc59-hl86n\" (UID: \"decaaef4-8799-4958-ac74-1c0562838d9c\") " pod="openshift-ingress/e2e-reuse-5b7734-openshift-default-5d48bfbc59-hl86n"
Jul 05 00:45:56.336742 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:45:56.336665    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/decaaef4-8799-4958-ac74-1c0562838d9c-credential-socket\") pod \"e2e-reuse-5b7734-openshift-default-5d48bfbc59-hl86n\" (UID: \"decaaef4-8799-4958-ac74-1c0562838d9c\") " pod="openshift-ingress/e2e-reuse-5b7734-openshift-default-5d48bfbc59-hl86n"
Jul 05 00:45:56.336742 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:45:56.336702    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/decaaef4-8799-4958-ac74-1c0562838d9c-istio-podinfo\") pod \"e2e-reuse-5b7734-openshift-default-5d48bfbc59-hl86n\" (UID: \"decaaef4-8799-4958-ac74-1c0562838d9c\") " pod="openshift-ingress/e2e-reuse-5b7734-openshift-default-5d48bfbc59-hl86n"
Jul 05 00:45:56.336742 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:45:56.336731    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/decaaef4-8799-4958-ac74-1c0562838d9c-istio-data\") pod \"e2e-reuse-5b7734-openshift-default-5d48bfbc59-hl86n\" (UID: \"decaaef4-8799-4958-ac74-1c0562838d9c\") " pod="openshift-ingress/e2e-reuse-5b7734-openshift-default-5d48bfbc59-hl86n"
Jul 05 00:45:56.337018 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:45:56.336747    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rhprg\" (UniqueName: \"kubernetes.io/projected/decaaef4-8799-4958-ac74-1c0562838d9c-kube-api-access-rhprg\") pod \"e2e-reuse-5b7734-openshift-default-5d48bfbc59-hl86n\" (UID: \"decaaef4-8799-4958-ac74-1c0562838d9c\") " pod="openshift-ingress/e2e-reuse-5b7734-openshift-default-5d48bfbc59-hl86n"
Jul 05 00:45:56.337018 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:45:56.336767    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/decaaef4-8799-4958-ac74-1c0562838d9c-workload-socket\") pod \"e2e-reuse-5b7734-openshift-default-5d48bfbc59-hl86n\" (UID: \"decaaef4-8799-4958-ac74-1c0562838d9c\") " pod="openshift-ingress/e2e-reuse-5b7734-openshift-default-5d48bfbc59-hl86n"
Jul 05 00:45:56.337018 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:45:56.336792    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/decaaef4-8799-4958-ac74-1c0562838d9c-workload-certs\") pod \"e2e-reuse-5b7734-openshift-default-5d48bfbc59-hl86n\" (UID: \"decaaef4-8799-4958-ac74-1c0562838d9c\") " pod="openshift-ingress/e2e-reuse-5b7734-openshift-default-5d48bfbc59-hl86n"
Jul 05 00:45:56.337018 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:45:56.336815    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/decaaef4-8799-4958-ac74-1c0562838d9c-istio-envoy\") pod \"e2e-reuse-5b7734-openshift-default-5d48bfbc59-hl86n\" (UID: \"decaaef4-8799-4958-ac74-1c0562838d9c\") " pod="openshift-ingress/e2e-reuse-5b7734-openshift-default-5d48bfbc59-hl86n"
Jul 05 00:45:56.337018 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:45:56.336856    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/decaaef4-8799-4958-ac74-1c0562838d9c-istio-token\") pod \"e2e-reuse-5b7734-openshift-default-5d48bfbc59-hl86n\" (UID: \"decaaef4-8799-4958-ac74-1c0562838d9c\") " pod="openshift-ingress/e2e-reuse-5b7734-openshift-default-5d48bfbc59-hl86n"
Jul 05 00:45:56.337018 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:45:56.336882    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/decaaef4-8799-4958-ac74-1c0562838d9c-istiod-ca-cert\") pod \"e2e-reuse-5b7734-openshift-default-5d48bfbc59-hl86n\" (UID: \"decaaef4-8799-4958-ac74-1c0562838d9c\") " pod="openshift-ingress/e2e-reuse-5b7734-openshift-default-5d48bfbc59-hl86n"
Jul 05 00:45:56.337370 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:45:56.337109    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/decaaef4-8799-4958-ac74-1c0562838d9c-credential-socket\") pod \"e2e-reuse-5b7734-openshift-default-5d48bfbc59-hl86n\" (UID: \"decaaef4-8799-4958-ac74-1c0562838d9c\") " pod="openshift-ingress/e2e-reuse-5b7734-openshift-default-5d48bfbc59-hl86n"
Jul 05 00:45:56.337370 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:45:56.337190    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/decaaef4-8799-4958-ac74-1c0562838d9c-istio-data\") pod \"e2e-reuse-5b7734-openshift-default-5d48bfbc59-hl86n\" (UID: \"decaaef4-8799-4958-ac74-1c0562838d9c\") " pod="openshift-ingress/e2e-reuse-5b7734-openshift-default-5d48bfbc59-hl86n"
Jul 05 00:45:56.337370 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:45:56.337210    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/decaaef4-8799-4958-ac74-1c0562838d9c-workload-socket\") pod \"e2e-reuse-5b7734-openshift-default-5d48bfbc59-hl86n\" (UID: \"decaaef4-8799-4958-ac74-1c0562838d9c\") " pod="openshift-ingress/e2e-reuse-5b7734-openshift-default-5d48bfbc59-hl86n"
Jul 05 00:45:56.337370 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:45:56.337288    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/decaaef4-8799-4958-ac74-1c0562838d9c-workload-certs\") pod \"e2e-reuse-5b7734-openshift-default-5d48bfbc59-hl86n\" (UID: \"decaaef4-8799-4958-ac74-1c0562838d9c\") " pod="openshift-ingress/e2e-reuse-5b7734-openshift-default-5d48bfbc59-hl86n"
Jul 05 00:45:56.337776 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:45:56.337758    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/decaaef4-8799-4958-ac74-1c0562838d9c-istiod-ca-cert\") pod \"e2e-reuse-5b7734-openshift-default-5d48bfbc59-hl86n\" (UID: \"decaaef4-8799-4958-ac74-1c0562838d9c\") " pod="openshift-ingress/e2e-reuse-5b7734-openshift-default-5d48bfbc59-hl86n"
Jul 05 00:45:56.339136 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:45:56.339111    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/decaaef4-8799-4958-ac74-1c0562838d9c-istio-envoy\") pod \"e2e-reuse-5b7734-openshift-default-5d48bfbc59-hl86n\" (UID: \"decaaef4-8799-4958-ac74-1c0562838d9c\") " pod="openshift-ingress/e2e-reuse-5b7734-openshift-default-5d48bfbc59-hl86n"
Jul 05 00:45:56.339310 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:45:56.339288    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/decaaef4-8799-4958-ac74-1c0562838d9c-istio-podinfo\") pod \"e2e-reuse-5b7734-openshift-default-5d48bfbc59-hl86n\" (UID: \"decaaef4-8799-4958-ac74-1c0562838d9c\") " pod="openshift-ingress/e2e-reuse-5b7734-openshift-default-5d48bfbc59-hl86n"
Jul 05 00:45:56.343724 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:45:56.343702    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-rhprg\" (UniqueName: \"kubernetes.io/projected/decaaef4-8799-4958-ac74-1c0562838d9c-kube-api-access-rhprg\") pod \"e2e-reuse-5b7734-openshift-default-5d48bfbc59-hl86n\" (UID: \"decaaef4-8799-4958-ac74-1c0562838d9c\") " pod="openshift-ingress/e2e-reuse-5b7734-openshift-default-5d48bfbc59-hl86n"
Jul 05 00:45:56.343968 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:45:56.343948    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/decaaef4-8799-4958-ac74-1c0562838d9c-istio-token\") pod \"e2e-reuse-5b7734-openshift-default-5d48bfbc59-hl86n\" (UID: \"decaaef4-8799-4958-ac74-1c0562838d9c\") " pod="openshift-ingress/e2e-reuse-5b7734-openshift-default-5d48bfbc59-hl86n"
Jul 05 00:45:56.398506 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:45:56.398485    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-reuse-5b7734-openshift-default-5d48bfbc59-hl86n"
Jul 05 00:45:56.531418 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:45:56.531395    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-reuse-5b7734-openshift-default-5d48bfbc59-hl86n"]
Jul 05 00:45:56.533158 ip-10-0-142-202 kubenswrapper[2563]: W0705 00:45:56.533123    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddecaaef4_8799_4958_ac74_1c0562838d9c.slice/crio-74e1588212288bf3b7a8953640244a78e2d25999c7c405b1ae010d375bf4bc56 WatchSource:0}: Error finding container 74e1588212288bf3b7a8953640244a78e2d25999c7c405b1ae010d375bf4bc56: Status 404 returned error can't find the container with id 74e1588212288bf3b7a8953640244a78e2d25999c7c405b1ae010d375bf4bc56
Jul 05 00:45:56.535346 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:45:56.535313    2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"}
Jul 05 00:45:56.535439 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:45:56.535374    2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"}
Jul 05 00:45:56.535439 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:45:56.535400    2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"}
Jul 05 00:45:57.372687 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:45:57.372653    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-reuse-5b7734-openshift-default-5d48bfbc59-hl86n" event={"ID":"decaaef4-8799-4958-ac74-1c0562838d9c","Type":"ContainerStarted","Data":"a3fc2b2f98f14dff3e4b48a0ebb9637176804c45f64af3de6bd802ea3e1b3f5d"}
Jul 05 00:45:57.373075 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:45:57.372702    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-reuse-5b7734-openshift-default-5d48bfbc59-hl86n" event={"ID":"decaaef4-8799-4958-ac74-1c0562838d9c","Type":"ContainerStarted","Data":"74e1588212288bf3b7a8953640244a78e2d25999c7c405b1ae010d375bf4bc56"}
Jul 05 00:45:57.389812 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:45:57.389768    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-reuse-5b7734-openshift-default-5d48bfbc59-hl86n" podStartSLOduration=1.389750871 podStartE2EDuration="1.389750871s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-05 00:45:56 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-05 00:45:57.388194743 +0000 UTC m=+2205.730523821" watchObservedRunningTime="2026-07-05 00:45:57.389750871 +0000 UTC m=+2205.732079931"
Jul 05 00:45:57.398798 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:45:57.398777    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-reuse-5b7734-openshift-default-5d48bfbc59-hl86n"
Jul 05 00:45:57.403614 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:45:57.403593    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-reuse-5b7734-openshift-default-5d48bfbc59-hl86n"
Jul 05 00:45:58.377591 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:45:58.377566    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-reuse-5b7734-openshift-default-5d48bfbc59-hl86n"
Jul 05 00:45:58.378569 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:45:58.378548    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-reuse-5b7734-openshift-default-5d48bfbc59-hl86n"
Jul 05 00:46:05.302472 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:46:05.302418    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-reuse-5b7734/test-model-5b7734-kserve-5977fc877-dzpdk"]
Jul 05 00:46:05.305841 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:46:05.305825    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-reuse-5b7734/test-model-5b7734-kserve-5977fc877-dzpdk"
Jul 05 00:46:05.308034 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:46:05.308010    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-reuse-5b7734\"/\"kube-root-ca.crt\""
Jul 05 00:46:05.308167 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:46:05.308010    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-reuse-5b7734\"/\"openshift-service-ca.crt\""
Jul 05 00:46:05.308676 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:46:05.308656    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-reuse-5b7734\"/\"test-model-5b7734-kserve-self-signed-certs\""
Jul 05 00:46:05.308781 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:46:05.308656    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-reuse-5b7734\"/\"default-dockercfg-8d8d7\""
Jul 05 00:46:05.313240 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:46:05.313195    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-reuse-5b7734/test-model-5b7734-kserve-5977fc877-dzpdk"]
Jul 05 00:46:05.412799 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:46:05.412765    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nkpzl\" (UniqueName: \"kubernetes.io/projected/a20027d2-b79a-4b92-8182-8d4589f961a6-kube-api-access-nkpzl\") pod \"test-model-5b7734-kserve-5977fc877-dzpdk\" (UID: \"a20027d2-b79a-4b92-8182-8d4589f961a6\") " pod="ai-tenant-e2e-reuse-5b7734/test-model-5b7734-kserve-5977fc877-dzpdk"
Jul 05 00:46:05.412986 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:46:05.412805    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a20027d2-b79a-4b92-8182-8d4589f961a6-tls-certs\") pod \"test-model-5b7734-kserve-5977fc877-dzpdk\" (UID: \"a20027d2-b79a-4b92-8182-8d4589f961a6\") " pod="ai-tenant-e2e-reuse-5b7734/test-model-5b7734-kserve-5977fc877-dzpdk"
Jul 05 00:46:05.412986 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:46:05.412866    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a20027d2-b79a-4b92-8182-8d4589f961a6-dshm\") pod \"test-model-5b7734-kserve-5977fc877-dzpdk\" (UID: \"a20027d2-b79a-4b92-8182-8d4589f961a6\") " pod="ai-tenant-e2e-reuse-5b7734/test-model-5b7734-kserve-5977fc877-dzpdk"
Jul 05 00:46:05.412986 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:46:05.412906    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a20027d2-b79a-4b92-8182-8d4589f961a6-home\") pod \"test-model-5b7734-kserve-5977fc877-dzpdk\" (UID: \"a20027d2-b79a-4b92-8182-8d4589f961a6\") " pod="ai-tenant-e2e-reuse-5b7734/test-model-5b7734-kserve-5977fc877-dzpdk"
Jul 05 00:46:05.412986 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:46:05.412959    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a20027d2-b79a-4b92-8182-8d4589f961a6-tmp-dir\") pod \"test-model-5b7734-kserve-5977fc877-dzpdk\" (UID: \"a20027d2-b79a-4b92-8182-8d4589f961a6\") " pod="ai-tenant-e2e-reuse-5b7734/test-model-5b7734-kserve-5977fc877-dzpdk"
Jul 05 00:46:05.413146 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:46:05.413008    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a20027d2-b79a-4b92-8182-8d4589f961a6-model-cache\") pod \"test-model-5b7734-kserve-5977fc877-dzpdk\" (UID: \"a20027d2-b79a-4b92-8182-8d4589f961a6\") " pod="ai-tenant-e2e-reuse-5b7734/test-model-5b7734-kserve-5977fc877-dzpdk"
Jul 05 00:46:05.514231 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:46:05.514193    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a20027d2-b79a-4b92-8182-8d4589f961a6-tmp-dir\") pod \"test-model-5b7734-kserve-5977fc877-dzpdk\" (UID: \"a20027d2-b79a-4b92-8182-8d4589f961a6\") " pod="ai-tenant-e2e-reuse-5b7734/test-model-5b7734-kserve-5977fc877-dzpdk"
Jul 05 00:46:05.514404 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:46:05.514307    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a20027d2-b79a-4b92-8182-8d4589f961a6-model-cache\") pod \"test-model-5b7734-kserve-5977fc877-dzpdk\" (UID: \"a20027d2-b79a-4b92-8182-8d4589f961a6\") " pod="ai-tenant-e2e-reuse-5b7734/test-model-5b7734-kserve-5977fc877-dzpdk"
Jul 05 00:46:05.514404 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:46:05.514347    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nkpzl\" (UniqueName: \"kubernetes.io/projected/a20027d2-b79a-4b92-8182-8d4589f961a6-kube-api-access-nkpzl\") pod \"test-model-5b7734-kserve-5977fc877-dzpdk\" (UID: \"a20027d2-b79a-4b92-8182-8d4589f961a6\") " pod="ai-tenant-e2e-reuse-5b7734/test-model-5b7734-kserve-5977fc877-dzpdk"
Jul 05 00:46:05.514404 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:46:05.514367    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a20027d2-b79a-4b92-8182-8d4589f961a6-tls-certs\") pod \"test-model-5b7734-kserve-5977fc877-dzpdk\" (UID: \"a20027d2-b79a-4b92-8182-8d4589f961a6\") " pod="ai-tenant-e2e-reuse-5b7734/test-model-5b7734-kserve-5977fc877-dzpdk"
Jul 05 00:46:05.514404 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:46:05.514396    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a20027d2-b79a-4b92-8182-8d4589f961a6-dshm\") pod \"test-model-5b7734-kserve-5977fc877-dzpdk\" (UID: \"a20027d2-b79a-4b92-8182-8d4589f961a6\") " pod="ai-tenant-e2e-reuse-5b7734/test-model-5b7734-kserve-5977fc877-dzpdk"
Jul 05 00:46:05.514568 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:46:05.514526    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a20027d2-b79a-4b92-8182-8d4589f961a6-home\") pod \"test-model-5b7734-kserve-5977fc877-dzpdk\" (UID: \"a20027d2-b79a-4b92-8182-8d4589f961a6\") " pod="ai-tenant-e2e-reuse-5b7734/test-model-5b7734-kserve-5977fc877-dzpdk"
Jul 05 00:46:05.514674 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:46:05.514654    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a20027d2-b79a-4b92-8182-8d4589f961a6-tmp-dir\") pod \"test-model-5b7734-kserve-5977fc877-dzpdk\" (UID: \"a20027d2-b79a-4b92-8182-8d4589f961a6\") " pod="ai-tenant-e2e-reuse-5b7734/test-model-5b7734-kserve-5977fc877-dzpdk"
Jul 05 00:46:05.514786 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:46:05.514688    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a20027d2-b79a-4b92-8182-8d4589f961a6-model-cache\") pod \"test-model-5b7734-kserve-5977fc877-dzpdk\" (UID: \"a20027d2-b79a-4b92-8182-8d4589f961a6\") " pod="ai-tenant-e2e-reuse-5b7734/test-model-5b7734-kserve-5977fc877-dzpdk"
Jul 05 00:46:05.514902 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:46:05.514883    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a20027d2-b79a-4b92-8182-8d4589f961a6-home\") pod \"test-model-5b7734-kserve-5977fc877-dzpdk\" (UID: \"a20027d2-b79a-4b92-8182-8d4589f961a6\") " pod="ai-tenant-e2e-reuse-5b7734/test-model-5b7734-kserve-5977fc877-dzpdk"
Jul 05 00:46:05.516691 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:46:05.516668    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a20027d2-b79a-4b92-8182-8d4589f961a6-dshm\") pod \"test-model-5b7734-kserve-5977fc877-dzpdk\" (UID: \"a20027d2-b79a-4b92-8182-8d4589f961a6\") " pod="ai-tenant-e2e-reuse-5b7734/test-model-5b7734-kserve-5977fc877-dzpdk"
Jul 05 00:46:05.516825 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:46:05.516812    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a20027d2-b79a-4b92-8182-8d4589f961a6-tls-certs\") pod \"test-model-5b7734-kserve-5977fc877-dzpdk\" (UID: \"a20027d2-b79a-4b92-8182-8d4589f961a6\") " pod="ai-tenant-e2e-reuse-5b7734/test-model-5b7734-kserve-5977fc877-dzpdk"
Jul 05 00:46:05.521101 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:46:05.521084    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-nkpzl\" (UniqueName: \"kubernetes.io/projected/a20027d2-b79a-4b92-8182-8d4589f961a6-kube-api-access-nkpzl\") pod \"test-model-5b7734-kserve-5977fc877-dzpdk\" (UID: \"a20027d2-b79a-4b92-8182-8d4589f961a6\") " pod="ai-tenant-e2e-reuse-5b7734/test-model-5b7734-kserve-5977fc877-dzpdk"
Jul 05 00:46:05.616953 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:46:05.616884    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-reuse-5b7734/test-model-5b7734-kserve-5977fc877-dzpdk"
Jul 05 00:46:05.744381 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:46:05.744310    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-reuse-5b7734/test-model-5b7734-kserve-5977fc877-dzpdk"]
Jul 05 00:46:05.748497 ip-10-0-142-202 kubenswrapper[2563]: W0705 00:46:05.748468    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-poda20027d2_b79a_4b92_8182_8d4589f961a6.slice/crio-f51f196b3de0a3a3c1c9ba49ba3ee5464f5a58c5bc7c37f69d7708f448ce0841 WatchSource:0}: Error finding container f51f196b3de0a3a3c1c9ba49ba3ee5464f5a58c5bc7c37f69d7708f448ce0841: Status 404 returned error can't find the container with id f51f196b3de0a3a3c1c9ba49ba3ee5464f5a58c5bc7c37f69d7708f448ce0841
Jul 05 00:46:06.416478 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:46:06.416445    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-reuse-5b7734/test-model-5b7734-kserve-5977fc877-dzpdk" event={"ID":"a20027d2-b79a-4b92-8182-8d4589f961a6","Type":"ContainerStarted","Data":"0148e4545d1e911547088a9b712587ceb3d9cea821fa357818bca0309944d5a3"}
Jul 05 00:46:06.416478 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:46:06.416486    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-reuse-5b7734/test-model-5b7734-kserve-5977fc877-dzpdk" event={"ID":"a20027d2-b79a-4b92-8182-8d4589f961a6","Type":"ContainerStarted","Data":"f51f196b3de0a3a3c1c9ba49ba3ee5464f5a58c5bc7c37f69d7708f448ce0841"}
Jul 05 00:46:06.431797 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:46:06.431752    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-reuse-5b7734/test-model-5b7734-kserve-5977fc877-dzpdk" podStartSLOduration=1.431736667 podStartE2EDuration="1.431736667s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-05 00:46:05 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-05 00:46:06.429721291 +0000 UTC m=+2214.772050372" watchObservedRunningTime="2026-07-05 00:46:06.431736667 +0000 UTC m=+2214.774065726"
Jul 05 00:46:13.931157 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:46:13.931122    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-x5szm"]
Jul 05 00:46:15.617328 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:46:15.617290    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-reuse-5b7734/test-model-5b7734-kserve-5977fc877-dzpdk"
Jul 05 00:46:15.617328 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:46:15.617338    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-reuse-5b7734/test-model-5b7734-kserve-5977fc877-dzpdk"
Jul 05 00:46:15.629825 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:46:15.629798    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-reuse-5b7734/test-model-5b7734-kserve-5977fc877-dzpdk"
Jul 05 00:46:16.495464 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:46:16.495435    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-reuse-5b7734/test-model-5b7734-kserve-5977fc877-dzpdk"
Jul 05 00:46:22.497389 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:46:22.497311    2563 generic.go:356] "Generic (PLEG): container finished" podID="decaaef4-8799-4958-ac74-1c0562838d9c" containerID="a3fc2b2f98f14dff3e4b48a0ebb9637176804c45f64af3de6bd802ea3e1b3f5d" exitCode=0
Jul 05 00:46:22.497753 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:46:22.497380    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-reuse-5b7734-openshift-default-5d48bfbc59-hl86n" event={"ID":"decaaef4-8799-4958-ac74-1c0562838d9c","Type":"ContainerDied","Data":"a3fc2b2f98f14dff3e4b48a0ebb9637176804c45f64af3de6bd802ea3e1b3f5d"}
Jul 05 00:46:22.497753 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:46:22.497752    2563 scope.go:117] "RemoveContainer" containerID="a3fc2b2f98f14dff3e4b48a0ebb9637176804c45f64af3de6bd802ea3e1b3f5d"
Jul 05 00:46:22.498744 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:46:22.498714    2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"}
Jul 05 00:46:22.498796 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:46:22.498784    2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"}
Jul 05 00:46:22.498830 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:46:22.498819    2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"}
Jul 05 00:46:23.503678 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:46:23.503643    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-reuse-5b7734-openshift-default-5d48bfbc59-hl86n" event={"ID":"decaaef4-8799-4958-ac74-1c0562838d9c","Type":"ContainerStarted","Data":"a457675cfef4fec259667387fbe6ef7b684623ccd1452f1ccfd188823a9a7841"}
Jul 05 00:46:24.398674 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:46:24.398621    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-reuse-5b7734-openshift-default-5d48bfbc59-hl86n"
Jul 05 00:46:24.399882 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:46:24.399860    2563 patch_prober.go:28] interesting pod/e2e-reuse-5b7734-openshift-default-5d48bfbc59-hl86n container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.40:15021/healthz/ready\": dial tcp 10.133.0.40:15021: connect: connection refused" start-of-body=
Jul 05 00:46:24.399937 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:46:24.399909    2563 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-reuse-5b7734-openshift-default-5d48bfbc59-hl86n" podUID="decaaef4-8799-4958-ac74-1c0562838d9c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.40:15021/healthz/ready\": dial tcp 10.133.0.40:15021: connect: connection refused"
Jul 05 00:46:25.399542 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:46:25.399506    2563 patch_prober.go:28] interesting pod/e2e-reuse-5b7734-openshift-default-5d48bfbc59-hl86n container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.40:15021/healthz/ready\": dial tcp 10.133.0.40:15021: connect: connection refused" start-of-body=
Jul 05 00:46:25.399921 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:46:25.399572    2563 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-reuse-5b7734-openshift-default-5d48bfbc59-hl86n" podUID="decaaef4-8799-4958-ac74-1c0562838d9c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.40:15021/healthz/ready\": dial tcp 10.133.0.40:15021: connect: connection refused"
Jul 05 00:46:26.399487 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:46:26.399448    2563 patch_prober.go:28] interesting pod/e2e-reuse-5b7734-openshift-default-5d48bfbc59-hl86n container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.40:15021/healthz/ready\": dial tcp 10.133.0.40:15021: connect: connection refused" start-of-body=
Jul 05 00:46:26.399666 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:46:26.399507    2563 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-reuse-5b7734-openshift-default-5d48bfbc59-hl86n" podUID="decaaef4-8799-4958-ac74-1c0562838d9c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.40:15021/healthz/ready\": dial tcp 10.133.0.40:15021: connect: connection refused"
Jul 05 00:46:27.403310 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:46:27.403278    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-reuse-5b7734-openshift-default-5d48bfbc59-hl86n"
Jul 05 00:46:27.403682 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:46:27.403563    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-reuse-5b7734-openshift-default-5d48bfbc59-hl86n"
Jul 05 00:46:27.404175 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:46:27.404159    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-reuse-5b7734-openshift-default-5d48bfbc59-hl86n"
Jul 05 00:46:36.137079 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:46:36.137050    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-reuse-5b7734/test-model-5b7734-kserve-5977fc877-dzpdk"]
Jul 05 00:46:38.621563 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:46:38.621521    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-x5szm"]
Jul 05 00:46:51.158139 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:46:51.158072    2563 kuberuntime_container.go:872] "Killing container with a grace period" pod="ai-tenant-e2e-reuse-5b7734/test-model-5b7734-kserve-5977fc877-dzpdk" podUID="a20027d2-b79a-4b92-8182-8d4589f961a6" containerName="main" containerID="cri-o://0148e4545d1e911547088a9b712587ceb3d9cea821fa357818bca0309944d5a3" gracePeriod=45
Jul 05 00:46:51.413426 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:46:51.413370    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-reuse-5b7734/test-model-5b7734-kserve-5977fc877-dzpdk"
Jul 05 00:46:51.532475 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:46:51.532447    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-nkpzl\" (UniqueName: \"kubernetes.io/projected/a20027d2-b79a-4b92-8182-8d4589f961a6-kube-api-access-nkpzl\") pod \"a20027d2-b79a-4b92-8182-8d4589f961a6\" (UID: \"a20027d2-b79a-4b92-8182-8d4589f961a6\") "
Jul 05 00:46:51.532714 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:46:51.532488    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a20027d2-b79a-4b92-8182-8d4589f961a6-tls-certs\") pod \"a20027d2-b79a-4b92-8182-8d4589f961a6\" (UID: \"a20027d2-b79a-4b92-8182-8d4589f961a6\") "
Jul 05 00:46:51.532714 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:46:51.532506    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a20027d2-b79a-4b92-8182-8d4589f961a6-dshm\") pod \"a20027d2-b79a-4b92-8182-8d4589f961a6\" (UID: \"a20027d2-b79a-4b92-8182-8d4589f961a6\") "
Jul 05 00:46:51.532714 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:46:51.532556    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a20027d2-b79a-4b92-8182-8d4589f961a6-home\") pod \"a20027d2-b79a-4b92-8182-8d4589f961a6\" (UID: \"a20027d2-b79a-4b92-8182-8d4589f961a6\") "
Jul 05 00:46:51.532714 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:46:51.532574    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a20027d2-b79a-4b92-8182-8d4589f961a6-tmp-dir\") pod \"a20027d2-b79a-4b92-8182-8d4589f961a6\" (UID: \"a20027d2-b79a-4b92-8182-8d4589f961a6\") "
Jul 05 00:46:51.532714 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:46:51.532590    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a20027d2-b79a-4b92-8182-8d4589f961a6-model-cache\") pod \"a20027d2-b79a-4b92-8182-8d4589f961a6\" (UID: \"a20027d2-b79a-4b92-8182-8d4589f961a6\") "
Jul 05 00:46:51.533000 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:46:51.532865    2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a20027d2-b79a-4b92-8182-8d4589f961a6-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "a20027d2-b79a-4b92-8182-8d4589f961a6" (UID: "a20027d2-b79a-4b92-8182-8d4589f961a6"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 05 00:46:51.533000 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:46:51.532900    2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a20027d2-b79a-4b92-8182-8d4589f961a6-home" (OuterVolumeSpecName: "home") pod "a20027d2-b79a-4b92-8182-8d4589f961a6" (UID: "a20027d2-b79a-4b92-8182-8d4589f961a6"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 05 00:46:51.533000 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:46:51.532941    2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a20027d2-b79a-4b92-8182-8d4589f961a6-model-cache" (OuterVolumeSpecName: "model-cache") pod "a20027d2-b79a-4b92-8182-8d4589f961a6" (UID: "a20027d2-b79a-4b92-8182-8d4589f961a6"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 05 00:46:51.535109 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:46:51.535075    2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a20027d2-b79a-4b92-8182-8d4589f961a6-dshm" (OuterVolumeSpecName: "dshm") pod "a20027d2-b79a-4b92-8182-8d4589f961a6" (UID: "a20027d2-b79a-4b92-8182-8d4589f961a6"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 05 00:46:51.535528 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:46:51.535505    2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a20027d2-b79a-4b92-8182-8d4589f961a6-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "a20027d2-b79a-4b92-8182-8d4589f961a6" (UID: "a20027d2-b79a-4b92-8182-8d4589f961a6"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 05 00:46:51.535528 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:46:51.535526    2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a20027d2-b79a-4b92-8182-8d4589f961a6-kube-api-access-nkpzl" (OuterVolumeSpecName: "kube-api-access-nkpzl") pod "a20027d2-b79a-4b92-8182-8d4589f961a6" (UID: "a20027d2-b79a-4b92-8182-8d4589f961a6"). InnerVolumeSpecName "kube-api-access-nkpzl". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 05 00:46:51.633241 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:46:51.633192    2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-nkpzl\" (UniqueName: \"kubernetes.io/projected/a20027d2-b79a-4b92-8182-8d4589f961a6-kube-api-access-nkpzl\") on node \"ip-10-0-142-202.ec2.internal\" DevicePath \"\""
Jul 05 00:46:51.633241 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:46:51.633231    2563 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a20027d2-b79a-4b92-8182-8d4589f961a6-tls-certs\") on node \"ip-10-0-142-202.ec2.internal\" DevicePath \"\""
Jul 05 00:46:51.633241 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:46:51.633243    2563 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a20027d2-b79a-4b92-8182-8d4589f961a6-dshm\") on node \"ip-10-0-142-202.ec2.internal\" DevicePath \"\""
Jul 05 00:46:51.633435 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:46:51.633251    2563 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a20027d2-b79a-4b92-8182-8d4589f961a6-home\") on node \"ip-10-0-142-202.ec2.internal\" DevicePath \"\""
Jul 05 00:46:51.633435 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:46:51.633259    2563 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a20027d2-b79a-4b92-8182-8d4589f961a6-tmp-dir\") on node \"ip-10-0-142-202.ec2.internal\" DevicePath \"\""
Jul 05 00:46:51.633435 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:46:51.633266    2563 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a20027d2-b79a-4b92-8182-8d4589f961a6-model-cache\") on node \"ip-10-0-142-202.ec2.internal\" DevicePath \"\""
Jul 05 00:46:51.635118 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:46:51.635094    2563 generic.go:356] "Generic (PLEG): container finished" podID="a20027d2-b79a-4b92-8182-8d4589f961a6" containerID="0148e4545d1e911547088a9b712587ceb3d9cea821fa357818bca0309944d5a3" exitCode=0
Jul 05 00:46:51.635200 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:46:51.635137    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-reuse-5b7734/test-model-5b7734-kserve-5977fc877-dzpdk"
Jul 05 00:46:51.635200 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:46:51.635183    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-reuse-5b7734/test-model-5b7734-kserve-5977fc877-dzpdk" event={"ID":"a20027d2-b79a-4b92-8182-8d4589f961a6","Type":"ContainerDied","Data":"0148e4545d1e911547088a9b712587ceb3d9cea821fa357818bca0309944d5a3"}
Jul 05 00:46:51.635286 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:46:51.635231    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-reuse-5b7734/test-model-5b7734-kserve-5977fc877-dzpdk" event={"ID":"a20027d2-b79a-4b92-8182-8d4589f961a6","Type":"ContainerDied","Data":"f51f196b3de0a3a3c1c9ba49ba3ee5464f5a58c5bc7c37f69d7708f448ce0841"}
Jul 05 00:46:51.635286 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:46:51.635250    2563 scope.go:117] "RemoveContainer" containerID="0148e4545d1e911547088a9b712587ceb3d9cea821fa357818bca0309944d5a3"
Jul 05 00:46:51.644502 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:46:51.644487    2563 scope.go:117] "RemoveContainer" containerID="0148e4545d1e911547088a9b712587ceb3d9cea821fa357818bca0309944d5a3"
Jul 05 00:46:51.644745 ip-10-0-142-202 kubenswrapper[2563]: E0705 00:46:51.644724    2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0148e4545d1e911547088a9b712587ceb3d9cea821fa357818bca0309944d5a3\": container with ID starting with 0148e4545d1e911547088a9b712587ceb3d9cea821fa357818bca0309944d5a3 not found: ID does not exist" containerID="0148e4545d1e911547088a9b712587ceb3d9cea821fa357818bca0309944d5a3"
Jul 05 00:46:51.644806 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:46:51.644759    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0148e4545d1e911547088a9b712587ceb3d9cea821fa357818bca0309944d5a3"} err="failed to get container status \"0148e4545d1e911547088a9b712587ceb3d9cea821fa357818bca0309944d5a3\": rpc error: code = NotFound desc = could not find container \"0148e4545d1e911547088a9b712587ceb3d9cea821fa357818bca0309944d5a3\": container with ID starting with 0148e4545d1e911547088a9b712587ceb3d9cea821fa357818bca0309944d5a3 not found: ID does not exist"
Jul 05 00:46:51.654283 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:46:51.654261    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-reuse-5b7734/test-model-5b7734-kserve-5977fc877-dzpdk"]
Jul 05 00:46:51.657424 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:46:51.657406    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-reuse-5b7734/test-model-5b7734-kserve-5977fc877-dzpdk"]
Jul 05 00:46:52.126580 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:46:52.126549    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a20027d2-b79a-4b92-8182-8d4589f961a6" path="/var/lib/kubelet/pods/a20027d2-b79a-4b92-8182-8d4589f961a6/volumes"
Jul 05 00:47:16.720013 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:47:16.719980    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-x5szm"]
Jul 05 00:47:18.879299 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:47:18.879268    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-reuse-5b7734-openshift-default-5d48bfbc59-hl86n"]
Jul 05 00:47:18.879760 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:47:18.879478    2563 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-ingress/e2e-reuse-5b7734-openshift-default-5d48bfbc59-hl86n" podUID="decaaef4-8799-4958-ac74-1c0562838d9c" containerName="istio-proxy" containerID="cri-o://a457675cfef4fec259667387fbe6ef7b684623ccd1452f1ccfd188823a9a7841" gracePeriod=30
Jul 05 00:47:20.010999 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:47:20.010967    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-a-4e7441/test-model-4e7441-kserve-695f7d9fff-gv7fq"]
Jul 05 00:47:20.887298 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:47:20.887267    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-b-11db4a/test-model-11db4a-kserve-5d6c4f5fd7-t827w"]
Jul 05 00:47:21.699349 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:47:21.699318    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-api-key-cleanup-e2e-inf-a-4e7441-29720205-hqggk"]
Jul 05 00:47:21.704910 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:47:21.704887    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-api-key-cleanup-e2e-inf-a-4e7441-29720205-hqggk"]
Jul 05 00:47:22.126066 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:47:22.126031    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e72b157a-2968-4b65-9709-d077e8293eaa" path="/var/lib/kubelet/pods/e72b157a-2968-4b65-9709-d077e8293eaa/volumes"
Jul 05 00:47:24.124102 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:47:24.124080    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-reuse-5b7734-openshift-default-5d48bfbc59-hl86n"
Jul 05 00:47:24.216961 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:47:24.216884    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/decaaef4-8799-4958-ac74-1c0562838d9c-workload-certs\") pod \"decaaef4-8799-4958-ac74-1c0562838d9c\" (UID: \"decaaef4-8799-4958-ac74-1c0562838d9c\") "
Jul 05 00:47:24.216961 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:47:24.216940    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/decaaef4-8799-4958-ac74-1c0562838d9c-istio-envoy\") pod \"decaaef4-8799-4958-ac74-1c0562838d9c\" (UID: \"decaaef4-8799-4958-ac74-1c0562838d9c\") "
Jul 05 00:47:24.217181 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:47:24.216978    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/decaaef4-8799-4958-ac74-1c0562838d9c-credential-socket\") pod \"decaaef4-8799-4958-ac74-1c0562838d9c\" (UID: \"decaaef4-8799-4958-ac74-1c0562838d9c\") "
Jul 05 00:47:24.217181 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:47:24.217004    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-rhprg\" (UniqueName: \"kubernetes.io/projected/decaaef4-8799-4958-ac74-1c0562838d9c-kube-api-access-rhprg\") pod \"decaaef4-8799-4958-ac74-1c0562838d9c\" (UID: \"decaaef4-8799-4958-ac74-1c0562838d9c\") "
Jul 05 00:47:24.217181 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:47:24.217024    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/decaaef4-8799-4958-ac74-1c0562838d9c-istio-token\") pod \"decaaef4-8799-4958-ac74-1c0562838d9c\" (UID: \"decaaef4-8799-4958-ac74-1c0562838d9c\") "
Jul 05 00:47:24.217181 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:47:24.217041    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/decaaef4-8799-4958-ac74-1c0562838d9c-istio-podinfo\") pod \"decaaef4-8799-4958-ac74-1c0562838d9c\" (UID: \"decaaef4-8799-4958-ac74-1c0562838d9c\") "
Jul 05 00:47:24.217181 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:47:24.217125    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/decaaef4-8799-4958-ac74-1c0562838d9c-istio-data\") pod \"decaaef4-8799-4958-ac74-1c0562838d9c\" (UID: \"decaaef4-8799-4958-ac74-1c0562838d9c\") "
Jul 05 00:47:24.217181 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:47:24.217157    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/decaaef4-8799-4958-ac74-1c0562838d9c-workload-socket\") pod \"decaaef4-8799-4958-ac74-1c0562838d9c\" (UID: \"decaaef4-8799-4958-ac74-1c0562838d9c\") "
Jul 05 00:47:24.217506 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:47:24.217170    2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/decaaef4-8799-4958-ac74-1c0562838d9c-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "decaaef4-8799-4958-ac74-1c0562838d9c" (UID: "decaaef4-8799-4958-ac74-1c0562838d9c"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 05 00:47:24.217506 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:47:24.217191    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/decaaef4-8799-4958-ac74-1c0562838d9c-istiod-ca-cert\") pod \"decaaef4-8799-4958-ac74-1c0562838d9c\" (UID: \"decaaef4-8799-4958-ac74-1c0562838d9c\") "
Jul 05 00:47:24.217613 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:47:24.217509    2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/decaaef4-8799-4958-ac74-1c0562838d9c-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "decaaef4-8799-4958-ac74-1c0562838d9c" (UID: "decaaef4-8799-4958-ac74-1c0562838d9c"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 05 00:47:24.217671 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:47:24.217610    2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/decaaef4-8799-4958-ac74-1c0562838d9c-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "decaaef4-8799-4958-ac74-1c0562838d9c" (UID: "decaaef4-8799-4958-ac74-1c0562838d9c"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 05 00:47:24.217671 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:47:24.217654    2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/decaaef4-8799-4958-ac74-1c0562838d9c-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "decaaef4-8799-4958-ac74-1c0562838d9c" (UID: "decaaef4-8799-4958-ac74-1c0562838d9c"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 05 00:47:24.217917 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:47:24.217893    2563 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/decaaef4-8799-4958-ac74-1c0562838d9c-workload-certs\") on node \"ip-10-0-142-202.ec2.internal\" DevicePath \"\""
Jul 05 00:47:24.218004 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:47:24.217920    2563 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/decaaef4-8799-4958-ac74-1c0562838d9c-credential-socket\") on node \"ip-10-0-142-202.ec2.internal\" DevicePath \"\""
Jul 05 00:47:24.218004 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:47:24.217936    2563 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/decaaef4-8799-4958-ac74-1c0562838d9c-workload-socket\") on node \"ip-10-0-142-202.ec2.internal\" DevicePath \"\""
Jul 05 00:47:24.218004 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:47:24.217949    2563 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/decaaef4-8799-4958-ac74-1c0562838d9c-istiod-ca-cert\") on node \"ip-10-0-142-202.ec2.internal\" DevicePath \"\""
Jul 05 00:47:24.218004 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:47:24.217916    2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/decaaef4-8799-4958-ac74-1c0562838d9c-istio-data" (OuterVolumeSpecName: "istio-data") pod "decaaef4-8799-4958-ac74-1c0562838d9c" (UID: "decaaef4-8799-4958-ac74-1c0562838d9c"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 05 00:47:24.219506 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:47:24.219480    2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/decaaef4-8799-4958-ac74-1c0562838d9c-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "decaaef4-8799-4958-ac74-1c0562838d9c" (UID: "decaaef4-8799-4958-ac74-1c0562838d9c"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Jul 05 00:47:24.219610 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:47:24.219544    2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/decaaef4-8799-4958-ac74-1c0562838d9c-kube-api-access-rhprg" (OuterVolumeSpecName: "kube-api-access-rhprg") pod "decaaef4-8799-4958-ac74-1c0562838d9c" (UID: "decaaef4-8799-4958-ac74-1c0562838d9c"). InnerVolumeSpecName "kube-api-access-rhprg". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 05 00:47:24.219610 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:47:24.219599    2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/decaaef4-8799-4958-ac74-1c0562838d9c-istio-token" (OuterVolumeSpecName: "istio-token") pod "decaaef4-8799-4958-ac74-1c0562838d9c" (UID: "decaaef4-8799-4958-ac74-1c0562838d9c"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 05 00:47:24.219687 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:47:24.219628    2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/decaaef4-8799-4958-ac74-1c0562838d9c-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "decaaef4-8799-4958-ac74-1c0562838d9c" (UID: "decaaef4-8799-4958-ac74-1c0562838d9c"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 05 00:47:24.319209 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:47:24.319177    2563 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/decaaef4-8799-4958-ac74-1c0562838d9c-istio-envoy\") on node \"ip-10-0-142-202.ec2.internal\" DevicePath \"\""
Jul 05 00:47:24.319209 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:47:24.319205    2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-rhprg\" (UniqueName: \"kubernetes.io/projected/decaaef4-8799-4958-ac74-1c0562838d9c-kube-api-access-rhprg\") on node \"ip-10-0-142-202.ec2.internal\" DevicePath \"\""
Jul 05 00:47:24.319428 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:47:24.319214    2563 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/decaaef4-8799-4958-ac74-1c0562838d9c-istio-token\") on node \"ip-10-0-142-202.ec2.internal\" DevicePath \"\""
Jul 05 00:47:24.319428 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:47:24.319265    2563 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/decaaef4-8799-4958-ac74-1c0562838d9c-istio-podinfo\") on node \"ip-10-0-142-202.ec2.internal\" DevicePath \"\""
Jul 05 00:47:24.319428 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:47:24.319273    2563 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/decaaef4-8799-4958-ac74-1c0562838d9c-istio-data\") on node \"ip-10-0-142-202.ec2.internal\" DevicePath \"\""
Jul 05 00:47:24.789944 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:47:24.789907    2563 generic.go:356] "Generic (PLEG): container finished" podID="decaaef4-8799-4958-ac74-1c0562838d9c" containerID="a457675cfef4fec259667387fbe6ef7b684623ccd1452f1ccfd188823a9a7841" exitCode=0
Jul 05 00:47:24.790133 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:47:24.789950    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-reuse-5b7734-openshift-default-5d48bfbc59-hl86n"
Jul 05 00:47:24.790133 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:47:24.789984    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-reuse-5b7734-openshift-default-5d48bfbc59-hl86n" event={"ID":"decaaef4-8799-4958-ac74-1c0562838d9c","Type":"ContainerDied","Data":"a457675cfef4fec259667387fbe6ef7b684623ccd1452f1ccfd188823a9a7841"}
Jul 05 00:47:24.790133 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:47:24.790039    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-reuse-5b7734-openshift-default-5d48bfbc59-hl86n" event={"ID":"decaaef4-8799-4958-ac74-1c0562838d9c","Type":"ContainerDied","Data":"74e1588212288bf3b7a8953640244a78e2d25999c7c405b1ae010d375bf4bc56"}
Jul 05 00:47:24.790133 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:47:24.790062    2563 scope.go:117] "RemoveContainer" containerID="a457675cfef4fec259667387fbe6ef7b684623ccd1452f1ccfd188823a9a7841"
Jul 05 00:47:24.805872 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:47:24.805850    2563 scope.go:117] "RemoveContainer" containerID="a3fc2b2f98f14dff3e4b48a0ebb9637176804c45f64af3de6bd802ea3e1b3f5d"
Jul 05 00:47:24.813639 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:47:24.813616    2563 scope.go:117] "RemoveContainer" containerID="a457675cfef4fec259667387fbe6ef7b684623ccd1452f1ccfd188823a9a7841"
Jul 05 00:47:24.813916 ip-10-0-142-202 kubenswrapper[2563]: E0705 00:47:24.813893    2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a457675cfef4fec259667387fbe6ef7b684623ccd1452f1ccfd188823a9a7841\": container with ID starting with a457675cfef4fec259667387fbe6ef7b684623ccd1452f1ccfd188823a9a7841 not found: ID does not exist" containerID="a457675cfef4fec259667387fbe6ef7b684623ccd1452f1ccfd188823a9a7841"
Jul 05 00:47:24.813973 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:47:24.813925    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a457675cfef4fec259667387fbe6ef7b684623ccd1452f1ccfd188823a9a7841"} err="failed to get container status \"a457675cfef4fec259667387fbe6ef7b684623ccd1452f1ccfd188823a9a7841\": rpc error: code = NotFound desc = could not find container \"a457675cfef4fec259667387fbe6ef7b684623ccd1452f1ccfd188823a9a7841\": container with ID starting with a457675cfef4fec259667387fbe6ef7b684623ccd1452f1ccfd188823a9a7841 not found: ID does not exist"
Jul 05 00:47:24.813973 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:47:24.813946    2563 scope.go:117] "RemoveContainer" containerID="a3fc2b2f98f14dff3e4b48a0ebb9637176804c45f64af3de6bd802ea3e1b3f5d"
Jul 05 00:47:24.814169 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:47:24.814149    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-reuse-5b7734-openshift-default-5d48bfbc59-hl86n"]
Jul 05 00:47:24.814268 ip-10-0-142-202 kubenswrapper[2563]: E0705 00:47:24.814232    2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a3fc2b2f98f14dff3e4b48a0ebb9637176804c45f64af3de6bd802ea3e1b3f5d\": container with ID starting with a3fc2b2f98f14dff3e4b48a0ebb9637176804c45f64af3de6bd802ea3e1b3f5d not found: ID does not exist" containerID="a3fc2b2f98f14dff3e4b48a0ebb9637176804c45f64af3de6bd802ea3e1b3f5d"
Jul 05 00:47:24.814268 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:47:24.814256    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a3fc2b2f98f14dff3e4b48a0ebb9637176804c45f64af3de6bd802ea3e1b3f5d"} err="failed to get container status \"a3fc2b2f98f14dff3e4b48a0ebb9637176804c45f64af3de6bd802ea3e1b3f5d\": rpc error: code = NotFound desc = could not find container \"a3fc2b2f98f14dff3e4b48a0ebb9637176804c45f64af3de6bd802ea3e1b3f5d\": container with ID starting with a3fc2b2f98f14dff3e4b48a0ebb9637176804c45f64af3de6bd802ea3e1b3f5d not found: ID does not exist"
Jul 05 00:47:24.819698 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:47:24.819678    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-reuse-5b7734-openshift-default-5d48bfbc59-hl86n"]
Jul 05 00:47:26.126054 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:47:26.126023    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="decaaef4-8799-4958-ac74-1c0562838d9c" path="/var/lib/kubelet/pods/decaaef4-8799-4958-ac74-1c0562838d9c/volumes"
Jul 05 00:47:32.821355 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:47:32.821323    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-x5szm"]
Jul 05 00:47:35.031184 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:47:35.031140    2563 kuberuntime_container.go:872] "Killing container with a grace period" pod="ai-tenant-e2e-inf-a-4e7441/test-model-4e7441-kserve-695f7d9fff-gv7fq" podUID="bf8ed536-a450-45a6-9758-df18c69a4de1" containerName="main" containerID="cri-o://63eccb2a1baf95b3a0d294cf3a4f1c045185617a8aec168a171d9f64c441816b" gracePeriod=45
Jul 05 00:47:35.277240 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:47:35.277204    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-4e7441/test-model-4e7441-kserve-695f7d9fff-gv7fq"
Jul 05 00:47:35.414693 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:47:35.414590    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/bf8ed536-a450-45a6-9758-df18c69a4de1-dshm\") pod \"bf8ed536-a450-45a6-9758-df18c69a4de1\" (UID: \"bf8ed536-a450-45a6-9758-df18c69a4de1\") "
Jul 05 00:47:35.414693 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:47:35.414640    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-2nf6j\" (UniqueName: \"kubernetes.io/projected/bf8ed536-a450-45a6-9758-df18c69a4de1-kube-api-access-2nf6j\") pod \"bf8ed536-a450-45a6-9758-df18c69a4de1\" (UID: \"bf8ed536-a450-45a6-9758-df18c69a4de1\") "
Jul 05 00:47:35.414693 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:47:35.414665    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/bf8ed536-a450-45a6-9758-df18c69a4de1-tls-certs\") pod \"bf8ed536-a450-45a6-9758-df18c69a4de1\" (UID: \"bf8ed536-a450-45a6-9758-df18c69a4de1\") "
Jul 05 00:47:35.414693 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:47:35.414697    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/bf8ed536-a450-45a6-9758-df18c69a4de1-tmp-dir\") pod \"bf8ed536-a450-45a6-9758-df18c69a4de1\" (UID: \"bf8ed536-a450-45a6-9758-df18c69a4de1\") "
Jul 05 00:47:35.415049 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:47:35.414719    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/bf8ed536-a450-45a6-9758-df18c69a4de1-model-cache\") pod \"bf8ed536-a450-45a6-9758-df18c69a4de1\" (UID: \"bf8ed536-a450-45a6-9758-df18c69a4de1\") "
Jul 05 00:47:35.415049 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:47:35.414810    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/bf8ed536-a450-45a6-9758-df18c69a4de1-home\") pod \"bf8ed536-a450-45a6-9758-df18c69a4de1\" (UID: \"bf8ed536-a450-45a6-9758-df18c69a4de1\") "
Jul 05 00:47:35.415049 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:47:35.415003    2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/bf8ed536-a450-45a6-9758-df18c69a4de1-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "bf8ed536-a450-45a6-9758-df18c69a4de1" (UID: "bf8ed536-a450-45a6-9758-df18c69a4de1"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 05 00:47:35.415194 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:47:35.415048    2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/bf8ed536-a450-45a6-9758-df18c69a4de1-model-cache" (OuterVolumeSpecName: "model-cache") pod "bf8ed536-a450-45a6-9758-df18c69a4de1" (UID: "bf8ed536-a450-45a6-9758-df18c69a4de1"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 05 00:47:35.415194 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:47:35.415114    2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/bf8ed536-a450-45a6-9758-df18c69a4de1-home" (OuterVolumeSpecName: "home") pod "bf8ed536-a450-45a6-9758-df18c69a4de1" (UID: "bf8ed536-a450-45a6-9758-df18c69a4de1"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 05 00:47:35.415194 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:47:35.415127    2563 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/bf8ed536-a450-45a6-9758-df18c69a4de1-tmp-dir\") on node \"ip-10-0-142-202.ec2.internal\" DevicePath \"\""
Jul 05 00:47:35.416842 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:47:35.416819    2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/bf8ed536-a450-45a6-9758-df18c69a4de1-kube-api-access-2nf6j" (OuterVolumeSpecName: "kube-api-access-2nf6j") pod "bf8ed536-a450-45a6-9758-df18c69a4de1" (UID: "bf8ed536-a450-45a6-9758-df18c69a4de1"). InnerVolumeSpecName "kube-api-access-2nf6j". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 05 00:47:35.416842 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:47:35.416833    2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/bf8ed536-a450-45a6-9758-df18c69a4de1-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "bf8ed536-a450-45a6-9758-df18c69a4de1" (UID: "bf8ed536-a450-45a6-9758-df18c69a4de1"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 05 00:47:35.416842 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:47:35.416821    2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/bf8ed536-a450-45a6-9758-df18c69a4de1-dshm" (OuterVolumeSpecName: "dshm") pod "bf8ed536-a450-45a6-9758-df18c69a4de1" (UID: "bf8ed536-a450-45a6-9758-df18c69a4de1"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 05 00:47:35.515993 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:47:35.515963    2563 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/bf8ed536-a450-45a6-9758-df18c69a4de1-tls-certs\") on node \"ip-10-0-142-202.ec2.internal\" DevicePath \"\""
Jul 05 00:47:35.515993 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:47:35.515988    2563 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/bf8ed536-a450-45a6-9758-df18c69a4de1-model-cache\") on node \"ip-10-0-142-202.ec2.internal\" DevicePath \"\""
Jul 05 00:47:35.515993 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:47:35.515997    2563 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/bf8ed536-a450-45a6-9758-df18c69a4de1-home\") on node \"ip-10-0-142-202.ec2.internal\" DevicePath \"\""
Jul 05 00:47:35.515993 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:47:35.516004    2563 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/bf8ed536-a450-45a6-9758-df18c69a4de1-dshm\") on node \"ip-10-0-142-202.ec2.internal\" DevicePath \"\""
Jul 05 00:47:35.516340 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:47:35.516013    2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-2nf6j\" (UniqueName: \"kubernetes.io/projected/bf8ed536-a450-45a6-9758-df18c69a4de1-kube-api-access-2nf6j\") on node \"ip-10-0-142-202.ec2.internal\" DevicePath \"\""
Jul 05 00:47:35.843899 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:47:35.843863    2563 generic.go:356] "Generic (PLEG): container finished" podID="bf8ed536-a450-45a6-9758-df18c69a4de1" containerID="63eccb2a1baf95b3a0d294cf3a4f1c045185617a8aec168a171d9f64c441816b" exitCode=0
Jul 05 00:47:35.844081 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:47:35.843906    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-4e7441/test-model-4e7441-kserve-695f7d9fff-gv7fq"
Jul 05 00:47:35.844081 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:47:35.843924    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-4e7441/test-model-4e7441-kserve-695f7d9fff-gv7fq" event={"ID":"bf8ed536-a450-45a6-9758-df18c69a4de1","Type":"ContainerDied","Data":"63eccb2a1baf95b3a0d294cf3a4f1c045185617a8aec168a171d9f64c441816b"}
Jul 05 00:47:35.844081 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:47:35.843959    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-4e7441/test-model-4e7441-kserve-695f7d9fff-gv7fq" event={"ID":"bf8ed536-a450-45a6-9758-df18c69a4de1","Type":"ContainerDied","Data":"1dd5806229129f36d435b9fbf776b48b9fae784d1c5e653a4add9f159115d758"}
Jul 05 00:47:35.844081 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:47:35.843978    2563 scope.go:117] "RemoveContainer" containerID="63eccb2a1baf95b3a0d294cf3a4f1c045185617a8aec168a171d9f64c441816b"
Jul 05 00:47:35.853063 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:47:35.853046    2563 scope.go:117] "RemoveContainer" containerID="63eccb2a1baf95b3a0d294cf3a4f1c045185617a8aec168a171d9f64c441816b"
Jul 05 00:47:35.853317 ip-10-0-142-202 kubenswrapper[2563]: E0705 00:47:35.853297    2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"63eccb2a1baf95b3a0d294cf3a4f1c045185617a8aec168a171d9f64c441816b\": container with ID starting with 63eccb2a1baf95b3a0d294cf3a4f1c045185617a8aec168a171d9f64c441816b not found: ID does not exist" containerID="63eccb2a1baf95b3a0d294cf3a4f1c045185617a8aec168a171d9f64c441816b"
Jul 05 00:47:35.853428 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:47:35.853321    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"63eccb2a1baf95b3a0d294cf3a4f1c045185617a8aec168a171d9f64c441816b"} err="failed to get container status \"63eccb2a1baf95b3a0d294cf3a4f1c045185617a8aec168a171d9f64c441816b\": rpc error: code = NotFound desc = could not find container \"63eccb2a1baf95b3a0d294cf3a4f1c045185617a8aec168a171d9f64c441816b\": container with ID starting with 63eccb2a1baf95b3a0d294cf3a4f1c045185617a8aec168a171d9f64c441816b not found: ID does not exist"
Jul 05 00:47:35.862755 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:47:35.862733    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-a-4e7441/test-model-4e7441-kserve-695f7d9fff-gv7fq"]
Jul 05 00:47:35.866414 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:47:35.866395    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-inf-a-4e7441/test-model-4e7441-kserve-695f7d9fff-gv7fq"]
Jul 05 00:47:35.907621 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:47:35.907589    2563 kuberuntime_container.go:872] "Killing container with a grace period" pod="ai-tenant-e2e-inf-b-11db4a/test-model-11db4a-kserve-5d6c4f5fd7-t827w" podUID="6715015b-de5b-456d-bf58-e07940749636" containerName="main" containerID="cri-o://12b00f182bd80e24918c7070ba3d3e37d8366acf66429e4435378d9be42af6d5" gracePeriod=45
Jul 05 00:47:36.127854 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:47:36.127828    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="bf8ed536-a450-45a6-9758-df18c69a4de1" path="/var/lib/kubelet/pods/bf8ed536-a450-45a6-9758-df18c69a4de1/volumes"
Jul 05 00:47:36.143720 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:47:36.143693    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-11db4a/test-model-11db4a-kserve-5d6c4f5fd7-t827w"
Jul 05 00:47:36.323567 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:47:36.323534    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/6715015b-de5b-456d-bf58-e07940749636-home\") pod \"6715015b-de5b-456d-bf58-e07940749636\" (UID: \"6715015b-de5b-456d-bf58-e07940749636\") "
Jul 05 00:47:36.323717 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:47:36.323585    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-ghn6p\" (UniqueName: \"kubernetes.io/projected/6715015b-de5b-456d-bf58-e07940749636-kube-api-access-ghn6p\") pod \"6715015b-de5b-456d-bf58-e07940749636\" (UID: \"6715015b-de5b-456d-bf58-e07940749636\") "
Jul 05 00:47:36.323717 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:47:36.323620    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/6715015b-de5b-456d-bf58-e07940749636-tls-certs\") pod \"6715015b-de5b-456d-bf58-e07940749636\" (UID: \"6715015b-de5b-456d-bf58-e07940749636\") "
Jul 05 00:47:36.323717 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:47:36.323649    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/6715015b-de5b-456d-bf58-e07940749636-tmp-dir\") pod \"6715015b-de5b-456d-bf58-e07940749636\" (UID: \"6715015b-de5b-456d-bf58-e07940749636\") "
Jul 05 00:47:36.323717 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:47:36.323704    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/6715015b-de5b-456d-bf58-e07940749636-dshm\") pod \"6715015b-de5b-456d-bf58-e07940749636\" (UID: \"6715015b-de5b-456d-bf58-e07940749636\") "
Jul 05 00:47:36.323998 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:47:36.323759    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/6715015b-de5b-456d-bf58-e07940749636-model-cache\") pod \"6715015b-de5b-456d-bf58-e07940749636\" (UID: \"6715015b-de5b-456d-bf58-e07940749636\") "
Jul 05 00:47:36.323998 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:47:36.323847    2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6715015b-de5b-456d-bf58-e07940749636-home" (OuterVolumeSpecName: "home") pod "6715015b-de5b-456d-bf58-e07940749636" (UID: "6715015b-de5b-456d-bf58-e07940749636"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 05 00:47:36.323998 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:47:36.323897    2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6715015b-de5b-456d-bf58-e07940749636-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "6715015b-de5b-456d-bf58-e07940749636" (UID: "6715015b-de5b-456d-bf58-e07940749636"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 05 00:47:36.324150 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:47:36.324054    2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6715015b-de5b-456d-bf58-e07940749636-model-cache" (OuterVolumeSpecName: "model-cache") pod "6715015b-de5b-456d-bf58-e07940749636" (UID: "6715015b-de5b-456d-bf58-e07940749636"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 05 00:47:36.324150 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:47:36.324086    2563 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/6715015b-de5b-456d-bf58-e07940749636-home\") on node \"ip-10-0-142-202.ec2.internal\" DevicePath \"\""
Jul 05 00:47:36.324150 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:47:36.324100    2563 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/6715015b-de5b-456d-bf58-e07940749636-tmp-dir\") on node \"ip-10-0-142-202.ec2.internal\" DevicePath \"\""
Jul 05 00:47:36.325875 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:47:36.325843    2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6715015b-de5b-456d-bf58-e07940749636-dshm" (OuterVolumeSpecName: "dshm") pod "6715015b-de5b-456d-bf58-e07940749636" (UID: "6715015b-de5b-456d-bf58-e07940749636"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 05 00:47:36.325983 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:47:36.325919    2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6715015b-de5b-456d-bf58-e07940749636-kube-api-access-ghn6p" (OuterVolumeSpecName: "kube-api-access-ghn6p") pod "6715015b-de5b-456d-bf58-e07940749636" (UID: "6715015b-de5b-456d-bf58-e07940749636"). InnerVolumeSpecName "kube-api-access-ghn6p". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 05 00:47:36.325983 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:47:36.325922    2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/6715015b-de5b-456d-bf58-e07940749636-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "6715015b-de5b-456d-bf58-e07940749636" (UID: "6715015b-de5b-456d-bf58-e07940749636"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 05 00:47:36.424870 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:47:36.424785    2563 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/6715015b-de5b-456d-bf58-e07940749636-dshm\") on node \"ip-10-0-142-202.ec2.internal\" DevicePath \"\""
Jul 05 00:47:36.424870 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:47:36.424817    2563 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/6715015b-de5b-456d-bf58-e07940749636-model-cache\") on node \"ip-10-0-142-202.ec2.internal\" DevicePath \"\""
Jul 05 00:47:36.424870 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:47:36.424833    2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-ghn6p\" (UniqueName: \"kubernetes.io/projected/6715015b-de5b-456d-bf58-e07940749636-kube-api-access-ghn6p\") on node \"ip-10-0-142-202.ec2.internal\" DevicePath \"\""
Jul 05 00:47:36.424870 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:47:36.424847    2563 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/6715015b-de5b-456d-bf58-e07940749636-tls-certs\") on node \"ip-10-0-142-202.ec2.internal\" DevicePath \"\""
Jul 05 00:47:36.851317 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:47:36.851281    2563 generic.go:356] "Generic (PLEG): container finished" podID="6715015b-de5b-456d-bf58-e07940749636" containerID="12b00f182bd80e24918c7070ba3d3e37d8366acf66429e4435378d9be42af6d5" exitCode=0
Jul 05 00:47:36.851500 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:47:36.851323    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-11db4a/test-model-11db4a-kserve-5d6c4f5fd7-t827w"
Jul 05 00:47:36.851500 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:47:36.851330    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-11db4a/test-model-11db4a-kserve-5d6c4f5fd7-t827w" event={"ID":"6715015b-de5b-456d-bf58-e07940749636","Type":"ContainerDied","Data":"12b00f182bd80e24918c7070ba3d3e37d8366acf66429e4435378d9be42af6d5"}
Jul 05 00:47:36.851500 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:47:36.851370    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-11db4a/test-model-11db4a-kserve-5d6c4f5fd7-t827w" event={"ID":"6715015b-de5b-456d-bf58-e07940749636","Type":"ContainerDied","Data":"28ce677af8a3383c30eaf5ef62c84487add1349967cb20c37f288068ca0d744a"}
Jul 05 00:47:36.851500 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:47:36.851393    2563 scope.go:117] "RemoveContainer" containerID="12b00f182bd80e24918c7070ba3d3e37d8366acf66429e4435378d9be42af6d5"
Jul 05 00:47:36.861597 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:47:36.861576    2563 scope.go:117] "RemoveContainer" containerID="12b00f182bd80e24918c7070ba3d3e37d8366acf66429e4435378d9be42af6d5"
Jul 05 00:47:36.861822 ip-10-0-142-202 kubenswrapper[2563]: E0705 00:47:36.861805    2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"12b00f182bd80e24918c7070ba3d3e37d8366acf66429e4435378d9be42af6d5\": container with ID starting with 12b00f182bd80e24918c7070ba3d3e37d8366acf66429e4435378d9be42af6d5 not found: ID does not exist" containerID="12b00f182bd80e24918c7070ba3d3e37d8366acf66429e4435378d9be42af6d5"
Jul 05 00:47:36.861869 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:47:36.861830    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"12b00f182bd80e24918c7070ba3d3e37d8366acf66429e4435378d9be42af6d5"} err="failed to get container status \"12b00f182bd80e24918c7070ba3d3e37d8366acf66429e4435378d9be42af6d5\": rpc error: code = NotFound desc = could not find container \"12b00f182bd80e24918c7070ba3d3e37d8366acf66429e4435378d9be42af6d5\": container with ID starting with 12b00f182bd80e24918c7070ba3d3e37d8366acf66429e4435378d9be42af6d5 not found: ID does not exist"
Jul 05 00:47:36.870873 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:47:36.870853    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-b-11db4a/test-model-11db4a-kserve-5d6c4f5fd7-t827w"]
Jul 05 00:47:36.874051 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:47:36.874032    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-inf-b-11db4a/test-model-11db4a-kserve-5d6c4f5fd7-t827w"]
Jul 05 00:47:38.126491 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:47:38.126460    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="6715015b-de5b-456d-bf58-e07940749636" path="/var/lib/kubelet/pods/6715015b-de5b-456d-bf58-e07940749636/volumes"
Jul 05 00:48:09.524487 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:48:09.524404    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-x5szm"]
Jul 05 00:48:11.118981 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:48:11.118952    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-api-a-174d7d-openshift-default-859476d8d9-cjgmw"]
Jul 05 00:48:11.119373 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:48:11.119304    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="decaaef4-8799-4958-ac74-1c0562838d9c" containerName="istio-proxy"
Jul 05 00:48:11.119373 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:48:11.119317    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="decaaef4-8799-4958-ac74-1c0562838d9c" containerName="istio-proxy"
Jul 05 00:48:11.119373 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:48:11.119326    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="bf8ed536-a450-45a6-9758-df18c69a4de1" containerName="main"
Jul 05 00:48:11.119373 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:48:11.119331    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="bf8ed536-a450-45a6-9758-df18c69a4de1" containerName="main"
Jul 05 00:48:11.119373 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:48:11.119343    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a20027d2-b79a-4b92-8182-8d4589f961a6" containerName="main"
Jul 05 00:48:11.119373 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:48:11.119349    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="a20027d2-b79a-4b92-8182-8d4589f961a6" containerName="main"
Jul 05 00:48:11.119373 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:48:11.119356    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="decaaef4-8799-4958-ac74-1c0562838d9c" containerName="istio-proxy"
Jul 05 00:48:11.119373 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:48:11.119362    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="decaaef4-8799-4958-ac74-1c0562838d9c" containerName="istio-proxy"
Jul 05 00:48:11.119373 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:48:11.119376    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6715015b-de5b-456d-bf58-e07940749636" containerName="main"
Jul 05 00:48:11.119682 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:48:11.119382    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="6715015b-de5b-456d-bf58-e07940749636" containerName="main"
Jul 05 00:48:11.119682 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:48:11.119426    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="a20027d2-b79a-4b92-8182-8d4589f961a6" containerName="main"
Jul 05 00:48:11.119682 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:48:11.119435    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="decaaef4-8799-4958-ac74-1c0562838d9c" containerName="istio-proxy"
Jul 05 00:48:11.119682 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:48:11.119442    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="6715015b-de5b-456d-bf58-e07940749636" containerName="main"
Jul 05 00:48:11.119682 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:48:11.119447    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="decaaef4-8799-4958-ac74-1c0562838d9c" containerName="istio-proxy"
Jul 05 00:48:11.119682 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:48:11.119454    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="bf8ed536-a450-45a6-9758-df18c69a4de1" containerName="main"
Jul 05 00:48:11.122335 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:48:11.122316    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-174d7d-openshift-default-859476d8d9-cjgmw"
Jul 05 00:48:11.124126 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:48:11.124107    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-api-a-174d7d-openshift-default-dockercfg-w2nfm\""
Jul 05 00:48:11.131875 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:48:11.131850    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-a-174d7d-openshift-default-859476d8d9-cjgmw"]
Jul 05 00:48:11.211792 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:48:11.211758    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a3819c3a-ce70-4665-a99f-f61e9999774a-credential-socket\") pod \"e2e-api-a-174d7d-openshift-default-859476d8d9-cjgmw\" (UID: \"a3819c3a-ce70-4665-a99f-f61e9999774a\") " pod="openshift-ingress/e2e-api-a-174d7d-openshift-default-859476d8d9-cjgmw"
Jul 05 00:48:11.211792 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:48:11.211803    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-n2zx5\" (UniqueName: \"kubernetes.io/projected/a3819c3a-ce70-4665-a99f-f61e9999774a-kube-api-access-n2zx5\") pod \"e2e-api-a-174d7d-openshift-default-859476d8d9-cjgmw\" (UID: \"a3819c3a-ce70-4665-a99f-f61e9999774a\") " pod="openshift-ingress/e2e-api-a-174d7d-openshift-default-859476d8d9-cjgmw"
Jul 05 00:48:11.212019 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:48:11.211827    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a3819c3a-ce70-4665-a99f-f61e9999774a-workload-certs\") pod \"e2e-api-a-174d7d-openshift-default-859476d8d9-cjgmw\" (UID: \"a3819c3a-ce70-4665-a99f-f61e9999774a\") " pod="openshift-ingress/e2e-api-a-174d7d-openshift-default-859476d8d9-cjgmw"
Jul 05 00:48:11.212019 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:48:11.211851    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a3819c3a-ce70-4665-a99f-f61e9999774a-istio-podinfo\") pod \"e2e-api-a-174d7d-openshift-default-859476d8d9-cjgmw\" (UID: \"a3819c3a-ce70-4665-a99f-f61e9999774a\") " pod="openshift-ingress/e2e-api-a-174d7d-openshift-default-859476d8d9-cjgmw"
Jul 05 00:48:11.212019 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:48:11.211926    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a3819c3a-ce70-4665-a99f-f61e9999774a-istio-envoy\") pod \"e2e-api-a-174d7d-openshift-default-859476d8d9-cjgmw\" (UID: \"a3819c3a-ce70-4665-a99f-f61e9999774a\") " pod="openshift-ingress/e2e-api-a-174d7d-openshift-default-859476d8d9-cjgmw"
Jul 05 00:48:11.212019 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:48:11.211960    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a3819c3a-ce70-4665-a99f-f61e9999774a-istio-data\") pod \"e2e-api-a-174d7d-openshift-default-859476d8d9-cjgmw\" (UID: \"a3819c3a-ce70-4665-a99f-f61e9999774a\") " pod="openshift-ingress/e2e-api-a-174d7d-openshift-default-859476d8d9-cjgmw"
Jul 05 00:48:11.212019 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:48:11.211996    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a3819c3a-ce70-4665-a99f-f61e9999774a-istiod-ca-cert\") pod \"e2e-api-a-174d7d-openshift-default-859476d8d9-cjgmw\" (UID: \"a3819c3a-ce70-4665-a99f-f61e9999774a\") " pod="openshift-ingress/e2e-api-a-174d7d-openshift-default-859476d8d9-cjgmw"
Jul 05 00:48:11.212019 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:48:11.212018    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a3819c3a-ce70-4665-a99f-f61e9999774a-istio-token\") pod \"e2e-api-a-174d7d-openshift-default-859476d8d9-cjgmw\" (UID: \"a3819c3a-ce70-4665-a99f-f61e9999774a\") " pod="openshift-ingress/e2e-api-a-174d7d-openshift-default-859476d8d9-cjgmw"
Jul 05 00:48:11.212207 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:48:11.212049    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a3819c3a-ce70-4665-a99f-f61e9999774a-workload-socket\") pod \"e2e-api-a-174d7d-openshift-default-859476d8d9-cjgmw\" (UID: \"a3819c3a-ce70-4665-a99f-f61e9999774a\") " pod="openshift-ingress/e2e-api-a-174d7d-openshift-default-859476d8d9-cjgmw"
Jul 05 00:48:11.312785 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:48:11.312753    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a3819c3a-ce70-4665-a99f-f61e9999774a-workload-certs\") pod \"e2e-api-a-174d7d-openshift-default-859476d8d9-cjgmw\" (UID: \"a3819c3a-ce70-4665-a99f-f61e9999774a\") " pod="openshift-ingress/e2e-api-a-174d7d-openshift-default-859476d8d9-cjgmw"
Jul 05 00:48:11.312969 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:48:11.312792    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a3819c3a-ce70-4665-a99f-f61e9999774a-istio-podinfo\") pod \"e2e-api-a-174d7d-openshift-default-859476d8d9-cjgmw\" (UID: \"a3819c3a-ce70-4665-a99f-f61e9999774a\") " pod="openshift-ingress/e2e-api-a-174d7d-openshift-default-859476d8d9-cjgmw"
Jul 05 00:48:11.312969 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:48:11.312838    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a3819c3a-ce70-4665-a99f-f61e9999774a-istio-envoy\") pod \"e2e-api-a-174d7d-openshift-default-859476d8d9-cjgmw\" (UID: \"a3819c3a-ce70-4665-a99f-f61e9999774a\") " pod="openshift-ingress/e2e-api-a-174d7d-openshift-default-859476d8d9-cjgmw"
Jul 05 00:48:11.312969 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:48:11.312861    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a3819c3a-ce70-4665-a99f-f61e9999774a-istio-data\") pod \"e2e-api-a-174d7d-openshift-default-859476d8d9-cjgmw\" (UID: \"a3819c3a-ce70-4665-a99f-f61e9999774a\") " pod="openshift-ingress/e2e-api-a-174d7d-openshift-default-859476d8d9-cjgmw"
Jul 05 00:48:11.312969 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:48:11.312903    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a3819c3a-ce70-4665-a99f-f61e9999774a-istiod-ca-cert\") pod \"e2e-api-a-174d7d-openshift-default-859476d8d9-cjgmw\" (UID: \"a3819c3a-ce70-4665-a99f-f61e9999774a\") " pod="openshift-ingress/e2e-api-a-174d7d-openshift-default-859476d8d9-cjgmw"
Jul 05 00:48:11.312969 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:48:11.312932    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a3819c3a-ce70-4665-a99f-f61e9999774a-istio-token\") pod \"e2e-api-a-174d7d-openshift-default-859476d8d9-cjgmw\" (UID: \"a3819c3a-ce70-4665-a99f-f61e9999774a\") " pod="openshift-ingress/e2e-api-a-174d7d-openshift-default-859476d8d9-cjgmw"
Jul 05 00:48:11.312969 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:48:11.312959    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a3819c3a-ce70-4665-a99f-f61e9999774a-workload-socket\") pod \"e2e-api-a-174d7d-openshift-default-859476d8d9-cjgmw\" (UID: \"a3819c3a-ce70-4665-a99f-f61e9999774a\") " pod="openshift-ingress/e2e-api-a-174d7d-openshift-default-859476d8d9-cjgmw"
Jul 05 00:48:11.313351 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:48:11.312999    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a3819c3a-ce70-4665-a99f-f61e9999774a-credential-socket\") pod \"e2e-api-a-174d7d-openshift-default-859476d8d9-cjgmw\" (UID: \"a3819c3a-ce70-4665-a99f-f61e9999774a\") " pod="openshift-ingress/e2e-api-a-174d7d-openshift-default-859476d8d9-cjgmw"
Jul 05 00:48:11.313351 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:48:11.313051    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-n2zx5\" (UniqueName: \"kubernetes.io/projected/a3819c3a-ce70-4665-a99f-f61e9999774a-kube-api-access-n2zx5\") pod \"e2e-api-a-174d7d-openshift-default-859476d8d9-cjgmw\" (UID: \"a3819c3a-ce70-4665-a99f-f61e9999774a\") " pod="openshift-ingress/e2e-api-a-174d7d-openshift-default-859476d8d9-cjgmw"
Jul 05 00:48:11.313351 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:48:11.313268    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a3819c3a-ce70-4665-a99f-f61e9999774a-workload-certs\") pod \"e2e-api-a-174d7d-openshift-default-859476d8d9-cjgmw\" (UID: \"a3819c3a-ce70-4665-a99f-f61e9999774a\") " pod="openshift-ingress/e2e-api-a-174d7d-openshift-default-859476d8d9-cjgmw"
Jul 05 00:48:11.313458 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:48:11.313349    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a3819c3a-ce70-4665-a99f-f61e9999774a-istio-data\") pod \"e2e-api-a-174d7d-openshift-default-859476d8d9-cjgmw\" (UID: \"a3819c3a-ce70-4665-a99f-f61e9999774a\") " pod="openshift-ingress/e2e-api-a-174d7d-openshift-default-859476d8d9-cjgmw"
Jul 05 00:48:11.313546 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:48:11.313526    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a3819c3a-ce70-4665-a99f-f61e9999774a-credential-socket\") pod \"e2e-api-a-174d7d-openshift-default-859476d8d9-cjgmw\" (UID: \"a3819c3a-ce70-4665-a99f-f61e9999774a\") " pod="openshift-ingress/e2e-api-a-174d7d-openshift-default-859476d8d9-cjgmw"
Jul 05 00:48:11.313623 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:48:11.313603    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a3819c3a-ce70-4665-a99f-f61e9999774a-workload-socket\") pod \"e2e-api-a-174d7d-openshift-default-859476d8d9-cjgmw\" (UID: \"a3819c3a-ce70-4665-a99f-f61e9999774a\") " pod="openshift-ingress/e2e-api-a-174d7d-openshift-default-859476d8d9-cjgmw"
Jul 05 00:48:11.313716 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:48:11.313699    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a3819c3a-ce70-4665-a99f-f61e9999774a-istiod-ca-cert\") pod \"e2e-api-a-174d7d-openshift-default-859476d8d9-cjgmw\" (UID: \"a3819c3a-ce70-4665-a99f-f61e9999774a\") " pod="openshift-ingress/e2e-api-a-174d7d-openshift-default-859476d8d9-cjgmw"
Jul 05 00:48:11.315235 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:48:11.315207    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a3819c3a-ce70-4665-a99f-f61e9999774a-istio-envoy\") pod \"e2e-api-a-174d7d-openshift-default-859476d8d9-cjgmw\" (UID: \"a3819c3a-ce70-4665-a99f-f61e9999774a\") " pod="openshift-ingress/e2e-api-a-174d7d-openshift-default-859476d8d9-cjgmw"
Jul 05 00:48:11.315374 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:48:11.315359    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a3819c3a-ce70-4665-a99f-f61e9999774a-istio-podinfo\") pod \"e2e-api-a-174d7d-openshift-default-859476d8d9-cjgmw\" (UID: \"a3819c3a-ce70-4665-a99f-f61e9999774a\") " pod="openshift-ingress/e2e-api-a-174d7d-openshift-default-859476d8d9-cjgmw"
Jul 05 00:48:11.319849 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:48:11.319821    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a3819c3a-ce70-4665-a99f-f61e9999774a-istio-token\") pod \"e2e-api-a-174d7d-openshift-default-859476d8d9-cjgmw\" (UID: \"a3819c3a-ce70-4665-a99f-f61e9999774a\") " pod="openshift-ingress/e2e-api-a-174d7d-openshift-default-859476d8d9-cjgmw"
Jul 05 00:48:11.320067 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:48:11.320051    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-n2zx5\" (UniqueName: \"kubernetes.io/projected/a3819c3a-ce70-4665-a99f-f61e9999774a-kube-api-access-n2zx5\") pod \"e2e-api-a-174d7d-openshift-default-859476d8d9-cjgmw\" (UID: \"a3819c3a-ce70-4665-a99f-f61e9999774a\") " pod="openshift-ingress/e2e-api-a-174d7d-openshift-default-859476d8d9-cjgmw"
Jul 05 00:48:11.433292 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:48:11.433200    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-174d7d-openshift-default-859476d8d9-cjgmw"
Jul 05 00:48:11.565960 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:48:11.565927    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-a-174d7d-openshift-default-859476d8d9-cjgmw"]
Jul 05 00:48:11.569390 ip-10-0-142-202 kubenswrapper[2563]: W0705 00:48:11.569365    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda3819c3a_ce70_4665_a99f_f61e9999774a.slice/crio-26633a33a4e23d1c1b44866fb4a7571df329cc6d899000652a488683f5b015db WatchSource:0}: Error finding container 26633a33a4e23d1c1b44866fb4a7571df329cc6d899000652a488683f5b015db: Status 404 returned error can't find the container with id 26633a33a4e23d1c1b44866fb4a7571df329cc6d899000652a488683f5b015db
Jul 05 00:48:11.571672 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:48:11.571643    2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"}
Jul 05 00:48:11.571738 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:48:11.571712    2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"}
Jul 05 00:48:11.571776 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:48:11.571741    2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"}
Jul 05 00:48:12.011623 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:48:12.011591    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-174d7d-openshift-default-859476d8d9-cjgmw" event={"ID":"a3819c3a-ce70-4665-a99f-f61e9999774a","Type":"ContainerStarted","Data":"ee6dd6768a27e8c76f3ee08321b5c58296662177fcc1a63e2213266741aa4b38"}
Jul 05 00:48:12.011797 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:48:12.011663    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-174d7d-openshift-default-859476d8d9-cjgmw" event={"ID":"a3819c3a-ce70-4665-a99f-f61e9999774a","Type":"ContainerStarted","Data":"26633a33a4e23d1c1b44866fb4a7571df329cc6d899000652a488683f5b015db"}
Jul 05 00:48:12.029932 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:48:12.029792    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-api-a-174d7d-openshift-default-859476d8d9-cjgmw" podStartSLOduration=1.029776175 podStartE2EDuration="1.029776175s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-05 00:48:11 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-05 00:48:12.027571182 +0000 UTC m=+2340.369900257" watchObservedRunningTime="2026-07-05 00:48:12.029776175 +0000 UTC m=+2340.372105235"
Jul 05 00:48:12.434298 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:48:12.434253    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-a-174d7d-openshift-default-859476d8d9-cjgmw"
Jul 05 00:48:12.439128 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:48:12.439103    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-a-174d7d-openshift-default-859476d8d9-cjgmw"
Jul 05 00:48:13.016243 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:48:13.016135    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-a-174d7d-openshift-default-859476d8d9-cjgmw"
Jul 05 00:48:13.017194 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:48:13.017171    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-a-174d7d-openshift-default-859476d8d9-cjgmw"
Jul 05 00:48:21.524547 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:48:21.524515    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-x5szm"]
Jul 05 00:48:29.058414 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:48:29.058373    2563 patch_prober.go:28] interesting pod/e2e-api-a-174d7d-openshift-default-859476d8d9-cjgmw container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.133.0.42:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Jul 05 00:48:29.058865 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:48:29.058450    2563 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-api-a-174d7d-openshift-default-859476d8d9-cjgmw" podUID="a3819c3a-ce70-4665-a99f-f61e9999774a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.42:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Jul 05 00:48:30.107849 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:48:30.107819    2563 generic.go:356] "Generic (PLEG): container finished" podID="a3819c3a-ce70-4665-a99f-f61e9999774a" containerID="ee6dd6768a27e8c76f3ee08321b5c58296662177fcc1a63e2213266741aa4b38" exitCode=0
Jul 05 00:48:30.108312 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:48:30.107876    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-174d7d-openshift-default-859476d8d9-cjgmw" event={"ID":"a3819c3a-ce70-4665-a99f-f61e9999774a","Type":"ContainerDied","Data":"ee6dd6768a27e8c76f3ee08321b5c58296662177fcc1a63e2213266741aa4b38"}
Jul 05 00:48:30.108312 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:48:30.108293    2563 scope.go:117] "RemoveContainer" containerID="ee6dd6768a27e8c76f3ee08321b5c58296662177fcc1a63e2213266741aa4b38"
Jul 05 00:48:30.109379 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:48:30.109337    2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"}
Jul 05 00:48:30.109470 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:48:30.109443    2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"}
Jul 05 00:48:30.109509 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:48:30.109490    2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"}
Jul 05 00:48:31.114809 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:48:31.114778    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-174d7d-openshift-default-859476d8d9-cjgmw" event={"ID":"a3819c3a-ce70-4665-a99f-f61e9999774a","Type":"ContainerStarted","Data":"4bc1eca34acd5a31e58f90f6472708bf09f7f038c280fcbb6f3e9e4d48859885"}
Jul 05 00:48:31.433818 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:48:31.433735    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-a-174d7d-openshift-default-859476d8d9-cjgmw"
Jul 05 00:48:31.434008 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:48:31.433985    2563 patch_prober.go:28] interesting pod/e2e-api-a-174d7d-openshift-default-859476d8d9-cjgmw container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.42:15021/healthz/ready\": dial tcp 10.133.0.42:15021: connect: connection refused" start-of-body=
Jul 05 00:48:31.434104 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:48:31.434079    2563 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-174d7d-openshift-default-859476d8d9-cjgmw" podUID="a3819c3a-ce70-4665-a99f-f61e9999774a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.42:15021/healthz/ready\": dial tcp 10.133.0.42:15021: connect: connection refused"
Jul 05 00:48:32.433953 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:48:32.433918    2563 patch_prober.go:28] interesting pod/e2e-api-a-174d7d-openshift-default-859476d8d9-cjgmw container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.42:15021/healthz/ready\": dial tcp 10.133.0.42:15021: connect: connection refused" start-of-body=
Jul 05 00:48:32.434410 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:48:32.433975    2563 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-174d7d-openshift-default-859476d8d9-cjgmw" podUID="a3819c3a-ce70-4665-a99f-f61e9999774a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.42:15021/healthz/ready\": dial tcp 10.133.0.42:15021: connect: connection refused"
Jul 05 00:48:33.433910 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:48:33.433868    2563 patch_prober.go:28] interesting pod/e2e-api-a-174d7d-openshift-default-859476d8d9-cjgmw container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.42:15021/healthz/ready\": dial tcp 10.133.0.42:15021: connect: connection refused" start-of-body=
Jul 05 00:48:33.434086 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:48:33.433934    2563 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-174d7d-openshift-default-859476d8d9-cjgmw" podUID="a3819c3a-ce70-4665-a99f-f61e9999774a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.42:15021/healthz/ready\": dial tcp 10.133.0.42:15021: connect: connection refused"
Jul 05 00:48:34.426500 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:48:34.426386    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-x5szm"]
Jul 05 00:48:34.433848 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:48:34.433815    2563 patch_prober.go:28] interesting pod/e2e-api-a-174d7d-openshift-default-859476d8d9-cjgmw container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.42:15021/healthz/ready\": dial tcp 10.133.0.42:15021: connect: connection refused" start-of-body=
Jul 05 00:48:34.433993 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:48:34.433871    2563 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-174d7d-openshift-default-859476d8d9-cjgmw" podUID="a3819c3a-ce70-4665-a99f-f61e9999774a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.42:15021/healthz/ready\": dial tcp 10.133.0.42:15021: connect: connection refused"
Jul 05 00:48:35.437582 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:48:35.437549    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-a-174d7d-openshift-default-859476d8d9-cjgmw"
Jul 05 00:48:35.438026 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:48:35.437811    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-a-174d7d-openshift-default-859476d8d9-cjgmw"
Jul 05 00:48:35.438662 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:48:35.438641    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-a-174d7d-openshift-default-859476d8d9-cjgmw"
Jul 05 00:48:41.679073 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:48:41.679041    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-a-174d7d-openshift-default-859476d8d9-cjgmw"]
Jul 05 00:48:41.679610 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:48:41.679259    2563 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-ingress/e2e-api-a-174d7d-openshift-default-859476d8d9-cjgmw" podUID="a3819c3a-ce70-4665-a99f-f61e9999774a" containerName="istio-proxy" containerID="cri-o://4bc1eca34acd5a31e58f90f6472708bf09f7f038c280fcbb6f3e9e4d48859885" gracePeriod=30
Jul 05 00:48:45.122266 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:48:45.122231    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-x5szm"]
Jul 05 00:48:46.926250 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:48:46.926209    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-174d7d-openshift-default-859476d8d9-cjgmw"
Jul 05 00:48:47.043955 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:48:47.043929    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a3819c3a-ce70-4665-a99f-f61e9999774a-workload-certs\") pod \"a3819c3a-ce70-4665-a99f-f61e9999774a\" (UID: \"a3819c3a-ce70-4665-a99f-f61e9999774a\") "
Jul 05 00:48:47.044144 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:48:47.044006    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a3819c3a-ce70-4665-a99f-f61e9999774a-istio-token\") pod \"a3819c3a-ce70-4665-a99f-f61e9999774a\" (UID: \"a3819c3a-ce70-4665-a99f-f61e9999774a\") "
Jul 05 00:48:47.044144 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:48:47.044054    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a3819c3a-ce70-4665-a99f-f61e9999774a-workload-socket\") pod \"a3819c3a-ce70-4665-a99f-f61e9999774a\" (UID: \"a3819c3a-ce70-4665-a99f-f61e9999774a\") "
Jul 05 00:48:47.044144 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:48:47.044100    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-n2zx5\" (UniqueName: \"kubernetes.io/projected/a3819c3a-ce70-4665-a99f-f61e9999774a-kube-api-access-n2zx5\") pod \"a3819c3a-ce70-4665-a99f-f61e9999774a\" (UID: \"a3819c3a-ce70-4665-a99f-f61e9999774a\") "
Jul 05 00:48:47.044345 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:48:47.044151    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a3819c3a-ce70-4665-a99f-f61e9999774a-credential-socket\") pod \"a3819c3a-ce70-4665-a99f-f61e9999774a\" (UID: \"a3819c3a-ce70-4665-a99f-f61e9999774a\") "
Jul 05 00:48:47.044345 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:48:47.044174    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a3819c3a-ce70-4665-a99f-f61e9999774a-istiod-ca-cert\") pod \"a3819c3a-ce70-4665-a99f-f61e9999774a\" (UID: \"a3819c3a-ce70-4665-a99f-f61e9999774a\") "
Jul 05 00:48:47.044345 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:48:47.044204    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a3819c3a-ce70-4665-a99f-f61e9999774a-istio-podinfo\") pod \"a3819c3a-ce70-4665-a99f-f61e9999774a\" (UID: \"a3819c3a-ce70-4665-a99f-f61e9999774a\") "
Jul 05 00:48:47.044345 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:48:47.044197    2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a3819c3a-ce70-4665-a99f-f61e9999774a-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "a3819c3a-ce70-4665-a99f-f61e9999774a" (UID: "a3819c3a-ce70-4665-a99f-f61e9999774a"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 05 00:48:47.044345 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:48:47.044264    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a3819c3a-ce70-4665-a99f-f61e9999774a-istio-data\") pod \"a3819c3a-ce70-4665-a99f-f61e9999774a\" (UID: \"a3819c3a-ce70-4665-a99f-f61e9999774a\") "
Jul 05 00:48:47.044345 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:48:47.044295    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a3819c3a-ce70-4665-a99f-f61e9999774a-istio-envoy\") pod \"a3819c3a-ce70-4665-a99f-f61e9999774a\" (UID: \"a3819c3a-ce70-4665-a99f-f61e9999774a\") "
Jul 05 00:48:47.044642 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:48:47.044430    2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a3819c3a-ce70-4665-a99f-f61e9999774a-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "a3819c3a-ce70-4665-a99f-f61e9999774a" (UID: "a3819c3a-ce70-4665-a99f-f61e9999774a"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 05 00:48:47.044642 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:48:47.044450    2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a3819c3a-ce70-4665-a99f-f61e9999774a-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "a3819c3a-ce70-4665-a99f-f61e9999774a" (UID: "a3819c3a-ce70-4665-a99f-f61e9999774a"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 05 00:48:47.044642 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:48:47.044512    2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a3819c3a-ce70-4665-a99f-f61e9999774a-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "a3819c3a-ce70-4665-a99f-f61e9999774a" (UID: "a3819c3a-ce70-4665-a99f-f61e9999774a"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 05 00:48:47.044822 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:48:47.044692    2563 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a3819c3a-ce70-4665-a99f-f61e9999774a-credential-socket\") on node \"ip-10-0-142-202.ec2.internal\" DevicePath \"\""
Jul 05 00:48:47.044822 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:48:47.044712    2563 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a3819c3a-ce70-4665-a99f-f61e9999774a-istiod-ca-cert\") on node \"ip-10-0-142-202.ec2.internal\" DevicePath \"\""
Jul 05 00:48:47.044822 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:48:47.044725    2563 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a3819c3a-ce70-4665-a99f-f61e9999774a-workload-certs\") on node \"ip-10-0-142-202.ec2.internal\" DevicePath \"\""
Jul 05 00:48:47.044822 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:48:47.044737    2563 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a3819c3a-ce70-4665-a99f-f61e9999774a-workload-socket\") on node \"ip-10-0-142-202.ec2.internal\" DevicePath \"\""
Jul 05 00:48:47.045299 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:48:47.045274    2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a3819c3a-ce70-4665-a99f-f61e9999774a-istio-data" (OuterVolumeSpecName: "istio-data") pod "a3819c3a-ce70-4665-a99f-f61e9999774a" (UID: "a3819c3a-ce70-4665-a99f-f61e9999774a"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 05 00:48:47.046522 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:48:47.046476    2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a3819c3a-ce70-4665-a99f-f61e9999774a-istio-token" (OuterVolumeSpecName: "istio-token") pod "a3819c3a-ce70-4665-a99f-f61e9999774a" (UID: "a3819c3a-ce70-4665-a99f-f61e9999774a"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 05 00:48:47.046522 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:48:47.046478    2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a3819c3a-ce70-4665-a99f-f61e9999774a-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "a3819c3a-ce70-4665-a99f-f61e9999774a" (UID: "a3819c3a-ce70-4665-a99f-f61e9999774a"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 05 00:48:47.046692 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:48:47.046672    2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a3819c3a-ce70-4665-a99f-f61e9999774a-kube-api-access-n2zx5" (OuterVolumeSpecName: "kube-api-access-n2zx5") pod "a3819c3a-ce70-4665-a99f-f61e9999774a" (UID: "a3819c3a-ce70-4665-a99f-f61e9999774a"). InnerVolumeSpecName "kube-api-access-n2zx5". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 05 00:48:47.046738 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:48:47.046676    2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/a3819c3a-ce70-4665-a99f-f61e9999774a-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "a3819c3a-ce70-4665-a99f-f61e9999774a" (UID: "a3819c3a-ce70-4665-a99f-f61e9999774a"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Jul 05 00:48:47.145385 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:48:47.145356    2563 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a3819c3a-ce70-4665-a99f-f61e9999774a-istio-data\") on node \"ip-10-0-142-202.ec2.internal\" DevicePath \"\""
Jul 05 00:48:47.145385 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:48:47.145382    2563 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a3819c3a-ce70-4665-a99f-f61e9999774a-istio-envoy\") on node \"ip-10-0-142-202.ec2.internal\" DevicePath \"\""
Jul 05 00:48:47.145579 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:48:47.145394    2563 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a3819c3a-ce70-4665-a99f-f61e9999774a-istio-token\") on node \"ip-10-0-142-202.ec2.internal\" DevicePath \"\""
Jul 05 00:48:47.145579 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:48:47.145407    2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-n2zx5\" (UniqueName: \"kubernetes.io/projected/a3819c3a-ce70-4665-a99f-f61e9999774a-kube-api-access-n2zx5\") on node \"ip-10-0-142-202.ec2.internal\" DevicePath \"\""
Jul 05 00:48:47.145579 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:48:47.145419    2563 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a3819c3a-ce70-4665-a99f-f61e9999774a-istio-podinfo\") on node \"ip-10-0-142-202.ec2.internal\" DevicePath \"\""
Jul 05 00:48:47.189236 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:48:47.189180    2563 generic.go:356] "Generic (PLEG): container finished" podID="a3819c3a-ce70-4665-a99f-f61e9999774a" containerID="4bc1eca34acd5a31e58f90f6472708bf09f7f038c280fcbb6f3e9e4d48859885" exitCode=0
Jul 05 00:48:47.189366 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:48:47.189247    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-174d7d-openshift-default-859476d8d9-cjgmw"
Jul 05 00:48:47.189366 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:48:47.189253    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-174d7d-openshift-default-859476d8d9-cjgmw" event={"ID":"a3819c3a-ce70-4665-a99f-f61e9999774a","Type":"ContainerDied","Data":"4bc1eca34acd5a31e58f90f6472708bf09f7f038c280fcbb6f3e9e4d48859885"}
Jul 05 00:48:47.189366 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:48:47.189314    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-174d7d-openshift-default-859476d8d9-cjgmw" event={"ID":"a3819c3a-ce70-4665-a99f-f61e9999774a","Type":"ContainerDied","Data":"26633a33a4e23d1c1b44866fb4a7571df329cc6d899000652a488683f5b015db"}
Jul 05 00:48:47.189366 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:48:47.189345    2563 scope.go:117] "RemoveContainer" containerID="4bc1eca34acd5a31e58f90f6472708bf09f7f038c280fcbb6f3e9e4d48859885"
Jul 05 00:48:47.201742 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:48:47.201726    2563 scope.go:117] "RemoveContainer" containerID="ee6dd6768a27e8c76f3ee08321b5c58296662177fcc1a63e2213266741aa4b38"
Jul 05 00:48:47.210341 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:48:47.210280    2563 scope.go:117] "RemoveContainer" containerID="4bc1eca34acd5a31e58f90f6472708bf09f7f038c280fcbb6f3e9e4d48859885"
Jul 05 00:48:47.210589 ip-10-0-142-202 kubenswrapper[2563]: E0705 00:48:47.210562    2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4bc1eca34acd5a31e58f90f6472708bf09f7f038c280fcbb6f3e9e4d48859885\": container with ID starting with 4bc1eca34acd5a31e58f90f6472708bf09f7f038c280fcbb6f3e9e4d48859885 not found: ID does not exist" containerID="4bc1eca34acd5a31e58f90f6472708bf09f7f038c280fcbb6f3e9e4d48859885"
Jul 05 00:48:47.210660 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:48:47.210599    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4bc1eca34acd5a31e58f90f6472708bf09f7f038c280fcbb6f3e9e4d48859885"} err="failed to get container status \"4bc1eca34acd5a31e58f90f6472708bf09f7f038c280fcbb6f3e9e4d48859885\": rpc error: code = NotFound desc = could not find container \"4bc1eca34acd5a31e58f90f6472708bf09f7f038c280fcbb6f3e9e4d48859885\": container with ID starting with 4bc1eca34acd5a31e58f90f6472708bf09f7f038c280fcbb6f3e9e4d48859885 not found: ID does not exist"
Jul 05 00:48:47.210660 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:48:47.210622    2563 scope.go:117] "RemoveContainer" containerID="ee6dd6768a27e8c76f3ee08321b5c58296662177fcc1a63e2213266741aa4b38"
Jul 05 00:48:47.210887 ip-10-0-142-202 kubenswrapper[2563]: E0705 00:48:47.210871    2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ee6dd6768a27e8c76f3ee08321b5c58296662177fcc1a63e2213266741aa4b38\": container with ID starting with ee6dd6768a27e8c76f3ee08321b5c58296662177fcc1a63e2213266741aa4b38 not found: ID does not exist" containerID="ee6dd6768a27e8c76f3ee08321b5c58296662177fcc1a63e2213266741aa4b38"
Jul 05 00:48:47.210929 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:48:47.210892    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ee6dd6768a27e8c76f3ee08321b5c58296662177fcc1a63e2213266741aa4b38"} err="failed to get container status \"ee6dd6768a27e8c76f3ee08321b5c58296662177fcc1a63e2213266741aa4b38\": rpc error: code = NotFound desc = could not find container \"ee6dd6768a27e8c76f3ee08321b5c58296662177fcc1a63e2213266741aa4b38\": container with ID starting with ee6dd6768a27e8c76f3ee08321b5c58296662177fcc1a63e2213266741aa4b38 not found: ID does not exist"
Jul 05 00:48:47.211950 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:48:47.211932    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-a-174d7d-openshift-default-859476d8d9-cjgmw"]
Jul 05 00:48:47.215121 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:48:47.215101    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-api-a-174d7d-openshift-default-859476d8d9-cjgmw"]
Jul 05 00:48:48.126873 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:48:48.126841    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a3819c3a-ce70-4665-a99f-f61e9999774a" path="/var/lib/kubelet/pods/a3819c3a-ce70-4665-a99f-f61e9999774a/volumes"
Jul 05 00:48:49.366870 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:48:49.366842    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-shared-a-3575e1-openshift-default-5c7fb45bd7-gp87d"]
Jul 05 00:48:49.367347 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:48:49.367330    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a3819c3a-ce70-4665-a99f-f61e9999774a" containerName="istio-proxy"
Jul 05 00:48:49.367421 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:48:49.367351    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="a3819c3a-ce70-4665-a99f-f61e9999774a" containerName="istio-proxy"
Jul 05 00:48:49.367421 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:48:49.367366    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a3819c3a-ce70-4665-a99f-f61e9999774a" containerName="istio-proxy"
Jul 05 00:48:49.367421 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:48:49.367374    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="a3819c3a-ce70-4665-a99f-f61e9999774a" containerName="istio-proxy"
Jul 05 00:48:49.367566 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:48:49.367469    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="a3819c3a-ce70-4665-a99f-f61e9999774a" containerName="istio-proxy"
Jul 05 00:48:49.370530 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:48:49.370511    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-3575e1-openshift-default-5c7fb45bd7-gp87d"
Jul 05 00:48:49.372450 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:48:49.372434    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-shared-a-3575e1-openshift-default-dockercfg-skth8\""
Jul 05 00:48:49.382562 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:48:49.382543    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-a-3575e1-openshift-default-5c7fb45bd7-gp87d"]
Jul 05 00:48:49.465536 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:48:49.465509    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e6b5817b-48da-49eb-911b-dc93e86b6df5-istio-podinfo\") pod \"e2e-shared-a-3575e1-openshift-default-5c7fb45bd7-gp87d\" (UID: \"e6b5817b-48da-49eb-911b-dc93e86b6df5\") " pod="openshift-ingress/e2e-shared-a-3575e1-openshift-default-5c7fb45bd7-gp87d"
Jul 05 00:48:49.465536 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:48:49.465540    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e6b5817b-48da-49eb-911b-dc93e86b6df5-credential-socket\") pod \"e2e-shared-a-3575e1-openshift-default-5c7fb45bd7-gp87d\" (UID: \"e6b5817b-48da-49eb-911b-dc93e86b6df5\") " pod="openshift-ingress/e2e-shared-a-3575e1-openshift-default-5c7fb45bd7-gp87d"
Jul 05 00:48:49.465790 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:48:49.465564    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vpbdg\" (UniqueName: \"kubernetes.io/projected/e6b5817b-48da-49eb-911b-dc93e86b6df5-kube-api-access-vpbdg\") pod \"e2e-shared-a-3575e1-openshift-default-5c7fb45bd7-gp87d\" (UID: \"e6b5817b-48da-49eb-911b-dc93e86b6df5\") " pod="openshift-ingress/e2e-shared-a-3575e1-openshift-default-5c7fb45bd7-gp87d"
Jul 05 00:48:49.465790 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:48:49.465583    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e6b5817b-48da-49eb-911b-dc93e86b6df5-workload-socket\") pod \"e2e-shared-a-3575e1-openshift-default-5c7fb45bd7-gp87d\" (UID: \"e6b5817b-48da-49eb-911b-dc93e86b6df5\") " pod="openshift-ingress/e2e-shared-a-3575e1-openshift-default-5c7fb45bd7-gp87d"
Jul 05 00:48:49.465790 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:48:49.465669    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e6b5817b-48da-49eb-911b-dc93e86b6df5-istio-envoy\") pod \"e2e-shared-a-3575e1-openshift-default-5c7fb45bd7-gp87d\" (UID: \"e6b5817b-48da-49eb-911b-dc93e86b6df5\") " pod="openshift-ingress/e2e-shared-a-3575e1-openshift-default-5c7fb45bd7-gp87d"
Jul 05 00:48:49.465790 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:48:49.465727    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e6b5817b-48da-49eb-911b-dc93e86b6df5-workload-certs\") pod \"e2e-shared-a-3575e1-openshift-default-5c7fb45bd7-gp87d\" (UID: \"e6b5817b-48da-49eb-911b-dc93e86b6df5\") " pod="openshift-ingress/e2e-shared-a-3575e1-openshift-default-5c7fb45bd7-gp87d"
Jul 05 00:48:49.465790 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:48:49.465761    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e6b5817b-48da-49eb-911b-dc93e86b6df5-istiod-ca-cert\") pod \"e2e-shared-a-3575e1-openshift-default-5c7fb45bd7-gp87d\" (UID: \"e6b5817b-48da-49eb-911b-dc93e86b6df5\") " pod="openshift-ingress/e2e-shared-a-3575e1-openshift-default-5c7fb45bd7-gp87d"
Jul 05 00:48:49.466013 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:48:49.465808    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e6b5817b-48da-49eb-911b-dc93e86b6df5-istio-token\") pod \"e2e-shared-a-3575e1-openshift-default-5c7fb45bd7-gp87d\" (UID: \"e6b5817b-48da-49eb-911b-dc93e86b6df5\") " pod="openshift-ingress/e2e-shared-a-3575e1-openshift-default-5c7fb45bd7-gp87d"
Jul 05 00:48:49.466013 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:48:49.465840    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e6b5817b-48da-49eb-911b-dc93e86b6df5-istio-data\") pod \"e2e-shared-a-3575e1-openshift-default-5c7fb45bd7-gp87d\" (UID: \"e6b5817b-48da-49eb-911b-dc93e86b6df5\") " pod="openshift-ingress/e2e-shared-a-3575e1-openshift-default-5c7fb45bd7-gp87d"
Jul 05 00:48:49.567086 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:48:49.567053    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e6b5817b-48da-49eb-911b-dc93e86b6df5-credential-socket\") pod \"e2e-shared-a-3575e1-openshift-default-5c7fb45bd7-gp87d\" (UID: \"e6b5817b-48da-49eb-911b-dc93e86b6df5\") " pod="openshift-ingress/e2e-shared-a-3575e1-openshift-default-5c7fb45bd7-gp87d"
Jul 05 00:48:49.567086 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:48:49.567087    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vpbdg\" (UniqueName: \"kubernetes.io/projected/e6b5817b-48da-49eb-911b-dc93e86b6df5-kube-api-access-vpbdg\") pod \"e2e-shared-a-3575e1-openshift-default-5c7fb45bd7-gp87d\" (UID: \"e6b5817b-48da-49eb-911b-dc93e86b6df5\") " pod="openshift-ingress/e2e-shared-a-3575e1-openshift-default-5c7fb45bd7-gp87d"
Jul 05 00:48:49.567359 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:48:49.567108    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e6b5817b-48da-49eb-911b-dc93e86b6df5-workload-socket\") pod \"e2e-shared-a-3575e1-openshift-default-5c7fb45bd7-gp87d\" (UID: \"e6b5817b-48da-49eb-911b-dc93e86b6df5\") " pod="openshift-ingress/e2e-shared-a-3575e1-openshift-default-5c7fb45bd7-gp87d"
Jul 05 00:48:49.567359 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:48:49.567238    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e6b5817b-48da-49eb-911b-dc93e86b6df5-istio-envoy\") pod \"e2e-shared-a-3575e1-openshift-default-5c7fb45bd7-gp87d\" (UID: \"e6b5817b-48da-49eb-911b-dc93e86b6df5\") " pod="openshift-ingress/e2e-shared-a-3575e1-openshift-default-5c7fb45bd7-gp87d"
Jul 05 00:48:49.567359 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:48:49.567311    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e6b5817b-48da-49eb-911b-dc93e86b6df5-workload-certs\") pod \"e2e-shared-a-3575e1-openshift-default-5c7fb45bd7-gp87d\" (UID: \"e6b5817b-48da-49eb-911b-dc93e86b6df5\") " pod="openshift-ingress/e2e-shared-a-3575e1-openshift-default-5c7fb45bd7-gp87d"
Jul 05 00:48:49.567359 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:48:49.567336    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e6b5817b-48da-49eb-911b-dc93e86b6df5-istiod-ca-cert\") pod \"e2e-shared-a-3575e1-openshift-default-5c7fb45bd7-gp87d\" (UID: \"e6b5817b-48da-49eb-911b-dc93e86b6df5\") " pod="openshift-ingress/e2e-shared-a-3575e1-openshift-default-5c7fb45bd7-gp87d"
Jul 05 00:48:49.567546 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:48:49.567466    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e6b5817b-48da-49eb-911b-dc93e86b6df5-istio-token\") pod \"e2e-shared-a-3575e1-openshift-default-5c7fb45bd7-gp87d\" (UID: \"e6b5817b-48da-49eb-911b-dc93e86b6df5\") " pod="openshift-ingress/e2e-shared-a-3575e1-openshift-default-5c7fb45bd7-gp87d"
Jul 05 00:48:49.567546 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:48:49.567519    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e6b5817b-48da-49eb-911b-dc93e86b6df5-istio-data\") pod \"e2e-shared-a-3575e1-openshift-default-5c7fb45bd7-gp87d\" (UID: \"e6b5817b-48da-49eb-911b-dc93e86b6df5\") " pod="openshift-ingress/e2e-shared-a-3575e1-openshift-default-5c7fb45bd7-gp87d"
Jul 05 00:48:49.567546 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:48:49.567524    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e6b5817b-48da-49eb-911b-dc93e86b6df5-credential-socket\") pod \"e2e-shared-a-3575e1-openshift-default-5c7fb45bd7-gp87d\" (UID: \"e6b5817b-48da-49eb-911b-dc93e86b6df5\") " pod="openshift-ingress/e2e-shared-a-3575e1-openshift-default-5c7fb45bd7-gp87d"
Jul 05 00:48:49.567546 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:48:49.567470    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e6b5817b-48da-49eb-911b-dc93e86b6df5-workload-socket\") pod \"e2e-shared-a-3575e1-openshift-default-5c7fb45bd7-gp87d\" (UID: \"e6b5817b-48da-49eb-911b-dc93e86b6df5\") " pod="openshift-ingress/e2e-shared-a-3575e1-openshift-default-5c7fb45bd7-gp87d"
Jul 05 00:48:49.567744 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:48:49.567609    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e6b5817b-48da-49eb-911b-dc93e86b6df5-istio-podinfo\") pod \"e2e-shared-a-3575e1-openshift-default-5c7fb45bd7-gp87d\" (UID: \"e6b5817b-48da-49eb-911b-dc93e86b6df5\") " pod="openshift-ingress/e2e-shared-a-3575e1-openshift-default-5c7fb45bd7-gp87d"
Jul 05 00:48:49.567744 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:48:49.567688    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e6b5817b-48da-49eb-911b-dc93e86b6df5-workload-certs\") pod \"e2e-shared-a-3575e1-openshift-default-5c7fb45bd7-gp87d\" (UID: \"e6b5817b-48da-49eb-911b-dc93e86b6df5\") " pod="openshift-ingress/e2e-shared-a-3575e1-openshift-default-5c7fb45bd7-gp87d"
Jul 05 00:48:49.567846 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:48:49.567829    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e6b5817b-48da-49eb-911b-dc93e86b6df5-istio-data\") pod \"e2e-shared-a-3575e1-openshift-default-5c7fb45bd7-gp87d\" (UID: \"e6b5817b-48da-49eb-911b-dc93e86b6df5\") " pod="openshift-ingress/e2e-shared-a-3575e1-openshift-default-5c7fb45bd7-gp87d"
Jul 05 00:48:49.567988 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:48:49.567970    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e6b5817b-48da-49eb-911b-dc93e86b6df5-istiod-ca-cert\") pod \"e2e-shared-a-3575e1-openshift-default-5c7fb45bd7-gp87d\" (UID: \"e6b5817b-48da-49eb-911b-dc93e86b6df5\") " pod="openshift-ingress/e2e-shared-a-3575e1-openshift-default-5c7fb45bd7-gp87d"
Jul 05 00:48:49.569454 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:48:49.569434    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e6b5817b-48da-49eb-911b-dc93e86b6df5-istio-envoy\") pod \"e2e-shared-a-3575e1-openshift-default-5c7fb45bd7-gp87d\" (UID: \"e6b5817b-48da-49eb-911b-dc93e86b6df5\") " pod="openshift-ingress/e2e-shared-a-3575e1-openshift-default-5c7fb45bd7-gp87d"
Jul 05 00:48:49.569932 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:48:49.569917    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e6b5817b-48da-49eb-911b-dc93e86b6df5-istio-podinfo\") pod \"e2e-shared-a-3575e1-openshift-default-5c7fb45bd7-gp87d\" (UID: \"e6b5817b-48da-49eb-911b-dc93e86b6df5\") " pod="openshift-ingress/e2e-shared-a-3575e1-openshift-default-5c7fb45bd7-gp87d"
Jul 05 00:48:49.574202 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:48:49.574177    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e6b5817b-48da-49eb-911b-dc93e86b6df5-istio-token\") pod \"e2e-shared-a-3575e1-openshift-default-5c7fb45bd7-gp87d\" (UID: \"e6b5817b-48da-49eb-911b-dc93e86b6df5\") " pod="openshift-ingress/e2e-shared-a-3575e1-openshift-default-5c7fb45bd7-gp87d"
Jul 05 00:48:49.574551 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:48:49.574527    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-vpbdg\" (UniqueName: \"kubernetes.io/projected/e6b5817b-48da-49eb-911b-dc93e86b6df5-kube-api-access-vpbdg\") pod \"e2e-shared-a-3575e1-openshift-default-5c7fb45bd7-gp87d\" (UID: \"e6b5817b-48da-49eb-911b-dc93e86b6df5\") " pod="openshift-ingress/e2e-shared-a-3575e1-openshift-default-5c7fb45bd7-gp87d"
Jul 05 00:48:49.680602 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:48:49.680525    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-3575e1-openshift-default-5c7fb45bd7-gp87d"
Jul 05 00:48:49.812996 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:48:49.812972    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-a-3575e1-openshift-default-5c7fb45bd7-gp87d"]
Jul 05 00:48:49.815266 ip-10-0-142-202 kubenswrapper[2563]: W0705 00:48:49.815240    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode6b5817b_48da_49eb_911b_dc93e86b6df5.slice/crio-98e33fda3a6a1ed3c22a82afa3aa8611731be72a594841451ae6637fbb809711 WatchSource:0}: Error finding container 98e33fda3a6a1ed3c22a82afa3aa8611731be72a594841451ae6637fbb809711: Status 404 returned error can't find the container with id 98e33fda3a6a1ed3c22a82afa3aa8611731be72a594841451ae6637fbb809711
Jul 05 00:48:49.817804 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:48:49.817776    2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"}
Jul 05 00:48:49.817868 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:48:49.817834    2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"}
Jul 05 00:48:49.817907 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:48:49.817866    2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"}
Jul 05 00:48:50.206204 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:48:50.206168    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-3575e1-openshift-default-5c7fb45bd7-gp87d" event={"ID":"e6b5817b-48da-49eb-911b-dc93e86b6df5","Type":"ContainerStarted","Data":"8d818e001e4702e4e9de5a97e5ed91c6439f6581381f6a281ed8aa0df5277c20"}
Jul 05 00:48:50.206431 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:48:50.206241    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-3575e1-openshift-default-5c7fb45bd7-gp87d" event={"ID":"e6b5817b-48da-49eb-911b-dc93e86b6df5","Type":"ContainerStarted","Data":"98e33fda3a6a1ed3c22a82afa3aa8611731be72a594841451ae6637fbb809711"}
Jul 05 00:48:50.224525 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:48:50.224477    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-shared-a-3575e1-openshift-default-5c7fb45bd7-gp87d" podStartSLOduration=1.224458849 podStartE2EDuration="1.224458849s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-05 00:48:49 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-05 00:48:50.22272646 +0000 UTC m=+2378.565055521" watchObservedRunningTime="2026-07-05 00:48:50.224458849 +0000 UTC m=+2378.566787908"
Jul 05 00:48:50.681408 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:48:50.681369    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-a-3575e1-openshift-default-5c7fb45bd7-gp87d"
Jul 05 00:48:50.686590 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:48:50.686568    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-a-3575e1-openshift-default-5c7fb45bd7-gp87d"
Jul 05 00:48:51.210854 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:48:51.210824    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-a-3575e1-openshift-default-5c7fb45bd7-gp87d"
Jul 05 00:48:51.211729 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:48:51.211707    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-a-3575e1-openshift-default-5c7fb45bd7-gp87d"
Jul 05 00:48:57.530118 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:48:57.530077    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-x5szm"]
Jul 05 00:49:05.274739 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:49:05.274627    2563 generic.go:356] "Generic (PLEG): container finished" podID="e6b5817b-48da-49eb-911b-dc93e86b6df5" containerID="8d818e001e4702e4e9de5a97e5ed91c6439f6581381f6a281ed8aa0df5277c20" exitCode=0
Jul 05 00:49:05.274739 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:49:05.274708    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-3575e1-openshift-default-5c7fb45bd7-gp87d" event={"ID":"e6b5817b-48da-49eb-911b-dc93e86b6df5","Type":"ContainerDied","Data":"8d818e001e4702e4e9de5a97e5ed91c6439f6581381f6a281ed8aa0df5277c20"}
Jul 05 00:49:05.275357 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:49:05.275260    2563 scope.go:117] "RemoveContainer" containerID="8d818e001e4702e4e9de5a97e5ed91c6439f6581381f6a281ed8aa0df5277c20"
Jul 05 00:49:05.276262 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:49:05.276235    2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"}
Jul 05 00:49:05.276366 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:49:05.276299    2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"}
Jul 05 00:49:05.276366 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:49:05.276335    2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"}
Jul 05 00:49:05.831279 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:49:05.831246    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-x5szm"]
Jul 05 00:49:06.211189 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:49:06.211114    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-a-3575e1-openshift-default-5c7fb45bd7-gp87d"
Jul 05 00:49:06.283693 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:49:06.283619    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-3575e1-openshift-default-5c7fb45bd7-gp87d" event={"ID":"e6b5817b-48da-49eb-911b-dc93e86b6df5","Type":"ContainerStarted","Data":"c272dc44518b6cba4738d5b0bf477219c796cc840c0d919be07f81a52f5c00bd"}
Jul 05 00:49:06.681311 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:49:06.681268    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-a-3575e1-openshift-default-5c7fb45bd7-gp87d"
Jul 05 00:49:06.682684 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:49:06.682654    2563 patch_prober.go:28] interesting pod/e2e-shared-a-3575e1-openshift-default-5c7fb45bd7-gp87d container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.43:15021/healthz/ready\": dial tcp 10.133.0.43:15021: connect: connection refused" start-of-body=
Jul 05 00:49:06.682889 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:49:06.682861    2563 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-3575e1-openshift-default-5c7fb45bd7-gp87d" podUID="e6b5817b-48da-49eb-911b-dc93e86b6df5" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.43:15021/healthz/ready\": dial tcp 10.133.0.43:15021: connect: connection refused"
Jul 05 00:49:07.681057 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:49:07.681024    2563 patch_prober.go:28] interesting pod/e2e-shared-a-3575e1-openshift-default-5c7fb45bd7-gp87d container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.43:15021/healthz/ready\": dial tcp 10.133.0.43:15021: connect: connection refused" start-of-body=
Jul 05 00:49:07.681666 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:49:07.681627    2563 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-3575e1-openshift-default-5c7fb45bd7-gp87d" podUID="e6b5817b-48da-49eb-911b-dc93e86b6df5" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.43:15021/healthz/ready\": dial tcp 10.133.0.43:15021: connect: connection refused"
Jul 05 00:49:08.681165 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:49:08.681123    2563 patch_prober.go:28] interesting pod/e2e-shared-a-3575e1-openshift-default-5c7fb45bd7-gp87d container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.43:15021/healthz/ready\": dial tcp 10.133.0.43:15021: connect: connection refused" start-of-body=
Jul 05 00:49:08.681676 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:49:08.681186    2563 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-3575e1-openshift-default-5c7fb45bd7-gp87d" podUID="e6b5817b-48da-49eb-911b-dc93e86b6df5" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.43:15021/healthz/ready\": dial tcp 10.133.0.43:15021: connect: connection refused"
Jul 05 00:49:09.681897 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:49:09.681856    2563 patch_prober.go:28] interesting pod/e2e-shared-a-3575e1-openshift-default-5c7fb45bd7-gp87d container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.43:15021/healthz/ready\": dial tcp 10.133.0.43:15021: connect: connection refused" start-of-body=
Jul 05 00:49:09.682953 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:49:09.682539    2563 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-3575e1-openshift-default-5c7fb45bd7-gp87d" podUID="e6b5817b-48da-49eb-911b-dc93e86b6df5" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.43:15021/healthz/ready\": dial tcp 10.133.0.43:15021: connect: connection refused"
Jul 05 00:49:10.684865 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:49:10.684833    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-a-3575e1-openshift-default-5c7fb45bd7-gp87d"
Jul 05 00:49:10.685251 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:49:10.685170    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-a-3575e1-openshift-default-5c7fb45bd7-gp87d"
Jul 05 00:49:10.685905 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:49:10.685889    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-a-3575e1-openshift-default-5c7fb45bd7-gp87d"
Jul 05 00:49:13.082340 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:49:13.082312    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-7sxq4_b96bfb4d-2fd3-4be3-9064-e24efeb904f4/ovn-acl-logging/0.log"
Jul 05 00:49:13.088203 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:49:13.088184    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-7sxq4_b96bfb4d-2fd3-4be3-9064-e24efeb904f4/ovn-acl-logging/0.log"
Jul 05 00:49:41.007727 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:49:41.007689    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-a-3575e1/auth-test-model-3575e1-kserve-8497c7559f-f24r6"]
Jul 05 00:49:41.008273 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:49:41.008063    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="a3819c3a-ce70-4665-a99f-f61e9999774a" containerName="istio-proxy"
Jul 05 00:49:41.011079 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:49:41.011060    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-3575e1/auth-test-model-3575e1-kserve-8497c7559f-f24r6"
Jul 05 00:49:41.013499 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:49:41.013474    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-3575e1\"/\"auth-test-model-3575e1-kserve-self-signed-certs\""
Jul 05 00:49:41.013625 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:49:41.013499    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-3575e1\"/\"kube-root-ca.crt\""
Jul 05 00:49:41.013625 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:49:41.013482    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-3575e1\"/\"openshift-service-ca.crt\""
Jul 05 00:49:41.013625 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:49:41.013474    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-3575e1\"/\"default-dockercfg-8f6t6\""
Jul 05 00:49:41.018140 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:49:41.018122    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-3575e1/auth-test-model-3575e1-kserve-8497c7559f-f24r6"]
Jul 05 00:49:41.129960 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:49:41.129929    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/7778d5cf-4af4-4400-8972-7505332d367d-tmp-dir\") pod \"auth-test-model-3575e1-kserve-8497c7559f-f24r6\" (UID: \"7778d5cf-4af4-4400-8972-7505332d367d\") " pod="ai-tenant-e2e-shared-a-3575e1/auth-test-model-3575e1-kserve-8497c7559f-f24r6"
Jul 05 00:49:41.129960 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:49:41.129963    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-snwgh\" (UniqueName: \"kubernetes.io/projected/7778d5cf-4af4-4400-8972-7505332d367d-kube-api-access-snwgh\") pod \"auth-test-model-3575e1-kserve-8497c7559f-f24r6\" (UID: \"7778d5cf-4af4-4400-8972-7505332d367d\") " pod="ai-tenant-e2e-shared-a-3575e1/auth-test-model-3575e1-kserve-8497c7559f-f24r6"
Jul 05 00:49:41.130166 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:49:41.130064    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/7778d5cf-4af4-4400-8972-7505332d367d-tls-certs\") pod \"auth-test-model-3575e1-kserve-8497c7559f-f24r6\" (UID: \"7778d5cf-4af4-4400-8972-7505332d367d\") " pod="ai-tenant-e2e-shared-a-3575e1/auth-test-model-3575e1-kserve-8497c7559f-f24r6"
Jul 05 00:49:41.130166 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:49:41.130111    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/7778d5cf-4af4-4400-8972-7505332d367d-dshm\") pod \"auth-test-model-3575e1-kserve-8497c7559f-f24r6\" (UID: \"7778d5cf-4af4-4400-8972-7505332d367d\") " pod="ai-tenant-e2e-shared-a-3575e1/auth-test-model-3575e1-kserve-8497c7559f-f24r6"
Jul 05 00:49:41.130166 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:49:41.130145    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/7778d5cf-4af4-4400-8972-7505332d367d-home\") pod \"auth-test-model-3575e1-kserve-8497c7559f-f24r6\" (UID: \"7778d5cf-4af4-4400-8972-7505332d367d\") " pod="ai-tenant-e2e-shared-a-3575e1/auth-test-model-3575e1-kserve-8497c7559f-f24r6"
Jul 05 00:49:41.130287 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:49:41.130201    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/7778d5cf-4af4-4400-8972-7505332d367d-model-cache\") pod \"auth-test-model-3575e1-kserve-8497c7559f-f24r6\" (UID: \"7778d5cf-4af4-4400-8972-7505332d367d\") " pod="ai-tenant-e2e-shared-a-3575e1/auth-test-model-3575e1-kserve-8497c7559f-f24r6"
Jul 05 00:49:41.231000 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:49:41.230969    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/7778d5cf-4af4-4400-8972-7505332d367d-tmp-dir\") pod \"auth-test-model-3575e1-kserve-8497c7559f-f24r6\" (UID: \"7778d5cf-4af4-4400-8972-7505332d367d\") " pod="ai-tenant-e2e-shared-a-3575e1/auth-test-model-3575e1-kserve-8497c7559f-f24r6"
Jul 05 00:49:41.231000 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:49:41.231005    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-snwgh\" (UniqueName: \"kubernetes.io/projected/7778d5cf-4af4-4400-8972-7505332d367d-kube-api-access-snwgh\") pod \"auth-test-model-3575e1-kserve-8497c7559f-f24r6\" (UID: \"7778d5cf-4af4-4400-8972-7505332d367d\") " pod="ai-tenant-e2e-shared-a-3575e1/auth-test-model-3575e1-kserve-8497c7559f-f24r6"
Jul 05 00:49:41.231282 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:49:41.231049    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/7778d5cf-4af4-4400-8972-7505332d367d-tls-certs\") pod \"auth-test-model-3575e1-kserve-8497c7559f-f24r6\" (UID: \"7778d5cf-4af4-4400-8972-7505332d367d\") " pod="ai-tenant-e2e-shared-a-3575e1/auth-test-model-3575e1-kserve-8497c7559f-f24r6"
Jul 05 00:49:41.231282 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:49:41.231083    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/7778d5cf-4af4-4400-8972-7505332d367d-dshm\") pod \"auth-test-model-3575e1-kserve-8497c7559f-f24r6\" (UID: \"7778d5cf-4af4-4400-8972-7505332d367d\") " pod="ai-tenant-e2e-shared-a-3575e1/auth-test-model-3575e1-kserve-8497c7559f-f24r6"
Jul 05 00:49:41.231282 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:49:41.231127    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/7778d5cf-4af4-4400-8972-7505332d367d-home\") pod \"auth-test-model-3575e1-kserve-8497c7559f-f24r6\" (UID: \"7778d5cf-4af4-4400-8972-7505332d367d\") " pod="ai-tenant-e2e-shared-a-3575e1/auth-test-model-3575e1-kserve-8497c7559f-f24r6"
Jul 05 00:49:41.231282 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:49:41.231161    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/7778d5cf-4af4-4400-8972-7505332d367d-model-cache\") pod \"auth-test-model-3575e1-kserve-8497c7559f-f24r6\" (UID: \"7778d5cf-4af4-4400-8972-7505332d367d\") " pod="ai-tenant-e2e-shared-a-3575e1/auth-test-model-3575e1-kserve-8497c7559f-f24r6"
Jul 05 00:49:41.231521 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:49:41.231441    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/7778d5cf-4af4-4400-8972-7505332d367d-tmp-dir\") pod \"auth-test-model-3575e1-kserve-8497c7559f-f24r6\" (UID: \"7778d5cf-4af4-4400-8972-7505332d367d\") " pod="ai-tenant-e2e-shared-a-3575e1/auth-test-model-3575e1-kserve-8497c7559f-f24r6"
Jul 05 00:49:41.231521 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:49:41.231493    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/7778d5cf-4af4-4400-8972-7505332d367d-model-cache\") pod \"auth-test-model-3575e1-kserve-8497c7559f-f24r6\" (UID: \"7778d5cf-4af4-4400-8972-7505332d367d\") " pod="ai-tenant-e2e-shared-a-3575e1/auth-test-model-3575e1-kserve-8497c7559f-f24r6"
Jul 05 00:49:41.231596 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:49:41.231563    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/7778d5cf-4af4-4400-8972-7505332d367d-home\") pod \"auth-test-model-3575e1-kserve-8497c7559f-f24r6\" (UID: \"7778d5cf-4af4-4400-8972-7505332d367d\") " pod="ai-tenant-e2e-shared-a-3575e1/auth-test-model-3575e1-kserve-8497c7559f-f24r6"
Jul 05 00:49:41.233294 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:49:41.233274    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/7778d5cf-4af4-4400-8972-7505332d367d-dshm\") pod \"auth-test-model-3575e1-kserve-8497c7559f-f24r6\" (UID: \"7778d5cf-4af4-4400-8972-7505332d367d\") " pod="ai-tenant-e2e-shared-a-3575e1/auth-test-model-3575e1-kserve-8497c7559f-f24r6"
Jul 05 00:49:41.233634 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:49:41.233614    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/7778d5cf-4af4-4400-8972-7505332d367d-tls-certs\") pod \"auth-test-model-3575e1-kserve-8497c7559f-f24r6\" (UID: \"7778d5cf-4af4-4400-8972-7505332d367d\") " pod="ai-tenant-e2e-shared-a-3575e1/auth-test-model-3575e1-kserve-8497c7559f-f24r6"
Jul 05 00:49:41.238076 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:49:41.238053    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-snwgh\" (UniqueName: \"kubernetes.io/projected/7778d5cf-4af4-4400-8972-7505332d367d-kube-api-access-snwgh\") pod \"auth-test-model-3575e1-kserve-8497c7559f-f24r6\" (UID: \"7778d5cf-4af4-4400-8972-7505332d367d\") " pod="ai-tenant-e2e-shared-a-3575e1/auth-test-model-3575e1-kserve-8497c7559f-f24r6"
Jul 05 00:49:41.322159 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:49:41.322127    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-3575e1/auth-test-model-3575e1-kserve-8497c7559f-f24r6"
Jul 05 00:49:41.448336 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:49:41.448314    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-3575e1/auth-test-model-3575e1-kserve-8497c7559f-f24r6"]
Jul 05 00:49:41.450724 ip-10-0-142-202 kubenswrapper[2563]: W0705 00:49:41.450695    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod7778d5cf_4af4_4400_8972_7505332d367d.slice/crio-b17062a4ab5a85c06d9ffdd3ba1ebd54a708ddca2364a9b5dcdeb092d89667bd WatchSource:0}: Error finding container b17062a4ab5a85c06d9ffdd3ba1ebd54a708ddca2364a9b5dcdeb092d89667bd: Status 404 returned error can't find the container with id b17062a4ab5a85c06d9ffdd3ba1ebd54a708ddca2364a9b5dcdeb092d89667bd
Jul 05 00:49:42.445661 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:49:42.445624    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-3575e1/auth-test-model-3575e1-kserve-8497c7559f-f24r6" event={"ID":"7778d5cf-4af4-4400-8972-7505332d367d","Type":"ContainerStarted","Data":"63ef76166dfb3ae820e35b0b80dc5498cfd8305d7ddf9b42d0861efb4a0e20cc"}
Jul 05 00:49:42.445661 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:49:42.445664    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-3575e1/auth-test-model-3575e1-kserve-8497c7559f-f24r6" event={"ID":"7778d5cf-4af4-4400-8972-7505332d367d","Type":"ContainerStarted","Data":"b17062a4ab5a85c06d9ffdd3ba1ebd54a708ddca2364a9b5dcdeb092d89667bd"}
Jul 05 00:49:42.461171 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:49:42.461128    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-a-3575e1/auth-test-model-3575e1-kserve-8497c7559f-f24r6" podStartSLOduration=2.4611141659999998 podStartE2EDuration="2.461114166s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-05 00:49:40 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-05 00:49:42.458916884 +0000 UTC m=+2430.801245943" watchObservedRunningTime="2026-07-05 00:49:42.461114166 +0000 UTC m=+2430.803443226"
Jul 05 00:49:44.421551 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:49:44.421517    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-x5szm"]
Jul 05 00:49:51.322387 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:49:51.322351    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-a-3575e1/auth-test-model-3575e1-kserve-8497c7559f-f24r6"
Jul 05 00:49:51.322781 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:49:51.322401    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-a-3575e1/auth-test-model-3575e1-kserve-8497c7559f-f24r6"
Jul 05 00:49:51.334858 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:49:51.334831    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-a-3575e1/auth-test-model-3575e1-kserve-8497c7559f-f24r6"
Jul 05 00:49:51.496518 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:49:51.496490    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-a-3575e1/auth-test-model-3575e1-kserve-8497c7559f-f24r6"
Jul 05 00:50:05.305498 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:50:05.305467    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-b-1a1f49/auth-test-model-1a1f49-kserve-8484849d7f-f62cz"]
Jul 05 00:50:05.309026 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:50:05.309006    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-1a1f49/auth-test-model-1a1f49-kserve-8484849d7f-f62cz"
Jul 05 00:50:05.311691 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:50:05.311665    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-1a1f49\"/\"default-dockercfg-txbcj\""
Jul 05 00:50:05.311819 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:50:05.311747    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-1a1f49\"/\"kube-root-ca.crt\""
Jul 05 00:50:05.311895 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:50:05.311692    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-1a1f49\"/\"auth-test-model-1a1f49-kserve-self-signed-certs\""
Jul 05 00:50:05.311976 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:50:05.311686    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-1a1f49\"/\"openshift-service-ca.crt\""
Jul 05 00:50:05.316351 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:50:05.316332    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-1a1f49/auth-test-model-1a1f49-kserve-8484849d7f-f62cz"]
Jul 05 00:50:05.339768 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:50:05.339741    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/71786709-d882-4896-9f9a-606df1a84822-tls-certs\") pod \"auth-test-model-1a1f49-kserve-8484849d7f-f62cz\" (UID: \"71786709-d882-4896-9f9a-606df1a84822\") " pod="ai-tenant-e2e-shared-b-1a1f49/auth-test-model-1a1f49-kserve-8484849d7f-f62cz"
Jul 05 00:50:05.339768 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:50:05.339768    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nbmt9\" (UniqueName: \"kubernetes.io/projected/71786709-d882-4896-9f9a-606df1a84822-kube-api-access-nbmt9\") pod \"auth-test-model-1a1f49-kserve-8484849d7f-f62cz\" (UID: \"71786709-d882-4896-9f9a-606df1a84822\") " pod="ai-tenant-e2e-shared-b-1a1f49/auth-test-model-1a1f49-kserve-8484849d7f-f62cz"
Jul 05 00:50:05.339951 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:50:05.339792    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/71786709-d882-4896-9f9a-606df1a84822-tmp-dir\") pod \"auth-test-model-1a1f49-kserve-8484849d7f-f62cz\" (UID: \"71786709-d882-4896-9f9a-606df1a84822\") " pod="ai-tenant-e2e-shared-b-1a1f49/auth-test-model-1a1f49-kserve-8484849d7f-f62cz"
Jul 05 00:50:05.339951 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:50:05.339873    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/71786709-d882-4896-9f9a-606df1a84822-model-cache\") pod \"auth-test-model-1a1f49-kserve-8484849d7f-f62cz\" (UID: \"71786709-d882-4896-9f9a-606df1a84822\") " pod="ai-tenant-e2e-shared-b-1a1f49/auth-test-model-1a1f49-kserve-8484849d7f-f62cz"
Jul 05 00:50:05.339951 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:50:05.339916    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/71786709-d882-4896-9f9a-606df1a84822-dshm\") pod \"auth-test-model-1a1f49-kserve-8484849d7f-f62cz\" (UID: \"71786709-d882-4896-9f9a-606df1a84822\") " pod="ai-tenant-e2e-shared-b-1a1f49/auth-test-model-1a1f49-kserve-8484849d7f-f62cz"
Jul 05 00:50:05.339951 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:50:05.339935    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/71786709-d882-4896-9f9a-606df1a84822-home\") pod \"auth-test-model-1a1f49-kserve-8484849d7f-f62cz\" (UID: \"71786709-d882-4896-9f9a-606df1a84822\") " pod="ai-tenant-e2e-shared-b-1a1f49/auth-test-model-1a1f49-kserve-8484849d7f-f62cz"
Jul 05 00:50:05.440661 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:50:05.440625    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/71786709-d882-4896-9f9a-606df1a84822-tls-certs\") pod \"auth-test-model-1a1f49-kserve-8484849d7f-f62cz\" (UID: \"71786709-d882-4896-9f9a-606df1a84822\") " pod="ai-tenant-e2e-shared-b-1a1f49/auth-test-model-1a1f49-kserve-8484849d7f-f62cz"
Jul 05 00:50:05.440661 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:50:05.440662    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nbmt9\" (UniqueName: \"kubernetes.io/projected/71786709-d882-4896-9f9a-606df1a84822-kube-api-access-nbmt9\") pod \"auth-test-model-1a1f49-kserve-8484849d7f-f62cz\" (UID: \"71786709-d882-4896-9f9a-606df1a84822\") " pod="ai-tenant-e2e-shared-b-1a1f49/auth-test-model-1a1f49-kserve-8484849d7f-f62cz"
Jul 05 00:50:05.440904 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:50:05.440690    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/71786709-d882-4896-9f9a-606df1a84822-tmp-dir\") pod \"auth-test-model-1a1f49-kserve-8484849d7f-f62cz\" (UID: \"71786709-d882-4896-9f9a-606df1a84822\") " pod="ai-tenant-e2e-shared-b-1a1f49/auth-test-model-1a1f49-kserve-8484849d7f-f62cz"
Jul 05 00:50:05.440904 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:50:05.440723    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/71786709-d882-4896-9f9a-606df1a84822-model-cache\") pod \"auth-test-model-1a1f49-kserve-8484849d7f-f62cz\" (UID: \"71786709-d882-4896-9f9a-606df1a84822\") " pod="ai-tenant-e2e-shared-b-1a1f49/auth-test-model-1a1f49-kserve-8484849d7f-f62cz"
Jul 05 00:50:05.440904 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:50:05.440760    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/71786709-d882-4896-9f9a-606df1a84822-dshm\") pod \"auth-test-model-1a1f49-kserve-8484849d7f-f62cz\" (UID: \"71786709-d882-4896-9f9a-606df1a84822\") " pod="ai-tenant-e2e-shared-b-1a1f49/auth-test-model-1a1f49-kserve-8484849d7f-f62cz"
Jul 05 00:50:05.440904 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:50:05.440777    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/71786709-d882-4896-9f9a-606df1a84822-home\") pod \"auth-test-model-1a1f49-kserve-8484849d7f-f62cz\" (UID: \"71786709-d882-4896-9f9a-606df1a84822\") " pod="ai-tenant-e2e-shared-b-1a1f49/auth-test-model-1a1f49-kserve-8484849d7f-f62cz"
Jul 05 00:50:05.441202 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:50:05.441181    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/71786709-d882-4896-9f9a-606df1a84822-model-cache\") pod \"auth-test-model-1a1f49-kserve-8484849d7f-f62cz\" (UID: \"71786709-d882-4896-9f9a-606df1a84822\") " pod="ai-tenant-e2e-shared-b-1a1f49/auth-test-model-1a1f49-kserve-8484849d7f-f62cz"
Jul 05 00:50:05.441202 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:50:05.441197    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/71786709-d882-4896-9f9a-606df1a84822-home\") pod \"auth-test-model-1a1f49-kserve-8484849d7f-f62cz\" (UID: \"71786709-d882-4896-9f9a-606df1a84822\") " pod="ai-tenant-e2e-shared-b-1a1f49/auth-test-model-1a1f49-kserve-8484849d7f-f62cz"
Jul 05 00:50:05.441354 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:50:05.441291    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/71786709-d882-4896-9f9a-606df1a84822-tmp-dir\") pod \"auth-test-model-1a1f49-kserve-8484849d7f-f62cz\" (UID: \"71786709-d882-4896-9f9a-606df1a84822\") " pod="ai-tenant-e2e-shared-b-1a1f49/auth-test-model-1a1f49-kserve-8484849d7f-f62cz"
Jul 05 00:50:05.443090 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:50:05.443067    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/71786709-d882-4896-9f9a-606df1a84822-dshm\") pod \"auth-test-model-1a1f49-kserve-8484849d7f-f62cz\" (UID: \"71786709-d882-4896-9f9a-606df1a84822\") " pod="ai-tenant-e2e-shared-b-1a1f49/auth-test-model-1a1f49-kserve-8484849d7f-f62cz"
Jul 05 00:50:05.443269 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:50:05.443254    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/71786709-d882-4896-9f9a-606df1a84822-tls-certs\") pod \"auth-test-model-1a1f49-kserve-8484849d7f-f62cz\" (UID: \"71786709-d882-4896-9f9a-606df1a84822\") " pod="ai-tenant-e2e-shared-b-1a1f49/auth-test-model-1a1f49-kserve-8484849d7f-f62cz"
Jul 05 00:50:05.447626 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:50:05.447604    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-nbmt9\" (UniqueName: \"kubernetes.io/projected/71786709-d882-4896-9f9a-606df1a84822-kube-api-access-nbmt9\") pod \"auth-test-model-1a1f49-kserve-8484849d7f-f62cz\" (UID: \"71786709-d882-4896-9f9a-606df1a84822\") " pod="ai-tenant-e2e-shared-b-1a1f49/auth-test-model-1a1f49-kserve-8484849d7f-f62cz"
Jul 05 00:50:05.621471 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:50:05.621374    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-1a1f49/auth-test-model-1a1f49-kserve-8484849d7f-f62cz"
Jul 05 00:50:05.749078 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:50:05.749055    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-1a1f49/auth-test-model-1a1f49-kserve-8484849d7f-f62cz"]
Jul 05 00:50:05.751578 ip-10-0-142-202 kubenswrapper[2563]: W0705 00:50:05.751550    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod71786709_d882_4896_9f9a_606df1a84822.slice/crio-7737aecebe7f0667fd09a8d943c55cbe006bfafd30881d7f921e1d0cc32dc9d0 WatchSource:0}: Error finding container 7737aecebe7f0667fd09a8d943c55cbe006bfafd30881d7f921e1d0cc32dc9d0: Status 404 returned error can't find the container with id 7737aecebe7f0667fd09a8d943c55cbe006bfafd30881d7f921e1d0cc32dc9d0
Jul 05 00:50:06.553332 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:50:06.553297    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-1a1f49/auth-test-model-1a1f49-kserve-8484849d7f-f62cz" event={"ID":"71786709-d882-4896-9f9a-606df1a84822","Type":"ContainerStarted","Data":"90f6ce2205858c95a261e97f191c1192d4753db32ea205894468c36f664ee785"}
Jul 05 00:50:06.553332 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:50:06.553337    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-1a1f49/auth-test-model-1a1f49-kserve-8484849d7f-f62cz" event={"ID":"71786709-d882-4896-9f9a-606df1a84822","Type":"ContainerStarted","Data":"7737aecebe7f0667fd09a8d943c55cbe006bfafd30881d7f921e1d0cc32dc9d0"}
Jul 05 00:50:06.568977 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:50:06.568931    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-b-1a1f49/auth-test-model-1a1f49-kserve-8484849d7f-f62cz" podStartSLOduration=1.568914814 podStartE2EDuration="1.568914814s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-05 00:50:05 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-05 00:50:06.566321932 +0000 UTC m=+2454.908651010" watchObservedRunningTime="2026-07-05 00:50:06.568914814 +0000 UTC m=+2454.911243874"
Jul 05 00:50:08.932793 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:50:08.932761    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-x5szm"]
Jul 05 00:50:15.622138 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:50:15.622103    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-b-1a1f49/auth-test-model-1a1f49-kserve-8484849d7f-f62cz"
Jul 05 00:50:15.622814 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:50:15.622157    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-b-1a1f49/auth-test-model-1a1f49-kserve-8484849d7f-f62cz"
Jul 05 00:50:15.634207 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:50:15.634187    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-b-1a1f49/auth-test-model-1a1f49-kserve-8484849d7f-f62cz"
Jul 05 00:50:16.615598 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:50:16.615573    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-b-1a1f49/auth-test-model-1a1f49-kserve-8484849d7f-f62cz"
Jul 05 00:50:26.027802 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:50:26.027764    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-x5szm"]
Jul 05 00:50:39.927420 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:50:39.927381    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-x5szm"]
Jul 05 00:50:56.521213 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:50:56.521177    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-x5szm"]
Jul 05 00:51:12.127516 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:51:12.127449    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-x5szm"]
Jul 05 00:51:13.141079 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:51:13.141046    2563 scope.go:117] "RemoveContainer" containerID="bc96216c730160bb60f64730a08979ff3c7b14b8d378ba2a59d7098e594f5fca"
Jul 05 00:51:29.021990 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:51:29.021955    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-x5szm"]
Jul 05 00:51:45.121902 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:51:45.121867    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-x5szm"]
Jul 05 00:51:54.459112 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:51:54.459075    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-3575e1/auth-test-model-3575e1-kserve-8497c7559f-f24r6"]
Jul 05 00:51:55.405626 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:51:55.405594    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-1a1f49/auth-test-model-1a1f49-kserve-8484849d7f-f62cz"]
Jul 05 00:52:00.227086 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:52:00.227045    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-x5szm"]
Jul 05 00:52:06.401045 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:52:06.401015    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-a-3575e1/sub-test-model-3575e1-kserve-7c7fc75499-4tw4z"]
Jul 05 00:52:06.404628 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:52:06.404608    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-3575e1/sub-test-model-3575e1-kserve-7c7fc75499-4tw4z"
Jul 05 00:52:06.406640 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:52:06.406616    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-3575e1\"/\"sub-test-model-3575e1-kserve-self-signed-certs\""
Jul 05 00:52:06.413514 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:52:06.413490    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-3575e1/sub-test-model-3575e1-kserve-7c7fc75499-4tw4z"]
Jul 05 00:52:06.505605 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:52:06.505573    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/41febbc1-d93c-4cdc-882d-5f68174b00fe-home\") pod \"sub-test-model-3575e1-kserve-7c7fc75499-4tw4z\" (UID: \"41febbc1-d93c-4cdc-882d-5f68174b00fe\") " pod="ai-tenant-e2e-shared-a-3575e1/sub-test-model-3575e1-kserve-7c7fc75499-4tw4z"
Jul 05 00:52:06.505801 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:52:06.505611    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/41febbc1-d93c-4cdc-882d-5f68174b00fe-model-cache\") pod \"sub-test-model-3575e1-kserve-7c7fc75499-4tw4z\" (UID: \"41febbc1-d93c-4cdc-882d-5f68174b00fe\") " pod="ai-tenant-e2e-shared-a-3575e1/sub-test-model-3575e1-kserve-7c7fc75499-4tw4z"
Jul 05 00:52:06.505801 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:52:06.505642    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/41febbc1-d93c-4cdc-882d-5f68174b00fe-tls-certs\") pod \"sub-test-model-3575e1-kserve-7c7fc75499-4tw4z\" (UID: \"41febbc1-d93c-4cdc-882d-5f68174b00fe\") " pod="ai-tenant-e2e-shared-a-3575e1/sub-test-model-3575e1-kserve-7c7fc75499-4tw4z"
Jul 05 00:52:06.505801 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:52:06.505768    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/41febbc1-d93c-4cdc-882d-5f68174b00fe-tmp-dir\") pod \"sub-test-model-3575e1-kserve-7c7fc75499-4tw4z\" (UID: \"41febbc1-d93c-4cdc-882d-5f68174b00fe\") " pod="ai-tenant-e2e-shared-a-3575e1/sub-test-model-3575e1-kserve-7c7fc75499-4tw4z"
Jul 05 00:52:06.505947 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:52:06.505807    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/41febbc1-d93c-4cdc-882d-5f68174b00fe-dshm\") pod \"sub-test-model-3575e1-kserve-7c7fc75499-4tw4z\" (UID: \"41febbc1-d93c-4cdc-882d-5f68174b00fe\") " pod="ai-tenant-e2e-shared-a-3575e1/sub-test-model-3575e1-kserve-7c7fc75499-4tw4z"
Jul 05 00:52:06.505947 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:52:06.505869    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5rlr6\" (UniqueName: \"kubernetes.io/projected/41febbc1-d93c-4cdc-882d-5f68174b00fe-kube-api-access-5rlr6\") pod \"sub-test-model-3575e1-kserve-7c7fc75499-4tw4z\" (UID: \"41febbc1-d93c-4cdc-882d-5f68174b00fe\") " pod="ai-tenant-e2e-shared-a-3575e1/sub-test-model-3575e1-kserve-7c7fc75499-4tw4z"
Jul 05 00:52:06.606604 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:52:06.606575    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/41febbc1-d93c-4cdc-882d-5f68174b00fe-tls-certs\") pod \"sub-test-model-3575e1-kserve-7c7fc75499-4tw4z\" (UID: \"41febbc1-d93c-4cdc-882d-5f68174b00fe\") " pod="ai-tenant-e2e-shared-a-3575e1/sub-test-model-3575e1-kserve-7c7fc75499-4tw4z"
Jul 05 00:52:06.606776 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:52:06.606649    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/41febbc1-d93c-4cdc-882d-5f68174b00fe-tmp-dir\") pod \"sub-test-model-3575e1-kserve-7c7fc75499-4tw4z\" (UID: \"41febbc1-d93c-4cdc-882d-5f68174b00fe\") " pod="ai-tenant-e2e-shared-a-3575e1/sub-test-model-3575e1-kserve-7c7fc75499-4tw4z"
Jul 05 00:52:06.606776 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:52:06.606677    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/41febbc1-d93c-4cdc-882d-5f68174b00fe-dshm\") pod \"sub-test-model-3575e1-kserve-7c7fc75499-4tw4z\" (UID: \"41febbc1-d93c-4cdc-882d-5f68174b00fe\") " pod="ai-tenant-e2e-shared-a-3575e1/sub-test-model-3575e1-kserve-7c7fc75499-4tw4z"
Jul 05 00:52:06.606776 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:52:06.606704    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5rlr6\" (UniqueName: \"kubernetes.io/projected/41febbc1-d93c-4cdc-882d-5f68174b00fe-kube-api-access-5rlr6\") pod \"sub-test-model-3575e1-kserve-7c7fc75499-4tw4z\" (UID: \"41febbc1-d93c-4cdc-882d-5f68174b00fe\") " pod="ai-tenant-e2e-shared-a-3575e1/sub-test-model-3575e1-kserve-7c7fc75499-4tw4z"
Jul 05 00:52:06.606776 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:52:06.606734    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/41febbc1-d93c-4cdc-882d-5f68174b00fe-home\") pod \"sub-test-model-3575e1-kserve-7c7fc75499-4tw4z\" (UID: \"41febbc1-d93c-4cdc-882d-5f68174b00fe\") " pod="ai-tenant-e2e-shared-a-3575e1/sub-test-model-3575e1-kserve-7c7fc75499-4tw4z"
Jul 05 00:52:06.606985 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:52:06.606866    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/41febbc1-d93c-4cdc-882d-5f68174b00fe-model-cache\") pod \"sub-test-model-3575e1-kserve-7c7fc75499-4tw4z\" (UID: \"41febbc1-d93c-4cdc-882d-5f68174b00fe\") " pod="ai-tenant-e2e-shared-a-3575e1/sub-test-model-3575e1-kserve-7c7fc75499-4tw4z"
Jul 05 00:52:06.607147 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:52:06.607126    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/41febbc1-d93c-4cdc-882d-5f68174b00fe-home\") pod \"sub-test-model-3575e1-kserve-7c7fc75499-4tw4z\" (UID: \"41febbc1-d93c-4cdc-882d-5f68174b00fe\") " pod="ai-tenant-e2e-shared-a-3575e1/sub-test-model-3575e1-kserve-7c7fc75499-4tw4z"
Jul 05 00:52:06.607333 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:52:06.607306    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/41febbc1-d93c-4cdc-882d-5f68174b00fe-model-cache\") pod \"sub-test-model-3575e1-kserve-7c7fc75499-4tw4z\" (UID: \"41febbc1-d93c-4cdc-882d-5f68174b00fe\") " pod="ai-tenant-e2e-shared-a-3575e1/sub-test-model-3575e1-kserve-7c7fc75499-4tw4z"
Jul 05 00:52:06.607333 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:52:06.607317    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/41febbc1-d93c-4cdc-882d-5f68174b00fe-tmp-dir\") pod \"sub-test-model-3575e1-kserve-7c7fc75499-4tw4z\" (UID: \"41febbc1-d93c-4cdc-882d-5f68174b00fe\") " pod="ai-tenant-e2e-shared-a-3575e1/sub-test-model-3575e1-kserve-7c7fc75499-4tw4z"
Jul 05 00:52:06.608946 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:52:06.608921    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/41febbc1-d93c-4cdc-882d-5f68174b00fe-dshm\") pod \"sub-test-model-3575e1-kserve-7c7fc75499-4tw4z\" (UID: \"41febbc1-d93c-4cdc-882d-5f68174b00fe\") " pod="ai-tenant-e2e-shared-a-3575e1/sub-test-model-3575e1-kserve-7c7fc75499-4tw4z"
Jul 05 00:52:06.609320 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:52:06.609301    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/41febbc1-d93c-4cdc-882d-5f68174b00fe-tls-certs\") pod \"sub-test-model-3575e1-kserve-7c7fc75499-4tw4z\" (UID: \"41febbc1-d93c-4cdc-882d-5f68174b00fe\") " pod="ai-tenant-e2e-shared-a-3575e1/sub-test-model-3575e1-kserve-7c7fc75499-4tw4z"
Jul 05 00:52:06.613879 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:52:06.613859    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-5rlr6\" (UniqueName: \"kubernetes.io/projected/41febbc1-d93c-4cdc-882d-5f68174b00fe-kube-api-access-5rlr6\") pod \"sub-test-model-3575e1-kserve-7c7fc75499-4tw4z\" (UID: \"41febbc1-d93c-4cdc-882d-5f68174b00fe\") " pod="ai-tenant-e2e-shared-a-3575e1/sub-test-model-3575e1-kserve-7c7fc75499-4tw4z"
Jul 05 00:52:06.714251 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:52:06.714158    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-3575e1/sub-test-model-3575e1-kserve-7c7fc75499-4tw4z"
Jul 05 00:52:06.844488 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:52:06.844454    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-3575e1/sub-test-model-3575e1-kserve-7c7fc75499-4tw4z"]
Jul 05 00:52:06.845354 ip-10-0-142-202 kubenswrapper[2563]: W0705 00:52:06.845323    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod41febbc1_d93c_4cdc_882d_5f68174b00fe.slice/crio-3f54b30be755b2352991deee86a0c201c54452824351d431269f04e8226b2cb9 WatchSource:0}: Error finding container 3f54b30be755b2352991deee86a0c201c54452824351d431269f04e8226b2cb9: Status 404 returned error can't find the container with id 3f54b30be755b2352991deee86a0c201c54452824351d431269f04e8226b2cb9
Jul 05 00:52:07.105800 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:52:07.105764    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-3575e1/sub-test-model-3575e1-kserve-7c7fc75499-4tw4z" event={"ID":"41febbc1-d93c-4cdc-882d-5f68174b00fe","Type":"ContainerStarted","Data":"e6ea6a059c6c55b890ab05152a559ab436ae7f9a20d121d0a9af7c3de211bfb0"}
Jul 05 00:52:07.105800 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:52:07.105806    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-3575e1/sub-test-model-3575e1-kserve-7c7fc75499-4tw4z" event={"ID":"41febbc1-d93c-4cdc-882d-5f68174b00fe","Type":"ContainerStarted","Data":"3f54b30be755b2352991deee86a0c201c54452824351d431269f04e8226b2cb9"}
Jul 05 00:52:07.120809 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:52:07.120752    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-a-3575e1/sub-test-model-3575e1-kserve-7c7fc75499-4tw4z" podStartSLOduration=1.120732925 podStartE2EDuration="1.120732925s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-05 00:52:06 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-05 00:52:07.120247838 +0000 UTC m=+2575.462576907" watchObservedRunningTime="2026-07-05 00:52:07.120732925 +0000 UTC m=+2575.463062022"
Jul 05 00:52:09.478963 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:52:09.478924    2563 kuberuntime_container.go:872] "Killing container with a grace period" pod="ai-tenant-e2e-shared-a-3575e1/auth-test-model-3575e1-kserve-8497c7559f-f24r6" podUID="7778d5cf-4af4-4400-8972-7505332d367d" containerName="main" containerID="cri-o://63ef76166dfb3ae820e35b0b80dc5498cfd8305d7ddf9b42d0861efb4a0e20cc" gracePeriod=45
Jul 05 00:52:09.724659 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:52:09.724635    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-3575e1/auth-test-model-3575e1-kserve-8497c7559f-f24r6"
Jul 05 00:52:09.837333 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:52:09.837301    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/7778d5cf-4af4-4400-8972-7505332d367d-model-cache\") pod \"7778d5cf-4af4-4400-8972-7505332d367d\" (UID: \"7778d5cf-4af4-4400-8972-7505332d367d\") "
Jul 05 00:52:09.837578 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:52:09.837342    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/7778d5cf-4af4-4400-8972-7505332d367d-dshm\") pod \"7778d5cf-4af4-4400-8972-7505332d367d\" (UID: \"7778d5cf-4af4-4400-8972-7505332d367d\") "
Jul 05 00:52:09.837578 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:52:09.837381    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/7778d5cf-4af4-4400-8972-7505332d367d-tls-certs\") pod \"7778d5cf-4af4-4400-8972-7505332d367d\" (UID: \"7778d5cf-4af4-4400-8972-7505332d367d\") "
Jul 05 00:52:09.837578 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:52:09.837427    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/7778d5cf-4af4-4400-8972-7505332d367d-tmp-dir\") pod \"7778d5cf-4af4-4400-8972-7505332d367d\" (UID: \"7778d5cf-4af4-4400-8972-7505332d367d\") "
Jul 05 00:52:09.837578 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:52:09.837476    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-snwgh\" (UniqueName: \"kubernetes.io/projected/7778d5cf-4af4-4400-8972-7505332d367d-kube-api-access-snwgh\") pod \"7778d5cf-4af4-4400-8972-7505332d367d\" (UID: \"7778d5cf-4af4-4400-8972-7505332d367d\") "
Jul 05 00:52:09.837578 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:52:09.837530    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/7778d5cf-4af4-4400-8972-7505332d367d-home\") pod \"7778d5cf-4af4-4400-8972-7505332d367d\" (UID: \"7778d5cf-4af4-4400-8972-7505332d367d\") "
Jul 05 00:52:09.837807 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:52:09.837635    2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7778d5cf-4af4-4400-8972-7505332d367d-model-cache" (OuterVolumeSpecName: "model-cache") pod "7778d5cf-4af4-4400-8972-7505332d367d" (UID: "7778d5cf-4af4-4400-8972-7505332d367d"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 05 00:52:09.837860 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:52:09.837839    2563 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/7778d5cf-4af4-4400-8972-7505332d367d-model-cache\") on node \"ip-10-0-142-202.ec2.internal\" DevicePath \"\""
Jul 05 00:52:09.838069 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:52:09.838009    2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7778d5cf-4af4-4400-8972-7505332d367d-home" (OuterVolumeSpecName: "home") pod "7778d5cf-4af4-4400-8972-7505332d367d" (UID: "7778d5cf-4af4-4400-8972-7505332d367d"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 05 00:52:09.838069 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:52:09.838015    2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7778d5cf-4af4-4400-8972-7505332d367d-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "7778d5cf-4af4-4400-8972-7505332d367d" (UID: "7778d5cf-4af4-4400-8972-7505332d367d"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 05 00:52:09.839751 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:52:09.839730    2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7778d5cf-4af4-4400-8972-7505332d367d-dshm" (OuterVolumeSpecName: "dshm") pod "7778d5cf-4af4-4400-8972-7505332d367d" (UID: "7778d5cf-4af4-4400-8972-7505332d367d"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 05 00:52:09.839822 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:52:09.839784    2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/7778d5cf-4af4-4400-8972-7505332d367d-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "7778d5cf-4af4-4400-8972-7505332d367d" (UID: "7778d5cf-4af4-4400-8972-7505332d367d"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 05 00:52:09.839822 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:52:09.839795    2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7778d5cf-4af4-4400-8972-7505332d367d-kube-api-access-snwgh" (OuterVolumeSpecName: "kube-api-access-snwgh") pod "7778d5cf-4af4-4400-8972-7505332d367d" (UID: "7778d5cf-4af4-4400-8972-7505332d367d"). InnerVolumeSpecName "kube-api-access-snwgh". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 05 00:52:09.939141 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:52:09.939108    2563 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/7778d5cf-4af4-4400-8972-7505332d367d-dshm\") on node \"ip-10-0-142-202.ec2.internal\" DevicePath \"\""
Jul 05 00:52:09.939141 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:52:09.939134    2563 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/7778d5cf-4af4-4400-8972-7505332d367d-tls-certs\") on node \"ip-10-0-142-202.ec2.internal\" DevicePath \"\""
Jul 05 00:52:09.939141 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:52:09.939144    2563 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/7778d5cf-4af4-4400-8972-7505332d367d-tmp-dir\") on node \"ip-10-0-142-202.ec2.internal\" DevicePath \"\""
Jul 05 00:52:09.939141 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:52:09.939152    2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-snwgh\" (UniqueName: \"kubernetes.io/projected/7778d5cf-4af4-4400-8972-7505332d367d-kube-api-access-snwgh\") on node \"ip-10-0-142-202.ec2.internal\" DevicePath \"\""
Jul 05 00:52:09.939426 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:52:09.939160    2563 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/7778d5cf-4af4-4400-8972-7505332d367d-home\") on node \"ip-10-0-142-202.ec2.internal\" DevicePath \"\""
Jul 05 00:52:10.120965 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:52:10.120868    2563 generic.go:356] "Generic (PLEG): container finished" podID="7778d5cf-4af4-4400-8972-7505332d367d" containerID="63ef76166dfb3ae820e35b0b80dc5498cfd8305d7ddf9b42d0861efb4a0e20cc" exitCode=0
Jul 05 00:52:10.121131 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:52:10.120979    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-3575e1/auth-test-model-3575e1-kserve-8497c7559f-f24r6"
Jul 05 00:52:10.125934 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:52:10.125908    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-3575e1/auth-test-model-3575e1-kserve-8497c7559f-f24r6" event={"ID":"7778d5cf-4af4-4400-8972-7505332d367d","Type":"ContainerDied","Data":"63ef76166dfb3ae820e35b0b80dc5498cfd8305d7ddf9b42d0861efb4a0e20cc"}
Jul 05 00:52:10.126060 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:52:10.125940    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-3575e1/auth-test-model-3575e1-kserve-8497c7559f-f24r6" event={"ID":"7778d5cf-4af4-4400-8972-7505332d367d","Type":"ContainerDied","Data":"b17062a4ab5a85c06d9ffdd3ba1ebd54a708ddca2364a9b5dcdeb092d89667bd"}
Jul 05 00:52:10.126060 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:52:10.125957    2563 scope.go:117] "RemoveContainer" containerID="63ef76166dfb3ae820e35b0b80dc5498cfd8305d7ddf9b42d0861efb4a0e20cc"
Jul 05 00:52:10.142478 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:52:10.142458    2563 scope.go:117] "RemoveContainer" containerID="63ef76166dfb3ae820e35b0b80dc5498cfd8305d7ddf9b42d0861efb4a0e20cc"
Jul 05 00:52:10.142750 ip-10-0-142-202 kubenswrapper[2563]: E0705 00:52:10.142729    2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"63ef76166dfb3ae820e35b0b80dc5498cfd8305d7ddf9b42d0861efb4a0e20cc\": container with ID starting with 63ef76166dfb3ae820e35b0b80dc5498cfd8305d7ddf9b42d0861efb4a0e20cc not found: ID does not exist" containerID="63ef76166dfb3ae820e35b0b80dc5498cfd8305d7ddf9b42d0861efb4a0e20cc"
Jul 05 00:52:10.142818 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:52:10.142762    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"63ef76166dfb3ae820e35b0b80dc5498cfd8305d7ddf9b42d0861efb4a0e20cc"} err="failed to get container status \"63ef76166dfb3ae820e35b0b80dc5498cfd8305d7ddf9b42d0861efb4a0e20cc\": rpc error: code = NotFound desc = could not find container \"63ef76166dfb3ae820e35b0b80dc5498cfd8305d7ddf9b42d0861efb4a0e20cc\": container with ID starting with 63ef76166dfb3ae820e35b0b80dc5498cfd8305d7ddf9b42d0861efb4a0e20cc not found: ID does not exist"
Jul 05 00:52:10.148853 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:52:10.148833    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-3575e1/auth-test-model-3575e1-kserve-8497c7559f-f24r6"]
Jul 05 00:52:10.154097 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:52:10.154076    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-a-3575e1/auth-test-model-3575e1-kserve-8497c7559f-f24r6"]
Jul 05 00:52:10.426387 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:52:10.426279    2563 kuberuntime_container.go:872] "Killing container with a grace period" pod="ai-tenant-e2e-shared-b-1a1f49/auth-test-model-1a1f49-kserve-8484849d7f-f62cz" podUID="71786709-d882-4896-9f9a-606df1a84822" containerName="main" containerID="cri-o://90f6ce2205858c95a261e97f191c1192d4753db32ea205894468c36f664ee785" gracePeriod=45
Jul 05 00:52:10.630395 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:52:10.630367    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-x5szm"]
Jul 05 00:52:10.690114 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:52:10.690056    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-1a1f49/auth-test-model-1a1f49-kserve-8484849d7f-f62cz"
Jul 05 00:52:10.848787 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:52:10.848758    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/71786709-d882-4896-9f9a-606df1a84822-tls-certs\") pod \"71786709-d882-4896-9f9a-606df1a84822\" (UID: \"71786709-d882-4896-9f9a-606df1a84822\") "
Jul 05 00:52:10.848787 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:52:10.848792    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/71786709-d882-4896-9f9a-606df1a84822-home\") pod \"71786709-d882-4896-9f9a-606df1a84822\" (UID: \"71786709-d882-4896-9f9a-606df1a84822\") "
Jul 05 00:52:10.849035 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:52:10.848821    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/71786709-d882-4896-9f9a-606df1a84822-model-cache\") pod \"71786709-d882-4896-9f9a-606df1a84822\" (UID: \"71786709-d882-4896-9f9a-606df1a84822\") "
Jul 05 00:52:10.849035 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:52:10.848844    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/71786709-d882-4896-9f9a-606df1a84822-tmp-dir\") pod \"71786709-d882-4896-9f9a-606df1a84822\" (UID: \"71786709-d882-4896-9f9a-606df1a84822\") "
Jul 05 00:52:10.849035 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:52:10.848908    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/71786709-d882-4896-9f9a-606df1a84822-dshm\") pod \"71786709-d882-4896-9f9a-606df1a84822\" (UID: \"71786709-d882-4896-9f9a-606df1a84822\") "
Jul 05 00:52:10.849035 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:52:10.848949    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-nbmt9\" (UniqueName: \"kubernetes.io/projected/71786709-d882-4896-9f9a-606df1a84822-kube-api-access-nbmt9\") pod \"71786709-d882-4896-9f9a-606df1a84822\" (UID: \"71786709-d882-4896-9f9a-606df1a84822\") "
Jul 05 00:52:10.849269 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:52:10.849088    2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/71786709-d882-4896-9f9a-606df1a84822-home" (OuterVolumeSpecName: "home") pod "71786709-d882-4896-9f9a-606df1a84822" (UID: "71786709-d882-4896-9f9a-606df1a84822"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 05 00:52:10.849269 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:52:10.849098    2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/71786709-d882-4896-9f9a-606df1a84822-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "71786709-d882-4896-9f9a-606df1a84822" (UID: "71786709-d882-4896-9f9a-606df1a84822"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 05 00:52:10.849269 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:52:10.849109    2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/71786709-d882-4896-9f9a-606df1a84822-model-cache" (OuterVolumeSpecName: "model-cache") pod "71786709-d882-4896-9f9a-606df1a84822" (UID: "71786709-d882-4896-9f9a-606df1a84822"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 05 00:52:10.849416 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:52:10.849297    2563 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/71786709-d882-4896-9f9a-606df1a84822-home\") on node \"ip-10-0-142-202.ec2.internal\" DevicePath \"\""
Jul 05 00:52:10.849416 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:52:10.849318    2563 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/71786709-d882-4896-9f9a-606df1a84822-model-cache\") on node \"ip-10-0-142-202.ec2.internal\" DevicePath \"\""
Jul 05 00:52:10.849416 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:52:10.849333    2563 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/71786709-d882-4896-9f9a-606df1a84822-tmp-dir\") on node \"ip-10-0-142-202.ec2.internal\" DevicePath \"\""
Jul 05 00:52:10.851074 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:52:10.851050    2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/71786709-d882-4896-9f9a-606df1a84822-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "71786709-d882-4896-9f9a-606df1a84822" (UID: "71786709-d882-4896-9f9a-606df1a84822"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 05 00:52:10.851193 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:52:10.851090    2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/71786709-d882-4896-9f9a-606df1a84822-dshm" (OuterVolumeSpecName: "dshm") pod "71786709-d882-4896-9f9a-606df1a84822" (UID: "71786709-d882-4896-9f9a-606df1a84822"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 05 00:52:10.851193 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:52:10.851104    2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/71786709-d882-4896-9f9a-606df1a84822-kube-api-access-nbmt9" (OuterVolumeSpecName: "kube-api-access-nbmt9") pod "71786709-d882-4896-9f9a-606df1a84822" (UID: "71786709-d882-4896-9f9a-606df1a84822"). InnerVolumeSpecName "kube-api-access-nbmt9". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 05 00:52:10.950692 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:52:10.950611    2563 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/71786709-d882-4896-9f9a-606df1a84822-tls-certs\") on node \"ip-10-0-142-202.ec2.internal\" DevicePath \"\""
Jul 05 00:52:10.950692 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:52:10.950642    2563 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/71786709-d882-4896-9f9a-606df1a84822-dshm\") on node \"ip-10-0-142-202.ec2.internal\" DevicePath \"\""
Jul 05 00:52:10.950692 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:52:10.950653    2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-nbmt9\" (UniqueName: \"kubernetes.io/projected/71786709-d882-4896-9f9a-606df1a84822-kube-api-access-nbmt9\") on node \"ip-10-0-142-202.ec2.internal\" DevicePath \"\""
Jul 05 00:52:11.129812 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:52:11.129776    2563 generic.go:356] "Generic (PLEG): container finished" podID="71786709-d882-4896-9f9a-606df1a84822" containerID="90f6ce2205858c95a261e97f191c1192d4753db32ea205894468c36f664ee785" exitCode=0
Jul 05 00:52:11.130022 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:52:11.129825    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-1a1f49/auth-test-model-1a1f49-kserve-8484849d7f-f62cz"
Jul 05 00:52:11.130022 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:52:11.129875    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-1a1f49/auth-test-model-1a1f49-kserve-8484849d7f-f62cz" event={"ID":"71786709-d882-4896-9f9a-606df1a84822","Type":"ContainerDied","Data":"90f6ce2205858c95a261e97f191c1192d4753db32ea205894468c36f664ee785"}
Jul 05 00:52:11.130022 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:52:11.129958    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-1a1f49/auth-test-model-1a1f49-kserve-8484849d7f-f62cz" event={"ID":"71786709-d882-4896-9f9a-606df1a84822","Type":"ContainerDied","Data":"7737aecebe7f0667fd09a8d943c55cbe006bfafd30881d7f921e1d0cc32dc9d0"}
Jul 05 00:52:11.130022 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:52:11.129988    2563 scope.go:117] "RemoveContainer" containerID="90f6ce2205858c95a261e97f191c1192d4753db32ea205894468c36f664ee785"
Jul 05 00:52:11.139702 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:52:11.139682    2563 scope.go:117] "RemoveContainer" containerID="90f6ce2205858c95a261e97f191c1192d4753db32ea205894468c36f664ee785"
Jul 05 00:52:11.139973 ip-10-0-142-202 kubenswrapper[2563]: E0705 00:52:11.139954    2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"90f6ce2205858c95a261e97f191c1192d4753db32ea205894468c36f664ee785\": container with ID starting with 90f6ce2205858c95a261e97f191c1192d4753db32ea205894468c36f664ee785 not found: ID does not exist" containerID="90f6ce2205858c95a261e97f191c1192d4753db32ea205894468c36f664ee785"
Jul 05 00:52:11.140036 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:52:11.139980    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"90f6ce2205858c95a261e97f191c1192d4753db32ea205894468c36f664ee785"} err="failed to get container status \"90f6ce2205858c95a261e97f191c1192d4753db32ea205894468c36f664ee785\": rpc error: code = NotFound desc = could not find container \"90f6ce2205858c95a261e97f191c1192d4753db32ea205894468c36f664ee785\": container with ID starting with 90f6ce2205858c95a261e97f191c1192d4753db32ea205894468c36f664ee785 not found: ID does not exist"
Jul 05 00:52:11.151865 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:52:11.151840    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-1a1f49/auth-test-model-1a1f49-kserve-8484849d7f-f62cz"]
Jul 05 00:52:11.155612 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:52:11.155588    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-b-1a1f49/auth-test-model-1a1f49-kserve-8484849d7f-f62cz"]
Jul 05 00:52:12.126624 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:52:12.126591    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="71786709-d882-4896-9f9a-606df1a84822" path="/var/lib/kubelet/pods/71786709-d882-4896-9f9a-606df1a84822/volumes"
Jul 05 00:52:12.126982 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:52:12.126968    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="7778d5cf-4af4-4400-8972-7505332d367d" path="/var/lib/kubelet/pods/7778d5cf-4af4-4400-8972-7505332d367d/volumes"
Jul 05 00:52:13.158749 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:52:13.158720    2563 scope.go:117] "RemoveContainer" containerID="ce34358c677044defac3f93deeb3800c81974f4de28a955eebd08604b1075e92"
Jul 05 00:52:16.714467 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:52:16.714430    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-a-3575e1/sub-test-model-3575e1-kserve-7c7fc75499-4tw4z"
Jul 05 00:52:16.714467 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:52:16.714476    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-a-3575e1/sub-test-model-3575e1-kserve-7c7fc75499-4tw4z"
Jul 05 00:52:16.726940 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:52:16.726916    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-a-3575e1/sub-test-model-3575e1-kserve-7c7fc75499-4tw4z"
Jul 05 00:52:17.168587 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:52:17.168561    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-a-3575e1/sub-test-model-3575e1-kserve-7c7fc75499-4tw4z"
Jul 05 00:52:36.402242 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:52:36.402142    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-b-1a1f49/sub-test-model-1a1f49-kserve-7f9bfb8c55-6bhlv"]
Jul 05 00:52:36.402716 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:52:36.402617    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="71786709-d882-4896-9f9a-606df1a84822" containerName="main"
Jul 05 00:52:36.402716 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:52:36.402635    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="71786709-d882-4896-9f9a-606df1a84822" containerName="main"
Jul 05 00:52:36.402716 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:52:36.402655    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7778d5cf-4af4-4400-8972-7505332d367d" containerName="main"
Jul 05 00:52:36.402716 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:52:36.402664    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="7778d5cf-4af4-4400-8972-7505332d367d" containerName="main"
Jul 05 00:52:36.402936 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:52:36.402763    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="7778d5cf-4af4-4400-8972-7505332d367d" containerName="main"
Jul 05 00:52:36.402936 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:52:36.402775    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="71786709-d882-4896-9f9a-606df1a84822" containerName="main"
Jul 05 00:52:36.405439 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:52:36.405419    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-1a1f49/sub-test-model-1a1f49-kserve-7f9bfb8c55-6bhlv"
Jul 05 00:52:36.407319 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:52:36.407299    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-1a1f49\"/\"default-dockercfg-txbcj\""
Jul 05 00:52:36.407838 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:52:36.407819    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-1a1f49\"/\"kube-root-ca.crt\""
Jul 05 00:52:36.407909 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:52:36.407817    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-1a1f49\"/\"openshift-service-ca.crt\""
Jul 05 00:52:36.407909 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:52:36.407826    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-1a1f49\"/\"sub-test-model-1a1f49-kserve-self-signed-certs\""
Jul 05 00:52:36.414883 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:52:36.414858    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-1a1f49/sub-test-model-1a1f49-kserve-7f9bfb8c55-6bhlv"]
Jul 05 00:52:36.470476 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:52:36.470438    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/fb220d96-0ff0-4e27-81da-ef587e617d56-tmp-dir\") pod \"sub-test-model-1a1f49-kserve-7f9bfb8c55-6bhlv\" (UID: \"fb220d96-0ff0-4e27-81da-ef587e617d56\") " pod="ai-tenant-e2e-shared-b-1a1f49/sub-test-model-1a1f49-kserve-7f9bfb8c55-6bhlv"
Jul 05 00:52:36.470668 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:52:36.470576    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7wh5n\" (UniqueName: \"kubernetes.io/projected/fb220d96-0ff0-4e27-81da-ef587e617d56-kube-api-access-7wh5n\") pod \"sub-test-model-1a1f49-kserve-7f9bfb8c55-6bhlv\" (UID: \"fb220d96-0ff0-4e27-81da-ef587e617d56\") " pod="ai-tenant-e2e-shared-b-1a1f49/sub-test-model-1a1f49-kserve-7f9bfb8c55-6bhlv"
Jul 05 00:52:36.470668 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:52:36.470646    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/fb220d96-0ff0-4e27-81da-ef587e617d56-dshm\") pod \"sub-test-model-1a1f49-kserve-7f9bfb8c55-6bhlv\" (UID: \"fb220d96-0ff0-4e27-81da-ef587e617d56\") " pod="ai-tenant-e2e-shared-b-1a1f49/sub-test-model-1a1f49-kserve-7f9bfb8c55-6bhlv"
Jul 05 00:52:36.470792 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:52:36.470672    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/fb220d96-0ff0-4e27-81da-ef587e617d56-tls-certs\") pod \"sub-test-model-1a1f49-kserve-7f9bfb8c55-6bhlv\" (UID: \"fb220d96-0ff0-4e27-81da-ef587e617d56\") " pod="ai-tenant-e2e-shared-b-1a1f49/sub-test-model-1a1f49-kserve-7f9bfb8c55-6bhlv"
Jul 05 00:52:36.470792 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:52:36.470729    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/fb220d96-0ff0-4e27-81da-ef587e617d56-model-cache\") pod \"sub-test-model-1a1f49-kserve-7f9bfb8c55-6bhlv\" (UID: \"fb220d96-0ff0-4e27-81da-ef587e617d56\") " pod="ai-tenant-e2e-shared-b-1a1f49/sub-test-model-1a1f49-kserve-7f9bfb8c55-6bhlv"
Jul 05 00:52:36.470792 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:52:36.470756    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/fb220d96-0ff0-4e27-81da-ef587e617d56-home\") pod \"sub-test-model-1a1f49-kserve-7f9bfb8c55-6bhlv\" (UID: \"fb220d96-0ff0-4e27-81da-ef587e617d56\") " pod="ai-tenant-e2e-shared-b-1a1f49/sub-test-model-1a1f49-kserve-7f9bfb8c55-6bhlv"
Jul 05 00:52:36.571870 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:52:36.571836    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7wh5n\" (UniqueName: \"kubernetes.io/projected/fb220d96-0ff0-4e27-81da-ef587e617d56-kube-api-access-7wh5n\") pod \"sub-test-model-1a1f49-kserve-7f9bfb8c55-6bhlv\" (UID: \"fb220d96-0ff0-4e27-81da-ef587e617d56\") " pod="ai-tenant-e2e-shared-b-1a1f49/sub-test-model-1a1f49-kserve-7f9bfb8c55-6bhlv"
Jul 05 00:52:36.572055 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:52:36.571886    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/fb220d96-0ff0-4e27-81da-ef587e617d56-dshm\") pod \"sub-test-model-1a1f49-kserve-7f9bfb8c55-6bhlv\" (UID: \"fb220d96-0ff0-4e27-81da-ef587e617d56\") " pod="ai-tenant-e2e-shared-b-1a1f49/sub-test-model-1a1f49-kserve-7f9bfb8c55-6bhlv"
Jul 05 00:52:36.572055 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:52:36.571903    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/fb220d96-0ff0-4e27-81da-ef587e617d56-tls-certs\") pod \"sub-test-model-1a1f49-kserve-7f9bfb8c55-6bhlv\" (UID: \"fb220d96-0ff0-4e27-81da-ef587e617d56\") " pod="ai-tenant-e2e-shared-b-1a1f49/sub-test-model-1a1f49-kserve-7f9bfb8c55-6bhlv"
Jul 05 00:52:36.572055 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:52:36.571973    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/fb220d96-0ff0-4e27-81da-ef587e617d56-model-cache\") pod \"sub-test-model-1a1f49-kserve-7f9bfb8c55-6bhlv\" (UID: \"fb220d96-0ff0-4e27-81da-ef587e617d56\") " pod="ai-tenant-e2e-shared-b-1a1f49/sub-test-model-1a1f49-kserve-7f9bfb8c55-6bhlv"
Jul 05 00:52:36.572055 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:52:36.572003    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/fb220d96-0ff0-4e27-81da-ef587e617d56-home\") pod \"sub-test-model-1a1f49-kserve-7f9bfb8c55-6bhlv\" (UID: \"fb220d96-0ff0-4e27-81da-ef587e617d56\") " pod="ai-tenant-e2e-shared-b-1a1f49/sub-test-model-1a1f49-kserve-7f9bfb8c55-6bhlv"
Jul 05 00:52:36.572055 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:52:36.572029    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/fb220d96-0ff0-4e27-81da-ef587e617d56-tmp-dir\") pod \"sub-test-model-1a1f49-kserve-7f9bfb8c55-6bhlv\" (UID: \"fb220d96-0ff0-4e27-81da-ef587e617d56\") " pod="ai-tenant-e2e-shared-b-1a1f49/sub-test-model-1a1f49-kserve-7f9bfb8c55-6bhlv"
Jul 05 00:52:36.572390 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:52:36.572373    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/fb220d96-0ff0-4e27-81da-ef587e617d56-home\") pod \"sub-test-model-1a1f49-kserve-7f9bfb8c55-6bhlv\" (UID: \"fb220d96-0ff0-4e27-81da-ef587e617d56\") " pod="ai-tenant-e2e-shared-b-1a1f49/sub-test-model-1a1f49-kserve-7f9bfb8c55-6bhlv"
Jul 05 00:52:36.572450 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:52:36.572425    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/fb220d96-0ff0-4e27-81da-ef587e617d56-model-cache\") pod \"sub-test-model-1a1f49-kserve-7f9bfb8c55-6bhlv\" (UID: \"fb220d96-0ff0-4e27-81da-ef587e617d56\") " pod="ai-tenant-e2e-shared-b-1a1f49/sub-test-model-1a1f49-kserve-7f9bfb8c55-6bhlv"
Jul 05 00:52:36.572497 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:52:36.572463    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/fb220d96-0ff0-4e27-81da-ef587e617d56-tmp-dir\") pod \"sub-test-model-1a1f49-kserve-7f9bfb8c55-6bhlv\" (UID: \"fb220d96-0ff0-4e27-81da-ef587e617d56\") " pod="ai-tenant-e2e-shared-b-1a1f49/sub-test-model-1a1f49-kserve-7f9bfb8c55-6bhlv"
Jul 05 00:52:36.574304 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:52:36.574284    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/fb220d96-0ff0-4e27-81da-ef587e617d56-dshm\") pod \"sub-test-model-1a1f49-kserve-7f9bfb8c55-6bhlv\" (UID: \"fb220d96-0ff0-4e27-81da-ef587e617d56\") " pod="ai-tenant-e2e-shared-b-1a1f49/sub-test-model-1a1f49-kserve-7f9bfb8c55-6bhlv"
Jul 05 00:52:36.574578 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:52:36.574562    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/fb220d96-0ff0-4e27-81da-ef587e617d56-tls-certs\") pod \"sub-test-model-1a1f49-kserve-7f9bfb8c55-6bhlv\" (UID: \"fb220d96-0ff0-4e27-81da-ef587e617d56\") " pod="ai-tenant-e2e-shared-b-1a1f49/sub-test-model-1a1f49-kserve-7f9bfb8c55-6bhlv"
Jul 05 00:52:36.579068 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:52:36.579047    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-7wh5n\" (UniqueName: \"kubernetes.io/projected/fb220d96-0ff0-4e27-81da-ef587e617d56-kube-api-access-7wh5n\") pod \"sub-test-model-1a1f49-kserve-7f9bfb8c55-6bhlv\" (UID: \"fb220d96-0ff0-4e27-81da-ef587e617d56\") " pod="ai-tenant-e2e-shared-b-1a1f49/sub-test-model-1a1f49-kserve-7f9bfb8c55-6bhlv"
Jul 05 00:52:36.714945 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:52:36.714858    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-1a1f49/sub-test-model-1a1f49-kserve-7f9bfb8c55-6bhlv"
Jul 05 00:52:36.846005 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:52:36.845973    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-1a1f49/sub-test-model-1a1f49-kserve-7f9bfb8c55-6bhlv"]
Jul 05 00:52:36.847714 ip-10-0-142-202 kubenswrapper[2563]: W0705 00:52:36.847692    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podfb220d96_0ff0_4e27_81da_ef587e617d56.slice/crio-68573bcf5e483c0623ef09dfaadf522f5a3943a46ac5a557e71b1a2398a94194 WatchSource:0}: Error finding container 68573bcf5e483c0623ef09dfaadf522f5a3943a46ac5a557e71b1a2398a94194: Status 404 returned error can't find the container with id 68573bcf5e483c0623ef09dfaadf522f5a3943a46ac5a557e71b1a2398a94194
Jul 05 00:52:37.251483 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:52:37.251447    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-1a1f49/sub-test-model-1a1f49-kserve-7f9bfb8c55-6bhlv" event={"ID":"fb220d96-0ff0-4e27-81da-ef587e617d56","Type":"ContainerStarted","Data":"2a550cfa085f1f78a632bcba7affbd862404c3a306d29aab23c7dc0eb7cd3f19"}
Jul 05 00:52:37.251661 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:52:37.251493    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-1a1f49/sub-test-model-1a1f49-kserve-7f9bfb8c55-6bhlv" event={"ID":"fb220d96-0ff0-4e27-81da-ef587e617d56","Type":"ContainerStarted","Data":"68573bcf5e483c0623ef09dfaadf522f5a3943a46ac5a557e71b1a2398a94194"}
Jul 05 00:52:37.266856 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:52:37.266810    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-b-1a1f49/sub-test-model-1a1f49-kserve-7f9bfb8c55-6bhlv" podStartSLOduration=1.266794819 podStartE2EDuration="1.266794819s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-05 00:52:36 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-05 00:52:37.265014621 +0000 UTC m=+2605.607343670" watchObservedRunningTime="2026-07-05 00:52:37.266794819 +0000 UTC m=+2605.609123878"
Jul 05 00:52:40.031857 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:52:40.031819    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-x5szm"]
Jul 05 00:52:46.715064 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:52:46.715007    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-b-1a1f49/sub-test-model-1a1f49-kserve-7f9bfb8c55-6bhlv"
Jul 05 00:52:46.715064 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:52:46.715072    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-b-1a1f49/sub-test-model-1a1f49-kserve-7f9bfb8c55-6bhlv"
Jul 05 00:52:46.727628 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:52:46.727606    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-b-1a1f49/sub-test-model-1a1f49-kserve-7f9bfb8c55-6bhlv"
Jul 05 00:52:47.310437 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:52:47.310410    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-b-1a1f49/sub-test-model-1a1f49-kserve-7f9bfb8c55-6bhlv"
Jul 05 00:52:58.531542 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:52:58.531506    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-x5szm"]
Jul 05 00:53:16.130506 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:53:16.130479    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-x5szm"]
Jul 05 00:53:22.960139 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:53:22.960103    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-3575e1/sub-test-model-3575e1-kserve-7c7fc75499-4tw4z"]
Jul 05 00:53:23.913658 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:53:23.913625    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-1a1f49/sub-test-model-1a1f49-kserve-7f9bfb8c55-6bhlv"]
Jul 05 00:53:28.011055 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:53:28.011022    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-a-3575e1/rate-test-model-3575e1-kserve-7564d5f659-qzvd5"]
Jul 05 00:53:28.014561 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:53:28.014539    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-3575e1/rate-test-model-3575e1-kserve-7564d5f659-qzvd5"
Jul 05 00:53:28.016425 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:53:28.016406    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-3575e1\"/\"rate-test-model-3575e1-kserve-self-signed-certs\""
Jul 05 00:53:28.022460 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:53:28.022436    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-3575e1/rate-test-model-3575e1-kserve-7564d5f659-qzvd5"]
Jul 05 00:53:28.128002 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:53:28.127974    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-x2xz4\" (UniqueName: \"kubernetes.io/projected/5bdc4747-ff60-43d0-9c17-ba28acdf331d-kube-api-access-x2xz4\") pod \"rate-test-model-3575e1-kserve-7564d5f659-qzvd5\" (UID: \"5bdc4747-ff60-43d0-9c17-ba28acdf331d\") " pod="ai-tenant-e2e-shared-a-3575e1/rate-test-model-3575e1-kserve-7564d5f659-qzvd5"
Jul 05 00:53:28.128142 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:53:28.128014    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/5bdc4747-ff60-43d0-9c17-ba28acdf331d-dshm\") pod \"rate-test-model-3575e1-kserve-7564d5f659-qzvd5\" (UID: \"5bdc4747-ff60-43d0-9c17-ba28acdf331d\") " pod="ai-tenant-e2e-shared-a-3575e1/rate-test-model-3575e1-kserve-7564d5f659-qzvd5"
Jul 05 00:53:28.128142 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:53:28.128033    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/5bdc4747-ff60-43d0-9c17-ba28acdf331d-tmp-dir\") pod \"rate-test-model-3575e1-kserve-7564d5f659-qzvd5\" (UID: \"5bdc4747-ff60-43d0-9c17-ba28acdf331d\") " pod="ai-tenant-e2e-shared-a-3575e1/rate-test-model-3575e1-kserve-7564d5f659-qzvd5"
Jul 05 00:53:28.128142 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:53:28.128104    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/5bdc4747-ff60-43d0-9c17-ba28acdf331d-model-cache\") pod \"rate-test-model-3575e1-kserve-7564d5f659-qzvd5\" (UID: \"5bdc4747-ff60-43d0-9c17-ba28acdf331d\") " pod="ai-tenant-e2e-shared-a-3575e1/rate-test-model-3575e1-kserve-7564d5f659-qzvd5"
Jul 05 00:53:28.128142 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:53:28.128136    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/5bdc4747-ff60-43d0-9c17-ba28acdf331d-tls-certs\") pod \"rate-test-model-3575e1-kserve-7564d5f659-qzvd5\" (UID: \"5bdc4747-ff60-43d0-9c17-ba28acdf331d\") " pod="ai-tenant-e2e-shared-a-3575e1/rate-test-model-3575e1-kserve-7564d5f659-qzvd5"
Jul 05 00:53:28.128301 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:53:28.128208    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/5bdc4747-ff60-43d0-9c17-ba28acdf331d-home\") pod \"rate-test-model-3575e1-kserve-7564d5f659-qzvd5\" (UID: \"5bdc4747-ff60-43d0-9c17-ba28acdf331d\") " pod="ai-tenant-e2e-shared-a-3575e1/rate-test-model-3575e1-kserve-7564d5f659-qzvd5"
Jul 05 00:53:28.229608 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:53:28.229573    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-x2xz4\" (UniqueName: \"kubernetes.io/projected/5bdc4747-ff60-43d0-9c17-ba28acdf331d-kube-api-access-x2xz4\") pod \"rate-test-model-3575e1-kserve-7564d5f659-qzvd5\" (UID: \"5bdc4747-ff60-43d0-9c17-ba28acdf331d\") " pod="ai-tenant-e2e-shared-a-3575e1/rate-test-model-3575e1-kserve-7564d5f659-qzvd5"
Jul 05 00:53:28.229608 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:53:28.229622    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/5bdc4747-ff60-43d0-9c17-ba28acdf331d-dshm\") pod \"rate-test-model-3575e1-kserve-7564d5f659-qzvd5\" (UID: \"5bdc4747-ff60-43d0-9c17-ba28acdf331d\") " pod="ai-tenant-e2e-shared-a-3575e1/rate-test-model-3575e1-kserve-7564d5f659-qzvd5"
Jul 05 00:53:28.229856 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:53:28.229641    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/5bdc4747-ff60-43d0-9c17-ba28acdf331d-tmp-dir\") pod \"rate-test-model-3575e1-kserve-7564d5f659-qzvd5\" (UID: \"5bdc4747-ff60-43d0-9c17-ba28acdf331d\") " pod="ai-tenant-e2e-shared-a-3575e1/rate-test-model-3575e1-kserve-7564d5f659-qzvd5"
Jul 05 00:53:28.229856 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:53:28.229716    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/5bdc4747-ff60-43d0-9c17-ba28acdf331d-model-cache\") pod \"rate-test-model-3575e1-kserve-7564d5f659-qzvd5\" (UID: \"5bdc4747-ff60-43d0-9c17-ba28acdf331d\") " pod="ai-tenant-e2e-shared-a-3575e1/rate-test-model-3575e1-kserve-7564d5f659-qzvd5"
Jul 05 00:53:28.229856 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:53:28.229765    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/5bdc4747-ff60-43d0-9c17-ba28acdf331d-tls-certs\") pod \"rate-test-model-3575e1-kserve-7564d5f659-qzvd5\" (UID: \"5bdc4747-ff60-43d0-9c17-ba28acdf331d\") " pod="ai-tenant-e2e-shared-a-3575e1/rate-test-model-3575e1-kserve-7564d5f659-qzvd5"
Jul 05 00:53:28.229856 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:53:28.229854    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/5bdc4747-ff60-43d0-9c17-ba28acdf331d-home\") pod \"rate-test-model-3575e1-kserve-7564d5f659-qzvd5\" (UID: \"5bdc4747-ff60-43d0-9c17-ba28acdf331d\") " pod="ai-tenant-e2e-shared-a-3575e1/rate-test-model-3575e1-kserve-7564d5f659-qzvd5"
Jul 05 00:53:28.230014 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:53:28.229984    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/5bdc4747-ff60-43d0-9c17-ba28acdf331d-tmp-dir\") pod \"rate-test-model-3575e1-kserve-7564d5f659-qzvd5\" (UID: \"5bdc4747-ff60-43d0-9c17-ba28acdf331d\") " pod="ai-tenant-e2e-shared-a-3575e1/rate-test-model-3575e1-kserve-7564d5f659-qzvd5"
Jul 05 00:53:28.230128 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:53:28.230110    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/5bdc4747-ff60-43d0-9c17-ba28acdf331d-model-cache\") pod \"rate-test-model-3575e1-kserve-7564d5f659-qzvd5\" (UID: \"5bdc4747-ff60-43d0-9c17-ba28acdf331d\") " pod="ai-tenant-e2e-shared-a-3575e1/rate-test-model-3575e1-kserve-7564d5f659-qzvd5"
Jul 05 00:53:28.230234 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:53:28.230205    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/5bdc4747-ff60-43d0-9c17-ba28acdf331d-home\") pod \"rate-test-model-3575e1-kserve-7564d5f659-qzvd5\" (UID: \"5bdc4747-ff60-43d0-9c17-ba28acdf331d\") " pod="ai-tenant-e2e-shared-a-3575e1/rate-test-model-3575e1-kserve-7564d5f659-qzvd5"
Jul 05 00:53:28.232022 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:53:28.232001    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/5bdc4747-ff60-43d0-9c17-ba28acdf331d-dshm\") pod \"rate-test-model-3575e1-kserve-7564d5f659-qzvd5\" (UID: \"5bdc4747-ff60-43d0-9c17-ba28acdf331d\") " pod="ai-tenant-e2e-shared-a-3575e1/rate-test-model-3575e1-kserve-7564d5f659-qzvd5"
Jul 05 00:53:28.232334 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:53:28.232314    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/5bdc4747-ff60-43d0-9c17-ba28acdf331d-tls-certs\") pod \"rate-test-model-3575e1-kserve-7564d5f659-qzvd5\" (UID: \"5bdc4747-ff60-43d0-9c17-ba28acdf331d\") " pod="ai-tenant-e2e-shared-a-3575e1/rate-test-model-3575e1-kserve-7564d5f659-qzvd5"
Jul 05 00:53:28.236877 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:53:28.236858    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-x2xz4\" (UniqueName: \"kubernetes.io/projected/5bdc4747-ff60-43d0-9c17-ba28acdf331d-kube-api-access-x2xz4\") pod \"rate-test-model-3575e1-kserve-7564d5f659-qzvd5\" (UID: \"5bdc4747-ff60-43d0-9c17-ba28acdf331d\") " pod="ai-tenant-e2e-shared-a-3575e1/rate-test-model-3575e1-kserve-7564d5f659-qzvd5"
Jul 05 00:53:28.324718 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:53:28.324682    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-3575e1/rate-test-model-3575e1-kserve-7564d5f659-qzvd5"
Jul 05 00:53:28.454187 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:53:28.454160    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-3575e1/rate-test-model-3575e1-kserve-7564d5f659-qzvd5"]
Jul 05 00:53:28.455947 ip-10-0-142-202 kubenswrapper[2563]: W0705 00:53:28.455919    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod5bdc4747_ff60_43d0_9c17_ba28acdf331d.slice/crio-8bdf6a88bbdec3dd6fa60f0c0a3fc21d96b6ef73b97cac2d9b460a5f5f013d5d WatchSource:0}: Error finding container 8bdf6a88bbdec3dd6fa60f0c0a3fc21d96b6ef73b97cac2d9b460a5f5f013d5d: Status 404 returned error can't find the container with id 8bdf6a88bbdec3dd6fa60f0c0a3fc21d96b6ef73b97cac2d9b460a5f5f013d5d
Jul 05 00:53:28.480547 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:53:28.480515    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-3575e1/rate-test-model-3575e1-kserve-7564d5f659-qzvd5" event={"ID":"5bdc4747-ff60-43d0-9c17-ba28acdf331d","Type":"ContainerStarted","Data":"8bdf6a88bbdec3dd6fa60f0c0a3fc21d96b6ef73b97cac2d9b460a5f5f013d5d"}
Jul 05 00:53:29.486617 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:53:29.486585    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-3575e1/rate-test-model-3575e1-kserve-7564d5f659-qzvd5" event={"ID":"5bdc4747-ff60-43d0-9c17-ba28acdf331d","Type":"ContainerStarted","Data":"3db551e17ac70205613ed050a1b673c2633f27f06b9ae7ed5bd6e2a9006488d7"}
Jul 05 00:53:29.502541 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:53:29.502497    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-a-3575e1/rate-test-model-3575e1-kserve-7564d5f659-qzvd5" podStartSLOduration=2.502482799 podStartE2EDuration="2.502482799s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-05 00:53:27 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-05 00:53:29.500604384 +0000 UTC m=+2657.842933446" watchObservedRunningTime="2026-07-05 00:53:29.502482799 +0000 UTC m=+2657.844811886"
Jul 05 00:53:31.323416 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:53:31.323380    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-x5szm"]
Jul 05 00:53:37.979762 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:53:37.979718    2563 kuberuntime_container.go:872] "Killing container with a grace period" pod="ai-tenant-e2e-shared-a-3575e1/sub-test-model-3575e1-kserve-7c7fc75499-4tw4z" podUID="41febbc1-d93c-4cdc-882d-5f68174b00fe" containerName="main" containerID="cri-o://e6ea6a059c6c55b890ab05152a559ab436ae7f9a20d121d0a9af7c3de211bfb0" gracePeriod=45
Jul 05 00:53:38.232682 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:53:38.232622    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-3575e1/sub-test-model-3575e1-kserve-7c7fc75499-4tw4z"
Jul 05 00:53:38.320076 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:53:38.320045    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/41febbc1-d93c-4cdc-882d-5f68174b00fe-tls-certs\") pod \"41febbc1-d93c-4cdc-882d-5f68174b00fe\" (UID: \"41febbc1-d93c-4cdc-882d-5f68174b00fe\") "
Jul 05 00:53:38.320274 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:53:38.320095    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/41febbc1-d93c-4cdc-882d-5f68174b00fe-tmp-dir\") pod \"41febbc1-d93c-4cdc-882d-5f68174b00fe\" (UID: \"41febbc1-d93c-4cdc-882d-5f68174b00fe\") "
Jul 05 00:53:38.320274 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:53:38.320126    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/41febbc1-d93c-4cdc-882d-5f68174b00fe-dshm\") pod \"41febbc1-d93c-4cdc-882d-5f68174b00fe\" (UID: \"41febbc1-d93c-4cdc-882d-5f68174b00fe\") "
Jul 05 00:53:38.320274 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:53:38.320153    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/41febbc1-d93c-4cdc-882d-5f68174b00fe-model-cache\") pod \"41febbc1-d93c-4cdc-882d-5f68174b00fe\" (UID: \"41febbc1-d93c-4cdc-882d-5f68174b00fe\") "
Jul 05 00:53:38.320274 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:53:38.320211    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/41febbc1-d93c-4cdc-882d-5f68174b00fe-home\") pod \"41febbc1-d93c-4cdc-882d-5f68174b00fe\" (UID: \"41febbc1-d93c-4cdc-882d-5f68174b00fe\") "
Jul 05 00:53:38.320274 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:53:38.320258    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-5rlr6\" (UniqueName: \"kubernetes.io/projected/41febbc1-d93c-4cdc-882d-5f68174b00fe-kube-api-access-5rlr6\") pod \"41febbc1-d93c-4cdc-882d-5f68174b00fe\" (UID: \"41febbc1-d93c-4cdc-882d-5f68174b00fe\") "
Jul 05 00:53:38.320532 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:53:38.320391    2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/41febbc1-d93c-4cdc-882d-5f68174b00fe-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "41febbc1-d93c-4cdc-882d-5f68174b00fe" (UID: "41febbc1-d93c-4cdc-882d-5f68174b00fe"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 05 00:53:38.320532 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:53:38.320487    2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/41febbc1-d93c-4cdc-882d-5f68174b00fe-home" (OuterVolumeSpecName: "home") pod "41febbc1-d93c-4cdc-882d-5f68174b00fe" (UID: "41febbc1-d93c-4cdc-882d-5f68174b00fe"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 05 00:53:38.320532 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:53:38.320496    2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/41febbc1-d93c-4cdc-882d-5f68174b00fe-model-cache" (OuterVolumeSpecName: "model-cache") pod "41febbc1-d93c-4cdc-882d-5f68174b00fe" (UID: "41febbc1-d93c-4cdc-882d-5f68174b00fe"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 05 00:53:38.320693 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:53:38.320622    2563 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/41febbc1-d93c-4cdc-882d-5f68174b00fe-home\") on node \"ip-10-0-142-202.ec2.internal\" DevicePath \"\""
Jul 05 00:53:38.320693 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:53:38.320637    2563 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/41febbc1-d93c-4cdc-882d-5f68174b00fe-tmp-dir\") on node \"ip-10-0-142-202.ec2.internal\" DevicePath \"\""
Jul 05 00:53:38.320693 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:53:38.320647    2563 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/41febbc1-d93c-4cdc-882d-5f68174b00fe-model-cache\") on node \"ip-10-0-142-202.ec2.internal\" DevicePath \"\""
Jul 05 00:53:38.322206 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:53:38.322187    2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/41febbc1-d93c-4cdc-882d-5f68174b00fe-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "41febbc1-d93c-4cdc-882d-5f68174b00fe" (UID: "41febbc1-d93c-4cdc-882d-5f68174b00fe"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 05 00:53:38.322334 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:53:38.322289    2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/41febbc1-d93c-4cdc-882d-5f68174b00fe-dshm" (OuterVolumeSpecName: "dshm") pod "41febbc1-d93c-4cdc-882d-5f68174b00fe" (UID: "41febbc1-d93c-4cdc-882d-5f68174b00fe"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 05 00:53:38.322334 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:53:38.322292    2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/41febbc1-d93c-4cdc-882d-5f68174b00fe-kube-api-access-5rlr6" (OuterVolumeSpecName: "kube-api-access-5rlr6") pod "41febbc1-d93c-4cdc-882d-5f68174b00fe" (UID: "41febbc1-d93c-4cdc-882d-5f68174b00fe"). InnerVolumeSpecName "kube-api-access-5rlr6". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 05 00:53:38.325417 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:53:38.325400    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-a-3575e1/rate-test-model-3575e1-kserve-7564d5f659-qzvd5"
Jul 05 00:53:38.325480 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:53:38.325442    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-a-3575e1/rate-test-model-3575e1-kserve-7564d5f659-qzvd5"
Jul 05 00:53:38.337966 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:53:38.337944    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-a-3575e1/rate-test-model-3575e1-kserve-7564d5f659-qzvd5"
Jul 05 00:53:38.421790 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:53:38.421765    2563 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/41febbc1-d93c-4cdc-882d-5f68174b00fe-tls-certs\") on node \"ip-10-0-142-202.ec2.internal\" DevicePath \"\""
Jul 05 00:53:38.421790 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:53:38.421786    2563 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/41febbc1-d93c-4cdc-882d-5f68174b00fe-dshm\") on node \"ip-10-0-142-202.ec2.internal\" DevicePath \"\""
Jul 05 00:53:38.421790 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:53:38.421795    2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-5rlr6\" (UniqueName: \"kubernetes.io/projected/41febbc1-d93c-4cdc-882d-5f68174b00fe-kube-api-access-5rlr6\") on node \"ip-10-0-142-202.ec2.internal\" DevicePath \"\""
Jul 05 00:53:38.527722 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:53:38.527697    2563 generic.go:356] "Generic (PLEG): container finished" podID="41febbc1-d93c-4cdc-882d-5f68174b00fe" containerID="e6ea6a059c6c55b890ab05152a559ab436ae7f9a20d121d0a9af7c3de211bfb0" exitCode=0
Jul 05 00:53:38.527867 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:53:38.527739    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-3575e1/sub-test-model-3575e1-kserve-7c7fc75499-4tw4z"
Jul 05 00:53:38.527867 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:53:38.527783    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-3575e1/sub-test-model-3575e1-kserve-7c7fc75499-4tw4z" event={"ID":"41febbc1-d93c-4cdc-882d-5f68174b00fe","Type":"ContainerDied","Data":"e6ea6a059c6c55b890ab05152a559ab436ae7f9a20d121d0a9af7c3de211bfb0"}
Jul 05 00:53:38.527867 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:53:38.527824    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-3575e1/sub-test-model-3575e1-kserve-7c7fc75499-4tw4z" event={"ID":"41febbc1-d93c-4cdc-882d-5f68174b00fe","Type":"ContainerDied","Data":"3f54b30be755b2352991deee86a0c201c54452824351d431269f04e8226b2cb9"}
Jul 05 00:53:38.527867 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:53:38.527843    2563 scope.go:117] "RemoveContainer" containerID="e6ea6a059c6c55b890ab05152a559ab436ae7f9a20d121d0a9af7c3de211bfb0"
Jul 05 00:53:38.537380 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:53:38.537327    2563 scope.go:117] "RemoveContainer" containerID="e6ea6a059c6c55b890ab05152a559ab436ae7f9a20d121d0a9af7c3de211bfb0"
Jul 05 00:53:38.537592 ip-10-0-142-202 kubenswrapper[2563]: E0705 00:53:38.537574    2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e6ea6a059c6c55b890ab05152a559ab436ae7f9a20d121d0a9af7c3de211bfb0\": container with ID starting with e6ea6a059c6c55b890ab05152a559ab436ae7f9a20d121d0a9af7c3de211bfb0 not found: ID does not exist" containerID="e6ea6a059c6c55b890ab05152a559ab436ae7f9a20d121d0a9af7c3de211bfb0"
Jul 05 00:53:38.537635 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:53:38.537599    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e6ea6a059c6c55b890ab05152a559ab436ae7f9a20d121d0a9af7c3de211bfb0"} err="failed to get container status \"e6ea6a059c6c55b890ab05152a559ab436ae7f9a20d121d0a9af7c3de211bfb0\": rpc error: code = NotFound desc = could not find container \"e6ea6a059c6c55b890ab05152a559ab436ae7f9a20d121d0a9af7c3de211bfb0\": container with ID starting with e6ea6a059c6c55b890ab05152a559ab436ae7f9a20d121d0a9af7c3de211bfb0 not found: ID does not exist"
Jul 05 00:53:38.540146 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:53:38.540126    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-a-3575e1/rate-test-model-3575e1-kserve-7564d5f659-qzvd5"
Jul 05 00:53:38.548309 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:53:38.548290    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-3575e1/sub-test-model-3575e1-kserve-7c7fc75499-4tw4z"]
Jul 05 00:53:38.550830 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:53:38.550811    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-a-3575e1/sub-test-model-3575e1-kserve-7c7fc75499-4tw4z"]
Jul 05 00:53:38.935603 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:53:38.935515    2563 kuberuntime_container.go:872] "Killing container with a grace period" pod="ai-tenant-e2e-shared-b-1a1f49/sub-test-model-1a1f49-kserve-7f9bfb8c55-6bhlv" podUID="fb220d96-0ff0-4e27-81da-ef587e617d56" containerName="main" containerID="cri-o://2a550cfa085f1f78a632bcba7affbd862404c3a306d29aab23c7dc0eb7cd3f19" gracePeriod=45
Jul 05 00:53:39.184678 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:53:39.184653    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-1a1f49/sub-test-model-1a1f49-kserve-7f9bfb8c55-6bhlv"
Jul 05 00:53:39.327405 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:53:39.327378    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/fb220d96-0ff0-4e27-81da-ef587e617d56-dshm\") pod \"fb220d96-0ff0-4e27-81da-ef587e617d56\" (UID: \"fb220d96-0ff0-4e27-81da-ef587e617d56\") "
Jul 05 00:53:39.327590 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:53:39.327417    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-7wh5n\" (UniqueName: \"kubernetes.io/projected/fb220d96-0ff0-4e27-81da-ef587e617d56-kube-api-access-7wh5n\") pod \"fb220d96-0ff0-4e27-81da-ef587e617d56\" (UID: \"fb220d96-0ff0-4e27-81da-ef587e617d56\") "
Jul 05 00:53:39.327590 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:53:39.327449    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/fb220d96-0ff0-4e27-81da-ef587e617d56-model-cache\") pod \"fb220d96-0ff0-4e27-81da-ef587e617d56\" (UID: \"fb220d96-0ff0-4e27-81da-ef587e617d56\") "
Jul 05 00:53:39.327590 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:53:39.327471    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/fb220d96-0ff0-4e27-81da-ef587e617d56-home\") pod \"fb220d96-0ff0-4e27-81da-ef587e617d56\" (UID: \"fb220d96-0ff0-4e27-81da-ef587e617d56\") "
Jul 05 00:53:39.327590 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:53:39.327546    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/fb220d96-0ff0-4e27-81da-ef587e617d56-tmp-dir\") pod \"fb220d96-0ff0-4e27-81da-ef587e617d56\" (UID: \"fb220d96-0ff0-4e27-81da-ef587e617d56\") "
Jul 05 00:53:39.327590 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:53:39.327569    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/fb220d96-0ff0-4e27-81da-ef587e617d56-tls-certs\") pod \"fb220d96-0ff0-4e27-81da-ef587e617d56\" (UID: \"fb220d96-0ff0-4e27-81da-ef587e617d56\") "
Jul 05 00:53:39.327844 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:53:39.327734    2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fb220d96-0ff0-4e27-81da-ef587e617d56-model-cache" (OuterVolumeSpecName: "model-cache") pod "fb220d96-0ff0-4e27-81da-ef587e617d56" (UID: "fb220d96-0ff0-4e27-81da-ef587e617d56"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 05 00:53:39.327844 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:53:39.327768    2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fb220d96-0ff0-4e27-81da-ef587e617d56-home" (OuterVolumeSpecName: "home") pod "fb220d96-0ff0-4e27-81da-ef587e617d56" (UID: "fb220d96-0ff0-4e27-81da-ef587e617d56"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 05 00:53:39.327844 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:53:39.327800    2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fb220d96-0ff0-4e27-81da-ef587e617d56-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "fb220d96-0ff0-4e27-81da-ef587e617d56" (UID: "fb220d96-0ff0-4e27-81da-ef587e617d56"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 05 00:53:39.329473 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:53:39.329450    2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/fb220d96-0ff0-4e27-81da-ef587e617d56-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "fb220d96-0ff0-4e27-81da-ef587e617d56" (UID: "fb220d96-0ff0-4e27-81da-ef587e617d56"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 05 00:53:39.329583 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:53:39.329538    2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fb220d96-0ff0-4e27-81da-ef587e617d56-dshm" (OuterVolumeSpecName: "dshm") pod "fb220d96-0ff0-4e27-81da-ef587e617d56" (UID: "fb220d96-0ff0-4e27-81da-ef587e617d56"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 05 00:53:39.329583 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:53:39.329546    2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/fb220d96-0ff0-4e27-81da-ef587e617d56-kube-api-access-7wh5n" (OuterVolumeSpecName: "kube-api-access-7wh5n") pod "fb220d96-0ff0-4e27-81da-ef587e617d56" (UID: "fb220d96-0ff0-4e27-81da-ef587e617d56"). InnerVolumeSpecName "kube-api-access-7wh5n". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 05 00:53:39.428659 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:53:39.428622    2563 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/fb220d96-0ff0-4e27-81da-ef587e617d56-tmp-dir\") on node \"ip-10-0-142-202.ec2.internal\" DevicePath \"\""
Jul 05 00:53:39.428659 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:53:39.428657    2563 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/fb220d96-0ff0-4e27-81da-ef587e617d56-tls-certs\") on node \"ip-10-0-142-202.ec2.internal\" DevicePath \"\""
Jul 05 00:53:39.428659 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:53:39.428665    2563 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/fb220d96-0ff0-4e27-81da-ef587e617d56-dshm\") on node \"ip-10-0-142-202.ec2.internal\" DevicePath \"\""
Jul 05 00:53:39.428852 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:53:39.428676    2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-7wh5n\" (UniqueName: \"kubernetes.io/projected/fb220d96-0ff0-4e27-81da-ef587e617d56-kube-api-access-7wh5n\") on node \"ip-10-0-142-202.ec2.internal\" DevicePath \"\""
Jul 05 00:53:39.428852 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:53:39.428685    2563 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/fb220d96-0ff0-4e27-81da-ef587e617d56-model-cache\") on node \"ip-10-0-142-202.ec2.internal\" DevicePath \"\""
Jul 05 00:53:39.428852 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:53:39.428692    2563 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/fb220d96-0ff0-4e27-81da-ef587e617d56-home\") on node \"ip-10-0-142-202.ec2.internal\" DevicePath \"\""
Jul 05 00:53:39.534704 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:53:39.534674    2563 generic.go:356] "Generic (PLEG): container finished" podID="fb220d96-0ff0-4e27-81da-ef587e617d56" containerID="2a550cfa085f1f78a632bcba7affbd862404c3a306d29aab23c7dc0eb7cd3f19" exitCode=0
Jul 05 00:53:39.534861 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:53:39.534736    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-1a1f49/sub-test-model-1a1f49-kserve-7f9bfb8c55-6bhlv"
Jul 05 00:53:39.534861 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:53:39.534757    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-1a1f49/sub-test-model-1a1f49-kserve-7f9bfb8c55-6bhlv" event={"ID":"fb220d96-0ff0-4e27-81da-ef587e617d56","Type":"ContainerDied","Data":"2a550cfa085f1f78a632bcba7affbd862404c3a306d29aab23c7dc0eb7cd3f19"}
Jul 05 00:53:39.534861 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:53:39.534798    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-1a1f49/sub-test-model-1a1f49-kserve-7f9bfb8c55-6bhlv" event={"ID":"fb220d96-0ff0-4e27-81da-ef587e617d56","Type":"ContainerDied","Data":"68573bcf5e483c0623ef09dfaadf522f5a3943a46ac5a557e71b1a2398a94194"}
Jul 05 00:53:39.534861 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:53:39.534815    2563 scope.go:117] "RemoveContainer" containerID="2a550cfa085f1f78a632bcba7affbd862404c3a306d29aab23c7dc0eb7cd3f19"
Jul 05 00:53:39.543569 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:53:39.543552    2563 scope.go:117] "RemoveContainer" containerID="2a550cfa085f1f78a632bcba7affbd862404c3a306d29aab23c7dc0eb7cd3f19"
Jul 05 00:53:39.543803 ip-10-0-142-202 kubenswrapper[2563]: E0705 00:53:39.543786    2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2a550cfa085f1f78a632bcba7affbd862404c3a306d29aab23c7dc0eb7cd3f19\": container with ID starting with 2a550cfa085f1f78a632bcba7affbd862404c3a306d29aab23c7dc0eb7cd3f19 not found: ID does not exist" containerID="2a550cfa085f1f78a632bcba7affbd862404c3a306d29aab23c7dc0eb7cd3f19"
Jul 05 00:53:39.543866 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:53:39.543809    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2a550cfa085f1f78a632bcba7affbd862404c3a306d29aab23c7dc0eb7cd3f19"} err="failed to get container status \"2a550cfa085f1f78a632bcba7affbd862404c3a306d29aab23c7dc0eb7cd3f19\": rpc error: code = NotFound desc = could not find container \"2a550cfa085f1f78a632bcba7affbd862404c3a306d29aab23c7dc0eb7cd3f19\": container with ID starting with 2a550cfa085f1f78a632bcba7affbd862404c3a306d29aab23c7dc0eb7cd3f19 not found: ID does not exist"
Jul 05 00:53:39.553476 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:53:39.553453    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-1a1f49/sub-test-model-1a1f49-kserve-7f9bfb8c55-6bhlv"]
Jul 05 00:53:39.557060 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:53:39.557039    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-b-1a1f49/sub-test-model-1a1f49-kserve-7f9bfb8c55-6bhlv"]
Jul 05 00:53:40.125868 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:53:40.125833    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="41febbc1-d93c-4cdc-882d-5f68174b00fe" path="/var/lib/kubelet/pods/41febbc1-d93c-4cdc-882d-5f68174b00fe/volumes"
Jul 05 00:53:40.126240 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:53:40.126209    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="fb220d96-0ff0-4e27-81da-ef587e617d56" path="/var/lib/kubelet/pods/fb220d96-0ff0-4e27-81da-ef587e617d56/volumes"
Jul 05 00:53:56.794902 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:53:56.794873    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-b-1a1f49/rate-test-model-1a1f49-kserve-745f88fb7f-nb92h"]
Jul 05 00:53:56.795571 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:53:56.795207    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="41febbc1-d93c-4cdc-882d-5f68174b00fe" containerName="main"
Jul 05 00:53:56.795571 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:53:56.795240    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="41febbc1-d93c-4cdc-882d-5f68174b00fe" containerName="main"
Jul 05 00:53:56.795571 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:53:56.795261    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="fb220d96-0ff0-4e27-81da-ef587e617d56" containerName="main"
Jul 05 00:53:56.795571 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:53:56.795266    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="fb220d96-0ff0-4e27-81da-ef587e617d56" containerName="main"
Jul 05 00:53:56.795571 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:53:56.795335    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="41febbc1-d93c-4cdc-882d-5f68174b00fe" containerName="main"
Jul 05 00:53:56.795571 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:53:56.795348    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="fb220d96-0ff0-4e27-81da-ef587e617d56" containerName="main"
Jul 05 00:53:56.798259 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:53:56.798240    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-1a1f49/rate-test-model-1a1f49-kserve-745f88fb7f-nb92h"
Jul 05 00:53:56.801899 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:53:56.801822    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-1a1f49\"/\"rate-test-model-1a1f49-kserve-self-signed-certs\""
Jul 05 00:53:56.801899 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:53:56.801875    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-1a1f49\"/\"kube-root-ca.crt\""
Jul 05 00:53:56.801899 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:53:56.801875    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-1a1f49\"/\"openshift-service-ca.crt\""
Jul 05 00:53:56.802139 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:53:56.801876    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-1a1f49\"/\"default-dockercfg-txbcj\""
Jul 05 00:53:56.810264 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:53:56.810241    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-1a1f49/rate-test-model-1a1f49-kserve-745f88fb7f-nb92h"]
Jul 05 00:53:56.980873 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:53:56.980837    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a48e02c1-be8a-454e-8046-04e6f33638ec-tls-certs\") pod \"rate-test-model-1a1f49-kserve-745f88fb7f-nb92h\" (UID: \"a48e02c1-be8a-454e-8046-04e6f33638ec\") " pod="ai-tenant-e2e-shared-b-1a1f49/rate-test-model-1a1f49-kserve-745f88fb7f-nb92h"
Jul 05 00:53:56.980873 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:53:56.980880    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a48e02c1-be8a-454e-8046-04e6f33638ec-model-cache\") pod \"rate-test-model-1a1f49-kserve-745f88fb7f-nb92h\" (UID: \"a48e02c1-be8a-454e-8046-04e6f33638ec\") " pod="ai-tenant-e2e-shared-b-1a1f49/rate-test-model-1a1f49-kserve-745f88fb7f-nb92h"
Jul 05 00:53:56.981088 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:53:56.980900    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gxwx5\" (UniqueName: \"kubernetes.io/projected/a48e02c1-be8a-454e-8046-04e6f33638ec-kube-api-access-gxwx5\") pod \"rate-test-model-1a1f49-kserve-745f88fb7f-nb92h\" (UID: \"a48e02c1-be8a-454e-8046-04e6f33638ec\") " pod="ai-tenant-e2e-shared-b-1a1f49/rate-test-model-1a1f49-kserve-745f88fb7f-nb92h"
Jul 05 00:53:56.981088 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:53:56.980980    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a48e02c1-be8a-454e-8046-04e6f33638ec-tmp-dir\") pod \"rate-test-model-1a1f49-kserve-745f88fb7f-nb92h\" (UID: \"a48e02c1-be8a-454e-8046-04e6f33638ec\") " pod="ai-tenant-e2e-shared-b-1a1f49/rate-test-model-1a1f49-kserve-745f88fb7f-nb92h"
Jul 05 00:53:56.981161 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:53:56.981098    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a48e02c1-be8a-454e-8046-04e6f33638ec-home\") pod \"rate-test-model-1a1f49-kserve-745f88fb7f-nb92h\" (UID: \"a48e02c1-be8a-454e-8046-04e6f33638ec\") " pod="ai-tenant-e2e-shared-b-1a1f49/rate-test-model-1a1f49-kserve-745f88fb7f-nb92h"
Jul 05 00:53:56.981161 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:53:56.981146    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a48e02c1-be8a-454e-8046-04e6f33638ec-dshm\") pod \"rate-test-model-1a1f49-kserve-745f88fb7f-nb92h\" (UID: \"a48e02c1-be8a-454e-8046-04e6f33638ec\") " pod="ai-tenant-e2e-shared-b-1a1f49/rate-test-model-1a1f49-kserve-745f88fb7f-nb92h"
Jul 05 00:53:57.082404 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:53:57.082313    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a48e02c1-be8a-454e-8046-04e6f33638ec-tls-certs\") pod \"rate-test-model-1a1f49-kserve-745f88fb7f-nb92h\" (UID: \"a48e02c1-be8a-454e-8046-04e6f33638ec\") " pod="ai-tenant-e2e-shared-b-1a1f49/rate-test-model-1a1f49-kserve-745f88fb7f-nb92h"
Jul 05 00:53:57.082576 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:53:57.082409    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a48e02c1-be8a-454e-8046-04e6f33638ec-model-cache\") pod \"rate-test-model-1a1f49-kserve-745f88fb7f-nb92h\" (UID: \"a48e02c1-be8a-454e-8046-04e6f33638ec\") " pod="ai-tenant-e2e-shared-b-1a1f49/rate-test-model-1a1f49-kserve-745f88fb7f-nb92h"
Jul 05 00:53:57.082576 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:53:57.082441    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gxwx5\" (UniqueName: \"kubernetes.io/projected/a48e02c1-be8a-454e-8046-04e6f33638ec-kube-api-access-gxwx5\") pod \"rate-test-model-1a1f49-kserve-745f88fb7f-nb92h\" (UID: \"a48e02c1-be8a-454e-8046-04e6f33638ec\") " pod="ai-tenant-e2e-shared-b-1a1f49/rate-test-model-1a1f49-kserve-745f88fb7f-nb92h"
Jul 05 00:53:57.082576 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:53:57.082561    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a48e02c1-be8a-454e-8046-04e6f33638ec-tmp-dir\") pod \"rate-test-model-1a1f49-kserve-745f88fb7f-nb92h\" (UID: \"a48e02c1-be8a-454e-8046-04e6f33638ec\") " pod="ai-tenant-e2e-shared-b-1a1f49/rate-test-model-1a1f49-kserve-745f88fb7f-nb92h"
Jul 05 00:53:57.082754 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:53:57.082674    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a48e02c1-be8a-454e-8046-04e6f33638ec-home\") pod \"rate-test-model-1a1f49-kserve-745f88fb7f-nb92h\" (UID: \"a48e02c1-be8a-454e-8046-04e6f33638ec\") " pod="ai-tenant-e2e-shared-b-1a1f49/rate-test-model-1a1f49-kserve-745f88fb7f-nb92h"
Jul 05 00:53:57.082754 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:53:57.082718    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a48e02c1-be8a-454e-8046-04e6f33638ec-dshm\") pod \"rate-test-model-1a1f49-kserve-745f88fb7f-nb92h\" (UID: \"a48e02c1-be8a-454e-8046-04e6f33638ec\") " pod="ai-tenant-e2e-shared-b-1a1f49/rate-test-model-1a1f49-kserve-745f88fb7f-nb92h"
Jul 05 00:53:57.082890 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:53:57.082875    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a48e02c1-be8a-454e-8046-04e6f33638ec-model-cache\") pod \"rate-test-model-1a1f49-kserve-745f88fb7f-nb92h\" (UID: \"a48e02c1-be8a-454e-8046-04e6f33638ec\") " pod="ai-tenant-e2e-shared-b-1a1f49/rate-test-model-1a1f49-kserve-745f88fb7f-nb92h"
Jul 05 00:53:57.082953 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:53:57.082929    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a48e02c1-be8a-454e-8046-04e6f33638ec-tmp-dir\") pod \"rate-test-model-1a1f49-kserve-745f88fb7f-nb92h\" (UID: \"a48e02c1-be8a-454e-8046-04e6f33638ec\") " pod="ai-tenant-e2e-shared-b-1a1f49/rate-test-model-1a1f49-kserve-745f88fb7f-nb92h"
Jul 05 00:53:57.083019 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:53:57.082975    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a48e02c1-be8a-454e-8046-04e6f33638ec-home\") pod \"rate-test-model-1a1f49-kserve-745f88fb7f-nb92h\" (UID: \"a48e02c1-be8a-454e-8046-04e6f33638ec\") " pod="ai-tenant-e2e-shared-b-1a1f49/rate-test-model-1a1f49-kserve-745f88fb7f-nb92h"
Jul 05 00:53:57.084804 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:53:57.084773    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a48e02c1-be8a-454e-8046-04e6f33638ec-dshm\") pod \"rate-test-model-1a1f49-kserve-745f88fb7f-nb92h\" (UID: \"a48e02c1-be8a-454e-8046-04e6f33638ec\") " pod="ai-tenant-e2e-shared-b-1a1f49/rate-test-model-1a1f49-kserve-745f88fb7f-nb92h"
Jul 05 00:53:57.084943 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:53:57.084905    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a48e02c1-be8a-454e-8046-04e6f33638ec-tls-certs\") pod \"rate-test-model-1a1f49-kserve-745f88fb7f-nb92h\" (UID: \"a48e02c1-be8a-454e-8046-04e6f33638ec\") " pod="ai-tenant-e2e-shared-b-1a1f49/rate-test-model-1a1f49-kserve-745f88fb7f-nb92h"
Jul 05 00:53:57.089454 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:53:57.089431    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-gxwx5\" (UniqueName: \"kubernetes.io/projected/a48e02c1-be8a-454e-8046-04e6f33638ec-kube-api-access-gxwx5\") pod \"rate-test-model-1a1f49-kserve-745f88fb7f-nb92h\" (UID: \"a48e02c1-be8a-454e-8046-04e6f33638ec\") " pod="ai-tenant-e2e-shared-b-1a1f49/rate-test-model-1a1f49-kserve-745f88fb7f-nb92h"
Jul 05 00:53:57.111238 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:53:57.111198    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-1a1f49/rate-test-model-1a1f49-kserve-745f88fb7f-nb92h"
Jul 05 00:53:57.240579 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:53:57.240552    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-1a1f49/rate-test-model-1a1f49-kserve-745f88fb7f-nb92h"]
Jul 05 00:53:57.242474 ip-10-0-142-202 kubenswrapper[2563]: W0705 00:53:57.242444    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-poda48e02c1_be8a_454e_8046_04e6f33638ec.slice/crio-b5d302481f10417d29d9cc73f36474291cefd59efb31e86a1606c49609857bcb WatchSource:0}: Error finding container b5d302481f10417d29d9cc73f36474291cefd59efb31e86a1606c49609857bcb: Status 404 returned error can't find the container with id b5d302481f10417d29d9cc73f36474291cefd59efb31e86a1606c49609857bcb
Jul 05 00:53:57.625007 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:53:57.624968    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-1a1f49/rate-test-model-1a1f49-kserve-745f88fb7f-nb92h" event={"ID":"a48e02c1-be8a-454e-8046-04e6f33638ec","Type":"ContainerStarted","Data":"c3bb78c37c700e2431f0a17f483fd547af4577624c702187e0b2b94a123fb42d"}
Jul 05 00:53:57.625204 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:53:57.625014    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-1a1f49/rate-test-model-1a1f49-kserve-745f88fb7f-nb92h" event={"ID":"a48e02c1-be8a-454e-8046-04e6f33638ec","Type":"ContainerStarted","Data":"b5d302481f10417d29d9cc73f36474291cefd59efb31e86a1606c49609857bcb"}
Jul 05 00:53:57.640060 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:53:57.640016    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-b-1a1f49/rate-test-model-1a1f49-kserve-745f88fb7f-nb92h" podStartSLOduration=1.640002204 podStartE2EDuration="1.640002204s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-05 00:53:56 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-05 00:53:57.638906572 +0000 UTC m=+2685.981235632" watchObservedRunningTime="2026-07-05 00:53:57.640002204 +0000 UTC m=+2685.982331264"
Jul 05 00:54:01.131437 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:54:01.131409    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-x5szm"]
Jul 05 00:54:07.111493 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:54:07.111457    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-b-1a1f49/rate-test-model-1a1f49-kserve-745f88fb7f-nb92h"
Jul 05 00:54:07.113536 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:54:07.111528    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-b-1a1f49/rate-test-model-1a1f49-kserve-745f88fb7f-nb92h"
Jul 05 00:54:07.124357 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:54:07.124327    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-b-1a1f49/rate-test-model-1a1f49-kserve-745f88fb7f-nb92h"
Jul 05 00:54:07.681321 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:54:07.681292    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-b-1a1f49/rate-test-model-1a1f49-kserve-745f88fb7f-nb92h"
Jul 05 00:54:13.110102 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:54:13.110074    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-7sxq4_b96bfb4d-2fd3-4be3-9064-e24efeb904f4/ovn-acl-logging/0.log"
Jul 05 00:54:13.115548 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:54:13.115529    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-7sxq4_b96bfb4d-2fd3-4be3-9064-e24efeb904f4/ovn-acl-logging/0.log"
Jul 05 00:54:19.436349 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:54:19.436317    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-x5szm"]
Jul 05 00:54:33.420519 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:54:33.420483    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-x5szm"]
Jul 05 00:54:40.489640 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:54:40.489603    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-3575e1/rate-test-model-3575e1-kserve-7564d5f659-qzvd5"]
Jul 05 00:54:41.295153 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:54:41.295120    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-1a1f49/rate-test-model-1a1f49-kserve-745f88fb7f-nb92h"]
Jul 05 00:54:45.627764 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:54:45.627728    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-x5szm"]
Jul 05 00:54:55.509658 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:54:55.509594    2563 kuberuntime_container.go:872] "Killing container with a grace period" pod="ai-tenant-e2e-shared-a-3575e1/rate-test-model-3575e1-kserve-7564d5f659-qzvd5" podUID="5bdc4747-ff60-43d0-9c17-ba28acdf331d" containerName="main" containerID="cri-o://3db551e17ac70205613ed050a1b673c2633f27f06b9ae7ed5bd6e2a9006488d7" gracePeriod=45
Jul 05 00:54:55.625986 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:54:55.625958    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-x5szm"]
Jul 05 00:54:55.756131 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:54:55.756110    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-3575e1/rate-test-model-3575e1-kserve-7564d5f659-qzvd5"
Jul 05 00:54:55.866239 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:54:55.866187    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-x2xz4\" (UniqueName: \"kubernetes.io/projected/5bdc4747-ff60-43d0-9c17-ba28acdf331d-kube-api-access-x2xz4\") pod \"5bdc4747-ff60-43d0-9c17-ba28acdf331d\" (UID: \"5bdc4747-ff60-43d0-9c17-ba28acdf331d\") "
Jul 05 00:54:55.866426 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:54:55.866250    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/5bdc4747-ff60-43d0-9c17-ba28acdf331d-tls-certs\") pod \"5bdc4747-ff60-43d0-9c17-ba28acdf331d\" (UID: \"5bdc4747-ff60-43d0-9c17-ba28acdf331d\") "
Jul 05 00:54:55.866426 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:54:55.866280    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/5bdc4747-ff60-43d0-9c17-ba28acdf331d-tmp-dir\") pod \"5bdc4747-ff60-43d0-9c17-ba28acdf331d\" (UID: \"5bdc4747-ff60-43d0-9c17-ba28acdf331d\") "
Jul 05 00:54:55.866426 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:54:55.866341    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/5bdc4747-ff60-43d0-9c17-ba28acdf331d-model-cache\") pod \"5bdc4747-ff60-43d0-9c17-ba28acdf331d\" (UID: \"5bdc4747-ff60-43d0-9c17-ba28acdf331d\") "
Jul 05 00:54:55.866426 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:54:55.866402    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/5bdc4747-ff60-43d0-9c17-ba28acdf331d-home\") pod \"5bdc4747-ff60-43d0-9c17-ba28acdf331d\" (UID: \"5bdc4747-ff60-43d0-9c17-ba28acdf331d\") "
Jul 05 00:54:55.866626 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:54:55.866427    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/5bdc4747-ff60-43d0-9c17-ba28acdf331d-dshm\") pod \"5bdc4747-ff60-43d0-9c17-ba28acdf331d\" (UID: \"5bdc4747-ff60-43d0-9c17-ba28acdf331d\") "
Jul 05 00:54:55.866626 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:54:55.866565    2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5bdc4747-ff60-43d0-9c17-ba28acdf331d-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "5bdc4747-ff60-43d0-9c17-ba28acdf331d" (UID: "5bdc4747-ff60-43d0-9c17-ba28acdf331d"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 05 00:54:55.866714 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:54:55.866635    2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5bdc4747-ff60-43d0-9c17-ba28acdf331d-model-cache" (OuterVolumeSpecName: "model-cache") pod "5bdc4747-ff60-43d0-9c17-ba28acdf331d" (UID: "5bdc4747-ff60-43d0-9c17-ba28acdf331d"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 05 00:54:55.866714 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:54:55.866644    2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5bdc4747-ff60-43d0-9c17-ba28acdf331d-home" (OuterVolumeSpecName: "home") pod "5bdc4747-ff60-43d0-9c17-ba28acdf331d" (UID: "5bdc4747-ff60-43d0-9c17-ba28acdf331d"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 05 00:54:55.866810 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:54:55.866752    2563 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/5bdc4747-ff60-43d0-9c17-ba28acdf331d-home\") on node \"ip-10-0-142-202.ec2.internal\" DevicePath \"\""
Jul 05 00:54:55.866810 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:54:55.866764    2563 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/5bdc4747-ff60-43d0-9c17-ba28acdf331d-tmp-dir\") on node \"ip-10-0-142-202.ec2.internal\" DevicePath \"\""
Jul 05 00:54:55.866810 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:54:55.866777    2563 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/5bdc4747-ff60-43d0-9c17-ba28acdf331d-model-cache\") on node \"ip-10-0-142-202.ec2.internal\" DevicePath \"\""
Jul 05 00:54:55.868452 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:54:55.868424    2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5bdc4747-ff60-43d0-9c17-ba28acdf331d-dshm" (OuterVolumeSpecName: "dshm") pod "5bdc4747-ff60-43d0-9c17-ba28acdf331d" (UID: "5bdc4747-ff60-43d0-9c17-ba28acdf331d"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 05 00:54:55.868452 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:54:55.868427    2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5bdc4747-ff60-43d0-9c17-ba28acdf331d-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "5bdc4747-ff60-43d0-9c17-ba28acdf331d" (UID: "5bdc4747-ff60-43d0-9c17-ba28acdf331d"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 05 00:54:55.868595 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:54:55.868474    2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5bdc4747-ff60-43d0-9c17-ba28acdf331d-kube-api-access-x2xz4" (OuterVolumeSpecName: "kube-api-access-x2xz4") pod "5bdc4747-ff60-43d0-9c17-ba28acdf331d" (UID: "5bdc4747-ff60-43d0-9c17-ba28acdf331d"). InnerVolumeSpecName "kube-api-access-x2xz4". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 05 00:54:55.892652 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:54:55.892624    2563 generic.go:356] "Generic (PLEG): container finished" podID="5bdc4747-ff60-43d0-9c17-ba28acdf331d" containerID="3db551e17ac70205613ed050a1b673c2633f27f06b9ae7ed5bd6e2a9006488d7" exitCode=0
Jul 05 00:54:55.892780 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:54:55.892669    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-3575e1/rate-test-model-3575e1-kserve-7564d5f659-qzvd5"
Jul 05 00:54:55.892780 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:54:55.892710    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-3575e1/rate-test-model-3575e1-kserve-7564d5f659-qzvd5" event={"ID":"5bdc4747-ff60-43d0-9c17-ba28acdf331d","Type":"ContainerDied","Data":"3db551e17ac70205613ed050a1b673c2633f27f06b9ae7ed5bd6e2a9006488d7"}
Jul 05 00:54:55.892780 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:54:55.892759    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-3575e1/rate-test-model-3575e1-kserve-7564d5f659-qzvd5" event={"ID":"5bdc4747-ff60-43d0-9c17-ba28acdf331d","Type":"ContainerDied","Data":"8bdf6a88bbdec3dd6fa60f0c0a3fc21d96b6ef73b97cac2d9b460a5f5f013d5d"}
Jul 05 00:54:55.892904 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:54:55.892781    2563 scope.go:117] "RemoveContainer" containerID="3db551e17ac70205613ed050a1b673c2633f27f06b9ae7ed5bd6e2a9006488d7"
Jul 05 00:54:55.901506 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:54:55.901490    2563 scope.go:117] "RemoveContainer" containerID="3db551e17ac70205613ed050a1b673c2633f27f06b9ae7ed5bd6e2a9006488d7"
Jul 05 00:54:55.901753 ip-10-0-142-202 kubenswrapper[2563]: E0705 00:54:55.901735    2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3db551e17ac70205613ed050a1b673c2633f27f06b9ae7ed5bd6e2a9006488d7\": container with ID starting with 3db551e17ac70205613ed050a1b673c2633f27f06b9ae7ed5bd6e2a9006488d7 not found: ID does not exist" containerID="3db551e17ac70205613ed050a1b673c2633f27f06b9ae7ed5bd6e2a9006488d7"
Jul 05 00:54:55.901814 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:54:55.901759    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3db551e17ac70205613ed050a1b673c2633f27f06b9ae7ed5bd6e2a9006488d7"} err="failed to get container status \"3db551e17ac70205613ed050a1b673c2633f27f06b9ae7ed5bd6e2a9006488d7\": rpc error: code = NotFound desc = could not find container \"3db551e17ac70205613ed050a1b673c2633f27f06b9ae7ed5bd6e2a9006488d7\": container with ID starting with 3db551e17ac70205613ed050a1b673c2633f27f06b9ae7ed5bd6e2a9006488d7 not found: ID does not exist"
Jul 05 00:54:55.911898 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:54:55.911872    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-3575e1/rate-test-model-3575e1-kserve-7564d5f659-qzvd5"]
Jul 05 00:54:55.914011 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:54:55.913991    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-a-3575e1/rate-test-model-3575e1-kserve-7564d5f659-qzvd5"]
Jul 05 00:54:55.967658 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:54:55.967637    2563 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/5bdc4747-ff60-43d0-9c17-ba28acdf331d-dshm\") on node \"ip-10-0-142-202.ec2.internal\" DevicePath \"\""
Jul 05 00:54:55.967658 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:54:55.967661    2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-x2xz4\" (UniqueName: \"kubernetes.io/projected/5bdc4747-ff60-43d0-9c17-ba28acdf331d-kube-api-access-x2xz4\") on node \"ip-10-0-142-202.ec2.internal\" DevicePath \"\""
Jul 05 00:54:55.967807 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:54:55.967670    2563 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/5bdc4747-ff60-43d0-9c17-ba28acdf331d-tls-certs\") on node \"ip-10-0-142-202.ec2.internal\" DevicePath \"\""
Jul 05 00:54:56.129686 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:54:56.128253    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="5bdc4747-ff60-43d0-9c17-ba28acdf331d" path="/var/lib/kubelet/pods/5bdc4747-ff60-43d0-9c17-ba28acdf331d/volumes"
Jul 05 00:54:56.315011 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:54:56.314948    2563 kuberuntime_container.go:872] "Killing container with a grace period" pod="ai-tenant-e2e-shared-b-1a1f49/rate-test-model-1a1f49-kserve-745f88fb7f-nb92h" podUID="a48e02c1-be8a-454e-8046-04e6f33638ec" containerName="main" containerID="cri-o://c3bb78c37c700e2431f0a17f483fd547af4577624c702187e0b2b94a123fb42d" gracePeriod=45
Jul 05 00:54:56.566666 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:54:56.566639    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-1a1f49/rate-test-model-1a1f49-kserve-745f88fb7f-nb92h"
Jul 05 00:54:56.673790 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:54:56.673759    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-gxwx5\" (UniqueName: \"kubernetes.io/projected/a48e02c1-be8a-454e-8046-04e6f33638ec-kube-api-access-gxwx5\") pod \"a48e02c1-be8a-454e-8046-04e6f33638ec\" (UID: \"a48e02c1-be8a-454e-8046-04e6f33638ec\") "
Jul 05 00:54:56.673969 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:54:56.673842    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a48e02c1-be8a-454e-8046-04e6f33638ec-home\") pod \"a48e02c1-be8a-454e-8046-04e6f33638ec\" (UID: \"a48e02c1-be8a-454e-8046-04e6f33638ec\") "
Jul 05 00:54:56.673969 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:54:56.673862    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a48e02c1-be8a-454e-8046-04e6f33638ec-tmp-dir\") pod \"a48e02c1-be8a-454e-8046-04e6f33638ec\" (UID: \"a48e02c1-be8a-454e-8046-04e6f33638ec\") "
Jul 05 00:54:56.673969 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:54:56.673901    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a48e02c1-be8a-454e-8046-04e6f33638ec-model-cache\") pod \"a48e02c1-be8a-454e-8046-04e6f33638ec\" (UID: \"a48e02c1-be8a-454e-8046-04e6f33638ec\") "
Jul 05 00:54:56.673969 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:54:56.673923    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a48e02c1-be8a-454e-8046-04e6f33638ec-dshm\") pod \"a48e02c1-be8a-454e-8046-04e6f33638ec\" (UID: \"a48e02c1-be8a-454e-8046-04e6f33638ec\") "
Jul 05 00:54:56.673969 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:54:56.673954    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a48e02c1-be8a-454e-8046-04e6f33638ec-tls-certs\") pod \"a48e02c1-be8a-454e-8046-04e6f33638ec\" (UID: \"a48e02c1-be8a-454e-8046-04e6f33638ec\") "
Jul 05 00:54:56.674247 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:54:56.674156    2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a48e02c1-be8a-454e-8046-04e6f33638ec-home" (OuterVolumeSpecName: "home") pod "a48e02c1-be8a-454e-8046-04e6f33638ec" (UID: "a48e02c1-be8a-454e-8046-04e6f33638ec"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 05 00:54:56.674247 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:54:56.674163    2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a48e02c1-be8a-454e-8046-04e6f33638ec-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "a48e02c1-be8a-454e-8046-04e6f33638ec" (UID: "a48e02c1-be8a-454e-8046-04e6f33638ec"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 05 00:54:56.674247 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:54:56.674179    2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a48e02c1-be8a-454e-8046-04e6f33638ec-model-cache" (OuterVolumeSpecName: "model-cache") pod "a48e02c1-be8a-454e-8046-04e6f33638ec" (UID: "a48e02c1-be8a-454e-8046-04e6f33638ec"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 05 00:54:56.675970 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:54:56.675941    2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a48e02c1-be8a-454e-8046-04e6f33638ec-dshm" (OuterVolumeSpecName: "dshm") pod "a48e02c1-be8a-454e-8046-04e6f33638ec" (UID: "a48e02c1-be8a-454e-8046-04e6f33638ec"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 05 00:54:56.675970 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:54:56.675954    2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a48e02c1-be8a-454e-8046-04e6f33638ec-kube-api-access-gxwx5" (OuterVolumeSpecName: "kube-api-access-gxwx5") pod "a48e02c1-be8a-454e-8046-04e6f33638ec" (UID: "a48e02c1-be8a-454e-8046-04e6f33638ec"). InnerVolumeSpecName "kube-api-access-gxwx5". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 05 00:54:56.676112 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:54:56.676023    2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a48e02c1-be8a-454e-8046-04e6f33638ec-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "a48e02c1-be8a-454e-8046-04e6f33638ec" (UID: "a48e02c1-be8a-454e-8046-04e6f33638ec"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 05 00:54:56.774819 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:54:56.774741    2563 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a48e02c1-be8a-454e-8046-04e6f33638ec-model-cache\") on node \"ip-10-0-142-202.ec2.internal\" DevicePath \"\""
Jul 05 00:54:56.774819 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:54:56.774766    2563 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a48e02c1-be8a-454e-8046-04e6f33638ec-dshm\") on node \"ip-10-0-142-202.ec2.internal\" DevicePath \"\""
Jul 05 00:54:56.774819 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:54:56.774775    2563 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a48e02c1-be8a-454e-8046-04e6f33638ec-tls-certs\") on node \"ip-10-0-142-202.ec2.internal\" DevicePath \"\""
Jul 05 00:54:56.774819 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:54:56.774784    2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-gxwx5\" (UniqueName: \"kubernetes.io/projected/a48e02c1-be8a-454e-8046-04e6f33638ec-kube-api-access-gxwx5\") on node \"ip-10-0-142-202.ec2.internal\" DevicePath \"\""
Jul 05 00:54:56.774819 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:54:56.774795    2563 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a48e02c1-be8a-454e-8046-04e6f33638ec-home\") on node \"ip-10-0-142-202.ec2.internal\" DevicePath \"\""
Jul 05 00:54:56.774819 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:54:56.774804    2563 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a48e02c1-be8a-454e-8046-04e6f33638ec-tmp-dir\") on node \"ip-10-0-142-202.ec2.internal\" DevicePath \"\""
Jul 05 00:54:56.898054 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:54:56.898019    2563 generic.go:356] "Generic (PLEG): container finished" podID="a48e02c1-be8a-454e-8046-04e6f33638ec" containerID="c3bb78c37c700e2431f0a17f483fd547af4577624c702187e0b2b94a123fb42d" exitCode=0
Jul 05 00:54:56.898054 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:54:56.898056    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-1a1f49/rate-test-model-1a1f49-kserve-745f88fb7f-nb92h"
Jul 05 00:54:56.898349 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:54:56.898104    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-1a1f49/rate-test-model-1a1f49-kserve-745f88fb7f-nb92h" event={"ID":"a48e02c1-be8a-454e-8046-04e6f33638ec","Type":"ContainerDied","Data":"c3bb78c37c700e2431f0a17f483fd547af4577624c702187e0b2b94a123fb42d"}
Jul 05 00:54:56.898349 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:54:56.898148    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-1a1f49/rate-test-model-1a1f49-kserve-745f88fb7f-nb92h" event={"ID":"a48e02c1-be8a-454e-8046-04e6f33638ec","Type":"ContainerDied","Data":"b5d302481f10417d29d9cc73f36474291cefd59efb31e86a1606c49609857bcb"}
Jul 05 00:54:56.898349 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:54:56.898174    2563 scope.go:117] "RemoveContainer" containerID="c3bb78c37c700e2431f0a17f483fd547af4577624c702187e0b2b94a123fb42d"
Jul 05 00:54:56.906718 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:54:56.906704    2563 scope.go:117] "RemoveContainer" containerID="c3bb78c37c700e2431f0a17f483fd547af4577624c702187e0b2b94a123fb42d"
Jul 05 00:54:56.906949 ip-10-0-142-202 kubenswrapper[2563]: E0705 00:54:56.906930    2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c3bb78c37c700e2431f0a17f483fd547af4577624c702187e0b2b94a123fb42d\": container with ID starting with c3bb78c37c700e2431f0a17f483fd547af4577624c702187e0b2b94a123fb42d not found: ID does not exist" containerID="c3bb78c37c700e2431f0a17f483fd547af4577624c702187e0b2b94a123fb42d"
Jul 05 00:54:56.906994 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:54:56.906957    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c3bb78c37c700e2431f0a17f483fd547af4577624c702187e0b2b94a123fb42d"} err="failed to get container status \"c3bb78c37c700e2431f0a17f483fd547af4577624c702187e0b2b94a123fb42d\": rpc error: code = NotFound desc = could not find container \"c3bb78c37c700e2431f0a17f483fd547af4577624c702187e0b2b94a123fb42d\": container with ID starting with c3bb78c37c700e2431f0a17f483fd547af4577624c702187e0b2b94a123fb42d not found: ID does not exist"
Jul 05 00:54:56.916929 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:54:56.916911    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-1a1f49/rate-test-model-1a1f49-kserve-745f88fb7f-nb92h"]
Jul 05 00:54:56.920534 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:54:56.920515    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-b-1a1f49/rate-test-model-1a1f49-kserve-745f88fb7f-nb92h"]
Jul 05 00:54:58.126643 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:54:58.126612    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a48e02c1-be8a-454e-8046-04e6f33638ec" path="/var/lib/kubelet/pods/a48e02c1-be8a-454e-8046-04e6f33638ec/volumes"
Jul 05 00:54:58.467268 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:54:58.467159    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-a-3575e1-openshift-default-5c7fb45bd7-gp87d"]
Jul 05 00:54:58.467512 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:54:58.467482    2563 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-ingress/e2e-shared-a-3575e1-openshift-default-5c7fb45bd7-gp87d" podUID="e6b5817b-48da-49eb-911b-dc93e86b6df5" containerName="istio-proxy" containerID="cri-o://c272dc44518b6cba4738d5b0bf477219c796cc840c0d919be07f81a52f5c00bd" gracePeriod=30
Jul 05 00:55:03.328085 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:55:03.325464    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-x5szm"]
Jul 05 00:55:03.715466 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:55:03.715443    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-3575e1-openshift-default-5c7fb45bd7-gp87d"
Jul 05 00:55:03.837800 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:55:03.837731    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e6b5817b-48da-49eb-911b-dc93e86b6df5-istiod-ca-cert\") pod \"e6b5817b-48da-49eb-911b-dc93e86b6df5\" (UID: \"e6b5817b-48da-49eb-911b-dc93e86b6df5\") "
Jul 05 00:55:03.837800 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:55:03.837786    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e6b5817b-48da-49eb-911b-dc93e86b6df5-istio-data\") pod \"e6b5817b-48da-49eb-911b-dc93e86b6df5\" (UID: \"e6b5817b-48da-49eb-911b-dc93e86b6df5\") "
Jul 05 00:55:03.838020 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:55:03.837815    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-vpbdg\" (UniqueName: \"kubernetes.io/projected/e6b5817b-48da-49eb-911b-dc93e86b6df5-kube-api-access-vpbdg\") pod \"e6b5817b-48da-49eb-911b-dc93e86b6df5\" (UID: \"e6b5817b-48da-49eb-911b-dc93e86b6df5\") "
Jul 05 00:55:03.838020 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:55:03.837990    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e6b5817b-48da-49eb-911b-dc93e86b6df5-istio-envoy\") pod \"e6b5817b-48da-49eb-911b-dc93e86b6df5\" (UID: \"e6b5817b-48da-49eb-911b-dc93e86b6df5\") "
Jul 05 00:55:03.838143 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:55:03.838034    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e6b5817b-48da-49eb-911b-dc93e86b6df5-istio-token\") pod \"e6b5817b-48da-49eb-911b-dc93e86b6df5\" (UID: \"e6b5817b-48da-49eb-911b-dc93e86b6df5\") "
Jul 05 00:55:03.838143 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:55:03.838061    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e6b5817b-48da-49eb-911b-dc93e86b6df5-workload-socket\") pod \"e6b5817b-48da-49eb-911b-dc93e86b6df5\" (UID: \"e6b5817b-48da-49eb-911b-dc93e86b6df5\") "
Jul 05 00:55:03.838143 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:55:03.838086    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e6b5817b-48da-49eb-911b-dc93e86b6df5-workload-certs\") pod \"e6b5817b-48da-49eb-911b-dc93e86b6df5\" (UID: \"e6b5817b-48da-49eb-911b-dc93e86b6df5\") "
Jul 05 00:55:03.838143 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:55:03.838115    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e6b5817b-48da-49eb-911b-dc93e86b6df5-istio-podinfo\") pod \"e6b5817b-48da-49eb-911b-dc93e86b6df5\" (UID: \"e6b5817b-48da-49eb-911b-dc93e86b6df5\") "
Jul 05 00:55:03.838355 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:55:03.838147    2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e6b5817b-48da-49eb-911b-dc93e86b6df5-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "e6b5817b-48da-49eb-911b-dc93e86b6df5" (UID: "e6b5817b-48da-49eb-911b-dc93e86b6df5"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 05 00:55:03.838355 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:55:03.838167    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e6b5817b-48da-49eb-911b-dc93e86b6df5-credential-socket\") pod \"e6b5817b-48da-49eb-911b-dc93e86b6df5\" (UID: \"e6b5817b-48da-49eb-911b-dc93e86b6df5\") "
Jul 05 00:55:03.838462 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:55:03.838361    2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e6b5817b-48da-49eb-911b-dc93e86b6df5-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "e6b5817b-48da-49eb-911b-dc93e86b6df5" (UID: "e6b5817b-48da-49eb-911b-dc93e86b6df5"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 05 00:55:03.838462 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:55:03.838369    2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e6b5817b-48da-49eb-911b-dc93e86b6df5-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "e6b5817b-48da-49eb-911b-dc93e86b6df5" (UID: "e6b5817b-48da-49eb-911b-dc93e86b6df5"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 05 00:55:03.838629 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:55:03.838589    2563 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e6b5817b-48da-49eb-911b-dc93e86b6df5-workload-certs\") on node \"ip-10-0-142-202.ec2.internal\" DevicePath \"\""
Jul 05 00:55:03.838629 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:55:03.838614    2563 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e6b5817b-48da-49eb-911b-dc93e86b6df5-credential-socket\") on node \"ip-10-0-142-202.ec2.internal\" DevicePath \"\""
Jul 05 00:55:03.838752 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:55:03.838629    2563 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e6b5817b-48da-49eb-911b-dc93e86b6df5-istiod-ca-cert\") on node \"ip-10-0-142-202.ec2.internal\" DevicePath \"\""
Jul 05 00:55:03.838805 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:55:03.838757    2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e6b5817b-48da-49eb-911b-dc93e86b6df5-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "e6b5817b-48da-49eb-911b-dc93e86b6df5" (UID: "e6b5817b-48da-49eb-911b-dc93e86b6df5"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 05 00:55:03.838854 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:55:03.838801    2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e6b5817b-48da-49eb-911b-dc93e86b6df5-istio-data" (OuterVolumeSpecName: "istio-data") pod "e6b5817b-48da-49eb-911b-dc93e86b6df5" (UID: "e6b5817b-48da-49eb-911b-dc93e86b6df5"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 05 00:55:03.840551 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:55:03.840524    2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e6b5817b-48da-49eb-911b-dc93e86b6df5-kube-api-access-vpbdg" (OuterVolumeSpecName: "kube-api-access-vpbdg") pod "e6b5817b-48da-49eb-911b-dc93e86b6df5" (UID: "e6b5817b-48da-49eb-911b-dc93e86b6df5"). InnerVolumeSpecName "kube-api-access-vpbdg". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 05 00:55:03.840661 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:55:03.840524    2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e6b5817b-48da-49eb-911b-dc93e86b6df5-istio-token" (OuterVolumeSpecName: "istio-token") pod "e6b5817b-48da-49eb-911b-dc93e86b6df5" (UID: "e6b5817b-48da-49eb-911b-dc93e86b6df5"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 05 00:55:03.840661 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:55:03.840556    2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/e6b5817b-48da-49eb-911b-dc93e86b6df5-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "e6b5817b-48da-49eb-911b-dc93e86b6df5" (UID: "e6b5817b-48da-49eb-911b-dc93e86b6df5"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Jul 05 00:55:03.840661 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:55:03.840572    2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e6b5817b-48da-49eb-911b-dc93e86b6df5-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "e6b5817b-48da-49eb-911b-dc93e86b6df5" (UID: "e6b5817b-48da-49eb-911b-dc93e86b6df5"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 05 00:55:03.933413 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:55:03.933385    2563 generic.go:356] "Generic (PLEG): container finished" podID="e6b5817b-48da-49eb-911b-dc93e86b6df5" containerID="c272dc44518b6cba4738d5b0bf477219c796cc840c0d919be07f81a52f5c00bd" exitCode=0
Jul 05 00:55:03.933540 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:55:03.933425    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-3575e1-openshift-default-5c7fb45bd7-gp87d" event={"ID":"e6b5817b-48da-49eb-911b-dc93e86b6df5","Type":"ContainerDied","Data":"c272dc44518b6cba4738d5b0bf477219c796cc840c0d919be07f81a52f5c00bd"}
Jul 05 00:55:03.933540 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:55:03.933431    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-3575e1-openshift-default-5c7fb45bd7-gp87d"
Jul 05 00:55:03.933540 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:55:03.933466    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-3575e1-openshift-default-5c7fb45bd7-gp87d" event={"ID":"e6b5817b-48da-49eb-911b-dc93e86b6df5","Type":"ContainerDied","Data":"98e33fda3a6a1ed3c22a82afa3aa8611731be72a594841451ae6637fbb809711"}
Jul 05 00:55:03.933540 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:55:03.933488    2563 scope.go:117] "RemoveContainer" containerID="c272dc44518b6cba4738d5b0bf477219c796cc840c0d919be07f81a52f5c00bd"
Jul 05 00:55:03.939195 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:55:03.939175    2563 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e6b5817b-48da-49eb-911b-dc93e86b6df5-workload-socket\") on node \"ip-10-0-142-202.ec2.internal\" DevicePath \"\""
Jul 05 00:55:03.939322 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:55:03.939196    2563 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e6b5817b-48da-49eb-911b-dc93e86b6df5-istio-podinfo\") on node \"ip-10-0-142-202.ec2.internal\" DevicePath \"\""
Jul 05 00:55:03.939322 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:55:03.939210    2563 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e6b5817b-48da-49eb-911b-dc93e86b6df5-istio-data\") on node \"ip-10-0-142-202.ec2.internal\" DevicePath \"\""
Jul 05 00:55:03.939322 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:55:03.939241    2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-vpbdg\" (UniqueName: \"kubernetes.io/projected/e6b5817b-48da-49eb-911b-dc93e86b6df5-kube-api-access-vpbdg\") on node \"ip-10-0-142-202.ec2.internal\" DevicePath \"\""
Jul 05 00:55:03.939322 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:55:03.939256    2563 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e6b5817b-48da-49eb-911b-dc93e86b6df5-istio-envoy\") on node \"ip-10-0-142-202.ec2.internal\" DevicePath \"\""
Jul 05 00:55:03.939322 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:55:03.939268    2563 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e6b5817b-48da-49eb-911b-dc93e86b6df5-istio-token\") on node \"ip-10-0-142-202.ec2.internal\" DevicePath \"\""
Jul 05 00:55:03.942313 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:55:03.942297    2563 scope.go:117] "RemoveContainer" containerID="8d818e001e4702e4e9de5a97e5ed91c6439f6581381f6a281ed8aa0df5277c20"
Jul 05 00:55:03.949736 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:55:03.949721    2563 scope.go:117] "RemoveContainer" containerID="c272dc44518b6cba4738d5b0bf477219c796cc840c0d919be07f81a52f5c00bd"
Jul 05 00:55:03.949982 ip-10-0-142-202 kubenswrapper[2563]: E0705 00:55:03.949966    2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c272dc44518b6cba4738d5b0bf477219c796cc840c0d919be07f81a52f5c00bd\": container with ID starting with c272dc44518b6cba4738d5b0bf477219c796cc840c0d919be07f81a52f5c00bd not found: ID does not exist" containerID="c272dc44518b6cba4738d5b0bf477219c796cc840c0d919be07f81a52f5c00bd"
Jul 05 00:55:03.950021 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:55:03.949989    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c272dc44518b6cba4738d5b0bf477219c796cc840c0d919be07f81a52f5c00bd"} err="failed to get container status \"c272dc44518b6cba4738d5b0bf477219c796cc840c0d919be07f81a52f5c00bd\": rpc error: code = NotFound desc = could not find container \"c272dc44518b6cba4738d5b0bf477219c796cc840c0d919be07f81a52f5c00bd\": container with ID starting with c272dc44518b6cba4738d5b0bf477219c796cc840c0d919be07f81a52f5c00bd not found: ID does not exist"
Jul 05 00:55:03.950021 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:55:03.950004    2563 scope.go:117] "RemoveContainer" containerID="8d818e001e4702e4e9de5a97e5ed91c6439f6581381f6a281ed8aa0df5277c20"
Jul 05 00:55:03.950200 ip-10-0-142-202 kubenswrapper[2563]: E0705 00:55:03.950184    2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8d818e001e4702e4e9de5a97e5ed91c6439f6581381f6a281ed8aa0df5277c20\": container with ID starting with 8d818e001e4702e4e9de5a97e5ed91c6439f6581381f6a281ed8aa0df5277c20 not found: ID does not exist" containerID="8d818e001e4702e4e9de5a97e5ed91c6439f6581381f6a281ed8aa0df5277c20"
Jul 05 00:55:03.950260 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:55:03.950204    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8d818e001e4702e4e9de5a97e5ed91c6439f6581381f6a281ed8aa0df5277c20"} err="failed to get container status \"8d818e001e4702e4e9de5a97e5ed91c6439f6581381f6a281ed8aa0df5277c20\": rpc error: code = NotFound desc = could not find container \"8d818e001e4702e4e9de5a97e5ed91c6439f6581381f6a281ed8aa0df5277c20\": container with ID starting with 8d818e001e4702e4e9de5a97e5ed91c6439f6581381f6a281ed8aa0df5277c20 not found: ID does not exist"
Jul 05 00:55:03.953413 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:55:03.953392    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-a-3575e1-openshift-default-5c7fb45bd7-gp87d"]
Jul 05 00:55:03.956571 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:55:03.956553    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-shared-a-3575e1-openshift-default-5c7fb45bd7-gp87d"]
Jul 05 00:55:04.125985 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:55:04.125909    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e6b5817b-48da-49eb-911b-dc93e86b6df5" path="/var/lib/kubelet/pods/e6b5817b-48da-49eb-911b-dc93e86b6df5/volumes"
Jul 05 00:55:06.061475 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:55:06.061445    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-7694cb889c-r9vqc_aa1985d1-619d-40d7-8734-5ae6e64d24ee/authorino/0.log"
Jul 05 00:55:11.546209 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:55:11.546180    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-api-key-cleanup-29720190-jnlkj_1837769c-81b2-45b4-9a21-cd351361f660/cleanup/0.log"
Jul 05 00:55:11.657717 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:55:11.657688    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-api-key-cleanup-29720205-jdftl_33370136-cdfa-45c3-9f8f-a10e21bb37e4/cleanup/0.log"
Jul 05 00:55:12.318663 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:55:12.318638    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-674c4bd948-9brxw_07bb806b-cabf-4e75-8758-19c65226e9ad/copy-manifests/0.log"
Jul 05 00:55:12.324390 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:55:12.324361    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-674c4bd948-9brxw_07bb806b-cabf-4e75-8758-19c65226e9ad/manager/0.log"
Jul 05 00:55:13.962707 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:55:13.962680    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-7694cb889c-r9vqc_aa1985d1-619d-40d7-8734-5ae6e64d24ee/authorino/0.log"
Jul 05 00:55:14.740663 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:55:14.740636    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-limitador-54d6d988d5-x5szm_cacf0cb6-3f72-44ee-a6d9-77d3a0d89616/limitador/0.log"
Jul 05 00:55:15.506773 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:55:15.506739    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-6bddf4ddb8-mb7vs_6c8d44d6-8738-4ab3-8950-f2b687d3026b/kube-auth-proxy/0.log"
Jul 05 00:55:15.777060 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:55:15.777035    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-56654dcf7-wzqkx_a95fc6e8-4791-4960-b7f5-2d8e09342c4a/istio-proxy/1.log"
Jul 05 00:55:16.971366 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:55:16.971338    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_facebook-opt-125m-simulated-kserve-74b68447df-2p6f8_5669b7cd-6e37-4231-bce1-83061f981f12/main/0.log"
Jul 05 00:55:17.097086 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:55:17.097057    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_premium-simulated-simulated-premium-kserve-5fc6fb456f-ql2w2_731bd138-d4b7-4193-9f52-4bf2c656c123/main/0.log"
Jul 05 00:55:24.550543 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:55:24.550515    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-dkwx5_00bb9c2a-9bed-4d69-9ae4-e4f1fad50578/global-pull-secret-syncer/0.log"
Jul 05 00:55:24.719804 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:55:24.719779    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-rwbw4_1f3c24e8-591d-4e53-9b63-091f6f1dec51/konnectivity-agent/0.log"
Jul 05 00:55:24.775066 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:55:24.775041    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-142-202.ec2.internal_ec580e48b6696694176fa0643f6420e5/haproxy/0.log"
Jul 05 00:55:29.928330 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:55:29.928292    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-7694cb889c-r9vqc_aa1985d1-619d-40d7-8734-5ae6e64d24ee/authorino/0.log"
Jul 05 00:55:30.267195 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:55:30.267114    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-limitador-54d6d988d5-x5szm_cacf0cb6-3f72-44ee-a6d9-77d3a0d89616/limitador/0.log"
Jul 05 00:55:32.266281 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:55:32.266196    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-7c9dbb8b7d-2pcmn_b323372b-85f9-406b-ae16-1f71d04a90f2/kube-state-metrics/0.log"
Jul 05 00:55:32.281843 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:55:32.281822    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-7c9dbb8b7d-2pcmn_b323372b-85f9-406b-ae16-1f71d04a90f2/kube-rbac-proxy-main/0.log"
Jul 05 00:55:32.299365 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:55:32.299345    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-7c9dbb8b7d-2pcmn_b323372b-85f9-406b-ae16-1f71d04a90f2/kube-rbac-proxy-self/0.log"
Jul 05 00:55:32.489735 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:55:32.489697    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-zfrjf_2a8bcefb-0550-490f-b3a0-6421287ee9d9/node-exporter/0.log"
Jul 05 00:55:32.503906 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:55:32.503882    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-zfrjf_2a8bcefb-0550-490f-b3a0-6421287ee9d9/kube-rbac-proxy/0.log"
Jul 05 00:55:32.522259 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:55:32.522190    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-zfrjf_2a8bcefb-0550-490f-b3a0-6421287ee9d9/init-textfile/0.log"
Jul 05 00:55:32.606954 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:55:32.606934    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_d4aa743c-f71b-4023-97bd-05452c4e4e15/prometheus/0.log"
Jul 05 00:55:32.622807 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:55:32.622792    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_d4aa743c-f71b-4023-97bd-05452c4e4e15/config-reloader/0.log"
Jul 05 00:55:32.639138 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:55:32.639118    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_d4aa743c-f71b-4023-97bd-05452c4e4e15/thanos-sidecar/0.log"
Jul 05 00:55:32.653723 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:55:32.653709    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_d4aa743c-f71b-4023-97bd-05452c4e4e15/kube-rbac-proxy-web/0.log"
Jul 05 00:55:32.668618 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:55:32.668601    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_d4aa743c-f71b-4023-97bd-05452c4e4e15/kube-rbac-proxy/0.log"
Jul 05 00:55:32.684362 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:55:32.684345    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_d4aa743c-f71b-4023-97bd-05452c4e4e15/kube-rbac-proxy-thanos/0.log"
Jul 05 00:55:32.700841 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:55:32.700823    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_d4aa743c-f71b-4023-97bd-05452c4e4e15/init-config-reloader/0.log"
Jul 05 00:55:32.723417 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:55:32.723394    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-567f7b8b7c-qch55_483ae7bd-5811-4563-8bdb-c3a3c31e6d27/prometheus-operator/0.log"
Jul 05 00:55:32.741122 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:55:32.741103    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-567f7b8b7c-qch55_483ae7bd-5811-4563-8bdb-c3a3c31e6d27/kube-rbac-proxy/0.log"
Jul 05 00:55:32.762089 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:55:32.762066    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-admission-webhook-d559d66d5-x2tdh_bd0ac8a5-2f74-481a-824a-db88b85f3078/prometheus-operator-admission-webhook/0.log"
Jul 05 00:55:34.912610 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:55:34.912528    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-6nfx9/perf-node-gather-daemonset-xlhcx"]
Jul 05 00:55:34.913104 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:55:34.912983    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e6b5817b-48da-49eb-911b-dc93e86b6df5" containerName="istio-proxy"
Jul 05 00:55:34.913104 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:55:34.913010    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="e6b5817b-48da-49eb-911b-dc93e86b6df5" containerName="istio-proxy"
Jul 05 00:55:34.913104 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:55:34.913027    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a48e02c1-be8a-454e-8046-04e6f33638ec" containerName="main"
Jul 05 00:55:34.913104 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:55:34.913035    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="a48e02c1-be8a-454e-8046-04e6f33638ec" containerName="main"
Jul 05 00:55:34.913104 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:55:34.913083    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5bdc4747-ff60-43d0-9c17-ba28acdf331d" containerName="main"
Jul 05 00:55:34.913104 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:55:34.913093    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="5bdc4747-ff60-43d0-9c17-ba28acdf331d" containerName="main"
Jul 05 00:55:34.913472 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:55:34.913203    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="e6b5817b-48da-49eb-911b-dc93e86b6df5" containerName="istio-proxy"
Jul 05 00:55:34.913472 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:55:34.913233    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="e6b5817b-48da-49eb-911b-dc93e86b6df5" containerName="istio-proxy"
Jul 05 00:55:34.913472 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:55:34.913245    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="a48e02c1-be8a-454e-8046-04e6f33638ec" containerName="main"
Jul 05 00:55:34.913472 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:55:34.913254    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="5bdc4747-ff60-43d0-9c17-ba28acdf331d" containerName="main"
Jul 05 00:55:34.916817 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:55:34.916799    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-6nfx9/perf-node-gather-daemonset-xlhcx"
Jul 05 00:55:34.918848 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:55:34.918822    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-6nfx9\"/\"kube-root-ca.crt\""
Jul 05 00:55:34.918944 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:55:34.918870    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-6nfx9\"/\"default-dockercfg-jzs6w\""
Jul 05 00:55:34.918944 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:55:34.918936    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-6nfx9\"/\"openshift-service-ca.crt\""
Jul 05 00:55:34.922275 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:55:34.922256    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-6nfx9/perf-node-gather-daemonset-xlhcx"]
Jul 05 00:55:34.970897 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:55:34.970867    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/1c95b23c-e45e-4c06-b9ca-645ae34bf926-proc\") pod \"perf-node-gather-daemonset-xlhcx\" (UID: \"1c95b23c-e45e-4c06-b9ca-645ae34bf926\") " pod="openshift-must-gather-6nfx9/perf-node-gather-daemonset-xlhcx"
Jul 05 00:55:34.971052 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:55:34.970908    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/1c95b23c-e45e-4c06-b9ca-645ae34bf926-podres\") pod \"perf-node-gather-daemonset-xlhcx\" (UID: \"1c95b23c-e45e-4c06-b9ca-645ae34bf926\") " pod="openshift-must-gather-6nfx9/perf-node-gather-daemonset-xlhcx"
Jul 05 00:55:34.971052 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:55:34.970964    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/1c95b23c-e45e-4c06-b9ca-645ae34bf926-lib-modules\") pod \"perf-node-gather-daemonset-xlhcx\" (UID: \"1c95b23c-e45e-4c06-b9ca-645ae34bf926\") " pod="openshift-must-gather-6nfx9/perf-node-gather-daemonset-xlhcx"
Jul 05 00:55:34.971052 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:55:34.970991    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/1c95b23c-e45e-4c06-b9ca-645ae34bf926-sys\") pod \"perf-node-gather-daemonset-xlhcx\" (UID: \"1c95b23c-e45e-4c06-b9ca-645ae34bf926\") " pod="openshift-must-gather-6nfx9/perf-node-gather-daemonset-xlhcx"
Jul 05 00:55:34.971052 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:55:34.971022    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qnt4q\" (UniqueName: \"kubernetes.io/projected/1c95b23c-e45e-4c06-b9ca-645ae34bf926-kube-api-access-qnt4q\") pod \"perf-node-gather-daemonset-xlhcx\" (UID: \"1c95b23c-e45e-4c06-b9ca-645ae34bf926\") " pod="openshift-must-gather-6nfx9/perf-node-gather-daemonset-xlhcx"
Jul 05 00:55:35.072148 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:55:35.072112    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qnt4q\" (UniqueName: \"kubernetes.io/projected/1c95b23c-e45e-4c06-b9ca-645ae34bf926-kube-api-access-qnt4q\") pod \"perf-node-gather-daemonset-xlhcx\" (UID: \"1c95b23c-e45e-4c06-b9ca-645ae34bf926\") " pod="openshift-must-gather-6nfx9/perf-node-gather-daemonset-xlhcx"
Jul 05 00:55:35.072354 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:55:35.072162    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/1c95b23c-e45e-4c06-b9ca-645ae34bf926-proc\") pod \"perf-node-gather-daemonset-xlhcx\" (UID: \"1c95b23c-e45e-4c06-b9ca-645ae34bf926\") " pod="openshift-must-gather-6nfx9/perf-node-gather-daemonset-xlhcx"
Jul 05 00:55:35.072354 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:55:35.072205    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/1c95b23c-e45e-4c06-b9ca-645ae34bf926-podres\") pod \"perf-node-gather-daemonset-xlhcx\" (UID: \"1c95b23c-e45e-4c06-b9ca-645ae34bf926\") " pod="openshift-must-gather-6nfx9/perf-node-gather-daemonset-xlhcx"
Jul 05 00:55:35.072354 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:55:35.072282    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/1c95b23c-e45e-4c06-b9ca-645ae34bf926-lib-modules\") pod \"perf-node-gather-daemonset-xlhcx\" (UID: \"1c95b23c-e45e-4c06-b9ca-645ae34bf926\") " pod="openshift-must-gather-6nfx9/perf-node-gather-daemonset-xlhcx"
Jul 05 00:55:35.072354 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:55:35.072301    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/1c95b23c-e45e-4c06-b9ca-645ae34bf926-proc\") pod \"perf-node-gather-daemonset-xlhcx\" (UID: \"1c95b23c-e45e-4c06-b9ca-645ae34bf926\") " pod="openshift-must-gather-6nfx9/perf-node-gather-daemonset-xlhcx"
Jul 05 00:55:35.072354 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:55:35.072319    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/1c95b23c-e45e-4c06-b9ca-645ae34bf926-sys\") pod \"perf-node-gather-daemonset-xlhcx\" (UID: \"1c95b23c-e45e-4c06-b9ca-645ae34bf926\") " pod="openshift-must-gather-6nfx9/perf-node-gather-daemonset-xlhcx"
Jul 05 00:55:35.072567 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:55:35.072398    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/1c95b23c-e45e-4c06-b9ca-645ae34bf926-sys\") pod \"perf-node-gather-daemonset-xlhcx\" (UID: \"1c95b23c-e45e-4c06-b9ca-645ae34bf926\") " pod="openshift-must-gather-6nfx9/perf-node-gather-daemonset-xlhcx"
Jul 05 00:55:35.072567 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:55:35.072405    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/1c95b23c-e45e-4c06-b9ca-645ae34bf926-lib-modules\") pod \"perf-node-gather-daemonset-xlhcx\" (UID: \"1c95b23c-e45e-4c06-b9ca-645ae34bf926\") " pod="openshift-must-gather-6nfx9/perf-node-gather-daemonset-xlhcx"
Jul 05 00:55:35.072567 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:55:35.072423    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/1c95b23c-e45e-4c06-b9ca-645ae34bf926-podres\") pod \"perf-node-gather-daemonset-xlhcx\" (UID: \"1c95b23c-e45e-4c06-b9ca-645ae34bf926\") " pod="openshift-must-gather-6nfx9/perf-node-gather-daemonset-xlhcx"
Jul 05 00:55:35.079537 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:55:35.079508    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-qnt4q\" (UniqueName: \"kubernetes.io/projected/1c95b23c-e45e-4c06-b9ca-645ae34bf926-kube-api-access-qnt4q\") pod \"perf-node-gather-daemonset-xlhcx\" (UID: \"1c95b23c-e45e-4c06-b9ca-645ae34bf926\") " pod="openshift-must-gather-6nfx9/perf-node-gather-daemonset-xlhcx"
Jul 05 00:55:35.227387 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:55:35.227297    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-6nfx9/perf-node-gather-daemonset-xlhcx"
Jul 05 00:55:35.356282 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:55:35.356247    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-6nfx9/perf-node-gather-daemonset-xlhcx"]
Jul 05 00:55:35.359070 ip-10-0-142-202 kubenswrapper[2563]: W0705 00:55:35.359034    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-pod1c95b23c_e45e_4c06_b9ca_645ae34bf926.slice/crio-c243891b845384b28fd58bf43e6e3f82991da9eee518c85a24a1a9d58c90c6b1 WatchSource:0}: Error finding container c243891b845384b28fd58bf43e6e3f82991da9eee518c85a24a1a9d58c90c6b1: Status 404 returned error can't find the container with id c243891b845384b28fd58bf43e6e3f82991da9eee518c85a24a1a9d58c90c6b1
Jul 05 00:55:36.086443 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:55:36.086384    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-6nfx9/perf-node-gather-daemonset-xlhcx" event={"ID":"1c95b23c-e45e-4c06-b9ca-645ae34bf926","Type":"ContainerStarted","Data":"17764a7891bdcafcff7c2c6bebfed5799613143da304523666d292ebffd9a40c"}
Jul 05 00:55:36.086840 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:55:36.086459    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-6nfx9/perf-node-gather-daemonset-xlhcx"
Jul 05 00:55:36.086840 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:55:36.086475    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-6nfx9/perf-node-gather-daemonset-xlhcx" event={"ID":"1c95b23c-e45e-4c06-b9ca-645ae34bf926","Type":"ContainerStarted","Data":"c243891b845384b28fd58bf43e6e3f82991da9eee518c85a24a1a9d58c90c6b1"}
Jul 05 00:55:36.102116 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:55:36.102074    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-6nfx9/perf-node-gather-daemonset-xlhcx" podStartSLOduration=2.1020596 podStartE2EDuration="2.1020596s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-05 00:55:34 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-05 00:55:36.099998908 +0000 UTC m=+2784.442327968" watchObservedRunningTime="2026-07-05 00:55:36.1020596 +0000 UTC m=+2784.444388660"
Jul 05 00:55:36.738681 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:55:36.738659    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-snsng_0d75ce89-37fb-4d36-ae84-aee968c71dfa/dns/0.log"
Jul 05 00:55:36.753093 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:55:36.753052    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-snsng_0d75ce89-37fb-4d36-ae84-aee968c71dfa/kube-rbac-proxy/0.log"
Jul 05 00:55:36.789996 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:55:36.789972    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-kjz9w_cecee17b-8d41-4a16-a3b2-53919db096bb/dns-node-resolver/0.log"
Jul 05 00:55:37.355736 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:55:37.355707    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-s547v_d3a9b78d-cf60-4acc-a3c5-016ee34903f8/node-ca/0.log"
Jul 05 00:55:38.381651 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:55:38.381626    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-6bddf4ddb8-mb7vs_6c8d44d6-8738-4ab3-8950-f2b687d3026b/kube-auth-proxy/0.log"
Jul 05 00:55:38.426517 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:55:38.426481    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-56654dcf7-wzqkx_a95fc6e8-4791-4960-b7f5-2d8e09342c4a/istio-proxy/0.log"
Jul 05 00:55:38.453647 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:55:38.453622    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-56654dcf7-wzqkx_a95fc6e8-4791-4960-b7f5-2d8e09342c4a/istio-proxy/1.log"
Jul 05 00:55:39.235108 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:55:39.235082    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-xqpm2_63e680ec-84d5-4c0a-b259-84c64914aab7/serve-healthcheck-canary/0.log"
Jul 05 00:55:39.756403 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:55:39.756374    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-snrdh_7f6da6e5-bbfb-48e4-bb8a-b471c187e3ee/kube-rbac-proxy/0.log"
Jul 05 00:55:39.770346 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:55:39.770320    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-snrdh_7f6da6e5-bbfb-48e4-bb8a-b471c187e3ee/exporter/0.log"
Jul 05 00:55:39.784836 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:55:39.784816    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-snrdh_7f6da6e5-bbfb-48e4-bb8a-b471c187e3ee/extractor/0.log"
Jul 05 00:55:42.100926 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:55:42.100902    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-6nfx9/perf-node-gather-daemonset-xlhcx"
Jul 05 00:55:42.107471 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:55:42.107450    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-api-key-cleanup-29720190-jnlkj_1837769c-81b2-45b4-9a21-cd351361f660/cleanup/0.log"
Jul 05 00:55:42.128757 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:55:42.128737    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-api-key-cleanup-29720205-jdftl_33370136-cdfa-45c3-9f8f-a10e21bb37e4/cleanup/0.log"
Jul 05 00:55:42.482606 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:55:42.482539    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-674c4bd948-9brxw_07bb806b-cabf-4e75-8758-19c65226e9ad/manager/0.log"
Jul 05 00:55:42.497130 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:55:42.497112    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-674c4bd948-9brxw_07bb806b-cabf-4e75-8758-19c65226e9ad/copy-manifests/0.log"
Jul 05 00:55:44.031087 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:55:44.031063    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-lws-operator_lws-controller-manager-7dcb9fcb66-btmss_4daa43eb-7628-421d-96e8-fbb8e3051e5b/manager/0.log"
Jul 05 00:55:44.992378 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:55:44.992355    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-6bddf4ddb8-mb7vs_6c8d44d6-8738-4ab3-8950-f2b687d3026b/kube-auth-proxy/0.log"
Jul 05 00:55:45.033193 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:55:45.033162    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-56654dcf7-wzqkx_a95fc6e8-4791-4960-b7f5-2d8e09342c4a/istio-proxy/0.log"
Jul 05 00:55:45.059564 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:55:45.059523    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-56654dcf7-wzqkx_a95fc6e8-4791-4960-b7f5-2d8e09342c4a/istio-proxy/1.log"
Jul 05 00:55:46.534596 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:55:46.534571    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-6bddf4ddb8-mb7vs_6c8d44d6-8738-4ab3-8950-f2b687d3026b/kube-auth-proxy/0.log"
Jul 05 00:55:46.564486 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:55:46.564464    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-56654dcf7-wzqkx_a95fc6e8-4791-4960-b7f5-2d8e09342c4a/istio-proxy/0.log"
Jul 05 00:55:46.589708 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:55:46.589684    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-56654dcf7-wzqkx_a95fc6e8-4791-4960-b7f5-2d8e09342c4a/istio-proxy/1.log"
Jul 05 00:55:48.966834 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:55:48.966803    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator_migrator-d57d58cc9-9qgkt_b0d434b5-78a3-4a5e-bf15-082da796dfdf/migrator/0.log"
Jul 05 00:55:48.981619 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:55:48.981591    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator_migrator-d57d58cc9-9qgkt_b0d434b5-78a3-4a5e-bf15-082da796dfdf/graceful-termination/0.log"
Jul 05 00:55:50.273980 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:55:50.273952    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-g568x_aeaa1e13-243b-4c7f-8045-85ba45081c42/kube-multus-additional-cni-plugins/0.log"
Jul 05 00:55:50.288902 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:55:50.288880    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-g568x_aeaa1e13-243b-4c7f-8045-85ba45081c42/egress-router-binary-copy/0.log"
Jul 05 00:55:50.302777 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:55:50.302756    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-g568x_aeaa1e13-243b-4c7f-8045-85ba45081c42/cni-plugins/0.log"
Jul 05 00:55:50.317658 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:55:50.317638    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-g568x_aeaa1e13-243b-4c7f-8045-85ba45081c42/bond-cni-plugin/0.log"
Jul 05 00:55:50.332540 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:55:50.332517    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-g568x_aeaa1e13-243b-4c7f-8045-85ba45081c42/routeoverride-cni/0.log"
Jul 05 00:55:50.350204 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:55:50.350186    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-g568x_aeaa1e13-243b-4c7f-8045-85ba45081c42/whereabouts-cni-bincopy/0.log"
Jul 05 00:55:50.367669 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:55:50.367642    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-g568x_aeaa1e13-243b-4c7f-8045-85ba45081c42/whereabouts-cni/0.log"
Jul 05 00:55:50.517589 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:55:50.517565    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-kwgzc_aa4848f6-1480-4acd-af73-e8e055306078/kube-multus/0.log"
Jul 05 00:55:50.610628 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:55:50.610602    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-5fn94_c085d4be-fcda-4931-a018-02cbaca17c59/network-metrics-daemon/0.log"
Jul 05 00:55:50.623686 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:55:50.623661    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-5fn94_c085d4be-fcda-4931-a018-02cbaca17c59/kube-rbac-proxy/0.log"
Jul 05 00:55:51.616279 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:55:51.616252    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-7sxq4_b96bfb4d-2fd3-4be3-9064-e24efeb904f4/ovn-controller/0.log"
Jul 05 00:55:51.627700 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:55:51.627676    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-7sxq4_b96bfb4d-2fd3-4be3-9064-e24efeb904f4/ovn-acl-logging/1.log"
Jul 05 00:55:51.627700 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:55:51.627695    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-7sxq4_b96bfb4d-2fd3-4be3-9064-e24efeb904f4/ovn-acl-logging/0.log"
Jul 05 00:55:51.643713 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:55:51.643696    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-7sxq4_b96bfb4d-2fd3-4be3-9064-e24efeb904f4/kube-rbac-proxy-node/0.log"
Jul 05 00:55:51.659440 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:55:51.659418    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-7sxq4_b96bfb4d-2fd3-4be3-9064-e24efeb904f4/kube-rbac-proxy-ovn-metrics/0.log"
Jul 05 00:55:51.672814 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:55:51.672799    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-7sxq4_b96bfb4d-2fd3-4be3-9064-e24efeb904f4/northd/0.log"
Jul 05 00:55:51.687896 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:55:51.687880    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-7sxq4_b96bfb4d-2fd3-4be3-9064-e24efeb904f4/nbdb/0.log"
Jul 05 00:55:51.701585 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:55:51.701568    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-7sxq4_b96bfb4d-2fd3-4be3-9064-e24efeb904f4/sbdb/0.log"
Jul 05 00:55:51.745465 ip-10-0-142-202 kubenswrapper[2563]: I0705 00:55:51.745444    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-7sxq4_b96bfb4d-2fd3-4be3-9064-e24efeb904f4/ovnkube-controller/0.log"