Jul 06 18:53:14.491073 ip-10-0-130-59 systemd[1]: kubelet.service: Failed to load environment files: No such file or directory Jul 06 18:53:14.491087 ip-10-0-130-59 systemd[1]: kubelet.service: Failed to run 'start-pre' task: No such file or directory Jul 06 18:53:14.491096 ip-10-0-130-59 systemd[1]: kubelet.service: Failed with result 'resources'. Jul 06 18:53:14.491394 ip-10-0-130-59 systemd[1]: Failed to start Kubernetes Kubelet. Jul 06 18:53:24.740072 ip-10-0-130-59 systemd[1]: kubelet.service: Failed to schedule restart job: Unit crio.service not found. Jul 06 18:53:24.740088 ip-10-0-130-59 systemd[1]: kubelet.service: Failed with result 'resources'. -- Boot aea452ed01d247dbbc5663d2d6d0bf0c -- Jul 06 18:55:18.543651 ip-10-0-130-59 systemd[1]: Starting Kubernetes Kubelet... Jul 06 18:55:20.809445 ip-10-0-130-59 kubenswrapper[2570]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 06 18:55:20.809445 ip-10-0-130-59 kubenswrapper[2570]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version. Jul 06 18:55:20.809445 ip-10-0-130-59 kubenswrapper[2570]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 06 18:55:20.809445 ip-10-0-130-59 kubenswrapper[2570]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI. Jul 06 18:55:20.809445 ip-10-0-130-59 kubenswrapper[2570]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 06 18:55:20.809445 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:55:20.738101 2570 server.go:213] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime" Jul 06 18:55:20.809445 ip-10-0-130-59 kubenswrapper[2570]: W0706 18:55:20.744163 2570 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 06 18:55:20.809445 ip-10-0-130-59 kubenswrapper[2570]: W0706 18:55:20.744173 2570 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 06 18:55:20.809445 ip-10-0-130-59 kubenswrapper[2570]: W0706 18:55:20.744177 2570 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 06 18:55:20.809445 ip-10-0-130-59 kubenswrapper[2570]: W0706 18:55:20.744181 2570 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 06 18:55:20.809445 ip-10-0-130-59 kubenswrapper[2570]: W0706 18:55:20.744184 2570 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 06 18:55:20.809445 ip-10-0-130-59 kubenswrapper[2570]: W0706 18:55:20.744187 2570 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 06 18:55:20.809445 ip-10-0-130-59 kubenswrapper[2570]: W0706 18:55:20.744190 2570 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 06 18:55:20.809445 ip-10-0-130-59 kubenswrapper[2570]: W0706 18:55:20.744193 2570 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 06 18:55:20.811563 ip-10-0-130-59 kubenswrapper[2570]: W0706 18:55:20.744196 2570 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 06 18:55:20.811563 ip-10-0-130-59 kubenswrapper[2570]: W0706 18:55:20.744198 2570 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 06 18:55:20.811563 ip-10-0-130-59 kubenswrapper[2570]: W0706 18:55:20.744201 2570 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 06 18:55:20.811563 ip-10-0-130-59 kubenswrapper[2570]: W0706 18:55:20.744206 2570 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 06 18:55:20.811563 ip-10-0-130-59 kubenswrapper[2570]: W0706 18:55:20.744210 2570 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 06 18:55:20.811563 ip-10-0-130-59 kubenswrapper[2570]: W0706 18:55:20.744213 2570 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 06 18:55:20.811563 ip-10-0-130-59 kubenswrapper[2570]: W0706 18:55:20.744216 2570 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 06 18:55:20.811563 ip-10-0-130-59 kubenswrapper[2570]: W0706 18:55:20.744218 2570 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 06 18:55:20.811563 ip-10-0-130-59 kubenswrapper[2570]: W0706 18:55:20.744221 2570 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 06 18:55:20.811563 ip-10-0-130-59 kubenswrapper[2570]: W0706 18:55:20.744223 2570 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 06 18:55:20.811563 ip-10-0-130-59 kubenswrapper[2570]: W0706 18:55:20.744226 2570 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 06 18:55:20.811563 ip-10-0-130-59 kubenswrapper[2570]: W0706 18:55:20.744230 2570 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 06 18:55:20.811563 ip-10-0-130-59 kubenswrapper[2570]: W0706 18:55:20.744232 2570 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 06 18:55:20.811563 ip-10-0-130-59 kubenswrapper[2570]: W0706 18:55:20.744235 2570 feature_gate.go:328] unrecognized feature gate: Example2 Jul 06 18:55:20.811563 ip-10-0-130-59 kubenswrapper[2570]: W0706 18:55:20.744238 2570 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 06 18:55:20.811563 ip-10-0-130-59 kubenswrapper[2570]: W0706 18:55:20.744241 2570 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 06 18:55:20.811563 ip-10-0-130-59 kubenswrapper[2570]: W0706 18:55:20.744243 2570 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 06 18:55:20.811563 ip-10-0-130-59 kubenswrapper[2570]: W0706 18:55:20.744246 2570 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 06 18:55:20.811563 ip-10-0-130-59 kubenswrapper[2570]: W0706 18:55:20.744249 2570 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 06 18:55:20.812198 ip-10-0-130-59 kubenswrapper[2570]: W0706 18:55:20.744252 2570 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 06 18:55:20.812198 ip-10-0-130-59 kubenswrapper[2570]: W0706 18:55:20.744254 2570 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 06 18:55:20.812198 ip-10-0-130-59 kubenswrapper[2570]: W0706 18:55:20.744257 2570 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 06 18:55:20.812198 ip-10-0-130-59 kubenswrapper[2570]: W0706 18:55:20.744260 2570 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 06 18:55:20.812198 ip-10-0-130-59 kubenswrapper[2570]: W0706 18:55:20.744262 2570 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 06 18:55:20.812198 ip-10-0-130-59 kubenswrapper[2570]: W0706 18:55:20.744265 2570 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 06 18:55:20.812198 ip-10-0-130-59 kubenswrapper[2570]: W0706 18:55:20.744267 2570 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 06 18:55:20.812198 ip-10-0-130-59 kubenswrapper[2570]: W0706 18:55:20.744270 2570 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 06 18:55:20.812198 ip-10-0-130-59 kubenswrapper[2570]: W0706 18:55:20.744273 2570 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 06 18:55:20.812198 ip-10-0-130-59 kubenswrapper[2570]: W0706 18:55:20.744275 2570 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 06 18:55:20.812198 ip-10-0-130-59 kubenswrapper[2570]: W0706 18:55:20.744278 2570 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 06 18:55:20.812198 ip-10-0-130-59 kubenswrapper[2570]: W0706 18:55:20.744281 2570 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 06 18:55:20.812198 ip-10-0-130-59 kubenswrapper[2570]: W0706 18:55:20.744283 2570 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 06 18:55:20.812198 ip-10-0-130-59 kubenswrapper[2570]: W0706 18:55:20.744286 2570 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 06 18:55:20.812198 ip-10-0-130-59 kubenswrapper[2570]: W0706 18:55:20.744289 2570 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 06 18:55:20.812198 ip-10-0-130-59 kubenswrapper[2570]: W0706 18:55:20.744291 2570 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 06 18:55:20.812198 ip-10-0-130-59 kubenswrapper[2570]: W0706 18:55:20.744293 2570 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 06 18:55:20.812198 ip-10-0-130-59 kubenswrapper[2570]: W0706 18:55:20.744296 2570 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 06 18:55:20.812198 ip-10-0-130-59 kubenswrapper[2570]: W0706 18:55:20.744298 2570 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 06 18:55:20.812198 ip-10-0-130-59 kubenswrapper[2570]: W0706 18:55:20.744301 2570 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 06 18:55:20.813088 ip-10-0-130-59 kubenswrapper[2570]: W0706 18:55:20.744303 2570 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 06 18:55:20.813088 ip-10-0-130-59 kubenswrapper[2570]: W0706 18:55:20.744306 2570 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 06 18:55:20.813088 ip-10-0-130-59 kubenswrapper[2570]: W0706 18:55:20.744308 2570 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 06 18:55:20.813088 ip-10-0-130-59 kubenswrapper[2570]: W0706 18:55:20.744311 2570 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 06 18:55:20.813088 ip-10-0-130-59 kubenswrapper[2570]: W0706 18:55:20.744313 2570 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 06 18:55:20.813088 ip-10-0-130-59 kubenswrapper[2570]: W0706 18:55:20.744324 2570 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 06 18:55:20.813088 ip-10-0-130-59 kubenswrapper[2570]: W0706 18:55:20.744327 2570 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 06 18:55:20.813088 ip-10-0-130-59 kubenswrapper[2570]: W0706 18:55:20.744330 2570 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 06 18:55:20.813088 ip-10-0-130-59 kubenswrapper[2570]: W0706 18:55:20.744333 2570 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 06 18:55:20.813088 ip-10-0-130-59 kubenswrapper[2570]: W0706 18:55:20.744335 2570 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 06 18:55:20.813088 ip-10-0-130-59 kubenswrapper[2570]: W0706 18:55:20.744338 2570 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 06 18:55:20.813088 ip-10-0-130-59 kubenswrapper[2570]: W0706 18:55:20.744341 2570 feature_gate.go:328] unrecognized feature gate: Example Jul 06 18:55:20.813088 ip-10-0-130-59 kubenswrapper[2570]: W0706 18:55:20.744343 2570 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 06 18:55:20.813088 ip-10-0-130-59 kubenswrapper[2570]: W0706 18:55:20.744346 2570 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 06 18:55:20.813088 ip-10-0-130-59 kubenswrapper[2570]: W0706 18:55:20.744348 2570 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 06 18:55:20.813088 ip-10-0-130-59 kubenswrapper[2570]: W0706 18:55:20.744350 2570 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 06 18:55:20.813088 ip-10-0-130-59 kubenswrapper[2570]: W0706 18:55:20.744353 2570 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 06 18:55:20.813088 ip-10-0-130-59 kubenswrapper[2570]: W0706 18:55:20.744358 2570 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 06 18:55:20.813088 ip-10-0-130-59 kubenswrapper[2570]: W0706 18:55:20.744361 2570 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 06 18:55:20.813088 ip-10-0-130-59 kubenswrapper[2570]: W0706 18:55:20.744365 2570 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 06 18:55:20.813988 ip-10-0-130-59 kubenswrapper[2570]: W0706 18:55:20.744368 2570 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 06 18:55:20.813988 ip-10-0-130-59 kubenswrapper[2570]: W0706 18:55:20.744370 2570 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 06 18:55:20.813988 ip-10-0-130-59 kubenswrapper[2570]: W0706 18:55:20.744374 2570 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 06 18:55:20.813988 ip-10-0-130-59 kubenswrapper[2570]: W0706 18:55:20.744377 2570 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 06 18:55:20.813988 ip-10-0-130-59 kubenswrapper[2570]: W0706 18:55:20.744380 2570 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 06 18:55:20.813988 ip-10-0-130-59 kubenswrapper[2570]: W0706 18:55:20.744382 2570 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 06 18:55:20.813988 ip-10-0-130-59 kubenswrapper[2570]: W0706 18:55:20.744384 2570 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 06 18:55:20.813988 ip-10-0-130-59 kubenswrapper[2570]: W0706 18:55:20.744387 2570 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 06 18:55:20.813988 ip-10-0-130-59 kubenswrapper[2570]: W0706 18:55:20.744390 2570 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 06 18:55:20.813988 ip-10-0-130-59 kubenswrapper[2570]: W0706 18:55:20.744392 2570 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 06 18:55:20.813988 ip-10-0-130-59 kubenswrapper[2570]: W0706 18:55:20.744395 2570 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 06 18:55:20.813988 ip-10-0-130-59 kubenswrapper[2570]: W0706 18:55:20.744397 2570 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 06 18:55:20.813988 ip-10-0-130-59 kubenswrapper[2570]: W0706 18:55:20.744400 2570 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 06 18:55:20.813988 ip-10-0-130-59 kubenswrapper[2570]: W0706 18:55:20.744402 2570 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 06 18:55:20.813988 ip-10-0-130-59 kubenswrapper[2570]: W0706 18:55:20.744405 2570 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 06 18:55:20.813988 ip-10-0-130-59 kubenswrapper[2570]: W0706 18:55:20.744407 2570 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 06 18:55:20.813988 ip-10-0-130-59 kubenswrapper[2570]: W0706 18:55:20.744410 2570 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 06 18:55:20.813988 ip-10-0-130-59 kubenswrapper[2570]: W0706 18:55:20.744412 2570 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 06 18:55:20.813988 ip-10-0-130-59 kubenswrapper[2570]: W0706 18:55:20.744416 2570 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 06 18:55:20.813988 ip-10-0-130-59 kubenswrapper[2570]: W0706 18:55:20.744419 2570 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 06 18:55:20.814884 ip-10-0-130-59 kubenswrapper[2570]: W0706 18:55:20.744422 2570 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 06 18:55:20.814884 ip-10-0-130-59 kubenswrapper[2570]: W0706 18:55:20.744425 2570 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 06 18:55:20.814884 ip-10-0-130-59 kubenswrapper[2570]: W0706 18:55:20.744428 2570 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 06 18:55:20.814884 ip-10-0-130-59 kubenswrapper[2570]: W0706 18:55:20.744430 2570 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 06 18:55:20.814884 ip-10-0-130-59 kubenswrapper[2570]: W0706 18:55:20.744433 2570 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 06 18:55:20.814884 ip-10-0-130-59 kubenswrapper[2570]: W0706 18:55:20.744435 2570 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 06 18:55:20.814884 ip-10-0-130-59 kubenswrapper[2570]: W0706 18:55:20.744438 2570 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 06 18:55:20.814884 ip-10-0-130-59 kubenswrapper[2570]: W0706 18:55:20.744440 2570 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 06 18:55:20.814884 ip-10-0-130-59 kubenswrapper[2570]: W0706 18:55:20.744443 2570 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 06 18:55:20.814884 ip-10-0-130-59 kubenswrapper[2570]: W0706 18:55:20.744445 2570 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 06 18:55:20.814884 ip-10-0-130-59 kubenswrapper[2570]: W0706 18:55:20.744448 2570 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 06 18:55:20.814884 ip-10-0-130-59 kubenswrapper[2570]: W0706 18:55:20.744451 2570 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 06 18:55:20.814884 ip-10-0-130-59 kubenswrapper[2570]: W0706 18:55:20.745182 2570 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 06 18:55:20.814884 ip-10-0-130-59 kubenswrapper[2570]: W0706 18:55:20.745189 2570 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 06 18:55:20.814884 ip-10-0-130-59 kubenswrapper[2570]: W0706 18:55:20.745192 2570 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 06 18:55:20.814884 ip-10-0-130-59 kubenswrapper[2570]: W0706 18:55:20.745195 2570 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 06 18:55:20.814884 ip-10-0-130-59 kubenswrapper[2570]: W0706 18:55:20.745198 2570 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 06 18:55:20.814884 ip-10-0-130-59 kubenswrapper[2570]: W0706 18:55:20.745201 2570 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 06 18:55:20.814884 ip-10-0-130-59 kubenswrapper[2570]: W0706 18:55:20.745203 2570 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 06 18:55:20.814884 ip-10-0-130-59 kubenswrapper[2570]: W0706 18:55:20.745205 2570 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 06 18:55:20.815518 ip-10-0-130-59 kubenswrapper[2570]: W0706 18:55:20.745208 2570 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 06 18:55:20.815518 ip-10-0-130-59 kubenswrapper[2570]: W0706 18:55:20.745211 2570 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 06 18:55:20.815518 ip-10-0-130-59 kubenswrapper[2570]: W0706 18:55:20.745213 2570 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 06 18:55:20.815518 ip-10-0-130-59 kubenswrapper[2570]: W0706 18:55:20.745216 2570 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 06 18:55:20.815518 ip-10-0-130-59 kubenswrapper[2570]: W0706 18:55:20.745218 2570 feature_gate.go:328] unrecognized feature gate: Example Jul 06 18:55:20.815518 ip-10-0-130-59 kubenswrapper[2570]: W0706 18:55:20.745221 2570 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 06 18:55:20.815518 ip-10-0-130-59 kubenswrapper[2570]: W0706 18:55:20.745224 2570 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 06 18:55:20.815518 ip-10-0-130-59 kubenswrapper[2570]: W0706 18:55:20.745227 2570 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 06 18:55:20.815518 ip-10-0-130-59 kubenswrapper[2570]: W0706 18:55:20.745230 2570 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 06 18:55:20.815518 ip-10-0-130-59 kubenswrapper[2570]: W0706 18:55:20.745232 2570 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 06 18:55:20.815518 ip-10-0-130-59 kubenswrapper[2570]: W0706 18:55:20.745235 2570 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 06 18:55:20.815518 ip-10-0-130-59 kubenswrapper[2570]: W0706 18:55:20.745239 2570 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 06 18:55:20.815518 ip-10-0-130-59 kubenswrapper[2570]: W0706 18:55:20.745242 2570 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 06 18:55:20.815518 ip-10-0-130-59 kubenswrapper[2570]: W0706 18:55:20.745244 2570 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 06 18:55:20.815518 ip-10-0-130-59 kubenswrapper[2570]: W0706 18:55:20.745249 2570 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 06 18:55:20.815518 ip-10-0-130-59 kubenswrapper[2570]: W0706 18:55:20.745253 2570 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 06 18:55:20.815518 ip-10-0-130-59 kubenswrapper[2570]: W0706 18:55:20.745256 2570 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 06 18:55:20.815518 ip-10-0-130-59 kubenswrapper[2570]: W0706 18:55:20.745259 2570 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 06 18:55:20.815518 ip-10-0-130-59 kubenswrapper[2570]: W0706 18:55:20.745262 2570 feature_gate.go:328] unrecognized feature gate: Example2 Jul 06 18:55:20.815518 ip-10-0-130-59 kubenswrapper[2570]: W0706 18:55:20.745265 2570 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 06 18:55:20.816163 ip-10-0-130-59 kubenswrapper[2570]: W0706 18:55:20.745268 2570 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 06 18:55:20.816163 ip-10-0-130-59 kubenswrapper[2570]: W0706 18:55:20.745271 2570 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 06 18:55:20.816163 ip-10-0-130-59 kubenswrapper[2570]: W0706 18:55:20.745273 2570 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 06 18:55:20.816163 ip-10-0-130-59 kubenswrapper[2570]: W0706 18:55:20.745276 2570 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 06 18:55:20.816163 ip-10-0-130-59 kubenswrapper[2570]: W0706 18:55:20.745278 2570 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 06 18:55:20.816163 ip-10-0-130-59 kubenswrapper[2570]: W0706 18:55:20.745281 2570 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 06 18:55:20.816163 ip-10-0-130-59 kubenswrapper[2570]: W0706 18:55:20.745283 2570 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 06 18:55:20.816163 ip-10-0-130-59 kubenswrapper[2570]: W0706 18:55:20.745286 2570 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 06 18:55:20.816163 ip-10-0-130-59 kubenswrapper[2570]: W0706 18:55:20.745288 2570 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 06 18:55:20.816163 ip-10-0-130-59 kubenswrapper[2570]: W0706 18:55:20.745291 2570 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 06 18:55:20.816163 ip-10-0-130-59 kubenswrapper[2570]: W0706 18:55:20.745293 2570 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 06 18:55:20.816163 ip-10-0-130-59 kubenswrapper[2570]: W0706 18:55:20.745296 2570 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 06 18:55:20.816163 ip-10-0-130-59 kubenswrapper[2570]: W0706 18:55:20.745298 2570 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 06 18:55:20.816163 ip-10-0-130-59 kubenswrapper[2570]: W0706 18:55:20.745301 2570 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 06 18:55:20.816163 ip-10-0-130-59 kubenswrapper[2570]: W0706 18:55:20.745304 2570 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 06 18:55:20.816163 ip-10-0-130-59 kubenswrapper[2570]: W0706 18:55:20.745306 2570 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 06 18:55:20.816163 ip-10-0-130-59 kubenswrapper[2570]: W0706 18:55:20.745309 2570 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 06 18:55:20.816163 ip-10-0-130-59 kubenswrapper[2570]: W0706 18:55:20.745311 2570 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 06 18:55:20.816163 ip-10-0-130-59 kubenswrapper[2570]: W0706 18:55:20.745314 2570 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 06 18:55:20.816163 ip-10-0-130-59 kubenswrapper[2570]: W0706 18:55:20.745317 2570 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 06 18:55:20.816647 ip-10-0-130-59 systemd[1]: Started Kubernetes Kubelet. Jul 06 18:55:20.816849 ip-10-0-130-59 kubenswrapper[2570]: W0706 18:55:20.745319 2570 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 06 18:55:20.816849 ip-10-0-130-59 kubenswrapper[2570]: W0706 18:55:20.745322 2570 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 06 18:55:20.816849 ip-10-0-130-59 kubenswrapper[2570]: W0706 18:55:20.745324 2570 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 06 18:55:20.816849 ip-10-0-130-59 kubenswrapper[2570]: W0706 18:55:20.745328 2570 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 06 18:55:20.816849 ip-10-0-130-59 kubenswrapper[2570]: W0706 18:55:20.745337 2570 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 06 18:55:20.816849 ip-10-0-130-59 kubenswrapper[2570]: W0706 18:55:20.745340 2570 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 06 18:55:20.816849 ip-10-0-130-59 kubenswrapper[2570]: W0706 18:55:20.745342 2570 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 06 18:55:20.816849 ip-10-0-130-59 kubenswrapper[2570]: W0706 18:55:20.745345 2570 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 06 18:55:20.816849 ip-10-0-130-59 kubenswrapper[2570]: W0706 18:55:20.745347 2570 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 06 18:55:20.816849 ip-10-0-130-59 kubenswrapper[2570]: W0706 18:55:20.745350 2570 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 06 18:55:20.816849 ip-10-0-130-59 kubenswrapper[2570]: W0706 18:55:20.745352 2570 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 06 18:55:20.816849 ip-10-0-130-59 kubenswrapper[2570]: W0706 18:55:20.745355 2570 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 06 18:55:20.816849 ip-10-0-130-59 kubenswrapper[2570]: W0706 18:55:20.745359 2570 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 06 18:55:20.816849 ip-10-0-130-59 kubenswrapper[2570]: W0706 18:55:20.745362 2570 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 06 18:55:20.816849 ip-10-0-130-59 kubenswrapper[2570]: W0706 18:55:20.745365 2570 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 06 18:55:20.816849 ip-10-0-130-59 kubenswrapper[2570]: W0706 18:55:20.745368 2570 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 06 18:55:20.816849 ip-10-0-130-59 kubenswrapper[2570]: W0706 18:55:20.745370 2570 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 06 18:55:20.816849 ip-10-0-130-59 kubenswrapper[2570]: W0706 18:55:20.745385 2570 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 06 18:55:20.816849 ip-10-0-130-59 kubenswrapper[2570]: W0706 18:55:20.745388 2570 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 06 18:55:20.817952 ip-10-0-130-59 kubenswrapper[2570]: W0706 18:55:20.745391 2570 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 06 18:55:20.817952 ip-10-0-130-59 kubenswrapper[2570]: W0706 18:55:20.745394 2570 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 06 18:55:20.817952 ip-10-0-130-59 kubenswrapper[2570]: W0706 18:55:20.745397 2570 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 06 18:55:20.817952 ip-10-0-130-59 kubenswrapper[2570]: W0706 18:55:20.745399 2570 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 06 18:55:20.817952 ip-10-0-130-59 kubenswrapper[2570]: W0706 18:55:20.745402 2570 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 06 18:55:20.817952 ip-10-0-130-59 kubenswrapper[2570]: W0706 18:55:20.745405 2570 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 06 18:55:20.817952 ip-10-0-130-59 kubenswrapper[2570]: W0706 18:55:20.745409 2570 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 06 18:55:20.817952 ip-10-0-130-59 kubenswrapper[2570]: W0706 18:55:20.745412 2570 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 06 18:55:20.817952 ip-10-0-130-59 kubenswrapper[2570]: W0706 18:55:20.745415 2570 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 06 18:55:20.817952 ip-10-0-130-59 kubenswrapper[2570]: W0706 18:55:20.745418 2570 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 06 18:55:20.817952 ip-10-0-130-59 kubenswrapper[2570]: W0706 18:55:20.745421 2570 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 06 18:55:20.817952 ip-10-0-130-59 kubenswrapper[2570]: W0706 18:55:20.745424 2570 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 06 18:55:20.817952 ip-10-0-130-59 kubenswrapper[2570]: W0706 18:55:20.745426 2570 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 06 18:55:20.817952 ip-10-0-130-59 kubenswrapper[2570]: W0706 18:55:20.745428 2570 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 06 18:55:20.817952 ip-10-0-130-59 kubenswrapper[2570]: W0706 18:55:20.745431 2570 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 06 18:55:20.817952 ip-10-0-130-59 kubenswrapper[2570]: W0706 18:55:20.745433 2570 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 06 18:55:20.817952 ip-10-0-130-59 kubenswrapper[2570]: W0706 18:55:20.745436 2570 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 06 18:55:20.817952 ip-10-0-130-59 kubenswrapper[2570]: W0706 18:55:20.745439 2570 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 06 18:55:20.817952 ip-10-0-130-59 kubenswrapper[2570]: W0706 18:55:20.745447 2570 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 06 18:55:20.819016 ip-10-0-130-59 kubenswrapper[2570]: W0706 18:55:20.745450 2570 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 06 18:55:20.819016 ip-10-0-130-59 kubenswrapper[2570]: W0706 18:55:20.745452 2570 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 06 18:55:20.819016 ip-10-0-130-59 kubenswrapper[2570]: W0706 18:55:20.745455 2570 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 06 18:55:20.819016 ip-10-0-130-59 kubenswrapper[2570]: W0706 18:55:20.745458 2570 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 06 18:55:20.819016 ip-10-0-130-59 kubenswrapper[2570]: W0706 18:55:20.745460 2570 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 06 18:55:20.819016 ip-10-0-130-59 kubenswrapper[2570]: W0706 18:55:20.745463 2570 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 06 18:55:20.819016 ip-10-0-130-59 kubenswrapper[2570]: W0706 18:55:20.745465 2570 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 06 18:55:20.819016 ip-10-0-130-59 kubenswrapper[2570]: W0706 18:55:20.745468 2570 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 06 18:55:20.819016 ip-10-0-130-59 kubenswrapper[2570]: W0706 18:55:20.745471 2570 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 06 18:55:20.819016 ip-10-0-130-59 kubenswrapper[2570]: W0706 18:55:20.745474 2570 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 06 18:55:20.819016 ip-10-0-130-59 kubenswrapper[2570]: W0706 18:55:20.745476 2570 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 06 18:55:20.819016 ip-10-0-130-59 kubenswrapper[2570]: W0706 18:55:20.745479 2570 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 06 18:55:20.819016 ip-10-0-130-59 kubenswrapper[2570]: W0706 18:55:20.745481 2570 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 06 18:55:20.819016 ip-10-0-130-59 kubenswrapper[2570]: W0706 18:55:20.745899 2570 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 06 18:55:20.819016 ip-10-0-130-59 kubenswrapper[2570]: W0706 18:55:20.745909 2570 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 06 18:55:20.819016 ip-10-0-130-59 kubenswrapper[2570]: W0706 18:55:20.745912 2570 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 06 18:55:20.819016 ip-10-0-130-59 kubenswrapper[2570]: W0706 18:55:20.745915 2570 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 06 18:55:20.819016 ip-10-0-130-59 kubenswrapper[2570]: W0706 18:55:20.745918 2570 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 06 18:55:20.819016 ip-10-0-130-59 kubenswrapper[2570]: W0706 18:55:20.745920 2570 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 06 18:55:20.819016 ip-10-0-130-59 kubenswrapper[2570]: W0706 18:55:20.745923 2570 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 06 18:55:20.820052 ip-10-0-130-59 kubenswrapper[2570]: W0706 18:55:20.745927 2570 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 06 18:55:20.820052 ip-10-0-130-59 kubenswrapper[2570]: W0706 18:55:20.745930 2570 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 06 18:55:20.820052 ip-10-0-130-59 kubenswrapper[2570]: W0706 18:55:20.745932 2570 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 06 18:55:20.820052 ip-10-0-130-59 kubenswrapper[2570]: W0706 18:55:20.745935 2570 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 06 18:55:20.820052 ip-10-0-130-59 kubenswrapper[2570]: W0706 18:55:20.745937 2570 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 06 18:55:20.820052 ip-10-0-130-59 kubenswrapper[2570]: W0706 18:55:20.745940 2570 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 06 18:55:20.820052 ip-10-0-130-59 kubenswrapper[2570]: W0706 18:55:20.745942 2570 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 06 18:55:20.820052 ip-10-0-130-59 kubenswrapper[2570]: W0706 18:55:20.745945 2570 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 06 18:55:20.820052 ip-10-0-130-59 kubenswrapper[2570]: W0706 18:55:20.745948 2570 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 06 18:55:20.820052 ip-10-0-130-59 kubenswrapper[2570]: W0706 18:55:20.745950 2570 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 06 18:55:20.820052 ip-10-0-130-59 kubenswrapper[2570]: W0706 18:55:20.745953 2570 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 06 18:55:20.820052 ip-10-0-130-59 kubenswrapper[2570]: W0706 18:55:20.745959 2570 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 06 18:55:20.820052 ip-10-0-130-59 kubenswrapper[2570]: W0706 18:55:20.745961 2570 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 06 18:55:20.820052 ip-10-0-130-59 kubenswrapper[2570]: W0706 18:55:20.745966 2570 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 06 18:55:20.820052 ip-10-0-130-59 kubenswrapper[2570]: W0706 18:55:20.745969 2570 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 06 18:55:20.820052 ip-10-0-130-59 kubenswrapper[2570]: W0706 18:55:20.745972 2570 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 06 18:55:20.820052 ip-10-0-130-59 kubenswrapper[2570]: W0706 18:55:20.745975 2570 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 06 18:55:20.820052 ip-10-0-130-59 kubenswrapper[2570]: W0706 18:55:20.745978 2570 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 06 18:55:20.820052 ip-10-0-130-59 kubenswrapper[2570]: W0706 18:55:20.745980 2570 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 06 18:55:20.821035 ip-10-0-130-59 kubenswrapper[2570]: W0706 18:55:20.745983 2570 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 06 18:55:20.821035 ip-10-0-130-59 kubenswrapper[2570]: W0706 18:55:20.745989 2570 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 06 18:55:20.821035 ip-10-0-130-59 kubenswrapper[2570]: W0706 18:55:20.745991 2570 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 06 18:55:20.821035 ip-10-0-130-59 kubenswrapper[2570]: W0706 18:55:20.745994 2570 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 06 18:55:20.821035 ip-10-0-130-59 kubenswrapper[2570]: W0706 18:55:20.745997 2570 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 06 18:55:20.821035 ip-10-0-130-59 kubenswrapper[2570]: W0706 18:55:20.746001 2570 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 06 18:55:20.821035 ip-10-0-130-59 kubenswrapper[2570]: W0706 18:55:20.746004 2570 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 06 18:55:20.821035 ip-10-0-130-59 kubenswrapper[2570]: W0706 18:55:20.746007 2570 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 06 18:55:20.821035 ip-10-0-130-59 kubenswrapper[2570]: W0706 18:55:20.746010 2570 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 06 18:55:20.821035 ip-10-0-130-59 kubenswrapper[2570]: W0706 18:55:20.746014 2570 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 06 18:55:20.821035 ip-10-0-130-59 kubenswrapper[2570]: W0706 18:55:20.746016 2570 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 06 18:55:20.821035 ip-10-0-130-59 kubenswrapper[2570]: W0706 18:55:20.746019 2570 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 06 18:55:20.821035 ip-10-0-130-59 kubenswrapper[2570]: W0706 18:55:20.746022 2570 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 06 18:55:20.821035 ip-10-0-130-59 kubenswrapper[2570]: W0706 18:55:20.746024 2570 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 06 18:55:20.821035 ip-10-0-130-59 kubenswrapper[2570]: W0706 18:55:20.746027 2570 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 06 18:55:20.821035 ip-10-0-130-59 kubenswrapper[2570]: W0706 18:55:20.746029 2570 feature_gate.go:328] unrecognized feature gate: Example2 Jul 06 18:55:20.821035 ip-10-0-130-59 kubenswrapper[2570]: W0706 18:55:20.746031 2570 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 06 18:55:20.821035 ip-10-0-130-59 kubenswrapper[2570]: W0706 18:55:20.746034 2570 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 06 18:55:20.821035 ip-10-0-130-59 kubenswrapper[2570]: W0706 18:55:20.746036 2570 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 06 18:55:20.821035 ip-10-0-130-59 kubenswrapper[2570]: W0706 18:55:20.746039 2570 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 06 18:55:20.822413 ip-10-0-130-59 kubenswrapper[2570]: W0706 18:55:20.746041 2570 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 06 18:55:20.822413 ip-10-0-130-59 kubenswrapper[2570]: W0706 18:55:20.746044 2570 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 06 18:55:20.822413 ip-10-0-130-59 kubenswrapper[2570]: W0706 18:55:20.746047 2570 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 06 18:55:20.822413 ip-10-0-130-59 kubenswrapper[2570]: W0706 18:55:20.746049 2570 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 06 18:55:20.822413 ip-10-0-130-59 kubenswrapper[2570]: W0706 18:55:20.746052 2570 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 06 18:55:20.822413 ip-10-0-130-59 kubenswrapper[2570]: W0706 18:55:20.746061 2570 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 06 18:55:20.822413 ip-10-0-130-59 kubenswrapper[2570]: W0706 18:55:20.746064 2570 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 06 18:55:20.822413 ip-10-0-130-59 kubenswrapper[2570]: W0706 18:55:20.746067 2570 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 06 18:55:20.822413 ip-10-0-130-59 kubenswrapper[2570]: W0706 18:55:20.746069 2570 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 06 18:55:20.822413 ip-10-0-130-59 kubenswrapper[2570]: W0706 18:55:20.746071 2570 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 06 18:55:20.822413 ip-10-0-130-59 kubenswrapper[2570]: W0706 18:55:20.746076 2570 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 06 18:55:20.822413 ip-10-0-130-59 kubenswrapper[2570]: W0706 18:55:20.746078 2570 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 06 18:55:20.822413 ip-10-0-130-59 kubenswrapper[2570]: W0706 18:55:20.746081 2570 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 06 18:55:20.822413 ip-10-0-130-59 kubenswrapper[2570]: W0706 18:55:20.746084 2570 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 06 18:55:20.822413 ip-10-0-130-59 kubenswrapper[2570]: W0706 18:55:20.746086 2570 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 06 18:55:20.822413 ip-10-0-130-59 kubenswrapper[2570]: W0706 18:55:20.746089 2570 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 06 18:55:20.822413 ip-10-0-130-59 kubenswrapper[2570]: W0706 18:55:20.746091 2570 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 06 18:55:20.822413 ip-10-0-130-59 kubenswrapper[2570]: W0706 18:55:20.746094 2570 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 06 18:55:20.822413 ip-10-0-130-59 kubenswrapper[2570]: W0706 18:55:20.746097 2570 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 06 18:55:20.823287 ip-10-0-130-59 kubenswrapper[2570]: W0706 18:55:20.746101 2570 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 06 18:55:20.823287 ip-10-0-130-59 kubenswrapper[2570]: W0706 18:55:20.746103 2570 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 06 18:55:20.823287 ip-10-0-130-59 kubenswrapper[2570]: W0706 18:55:20.746106 2570 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 06 18:55:20.823287 ip-10-0-130-59 kubenswrapper[2570]: W0706 18:55:20.746108 2570 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 06 18:55:20.823287 ip-10-0-130-59 kubenswrapper[2570]: W0706 18:55:20.746111 2570 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 06 18:55:20.823287 ip-10-0-130-59 kubenswrapper[2570]: W0706 18:55:20.746113 2570 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 06 18:55:20.823287 ip-10-0-130-59 kubenswrapper[2570]: W0706 18:55:20.746116 2570 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 06 18:55:20.823287 ip-10-0-130-59 kubenswrapper[2570]: W0706 18:55:20.746118 2570 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 06 18:55:20.823287 ip-10-0-130-59 kubenswrapper[2570]: W0706 18:55:20.746121 2570 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 06 18:55:20.823287 ip-10-0-130-59 kubenswrapper[2570]: W0706 18:55:20.746124 2570 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 06 18:55:20.823287 ip-10-0-130-59 kubenswrapper[2570]: W0706 18:55:20.746126 2570 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 06 18:55:20.823287 ip-10-0-130-59 kubenswrapper[2570]: W0706 18:55:20.746129 2570 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 06 18:55:20.823287 ip-10-0-130-59 kubenswrapper[2570]: W0706 18:55:20.746132 2570 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 06 18:55:20.823287 ip-10-0-130-59 kubenswrapper[2570]: W0706 18:55:20.746138 2570 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 06 18:55:20.823287 ip-10-0-130-59 kubenswrapper[2570]: W0706 18:55:20.746141 2570 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 06 18:55:20.823287 ip-10-0-130-59 kubenswrapper[2570]: W0706 18:55:20.746144 2570 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 06 18:55:20.823287 ip-10-0-130-59 kubenswrapper[2570]: W0706 18:55:20.746146 2570 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 06 18:55:20.823287 ip-10-0-130-59 kubenswrapper[2570]: W0706 18:55:20.746149 2570 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 06 18:55:20.823287 ip-10-0-130-59 kubenswrapper[2570]: W0706 18:55:20.746151 2570 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 06 18:55:20.823287 ip-10-0-130-59 kubenswrapper[2570]: W0706 18:55:20.746159 2570 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 06 18:55:20.824075 ip-10-0-130-59 kubenswrapper[2570]: W0706 18:55:20.746162 2570 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 06 18:55:20.824075 ip-10-0-130-59 kubenswrapper[2570]: W0706 18:55:20.746164 2570 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 06 18:55:20.824075 ip-10-0-130-59 kubenswrapper[2570]: W0706 18:55:20.746169 2570 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 06 18:55:20.824075 ip-10-0-130-59 kubenswrapper[2570]: W0706 18:55:20.746173 2570 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 06 18:55:20.824075 ip-10-0-130-59 kubenswrapper[2570]: W0706 18:55:20.746175 2570 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 06 18:55:20.824075 ip-10-0-130-59 kubenswrapper[2570]: W0706 18:55:20.746177 2570 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 06 18:55:20.824075 ip-10-0-130-59 kubenswrapper[2570]: W0706 18:55:20.746180 2570 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 06 18:55:20.824075 ip-10-0-130-59 kubenswrapper[2570]: W0706 18:55:20.746183 2570 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 06 18:55:20.824075 ip-10-0-130-59 kubenswrapper[2570]: W0706 18:55:20.746185 2570 feature_gate.go:328] unrecognized feature gate: Example Jul 06 18:55:20.824075 ip-10-0-130-59 kubenswrapper[2570]: W0706 18:55:20.746187 2570 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 06 18:55:20.824075 ip-10-0-130-59 kubenswrapper[2570]: W0706 18:55:20.746190 2570 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 06 18:55:20.824075 ip-10-0-130-59 kubenswrapper[2570]: W0706 18:55:20.746193 2570 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 06 18:55:20.824075 ip-10-0-130-59 kubenswrapper[2570]: W0706 18:55:20.746195 2570 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 06 18:55:20.824075 ip-10-0-130-59 kubenswrapper[2570]: W0706 18:55:20.746198 2570 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 06 18:55:20.824075 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:55:20.746268 2570 flags.go:64] FLAG: --address="0.0.0.0" Jul 06 18:55:20.824075 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:55:20.746278 2570 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]" Jul 06 18:55:20.824075 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:55:20.746289 2570 flags.go:64] FLAG: --anonymous-auth="true" Jul 06 18:55:20.824075 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:55:20.746294 2570 flags.go:64] FLAG: --application-metrics-count-limit="100" Jul 06 18:55:20.824075 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:55:20.746298 2570 flags.go:64] FLAG: --authentication-token-webhook="false" Jul 06 18:55:20.824075 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:55:20.746301 2570 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s" Jul 06 18:55:20.824075 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:55:20.746305 2570 flags.go:64] FLAG: --authorization-mode="AlwaysAllow" Jul 06 18:55:20.824075 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:55:20.746309 2570 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s" Jul 06 18:55:20.824696 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:55:20.746312 2570 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s" Jul 06 18:55:20.824696 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:55:20.746315 2570 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id" Jul 06 18:55:20.824696 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:55:20.746318 2570 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig" Jul 06 18:55:20.824696 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:55:20.746321 2570 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki" Jul 06 18:55:20.824696 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:55:20.746325 2570 flags.go:64] FLAG: --cgroup-driver="cgroupfs" Jul 06 18:55:20.824696 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:55:20.746327 2570 flags.go:64] FLAG: --cgroup-root="" Jul 06 18:55:20.824696 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:55:20.746330 2570 flags.go:64] FLAG: --cgroups-per-qos="true" Jul 06 18:55:20.824696 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:55:20.746334 2570 flags.go:64] FLAG: --client-ca-file="" Jul 06 18:55:20.824696 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:55:20.746336 2570 flags.go:64] FLAG: --cloud-provider="external" Jul 06 18:55:20.824696 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:55:20.746339 2570 flags.go:64] FLAG: --cluster-dns="[]" Jul 06 18:55:20.824696 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:55:20.746352 2570 flags.go:64] FLAG: --cluster-domain="" Jul 06 18:55:20.824696 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:55:20.746355 2570 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf" Jul 06 18:55:20.824696 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:55:20.746360 2570 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d" Jul 06 18:55:20.824696 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:55:20.746364 2570 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json" Jul 06 18:55:20.824696 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:55:20.746367 2570 flags.go:64] FLAG: --container-log-max-files="5" Jul 06 18:55:20.824696 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:55:20.746371 2570 flags.go:64] FLAG: --container-log-max-size="10Mi" Jul 06 18:55:20.824696 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:55:20.746374 2570 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock" Jul 06 18:55:20.824696 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:55:20.746377 2570 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock" Jul 06 18:55:20.824696 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:55:20.746380 2570 flags.go:64] FLAG: --containerd-namespace="k8s.io" Jul 06 18:55:20.824696 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:55:20.746384 2570 flags.go:64] FLAG: --contention-profiling="false" Jul 06 18:55:20.824696 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:55:20.746386 2570 flags.go:64] FLAG: --cpu-cfs-quota="true" Jul 06 18:55:20.824696 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:55:20.746389 2570 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms" Jul 06 18:55:20.824696 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:55:20.746393 2570 flags.go:64] FLAG: --cpu-manager-policy="none" Jul 06 18:55:20.824696 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:55:20.746396 2570 flags.go:64] FLAG: --cpu-manager-policy-options="" Jul 06 18:55:20.825385 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:55:20.746400 2570 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s" Jul 06 18:55:20.825385 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:55:20.746403 2570 flags.go:64] FLAG: --enable-controller-attach-detach="true" Jul 06 18:55:20.825385 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:55:20.746405 2570 flags.go:64] FLAG: --enable-debugging-handlers="true" Jul 06 18:55:20.825385 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:55:20.746408 2570 flags.go:64] FLAG: --enable-load-reader="false" Jul 06 18:55:20.825385 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:55:20.746411 2570 flags.go:64] FLAG: --enable-server="true" Jul 06 18:55:20.825385 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:55:20.746414 2570 flags.go:64] FLAG: --enforce-node-allocatable="[pods]" Jul 06 18:55:20.825385 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:55:20.746420 2570 flags.go:64] FLAG: --event-burst="100" Jul 06 18:55:20.825385 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:55:20.746423 2570 flags.go:64] FLAG: --event-qps="50" Jul 06 18:55:20.825385 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:55:20.746426 2570 flags.go:64] FLAG: --event-storage-age-limit="default=0" Jul 06 18:55:20.825385 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:55:20.746429 2570 flags.go:64] FLAG: --event-storage-event-limit="default=0" Jul 06 18:55:20.825385 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:55:20.746432 2570 flags.go:64] FLAG: --eviction-hard="" Jul 06 18:55:20.825385 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:55:20.746435 2570 flags.go:64] FLAG: --eviction-max-pod-grace-period="0" Jul 06 18:55:20.825385 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:55:20.746438 2570 flags.go:64] FLAG: --eviction-minimum-reclaim="" Jul 06 18:55:20.825385 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:55:20.746441 2570 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s" Jul 06 18:55:20.825385 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:55:20.746444 2570 flags.go:64] FLAG: --eviction-soft="" Jul 06 18:55:20.825385 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:55:20.746447 2570 flags.go:64] FLAG: --eviction-soft-grace-period="" Jul 06 18:55:20.825385 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:55:20.746450 2570 flags.go:64] FLAG: --exit-on-lock-contention="false" Jul 06 18:55:20.825385 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:55:20.746453 2570 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false" Jul 06 18:55:20.825385 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:55:20.746455 2570 flags.go:64] FLAG: --experimental-mounter-path="" Jul 06 18:55:20.825385 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:55:20.746464 2570 flags.go:64] FLAG: --fail-cgroupv1="false" Jul 06 18:55:20.825385 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:55:20.746468 2570 flags.go:64] FLAG: --fail-swap-on="true" Jul 06 18:55:20.825385 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:55:20.746473 2570 flags.go:64] FLAG: --feature-gates="" Jul 06 18:55:20.825385 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:55:20.746476 2570 flags.go:64] FLAG: --file-check-frequency="20s" Jul 06 18:55:20.825385 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:55:20.746479 2570 flags.go:64] FLAG: --global-housekeeping-interval="1m0s" Jul 06 18:55:20.825385 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:55:20.746482 2570 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge" Jul 06 18:55:20.826073 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:55:20.746486 2570 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1" Jul 06 18:55:20.826073 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:55:20.746488 2570 flags.go:64] FLAG: --healthz-port="10248" Jul 06 18:55:20.826073 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:55:20.746491 2570 flags.go:64] FLAG: --help="false" Jul 06 18:55:20.826073 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:55:20.746494 2570 flags.go:64] FLAG: --hostname-override="ip-10-0-130-59.ec2.internal" Jul 06 18:55:20.826073 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:55:20.746497 2570 flags.go:64] FLAG: --housekeeping-interval="10s" Jul 06 18:55:20.826073 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:55:20.746500 2570 flags.go:64] FLAG: --http-check-frequency="20s" Jul 06 18:55:20.826073 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:55:20.746503 2570 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins" Jul 06 18:55:20.826073 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:55:20.746507 2570 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml" Jul 06 18:55:20.826073 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:55:20.746510 2570 flags.go:64] FLAG: --image-gc-high-threshold="85" Jul 06 18:55:20.826073 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:55:20.746513 2570 flags.go:64] FLAG: --image-gc-low-threshold="80" Jul 06 18:55:20.826073 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:55:20.746516 2570 flags.go:64] FLAG: --image-service-endpoint="" Jul 06 18:55:20.826073 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:55:20.746518 2570 flags.go:64] FLAG: --kernel-memcg-notification="false" Jul 06 18:55:20.826073 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:55:20.746521 2570 flags.go:64] FLAG: --kube-api-burst="100" Jul 06 18:55:20.826073 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:55:20.746524 2570 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf" Jul 06 18:55:20.826073 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:55:20.746527 2570 flags.go:64] FLAG: --kube-api-qps="50" Jul 06 18:55:20.826073 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:55:20.746530 2570 flags.go:64] FLAG: --kube-reserved="" Jul 06 18:55:20.826073 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:55:20.746533 2570 flags.go:64] FLAG: --kube-reserved-cgroup="" Jul 06 18:55:20.826073 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:55:20.746536 2570 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig" Jul 06 18:55:20.826073 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:55:20.746539 2570 flags.go:64] FLAG: --kubelet-cgroups="" Jul 06 18:55:20.826073 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:55:20.746541 2570 flags.go:64] FLAG: --local-storage-capacity-isolation="true" Jul 06 18:55:20.826073 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:55:20.746544 2570 flags.go:64] FLAG: --lock-file="" Jul 06 18:55:20.826073 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:55:20.746547 2570 flags.go:64] FLAG: --log-cadvisor-usage="false" Jul 06 18:55:20.826073 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:55:20.746550 2570 flags.go:64] FLAG: --log-flush-frequency="5s" Jul 06 18:55:20.826073 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:55:20.746553 2570 flags.go:64] FLAG: --log-json-info-buffer-size="0" Jul 06 18:55:20.826750 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:55:20.746558 2570 flags.go:64] FLAG: --log-json-split-stream="false" Jul 06 18:55:20.826750 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:55:20.746561 2570 flags.go:64] FLAG: --log-text-info-buffer-size="0" Jul 06 18:55:20.826750 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:55:20.746563 2570 flags.go:64] FLAG: --log-text-split-stream="false" Jul 06 18:55:20.826750 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:55:20.746575 2570 flags.go:64] FLAG: --logging-format="text" Jul 06 18:55:20.826750 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:55:20.746579 2570 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id" Jul 06 18:55:20.826750 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:55:20.746582 2570 flags.go:64] FLAG: --make-iptables-util-chains="true" Jul 06 18:55:20.826750 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:55:20.746585 2570 flags.go:64] FLAG: --manifest-url="" Jul 06 18:55:20.826750 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:55:20.746588 2570 flags.go:64] FLAG: --manifest-url-header="" Jul 06 18:55:20.826750 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:55:20.746592 2570 flags.go:64] FLAG: --max-housekeeping-interval="15s" Jul 06 18:55:20.826750 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:55:20.746595 2570 flags.go:64] FLAG: --max-open-files="1000000" Jul 06 18:55:20.826750 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:55:20.746599 2570 flags.go:64] FLAG: --max-pods="110" Jul 06 18:55:20.826750 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:55:20.746602 2570 flags.go:64] FLAG: --maximum-dead-containers="-1" Jul 06 18:55:20.826750 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:55:20.746605 2570 flags.go:64] FLAG: --maximum-dead-containers-per-container="1" Jul 06 18:55:20.826750 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:55:20.746625 2570 flags.go:64] FLAG: --memory-manager-policy="None" Jul 06 18:55:20.826750 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:55:20.746630 2570 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s" Jul 06 18:55:20.826750 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:55:20.746634 2570 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s" Jul 06 18:55:20.826750 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:55:20.746639 2570 flags.go:64] FLAG: --node-ip="0.0.0.0" Jul 06 18:55:20.826750 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:55:20.746642 2570 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel" Jul 06 18:55:20.826750 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:55:20.746649 2570 flags.go:64] FLAG: --node-status-max-images="50" Jul 06 18:55:20.826750 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:55:20.746652 2570 flags.go:64] FLAG: --node-status-update-frequency="10s" Jul 06 18:55:20.826750 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:55:20.746655 2570 flags.go:64] FLAG: --oom-score-adj="-999" Jul 06 18:55:20.826750 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:55:20.746658 2570 flags.go:64] FLAG: --pod-cidr="" Jul 06 18:55:20.826750 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:55:20.746661 2570 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:ce93f6b4197b9c62a26a3131f8057ffe9c9379c5079a4f62e5919b75151394e5" Jul 06 18:55:20.827662 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:55:20.746670 2570 flags.go:64] FLAG: --pod-manifest-path="" Jul 06 18:55:20.827662 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:55:20.746672 2570 flags.go:64] FLAG: --pod-max-pids="-1" Jul 06 18:55:20.827662 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:55:20.746675 2570 flags.go:64] FLAG: --pods-per-core="0" Jul 06 18:55:20.827662 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:55:20.746678 2570 flags.go:64] FLAG: --port="10250" Jul 06 18:55:20.827662 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:55:20.746681 2570 flags.go:64] FLAG: --protect-kernel-defaults="false" Jul 06 18:55:20.827662 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:55:20.746684 2570 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-07a005783adbb0c52" Jul 06 18:55:20.827662 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:55:20.746687 2570 flags.go:64] FLAG: --qos-reserved="" Jul 06 18:55:20.827662 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:55:20.746690 2570 flags.go:64] FLAG: --read-only-port="10255" Jul 06 18:55:20.827662 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:55:20.746693 2570 flags.go:64] FLAG: --register-node="true" Jul 06 18:55:20.827662 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:55:20.746696 2570 flags.go:64] FLAG: --register-with-taints="" Jul 06 18:55:20.827662 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:55:20.746699 2570 flags.go:64] FLAG: --registry-burst="10" Jul 06 18:55:20.827662 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:55:20.746702 2570 flags.go:64] FLAG: --registry-qps="5" Jul 06 18:55:20.827662 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:55:20.746705 2570 flags.go:64] FLAG: --reserved-cpus="" Jul 06 18:55:20.827662 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:55:20.746716 2570 flags.go:64] FLAG: --reserved-memory="" Jul 06 18:55:20.827662 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:55:20.746719 2570 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf" Jul 06 18:55:20.827662 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:55:20.746722 2570 flags.go:64] FLAG: --root-dir="/var/lib/kubelet" Jul 06 18:55:20.827662 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:55:20.746725 2570 flags.go:64] FLAG: --rotate-certificates="false" Jul 06 18:55:20.827662 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:55:20.746728 2570 flags.go:64] FLAG: --rotate-server-certificates="false" Jul 06 18:55:20.827662 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:55:20.746731 2570 flags.go:64] FLAG: --runonce="false" Jul 06 18:55:20.827662 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:55:20.746734 2570 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service" Jul 06 18:55:20.827662 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:55:20.746737 2570 flags.go:64] FLAG: --runtime-request-timeout="2m0s" Jul 06 18:55:20.827662 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:55:20.746740 2570 flags.go:64] FLAG: --seccomp-default="false" Jul 06 18:55:20.827662 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:55:20.746742 2570 flags.go:64] FLAG: --serialize-image-pulls="true" Jul 06 18:55:20.827662 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:55:20.746745 2570 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s" Jul 06 18:55:20.827662 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:55:20.746748 2570 flags.go:64] FLAG: --storage-driver-db="cadvisor" Jul 06 18:55:20.827662 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:55:20.746751 2570 flags.go:64] FLAG: --storage-driver-host="localhost:8086" Jul 06 18:55:20.828765 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:55:20.746754 2570 flags.go:64] FLAG: --storage-driver-password="root" Jul 06 18:55:20.828765 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:55:20.746756 2570 flags.go:64] FLAG: --storage-driver-secure="false" Jul 06 18:55:20.828765 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:55:20.746759 2570 flags.go:64] FLAG: --storage-driver-table="stats" Jul 06 18:55:20.828765 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:55:20.746762 2570 flags.go:64] FLAG: --storage-driver-user="root" Jul 06 18:55:20.828765 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:55:20.746765 2570 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s" Jul 06 18:55:20.828765 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:55:20.746768 2570 flags.go:64] FLAG: --sync-frequency="1m0s" Jul 06 18:55:20.828765 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:55:20.746771 2570 flags.go:64] FLAG: --system-cgroups="" Jul 06 18:55:20.828765 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:55:20.746773 2570 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi" Jul 06 18:55:20.828765 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:55:20.746778 2570 flags.go:64] FLAG: --system-reserved-cgroup="" Jul 06 18:55:20.828765 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:55:20.746781 2570 flags.go:64] FLAG: --tls-cert-file="" Jul 06 18:55:20.828765 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:55:20.746783 2570 flags.go:64] FLAG: --tls-cipher-suites="[]" Jul 06 18:55:20.828765 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:55:20.746790 2570 flags.go:64] FLAG: --tls-min-version="" Jul 06 18:55:20.828765 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:55:20.746793 2570 flags.go:64] FLAG: --tls-private-key-file="" Jul 06 18:55:20.828765 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:55:20.746796 2570 flags.go:64] FLAG: --topology-manager-policy="none" Jul 06 18:55:20.828765 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:55:20.746799 2570 flags.go:64] FLAG: --topology-manager-policy-options="" Jul 06 18:55:20.828765 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:55:20.746801 2570 flags.go:64] FLAG: --topology-manager-scope="container" Jul 06 18:55:20.828765 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:55:20.746804 2570 flags.go:64] FLAG: --v="2" Jul 06 18:55:20.828765 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:55:20.746808 2570 flags.go:64] FLAG: --version="false" Jul 06 18:55:20.828765 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:55:20.746811 2570 flags.go:64] FLAG: --vmodule="" Jul 06 18:55:20.828765 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:55:20.746818 2570 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec" Jul 06 18:55:20.828765 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:55:20.746827 2570 flags.go:64] FLAG: --volume-stats-agg-period="1m0s" Jul 06 18:55:20.828765 ip-10-0-130-59 kubenswrapper[2570]: W0706 18:55:20.747185 2570 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 06 18:55:20.828765 ip-10-0-130-59 kubenswrapper[2570]: W0706 18:55:20.747198 2570 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 06 18:55:20.828765 ip-10-0-130-59 kubenswrapper[2570]: W0706 18:55:20.747203 2570 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 06 18:55:20.828765 ip-10-0-130-59 kubenswrapper[2570]: W0706 18:55:20.747209 2570 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 06 18:55:20.830051 ip-10-0-130-59 kubenswrapper[2570]: W0706 18:55:20.747214 2570 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 06 18:55:20.830051 ip-10-0-130-59 kubenswrapper[2570]: W0706 18:55:20.747218 2570 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 06 18:55:20.830051 ip-10-0-130-59 kubenswrapper[2570]: W0706 18:55:20.747222 2570 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 06 18:55:20.830051 ip-10-0-130-59 kubenswrapper[2570]: W0706 18:55:20.747226 2570 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 06 18:55:20.830051 ip-10-0-130-59 kubenswrapper[2570]: W0706 18:55:20.747230 2570 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 06 18:55:20.830051 ip-10-0-130-59 kubenswrapper[2570]: W0706 18:55:20.747242 2570 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 06 18:55:20.830051 ip-10-0-130-59 kubenswrapper[2570]: W0706 18:55:20.747247 2570 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 06 18:55:20.830051 ip-10-0-130-59 kubenswrapper[2570]: W0706 18:55:20.747252 2570 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 06 18:55:20.830051 ip-10-0-130-59 kubenswrapper[2570]: W0706 18:55:20.747256 2570 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 06 18:55:20.830051 ip-10-0-130-59 kubenswrapper[2570]: W0706 18:55:20.747261 2570 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 06 18:55:20.830051 ip-10-0-130-59 kubenswrapper[2570]: W0706 18:55:20.747265 2570 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 06 18:55:20.830051 ip-10-0-130-59 kubenswrapper[2570]: W0706 18:55:20.747269 2570 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 06 18:55:20.830051 ip-10-0-130-59 kubenswrapper[2570]: W0706 18:55:20.747273 2570 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 06 18:55:20.830051 ip-10-0-130-59 kubenswrapper[2570]: W0706 18:55:20.747277 2570 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 06 18:55:20.830051 ip-10-0-130-59 kubenswrapper[2570]: W0706 18:55:20.747281 2570 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 06 18:55:20.830051 ip-10-0-130-59 kubenswrapper[2570]: W0706 18:55:20.747285 2570 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 06 18:55:20.830051 ip-10-0-130-59 kubenswrapper[2570]: W0706 18:55:20.747290 2570 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 06 18:55:20.830051 ip-10-0-130-59 kubenswrapper[2570]: W0706 18:55:20.747298 2570 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 06 18:55:20.830051 ip-10-0-130-59 kubenswrapper[2570]: W0706 18:55:20.747302 2570 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 06 18:55:20.830051 ip-10-0-130-59 kubenswrapper[2570]: W0706 18:55:20.747307 2570 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 06 18:55:20.831072 ip-10-0-130-59 kubenswrapper[2570]: W0706 18:55:20.747311 2570 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 06 18:55:20.831072 ip-10-0-130-59 kubenswrapper[2570]: W0706 18:55:20.747315 2570 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 06 18:55:20.831072 ip-10-0-130-59 kubenswrapper[2570]: W0706 18:55:20.747319 2570 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 06 18:55:20.831072 ip-10-0-130-59 kubenswrapper[2570]: W0706 18:55:20.747324 2570 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 06 18:55:20.831072 ip-10-0-130-59 kubenswrapper[2570]: W0706 18:55:20.747328 2570 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 06 18:55:20.831072 ip-10-0-130-59 kubenswrapper[2570]: W0706 18:55:20.747334 2570 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 06 18:55:20.831072 ip-10-0-130-59 kubenswrapper[2570]: W0706 18:55:20.747345 2570 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 06 18:55:20.831072 ip-10-0-130-59 kubenswrapper[2570]: W0706 18:55:20.747349 2570 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 06 18:55:20.831072 ip-10-0-130-59 kubenswrapper[2570]: W0706 18:55:20.747355 2570 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 06 18:55:20.831072 ip-10-0-130-59 kubenswrapper[2570]: W0706 18:55:20.747359 2570 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 06 18:55:20.831072 ip-10-0-130-59 kubenswrapper[2570]: W0706 18:55:20.747369 2570 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 06 18:55:20.831072 ip-10-0-130-59 kubenswrapper[2570]: W0706 18:55:20.747373 2570 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 06 18:55:20.831072 ip-10-0-130-59 kubenswrapper[2570]: W0706 18:55:20.747377 2570 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 06 18:55:20.831072 ip-10-0-130-59 kubenswrapper[2570]: W0706 18:55:20.747382 2570 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 06 18:55:20.831072 ip-10-0-130-59 kubenswrapper[2570]: W0706 18:55:20.747386 2570 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 06 18:55:20.831072 ip-10-0-130-59 kubenswrapper[2570]: W0706 18:55:20.747390 2570 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 06 18:55:20.831072 ip-10-0-130-59 kubenswrapper[2570]: W0706 18:55:20.747395 2570 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 06 18:55:20.831072 ip-10-0-130-59 kubenswrapper[2570]: W0706 18:55:20.747399 2570 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 06 18:55:20.831072 ip-10-0-130-59 kubenswrapper[2570]: W0706 18:55:20.747403 2570 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 06 18:55:20.831072 ip-10-0-130-59 kubenswrapper[2570]: W0706 18:55:20.747407 2570 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 06 18:55:20.832012 ip-10-0-130-59 kubenswrapper[2570]: W0706 18:55:20.747412 2570 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 06 18:55:20.832012 ip-10-0-130-59 kubenswrapper[2570]: W0706 18:55:20.747416 2570 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 06 18:55:20.832012 ip-10-0-130-59 kubenswrapper[2570]: W0706 18:55:20.747420 2570 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 06 18:55:20.832012 ip-10-0-130-59 kubenswrapper[2570]: W0706 18:55:20.747429 2570 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 06 18:55:20.832012 ip-10-0-130-59 kubenswrapper[2570]: W0706 18:55:20.747434 2570 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 06 18:55:20.832012 ip-10-0-130-59 kubenswrapper[2570]: W0706 18:55:20.747440 2570 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 06 18:55:20.832012 ip-10-0-130-59 kubenswrapper[2570]: W0706 18:55:20.747446 2570 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 06 18:55:20.832012 ip-10-0-130-59 kubenswrapper[2570]: W0706 18:55:20.747452 2570 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 06 18:55:20.832012 ip-10-0-130-59 kubenswrapper[2570]: W0706 18:55:20.747459 2570 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 06 18:55:20.832012 ip-10-0-130-59 kubenswrapper[2570]: W0706 18:55:20.747464 2570 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 06 18:55:20.832012 ip-10-0-130-59 kubenswrapper[2570]: W0706 18:55:20.747469 2570 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 06 18:55:20.832012 ip-10-0-130-59 kubenswrapper[2570]: W0706 18:55:20.747473 2570 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 06 18:55:20.832012 ip-10-0-130-59 kubenswrapper[2570]: W0706 18:55:20.747477 2570 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 06 18:55:20.832012 ip-10-0-130-59 kubenswrapper[2570]: W0706 18:55:20.747481 2570 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 06 18:55:20.832012 ip-10-0-130-59 kubenswrapper[2570]: W0706 18:55:20.747491 2570 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 06 18:55:20.832012 ip-10-0-130-59 kubenswrapper[2570]: W0706 18:55:20.747495 2570 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 06 18:55:20.832012 ip-10-0-130-59 kubenswrapper[2570]: W0706 18:55:20.747500 2570 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 06 18:55:20.832012 ip-10-0-130-59 kubenswrapper[2570]: W0706 18:55:20.747504 2570 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 06 18:55:20.832012 ip-10-0-130-59 kubenswrapper[2570]: W0706 18:55:20.747511 2570 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 06 18:55:20.929330 ip-10-0-130-59 kubenswrapper[2570]: W0706 18:55:20.747516 2570 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 06 18:55:20.929330 ip-10-0-130-59 kubenswrapper[2570]: W0706 18:55:20.747520 2570 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 06 18:55:20.929330 ip-10-0-130-59 kubenswrapper[2570]: W0706 18:55:20.747525 2570 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 06 18:55:20.929330 ip-10-0-130-59 kubenswrapper[2570]: W0706 18:55:20.747530 2570 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 06 18:55:20.929330 ip-10-0-130-59 kubenswrapper[2570]: W0706 18:55:20.747534 2570 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 06 18:55:20.929330 ip-10-0-130-59 kubenswrapper[2570]: W0706 18:55:20.747538 2570 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 06 18:55:20.929330 ip-10-0-130-59 kubenswrapper[2570]: W0706 18:55:20.747543 2570 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 06 18:55:20.929330 ip-10-0-130-59 kubenswrapper[2570]: W0706 18:55:20.747547 2570 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 06 18:55:20.929330 ip-10-0-130-59 kubenswrapper[2570]: W0706 18:55:20.747556 2570 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 06 18:55:20.929330 ip-10-0-130-59 kubenswrapper[2570]: W0706 18:55:20.747561 2570 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 06 18:55:20.929330 ip-10-0-130-59 kubenswrapper[2570]: W0706 18:55:20.747565 2570 feature_gate.go:328] unrecognized feature gate: Example2 Jul 06 18:55:20.929330 ip-10-0-130-59 kubenswrapper[2570]: W0706 18:55:20.747570 2570 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 06 18:55:20.929330 ip-10-0-130-59 kubenswrapper[2570]: W0706 18:55:20.747575 2570 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 06 18:55:20.929330 ip-10-0-130-59 kubenswrapper[2570]: W0706 18:55:20.747579 2570 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 06 18:55:20.929330 ip-10-0-130-59 kubenswrapper[2570]: W0706 18:55:20.747584 2570 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 06 18:55:20.929330 ip-10-0-130-59 kubenswrapper[2570]: W0706 18:55:20.747588 2570 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 06 18:55:20.929330 ip-10-0-130-59 kubenswrapper[2570]: W0706 18:55:20.747592 2570 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 06 18:55:20.929330 ip-10-0-130-59 kubenswrapper[2570]: W0706 18:55:20.747597 2570 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 06 18:55:20.929330 ip-10-0-130-59 kubenswrapper[2570]: W0706 18:55:20.747601 2570 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 06 18:55:20.929330 ip-10-0-130-59 kubenswrapper[2570]: W0706 18:55:20.747605 2570 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 06 18:55:20.930943 ip-10-0-130-59 kubenswrapper[2570]: W0706 18:55:20.747626 2570 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 06 18:55:20.930943 ip-10-0-130-59 kubenswrapper[2570]: W0706 18:55:20.747636 2570 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 06 18:55:20.930943 ip-10-0-130-59 kubenswrapper[2570]: W0706 18:55:20.747640 2570 feature_gate.go:328] unrecognized feature gate: Example Jul 06 18:55:20.930943 ip-10-0-130-59 kubenswrapper[2570]: W0706 18:55:20.747644 2570 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 06 18:55:20.930943 ip-10-0-130-59 kubenswrapper[2570]: W0706 18:55:20.747648 2570 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 06 18:55:20.930943 ip-10-0-130-59 kubenswrapper[2570]: W0706 18:55:20.747652 2570 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 06 18:55:20.930943 ip-10-0-130-59 kubenswrapper[2570]: W0706 18:55:20.747655 2570 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 06 18:55:20.930943 ip-10-0-130-59 kubenswrapper[2570]: W0706 18:55:20.747660 2570 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 06 18:55:20.930943 ip-10-0-130-59 kubenswrapper[2570]: W0706 18:55:20.747664 2570 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 06 18:55:20.930943 ip-10-0-130-59 kubenswrapper[2570]: W0706 18:55:20.747668 2570 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 06 18:55:20.930943 ip-10-0-130-59 kubenswrapper[2570]: W0706 18:55:20.747672 2570 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 06 18:55:20.930943 ip-10-0-130-59 kubenswrapper[2570]: W0706 18:55:20.747680 2570 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 06 18:55:20.930943 ip-10-0-130-59 kubenswrapper[2570]: W0706 18:55:20.747684 2570 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 06 18:55:20.930943 ip-10-0-130-59 kubenswrapper[2570]: W0706 18:55:20.747693 2570 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 06 18:55:20.930943 ip-10-0-130-59 kubenswrapper[2570]: W0706 18:55:20.747697 2570 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 06 18:55:20.930943 ip-10-0-130-59 kubenswrapper[2570]: W0706 18:55:20.747702 2570 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 06 18:55:20.931624 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:55:20.748510 2570 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 06 18:55:20.931624 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:55:20.755429 2570 server.go:529] "Kubelet version" kubeletVersion="v1.34.8" Jul 06 18:55:20.931624 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:55:20.755443 2570 server.go:531] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK="" Jul 06 18:55:20.931624 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:55:20.755461 2570 watchdog_linux.go:95] "Systemd watchdog is not enabled" Jul 06 18:55:20.931624 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:55:20.755466 2570 watchdog_linux.go:137] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started." Jul 06 18:55:20.931624 ip-10-0-130-59 kubenswrapper[2570]: W0706 18:55:20.755522 2570 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 06 18:55:20.931624 ip-10-0-130-59 kubenswrapper[2570]: W0706 18:55:20.755527 2570 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 06 18:55:20.931624 ip-10-0-130-59 kubenswrapper[2570]: W0706 18:55:20.755530 2570 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 06 18:55:20.931624 ip-10-0-130-59 kubenswrapper[2570]: W0706 18:55:20.755533 2570 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 06 18:55:20.931624 ip-10-0-130-59 kubenswrapper[2570]: W0706 18:55:20.755536 2570 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 06 18:55:20.931624 ip-10-0-130-59 kubenswrapper[2570]: W0706 18:55:20.755539 2570 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 06 18:55:20.931624 ip-10-0-130-59 kubenswrapper[2570]: W0706 18:55:20.755542 2570 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 06 18:55:20.931624 ip-10-0-130-59 kubenswrapper[2570]: W0706 18:55:20.755545 2570 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 06 18:55:20.931624 ip-10-0-130-59 kubenswrapper[2570]: W0706 18:55:20.755548 2570 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 06 18:55:20.932589 ip-10-0-130-59 kubenswrapper[2570]: W0706 18:55:20.755550 2570 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 06 18:55:20.932589 ip-10-0-130-59 kubenswrapper[2570]: W0706 18:55:20.755553 2570 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 06 18:55:20.932589 ip-10-0-130-59 kubenswrapper[2570]: W0706 18:55:20.755557 2570 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 06 18:55:20.932589 ip-10-0-130-59 kubenswrapper[2570]: W0706 18:55:20.755560 2570 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 06 18:55:20.932589 ip-10-0-130-59 kubenswrapper[2570]: W0706 18:55:20.755562 2570 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 06 18:55:20.932589 ip-10-0-130-59 kubenswrapper[2570]: W0706 18:55:20.755565 2570 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 06 18:55:20.932589 ip-10-0-130-59 kubenswrapper[2570]: W0706 18:55:20.755567 2570 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 06 18:55:20.932589 ip-10-0-130-59 kubenswrapper[2570]: W0706 18:55:20.755570 2570 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 06 18:55:20.932589 ip-10-0-130-59 kubenswrapper[2570]: W0706 18:55:20.755573 2570 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 06 18:55:20.932589 ip-10-0-130-59 kubenswrapper[2570]: W0706 18:55:20.755575 2570 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 06 18:55:20.932589 ip-10-0-130-59 kubenswrapper[2570]: W0706 18:55:20.755578 2570 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 06 18:55:20.932589 ip-10-0-130-59 kubenswrapper[2570]: W0706 18:55:20.755581 2570 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 06 18:55:20.932589 ip-10-0-130-59 kubenswrapper[2570]: W0706 18:55:20.755584 2570 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 06 18:55:20.932589 ip-10-0-130-59 kubenswrapper[2570]: W0706 18:55:20.755587 2570 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 06 18:55:20.932589 ip-10-0-130-59 kubenswrapper[2570]: W0706 18:55:20.755592 2570 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 06 18:55:20.932589 ip-10-0-130-59 kubenswrapper[2570]: W0706 18:55:20.755595 2570 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 06 18:55:20.932589 ip-10-0-130-59 kubenswrapper[2570]: W0706 18:55:20.755598 2570 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 06 18:55:20.932589 ip-10-0-130-59 kubenswrapper[2570]: W0706 18:55:20.755600 2570 feature_gate.go:328] unrecognized feature gate: Example2 Jul 06 18:55:20.932589 ip-10-0-130-59 kubenswrapper[2570]: W0706 18:55:20.755603 2570 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 06 18:55:20.932589 ip-10-0-130-59 kubenswrapper[2570]: W0706 18:55:20.755605 2570 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 06 18:55:20.933366 ip-10-0-130-59 kubenswrapper[2570]: W0706 18:55:20.755622 2570 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 06 18:55:20.933366 ip-10-0-130-59 kubenswrapper[2570]: W0706 18:55:20.755626 2570 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 06 18:55:20.933366 ip-10-0-130-59 kubenswrapper[2570]: W0706 18:55:20.755628 2570 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 06 18:55:20.933366 ip-10-0-130-59 kubenswrapper[2570]: W0706 18:55:20.755631 2570 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 06 18:55:20.933366 ip-10-0-130-59 kubenswrapper[2570]: W0706 18:55:20.755634 2570 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 06 18:55:20.933366 ip-10-0-130-59 kubenswrapper[2570]: W0706 18:55:20.755637 2570 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 06 18:55:20.933366 ip-10-0-130-59 kubenswrapper[2570]: W0706 18:55:20.755639 2570 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 06 18:55:20.933366 ip-10-0-130-59 kubenswrapper[2570]: W0706 18:55:20.755642 2570 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 06 18:55:20.933366 ip-10-0-130-59 kubenswrapper[2570]: W0706 18:55:20.755645 2570 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 06 18:55:20.933366 ip-10-0-130-59 kubenswrapper[2570]: W0706 18:55:20.755647 2570 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 06 18:55:20.933366 ip-10-0-130-59 kubenswrapper[2570]: W0706 18:55:20.755650 2570 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 06 18:55:20.933366 ip-10-0-130-59 kubenswrapper[2570]: W0706 18:55:20.755652 2570 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 06 18:55:20.933366 ip-10-0-130-59 kubenswrapper[2570]: W0706 18:55:20.755655 2570 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 06 18:55:20.933366 ip-10-0-130-59 kubenswrapper[2570]: W0706 18:55:20.755657 2570 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 06 18:55:20.933366 ip-10-0-130-59 kubenswrapper[2570]: W0706 18:55:20.755660 2570 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 06 18:55:20.933366 ip-10-0-130-59 kubenswrapper[2570]: W0706 18:55:20.755662 2570 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 06 18:55:20.933366 ip-10-0-130-59 kubenswrapper[2570]: W0706 18:55:20.755665 2570 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 06 18:55:20.933366 ip-10-0-130-59 kubenswrapper[2570]: W0706 18:55:20.755667 2570 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 06 18:55:20.933366 ip-10-0-130-59 kubenswrapper[2570]: W0706 18:55:20.755670 2570 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 06 18:55:20.933366 ip-10-0-130-59 kubenswrapper[2570]: W0706 18:55:20.755673 2570 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 06 18:55:20.934029 ip-10-0-130-59 kubenswrapper[2570]: W0706 18:55:20.755676 2570 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 06 18:55:20.934029 ip-10-0-130-59 kubenswrapper[2570]: W0706 18:55:20.755679 2570 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 06 18:55:20.934029 ip-10-0-130-59 kubenswrapper[2570]: W0706 18:55:20.755683 2570 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 06 18:55:20.934029 ip-10-0-130-59 kubenswrapper[2570]: W0706 18:55:20.755686 2570 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 06 18:55:20.934029 ip-10-0-130-59 kubenswrapper[2570]: W0706 18:55:20.755689 2570 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 06 18:55:20.934029 ip-10-0-130-59 kubenswrapper[2570]: W0706 18:55:20.755692 2570 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 06 18:55:20.934029 ip-10-0-130-59 kubenswrapper[2570]: W0706 18:55:20.755697 2570 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 06 18:55:20.934029 ip-10-0-130-59 kubenswrapper[2570]: W0706 18:55:20.755700 2570 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 06 18:55:20.934029 ip-10-0-130-59 kubenswrapper[2570]: W0706 18:55:20.755703 2570 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 06 18:55:20.934029 ip-10-0-130-59 kubenswrapper[2570]: W0706 18:55:20.755705 2570 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 06 18:55:20.934029 ip-10-0-130-59 kubenswrapper[2570]: W0706 18:55:20.755708 2570 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 06 18:55:20.934029 ip-10-0-130-59 kubenswrapper[2570]: W0706 18:55:20.755711 2570 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 06 18:55:20.934029 ip-10-0-130-59 kubenswrapper[2570]: W0706 18:55:20.755713 2570 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 06 18:55:20.934029 ip-10-0-130-59 kubenswrapper[2570]: W0706 18:55:20.755716 2570 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 06 18:55:20.934029 ip-10-0-130-59 kubenswrapper[2570]: W0706 18:55:20.755718 2570 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 06 18:55:20.934029 ip-10-0-130-59 kubenswrapper[2570]: W0706 18:55:20.755721 2570 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 06 18:55:20.934029 ip-10-0-130-59 kubenswrapper[2570]: W0706 18:55:20.755724 2570 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 06 18:55:20.934029 ip-10-0-130-59 kubenswrapper[2570]: W0706 18:55:20.755726 2570 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 06 18:55:20.934029 ip-10-0-130-59 kubenswrapper[2570]: W0706 18:55:20.755729 2570 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 06 18:55:20.934606 ip-10-0-130-59 kubenswrapper[2570]: W0706 18:55:20.755731 2570 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 06 18:55:20.934606 ip-10-0-130-59 kubenswrapper[2570]: W0706 18:55:20.755734 2570 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 06 18:55:20.934606 ip-10-0-130-59 kubenswrapper[2570]: W0706 18:55:20.755737 2570 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 06 18:55:20.934606 ip-10-0-130-59 kubenswrapper[2570]: W0706 18:55:20.755739 2570 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 06 18:55:20.934606 ip-10-0-130-59 kubenswrapper[2570]: W0706 18:55:20.755742 2570 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 06 18:55:20.934606 ip-10-0-130-59 kubenswrapper[2570]: W0706 18:55:20.755744 2570 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 06 18:55:20.934606 ip-10-0-130-59 kubenswrapper[2570]: W0706 18:55:20.755747 2570 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 06 18:55:20.934606 ip-10-0-130-59 kubenswrapper[2570]: W0706 18:55:20.755750 2570 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 06 18:55:20.934606 ip-10-0-130-59 kubenswrapper[2570]: W0706 18:55:20.755753 2570 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 06 18:55:20.934606 ip-10-0-130-59 kubenswrapper[2570]: W0706 18:55:20.755756 2570 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 06 18:55:20.934606 ip-10-0-130-59 kubenswrapper[2570]: W0706 18:55:20.755758 2570 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 06 18:55:20.934606 ip-10-0-130-59 kubenswrapper[2570]: W0706 18:55:20.755761 2570 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 06 18:55:20.934606 ip-10-0-130-59 kubenswrapper[2570]: W0706 18:55:20.755765 2570 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 06 18:55:20.934606 ip-10-0-130-59 kubenswrapper[2570]: W0706 18:55:20.755768 2570 feature_gate.go:328] unrecognized feature gate: Example Jul 06 18:55:20.934606 ip-10-0-130-59 kubenswrapper[2570]: W0706 18:55:20.755771 2570 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 06 18:55:20.934606 ip-10-0-130-59 kubenswrapper[2570]: W0706 18:55:20.755774 2570 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 06 18:55:20.934606 ip-10-0-130-59 kubenswrapper[2570]: W0706 18:55:20.755777 2570 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 06 18:55:20.934606 ip-10-0-130-59 kubenswrapper[2570]: W0706 18:55:20.755780 2570 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 06 18:55:20.934606 ip-10-0-130-59 kubenswrapper[2570]: W0706 18:55:20.755782 2570 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 06 18:55:20.934606 ip-10-0-130-59 kubenswrapper[2570]: W0706 18:55:20.755791 2570 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 06 18:55:20.935242 ip-10-0-130-59 kubenswrapper[2570]: W0706 18:55:20.755795 2570 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 06 18:55:20.935242 ip-10-0-130-59 kubenswrapper[2570]: W0706 18:55:20.755798 2570 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 06 18:55:20.935242 ip-10-0-130-59 kubenswrapper[2570]: W0706 18:55:20.755801 2570 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 06 18:55:20.935242 ip-10-0-130-59 kubenswrapper[2570]: W0706 18:55:20.755804 2570 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 06 18:55:20.935242 ip-10-0-130-59 kubenswrapper[2570]: W0706 18:55:20.755806 2570 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 06 18:55:20.935242 ip-10-0-130-59 kubenswrapper[2570]: W0706 18:55:20.755809 2570 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 06 18:55:20.935242 ip-10-0-130-59 kubenswrapper[2570]: W0706 18:55:20.755811 2570 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 06 18:55:20.935242 ip-10-0-130-59 kubenswrapper[2570]: W0706 18:55:20.755814 2570 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 06 18:55:20.935242 ip-10-0-130-59 kubenswrapper[2570]: W0706 18:55:20.755817 2570 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 06 18:55:20.935242 ip-10-0-130-59 kubenswrapper[2570]: W0706 18:55:20.755819 2570 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 06 18:55:20.935242 ip-10-0-130-59 kubenswrapper[2570]: W0706 18:55:20.755821 2570 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 06 18:55:20.935242 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:55:20.755826 2570 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 06 18:55:20.935242 ip-10-0-130-59 kubenswrapper[2570]: W0706 18:55:20.755975 2570 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 06 18:55:20.935242 ip-10-0-130-59 kubenswrapper[2570]: W0706 18:55:20.755980 2570 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 06 18:55:20.935673 ip-10-0-130-59 kubenswrapper[2570]: W0706 18:55:20.755983 2570 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 06 18:55:20.935673 ip-10-0-130-59 kubenswrapper[2570]: W0706 18:55:20.755986 2570 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 06 18:55:20.935673 ip-10-0-130-59 kubenswrapper[2570]: W0706 18:55:20.755989 2570 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 06 18:55:20.935673 ip-10-0-130-59 kubenswrapper[2570]: W0706 18:55:20.755992 2570 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 06 18:55:20.935673 ip-10-0-130-59 kubenswrapper[2570]: W0706 18:55:20.755994 2570 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 06 18:55:20.935673 ip-10-0-130-59 kubenswrapper[2570]: W0706 18:55:20.755997 2570 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 06 18:55:20.935673 ip-10-0-130-59 kubenswrapper[2570]: W0706 18:55:20.755999 2570 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 06 18:55:20.935673 ip-10-0-130-59 kubenswrapper[2570]: W0706 18:55:20.756003 2570 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 06 18:55:20.935673 ip-10-0-130-59 kubenswrapper[2570]: W0706 18:55:20.756007 2570 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 06 18:55:20.935673 ip-10-0-130-59 kubenswrapper[2570]: W0706 18:55:20.756010 2570 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 06 18:55:20.935673 ip-10-0-130-59 kubenswrapper[2570]: W0706 18:55:20.756013 2570 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 06 18:55:20.935673 ip-10-0-130-59 kubenswrapper[2570]: W0706 18:55:20.756015 2570 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 06 18:55:20.935673 ip-10-0-130-59 kubenswrapper[2570]: W0706 18:55:20.756018 2570 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 06 18:55:20.935673 ip-10-0-130-59 kubenswrapper[2570]: W0706 18:55:20.756020 2570 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 06 18:55:20.935673 ip-10-0-130-59 kubenswrapper[2570]: W0706 18:55:20.756023 2570 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 06 18:55:20.935673 ip-10-0-130-59 kubenswrapper[2570]: W0706 18:55:20.756026 2570 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 06 18:55:20.935673 ip-10-0-130-59 kubenswrapper[2570]: W0706 18:55:20.756028 2570 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 06 18:55:20.935673 ip-10-0-130-59 kubenswrapper[2570]: W0706 18:55:20.756032 2570 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 06 18:55:20.935673 ip-10-0-130-59 kubenswrapper[2570]: W0706 18:55:20.756036 2570 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 06 18:55:20.936221 ip-10-0-130-59 kubenswrapper[2570]: W0706 18:55:20.756038 2570 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 06 18:55:20.936221 ip-10-0-130-59 kubenswrapper[2570]: W0706 18:55:20.756041 2570 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 06 18:55:20.936221 ip-10-0-130-59 kubenswrapper[2570]: W0706 18:55:20.756044 2570 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 06 18:55:20.936221 ip-10-0-130-59 kubenswrapper[2570]: W0706 18:55:20.756046 2570 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 06 18:55:20.936221 ip-10-0-130-59 kubenswrapper[2570]: W0706 18:55:20.756049 2570 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 06 18:55:20.936221 ip-10-0-130-59 kubenswrapper[2570]: W0706 18:55:20.756051 2570 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 06 18:55:20.936221 ip-10-0-130-59 kubenswrapper[2570]: W0706 18:55:20.756054 2570 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 06 18:55:20.936221 ip-10-0-130-59 kubenswrapper[2570]: W0706 18:55:20.756057 2570 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 06 18:55:20.936221 ip-10-0-130-59 kubenswrapper[2570]: W0706 18:55:20.756059 2570 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 06 18:55:20.936221 ip-10-0-130-59 kubenswrapper[2570]: W0706 18:55:20.756062 2570 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 06 18:55:20.936221 ip-10-0-130-59 kubenswrapper[2570]: W0706 18:55:20.756064 2570 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 06 18:55:20.936221 ip-10-0-130-59 kubenswrapper[2570]: W0706 18:55:20.756067 2570 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 06 18:55:20.936221 ip-10-0-130-59 kubenswrapper[2570]: W0706 18:55:20.756070 2570 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 06 18:55:20.936221 ip-10-0-130-59 kubenswrapper[2570]: W0706 18:55:20.756072 2570 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 06 18:55:20.936221 ip-10-0-130-59 kubenswrapper[2570]: W0706 18:55:20.756075 2570 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 06 18:55:20.936221 ip-10-0-130-59 kubenswrapper[2570]: W0706 18:55:20.756077 2570 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 06 18:55:20.936221 ip-10-0-130-59 kubenswrapper[2570]: W0706 18:55:20.756080 2570 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 06 18:55:20.936221 ip-10-0-130-59 kubenswrapper[2570]: W0706 18:55:20.756082 2570 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 06 18:55:20.936221 ip-10-0-130-59 kubenswrapper[2570]: W0706 18:55:20.756085 2570 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 06 18:55:20.936814 ip-10-0-130-59 kubenswrapper[2570]: W0706 18:55:20.756088 2570 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 06 18:55:20.936814 ip-10-0-130-59 kubenswrapper[2570]: W0706 18:55:20.756090 2570 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 06 18:55:20.936814 ip-10-0-130-59 kubenswrapper[2570]: W0706 18:55:20.756093 2570 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 06 18:55:20.936814 ip-10-0-130-59 kubenswrapper[2570]: W0706 18:55:20.756096 2570 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 06 18:55:20.936814 ip-10-0-130-59 kubenswrapper[2570]: W0706 18:55:20.756098 2570 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 06 18:55:20.936814 ip-10-0-130-59 kubenswrapper[2570]: W0706 18:55:20.756101 2570 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 06 18:55:20.936814 ip-10-0-130-59 kubenswrapper[2570]: W0706 18:55:20.756103 2570 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 06 18:55:20.936814 ip-10-0-130-59 kubenswrapper[2570]: W0706 18:55:20.756105 2570 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 06 18:55:20.936814 ip-10-0-130-59 kubenswrapper[2570]: W0706 18:55:20.756108 2570 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 06 18:55:20.936814 ip-10-0-130-59 kubenswrapper[2570]: W0706 18:55:20.756110 2570 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 06 18:55:20.936814 ip-10-0-130-59 kubenswrapper[2570]: W0706 18:55:20.756113 2570 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 06 18:55:20.936814 ip-10-0-130-59 kubenswrapper[2570]: W0706 18:55:20.756117 2570 feature_gate.go:328] unrecognized feature gate: Example Jul 06 18:55:20.936814 ip-10-0-130-59 kubenswrapper[2570]: W0706 18:55:20.756121 2570 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 06 18:55:20.936814 ip-10-0-130-59 kubenswrapper[2570]: W0706 18:55:20.756124 2570 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 06 18:55:20.936814 ip-10-0-130-59 kubenswrapper[2570]: W0706 18:55:20.756126 2570 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 06 18:55:20.936814 ip-10-0-130-59 kubenswrapper[2570]: W0706 18:55:20.756129 2570 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 06 18:55:20.936814 ip-10-0-130-59 kubenswrapper[2570]: W0706 18:55:20.756132 2570 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 06 18:55:20.936814 ip-10-0-130-59 kubenswrapper[2570]: W0706 18:55:20.756134 2570 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 06 18:55:20.936814 ip-10-0-130-59 kubenswrapper[2570]: W0706 18:55:20.756137 2570 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 06 18:55:20.936814 ip-10-0-130-59 kubenswrapper[2570]: W0706 18:55:20.756139 2570 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 06 18:55:20.937447 ip-10-0-130-59 kubenswrapper[2570]: W0706 18:55:20.756142 2570 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 06 18:55:20.937447 ip-10-0-130-59 kubenswrapper[2570]: W0706 18:55:20.756144 2570 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 06 18:55:20.937447 ip-10-0-130-59 kubenswrapper[2570]: W0706 18:55:20.756147 2570 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 06 18:55:20.937447 ip-10-0-130-59 kubenswrapper[2570]: W0706 18:55:20.756149 2570 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 06 18:55:20.937447 ip-10-0-130-59 kubenswrapper[2570]: W0706 18:55:20.756152 2570 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 06 18:55:20.937447 ip-10-0-130-59 kubenswrapper[2570]: W0706 18:55:20.756155 2570 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 06 18:55:20.937447 ip-10-0-130-59 kubenswrapper[2570]: W0706 18:55:20.756157 2570 feature_gate.go:328] unrecognized feature gate: Example2 Jul 06 18:55:20.937447 ip-10-0-130-59 kubenswrapper[2570]: W0706 18:55:20.756159 2570 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 06 18:55:20.937447 ip-10-0-130-59 kubenswrapper[2570]: W0706 18:55:20.756162 2570 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 06 18:55:20.937447 ip-10-0-130-59 kubenswrapper[2570]: W0706 18:55:20.756165 2570 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 06 18:55:20.937447 ip-10-0-130-59 kubenswrapper[2570]: W0706 18:55:20.756167 2570 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 06 18:55:20.937447 ip-10-0-130-59 kubenswrapper[2570]: W0706 18:55:20.756170 2570 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 06 18:55:20.937447 ip-10-0-130-59 kubenswrapper[2570]: W0706 18:55:20.756172 2570 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 06 18:55:20.937447 ip-10-0-130-59 kubenswrapper[2570]: W0706 18:55:20.756175 2570 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 06 18:55:20.937447 ip-10-0-130-59 kubenswrapper[2570]: W0706 18:55:20.756177 2570 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 06 18:55:20.937447 ip-10-0-130-59 kubenswrapper[2570]: W0706 18:55:20.756180 2570 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 06 18:55:20.937447 ip-10-0-130-59 kubenswrapper[2570]: W0706 18:55:20.756182 2570 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 06 18:55:20.937447 ip-10-0-130-59 kubenswrapper[2570]: W0706 18:55:20.756185 2570 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 06 18:55:20.937447 ip-10-0-130-59 kubenswrapper[2570]: W0706 18:55:20.756189 2570 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 06 18:55:20.937986 ip-10-0-130-59 kubenswrapper[2570]: W0706 18:55:20.756192 2570 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 06 18:55:20.937986 ip-10-0-130-59 kubenswrapper[2570]: W0706 18:55:20.756195 2570 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 06 18:55:20.937986 ip-10-0-130-59 kubenswrapper[2570]: W0706 18:55:20.756198 2570 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 06 18:55:20.937986 ip-10-0-130-59 kubenswrapper[2570]: W0706 18:55:20.756202 2570 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 06 18:55:20.937986 ip-10-0-130-59 kubenswrapper[2570]: W0706 18:55:20.756206 2570 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 06 18:55:20.937986 ip-10-0-130-59 kubenswrapper[2570]: W0706 18:55:20.756210 2570 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 06 18:55:20.937986 ip-10-0-130-59 kubenswrapper[2570]: W0706 18:55:20.756213 2570 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 06 18:55:20.937986 ip-10-0-130-59 kubenswrapper[2570]: W0706 18:55:20.756215 2570 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 06 18:55:20.937986 ip-10-0-130-59 kubenswrapper[2570]: W0706 18:55:20.756218 2570 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 06 18:55:20.937986 ip-10-0-130-59 kubenswrapper[2570]: W0706 18:55:20.756220 2570 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 06 18:55:20.937986 ip-10-0-130-59 kubenswrapper[2570]: W0706 18:55:20.756223 2570 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 06 18:55:20.937986 ip-10-0-130-59 kubenswrapper[2570]: W0706 18:55:20.756226 2570 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 06 18:55:20.937986 ip-10-0-130-59 kubenswrapper[2570]: W0706 18:55:20.756228 2570 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 06 18:55:20.937986 ip-10-0-130-59 kubenswrapper[2570]: W0706 18:55:20.756231 2570 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 06 18:55:20.937986 ip-10-0-130-59 kubenswrapper[2570]: W0706 18:55:20.756234 2570 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 06 18:55:20.937986 ip-10-0-130-59 kubenswrapper[2570]: W0706 18:55:20.756236 2570 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 06 18:55:20.937986 ip-10-0-130-59 kubenswrapper[2570]: W0706 18:55:20.756239 2570 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 06 18:55:20.937986 ip-10-0-130-59 kubenswrapper[2570]: W0706 18:55:20.756241 2570 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 06 18:55:20.937986 ip-10-0-130-59 kubenswrapper[2570]: W0706 18:55:20.756244 2570 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 06 18:55:20.937986 ip-10-0-130-59 kubenswrapper[2570]: W0706 18:55:20.756246 2570 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 06 18:55:20.938500 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:55:20.756251 2570 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 06 18:55:20.938500 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:55:20.756345 2570 server.go:962] "Client rotation is on, will bootstrap in background" Jul 06 18:55:20.938500 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:55:20.758404 2570 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir" Jul 06 18:55:20.938500 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:55:20.759822 2570 server.go:1019] "Starting client certificate rotation" Jul 06 18:55:20.938500 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:55:20.759949 2570 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet" Jul 06 18:55:20.938500 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:55:20.759998 2570 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet" Jul 06 18:55:20.938500 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:55:20.762656 2570 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Jul 06 18:55:20.938500 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:55:20.763955 2570 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Jul 06 18:55:20.938500 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:55:20.779353 2570 log.go:25] "Validated CRI v1 runtime API" Jul 06 18:55:20.938500 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:55:20.781811 2570 log.go:25] "Validated CRI v1 image API" Jul 06 18:55:20.938500 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:55:20.783444 2570 server.go:1429] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd" Jul 06 18:55:20.938500 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:55:20.786005 2570 fs.go:135] Filesystem UUIDs: map[4c9bf886-041a-4267-ba2f-3c9267f75609:/dev/nvme0n1p3 654d39a1-3f8e-4961-b65e-f8a89c6b4ba2:/dev/nvme0n1p4 7B77-95E7:/dev/nvme0n1p2] Jul 06 18:55:20.938889 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:55:20.786022 2570 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}] Jul 06 18:55:20.938889 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:55:20.792953 2570 reflector.go:439] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Jul 06 18:55:20.939122 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:55:20.793150 2570 manager.go:216] Machine: {Timestamp:2026-07-06 18:55:20.791486942 +0000 UTC m=+0.346508129 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3099735 MemoryCapacity:33164488704 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec2be773f8c3362c633d5938152679a9 SystemUUID:ec2be773-f8c3-362c-633d-5938152679a9 BootID:aea452ed-01d2-47db-bc56-63d2d6d0bf0c Filesystems:[{Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6107136 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16582246400 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true} {Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16582242304 Type:vfs Inodes:4048399 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6632898560 Type:vfs Inodes:819200 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:f3:13:f1:37:b1 Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:f3:13:f1:37:b1 Speed:0 Mtu:9001} {Name:ovs-system MacAddress:9e:94:21:fc:79:96 Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:33164488704 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:37486592 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None} Jul 06 18:55:20.939122 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:55:20.793288 2570 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available. Jul 06 18:55:20.939122 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:55:20.793380 2570 manager.go:232] Version: {KernelVersion:5.14.0-570.125.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260630-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:} Jul 06 18:55:20.939122 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:55:20.797019 2570 container_manager_linux.go:273] "Container manager verified user specified cgroup-root exists" cgroupRoot=[] Jul 06 18:55:20.939122 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:55:20.797045 2570 container_manager_linux.go:278] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-130-59.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2} Jul 06 18:55:20.939122 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:55:20.798091 2570 topology_manager.go:138] "Creating topology manager with none policy" Jul 06 18:55:20.939122 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:55:20.798101 2570 container_manager_linux.go:309] "Creating device plugin manager" Jul 06 18:55:20.939122 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:55:20.798108 2570 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock" Jul 06 18:55:20.939122 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:55:20.798880 2570 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock" Jul 06 18:55:20.939122 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:55:20.800133 2570 container_manager_linux.go:318] "Creating Dynamic Resource Allocation (DRA) manager" Jul 06 18:55:20.939122 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:55:20.803882 2570 state_mem.go:36] "Initialized new in-memory state store" Jul 06 18:55:20.939122 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:55:20.803994 2570 server.go:1240] "Using root directory" path="/var/lib/kubelet" Jul 06 18:55:20.939122 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:55:20.804022 2570 kubelet.go:486] "Attempting to sync node with API server" Jul 06 18:55:20.939122 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:55:20.804046 2570 kubelet.go:387] "Adding static pod path" path="/etc/kubernetes/manifests" Jul 06 18:55:20.939122 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:55:20.804065 2570 file.go:69] "Watching path" path="/etc/kubernetes/manifests" Jul 06 18:55:20.939122 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:55:20.804079 2570 kubelet.go:398] "Adding apiserver pod source" Jul 06 18:55:20.939122 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:55:20.804095 2570 apiserver.go:42] "Waiting for node sync before watching apiserver pods" Jul 06 18:55:20.939122 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:55:20.804477 2570 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Jul 06 18:55:20.939707 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:55:20.804496 2570 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Jul 06 18:55:20.939707 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:55:20.806924 2570 kuberuntime_manager.go:301] "Container runtime initialized" containerRuntime="cri-o" version="1.34.8-3.rhaos4.21.git91b4b0b.el9" apiVersion="v1" Jul 06 18:55:20.939707 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:55:20.809460 2570 kubelet.go:960] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled" Jul 06 18:55:20.939707 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:55:20.809496 2570 kubelet.go:984] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled" Jul 06 18:55:20.939707 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:55:20.810789 2570 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume" Jul 06 18:55:20.939707 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:55:20.810806 2570 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir" Jul 06 18:55:20.939707 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:55:20.810815 2570 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo" Jul 06 18:55:20.939707 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:55:20.810823 2570 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path" Jul 06 18:55:20.939707 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:55:20.810835 2570 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs" Jul 06 18:55:20.939707 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:55:20.810843 2570 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret" Jul 06 18:55:20.939707 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:55:20.810853 2570 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi" Jul 06 18:55:20.939707 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:55:20.810861 2570 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api" Jul 06 18:55:20.939707 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:55:20.810870 2570 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc" Jul 06 18:55:20.939707 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:55:20.810878 2570 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap" Jul 06 18:55:20.939707 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:55:20.810899 2570 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected" Jul 06 18:55:20.939707 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:55:20.810908 2570 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume" Jul 06 18:55:20.939707 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:55:20.812527 2570 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi" Jul 06 18:55:20.939707 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:55:20.812537 2570 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image" Jul 06 18:55:20.939707 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:55:20.815998 2570 server.go:1268] "Started kubelet" Jul 06 18:55:20.939707 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:55:20.816402 2570 server.go:180] "Starting to listen" address="0.0.0.0" port=10250 Jul 06 18:55:20.939707 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:55:20.819159 2570 server.go:310] "Adding debug handlers to kubelet server" Jul 06 18:55:20.939707 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:55:20.820331 2570 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10 Jul 06 18:55:20.939707 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:55:20.820376 2570 server_v1.go:49] "podresources" method="list" useActivePods=true Jul 06 18:55:20.939707 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:55:20.822715 2570 server.go:249] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock" Jul 06 18:55:20.939707 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:55:20.823037 2570 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-wbw9p" Jul 06 18:55:20.939707 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:55:20.827848 2570 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-wbw9p" Jul 06 18:55:20.939707 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:55:20.828131 2570 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer" Jul 06 18:55:20.939707 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:55:20.828606 2570 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving" Jul 06 18:55:20.939707 ip-10-0-130-59 kubenswrapper[2570]: E0706 18:55:20.828837 2570 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-130-59.ec2.internal\" not found" Jul 06 18:55:20.939707 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:55:20.829539 2570 volume_manager.go:311] "The desired_state_of_world populator starts" Jul 06 18:55:20.939707 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:55:20.829558 2570 volume_manager.go:313] "Starting Kubelet Volume Manager" Jul 06 18:55:20.939707 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:55:20.829714 2570 desired_state_of_world_populator.go:146] "Desired state populator starts to run" Jul 06 18:55:20.939707 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:55:20.829819 2570 reconstruct.go:97] "Volume reconstruction finished" Jul 06 18:55:20.939707 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:55:20.829826 2570 reconciler.go:29] "Reconciler: start to sync state" Jul 06 18:55:20.939707 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:55:20.830386 2570 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-130-59.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope Jul 06 18:55:20.939707 ip-10-0-130-59 kubenswrapper[2570]: E0706 18:55:20.830396 2570 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service" Jul 06 18:55:20.939707 ip-10-0-130-59 kubenswrapper[2570]: E0706 18:55:20.830575 2570 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-130-59.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node" Jul 06 18:55:20.939707 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:55:20.830662 2570 factory.go:55] Registering systemd factory Jul 06 18:55:20.939707 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:55:20.830676 2570 factory.go:223] Registration of the systemd container factory successfully Jul 06 18:55:20.939707 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:55:20.831407 2570 factory.go:153] Registering CRI-O factory Jul 06 18:55:20.939707 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:55:20.831435 2570 factory.go:223] Registration of the crio container factory successfully Jul 06 18:55:20.939707 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:55:20.831492 2570 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory Jul 06 18:55:20.939707 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:55:20.831521 2570 factory.go:103] Registering Raw factory Jul 06 18:55:20.939707 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:55:20.831536 2570 manager.go:1196] Started watching for new ooms in manager Jul 06 18:55:20.939707 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:55:20.831986 2570 manager.go:349] Starting recovery of all containers Jul 06 18:55:20.939707 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:55:20.841576 2570 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160" Jul 06 18:55:20.939707 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:55:20.844839 2570 manager.go:354] Recovery completed Jul 06 18:55:20.939707 ip-10-0-130-59 kubenswrapper[2570]: E0706 18:55:20.848929 2570 nodelease.go:49] "Failed to get node when trying to set owner ref to the node lease" err="nodes \"ip-10-0-130-59.ec2.internal\" not found" node="ip-10-0-130-59.ec2.internal" Jul 06 18:55:20.940957 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:55:20.849556 2570 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 06 18:55:20.940957 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:55:20.851374 2570 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-59.ec2.internal" event="NodeHasSufficientMemory" Jul 06 18:55:20.940957 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:55:20.851400 2570 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-59.ec2.internal" event="NodeHasNoDiskPressure" Jul 06 18:55:20.940957 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:55:20.851410 2570 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-59.ec2.internal" event="NodeHasSufficientPID" Jul 06 18:55:20.940957 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:55:20.854392 2570 cpu_manager.go:222] "Starting CPU manager" policy="none" Jul 06 18:55:20.940957 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:55:20.854404 2570 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s" Jul 06 18:55:20.940957 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:55:20.854426 2570 state_mem.go:36] "Initialized new in-memory state store" Jul 06 18:55:20.940957 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:55:20.854429 2570 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 06 18:55:20.940957 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:55:20.855231 2570 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-59.ec2.internal" event="NodeHasSufficientMemory" Jul 06 18:55:20.940957 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:55:20.855258 2570 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-59.ec2.internal" event="NodeHasNoDiskPressure" Jul 06 18:55:20.940957 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:55:20.855273 2570 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-59.ec2.internal" event="NodeHasSufficientPID" Jul 06 18:55:20.940957 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:55:20.856184 2570 policy_none.go:49] "None policy: Start" Jul 06 18:55:20.940957 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:55:20.856198 2570 memory_manager.go:187] "Starting memorymanager" policy="None" Jul 06 18:55:20.940957 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:55:20.856208 2570 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint" Jul 06 18:55:20.940957 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:55:20.857326 2570 policy_none.go:47] "Start" Jul 06 18:55:20.940957 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:55:20.891276 2570 manager.go:337] "Starting Device Plugin manager" Jul 06 18:55:20.940957 ip-10-0-130-59 kubenswrapper[2570]: E0706 18:55:20.891319 2570 manager.go:513] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint" Jul 06 18:55:20.940957 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:55:20.891334 2570 server.go:87] "Starting device plugin registration server" Jul 06 18:55:20.940957 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:55:20.891597 2570 eviction_manager.go:189] "Eviction manager: starting control loop" Jul 06 18:55:20.940957 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:55:20.891648 2570 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s" Jul 06 18:55:20.940957 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:55:20.891788 2570 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry" Jul 06 18:55:20.940957 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:55:20.891977 2570 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts" Jul 06 18:55:20.940957 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:55:20.891988 2570 plugin_manager.go:118] "Starting Kubelet Plugin Manager" Jul 06 18:55:20.940957 ip-10-0-130-59 kubenswrapper[2570]: E0706 18:55:20.892367 2570 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\"" Jul 06 18:55:20.940957 ip-10-0-130-59 kubenswrapper[2570]: E0706 18:55:20.892430 2570 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-130-59.ec2.internal\" not found" Jul 06 18:55:21.074742 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:55:20.991896 2570 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 06 18:55:21.074742 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:55:20.992687 2570 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-59.ec2.internal" event="NodeHasSufficientMemory" Jul 06 18:55:21.074742 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:55:20.992716 2570 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-59.ec2.internal" event="NodeHasNoDiskPressure" Jul 06 18:55:21.074742 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:55:20.992729 2570 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-59.ec2.internal" event="NodeHasSufficientPID" Jul 06 18:55:21.074742 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:55:20.992758 2570 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-130-59.ec2.internal" Jul 06 18:55:21.074742 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:55:21.003271 2570 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-130-59.ec2.internal" Jul 06 18:55:21.074742 ip-10-0-130-59 kubenswrapper[2570]: E0706 18:55:21.003288 2570 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-130-59.ec2.internal\": node \"ip-10-0-130-59.ec2.internal\" not found" Jul 06 18:55:21.074742 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:55:21.020518 2570 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4" Jul 06 18:55:21.074742 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:55:21.021583 2570 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6" Jul 06 18:55:21.074742 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:55:21.021638 2570 status_manager.go:244] "Starting to sync pod status with apiserver" Jul 06 18:55:21.074742 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:55:21.021659 2570 kubelet.go:2448] "Starting kubelet main sync loop" Jul 06 18:55:21.074742 ip-10-0-130-59 kubenswrapper[2570]: E0706 18:55:21.021688 2570 kubelet.go:2472] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful" Jul 06 18:55:21.074742 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:55:21.024374 2570 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Jul 06 18:55:21.074742 ip-10-0-130-59 kubenswrapper[2570]: E0706 18:55:21.039853 2570 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-130-59.ec2.internal\" not found" Jul 06 18:55:21.221481 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:55:21.121939 2570 kubelet.go:2535] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-59.ec2.internal","kube-system/kube-apiserver-proxy-ip-10-0-130-59.ec2.internal"] Jul 06 18:55:21.221481 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:55:21.122144 2570 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 06 18:55:21.221481 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:55:21.123729 2570 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-59.ec2.internal" event="NodeHasSufficientMemory" Jul 06 18:55:21.221481 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:55:21.123766 2570 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-59.ec2.internal" event="NodeHasNoDiskPressure" Jul 06 18:55:21.221481 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:55:21.123786 2570 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-59.ec2.internal" event="NodeHasSufficientPID" Jul 06 18:55:21.221481 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:55:21.125343 2570 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 06 18:55:21.221481 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:55:21.125451 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-59.ec2.internal" Jul 06 18:55:21.221481 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:55:21.125486 2570 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 06 18:55:21.221481 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:55:21.126091 2570 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-59.ec2.internal" event="NodeHasSufficientMemory" Jul 06 18:55:21.221481 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:55:21.126123 2570 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-59.ec2.internal" event="NodeHasNoDiskPressure" Jul 06 18:55:21.221481 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:55:21.126132 2570 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-59.ec2.internal" event="NodeHasSufficientPID" Jul 06 18:55:21.221481 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:55:21.126096 2570 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-59.ec2.internal" event="NodeHasSufficientMemory" Jul 06 18:55:21.221481 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:55:21.126187 2570 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-59.ec2.internal" event="NodeHasNoDiskPressure" Jul 06 18:55:21.221481 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:55:21.126201 2570 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-59.ec2.internal" event="NodeHasSufficientPID" Jul 06 18:55:21.221481 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:55:21.127449 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-130-59.ec2.internal" Jul 06 18:55:21.221481 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:55:21.127472 2570 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 06 18:55:21.221481 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:55:21.128087 2570 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-59.ec2.internal" event="NodeHasSufficientMemory" Jul 06 18:55:21.221481 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:55:21.128110 2570 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-59.ec2.internal" event="NodeHasNoDiskPressure" Jul 06 18:55:21.221481 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:55:21.128121 2570 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-59.ec2.internal" event="NodeHasSufficientPID" Jul 06 18:55:21.221481 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:55:21.134733 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/f20193a4bb56b9075be1ec3e656dd631-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-130-59.ec2.internal\" (UID: \"f20193a4bb56b9075be1ec3e656dd631\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-59.ec2.internal" Jul 06 18:55:21.221481 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:55:21.134756 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/f20193a4bb56b9075be1ec3e656dd631-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-130-59.ec2.internal\" (UID: \"f20193a4bb56b9075be1ec3e656dd631\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-59.ec2.internal" Jul 06 18:55:21.221481 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:55:21.134798 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/d57d2f4948f6aacb3452bada80a2e242-config\") pod \"kube-apiserver-proxy-ip-10-0-130-59.ec2.internal\" (UID: \"d57d2f4948f6aacb3452bada80a2e242\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-130-59.ec2.internal" Jul 06 18:55:21.221481 ip-10-0-130-59 kubenswrapper[2570]: E0706 18:55:21.140700 2570 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-130-59.ec2.internal\" not found" Jul 06 18:55:21.221481 ip-10-0-130-59 kubenswrapper[2570]: E0706 18:55:21.164102 2570 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-130-59.ec2.internal\" not found" node="ip-10-0-130-59.ec2.internal" Jul 06 18:55:21.221481 ip-10-0-130-59 kubenswrapper[2570]: E0706 18:55:21.168535 2570 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-130-59.ec2.internal\" not found" node="ip-10-0-130-59.ec2.internal" Jul 06 18:55:21.387142 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:55:21.237395 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/f20193a4bb56b9075be1ec3e656dd631-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-130-59.ec2.internal\" (UID: \"f20193a4bb56b9075be1ec3e656dd631\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-59.ec2.internal" Jul 06 18:55:21.387142 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:55:21.237423 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/f20193a4bb56b9075be1ec3e656dd631-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-130-59.ec2.internal\" (UID: \"f20193a4bb56b9075be1ec3e656dd631\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-59.ec2.internal" Jul 06 18:55:21.387142 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:55:21.237440 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/d57d2f4948f6aacb3452bada80a2e242-config\") pod \"kube-apiserver-proxy-ip-10-0-130-59.ec2.internal\" (UID: \"d57d2f4948f6aacb3452bada80a2e242\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-130-59.ec2.internal" Jul 06 18:55:21.387142 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:55:21.237472 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/d57d2f4948f6aacb3452bada80a2e242-config\") pod \"kube-apiserver-proxy-ip-10-0-130-59.ec2.internal\" (UID: \"d57d2f4948f6aacb3452bada80a2e242\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-130-59.ec2.internal" Jul 06 18:55:21.387142 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:55:21.237495 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/f20193a4bb56b9075be1ec3e656dd631-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-130-59.ec2.internal\" (UID: \"f20193a4bb56b9075be1ec3e656dd631\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-59.ec2.internal" Jul 06 18:55:21.387142 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:55:21.237492 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/f20193a4bb56b9075be1ec3e656dd631-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-130-59.ec2.internal\" (UID: \"f20193a4bb56b9075be1ec3e656dd631\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-59.ec2.internal" Jul 06 18:55:21.387142 ip-10-0-130-59 kubenswrapper[2570]: E0706 18:55:21.241495 2570 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-130-59.ec2.internal\" not found" Jul 06 18:55:21.387142 ip-10-0-130-59 kubenswrapper[2570]: E0706 18:55:21.341738 2570 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-130-59.ec2.internal\" not found" Jul 06 18:55:21.495153 ip-10-0-130-59 kubenswrapper[2570]: E0706 18:55:21.442171 2570 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-130-59.ec2.internal\" not found" Jul 06 18:55:21.495153 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:55:21.467324 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-59.ec2.internal" Jul 06 18:55:21.495153 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:55:21.472303 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-130-59.ec2.internal" Jul 06 18:55:21.605258 ip-10-0-130-59 kubenswrapper[2570]: E0706 18:55:21.542418 2570 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-130-59.ec2.internal\" not found" Jul 06 18:55:21.708430 ip-10-0-130-59 kubenswrapper[2570]: E0706 18:55:21.642922 2570 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-130-59.ec2.internal\" not found" Jul 06 18:55:21.797220 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:55:21.740698 2570 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160" Jul 06 18:55:21.797220 ip-10-0-130-59 kubenswrapper[2570]: E0706 18:55:21.743334 2570 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-130-59.ec2.internal\" not found" Jul 06 18:55:21.797220 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:55:21.759471 2570 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials" Jul 06 18:55:21.797220 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:55:21.759567 2570 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received" Jul 06 18:55:21.797220 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:55:21.759601 2570 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received" Jul 06 18:55:21.797220 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:55:21.759631 2570 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received" Jul 06 18:55:21.830237 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:55:21.830206 2570 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-07-05 18:50:20 +0000 UTC" deadline="2028-02-15 08:20:18.81912333 +0000 UTC" Jul 06 18:55:21.830237 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:55:21.830235 2570 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="14125h24m56.988891995s" Jul 06 18:55:21.844291 ip-10-0-130-59 kubenswrapper[2570]: E0706 18:55:21.844275 2570 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-130-59.ec2.internal\" not found" Jul 06 18:55:21.944682 ip-10-0-130-59 kubenswrapper[2570]: E0706 18:55:21.944653 2570 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-130-59.ec2.internal\" not found" Jul 06 18:55:21.986908 ip-10-0-130-59 kubenswrapper[2570]: W0706 18:55:21.986867 2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd57d2f4948f6aacb3452bada80a2e242.slice/crio-7b61c6dd18915105a51d43cd0ebe0322b5dc7130b46c95a1e973e72ac234bb13 WatchSource:0}: Error finding container 7b61c6dd18915105a51d43cd0ebe0322b5dc7130b46c95a1e973e72ac234bb13: Status 404 returned error can't find the container with id 7b61c6dd18915105a51d43cd0ebe0322b5dc7130b46c95a1e973e72ac234bb13 Jul 06 18:55:21.987205 ip-10-0-130-59 kubenswrapper[2570]: W0706 18:55:21.987187 2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf20193a4bb56b9075be1ec3e656dd631.slice/crio-52cf21fc7289bfe5a3547223225e4acaba6ba7350b069d8c6466186d3ec3a398 WatchSource:0}: Error finding container 52cf21fc7289bfe5a3547223225e4acaba6ba7350b069d8c6466186d3ec3a398: Status 404 returned error can't find the container with id 52cf21fc7289bfe5a3547223225e4acaba6ba7350b069d8c6466186d3ec3a398 Jul 06 18:55:21.992505 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:55:21.992468 2570 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 06 18:55:22.024163 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:55:22.024108 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-130-59.ec2.internal" event={"ID":"d57d2f4948f6aacb3452bada80a2e242","Type":"ContainerStarted","Data":"7b61c6dd18915105a51d43cd0ebe0322b5dc7130b46c95a1e973e72ac234bb13"} Jul 06 18:55:22.025075 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:55:22.025051 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-59.ec2.internal" event={"ID":"f20193a4bb56b9075be1ec3e656dd631","Type":"ContainerStarted","Data":"52cf21fc7289bfe5a3547223225e4acaba6ba7350b069d8c6466186d3ec3a398"} Jul 06 18:55:22.045481 ip-10-0-130-59 kubenswrapper[2570]: E0706 18:55:22.045464 2570 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-130-59.ec2.internal\" not found" Jul 06 18:55:22.146021 ip-10-0-130-59 kubenswrapper[2570]: E0706 18:55:22.145999 2570 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-130-59.ec2.internal\" not found" Jul 06 18:55:22.246516 ip-10-0-130-59 kubenswrapper[2570]: E0706 18:55:22.246457 2570 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-130-59.ec2.internal\" not found" Jul 06 18:55:22.347303 ip-10-0-130-59 kubenswrapper[2570]: E0706 18:55:22.347273 2570 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-130-59.ec2.internal\" not found" Jul 06 18:55:22.356942 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:55:22.356919 2570 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160" Jul 06 18:55:22.430098 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:55:22.430072 2570 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-59.ec2.internal" Jul 06 18:55:22.441002 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:55:22.440981 2570 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Jul 06 18:55:22.442222 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:55:22.442205 2570 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-130-59.ec2.internal" Jul 06 18:55:22.448811 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:55:22.448788 2570 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Jul 06 18:55:22.691822 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:55:22.691791 2570 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Jul 06 18:55:22.805344 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:55:22.805314 2570 apiserver.go:52] "Watching apiserver" Jul 06 18:55:22.813784 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:55:22.813759 2570 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66" Jul 06 18:55:22.815893 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:55:22.815861 2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kube-system/global-pull-secret-syncer-ln98s","kube-system/konnectivity-agent-k8fbq","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-k5knh","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-59.ec2.internal","openshift-multus/multus-additional-cni-plugins-wmxvz","openshift-multus/network-metrics-daemon-wh2jx","openshift-network-diagnostics/network-check-target-296mj","kube-system/kube-apiserver-proxy-ip-10-0-130-59.ec2.internal","openshift-cluster-node-tuning-operator/tuned-fvvmf","openshift-dns/node-resolver-4qk4s","openshift-image-registry/node-ca-sxf4k","openshift-multus/multus-b66m9","openshift-network-operator/iptables-alerter-qnhx2","openshift-ovn-kubernetes/ovnkube-node-rr76d"] Jul 06 18:55:22.817294 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:55:22.817270 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-4qk4s" Jul 06 18:55:22.818540 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:55:22.818521 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-qnhx2" Jul 06 18:55:22.820251 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:55:22.819804 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-k5knh" Jul 06 18:55:22.820251 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:55:22.819830 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-jgkxq\"" Jul 06 18:55:22.820251 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:55:22.819895 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\"" Jul 06 18:55:22.820251 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:55:22.820050 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\"" Jul 06 18:55:22.821337 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:55:22.821213 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\"" Jul 06 18:55:22.821602 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:55:22.821583 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-sdvsd\"" Jul 06 18:55:22.821742 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:55:22.821631 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\"" Jul 06 18:55:22.822325 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:55:22.822310 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\"" Jul 06 18:55:22.822415 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:55:22.822398 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\"" Jul 06 18:55:22.822596 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:55:22.822574 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\"" Jul 06 18:55:22.822898 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:55:22.822882 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\"" Jul 06 18:55:22.822999 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:55:22.822986 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-zdq77\"" Jul 06 18:55:22.824918 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:55:22.824898 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-rr76d" Jul 06 18:55:22.827272 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:55:22.826974 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-wmxvz" Jul 06 18:55:22.827272 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:55:22.827240 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\"" Jul 06 18:55:22.827425 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:55:22.827394 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\"" Jul 06 18:55:22.827577 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:55:22.827542 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\"" Jul 06 18:55:22.827693 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:55:22.827640 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-rbhcb\"" Jul 06 18:55:22.828490 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:55:22.828358 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\"" Jul 06 18:55:22.828490 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:55:22.828399 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\"" Jul 06 18:55:22.828950 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:55:22.828799 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\"" Jul 06 18:55:22.828950 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:55:22.828906 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-wh2jx" Jul 06 18:55:22.829076 ip-10-0-130-59 kubenswrapper[2570]: E0706 18:55:22.828989 2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-wh2jx" podUID="9262d667-2ed4-45aa-98f5-2d8158a66ca1" Jul 06 18:55:22.829305 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:55:22.829284 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\"" Jul 06 18:55:22.829305 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:55:22.829300 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\"" Jul 06 18:55:22.829436 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:55:22.829307 2570 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving" Jul 06 18:55:22.829436 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:55:22.829290 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\"" Jul 06 18:55:22.829436 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:55:22.829431 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\"" Jul 06 18:55:22.829725 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:55:22.829708 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-nx285\"" Jul 06 18:55:22.829725 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:55:22.829720 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\"" Jul 06 18:55:22.830366 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:55:22.830349 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-fvvmf" Jul 06 18:55:22.832011 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:55:22.831996 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-296mj" Jul 06 18:55:22.832155 ip-10-0-130-59 kubenswrapper[2570]: E0706 18:55:22.832135 2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-296mj" podUID="f7bc558b-d33b-448d-929c-3652d096c3d5" Jul 06 18:55:22.832467 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:55:22.832423 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\"" Jul 06 18:55:22.832467 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:55:22.832453 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\"" Jul 06 18:55:22.832625 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:55:22.832473 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-flw4g\"" Jul 06 18:55:22.833771 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:55:22.833752 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-k8fbq" Jul 06 18:55:22.835006 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:55:22.834992 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-ln98s" Jul 06 18:55:22.835083 ip-10-0-130-59 kubenswrapper[2570]: E0706 18:55:22.835049 2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-ln98s" podUID="4beea8e8-7750-4a7a-bc49-249b3d7e3ab3" Jul 06 18:55:22.835985 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:55:22.835967 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\"" Jul 06 18:55:22.836075 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:55:22.835997 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-2bzrn\"" Jul 06 18:55:22.836075 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:55:22.836022 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\"" Jul 06 18:55:22.836559 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:55:22.836533 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-sxf4k" Jul 06 18:55:22.838139 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:55:22.838103 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-b66m9" Jul 06 18:55:22.838593 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:55:22.838578 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\"" Jul 06 18:55:22.838828 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:55:22.838812 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-75r2r\"" Jul 06 18:55:22.839170 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:55:22.839152 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\"" Jul 06 18:55:22.840002 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:55:22.839967 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\"" Jul 06 18:55:22.840763 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:55:22.840745 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\"" Jul 06 18:55:22.840899 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:55:22.840869 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-4lq47\"" Jul 06 18:55:22.843947 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:55:22.843929 2570 reflector.go:439] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Jul 06 18:55:22.846015 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:55:22.845996 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-d96dd\" (UniqueName: \"kubernetes.io/projected/ac420ddc-b839-4613-943d-50d18a911d56-kube-api-access-d96dd\") pod \"ovnkube-node-rr76d\" (UID: \"ac420ddc-b839-4613-943d-50d18a911d56\") " pod="openshift-ovn-kubernetes/ovnkube-node-rr76d" Jul 06 18:55:22.846094 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:55:22.846025 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/4f1bcb6b-7c73-4bff-a246-78ddc5ca22ea-system-cni-dir\") pod \"multus-additional-cni-plugins-wmxvz\" (UID: \"4f1bcb6b-7c73-4bff-a246-78ddc5ca22ea\") " pod="openshift-multus/multus-additional-cni-plugins-wmxvz" Jul 06 18:55:22.846094 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:55:22.846053 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a88da320-d446-42e3-b155-418a20f35893-tmp-dir\") pod \"node-resolver-4qk4s\" (UID: \"a88da320-d446-42e3-b155-418a20f35893\") " pod="openshift-dns/node-resolver-4qk4s" Jul 06 18:55:22.846094 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:55:22.846070 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/17c6ff8a-2c68-400e-b46d-7f73f3673088-host-slash\") pod \"iptables-alerter-qnhx2\" (UID: \"17c6ff8a-2c68-400e-b46d-7f73f3673088\") " pod="openshift-network-operator/iptables-alerter-qnhx2" Jul 06 18:55:22.846251 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:55:22.846107 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/92e4be75-799a-44f0-aaca-22ef4bca49fc-etc-modprobe-d\") pod \"tuned-fvvmf\" (UID: \"92e4be75-799a-44f0-aaca-22ef4bca49fc\") " pod="openshift-cluster-node-tuning-operator/tuned-fvvmf" Jul 06 18:55:22.846251 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:55:22.846136 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/92e4be75-799a-44f0-aaca-22ef4bca49fc-etc-systemd\") pod \"tuned-fvvmf\" (UID: \"92e4be75-799a-44f0-aaca-22ef4bca49fc\") " pod="openshift-cluster-node-tuning-operator/tuned-fvvmf" Jul 06 18:55:22.846251 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:55:22.846175 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/92e4be75-799a-44f0-aaca-22ef4bca49fc-lib-modules\") pod \"tuned-fvvmf\" (UID: \"92e4be75-799a-44f0-aaca-22ef4bca49fc\") " pod="openshift-cluster-node-tuning-operator/tuned-fvvmf" Jul 06 18:55:22.846251 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:55:22.846222 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/b9b9a09a-8299-41aa-9b59-51ed106f6802-sys-fs\") pod \"aws-ebs-csi-driver-node-k5knh\" (UID: \"b9b9a09a-8299-41aa-9b59-51ed106f6802\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-k5knh" Jul 06 18:55:22.846526 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:55:22.846252 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fzqfj\" (UniqueName: \"kubernetes.io/projected/a88da320-d446-42e3-b155-418a20f35893-kube-api-access-fzqfj\") pod \"node-resolver-4qk4s\" (UID: \"a88da320-d446-42e3-b155-418a20f35893\") " pod="openshift-dns/node-resolver-4qk4s" Jul 06 18:55:22.846526 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:55:22.846283 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/92e4be75-799a-44f0-aaca-22ef4bca49fc-sys\") pod \"tuned-fvvmf\" (UID: \"92e4be75-799a-44f0-aaca-22ef4bca49fc\") " pod="openshift-cluster-node-tuning-operator/tuned-fvvmf" Jul 06 18:55:22.846526 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:55:22.846305 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/ac420ddc-b839-4613-943d-50d18a911d56-host-cni-netd\") pod \"ovnkube-node-rr76d\" (UID: \"ac420ddc-b839-4613-943d-50d18a911d56\") " pod="openshift-ovn-kubernetes/ovnkube-node-rr76d" Jul 06 18:55:22.846526 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:55:22.846328 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/ac420ddc-b839-4613-943d-50d18a911d56-ovn-node-metrics-cert\") pod \"ovnkube-node-rr76d\" (UID: \"ac420ddc-b839-4613-943d-50d18a911d56\") " pod="openshift-ovn-kubernetes/ovnkube-node-rr76d" Jul 06 18:55:22.846526 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:55:22.846368 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/92e4be75-799a-44f0-aaca-22ef4bca49fc-etc-tuned\") pod \"tuned-fvvmf\" (UID: \"92e4be75-799a-44f0-aaca-22ef4bca49fc\") " pod="openshift-cluster-node-tuning-operator/tuned-fvvmf" Jul 06 18:55:22.846526 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:55:22.846397 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/ac420ddc-b839-4613-943d-50d18a911d56-run-ovn\") pod \"ovnkube-node-rr76d\" (UID: \"ac420ddc-b839-4613-943d-50d18a911d56\") " pod="openshift-ovn-kubernetes/ovnkube-node-rr76d" Jul 06 18:55:22.846526 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:55:22.846417 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/ac420ddc-b839-4613-943d-50d18a911d56-node-log\") pod \"ovnkube-node-rr76d\" (UID: \"ac420ddc-b839-4613-943d-50d18a911d56\") " pod="openshift-ovn-kubernetes/ovnkube-node-rr76d" Jul 06 18:55:22.846526 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:55:22.846432 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/4f1bcb6b-7c73-4bff-a246-78ddc5ca22ea-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-wmxvz\" (UID: \"4f1bcb6b-7c73-4bff-a246-78ddc5ca22ea\") " pod="openshift-multus/multus-additional-cni-plugins-wmxvz" Jul 06 18:55:22.846526 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:55:22.846472 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-z65sk\" (UniqueName: \"kubernetes.io/projected/17c6ff8a-2c68-400e-b46d-7f73f3673088-kube-api-access-z65sk\") pod \"iptables-alerter-qnhx2\" (UID: \"17c6ff8a-2c68-400e-b46d-7f73f3673088\") " pod="openshift-network-operator/iptables-alerter-qnhx2" Jul 06 18:55:22.846526 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:55:22.846501 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/9262d667-2ed4-45aa-98f5-2d8158a66ca1-metrics-certs\") pod \"network-metrics-daemon-wh2jx\" (UID: \"9262d667-2ed4-45aa-98f5-2d8158a66ca1\") " pod="openshift-multus/network-metrics-daemon-wh2jx" Jul 06 18:55:22.846900 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:55:22.846523 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/92e4be75-799a-44f0-aaca-22ef4bca49fc-etc-kubernetes\") pod \"tuned-fvvmf\" (UID: \"92e4be75-799a-44f0-aaca-22ef4bca49fc\") " pod="openshift-cluster-node-tuning-operator/tuned-fvvmf" Jul 06 18:55:22.846900 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:55:22.846554 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/ac420ddc-b839-4613-943d-50d18a911d56-log-socket\") pod \"ovnkube-node-rr76d\" (UID: \"ac420ddc-b839-4613-943d-50d18a911d56\") " pod="openshift-ovn-kubernetes/ovnkube-node-rr76d" Jul 06 18:55:22.846900 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:55:22.846588 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/ac420ddc-b839-4613-943d-50d18a911d56-host-run-ovn-kubernetes\") pod \"ovnkube-node-rr76d\" (UID: \"ac420ddc-b839-4613-943d-50d18a911d56\") " pod="openshift-ovn-kubernetes/ovnkube-node-rr76d" Jul 06 18:55:22.846900 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:55:22.846626 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/ac420ddc-b839-4613-943d-50d18a911d56-ovnkube-config\") pod \"ovnkube-node-rr76d\" (UID: \"ac420ddc-b839-4613-943d-50d18a911d56\") " pod="openshift-ovn-kubernetes/ovnkube-node-rr76d" Jul 06 18:55:22.846900 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:55:22.846663 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/17c6ff8a-2c68-400e-b46d-7f73f3673088-iptables-alerter-script\") pod \"iptables-alerter-qnhx2\" (UID: \"17c6ff8a-2c68-400e-b46d-7f73f3673088\") " pod="openshift-network-operator/iptables-alerter-qnhx2" Jul 06 18:55:22.846900 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:55:22.846700 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/92e4be75-799a-44f0-aaca-22ef4bca49fc-etc-sysconfig\") pod \"tuned-fvvmf\" (UID: \"92e4be75-799a-44f0-aaca-22ef4bca49fc\") " pod="openshift-cluster-node-tuning-operator/tuned-fvvmf" Jul 06 18:55:22.846900 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:55:22.846744 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/ac420ddc-b839-4613-943d-50d18a911d56-host-cni-bin\") pod \"ovnkube-node-rr76d\" (UID: \"ac420ddc-b839-4613-943d-50d18a911d56\") " pod="openshift-ovn-kubernetes/ovnkube-node-rr76d" Jul 06 18:55:22.846900 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:55:22.846770 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vzd9s\" (UniqueName: \"kubernetes.io/projected/4f1bcb6b-7c73-4bff-a246-78ddc5ca22ea-kube-api-access-vzd9s\") pod \"multus-additional-cni-plugins-wmxvz\" (UID: \"4f1bcb6b-7c73-4bff-a246-78ddc5ca22ea\") " pod="openshift-multus/multus-additional-cni-plugins-wmxvz" Jul 06 18:55:22.846900 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:55:22.846793 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/ac420ddc-b839-4613-943d-50d18a911d56-host-slash\") pod \"ovnkube-node-rr76d\" (UID: \"ac420ddc-b839-4613-943d-50d18a911d56\") " pod="openshift-ovn-kubernetes/ovnkube-node-rr76d" Jul 06 18:55:22.846900 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:55:22.846815 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/4f1bcb6b-7c73-4bff-a246-78ddc5ca22ea-cnibin\") pod \"multus-additional-cni-plugins-wmxvz\" (UID: \"4f1bcb6b-7c73-4bff-a246-78ddc5ca22ea\") " pod="openshift-multus/multus-additional-cni-plugins-wmxvz" Jul 06 18:55:22.846900 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:55:22.846857 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/4f1bcb6b-7c73-4bff-a246-78ddc5ca22ea-os-release\") pod \"multus-additional-cni-plugins-wmxvz\" (UID: \"4f1bcb6b-7c73-4bff-a246-78ddc5ca22ea\") " pod="openshift-multus/multus-additional-cni-plugins-wmxvz" Jul 06 18:55:22.846900 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:55:22.846889 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/ac420ddc-b839-4613-943d-50d18a911d56-run-systemd\") pod \"ovnkube-node-rr76d\" (UID: \"ac420ddc-b839-4613-943d-50d18a911d56\") " pod="openshift-ovn-kubernetes/ovnkube-node-rr76d" Jul 06 18:55:22.847316 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:55:22.846926 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/ac420ddc-b839-4613-943d-50d18a911d56-env-overrides\") pod \"ovnkube-node-rr76d\" (UID: \"ac420ddc-b839-4613-943d-50d18a911d56\") " pod="openshift-ovn-kubernetes/ovnkube-node-rr76d" Jul 06 18:55:22.847316 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:55:22.846950 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/4f1bcb6b-7c73-4bff-a246-78ddc5ca22ea-tuning-conf-dir\") pod \"multus-additional-cni-plugins-wmxvz\" (UID: \"4f1bcb6b-7c73-4bff-a246-78ddc5ca22ea\") " pod="openshift-multus/multus-additional-cni-plugins-wmxvz" Jul 06 18:55:22.847316 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:55:22.846979 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/4f1bcb6b-7c73-4bff-a246-78ddc5ca22ea-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-wmxvz\" (UID: \"4f1bcb6b-7c73-4bff-a246-78ddc5ca22ea\") " pod="openshift-multus/multus-additional-cni-plugins-wmxvz" Jul 06 18:55:22.847316 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:55:22.847004 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/92e4be75-799a-44f0-aaca-22ef4bca49fc-run\") pod \"tuned-fvvmf\" (UID: \"92e4be75-799a-44f0-aaca-22ef4bca49fc\") " pod="openshift-cluster-node-tuning-operator/tuned-fvvmf" Jul 06 18:55:22.847316 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:55:22.847026 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/b9b9a09a-8299-41aa-9b59-51ed106f6802-kubelet-dir\") pod \"aws-ebs-csi-driver-node-k5knh\" (UID: \"b9b9a09a-8299-41aa-9b59-51ed106f6802\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-k5knh" Jul 06 18:55:22.847316 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:55:22.847051 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/92e4be75-799a-44f0-aaca-22ef4bca49fc-etc-sysctl-d\") pod \"tuned-fvvmf\" (UID: \"92e4be75-799a-44f0-aaca-22ef4bca49fc\") " pod="openshift-cluster-node-tuning-operator/tuned-fvvmf" Jul 06 18:55:22.847316 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:55:22.847076 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-r285c\" (UniqueName: \"kubernetes.io/projected/92e4be75-799a-44f0-aaca-22ef4bca49fc-kube-api-access-r285c\") pod \"tuned-fvvmf\" (UID: \"92e4be75-799a-44f0-aaca-22ef4bca49fc\") " pod="openshift-cluster-node-tuning-operator/tuned-fvvmf" Jul 06 18:55:22.847316 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:55:22.847098 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/b9b9a09a-8299-41aa-9b59-51ed106f6802-device-dir\") pod \"aws-ebs-csi-driver-node-k5knh\" (UID: \"b9b9a09a-8299-41aa-9b59-51ed106f6802\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-k5knh" Jul 06 18:55:22.847316 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:55:22.847122 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-m2ckm\" (UniqueName: \"kubernetes.io/projected/9262d667-2ed4-45aa-98f5-2d8158a66ca1-kube-api-access-m2ckm\") pod \"network-metrics-daemon-wh2jx\" (UID: \"9262d667-2ed4-45aa-98f5-2d8158a66ca1\") " pod="openshift-multus/network-metrics-daemon-wh2jx" Jul 06 18:55:22.847316 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:55:22.847149 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/92e4be75-799a-44f0-aaca-22ef4bca49fc-etc-sysctl-conf\") pod \"tuned-fvvmf\" (UID: \"92e4be75-799a-44f0-aaca-22ef4bca49fc\") " pod="openshift-cluster-node-tuning-operator/tuned-fvvmf" Jul 06 18:55:22.847316 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:55:22.847169 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/92e4be75-799a-44f0-aaca-22ef4bca49fc-var-lib-kubelet\") pod \"tuned-fvvmf\" (UID: \"92e4be75-799a-44f0-aaca-22ef4bca49fc\") " pod="openshift-cluster-node-tuning-operator/tuned-fvvmf" Jul 06 18:55:22.847316 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:55:22.847193 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/ac420ddc-b839-4613-943d-50d18a911d56-host-kubelet\") pod \"ovnkube-node-rr76d\" (UID: \"ac420ddc-b839-4613-943d-50d18a911d56\") " pod="openshift-ovn-kubernetes/ovnkube-node-rr76d" Jul 06 18:55:22.847316 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:55:22.847286 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/ac420ddc-b839-4613-943d-50d18a911d56-host-run-netns\") pod \"ovnkube-node-rr76d\" (UID: \"ac420ddc-b839-4613-943d-50d18a911d56\") " pod="openshift-ovn-kubernetes/ovnkube-node-rr76d" Jul 06 18:55:22.847890 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:55:22.847329 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/ac420ddc-b839-4613-943d-50d18a911d56-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-rr76d\" (UID: \"ac420ddc-b839-4613-943d-50d18a911d56\") " pod="openshift-ovn-kubernetes/ovnkube-node-rr76d" Jul 06 18:55:22.847890 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:55:22.847360 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/4f1bcb6b-7c73-4bff-a246-78ddc5ca22ea-cni-binary-copy\") pod \"multus-additional-cni-plugins-wmxvz\" (UID: \"4f1bcb6b-7c73-4bff-a246-78ddc5ca22ea\") " pod="openshift-multus/multus-additional-cni-plugins-wmxvz" Jul 06 18:55:22.847890 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:55:22.847385 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/b9b9a09a-8299-41aa-9b59-51ed106f6802-etc-selinux\") pod \"aws-ebs-csi-driver-node-k5knh\" (UID: \"b9b9a09a-8299-41aa-9b59-51ed106f6802\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-k5knh" Jul 06 18:55:22.847890 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:55:22.847409 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/ac420ddc-b839-4613-943d-50d18a911d56-systemd-units\") pod \"ovnkube-node-rr76d\" (UID: \"ac420ddc-b839-4613-943d-50d18a911d56\") " pod="openshift-ovn-kubernetes/ovnkube-node-rr76d" Jul 06 18:55:22.847890 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:55:22.847433 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ptkmq\" (UniqueName: \"kubernetes.io/projected/b9b9a09a-8299-41aa-9b59-51ed106f6802-kube-api-access-ptkmq\") pod \"aws-ebs-csi-driver-node-k5knh\" (UID: \"b9b9a09a-8299-41aa-9b59-51ed106f6802\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-k5knh" Jul 06 18:55:22.847890 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:55:22.847457 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/ac420ddc-b839-4613-943d-50d18a911d56-var-lib-openvswitch\") pod \"ovnkube-node-rr76d\" (UID: \"ac420ddc-b839-4613-943d-50d18a911d56\") " pod="openshift-ovn-kubernetes/ovnkube-node-rr76d" Jul 06 18:55:22.847890 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:55:22.847480 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/ac420ddc-b839-4613-943d-50d18a911d56-ovnkube-script-lib\") pod \"ovnkube-node-rr76d\" (UID: \"ac420ddc-b839-4613-943d-50d18a911d56\") " pod="openshift-ovn-kubernetes/ovnkube-node-rr76d" Jul 06 18:55:22.847890 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:55:22.847520 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/92e4be75-799a-44f0-aaca-22ef4bca49fc-tmp\") pod \"tuned-fvvmf\" (UID: \"92e4be75-799a-44f0-aaca-22ef4bca49fc\") " pod="openshift-cluster-node-tuning-operator/tuned-fvvmf" Jul 06 18:55:22.847890 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:55:22.847545 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/ac420ddc-b839-4613-943d-50d18a911d56-etc-openvswitch\") pod \"ovnkube-node-rr76d\" (UID: \"ac420ddc-b839-4613-943d-50d18a911d56\") " pod="openshift-ovn-kubernetes/ovnkube-node-rr76d" Jul 06 18:55:22.847890 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:55:22.847592 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/ac420ddc-b839-4613-943d-50d18a911d56-run-openvswitch\") pod \"ovnkube-node-rr76d\" (UID: \"ac420ddc-b839-4613-943d-50d18a911d56\") " pod="openshift-ovn-kubernetes/ovnkube-node-rr76d" Jul 06 18:55:22.847890 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:55:22.847658 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/92e4be75-799a-44f0-aaca-22ef4bca49fc-host\") pod \"tuned-fvvmf\" (UID: \"92e4be75-799a-44f0-aaca-22ef4bca49fc\") " pod="openshift-cluster-node-tuning-operator/tuned-fvvmf" Jul 06 18:55:22.847890 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:55:22.847682 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/a88da320-d446-42e3-b155-418a20f35893-hosts-file\") pod \"node-resolver-4qk4s\" (UID: \"a88da320-d446-42e3-b155-418a20f35893\") " pod="openshift-dns/node-resolver-4qk4s" Jul 06 18:55:22.847890 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:55:22.847703 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/b9b9a09a-8299-41aa-9b59-51ed106f6802-socket-dir\") pod \"aws-ebs-csi-driver-node-k5knh\" (UID: \"b9b9a09a-8299-41aa-9b59-51ed106f6802\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-k5knh" Jul 06 18:55:22.847890 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:55:22.847743 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/b9b9a09a-8299-41aa-9b59-51ed106f6802-registration-dir\") pod \"aws-ebs-csi-driver-node-k5knh\" (UID: \"b9b9a09a-8299-41aa-9b59-51ed106f6802\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-k5knh" Jul 06 18:55:22.869962 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:55:22.869944 2570 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-mcdwg" Jul 06 18:55:22.879158 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:55:22.879137 2570 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-mcdwg" Jul 06 18:55:22.931526 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:55:22.931501 2570 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world" Jul 06 18:55:22.948635 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:55:22.948546 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/92e4be75-799a-44f0-aaca-22ef4bca49fc-tmp\") pod \"tuned-fvvmf\" (UID: \"92e4be75-799a-44f0-aaca-22ef4bca49fc\") " pod="openshift-cluster-node-tuning-operator/tuned-fvvmf" Jul 06 18:55:22.948764 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:55:22.948643 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/ac420ddc-b839-4613-943d-50d18a911d56-etc-openvswitch\") pod \"ovnkube-node-rr76d\" (UID: \"ac420ddc-b839-4613-943d-50d18a911d56\") " pod="openshift-ovn-kubernetes/ovnkube-node-rr76d" Jul 06 18:55:22.948764 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:55:22.948671 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/ac420ddc-b839-4613-943d-50d18a911d56-run-openvswitch\") pod \"ovnkube-node-rr76d\" (UID: \"ac420ddc-b839-4613-943d-50d18a911d56\") " pod="openshift-ovn-kubernetes/ovnkube-node-rr76d" Jul 06 18:55:22.948764 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:55:22.948697 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/da433510-33a8-4e5e-9a3d-29d8d0ec0502-host\") pod \"node-ca-sxf4k\" (UID: \"da433510-33a8-4e5e-9a3d-29d8d0ec0502\") " pod="openshift-image-registry/node-ca-sxf4k" Jul 06 18:55:22.948764 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:55:22.948720 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/92e4be75-799a-44f0-aaca-22ef4bca49fc-host\") pod \"tuned-fvvmf\" (UID: \"92e4be75-799a-44f0-aaca-22ef4bca49fc\") " pod="openshift-cluster-node-tuning-operator/tuned-fvvmf" Jul 06 18:55:22.948936 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:55:22.948759 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/ac420ddc-b839-4613-943d-50d18a911d56-etc-openvswitch\") pod \"ovnkube-node-rr76d\" (UID: \"ac420ddc-b839-4613-943d-50d18a911d56\") " pod="openshift-ovn-kubernetes/ovnkube-node-rr76d" Jul 06 18:55:22.948936 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:55:22.948774 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/a88da320-d446-42e3-b155-418a20f35893-hosts-file\") pod \"node-resolver-4qk4s\" (UID: \"a88da320-d446-42e3-b155-418a20f35893\") " pod="openshift-dns/node-resolver-4qk4s" Jul 06 18:55:22.948936 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:55:22.948799 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/b9b9a09a-8299-41aa-9b59-51ed106f6802-socket-dir\") pod \"aws-ebs-csi-driver-node-k5knh\" (UID: \"b9b9a09a-8299-41aa-9b59-51ed106f6802\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-k5knh" Jul 06 18:55:22.948936 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:55:22.948855 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/92e4be75-799a-44f0-aaca-22ef4bca49fc-host\") pod \"tuned-fvvmf\" (UID: \"92e4be75-799a-44f0-aaca-22ef4bca49fc\") " pod="openshift-cluster-node-tuning-operator/tuned-fvvmf" Jul 06 18:55:22.948936 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:55:22.948897 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/ac420ddc-b839-4613-943d-50d18a911d56-run-openvswitch\") pod \"ovnkube-node-rr76d\" (UID: \"ac420ddc-b839-4613-943d-50d18a911d56\") " pod="openshift-ovn-kubernetes/ovnkube-node-rr76d" Jul 06 18:55:22.949173 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:55:22.948944 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/b9b9a09a-8299-41aa-9b59-51ed106f6802-socket-dir\") pod \"aws-ebs-csi-driver-node-k5knh\" (UID: \"b9b9a09a-8299-41aa-9b59-51ed106f6802\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-k5knh" Jul 06 18:55:22.949173 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:55:22.948941 2570 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory" Jul 06 18:55:22.949173 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:55:22.948951 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/0b813b17-4a1b-4efa-aaa2-2c5e926196e3-cni-binary-copy\") pod \"multus-b66m9\" (UID: \"0b813b17-4a1b-4efa-aaa2-2c5e926196e3\") " pod="openshift-multus/multus-b66m9" Jul 06 18:55:22.949173 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:55:22.948968 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/a88da320-d446-42e3-b155-418a20f35893-hosts-file\") pod \"node-resolver-4qk4s\" (UID: \"a88da320-d446-42e3-b155-418a20f35893\") " pod="openshift-dns/node-resolver-4qk4s" Jul 06 18:55:22.949173 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:55:22.949000 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qfrb6\" (UniqueName: \"kubernetes.io/projected/f7bc558b-d33b-448d-929c-3652d096c3d5-kube-api-access-qfrb6\") pod \"network-check-target-296mj\" (UID: \"f7bc558b-d33b-448d-929c-3652d096c3d5\") " pod="openshift-network-diagnostics/network-check-target-296mj" Jul 06 18:55:22.949173 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:55:22.949031 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/b9b9a09a-8299-41aa-9b59-51ed106f6802-registration-dir\") pod \"aws-ebs-csi-driver-node-k5knh\" (UID: \"b9b9a09a-8299-41aa-9b59-51ed106f6802\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-k5knh" Jul 06 18:55:22.949173 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:55:22.949105 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/b9b9a09a-8299-41aa-9b59-51ed106f6802-registration-dir\") pod \"aws-ebs-csi-driver-node-k5knh\" (UID: \"b9b9a09a-8299-41aa-9b59-51ed106f6802\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-k5knh" Jul 06 18:55:22.949173 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:55:22.949137 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-d96dd\" (UniqueName: \"kubernetes.io/projected/ac420ddc-b839-4613-943d-50d18a911d56-kube-api-access-d96dd\") pod \"ovnkube-node-rr76d\" (UID: \"ac420ddc-b839-4613-943d-50d18a911d56\") " pod="openshift-ovn-kubernetes/ovnkube-node-rr76d" Jul 06 18:55:22.949173 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:55:22.949167 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/4f1bcb6b-7c73-4bff-a246-78ddc5ca22ea-system-cni-dir\") pod \"multus-additional-cni-plugins-wmxvz\" (UID: \"4f1bcb6b-7c73-4bff-a246-78ddc5ca22ea\") " pod="openshift-multus/multus-additional-cni-plugins-wmxvz" Jul 06 18:55:22.949642 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:55:22.949190 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a88da320-d446-42e3-b155-418a20f35893-tmp-dir\") pod \"node-resolver-4qk4s\" (UID: \"a88da320-d446-42e3-b155-418a20f35893\") " pod="openshift-dns/node-resolver-4qk4s" Jul 06 18:55:22.949642 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:55:22.949236 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/17c6ff8a-2c68-400e-b46d-7f73f3673088-host-slash\") pod \"iptables-alerter-qnhx2\" (UID: \"17c6ff8a-2c68-400e-b46d-7f73f3673088\") " pod="openshift-network-operator/iptables-alerter-qnhx2" Jul 06 18:55:22.949642 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:55:22.949264 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/92e4be75-799a-44f0-aaca-22ef4bca49fc-etc-modprobe-d\") pod \"tuned-fvvmf\" (UID: \"92e4be75-799a-44f0-aaca-22ef4bca49fc\") " pod="openshift-cluster-node-tuning-operator/tuned-fvvmf" Jul 06 18:55:22.949642 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:55:22.949325 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/17c6ff8a-2c68-400e-b46d-7f73f3673088-host-slash\") pod \"iptables-alerter-qnhx2\" (UID: \"17c6ff8a-2c68-400e-b46d-7f73f3673088\") " pod="openshift-network-operator/iptables-alerter-qnhx2" Jul 06 18:55:22.949642 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:55:22.949399 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/92e4be75-799a-44f0-aaca-22ef4bca49fc-etc-modprobe-d\") pod \"tuned-fvvmf\" (UID: \"92e4be75-799a-44f0-aaca-22ef4bca49fc\") " pod="openshift-cluster-node-tuning-operator/tuned-fvvmf" Jul 06 18:55:22.949642 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:55:22.949444 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/92e4be75-799a-44f0-aaca-22ef4bca49fc-etc-systemd\") pod \"tuned-fvvmf\" (UID: \"92e4be75-799a-44f0-aaca-22ef4bca49fc\") " pod="openshift-cluster-node-tuning-operator/tuned-fvvmf" Jul 06 18:55:22.949642 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:55:22.949468 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/92e4be75-799a-44f0-aaca-22ef4bca49fc-lib-modules\") pod \"tuned-fvvmf\" (UID: \"92e4be75-799a-44f0-aaca-22ef4bca49fc\") " pod="openshift-cluster-node-tuning-operator/tuned-fvvmf" Jul 06 18:55:22.949642 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:55:22.949490 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/b9b9a09a-8299-41aa-9b59-51ed106f6802-sys-fs\") pod \"aws-ebs-csi-driver-node-k5knh\" (UID: \"b9b9a09a-8299-41aa-9b59-51ed106f6802\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-k5knh" Jul 06 18:55:22.949642 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:55:22.949518 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fzqfj\" (UniqueName: \"kubernetes.io/projected/a88da320-d446-42e3-b155-418a20f35893-kube-api-access-fzqfj\") pod \"node-resolver-4qk4s\" (UID: \"a88da320-d446-42e3-b155-418a20f35893\") " pod="openshift-dns/node-resolver-4qk4s" Jul 06 18:55:22.949642 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:55:22.949538 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/92e4be75-799a-44f0-aaca-22ef4bca49fc-sys\") pod \"tuned-fvvmf\" (UID: \"92e4be75-799a-44f0-aaca-22ef4bca49fc\") " pod="openshift-cluster-node-tuning-operator/tuned-fvvmf" Jul 06 18:55:22.949642 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:55:22.949555 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/ac420ddc-b839-4613-943d-50d18a911d56-host-cni-netd\") pod \"ovnkube-node-rr76d\" (UID: \"ac420ddc-b839-4613-943d-50d18a911d56\") " pod="openshift-ovn-kubernetes/ovnkube-node-rr76d" Jul 06 18:55:22.949642 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:55:22.949569 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/ac420ddc-b839-4613-943d-50d18a911d56-ovn-node-metrics-cert\") pod \"ovnkube-node-rr76d\" (UID: \"ac420ddc-b839-4613-943d-50d18a911d56\") " pod="openshift-ovn-kubernetes/ovnkube-node-rr76d" Jul 06 18:55:22.949642 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:55:22.949568 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/92e4be75-799a-44f0-aaca-22ef4bca49fc-etc-systemd\") pod \"tuned-fvvmf\" (UID: \"92e4be75-799a-44f0-aaca-22ef4bca49fc\") " pod="openshift-cluster-node-tuning-operator/tuned-fvvmf" Jul 06 18:55:22.949642 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:55:22.949585 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/0b813b17-4a1b-4efa-aaa2-2c5e926196e3-os-release\") pod \"multus-b66m9\" (UID: \"0b813b17-4a1b-4efa-aaa2-2c5e926196e3\") " pod="openshift-multus/multus-b66m9" Jul 06 18:55:22.949642 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:55:22.949623 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/0b813b17-4a1b-4efa-aaa2-2c5e926196e3-etc-kubernetes\") pod \"multus-b66m9\" (UID: \"0b813b17-4a1b-4efa-aaa2-2c5e926196e3\") " pod="openshift-multus/multus-b66m9" Jul 06 18:55:22.949642 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:55:22.949642 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/92e4be75-799a-44f0-aaca-22ef4bca49fc-etc-tuned\") pod \"tuned-fvvmf\" (UID: \"92e4be75-799a-44f0-aaca-22ef4bca49fc\") " pod="openshift-cluster-node-tuning-operator/tuned-fvvmf" Jul 06 18:55:22.950376 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:55:22.949686 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/ac420ddc-b839-4613-943d-50d18a911d56-run-ovn\") pod \"ovnkube-node-rr76d\" (UID: \"ac420ddc-b839-4613-943d-50d18a911d56\") " pod="openshift-ovn-kubernetes/ovnkube-node-rr76d" Jul 06 18:55:22.950376 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:55:22.949710 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/ac420ddc-b839-4613-943d-50d18a911d56-node-log\") pod \"ovnkube-node-rr76d\" (UID: \"ac420ddc-b839-4613-943d-50d18a911d56\") " pod="openshift-ovn-kubernetes/ovnkube-node-rr76d" Jul 06 18:55:22.950376 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:55:22.949743 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/4f1bcb6b-7c73-4bff-a246-78ddc5ca22ea-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-wmxvz\" (UID: \"4f1bcb6b-7c73-4bff-a246-78ddc5ca22ea\") " pod="openshift-multus/multus-additional-cni-plugins-wmxvz" Jul 06 18:55:22.950376 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:55:22.949771 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/0b813b17-4a1b-4efa-aaa2-2c5e926196e3-multus-socket-dir-parent\") pod \"multus-b66m9\" (UID: \"0b813b17-4a1b-4efa-aaa2-2c5e926196e3\") " pod="openshift-multus/multus-b66m9" Jul 06 18:55:22.950376 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:55:22.949796 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/0b813b17-4a1b-4efa-aaa2-2c5e926196e3-host-run-netns\") pod \"multus-b66m9\" (UID: \"0b813b17-4a1b-4efa-aaa2-2c5e926196e3\") " pod="openshift-multus/multus-b66m9" Jul 06 18:55:22.950376 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:55:22.949817 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/0b813b17-4a1b-4efa-aaa2-2c5e926196e3-host-var-lib-kubelet\") pod \"multus-b66m9\" (UID: \"0b813b17-4a1b-4efa-aaa2-2c5e926196e3\") " pod="openshift-multus/multus-b66m9" Jul 06 18:55:22.950376 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:55:22.949843 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-z65sk\" (UniqueName: \"kubernetes.io/projected/17c6ff8a-2c68-400e-b46d-7f73f3673088-kube-api-access-z65sk\") pod \"iptables-alerter-qnhx2\" (UID: \"17c6ff8a-2c68-400e-b46d-7f73f3673088\") " pod="openshift-network-operator/iptables-alerter-qnhx2" Jul 06 18:55:22.950376 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:55:22.949858 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a88da320-d446-42e3-b155-418a20f35893-tmp-dir\") pod \"node-resolver-4qk4s\" (UID: \"a88da320-d446-42e3-b155-418a20f35893\") " pod="openshift-dns/node-resolver-4qk4s" Jul 06 18:55:22.950376 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:55:22.949868 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/9262d667-2ed4-45aa-98f5-2d8158a66ca1-metrics-certs\") pod \"network-metrics-daemon-wh2jx\" (UID: \"9262d667-2ed4-45aa-98f5-2d8158a66ca1\") " pod="openshift-multus/network-metrics-daemon-wh2jx" Jul 06 18:55:22.950376 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:55:22.949920 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/92e4be75-799a-44f0-aaca-22ef4bca49fc-etc-kubernetes\") pod \"tuned-fvvmf\" (UID: \"92e4be75-799a-44f0-aaca-22ef4bca49fc\") " pod="openshift-cluster-node-tuning-operator/tuned-fvvmf" Jul 06 18:55:22.950376 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:55:22.949929 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/4f1bcb6b-7c73-4bff-a246-78ddc5ca22ea-system-cni-dir\") pod \"multus-additional-cni-plugins-wmxvz\" (UID: \"4f1bcb6b-7c73-4bff-a246-78ddc5ca22ea\") " pod="openshift-multus/multus-additional-cni-plugins-wmxvz" Jul 06 18:55:22.950376 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:55:22.949946 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/ac420ddc-b839-4613-943d-50d18a911d56-log-socket\") pod \"ovnkube-node-rr76d\" (UID: \"ac420ddc-b839-4613-943d-50d18a911d56\") " pod="openshift-ovn-kubernetes/ovnkube-node-rr76d" Jul 06 18:55:22.950376 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:55:22.949972 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/ac420ddc-b839-4613-943d-50d18a911d56-host-run-ovn-kubernetes\") pod \"ovnkube-node-rr76d\" (UID: \"ac420ddc-b839-4613-943d-50d18a911d56\") " pod="openshift-ovn-kubernetes/ovnkube-node-rr76d" Jul 06 18:55:22.950376 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:55:22.949997 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/ac420ddc-b839-4613-943d-50d18a911d56-ovnkube-config\") pod \"ovnkube-node-rr76d\" (UID: \"ac420ddc-b839-4613-943d-50d18a911d56\") " pod="openshift-ovn-kubernetes/ovnkube-node-rr76d" Jul 06 18:55:22.950376 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:55:22.950036 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/4beea8e8-7750-4a7a-bc49-249b3d7e3ab3-kubelet-config\") pod \"global-pull-secret-syncer-ln98s\" (UID: \"4beea8e8-7750-4a7a-bc49-249b3d7e3ab3\") " pod="kube-system/global-pull-secret-syncer-ln98s" Jul 06 18:55:22.950376 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:55:22.950083 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/17c6ff8a-2c68-400e-b46d-7f73f3673088-iptables-alerter-script\") pod \"iptables-alerter-qnhx2\" (UID: \"17c6ff8a-2c68-400e-b46d-7f73f3673088\") " pod="openshift-network-operator/iptables-alerter-qnhx2" Jul 06 18:55:22.950376 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:55:22.950121 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/92e4be75-799a-44f0-aaca-22ef4bca49fc-etc-sysconfig\") pod \"tuned-fvvmf\" (UID: \"92e4be75-799a-44f0-aaca-22ef4bca49fc\") " pod="openshift-cluster-node-tuning-operator/tuned-fvvmf" Jul 06 18:55:22.951222 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:55:22.950154 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/ac420ddc-b839-4613-943d-50d18a911d56-host-cni-bin\") pod \"ovnkube-node-rr76d\" (UID: \"ac420ddc-b839-4613-943d-50d18a911d56\") " pod="openshift-ovn-kubernetes/ovnkube-node-rr76d" Jul 06 18:55:22.951222 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:55:22.950179 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vzd9s\" (UniqueName: \"kubernetes.io/projected/4f1bcb6b-7c73-4bff-a246-78ddc5ca22ea-kube-api-access-vzd9s\") pod \"multus-additional-cni-plugins-wmxvz\" (UID: \"4f1bcb6b-7c73-4bff-a246-78ddc5ca22ea\") " pod="openshift-multus/multus-additional-cni-plugins-wmxvz" Jul 06 18:55:22.951222 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:55:22.950217 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/0b813b17-4a1b-4efa-aaa2-2c5e926196e3-system-cni-dir\") pod \"multus-b66m9\" (UID: \"0b813b17-4a1b-4efa-aaa2-2c5e926196e3\") " pod="openshift-multus/multus-b66m9" Jul 06 18:55:22.951222 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:55:22.950248 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/92e4be75-799a-44f0-aaca-22ef4bca49fc-sys\") pod \"tuned-fvvmf\" (UID: \"92e4be75-799a-44f0-aaca-22ef4bca49fc\") " pod="openshift-cluster-node-tuning-operator/tuned-fvvmf" Jul 06 18:55:22.951222 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:55:22.950310 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/92e4be75-799a-44f0-aaca-22ef4bca49fc-etc-kubernetes\") pod \"tuned-fvvmf\" (UID: \"92e4be75-799a-44f0-aaca-22ef4bca49fc\") " pod="openshift-cluster-node-tuning-operator/tuned-fvvmf" Jul 06 18:55:22.951222 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:55:22.950316 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/0b813b17-4a1b-4efa-aaa2-2c5e926196e3-host-run-k8s-cni-cncf-io\") pod \"multus-b66m9\" (UID: \"0b813b17-4a1b-4efa-aaa2-2c5e926196e3\") " pod="openshift-multus/multus-b66m9" Jul 06 18:55:22.951222 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:55:22.950346 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/92e4be75-799a-44f0-aaca-22ef4bca49fc-lib-modules\") pod \"tuned-fvvmf\" (UID: \"92e4be75-799a-44f0-aaca-22ef4bca49fc\") " pod="openshift-cluster-node-tuning-operator/tuned-fvvmf" Jul 06 18:55:22.951222 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:55:22.950354 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/0b813b17-4a1b-4efa-aaa2-2c5e926196e3-hostroot\") pod \"multus-b66m9\" (UID: \"0b813b17-4a1b-4efa-aaa2-2c5e926196e3\") " pod="openshift-multus/multus-b66m9" Jul 06 18:55:22.951222 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:55:22.950388 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/ac420ddc-b839-4613-943d-50d18a911d56-host-run-ovn-kubernetes\") pod \"ovnkube-node-rr76d\" (UID: \"ac420ddc-b839-4613-943d-50d18a911d56\") " pod="openshift-ovn-kubernetes/ovnkube-node-rr76d" Jul 06 18:55:22.951222 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:55:22.950399 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/b9b9a09a-8299-41aa-9b59-51ed106f6802-sys-fs\") pod \"aws-ebs-csi-driver-node-k5knh\" (UID: \"b9b9a09a-8299-41aa-9b59-51ed106f6802\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-k5knh" Jul 06 18:55:22.951222 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:55:22.950453 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/ac420ddc-b839-4613-943d-50d18a911d56-host-cni-netd\") pod \"ovnkube-node-rr76d\" (UID: \"ac420ddc-b839-4613-943d-50d18a911d56\") " pod="openshift-ovn-kubernetes/ovnkube-node-rr76d" Jul 06 18:55:22.951222 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:55:22.950523 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/ac420ddc-b839-4613-943d-50d18a911d56-log-socket\") pod \"ovnkube-node-rr76d\" (UID: \"ac420ddc-b839-4613-943d-50d18a911d56\") " pod="openshift-ovn-kubernetes/ovnkube-node-rr76d" Jul 06 18:55:22.951222 ip-10-0-130-59 kubenswrapper[2570]: E0706 18:55:22.950906 2570 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 06 18:55:22.951222 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:55:22.951009 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/4f1bcb6b-7c73-4bff-a246-78ddc5ca22ea-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-wmxvz\" (UID: \"4f1bcb6b-7c73-4bff-a246-78ddc5ca22ea\") " pod="openshift-multus/multus-additional-cni-plugins-wmxvz" Jul 06 18:55:22.951222 ip-10-0-130-59 kubenswrapper[2570]: E0706 18:55:22.951042 2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9262d667-2ed4-45aa-98f5-2d8158a66ca1-metrics-certs podName:9262d667-2ed4-45aa-98f5-2d8158a66ca1 nodeName:}" failed. No retries permitted until 2026-07-06 18:55:23.451023732 +0000 UTC m=+3.006044895 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/9262d667-2ed4-45aa-98f5-2d8158a66ca1-metrics-certs") pod "network-metrics-daemon-wh2jx" (UID: "9262d667-2ed4-45aa-98f5-2d8158a66ca1") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 06 18:55:22.951222 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:55:22.951042 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/ac420ddc-b839-4613-943d-50d18a911d56-node-log\") pod \"ovnkube-node-rr76d\" (UID: \"ac420ddc-b839-4613-943d-50d18a911d56\") " pod="openshift-ovn-kubernetes/ovnkube-node-rr76d" Jul 06 18:55:22.951222 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:55:22.951095 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/ac420ddc-b839-4613-943d-50d18a911d56-host-cni-bin\") pod \"ovnkube-node-rr76d\" (UID: \"ac420ddc-b839-4613-943d-50d18a911d56\") " pod="openshift-ovn-kubernetes/ovnkube-node-rr76d" Jul 06 18:55:22.952032 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:55:22.951102 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/17c6ff8a-2c68-400e-b46d-7f73f3673088-iptables-alerter-script\") pod \"iptables-alerter-qnhx2\" (UID: \"17c6ff8a-2c68-400e-b46d-7f73f3673088\") " pod="openshift-network-operator/iptables-alerter-qnhx2" Jul 06 18:55:22.952032 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:55:22.950388 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/0b813b17-4a1b-4efa-aaa2-2c5e926196e3-host-run-multus-certs\") pod \"multus-b66m9\" (UID: \"0b813b17-4a1b-4efa-aaa2-2c5e926196e3\") " pod="openshift-multus/multus-b66m9" Jul 06 18:55:22.952032 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:55:22.951160 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/ac420ddc-b839-4613-943d-50d18a911d56-host-slash\") pod \"ovnkube-node-rr76d\" (UID: \"ac420ddc-b839-4613-943d-50d18a911d56\") " pod="openshift-ovn-kubernetes/ovnkube-node-rr76d" Jul 06 18:55:22.952032 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:55:22.951177 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/ac420ddc-b839-4613-943d-50d18a911d56-run-ovn\") pod \"ovnkube-node-rr76d\" (UID: \"ac420ddc-b839-4613-943d-50d18a911d56\") " pod="openshift-ovn-kubernetes/ovnkube-node-rr76d" Jul 06 18:55:22.952032 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:55:22.951215 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/4f1bcb6b-7c73-4bff-a246-78ddc5ca22ea-cnibin\") pod \"multus-additional-cni-plugins-wmxvz\" (UID: \"4f1bcb6b-7c73-4bff-a246-78ddc5ca22ea\") " pod="openshift-multus/multus-additional-cni-plugins-wmxvz" Jul 06 18:55:22.952032 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:55:22.951230 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/92e4be75-799a-44f0-aaca-22ef4bca49fc-etc-sysconfig\") pod \"tuned-fvvmf\" (UID: \"92e4be75-799a-44f0-aaca-22ef4bca49fc\") " pod="openshift-cluster-node-tuning-operator/tuned-fvvmf" Jul 06 18:55:22.952032 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:55:22.951248 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/ac420ddc-b839-4613-943d-50d18a911d56-host-slash\") pod \"ovnkube-node-rr76d\" (UID: \"ac420ddc-b839-4613-943d-50d18a911d56\") " pod="openshift-ovn-kubernetes/ovnkube-node-rr76d" Jul 06 18:55:22.952032 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:55:22.951422 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/4f1bcb6b-7c73-4bff-a246-78ddc5ca22ea-os-release\") pod \"multus-additional-cni-plugins-wmxvz\" (UID: \"4f1bcb6b-7c73-4bff-a246-78ddc5ca22ea\") " pod="openshift-multus/multus-additional-cni-plugins-wmxvz" Jul 06 18:55:22.952032 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:55:22.951428 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/4f1bcb6b-7c73-4bff-a246-78ddc5ca22ea-cnibin\") pod \"multus-additional-cni-plugins-wmxvz\" (UID: \"4f1bcb6b-7c73-4bff-a246-78ddc5ca22ea\") " pod="openshift-multus/multus-additional-cni-plugins-wmxvz" Jul 06 18:55:22.952032 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:55:22.951562 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/0b813b17-4a1b-4efa-aaa2-2c5e926196e3-multus-cni-dir\") pod \"multus-b66m9\" (UID: \"0b813b17-4a1b-4efa-aaa2-2c5e926196e3\") " pod="openshift-multus/multus-b66m9" Jul 06 18:55:22.952032 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:55:22.951593 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/4beea8e8-7750-4a7a-bc49-249b3d7e3ab3-original-pull-secret\") pod \"global-pull-secret-syncer-ln98s\" (UID: \"4beea8e8-7750-4a7a-bc49-249b3d7e3ab3\") " pod="kube-system/global-pull-secret-syncer-ln98s" Jul 06 18:55:22.952032 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:55:22.951643 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-n7gwg\" (UniqueName: \"kubernetes.io/projected/da433510-33a8-4e5e-9a3d-29d8d0ec0502-kube-api-access-n7gwg\") pod \"node-ca-sxf4k\" (UID: \"da433510-33a8-4e5e-9a3d-29d8d0ec0502\") " pod="openshift-image-registry/node-ca-sxf4k" Jul 06 18:55:22.952032 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:55:22.951674 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/ac420ddc-b839-4613-943d-50d18a911d56-run-systemd\") pod \"ovnkube-node-rr76d\" (UID: \"ac420ddc-b839-4613-943d-50d18a911d56\") " pod="openshift-ovn-kubernetes/ovnkube-node-rr76d" Jul 06 18:55:22.952032 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:55:22.951702 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/4f1bcb6b-7c73-4bff-a246-78ddc5ca22ea-os-release\") pod \"multus-additional-cni-plugins-wmxvz\" (UID: \"4f1bcb6b-7c73-4bff-a246-78ddc5ca22ea\") " pod="openshift-multus/multus-additional-cni-plugins-wmxvz" Jul 06 18:55:22.952032 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:55:22.951707 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/ac420ddc-b839-4613-943d-50d18a911d56-env-overrides\") pod \"ovnkube-node-rr76d\" (UID: \"ac420ddc-b839-4613-943d-50d18a911d56\") " pod="openshift-ovn-kubernetes/ovnkube-node-rr76d" Jul 06 18:55:22.952032 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:55:22.951703 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/ac420ddc-b839-4613-943d-50d18a911d56-ovnkube-config\") pod \"ovnkube-node-rr76d\" (UID: \"ac420ddc-b839-4613-943d-50d18a911d56\") " pod="openshift-ovn-kubernetes/ovnkube-node-rr76d" Jul 06 18:55:22.952032 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:55:22.951734 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/ac420ddc-b839-4613-943d-50d18a911d56-run-systemd\") pod \"ovnkube-node-rr76d\" (UID: \"ac420ddc-b839-4613-943d-50d18a911d56\") " pod="openshift-ovn-kubernetes/ovnkube-node-rr76d" Jul 06 18:55:22.952837 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:55:22.951778 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/4f1bcb6b-7c73-4bff-a246-78ddc5ca22ea-tuning-conf-dir\") pod \"multus-additional-cni-plugins-wmxvz\" (UID: \"4f1bcb6b-7c73-4bff-a246-78ddc5ca22ea\") " pod="openshift-multus/multus-additional-cni-plugins-wmxvz" Jul 06 18:55:22.952837 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:55:22.951878 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/4f1bcb6b-7c73-4bff-a246-78ddc5ca22ea-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-wmxvz\" (UID: \"4f1bcb6b-7c73-4bff-a246-78ddc5ca22ea\") " pod="openshift-multus/multus-additional-cni-plugins-wmxvz" Jul 06 18:55:22.952837 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:55:22.951937 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/0b813b17-4a1b-4efa-aaa2-2c5e926196e3-host-var-lib-cni-multus\") pod \"multus-b66m9\" (UID: \"0b813b17-4a1b-4efa-aaa2-2c5e926196e3\") " pod="openshift-multus/multus-b66m9" Jul 06 18:55:22.952837 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:55:22.951964 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/0b813b17-4a1b-4efa-aaa2-2c5e926196e3-multus-conf-dir\") pod \"multus-b66m9\" (UID: \"0b813b17-4a1b-4efa-aaa2-2c5e926196e3\") " pod="openshift-multus/multus-b66m9" Jul 06 18:55:22.952837 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:55:22.951990 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/4f1bcb6b-7c73-4bff-a246-78ddc5ca22ea-tuning-conf-dir\") pod \"multus-additional-cni-plugins-wmxvz\" (UID: \"4f1bcb6b-7c73-4bff-a246-78ddc5ca22ea\") " pod="openshift-multus/multus-additional-cni-plugins-wmxvz" Jul 06 18:55:22.952837 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:55:22.951997 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/bc7057bd-88da-48af-ba32-e6efdc6e8362-agent-certs\") pod \"konnectivity-agent-k8fbq\" (UID: \"bc7057bd-88da-48af-ba32-e6efdc6e8362\") " pod="kube-system/konnectivity-agent-k8fbq" Jul 06 18:55:22.952837 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:55:22.952091 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/92e4be75-799a-44f0-aaca-22ef4bca49fc-run\") pod \"tuned-fvvmf\" (UID: \"92e4be75-799a-44f0-aaca-22ef4bca49fc\") " pod="openshift-cluster-node-tuning-operator/tuned-fvvmf" Jul 06 18:55:22.952837 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:55:22.952117 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/b9b9a09a-8299-41aa-9b59-51ed106f6802-kubelet-dir\") pod \"aws-ebs-csi-driver-node-k5knh\" (UID: \"b9b9a09a-8299-41aa-9b59-51ed106f6802\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-k5knh" Jul 06 18:55:22.952837 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:55:22.952142 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/bc7057bd-88da-48af-ba32-e6efdc6e8362-konnectivity-ca\") pod \"konnectivity-agent-k8fbq\" (UID: \"bc7057bd-88da-48af-ba32-e6efdc6e8362\") " pod="kube-system/konnectivity-agent-k8fbq" Jul 06 18:55:22.952837 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:55:22.952164 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/4beea8e8-7750-4a7a-bc49-249b3d7e3ab3-dbus\") pod \"global-pull-secret-syncer-ln98s\" (UID: \"4beea8e8-7750-4a7a-bc49-249b3d7e3ab3\") " pod="kube-system/global-pull-secret-syncer-ln98s" Jul 06 18:55:22.952837 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:55:22.952187 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/da433510-33a8-4e5e-9a3d-29d8d0ec0502-serviceca\") pod \"node-ca-sxf4k\" (UID: \"da433510-33a8-4e5e-9a3d-29d8d0ec0502\") " pod="openshift-image-registry/node-ca-sxf4k" Jul 06 18:55:22.952837 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:55:22.952207 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/ac420ddc-b839-4613-943d-50d18a911d56-env-overrides\") pod \"ovnkube-node-rr76d\" (UID: \"ac420ddc-b839-4613-943d-50d18a911d56\") " pod="openshift-ovn-kubernetes/ovnkube-node-rr76d" Jul 06 18:55:22.952837 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:55:22.952212 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/92e4be75-799a-44f0-aaca-22ef4bca49fc-etc-sysctl-d\") pod \"tuned-fvvmf\" (UID: \"92e4be75-799a-44f0-aaca-22ef4bca49fc\") " pod="openshift-cluster-node-tuning-operator/tuned-fvvmf" Jul 06 18:55:22.952837 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:55:22.952235 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-r285c\" (UniqueName: \"kubernetes.io/projected/92e4be75-799a-44f0-aaca-22ef4bca49fc-kube-api-access-r285c\") pod \"tuned-fvvmf\" (UID: \"92e4be75-799a-44f0-aaca-22ef4bca49fc\") " pod="openshift-cluster-node-tuning-operator/tuned-fvvmf" Jul 06 18:55:22.952837 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:55:22.952257 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/b9b9a09a-8299-41aa-9b59-51ed106f6802-device-dir\") pod \"aws-ebs-csi-driver-node-k5knh\" (UID: \"b9b9a09a-8299-41aa-9b59-51ed106f6802\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-k5knh" Jul 06 18:55:22.952837 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:55:22.952276 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/0b813b17-4a1b-4efa-aaa2-2c5e926196e3-multus-daemon-config\") pod \"multus-b66m9\" (UID: \"0b813b17-4a1b-4efa-aaa2-2c5e926196e3\") " pod="openshift-multus/multus-b66m9" Jul 06 18:55:22.952837 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:55:22.952301 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-m2ckm\" (UniqueName: \"kubernetes.io/projected/9262d667-2ed4-45aa-98f5-2d8158a66ca1-kube-api-access-m2ckm\") pod \"network-metrics-daemon-wh2jx\" (UID: \"9262d667-2ed4-45aa-98f5-2d8158a66ca1\") " pod="openshift-multus/network-metrics-daemon-wh2jx" Jul 06 18:55:22.953621 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:55:22.952310 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/92e4be75-799a-44f0-aaca-22ef4bca49fc-run\") pod \"tuned-fvvmf\" (UID: \"92e4be75-799a-44f0-aaca-22ef4bca49fc\") " pod="openshift-cluster-node-tuning-operator/tuned-fvvmf" Jul 06 18:55:22.953621 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:55:22.952326 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/92e4be75-799a-44f0-aaca-22ef4bca49fc-etc-sysctl-conf\") pod \"tuned-fvvmf\" (UID: \"92e4be75-799a-44f0-aaca-22ef4bca49fc\") " pod="openshift-cluster-node-tuning-operator/tuned-fvvmf" Jul 06 18:55:22.953621 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:55:22.952352 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/92e4be75-799a-44f0-aaca-22ef4bca49fc-var-lib-kubelet\") pod \"tuned-fvvmf\" (UID: \"92e4be75-799a-44f0-aaca-22ef4bca49fc\") " pod="openshift-cluster-node-tuning-operator/tuned-fvvmf" Jul 06 18:55:22.953621 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:55:22.952373 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/ac420ddc-b839-4613-943d-50d18a911d56-host-kubelet\") pod \"ovnkube-node-rr76d\" (UID: \"ac420ddc-b839-4613-943d-50d18a911d56\") " pod="openshift-ovn-kubernetes/ovnkube-node-rr76d" Jul 06 18:55:22.953621 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:55:22.952394 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/ac420ddc-b839-4613-943d-50d18a911d56-host-run-netns\") pod \"ovnkube-node-rr76d\" (UID: \"ac420ddc-b839-4613-943d-50d18a911d56\") " pod="openshift-ovn-kubernetes/ovnkube-node-rr76d" Jul 06 18:55:22.953621 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:55:22.952419 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/ac420ddc-b839-4613-943d-50d18a911d56-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-rr76d\" (UID: \"ac420ddc-b839-4613-943d-50d18a911d56\") " pod="openshift-ovn-kubernetes/ovnkube-node-rr76d" Jul 06 18:55:22.953621 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:55:22.952442 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/4f1bcb6b-7c73-4bff-a246-78ddc5ca22ea-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-wmxvz\" (UID: \"4f1bcb6b-7c73-4bff-a246-78ddc5ca22ea\") " pod="openshift-multus/multus-additional-cni-plugins-wmxvz" Jul 06 18:55:22.953621 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:55:22.952444 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/4f1bcb6b-7c73-4bff-a246-78ddc5ca22ea-cni-binary-copy\") pod \"multus-additional-cni-plugins-wmxvz\" (UID: \"4f1bcb6b-7c73-4bff-a246-78ddc5ca22ea\") " pod="openshift-multus/multus-additional-cni-plugins-wmxvz" Jul 06 18:55:22.953621 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:55:22.952487 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/0b813b17-4a1b-4efa-aaa2-2c5e926196e3-host-var-lib-cni-bin\") pod \"multus-b66m9\" (UID: \"0b813b17-4a1b-4efa-aaa2-2c5e926196e3\") " pod="openshift-multus/multus-b66m9" Jul 06 18:55:22.953621 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:55:22.952516 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/b9b9a09a-8299-41aa-9b59-51ed106f6802-etc-selinux\") pod \"aws-ebs-csi-driver-node-k5knh\" (UID: \"b9b9a09a-8299-41aa-9b59-51ed106f6802\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-k5knh" Jul 06 18:55:22.953621 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:55:22.952555 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/0b813b17-4a1b-4efa-aaa2-2c5e926196e3-cnibin\") pod \"multus-b66m9\" (UID: \"0b813b17-4a1b-4efa-aaa2-2c5e926196e3\") " pod="openshift-multus/multus-b66m9" Jul 06 18:55:22.953621 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:55:22.952589 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nkcgk\" (UniqueName: \"kubernetes.io/projected/0b813b17-4a1b-4efa-aaa2-2c5e926196e3-kube-api-access-nkcgk\") pod \"multus-b66m9\" (UID: \"0b813b17-4a1b-4efa-aaa2-2c5e926196e3\") " pod="openshift-multus/multus-b66m9" Jul 06 18:55:22.953621 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:55:22.952632 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/ac420ddc-b839-4613-943d-50d18a911d56-systemd-units\") pod \"ovnkube-node-rr76d\" (UID: \"ac420ddc-b839-4613-943d-50d18a911d56\") " pod="openshift-ovn-kubernetes/ovnkube-node-rr76d" Jul 06 18:55:22.953621 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:55:22.952672 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ptkmq\" (UniqueName: \"kubernetes.io/projected/b9b9a09a-8299-41aa-9b59-51ed106f6802-kube-api-access-ptkmq\") pod \"aws-ebs-csi-driver-node-k5knh\" (UID: \"b9b9a09a-8299-41aa-9b59-51ed106f6802\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-k5knh" Jul 06 18:55:22.953621 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:55:22.952728 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/ac420ddc-b839-4613-943d-50d18a911d56-host-run-netns\") pod \"ovnkube-node-rr76d\" (UID: \"ac420ddc-b839-4613-943d-50d18a911d56\") " pod="openshift-ovn-kubernetes/ovnkube-node-rr76d" Jul 06 18:55:22.953621 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:55:22.952741 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/b9b9a09a-8299-41aa-9b59-51ed106f6802-device-dir\") pod \"aws-ebs-csi-driver-node-k5knh\" (UID: \"b9b9a09a-8299-41aa-9b59-51ed106f6802\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-k5knh" Jul 06 18:55:22.953621 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:55:22.952767 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/ac420ddc-b839-4613-943d-50d18a911d56-host-kubelet\") pod \"ovnkube-node-rr76d\" (UID: \"ac420ddc-b839-4613-943d-50d18a911d56\") " pod="openshift-ovn-kubernetes/ovnkube-node-rr76d" Jul 06 18:55:22.954412 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:55:22.952776 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/ac420ddc-b839-4613-943d-50d18a911d56-systemd-units\") pod \"ovnkube-node-rr76d\" (UID: \"ac420ddc-b839-4613-943d-50d18a911d56\") " pod="openshift-ovn-kubernetes/ovnkube-node-rr76d" Jul 06 18:55:22.954412 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:55:22.952864 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/92e4be75-799a-44f0-aaca-22ef4bca49fc-etc-sysctl-conf\") pod \"tuned-fvvmf\" (UID: \"92e4be75-799a-44f0-aaca-22ef4bca49fc\") " pod="openshift-cluster-node-tuning-operator/tuned-fvvmf" Jul 06 18:55:22.954412 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:55:22.952904 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/4f1bcb6b-7c73-4bff-a246-78ddc5ca22ea-cni-binary-copy\") pod \"multus-additional-cni-plugins-wmxvz\" (UID: \"4f1bcb6b-7c73-4bff-a246-78ddc5ca22ea\") " pod="openshift-multus/multus-additional-cni-plugins-wmxvz" Jul 06 18:55:22.954412 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:55:22.952945 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/b9b9a09a-8299-41aa-9b59-51ed106f6802-etc-selinux\") pod \"aws-ebs-csi-driver-node-k5knh\" (UID: \"b9b9a09a-8299-41aa-9b59-51ed106f6802\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-k5knh" Jul 06 18:55:22.954412 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:55:22.952969 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/ac420ddc-b839-4613-943d-50d18a911d56-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-rr76d\" (UID: \"ac420ddc-b839-4613-943d-50d18a911d56\") " pod="openshift-ovn-kubernetes/ovnkube-node-rr76d" Jul 06 18:55:22.954412 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:55:22.953003 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/92e4be75-799a-44f0-aaca-22ef4bca49fc-var-lib-kubelet\") pod \"tuned-fvvmf\" (UID: \"92e4be75-799a-44f0-aaca-22ef4bca49fc\") " pod="openshift-cluster-node-tuning-operator/tuned-fvvmf" Jul 06 18:55:22.954412 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:55:22.953008 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/ac420ddc-b839-4613-943d-50d18a911d56-var-lib-openvswitch\") pod \"ovnkube-node-rr76d\" (UID: \"ac420ddc-b839-4613-943d-50d18a911d56\") " pod="openshift-ovn-kubernetes/ovnkube-node-rr76d" Jul 06 18:55:22.954412 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:55:22.952969 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/ac420ddc-b839-4613-943d-50d18a911d56-var-lib-openvswitch\") pod \"ovnkube-node-rr76d\" (UID: \"ac420ddc-b839-4613-943d-50d18a911d56\") " pod="openshift-ovn-kubernetes/ovnkube-node-rr76d" Jul 06 18:55:22.954412 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:55:22.953044 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/ac420ddc-b839-4613-943d-50d18a911d56-ovnkube-script-lib\") pod \"ovnkube-node-rr76d\" (UID: \"ac420ddc-b839-4613-943d-50d18a911d56\") " pod="openshift-ovn-kubernetes/ovnkube-node-rr76d" Jul 06 18:55:22.954412 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:55:22.953064 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/92e4be75-799a-44f0-aaca-22ef4bca49fc-etc-sysctl-d\") pod \"tuned-fvvmf\" (UID: \"92e4be75-799a-44f0-aaca-22ef4bca49fc\") " pod="openshift-cluster-node-tuning-operator/tuned-fvvmf" Jul 06 18:55:22.954412 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:55:22.953514 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/92e4be75-799a-44f0-aaca-22ef4bca49fc-tmp\") pod \"tuned-fvvmf\" (UID: \"92e4be75-799a-44f0-aaca-22ef4bca49fc\") " pod="openshift-cluster-node-tuning-operator/tuned-fvvmf" Jul 06 18:55:22.954412 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:55:22.953575 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/ac420ddc-b839-4613-943d-50d18a911d56-ovnkube-script-lib\") pod \"ovnkube-node-rr76d\" (UID: \"ac420ddc-b839-4613-943d-50d18a911d56\") " pod="openshift-ovn-kubernetes/ovnkube-node-rr76d" Jul 06 18:55:22.954412 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:55:22.953974 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/ac420ddc-b839-4613-943d-50d18a911d56-ovn-node-metrics-cert\") pod \"ovnkube-node-rr76d\" (UID: \"ac420ddc-b839-4613-943d-50d18a911d56\") " pod="openshift-ovn-kubernetes/ovnkube-node-rr76d" Jul 06 18:55:22.955042 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:55:22.954433 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/b9b9a09a-8299-41aa-9b59-51ed106f6802-kubelet-dir\") pod \"aws-ebs-csi-driver-node-k5knh\" (UID: \"b9b9a09a-8299-41aa-9b59-51ed106f6802\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-k5knh" Jul 06 18:55:22.955692 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:55:22.955669 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/92e4be75-799a-44f0-aaca-22ef4bca49fc-etc-tuned\") pod \"tuned-fvvmf\" (UID: \"92e4be75-799a-44f0-aaca-22ef4bca49fc\") " pod="openshift-cluster-node-tuning-operator/tuned-fvvmf" Jul 06 18:55:22.957484 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:55:22.957460 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-d96dd\" (UniqueName: \"kubernetes.io/projected/ac420ddc-b839-4613-943d-50d18a911d56-kube-api-access-d96dd\") pod \"ovnkube-node-rr76d\" (UID: \"ac420ddc-b839-4613-943d-50d18a911d56\") " pod="openshift-ovn-kubernetes/ovnkube-node-rr76d" Jul 06 18:55:22.959564 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:55:22.959538 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-vzd9s\" (UniqueName: \"kubernetes.io/projected/4f1bcb6b-7c73-4bff-a246-78ddc5ca22ea-kube-api-access-vzd9s\") pod \"multus-additional-cni-plugins-wmxvz\" (UID: \"4f1bcb6b-7c73-4bff-a246-78ddc5ca22ea\") " pod="openshift-multus/multus-additional-cni-plugins-wmxvz" Jul 06 18:55:22.960005 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:55:22.959978 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-fzqfj\" (UniqueName: \"kubernetes.io/projected/a88da320-d446-42e3-b155-418a20f35893-kube-api-access-fzqfj\") pod \"node-resolver-4qk4s\" (UID: \"a88da320-d446-42e3-b155-418a20f35893\") " pod="openshift-dns/node-resolver-4qk4s" Jul 06 18:55:22.960859 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:55:22.960817 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-z65sk\" (UniqueName: \"kubernetes.io/projected/17c6ff8a-2c68-400e-b46d-7f73f3673088-kube-api-access-z65sk\") pod \"iptables-alerter-qnhx2\" (UID: \"17c6ff8a-2c68-400e-b46d-7f73f3673088\") " pod="openshift-network-operator/iptables-alerter-qnhx2" Jul 06 18:55:22.961039 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:55:22.961015 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-r285c\" (UniqueName: \"kubernetes.io/projected/92e4be75-799a-44f0-aaca-22ef4bca49fc-kube-api-access-r285c\") pod \"tuned-fvvmf\" (UID: \"92e4be75-799a-44f0-aaca-22ef4bca49fc\") " pod="openshift-cluster-node-tuning-operator/tuned-fvvmf" Jul 06 18:55:22.961337 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:55:22.961318 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-ptkmq\" (UniqueName: \"kubernetes.io/projected/b9b9a09a-8299-41aa-9b59-51ed106f6802-kube-api-access-ptkmq\") pod \"aws-ebs-csi-driver-node-k5knh\" (UID: \"b9b9a09a-8299-41aa-9b59-51ed106f6802\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-k5knh" Jul 06 18:55:22.962562 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:55:22.962541 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-m2ckm\" (UniqueName: \"kubernetes.io/projected/9262d667-2ed4-45aa-98f5-2d8158a66ca1-kube-api-access-m2ckm\") pod \"network-metrics-daemon-wh2jx\" (UID: \"9262d667-2ed4-45aa-98f5-2d8158a66ca1\") " pod="openshift-multus/network-metrics-daemon-wh2jx" Jul 06 18:55:22.992182 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:55:22.992152 2570 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160" Jul 06 18:55:23.054940 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:55:23.054181 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/0b813b17-4a1b-4efa-aaa2-2c5e926196e3-host-var-lib-cni-multus\") pod \"multus-b66m9\" (UID: \"0b813b17-4a1b-4efa-aaa2-2c5e926196e3\") " pod="openshift-multus/multus-b66m9" Jul 06 18:55:23.054940 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:55:23.054227 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/0b813b17-4a1b-4efa-aaa2-2c5e926196e3-multus-conf-dir\") pod \"multus-b66m9\" (UID: \"0b813b17-4a1b-4efa-aaa2-2c5e926196e3\") " pod="openshift-multus/multus-b66m9" Jul 06 18:55:23.054940 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:55:23.054255 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/bc7057bd-88da-48af-ba32-e6efdc6e8362-agent-certs\") pod \"konnectivity-agent-k8fbq\" (UID: \"bc7057bd-88da-48af-ba32-e6efdc6e8362\") " pod="kube-system/konnectivity-agent-k8fbq" Jul 06 18:55:23.054940 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:55:23.054281 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/bc7057bd-88da-48af-ba32-e6efdc6e8362-konnectivity-ca\") pod \"konnectivity-agent-k8fbq\" (UID: \"bc7057bd-88da-48af-ba32-e6efdc6e8362\") " pod="kube-system/konnectivity-agent-k8fbq" Jul 06 18:55:23.054940 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:55:23.054304 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/4beea8e8-7750-4a7a-bc49-249b3d7e3ab3-dbus\") pod \"global-pull-secret-syncer-ln98s\" (UID: \"4beea8e8-7750-4a7a-bc49-249b3d7e3ab3\") " pod="kube-system/global-pull-secret-syncer-ln98s" Jul 06 18:55:23.054940 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:55:23.054327 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/da433510-33a8-4e5e-9a3d-29d8d0ec0502-serviceca\") pod \"node-ca-sxf4k\" (UID: \"da433510-33a8-4e5e-9a3d-29d8d0ec0502\") " pod="openshift-image-registry/node-ca-sxf4k" Jul 06 18:55:23.054940 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:55:23.054344 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/0b813b17-4a1b-4efa-aaa2-2c5e926196e3-multus-daemon-config\") pod \"multus-b66m9\" (UID: \"0b813b17-4a1b-4efa-aaa2-2c5e926196e3\") " pod="openshift-multus/multus-b66m9" Jul 06 18:55:23.054940 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:55:23.054370 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/0b813b17-4a1b-4efa-aaa2-2c5e926196e3-host-var-lib-cni-bin\") pod \"multus-b66m9\" (UID: \"0b813b17-4a1b-4efa-aaa2-2c5e926196e3\") " pod="openshift-multus/multus-b66m9" Jul 06 18:55:23.054940 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:55:23.054394 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/0b813b17-4a1b-4efa-aaa2-2c5e926196e3-cnibin\") pod \"multus-b66m9\" (UID: \"0b813b17-4a1b-4efa-aaa2-2c5e926196e3\") " pod="openshift-multus/multus-b66m9" Jul 06 18:55:23.054940 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:55:23.054409 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nkcgk\" (UniqueName: \"kubernetes.io/projected/0b813b17-4a1b-4efa-aaa2-2c5e926196e3-kube-api-access-nkcgk\") pod \"multus-b66m9\" (UID: \"0b813b17-4a1b-4efa-aaa2-2c5e926196e3\") " pod="openshift-multus/multus-b66m9" Jul 06 18:55:23.054940 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:55:23.054436 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/da433510-33a8-4e5e-9a3d-29d8d0ec0502-host\") pod \"node-ca-sxf4k\" (UID: \"da433510-33a8-4e5e-9a3d-29d8d0ec0502\") " pod="openshift-image-registry/node-ca-sxf4k" Jul 06 18:55:23.054940 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:55:23.054454 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/0b813b17-4a1b-4efa-aaa2-2c5e926196e3-cni-binary-copy\") pod \"multus-b66m9\" (UID: \"0b813b17-4a1b-4efa-aaa2-2c5e926196e3\") " pod="openshift-multus/multus-b66m9" Jul 06 18:55:23.054940 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:55:23.054471 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qfrb6\" (UniqueName: \"kubernetes.io/projected/f7bc558b-d33b-448d-929c-3652d096c3d5-kube-api-access-qfrb6\") pod \"network-check-target-296mj\" (UID: \"f7bc558b-d33b-448d-929c-3652d096c3d5\") " pod="openshift-network-diagnostics/network-check-target-296mj" Jul 06 18:55:23.054940 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:55:23.054513 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/0b813b17-4a1b-4efa-aaa2-2c5e926196e3-os-release\") pod \"multus-b66m9\" (UID: \"0b813b17-4a1b-4efa-aaa2-2c5e926196e3\") " pod="openshift-multus/multus-b66m9" Jul 06 18:55:23.054940 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:55:23.054537 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/0b813b17-4a1b-4efa-aaa2-2c5e926196e3-etc-kubernetes\") pod \"multus-b66m9\" (UID: \"0b813b17-4a1b-4efa-aaa2-2c5e926196e3\") " pod="openshift-multus/multus-b66m9" Jul 06 18:55:23.054940 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:55:23.054558 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/0b813b17-4a1b-4efa-aaa2-2c5e926196e3-multus-socket-dir-parent\") pod \"multus-b66m9\" (UID: \"0b813b17-4a1b-4efa-aaa2-2c5e926196e3\") " pod="openshift-multus/multus-b66m9" Jul 06 18:55:23.054940 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:55:23.054577 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/0b813b17-4a1b-4efa-aaa2-2c5e926196e3-host-run-netns\") pod \"multus-b66m9\" (UID: \"0b813b17-4a1b-4efa-aaa2-2c5e926196e3\") " pod="openshift-multus/multus-b66m9" Jul 06 18:55:23.054940 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:55:23.054594 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/0b813b17-4a1b-4efa-aaa2-2c5e926196e3-host-var-lib-kubelet\") pod \"multus-b66m9\" (UID: \"0b813b17-4a1b-4efa-aaa2-2c5e926196e3\") " pod="openshift-multus/multus-b66m9" Jul 06 18:55:23.055704 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:55:23.054691 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/0b813b17-4a1b-4efa-aaa2-2c5e926196e3-cnibin\") pod \"multus-b66m9\" (UID: \"0b813b17-4a1b-4efa-aaa2-2c5e926196e3\") " pod="openshift-multus/multus-b66m9" Jul 06 18:55:23.055704 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:55:23.054709 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/0b813b17-4a1b-4efa-aaa2-2c5e926196e3-multus-socket-dir-parent\") pod \"multus-b66m9\" (UID: \"0b813b17-4a1b-4efa-aaa2-2c5e926196e3\") " pod="openshift-multus/multus-b66m9" Jul 06 18:55:23.055704 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:55:23.054748 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/0b813b17-4a1b-4efa-aaa2-2c5e926196e3-multus-conf-dir\") pod \"multus-b66m9\" (UID: \"0b813b17-4a1b-4efa-aaa2-2c5e926196e3\") " pod="openshift-multus/multus-b66m9" Jul 06 18:55:23.055704 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:55:23.054786 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/0b813b17-4a1b-4efa-aaa2-2c5e926196e3-host-run-netns\") pod \"multus-b66m9\" (UID: \"0b813b17-4a1b-4efa-aaa2-2c5e926196e3\") " pod="openshift-multus/multus-b66m9" Jul 06 18:55:23.055704 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:55:23.054839 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/0b813b17-4a1b-4efa-aaa2-2c5e926196e3-os-release\") pod \"multus-b66m9\" (UID: \"0b813b17-4a1b-4efa-aaa2-2c5e926196e3\") " pod="openshift-multus/multus-b66m9" Jul 06 18:55:23.055704 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:55:23.054875 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/0b813b17-4a1b-4efa-aaa2-2c5e926196e3-etc-kubernetes\") pod \"multus-b66m9\" (UID: \"0b813b17-4a1b-4efa-aaa2-2c5e926196e3\") " pod="openshift-multus/multus-b66m9" Jul 06 18:55:23.055704 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:55:23.054909 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/0b813b17-4a1b-4efa-aaa2-2c5e926196e3-host-var-lib-cni-multus\") pod \"multus-b66m9\" (UID: \"0b813b17-4a1b-4efa-aaa2-2c5e926196e3\") " pod="openshift-multus/multus-b66m9" Jul 06 18:55:23.055704 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:55:23.055067 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/da433510-33a8-4e5e-9a3d-29d8d0ec0502-host\") pod \"node-ca-sxf4k\" (UID: \"da433510-33a8-4e5e-9a3d-29d8d0ec0502\") " pod="openshift-image-registry/node-ca-sxf4k" Jul 06 18:55:23.055704 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:55:23.055125 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/0b813b17-4a1b-4efa-aaa2-2c5e926196e3-host-var-lib-kubelet\") pod \"multus-b66m9\" (UID: \"0b813b17-4a1b-4efa-aaa2-2c5e926196e3\") " pod="openshift-multus/multus-b66m9" Jul 06 18:55:23.055704 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:55:23.055169 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/0b813b17-4a1b-4efa-aaa2-2c5e926196e3-host-var-lib-cni-bin\") pod \"multus-b66m9\" (UID: \"0b813b17-4a1b-4efa-aaa2-2c5e926196e3\") " pod="openshift-multus/multus-b66m9" Jul 06 18:55:23.055704 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:55:23.055333 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/da433510-33a8-4e5e-9a3d-29d8d0ec0502-serviceca\") pod \"node-ca-sxf4k\" (UID: \"da433510-33a8-4e5e-9a3d-29d8d0ec0502\") " pod="openshift-image-registry/node-ca-sxf4k" Jul 06 18:55:23.055704 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:55:23.055480 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/4beea8e8-7750-4a7a-bc49-249b3d7e3ab3-dbus\") pod \"global-pull-secret-syncer-ln98s\" (UID: \"4beea8e8-7750-4a7a-bc49-249b3d7e3ab3\") " pod="kube-system/global-pull-secret-syncer-ln98s" Jul 06 18:55:23.055704 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:55:23.055525 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/4beea8e8-7750-4a7a-bc49-249b3d7e3ab3-kubelet-config\") pod \"global-pull-secret-syncer-ln98s\" (UID: \"4beea8e8-7750-4a7a-bc49-249b3d7e3ab3\") " pod="kube-system/global-pull-secret-syncer-ln98s" Jul 06 18:55:23.055704 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:55:23.055556 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/0b813b17-4a1b-4efa-aaa2-2c5e926196e3-system-cni-dir\") pod \"multus-b66m9\" (UID: \"0b813b17-4a1b-4efa-aaa2-2c5e926196e3\") " pod="openshift-multus/multus-b66m9" Jul 06 18:55:23.055704 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:55:23.055581 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/0b813b17-4a1b-4efa-aaa2-2c5e926196e3-host-run-k8s-cni-cncf-io\") pod \"multus-b66m9\" (UID: \"0b813b17-4a1b-4efa-aaa2-2c5e926196e3\") " pod="openshift-multus/multus-b66m9" Jul 06 18:55:23.055704 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:55:23.055596 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/0b813b17-4a1b-4efa-aaa2-2c5e926196e3-multus-daemon-config\") pod \"multus-b66m9\" (UID: \"0b813b17-4a1b-4efa-aaa2-2c5e926196e3\") " pod="openshift-multus/multus-b66m9" Jul 06 18:55:23.055704 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:55:23.055624 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/0b813b17-4a1b-4efa-aaa2-2c5e926196e3-hostroot\") pod \"multus-b66m9\" (UID: \"0b813b17-4a1b-4efa-aaa2-2c5e926196e3\") " pod="openshift-multus/multus-b66m9" Jul 06 18:55:23.055704 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:55:23.055643 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/0b813b17-4a1b-4efa-aaa2-2c5e926196e3-cni-binary-copy\") pod \"multus-b66m9\" (UID: \"0b813b17-4a1b-4efa-aaa2-2c5e926196e3\") " pod="openshift-multus/multus-b66m9" Jul 06 18:55:23.056522 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:55:23.055652 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/0b813b17-4a1b-4efa-aaa2-2c5e926196e3-host-run-multus-certs\") pod \"multus-b66m9\" (UID: \"0b813b17-4a1b-4efa-aaa2-2c5e926196e3\") " pod="openshift-multus/multus-b66m9" Jul 06 18:55:23.056522 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:55:23.055678 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/bc7057bd-88da-48af-ba32-e6efdc6e8362-konnectivity-ca\") pod \"konnectivity-agent-k8fbq\" (UID: \"bc7057bd-88da-48af-ba32-e6efdc6e8362\") " pod="kube-system/konnectivity-agent-k8fbq" Jul 06 18:55:23.056522 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:55:23.055687 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/0b813b17-4a1b-4efa-aaa2-2c5e926196e3-system-cni-dir\") pod \"multus-b66m9\" (UID: \"0b813b17-4a1b-4efa-aaa2-2c5e926196e3\") " pod="openshift-multus/multus-b66m9" Jul 06 18:55:23.056522 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:55:23.055687 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/0b813b17-4a1b-4efa-aaa2-2c5e926196e3-host-run-multus-certs\") pod \"multus-b66m9\" (UID: \"0b813b17-4a1b-4efa-aaa2-2c5e926196e3\") " pod="openshift-multus/multus-b66m9" Jul 06 18:55:23.056522 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:55:23.055700 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/0b813b17-4a1b-4efa-aaa2-2c5e926196e3-multus-cni-dir\") pod \"multus-b66m9\" (UID: \"0b813b17-4a1b-4efa-aaa2-2c5e926196e3\") " pod="openshift-multus/multus-b66m9" Jul 06 18:55:23.056522 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:55:23.055720 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/4beea8e8-7750-4a7a-bc49-249b3d7e3ab3-original-pull-secret\") pod \"global-pull-secret-syncer-ln98s\" (UID: \"4beea8e8-7750-4a7a-bc49-249b3d7e3ab3\") " pod="kube-system/global-pull-secret-syncer-ln98s" Jul 06 18:55:23.056522 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:55:23.055723 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/0b813b17-4a1b-4efa-aaa2-2c5e926196e3-host-run-k8s-cni-cncf-io\") pod \"multus-b66m9\" (UID: \"0b813b17-4a1b-4efa-aaa2-2c5e926196e3\") " pod="openshift-multus/multus-b66m9" Jul 06 18:55:23.056522 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:55:23.055738 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/4beea8e8-7750-4a7a-bc49-249b3d7e3ab3-kubelet-config\") pod \"global-pull-secret-syncer-ln98s\" (UID: \"4beea8e8-7750-4a7a-bc49-249b3d7e3ab3\") " pod="kube-system/global-pull-secret-syncer-ln98s" Jul 06 18:55:23.056522 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:55:23.055743 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-n7gwg\" (UniqueName: \"kubernetes.io/projected/da433510-33a8-4e5e-9a3d-29d8d0ec0502-kube-api-access-n7gwg\") pod \"node-ca-sxf4k\" (UID: \"da433510-33a8-4e5e-9a3d-29d8d0ec0502\") " pod="openshift-image-registry/node-ca-sxf4k" Jul 06 18:55:23.056522 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:55:23.055745 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/0b813b17-4a1b-4efa-aaa2-2c5e926196e3-hostroot\") pod \"multus-b66m9\" (UID: \"0b813b17-4a1b-4efa-aaa2-2c5e926196e3\") " pod="openshift-multus/multus-b66m9" Jul 06 18:55:23.056522 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:55:23.055781 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/0b813b17-4a1b-4efa-aaa2-2c5e926196e3-multus-cni-dir\") pod \"multus-b66m9\" (UID: \"0b813b17-4a1b-4efa-aaa2-2c5e926196e3\") " pod="openshift-multus/multus-b66m9" Jul 06 18:55:23.056522 ip-10-0-130-59 kubenswrapper[2570]: E0706 18:55:23.055880 2570 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 06 18:55:23.056522 ip-10-0-130-59 kubenswrapper[2570]: E0706 18:55:23.055999 2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4beea8e8-7750-4a7a-bc49-249b3d7e3ab3-original-pull-secret podName:4beea8e8-7750-4a7a-bc49-249b3d7e3ab3 nodeName:}" failed. No retries permitted until 2026-07-06 18:55:23.555975759 +0000 UTC m=+3.110996927 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/4beea8e8-7750-4a7a-bc49-249b3d7e3ab3-original-pull-secret") pod "global-pull-secret-syncer-ln98s" (UID: "4beea8e8-7750-4a7a-bc49-249b3d7e3ab3") : object "kube-system"/"original-pull-secret" not registered Jul 06 18:55:23.057417 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:55:23.057381 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/bc7057bd-88da-48af-ba32-e6efdc6e8362-agent-certs\") pod \"konnectivity-agent-k8fbq\" (UID: \"bc7057bd-88da-48af-ba32-e6efdc6e8362\") " pod="kube-system/konnectivity-agent-k8fbq" Jul 06 18:55:23.063546 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:55:23.063524 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-nkcgk\" (UniqueName: \"kubernetes.io/projected/0b813b17-4a1b-4efa-aaa2-2c5e926196e3-kube-api-access-nkcgk\") pod \"multus-b66m9\" (UID: \"0b813b17-4a1b-4efa-aaa2-2c5e926196e3\") " pod="openshift-multus/multus-b66m9" Jul 06 18:55:23.063774 ip-10-0-130-59 kubenswrapper[2570]: E0706 18:55:23.063752 2570 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 06 18:55:23.063865 ip-10-0-130-59 kubenswrapper[2570]: E0706 18:55:23.063777 2570 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 06 18:55:23.063865 ip-10-0-130-59 kubenswrapper[2570]: E0706 18:55:23.063792 2570 projected.go:196] Error preparing data for projected volume kube-api-access-qfrb6 for pod openshift-network-diagnostics/network-check-target-296mj: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 06 18:55:23.063865 ip-10-0-130-59 kubenswrapper[2570]: E0706 18:55:23.063845 2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/f7bc558b-d33b-448d-929c-3652d096c3d5-kube-api-access-qfrb6 podName:f7bc558b-d33b-448d-929c-3652d096c3d5 nodeName:}" failed. No retries permitted until 2026-07-06 18:55:23.563833871 +0000 UTC m=+3.118855040 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-qfrb6" (UniqueName: "kubernetes.io/projected/f7bc558b-d33b-448d-929c-3652d096c3d5-kube-api-access-qfrb6") pod "network-check-target-296mj" (UID: "f7bc558b-d33b-448d-929c-3652d096c3d5") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 06 18:55:23.066182 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:55:23.066160 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-n7gwg\" (UniqueName: \"kubernetes.io/projected/da433510-33a8-4e5e-9a3d-29d8d0ec0502-kube-api-access-n7gwg\") pod \"node-ca-sxf4k\" (UID: \"da433510-33a8-4e5e-9a3d-29d8d0ec0502\") " pod="openshift-image-registry/node-ca-sxf4k" Jul 06 18:55:23.128968 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:55:23.128945 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-4qk4s" Jul 06 18:55:23.138292 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:55:23.138273 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-qnhx2" Jul 06 18:55:23.146495 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:55:23.146476 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-k5knh" Jul 06 18:55:23.153695 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:55:23.153678 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-b66m9" Jul 06 18:55:23.166927 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:55:23.166901 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-rr76d" Jul 06 18:55:23.174127 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:55:23.174110 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-wmxvz" Jul 06 18:55:23.181438 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:55:23.181419 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-fvvmf" Jul 06 18:55:23.187676 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:55:23.187662 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-k8fbq" Jul 06 18:55:23.192061 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:55:23.192043 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-sxf4k" Jul 06 18:55:23.458697 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:55:23.458667 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/9262d667-2ed4-45aa-98f5-2d8158a66ca1-metrics-certs\") pod \"network-metrics-daemon-wh2jx\" (UID: \"9262d667-2ed4-45aa-98f5-2d8158a66ca1\") " pod="openshift-multus/network-metrics-daemon-wh2jx" Jul 06 18:55:23.458879 ip-10-0-130-59 kubenswrapper[2570]: E0706 18:55:23.458782 2570 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 06 18:55:23.458879 ip-10-0-130-59 kubenswrapper[2570]: E0706 18:55:23.458853 2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9262d667-2ed4-45aa-98f5-2d8158a66ca1-metrics-certs podName:9262d667-2ed4-45aa-98f5-2d8158a66ca1 nodeName:}" failed. No retries permitted until 2026-07-06 18:55:24.458839554 +0000 UTC m=+4.013860719 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/9262d667-2ed4-45aa-98f5-2d8158a66ca1-metrics-certs") pod "network-metrics-daemon-wh2jx" (UID: "9262d667-2ed4-45aa-98f5-2d8158a66ca1") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 06 18:55:23.559384 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:55:23.559346 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/4beea8e8-7750-4a7a-bc49-249b3d7e3ab3-original-pull-secret\") pod \"global-pull-secret-syncer-ln98s\" (UID: \"4beea8e8-7750-4a7a-bc49-249b3d7e3ab3\") " pod="kube-system/global-pull-secret-syncer-ln98s" Jul 06 18:55:23.559532 ip-10-0-130-59 kubenswrapper[2570]: E0706 18:55:23.559499 2570 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 06 18:55:23.559585 ip-10-0-130-59 kubenswrapper[2570]: E0706 18:55:23.559569 2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4beea8e8-7750-4a7a-bc49-249b3d7e3ab3-original-pull-secret podName:4beea8e8-7750-4a7a-bc49-249b3d7e3ab3 nodeName:}" failed. No retries permitted until 2026-07-06 18:55:24.559553351 +0000 UTC m=+4.114574503 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/4beea8e8-7750-4a7a-bc49-249b3d7e3ab3-original-pull-secret") pod "global-pull-secret-syncer-ln98s" (UID: "4beea8e8-7750-4a7a-bc49-249b3d7e3ab3") : object "kube-system"/"original-pull-secret" not registered Jul 06 18:55:23.660382 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:55:23.660348 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qfrb6\" (UniqueName: \"kubernetes.io/projected/f7bc558b-d33b-448d-929c-3652d096c3d5-kube-api-access-qfrb6\") pod \"network-check-target-296mj\" (UID: \"f7bc558b-d33b-448d-929c-3652d096c3d5\") " pod="openshift-network-diagnostics/network-check-target-296mj" Jul 06 18:55:23.660554 ip-10-0-130-59 kubenswrapper[2570]: E0706 18:55:23.660521 2570 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 06 18:55:23.660554 ip-10-0-130-59 kubenswrapper[2570]: E0706 18:55:23.660547 2570 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 06 18:55:23.660693 ip-10-0-130-59 kubenswrapper[2570]: E0706 18:55:23.660561 2570 projected.go:196] Error preparing data for projected volume kube-api-access-qfrb6 for pod openshift-network-diagnostics/network-check-target-296mj: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 06 18:55:23.660693 ip-10-0-130-59 kubenswrapper[2570]: E0706 18:55:23.660640 2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/f7bc558b-d33b-448d-929c-3652d096c3d5-kube-api-access-qfrb6 podName:f7bc558b-d33b-448d-929c-3652d096c3d5 nodeName:}" failed. No retries permitted until 2026-07-06 18:55:24.660621274 +0000 UTC m=+4.215642438 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-qfrb6" (UniqueName: "kubernetes.io/projected/f7bc558b-d33b-448d-929c-3652d096c3d5-kube-api-access-qfrb6") pod "network-check-target-296mj" (UID: "f7bc558b-d33b-448d-929c-3652d096c3d5") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 06 18:55:23.881803 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:55:23.881729 2570 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-05 18:50:22 +0000 UTC" deadline="2028-04-01 05:47:23.254554252 +0000 UTC" Jul 06 18:55:23.881803 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:55:23.881758 2570 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="15226h51m59.372801187s" Jul 06 18:55:23.990736 ip-10-0-130-59 kubenswrapper[2570]: W0706 18:55:23.990700 2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod92e4be75_799a_44f0_aaca_22ef4bca49fc.slice/crio-589a2e381b4722794691cb22b69d71c71b4d15af00d6ed8e9585314f5b3822b3 WatchSource:0}: Error finding container 589a2e381b4722794691cb22b69d71c71b4d15af00d6ed8e9585314f5b3822b3: Status 404 returned error can't find the container with id 589a2e381b4722794691cb22b69d71c71b4d15af00d6ed8e9585314f5b3822b3 Jul 06 18:55:23.995332 ip-10-0-130-59 kubenswrapper[2570]: W0706 18:55:23.995307 2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0b813b17_4a1b_4efa_aaa2_2c5e926196e3.slice/crio-21859eee202deadffc76c00c99df6c34271dd3cf30ec71a4d907935fc8d21137 WatchSource:0}: Error finding container 21859eee202deadffc76c00c99df6c34271dd3cf30ec71a4d907935fc8d21137: Status 404 returned error can't find the container with id 21859eee202deadffc76c00c99df6c34271dd3cf30ec71a4d907935fc8d21137 Jul 06 18:55:23.995564 ip-10-0-130-59 kubenswrapper[2570]: W0706 18:55:23.995525 2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podda433510_33a8_4e5e_9a3d_29d8d0ec0502.slice/crio-1863e455aa682f89d1c16c3b059189487ac21fc764ff239ef24d70446f8f075d WatchSource:0}: Error finding container 1863e455aa682f89d1c16c3b059189487ac21fc764ff239ef24d70446f8f075d: Status 404 returned error can't find the container with id 1863e455aa682f89d1c16c3b059189487ac21fc764ff239ef24d70446f8f075d Jul 06 18:55:23.997189 ip-10-0-130-59 kubenswrapper[2570]: W0706 18:55:23.997066 2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4f1bcb6b_7c73_4bff_a246_78ddc5ca22ea.slice/crio-a142a3db198c0b4095426876d52a5ccbe81d32636060cc9d872626bc78cb82ec WatchSource:0}: Error finding container a142a3db198c0b4095426876d52a5ccbe81d32636060cc9d872626bc78cb82ec: Status 404 returned error can't find the container with id a142a3db198c0b4095426876d52a5ccbe81d32636060cc9d872626bc78cb82ec Jul 06 18:55:23.997939 ip-10-0-130-59 kubenswrapper[2570]: W0706 18:55:23.997897 2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb9b9a09a_8299_41aa_9b59_51ed106f6802.slice/crio-14571f70c45aae813f95a5374672fe577dfc58dea510cf54e4e360b1da623643 WatchSource:0}: Error finding container 14571f70c45aae813f95a5374672fe577dfc58dea510cf54e4e360b1da623643: Status 404 returned error can't find the container with id 14571f70c45aae813f95a5374672fe577dfc58dea510cf54e4e360b1da623643 Jul 06 18:55:23.999257 ip-10-0-130-59 kubenswrapper[2570]: W0706 18:55:23.999232 2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbc7057bd_88da_48af_ba32_e6efdc6e8362.slice/crio-74b008ad967b33dbfe66ff5256890046f3406e47af95e7d6cabee2703d40618b WatchSource:0}: Error finding container 74b008ad967b33dbfe66ff5256890046f3406e47af95e7d6cabee2703d40618b: Status 404 returned error can't find the container with id 74b008ad967b33dbfe66ff5256890046f3406e47af95e7d6cabee2703d40618b Jul 06 18:55:24.000996 ip-10-0-130-59 kubenswrapper[2570]: W0706 18:55:24.000897 2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod17c6ff8a_2c68_400e_b46d_7f73f3673088.slice/crio-600ae1b64ea7b76fb2272c0c8b916d08a6aa60f3710acbb56e03b1936ed213f3 WatchSource:0}: Error finding container 600ae1b64ea7b76fb2272c0c8b916d08a6aa60f3710acbb56e03b1936ed213f3: Status 404 returned error can't find the container with id 600ae1b64ea7b76fb2272c0c8b916d08a6aa60f3710acbb56e03b1936ed213f3 Jul 06 18:55:24.021465 ip-10-0-130-59 kubenswrapper[2570]: W0706 18:55:24.021442 2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda88da320_d446_42e3_b155_418a20f35893.slice/crio-dab2a87d7ca6a4b94291d3ea30048c21604e73ed2c09931f4d4642592375105c WatchSource:0}: Error finding container dab2a87d7ca6a4b94291d3ea30048c21604e73ed2c09931f4d4642592375105c: Status 404 returned error can't find the container with id dab2a87d7ca6a4b94291d3ea30048c21604e73ed2c09931f4d4642592375105c Jul 06 18:55:24.022106 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:55:24.021945 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-ln98s" Jul 06 18:55:24.022106 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:55:24.021974 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-wh2jx" Jul 06 18:55:24.022106 ip-10-0-130-59 kubenswrapper[2570]: E0706 18:55:24.022063 2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-ln98s" podUID="4beea8e8-7750-4a7a-bc49-249b3d7e3ab3" Jul 06 18:55:24.022286 ip-10-0-130-59 kubenswrapper[2570]: E0706 18:55:24.022183 2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-wh2jx" podUID="9262d667-2ed4-45aa-98f5-2d8158a66ca1" Jul 06 18:55:24.022286 ip-10-0-130-59 kubenswrapper[2570]: W0706 18:55:24.022258 2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podac420ddc_b839_4613_943d_50d18a911d56.slice/crio-91b33b5c7b22d4e35ef489ce13fc2026dcfdfbee32d3c8c68d77b089d7d0880c WatchSource:0}: Error finding container 91b33b5c7b22d4e35ef489ce13fc2026dcfdfbee32d3c8c68d77b089d7d0880c: Status 404 returned error can't find the container with id 91b33b5c7b22d4e35ef489ce13fc2026dcfdfbee32d3c8c68d77b089d7d0880c Jul 06 18:55:24.028245 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:55:24.028143 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-4qk4s" event={"ID":"a88da320-d446-42e3-b155-418a20f35893","Type":"ContainerStarted","Data":"dab2a87d7ca6a4b94291d3ea30048c21604e73ed2c09931f4d4642592375105c"} Jul 06 18:55:24.029847 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:55:24.029821 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-wmxvz" event={"ID":"4f1bcb6b-7c73-4bff-a246-78ddc5ca22ea","Type":"ContainerStarted","Data":"a142a3db198c0b4095426876d52a5ccbe81d32636060cc9d872626bc78cb82ec"} Jul 06 18:55:24.031040 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:55:24.031015 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-sxf4k" event={"ID":"da433510-33a8-4e5e-9a3d-29d8d0ec0502","Type":"ContainerStarted","Data":"1863e455aa682f89d1c16c3b059189487ac21fc764ff239ef24d70446f8f075d"} Jul 06 18:55:24.032120 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:55:24.032093 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-b66m9" event={"ID":"0b813b17-4a1b-4efa-aaa2-2c5e926196e3","Type":"ContainerStarted","Data":"21859eee202deadffc76c00c99df6c34271dd3cf30ec71a4d907935fc8d21137"} Jul 06 18:55:24.036433 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:55:24.036341 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-fvvmf" event={"ID":"92e4be75-799a-44f0-aaca-22ef4bca49fc","Type":"ContainerStarted","Data":"589a2e381b4722794691cb22b69d71c71b4d15af00d6ed8e9585314f5b3822b3"} Jul 06 18:55:24.037542 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:55:24.037516 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-rr76d" event={"ID":"ac420ddc-b839-4613-943d-50d18a911d56","Type":"ContainerStarted","Data":"91b33b5c7b22d4e35ef489ce13fc2026dcfdfbee32d3c8c68d77b089d7d0880c"} Jul 06 18:55:24.039059 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:55:24.039039 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-qnhx2" event={"ID":"17c6ff8a-2c68-400e-b46d-7f73f3673088","Type":"ContainerStarted","Data":"600ae1b64ea7b76fb2272c0c8b916d08a6aa60f3710acbb56e03b1936ed213f3"} Jul 06 18:55:24.040193 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:55:24.040168 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-k8fbq" event={"ID":"bc7057bd-88da-48af-ba32-e6efdc6e8362","Type":"ContainerStarted","Data":"74b008ad967b33dbfe66ff5256890046f3406e47af95e7d6cabee2703d40618b"} Jul 06 18:55:24.041298 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:55:24.041279 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-k5knh" event={"ID":"b9b9a09a-8299-41aa-9b59-51ed106f6802","Type":"ContainerStarted","Data":"14571f70c45aae813f95a5374672fe577dfc58dea510cf54e4e360b1da623643"} Jul 06 18:55:24.468334 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:55:24.468024 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/9262d667-2ed4-45aa-98f5-2d8158a66ca1-metrics-certs\") pod \"network-metrics-daemon-wh2jx\" (UID: \"9262d667-2ed4-45aa-98f5-2d8158a66ca1\") " pod="openshift-multus/network-metrics-daemon-wh2jx" Jul 06 18:55:24.468334 ip-10-0-130-59 kubenswrapper[2570]: E0706 18:55:24.468254 2570 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 06 18:55:24.468334 ip-10-0-130-59 kubenswrapper[2570]: E0706 18:55:24.468330 2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9262d667-2ed4-45aa-98f5-2d8158a66ca1-metrics-certs podName:9262d667-2ed4-45aa-98f5-2d8158a66ca1 nodeName:}" failed. No retries permitted until 2026-07-06 18:55:26.468308815 +0000 UTC m=+6.023329977 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/9262d667-2ed4-45aa-98f5-2d8158a66ca1-metrics-certs") pod "network-metrics-daemon-wh2jx" (UID: "9262d667-2ed4-45aa-98f5-2d8158a66ca1") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 06 18:55:24.569197 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:55:24.569110 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/4beea8e8-7750-4a7a-bc49-249b3d7e3ab3-original-pull-secret\") pod \"global-pull-secret-syncer-ln98s\" (UID: \"4beea8e8-7750-4a7a-bc49-249b3d7e3ab3\") " pod="kube-system/global-pull-secret-syncer-ln98s" Jul 06 18:55:24.569352 ip-10-0-130-59 kubenswrapper[2570]: E0706 18:55:24.569314 2570 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 06 18:55:24.569430 ip-10-0-130-59 kubenswrapper[2570]: E0706 18:55:24.569375 2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4beea8e8-7750-4a7a-bc49-249b3d7e3ab3-original-pull-secret podName:4beea8e8-7750-4a7a-bc49-249b3d7e3ab3 nodeName:}" failed. No retries permitted until 2026-07-06 18:55:26.569357212 +0000 UTC m=+6.124378375 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/4beea8e8-7750-4a7a-bc49-249b3d7e3ab3-original-pull-secret") pod "global-pull-secret-syncer-ln98s" (UID: "4beea8e8-7750-4a7a-bc49-249b3d7e3ab3") : object "kube-system"/"original-pull-secret" not registered Jul 06 18:55:24.670666 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:55:24.670629 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qfrb6\" (UniqueName: \"kubernetes.io/projected/f7bc558b-d33b-448d-929c-3652d096c3d5-kube-api-access-qfrb6\") pod \"network-check-target-296mj\" (UID: \"f7bc558b-d33b-448d-929c-3652d096c3d5\") " pod="openshift-network-diagnostics/network-check-target-296mj" Jul 06 18:55:24.670844 ip-10-0-130-59 kubenswrapper[2570]: E0706 18:55:24.670802 2570 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 06 18:55:24.670844 ip-10-0-130-59 kubenswrapper[2570]: E0706 18:55:24.670820 2570 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 06 18:55:24.670844 ip-10-0-130-59 kubenswrapper[2570]: E0706 18:55:24.670832 2570 projected.go:196] Error preparing data for projected volume kube-api-access-qfrb6 for pod openshift-network-diagnostics/network-check-target-296mj: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 06 18:55:24.671007 ip-10-0-130-59 kubenswrapper[2570]: E0706 18:55:24.670890 2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/f7bc558b-d33b-448d-929c-3652d096c3d5-kube-api-access-qfrb6 podName:f7bc558b-d33b-448d-929c-3652d096c3d5 nodeName:}" failed. No retries permitted until 2026-07-06 18:55:26.670871983 +0000 UTC m=+6.225893156 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-qfrb6" (UniqueName: "kubernetes.io/projected/f7bc558b-d33b-448d-929c-3652d096c3d5-kube-api-access-qfrb6") pod "network-check-target-296mj" (UID: "f7bc558b-d33b-448d-929c-3652d096c3d5") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 06 18:55:24.711509 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:55:24.711480 2570 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160" Jul 06 18:55:24.882822 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:55:24.882689 2570 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-05 18:50:22 +0000 UTC" deadline="2028-02-20 02:52:14.986915329 +0000 UTC" Jul 06 18:55:24.882822 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:55:24.882746 2570 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="14239h56m50.104173314s" Jul 06 18:55:25.024741 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:55:25.024709 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-296mj" Jul 06 18:55:25.024907 ip-10-0-130-59 kubenswrapper[2570]: E0706 18:55:25.024842 2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-296mj" podUID="f7bc558b-d33b-448d-929c-3652d096c3d5" Jul 06 18:55:25.063658 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:55:25.063217 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-130-59.ec2.internal" event={"ID":"d57d2f4948f6aacb3452bada80a2e242","Type":"ContainerStarted","Data":"28917a0e6091ecfcb20c72859793ae45e0823f5be1a84557a74430323f18837e"} Jul 06 18:55:25.075053 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:55:25.074965 2570 generic.go:356] "Generic (PLEG): container finished" podID="f20193a4bb56b9075be1ec3e656dd631" containerID="4e0baadfcfd9e604bacd637ce87cd2613e5ed18d40f1343d17bd6d90c9a25c4c" exitCode=0 Jul 06 18:55:25.075053 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:55:25.075010 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-59.ec2.internal" event={"ID":"f20193a4bb56b9075be1ec3e656dd631","Type":"ContainerDied","Data":"4e0baadfcfd9e604bacd637ce87cd2613e5ed18d40f1343d17bd6d90c9a25c4c"} Jul 06 18:55:25.078250 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:55:25.078201 2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-130-59.ec2.internal" podStartSLOduration=3.078184548 podStartE2EDuration="3.078184548s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-06 18:55:22 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 18:55:25.077996074 +0000 UTC m=+4.633017247" watchObservedRunningTime="2026-07-06 18:55:25.078184548 +0000 UTC m=+4.633205721" Jul 06 18:55:26.022502 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:55:26.022469 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-wh2jx" Jul 06 18:55:26.022993 ip-10-0-130-59 kubenswrapper[2570]: E0706 18:55:26.022628 2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-wh2jx" podUID="9262d667-2ed4-45aa-98f5-2d8158a66ca1" Jul 06 18:55:26.023060 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:55:26.023011 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-ln98s" Jul 06 18:55:26.024698 ip-10-0-130-59 kubenswrapper[2570]: E0706 18:55:26.023101 2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-ln98s" podUID="4beea8e8-7750-4a7a-bc49-249b3d7e3ab3" Jul 06 18:55:26.089189 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:55:26.089116 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-59.ec2.internal" event={"ID":"f20193a4bb56b9075be1ec3e656dd631","Type":"ContainerStarted","Data":"4371cd08cd54828feb5e3a144602cc580eb976e7dde7d07c0d916d49d29cd991"} Jul 06 18:55:26.485348 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:55:26.485229 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/9262d667-2ed4-45aa-98f5-2d8158a66ca1-metrics-certs\") pod \"network-metrics-daemon-wh2jx\" (UID: \"9262d667-2ed4-45aa-98f5-2d8158a66ca1\") " pod="openshift-multus/network-metrics-daemon-wh2jx" Jul 06 18:55:26.485536 ip-10-0-130-59 kubenswrapper[2570]: E0706 18:55:26.485381 2570 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 06 18:55:26.485536 ip-10-0-130-59 kubenswrapper[2570]: E0706 18:55:26.485463 2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9262d667-2ed4-45aa-98f5-2d8158a66ca1-metrics-certs podName:9262d667-2ed4-45aa-98f5-2d8158a66ca1 nodeName:}" failed. No retries permitted until 2026-07-06 18:55:30.485441444 +0000 UTC m=+10.040462607 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/9262d667-2ed4-45aa-98f5-2d8158a66ca1-metrics-certs") pod "network-metrics-daemon-wh2jx" (UID: "9262d667-2ed4-45aa-98f5-2d8158a66ca1") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 06 18:55:26.586716 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:55:26.586683 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/4beea8e8-7750-4a7a-bc49-249b3d7e3ab3-original-pull-secret\") pod \"global-pull-secret-syncer-ln98s\" (UID: \"4beea8e8-7750-4a7a-bc49-249b3d7e3ab3\") " pod="kube-system/global-pull-secret-syncer-ln98s" Jul 06 18:55:26.586881 ip-10-0-130-59 kubenswrapper[2570]: E0706 18:55:26.586830 2570 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 06 18:55:26.586935 ip-10-0-130-59 kubenswrapper[2570]: E0706 18:55:26.586888 2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4beea8e8-7750-4a7a-bc49-249b3d7e3ab3-original-pull-secret podName:4beea8e8-7750-4a7a-bc49-249b3d7e3ab3 nodeName:}" failed. No retries permitted until 2026-07-06 18:55:30.586870829 +0000 UTC m=+10.141892002 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/4beea8e8-7750-4a7a-bc49-249b3d7e3ab3-original-pull-secret") pod "global-pull-secret-syncer-ln98s" (UID: "4beea8e8-7750-4a7a-bc49-249b3d7e3ab3") : object "kube-system"/"original-pull-secret" not registered Jul 06 18:55:26.687495 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:55:26.687457 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qfrb6\" (UniqueName: \"kubernetes.io/projected/f7bc558b-d33b-448d-929c-3652d096c3d5-kube-api-access-qfrb6\") pod \"network-check-target-296mj\" (UID: \"f7bc558b-d33b-448d-929c-3652d096c3d5\") " pod="openshift-network-diagnostics/network-check-target-296mj" Jul 06 18:55:26.687705 ip-10-0-130-59 kubenswrapper[2570]: E0706 18:55:26.687641 2570 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 06 18:55:26.687705 ip-10-0-130-59 kubenswrapper[2570]: E0706 18:55:26.687666 2570 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 06 18:55:26.687705 ip-10-0-130-59 kubenswrapper[2570]: E0706 18:55:26.687680 2570 projected.go:196] Error preparing data for projected volume kube-api-access-qfrb6 for pod openshift-network-diagnostics/network-check-target-296mj: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 06 18:55:26.687887 ip-10-0-130-59 kubenswrapper[2570]: E0706 18:55:26.687738 2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/f7bc558b-d33b-448d-929c-3652d096c3d5-kube-api-access-qfrb6 podName:f7bc558b-d33b-448d-929c-3652d096c3d5 nodeName:}" failed. No retries permitted until 2026-07-06 18:55:30.687719167 +0000 UTC m=+10.242740339 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-qfrb6" (UniqueName: "kubernetes.io/projected/f7bc558b-d33b-448d-929c-3652d096c3d5-kube-api-access-qfrb6") pod "network-check-target-296mj" (UID: "f7bc558b-d33b-448d-929c-3652d096c3d5") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 06 18:55:27.025127 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:55:27.025095 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-296mj" Jul 06 18:55:27.025586 ip-10-0-130-59 kubenswrapper[2570]: E0706 18:55:27.025226 2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-296mj" podUID="f7bc558b-d33b-448d-929c-3652d096c3d5" Jul 06 18:55:28.023139 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:55:28.022890 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-wh2jx" Jul 06 18:55:28.023139 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:55:28.022938 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-ln98s" Jul 06 18:55:28.023139 ip-10-0-130-59 kubenswrapper[2570]: E0706 18:55:28.023040 2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-wh2jx" podUID="9262d667-2ed4-45aa-98f5-2d8158a66ca1" Jul 06 18:55:28.023506 ip-10-0-130-59 kubenswrapper[2570]: E0706 18:55:28.023446 2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-ln98s" podUID="4beea8e8-7750-4a7a-bc49-249b3d7e3ab3" Jul 06 18:55:29.022706 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:55:29.022675 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-296mj" Jul 06 18:55:29.023156 ip-10-0-130-59 kubenswrapper[2570]: E0706 18:55:29.022807 2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-296mj" podUID="f7bc558b-d33b-448d-929c-3652d096c3d5" Jul 06 18:55:30.022719 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:55:30.022688 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-wh2jx" Jul 06 18:55:30.023200 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:55:30.022688 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-ln98s" Jul 06 18:55:30.023200 ip-10-0-130-59 kubenswrapper[2570]: E0706 18:55:30.022834 2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-wh2jx" podUID="9262d667-2ed4-45aa-98f5-2d8158a66ca1" Jul 06 18:55:30.023200 ip-10-0-130-59 kubenswrapper[2570]: E0706 18:55:30.022928 2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-ln98s" podUID="4beea8e8-7750-4a7a-bc49-249b3d7e3ab3" Jul 06 18:55:30.517770 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:55:30.517656 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/9262d667-2ed4-45aa-98f5-2d8158a66ca1-metrics-certs\") pod \"network-metrics-daemon-wh2jx\" (UID: \"9262d667-2ed4-45aa-98f5-2d8158a66ca1\") " pod="openshift-multus/network-metrics-daemon-wh2jx" Jul 06 18:55:30.517972 ip-10-0-130-59 kubenswrapper[2570]: E0706 18:55:30.517827 2570 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 06 18:55:30.517972 ip-10-0-130-59 kubenswrapper[2570]: E0706 18:55:30.517887 2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9262d667-2ed4-45aa-98f5-2d8158a66ca1-metrics-certs podName:9262d667-2ed4-45aa-98f5-2d8158a66ca1 nodeName:}" failed. No retries permitted until 2026-07-06 18:55:38.517868404 +0000 UTC m=+18.072889567 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/9262d667-2ed4-45aa-98f5-2d8158a66ca1-metrics-certs") pod "network-metrics-daemon-wh2jx" (UID: "9262d667-2ed4-45aa-98f5-2d8158a66ca1") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 06 18:55:30.618300 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:55:30.618267 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/4beea8e8-7750-4a7a-bc49-249b3d7e3ab3-original-pull-secret\") pod \"global-pull-secret-syncer-ln98s\" (UID: \"4beea8e8-7750-4a7a-bc49-249b3d7e3ab3\") " pod="kube-system/global-pull-secret-syncer-ln98s" Jul 06 18:55:30.618464 ip-10-0-130-59 kubenswrapper[2570]: E0706 18:55:30.618390 2570 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 06 18:55:30.618464 ip-10-0-130-59 kubenswrapper[2570]: E0706 18:55:30.618455 2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4beea8e8-7750-4a7a-bc49-249b3d7e3ab3-original-pull-secret podName:4beea8e8-7750-4a7a-bc49-249b3d7e3ab3 nodeName:}" failed. No retries permitted until 2026-07-06 18:55:38.618437843 +0000 UTC m=+18.173459029 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/4beea8e8-7750-4a7a-bc49-249b3d7e3ab3-original-pull-secret") pod "global-pull-secret-syncer-ln98s" (UID: "4beea8e8-7750-4a7a-bc49-249b3d7e3ab3") : object "kube-system"/"original-pull-secret" not registered Jul 06 18:55:30.718826 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:55:30.718791 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qfrb6\" (UniqueName: \"kubernetes.io/projected/f7bc558b-d33b-448d-929c-3652d096c3d5-kube-api-access-qfrb6\") pod \"network-check-target-296mj\" (UID: \"f7bc558b-d33b-448d-929c-3652d096c3d5\") " pod="openshift-network-diagnostics/network-check-target-296mj" Jul 06 18:55:30.718999 ip-10-0-130-59 kubenswrapper[2570]: E0706 18:55:30.718972 2570 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 06 18:55:30.718999 ip-10-0-130-59 kubenswrapper[2570]: E0706 18:55:30.718995 2570 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 06 18:55:30.719104 ip-10-0-130-59 kubenswrapper[2570]: E0706 18:55:30.719008 2570 projected.go:196] Error preparing data for projected volume kube-api-access-qfrb6 for pod openshift-network-diagnostics/network-check-target-296mj: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 06 18:55:30.719104 ip-10-0-130-59 kubenswrapper[2570]: E0706 18:55:30.719074 2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/f7bc558b-d33b-448d-929c-3652d096c3d5-kube-api-access-qfrb6 podName:f7bc558b-d33b-448d-929c-3652d096c3d5 nodeName:}" failed. No retries permitted until 2026-07-06 18:55:38.719054949 +0000 UTC m=+18.274076125 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-qfrb6" (UniqueName: "kubernetes.io/projected/f7bc558b-d33b-448d-929c-3652d096c3d5-kube-api-access-qfrb6") pod "network-check-target-296mj" (UID: "f7bc558b-d33b-448d-929c-3652d096c3d5") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 06 18:55:31.022550 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:55:31.022516 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-296mj" Jul 06 18:55:31.022758 ip-10-0-130-59 kubenswrapper[2570]: E0706 18:55:31.022632 2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-296mj" podUID="f7bc558b-d33b-448d-929c-3652d096c3d5" Jul 06 18:55:32.022758 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:55:32.022673 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-ln98s" Jul 06 18:55:32.023207 ip-10-0-130-59 kubenswrapper[2570]: E0706 18:55:32.022808 2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-ln98s" podUID="4beea8e8-7750-4a7a-bc49-249b3d7e3ab3" Jul 06 18:55:32.023207 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:55:32.022905 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-wh2jx" Jul 06 18:55:32.023207 ip-10-0-130-59 kubenswrapper[2570]: E0706 18:55:32.023002 2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-wh2jx" podUID="9262d667-2ed4-45aa-98f5-2d8158a66ca1" Jul 06 18:55:33.022293 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:55:33.022256 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-296mj" Jul 06 18:55:33.022478 ip-10-0-130-59 kubenswrapper[2570]: E0706 18:55:33.022392 2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-296mj" podUID="f7bc558b-d33b-448d-929c-3652d096c3d5" Jul 06 18:55:34.022513 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:55:34.022482 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-ln98s" Jul 06 18:55:34.022980 ip-10-0-130-59 kubenswrapper[2570]: E0706 18:55:34.022631 2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-ln98s" podUID="4beea8e8-7750-4a7a-bc49-249b3d7e3ab3" Jul 06 18:55:34.022980 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:55:34.022645 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-wh2jx" Jul 06 18:55:34.022980 ip-10-0-130-59 kubenswrapper[2570]: E0706 18:55:34.022797 2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-wh2jx" podUID="9262d667-2ed4-45aa-98f5-2d8158a66ca1" Jul 06 18:55:35.022891 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:55:35.022853 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-296mj" Jul 06 18:55:35.023351 ip-10-0-130-59 kubenswrapper[2570]: E0706 18:55:35.022985 2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-296mj" podUID="f7bc558b-d33b-448d-929c-3652d096c3d5" Jul 06 18:55:36.022775 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:55:36.022745 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-ln98s" Jul 06 18:55:36.022953 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:55:36.022745 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-wh2jx" Jul 06 18:55:36.022953 ip-10-0-130-59 kubenswrapper[2570]: E0706 18:55:36.022866 2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-ln98s" podUID="4beea8e8-7750-4a7a-bc49-249b3d7e3ab3" Jul 06 18:55:36.023273 ip-10-0-130-59 kubenswrapper[2570]: E0706 18:55:36.022977 2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-wh2jx" podUID="9262d667-2ed4-45aa-98f5-2d8158a66ca1" Jul 06 18:55:37.022233 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:55:37.022194 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-296mj" Jul 06 18:55:37.022415 ip-10-0-130-59 kubenswrapper[2570]: E0706 18:55:37.022311 2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-296mj" podUID="f7bc558b-d33b-448d-929c-3652d096c3d5" Jul 06 18:55:38.022448 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:55:38.022415 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-wh2jx" Jul 06 18:55:38.022448 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:55:38.022443 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-ln98s" Jul 06 18:55:38.023029 ip-10-0-130-59 kubenswrapper[2570]: E0706 18:55:38.022529 2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-wh2jx" podUID="9262d667-2ed4-45aa-98f5-2d8158a66ca1" Jul 06 18:55:38.023029 ip-10-0-130-59 kubenswrapper[2570]: E0706 18:55:38.022674 2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-ln98s" podUID="4beea8e8-7750-4a7a-bc49-249b3d7e3ab3" Jul 06 18:55:38.576929 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:55:38.576847 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/9262d667-2ed4-45aa-98f5-2d8158a66ca1-metrics-certs\") pod \"network-metrics-daemon-wh2jx\" (UID: \"9262d667-2ed4-45aa-98f5-2d8158a66ca1\") " pod="openshift-multus/network-metrics-daemon-wh2jx" Jul 06 18:55:38.577122 ip-10-0-130-59 kubenswrapper[2570]: E0706 18:55:38.577042 2570 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 06 18:55:38.577122 ip-10-0-130-59 kubenswrapper[2570]: E0706 18:55:38.577120 2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9262d667-2ed4-45aa-98f5-2d8158a66ca1-metrics-certs podName:9262d667-2ed4-45aa-98f5-2d8158a66ca1 nodeName:}" failed. No retries permitted until 2026-07-06 18:55:54.5770995 +0000 UTC m=+34.132120652 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/9262d667-2ed4-45aa-98f5-2d8158a66ca1-metrics-certs") pod "network-metrics-daemon-wh2jx" (UID: "9262d667-2ed4-45aa-98f5-2d8158a66ca1") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 06 18:55:38.677657 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:55:38.677624 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/4beea8e8-7750-4a7a-bc49-249b3d7e3ab3-original-pull-secret\") pod \"global-pull-secret-syncer-ln98s\" (UID: \"4beea8e8-7750-4a7a-bc49-249b3d7e3ab3\") " pod="kube-system/global-pull-secret-syncer-ln98s" Jul 06 18:55:38.677813 ip-10-0-130-59 kubenswrapper[2570]: E0706 18:55:38.677798 2570 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 06 18:55:38.677878 ip-10-0-130-59 kubenswrapper[2570]: E0706 18:55:38.677866 2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4beea8e8-7750-4a7a-bc49-249b3d7e3ab3-original-pull-secret podName:4beea8e8-7750-4a7a-bc49-249b3d7e3ab3 nodeName:}" failed. No retries permitted until 2026-07-06 18:55:54.67784969 +0000 UTC m=+34.232870843 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/4beea8e8-7750-4a7a-bc49-249b3d7e3ab3-original-pull-secret") pod "global-pull-secret-syncer-ln98s" (UID: "4beea8e8-7750-4a7a-bc49-249b3d7e3ab3") : object "kube-system"/"original-pull-secret" not registered Jul 06 18:55:38.778222 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:55:38.778185 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qfrb6\" (UniqueName: \"kubernetes.io/projected/f7bc558b-d33b-448d-929c-3652d096c3d5-kube-api-access-qfrb6\") pod \"network-check-target-296mj\" (UID: \"f7bc558b-d33b-448d-929c-3652d096c3d5\") " pod="openshift-network-diagnostics/network-check-target-296mj" Jul 06 18:55:38.778394 ip-10-0-130-59 kubenswrapper[2570]: E0706 18:55:38.778357 2570 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 06 18:55:38.778394 ip-10-0-130-59 kubenswrapper[2570]: E0706 18:55:38.778383 2570 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 06 18:55:38.778394 ip-10-0-130-59 kubenswrapper[2570]: E0706 18:55:38.778396 2570 projected.go:196] Error preparing data for projected volume kube-api-access-qfrb6 for pod openshift-network-diagnostics/network-check-target-296mj: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 06 18:55:38.778540 ip-10-0-130-59 kubenswrapper[2570]: E0706 18:55:38.778461 2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/f7bc558b-d33b-448d-929c-3652d096c3d5-kube-api-access-qfrb6 podName:f7bc558b-d33b-448d-929c-3652d096c3d5 nodeName:}" failed. No retries permitted until 2026-07-06 18:55:54.77844666 +0000 UTC m=+34.333467830 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-qfrb6" (UniqueName: "kubernetes.io/projected/f7bc558b-d33b-448d-929c-3652d096c3d5-kube-api-access-qfrb6") pod "network-check-target-296mj" (UID: "f7bc558b-d33b-448d-929c-3652d096c3d5") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 06 18:55:39.022321 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:55:39.022285 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-296mj" Jul 06 18:55:39.022505 ip-10-0-130-59 kubenswrapper[2570]: E0706 18:55:39.022437 2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-296mj" podUID="f7bc558b-d33b-448d-929c-3652d096c3d5" Jul 06 18:55:40.021992 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:55:40.021905 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-ln98s" Jul 06 18:55:40.022194 ip-10-0-130-59 kubenswrapper[2570]: E0706 18:55:40.022035 2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-ln98s" podUID="4beea8e8-7750-4a7a-bc49-249b3d7e3ab3" Jul 06 18:55:40.022194 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:55:40.022091 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-wh2jx" Jul 06 18:55:40.022313 ip-10-0-130-59 kubenswrapper[2570]: E0706 18:55:40.022210 2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-wh2jx" podUID="9262d667-2ed4-45aa-98f5-2d8158a66ca1" Jul 06 18:55:41.022966 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:55:41.022917 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-296mj" Jul 06 18:55:41.023390 ip-10-0-130-59 kubenswrapper[2570]: E0706 18:55:41.023023 2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-296mj" podUID="f7bc558b-d33b-448d-929c-3652d096c3d5" Jul 06 18:55:42.022536 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:55:42.022504 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-ln98s" Jul 06 18:55:42.022536 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:55:42.022535 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-wh2jx" Jul 06 18:55:42.022806 ip-10-0-130-59 kubenswrapper[2570]: E0706 18:55:42.022645 2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-ln98s" podUID="4beea8e8-7750-4a7a-bc49-249b3d7e3ab3" Jul 06 18:55:42.022806 ip-10-0-130-59 kubenswrapper[2570]: E0706 18:55:42.022696 2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-wh2jx" podUID="9262d667-2ed4-45aa-98f5-2d8158a66ca1" Jul 06 18:55:43.022738 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:55:43.022206 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-296mj" Jul 06 18:55:43.022738 ip-10-0-130-59 kubenswrapper[2570]: E0706 18:55:43.022400 2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-296mj" podUID="f7bc558b-d33b-448d-929c-3652d096c3d5" Jul 06 18:55:43.121850 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:55:43.121641 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-rr76d" event={"ID":"ac420ddc-b839-4613-943d-50d18a911d56","Type":"ContainerStarted","Data":"2509780d467935226916c9eaec0baa79e9b0b7794123af43ea7637721f600ab2"} Jul 06 18:55:43.121964 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:55:43.121908 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-rr76d" event={"ID":"ac420ddc-b839-4613-943d-50d18a911d56","Type":"ContainerStarted","Data":"40aa8b27c59259af126bd227a17feb129b81c12f886a7eee77dfc648e8940a27"} Jul 06 18:55:43.123022 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:55:43.122999 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-k8fbq" event={"ID":"bc7057bd-88da-48af-ba32-e6efdc6e8362","Type":"ContainerStarted","Data":"f86597e3595d14300b5d2a4936ba0d6c83611fe7cd3302ef803ef87fa9aeb427"} Jul 06 18:55:43.124539 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:55:43.124520 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-k5knh" event={"ID":"b9b9a09a-8299-41aa-9b59-51ed106f6802","Type":"ContainerStarted","Data":"c80fc8e2ecd6e427dd82e5ad23e6e66a52fda98e3392e94d4b41d7ae3c110a6d"} Jul 06 18:55:43.125933 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:55:43.125910 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-4qk4s" event={"ID":"a88da320-d446-42e3-b155-418a20f35893","Type":"ContainerStarted","Data":"20d22234c23016a45c77a5f29ed3de109cbcf9b98185e8b76929c398da12f06c"} Jul 06 18:55:43.127491 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:55:43.127470 2570 generic.go:356] "Generic (PLEG): container finished" podID="4f1bcb6b-7c73-4bff-a246-78ddc5ca22ea" containerID="fd679e94187b8b0af15d09c9b8ae04ec6b2db9b7e893503add3322cdec8d2b2c" exitCode=0 Jul 06 18:55:43.127591 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:55:43.127531 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-wmxvz" event={"ID":"4f1bcb6b-7c73-4bff-a246-78ddc5ca22ea","Type":"ContainerDied","Data":"fd679e94187b8b0af15d09c9b8ae04ec6b2db9b7e893503add3322cdec8d2b2c"} Jul 06 18:55:43.129181 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:55:43.129154 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-sxf4k" event={"ID":"da433510-33a8-4e5e-9a3d-29d8d0ec0502","Type":"ContainerStarted","Data":"df9e5590ffab1be27d99109ef56c3e41e5e48d26cd68cab17bf6562fc6c5f69e"} Jul 06 18:55:43.130655 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:55:43.130636 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-b66m9" event={"ID":"0b813b17-4a1b-4efa-aaa2-2c5e926196e3","Type":"ContainerStarted","Data":"531e728736610cfce4729736d9cfacaf6b34cad54957d530d58cd94395087110"} Jul 06 18:55:43.132106 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:55:43.132087 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-fvvmf" event={"ID":"92e4be75-799a-44f0-aaca-22ef4bca49fc","Type":"ContainerStarted","Data":"d392f43ca63b071ba7e6c908e9d1b1719349523387a01f1c03a35e7621b2f8ec"} Jul 06 18:55:43.142939 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:55:43.142884 2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-59.ec2.internal" podStartSLOduration=21.142871137 podStartE2EDuration="21.142871137s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-06 18:55:22 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 18:55:26.105667389 +0000 UTC m=+5.660688582" watchObservedRunningTime="2026-07-06 18:55:43.142871137 +0000 UTC m=+22.697892308" Jul 06 18:55:43.165017 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:55:43.164973 2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/konnectivity-agent-k8fbq" podStartSLOduration=7.905961469 podStartE2EDuration="22.164957486s" totalImagesPullingTime="14.258996017s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 18:55:21 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 18:55:43.143534852 +0000 UTC m=+22.698556023" watchObservedRunningTime="2026-07-06 18:55:43.164957486 +0000 UTC m=+22.719978652" Jul 06 18:55:43.165367 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:55:43.165338 2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-fvvmf" podStartSLOduration=3.402575525 podStartE2EDuration="22.165327523s" totalImagesPullingTime="18.762751998s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-06 18:55:21 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 18:55:43.164897149 +0000 UTC m=+22.719918319" watchObservedRunningTime="2026-07-06 18:55:43.165327523 +0000 UTC m=+22.720348698" Jul 06 18:55:43.193405 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:55:43.193370 2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-b66m9" podStartSLOduration=3.372778772 podStartE2EDuration="22.193358053s" totalImagesPullingTime="18.820579281s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-06 18:55:21 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 18:55:43.191072008 +0000 UTC m=+22.746093180" watchObservedRunningTime="2026-07-06 18:55:43.193358053 +0000 UTC m=+22.748379226" Jul 06 18:55:43.224477 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:55:43.224432 2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/node-ca-sxf4k" podStartSLOduration=3.466381449 podStartE2EDuration="22.22440798s" totalImagesPullingTime="18.758026531s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-06 18:55:21 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 18:55:43.207940246 +0000 UTC m=+22.762961418" watchObservedRunningTime="2026-07-06 18:55:43.22440798 +0000 UTC m=+22.779429204" Jul 06 18:55:43.249183 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:55:43.249150 2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/node-resolver-4qk4s" podStartSLOduration=3.517396754 podStartE2EDuration="22.249137957s" totalImagesPullingTime="18.731741203s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-06 18:55:21 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 18:55:43.224556593 +0000 UTC m=+22.779577766" watchObservedRunningTime="2026-07-06 18:55:43.249137957 +0000 UTC m=+22.804159128" Jul 06 18:55:43.547584 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:55:43.547558 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k8fbq_bc7057bd-88da-48af-ba32-e6efdc6e8362/konnectivity-agent/0.log" limit=1024 Jul 06 18:55:43.648932 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:55:43.648899 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k8fbq_bc7057bd-88da-48af-ba32-e6efdc6e8362/konnectivity-agent/0.log" limit=1024 Jul 06 18:55:43.844592 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:55:43.844567 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k8fbq_bc7057bd-88da-48af-ba32-e6efdc6e8362/konnectivity-agent/0.log" limit=1024 Jul 06 18:55:44.022470 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:55:44.022443 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-wh2jx" Jul 06 18:55:44.022567 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:55:44.022478 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-ln98s" Jul 06 18:55:44.022621 ip-10-0-130-59 kubenswrapper[2570]: E0706 18:55:44.022578 2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-wh2jx" podUID="9262d667-2ed4-45aa-98f5-2d8158a66ca1" Jul 06 18:55:44.022737 ip-10-0-130-59 kubenswrapper[2570]: E0706 18:55:44.022695 2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-ln98s" podUID="4beea8e8-7750-4a7a-bc49-249b3d7e3ab3" Jul 06 18:55:44.134535 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:55:44.134515 2570 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock" Jul 06 18:55:44.136349 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:55:44.136324 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-qnhx2" event={"ID":"17c6ff8a-2c68-400e-b46d-7f73f3673088","Type":"ContainerStarted","Data":"d92fac321207cc17188660fc491e0f8493d2021fb3cf2fbc95486d6862a54b2b"} Jul 06 18:55:44.138195 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:55:44.138171 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-k5knh" event={"ID":"b9b9a09a-8299-41aa-9b59-51ed106f6802","Type":"ContainerStarted","Data":"05853e4e49f583b20889982ca139264507d3def7b6d0a7b7ada0b25633112df6"} Jul 06 18:55:44.140967 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:55:44.140943 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-rr76d" event={"ID":"ac420ddc-b839-4613-943d-50d18a911d56","Type":"ContainerStarted","Data":"286074d668b3e1963639938fcbbeae77d695e83e1b5bb54c80684927c52f2cf6"} Jul 06 18:55:44.141047 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:55:44.141010 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-rr76d" event={"ID":"ac420ddc-b839-4613-943d-50d18a911d56","Type":"ContainerStarted","Data":"b34f3f9176f580f1166867fa70c0361568a85f20a7ff07a920416cd6ae5595d8"} Jul 06 18:55:44.141047 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:55:44.141043 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-rr76d" event={"ID":"ac420ddc-b839-4613-943d-50d18a911d56","Type":"ContainerStarted","Data":"9157530ee4468b2bc6c8653d54c8c1a8fd33f89e61eec8a8b5ff5848b67b8094"} Jul 06 18:55:44.141129 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:55:44.141081 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-rr76d" event={"ID":"ac420ddc-b839-4613-943d-50d18a911d56","Type":"ContainerStarted","Data":"79bee7d2a5a96540f16adcc21b9b262163c255bee1828b58aeacf8ba00a74832"} Jul 06 18:55:44.781415 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:55:44.781268 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k8fbq_bc7057bd-88da-48af-ba32-e6efdc6e8362/konnectivity-agent/0.log" limit=1024 Jul 06 18:55:44.908776 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:55:44.908661 2570 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-07-06T18:55:44.134531098Z","UUID":"098faae2-9837-4299-8e9f-7941473574b3","Handler":null,"Name":"","Endpoint":""} Jul 06 18:55:44.911121 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:55:44.911101 2570 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0 Jul 06 18:55:44.911242 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:55:44.911135 2570 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock Jul 06 18:55:44.921556 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:55:44.921528 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k8fbq_bc7057bd-88da-48af-ba32-e6efdc6e8362/konnectivity-agent/0.log" limit=1024 Jul 06 18:55:45.022747 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:55:45.022716 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-296mj" Jul 06 18:55:45.023260 ip-10-0-130-59 kubenswrapper[2570]: E0706 18:55:45.022837 2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-296mj" podUID="f7bc558b-d33b-448d-929c-3652d096c3d5" Jul 06 18:55:45.025573 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:55:45.025546 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k8fbq_bc7057bd-88da-48af-ba32-e6efdc6e8362/konnectivity-agent/0.log" limit=1024 Jul 06 18:55:45.161970 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:55:45.161885 2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-qnhx2" podStartSLOduration=5.427762061 podStartE2EDuration="24.161866077s" totalImagesPullingTime="18.734104016s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-06 18:55:21 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 18:55:45.161756976 +0000 UTC m=+24.716778148" watchObservedRunningTime="2026-07-06 18:55:45.161866077 +0000 UTC m=+24.716887250" Jul 06 18:55:45.233889 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:55:45.233856 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k8fbq_bc7057bd-88da-48af-ba32-e6efdc6e8362/konnectivity-agent/0.log" limit=1024 Jul 06 18:55:45.424692 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:55:45.424664 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k8fbq_bc7057bd-88da-48af-ba32-e6efdc6e8362/konnectivity-agent/0.log" limit=1024 Jul 06 18:55:45.527459 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:55:45.527409 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k8fbq_bc7057bd-88da-48af-ba32-e6efdc6e8362/konnectivity-agent/0.log" limit=1024 Jul 06 18:55:45.625966 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:55:45.625941 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k8fbq_bc7057bd-88da-48af-ba32-e6efdc6e8362/konnectivity-agent/0.log" limit=1024 Jul 06 18:55:46.006006 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:55:46.005974 2570 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-k8fbq" Jul 06 18:55:46.006580 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:55:46.006559 2570 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-k8fbq" Jul 06 18:55:46.022543 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:55:46.022522 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-wh2jx" Jul 06 18:55:46.022543 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:55:46.022536 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-ln98s" Jul 06 18:55:46.022709 ip-10-0-130-59 kubenswrapper[2570]: E0706 18:55:46.022672 2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-wh2jx" podUID="9262d667-2ed4-45aa-98f5-2d8158a66ca1" Jul 06 18:55:46.022825 ip-10-0-130-59 kubenswrapper[2570]: E0706 18:55:46.022802 2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-ln98s" podUID="4beea8e8-7750-4a7a-bc49-249b3d7e3ab3" Jul 06 18:55:46.146498 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:55:46.146460 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-k5knh" event={"ID":"b9b9a09a-8299-41aa-9b59-51ed106f6802","Type":"ContainerStarted","Data":"5ea5b897565b4a79932a3b061075ee5395cd91af61ebf7093bf470b641dd7a60"} Jul 06 18:55:46.149505 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:55:46.149481 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-rr76d" event={"ID":"ac420ddc-b839-4613-943d-50d18a911d56","Type":"ContainerStarted","Data":"61e547f51850615eea9cd0e53b9c391850d72c912d6a18f7dfdd1fa8ac4806fc"} Jul 06 18:55:46.149637 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:55:46.149547 2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-k8fbq" Jul 06 18:55:46.150000 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:55:46.149983 2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-k8fbq" Jul 06 18:55:46.169347 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:55:46.169306 2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-k5knh" podStartSLOduration=4.262309481 podStartE2EDuration="25.169293637s" totalImagesPullingTime="20.906984156s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-06 18:55:21 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 18:55:46.167955188 +0000 UTC m=+25.722976360" watchObservedRunningTime="2026-07-06 18:55:46.169293637 +0000 UTC m=+25.724314842" Jul 06 18:55:46.215904 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:55:46.215884 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k8fbq_bc7057bd-88da-48af-ba32-e6efdc6e8362/konnectivity-agent/0.log" limit=1024 Jul 06 18:55:46.317047 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:55:46.316978 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k8fbq_bc7057bd-88da-48af-ba32-e6efdc6e8362/konnectivity-agent/0.log" limit=1024 Jul 06 18:55:46.835426 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:55:46.835397 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k8fbq_bc7057bd-88da-48af-ba32-e6efdc6e8362/konnectivity-agent/0.log" limit=1024 Jul 06 18:55:47.021920 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:55:47.021884 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-296mj" Jul 06 18:55:47.022097 ip-10-0-130-59 kubenswrapper[2570]: E0706 18:55:47.021999 2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-296mj" podUID="f7bc558b-d33b-448d-929c-3652d096c3d5" Jul 06 18:55:47.041004 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:55:47.040975 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k8fbq_bc7057bd-88da-48af-ba32-e6efdc6e8362/konnectivity-agent/0.log" limit=1024 Jul 06 18:55:47.243712 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:55:47.243681 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k8fbq_bc7057bd-88da-48af-ba32-e6efdc6e8362/konnectivity-agent/0.log" limit=1024 Jul 06 18:55:47.457097 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:55:47.457070 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k8fbq_bc7057bd-88da-48af-ba32-e6efdc6e8362/konnectivity-agent/0.log" limit=1024 Jul 06 18:55:48.022405 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:55:48.022174 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-ln98s" Jul 06 18:55:48.022579 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:55:48.022225 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-wh2jx" Jul 06 18:55:48.022579 ip-10-0-130-59 kubenswrapper[2570]: E0706 18:55:48.022508 2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-ln98s" podUID="4beea8e8-7750-4a7a-bc49-249b3d7e3ab3" Jul 06 18:55:48.022715 ip-10-0-130-59 kubenswrapper[2570]: E0706 18:55:48.022638 2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-wh2jx" podUID="9262d667-2ed4-45aa-98f5-2d8158a66ca1" Jul 06 18:55:48.157051 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:55:48.157017 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-rr76d" event={"ID":"ac420ddc-b839-4613-943d-50d18a911d56","Type":"ContainerStarted","Data":"73e95fd8badcc429a1da7b7178d1b649fb1ee88032349ed6b0a19784f5020475"} Jul 06 18:55:48.186462 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:55:48.186417 2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-rr76d" podStartSLOduration=8.392453362 podStartE2EDuration="27.186402465s" totalImagesPullingTime="18.793949103s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-06 18:55:21 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 18:55:48.185947302 +0000 UTC m=+27.740968475" watchObservedRunningTime="2026-07-06 18:55:48.186402465 +0000 UTC m=+27.741423637" Jul 06 18:55:48.432127 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:55:48.432090 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k8fbq_bc7057bd-88da-48af-ba32-e6efdc6e8362/konnectivity-agent/0.log" limit=1024 Jul 06 18:55:48.630422 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:55:48.630336 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k8fbq_bc7057bd-88da-48af-ba32-e6efdc6e8362/konnectivity-agent/0.log" limit=1024 Jul 06 18:55:48.730997 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:55:48.730964 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k8fbq_bc7057bd-88da-48af-ba32-e6efdc6e8362/konnectivity-agent/0.log" limit=1024 Jul 06 18:55:48.830042 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:55:48.830007 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k8fbq_bc7057bd-88da-48af-ba32-e6efdc6e8362/konnectivity-agent/0.log" limit=1024 Jul 06 18:55:49.022739 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:55:49.022707 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-296mj" Jul 06 18:55:49.022901 ip-10-0-130-59 kubenswrapper[2570]: E0706 18:55:49.022846 2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-296mj" podUID="f7bc558b-d33b-448d-929c-3652d096c3d5" Jul 06 18:55:49.037593 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:55:49.037553 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k8fbq_bc7057bd-88da-48af-ba32-e6efdc6e8362/konnectivity-agent/0.log" limit=1024 Jul 06 18:55:49.160139 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:55:49.160111 2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-rr76d" Jul 06 18:55:49.160721 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:55:49.160195 2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-rr76d" Jul 06 18:55:49.160721 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:55:49.160237 2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-rr76d" Jul 06 18:55:49.177171 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:55:49.177145 2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-rr76d" Jul 06 18:55:49.177312 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:55:49.177252 2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-rr76d" Jul 06 18:55:49.237399 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:55:49.237366 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k8fbq_bc7057bd-88da-48af-ba32-e6efdc6e8362/konnectivity-agent/0.log" limit=1024 Jul 06 18:55:49.440492 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:55:49.440288 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k8fbq_bc7057bd-88da-48af-ba32-e6efdc6e8362/konnectivity-agent/0.log" limit=1024 Jul 06 18:55:49.540575 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:55:49.540546 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k8fbq_bc7057bd-88da-48af-ba32-e6efdc6e8362/konnectivity-agent/0.log" limit=1024 Jul 06 18:55:50.022255 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:55:50.022230 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-wh2jx" Jul 06 18:55:50.022387 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:55:50.022230 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-ln98s" Jul 06 18:55:50.022387 ip-10-0-130-59 kubenswrapper[2570]: E0706 18:55:50.022331 2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-wh2jx" podUID="9262d667-2ed4-45aa-98f5-2d8158a66ca1" Jul 06 18:55:50.022502 ip-10-0-130-59 kubenswrapper[2570]: E0706 18:55:50.022384 2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-ln98s" podUID="4beea8e8-7750-4a7a-bc49-249b3d7e3ab3" Jul 06 18:55:50.450492 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:55:50.450463 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k8fbq_bc7057bd-88da-48af-ba32-e6efdc6e8362/konnectivity-agent/0.log" limit=1024 Jul 06 18:55:50.832517 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:55:50.832477 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k8fbq_bc7057bd-88da-48af-ba32-e6efdc6e8362/konnectivity-agent/0.log" limit=1024 Jul 06 18:55:50.954527 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:55:50.954496 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k8fbq_bc7057bd-88da-48af-ba32-e6efdc6e8362/konnectivity-agent/0.log" limit=1024 Jul 06 18:55:51.023149 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:55:51.023126 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-296mj" Jul 06 18:55:51.023270 ip-10-0-130-59 kubenswrapper[2570]: E0706 18:55:51.023196 2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-296mj" podUID="f7bc558b-d33b-448d-929c-3652d096c3d5" Jul 06 18:55:51.055058 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:55:51.055033 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k8fbq_bc7057bd-88da-48af-ba32-e6efdc6e8362/konnectivity-agent/0.log" limit=1024 Jul 06 18:55:51.165245 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:55:51.165189 2570 generic.go:356] "Generic (PLEG): container finished" podID="4f1bcb6b-7c73-4bff-a246-78ddc5ca22ea" containerID="466c66685ece9f44b84b8bf2f003e3a4f69d9f500f25f0a653ecf10a83b38046" exitCode=0 Jul 06 18:55:51.165360 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:55:51.165279 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-wmxvz" event={"ID":"4f1bcb6b-7c73-4bff-a246-78ddc5ca22ea","Type":"ContainerDied","Data":"466c66685ece9f44b84b8bf2f003e3a4f69d9f500f25f0a653ecf10a83b38046"} Jul 06 18:55:51.237743 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:55:51.237723 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k8fbq_bc7057bd-88da-48af-ba32-e6efdc6e8362/konnectivity-agent/0.log" limit=1024 Jul 06 18:55:51.645906 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:55:51.645872 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k8fbq_bc7057bd-88da-48af-ba32-e6efdc6e8362/konnectivity-agent/0.log" limit=1024 Jul 06 18:55:52.022475 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:55:52.022450 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-ln98s" Jul 06 18:55:52.022653 ip-10-0-130-59 kubenswrapper[2570]: E0706 18:55:52.022555 2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-ln98s" podUID="4beea8e8-7750-4a7a-bc49-249b3d7e3ab3" Jul 06 18:55:52.022653 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:55:52.022624 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-wh2jx" Jul 06 18:55:52.022737 ip-10-0-130-59 kubenswrapper[2570]: E0706 18:55:52.022723 2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-wh2jx" podUID="9262d667-2ed4-45aa-98f5-2d8158a66ca1" Jul 06 18:55:52.276250 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:55:52.276194 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k8fbq_bc7057bd-88da-48af-ba32-e6efdc6e8362/konnectivity-agent/0.log" limit=1024 Jul 06 18:55:52.639975 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:55:52.639875 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k8fbq_bc7057bd-88da-48af-ba32-e6efdc6e8362/konnectivity-agent/0.log" limit=1024 Jul 06 18:55:52.826635 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:55:52.826590 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k8fbq_bc7057bd-88da-48af-ba32-e6efdc6e8362/konnectivity-agent/0.log" limit=1024 Jul 06 18:55:53.022811 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:55:53.022786 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-296mj" Jul 06 18:55:53.022950 ip-10-0-130-59 kubenswrapper[2570]: E0706 18:55:53.022899 2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-296mj" podUID="f7bc558b-d33b-448d-929c-3652d096c3d5" Jul 06 18:55:53.035233 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:55:53.035211 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k8fbq_bc7057bd-88da-48af-ba32-e6efdc6e8362/konnectivity-agent/0.log" limit=1024 Jul 06 18:55:53.171087 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:55:53.171060 2570 generic.go:356] "Generic (PLEG): container finished" podID="4f1bcb6b-7c73-4bff-a246-78ddc5ca22ea" containerID="4182a0a3a7a712accb31bd760572800c8b67995613803e8053e09f7228b44c53" exitCode=0 Jul 06 18:55:53.171197 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:55:53.171109 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-wmxvz" event={"ID":"4f1bcb6b-7c73-4bff-a246-78ddc5ca22ea","Type":"ContainerDied","Data":"4182a0a3a7a712accb31bd760572800c8b67995613803e8053e09f7228b44c53"} Jul 06 18:55:53.237122 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:55:53.237098 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k8fbq_bc7057bd-88da-48af-ba32-e6efdc6e8362/konnectivity-agent/0.log" limit=1024 Jul 06 18:55:54.022263 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:55:54.022244 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-wh2jx" Jul 06 18:55:54.022531 ip-10-0-130-59 kubenswrapper[2570]: E0706 18:55:54.022336 2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-wh2jx" podUID="9262d667-2ed4-45aa-98f5-2d8158a66ca1" Jul 06 18:55:54.022531 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:55:54.022395 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-ln98s" Jul 06 18:55:54.022531 ip-10-0-130-59 kubenswrapper[2570]: E0706 18:55:54.022476 2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-ln98s" podUID="4beea8e8-7750-4a7a-bc49-249b3d7e3ab3" Jul 06 18:55:54.036889 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:55:54.036871 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k8fbq_bc7057bd-88da-48af-ba32-e6efdc6e8362/konnectivity-agent/0.log" limit=1024 Jul 06 18:55:54.175572 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:55:54.175546 2570 generic.go:356] "Generic (PLEG): container finished" podID="4f1bcb6b-7c73-4bff-a246-78ddc5ca22ea" containerID="3e9941e22e55b4efbafab092780bd07aded5bfe9faaee18350b06aca5e91bd4a" exitCode=0 Jul 06 18:55:54.175681 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:55:54.175600 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-wmxvz" event={"ID":"4f1bcb6b-7c73-4bff-a246-78ddc5ca22ea","Type":"ContainerDied","Data":"3e9941e22e55b4efbafab092780bd07aded5bfe9faaee18350b06aca5e91bd4a"} Jul 06 18:55:54.447298 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:55:54.447208 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k8fbq_bc7057bd-88da-48af-ba32-e6efdc6e8362/konnectivity-agent/0.log" limit=1024 Jul 06 18:55:54.600279 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:55:54.600252 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/9262d667-2ed4-45aa-98f5-2d8158a66ca1-metrics-certs\") pod \"network-metrics-daemon-wh2jx\" (UID: \"9262d667-2ed4-45aa-98f5-2d8158a66ca1\") " pod="openshift-multus/network-metrics-daemon-wh2jx" Jul 06 18:55:54.600441 ip-10-0-130-59 kubenswrapper[2570]: E0706 18:55:54.600423 2570 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 06 18:55:54.600508 ip-10-0-130-59 kubenswrapper[2570]: E0706 18:55:54.600493 2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9262d667-2ed4-45aa-98f5-2d8158a66ca1-metrics-certs podName:9262d667-2ed4-45aa-98f5-2d8158a66ca1 nodeName:}" failed. No retries permitted until 2026-07-06 18:56:26.600472912 +0000 UTC m=+66.155494063 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/9262d667-2ed4-45aa-98f5-2d8158a66ca1-metrics-certs") pod "network-metrics-daemon-wh2jx" (UID: "9262d667-2ed4-45aa-98f5-2d8158a66ca1") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 06 18:55:54.690924 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:55:54.690891 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k8fbq_bc7057bd-88da-48af-ba32-e6efdc6e8362/konnectivity-agent/0.log" limit=1024 Jul 06 18:55:54.701041 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:55:54.700964 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/4beea8e8-7750-4a7a-bc49-249b3d7e3ab3-original-pull-secret\") pod \"global-pull-secret-syncer-ln98s\" (UID: \"4beea8e8-7750-4a7a-bc49-249b3d7e3ab3\") " pod="kube-system/global-pull-secret-syncer-ln98s" Jul 06 18:55:54.701202 ip-10-0-130-59 kubenswrapper[2570]: E0706 18:55:54.701118 2570 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 06 18:55:54.701202 ip-10-0-130-59 kubenswrapper[2570]: E0706 18:55:54.701180 2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4beea8e8-7750-4a7a-bc49-249b3d7e3ab3-original-pull-secret podName:4beea8e8-7750-4a7a-bc49-249b3d7e3ab3 nodeName:}" failed. No retries permitted until 2026-07-06 18:56:26.701163309 +0000 UTC m=+66.256184459 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/4beea8e8-7750-4a7a-bc49-249b3d7e3ab3-original-pull-secret") pod "global-pull-secret-syncer-ln98s" (UID: "4beea8e8-7750-4a7a-bc49-249b3d7e3ab3") : object "kube-system"/"original-pull-secret" not registered Jul 06 18:55:54.801380 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:55:54.801349 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qfrb6\" (UniqueName: \"kubernetes.io/projected/f7bc558b-d33b-448d-929c-3652d096c3d5-kube-api-access-qfrb6\") pod \"network-check-target-296mj\" (UID: \"f7bc558b-d33b-448d-929c-3652d096c3d5\") " pod="openshift-network-diagnostics/network-check-target-296mj" Jul 06 18:55:54.801512 ip-10-0-130-59 kubenswrapper[2570]: E0706 18:55:54.801482 2570 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 06 18:55:54.801512 ip-10-0-130-59 kubenswrapper[2570]: E0706 18:55:54.801502 2570 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 06 18:55:54.801651 ip-10-0-130-59 kubenswrapper[2570]: E0706 18:55:54.801514 2570 projected.go:196] Error preparing data for projected volume kube-api-access-qfrb6 for pod openshift-network-diagnostics/network-check-target-296mj: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 06 18:55:54.801651 ip-10-0-130-59 kubenswrapper[2570]: E0706 18:55:54.801571 2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/f7bc558b-d33b-448d-929c-3652d096c3d5-kube-api-access-qfrb6 podName:f7bc558b-d33b-448d-929c-3652d096c3d5 nodeName:}" failed. No retries permitted until 2026-07-06 18:56:26.801554855 +0000 UTC m=+66.356576068 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-qfrb6" (UniqueName: "kubernetes.io/projected/f7bc558b-d33b-448d-929c-3652d096c3d5-kube-api-access-qfrb6") pod "network-check-target-296mj" (UID: "f7bc558b-d33b-448d-929c-3652d096c3d5") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 06 18:55:54.829189 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:55:54.829160 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k8fbq_bc7057bd-88da-48af-ba32-e6efdc6e8362/konnectivity-agent/0.log" limit=1024 Jul 06 18:55:55.022574 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:55:55.022515 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-296mj" Jul 06 18:55:55.022905 ip-10-0-130-59 kubenswrapper[2570]: E0706 18:55:55.022649 2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-296mj" podUID="f7bc558b-d33b-448d-929c-3652d096c3d5" Jul 06 18:55:55.029595 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:55:55.029570 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k8fbq_bc7057bd-88da-48af-ba32-e6efdc6e8362/konnectivity-agent/0.log" limit=1024 Jul 06 18:55:55.497957 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:55:55.497928 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k8fbq_bc7057bd-88da-48af-ba32-e6efdc6e8362/konnectivity-agent/0.log" limit=1024 Jul 06 18:55:56.022837 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:55:56.022808 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-ln98s" Jul 06 18:55:56.022837 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:55:56.022825 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-wh2jx" Jul 06 18:55:56.023377 ip-10-0-130-59 kubenswrapper[2570]: E0706 18:55:56.022916 2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-ln98s" podUID="4beea8e8-7750-4a7a-bc49-249b3d7e3ab3" Jul 06 18:55:56.023377 ip-10-0-130-59 kubenswrapper[2570]: E0706 18:55:56.023009 2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-wh2jx" podUID="9262d667-2ed4-45aa-98f5-2d8158a66ca1" Jul 06 18:55:57.021919 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:55:57.021883 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-296mj" Jul 06 18:55:57.022087 ip-10-0-130-59 kubenswrapper[2570]: E0706 18:55:57.022009 2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-296mj" podUID="f7bc558b-d33b-448d-929c-3652d096c3d5" Jul 06 18:55:58.022900 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:55:58.022865 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-wh2jx" Jul 06 18:55:58.023324 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:55:58.022865 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-ln98s" Jul 06 18:55:58.023324 ip-10-0-130-59 kubenswrapper[2570]: E0706 18:55:58.023005 2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-wh2jx" podUID="9262d667-2ed4-45aa-98f5-2d8158a66ca1" Jul 06 18:55:58.023324 ip-10-0-130-59 kubenswrapper[2570]: E0706 18:55:58.023091 2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-ln98s" podUID="4beea8e8-7750-4a7a-bc49-249b3d7e3ab3" Jul 06 18:55:59.021983 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:55:59.021955 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-296mj" Jul 06 18:55:59.022131 ip-10-0-130-59 kubenswrapper[2570]: E0706 18:55:59.022078 2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-296mj" podUID="f7bc558b-d33b-448d-929c-3652d096c3d5" Jul 06 18:55:59.189944 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:55:59.189914 2570 generic.go:356] "Generic (PLEG): container finished" podID="4f1bcb6b-7c73-4bff-a246-78ddc5ca22ea" containerID="d2d7663b649ab2adf163e17fa99e401ce527cbef3425e06355233706c1415002" exitCode=0 Jul 06 18:55:59.190375 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:55:59.189979 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-wmxvz" event={"ID":"4f1bcb6b-7c73-4bff-a246-78ddc5ca22ea","Type":"ContainerDied","Data":"d2d7663b649ab2adf163e17fa99e401ce527cbef3425e06355233706c1415002"} Jul 06 18:56:00.022390 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:56:00.022366 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-ln98s" Jul 06 18:56:00.022562 ip-10-0-130-59 kubenswrapper[2570]: E0706 18:56:00.022478 2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-ln98s" podUID="4beea8e8-7750-4a7a-bc49-249b3d7e3ab3" Jul 06 18:56:00.022562 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:56:00.022531 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-wh2jx" Jul 06 18:56:00.022728 ip-10-0-130-59 kubenswrapper[2570]: E0706 18:56:00.022633 2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-wh2jx" podUID="9262d667-2ed4-45aa-98f5-2d8158a66ca1" Jul 06 18:56:00.195103 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:56:00.195074 2570 generic.go:356] "Generic (PLEG): container finished" podID="4f1bcb6b-7c73-4bff-a246-78ddc5ca22ea" containerID="bcf81c1bd3d1d991976b69166965a7d0b864759c185dd64885c87f3a98535450" exitCode=0 Jul 06 18:56:00.195444 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:56:00.195111 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-wmxvz" event={"ID":"4f1bcb6b-7c73-4bff-a246-78ddc5ca22ea","Type":"ContainerDied","Data":"bcf81c1bd3d1d991976b69166965a7d0b864759c185dd64885c87f3a98535450"} Jul 06 18:56:01.025353 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:56:01.025326 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-296mj" Jul 06 18:56:01.025504 ip-10-0-130-59 kubenswrapper[2570]: E0706 18:56:01.025405 2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-296mj" podUID="f7bc558b-d33b-448d-929c-3652d096c3d5" Jul 06 18:56:01.199157 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:56:01.199125 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k8fbq_bc7057bd-88da-48af-ba32-e6efdc6e8362/konnectivity-agent/0.log" limit=1024 Jul 06 18:56:01.200564 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:56:01.200539 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-wmxvz" event={"ID":"4f1bcb6b-7c73-4bff-a246-78ddc5ca22ea","Type":"ContainerStarted","Data":"c703c5b056c3e0e4aae08eadd89ed8bff0d96a8c678a7a8bcd0a483be98106fb"} Jul 06 18:56:01.223977 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:56:01.223939 2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-wmxvz" podStartSLOduration=8.291229763 podStartE2EDuration="40.223925128s" totalImagesPullingTime="31.932695365s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-06 18:55:21 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 18:56:01.223485567 +0000 UTC m=+40.778506739" watchObservedRunningTime="2026-07-06 18:56:01.223925128 +0000 UTC m=+40.778946299" Jul 06 18:56:01.255350 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:56:01.255328 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k8fbq_bc7057bd-88da-48af-ba32-e6efdc6e8362/konnectivity-agent/0.log" limit=1024 Jul 06 18:56:02.022217 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:56:02.022186 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-wh2jx" Jul 06 18:56:02.022365 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:56:02.022186 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-ln98s" Jul 06 18:56:02.022365 ip-10-0-130-59 kubenswrapper[2570]: E0706 18:56:02.022310 2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-wh2jx" podUID="9262d667-2ed4-45aa-98f5-2d8158a66ca1" Jul 06 18:56:02.022426 ip-10-0-130-59 kubenswrapper[2570]: E0706 18:56:02.022361 2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-ln98s" podUID="4beea8e8-7750-4a7a-bc49-249b3d7e3ab3" Jul 06 18:56:02.362841 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:56:02.362768 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k8fbq_bc7057bd-88da-48af-ba32-e6efdc6e8362/konnectivity-agent/0.log" limit=1024 Jul 06 18:56:02.463803 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:56:02.463773 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k8fbq_bc7057bd-88da-48af-ba32-e6efdc6e8362/konnectivity-agent/0.log" limit=1024 Jul 06 18:56:02.523841 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:56:02.523812 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k8fbq_bc7057bd-88da-48af-ba32-e6efdc6e8362/konnectivity-agent/0.log" limit=1024 Jul 06 18:56:02.646479 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:56:02.646381 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k8fbq_bc7057bd-88da-48af-ba32-e6efdc6e8362/konnectivity-agent/0.log" limit=1024 Jul 06 18:56:02.822425 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:56:02.822396 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k8fbq_bc7057bd-88da-48af-ba32-e6efdc6e8362/konnectivity-agent/0.log" limit=1024 Jul 06 18:56:02.878036 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:56:02.877998 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k8fbq_bc7057bd-88da-48af-ba32-e6efdc6e8362/konnectivity-agent/0.log" limit=1024 Jul 06 18:56:03.022650 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:56:03.022596 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-296mj" Jul 06 18:56:03.022816 ip-10-0-130-59 kubenswrapper[2570]: E0706 18:56:03.022717 2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-296mj" podUID="f7bc558b-d33b-448d-929c-3652d096c3d5" Jul 06 18:56:03.048044 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:56:03.048012 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k8fbq_bc7057bd-88da-48af-ba32-e6efdc6e8362/konnectivity-agent/0.log" limit=1024 Jul 06 18:56:03.268038 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:56:03.267999 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k8fbq_bc7057bd-88da-48af-ba32-e6efdc6e8362/konnectivity-agent/0.log" limit=1024 Jul 06 18:56:03.463388 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:56:03.463362 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k8fbq_bc7057bd-88da-48af-ba32-e6efdc6e8362/konnectivity-agent/0.log" limit=1024 Jul 06 18:56:03.538165 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:56:03.538136 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k8fbq_bc7057bd-88da-48af-ba32-e6efdc6e8362/konnectivity-agent/0.log" limit=1024 Jul 06 18:56:03.851830 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:56:03.851766 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k8fbq_bc7057bd-88da-48af-ba32-e6efdc6e8362/konnectivity-agent/0.log" limit=1024 Jul 06 18:56:04.021969 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:56:04.021944 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-wh2jx" Jul 06 18:56:04.022205 ip-10-0-130-59 kubenswrapper[2570]: E0706 18:56:04.022034 2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-wh2jx" podUID="9262d667-2ed4-45aa-98f5-2d8158a66ca1" Jul 06 18:56:04.022205 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:56:04.022060 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-ln98s" Jul 06 18:56:04.022205 ip-10-0-130-59 kubenswrapper[2570]: E0706 18:56:04.022108 2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-ln98s" podUID="4beea8e8-7750-4a7a-bc49-249b3d7e3ab3" Jul 06 18:56:04.239007 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:56:04.238983 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k8fbq_bc7057bd-88da-48af-ba32-e6efdc6e8362/konnectivity-agent/0.log" limit=1024 Jul 06 18:56:04.296080 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:56:04.296055 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k8fbq_bc7057bd-88da-48af-ba32-e6efdc6e8362/konnectivity-agent/0.log" limit=1024 Jul 06 18:56:05.022370 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:56:05.022336 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-296mj" Jul 06 18:56:05.022744 ip-10-0-130-59 kubenswrapper[2570]: E0706 18:56:05.022434 2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-296mj" podUID="f7bc558b-d33b-448d-929c-3652d096c3d5" Jul 06 18:56:05.090809 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:56:05.090776 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k8fbq_bc7057bd-88da-48af-ba32-e6efdc6e8362/konnectivity-agent/0.log" limit=1024 Jul 06 18:56:05.159707 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:56:05.159679 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k8fbq_bc7057bd-88da-48af-ba32-e6efdc6e8362/konnectivity-agent/0.log" limit=1024 Jul 06 18:56:05.253734 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:56:05.253704 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k8fbq_bc7057bd-88da-48af-ba32-e6efdc6e8362/konnectivity-agent/0.log" limit=1024 Jul 06 18:56:05.497242 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:56:05.497214 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k8fbq_bc7057bd-88da-48af-ba32-e6efdc6e8362/konnectivity-agent/0.log" limit=1024 Jul 06 18:56:05.699566 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:56:05.699535 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k8fbq_bc7057bd-88da-48af-ba32-e6efdc6e8362/konnectivity-agent/0.log" limit=1024 Jul 06 18:56:05.872412 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:56:05.872333 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k8fbq_bc7057bd-88da-48af-ba32-e6efdc6e8362/konnectivity-agent/0.log" limit=1024 Jul 06 18:56:06.022679 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:56:06.022652 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-ln98s" Jul 06 18:56:06.022679 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:56:06.022680 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-wh2jx" Jul 06 18:56:06.023044 ip-10-0-130-59 kubenswrapper[2570]: E0706 18:56:06.022753 2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-ln98s" podUID="4beea8e8-7750-4a7a-bc49-249b3d7e3ab3" Jul 06 18:56:06.023044 ip-10-0-130-59 kubenswrapper[2570]: E0706 18:56:06.022868 2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-wh2jx" podUID="9262d667-2ed4-45aa-98f5-2d8158a66ca1" Jul 06 18:56:06.045754 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:56:06.045729 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k8fbq_bc7057bd-88da-48af-ba32-e6efdc6e8362/konnectivity-agent/0.log" limit=1024 Jul 06 18:56:06.170874 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:56:06.170847 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k8fbq_bc7057bd-88da-48af-ba32-e6efdc6e8362/konnectivity-agent/0.log" limit=1024 Jul 06 18:56:06.266487 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:56:06.266453 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k8fbq_bc7057bd-88da-48af-ba32-e6efdc6e8362/konnectivity-agent/0.log" limit=1024 Jul 06 18:56:06.604846 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:56:06.604641 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k8fbq_bc7057bd-88da-48af-ba32-e6efdc6e8362/konnectivity-agent/0.log" limit=1024 Jul 06 18:56:06.984511 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:56:06.984470 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-wh2jx"] Jul 06 18:56:06.984736 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:56:06.984665 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-wh2jx" Jul 06 18:56:06.984819 ip-10-0-130-59 kubenswrapper[2570]: E0706 18:56:06.984792 2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-wh2jx" podUID="9262d667-2ed4-45aa-98f5-2d8158a66ca1" Jul 06 18:56:06.985119 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:56:06.985102 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-ln98s"] Jul 06 18:56:06.985212 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:56:06.985187 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-ln98s" Jul 06 18:56:06.985301 ip-10-0-130-59 kubenswrapper[2570]: E0706 18:56:06.985283 2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-ln98s" podUID="4beea8e8-7750-4a7a-bc49-249b3d7e3ab3" Jul 06 18:56:06.989400 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:56:06.989379 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-296mj"] Jul 06 18:56:06.989497 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:56:06.989459 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-296mj" Jul 06 18:56:06.989566 ip-10-0-130-59 kubenswrapper[2570]: E0706 18:56:06.989546 2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-296mj" podUID="f7bc558b-d33b-448d-929c-3652d096c3d5" Jul 06 18:56:07.009720 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:56:07.009690 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k8fbq_bc7057bd-88da-48af-ba32-e6efdc6e8362/konnectivity-agent/0.log" limit=1024 Jul 06 18:56:07.073557 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:56:07.073521 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k8fbq_bc7057bd-88da-48af-ba32-e6efdc6e8362/konnectivity-agent/0.log" limit=1024 Jul 06 18:56:08.304812 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:56:08.304776 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k8fbq_bc7057bd-88da-48af-ba32-e6efdc6e8362/konnectivity-agent/0.log" limit=1024 Jul 06 18:56:08.401243 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:56:08.401189 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k8fbq_bc7057bd-88da-48af-ba32-e6efdc6e8362/konnectivity-agent/0.log" limit=1024 Jul 06 18:56:08.806747 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:56:08.806716 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k8fbq_bc7057bd-88da-48af-ba32-e6efdc6e8362/konnectivity-agent/0.log" limit=1024 Jul 06 18:56:09.022593 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:56:09.022510 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-wh2jx" Jul 06 18:56:09.022729 ip-10-0-130-59 kubenswrapper[2570]: E0706 18:56:09.022635 2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-wh2jx" podUID="9262d667-2ed4-45aa-98f5-2d8158a66ca1" Jul 06 18:56:09.022729 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:56:09.022670 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-ln98s" Jul 06 18:56:09.022729 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:56:09.022701 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-296mj" Jul 06 18:56:09.022870 ip-10-0-130-59 kubenswrapper[2570]: E0706 18:56:09.022747 2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-296mj" podUID="f7bc558b-d33b-448d-929c-3652d096c3d5" Jul 06 18:56:09.022870 ip-10-0-130-59 kubenswrapper[2570]: E0706 18:56:09.022808 2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-ln98s" podUID="4beea8e8-7750-4a7a-bc49-249b3d7e3ab3" Jul 06 18:56:10.710732 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:56:10.710708 2570 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-59.ec2.internal" event="NodeReady" Jul 06 18:56:10.711274 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:56:10.710804 2570 kubelet_node_status.go:489] "Fast updating node status as it just became ready" Jul 06 18:56:10.759255 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:56:10.759231 2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-22lv8"] Jul 06 18:56:10.773570 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:56:10.773548 2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-mkrnp"] Jul 06 18:56:10.773725 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:56:10.773680 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-22lv8" Jul 06 18:56:10.776393 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:56:10.776369 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-ft894\"" Jul 06 18:56:10.776677 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:56:10.776654 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\"" Jul 06 18:56:10.776791 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:56:10.776699 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\"" Jul 06 18:56:10.776791 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:56:10.776710 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\"" Jul 06 18:56:10.792081 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:56:10.792060 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-22lv8"] Jul 06 18:56:10.792181 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:56:10.792085 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-mkrnp"] Jul 06 18:56:10.792240 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:56:10.792177 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-mkrnp" Jul 06 18:56:10.794759 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:56:10.794743 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-tlz5r\"" Jul 06 18:56:10.794865 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:56:10.794782 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\"" Jul 06 18:56:10.794865 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:56:10.794743 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\"" Jul 06 18:56:10.794865 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:56:10.794835 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\"" Jul 06 18:56:10.795289 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:56:10.795262 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\"" Jul 06 18:56:10.816988 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:56:10.816965 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/5f316223-4aea-420d-be1b-c2921c1f994f-data-volume\") pod \"insights-runtime-extractor-mkrnp\" (UID: \"5f316223-4aea-420d-be1b-c2921c1f994f\") " pod="openshift-insights/insights-runtime-extractor-mkrnp" Jul 06 18:56:10.817090 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:56:10.817001 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/5f316223-4aea-420d-be1b-c2921c1f994f-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-mkrnp\" (UID: \"5f316223-4aea-420d-be1b-c2921c1f994f\") " pod="openshift-insights/insights-runtime-extractor-mkrnp" Jul 06 18:56:10.817090 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:56:10.817031 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/5f316223-4aea-420d-be1b-c2921c1f994f-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-mkrnp\" (UID: \"5f316223-4aea-420d-be1b-c2921c1f994f\") " pod="openshift-insights/insights-runtime-extractor-mkrnp" Jul 06 18:56:10.817090 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:56:10.817046 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/60870efe-6ea8-49ed-a879-7a1d7dee12d0-cert\") pod \"ingress-canary-22lv8\" (UID: \"60870efe-6ea8-49ed-a879-7a1d7dee12d0\") " pod="openshift-ingress-canary/ingress-canary-22lv8" Jul 06 18:56:10.817207 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:56:10.817096 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-twjnk\" (UniqueName: \"kubernetes.io/projected/60870efe-6ea8-49ed-a879-7a1d7dee12d0-kube-api-access-twjnk\") pod \"ingress-canary-22lv8\" (UID: \"60870efe-6ea8-49ed-a879-7a1d7dee12d0\") " pod="openshift-ingress-canary/ingress-canary-22lv8" Jul 06 18:56:10.817207 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:56:10.817181 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/5f316223-4aea-420d-be1b-c2921c1f994f-crio-socket\") pod \"insights-runtime-extractor-mkrnp\" (UID: \"5f316223-4aea-420d-be1b-c2921c1f994f\") " pod="openshift-insights/insights-runtime-extractor-mkrnp" Jul 06 18:56:10.817306 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:56:10.817206 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-d4954\" (UniqueName: \"kubernetes.io/projected/5f316223-4aea-420d-be1b-c2921c1f994f-kube-api-access-d4954\") pod \"insights-runtime-extractor-mkrnp\" (UID: \"5f316223-4aea-420d-be1b-c2921c1f994f\") " pod="openshift-insights/insights-runtime-extractor-mkrnp" Jul 06 18:56:10.862010 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:56:10.861991 2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-dk4j7"] Jul 06 18:56:10.879183 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:56:10.879164 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-dk4j7"] Jul 06 18:56:10.879261 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:56:10.879240 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-dk4j7" Jul 06 18:56:10.881904 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:56:10.881885 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\"" Jul 06 18:56:10.882004 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:56:10.881883 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-q4rjh\"" Jul 06 18:56:10.882004 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:56:10.881883 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\"" Jul 06 18:56:10.918026 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:56:10.918001 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/5f316223-4aea-420d-be1b-c2921c1f994f-data-volume\") pod \"insights-runtime-extractor-mkrnp\" (UID: \"5f316223-4aea-420d-be1b-c2921c1f994f\") " pod="openshift-insights/insights-runtime-extractor-mkrnp" Jul 06 18:56:10.918126 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:56:10.918040 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/5f316223-4aea-420d-be1b-c2921c1f994f-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-mkrnp\" (UID: \"5f316223-4aea-420d-be1b-c2921c1f994f\") " pod="openshift-insights/insights-runtime-extractor-mkrnp" Jul 06 18:56:10.918126 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:56:10.918079 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/5f316223-4aea-420d-be1b-c2921c1f994f-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-mkrnp\" (UID: \"5f316223-4aea-420d-be1b-c2921c1f994f\") " pod="openshift-insights/insights-runtime-extractor-mkrnp" Jul 06 18:56:10.918126 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:56:10.918108 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/60870efe-6ea8-49ed-a879-7a1d7dee12d0-cert\") pod \"ingress-canary-22lv8\" (UID: \"60870efe-6ea8-49ed-a879-7a1d7dee12d0\") " pod="openshift-ingress-canary/ingress-canary-22lv8" Jul 06 18:56:10.918267 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:56:10.918254 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-twjnk\" (UniqueName: \"kubernetes.io/projected/60870efe-6ea8-49ed-a879-7a1d7dee12d0-kube-api-access-twjnk\") pod \"ingress-canary-22lv8\" (UID: \"60870efe-6ea8-49ed-a879-7a1d7dee12d0\") " pod="openshift-ingress-canary/ingress-canary-22lv8" Jul 06 18:56:10.918321 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:56:10.918284 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-q5x2p\" (UniqueName: \"kubernetes.io/projected/c68e3e9c-2b68-4ed9-99c5-f76eb52b09c3-kube-api-access-q5x2p\") pod \"dns-default-dk4j7\" (UID: \"c68e3e9c-2b68-4ed9-99c5-f76eb52b09c3\") " pod="openshift-dns/dns-default-dk4j7" Jul 06 18:56:10.918321 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:56:10.918310 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/c68e3e9c-2b68-4ed9-99c5-f76eb52b09c3-config-volume\") pod \"dns-default-dk4j7\" (UID: \"c68e3e9c-2b68-4ed9-99c5-f76eb52b09c3\") " pod="openshift-dns/dns-default-dk4j7" Jul 06 18:56:10.918426 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:56:10.918381 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c68e3e9c-2b68-4ed9-99c5-f76eb52b09c3-tmp-dir\") pod \"dns-default-dk4j7\" (UID: \"c68e3e9c-2b68-4ed9-99c5-f76eb52b09c3\") " pod="openshift-dns/dns-default-dk4j7" Jul 06 18:56:10.918426 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:56:10.918409 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/5f316223-4aea-420d-be1b-c2921c1f994f-data-volume\") pod \"insights-runtime-extractor-mkrnp\" (UID: \"5f316223-4aea-420d-be1b-c2921c1f994f\") " pod="openshift-insights/insights-runtime-extractor-mkrnp" Jul 06 18:56:10.918526 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:56:10.918455 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/5f316223-4aea-420d-be1b-c2921c1f994f-crio-socket\") pod \"insights-runtime-extractor-mkrnp\" (UID: \"5f316223-4aea-420d-be1b-c2921c1f994f\") " pod="openshift-insights/insights-runtime-extractor-mkrnp" Jul 06 18:56:10.918526 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:56:10.918497 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-d4954\" (UniqueName: \"kubernetes.io/projected/5f316223-4aea-420d-be1b-c2921c1f994f-kube-api-access-d4954\") pod \"insights-runtime-extractor-mkrnp\" (UID: \"5f316223-4aea-420d-be1b-c2921c1f994f\") " pod="openshift-insights/insights-runtime-extractor-mkrnp" Jul 06 18:56:10.918642 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:56:10.918620 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/c68e3e9c-2b68-4ed9-99c5-f76eb52b09c3-metrics-tls\") pod \"dns-default-dk4j7\" (UID: \"c68e3e9c-2b68-4ed9-99c5-f76eb52b09c3\") " pod="openshift-dns/dns-default-dk4j7" Jul 06 18:56:10.918705 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:56:10.918675 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/5f316223-4aea-420d-be1b-c2921c1f994f-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-mkrnp\" (UID: \"5f316223-4aea-420d-be1b-c2921c1f994f\") " pod="openshift-insights/insights-runtime-extractor-mkrnp" Jul 06 18:56:10.918771 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:56:10.918750 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/5f316223-4aea-420d-be1b-c2921c1f994f-crio-socket\") pod \"insights-runtime-extractor-mkrnp\" (UID: \"5f316223-4aea-420d-be1b-c2921c1f994f\") " pod="openshift-insights/insights-runtime-extractor-mkrnp" Jul 06 18:56:10.921888 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:56:10.921867 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/5f316223-4aea-420d-be1b-c2921c1f994f-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-mkrnp\" (UID: \"5f316223-4aea-420d-be1b-c2921c1f994f\") " pod="openshift-insights/insights-runtime-extractor-mkrnp" Jul 06 18:56:10.921964 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:56:10.921906 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/60870efe-6ea8-49ed-a879-7a1d7dee12d0-cert\") pod \"ingress-canary-22lv8\" (UID: \"60870efe-6ea8-49ed-a879-7a1d7dee12d0\") " pod="openshift-ingress-canary/ingress-canary-22lv8" Jul 06 18:56:10.926637 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:56:10.926593 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-twjnk\" (UniqueName: \"kubernetes.io/projected/60870efe-6ea8-49ed-a879-7a1d7dee12d0-kube-api-access-twjnk\") pod \"ingress-canary-22lv8\" (UID: \"60870efe-6ea8-49ed-a879-7a1d7dee12d0\") " pod="openshift-ingress-canary/ingress-canary-22lv8" Jul 06 18:56:10.926789 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:56:10.926770 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-d4954\" (UniqueName: \"kubernetes.io/projected/5f316223-4aea-420d-be1b-c2921c1f994f-kube-api-access-d4954\") pod \"insights-runtime-extractor-mkrnp\" (UID: \"5f316223-4aea-420d-be1b-c2921c1f994f\") " pod="openshift-insights/insights-runtime-extractor-mkrnp" Jul 06 18:56:10.964380 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:56:10.964329 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k8fbq_bc7057bd-88da-48af-ba32-e6efdc6e8362/konnectivity-agent/0.log" limit=1024 Jul 06 18:56:11.019831 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:56:11.019808 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c68e3e9c-2b68-4ed9-99c5-f76eb52b09c3-tmp-dir\") pod \"dns-default-dk4j7\" (UID: \"c68e3e9c-2b68-4ed9-99c5-f76eb52b09c3\") " pod="openshift-dns/dns-default-dk4j7" Jul 06 18:56:11.019912 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:56:11.019834 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/c68e3e9c-2b68-4ed9-99c5-f76eb52b09c3-metrics-tls\") pod \"dns-default-dk4j7\" (UID: \"c68e3e9c-2b68-4ed9-99c5-f76eb52b09c3\") " pod="openshift-dns/dns-default-dk4j7" Jul 06 18:56:11.019990 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:56:11.019973 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-q5x2p\" (UniqueName: \"kubernetes.io/projected/c68e3e9c-2b68-4ed9-99c5-f76eb52b09c3-kube-api-access-q5x2p\") pod \"dns-default-dk4j7\" (UID: \"c68e3e9c-2b68-4ed9-99c5-f76eb52b09c3\") " pod="openshift-dns/dns-default-dk4j7" Jul 06 18:56:11.020051 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:56:11.020018 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/c68e3e9c-2b68-4ed9-99c5-f76eb52b09c3-config-volume\") pod \"dns-default-dk4j7\" (UID: \"c68e3e9c-2b68-4ed9-99c5-f76eb52b09c3\") " pod="openshift-dns/dns-default-dk4j7" Jul 06 18:56:11.020114 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:56:11.020093 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c68e3e9c-2b68-4ed9-99c5-f76eb52b09c3-tmp-dir\") pod \"dns-default-dk4j7\" (UID: \"c68e3e9c-2b68-4ed9-99c5-f76eb52b09c3\") " pod="openshift-dns/dns-default-dk4j7" Jul 06 18:56:11.021721 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:56:11.021704 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/c68e3e9c-2b68-4ed9-99c5-f76eb52b09c3-metrics-tls\") pod \"dns-default-dk4j7\" (UID: \"c68e3e9c-2b68-4ed9-99c5-f76eb52b09c3\") " pod="openshift-dns/dns-default-dk4j7" Jul 06 18:56:11.023350 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:56:11.023332 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-ln98s" Jul 06 18:56:11.023419 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:56:11.023407 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-wh2jx" Jul 06 18:56:11.023463 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:56:11.023452 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-296mj" Jul 06 18:56:11.025919 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:56:11.025864 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-gk8fr\"" Jul 06 18:56:11.026126 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:56:11.026111 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-w7zz5\"" Jul 06 18:56:11.026198 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:56:11.026162 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Jul 06 18:56:11.026252 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:56:11.026213 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Jul 06 18:56:11.026306 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:56:11.026282 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" Jul 06 18:56:11.026306 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:56:11.026284 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" Jul 06 18:56:11.030140 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:56:11.030121 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/c68e3e9c-2b68-4ed9-99c5-f76eb52b09c3-config-volume\") pod \"dns-default-dk4j7\" (UID: \"c68e3e9c-2b68-4ed9-99c5-f76eb52b09c3\") " pod="openshift-dns/dns-default-dk4j7" Jul 06 18:56:11.031545 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:56:11.031522 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k8fbq_bc7057bd-88da-48af-ba32-e6efdc6e8362/konnectivity-agent/0.log" limit=1024 Jul 06 18:56:11.033930 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:56:11.033909 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-q5x2p\" (UniqueName: \"kubernetes.io/projected/c68e3e9c-2b68-4ed9-99c5-f76eb52b09c3-kube-api-access-q5x2p\") pod \"dns-default-dk4j7\" (UID: \"c68e3e9c-2b68-4ed9-99c5-f76eb52b09c3\") " pod="openshift-dns/dns-default-dk4j7" Jul 06 18:56:11.083160 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:56:11.083130 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-22lv8" Jul 06 18:56:11.102707 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:56:11.102691 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-mkrnp" Jul 06 18:56:11.187467 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:56:11.187268 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-dk4j7" Jul 06 18:56:11.278800 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:56:11.278774 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-22lv8"] Jul 06 18:56:11.281977 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:56:11.281951 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-mkrnp"] Jul 06 18:56:11.285387 ip-10-0-130-59 kubenswrapper[2570]: W0706 18:56:11.283255 2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod60870efe_6ea8_49ed_a879_7a1d7dee12d0.slice/crio-bbc2b827f99c09f013263fd28e2200fdb14cec6b16ba2508dac6c010be5e66cb WatchSource:0}: Error finding container bbc2b827f99c09f013263fd28e2200fdb14cec6b16ba2508dac6c010be5e66cb: Status 404 returned error can't find the container with id bbc2b827f99c09f013263fd28e2200fdb14cec6b16ba2508dac6c010be5e66cb Jul 06 18:56:11.287860 ip-10-0-130-59 kubenswrapper[2570]: W0706 18:56:11.287604 2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5f316223_4aea_420d_be1b_c2921c1f994f.slice/crio-0262a647e0fe5ba4f6b30a3d184632f8ede52eb30286d789bab9097ae0a0a103 WatchSource:0}: Error finding container 0262a647e0fe5ba4f6b30a3d184632f8ede52eb30286d789bab9097ae0a0a103: Status 404 returned error can't find the container with id 0262a647e0fe5ba4f6b30a3d184632f8ede52eb30286d789bab9097ae0a0a103 Jul 06 18:56:11.335972 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:56:11.335945 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-dk4j7"] Jul 06 18:56:11.343794 ip-10-0-130-59 kubenswrapper[2570]: W0706 18:56:11.343772 2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc68e3e9c_2b68_4ed9_99c5_f76eb52b09c3.slice/crio-966c669512d04ed0bae23f97002e9ba37a913480c3c0f92200c7d101031e645d WatchSource:0}: Error finding container 966c669512d04ed0bae23f97002e9ba37a913480c3c0f92200c7d101031e645d: Status 404 returned error can't find the container with id 966c669512d04ed0bae23f97002e9ba37a913480c3c0f92200c7d101031e645d Jul 06 18:56:11.662678 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:56:11.662586 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k8fbq_bc7057bd-88da-48af-ba32-e6efdc6e8362/konnectivity-agent/0.log" limit=1024 Jul 06 18:56:11.864023 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:56:11.863995 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k8fbq_bc7057bd-88da-48af-ba32-e6efdc6e8362/konnectivity-agent/0.log" limit=1024 Jul 06 18:56:11.923256 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:56:11.923221 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k8fbq_bc7057bd-88da-48af-ba32-e6efdc6e8362/konnectivity-agent/0.log" limit=1024 Jul 06 18:56:12.228267 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:56:12.228107 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-dk4j7" event={"ID":"c68e3e9c-2b68-4ed9-99c5-f76eb52b09c3","Type":"ContainerStarted","Data":"966c669512d04ed0bae23f97002e9ba37a913480c3c0f92200c7d101031e645d"} Jul 06 18:56:12.230237 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:56:12.230147 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-mkrnp" event={"ID":"5f316223-4aea-420d-be1b-c2921c1f994f","Type":"ContainerStarted","Data":"0262a647e0fe5ba4f6b30a3d184632f8ede52eb30286d789bab9097ae0a0a103"} Jul 06 18:56:12.232047 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:56:12.232022 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-22lv8" event={"ID":"60870efe-6ea8-49ed-a879-7a1d7dee12d0","Type":"ContainerStarted","Data":"bbc2b827f99c09f013263fd28e2200fdb14cec6b16ba2508dac6c010be5e66cb"} Jul 06 18:56:13.272927 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:56:13.272867 2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-6c6c7cc65c-tp9k6"] Jul 06 18:56:13.290744 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:56:13.290696 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-6c6c7cc65c-tp9k6"] Jul 06 18:56:13.290887 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:56:13.290835 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6c6c7cc65c-tp9k6" Jul 06 18:56:13.294272 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:56:13.293825 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"kube-root-ca.crt\"" Jul 06 18:56:13.294272 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:56:13.293991 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"openshift-service-ca.crt\"" Jul 06 18:56:13.294272 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:56:13.294037 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"oauth-serving-cert\"" Jul 06 18:56:13.294272 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:56:13.294095 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"console-config\"" Jul 06 18:56:13.294272 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:56:13.294129 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-serving-cert\"" Jul 06 18:56:13.294712 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:56:13.294694 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-oauth-config\"" Jul 06 18:56:13.294804 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:56:13.294725 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"service-ca\"" Jul 06 18:56:13.294863 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:56:13.294854 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-dockercfg-9l9vz\"" Jul 06 18:56:13.348544 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:56:13.348489 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/3afaab40-a812-4cd4-ab65-bc0da82cb738-console-oauth-config\") pod \"console-6c6c7cc65c-tp9k6\" (UID: \"3afaab40-a812-4cd4-ab65-bc0da82cb738\") " pod="openshift-console/console-6c6c7cc65c-tp9k6" Jul 06 18:56:13.348743 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:56:13.348594 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/3afaab40-a812-4cd4-ab65-bc0da82cb738-service-ca\") pod \"console-6c6c7cc65c-tp9k6\" (UID: \"3afaab40-a812-4cd4-ab65-bc0da82cb738\") " pod="openshift-console/console-6c6c7cc65c-tp9k6" Jul 06 18:56:13.348808 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:56:13.348731 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/3afaab40-a812-4cd4-ab65-bc0da82cb738-console-serving-cert\") pod \"console-6c6c7cc65c-tp9k6\" (UID: \"3afaab40-a812-4cd4-ab65-bc0da82cb738\") " pod="openshift-console/console-6c6c7cc65c-tp9k6" Jul 06 18:56:13.348909 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:56:13.348886 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-scdhh\" (UniqueName: \"kubernetes.io/projected/3afaab40-a812-4cd4-ab65-bc0da82cb738-kube-api-access-scdhh\") pod \"console-6c6c7cc65c-tp9k6\" (UID: \"3afaab40-a812-4cd4-ab65-bc0da82cb738\") " pod="openshift-console/console-6c6c7cc65c-tp9k6" Jul 06 18:56:13.349047 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:56:13.348928 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/3afaab40-a812-4cd4-ab65-bc0da82cb738-console-config\") pod \"console-6c6c7cc65c-tp9k6\" (UID: \"3afaab40-a812-4cd4-ab65-bc0da82cb738\") " pod="openshift-console/console-6c6c7cc65c-tp9k6" Jul 06 18:56:13.349047 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:56:13.348998 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/3afaab40-a812-4cd4-ab65-bc0da82cb738-oauth-serving-cert\") pod \"console-6c6c7cc65c-tp9k6\" (UID: \"3afaab40-a812-4cd4-ab65-bc0da82cb738\") " pod="openshift-console/console-6c6c7cc65c-tp9k6" Jul 06 18:56:13.449917 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:56:13.449878 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-scdhh\" (UniqueName: \"kubernetes.io/projected/3afaab40-a812-4cd4-ab65-bc0da82cb738-kube-api-access-scdhh\") pod \"console-6c6c7cc65c-tp9k6\" (UID: \"3afaab40-a812-4cd4-ab65-bc0da82cb738\") " pod="openshift-console/console-6c6c7cc65c-tp9k6" Jul 06 18:56:13.449917 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:56:13.449922 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/3afaab40-a812-4cd4-ab65-bc0da82cb738-console-config\") pod \"console-6c6c7cc65c-tp9k6\" (UID: \"3afaab40-a812-4cd4-ab65-bc0da82cb738\") " pod="openshift-console/console-6c6c7cc65c-tp9k6" Jul 06 18:56:13.450228 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:56:13.449951 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/3afaab40-a812-4cd4-ab65-bc0da82cb738-oauth-serving-cert\") pod \"console-6c6c7cc65c-tp9k6\" (UID: \"3afaab40-a812-4cd4-ab65-bc0da82cb738\") " pod="openshift-console/console-6c6c7cc65c-tp9k6" Jul 06 18:56:13.450228 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:56:13.449975 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/3afaab40-a812-4cd4-ab65-bc0da82cb738-console-oauth-config\") pod \"console-6c6c7cc65c-tp9k6\" (UID: \"3afaab40-a812-4cd4-ab65-bc0da82cb738\") " pod="openshift-console/console-6c6c7cc65c-tp9k6" Jul 06 18:56:13.450228 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:56:13.449993 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/3afaab40-a812-4cd4-ab65-bc0da82cb738-service-ca\") pod \"console-6c6c7cc65c-tp9k6\" (UID: \"3afaab40-a812-4cd4-ab65-bc0da82cb738\") " pod="openshift-console/console-6c6c7cc65c-tp9k6" Jul 06 18:56:13.450228 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:56:13.450018 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/3afaab40-a812-4cd4-ab65-bc0da82cb738-console-serving-cert\") pod \"console-6c6c7cc65c-tp9k6\" (UID: \"3afaab40-a812-4cd4-ab65-bc0da82cb738\") " pod="openshift-console/console-6c6c7cc65c-tp9k6" Jul 06 18:56:13.450692 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:56:13.450667 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/3afaab40-a812-4cd4-ab65-bc0da82cb738-console-config\") pod \"console-6c6c7cc65c-tp9k6\" (UID: \"3afaab40-a812-4cd4-ab65-bc0da82cb738\") " pod="openshift-console/console-6c6c7cc65c-tp9k6" Jul 06 18:56:13.450862 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:56:13.450841 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/3afaab40-a812-4cd4-ab65-bc0da82cb738-service-ca\") pod \"console-6c6c7cc65c-tp9k6\" (UID: \"3afaab40-a812-4cd4-ab65-bc0da82cb738\") " pod="openshift-console/console-6c6c7cc65c-tp9k6" Jul 06 18:56:13.451459 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:56:13.451389 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/3afaab40-a812-4cd4-ab65-bc0da82cb738-oauth-serving-cert\") pod \"console-6c6c7cc65c-tp9k6\" (UID: \"3afaab40-a812-4cd4-ab65-bc0da82cb738\") " pod="openshift-console/console-6c6c7cc65c-tp9k6" Jul 06 18:56:13.454322 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:56:13.454301 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/3afaab40-a812-4cd4-ab65-bc0da82cb738-console-oauth-config\") pod \"console-6c6c7cc65c-tp9k6\" (UID: \"3afaab40-a812-4cd4-ab65-bc0da82cb738\") " pod="openshift-console/console-6c6c7cc65c-tp9k6" Jul 06 18:56:13.454420 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:56:13.454321 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/3afaab40-a812-4cd4-ab65-bc0da82cb738-console-serving-cert\") pod \"console-6c6c7cc65c-tp9k6\" (UID: \"3afaab40-a812-4cd4-ab65-bc0da82cb738\") " pod="openshift-console/console-6c6c7cc65c-tp9k6" Jul 06 18:56:13.471180 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:56:13.471151 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k8fbq_bc7057bd-88da-48af-ba32-e6efdc6e8362/konnectivity-agent/0.log" limit=1024 Jul 06 18:56:13.474073 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:56:13.474046 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-scdhh\" (UniqueName: \"kubernetes.io/projected/3afaab40-a812-4cd4-ab65-bc0da82cb738-kube-api-access-scdhh\") pod \"console-6c6c7cc65c-tp9k6\" (UID: \"3afaab40-a812-4cd4-ab65-bc0da82cb738\") " pod="openshift-console/console-6c6c7cc65c-tp9k6" Jul 06 18:56:13.603155 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:56:13.603072 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6c6c7cc65c-tp9k6" Jul 06 18:56:13.720835 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:56:13.720803 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k8fbq_bc7057bd-88da-48af-ba32-e6efdc6e8362/konnectivity-agent/0.log" limit=1024 Jul 06 18:56:13.753926 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:56:13.753889 2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/openshift-state-metrics-54cf96dfcf-h62zc"] Jul 06 18:56:13.766336 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:56:13.766311 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-h62zc" Jul 06 18:56:13.770595 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:56:13.770573 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-kube-rbac-proxy-config\"" Jul 06 18:56:13.770736 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:56:13.770626 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\"" Jul 06 18:56:13.770808 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:56:13.770742 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\"" Jul 06 18:56:13.770808 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:56:13.770769 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-tls\"" Jul 06 18:56:13.771485 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:56:13.771465 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-dockercfg-lphkc\"" Jul 06 18:56:13.772064 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:56:13.771952 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\"" Jul 06 18:56:13.773022 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:56:13.772998 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/openshift-state-metrics-54cf96dfcf-h62zc"] Jul 06 18:56:13.779099 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:56:13.779076 2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/kube-state-metrics-7c9dbb8b7d-lrhkk"] Jul 06 18:56:13.794470 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:56:13.794436 2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-2fd54"] Jul 06 18:56:13.794650 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:56:13.794626 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-lrhkk" Jul 06 18:56:13.801143 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:56:13.801120 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-kube-rbac-proxy-config\"" Jul 06 18:56:13.801503 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:56:13.801436 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-dockercfg-wfnbv\"" Jul 06 18:56:13.801687 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:56:13.801666 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-tls\"" Jul 06 18:56:13.801921 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:56:13.801866 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-custom-resource-state-configmap\"" Jul 06 18:56:13.808915 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:56:13.808893 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/kube-state-metrics-7c9dbb8b7d-lrhkk"] Jul 06 18:56:13.809042 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:56:13.809025 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-2fd54" Jul 06 18:56:13.812088 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:56:13.812037 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-s2prm\"" Jul 06 18:56:13.812088 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:56:13.812082 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\"" Jul 06 18:56:13.812663 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:56:13.812336 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\"" Jul 06 18:56:13.812663 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:56:13.812437 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\"" Jul 06 18:56:13.817095 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:56:13.817038 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k8fbq_bc7057bd-88da-48af-ba32-e6efdc6e8362/konnectivity-agent/0.log" limit=1024 Jul 06 18:56:13.853730 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:56:13.853652 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/30d171a0-0417-4444-8af2-154f0d6c82f9-volume-directive-shadow\") pod \"kube-state-metrics-7c9dbb8b7d-lrhkk\" (UID: \"30d171a0-0417-4444-8af2-154f0d6c82f9\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-lrhkk" Jul 06 18:56:13.853730 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:56:13.853696 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/910d0e65-b069-41b4-a8d5-2881341454e7-openshift-state-metrics-tls\") pod \"openshift-state-metrics-54cf96dfcf-h62zc\" (UID: \"910d0e65-b069-41b4-a8d5-2881341454e7\") " pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-h62zc" Jul 06 18:56:13.853925 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:56:13.853739 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/b3ab8883-a662-4f63-9429-13b7f1c3a3b3-node-exporter-accelerators-collector-config\") pod \"node-exporter-2fd54\" (UID: \"b3ab8883-a662-4f63-9429-13b7f1c3a3b3\") " pod="openshift-monitoring/node-exporter-2fd54" Jul 06 18:56:13.853925 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:56:13.853766 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/30d171a0-0417-4444-8af2-154f0d6c82f9-kube-state-metrics-tls\") pod \"kube-state-metrics-7c9dbb8b7d-lrhkk\" (UID: \"30d171a0-0417-4444-8af2-154f0d6c82f9\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-lrhkk" Jul 06 18:56:13.853925 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:56:13.853787 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/910d0e65-b069-41b4-a8d5-2881341454e7-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-54cf96dfcf-h62zc\" (UID: \"910d0e65-b069-41b4-a8d5-2881341454e7\") " pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-h62zc" Jul 06 18:56:13.853925 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:56:13.853811 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/b3ab8883-a662-4f63-9429-13b7f1c3a3b3-metrics-client-ca\") pod \"node-exporter-2fd54\" (UID: \"b3ab8883-a662-4f63-9429-13b7f1c3a3b3\") " pod="openshift-monitoring/node-exporter-2fd54" Jul 06 18:56:13.853925 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:56:13.853842 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/30d171a0-0417-4444-8af2-154f0d6c82f9-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-7c9dbb8b7d-lrhkk\" (UID: \"30d171a0-0417-4444-8af2-154f0d6c82f9\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-lrhkk" Jul 06 18:56:13.853925 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:56:13.853867 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/30d171a0-0417-4444-8af2-154f0d6c82f9-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-7c9dbb8b7d-lrhkk\" (UID: \"30d171a0-0417-4444-8af2-154f0d6c82f9\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-lrhkk" Jul 06 18:56:13.854138 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:56:13.853942 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/30d171a0-0417-4444-8af2-154f0d6c82f9-metrics-client-ca\") pod \"kube-state-metrics-7c9dbb8b7d-lrhkk\" (UID: \"30d171a0-0417-4444-8af2-154f0d6c82f9\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-lrhkk" Jul 06 18:56:13.854138 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:56:13.853973 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/b3ab8883-a662-4f63-9429-13b7f1c3a3b3-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-2fd54\" (UID: \"b3ab8883-a662-4f63-9429-13b7f1c3a3b3\") " pod="openshift-monitoring/node-exporter-2fd54" Jul 06 18:56:13.854138 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:56:13.853997 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/b3ab8883-a662-4f63-9429-13b7f1c3a3b3-node-exporter-textfile\") pod \"node-exporter-2fd54\" (UID: \"b3ab8883-a662-4f63-9429-13b7f1c3a3b3\") " pod="openshift-monitoring/node-exporter-2fd54" Jul 06 18:56:13.854138 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:56:13.854017 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/b3ab8883-a662-4f63-9429-13b7f1c3a3b3-sys\") pod \"node-exporter-2fd54\" (UID: \"b3ab8883-a662-4f63-9429-13b7f1c3a3b3\") " pod="openshift-monitoring/node-exporter-2fd54" Jul 06 18:56:13.854138 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:56:13.854038 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/b3ab8883-a662-4f63-9429-13b7f1c3a3b3-node-exporter-wtmp\") pod \"node-exporter-2fd54\" (UID: \"b3ab8883-a662-4f63-9429-13b7f1c3a3b3\") " pod="openshift-monitoring/node-exporter-2fd54" Jul 06 18:56:13.854138 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:56:13.854111 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/910d0e65-b069-41b4-a8d5-2881341454e7-metrics-client-ca\") pod \"openshift-state-metrics-54cf96dfcf-h62zc\" (UID: \"910d0e65-b069-41b4-a8d5-2881341454e7\") " pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-h62zc" Jul 06 18:56:13.854395 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:56:13.854162 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cht2t\" (UniqueName: \"kubernetes.io/projected/910d0e65-b069-41b4-a8d5-2881341454e7-kube-api-access-cht2t\") pod \"openshift-state-metrics-54cf96dfcf-h62zc\" (UID: \"910d0e65-b069-41b4-a8d5-2881341454e7\") " pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-h62zc" Jul 06 18:56:13.854395 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:56:13.854182 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dnt7f\" (UniqueName: \"kubernetes.io/projected/30d171a0-0417-4444-8af2-154f0d6c82f9-kube-api-access-dnt7f\") pod \"kube-state-metrics-7c9dbb8b7d-lrhkk\" (UID: \"30d171a0-0417-4444-8af2-154f0d6c82f9\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-lrhkk" Jul 06 18:56:13.854395 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:56:13.854250 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/b3ab8883-a662-4f63-9429-13b7f1c3a3b3-node-exporter-tls\") pod \"node-exporter-2fd54\" (UID: \"b3ab8883-a662-4f63-9429-13b7f1c3a3b3\") " pod="openshift-monitoring/node-exporter-2fd54" Jul 06 18:56:13.854395 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:56:13.854281 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/b3ab8883-a662-4f63-9429-13b7f1c3a3b3-root\") pod \"node-exporter-2fd54\" (UID: \"b3ab8883-a662-4f63-9429-13b7f1c3a3b3\") " pod="openshift-monitoring/node-exporter-2fd54" Jul 06 18:56:13.854395 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:56:13.854310 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pmqp7\" (UniqueName: \"kubernetes.io/projected/b3ab8883-a662-4f63-9429-13b7f1c3a3b3-kube-api-access-pmqp7\") pod \"node-exporter-2fd54\" (UID: \"b3ab8883-a662-4f63-9429-13b7f1c3a3b3\") " pod="openshift-monitoring/node-exporter-2fd54" Jul 06 18:56:13.954784 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:56:13.954749 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/30d171a0-0417-4444-8af2-154f0d6c82f9-metrics-client-ca\") pod \"kube-state-metrics-7c9dbb8b7d-lrhkk\" (UID: \"30d171a0-0417-4444-8af2-154f0d6c82f9\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-lrhkk" Jul 06 18:56:13.954959 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:56:13.954795 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/b3ab8883-a662-4f63-9429-13b7f1c3a3b3-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-2fd54\" (UID: \"b3ab8883-a662-4f63-9429-13b7f1c3a3b3\") " pod="openshift-monitoring/node-exporter-2fd54" Jul 06 18:56:13.954959 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:56:13.954827 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/b3ab8883-a662-4f63-9429-13b7f1c3a3b3-node-exporter-textfile\") pod \"node-exporter-2fd54\" (UID: \"b3ab8883-a662-4f63-9429-13b7f1c3a3b3\") " pod="openshift-monitoring/node-exporter-2fd54" Jul 06 18:56:13.954959 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:56:13.954850 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/b3ab8883-a662-4f63-9429-13b7f1c3a3b3-sys\") pod \"node-exporter-2fd54\" (UID: \"b3ab8883-a662-4f63-9429-13b7f1c3a3b3\") " pod="openshift-monitoring/node-exporter-2fd54" Jul 06 18:56:13.954959 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:56:13.954868 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/b3ab8883-a662-4f63-9429-13b7f1c3a3b3-node-exporter-wtmp\") pod \"node-exporter-2fd54\" (UID: \"b3ab8883-a662-4f63-9429-13b7f1c3a3b3\") " pod="openshift-monitoring/node-exporter-2fd54" Jul 06 18:56:13.954959 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:56:13.954916 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/910d0e65-b069-41b4-a8d5-2881341454e7-metrics-client-ca\") pod \"openshift-state-metrics-54cf96dfcf-h62zc\" (UID: \"910d0e65-b069-41b4-a8d5-2881341454e7\") " pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-h62zc" Jul 06 18:56:13.955163 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:56:13.954964 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cht2t\" (UniqueName: \"kubernetes.io/projected/910d0e65-b069-41b4-a8d5-2881341454e7-kube-api-access-cht2t\") pod \"openshift-state-metrics-54cf96dfcf-h62zc\" (UID: \"910d0e65-b069-41b4-a8d5-2881341454e7\") " pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-h62zc" Jul 06 18:56:13.955163 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:56:13.954986 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dnt7f\" (UniqueName: \"kubernetes.io/projected/30d171a0-0417-4444-8af2-154f0d6c82f9-kube-api-access-dnt7f\") pod \"kube-state-metrics-7c9dbb8b7d-lrhkk\" (UID: \"30d171a0-0417-4444-8af2-154f0d6c82f9\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-lrhkk" Jul 06 18:56:13.955163 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:56:13.955019 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/b3ab8883-a662-4f63-9429-13b7f1c3a3b3-node-exporter-tls\") pod \"node-exporter-2fd54\" (UID: \"b3ab8883-a662-4f63-9429-13b7f1c3a3b3\") " pod="openshift-monitoring/node-exporter-2fd54" Jul 06 18:56:13.955163 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:56:13.955054 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/b3ab8883-a662-4f63-9429-13b7f1c3a3b3-root\") pod \"node-exporter-2fd54\" (UID: \"b3ab8883-a662-4f63-9429-13b7f1c3a3b3\") " pod="openshift-monitoring/node-exporter-2fd54" Jul 06 18:56:13.955163 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:56:13.955079 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pmqp7\" (UniqueName: \"kubernetes.io/projected/b3ab8883-a662-4f63-9429-13b7f1c3a3b3-kube-api-access-pmqp7\") pod \"node-exporter-2fd54\" (UID: \"b3ab8883-a662-4f63-9429-13b7f1c3a3b3\") " pod="openshift-monitoring/node-exporter-2fd54" Jul 06 18:56:13.955163 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:56:13.955117 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/30d171a0-0417-4444-8af2-154f0d6c82f9-volume-directive-shadow\") pod \"kube-state-metrics-7c9dbb8b7d-lrhkk\" (UID: \"30d171a0-0417-4444-8af2-154f0d6c82f9\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-lrhkk" Jul 06 18:56:13.955163 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:56:13.955143 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/910d0e65-b069-41b4-a8d5-2881341454e7-openshift-state-metrics-tls\") pod \"openshift-state-metrics-54cf96dfcf-h62zc\" (UID: \"910d0e65-b069-41b4-a8d5-2881341454e7\") " pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-h62zc" Jul 06 18:56:13.955382 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:56:13.955172 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/b3ab8883-a662-4f63-9429-13b7f1c3a3b3-node-exporter-accelerators-collector-config\") pod \"node-exporter-2fd54\" (UID: \"b3ab8883-a662-4f63-9429-13b7f1c3a3b3\") " pod="openshift-monitoring/node-exporter-2fd54" Jul 06 18:56:13.955382 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:56:13.955202 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/b3ab8883-a662-4f63-9429-13b7f1c3a3b3-node-exporter-textfile\") pod \"node-exporter-2fd54\" (UID: \"b3ab8883-a662-4f63-9429-13b7f1c3a3b3\") " pod="openshift-monitoring/node-exporter-2fd54" Jul 06 18:56:13.955382 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:56:13.955209 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/30d171a0-0417-4444-8af2-154f0d6c82f9-kube-state-metrics-tls\") pod \"kube-state-metrics-7c9dbb8b7d-lrhkk\" (UID: \"30d171a0-0417-4444-8af2-154f0d6c82f9\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-lrhkk" Jul 06 18:56:13.955382 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:56:13.955238 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/910d0e65-b069-41b4-a8d5-2881341454e7-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-54cf96dfcf-h62zc\" (UID: \"910d0e65-b069-41b4-a8d5-2881341454e7\") " pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-h62zc" Jul 06 18:56:13.955382 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:56:13.955265 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/b3ab8883-a662-4f63-9429-13b7f1c3a3b3-metrics-client-ca\") pod \"node-exporter-2fd54\" (UID: \"b3ab8883-a662-4f63-9429-13b7f1c3a3b3\") " pod="openshift-monitoring/node-exporter-2fd54" Jul 06 18:56:13.955382 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:56:13.955295 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/30d171a0-0417-4444-8af2-154f0d6c82f9-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-7c9dbb8b7d-lrhkk\" (UID: \"30d171a0-0417-4444-8af2-154f0d6c82f9\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-lrhkk" Jul 06 18:56:13.955382 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:56:13.955328 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/30d171a0-0417-4444-8af2-154f0d6c82f9-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-7c9dbb8b7d-lrhkk\" (UID: \"30d171a0-0417-4444-8af2-154f0d6c82f9\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-lrhkk" Jul 06 18:56:13.956577 ip-10-0-130-59 kubenswrapper[2570]: E0706 18:56:13.955835 2570 secret.go:189] Couldn't get secret openshift-monitoring/kube-state-metrics-tls: secret "kube-state-metrics-tls" not found Jul 06 18:56:13.956577 ip-10-0-130-59 kubenswrapper[2570]: E0706 18:56:13.955904 2570 secret.go:189] Couldn't get secret openshift-monitoring/openshift-state-metrics-tls: secret "openshift-state-metrics-tls" not found Jul 06 18:56:13.956577 ip-10-0-130-59 kubenswrapper[2570]: E0706 18:56:13.955920 2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/30d171a0-0417-4444-8af2-154f0d6c82f9-kube-state-metrics-tls podName:30d171a0-0417-4444-8af2-154f0d6c82f9 nodeName:}" failed. No retries permitted until 2026-07-06 18:56:14.455901548 +0000 UTC m=+54.010922700 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-state-metrics-tls" (UniqueName: "kubernetes.io/secret/30d171a0-0417-4444-8af2-154f0d6c82f9-kube-state-metrics-tls") pod "kube-state-metrics-7c9dbb8b7d-lrhkk" (UID: "30d171a0-0417-4444-8af2-154f0d6c82f9") : secret "kube-state-metrics-tls" not found Jul 06 18:56:13.956577 ip-10-0-130-59 kubenswrapper[2570]: E0706 18:56:13.955946 2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/910d0e65-b069-41b4-a8d5-2881341454e7-openshift-state-metrics-tls podName:910d0e65-b069-41b4-a8d5-2881341454e7 nodeName:}" failed. No retries permitted until 2026-07-06 18:56:14.455933537 +0000 UTC m=+54.010954706 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "openshift-state-metrics-tls" (UniqueName: "kubernetes.io/secret/910d0e65-b069-41b4-a8d5-2881341454e7-openshift-state-metrics-tls") pod "openshift-state-metrics-54cf96dfcf-h62zc" (UID: "910d0e65-b069-41b4-a8d5-2881341454e7") : secret "openshift-state-metrics-tls" not found Jul 06 18:56:13.956577 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:56:13.955994 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/b3ab8883-a662-4f63-9429-13b7f1c3a3b3-sys\") pod \"node-exporter-2fd54\" (UID: \"b3ab8883-a662-4f63-9429-13b7f1c3a3b3\") " pod="openshift-monitoring/node-exporter-2fd54" Jul 06 18:56:13.956577 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:56:13.955990 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/30d171a0-0417-4444-8af2-154f0d6c82f9-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-7c9dbb8b7d-lrhkk\" (UID: \"30d171a0-0417-4444-8af2-154f0d6c82f9\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-lrhkk" Jul 06 18:56:13.956577 ip-10-0-130-59 kubenswrapper[2570]: E0706 18:56:13.956100 2570 secret.go:189] Couldn't get secret openshift-monitoring/node-exporter-tls: secret "node-exporter-tls" not found Jul 06 18:56:13.956577 ip-10-0-130-59 kubenswrapper[2570]: E0706 18:56:13.956163 2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b3ab8883-a662-4f63-9429-13b7f1c3a3b3-node-exporter-tls podName:b3ab8883-a662-4f63-9429-13b7f1c3a3b3 nodeName:}" failed. No retries permitted until 2026-07-06 18:56:14.456146185 +0000 UTC m=+54.011167350 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "node-exporter-tls" (UniqueName: "kubernetes.io/secret/b3ab8883-a662-4f63-9429-13b7f1c3a3b3-node-exporter-tls") pod "node-exporter-2fd54" (UID: "b3ab8883-a662-4f63-9429-13b7f1c3a3b3") : secret "node-exporter-tls" not found Jul 06 18:56:13.956577 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:56:13.956212 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/b3ab8883-a662-4f63-9429-13b7f1c3a3b3-root\") pod \"node-exporter-2fd54\" (UID: \"b3ab8883-a662-4f63-9429-13b7f1c3a3b3\") " pod="openshift-monitoring/node-exporter-2fd54" Jul 06 18:56:13.956577 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:56:13.956332 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/b3ab8883-a662-4f63-9429-13b7f1c3a3b3-node-exporter-wtmp\") pod \"node-exporter-2fd54\" (UID: \"b3ab8883-a662-4f63-9429-13b7f1c3a3b3\") " pod="openshift-monitoring/node-exporter-2fd54" Jul 06 18:56:13.956577 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:56:13.956337 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/30d171a0-0417-4444-8af2-154f0d6c82f9-volume-directive-shadow\") pod \"kube-state-metrics-7c9dbb8b7d-lrhkk\" (UID: \"30d171a0-0417-4444-8af2-154f0d6c82f9\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-lrhkk" Jul 06 18:56:13.956577 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:56:13.956526 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/b3ab8883-a662-4f63-9429-13b7f1c3a3b3-metrics-client-ca\") pod \"node-exporter-2fd54\" (UID: \"b3ab8883-a662-4f63-9429-13b7f1c3a3b3\") " pod="openshift-monitoring/node-exporter-2fd54" Jul 06 18:56:13.957255 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:56:13.956702 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/910d0e65-b069-41b4-a8d5-2881341454e7-metrics-client-ca\") pod \"openshift-state-metrics-54cf96dfcf-h62zc\" (UID: \"910d0e65-b069-41b4-a8d5-2881341454e7\") " pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-h62zc" Jul 06 18:56:13.957255 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:56:13.956713 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/b3ab8883-a662-4f63-9429-13b7f1c3a3b3-node-exporter-accelerators-collector-config\") pod \"node-exporter-2fd54\" (UID: \"b3ab8883-a662-4f63-9429-13b7f1c3a3b3\") " pod="openshift-monitoring/node-exporter-2fd54" Jul 06 18:56:13.957255 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:56:13.956879 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/30d171a0-0417-4444-8af2-154f0d6c82f9-metrics-client-ca\") pod \"kube-state-metrics-7c9dbb8b7d-lrhkk\" (UID: \"30d171a0-0417-4444-8af2-154f0d6c82f9\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-lrhkk" Jul 06 18:56:13.959279 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:56:13.959234 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/b3ab8883-a662-4f63-9429-13b7f1c3a3b3-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-2fd54\" (UID: \"b3ab8883-a662-4f63-9429-13b7f1c3a3b3\") " pod="openshift-monitoring/node-exporter-2fd54" Jul 06 18:56:13.959279 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:56:13.959255 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/30d171a0-0417-4444-8af2-154f0d6c82f9-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-7c9dbb8b7d-lrhkk\" (UID: \"30d171a0-0417-4444-8af2-154f0d6c82f9\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-lrhkk" Jul 06 18:56:13.959451 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:56:13.959408 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/910d0e65-b069-41b4-a8d5-2881341454e7-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-54cf96dfcf-h62zc\" (UID: \"910d0e65-b069-41b4-a8d5-2881341454e7\") " pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-h62zc" Jul 06 18:56:13.964667 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:56:13.964640 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-pmqp7\" (UniqueName: \"kubernetes.io/projected/b3ab8883-a662-4f63-9429-13b7f1c3a3b3-kube-api-access-pmqp7\") pod \"node-exporter-2fd54\" (UID: \"b3ab8883-a662-4f63-9429-13b7f1c3a3b3\") " pod="openshift-monitoring/node-exporter-2fd54" Jul 06 18:56:13.964853 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:56:13.964837 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-cht2t\" (UniqueName: \"kubernetes.io/projected/910d0e65-b069-41b4-a8d5-2881341454e7-kube-api-access-cht2t\") pod \"openshift-state-metrics-54cf96dfcf-h62zc\" (UID: \"910d0e65-b069-41b4-a8d5-2881341454e7\") " pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-h62zc" Jul 06 18:56:13.965035 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:56:13.965015 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-dnt7f\" (UniqueName: \"kubernetes.io/projected/30d171a0-0417-4444-8af2-154f0d6c82f9-kube-api-access-dnt7f\") pod \"kube-state-metrics-7c9dbb8b7d-lrhkk\" (UID: \"30d171a0-0417-4444-8af2-154f0d6c82f9\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-lrhkk" Jul 06 18:56:13.989426 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:56:13.989393 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k8fbq_bc7057bd-88da-48af-ba32-e6efdc6e8362/konnectivity-agent/0.log" limit=1024 Jul 06 18:56:14.105570 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:56:14.105474 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k8fbq_bc7057bd-88da-48af-ba32-e6efdc6e8362/konnectivity-agent/0.log" limit=1024 Jul 06 18:56:14.339571 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:56:14.339448 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k8fbq_bc7057bd-88da-48af-ba32-e6efdc6e8362/konnectivity-agent/0.log" limit=1024 Jul 06 18:56:14.429523 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:56:14.429494 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k8fbq_bc7057bd-88da-48af-ba32-e6efdc6e8362/konnectivity-agent/0.log" limit=1024 Jul 06 18:56:14.459785 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:56:14.459748 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/b3ab8883-a662-4f63-9429-13b7f1c3a3b3-node-exporter-tls\") pod \"node-exporter-2fd54\" (UID: \"b3ab8883-a662-4f63-9429-13b7f1c3a3b3\") " pod="openshift-monitoring/node-exporter-2fd54" Jul 06 18:56:14.459912 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:56:14.459807 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/910d0e65-b069-41b4-a8d5-2881341454e7-openshift-state-metrics-tls\") pod \"openshift-state-metrics-54cf96dfcf-h62zc\" (UID: \"910d0e65-b069-41b4-a8d5-2881341454e7\") " pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-h62zc" Jul 06 18:56:14.459912 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:56:14.459846 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/30d171a0-0417-4444-8af2-154f0d6c82f9-kube-state-metrics-tls\") pod \"kube-state-metrics-7c9dbb8b7d-lrhkk\" (UID: \"30d171a0-0417-4444-8af2-154f0d6c82f9\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-lrhkk" Jul 06 18:56:14.462367 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:56:14.462337 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/b3ab8883-a662-4f63-9429-13b7f1c3a3b3-node-exporter-tls\") pod \"node-exporter-2fd54\" (UID: \"b3ab8883-a662-4f63-9429-13b7f1c3a3b3\") " pod="openshift-monitoring/node-exporter-2fd54" Jul 06 18:56:14.462367 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:56:14.462364 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/910d0e65-b069-41b4-a8d5-2881341454e7-openshift-state-metrics-tls\") pod \"openshift-state-metrics-54cf96dfcf-h62zc\" (UID: \"910d0e65-b069-41b4-a8d5-2881341454e7\") " pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-h62zc" Jul 06 18:56:14.462500 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:56:14.462397 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/30d171a0-0417-4444-8af2-154f0d6c82f9-kube-state-metrics-tls\") pod \"kube-state-metrics-7c9dbb8b7d-lrhkk\" (UID: \"30d171a0-0417-4444-8af2-154f0d6c82f9\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-lrhkk" Jul 06 18:56:14.531909 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:56:14.531875 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k8fbq_bc7057bd-88da-48af-ba32-e6efdc6e8362/konnectivity-agent/0.log" limit=1024 Jul 06 18:56:14.671978 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:56:14.671949 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k8fbq_bc7057bd-88da-48af-ba32-e6efdc6e8362/konnectivity-agent/0.log" limit=1024 Jul 06 18:56:14.677988 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:56:14.677959 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-h62zc" Jul 06 18:56:14.705963 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:56:14.705935 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-lrhkk" Jul 06 18:56:14.720090 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:56:14.720068 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-2fd54" Jul 06 18:56:14.772058 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:56:14.772023 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k8fbq_bc7057bd-88da-48af-ba32-e6efdc6e8362/konnectivity-agent/0.log" limit=1024 Jul 06 18:56:14.883005 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:56:14.882969 2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 06 18:56:14.923703 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:56:14.923669 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k8fbq_bc7057bd-88da-48af-ba32-e6efdc6e8362/konnectivity-agent/0.log" limit=1024 Jul 06 18:56:14.925646 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:56:14.925623 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Jul 06 18:56:14.928177 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:56:14.928149 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 06 18:56:14.928313 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:56:14.928212 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-web\"" Jul 06 18:56:14.928459 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:56:14.928426 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls-assets-0\"" Jul 06 18:56:14.928459 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:56:14.928449 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-cluster-tls-config\"" Jul 06 18:56:14.928853 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:56:14.928808 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-metric\"" Jul 06 18:56:14.929576 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:56:14.929403 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-generated\"" Jul 06 18:56:14.929576 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:56:14.929415 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"alertmanager-trusted-ca-bundle\"" Jul 06 18:56:14.929576 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:56:14.929430 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-web-config\"" Jul 06 18:56:14.929576 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:56:14.929447 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls\"" Jul 06 18:56:14.929576 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:56:14.929408 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy\"" Jul 06 18:56:14.929576 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:56:14.929422 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-dockercfg-m4nh6\"" Jul 06 18:56:14.964063 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:56:14.963998 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/06b795c7-2ff5-4057-ab00-489a7cf9457b-tls-assets\") pod \"alertmanager-main-0\" (UID: \"06b795c7-2ff5-4057-ab00-489a7cf9457b\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 18:56:14.964063 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:56:14.964038 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/06b795c7-2ff5-4057-ab00-489a7cf9457b-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"06b795c7-2ff5-4057-ab00-489a7cf9457b\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 18:56:14.964226 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:56:14.964073 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/06b795c7-2ff5-4057-ab00-489a7cf9457b-config-volume\") pod \"alertmanager-main-0\" (UID: \"06b795c7-2ff5-4057-ab00-489a7cf9457b\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 18:56:14.964226 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:56:14.964096 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/06b795c7-2ff5-4057-ab00-489a7cf9457b-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"06b795c7-2ff5-4057-ab00-489a7cf9457b\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 18:56:14.964226 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:56:14.964119 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sp6c4\" (UniqueName: \"kubernetes.io/projected/06b795c7-2ff5-4057-ab00-489a7cf9457b-kube-api-access-sp6c4\") pod \"alertmanager-main-0\" (UID: \"06b795c7-2ff5-4057-ab00-489a7cf9457b\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 18:56:14.964226 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:56:14.964182 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/06b795c7-2ff5-4057-ab00-489a7cf9457b-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"06b795c7-2ff5-4057-ab00-489a7cf9457b\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 18:56:14.964226 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:56:14.964209 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/06b795c7-2ff5-4057-ab00-489a7cf9457b-config-out\") pod \"alertmanager-main-0\" (UID: \"06b795c7-2ff5-4057-ab00-489a7cf9457b\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 18:56:14.964459 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:56:14.964237 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/06b795c7-2ff5-4057-ab00-489a7cf9457b-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"06b795c7-2ff5-4057-ab00-489a7cf9457b\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 18:56:14.964459 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:56:14.964273 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/06b795c7-2ff5-4057-ab00-489a7cf9457b-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"06b795c7-2ff5-4057-ab00-489a7cf9457b\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 18:56:14.964459 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:56:14.964306 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/06b795c7-2ff5-4057-ab00-489a7cf9457b-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"06b795c7-2ff5-4057-ab00-489a7cf9457b\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 18:56:14.964459 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:56:14.964347 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/06b795c7-2ff5-4057-ab00-489a7cf9457b-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"06b795c7-2ff5-4057-ab00-489a7cf9457b\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 18:56:14.964459 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:56:14.964378 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/06b795c7-2ff5-4057-ab00-489a7cf9457b-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"06b795c7-2ff5-4057-ab00-489a7cf9457b\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 18:56:14.964459 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:56:14.964413 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/06b795c7-2ff5-4057-ab00-489a7cf9457b-web-config\") pod \"alertmanager-main-0\" (UID: \"06b795c7-2ff5-4057-ab00-489a7cf9457b\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 18:56:15.065378 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:56:15.065351 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/06b795c7-2ff5-4057-ab00-489a7cf9457b-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"06b795c7-2ff5-4057-ab00-489a7cf9457b\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 18:56:15.065533 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:56:15.065387 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/06b795c7-2ff5-4057-ab00-489a7cf9457b-config-volume\") pod \"alertmanager-main-0\" (UID: \"06b795c7-2ff5-4057-ab00-489a7cf9457b\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 18:56:15.065533 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:56:15.065411 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/06b795c7-2ff5-4057-ab00-489a7cf9457b-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"06b795c7-2ff5-4057-ab00-489a7cf9457b\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 18:56:15.065666 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:56:15.065528 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sp6c4\" (UniqueName: \"kubernetes.io/projected/06b795c7-2ff5-4057-ab00-489a7cf9457b-kube-api-access-sp6c4\") pod \"alertmanager-main-0\" (UID: \"06b795c7-2ff5-4057-ab00-489a7cf9457b\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 18:56:15.065666 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:56:15.065628 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/06b795c7-2ff5-4057-ab00-489a7cf9457b-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"06b795c7-2ff5-4057-ab00-489a7cf9457b\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 18:56:15.065666 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:56:15.065661 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/06b795c7-2ff5-4057-ab00-489a7cf9457b-config-out\") pod \"alertmanager-main-0\" (UID: \"06b795c7-2ff5-4057-ab00-489a7cf9457b\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 18:56:15.065812 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:56:15.065692 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/06b795c7-2ff5-4057-ab00-489a7cf9457b-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"06b795c7-2ff5-4057-ab00-489a7cf9457b\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 18:56:15.065812 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:56:15.065729 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/06b795c7-2ff5-4057-ab00-489a7cf9457b-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"06b795c7-2ff5-4057-ab00-489a7cf9457b\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 18:56:15.065812 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:56:15.065761 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/06b795c7-2ff5-4057-ab00-489a7cf9457b-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"06b795c7-2ff5-4057-ab00-489a7cf9457b\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 18:56:15.065812 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:56:15.065804 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/06b795c7-2ff5-4057-ab00-489a7cf9457b-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"06b795c7-2ff5-4057-ab00-489a7cf9457b\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 18:56:15.066047 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:56:15.065840 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/06b795c7-2ff5-4057-ab00-489a7cf9457b-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"06b795c7-2ff5-4057-ab00-489a7cf9457b\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 18:56:15.066047 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:56:15.065878 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/06b795c7-2ff5-4057-ab00-489a7cf9457b-web-config\") pod \"alertmanager-main-0\" (UID: \"06b795c7-2ff5-4057-ab00-489a7cf9457b\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 18:56:15.066047 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:56:15.065935 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/06b795c7-2ff5-4057-ab00-489a7cf9457b-tls-assets\") pod \"alertmanager-main-0\" (UID: \"06b795c7-2ff5-4057-ab00-489a7cf9457b\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 18:56:15.067031 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:56:15.067001 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/06b795c7-2ff5-4057-ab00-489a7cf9457b-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"06b795c7-2ff5-4057-ab00-489a7cf9457b\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 18:56:15.067151 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:56:15.067126 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/06b795c7-2ff5-4057-ab00-489a7cf9457b-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"06b795c7-2ff5-4057-ab00-489a7cf9457b\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 18:56:15.068080 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:56:15.068023 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/06b795c7-2ff5-4057-ab00-489a7cf9457b-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"06b795c7-2ff5-4057-ab00-489a7cf9457b\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 18:56:15.068592 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:56:15.068559 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/06b795c7-2ff5-4057-ab00-489a7cf9457b-config-volume\") pod \"alertmanager-main-0\" (UID: \"06b795c7-2ff5-4057-ab00-489a7cf9457b\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 18:56:15.068888 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:56:15.068854 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/06b795c7-2ff5-4057-ab00-489a7cf9457b-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"06b795c7-2ff5-4057-ab00-489a7cf9457b\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 18:56:15.069692 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:56:15.069667 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/06b795c7-2ff5-4057-ab00-489a7cf9457b-config-out\") pod \"alertmanager-main-0\" (UID: \"06b795c7-2ff5-4057-ab00-489a7cf9457b\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 18:56:15.069823 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:56:15.069802 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/06b795c7-2ff5-4057-ab00-489a7cf9457b-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"06b795c7-2ff5-4057-ab00-489a7cf9457b\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 18:56:15.069888 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:56:15.069806 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/06b795c7-2ff5-4057-ab00-489a7cf9457b-web-config\") pod \"alertmanager-main-0\" (UID: \"06b795c7-2ff5-4057-ab00-489a7cf9457b\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 18:56:15.070838 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:56:15.070808 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/06b795c7-2ff5-4057-ab00-489a7cf9457b-tls-assets\") pod \"alertmanager-main-0\" (UID: \"06b795c7-2ff5-4057-ab00-489a7cf9457b\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 18:56:15.071044 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:56:15.070969 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/06b795c7-2ff5-4057-ab00-489a7cf9457b-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"06b795c7-2ff5-4057-ab00-489a7cf9457b\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 18:56:15.071263 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:56:15.071238 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/06b795c7-2ff5-4057-ab00-489a7cf9457b-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"06b795c7-2ff5-4057-ab00-489a7cf9457b\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 18:56:15.071345 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:56:15.071330 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/06b795c7-2ff5-4057-ab00-489a7cf9457b-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"06b795c7-2ff5-4057-ab00-489a7cf9457b\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 18:56:15.075496 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:56:15.075477 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-sp6c4\" (UniqueName: \"kubernetes.io/projected/06b795c7-2ff5-4057-ab00-489a7cf9457b-kube-api-access-sp6c4\") pod \"alertmanager-main-0\" (UID: \"06b795c7-2ff5-4057-ab00-489a7cf9457b\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 18:56:15.114931 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:56:15.114894 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k8fbq_bc7057bd-88da-48af-ba32-e6efdc6e8362/konnectivity-agent/0.log" limit=1024 Jul 06 18:56:15.237276 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:56:15.237195 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Jul 06 18:56:15.564162 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:56:15.564079 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k8fbq_bc7057bd-88da-48af-ba32-e6efdc6e8362/konnectivity-agent/0.log" limit=1024 Jul 06 18:56:15.661032 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:56:15.660977 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k8fbq_bc7057bd-88da-48af-ba32-e6efdc6e8362/konnectivity-agent/0.log" limit=1024 Jul 06 18:56:15.762366 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:56:15.762335 2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/thanos-querier-7d85fbb9b-8c6ms"] Jul 06 18:56:15.883718 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:56:15.883637 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/thanos-querier-7d85fbb9b-8c6ms"] Jul 06 18:56:15.883850 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:56:15.883737 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/thanos-querier-7d85fbb9b-8c6ms" Jul 06 18:56:15.886490 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:56:15.886463 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-dockercfg-4cmmr\"" Jul 06 18:56:15.886625 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:56:15.886539 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-grpc-tls-dtdi590i6ltal\"" Jul 06 18:56:15.888342 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:56:15.888321 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-metrics\"" Jul 06 18:56:15.888456 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:56:15.888389 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy\"" Jul 06 18:56:15.888561 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:56:15.888546 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-rules\"" Jul 06 18:56:15.888664 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:56:15.888586 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-web\"" Jul 06 18:56:15.888664 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:56:15.888551 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-tls\"" Jul 06 18:56:15.974683 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:56:15.974653 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/3c3710b4-98b2-42cf-9425-5a1b35e7f794-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-7d85fbb9b-8c6ms\" (UID: \"3c3710b4-98b2-42cf-9425-5a1b35e7f794\") " pod="openshift-monitoring/thanos-querier-7d85fbb9b-8c6ms" Jul 06 18:56:15.974794 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:56:15.974694 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rzffc\" (UniqueName: \"kubernetes.io/projected/3c3710b4-98b2-42cf-9425-5a1b35e7f794-kube-api-access-rzffc\") pod \"thanos-querier-7d85fbb9b-8c6ms\" (UID: \"3c3710b4-98b2-42cf-9425-5a1b35e7f794\") " pod="openshift-monitoring/thanos-querier-7d85fbb9b-8c6ms" Jul 06 18:56:15.974794 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:56:15.974722 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/3c3710b4-98b2-42cf-9425-5a1b35e7f794-secret-thanos-querier-tls\") pod \"thanos-querier-7d85fbb9b-8c6ms\" (UID: \"3c3710b4-98b2-42cf-9425-5a1b35e7f794\") " pod="openshift-monitoring/thanos-querier-7d85fbb9b-8c6ms" Jul 06 18:56:15.974794 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:56:15.974776 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/3c3710b4-98b2-42cf-9425-5a1b35e7f794-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-7d85fbb9b-8c6ms\" (UID: \"3c3710b4-98b2-42cf-9425-5a1b35e7f794\") " pod="openshift-monitoring/thanos-querier-7d85fbb9b-8c6ms" Jul 06 18:56:15.974919 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:56:15.974866 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/3c3710b4-98b2-42cf-9425-5a1b35e7f794-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-7d85fbb9b-8c6ms\" (UID: \"3c3710b4-98b2-42cf-9425-5a1b35e7f794\") " pod="openshift-monitoring/thanos-querier-7d85fbb9b-8c6ms" Jul 06 18:56:15.974919 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:56:15.974895 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/3c3710b4-98b2-42cf-9425-5a1b35e7f794-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-7d85fbb9b-8c6ms\" (UID: \"3c3710b4-98b2-42cf-9425-5a1b35e7f794\") " pod="openshift-monitoring/thanos-querier-7d85fbb9b-8c6ms" Jul 06 18:56:15.974919 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:56:15.974913 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/3c3710b4-98b2-42cf-9425-5a1b35e7f794-secret-grpc-tls\") pod \"thanos-querier-7d85fbb9b-8c6ms\" (UID: \"3c3710b4-98b2-42cf-9425-5a1b35e7f794\") " pod="openshift-monitoring/thanos-querier-7d85fbb9b-8c6ms" Jul 06 18:56:15.975100 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:56:15.974933 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/3c3710b4-98b2-42cf-9425-5a1b35e7f794-metrics-client-ca\") pod \"thanos-querier-7d85fbb9b-8c6ms\" (UID: \"3c3710b4-98b2-42cf-9425-5a1b35e7f794\") " pod="openshift-monitoring/thanos-querier-7d85fbb9b-8c6ms" Jul 06 18:56:16.075639 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:56:16.075601 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/3c3710b4-98b2-42cf-9425-5a1b35e7f794-secret-grpc-tls\") pod \"thanos-querier-7d85fbb9b-8c6ms\" (UID: \"3c3710b4-98b2-42cf-9425-5a1b35e7f794\") " pod="openshift-monitoring/thanos-querier-7d85fbb9b-8c6ms" Jul 06 18:56:16.075727 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:56:16.075645 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/3c3710b4-98b2-42cf-9425-5a1b35e7f794-metrics-client-ca\") pod \"thanos-querier-7d85fbb9b-8c6ms\" (UID: \"3c3710b4-98b2-42cf-9425-5a1b35e7f794\") " pod="openshift-monitoring/thanos-querier-7d85fbb9b-8c6ms" Jul 06 18:56:16.075727 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:56:16.075683 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/3c3710b4-98b2-42cf-9425-5a1b35e7f794-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-7d85fbb9b-8c6ms\" (UID: \"3c3710b4-98b2-42cf-9425-5a1b35e7f794\") " pod="openshift-monitoring/thanos-querier-7d85fbb9b-8c6ms" Jul 06 18:56:16.075727 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:56:16.075701 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rzffc\" (UniqueName: \"kubernetes.io/projected/3c3710b4-98b2-42cf-9425-5a1b35e7f794-kube-api-access-rzffc\") pod \"thanos-querier-7d85fbb9b-8c6ms\" (UID: \"3c3710b4-98b2-42cf-9425-5a1b35e7f794\") " pod="openshift-monitoring/thanos-querier-7d85fbb9b-8c6ms" Jul 06 18:56:16.075727 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:56:16.075718 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/3c3710b4-98b2-42cf-9425-5a1b35e7f794-secret-thanos-querier-tls\") pod \"thanos-querier-7d85fbb9b-8c6ms\" (UID: \"3c3710b4-98b2-42cf-9425-5a1b35e7f794\") " pod="openshift-monitoring/thanos-querier-7d85fbb9b-8c6ms" Jul 06 18:56:16.075866 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:56:16.075745 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/3c3710b4-98b2-42cf-9425-5a1b35e7f794-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-7d85fbb9b-8c6ms\" (UID: \"3c3710b4-98b2-42cf-9425-5a1b35e7f794\") " pod="openshift-monitoring/thanos-querier-7d85fbb9b-8c6ms" Jul 06 18:56:16.075937 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:56:16.075905 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/3c3710b4-98b2-42cf-9425-5a1b35e7f794-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-7d85fbb9b-8c6ms\" (UID: \"3c3710b4-98b2-42cf-9425-5a1b35e7f794\") " pod="openshift-monitoring/thanos-querier-7d85fbb9b-8c6ms" Jul 06 18:56:16.076010 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:56:16.075957 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/3c3710b4-98b2-42cf-9425-5a1b35e7f794-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-7d85fbb9b-8c6ms\" (UID: \"3c3710b4-98b2-42cf-9425-5a1b35e7f794\") " pod="openshift-monitoring/thanos-querier-7d85fbb9b-8c6ms" Jul 06 18:56:16.076512 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:56:16.076485 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/3c3710b4-98b2-42cf-9425-5a1b35e7f794-metrics-client-ca\") pod \"thanos-querier-7d85fbb9b-8c6ms\" (UID: \"3c3710b4-98b2-42cf-9425-5a1b35e7f794\") " pod="openshift-monitoring/thanos-querier-7d85fbb9b-8c6ms" Jul 06 18:56:16.078214 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:56:16.078189 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/3c3710b4-98b2-42cf-9425-5a1b35e7f794-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-7d85fbb9b-8c6ms\" (UID: \"3c3710b4-98b2-42cf-9425-5a1b35e7f794\") " pod="openshift-monitoring/thanos-querier-7d85fbb9b-8c6ms" Jul 06 18:56:16.078423 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:56:16.078399 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/3c3710b4-98b2-42cf-9425-5a1b35e7f794-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-7d85fbb9b-8c6ms\" (UID: \"3c3710b4-98b2-42cf-9425-5a1b35e7f794\") " pod="openshift-monitoring/thanos-querier-7d85fbb9b-8c6ms" Jul 06 18:56:16.078580 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:56:16.078556 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/3c3710b4-98b2-42cf-9425-5a1b35e7f794-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-7d85fbb9b-8c6ms\" (UID: \"3c3710b4-98b2-42cf-9425-5a1b35e7f794\") " pod="openshift-monitoring/thanos-querier-7d85fbb9b-8c6ms" Jul 06 18:56:16.078685 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:56:16.078557 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/3c3710b4-98b2-42cf-9425-5a1b35e7f794-secret-thanos-querier-tls\") pod \"thanos-querier-7d85fbb9b-8c6ms\" (UID: \"3c3710b4-98b2-42cf-9425-5a1b35e7f794\") " pod="openshift-monitoring/thanos-querier-7d85fbb9b-8c6ms" Jul 06 18:56:16.078733 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:56:16.078693 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/3c3710b4-98b2-42cf-9425-5a1b35e7f794-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-7d85fbb9b-8c6ms\" (UID: \"3c3710b4-98b2-42cf-9425-5a1b35e7f794\") " pod="openshift-monitoring/thanos-querier-7d85fbb9b-8c6ms" Jul 06 18:56:16.078733 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:56:16.078705 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/3c3710b4-98b2-42cf-9425-5a1b35e7f794-secret-grpc-tls\") pod \"thanos-querier-7d85fbb9b-8c6ms\" (UID: \"3c3710b4-98b2-42cf-9425-5a1b35e7f794\") " pod="openshift-monitoring/thanos-querier-7d85fbb9b-8c6ms" Jul 06 18:56:16.084039 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:56:16.084020 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-rzffc\" (UniqueName: \"kubernetes.io/projected/3c3710b4-98b2-42cf-9425-5a1b35e7f794-kube-api-access-rzffc\") pod \"thanos-querier-7d85fbb9b-8c6ms\" (UID: \"3c3710b4-98b2-42cf-9425-5a1b35e7f794\") " pod="openshift-monitoring/thanos-querier-7d85fbb9b-8c6ms" Jul 06 18:56:16.192729 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:56:16.192697 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/thanos-querier-7d85fbb9b-8c6ms" Jul 06 18:56:16.211440 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:56:16.211416 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k8fbq_bc7057bd-88da-48af-ba32-e6efdc6e8362/konnectivity-agent/0.log" limit=1024 Jul 06 18:56:16.311229 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:56:16.311187 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k8fbq_bc7057bd-88da-48af-ba32-e6efdc6e8362/konnectivity-agent/0.log" limit=1024 Jul 06 18:56:16.745634 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:56:16.745582 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k8fbq_bc7057bd-88da-48af-ba32-e6efdc6e8362/konnectivity-agent/0.log" limit=1024 Jul 06 18:56:16.847256 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:56:16.847184 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k8fbq_bc7057bd-88da-48af-ba32-e6efdc6e8362/konnectivity-agent/0.log" limit=1024 Jul 06 18:56:17.142884 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:56:17.142329 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k8fbq_bc7057bd-88da-48af-ba32-e6efdc6e8362/konnectivity-agent/0.log" limit=1024 Jul 06 18:56:17.232525 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:56:17.232416 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k8fbq_bc7057bd-88da-48af-ba32-e6efdc6e8362/konnectivity-agent/0.log" limit=1024 Jul 06 18:56:17.257260 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:56:17.252512 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-2fd54" event={"ID":"b3ab8883-a662-4f63-9429-13b7f1c3a3b3","Type":"ContainerStarted","Data":"3e6beafef14ea7691a2493240eac3b1d317d2fcdb676a9f989859e650b216891"} Jul 06 18:56:17.257260 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:56:17.256773 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/kube-state-metrics-7c9dbb8b7d-lrhkk"] Jul 06 18:56:17.260861 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:56:17.259337 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-mkrnp" event={"ID":"5f316223-4aea-420d-be1b-c2921c1f994f","Type":"ContainerStarted","Data":"9bb3427a3a47d582d5276705b44b32327ddf05e7d885da3cc51ce2ad9f793307"} Jul 06 18:56:17.268419 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:56:17.268188 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-6c6c7cc65c-tp9k6"] Jul 06 18:56:17.271243 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:56:17.271048 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-22lv8" event={"ID":"60870efe-6ea8-49ed-a879-7a1d7dee12d0","Type":"ContainerStarted","Data":"bcb991de2bc91f5330bf85e0b69c726529fbe5cecbc783cf12ad85c25267450d"} Jul 06 18:56:17.275369 ip-10-0-130-59 kubenswrapper[2570]: W0706 18:56:17.274541 2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3afaab40_a812_4cd4_ab65_bc0da82cb738.slice/crio-21824889c72b262e9906b59165d050cddc387811e0b5e0bff1689a0eb9c6c39f WatchSource:0}: Error finding container 21824889c72b262e9906b59165d050cddc387811e0b5e0bff1689a0eb9c6c39f: Status 404 returned error can't find the container with id 21824889c72b262e9906b59165d050cddc387811e0b5e0bff1689a0eb9c6c39f Jul 06 18:56:17.285970 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:56:17.278639 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-dk4j7" event={"ID":"c68e3e9c-2b68-4ed9-99c5-f76eb52b09c3","Type":"ContainerStarted","Data":"b569421866a2dcffad681dd7fe5d3dbb713591b5575c4d545bdf9e1d4a075b03"} Jul 06 18:56:17.295763 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:56:17.295002 2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-22lv8" podStartSLOduration=1.544214439 podStartE2EDuration="7.294983103s" totalImagesPullingTime="5.750768664s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 18:56:10 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 18:56:17.293455738 +0000 UTC m=+56.848476910" watchObservedRunningTime="2026-07-06 18:56:17.294983103 +0000 UTC m=+56.850004270" Jul 06 18:56:17.330049 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:56:17.330002 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 06 18:56:17.332733 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:56:17.332660 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/openshift-state-metrics-54cf96dfcf-h62zc"] Jul 06 18:56:17.335101 ip-10-0-130-59 kubenswrapper[2570]: W0706 18:56:17.335073 2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod06b795c7_2ff5_4057_ab00_489a7cf9457b.slice/crio-3919886510a897a05da82605824404c74654d32b8b1f3ac98d55664d6e1e2f01 WatchSource:0}: Error finding container 3919886510a897a05da82605824404c74654d32b8b1f3ac98d55664d6e1e2f01: Status 404 returned error can't find the container with id 3919886510a897a05da82605824404c74654d32b8b1f3ac98d55664d6e1e2f01 Jul 06 18:56:17.338797 ip-10-0-130-59 kubenswrapper[2570]: W0706 18:56:17.338757 2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod910d0e65_b069_41b4_a8d5_2881341454e7.slice/crio-fe67b578e048a077e1c3bb1228ce222905d084a1ebfb9a6450bf496c412f7b8b WatchSource:0}: Error finding container fe67b578e048a077e1c3bb1228ce222905d084a1ebfb9a6450bf496c412f7b8b: Status 404 returned error can't find the container with id fe67b578e048a077e1c3bb1228ce222905d084a1ebfb9a6450bf496c412f7b8b Jul 06 18:56:17.469655 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:56:17.469628 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/thanos-querier-7d85fbb9b-8c6ms"] Jul 06 18:56:17.473436 ip-10-0-130-59 kubenswrapper[2570]: W0706 18:56:17.473411 2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3c3710b4_98b2_42cf_9425_5a1b35e7f794.slice/crio-6c315b17cb32e1d54006199f2aede5a1a9506a56b451dff7b99036f730b80b04 WatchSource:0}: Error finding container 6c315b17cb32e1d54006199f2aede5a1a9506a56b451dff7b99036f730b80b04: Status 404 returned error can't find the container with id 6c315b17cb32e1d54006199f2aede5a1a9506a56b451dff7b99036f730b80b04 Jul 06 18:56:18.039871 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:56:18.039836 2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/metrics-server-59d88b459-jqgzb"] Jul 06 18:56:18.043412 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:56:18.043390 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/metrics-server-59d88b459-jqgzb" Jul 06 18:56:18.046008 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:56:18.045985 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-client-certs\"" Jul 06 18:56:18.046269 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:56:18.046254 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kubelet-serving-ca-bundle\"" Jul 06 18:56:18.046383 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:56:18.046254 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-dockercfg-6hsnr\"" Jul 06 18:56:18.046383 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:56:18.046323 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-server-audit-profiles\"" Jul 06 18:56:18.047328 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:56:18.047307 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-tls\"" Jul 06 18:56:18.047440 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:56:18.047392 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-32tm9qves39fr\"" Jul 06 18:56:18.056046 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:56:18.056023 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/metrics-server-59d88b459-jqgzb"] Jul 06 18:56:18.096248 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:56:18.096219 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/f575cfd3-dd2e-4ecf-ab0f-1860884038a3-metrics-server-audit-profiles\") pod \"metrics-server-59d88b459-jqgzb\" (UID: \"f575cfd3-dd2e-4ecf-ab0f-1860884038a3\") " pod="openshift-monitoring/metrics-server-59d88b459-jqgzb" Jul 06 18:56:18.096364 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:56:18.096262 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f575cfd3-dd2e-4ecf-ab0f-1860884038a3-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-59d88b459-jqgzb\" (UID: \"f575cfd3-dd2e-4ecf-ab0f-1860884038a3\") " pod="openshift-monitoring/metrics-server-59d88b459-jqgzb" Jul 06 18:56:18.096364 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:56:18.096330 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/f575cfd3-dd2e-4ecf-ab0f-1860884038a3-audit-log\") pod \"metrics-server-59d88b459-jqgzb\" (UID: \"f575cfd3-dd2e-4ecf-ab0f-1860884038a3\") " pod="openshift-monitoring/metrics-server-59d88b459-jqgzb" Jul 06 18:56:18.096490 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:56:18.096409 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/f575cfd3-dd2e-4ecf-ab0f-1860884038a3-client-ca-bundle\") pod \"metrics-server-59d88b459-jqgzb\" (UID: \"f575cfd3-dd2e-4ecf-ab0f-1860884038a3\") " pod="openshift-monitoring/metrics-server-59d88b459-jqgzb" Jul 06 18:56:18.096490 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:56:18.096472 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kqghl\" (UniqueName: \"kubernetes.io/projected/f575cfd3-dd2e-4ecf-ab0f-1860884038a3-kube-api-access-kqghl\") pod \"metrics-server-59d88b459-jqgzb\" (UID: \"f575cfd3-dd2e-4ecf-ab0f-1860884038a3\") " pod="openshift-monitoring/metrics-server-59d88b459-jqgzb" Jul 06 18:56:18.096564 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:56:18.096518 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/f575cfd3-dd2e-4ecf-ab0f-1860884038a3-secret-metrics-server-client-certs\") pod \"metrics-server-59d88b459-jqgzb\" (UID: \"f575cfd3-dd2e-4ecf-ab0f-1860884038a3\") " pod="openshift-monitoring/metrics-server-59d88b459-jqgzb" Jul 06 18:56:18.096564 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:56:18.096536 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/f575cfd3-dd2e-4ecf-ab0f-1860884038a3-secret-metrics-server-tls\") pod \"metrics-server-59d88b459-jqgzb\" (UID: \"f575cfd3-dd2e-4ecf-ab0f-1860884038a3\") " pod="openshift-monitoring/metrics-server-59d88b459-jqgzb" Jul 06 18:56:18.197645 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:56:18.197601 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/f575cfd3-dd2e-4ecf-ab0f-1860884038a3-audit-log\") pod \"metrics-server-59d88b459-jqgzb\" (UID: \"f575cfd3-dd2e-4ecf-ab0f-1860884038a3\") " pod="openshift-monitoring/metrics-server-59d88b459-jqgzb" Jul 06 18:56:18.197750 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:56:18.197692 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/f575cfd3-dd2e-4ecf-ab0f-1860884038a3-client-ca-bundle\") pod \"metrics-server-59d88b459-jqgzb\" (UID: \"f575cfd3-dd2e-4ecf-ab0f-1860884038a3\") " pod="openshift-monitoring/metrics-server-59d88b459-jqgzb" Jul 06 18:56:18.197750 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:56:18.197733 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kqghl\" (UniqueName: \"kubernetes.io/projected/f575cfd3-dd2e-4ecf-ab0f-1860884038a3-kube-api-access-kqghl\") pod \"metrics-server-59d88b459-jqgzb\" (UID: \"f575cfd3-dd2e-4ecf-ab0f-1860884038a3\") " pod="openshift-monitoring/metrics-server-59d88b459-jqgzb" Jul 06 18:56:18.197866 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:56:18.197773 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/f575cfd3-dd2e-4ecf-ab0f-1860884038a3-secret-metrics-server-client-certs\") pod \"metrics-server-59d88b459-jqgzb\" (UID: \"f575cfd3-dd2e-4ecf-ab0f-1860884038a3\") " pod="openshift-monitoring/metrics-server-59d88b459-jqgzb" Jul 06 18:56:18.197866 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:56:18.197825 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/f575cfd3-dd2e-4ecf-ab0f-1860884038a3-secret-metrics-server-tls\") pod \"metrics-server-59d88b459-jqgzb\" (UID: \"f575cfd3-dd2e-4ecf-ab0f-1860884038a3\") " pod="openshift-monitoring/metrics-server-59d88b459-jqgzb" Jul 06 18:56:18.197954 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:56:18.197886 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/f575cfd3-dd2e-4ecf-ab0f-1860884038a3-metrics-server-audit-profiles\") pod \"metrics-server-59d88b459-jqgzb\" (UID: \"f575cfd3-dd2e-4ecf-ab0f-1860884038a3\") " pod="openshift-monitoring/metrics-server-59d88b459-jqgzb" Jul 06 18:56:18.197954 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:56:18.197935 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f575cfd3-dd2e-4ecf-ab0f-1860884038a3-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-59d88b459-jqgzb\" (UID: \"f575cfd3-dd2e-4ecf-ab0f-1860884038a3\") " pod="openshift-monitoring/metrics-server-59d88b459-jqgzb" Jul 06 18:56:18.198670 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:56:18.198208 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/f575cfd3-dd2e-4ecf-ab0f-1860884038a3-audit-log\") pod \"metrics-server-59d88b459-jqgzb\" (UID: \"f575cfd3-dd2e-4ecf-ab0f-1860884038a3\") " pod="openshift-monitoring/metrics-server-59d88b459-jqgzb" Jul 06 18:56:18.198670 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:56:18.198574 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f575cfd3-dd2e-4ecf-ab0f-1860884038a3-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-59d88b459-jqgzb\" (UID: \"f575cfd3-dd2e-4ecf-ab0f-1860884038a3\") " pod="openshift-monitoring/metrics-server-59d88b459-jqgzb" Jul 06 18:56:18.199892 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:56:18.199869 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/f575cfd3-dd2e-4ecf-ab0f-1860884038a3-metrics-server-audit-profiles\") pod \"metrics-server-59d88b459-jqgzb\" (UID: \"f575cfd3-dd2e-4ecf-ab0f-1860884038a3\") " pod="openshift-monitoring/metrics-server-59d88b459-jqgzb" Jul 06 18:56:18.201481 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:56:18.201440 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/f575cfd3-dd2e-4ecf-ab0f-1860884038a3-secret-metrics-server-tls\") pod \"metrics-server-59d88b459-jqgzb\" (UID: \"f575cfd3-dd2e-4ecf-ab0f-1860884038a3\") " pod="openshift-monitoring/metrics-server-59d88b459-jqgzb" Jul 06 18:56:18.202787 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:56:18.202687 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/f575cfd3-dd2e-4ecf-ab0f-1860884038a3-client-ca-bundle\") pod \"metrics-server-59d88b459-jqgzb\" (UID: \"f575cfd3-dd2e-4ecf-ab0f-1860884038a3\") " pod="openshift-monitoring/metrics-server-59d88b459-jqgzb" Jul 06 18:56:18.204779 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:56:18.204729 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/f575cfd3-dd2e-4ecf-ab0f-1860884038a3-secret-metrics-server-client-certs\") pod \"metrics-server-59d88b459-jqgzb\" (UID: \"f575cfd3-dd2e-4ecf-ab0f-1860884038a3\") " pod="openshift-monitoring/metrics-server-59d88b459-jqgzb" Jul 06 18:56:18.208185 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:56:18.207980 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-kqghl\" (UniqueName: \"kubernetes.io/projected/f575cfd3-dd2e-4ecf-ab0f-1860884038a3-kube-api-access-kqghl\") pod \"metrics-server-59d88b459-jqgzb\" (UID: \"f575cfd3-dd2e-4ecf-ab0f-1860884038a3\") " pod="openshift-monitoring/metrics-server-59d88b459-jqgzb" Jul 06 18:56:18.285140 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:56:18.285077 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-7d85fbb9b-8c6ms" event={"ID":"3c3710b4-98b2-42cf-9425-5a1b35e7f794","Type":"ContainerStarted","Data":"6c315b17cb32e1d54006199f2aede5a1a9506a56b451dff7b99036f730b80b04"} Jul 06 18:56:18.286684 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:56:18.286653 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"06b795c7-2ff5-4057-ab00-489a7cf9457b","Type":"ContainerStarted","Data":"3919886510a897a05da82605824404c74654d32b8b1f3ac98d55664d6e1e2f01"} Jul 06 18:56:18.289251 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:56:18.289167 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-dk4j7" event={"ID":"c68e3e9c-2b68-4ed9-99c5-f76eb52b09c3","Type":"ContainerStarted","Data":"4f719d730b30e2a2d07d03f0d3918b2063d2aa388097c42d606bf33c5e29bf16"} Jul 06 18:56:18.289472 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:56:18.289410 2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-dk4j7" Jul 06 18:56:18.291020 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:56:18.290952 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-lrhkk" event={"ID":"30d171a0-0417-4444-8af2-154f0d6c82f9","Type":"ContainerStarted","Data":"62c0f34d9e3f7052644dce736bb955799a21f5a67fcf69b7c32055bcbcb03465"} Jul 06 18:56:18.293100 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:56:18.292799 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-2fd54" event={"ID":"b3ab8883-a662-4f63-9429-13b7f1c3a3b3","Type":"ContainerStarted","Data":"927ff1f9905ab7aab964effd7a96b141396d7eafb5a95133c223aed0f3a6d261"} Jul 06 18:56:18.296131 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:56:18.296016 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-h62zc" event={"ID":"910d0e65-b069-41b4-a8d5-2881341454e7","Type":"ContainerStarted","Data":"69fcb7278ae7fef22d34dd214d0f721a414b07b84edbdf74bdc3b2d0a0465e4e"} Jul 06 18:56:18.296131 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:56:18.296063 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-h62zc" event={"ID":"910d0e65-b069-41b4-a8d5-2881341454e7","Type":"ContainerStarted","Data":"eeee55e94b6a02271b9612dabede5d38c1b4aea6cd3e3ea9051835e520b672f8"} Jul 06 18:56:18.296254 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:56:18.296131 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-h62zc" event={"ID":"910d0e65-b069-41b4-a8d5-2881341454e7","Type":"ContainerStarted","Data":"fe67b578e048a077e1c3bb1228ce222905d084a1ebfb9a6450bf496c412f7b8b"} Jul 06 18:56:18.298070 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:56:18.298046 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6c6c7cc65c-tp9k6" event={"ID":"3afaab40-a812-4cd4-ab65-bc0da82cb738","Type":"ContainerStarted","Data":"21824889c72b262e9906b59165d050cddc387811e0b5e0bff1689a0eb9c6c39f"} Jul 06 18:56:18.301236 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:56:18.301170 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-mkrnp" event={"ID":"5f316223-4aea-420d-be1b-c2921c1f994f","Type":"ContainerStarted","Data":"876cddc1ae18be800686b7baf421c786b4ee67f223b75aa4f6304e9df6d3842d"} Jul 06 18:56:18.354014 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:56:18.353830 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/metrics-server-59d88b459-jqgzb" Jul 06 18:56:18.356379 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:56:18.356326 2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/dns-default-dk4j7" podStartSLOduration=3.002995323 podStartE2EDuration="8.356307998s" totalImagesPullingTime="5.353312675s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 18:56:10 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 18:56:18.327665281 +0000 UTC m=+57.882686454" watchObservedRunningTime="2026-07-06 18:56:18.356307998 +0000 UTC m=+57.911329171" Jul 06 18:56:18.499055 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:56:18.498990 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k8fbq_bc7057bd-88da-48af-ba32-e6efdc6e8362/konnectivity-agent/0.log" limit=1024 Jul 06 18:56:18.506303 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:56:18.505151 2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/monitoring-plugin-d5dcff7cf-c9prf"] Jul 06 18:56:18.507817 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:56:18.507796 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/monitoring-plugin-d5dcff7cf-c9prf" Jul 06 18:56:18.510805 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:56:18.510783 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"monitoring-plugin-dockercfg-kj2p5\"" Jul 06 18:56:18.511169 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:56:18.510991 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"monitoring-plugin-cert\"" Jul 06 18:56:18.520266 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:56:18.520245 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/monitoring-plugin-d5dcff7cf-c9prf"] Jul 06 18:56:18.538376 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:56:18.538318 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/metrics-server-59d88b459-jqgzb"] Jul 06 18:56:18.602768 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:56:18.602534 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kz82h\" (UniqueName: \"kubernetes.io/projected/8c828f23-ece7-439d-9ca2-ef0642e73506-kube-api-access-kz82h\") pod \"monitoring-plugin-d5dcff7cf-c9prf\" (UID: \"8c828f23-ece7-439d-9ca2-ef0642e73506\") " pod="openshift-monitoring/monitoring-plugin-d5dcff7cf-c9prf" Jul 06 18:56:18.602768 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:56:18.602598 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/8c828f23-ece7-439d-9ca2-ef0642e73506-monitoring-plugin-cert\") pod \"monitoring-plugin-d5dcff7cf-c9prf\" (UID: \"8c828f23-ece7-439d-9ca2-ef0642e73506\") " pod="openshift-monitoring/monitoring-plugin-d5dcff7cf-c9prf" Jul 06 18:56:18.616066 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:56:18.615994 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k8fbq_bc7057bd-88da-48af-ba32-e6efdc6e8362/konnectivity-agent/0.log" limit=1024 Jul 06 18:56:18.704904 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:56:18.704007 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kz82h\" (UniqueName: \"kubernetes.io/projected/8c828f23-ece7-439d-9ca2-ef0642e73506-kube-api-access-kz82h\") pod \"monitoring-plugin-d5dcff7cf-c9prf\" (UID: \"8c828f23-ece7-439d-9ca2-ef0642e73506\") " pod="openshift-monitoring/monitoring-plugin-d5dcff7cf-c9prf" Jul 06 18:56:18.704904 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:56:18.704068 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/8c828f23-ece7-439d-9ca2-ef0642e73506-monitoring-plugin-cert\") pod \"monitoring-plugin-d5dcff7cf-c9prf\" (UID: \"8c828f23-ece7-439d-9ca2-ef0642e73506\") " pod="openshift-monitoring/monitoring-plugin-d5dcff7cf-c9prf" Jul 06 18:56:18.710802 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:56:18.710756 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/8c828f23-ece7-439d-9ca2-ef0642e73506-monitoring-plugin-cert\") pod \"monitoring-plugin-d5dcff7cf-c9prf\" (UID: \"8c828f23-ece7-439d-9ca2-ef0642e73506\") " pod="openshift-monitoring/monitoring-plugin-d5dcff7cf-c9prf" Jul 06 18:56:18.715864 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:56:18.715836 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k8fbq_bc7057bd-88da-48af-ba32-e6efdc6e8362/konnectivity-agent/0.log" limit=1024 Jul 06 18:56:18.727874 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:56:18.727805 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-kz82h\" (UniqueName: \"kubernetes.io/projected/8c828f23-ece7-439d-9ca2-ef0642e73506-kube-api-access-kz82h\") pod \"monitoring-plugin-d5dcff7cf-c9prf\" (UID: \"8c828f23-ece7-439d-9ca2-ef0642e73506\") " pod="openshift-monitoring/monitoring-plugin-d5dcff7cf-c9prf" Jul 06 18:56:18.822906 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:56:18.822873 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/monitoring-plugin-d5dcff7cf-c9prf" Jul 06 18:56:19.309182 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:56:19.309056 2570 generic.go:356] "Generic (PLEG): container finished" podID="b3ab8883-a662-4f63-9429-13b7f1c3a3b3" containerID="927ff1f9905ab7aab964effd7a96b141396d7eafb5a95133c223aed0f3a6d261" exitCode=0 Jul 06 18:56:19.309182 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:56:19.309174 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-2fd54" event={"ID":"b3ab8883-a662-4f63-9429-13b7f1c3a3b3","Type":"ContainerDied","Data":"927ff1f9905ab7aab964effd7a96b141396d7eafb5a95133c223aed0f3a6d261"} Jul 06 18:56:19.312659 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:56:19.312591 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/metrics-server-59d88b459-jqgzb" event={"ID":"f575cfd3-dd2e-4ecf-ab0f-1860884038a3","Type":"ContainerStarted","Data":"19dcceb1dc1e22ebf5d0ebd4f71bf2ab31aab7fd550cf0b5c77f7adbf078c863"} Jul 06 18:56:19.666719 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:56:19.666571 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k8fbq_bc7057bd-88da-48af-ba32-e6efdc6e8362/konnectivity-agent/0.log" limit=1024 Jul 06 18:56:19.747292 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:56:19.747259 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k8fbq_bc7057bd-88da-48af-ba32-e6efdc6e8362/konnectivity-agent/0.log" limit=1024 Jul 06 18:56:20.037019 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:56:20.035675 2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jul 06 18:56:20.037521 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:56:20.037451 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k8fbq_bc7057bd-88da-48af-ba32-e6efdc6e8362/konnectivity-agent/0.log" limit=1024 Jul 06 18:56:20.040158 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:56:20.040137 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" Jul 06 18:56:20.043838 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:56:20.043082 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls\"" Jul 06 18:56:20.043838 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:56:20.043274 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-sidecar-tls\"" Jul 06 18:56:20.043838 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:56:20.043280 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-prometheus-http-client-file\"" Jul 06 18:56:20.043838 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:56:20.043423 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-client-certs\"" Jul 06 18:56:20.043838 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:56:20.043592 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-web-config\"" Jul 06 18:56:20.047325 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:56:20.044500 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls-assets-0\"" Jul 06 18:56:20.047325 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:56:20.046777 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"serving-certs-ca-bundle\"" Jul 06 18:56:20.047471 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:56:20.047347 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-2\"" Jul 06 18:56:20.048322 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:56:20.048298 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-kube-rbac-proxy-web\"" Jul 06 18:56:20.048549 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:56:20.048528 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-rbac-proxy\"" Jul 06 18:56:20.049347 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:56:20.049322 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-1\"" Jul 06 18:56:20.050272 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:56:20.050249 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-grpc-tls-bkvq98edilq4q\"" Jul 06 18:56:20.054288 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:56:20.054157 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s\"" Jul 06 18:56:20.054477 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:56:20.054457 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-dockercfg-wkf8m\"" Jul 06 18:56:20.059832 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:56:20.059811 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-0\"" Jul 06 18:56:20.060551 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:56:20.060522 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jul 06 18:56:20.061194 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:56:20.061172 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-trusted-ca-bundle\"" Jul 06 18:56:20.119909 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:56:20.119877 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/45a3639f-79ab-4bb6-8a64-f2b4458f3375-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"45a3639f-79ab-4bb6-8a64-f2b4458f3375\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 18:56:20.120090 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:56:20.119918 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/45a3639f-79ab-4bb6-8a64-f2b4458f3375-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"45a3639f-79ab-4bb6-8a64-f2b4458f3375\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 18:56:20.120090 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:56:20.119945 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/45a3639f-79ab-4bb6-8a64-f2b4458f3375-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"45a3639f-79ab-4bb6-8a64-f2b4458f3375\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 18:56:20.120090 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:56:20.119970 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/45a3639f-79ab-4bb6-8a64-f2b4458f3375-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"45a3639f-79ab-4bb6-8a64-f2b4458f3375\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 18:56:20.120090 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:56:20.120012 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/45a3639f-79ab-4bb6-8a64-f2b4458f3375-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"45a3639f-79ab-4bb6-8a64-f2b4458f3375\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 18:56:20.120090 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:56:20.120037 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/45a3639f-79ab-4bb6-8a64-f2b4458f3375-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"45a3639f-79ab-4bb6-8a64-f2b4458f3375\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 18:56:20.120090 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:56:20.120070 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/45a3639f-79ab-4bb6-8a64-f2b4458f3375-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"45a3639f-79ab-4bb6-8a64-f2b4458f3375\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 18:56:20.120383 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:56:20.120100 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/45a3639f-79ab-4bb6-8a64-f2b4458f3375-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"45a3639f-79ab-4bb6-8a64-f2b4458f3375\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 18:56:20.120383 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:56:20.120130 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/45a3639f-79ab-4bb6-8a64-f2b4458f3375-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"45a3639f-79ab-4bb6-8a64-f2b4458f3375\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 18:56:20.120383 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:56:20.120159 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/45a3639f-79ab-4bb6-8a64-f2b4458f3375-config\") pod \"prometheus-k8s-0\" (UID: \"45a3639f-79ab-4bb6-8a64-f2b4458f3375\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 18:56:20.120383 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:56:20.120212 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/45a3639f-79ab-4bb6-8a64-f2b4458f3375-web-config\") pod \"prometheus-k8s-0\" (UID: \"45a3639f-79ab-4bb6-8a64-f2b4458f3375\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 18:56:20.120383 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:56:20.120240 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/45a3639f-79ab-4bb6-8a64-f2b4458f3375-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"45a3639f-79ab-4bb6-8a64-f2b4458f3375\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 18:56:20.120383 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:56:20.120279 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/45a3639f-79ab-4bb6-8a64-f2b4458f3375-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"45a3639f-79ab-4bb6-8a64-f2b4458f3375\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 18:56:20.120383 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:56:20.120303 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/45a3639f-79ab-4bb6-8a64-f2b4458f3375-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"45a3639f-79ab-4bb6-8a64-f2b4458f3375\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 18:56:20.120383 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:56:20.120334 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/45a3639f-79ab-4bb6-8a64-f2b4458f3375-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"45a3639f-79ab-4bb6-8a64-f2b4458f3375\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 18:56:20.120383 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:56:20.120375 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4zv9t\" (UniqueName: \"kubernetes.io/projected/45a3639f-79ab-4bb6-8a64-f2b4458f3375-kube-api-access-4zv9t\") pod \"prometheus-k8s-0\" (UID: \"45a3639f-79ab-4bb6-8a64-f2b4458f3375\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 18:56:20.120823 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:56:20.120489 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/45a3639f-79ab-4bb6-8a64-f2b4458f3375-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"45a3639f-79ab-4bb6-8a64-f2b4458f3375\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 18:56:20.120823 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:56:20.120520 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/45a3639f-79ab-4bb6-8a64-f2b4458f3375-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"45a3639f-79ab-4bb6-8a64-f2b4458f3375\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 18:56:20.120823 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:56:20.120559 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/45a3639f-79ab-4bb6-8a64-f2b4458f3375-config-out\") pod \"prometheus-k8s-0\" (UID: \"45a3639f-79ab-4bb6-8a64-f2b4458f3375\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 18:56:20.120823 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:56:20.120636 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/45a3639f-79ab-4bb6-8a64-f2b4458f3375-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"45a3639f-79ab-4bb6-8a64-f2b4458f3375\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 18:56:20.221585 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:56:20.221554 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/45a3639f-79ab-4bb6-8a64-f2b4458f3375-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"45a3639f-79ab-4bb6-8a64-f2b4458f3375\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 18:56:20.221585 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:56:20.221587 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/45a3639f-79ab-4bb6-8a64-f2b4458f3375-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"45a3639f-79ab-4bb6-8a64-f2b4458f3375\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 18:56:20.221823 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:56:20.221632 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/45a3639f-79ab-4bb6-8a64-f2b4458f3375-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"45a3639f-79ab-4bb6-8a64-f2b4458f3375\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 18:56:20.221823 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:56:20.221669 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4zv9t\" (UniqueName: \"kubernetes.io/projected/45a3639f-79ab-4bb6-8a64-f2b4458f3375-kube-api-access-4zv9t\") pod \"prometheus-k8s-0\" (UID: \"45a3639f-79ab-4bb6-8a64-f2b4458f3375\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 18:56:20.221823 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:56:20.221702 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/45a3639f-79ab-4bb6-8a64-f2b4458f3375-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"45a3639f-79ab-4bb6-8a64-f2b4458f3375\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 18:56:20.221823 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:56:20.221728 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/45a3639f-79ab-4bb6-8a64-f2b4458f3375-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"45a3639f-79ab-4bb6-8a64-f2b4458f3375\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 18:56:20.221823 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:56:20.221763 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/45a3639f-79ab-4bb6-8a64-f2b4458f3375-config-out\") pod \"prometheus-k8s-0\" (UID: \"45a3639f-79ab-4bb6-8a64-f2b4458f3375\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 18:56:20.221823 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:56:20.221808 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/45a3639f-79ab-4bb6-8a64-f2b4458f3375-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"45a3639f-79ab-4bb6-8a64-f2b4458f3375\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 18:56:20.222095 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:56:20.221839 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/45a3639f-79ab-4bb6-8a64-f2b4458f3375-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"45a3639f-79ab-4bb6-8a64-f2b4458f3375\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 18:56:20.222095 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:56:20.221862 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/45a3639f-79ab-4bb6-8a64-f2b4458f3375-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"45a3639f-79ab-4bb6-8a64-f2b4458f3375\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 18:56:20.222095 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:56:20.221886 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/45a3639f-79ab-4bb6-8a64-f2b4458f3375-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"45a3639f-79ab-4bb6-8a64-f2b4458f3375\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 18:56:20.222095 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:56:20.221916 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/45a3639f-79ab-4bb6-8a64-f2b4458f3375-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"45a3639f-79ab-4bb6-8a64-f2b4458f3375\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 18:56:20.222095 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:56:20.221949 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/45a3639f-79ab-4bb6-8a64-f2b4458f3375-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"45a3639f-79ab-4bb6-8a64-f2b4458f3375\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 18:56:20.222095 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:56:20.221975 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/45a3639f-79ab-4bb6-8a64-f2b4458f3375-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"45a3639f-79ab-4bb6-8a64-f2b4458f3375\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 18:56:20.222095 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:56:20.222008 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/45a3639f-79ab-4bb6-8a64-f2b4458f3375-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"45a3639f-79ab-4bb6-8a64-f2b4458f3375\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 18:56:20.222095 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:56:20.222040 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/45a3639f-79ab-4bb6-8a64-f2b4458f3375-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"45a3639f-79ab-4bb6-8a64-f2b4458f3375\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 18:56:20.222095 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:56:20.222073 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/45a3639f-79ab-4bb6-8a64-f2b4458f3375-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"45a3639f-79ab-4bb6-8a64-f2b4458f3375\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 18:56:20.222512 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:56:20.222106 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/45a3639f-79ab-4bb6-8a64-f2b4458f3375-config\") pod \"prometheus-k8s-0\" (UID: \"45a3639f-79ab-4bb6-8a64-f2b4458f3375\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 18:56:20.222512 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:56:20.222158 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/45a3639f-79ab-4bb6-8a64-f2b4458f3375-web-config\") pod \"prometheus-k8s-0\" (UID: \"45a3639f-79ab-4bb6-8a64-f2b4458f3375\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 18:56:20.222512 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:56:20.222185 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/45a3639f-79ab-4bb6-8a64-f2b4458f3375-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"45a3639f-79ab-4bb6-8a64-f2b4458f3375\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 18:56:20.222694 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:56:20.222543 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/45a3639f-79ab-4bb6-8a64-f2b4458f3375-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"45a3639f-79ab-4bb6-8a64-f2b4458f3375\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 18:56:20.222694 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:56:20.222546 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/45a3639f-79ab-4bb6-8a64-f2b4458f3375-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"45a3639f-79ab-4bb6-8a64-f2b4458f3375\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 18:56:20.223502 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:56:20.223225 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/45a3639f-79ab-4bb6-8a64-f2b4458f3375-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"45a3639f-79ab-4bb6-8a64-f2b4458f3375\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 18:56:20.223502 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:56:20.223286 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/45a3639f-79ab-4bb6-8a64-f2b4458f3375-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"45a3639f-79ab-4bb6-8a64-f2b4458f3375\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 18:56:20.224705 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:56:20.224678 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/45a3639f-79ab-4bb6-8a64-f2b4458f3375-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"45a3639f-79ab-4bb6-8a64-f2b4458f3375\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 18:56:20.225194 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:56:20.225175 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/45a3639f-79ab-4bb6-8a64-f2b4458f3375-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"45a3639f-79ab-4bb6-8a64-f2b4458f3375\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 18:56:20.225416 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:56:20.225386 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/45a3639f-79ab-4bb6-8a64-f2b4458f3375-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"45a3639f-79ab-4bb6-8a64-f2b4458f3375\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 18:56:20.225416 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:56:20.225405 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/45a3639f-79ab-4bb6-8a64-f2b4458f3375-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"45a3639f-79ab-4bb6-8a64-f2b4458f3375\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 18:56:20.226067 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:56:20.225821 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/45a3639f-79ab-4bb6-8a64-f2b4458f3375-config-out\") pod \"prometheus-k8s-0\" (UID: \"45a3639f-79ab-4bb6-8a64-f2b4458f3375\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 18:56:20.226067 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:56:20.225866 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/45a3639f-79ab-4bb6-8a64-f2b4458f3375-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"45a3639f-79ab-4bb6-8a64-f2b4458f3375\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 18:56:20.226067 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:56:20.226029 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/45a3639f-79ab-4bb6-8a64-f2b4458f3375-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"45a3639f-79ab-4bb6-8a64-f2b4458f3375\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 18:56:20.226551 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:56:20.226532 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/45a3639f-79ab-4bb6-8a64-f2b4458f3375-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"45a3639f-79ab-4bb6-8a64-f2b4458f3375\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 18:56:20.227334 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:56:20.227310 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/45a3639f-79ab-4bb6-8a64-f2b4458f3375-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"45a3639f-79ab-4bb6-8a64-f2b4458f3375\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 18:56:20.227477 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:56:20.227445 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/45a3639f-79ab-4bb6-8a64-f2b4458f3375-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"45a3639f-79ab-4bb6-8a64-f2b4458f3375\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 18:56:20.227848 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:56:20.227797 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/45a3639f-79ab-4bb6-8a64-f2b4458f3375-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"45a3639f-79ab-4bb6-8a64-f2b4458f3375\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 18:56:20.228782 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:56:20.228746 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/45a3639f-79ab-4bb6-8a64-f2b4458f3375-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"45a3639f-79ab-4bb6-8a64-f2b4458f3375\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 18:56:20.228782 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:56:20.228746 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/45a3639f-79ab-4bb6-8a64-f2b4458f3375-web-config\") pod \"prometheus-k8s-0\" (UID: \"45a3639f-79ab-4bb6-8a64-f2b4458f3375\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 18:56:20.229092 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:56:20.229069 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/secret/45a3639f-79ab-4bb6-8a64-f2b4458f3375-config\") pod \"prometheus-k8s-0\" (UID: \"45a3639f-79ab-4bb6-8a64-f2b4458f3375\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 18:56:20.229517 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:56:20.229468 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/45a3639f-79ab-4bb6-8a64-f2b4458f3375-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"45a3639f-79ab-4bb6-8a64-f2b4458f3375\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 18:56:20.230954 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:56:20.230935 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-4zv9t\" (UniqueName: \"kubernetes.io/projected/45a3639f-79ab-4bb6-8a64-f2b4458f3375-kube-api-access-4zv9t\") pod \"prometheus-k8s-0\" (UID: \"45a3639f-79ab-4bb6-8a64-f2b4458f3375\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 18:56:20.362043 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:56:20.361969 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" Jul 06 18:56:21.177319 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:56:21.177288 2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-rr76d" Jul 06 18:56:21.301302 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:56:21.301226 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k8fbq_bc7057bd-88da-48af-ba32-e6efdc6e8362/konnectivity-agent/0.log" limit=1024 Jul 06 18:56:21.363966 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:56:21.363928 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k8fbq_bc7057bd-88da-48af-ba32-e6efdc6e8362/konnectivity-agent/0.log" limit=1024 Jul 06 18:56:21.687558 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:56:21.687525 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/monitoring-plugin-d5dcff7cf-c9prf"] Jul 06 18:56:22.181673 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:56:22.181635 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k8fbq_bc7057bd-88da-48af-ba32-e6efdc6e8362/konnectivity-agent/0.log" limit=1024 Jul 06 18:56:22.239764 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:56:22.239730 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k8fbq_bc7057bd-88da-48af-ba32-e6efdc6e8362/konnectivity-agent/0.log" limit=1024 Jul 06 18:56:23.180333 ip-10-0-130-59 kubenswrapper[2570]: W0706 18:56:23.180304 2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8c828f23_ece7_439d_9ca2_ef0642e73506.slice/crio-d22b926905bd796c8ec45addf7b28a13790f24960d43b363515eead5043d9a66 WatchSource:0}: Error finding container d22b926905bd796c8ec45addf7b28a13790f24960d43b363515eead5043d9a66: Status 404 returned error can't find the container with id d22b926905bd796c8ec45addf7b28a13790f24960d43b363515eead5043d9a66 Jul 06 18:56:23.228176 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:56:23.228155 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k8fbq_bc7057bd-88da-48af-ba32-e6efdc6e8362/konnectivity-agent/0.log" limit=1024 Jul 06 18:56:23.329287 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:56:23.328537 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k8fbq_bc7057bd-88da-48af-ba32-e6efdc6e8362/konnectivity-agent/0.log" limit=1024 Jul 06 18:56:23.377897 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:56:23.377312 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/monitoring-plugin-d5dcff7cf-c9prf" event={"ID":"8c828f23-ece7-439d-9ca2-ef0642e73506","Type":"ContainerStarted","Data":"d22b926905bd796c8ec45addf7b28a13790f24960d43b363515eead5043d9a66"} Jul 06 18:56:23.428589 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:56:23.428496 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k8fbq_bc7057bd-88da-48af-ba32-e6efdc6e8362/konnectivity-agent/0.log" limit=1024 Jul 06 18:56:23.439178 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:56:23.438496 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jul 06 18:56:24.159202 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:56:24.159164 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k8fbq_bc7057bd-88da-48af-ba32-e6efdc6e8362/konnectivity-agent/0.log" limit=1024 Jul 06 18:56:24.226354 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:56:24.226318 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k8fbq_bc7057bd-88da-48af-ba32-e6efdc6e8362/konnectivity-agent/0.log" limit=1024 Jul 06 18:56:24.388892 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:56:24.388855 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-h62zc" event={"ID":"910d0e65-b069-41b4-a8d5-2881341454e7","Type":"ContainerStarted","Data":"a930c32aa271ae8216efa889c4f819c8df36b578885e28b7999e2a8134f63c6a"} Jul 06 18:56:24.393794 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:56:24.393677 2570 generic.go:356] "Generic (PLEG): container finished" podID="45a3639f-79ab-4bb6-8a64-f2b4458f3375" containerID="824e75a7716111edd36acdce16301eda65bfd510a905d6f2eba53141bc8c875b" exitCode=0 Jul 06 18:56:24.393794 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:56:24.393769 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"45a3639f-79ab-4bb6-8a64-f2b4458f3375","Type":"ContainerDied","Data":"824e75a7716111edd36acdce16301eda65bfd510a905d6f2eba53141bc8c875b"} Jul 06 18:56:24.394005 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:56:24.393901 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"45a3639f-79ab-4bb6-8a64-f2b4458f3375","Type":"ContainerStarted","Data":"500b03df330643f6796c8feeff80b2c051c61f2b085f060d7f2c778c4e83f455"} Jul 06 18:56:24.396956 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:56:24.396929 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6c6c7cc65c-tp9k6" event={"ID":"3afaab40-a812-4cd4-ab65-bc0da82cb738","Type":"ContainerStarted","Data":"c06b0d05baa189308f7a092e7298349d4572ba5f41ee39d6c303f0c0b4e6527c"} Jul 06 18:56:24.400519 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:56:24.400491 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-mkrnp" event={"ID":"5f316223-4aea-420d-be1b-c2921c1f994f","Type":"ContainerStarted","Data":"44a719c19099e9b5032f0d4dcc0db9e3ed24057677155024a09654530677ffbe"} Jul 06 18:56:24.403057 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:56:24.402889 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-7d85fbb9b-8c6ms" event={"ID":"3c3710b4-98b2-42cf-9425-5a1b35e7f794","Type":"ContainerStarted","Data":"08bd186073f972128ff7a08590907b9d45672632886b4b79be650c1a8f76441a"} Jul 06 18:56:24.403057 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:56:24.402971 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-7d85fbb9b-8c6ms" event={"ID":"3c3710b4-98b2-42cf-9425-5a1b35e7f794","Type":"ContainerStarted","Data":"42af45854f8c1d278a9acd31515a56c27ad0e818b68836dde7b383e3856a8de3"} Jul 06 18:56:24.403057 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:56:24.403013 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-7d85fbb9b-8c6ms" event={"ID":"3c3710b4-98b2-42cf-9425-5a1b35e7f794","Type":"ContainerStarted","Data":"846d4c215a7ff40579340ddfafa14c7bd065e1d180b28672d76b9026fb764eaa"} Jul 06 18:56:24.404482 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:56:24.404442 2570 generic.go:356] "Generic (PLEG): container finished" podID="06b795c7-2ff5-4057-ab00-489a7cf9457b" containerID="8509d0c990ed24211a43112e70124fc9b8f5d8407be2694638decb4cbab460ee" exitCode=0 Jul 06 18:56:24.404593 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:56:24.404475 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"06b795c7-2ff5-4057-ab00-489a7cf9457b","Type":"ContainerDied","Data":"8509d0c990ed24211a43112e70124fc9b8f5d8407be2694638decb4cbab460ee"} Jul 06 18:56:24.406773 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:56:24.406469 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/metrics-server-59d88b459-jqgzb" event={"ID":"f575cfd3-dd2e-4ecf-ab0f-1860884038a3","Type":"ContainerStarted","Data":"e2f280b6af0f42cdfcee73d1099b654ab79ec03ab04c358c98dccee3cdb233ab"} Jul 06 18:56:24.407812 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:56:24.407767 2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-h62zc" podStartSLOduration=6.124703633 podStartE2EDuration="11.407750113s" totalImagesPullingTime="5.28304648s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 18:56:13 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 18:56:24.405775591 +0000 UTC m=+63.960796774" watchObservedRunningTime="2026-07-06 18:56:24.407750113 +0000 UTC m=+63.962771285" Jul 06 18:56:24.409329 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:56:24.409275 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-lrhkk" event={"ID":"30d171a0-0417-4444-8af2-154f0d6c82f9","Type":"ContainerStarted","Data":"2d34b9e0b7ac8496a13e63b21955efe04175b0c63057f2f76dc217f60e4e2dce"} Jul 06 18:56:24.409419 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:56:24.409329 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-lrhkk" event={"ID":"30d171a0-0417-4444-8af2-154f0d6c82f9","Type":"ContainerStarted","Data":"3507e4bd3bcccd6defd40830abbf9516e22564d9eed8e00c24bd12596deb3d24"} Jul 06 18:56:24.409419 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:56:24.409409 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-lrhkk" event={"ID":"30d171a0-0417-4444-8af2-154f0d6c82f9","Type":"ContainerStarted","Data":"92305e40900acfd2a2934f6f5e148148ba694fd5e542596405535b31ee4baf29"} Jul 06 18:56:24.412134 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:56:24.412112 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-2fd54" event={"ID":"b3ab8883-a662-4f63-9429-13b7f1c3a3b3","Type":"ContainerStarted","Data":"650c620975e7bd66b52d6ea589549bb210475acb12b43f53999f26bd1b74cc32"} Jul 06 18:56:24.412199 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:56:24.412155 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-2fd54" event={"ID":"b3ab8883-a662-4f63-9429-13b7f1c3a3b3","Type":"ContainerStarted","Data":"6c585598a8ba66366c2e4f0cae6243e905f0d57e0e12cbd8ddc5fd3ee7367e0f"} Jul 06 18:56:24.424285 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:56:24.424242 2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-mkrnp" podStartSLOduration=2.8411914830000002 podStartE2EDuration="14.424228529s" totalImagesPullingTime="11.583037046s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-06 18:56:10 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 18:56:24.423658818 +0000 UTC m=+63.978680016" watchObservedRunningTime="2026-07-06 18:56:24.424228529 +0000 UTC m=+63.979249702" Jul 06 18:56:24.442974 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:56:24.442934 2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-6c6c7cc65c-tp9k6" podStartSLOduration=5.47991744 podStartE2EDuration="11.442919792s" totalImagesPullingTime="5.963002352s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 18:56:13 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 18:56:24.441597904 +0000 UTC m=+63.996619113" watchObservedRunningTime="2026-07-06 18:56:24.442919792 +0000 UTC m=+63.997940964" Jul 06 18:56:24.465054 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:56:24.465026 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k8fbq_bc7057bd-88da-48af-ba32-e6efdc6e8362/konnectivity-agent/0.log" limit=1024 Jul 06 18:56:24.527570 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:56:24.527519 2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/metrics-server-59d88b459-jqgzb" podStartSLOduration=1.823279285 podStartE2EDuration="6.52750376s" totalImagesPullingTime="4.704224475s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 18:56:18 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 18:56:24.526120346 +0000 UTC m=+64.081141519" watchObservedRunningTime="2026-07-06 18:56:24.52750376 +0000 UTC m=+64.082524932" Jul 06 18:56:24.549902 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:56:24.549852 2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-2fd54" podStartSLOduration=10.379127447 podStartE2EDuration="11.549834429s" totalImagesPullingTime="1.085384523s" totalInitContainerRuntime="85.322459ms" isStatefulPod=true podCreationTimestamp="2026-07-06 18:56:13 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 18:56:24.548809263 +0000 UTC m=+64.103830459" watchObservedRunningTime="2026-07-06 18:56:24.549834429 +0000 UTC m=+64.104855601" Jul 06 18:56:24.553769 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:56:24.553730 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k8fbq_bc7057bd-88da-48af-ba32-e6efdc6e8362/konnectivity-agent/0.log" limit=1024 Jul 06 18:56:24.568521 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:56:24.568487 2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-lrhkk" podStartSLOduration=5.613208133 podStartE2EDuration="11.568473815s" totalImagesPullingTime="5.955265682s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 18:56:13 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 18:56:24.567582763 +0000 UTC m=+64.122603952" watchObservedRunningTime="2026-07-06 18:56:24.568473815 +0000 UTC m=+64.123494992" Jul 06 18:56:25.421077 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:56:25.421039 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-7d85fbb9b-8c6ms" event={"ID":"3c3710b4-98b2-42cf-9425-5a1b35e7f794","Type":"ContainerStarted","Data":"f258f31cae56c00cef05da2b019da3858265faa929372080c1fbbffc37f97932"} Jul 06 18:56:25.421528 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:56:25.421132 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-7d85fbb9b-8c6ms" event={"ID":"3c3710b4-98b2-42cf-9425-5a1b35e7f794","Type":"ContainerStarted","Data":"306c6495e3c4dba8c607317603842e42d40da0c634628568e0e3e57ce8deb041"} Jul 06 18:56:25.421528 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:56:25.421182 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-7d85fbb9b-8c6ms" event={"ID":"3c3710b4-98b2-42cf-9425-5a1b35e7f794","Type":"ContainerStarted","Data":"587198690c8a491ba2f4624eba148cb7be9359c44ca57babc4df6a5986e256bd"} Jul 06 18:56:25.421528 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:56:25.421322 2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/thanos-querier-7d85fbb9b-8c6ms" Jul 06 18:56:25.423305 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:56:25.423276 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/monitoring-plugin-d5dcff7cf-c9prf" event={"ID":"8c828f23-ece7-439d-9ca2-ef0642e73506","Type":"ContainerStarted","Data":"2ab4a473a9010625d1308fd4b8f63228abfd80e7dea26bf82e295c8c39abdb86"} Jul 06 18:56:25.424548 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:56:25.424528 2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/monitoring-plugin-d5dcff7cf-c9prf" Jul 06 18:56:25.430710 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:56:25.430687 2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/monitoring-plugin-d5dcff7cf-c9prf" Jul 06 18:56:25.444213 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:56:25.443654 2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/thanos-querier-7d85fbb9b-8c6ms" podStartSLOduration=3.537625259 podStartE2EDuration="10.443636971s" totalImagesPullingTime="6.906011712s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 18:56:15 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 18:56:25.441779957 +0000 UTC m=+64.996801150" watchObservedRunningTime="2026-07-06 18:56:25.443636971 +0000 UTC m=+64.998658142" Jul 06 18:56:25.458328 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:56:25.458118 2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/monitoring-plugin-d5dcff7cf-c9prf" podStartSLOduration=5.895582028 podStartE2EDuration="7.458100883s" totalImagesPullingTime="1.562518855s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 18:56:18 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 18:56:25.456593484 +0000 UTC m=+65.011614681" watchObservedRunningTime="2026-07-06 18:56:25.458100883 +0000 UTC m=+65.013122058" Jul 06 18:56:26.693652 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:56:26.693549 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/9262d667-2ed4-45aa-98f5-2d8158a66ca1-metrics-certs\") pod \"network-metrics-daemon-wh2jx\" (UID: \"9262d667-2ed4-45aa-98f5-2d8158a66ca1\") " pod="openshift-multus/network-metrics-daemon-wh2jx" Jul 06 18:56:26.696636 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:56:26.696600 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Jul 06 18:56:26.707335 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:56:26.707309 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/9262d667-2ed4-45aa-98f5-2d8158a66ca1-metrics-certs\") pod \"network-metrics-daemon-wh2jx\" (UID: \"9262d667-2ed4-45aa-98f5-2d8158a66ca1\") " pod="openshift-multus/network-metrics-daemon-wh2jx" Jul 06 18:56:26.794903 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:56:26.794861 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/4beea8e8-7750-4a7a-bc49-249b3d7e3ab3-original-pull-secret\") pod \"global-pull-secret-syncer-ln98s\" (UID: \"4beea8e8-7750-4a7a-bc49-249b3d7e3ab3\") " pod="kube-system/global-pull-secret-syncer-ln98s" Jul 06 18:56:26.797773 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:56:26.797750 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Jul 06 18:56:26.808177 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:56:26.808156 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/4beea8e8-7750-4a7a-bc49-249b3d7e3ab3-original-pull-secret\") pod \"global-pull-secret-syncer-ln98s\" (UID: \"4beea8e8-7750-4a7a-bc49-249b3d7e3ab3\") " pod="kube-system/global-pull-secret-syncer-ln98s" Jul 06 18:56:26.895576 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:56:26.895541 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qfrb6\" (UniqueName: \"kubernetes.io/projected/f7bc558b-d33b-448d-929c-3652d096c3d5-kube-api-access-qfrb6\") pod \"network-check-target-296mj\" (UID: \"f7bc558b-d33b-448d-929c-3652d096c3d5\") " pod="openshift-network-diagnostics/network-check-target-296mj" Jul 06 18:56:26.898142 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:56:26.898122 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" Jul 06 18:56:26.908760 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:56:26.908739 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" Jul 06 18:56:26.920764 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:56:26.919770 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-qfrb6\" (UniqueName: \"kubernetes.io/projected/f7bc558b-d33b-448d-929c-3652d096c3d5-kube-api-access-qfrb6\") pod \"network-check-target-296mj\" (UID: \"f7bc558b-d33b-448d-929c-3652d096c3d5\") " pod="openshift-network-diagnostics/network-check-target-296mj" Jul 06 18:56:26.933771 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:56:26.933750 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-ln98s" Jul 06 18:56:26.946633 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:56:26.946563 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-gk8fr\"" Jul 06 18:56:26.950832 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:56:26.950818 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-w7zz5\"" Jul 06 18:56:26.954932 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:56:26.954911 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-296mj" Jul 06 18:56:26.958745 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:56:26.958726 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-wh2jx" Jul 06 18:56:27.434214 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:56:27.434182 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"06b795c7-2ff5-4057-ab00-489a7cf9457b","Type":"ContainerStarted","Data":"702bdcefebee8c3053467fdab3369df0fe728f3b39ca6b1af7c9b61687904729"} Jul 06 18:56:27.434359 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:56:27.434254 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"06b795c7-2ff5-4057-ab00-489a7cf9457b","Type":"ContainerStarted","Data":"09a3d2934fd8c9aa70ec05da33359b5009842c5513feecf5e645b544aa90af71"} Jul 06 18:56:27.847758 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:56:27.847691 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-ln98s"] Jul 06 18:56:27.854309 ip-10-0-130-59 kubenswrapper[2570]: W0706 18:56:27.853982 2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4beea8e8_7750_4a7a_bc49_249b3d7e3ab3.slice/crio-01cf21365b87b38f94f3283884444680841af64d1cfa52a040c6a3d1db10dd49 WatchSource:0}: Error finding container 01cf21365b87b38f94f3283884444680841af64d1cfa52a040c6a3d1db10dd49: Status 404 returned error can't find the container with id 01cf21365b87b38f94f3283884444680841af64d1cfa52a040c6a3d1db10dd49 Jul 06 18:56:27.866931 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:56:27.866815 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-wh2jx"] Jul 06 18:56:27.871636 ip-10-0-130-59 kubenswrapper[2570]: W0706 18:56:27.871575 2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9262d667_2ed4_45aa_98f5_2d8158a66ca1.slice/crio-9611c88a6ac643645e388fde0fcc200883b0ae88b1afa6d74a0fc37b1fc1a4ea WatchSource:0}: Error finding container 9611c88a6ac643645e388fde0fcc200883b0ae88b1afa6d74a0fc37b1fc1a4ea: Status 404 returned error can't find the container with id 9611c88a6ac643645e388fde0fcc200883b0ae88b1afa6d74a0fc37b1fc1a4ea Jul 06 18:56:27.898348 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:56:27.898067 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-296mj"] Jul 06 18:56:27.904476 ip-10-0-130-59 kubenswrapper[2570]: W0706 18:56:27.904406 2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf7bc558b_d33b_448d_929c_3652d096c3d5.slice/crio-68f1174e2532ad1c2f5a7479eb10d6d21f0f574e0dd70e06465694b75be36cfe WatchSource:0}: Error finding container 68f1174e2532ad1c2f5a7479eb10d6d21f0f574e0dd70e06465694b75be36cfe: Status 404 returned error can't find the container with id 68f1174e2532ad1c2f5a7479eb10d6d21f0f574e0dd70e06465694b75be36cfe Jul 06 18:56:28.162992 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:56:28.162947 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k8fbq_bc7057bd-88da-48af-ba32-e6efdc6e8362/konnectivity-agent/0.log" limit=1024 Jul 06 18:56:28.263558 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:56:28.263463 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k8fbq_bc7057bd-88da-48af-ba32-e6efdc6e8362/konnectivity-agent/0.log" limit=1024 Jul 06 18:56:28.315708 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:56:28.315007 2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-dk4j7" Jul 06 18:56:28.447220 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:56:28.447182 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"06b795c7-2ff5-4057-ab00-489a7cf9457b","Type":"ContainerStarted","Data":"090ca2e6ac974757f9c0720b6495efd4b3246a4ed03262d9440002358c05338d"} Jul 06 18:56:28.447372 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:56:28.447267 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"06b795c7-2ff5-4057-ab00-489a7cf9457b","Type":"ContainerStarted","Data":"34d7c70a78642e1ef5ef61eb3a721dc61438a8bf6a2d8afdcb31842aeedcc1f0"} Jul 06 18:56:28.447372 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:56:28.447309 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"06b795c7-2ff5-4057-ab00-489a7cf9457b","Type":"ContainerStarted","Data":"a32c5b0db91a42bbaf6723866951bd6014c5403d2a8cd03f6c69215cadb8395b"} Jul 06 18:56:28.447372 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:56:28.447347 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"06b795c7-2ff5-4057-ab00-489a7cf9457b","Type":"ContainerStarted","Data":"553b82728c7228e54f93eb9d35464feebbd33b9aa2c96d5e4ae48cef36947aae"} Jul 06 18:56:28.450873 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:56:28.450845 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-296mj" event={"ID":"f7bc558b-d33b-448d-929c-3652d096c3d5","Type":"ContainerStarted","Data":"68f1174e2532ad1c2f5a7479eb10d6d21f0f574e0dd70e06465694b75be36cfe"} Jul 06 18:56:28.453230 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:56:28.453204 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-wh2jx" event={"ID":"9262d667-2ed4-45aa-98f5-2d8158a66ca1","Type":"ContainerStarted","Data":"9611c88a6ac643645e388fde0fcc200883b0ae88b1afa6d74a0fc37b1fc1a4ea"} Jul 06 18:56:28.454908 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:56:28.454884 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-ln98s" event={"ID":"4beea8e8-7750-4a7a-bc49-249b3d7e3ab3","Type":"ContainerStarted","Data":"01cf21365b87b38f94f3283884444680841af64d1cfa52a040c6a3d1db10dd49"} Jul 06 18:56:28.461469 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:56:28.461445 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"45a3639f-79ab-4bb6-8a64-f2b4458f3375","Type":"ContainerStarted","Data":"e0f5261f30fd436f3cc5191bf5d2fe73764fdab9992b31134d2d7dbbf9bcc54c"} Jul 06 18:56:28.461593 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:56:28.461564 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"45a3639f-79ab-4bb6-8a64-f2b4458f3375","Type":"ContainerStarted","Data":"d50576bc26b15d78d3e1b9b182ecf8016762de33614fb8d8e237e55d260720e5"} Jul 06 18:56:28.461899 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:56:28.461721 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"45a3639f-79ab-4bb6-8a64-f2b4458f3375","Type":"ContainerStarted","Data":"ec3c3442dab43c6a7de794ed52c5c293b5c7b76635bd2bb41ef15a4cffaf184a"} Jul 06 18:56:28.461899 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:56:28.461776 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"45a3639f-79ab-4bb6-8a64-f2b4458f3375","Type":"ContainerStarted","Data":"2cb7f600ea6415e85375a9c11ea27b29e225aa43e7f430dace90788421c2467c"} Jul 06 18:56:28.461899 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:56:28.461816 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"45a3639f-79ab-4bb6-8a64-f2b4458f3375","Type":"ContainerStarted","Data":"7efcab14e9d4bed190760cf9ca3845e56cf9c917cdc6744f4b01234852be927a"} Jul 06 18:56:28.461899 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:56:28.461855 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"45a3639f-79ab-4bb6-8a64-f2b4458f3375","Type":"ContainerStarted","Data":"061222b702aae270175e5d72ed12de85bfcb3957293f36dab428c91346da5006"} Jul 06 18:56:28.539129 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:56:28.538545 2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/alertmanager-main-0" podStartSLOduration=7.084424951 podStartE2EDuration="14.538522073s" totalImagesPullingTime="7.401829629s" totalInitContainerRuntime="52.267493ms" isStatefulPod=false podCreationTimestamp="2026-07-06 18:56:14 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 18:56:28.485491985 +0000 UTC m=+68.040513158" watchObservedRunningTime="2026-07-06 18:56:28.538522073 +0000 UTC m=+68.093543247" Jul 06 18:56:28.539129 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:56:28.538830 2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-k8s-0" podStartSLOduration=5.20762757 podStartE2EDuration="8.538820315s" totalImagesPullingTime="3.294054873s" totalInitContainerRuntime="37.137872ms" isStatefulPod=false podCreationTimestamp="2026-07-06 18:56:20 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 18:56:28.536906368 +0000 UTC m=+68.091927565" watchObservedRunningTime="2026-07-06 18:56:28.538820315 +0000 UTC m=+68.093841488" Jul 06 18:56:29.469572 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:56:29.469388 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-wh2jx" event={"ID":"9262d667-2ed4-45aa-98f5-2d8158a66ca1","Type":"ContainerStarted","Data":"167182702a34b49fcbe771e6b978ce08cc4afdc0fbb8127ca4c364eb9e858756"} Jul 06 18:56:29.469572 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:56:29.469495 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-wh2jx" event={"ID":"9262d667-2ed4-45aa-98f5-2d8158a66ca1","Type":"ContainerStarted","Data":"fc32ec16efa58119262c7840501700d209b0b32eda276d11c23c08cdd2b9d1b3"} Jul 06 18:56:29.490440 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:56:29.490281 2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-wh2jx" podStartSLOduration=67.330137816 podStartE2EDuration="1m8.490263958s" totalImagesPullingTime="1.160126142s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 18:55:21 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 18:56:29.487012336 +0000 UTC m=+69.042033534" watchObservedRunningTime="2026-07-06 18:56:29.490263958 +0000 UTC m=+69.045285132" Jul 06 18:56:29.491719 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:56:29.491462 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k8fbq_bc7057bd-88da-48af-ba32-e6efdc6e8362/konnectivity-agent/0.log" limit=1024 Jul 06 18:56:29.594661 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:56:29.594574 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k8fbq_bc7057bd-88da-48af-ba32-e6efdc6e8362/konnectivity-agent/0.log" limit=1024 Jul 06 18:56:29.700252 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:56:29.700210 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k8fbq_bc7057bd-88da-48af-ba32-e6efdc6e8362/konnectivity-agent/0.log" limit=1024 Jul 06 18:56:30.133912 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:56:30.133844 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k8fbq_bc7057bd-88da-48af-ba32-e6efdc6e8362/konnectivity-agent/0.log" limit=1024 Jul 06 18:56:30.237071 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:56:30.237036 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k8fbq_bc7057bd-88da-48af-ba32-e6efdc6e8362/konnectivity-agent/0.log" limit=1024 Jul 06 18:56:30.362105 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:56:30.362073 2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-k8s-0" Jul 06 18:56:31.199563 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:56:31.199521 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k8fbq_bc7057bd-88da-48af-ba32-e6efdc6e8362/konnectivity-agent/0.log" limit=1024 Jul 06 18:56:31.299917 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:56:31.299875 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k8fbq_bc7057bd-88da-48af-ba32-e6efdc6e8362/konnectivity-agent/0.log" limit=1024 Jul 06 18:56:31.435047 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:56:31.435023 2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/thanos-querier-7d85fbb9b-8c6ms" Jul 06 18:56:33.489781 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:56:33.489745 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-296mj" event={"ID":"f7bc558b-d33b-448d-929c-3652d096c3d5","Type":"ContainerStarted","Data":"b35144815ae0c2c33bf78f180303c79c196f5ec6e8eadfeaa385801846e48960"} Jul 06 18:56:33.490272 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:56:33.489828 2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-296mj" Jul 06 18:56:33.491281 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:56:33.491258 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-ln98s" event={"ID":"4beea8e8-7750-4a7a-bc49-249b3d7e3ab3","Type":"ContainerStarted","Data":"72d8786baef69ebda389cf1dd13c72c6e5fedeceba0b2a75cee9f59521b62039"} Jul 06 18:56:33.506870 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:56:33.506819 2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-296mj" podStartSLOduration=67.845216263 podStartE2EDuration="1m12.506805971s" totalImagesPullingTime="4.661589708s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 18:55:21 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 18:56:33.505861809 +0000 UTC m=+73.060882979" watchObservedRunningTime="2026-07-06 18:56:33.506805971 +0000 UTC m=+73.061827121" Jul 06 18:56:33.523065 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:56:33.523025 2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-ln98s" podStartSLOduration=67.804992573 podStartE2EDuration="1m12.523012605s" totalImagesPullingTime="4.718020032s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-06 18:55:21 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 18:56:33.520847308 +0000 UTC m=+73.075868480" watchObservedRunningTime="2026-07-06 18:56:33.523012605 +0000 UTC m=+73.078033804" Jul 06 18:56:33.603659 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:56:33.603634 2570 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-6c6c7cc65c-tp9k6" Jul 06 18:56:33.603772 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:56:33.603687 2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-6c6c7cc65c-tp9k6" Jul 06 18:56:33.608241 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:56:33.608225 2570 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-6c6c7cc65c-tp9k6" Jul 06 18:56:34.232091 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:56:34.232055 2570 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-6c6c7cc65c-tp9k6"] Jul 06 18:56:34.499184 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:56:34.499111 2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-6c6c7cc65c-tp9k6" Jul 06 18:56:35.288388 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:56:35.288355 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k8fbq_bc7057bd-88da-48af-ba32-e6efdc6e8362/konnectivity-agent/0.log" limit=1024 Jul 06 18:56:35.356502 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:56:35.356467 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k8fbq_bc7057bd-88da-48af-ba32-e6efdc6e8362/konnectivity-agent/0.log" limit=1024 Jul 06 18:56:36.488297 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:56:36.488263 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k8fbq_bc7057bd-88da-48af-ba32-e6efdc6e8362/konnectivity-agent/0.log" limit=1024 Jul 06 18:56:36.585870 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:56:36.585834 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k8fbq_bc7057bd-88da-48af-ba32-e6efdc6e8362/konnectivity-agent/0.log" limit=1024 Jul 06 18:56:37.278916 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:56:37.278883 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k8fbq_bc7057bd-88da-48af-ba32-e6efdc6e8362/konnectivity-agent/0.log" limit=1024 Jul 06 18:56:37.335200 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:56:37.335178 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k8fbq_bc7057bd-88da-48af-ba32-e6efdc6e8362/konnectivity-agent/0.log" limit=1024 Jul 06 18:56:37.468595 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:56:37.468560 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k8fbq_bc7057bd-88da-48af-ba32-e6efdc6e8362/konnectivity-agent/0.log" limit=1024 Jul 06 18:56:37.529381 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:56:37.529288 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k8fbq_bc7057bd-88da-48af-ba32-e6efdc6e8362/konnectivity-agent/0.log" limit=1024 Jul 06 18:56:38.354408 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:56:38.354369 2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/metrics-server-59d88b459-jqgzb" Jul 06 18:56:38.354634 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:56:38.354429 2570 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/metrics-server-59d88b459-jqgzb" Jul 06 18:56:38.487968 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:56:38.487934 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k8fbq_bc7057bd-88da-48af-ba32-e6efdc6e8362/konnectivity-agent/0.log" limit=1024 Jul 06 18:56:38.554594 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:56:38.554561 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k8fbq_bc7057bd-88da-48af-ba32-e6efdc6e8362/konnectivity-agent/0.log" limit=1024 Jul 06 18:56:57.264235 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:56:57.264117 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k8fbq_bc7057bd-88da-48af-ba32-e6efdc6e8362/konnectivity-agent/0.log" limit=1024 Jul 06 18:56:57.343042 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:56:57.343013 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k8fbq_bc7057bd-88da-48af-ba32-e6efdc6e8362/konnectivity-agent/0.log" limit=1024 Jul 06 18:56:58.363513 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:56:58.363484 2570 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/metrics-server-59d88b459-jqgzb" Jul 06 18:56:58.367333 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:56:58.367315 2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/metrics-server-59d88b459-jqgzb" Jul 06 18:57:00.519687 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:57:00.519646 2570 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-console/console-6c6c7cc65c-tp9k6" podUID="3afaab40-a812-4cd4-ab65-bc0da82cb738" containerName="console" containerID="cri-o://c06b0d05baa189308f7a092e7298349d4572ba5f41ee39d6c303f0c0b4e6527c" gracePeriod=15 Jul 06 18:57:00.762426 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:57:00.762397 2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6c6c7cc65c-tp9k6" Jul 06 18:57:00.886091 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:57:00.886015 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/3afaab40-a812-4cd4-ab65-bc0da82cb738-console-oauth-config\") pod \"3afaab40-a812-4cd4-ab65-bc0da82cb738\" (UID: \"3afaab40-a812-4cd4-ab65-bc0da82cb738\") " Jul 06 18:57:00.886091 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:57:00.886051 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/3afaab40-a812-4cd4-ab65-bc0da82cb738-oauth-serving-cert\") pod \"3afaab40-a812-4cd4-ab65-bc0da82cb738\" (UID: \"3afaab40-a812-4cd4-ab65-bc0da82cb738\") " Jul 06 18:57:00.886091 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:57:00.886075 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/3afaab40-a812-4cd4-ab65-bc0da82cb738-console-serving-cert\") pod \"3afaab40-a812-4cd4-ab65-bc0da82cb738\" (UID: \"3afaab40-a812-4cd4-ab65-bc0da82cb738\") " Jul 06 18:57:00.886091 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:57:00.886093 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/3afaab40-a812-4cd4-ab65-bc0da82cb738-service-ca\") pod \"3afaab40-a812-4cd4-ab65-bc0da82cb738\" (UID: \"3afaab40-a812-4cd4-ab65-bc0da82cb738\") " Jul 06 18:57:00.886319 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:57:00.886221 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-scdhh\" (UniqueName: \"kubernetes.io/projected/3afaab40-a812-4cd4-ab65-bc0da82cb738-kube-api-access-scdhh\") pod \"3afaab40-a812-4cd4-ab65-bc0da82cb738\" (UID: \"3afaab40-a812-4cd4-ab65-bc0da82cb738\") " Jul 06 18:57:00.886364 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:57:00.886331 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/3afaab40-a812-4cd4-ab65-bc0da82cb738-console-config\") pod \"3afaab40-a812-4cd4-ab65-bc0da82cb738\" (UID: \"3afaab40-a812-4cd4-ab65-bc0da82cb738\") " Jul 06 18:57:00.886830 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:57:00.886476 2570 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3afaab40-a812-4cd4-ab65-bc0da82cb738-service-ca" (OuterVolumeSpecName: "service-ca") pod "3afaab40-a812-4cd4-ab65-bc0da82cb738" (UID: "3afaab40-a812-4cd4-ab65-bc0da82cb738"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 06 18:57:00.886830 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:57:00.886545 2570 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3afaab40-a812-4cd4-ab65-bc0da82cb738-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "3afaab40-a812-4cd4-ab65-bc0da82cb738" (UID: "3afaab40-a812-4cd4-ab65-bc0da82cb738"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 06 18:57:00.886830 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:57:00.886695 2570 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/3afaab40-a812-4cd4-ab65-bc0da82cb738-service-ca\") on node \"ip-10-0-130-59.ec2.internal\" DevicePath \"\"" Jul 06 18:57:00.886830 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:57:00.886786 2570 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3afaab40-a812-4cd4-ab65-bc0da82cb738-console-config" (OuterVolumeSpecName: "console-config") pod "3afaab40-a812-4cd4-ab65-bc0da82cb738" (UID: "3afaab40-a812-4cd4-ab65-bc0da82cb738"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 06 18:57:00.888511 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:57:00.888476 2570 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3afaab40-a812-4cd4-ab65-bc0da82cb738-kube-api-access-scdhh" (OuterVolumeSpecName: "kube-api-access-scdhh") pod "3afaab40-a812-4cd4-ab65-bc0da82cb738" (UID: "3afaab40-a812-4cd4-ab65-bc0da82cb738"). InnerVolumeSpecName "kube-api-access-scdhh". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 06 18:57:00.888511 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:57:00.888484 2570 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/3afaab40-a812-4cd4-ab65-bc0da82cb738-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "3afaab40-a812-4cd4-ab65-bc0da82cb738" (UID: "3afaab40-a812-4cd4-ab65-bc0da82cb738"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 06 18:57:00.888711 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:57:00.888518 2570 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/3afaab40-a812-4cd4-ab65-bc0da82cb738-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "3afaab40-a812-4cd4-ab65-bc0da82cb738" (UID: "3afaab40-a812-4cd4-ab65-bc0da82cb738"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 06 18:57:00.987312 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:57:00.987284 2570 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/3afaab40-a812-4cd4-ab65-bc0da82cb738-console-config\") on node \"ip-10-0-130-59.ec2.internal\" DevicePath \"\"" Jul 06 18:57:00.987312 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:57:00.987308 2570 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/3afaab40-a812-4cd4-ab65-bc0da82cb738-console-oauth-config\") on node \"ip-10-0-130-59.ec2.internal\" DevicePath \"\"" Jul 06 18:57:00.987501 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:57:00.987321 2570 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/3afaab40-a812-4cd4-ab65-bc0da82cb738-oauth-serving-cert\") on node \"ip-10-0-130-59.ec2.internal\" DevicePath \"\"" Jul 06 18:57:00.987501 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:57:00.987334 2570 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/3afaab40-a812-4cd4-ab65-bc0da82cb738-console-serving-cert\") on node \"ip-10-0-130-59.ec2.internal\" DevicePath \"\"" Jul 06 18:57:00.987501 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:57:00.987346 2570 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-scdhh\" (UniqueName: \"kubernetes.io/projected/3afaab40-a812-4cd4-ab65-bc0da82cb738-kube-api-access-scdhh\") on node \"ip-10-0-130-59.ec2.internal\" DevicePath \"\"" Jul 06 18:57:01.206195 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:57:01.206164 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k8fbq_bc7057bd-88da-48af-ba32-e6efdc6e8362/konnectivity-agent/0.log" limit=1024 Jul 06 18:57:01.273081 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:57:01.273054 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k8fbq_bc7057bd-88da-48af-ba32-e6efdc6e8362/konnectivity-agent/0.log" limit=1024 Jul 06 18:57:01.606253 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:57:01.606166 2570 generic.go:356] "Generic (PLEG): container finished" podID="3afaab40-a812-4cd4-ab65-bc0da82cb738" containerID="c06b0d05baa189308f7a092e7298349d4572ba5f41ee39d6c303f0c0b4e6527c" exitCode=0 Jul 06 18:57:01.606253 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:57:01.606221 2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6c6c7cc65c-tp9k6" Jul 06 18:57:01.606253 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:57:01.606242 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6c6c7cc65c-tp9k6" event={"ID":"3afaab40-a812-4cd4-ab65-bc0da82cb738","Type":"ContainerDied","Data":"c06b0d05baa189308f7a092e7298349d4572ba5f41ee39d6c303f0c0b4e6527c"} Jul 06 18:57:01.606733 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:57:01.606307 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6c6c7cc65c-tp9k6" event={"ID":"3afaab40-a812-4cd4-ab65-bc0da82cb738","Type":"ContainerDied","Data":"21824889c72b262e9906b59165d050cddc387811e0b5e0bff1689a0eb9c6c39f"} Jul 06 18:57:01.606733 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:57:01.606330 2570 scope.go:117] "RemoveContainer" containerID="c06b0d05baa189308f7a092e7298349d4572ba5f41ee39d6c303f0c0b4e6527c" Jul 06 18:57:01.614571 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:57:01.614551 2570 scope.go:117] "RemoveContainer" containerID="c06b0d05baa189308f7a092e7298349d4572ba5f41ee39d6c303f0c0b4e6527c" Jul 06 18:57:01.614856 ip-10-0-130-59 kubenswrapper[2570]: E0706 18:57:01.614834 2570 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c06b0d05baa189308f7a092e7298349d4572ba5f41ee39d6c303f0c0b4e6527c\": container with ID starting with c06b0d05baa189308f7a092e7298349d4572ba5f41ee39d6c303f0c0b4e6527c not found: ID does not exist" containerID="c06b0d05baa189308f7a092e7298349d4572ba5f41ee39d6c303f0c0b4e6527c" Jul 06 18:57:01.614910 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:57:01.614864 2570 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c06b0d05baa189308f7a092e7298349d4572ba5f41ee39d6c303f0c0b4e6527c"} err="failed to get container status \"c06b0d05baa189308f7a092e7298349d4572ba5f41ee39d6c303f0c0b4e6527c\": rpc error: code = NotFound desc = could not find container \"c06b0d05baa189308f7a092e7298349d4572ba5f41ee39d6c303f0c0b4e6527c\": container with ID starting with c06b0d05baa189308f7a092e7298349d4572ba5f41ee39d6c303f0c0b4e6527c not found: ID does not exist" Jul 06 18:57:01.623567 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:57:01.623544 2570 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-6c6c7cc65c-tp9k6"] Jul 06 18:57:01.631276 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:57:01.631256 2570 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-6c6c7cc65c-tp9k6"] Jul 06 18:57:03.027413 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:57:03.027377 2570 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="3afaab40-a812-4cd4-ab65-bc0da82cb738" path="/var/lib/kubelet/pods/3afaab40-a812-4cd4-ab65-bc0da82cb738/volumes" Jul 06 18:57:04.498103 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:57:04.498075 2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-296mj" Jul 06 18:57:27.655967 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:57:27.655925 2570 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/prometheus-k8s-0" Jul 06 18:57:27.675070 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:57:27.675040 2570 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/prometheus-k8s-0" Jul 06 18:57:27.730789 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:57:27.730763 2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-k8s-0" Jul 06 18:57:33.071767 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:57:33.071729 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k8fbq_bc7057bd-88da-48af-ba32-e6efdc6e8362/konnectivity-agent/0.log" limit=1024 Jul 06 18:57:33.289808 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:57:33.289777 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k8fbq_bc7057bd-88da-48af-ba32-e6efdc6e8362/konnectivity-agent/0.log" limit=1024 Jul 06 18:57:34.186266 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:57:34.186231 2570 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 06 18:57:34.186695 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:57:34.186674 2570 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="06b795c7-2ff5-4057-ab00-489a7cf9457b" containerName="alertmanager" containerID="cri-o://09a3d2934fd8c9aa70ec05da33359b5009842c5513feecf5e645b544aa90af71" gracePeriod=120 Jul 06 18:57:34.186787 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:57:34.186755 2570 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="06b795c7-2ff5-4057-ab00-489a7cf9457b" containerName="kube-rbac-proxy-metric" containerID="cri-o://34d7c70a78642e1ef5ef61eb3a721dc61438a8bf6a2d8afdcb31842aeedcc1f0" gracePeriod=120 Jul 06 18:57:34.186900 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:57:34.186862 2570 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="06b795c7-2ff5-4057-ab00-489a7cf9457b" containerName="kube-rbac-proxy" containerID="cri-o://a32c5b0db91a42bbaf6723866951bd6014c5403d2a8cd03f6c69215cadb8395b" gracePeriod=120 Jul 06 18:57:34.187669 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:57:34.186824 2570 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="06b795c7-2ff5-4057-ab00-489a7cf9457b" containerName="config-reloader" containerID="cri-o://702bdcefebee8c3053467fdab3369df0fe728f3b39ca6b1af7c9b61687904729" gracePeriod=120 Jul 06 18:57:34.187669 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:57:34.186834 2570 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="06b795c7-2ff5-4057-ab00-489a7cf9457b" containerName="prom-label-proxy" containerID="cri-o://090ca2e6ac974757f9c0720b6495efd4b3246a4ed03262d9440002358c05338d" gracePeriod=120 Jul 06 18:57:34.187669 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:57:34.186921 2570 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="06b795c7-2ff5-4057-ab00-489a7cf9457b" containerName="kube-rbac-proxy-web" containerID="cri-o://553b82728c7228e54f93eb9d35464feebbd33b9aa2c96d5e4ae48cef36947aae" gracePeriod=120 Jul 06 18:57:34.744743 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:57:34.744708 2570 generic.go:356] "Generic (PLEG): container finished" podID="06b795c7-2ff5-4057-ab00-489a7cf9457b" containerID="090ca2e6ac974757f9c0720b6495efd4b3246a4ed03262d9440002358c05338d" exitCode=0 Jul 06 18:57:34.744743 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:57:34.744735 2570 generic.go:356] "Generic (PLEG): container finished" podID="06b795c7-2ff5-4057-ab00-489a7cf9457b" containerID="34d7c70a78642e1ef5ef61eb3a721dc61438a8bf6a2d8afdcb31842aeedcc1f0" exitCode=0 Jul 06 18:57:34.744743 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:57:34.744742 2570 generic.go:356] "Generic (PLEG): container finished" podID="06b795c7-2ff5-4057-ab00-489a7cf9457b" containerID="a32c5b0db91a42bbaf6723866951bd6014c5403d2a8cd03f6c69215cadb8395b" exitCode=0 Jul 06 18:57:34.744743 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:57:34.744747 2570 generic.go:356] "Generic (PLEG): container finished" podID="06b795c7-2ff5-4057-ab00-489a7cf9457b" containerID="702bdcefebee8c3053467fdab3369df0fe728f3b39ca6b1af7c9b61687904729" exitCode=0 Jul 06 18:57:34.744743 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:57:34.744752 2570 generic.go:356] "Generic (PLEG): container finished" podID="06b795c7-2ff5-4057-ab00-489a7cf9457b" containerID="09a3d2934fd8c9aa70ec05da33359b5009842c5513feecf5e645b544aa90af71" exitCode=0 Jul 06 18:57:34.745028 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:57:34.744812 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"06b795c7-2ff5-4057-ab00-489a7cf9457b","Type":"ContainerDied","Data":"090ca2e6ac974757f9c0720b6495efd4b3246a4ed03262d9440002358c05338d"} Jul 06 18:57:34.745028 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:57:34.744866 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"06b795c7-2ff5-4057-ab00-489a7cf9457b","Type":"ContainerDied","Data":"34d7c70a78642e1ef5ef61eb3a721dc61438a8bf6a2d8afdcb31842aeedcc1f0"} Jul 06 18:57:34.745028 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:57:34.744890 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"06b795c7-2ff5-4057-ab00-489a7cf9457b","Type":"ContainerDied","Data":"a32c5b0db91a42bbaf6723866951bd6014c5403d2a8cd03f6c69215cadb8395b"} Jul 06 18:57:34.745028 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:57:34.744915 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"06b795c7-2ff5-4057-ab00-489a7cf9457b","Type":"ContainerDied","Data":"702bdcefebee8c3053467fdab3369df0fe728f3b39ca6b1af7c9b61687904729"} Jul 06 18:57:34.745028 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:57:34.744936 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"06b795c7-2ff5-4057-ab00-489a7cf9457b","Type":"ContainerDied","Data":"09a3d2934fd8c9aa70ec05da33359b5009842c5513feecf5e645b544aa90af71"} Jul 06 18:57:35.422810 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:57:35.422787 2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Jul 06 18:57:35.470198 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:57:35.470170 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/06b795c7-2ff5-4057-ab00-489a7cf9457b-alertmanager-trusted-ca-bundle\") pod \"06b795c7-2ff5-4057-ab00-489a7cf9457b\" (UID: \"06b795c7-2ff5-4057-ab00-489a7cf9457b\") " Jul 06 18:57:35.470348 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:57:35.470215 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/06b795c7-2ff5-4057-ab00-489a7cf9457b-tls-assets\") pod \"06b795c7-2ff5-4057-ab00-489a7cf9457b\" (UID: \"06b795c7-2ff5-4057-ab00-489a7cf9457b\") " Jul 06 18:57:35.470348 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:57:35.470245 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/06b795c7-2ff5-4057-ab00-489a7cf9457b-secret-alertmanager-kube-rbac-proxy\") pod \"06b795c7-2ff5-4057-ab00-489a7cf9457b\" (UID: \"06b795c7-2ff5-4057-ab00-489a7cf9457b\") " Jul 06 18:57:35.470348 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:57:35.470273 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/06b795c7-2ff5-4057-ab00-489a7cf9457b-metrics-client-ca\") pod \"06b795c7-2ff5-4057-ab00-489a7cf9457b\" (UID: \"06b795c7-2ff5-4057-ab00-489a7cf9457b\") " Jul 06 18:57:35.470348 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:57:35.470297 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/06b795c7-2ff5-4057-ab00-489a7cf9457b-alertmanager-main-db\") pod \"06b795c7-2ff5-4057-ab00-489a7cf9457b\" (UID: \"06b795c7-2ff5-4057-ab00-489a7cf9457b\") " Jul 06 18:57:35.470348 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:57:35.470326 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/06b795c7-2ff5-4057-ab00-489a7cf9457b-web-config\") pod \"06b795c7-2ff5-4057-ab00-489a7cf9457b\" (UID: \"06b795c7-2ff5-4057-ab00-489a7cf9457b\") " Jul 06 18:57:35.470714 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:57:35.470373 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/06b795c7-2ff5-4057-ab00-489a7cf9457b-cluster-tls-config\") pod \"06b795c7-2ff5-4057-ab00-489a7cf9457b\" (UID: \"06b795c7-2ff5-4057-ab00-489a7cf9457b\") " Jul 06 18:57:35.470714 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:57:35.470407 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-sp6c4\" (UniqueName: \"kubernetes.io/projected/06b795c7-2ff5-4057-ab00-489a7cf9457b-kube-api-access-sp6c4\") pod \"06b795c7-2ff5-4057-ab00-489a7cf9457b\" (UID: \"06b795c7-2ff5-4057-ab00-489a7cf9457b\") " Jul 06 18:57:35.470714 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:57:35.470445 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/06b795c7-2ff5-4057-ab00-489a7cf9457b-config-out\") pod \"06b795c7-2ff5-4057-ab00-489a7cf9457b\" (UID: \"06b795c7-2ff5-4057-ab00-489a7cf9457b\") " Jul 06 18:57:35.470714 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:57:35.470476 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/06b795c7-2ff5-4057-ab00-489a7cf9457b-secret-alertmanager-main-tls\") pod \"06b795c7-2ff5-4057-ab00-489a7cf9457b\" (UID: \"06b795c7-2ff5-4057-ab00-489a7cf9457b\") " Jul 06 18:57:35.470714 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:57:35.470545 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/06b795c7-2ff5-4057-ab00-489a7cf9457b-config-volume\") pod \"06b795c7-2ff5-4057-ab00-489a7cf9457b\" (UID: \"06b795c7-2ff5-4057-ab00-489a7cf9457b\") " Jul 06 18:57:35.470714 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:57:35.470575 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/06b795c7-2ff5-4057-ab00-489a7cf9457b-secret-alertmanager-kube-rbac-proxy-metric\") pod \"06b795c7-2ff5-4057-ab00-489a7cf9457b\" (UID: \"06b795c7-2ff5-4057-ab00-489a7cf9457b\") " Jul 06 18:57:35.470714 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:57:35.470587 2570 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/06b795c7-2ff5-4057-ab00-489a7cf9457b-alertmanager-trusted-ca-bundle" (OuterVolumeSpecName: "alertmanager-trusted-ca-bundle") pod "06b795c7-2ff5-4057-ab00-489a7cf9457b" (UID: "06b795c7-2ff5-4057-ab00-489a7cf9457b"). InnerVolumeSpecName "alertmanager-trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 06 18:57:35.471057 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:57:35.470621 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/06b795c7-2ff5-4057-ab00-489a7cf9457b-secret-alertmanager-kube-rbac-proxy-web\") pod \"06b795c7-2ff5-4057-ab00-489a7cf9457b\" (UID: \"06b795c7-2ff5-4057-ab00-489a7cf9457b\") " Jul 06 18:57:35.471057 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:57:35.470793 2570 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/06b795c7-2ff5-4057-ab00-489a7cf9457b-alertmanager-main-db" (OuterVolumeSpecName: "alertmanager-main-db") pod "06b795c7-2ff5-4057-ab00-489a7cf9457b" (UID: "06b795c7-2ff5-4057-ab00-489a7cf9457b"). InnerVolumeSpecName "alertmanager-main-db". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 18:57:35.471057 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:57:35.471040 2570 reconciler_common.go:299] "Volume detached for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/06b795c7-2ff5-4057-ab00-489a7cf9457b-alertmanager-trusted-ca-bundle\") on node \"ip-10-0-130-59.ec2.internal\" DevicePath \"\"" Jul 06 18:57:35.471207 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:57:35.471061 2570 reconciler_common.go:299] "Volume detached for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/06b795c7-2ff5-4057-ab00-489a7cf9457b-alertmanager-main-db\") on node \"ip-10-0-130-59.ec2.internal\" DevicePath \"\"" Jul 06 18:57:35.472301 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:57:35.472270 2570 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/06b795c7-2ff5-4057-ab00-489a7cf9457b-metrics-client-ca" (OuterVolumeSpecName: "metrics-client-ca") pod "06b795c7-2ff5-4057-ab00-489a7cf9457b" (UID: "06b795c7-2ff5-4057-ab00-489a7cf9457b"). InnerVolumeSpecName "metrics-client-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 06 18:57:35.473261 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:57:35.473223 2570 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/06b795c7-2ff5-4057-ab00-489a7cf9457b-config-volume" (OuterVolumeSpecName: "config-volume") pod "06b795c7-2ff5-4057-ab00-489a7cf9457b" (UID: "06b795c7-2ff5-4057-ab00-489a7cf9457b"). InnerVolumeSpecName "config-volume". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 06 18:57:35.474130 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:57:35.474095 2570 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/06b795c7-2ff5-4057-ab00-489a7cf9457b-secret-alertmanager-kube-rbac-proxy" (OuterVolumeSpecName: "secret-alertmanager-kube-rbac-proxy") pod "06b795c7-2ff5-4057-ab00-489a7cf9457b" (UID: "06b795c7-2ff5-4057-ab00-489a7cf9457b"). InnerVolumeSpecName "secret-alertmanager-kube-rbac-proxy". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 06 18:57:35.475936 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:57:35.475873 2570 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/06b795c7-2ff5-4057-ab00-489a7cf9457b-secret-alertmanager-main-tls" (OuterVolumeSpecName: "secret-alertmanager-main-tls") pod "06b795c7-2ff5-4057-ab00-489a7cf9457b" (UID: "06b795c7-2ff5-4057-ab00-489a7cf9457b"). InnerVolumeSpecName "secret-alertmanager-main-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 06 18:57:35.476116 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:57:35.476038 2570 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/06b795c7-2ff5-4057-ab00-489a7cf9457b-secret-alertmanager-kube-rbac-proxy-web" (OuterVolumeSpecName: "secret-alertmanager-kube-rbac-proxy-web") pod "06b795c7-2ff5-4057-ab00-489a7cf9457b" (UID: "06b795c7-2ff5-4057-ab00-489a7cf9457b"). InnerVolumeSpecName "secret-alertmanager-kube-rbac-proxy-web". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 06 18:57:35.476210 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:57:35.475879 2570 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/06b795c7-2ff5-4057-ab00-489a7cf9457b-tls-assets" (OuterVolumeSpecName: "tls-assets") pod "06b795c7-2ff5-4057-ab00-489a7cf9457b" (UID: "06b795c7-2ff5-4057-ab00-489a7cf9457b"). InnerVolumeSpecName "tls-assets". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 06 18:57:35.476291 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:57:35.476269 2570 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/06b795c7-2ff5-4057-ab00-489a7cf9457b-config-out" (OuterVolumeSpecName: "config-out") pod "06b795c7-2ff5-4057-ab00-489a7cf9457b" (UID: "06b795c7-2ff5-4057-ab00-489a7cf9457b"). InnerVolumeSpecName "config-out". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 18:57:35.476291 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:57:35.476271 2570 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/06b795c7-2ff5-4057-ab00-489a7cf9457b-kube-api-access-sp6c4" (OuterVolumeSpecName: "kube-api-access-sp6c4") pod "06b795c7-2ff5-4057-ab00-489a7cf9457b" (UID: "06b795c7-2ff5-4057-ab00-489a7cf9457b"). InnerVolumeSpecName "kube-api-access-sp6c4". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 06 18:57:35.476584 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:57:35.476564 2570 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/06b795c7-2ff5-4057-ab00-489a7cf9457b-secret-alertmanager-kube-rbac-proxy-metric" (OuterVolumeSpecName: "secret-alertmanager-kube-rbac-proxy-metric") pod "06b795c7-2ff5-4057-ab00-489a7cf9457b" (UID: "06b795c7-2ff5-4057-ab00-489a7cf9457b"). InnerVolumeSpecName "secret-alertmanager-kube-rbac-proxy-metric". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 06 18:57:35.480760 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:57:35.480644 2570 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/06b795c7-2ff5-4057-ab00-489a7cf9457b-cluster-tls-config" (OuterVolumeSpecName: "cluster-tls-config") pod "06b795c7-2ff5-4057-ab00-489a7cf9457b" (UID: "06b795c7-2ff5-4057-ab00-489a7cf9457b"). InnerVolumeSpecName "cluster-tls-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 06 18:57:35.486481 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:57:35.486432 2570 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/06b795c7-2ff5-4057-ab00-489a7cf9457b-web-config" (OuterVolumeSpecName: "web-config") pod "06b795c7-2ff5-4057-ab00-489a7cf9457b" (UID: "06b795c7-2ff5-4057-ab00-489a7cf9457b"). InnerVolumeSpecName "web-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 06 18:57:35.571532 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:57:35.571504 2570 reconciler_common.go:299] "Volume detached for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/06b795c7-2ff5-4057-ab00-489a7cf9457b-config-volume\") on node \"ip-10-0-130-59.ec2.internal\" DevicePath \"\"" Jul 06 18:57:35.571532 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:57:35.571527 2570 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/06b795c7-2ff5-4057-ab00-489a7cf9457b-secret-alertmanager-kube-rbac-proxy-metric\") on node \"ip-10-0-130-59.ec2.internal\" DevicePath \"\"" Jul 06 18:57:35.571687 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:57:35.571537 2570 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/06b795c7-2ff5-4057-ab00-489a7cf9457b-secret-alertmanager-kube-rbac-proxy-web\") on node \"ip-10-0-130-59.ec2.internal\" DevicePath \"\"" Jul 06 18:57:35.571687 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:57:35.571546 2570 reconciler_common.go:299] "Volume detached for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/06b795c7-2ff5-4057-ab00-489a7cf9457b-tls-assets\") on node \"ip-10-0-130-59.ec2.internal\" DevicePath \"\"" Jul 06 18:57:35.571687 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:57:35.571555 2570 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/06b795c7-2ff5-4057-ab00-489a7cf9457b-secret-alertmanager-kube-rbac-proxy\") on node \"ip-10-0-130-59.ec2.internal\" DevicePath \"\"" Jul 06 18:57:35.571687 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:57:35.571565 2570 reconciler_common.go:299] "Volume detached for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/06b795c7-2ff5-4057-ab00-489a7cf9457b-metrics-client-ca\") on node \"ip-10-0-130-59.ec2.internal\" DevicePath \"\"" Jul 06 18:57:35.571687 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:57:35.571573 2570 reconciler_common.go:299] "Volume detached for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/06b795c7-2ff5-4057-ab00-489a7cf9457b-web-config\") on node \"ip-10-0-130-59.ec2.internal\" DevicePath \"\"" Jul 06 18:57:35.571687 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:57:35.571581 2570 reconciler_common.go:299] "Volume detached for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/06b795c7-2ff5-4057-ab00-489a7cf9457b-cluster-tls-config\") on node \"ip-10-0-130-59.ec2.internal\" DevicePath \"\"" Jul 06 18:57:35.571687 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:57:35.571589 2570 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-sp6c4\" (UniqueName: \"kubernetes.io/projected/06b795c7-2ff5-4057-ab00-489a7cf9457b-kube-api-access-sp6c4\") on node \"ip-10-0-130-59.ec2.internal\" DevicePath \"\"" Jul 06 18:57:35.571687 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:57:35.571598 2570 reconciler_common.go:299] "Volume detached for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/06b795c7-2ff5-4057-ab00-489a7cf9457b-config-out\") on node \"ip-10-0-130-59.ec2.internal\" DevicePath \"\"" Jul 06 18:57:35.571687 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:57:35.571607 2570 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/06b795c7-2ff5-4057-ab00-489a7cf9457b-secret-alertmanager-main-tls\") on node \"ip-10-0-130-59.ec2.internal\" DevicePath \"\"" Jul 06 18:57:35.751379 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:57:35.751305 2570 generic.go:356] "Generic (PLEG): container finished" podID="06b795c7-2ff5-4057-ab00-489a7cf9457b" containerID="553b82728c7228e54f93eb9d35464feebbd33b9aa2c96d5e4ae48cef36947aae" exitCode=0 Jul 06 18:57:35.751506 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:57:35.751381 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"06b795c7-2ff5-4057-ab00-489a7cf9457b","Type":"ContainerDied","Data":"553b82728c7228e54f93eb9d35464feebbd33b9aa2c96d5e4ae48cef36947aae"} Jul 06 18:57:35.751506 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:57:35.751394 2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Jul 06 18:57:35.751506 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:57:35.751450 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"06b795c7-2ff5-4057-ab00-489a7cf9457b","Type":"ContainerDied","Data":"3919886510a897a05da82605824404c74654d32b8b1f3ac98d55664d6e1e2f01"} Jul 06 18:57:35.751506 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:57:35.751485 2570 scope.go:117] "RemoveContainer" containerID="090ca2e6ac974757f9c0720b6495efd4b3246a4ed03262d9440002358c05338d" Jul 06 18:57:35.759157 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:57:35.759044 2570 scope.go:117] "RemoveContainer" containerID="34d7c70a78642e1ef5ef61eb3a721dc61438a8bf6a2d8afdcb31842aeedcc1f0" Jul 06 18:57:35.766019 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:57:35.766002 2570 scope.go:117] "RemoveContainer" containerID="a32c5b0db91a42bbaf6723866951bd6014c5403d2a8cd03f6c69215cadb8395b" Jul 06 18:57:35.772824 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:57:35.772809 2570 scope.go:117] "RemoveContainer" containerID="553b82728c7228e54f93eb9d35464feebbd33b9aa2c96d5e4ae48cef36947aae" Jul 06 18:57:35.777425 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:57:35.777406 2570 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 06 18:57:35.780131 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:57:35.780112 2570 scope.go:117] "RemoveContainer" containerID="702bdcefebee8c3053467fdab3369df0fe728f3b39ca6b1af7c9b61687904729" Jul 06 18:57:35.784061 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:57:35.784039 2570 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 06 18:57:35.787348 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:57:35.787323 2570 scope.go:117] "RemoveContainer" containerID="09a3d2934fd8c9aa70ec05da33359b5009842c5513feecf5e645b544aa90af71" Jul 06 18:57:35.793965 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:57:35.793945 2570 scope.go:117] "RemoveContainer" containerID="8509d0c990ed24211a43112e70124fc9b8f5d8407be2694638decb4cbab460ee" Jul 06 18:57:35.800250 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:57:35.800235 2570 scope.go:117] "RemoveContainer" containerID="090ca2e6ac974757f9c0720b6495efd4b3246a4ed03262d9440002358c05338d" Jul 06 18:57:35.800480 ip-10-0-130-59 kubenswrapper[2570]: E0706 18:57:35.800463 2570 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"090ca2e6ac974757f9c0720b6495efd4b3246a4ed03262d9440002358c05338d\": container with ID starting with 090ca2e6ac974757f9c0720b6495efd4b3246a4ed03262d9440002358c05338d not found: ID does not exist" containerID="090ca2e6ac974757f9c0720b6495efd4b3246a4ed03262d9440002358c05338d" Jul 06 18:57:35.800525 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:57:35.800488 2570 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"090ca2e6ac974757f9c0720b6495efd4b3246a4ed03262d9440002358c05338d"} err="failed to get container status \"090ca2e6ac974757f9c0720b6495efd4b3246a4ed03262d9440002358c05338d\": rpc error: code = NotFound desc = could not find container \"090ca2e6ac974757f9c0720b6495efd4b3246a4ed03262d9440002358c05338d\": container with ID starting with 090ca2e6ac974757f9c0720b6495efd4b3246a4ed03262d9440002358c05338d not found: ID does not exist" Jul 06 18:57:35.800525 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:57:35.800505 2570 scope.go:117] "RemoveContainer" containerID="34d7c70a78642e1ef5ef61eb3a721dc61438a8bf6a2d8afdcb31842aeedcc1f0" Jul 06 18:57:35.800745 ip-10-0-130-59 kubenswrapper[2570]: E0706 18:57:35.800721 2570 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"34d7c70a78642e1ef5ef61eb3a721dc61438a8bf6a2d8afdcb31842aeedcc1f0\": container with ID starting with 34d7c70a78642e1ef5ef61eb3a721dc61438a8bf6a2d8afdcb31842aeedcc1f0 not found: ID does not exist" containerID="34d7c70a78642e1ef5ef61eb3a721dc61438a8bf6a2d8afdcb31842aeedcc1f0" Jul 06 18:57:35.800745 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:57:35.800740 2570 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"34d7c70a78642e1ef5ef61eb3a721dc61438a8bf6a2d8afdcb31842aeedcc1f0"} err="failed to get container status \"34d7c70a78642e1ef5ef61eb3a721dc61438a8bf6a2d8afdcb31842aeedcc1f0\": rpc error: code = NotFound desc = could not find container \"34d7c70a78642e1ef5ef61eb3a721dc61438a8bf6a2d8afdcb31842aeedcc1f0\": container with ID starting with 34d7c70a78642e1ef5ef61eb3a721dc61438a8bf6a2d8afdcb31842aeedcc1f0 not found: ID does not exist" Jul 06 18:57:35.800839 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:57:35.800752 2570 scope.go:117] "RemoveContainer" containerID="a32c5b0db91a42bbaf6723866951bd6014c5403d2a8cd03f6c69215cadb8395b" Jul 06 18:57:35.800951 ip-10-0-130-59 kubenswrapper[2570]: E0706 18:57:35.800935 2570 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a32c5b0db91a42bbaf6723866951bd6014c5403d2a8cd03f6c69215cadb8395b\": container with ID starting with a32c5b0db91a42bbaf6723866951bd6014c5403d2a8cd03f6c69215cadb8395b not found: ID does not exist" containerID="a32c5b0db91a42bbaf6723866951bd6014c5403d2a8cd03f6c69215cadb8395b" Jul 06 18:57:35.800984 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:57:35.800958 2570 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a32c5b0db91a42bbaf6723866951bd6014c5403d2a8cd03f6c69215cadb8395b"} err="failed to get container status \"a32c5b0db91a42bbaf6723866951bd6014c5403d2a8cd03f6c69215cadb8395b\": rpc error: code = NotFound desc = could not find container \"a32c5b0db91a42bbaf6723866951bd6014c5403d2a8cd03f6c69215cadb8395b\": container with ID starting with a32c5b0db91a42bbaf6723866951bd6014c5403d2a8cd03f6c69215cadb8395b not found: ID does not exist" Jul 06 18:57:35.800984 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:57:35.800981 2570 scope.go:117] "RemoveContainer" containerID="553b82728c7228e54f93eb9d35464feebbd33b9aa2c96d5e4ae48cef36947aae" Jul 06 18:57:35.801204 ip-10-0-130-59 kubenswrapper[2570]: E0706 18:57:35.801187 2570 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"553b82728c7228e54f93eb9d35464feebbd33b9aa2c96d5e4ae48cef36947aae\": container with ID starting with 553b82728c7228e54f93eb9d35464feebbd33b9aa2c96d5e4ae48cef36947aae not found: ID does not exist" containerID="553b82728c7228e54f93eb9d35464feebbd33b9aa2c96d5e4ae48cef36947aae" Jul 06 18:57:35.801255 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:57:35.801207 2570 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"553b82728c7228e54f93eb9d35464feebbd33b9aa2c96d5e4ae48cef36947aae"} err="failed to get container status \"553b82728c7228e54f93eb9d35464feebbd33b9aa2c96d5e4ae48cef36947aae\": rpc error: code = NotFound desc = could not find container \"553b82728c7228e54f93eb9d35464feebbd33b9aa2c96d5e4ae48cef36947aae\": container with ID starting with 553b82728c7228e54f93eb9d35464feebbd33b9aa2c96d5e4ae48cef36947aae not found: ID does not exist" Jul 06 18:57:35.801255 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:57:35.801220 2570 scope.go:117] "RemoveContainer" containerID="702bdcefebee8c3053467fdab3369df0fe728f3b39ca6b1af7c9b61687904729" Jul 06 18:57:35.801426 ip-10-0-130-59 kubenswrapper[2570]: E0706 18:57:35.801411 2570 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"702bdcefebee8c3053467fdab3369df0fe728f3b39ca6b1af7c9b61687904729\": container with ID starting with 702bdcefebee8c3053467fdab3369df0fe728f3b39ca6b1af7c9b61687904729 not found: ID does not exist" containerID="702bdcefebee8c3053467fdab3369df0fe728f3b39ca6b1af7c9b61687904729" Jul 06 18:57:35.801470 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:57:35.801429 2570 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"702bdcefebee8c3053467fdab3369df0fe728f3b39ca6b1af7c9b61687904729"} err="failed to get container status \"702bdcefebee8c3053467fdab3369df0fe728f3b39ca6b1af7c9b61687904729\": rpc error: code = NotFound desc = could not find container \"702bdcefebee8c3053467fdab3369df0fe728f3b39ca6b1af7c9b61687904729\": container with ID starting with 702bdcefebee8c3053467fdab3369df0fe728f3b39ca6b1af7c9b61687904729 not found: ID does not exist" Jul 06 18:57:35.801470 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:57:35.801442 2570 scope.go:117] "RemoveContainer" containerID="09a3d2934fd8c9aa70ec05da33359b5009842c5513feecf5e645b544aa90af71" Jul 06 18:57:35.801649 ip-10-0-130-59 kubenswrapper[2570]: E0706 18:57:35.801635 2570 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"09a3d2934fd8c9aa70ec05da33359b5009842c5513feecf5e645b544aa90af71\": container with ID starting with 09a3d2934fd8c9aa70ec05da33359b5009842c5513feecf5e645b544aa90af71 not found: ID does not exist" containerID="09a3d2934fd8c9aa70ec05da33359b5009842c5513feecf5e645b544aa90af71" Jul 06 18:57:35.801686 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:57:35.801652 2570 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"09a3d2934fd8c9aa70ec05da33359b5009842c5513feecf5e645b544aa90af71"} err="failed to get container status \"09a3d2934fd8c9aa70ec05da33359b5009842c5513feecf5e645b544aa90af71\": rpc error: code = NotFound desc = could not find container \"09a3d2934fd8c9aa70ec05da33359b5009842c5513feecf5e645b544aa90af71\": container with ID starting with 09a3d2934fd8c9aa70ec05da33359b5009842c5513feecf5e645b544aa90af71 not found: ID does not exist" Jul 06 18:57:35.801686 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:57:35.801664 2570 scope.go:117] "RemoveContainer" containerID="8509d0c990ed24211a43112e70124fc9b8f5d8407be2694638decb4cbab460ee" Jul 06 18:57:35.801925 ip-10-0-130-59 kubenswrapper[2570]: E0706 18:57:35.801911 2570 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8509d0c990ed24211a43112e70124fc9b8f5d8407be2694638decb4cbab460ee\": container with ID starting with 8509d0c990ed24211a43112e70124fc9b8f5d8407be2694638decb4cbab460ee not found: ID does not exist" containerID="8509d0c990ed24211a43112e70124fc9b8f5d8407be2694638decb4cbab460ee" Jul 06 18:57:35.801964 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:57:35.801928 2570 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8509d0c990ed24211a43112e70124fc9b8f5d8407be2694638decb4cbab460ee"} err="failed to get container status \"8509d0c990ed24211a43112e70124fc9b8f5d8407be2694638decb4cbab460ee\": rpc error: code = NotFound desc = could not find container \"8509d0c990ed24211a43112e70124fc9b8f5d8407be2694638decb4cbab460ee\": container with ID starting with 8509d0c990ed24211a43112e70124fc9b8f5d8407be2694638decb4cbab460ee not found: ID does not exist" Jul 06 18:57:35.813063 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:57:35.813043 2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 06 18:57:35.813319 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:57:35.813307 2570 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="06b795c7-2ff5-4057-ab00-489a7cf9457b" containerName="config-reloader" Jul 06 18:57:35.813319 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:57:35.813320 2570 state_mem.go:107] "Deleted CPUSet assignment" podUID="06b795c7-2ff5-4057-ab00-489a7cf9457b" containerName="config-reloader" Jul 06 18:57:35.813397 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:57:35.813330 2570 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="06b795c7-2ff5-4057-ab00-489a7cf9457b" containerName="kube-rbac-proxy" Jul 06 18:57:35.813397 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:57:35.813336 2570 state_mem.go:107] "Deleted CPUSet assignment" podUID="06b795c7-2ff5-4057-ab00-489a7cf9457b" containerName="kube-rbac-proxy" Jul 06 18:57:35.813397 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:57:35.813346 2570 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="06b795c7-2ff5-4057-ab00-489a7cf9457b" containerName="prom-label-proxy" Jul 06 18:57:35.813397 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:57:35.813352 2570 state_mem.go:107] "Deleted CPUSet assignment" podUID="06b795c7-2ff5-4057-ab00-489a7cf9457b" containerName="prom-label-proxy" Jul 06 18:57:35.813397 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:57:35.813359 2570 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="06b795c7-2ff5-4057-ab00-489a7cf9457b" containerName="kube-rbac-proxy-web" Jul 06 18:57:35.813397 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:57:35.813364 2570 state_mem.go:107] "Deleted CPUSet assignment" podUID="06b795c7-2ff5-4057-ab00-489a7cf9457b" containerName="kube-rbac-proxy-web" Jul 06 18:57:35.813397 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:57:35.813372 2570 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="06b795c7-2ff5-4057-ab00-489a7cf9457b" containerName="alertmanager" Jul 06 18:57:35.813397 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:57:35.813378 2570 state_mem.go:107] "Deleted CPUSet assignment" podUID="06b795c7-2ff5-4057-ab00-489a7cf9457b" containerName="alertmanager" Jul 06 18:57:35.813397 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:57:35.813384 2570 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="06b795c7-2ff5-4057-ab00-489a7cf9457b" containerName="init-config-reloader" Jul 06 18:57:35.813397 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:57:35.813390 2570 state_mem.go:107] "Deleted CPUSet assignment" podUID="06b795c7-2ff5-4057-ab00-489a7cf9457b" containerName="init-config-reloader" Jul 06 18:57:35.813397 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:57:35.813396 2570 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3afaab40-a812-4cd4-ab65-bc0da82cb738" containerName="console" Jul 06 18:57:35.813397 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:57:35.813400 2570 state_mem.go:107] "Deleted CPUSet assignment" podUID="3afaab40-a812-4cd4-ab65-bc0da82cb738" containerName="console" Jul 06 18:57:35.813770 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:57:35.813406 2570 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="06b795c7-2ff5-4057-ab00-489a7cf9457b" containerName="kube-rbac-proxy-metric" Jul 06 18:57:35.813770 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:57:35.813411 2570 state_mem.go:107] "Deleted CPUSet assignment" podUID="06b795c7-2ff5-4057-ab00-489a7cf9457b" containerName="kube-rbac-proxy-metric" Jul 06 18:57:35.813770 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:57:35.813467 2570 memory_manager.go:368] "RemoveStaleState removing state" podUID="3afaab40-a812-4cd4-ab65-bc0da82cb738" containerName="console" Jul 06 18:57:35.813770 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:57:35.813474 2570 memory_manager.go:368] "RemoveStaleState removing state" podUID="06b795c7-2ff5-4057-ab00-489a7cf9457b" containerName="kube-rbac-proxy" Jul 06 18:57:35.813770 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:57:35.813480 2570 memory_manager.go:368] "RemoveStaleState removing state" podUID="06b795c7-2ff5-4057-ab00-489a7cf9457b" containerName="alertmanager" Jul 06 18:57:35.813770 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:57:35.813485 2570 memory_manager.go:368] "RemoveStaleState removing state" podUID="06b795c7-2ff5-4057-ab00-489a7cf9457b" containerName="config-reloader" Jul 06 18:57:35.813770 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:57:35.813491 2570 memory_manager.go:368] "RemoveStaleState removing state" podUID="06b795c7-2ff5-4057-ab00-489a7cf9457b" containerName="kube-rbac-proxy-web" Jul 06 18:57:35.813770 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:57:35.813499 2570 memory_manager.go:368] "RemoveStaleState removing state" podUID="06b795c7-2ff5-4057-ab00-489a7cf9457b" containerName="prom-label-proxy" Jul 06 18:57:35.813770 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:57:35.813505 2570 memory_manager.go:368] "RemoveStaleState removing state" podUID="06b795c7-2ff5-4057-ab00-489a7cf9457b" containerName="kube-rbac-proxy-metric" Jul 06 18:57:35.818882 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:57:35.818504 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Jul 06 18:57:35.822268 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:57:35.822245 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-cluster-tls-config\"" Jul 06 18:57:35.822364 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:57:35.822245 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-web\"" Jul 06 18:57:35.822655 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:57:35.822602 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls-assets-0\"" Jul 06 18:57:35.822655 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:57:35.822602 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-generated\"" Jul 06 18:57:35.822801 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:57:35.822634 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy\"" Jul 06 18:57:35.822801 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:57:35.822758 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-dockercfg-m4nh6\"" Jul 06 18:57:35.822913 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:57:35.822896 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-web-config\"" Jul 06 18:57:35.823037 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:57:35.823022 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls\"" Jul 06 18:57:35.823863 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:57:35.823846 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-metric\"" Jul 06 18:57:35.827877 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:57:35.827859 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"alertmanager-trusted-ca-bundle\"" Jul 06 18:57:35.835361 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:57:35.835339 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 06 18:57:35.873415 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:57:35.873394 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/0b59ed61-5ebf-4244-8ae9-1bd151f4673f-config-volume\") pod \"alertmanager-main-0\" (UID: \"0b59ed61-5ebf-4244-8ae9-1bd151f4673f\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 18:57:35.873415 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:57:35.873422 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/0b59ed61-5ebf-4244-8ae9-1bd151f4673f-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"0b59ed61-5ebf-4244-8ae9-1bd151f4673f\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 18:57:35.873574 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:57:35.873440 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/0b59ed61-5ebf-4244-8ae9-1bd151f4673f-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"0b59ed61-5ebf-4244-8ae9-1bd151f4673f\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 18:57:35.873574 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:57:35.873459 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/0b59ed61-5ebf-4244-8ae9-1bd151f4673f-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"0b59ed61-5ebf-4244-8ae9-1bd151f4673f\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 18:57:35.873574 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:57:35.873484 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/0b59ed61-5ebf-4244-8ae9-1bd151f4673f-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"0b59ed61-5ebf-4244-8ae9-1bd151f4673f\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 18:57:35.873574 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:57:35.873505 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/0b59ed61-5ebf-4244-8ae9-1bd151f4673f-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"0b59ed61-5ebf-4244-8ae9-1bd151f4673f\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 18:57:35.873574 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:57:35.873524 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/0b59ed61-5ebf-4244-8ae9-1bd151f4673f-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"0b59ed61-5ebf-4244-8ae9-1bd151f4673f\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 18:57:35.873574 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:57:35.873537 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mtzqf\" (UniqueName: \"kubernetes.io/projected/0b59ed61-5ebf-4244-8ae9-1bd151f4673f-kube-api-access-mtzqf\") pod \"alertmanager-main-0\" (UID: \"0b59ed61-5ebf-4244-8ae9-1bd151f4673f\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 18:57:35.873574 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:57:35.873562 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/0b59ed61-5ebf-4244-8ae9-1bd151f4673f-web-config\") pod \"alertmanager-main-0\" (UID: \"0b59ed61-5ebf-4244-8ae9-1bd151f4673f\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 18:57:35.873918 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:57:35.873684 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/0b59ed61-5ebf-4244-8ae9-1bd151f4673f-tls-assets\") pod \"alertmanager-main-0\" (UID: \"0b59ed61-5ebf-4244-8ae9-1bd151f4673f\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 18:57:35.873918 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:57:35.873717 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/0b59ed61-5ebf-4244-8ae9-1bd151f4673f-config-out\") pod \"alertmanager-main-0\" (UID: \"0b59ed61-5ebf-4244-8ae9-1bd151f4673f\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 18:57:35.873918 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:57:35.873749 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/0b59ed61-5ebf-4244-8ae9-1bd151f4673f-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"0b59ed61-5ebf-4244-8ae9-1bd151f4673f\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 18:57:35.873918 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:57:35.873778 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/0b59ed61-5ebf-4244-8ae9-1bd151f4673f-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"0b59ed61-5ebf-4244-8ae9-1bd151f4673f\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 18:57:35.974482 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:57:35.974450 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/0b59ed61-5ebf-4244-8ae9-1bd151f4673f-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"0b59ed61-5ebf-4244-8ae9-1bd151f4673f\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 18:57:35.974588 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:57:35.974490 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/0b59ed61-5ebf-4244-8ae9-1bd151f4673f-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"0b59ed61-5ebf-4244-8ae9-1bd151f4673f\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 18:57:35.974588 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:57:35.974545 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/0b59ed61-5ebf-4244-8ae9-1bd151f4673f-config-volume\") pod \"alertmanager-main-0\" (UID: \"0b59ed61-5ebf-4244-8ae9-1bd151f4673f\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 18:57:35.974588 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:57:35.974578 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/0b59ed61-5ebf-4244-8ae9-1bd151f4673f-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"0b59ed61-5ebf-4244-8ae9-1bd151f4673f\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 18:57:35.974760 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:57:35.974606 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/0b59ed61-5ebf-4244-8ae9-1bd151f4673f-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"0b59ed61-5ebf-4244-8ae9-1bd151f4673f\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 18:57:35.974760 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:57:35.974731 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/0b59ed61-5ebf-4244-8ae9-1bd151f4673f-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"0b59ed61-5ebf-4244-8ae9-1bd151f4673f\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 18:57:35.974859 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:57:35.974769 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/0b59ed61-5ebf-4244-8ae9-1bd151f4673f-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"0b59ed61-5ebf-4244-8ae9-1bd151f4673f\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 18:57:35.974859 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:57:35.974794 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/0b59ed61-5ebf-4244-8ae9-1bd151f4673f-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"0b59ed61-5ebf-4244-8ae9-1bd151f4673f\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 18:57:35.974859 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:57:35.974819 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/0b59ed61-5ebf-4244-8ae9-1bd151f4673f-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"0b59ed61-5ebf-4244-8ae9-1bd151f4673f\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 18:57:35.974859 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:57:35.974839 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mtzqf\" (UniqueName: \"kubernetes.io/projected/0b59ed61-5ebf-4244-8ae9-1bd151f4673f-kube-api-access-mtzqf\") pod \"alertmanager-main-0\" (UID: \"0b59ed61-5ebf-4244-8ae9-1bd151f4673f\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 18:57:35.975792 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:57:35.975320 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/0b59ed61-5ebf-4244-8ae9-1bd151f4673f-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"0b59ed61-5ebf-4244-8ae9-1bd151f4673f\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 18:57:35.975792 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:57:35.975329 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/0b59ed61-5ebf-4244-8ae9-1bd151f4673f-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"0b59ed61-5ebf-4244-8ae9-1bd151f4673f\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 18:57:35.975792 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:57:35.975383 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/0b59ed61-5ebf-4244-8ae9-1bd151f4673f-web-config\") pod \"alertmanager-main-0\" (UID: \"0b59ed61-5ebf-4244-8ae9-1bd151f4673f\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 18:57:35.975792 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:57:35.975436 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/0b59ed61-5ebf-4244-8ae9-1bd151f4673f-tls-assets\") pod \"alertmanager-main-0\" (UID: \"0b59ed61-5ebf-4244-8ae9-1bd151f4673f\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 18:57:35.975792 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:57:35.975458 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/0b59ed61-5ebf-4244-8ae9-1bd151f4673f-config-out\") pod \"alertmanager-main-0\" (UID: \"0b59ed61-5ebf-4244-8ae9-1bd151f4673f\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 18:57:35.975792 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:57:35.975383 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/0b59ed61-5ebf-4244-8ae9-1bd151f4673f-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"0b59ed61-5ebf-4244-8ae9-1bd151f4673f\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 18:57:35.977743 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:57:35.977719 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/0b59ed61-5ebf-4244-8ae9-1bd151f4673f-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"0b59ed61-5ebf-4244-8ae9-1bd151f4673f\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 18:57:35.977836 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:57:35.977808 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/0b59ed61-5ebf-4244-8ae9-1bd151f4673f-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"0b59ed61-5ebf-4244-8ae9-1bd151f4673f\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 18:57:35.977895 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:57:35.977866 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/0b59ed61-5ebf-4244-8ae9-1bd151f4673f-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"0b59ed61-5ebf-4244-8ae9-1bd151f4673f\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 18:57:35.977957 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:57:35.977866 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/0b59ed61-5ebf-4244-8ae9-1bd151f4673f-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"0b59ed61-5ebf-4244-8ae9-1bd151f4673f\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 18:57:35.978111 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:57:35.978088 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/0b59ed61-5ebf-4244-8ae9-1bd151f4673f-config-volume\") pod \"alertmanager-main-0\" (UID: \"0b59ed61-5ebf-4244-8ae9-1bd151f4673f\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 18:57:35.978111 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:57:35.978099 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/0b59ed61-5ebf-4244-8ae9-1bd151f4673f-web-config\") pod \"alertmanager-main-0\" (UID: \"0b59ed61-5ebf-4244-8ae9-1bd151f4673f\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 18:57:35.978216 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:57:35.978121 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/0b59ed61-5ebf-4244-8ae9-1bd151f4673f-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"0b59ed61-5ebf-4244-8ae9-1bd151f4673f\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 18:57:35.978347 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:57:35.978334 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/0b59ed61-5ebf-4244-8ae9-1bd151f4673f-config-out\") pod \"alertmanager-main-0\" (UID: \"0b59ed61-5ebf-4244-8ae9-1bd151f4673f\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 18:57:35.979562 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:57:35.979545 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/0b59ed61-5ebf-4244-8ae9-1bd151f4673f-tls-assets\") pod \"alertmanager-main-0\" (UID: \"0b59ed61-5ebf-4244-8ae9-1bd151f4673f\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 18:57:35.984344 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:57:35.984324 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-mtzqf\" (UniqueName: \"kubernetes.io/projected/0b59ed61-5ebf-4244-8ae9-1bd151f4673f-kube-api-access-mtzqf\") pod \"alertmanager-main-0\" (UID: \"0b59ed61-5ebf-4244-8ae9-1bd151f4673f\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 18:57:36.128304 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:57:36.128245 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Jul 06 18:57:36.261998 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:57:36.261972 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 06 18:57:36.264580 ip-10-0-130-59 kubenswrapper[2570]: W0706 18:57:36.264552 2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0b59ed61_5ebf_4244_8ae9_1bd151f4673f.slice/crio-de96370302c617f4bc7c3320133f36e36e67caea865cecad8642f5e341c1fd14 WatchSource:0}: Error finding container de96370302c617f4bc7c3320133f36e36e67caea865cecad8642f5e341c1fd14: Status 404 returned error can't find the container with id de96370302c617f4bc7c3320133f36e36e67caea865cecad8642f5e341c1fd14 Jul 06 18:57:36.757009 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:57:36.756975 2570 generic.go:356] "Generic (PLEG): container finished" podID="0b59ed61-5ebf-4244-8ae9-1bd151f4673f" containerID="5e6a6328bd016aa57de4038017656a4c90680b3efe53a8a899144a2b47005d21" exitCode=0 Jul 06 18:57:36.757349 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:57:36.757064 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"0b59ed61-5ebf-4244-8ae9-1bd151f4673f","Type":"ContainerDied","Data":"5e6a6328bd016aa57de4038017656a4c90680b3efe53a8a899144a2b47005d21"} Jul 06 18:57:36.757349 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:57:36.757123 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"0b59ed61-5ebf-4244-8ae9-1bd151f4673f","Type":"ContainerStarted","Data":"de96370302c617f4bc7c3320133f36e36e67caea865cecad8642f5e341c1fd14"} Jul 06 18:57:37.027506 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:57:37.027480 2570 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="06b795c7-2ff5-4057-ab00-489a7cf9457b" path="/var/lib/kubelet/pods/06b795c7-2ff5-4057-ab00-489a7cf9457b/volumes" Jul 06 18:57:37.765018 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:57:37.764990 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"0b59ed61-5ebf-4244-8ae9-1bd151f4673f","Type":"ContainerStarted","Data":"8379eca48246aace610a4d0001835d88c7316d4ab2e7b3370db86662eecc1a26"} Jul 06 18:57:37.765362 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:57:37.765048 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"0b59ed61-5ebf-4244-8ae9-1bd151f4673f","Type":"ContainerStarted","Data":"f99f23a217c1f24d8857361405427ab377f3b6f69a487b07ec0f7a846ece969b"} Jul 06 18:57:37.765362 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:57:37.765076 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"0b59ed61-5ebf-4244-8ae9-1bd151f4673f","Type":"ContainerStarted","Data":"37723ffbd66d99cf50b90e250ddcf4adc30fa25c0ba8580ca53fd0f6eecaf23d"} Jul 06 18:57:37.765362 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:57:37.765101 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"0b59ed61-5ebf-4244-8ae9-1bd151f4673f","Type":"ContainerStarted","Data":"225812f3a2416c68de8c89d91f522229ff13b0256cc06c425cab096e0b5caa29"} Jul 06 18:57:37.765362 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:57:37.765124 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"0b59ed61-5ebf-4244-8ae9-1bd151f4673f","Type":"ContainerStarted","Data":"10f88c6641d6c4effb56da613c85a82438b8b2c25ab31f4a8adfe9d2d3c7fe49"} Jul 06 18:57:37.765362 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:57:37.765147 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"0b59ed61-5ebf-4244-8ae9-1bd151f4673f","Type":"ContainerStarted","Data":"7ab14229348970625350be62f6f9eadf994c8986555a62477c62212f1af69567"} Jul 06 18:57:37.799251 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:57:37.799198 2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/alertmanager-main-0" podStartSLOduration=2.783567578 podStartE2EDuration="2.799183334s" totalImagesPullingTime="0s" totalInitContainerRuntime="15.615756ms" isStatefulPod=false podCreationTimestamp="2026-07-06 18:57:35 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 18:57:37.796403706 +0000 UTC m=+137.351424916" watchObservedRunningTime="2026-07-06 18:57:37.799183334 +0000 UTC m=+137.354204507" Jul 06 18:57:38.185175 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:57:38.185142 2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/telemeter-client-7c47bc74cb-xrsz6"] Jul 06 18:57:38.187690 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:57:38.187672 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/telemeter-client-7c47bc74cb-xrsz6" Jul 06 18:57:38.190237 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:57:38.190219 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client\"" Jul 06 18:57:38.190330 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:57:38.190261 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-kube-rbac-proxy-config\"" Jul 06 18:57:38.190330 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:57:38.190319 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemeter-client-serving-certs-ca-bundle\"" Jul 06 18:57:38.190445 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:57:38.190319 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-tls\"" Jul 06 18:57:38.190445 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:57:38.190373 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-dockercfg-pzz9w\"" Jul 06 18:57:38.190445 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:57:38.190409 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"federate-client-certs\"" Jul 06 18:57:38.192569 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:57:38.192550 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/564f0d46-248e-4a95-9416-2a6d23478fb0-serving-certs-ca-bundle\") pod \"telemeter-client-7c47bc74cb-xrsz6\" (UID: \"564f0d46-248e-4a95-9416-2a6d23478fb0\") " pod="openshift-monitoring/telemeter-client-7c47bc74cb-xrsz6" Jul 06 18:57:38.192689 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:57:38.192576 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tg7x6\" (UniqueName: \"kubernetes.io/projected/564f0d46-248e-4a95-9416-2a6d23478fb0-kube-api-access-tg7x6\") pod \"telemeter-client-7c47bc74cb-xrsz6\" (UID: \"564f0d46-248e-4a95-9416-2a6d23478fb0\") " pod="openshift-monitoring/telemeter-client-7c47bc74cb-xrsz6" Jul 06 18:57:38.192689 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:57:38.192601 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/564f0d46-248e-4a95-9416-2a6d23478fb0-federate-client-tls\") pod \"telemeter-client-7c47bc74cb-xrsz6\" (UID: \"564f0d46-248e-4a95-9416-2a6d23478fb0\") " pod="openshift-monitoring/telemeter-client-7c47bc74cb-xrsz6" Jul 06 18:57:38.192806 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:57:38.192729 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/564f0d46-248e-4a95-9416-2a6d23478fb0-telemeter-trusted-ca-bundle\") pod \"telemeter-client-7c47bc74cb-xrsz6\" (UID: \"564f0d46-248e-4a95-9416-2a6d23478fb0\") " pod="openshift-monitoring/telemeter-client-7c47bc74cb-xrsz6" Jul 06 18:57:38.192806 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:57:38.192795 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/564f0d46-248e-4a95-9416-2a6d23478fb0-metrics-client-ca\") pod \"telemeter-client-7c47bc74cb-xrsz6\" (UID: \"564f0d46-248e-4a95-9416-2a6d23478fb0\") " pod="openshift-monitoring/telemeter-client-7c47bc74cb-xrsz6" Jul 06 18:57:38.192903 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:57:38.192843 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/564f0d46-248e-4a95-9416-2a6d23478fb0-secret-telemeter-client\") pod \"telemeter-client-7c47bc74cb-xrsz6\" (UID: \"564f0d46-248e-4a95-9416-2a6d23478fb0\") " pod="openshift-monitoring/telemeter-client-7c47bc74cb-xrsz6" Jul 06 18:57:38.192946 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:57:38.192912 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/564f0d46-248e-4a95-9416-2a6d23478fb0-telemeter-client-tls\") pod \"telemeter-client-7c47bc74cb-xrsz6\" (UID: \"564f0d46-248e-4a95-9416-2a6d23478fb0\") " pod="openshift-monitoring/telemeter-client-7c47bc74cb-xrsz6" Jul 06 18:57:38.192946 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:57:38.192933 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/564f0d46-248e-4a95-9416-2a6d23478fb0-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-7c47bc74cb-xrsz6\" (UID: \"564f0d46-248e-4a95-9416-2a6d23478fb0\") " pod="openshift-monitoring/telemeter-client-7c47bc74cb-xrsz6" Jul 06 18:57:38.195470 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:57:38.195452 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemeter-trusted-ca-bundle-8i12ta5c71j38\"" Jul 06 18:57:38.206088 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:57:38.206061 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/telemeter-client-7c47bc74cb-xrsz6"] Jul 06 18:57:38.293446 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:57:38.293419 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/564f0d46-248e-4a95-9416-2a6d23478fb0-telemeter-client-tls\") pod \"telemeter-client-7c47bc74cb-xrsz6\" (UID: \"564f0d46-248e-4a95-9416-2a6d23478fb0\") " pod="openshift-monitoring/telemeter-client-7c47bc74cb-xrsz6" Jul 06 18:57:38.293601 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:57:38.293452 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/564f0d46-248e-4a95-9416-2a6d23478fb0-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-7c47bc74cb-xrsz6\" (UID: \"564f0d46-248e-4a95-9416-2a6d23478fb0\") " pod="openshift-monitoring/telemeter-client-7c47bc74cb-xrsz6" Jul 06 18:57:38.293601 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:57:38.293474 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/564f0d46-248e-4a95-9416-2a6d23478fb0-serving-certs-ca-bundle\") pod \"telemeter-client-7c47bc74cb-xrsz6\" (UID: \"564f0d46-248e-4a95-9416-2a6d23478fb0\") " pod="openshift-monitoring/telemeter-client-7c47bc74cb-xrsz6" Jul 06 18:57:38.293601 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:57:38.293545 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tg7x6\" (UniqueName: \"kubernetes.io/projected/564f0d46-248e-4a95-9416-2a6d23478fb0-kube-api-access-tg7x6\") pod \"telemeter-client-7c47bc74cb-xrsz6\" (UID: \"564f0d46-248e-4a95-9416-2a6d23478fb0\") " pod="openshift-monitoring/telemeter-client-7c47bc74cb-xrsz6" Jul 06 18:57:38.293803 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:57:38.293598 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/564f0d46-248e-4a95-9416-2a6d23478fb0-federate-client-tls\") pod \"telemeter-client-7c47bc74cb-xrsz6\" (UID: \"564f0d46-248e-4a95-9416-2a6d23478fb0\") " pod="openshift-monitoring/telemeter-client-7c47bc74cb-xrsz6" Jul 06 18:57:38.293803 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:57:38.293671 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/564f0d46-248e-4a95-9416-2a6d23478fb0-telemeter-trusted-ca-bundle\") pod \"telemeter-client-7c47bc74cb-xrsz6\" (UID: \"564f0d46-248e-4a95-9416-2a6d23478fb0\") " pod="openshift-monitoring/telemeter-client-7c47bc74cb-xrsz6" Jul 06 18:57:38.293803 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:57:38.293733 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/564f0d46-248e-4a95-9416-2a6d23478fb0-metrics-client-ca\") pod \"telemeter-client-7c47bc74cb-xrsz6\" (UID: \"564f0d46-248e-4a95-9416-2a6d23478fb0\") " pod="openshift-monitoring/telemeter-client-7c47bc74cb-xrsz6" Jul 06 18:57:38.293803 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:57:38.293785 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/564f0d46-248e-4a95-9416-2a6d23478fb0-secret-telemeter-client\") pod \"telemeter-client-7c47bc74cb-xrsz6\" (UID: \"564f0d46-248e-4a95-9416-2a6d23478fb0\") " pod="openshift-monitoring/telemeter-client-7c47bc74cb-xrsz6" Jul 06 18:57:38.294308 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:57:38.294269 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/564f0d46-248e-4a95-9416-2a6d23478fb0-serving-certs-ca-bundle\") pod \"telemeter-client-7c47bc74cb-xrsz6\" (UID: \"564f0d46-248e-4a95-9416-2a6d23478fb0\") " pod="openshift-monitoring/telemeter-client-7c47bc74cb-xrsz6" Jul 06 18:57:38.294445 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:57:38.294423 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/564f0d46-248e-4a95-9416-2a6d23478fb0-metrics-client-ca\") pod \"telemeter-client-7c47bc74cb-xrsz6\" (UID: \"564f0d46-248e-4a95-9416-2a6d23478fb0\") " pod="openshift-monitoring/telemeter-client-7c47bc74cb-xrsz6" Jul 06 18:57:38.294587 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:57:38.294559 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/564f0d46-248e-4a95-9416-2a6d23478fb0-telemeter-trusted-ca-bundle\") pod \"telemeter-client-7c47bc74cb-xrsz6\" (UID: \"564f0d46-248e-4a95-9416-2a6d23478fb0\") " pod="openshift-monitoring/telemeter-client-7c47bc74cb-xrsz6" Jul 06 18:57:38.296007 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:57:38.295985 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/564f0d46-248e-4a95-9416-2a6d23478fb0-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-7c47bc74cb-xrsz6\" (UID: \"564f0d46-248e-4a95-9416-2a6d23478fb0\") " pod="openshift-monitoring/telemeter-client-7c47bc74cb-xrsz6" Jul 06 18:57:38.296165 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:57:38.296149 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/564f0d46-248e-4a95-9416-2a6d23478fb0-telemeter-client-tls\") pod \"telemeter-client-7c47bc74cb-xrsz6\" (UID: \"564f0d46-248e-4a95-9416-2a6d23478fb0\") " pod="openshift-monitoring/telemeter-client-7c47bc74cb-xrsz6" Jul 06 18:57:38.296237 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:57:38.296218 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/564f0d46-248e-4a95-9416-2a6d23478fb0-secret-telemeter-client\") pod \"telemeter-client-7c47bc74cb-xrsz6\" (UID: \"564f0d46-248e-4a95-9416-2a6d23478fb0\") " pod="openshift-monitoring/telemeter-client-7c47bc74cb-xrsz6" Jul 06 18:57:38.296381 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:57:38.296365 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/564f0d46-248e-4a95-9416-2a6d23478fb0-federate-client-tls\") pod \"telemeter-client-7c47bc74cb-xrsz6\" (UID: \"564f0d46-248e-4a95-9416-2a6d23478fb0\") " pod="openshift-monitoring/telemeter-client-7c47bc74cb-xrsz6" Jul 06 18:57:38.302208 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:57:38.302186 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-tg7x6\" (UniqueName: \"kubernetes.io/projected/564f0d46-248e-4a95-9416-2a6d23478fb0-kube-api-access-tg7x6\") pod \"telemeter-client-7c47bc74cb-xrsz6\" (UID: \"564f0d46-248e-4a95-9416-2a6d23478fb0\") " pod="openshift-monitoring/telemeter-client-7c47bc74cb-xrsz6" Jul 06 18:57:38.491793 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:57:38.491708 2570 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jul 06 18:57:38.492046 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:57:38.491995 2570 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="45a3639f-79ab-4bb6-8a64-f2b4458f3375" containerName="prometheus" containerID="cri-o://061222b702aae270175e5d72ed12de85bfcb3957293f36dab428c91346da5006" gracePeriod=600 Jul 06 18:57:38.492114 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:57:38.492047 2570 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="45a3639f-79ab-4bb6-8a64-f2b4458f3375" containerName="config-reloader" containerID="cri-o://7efcab14e9d4bed190760cf9ca3845e56cf9c917cdc6744f4b01234852be927a" gracePeriod=600 Jul 06 18:57:38.492114 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:57:38.492050 2570 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="45a3639f-79ab-4bb6-8a64-f2b4458f3375" containerName="kube-rbac-proxy-thanos" containerID="cri-o://e0f5261f30fd436f3cc5191bf5d2fe73764fdab9992b31134d2d7dbbf9bcc54c" gracePeriod=600 Jul 06 18:57:38.492114 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:57:38.492087 2570 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="45a3639f-79ab-4bb6-8a64-f2b4458f3375" containerName="thanos-sidecar" containerID="cri-o://2cb7f600ea6415e85375a9c11ea27b29e225aa43e7f430dace90788421c2467c" gracePeriod=600 Jul 06 18:57:38.492265 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:57:38.492024 2570 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="45a3639f-79ab-4bb6-8a64-f2b4458f3375" containerName="kube-rbac-proxy" containerID="cri-o://d50576bc26b15d78d3e1b9b182ecf8016762de33614fb8d8e237e55d260720e5" gracePeriod=600 Jul 06 18:57:38.492265 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:57:38.492083 2570 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="45a3639f-79ab-4bb6-8a64-f2b4458f3375" containerName="kube-rbac-proxy-web" containerID="cri-o://ec3c3442dab43c6a7de794ed52c5c293b5c7b76635bd2bb41ef15a4cffaf184a" gracePeriod=600 Jul 06 18:57:38.497883 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:57:38.497859 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/telemeter-client-7c47bc74cb-xrsz6" Jul 06 18:57:38.639812 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:57:38.639781 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/telemeter-client-7c47bc74cb-xrsz6"] Jul 06 18:57:38.644558 ip-10-0-130-59 kubenswrapper[2570]: W0706 18:57:38.644526 2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod564f0d46_248e_4a95_9416_2a6d23478fb0.slice/crio-4b20a2ba53d8eef5863ddf587175bcbdd769ec601c5c205706fc381d92eb1db7 WatchSource:0}: Error finding container 4b20a2ba53d8eef5863ddf587175bcbdd769ec601c5c205706fc381d92eb1db7: Status 404 returned error can't find the container with id 4b20a2ba53d8eef5863ddf587175bcbdd769ec601c5c205706fc381d92eb1db7 Jul 06 18:57:38.736606 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:57:38.736579 2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" Jul 06 18:57:38.772044 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:57:38.771963 2570 generic.go:356] "Generic (PLEG): container finished" podID="45a3639f-79ab-4bb6-8a64-f2b4458f3375" containerID="e0f5261f30fd436f3cc5191bf5d2fe73764fdab9992b31134d2d7dbbf9bcc54c" exitCode=0 Jul 06 18:57:38.772044 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:57:38.771987 2570 generic.go:356] "Generic (PLEG): container finished" podID="45a3639f-79ab-4bb6-8a64-f2b4458f3375" containerID="d50576bc26b15d78d3e1b9b182ecf8016762de33614fb8d8e237e55d260720e5" exitCode=0 Jul 06 18:57:38.772044 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:57:38.771993 2570 generic.go:356] "Generic (PLEG): container finished" podID="45a3639f-79ab-4bb6-8a64-f2b4458f3375" containerID="ec3c3442dab43c6a7de794ed52c5c293b5c7b76635bd2bb41ef15a4cffaf184a" exitCode=0 Jul 06 18:57:38.772044 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:57:38.771998 2570 generic.go:356] "Generic (PLEG): container finished" podID="45a3639f-79ab-4bb6-8a64-f2b4458f3375" containerID="2cb7f600ea6415e85375a9c11ea27b29e225aa43e7f430dace90788421c2467c" exitCode=0 Jul 06 18:57:38.772044 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:57:38.772003 2570 generic.go:356] "Generic (PLEG): container finished" podID="45a3639f-79ab-4bb6-8a64-f2b4458f3375" containerID="7efcab14e9d4bed190760cf9ca3845e56cf9c917cdc6744f4b01234852be927a" exitCode=0 Jul 06 18:57:38.772044 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:57:38.772008 2570 generic.go:356] "Generic (PLEG): container finished" podID="45a3639f-79ab-4bb6-8a64-f2b4458f3375" containerID="061222b702aae270175e5d72ed12de85bfcb3957293f36dab428c91346da5006" exitCode=0 Jul 06 18:57:38.772044 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:57:38.772038 2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" Jul 06 18:57:38.772652 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:57:38.772052 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"45a3639f-79ab-4bb6-8a64-f2b4458f3375","Type":"ContainerDied","Data":"e0f5261f30fd436f3cc5191bf5d2fe73764fdab9992b31134d2d7dbbf9bcc54c"} Jul 06 18:57:38.772652 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:57:38.772147 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"45a3639f-79ab-4bb6-8a64-f2b4458f3375","Type":"ContainerDied","Data":"d50576bc26b15d78d3e1b9b182ecf8016762de33614fb8d8e237e55d260720e5"} Jul 06 18:57:38.772652 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:57:38.772191 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"45a3639f-79ab-4bb6-8a64-f2b4458f3375","Type":"ContainerDied","Data":"ec3c3442dab43c6a7de794ed52c5c293b5c7b76635bd2bb41ef15a4cffaf184a"} Jul 06 18:57:38.772652 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:57:38.772200 2570 scope.go:117] "RemoveContainer" containerID="e0f5261f30fd436f3cc5191bf5d2fe73764fdab9992b31134d2d7dbbf9bcc54c" Jul 06 18:57:38.772652 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:57:38.772232 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"45a3639f-79ab-4bb6-8a64-f2b4458f3375","Type":"ContainerDied","Data":"2cb7f600ea6415e85375a9c11ea27b29e225aa43e7f430dace90788421c2467c"} Jul 06 18:57:38.772652 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:57:38.772275 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"45a3639f-79ab-4bb6-8a64-f2b4458f3375","Type":"ContainerDied","Data":"7efcab14e9d4bed190760cf9ca3845e56cf9c917cdc6744f4b01234852be927a"} Jul 06 18:57:38.772652 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:57:38.772305 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"45a3639f-79ab-4bb6-8a64-f2b4458f3375","Type":"ContainerDied","Data":"061222b702aae270175e5d72ed12de85bfcb3957293f36dab428c91346da5006"} Jul 06 18:57:38.772652 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:57:38.772334 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"45a3639f-79ab-4bb6-8a64-f2b4458f3375","Type":"ContainerDied","Data":"500b03df330643f6796c8feeff80b2c051c61f2b085f060d7f2c778c4e83f455"} Jul 06 18:57:38.773689 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:57:38.773556 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-7c47bc74cb-xrsz6" event={"ID":"564f0d46-248e-4a95-9416-2a6d23478fb0","Type":"ContainerStarted","Data":"4b20a2ba53d8eef5863ddf587175bcbdd769ec601c5c205706fc381d92eb1db7"} Jul 06 18:57:38.780965 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:57:38.780952 2570 scope.go:117] "RemoveContainer" containerID="d50576bc26b15d78d3e1b9b182ecf8016762de33614fb8d8e237e55d260720e5" Jul 06 18:57:38.788571 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:57:38.788549 2570 scope.go:117] "RemoveContainer" containerID="ec3c3442dab43c6a7de794ed52c5c293b5c7b76635bd2bb41ef15a4cffaf184a" Jul 06 18:57:38.796941 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:57:38.796919 2570 scope.go:117] "RemoveContainer" containerID="2cb7f600ea6415e85375a9c11ea27b29e225aa43e7f430dace90788421c2467c" Jul 06 18:57:38.799038 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:57:38.799021 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/45a3639f-79ab-4bb6-8a64-f2b4458f3375-prometheus-k8s-rulefiles-2\") pod \"45a3639f-79ab-4bb6-8a64-f2b4458f3375\" (UID: \"45a3639f-79ab-4bb6-8a64-f2b4458f3375\") " Jul 06 18:57:38.799138 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:57:38.799050 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/45a3639f-79ab-4bb6-8a64-f2b4458f3375-prometheus-k8s-db\") pod \"45a3639f-79ab-4bb6-8a64-f2b4458f3375\" (UID: \"45a3639f-79ab-4bb6-8a64-f2b4458f3375\") " Jul 06 18:57:38.799138 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:57:38.799072 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/45a3639f-79ab-4bb6-8a64-f2b4458f3375-thanos-prometheus-http-client-file\") pod \"45a3639f-79ab-4bb6-8a64-f2b4458f3375\" (UID: \"45a3639f-79ab-4bb6-8a64-f2b4458f3375\") " Jul 06 18:57:38.799138 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:57:38.799099 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/45a3639f-79ab-4bb6-8a64-f2b4458f3375-secret-metrics-client-certs\") pod \"45a3639f-79ab-4bb6-8a64-f2b4458f3375\" (UID: \"45a3639f-79ab-4bb6-8a64-f2b4458f3375\") " Jul 06 18:57:38.799138 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:57:38.799120 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/45a3639f-79ab-4bb6-8a64-f2b4458f3375-configmap-metrics-client-ca\") pod \"45a3639f-79ab-4bb6-8a64-f2b4458f3375\" (UID: \"45a3639f-79ab-4bb6-8a64-f2b4458f3375\") " Jul 06 18:57:38.799720 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:57:38.799144 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/45a3639f-79ab-4bb6-8a64-f2b4458f3375-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"45a3639f-79ab-4bb6-8a64-f2b4458f3375\" (UID: \"45a3639f-79ab-4bb6-8a64-f2b4458f3375\") " Jul 06 18:57:38.799720 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:57:38.799174 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/45a3639f-79ab-4bb6-8a64-f2b4458f3375-tls-assets\") pod \"45a3639f-79ab-4bb6-8a64-f2b4458f3375\" (UID: \"45a3639f-79ab-4bb6-8a64-f2b4458f3375\") " Jul 06 18:57:38.799720 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:57:38.799203 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/45a3639f-79ab-4bb6-8a64-f2b4458f3375-secret-kube-rbac-proxy\") pod \"45a3639f-79ab-4bb6-8a64-f2b4458f3375\" (UID: \"45a3639f-79ab-4bb6-8a64-f2b4458f3375\") " Jul 06 18:57:38.799720 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:57:38.799234 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/45a3639f-79ab-4bb6-8a64-f2b4458f3375-config-out\") pod \"45a3639f-79ab-4bb6-8a64-f2b4458f3375\" (UID: \"45a3639f-79ab-4bb6-8a64-f2b4458f3375\") " Jul 06 18:57:38.799720 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:57:38.799279 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/45a3639f-79ab-4bb6-8a64-f2b4458f3375-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"45a3639f-79ab-4bb6-8a64-f2b4458f3375\" (UID: \"45a3639f-79ab-4bb6-8a64-f2b4458f3375\") " Jul 06 18:57:38.799720 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:57:38.799308 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/45a3639f-79ab-4bb6-8a64-f2b4458f3375-configmap-serving-certs-ca-bundle\") pod \"45a3639f-79ab-4bb6-8a64-f2b4458f3375\" (UID: \"45a3639f-79ab-4bb6-8a64-f2b4458f3375\") " Jul 06 18:57:38.799720 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:57:38.799421 2570 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/45a3639f-79ab-4bb6-8a64-f2b4458f3375-prometheus-k8s-rulefiles-2" (OuterVolumeSpecName: "prometheus-k8s-rulefiles-2") pod "45a3639f-79ab-4bb6-8a64-f2b4458f3375" (UID: "45a3639f-79ab-4bb6-8a64-f2b4458f3375"). InnerVolumeSpecName "prometheus-k8s-rulefiles-2". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 06 18:57:38.799720 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:57:38.799707 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/45a3639f-79ab-4bb6-8a64-f2b4458f3375-prometheus-trusted-ca-bundle\") pod \"45a3639f-79ab-4bb6-8a64-f2b4458f3375\" (UID: \"45a3639f-79ab-4bb6-8a64-f2b4458f3375\") " Jul 06 18:57:38.800106 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:57:38.799736 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/45a3639f-79ab-4bb6-8a64-f2b4458f3375-secret-prometheus-k8s-tls\") pod \"45a3639f-79ab-4bb6-8a64-f2b4458f3375\" (UID: \"45a3639f-79ab-4bb6-8a64-f2b4458f3375\") " Jul 06 18:57:38.800106 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:57:38.799834 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/45a3639f-79ab-4bb6-8a64-f2b4458f3375-prometheus-k8s-rulefiles-0\") pod \"45a3639f-79ab-4bb6-8a64-f2b4458f3375\" (UID: \"45a3639f-79ab-4bb6-8a64-f2b4458f3375\") " Jul 06 18:57:38.800106 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:57:38.799868 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/45a3639f-79ab-4bb6-8a64-f2b4458f3375-web-config\") pod \"45a3639f-79ab-4bb6-8a64-f2b4458f3375\" (UID: \"45a3639f-79ab-4bb6-8a64-f2b4458f3375\") " Jul 06 18:57:38.800106 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:57:38.799900 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-4zv9t\" (UniqueName: \"kubernetes.io/projected/45a3639f-79ab-4bb6-8a64-f2b4458f3375-kube-api-access-4zv9t\") pod \"45a3639f-79ab-4bb6-8a64-f2b4458f3375\" (UID: \"45a3639f-79ab-4bb6-8a64-f2b4458f3375\") " Jul 06 18:57:38.800106 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:57:38.799911 2570 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/45a3639f-79ab-4bb6-8a64-f2b4458f3375-configmap-metrics-client-ca" (OuterVolumeSpecName: "configmap-metrics-client-ca") pod "45a3639f-79ab-4bb6-8a64-f2b4458f3375" (UID: "45a3639f-79ab-4bb6-8a64-f2b4458f3375"). InnerVolumeSpecName "configmap-metrics-client-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 06 18:57:38.800106 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:57:38.799940 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/45a3639f-79ab-4bb6-8a64-f2b4458f3375-prometheus-k8s-rulefiles-1\") pod \"45a3639f-79ab-4bb6-8a64-f2b4458f3375\" (UID: \"45a3639f-79ab-4bb6-8a64-f2b4458f3375\") " Jul 06 18:57:38.800106 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:57:38.799968 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/45a3639f-79ab-4bb6-8a64-f2b4458f3375-configmap-kubelet-serving-ca-bundle\") pod \"45a3639f-79ab-4bb6-8a64-f2b4458f3375\" (UID: \"45a3639f-79ab-4bb6-8a64-f2b4458f3375\") " Jul 06 18:57:38.800106 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:57:38.799998 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/45a3639f-79ab-4bb6-8a64-f2b4458f3375-config\") pod \"45a3639f-79ab-4bb6-8a64-f2b4458f3375\" (UID: \"45a3639f-79ab-4bb6-8a64-f2b4458f3375\") " Jul 06 18:57:38.800106 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:57:38.800028 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/45a3639f-79ab-4bb6-8a64-f2b4458f3375-secret-grpc-tls\") pod \"45a3639f-79ab-4bb6-8a64-f2b4458f3375\" (UID: \"45a3639f-79ab-4bb6-8a64-f2b4458f3375\") " Jul 06 18:57:38.800636 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:57:38.800364 2570 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/45a3639f-79ab-4bb6-8a64-f2b4458f3375-prometheus-k8s-db" (OuterVolumeSpecName: "prometheus-k8s-db") pod "45a3639f-79ab-4bb6-8a64-f2b4458f3375" (UID: "45a3639f-79ab-4bb6-8a64-f2b4458f3375"). InnerVolumeSpecName "prometheus-k8s-db". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 18:57:38.803532 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:57:38.801019 2570 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/45a3639f-79ab-4bb6-8a64-f2b4458f3375-prometheus-k8s-rulefiles-2\") on node \"ip-10-0-130-59.ec2.internal\" DevicePath \"\"" Jul 06 18:57:38.803532 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:57:38.801043 2570 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/45a3639f-79ab-4bb6-8a64-f2b4458f3375-prometheus-k8s-db\") on node \"ip-10-0-130-59.ec2.internal\" DevicePath \"\"" Jul 06 18:57:38.803532 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:57:38.801059 2570 reconciler_common.go:299] "Volume detached for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/45a3639f-79ab-4bb6-8a64-f2b4458f3375-configmap-metrics-client-ca\") on node \"ip-10-0-130-59.ec2.internal\" DevicePath \"\"" Jul 06 18:57:38.803532 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:57:38.801483 2570 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/45a3639f-79ab-4bb6-8a64-f2b4458f3375-configmap-serving-certs-ca-bundle" (OuterVolumeSpecName: "configmap-serving-certs-ca-bundle") pod "45a3639f-79ab-4bb6-8a64-f2b4458f3375" (UID: "45a3639f-79ab-4bb6-8a64-f2b4458f3375"). InnerVolumeSpecName "configmap-serving-certs-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 06 18:57:38.803532 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:57:38.802271 2570 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/45a3639f-79ab-4bb6-8a64-f2b4458f3375-prometheus-trusted-ca-bundle" (OuterVolumeSpecName: "prometheus-trusted-ca-bundle") pod "45a3639f-79ab-4bb6-8a64-f2b4458f3375" (UID: "45a3639f-79ab-4bb6-8a64-f2b4458f3375"). InnerVolumeSpecName "prometheus-trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 06 18:57:38.803532 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:57:38.802800 2570 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/45a3639f-79ab-4bb6-8a64-f2b4458f3375-configmap-kubelet-serving-ca-bundle" (OuterVolumeSpecName: "configmap-kubelet-serving-ca-bundle") pod "45a3639f-79ab-4bb6-8a64-f2b4458f3375" (UID: "45a3639f-79ab-4bb6-8a64-f2b4458f3375"). InnerVolumeSpecName "configmap-kubelet-serving-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 06 18:57:38.803532 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:57:38.803180 2570 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/45a3639f-79ab-4bb6-8a64-f2b4458f3375-prometheus-k8s-rulefiles-0" (OuterVolumeSpecName: "prometheus-k8s-rulefiles-0") pod "45a3639f-79ab-4bb6-8a64-f2b4458f3375" (UID: "45a3639f-79ab-4bb6-8a64-f2b4458f3375"). InnerVolumeSpecName "prometheus-k8s-rulefiles-0". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 06 18:57:38.803532 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:57:38.803507 2570 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/45a3639f-79ab-4bb6-8a64-f2b4458f3375-prometheus-k8s-rulefiles-1" (OuterVolumeSpecName: "prometheus-k8s-rulefiles-1") pod "45a3639f-79ab-4bb6-8a64-f2b4458f3375" (UID: "45a3639f-79ab-4bb6-8a64-f2b4458f3375"). InnerVolumeSpecName "prometheus-k8s-rulefiles-1". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 06 18:57:38.804724 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:57:38.804633 2570 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/45a3639f-79ab-4bb6-8a64-f2b4458f3375-thanos-prometheus-http-client-file" (OuterVolumeSpecName: "thanos-prometheus-http-client-file") pod "45a3639f-79ab-4bb6-8a64-f2b4458f3375" (UID: "45a3639f-79ab-4bb6-8a64-f2b4458f3375"). InnerVolumeSpecName "thanos-prometheus-http-client-file". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 06 18:57:38.804839 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:57:38.804812 2570 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/45a3639f-79ab-4bb6-8a64-f2b4458f3375-secret-kube-rbac-proxy" (OuterVolumeSpecName: "secret-kube-rbac-proxy") pod "45a3639f-79ab-4bb6-8a64-f2b4458f3375" (UID: "45a3639f-79ab-4bb6-8a64-f2b4458f3375"). InnerVolumeSpecName "secret-kube-rbac-proxy". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 06 18:57:38.805421 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:57:38.805391 2570 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/45a3639f-79ab-4bb6-8a64-f2b4458f3375-secret-metrics-client-certs" (OuterVolumeSpecName: "secret-metrics-client-certs") pod "45a3639f-79ab-4bb6-8a64-f2b4458f3375" (UID: "45a3639f-79ab-4bb6-8a64-f2b4458f3375"). InnerVolumeSpecName "secret-metrics-client-certs". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 06 18:57:38.805531 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:57:38.805510 2570 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/45a3639f-79ab-4bb6-8a64-f2b4458f3375-secret-prometheus-k8s-kube-rbac-proxy-web" (OuterVolumeSpecName: "secret-prometheus-k8s-kube-rbac-proxy-web") pod "45a3639f-79ab-4bb6-8a64-f2b4458f3375" (UID: "45a3639f-79ab-4bb6-8a64-f2b4458f3375"). InnerVolumeSpecName "secret-prometheus-k8s-kube-rbac-proxy-web". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 06 18:57:38.805601 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:57:38.805533 2570 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/45a3639f-79ab-4bb6-8a64-f2b4458f3375-kube-api-access-4zv9t" (OuterVolumeSpecName: "kube-api-access-4zv9t") pod "45a3639f-79ab-4bb6-8a64-f2b4458f3375" (UID: "45a3639f-79ab-4bb6-8a64-f2b4458f3375"). InnerVolumeSpecName "kube-api-access-4zv9t". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 06 18:57:38.806112 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:57:38.806076 2570 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/45a3639f-79ab-4bb6-8a64-f2b4458f3375-secret-prometheus-k8s-thanos-sidecar-tls" (OuterVolumeSpecName: "secret-prometheus-k8s-thanos-sidecar-tls") pod "45a3639f-79ab-4bb6-8a64-f2b4458f3375" (UID: "45a3639f-79ab-4bb6-8a64-f2b4458f3375"). InnerVolumeSpecName "secret-prometheus-k8s-thanos-sidecar-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 06 18:57:38.806194 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:57:38.806121 2570 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/45a3639f-79ab-4bb6-8a64-f2b4458f3375-config-out" (OuterVolumeSpecName: "config-out") pod "45a3639f-79ab-4bb6-8a64-f2b4458f3375" (UID: "45a3639f-79ab-4bb6-8a64-f2b4458f3375"). InnerVolumeSpecName "config-out". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 18:57:38.806396 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:57:38.806358 2570 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/45a3639f-79ab-4bb6-8a64-f2b4458f3375-tls-assets" (OuterVolumeSpecName: "tls-assets") pod "45a3639f-79ab-4bb6-8a64-f2b4458f3375" (UID: "45a3639f-79ab-4bb6-8a64-f2b4458f3375"). InnerVolumeSpecName "tls-assets". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 06 18:57:38.806396 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:57:38.806385 2570 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/45a3639f-79ab-4bb6-8a64-f2b4458f3375-config" (OuterVolumeSpecName: "config") pod "45a3639f-79ab-4bb6-8a64-f2b4458f3375" (UID: "45a3639f-79ab-4bb6-8a64-f2b4458f3375"). InnerVolumeSpecName "config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 06 18:57:38.806498 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:57:38.806472 2570 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/45a3639f-79ab-4bb6-8a64-f2b4458f3375-secret-prometheus-k8s-tls" (OuterVolumeSpecName: "secret-prometheus-k8s-tls") pod "45a3639f-79ab-4bb6-8a64-f2b4458f3375" (UID: "45a3639f-79ab-4bb6-8a64-f2b4458f3375"). InnerVolumeSpecName "secret-prometheus-k8s-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 06 18:57:38.807228 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:57:38.807203 2570 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/45a3639f-79ab-4bb6-8a64-f2b4458f3375-secret-grpc-tls" (OuterVolumeSpecName: "secret-grpc-tls") pod "45a3639f-79ab-4bb6-8a64-f2b4458f3375" (UID: "45a3639f-79ab-4bb6-8a64-f2b4458f3375"). InnerVolumeSpecName "secret-grpc-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 06 18:57:38.807688 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:57:38.807669 2570 scope.go:117] "RemoveContainer" containerID="7efcab14e9d4bed190760cf9ca3845e56cf9c917cdc6744f4b01234852be927a" Jul 06 18:57:38.815705 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:57:38.815683 2570 scope.go:117] "RemoveContainer" containerID="061222b702aae270175e5d72ed12de85bfcb3957293f36dab428c91346da5006" Jul 06 18:57:38.816360 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:57:38.816340 2570 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/45a3639f-79ab-4bb6-8a64-f2b4458f3375-web-config" (OuterVolumeSpecName: "web-config") pod "45a3639f-79ab-4bb6-8a64-f2b4458f3375" (UID: "45a3639f-79ab-4bb6-8a64-f2b4458f3375"). InnerVolumeSpecName "web-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 06 18:57:38.824264 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:57:38.824234 2570 scope.go:117] "RemoveContainer" containerID="824e75a7716111edd36acdce16301eda65bfd510a905d6f2eba53141bc8c875b" Jul 06 18:57:38.831905 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:57:38.831874 2570 scope.go:117] "RemoveContainer" containerID="e0f5261f30fd436f3cc5191bf5d2fe73764fdab9992b31134d2d7dbbf9bcc54c" Jul 06 18:57:38.832168 ip-10-0-130-59 kubenswrapper[2570]: E0706 18:57:38.832151 2570 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e0f5261f30fd436f3cc5191bf5d2fe73764fdab9992b31134d2d7dbbf9bcc54c\": container with ID starting with e0f5261f30fd436f3cc5191bf5d2fe73764fdab9992b31134d2d7dbbf9bcc54c not found: ID does not exist" containerID="e0f5261f30fd436f3cc5191bf5d2fe73764fdab9992b31134d2d7dbbf9bcc54c" Jul 06 18:57:38.832226 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:57:38.832176 2570 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e0f5261f30fd436f3cc5191bf5d2fe73764fdab9992b31134d2d7dbbf9bcc54c"} err="failed to get container status \"e0f5261f30fd436f3cc5191bf5d2fe73764fdab9992b31134d2d7dbbf9bcc54c\": rpc error: code = NotFound desc = could not find container \"e0f5261f30fd436f3cc5191bf5d2fe73764fdab9992b31134d2d7dbbf9bcc54c\": container with ID starting with e0f5261f30fd436f3cc5191bf5d2fe73764fdab9992b31134d2d7dbbf9bcc54c not found: ID does not exist" Jul 06 18:57:38.832226 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:57:38.832195 2570 scope.go:117] "RemoveContainer" containerID="d50576bc26b15d78d3e1b9b182ecf8016762de33614fb8d8e237e55d260720e5" Jul 06 18:57:38.832424 ip-10-0-130-59 kubenswrapper[2570]: E0706 18:57:38.832402 2570 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d50576bc26b15d78d3e1b9b182ecf8016762de33614fb8d8e237e55d260720e5\": container with ID starting with d50576bc26b15d78d3e1b9b182ecf8016762de33614fb8d8e237e55d260720e5 not found: ID does not exist" containerID="d50576bc26b15d78d3e1b9b182ecf8016762de33614fb8d8e237e55d260720e5" Jul 06 18:57:38.832491 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:57:38.832430 2570 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d50576bc26b15d78d3e1b9b182ecf8016762de33614fb8d8e237e55d260720e5"} err="failed to get container status \"d50576bc26b15d78d3e1b9b182ecf8016762de33614fb8d8e237e55d260720e5\": rpc error: code = NotFound desc = could not find container \"d50576bc26b15d78d3e1b9b182ecf8016762de33614fb8d8e237e55d260720e5\": container with ID starting with d50576bc26b15d78d3e1b9b182ecf8016762de33614fb8d8e237e55d260720e5 not found: ID does not exist" Jul 06 18:57:38.832491 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:57:38.832451 2570 scope.go:117] "RemoveContainer" containerID="ec3c3442dab43c6a7de794ed52c5c293b5c7b76635bd2bb41ef15a4cffaf184a" Jul 06 18:57:38.832760 ip-10-0-130-59 kubenswrapper[2570]: E0706 18:57:38.832741 2570 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ec3c3442dab43c6a7de794ed52c5c293b5c7b76635bd2bb41ef15a4cffaf184a\": container with ID starting with ec3c3442dab43c6a7de794ed52c5c293b5c7b76635bd2bb41ef15a4cffaf184a not found: ID does not exist" containerID="ec3c3442dab43c6a7de794ed52c5c293b5c7b76635bd2bb41ef15a4cffaf184a" Jul 06 18:57:38.832807 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:57:38.832767 2570 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ec3c3442dab43c6a7de794ed52c5c293b5c7b76635bd2bb41ef15a4cffaf184a"} err="failed to get container status \"ec3c3442dab43c6a7de794ed52c5c293b5c7b76635bd2bb41ef15a4cffaf184a\": rpc error: code = NotFound desc = could not find container \"ec3c3442dab43c6a7de794ed52c5c293b5c7b76635bd2bb41ef15a4cffaf184a\": container with ID starting with ec3c3442dab43c6a7de794ed52c5c293b5c7b76635bd2bb41ef15a4cffaf184a not found: ID does not exist" Jul 06 18:57:38.832807 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:57:38.832785 2570 scope.go:117] "RemoveContainer" containerID="2cb7f600ea6415e85375a9c11ea27b29e225aa43e7f430dace90788421c2467c" Jul 06 18:57:38.833021 ip-10-0-130-59 kubenswrapper[2570]: E0706 18:57:38.833003 2570 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2cb7f600ea6415e85375a9c11ea27b29e225aa43e7f430dace90788421c2467c\": container with ID starting with 2cb7f600ea6415e85375a9c11ea27b29e225aa43e7f430dace90788421c2467c not found: ID does not exist" containerID="2cb7f600ea6415e85375a9c11ea27b29e225aa43e7f430dace90788421c2467c" Jul 06 18:57:38.833062 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:57:38.833036 2570 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2cb7f600ea6415e85375a9c11ea27b29e225aa43e7f430dace90788421c2467c"} err="failed to get container status \"2cb7f600ea6415e85375a9c11ea27b29e225aa43e7f430dace90788421c2467c\": rpc error: code = NotFound desc = could not find container \"2cb7f600ea6415e85375a9c11ea27b29e225aa43e7f430dace90788421c2467c\": container with ID starting with 2cb7f600ea6415e85375a9c11ea27b29e225aa43e7f430dace90788421c2467c not found: ID does not exist" Jul 06 18:57:38.833062 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:57:38.833053 2570 scope.go:117] "RemoveContainer" containerID="7efcab14e9d4bed190760cf9ca3845e56cf9c917cdc6744f4b01234852be927a" Jul 06 18:57:38.833268 ip-10-0-130-59 kubenswrapper[2570]: E0706 18:57:38.833252 2570 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7efcab14e9d4bed190760cf9ca3845e56cf9c917cdc6744f4b01234852be927a\": container with ID starting with 7efcab14e9d4bed190760cf9ca3845e56cf9c917cdc6744f4b01234852be927a not found: ID does not exist" containerID="7efcab14e9d4bed190760cf9ca3845e56cf9c917cdc6744f4b01234852be927a" Jul 06 18:57:38.833333 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:57:38.833276 2570 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7efcab14e9d4bed190760cf9ca3845e56cf9c917cdc6744f4b01234852be927a"} err="failed to get container status \"7efcab14e9d4bed190760cf9ca3845e56cf9c917cdc6744f4b01234852be927a\": rpc error: code = NotFound desc = could not find container \"7efcab14e9d4bed190760cf9ca3845e56cf9c917cdc6744f4b01234852be927a\": container with ID starting with 7efcab14e9d4bed190760cf9ca3845e56cf9c917cdc6744f4b01234852be927a not found: ID does not exist" Jul 06 18:57:38.833333 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:57:38.833296 2570 scope.go:117] "RemoveContainer" containerID="061222b702aae270175e5d72ed12de85bfcb3957293f36dab428c91346da5006" Jul 06 18:57:38.833524 ip-10-0-130-59 kubenswrapper[2570]: E0706 18:57:38.833506 2570 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"061222b702aae270175e5d72ed12de85bfcb3957293f36dab428c91346da5006\": container with ID starting with 061222b702aae270175e5d72ed12de85bfcb3957293f36dab428c91346da5006 not found: ID does not exist" containerID="061222b702aae270175e5d72ed12de85bfcb3957293f36dab428c91346da5006" Jul 06 18:57:38.833561 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:57:38.833529 2570 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"061222b702aae270175e5d72ed12de85bfcb3957293f36dab428c91346da5006"} err="failed to get container status \"061222b702aae270175e5d72ed12de85bfcb3957293f36dab428c91346da5006\": rpc error: code = NotFound desc = could not find container \"061222b702aae270175e5d72ed12de85bfcb3957293f36dab428c91346da5006\": container with ID starting with 061222b702aae270175e5d72ed12de85bfcb3957293f36dab428c91346da5006 not found: ID does not exist" Jul 06 18:57:38.833561 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:57:38.833543 2570 scope.go:117] "RemoveContainer" containerID="824e75a7716111edd36acdce16301eda65bfd510a905d6f2eba53141bc8c875b" Jul 06 18:57:38.834057 ip-10-0-130-59 kubenswrapper[2570]: E0706 18:57:38.834032 2570 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"824e75a7716111edd36acdce16301eda65bfd510a905d6f2eba53141bc8c875b\": container with ID starting with 824e75a7716111edd36acdce16301eda65bfd510a905d6f2eba53141bc8c875b not found: ID does not exist" containerID="824e75a7716111edd36acdce16301eda65bfd510a905d6f2eba53141bc8c875b" Jul 06 18:57:38.834132 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:57:38.834062 2570 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"824e75a7716111edd36acdce16301eda65bfd510a905d6f2eba53141bc8c875b"} err="failed to get container status \"824e75a7716111edd36acdce16301eda65bfd510a905d6f2eba53141bc8c875b\": rpc error: code = NotFound desc = could not find container \"824e75a7716111edd36acdce16301eda65bfd510a905d6f2eba53141bc8c875b\": container with ID starting with 824e75a7716111edd36acdce16301eda65bfd510a905d6f2eba53141bc8c875b not found: ID does not exist" Jul 06 18:57:38.834132 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:57:38.834081 2570 scope.go:117] "RemoveContainer" containerID="e0f5261f30fd436f3cc5191bf5d2fe73764fdab9992b31134d2d7dbbf9bcc54c" Jul 06 18:57:38.834299 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:57:38.834279 2570 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e0f5261f30fd436f3cc5191bf5d2fe73764fdab9992b31134d2d7dbbf9bcc54c"} err="failed to get container status \"e0f5261f30fd436f3cc5191bf5d2fe73764fdab9992b31134d2d7dbbf9bcc54c\": rpc error: code = NotFound desc = could not find container \"e0f5261f30fd436f3cc5191bf5d2fe73764fdab9992b31134d2d7dbbf9bcc54c\": container with ID starting with e0f5261f30fd436f3cc5191bf5d2fe73764fdab9992b31134d2d7dbbf9bcc54c not found: ID does not exist" Jul 06 18:57:38.834346 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:57:38.834303 2570 scope.go:117] "RemoveContainer" containerID="d50576bc26b15d78d3e1b9b182ecf8016762de33614fb8d8e237e55d260720e5" Jul 06 18:57:38.834498 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:57:38.834482 2570 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d50576bc26b15d78d3e1b9b182ecf8016762de33614fb8d8e237e55d260720e5"} err="failed to get container status \"d50576bc26b15d78d3e1b9b182ecf8016762de33614fb8d8e237e55d260720e5\": rpc error: code = NotFound desc = could not find container \"d50576bc26b15d78d3e1b9b182ecf8016762de33614fb8d8e237e55d260720e5\": container with ID starting with d50576bc26b15d78d3e1b9b182ecf8016762de33614fb8d8e237e55d260720e5 not found: ID does not exist" Jul 06 18:57:38.834541 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:57:38.834498 2570 scope.go:117] "RemoveContainer" containerID="ec3c3442dab43c6a7de794ed52c5c293b5c7b76635bd2bb41ef15a4cffaf184a" Jul 06 18:57:38.834761 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:57:38.834743 2570 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ec3c3442dab43c6a7de794ed52c5c293b5c7b76635bd2bb41ef15a4cffaf184a"} err="failed to get container status \"ec3c3442dab43c6a7de794ed52c5c293b5c7b76635bd2bb41ef15a4cffaf184a\": rpc error: code = NotFound desc = could not find container \"ec3c3442dab43c6a7de794ed52c5c293b5c7b76635bd2bb41ef15a4cffaf184a\": container with ID starting with ec3c3442dab43c6a7de794ed52c5c293b5c7b76635bd2bb41ef15a4cffaf184a not found: ID does not exist" Jul 06 18:57:38.834826 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:57:38.834762 2570 scope.go:117] "RemoveContainer" containerID="2cb7f600ea6415e85375a9c11ea27b29e225aa43e7f430dace90788421c2467c" Jul 06 18:57:38.834967 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:57:38.834951 2570 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2cb7f600ea6415e85375a9c11ea27b29e225aa43e7f430dace90788421c2467c"} err="failed to get container status \"2cb7f600ea6415e85375a9c11ea27b29e225aa43e7f430dace90788421c2467c\": rpc error: code = NotFound desc = could not find container \"2cb7f600ea6415e85375a9c11ea27b29e225aa43e7f430dace90788421c2467c\": container with ID starting with 2cb7f600ea6415e85375a9c11ea27b29e225aa43e7f430dace90788421c2467c not found: ID does not exist" Jul 06 18:57:38.835039 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:57:38.834968 2570 scope.go:117] "RemoveContainer" containerID="7efcab14e9d4bed190760cf9ca3845e56cf9c917cdc6744f4b01234852be927a" Jul 06 18:57:38.835160 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:57:38.835143 2570 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7efcab14e9d4bed190760cf9ca3845e56cf9c917cdc6744f4b01234852be927a"} err="failed to get container status \"7efcab14e9d4bed190760cf9ca3845e56cf9c917cdc6744f4b01234852be927a\": rpc error: code = NotFound desc = could not find container \"7efcab14e9d4bed190760cf9ca3845e56cf9c917cdc6744f4b01234852be927a\": container with ID starting with 7efcab14e9d4bed190760cf9ca3845e56cf9c917cdc6744f4b01234852be927a not found: ID does not exist" Jul 06 18:57:38.835210 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:57:38.835162 2570 scope.go:117] "RemoveContainer" containerID="061222b702aae270175e5d72ed12de85bfcb3957293f36dab428c91346da5006" Jul 06 18:57:38.835343 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:57:38.835324 2570 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"061222b702aae270175e5d72ed12de85bfcb3957293f36dab428c91346da5006"} err="failed to get container status \"061222b702aae270175e5d72ed12de85bfcb3957293f36dab428c91346da5006\": rpc error: code = NotFound desc = could not find container \"061222b702aae270175e5d72ed12de85bfcb3957293f36dab428c91346da5006\": container with ID starting with 061222b702aae270175e5d72ed12de85bfcb3957293f36dab428c91346da5006 not found: ID does not exist" Jul 06 18:57:38.835398 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:57:38.835343 2570 scope.go:117] "RemoveContainer" containerID="824e75a7716111edd36acdce16301eda65bfd510a905d6f2eba53141bc8c875b" Jul 06 18:57:38.835520 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:57:38.835504 2570 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"824e75a7716111edd36acdce16301eda65bfd510a905d6f2eba53141bc8c875b"} err="failed to get container status \"824e75a7716111edd36acdce16301eda65bfd510a905d6f2eba53141bc8c875b\": rpc error: code = NotFound desc = could not find container \"824e75a7716111edd36acdce16301eda65bfd510a905d6f2eba53141bc8c875b\": container with ID starting with 824e75a7716111edd36acdce16301eda65bfd510a905d6f2eba53141bc8c875b not found: ID does not exist" Jul 06 18:57:38.835520 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:57:38.835519 2570 scope.go:117] "RemoveContainer" containerID="e0f5261f30fd436f3cc5191bf5d2fe73764fdab9992b31134d2d7dbbf9bcc54c" Jul 06 18:57:38.835721 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:57:38.835703 2570 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e0f5261f30fd436f3cc5191bf5d2fe73764fdab9992b31134d2d7dbbf9bcc54c"} err="failed to get container status \"e0f5261f30fd436f3cc5191bf5d2fe73764fdab9992b31134d2d7dbbf9bcc54c\": rpc error: code = NotFound desc = could not find container \"e0f5261f30fd436f3cc5191bf5d2fe73764fdab9992b31134d2d7dbbf9bcc54c\": container with ID starting with e0f5261f30fd436f3cc5191bf5d2fe73764fdab9992b31134d2d7dbbf9bcc54c not found: ID does not exist" Jul 06 18:57:38.835721 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:57:38.835720 2570 scope.go:117] "RemoveContainer" containerID="d50576bc26b15d78d3e1b9b182ecf8016762de33614fb8d8e237e55d260720e5" Jul 06 18:57:38.835919 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:57:38.835892 2570 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d50576bc26b15d78d3e1b9b182ecf8016762de33614fb8d8e237e55d260720e5"} err="failed to get container status \"d50576bc26b15d78d3e1b9b182ecf8016762de33614fb8d8e237e55d260720e5\": rpc error: code = NotFound desc = could not find container \"d50576bc26b15d78d3e1b9b182ecf8016762de33614fb8d8e237e55d260720e5\": container with ID starting with d50576bc26b15d78d3e1b9b182ecf8016762de33614fb8d8e237e55d260720e5 not found: ID does not exist" Jul 06 18:57:38.835969 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:57:38.835920 2570 scope.go:117] "RemoveContainer" containerID="ec3c3442dab43c6a7de794ed52c5c293b5c7b76635bd2bb41ef15a4cffaf184a" Jul 06 18:57:38.836126 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:57:38.836105 2570 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ec3c3442dab43c6a7de794ed52c5c293b5c7b76635bd2bb41ef15a4cffaf184a"} err="failed to get container status \"ec3c3442dab43c6a7de794ed52c5c293b5c7b76635bd2bb41ef15a4cffaf184a\": rpc error: code = NotFound desc = could not find container \"ec3c3442dab43c6a7de794ed52c5c293b5c7b76635bd2bb41ef15a4cffaf184a\": container with ID starting with ec3c3442dab43c6a7de794ed52c5c293b5c7b76635bd2bb41ef15a4cffaf184a not found: ID does not exist" Jul 06 18:57:38.836194 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:57:38.836127 2570 scope.go:117] "RemoveContainer" containerID="2cb7f600ea6415e85375a9c11ea27b29e225aa43e7f430dace90788421c2467c" Jul 06 18:57:38.836373 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:57:38.836355 2570 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2cb7f600ea6415e85375a9c11ea27b29e225aa43e7f430dace90788421c2467c"} err="failed to get container status \"2cb7f600ea6415e85375a9c11ea27b29e225aa43e7f430dace90788421c2467c\": rpc error: code = NotFound desc = could not find container \"2cb7f600ea6415e85375a9c11ea27b29e225aa43e7f430dace90788421c2467c\": container with ID starting with 2cb7f600ea6415e85375a9c11ea27b29e225aa43e7f430dace90788421c2467c not found: ID does not exist" Jul 06 18:57:38.836422 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:57:38.836373 2570 scope.go:117] "RemoveContainer" containerID="7efcab14e9d4bed190760cf9ca3845e56cf9c917cdc6744f4b01234852be927a" Jul 06 18:57:38.836570 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:57:38.836555 2570 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7efcab14e9d4bed190760cf9ca3845e56cf9c917cdc6744f4b01234852be927a"} err="failed to get container status \"7efcab14e9d4bed190760cf9ca3845e56cf9c917cdc6744f4b01234852be927a\": rpc error: code = NotFound desc = could not find container \"7efcab14e9d4bed190760cf9ca3845e56cf9c917cdc6744f4b01234852be927a\": container with ID starting with 7efcab14e9d4bed190760cf9ca3845e56cf9c917cdc6744f4b01234852be927a not found: ID does not exist" Jul 06 18:57:38.836633 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:57:38.836571 2570 scope.go:117] "RemoveContainer" containerID="061222b702aae270175e5d72ed12de85bfcb3957293f36dab428c91346da5006" Jul 06 18:57:38.836832 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:57:38.836814 2570 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"061222b702aae270175e5d72ed12de85bfcb3957293f36dab428c91346da5006"} err="failed to get container status \"061222b702aae270175e5d72ed12de85bfcb3957293f36dab428c91346da5006\": rpc error: code = NotFound desc = could not find container \"061222b702aae270175e5d72ed12de85bfcb3957293f36dab428c91346da5006\": container with ID starting with 061222b702aae270175e5d72ed12de85bfcb3957293f36dab428c91346da5006 not found: ID does not exist" Jul 06 18:57:38.836896 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:57:38.836832 2570 scope.go:117] "RemoveContainer" containerID="824e75a7716111edd36acdce16301eda65bfd510a905d6f2eba53141bc8c875b" Jul 06 18:57:38.837051 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:57:38.837032 2570 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"824e75a7716111edd36acdce16301eda65bfd510a905d6f2eba53141bc8c875b"} err="failed to get container status \"824e75a7716111edd36acdce16301eda65bfd510a905d6f2eba53141bc8c875b\": rpc error: code = NotFound desc = could not find container \"824e75a7716111edd36acdce16301eda65bfd510a905d6f2eba53141bc8c875b\": container with ID starting with 824e75a7716111edd36acdce16301eda65bfd510a905d6f2eba53141bc8c875b not found: ID does not exist" Jul 06 18:57:38.837125 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:57:38.837052 2570 scope.go:117] "RemoveContainer" containerID="e0f5261f30fd436f3cc5191bf5d2fe73764fdab9992b31134d2d7dbbf9bcc54c" Jul 06 18:57:38.837294 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:57:38.837277 2570 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e0f5261f30fd436f3cc5191bf5d2fe73764fdab9992b31134d2d7dbbf9bcc54c"} err="failed to get container status \"e0f5261f30fd436f3cc5191bf5d2fe73764fdab9992b31134d2d7dbbf9bcc54c\": rpc error: code = NotFound desc = could not find container \"e0f5261f30fd436f3cc5191bf5d2fe73764fdab9992b31134d2d7dbbf9bcc54c\": container with ID starting with e0f5261f30fd436f3cc5191bf5d2fe73764fdab9992b31134d2d7dbbf9bcc54c not found: ID does not exist" Jul 06 18:57:38.837362 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:57:38.837296 2570 scope.go:117] "RemoveContainer" containerID="d50576bc26b15d78d3e1b9b182ecf8016762de33614fb8d8e237e55d260720e5" Jul 06 18:57:38.837517 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:57:38.837500 2570 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d50576bc26b15d78d3e1b9b182ecf8016762de33614fb8d8e237e55d260720e5"} err="failed to get container status \"d50576bc26b15d78d3e1b9b182ecf8016762de33614fb8d8e237e55d260720e5\": rpc error: code = NotFound desc = could not find container \"d50576bc26b15d78d3e1b9b182ecf8016762de33614fb8d8e237e55d260720e5\": container with ID starting with d50576bc26b15d78d3e1b9b182ecf8016762de33614fb8d8e237e55d260720e5 not found: ID does not exist" Jul 06 18:57:38.837639 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:57:38.837517 2570 scope.go:117] "RemoveContainer" containerID="ec3c3442dab43c6a7de794ed52c5c293b5c7b76635bd2bb41ef15a4cffaf184a" Jul 06 18:57:38.837735 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:57:38.837713 2570 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ec3c3442dab43c6a7de794ed52c5c293b5c7b76635bd2bb41ef15a4cffaf184a"} err="failed to get container status \"ec3c3442dab43c6a7de794ed52c5c293b5c7b76635bd2bb41ef15a4cffaf184a\": rpc error: code = NotFound desc = could not find container \"ec3c3442dab43c6a7de794ed52c5c293b5c7b76635bd2bb41ef15a4cffaf184a\": container with ID starting with ec3c3442dab43c6a7de794ed52c5c293b5c7b76635bd2bb41ef15a4cffaf184a not found: ID does not exist" Jul 06 18:57:38.837801 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:57:38.837738 2570 scope.go:117] "RemoveContainer" containerID="2cb7f600ea6415e85375a9c11ea27b29e225aa43e7f430dace90788421c2467c" Jul 06 18:57:38.837998 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:57:38.837982 2570 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2cb7f600ea6415e85375a9c11ea27b29e225aa43e7f430dace90788421c2467c"} err="failed to get container status \"2cb7f600ea6415e85375a9c11ea27b29e225aa43e7f430dace90788421c2467c\": rpc error: code = NotFound desc = could not find container \"2cb7f600ea6415e85375a9c11ea27b29e225aa43e7f430dace90788421c2467c\": container with ID starting with 2cb7f600ea6415e85375a9c11ea27b29e225aa43e7f430dace90788421c2467c not found: ID does not exist" Jul 06 18:57:38.837998 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:57:38.837998 2570 scope.go:117] "RemoveContainer" containerID="7efcab14e9d4bed190760cf9ca3845e56cf9c917cdc6744f4b01234852be927a" Jul 06 18:57:38.838225 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:57:38.838209 2570 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7efcab14e9d4bed190760cf9ca3845e56cf9c917cdc6744f4b01234852be927a"} err="failed to get container status \"7efcab14e9d4bed190760cf9ca3845e56cf9c917cdc6744f4b01234852be927a\": rpc error: code = NotFound desc = could not find container \"7efcab14e9d4bed190760cf9ca3845e56cf9c917cdc6744f4b01234852be927a\": container with ID starting with 7efcab14e9d4bed190760cf9ca3845e56cf9c917cdc6744f4b01234852be927a not found: ID does not exist" Jul 06 18:57:38.838225 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:57:38.838224 2570 scope.go:117] "RemoveContainer" containerID="061222b702aae270175e5d72ed12de85bfcb3957293f36dab428c91346da5006" Jul 06 18:57:38.838407 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:57:38.838388 2570 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"061222b702aae270175e5d72ed12de85bfcb3957293f36dab428c91346da5006"} err="failed to get container status \"061222b702aae270175e5d72ed12de85bfcb3957293f36dab428c91346da5006\": rpc error: code = NotFound desc = could not find container \"061222b702aae270175e5d72ed12de85bfcb3957293f36dab428c91346da5006\": container with ID starting with 061222b702aae270175e5d72ed12de85bfcb3957293f36dab428c91346da5006 not found: ID does not exist" Jul 06 18:57:38.838407 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:57:38.838406 2570 scope.go:117] "RemoveContainer" containerID="824e75a7716111edd36acdce16301eda65bfd510a905d6f2eba53141bc8c875b" Jul 06 18:57:38.838643 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:57:38.838626 2570 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"824e75a7716111edd36acdce16301eda65bfd510a905d6f2eba53141bc8c875b"} err="failed to get container status \"824e75a7716111edd36acdce16301eda65bfd510a905d6f2eba53141bc8c875b\": rpc error: code = NotFound desc = could not find container \"824e75a7716111edd36acdce16301eda65bfd510a905d6f2eba53141bc8c875b\": container with ID starting with 824e75a7716111edd36acdce16301eda65bfd510a905d6f2eba53141bc8c875b not found: ID does not exist" Jul 06 18:57:38.838749 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:57:38.838644 2570 scope.go:117] "RemoveContainer" containerID="e0f5261f30fd436f3cc5191bf5d2fe73764fdab9992b31134d2d7dbbf9bcc54c" Jul 06 18:57:38.838870 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:57:38.838849 2570 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e0f5261f30fd436f3cc5191bf5d2fe73764fdab9992b31134d2d7dbbf9bcc54c"} err="failed to get container status \"e0f5261f30fd436f3cc5191bf5d2fe73764fdab9992b31134d2d7dbbf9bcc54c\": rpc error: code = NotFound desc = could not find container \"e0f5261f30fd436f3cc5191bf5d2fe73764fdab9992b31134d2d7dbbf9bcc54c\": container with ID starting with e0f5261f30fd436f3cc5191bf5d2fe73764fdab9992b31134d2d7dbbf9bcc54c not found: ID does not exist" Jul 06 18:57:38.838918 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:57:38.838872 2570 scope.go:117] "RemoveContainer" containerID="d50576bc26b15d78d3e1b9b182ecf8016762de33614fb8d8e237e55d260720e5" Jul 06 18:57:38.839077 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:57:38.839060 2570 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d50576bc26b15d78d3e1b9b182ecf8016762de33614fb8d8e237e55d260720e5"} err="failed to get container status \"d50576bc26b15d78d3e1b9b182ecf8016762de33614fb8d8e237e55d260720e5\": rpc error: code = NotFound desc = could not find container \"d50576bc26b15d78d3e1b9b182ecf8016762de33614fb8d8e237e55d260720e5\": container with ID starting with d50576bc26b15d78d3e1b9b182ecf8016762de33614fb8d8e237e55d260720e5 not found: ID does not exist" Jul 06 18:57:38.839077 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:57:38.839077 2570 scope.go:117] "RemoveContainer" containerID="ec3c3442dab43c6a7de794ed52c5c293b5c7b76635bd2bb41ef15a4cffaf184a" Jul 06 18:57:38.839269 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:57:38.839246 2570 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ec3c3442dab43c6a7de794ed52c5c293b5c7b76635bd2bb41ef15a4cffaf184a"} err="failed to get container status \"ec3c3442dab43c6a7de794ed52c5c293b5c7b76635bd2bb41ef15a4cffaf184a\": rpc error: code = NotFound desc = could not find container \"ec3c3442dab43c6a7de794ed52c5c293b5c7b76635bd2bb41ef15a4cffaf184a\": container with ID starting with ec3c3442dab43c6a7de794ed52c5c293b5c7b76635bd2bb41ef15a4cffaf184a not found: ID does not exist" Jul 06 18:57:38.839269 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:57:38.839264 2570 scope.go:117] "RemoveContainer" containerID="2cb7f600ea6415e85375a9c11ea27b29e225aa43e7f430dace90788421c2467c" Jul 06 18:57:38.839461 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:57:38.839446 2570 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2cb7f600ea6415e85375a9c11ea27b29e225aa43e7f430dace90788421c2467c"} err="failed to get container status \"2cb7f600ea6415e85375a9c11ea27b29e225aa43e7f430dace90788421c2467c\": rpc error: code = NotFound desc = could not find container \"2cb7f600ea6415e85375a9c11ea27b29e225aa43e7f430dace90788421c2467c\": container with ID starting with 2cb7f600ea6415e85375a9c11ea27b29e225aa43e7f430dace90788421c2467c not found: ID does not exist" Jul 06 18:57:38.839513 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:57:38.839461 2570 scope.go:117] "RemoveContainer" containerID="7efcab14e9d4bed190760cf9ca3845e56cf9c917cdc6744f4b01234852be927a" Jul 06 18:57:38.839700 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:57:38.839682 2570 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7efcab14e9d4bed190760cf9ca3845e56cf9c917cdc6744f4b01234852be927a"} err="failed to get container status \"7efcab14e9d4bed190760cf9ca3845e56cf9c917cdc6744f4b01234852be927a\": rpc error: code = NotFound desc = could not find container \"7efcab14e9d4bed190760cf9ca3845e56cf9c917cdc6744f4b01234852be927a\": container with ID starting with 7efcab14e9d4bed190760cf9ca3845e56cf9c917cdc6744f4b01234852be927a not found: ID does not exist" Jul 06 18:57:38.839700 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:57:38.839699 2570 scope.go:117] "RemoveContainer" containerID="061222b702aae270175e5d72ed12de85bfcb3957293f36dab428c91346da5006" Jul 06 18:57:38.839904 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:57:38.839889 2570 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"061222b702aae270175e5d72ed12de85bfcb3957293f36dab428c91346da5006"} err="failed to get container status \"061222b702aae270175e5d72ed12de85bfcb3957293f36dab428c91346da5006\": rpc error: code = NotFound desc = could not find container \"061222b702aae270175e5d72ed12de85bfcb3957293f36dab428c91346da5006\": container with ID starting with 061222b702aae270175e5d72ed12de85bfcb3957293f36dab428c91346da5006 not found: ID does not exist" Jul 06 18:57:38.839962 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:57:38.839904 2570 scope.go:117] "RemoveContainer" containerID="824e75a7716111edd36acdce16301eda65bfd510a905d6f2eba53141bc8c875b" Jul 06 18:57:38.840133 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:57:38.840117 2570 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"824e75a7716111edd36acdce16301eda65bfd510a905d6f2eba53141bc8c875b"} err="failed to get container status \"824e75a7716111edd36acdce16301eda65bfd510a905d6f2eba53141bc8c875b\": rpc error: code = NotFound desc = could not find container \"824e75a7716111edd36acdce16301eda65bfd510a905d6f2eba53141bc8c875b\": container with ID starting with 824e75a7716111edd36acdce16301eda65bfd510a905d6f2eba53141bc8c875b not found: ID does not exist" Jul 06 18:57:38.840133 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:57:38.840134 2570 scope.go:117] "RemoveContainer" containerID="e0f5261f30fd436f3cc5191bf5d2fe73764fdab9992b31134d2d7dbbf9bcc54c" Jul 06 18:57:38.840371 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:57:38.840354 2570 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e0f5261f30fd436f3cc5191bf5d2fe73764fdab9992b31134d2d7dbbf9bcc54c"} err="failed to get container status \"e0f5261f30fd436f3cc5191bf5d2fe73764fdab9992b31134d2d7dbbf9bcc54c\": rpc error: code = NotFound desc = could not find container \"e0f5261f30fd436f3cc5191bf5d2fe73764fdab9992b31134d2d7dbbf9bcc54c\": container with ID starting with e0f5261f30fd436f3cc5191bf5d2fe73764fdab9992b31134d2d7dbbf9bcc54c not found: ID does not exist" Jul 06 18:57:38.840414 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:57:38.840375 2570 scope.go:117] "RemoveContainer" containerID="d50576bc26b15d78d3e1b9b182ecf8016762de33614fb8d8e237e55d260720e5" Jul 06 18:57:38.840569 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:57:38.840554 2570 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d50576bc26b15d78d3e1b9b182ecf8016762de33614fb8d8e237e55d260720e5"} err="failed to get container status \"d50576bc26b15d78d3e1b9b182ecf8016762de33614fb8d8e237e55d260720e5\": rpc error: code = NotFound desc = could not find container \"d50576bc26b15d78d3e1b9b182ecf8016762de33614fb8d8e237e55d260720e5\": container with ID starting with d50576bc26b15d78d3e1b9b182ecf8016762de33614fb8d8e237e55d260720e5 not found: ID does not exist" Jul 06 18:57:38.840628 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:57:38.840570 2570 scope.go:117] "RemoveContainer" containerID="ec3c3442dab43c6a7de794ed52c5c293b5c7b76635bd2bb41ef15a4cffaf184a" Jul 06 18:57:38.840803 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:57:38.840787 2570 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ec3c3442dab43c6a7de794ed52c5c293b5c7b76635bd2bb41ef15a4cffaf184a"} err="failed to get container status \"ec3c3442dab43c6a7de794ed52c5c293b5c7b76635bd2bb41ef15a4cffaf184a\": rpc error: code = NotFound desc = could not find container \"ec3c3442dab43c6a7de794ed52c5c293b5c7b76635bd2bb41ef15a4cffaf184a\": container with ID starting with ec3c3442dab43c6a7de794ed52c5c293b5c7b76635bd2bb41ef15a4cffaf184a not found: ID does not exist" Jul 06 18:57:38.840855 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:57:38.840802 2570 scope.go:117] "RemoveContainer" containerID="2cb7f600ea6415e85375a9c11ea27b29e225aa43e7f430dace90788421c2467c" Jul 06 18:57:38.840997 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:57:38.840981 2570 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2cb7f600ea6415e85375a9c11ea27b29e225aa43e7f430dace90788421c2467c"} err="failed to get container status \"2cb7f600ea6415e85375a9c11ea27b29e225aa43e7f430dace90788421c2467c\": rpc error: code = NotFound desc = could not find container \"2cb7f600ea6415e85375a9c11ea27b29e225aa43e7f430dace90788421c2467c\": container with ID starting with 2cb7f600ea6415e85375a9c11ea27b29e225aa43e7f430dace90788421c2467c not found: ID does not exist" Jul 06 18:57:38.841034 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:57:38.840997 2570 scope.go:117] "RemoveContainer" containerID="7efcab14e9d4bed190760cf9ca3845e56cf9c917cdc6744f4b01234852be927a" Jul 06 18:57:38.841173 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:57:38.841159 2570 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7efcab14e9d4bed190760cf9ca3845e56cf9c917cdc6744f4b01234852be927a"} err="failed to get container status \"7efcab14e9d4bed190760cf9ca3845e56cf9c917cdc6744f4b01234852be927a\": rpc error: code = NotFound desc = could not find container \"7efcab14e9d4bed190760cf9ca3845e56cf9c917cdc6744f4b01234852be927a\": container with ID starting with 7efcab14e9d4bed190760cf9ca3845e56cf9c917cdc6744f4b01234852be927a not found: ID does not exist" Jul 06 18:57:38.841216 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:57:38.841173 2570 scope.go:117] "RemoveContainer" containerID="061222b702aae270175e5d72ed12de85bfcb3957293f36dab428c91346da5006" Jul 06 18:57:38.841360 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:57:38.841339 2570 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"061222b702aae270175e5d72ed12de85bfcb3957293f36dab428c91346da5006"} err="failed to get container status \"061222b702aae270175e5d72ed12de85bfcb3957293f36dab428c91346da5006\": rpc error: code = NotFound desc = could not find container \"061222b702aae270175e5d72ed12de85bfcb3957293f36dab428c91346da5006\": container with ID starting with 061222b702aae270175e5d72ed12de85bfcb3957293f36dab428c91346da5006 not found: ID does not exist" Jul 06 18:57:38.841401 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:57:38.841361 2570 scope.go:117] "RemoveContainer" containerID="824e75a7716111edd36acdce16301eda65bfd510a905d6f2eba53141bc8c875b" Jul 06 18:57:38.841526 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:57:38.841509 2570 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"824e75a7716111edd36acdce16301eda65bfd510a905d6f2eba53141bc8c875b"} err="failed to get container status \"824e75a7716111edd36acdce16301eda65bfd510a905d6f2eba53141bc8c875b\": rpc error: code = NotFound desc = could not find container \"824e75a7716111edd36acdce16301eda65bfd510a905d6f2eba53141bc8c875b\": container with ID starting with 824e75a7716111edd36acdce16301eda65bfd510a905d6f2eba53141bc8c875b not found: ID does not exist" Jul 06 18:57:38.902155 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:57:38.902125 2570 reconciler_common.go:299] "Volume detached for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/45a3639f-79ab-4bb6-8a64-f2b4458f3375-secret-metrics-client-certs\") on node \"ip-10-0-130-59.ec2.internal\" DevicePath \"\"" Jul 06 18:57:38.902155 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:57:38.902150 2570 reconciler_common.go:299] "Volume detached for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/45a3639f-79ab-4bb6-8a64-f2b4458f3375-secret-prometheus-k8s-kube-rbac-proxy-web\") on node \"ip-10-0-130-59.ec2.internal\" DevicePath \"\"" Jul 06 18:57:38.902155 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:57:38.902162 2570 reconciler_common.go:299] "Volume detached for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/45a3639f-79ab-4bb6-8a64-f2b4458f3375-tls-assets\") on node \"ip-10-0-130-59.ec2.internal\" DevicePath \"\"" Jul 06 18:57:38.902352 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:57:38.902171 2570 reconciler_common.go:299] "Volume detached for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/45a3639f-79ab-4bb6-8a64-f2b4458f3375-secret-kube-rbac-proxy\") on node \"ip-10-0-130-59.ec2.internal\" DevicePath \"\"" Jul 06 18:57:38.902352 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:57:38.902180 2570 reconciler_common.go:299] "Volume detached for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/45a3639f-79ab-4bb6-8a64-f2b4458f3375-config-out\") on node \"ip-10-0-130-59.ec2.internal\" DevicePath \"\"" Jul 06 18:57:38.902352 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:57:38.902188 2570 reconciler_common.go:299] "Volume detached for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/45a3639f-79ab-4bb6-8a64-f2b4458f3375-secret-prometheus-k8s-thanos-sidecar-tls\") on node \"ip-10-0-130-59.ec2.internal\" DevicePath \"\"" Jul 06 18:57:38.902352 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:57:38.902197 2570 reconciler_common.go:299] "Volume detached for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/45a3639f-79ab-4bb6-8a64-f2b4458f3375-configmap-serving-certs-ca-bundle\") on node \"ip-10-0-130-59.ec2.internal\" DevicePath \"\"" Jul 06 18:57:38.902352 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:57:38.902205 2570 reconciler_common.go:299] "Volume detached for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/45a3639f-79ab-4bb6-8a64-f2b4458f3375-prometheus-trusted-ca-bundle\") on node \"ip-10-0-130-59.ec2.internal\" DevicePath \"\"" Jul 06 18:57:38.902352 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:57:38.902213 2570 reconciler_common.go:299] "Volume detached for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/45a3639f-79ab-4bb6-8a64-f2b4458f3375-secret-prometheus-k8s-tls\") on node \"ip-10-0-130-59.ec2.internal\" DevicePath \"\"" Jul 06 18:57:38.902352 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:57:38.902222 2570 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/45a3639f-79ab-4bb6-8a64-f2b4458f3375-prometheus-k8s-rulefiles-0\") on node \"ip-10-0-130-59.ec2.internal\" DevicePath \"\"" Jul 06 18:57:38.902352 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:57:38.902230 2570 reconciler_common.go:299] "Volume detached for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/45a3639f-79ab-4bb6-8a64-f2b4458f3375-web-config\") on node \"ip-10-0-130-59.ec2.internal\" DevicePath \"\"" Jul 06 18:57:38.902352 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:57:38.902238 2570 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-4zv9t\" (UniqueName: \"kubernetes.io/projected/45a3639f-79ab-4bb6-8a64-f2b4458f3375-kube-api-access-4zv9t\") on node \"ip-10-0-130-59.ec2.internal\" DevicePath \"\"" Jul 06 18:57:38.902352 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:57:38.902247 2570 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/45a3639f-79ab-4bb6-8a64-f2b4458f3375-prometheus-k8s-rulefiles-1\") on node \"ip-10-0-130-59.ec2.internal\" DevicePath \"\"" Jul 06 18:57:38.902352 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:57:38.902255 2570 reconciler_common.go:299] "Volume detached for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/45a3639f-79ab-4bb6-8a64-f2b4458f3375-configmap-kubelet-serving-ca-bundle\") on node \"ip-10-0-130-59.ec2.internal\" DevicePath \"\"" Jul 06 18:57:38.902352 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:57:38.902265 2570 reconciler_common.go:299] "Volume detached for volume \"config\" (UniqueName: \"kubernetes.io/secret/45a3639f-79ab-4bb6-8a64-f2b4458f3375-config\") on node \"ip-10-0-130-59.ec2.internal\" DevicePath \"\"" Jul 06 18:57:38.902352 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:57:38.902272 2570 reconciler_common.go:299] "Volume detached for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/45a3639f-79ab-4bb6-8a64-f2b4458f3375-secret-grpc-tls\") on node \"ip-10-0-130-59.ec2.internal\" DevicePath \"\"" Jul 06 18:57:38.902352 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:57:38.902281 2570 reconciler_common.go:299] "Volume detached for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/45a3639f-79ab-4bb6-8a64-f2b4458f3375-thanos-prometheus-http-client-file\") on node \"ip-10-0-130-59.ec2.internal\" DevicePath \"\"" Jul 06 18:57:39.096867 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:57:39.096837 2570 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jul 06 18:57:39.101478 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:57:39.101452 2570 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jul 06 18:57:39.144808 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:57:39.144776 2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jul 06 18:57:39.145178 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:57:39.145160 2570 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="45a3639f-79ab-4bb6-8a64-f2b4458f3375" containerName="config-reloader" Jul 06 18:57:39.145178 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:57:39.145179 2570 state_mem.go:107] "Deleted CPUSet assignment" podUID="45a3639f-79ab-4bb6-8a64-f2b4458f3375" containerName="config-reloader" Jul 06 18:57:39.145301 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:57:39.145191 2570 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="45a3639f-79ab-4bb6-8a64-f2b4458f3375" containerName="prometheus" Jul 06 18:57:39.145301 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:57:39.145197 2570 state_mem.go:107] "Deleted CPUSet assignment" podUID="45a3639f-79ab-4bb6-8a64-f2b4458f3375" containerName="prometheus" Jul 06 18:57:39.145301 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:57:39.145206 2570 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="45a3639f-79ab-4bb6-8a64-f2b4458f3375" containerName="init-config-reloader" Jul 06 18:57:39.145301 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:57:39.145212 2570 state_mem.go:107] "Deleted CPUSet assignment" podUID="45a3639f-79ab-4bb6-8a64-f2b4458f3375" containerName="init-config-reloader" Jul 06 18:57:39.145301 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:57:39.145223 2570 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="45a3639f-79ab-4bb6-8a64-f2b4458f3375" containerName="thanos-sidecar" Jul 06 18:57:39.145301 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:57:39.145230 2570 state_mem.go:107] "Deleted CPUSet assignment" podUID="45a3639f-79ab-4bb6-8a64-f2b4458f3375" containerName="thanos-sidecar" Jul 06 18:57:39.145301 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:57:39.145236 2570 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="45a3639f-79ab-4bb6-8a64-f2b4458f3375" containerName="kube-rbac-proxy-web" Jul 06 18:57:39.145301 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:57:39.145241 2570 state_mem.go:107] "Deleted CPUSet assignment" podUID="45a3639f-79ab-4bb6-8a64-f2b4458f3375" containerName="kube-rbac-proxy-web" Jul 06 18:57:39.145301 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:57:39.145255 2570 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="45a3639f-79ab-4bb6-8a64-f2b4458f3375" containerName="kube-rbac-proxy-thanos" Jul 06 18:57:39.145301 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:57:39.145259 2570 state_mem.go:107] "Deleted CPUSet assignment" podUID="45a3639f-79ab-4bb6-8a64-f2b4458f3375" containerName="kube-rbac-proxy-thanos" Jul 06 18:57:39.145301 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:57:39.145266 2570 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="45a3639f-79ab-4bb6-8a64-f2b4458f3375" containerName="kube-rbac-proxy" Jul 06 18:57:39.145301 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:57:39.145270 2570 state_mem.go:107] "Deleted CPUSet assignment" podUID="45a3639f-79ab-4bb6-8a64-f2b4458f3375" containerName="kube-rbac-proxy" Jul 06 18:57:39.145699 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:57:39.145313 2570 memory_manager.go:368] "RemoveStaleState removing state" podUID="45a3639f-79ab-4bb6-8a64-f2b4458f3375" containerName="prometheus" Jul 06 18:57:39.145699 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:57:39.145321 2570 memory_manager.go:368] "RemoveStaleState removing state" podUID="45a3639f-79ab-4bb6-8a64-f2b4458f3375" containerName="kube-rbac-proxy-thanos" Jul 06 18:57:39.145699 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:57:39.145326 2570 memory_manager.go:368] "RemoveStaleState removing state" podUID="45a3639f-79ab-4bb6-8a64-f2b4458f3375" containerName="config-reloader" Jul 06 18:57:39.145699 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:57:39.145332 2570 memory_manager.go:368] "RemoveStaleState removing state" podUID="45a3639f-79ab-4bb6-8a64-f2b4458f3375" containerName="kube-rbac-proxy" Jul 06 18:57:39.145699 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:57:39.145339 2570 memory_manager.go:368] "RemoveStaleState removing state" podUID="45a3639f-79ab-4bb6-8a64-f2b4458f3375" containerName="thanos-sidecar" Jul 06 18:57:39.145699 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:57:39.145346 2570 memory_manager.go:368] "RemoveStaleState removing state" podUID="45a3639f-79ab-4bb6-8a64-f2b4458f3375" containerName="kube-rbac-proxy-web" Jul 06 18:57:39.149431 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:57:39.149415 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" Jul 06 18:57:39.153240 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:57:39.153218 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-dockercfg-wkf8m\"" Jul 06 18:57:39.153377 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:57:39.153290 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-rbac-proxy\"" Jul 06 18:57:39.153496 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:57:39.153482 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-1\"" Jul 06 18:57:39.153646 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:57:39.153630 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s\"" Jul 06 18:57:39.155163 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:57:39.155133 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-kube-rbac-proxy-web\"" Jul 06 18:57:39.155489 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:57:39.155321 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-web-config\"" Jul 06 18:57:39.155489 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:57:39.155412 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"serving-certs-ca-bundle\"" Jul 06 18:57:39.155682 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:57:39.155645 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-2\"" Jul 06 18:57:39.155682 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:57:39.155652 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-client-certs\"" Jul 06 18:57:39.155794 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:57:39.155651 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-sidecar-tls\"" Jul 06 18:57:39.156005 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:57:39.155981 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls-assets-0\"" Jul 06 18:57:39.156117 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:57:39.156100 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls\"" Jul 06 18:57:39.156396 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:57:39.156375 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-grpc-tls-bkvq98edilq4q\"" Jul 06 18:57:39.157224 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:57:39.157024 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-prometheus-http-client-file\"" Jul 06 18:57:39.159847 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:57:39.159826 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-0\"" Jul 06 18:57:39.162486 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:57:39.162465 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-trusted-ca-bundle\"" Jul 06 18:57:39.173924 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:57:39.173900 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jul 06 18:57:39.204787 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:57:39.204758 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/2907ab4e-a813-4e8a-9358-2021a20445a4-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"2907ab4e-a813-4e8a-9358-2021a20445a4\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 18:57:39.204934 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:57:39.204798 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/2907ab4e-a813-4e8a-9358-2021a20445a4-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"2907ab4e-a813-4e8a-9358-2021a20445a4\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 18:57:39.204934 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:57:39.204821 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-h9zj8\" (UniqueName: \"kubernetes.io/projected/2907ab4e-a813-4e8a-9358-2021a20445a4-kube-api-access-h9zj8\") pod \"prometheus-k8s-0\" (UID: \"2907ab4e-a813-4e8a-9358-2021a20445a4\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 18:57:39.204934 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:57:39.204844 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/2907ab4e-a813-4e8a-9358-2021a20445a4-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"2907ab4e-a813-4e8a-9358-2021a20445a4\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 18:57:39.204934 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:57:39.204908 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/2907ab4e-a813-4e8a-9358-2021a20445a4-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"2907ab4e-a813-4e8a-9358-2021a20445a4\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 18:57:39.205117 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:57:39.204936 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/2907ab4e-a813-4e8a-9358-2021a20445a4-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"2907ab4e-a813-4e8a-9358-2021a20445a4\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 18:57:39.205117 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:57:39.204961 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/2907ab4e-a813-4e8a-9358-2021a20445a4-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"2907ab4e-a813-4e8a-9358-2021a20445a4\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 18:57:39.205117 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:57:39.205001 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/2907ab4e-a813-4e8a-9358-2021a20445a4-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"2907ab4e-a813-4e8a-9358-2021a20445a4\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 18:57:39.205117 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:57:39.205024 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/2907ab4e-a813-4e8a-9358-2021a20445a4-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"2907ab4e-a813-4e8a-9358-2021a20445a4\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 18:57:39.205117 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:57:39.205059 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/2907ab4e-a813-4e8a-9358-2021a20445a4-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"2907ab4e-a813-4e8a-9358-2021a20445a4\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 18:57:39.205117 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:57:39.205088 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/2907ab4e-a813-4e8a-9358-2021a20445a4-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"2907ab4e-a813-4e8a-9358-2021a20445a4\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 18:57:39.205408 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:57:39.205154 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/2907ab4e-a813-4e8a-9358-2021a20445a4-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"2907ab4e-a813-4e8a-9358-2021a20445a4\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 18:57:39.205408 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:57:39.205190 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/2907ab4e-a813-4e8a-9358-2021a20445a4-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"2907ab4e-a813-4e8a-9358-2021a20445a4\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 18:57:39.205408 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:57:39.205217 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/2907ab4e-a813-4e8a-9358-2021a20445a4-web-config\") pod \"prometheus-k8s-0\" (UID: \"2907ab4e-a813-4e8a-9358-2021a20445a4\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 18:57:39.205408 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:57:39.205242 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/2907ab4e-a813-4e8a-9358-2021a20445a4-config-out\") pod \"prometheus-k8s-0\" (UID: \"2907ab4e-a813-4e8a-9358-2021a20445a4\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 18:57:39.205408 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:57:39.205265 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/2907ab4e-a813-4e8a-9358-2021a20445a4-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"2907ab4e-a813-4e8a-9358-2021a20445a4\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 18:57:39.205408 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:57:39.205290 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/2907ab4e-a813-4e8a-9358-2021a20445a4-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"2907ab4e-a813-4e8a-9358-2021a20445a4\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 18:57:39.205408 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:57:39.205310 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/2907ab4e-a813-4e8a-9358-2021a20445a4-config\") pod \"prometheus-k8s-0\" (UID: \"2907ab4e-a813-4e8a-9358-2021a20445a4\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 18:57:39.205408 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:57:39.205324 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/2907ab4e-a813-4e8a-9358-2021a20445a4-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"2907ab4e-a813-4e8a-9358-2021a20445a4\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 18:57:39.205408 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:57:39.205337 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/2907ab4e-a813-4e8a-9358-2021a20445a4-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"2907ab4e-a813-4e8a-9358-2021a20445a4\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 18:57:39.305827 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:57:39.305791 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-h9zj8\" (UniqueName: \"kubernetes.io/projected/2907ab4e-a813-4e8a-9358-2021a20445a4-kube-api-access-h9zj8\") pod \"prometheus-k8s-0\" (UID: \"2907ab4e-a813-4e8a-9358-2021a20445a4\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 18:57:39.305982 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:57:39.305845 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/2907ab4e-a813-4e8a-9358-2021a20445a4-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"2907ab4e-a813-4e8a-9358-2021a20445a4\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 18:57:39.305982 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:57:39.305883 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/2907ab4e-a813-4e8a-9358-2021a20445a4-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"2907ab4e-a813-4e8a-9358-2021a20445a4\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 18:57:39.305982 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:57:39.305910 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/2907ab4e-a813-4e8a-9358-2021a20445a4-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"2907ab4e-a813-4e8a-9358-2021a20445a4\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 18:57:39.305982 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:57:39.305934 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/2907ab4e-a813-4e8a-9358-2021a20445a4-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"2907ab4e-a813-4e8a-9358-2021a20445a4\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 18:57:39.306198 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:57:39.305984 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/2907ab4e-a813-4e8a-9358-2021a20445a4-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"2907ab4e-a813-4e8a-9358-2021a20445a4\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 18:57:39.306198 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:57:39.306009 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/2907ab4e-a813-4e8a-9358-2021a20445a4-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"2907ab4e-a813-4e8a-9358-2021a20445a4\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 18:57:39.306198 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:57:39.306058 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/2907ab4e-a813-4e8a-9358-2021a20445a4-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"2907ab4e-a813-4e8a-9358-2021a20445a4\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 18:57:39.306198 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:57:39.306092 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/2907ab4e-a813-4e8a-9358-2021a20445a4-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"2907ab4e-a813-4e8a-9358-2021a20445a4\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 18:57:39.306198 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:57:39.306124 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/2907ab4e-a813-4e8a-9358-2021a20445a4-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"2907ab4e-a813-4e8a-9358-2021a20445a4\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 18:57:39.306198 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:57:39.306168 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/2907ab4e-a813-4e8a-9358-2021a20445a4-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"2907ab4e-a813-4e8a-9358-2021a20445a4\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 18:57:39.306555 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:57:39.306211 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/2907ab4e-a813-4e8a-9358-2021a20445a4-web-config\") pod \"prometheus-k8s-0\" (UID: \"2907ab4e-a813-4e8a-9358-2021a20445a4\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 18:57:39.306555 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:57:39.306242 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/2907ab4e-a813-4e8a-9358-2021a20445a4-config-out\") pod \"prometheus-k8s-0\" (UID: \"2907ab4e-a813-4e8a-9358-2021a20445a4\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 18:57:39.306555 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:57:39.306266 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/2907ab4e-a813-4e8a-9358-2021a20445a4-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"2907ab4e-a813-4e8a-9358-2021a20445a4\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 18:57:39.306555 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:57:39.306290 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/2907ab4e-a813-4e8a-9358-2021a20445a4-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"2907ab4e-a813-4e8a-9358-2021a20445a4\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 18:57:39.306555 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:57:39.306317 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/2907ab4e-a813-4e8a-9358-2021a20445a4-config\") pod \"prometheus-k8s-0\" (UID: \"2907ab4e-a813-4e8a-9358-2021a20445a4\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 18:57:39.306555 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:57:39.306343 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/2907ab4e-a813-4e8a-9358-2021a20445a4-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"2907ab4e-a813-4e8a-9358-2021a20445a4\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 18:57:39.306555 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:57:39.306368 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/2907ab4e-a813-4e8a-9358-2021a20445a4-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"2907ab4e-a813-4e8a-9358-2021a20445a4\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 18:57:39.306555 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:57:39.306418 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/2907ab4e-a813-4e8a-9358-2021a20445a4-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"2907ab4e-a813-4e8a-9358-2021a20445a4\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 18:57:39.306555 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:57:39.306457 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/2907ab4e-a813-4e8a-9358-2021a20445a4-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"2907ab4e-a813-4e8a-9358-2021a20445a4\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 18:57:39.306555 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:57:39.306488 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/2907ab4e-a813-4e8a-9358-2021a20445a4-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"2907ab4e-a813-4e8a-9358-2021a20445a4\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 18:57:39.307053 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:57:39.306984 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/2907ab4e-a813-4e8a-9358-2021a20445a4-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"2907ab4e-a813-4e8a-9358-2021a20445a4\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 18:57:39.308439 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:57:39.308410 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/2907ab4e-a813-4e8a-9358-2021a20445a4-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"2907ab4e-a813-4e8a-9358-2021a20445a4\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 18:57:39.309375 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:57:39.309310 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/2907ab4e-a813-4e8a-9358-2021a20445a4-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"2907ab4e-a813-4e8a-9358-2021a20445a4\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 18:57:39.309375 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:57:39.309367 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/2907ab4e-a813-4e8a-9358-2021a20445a4-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"2907ab4e-a813-4e8a-9358-2021a20445a4\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 18:57:39.309542 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:57:39.309420 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/2907ab4e-a813-4e8a-9358-2021a20445a4-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"2907ab4e-a813-4e8a-9358-2021a20445a4\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 18:57:39.310676 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:57:39.309825 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/2907ab4e-a813-4e8a-9358-2021a20445a4-config-out\") pod \"prometheus-k8s-0\" (UID: \"2907ab4e-a813-4e8a-9358-2021a20445a4\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 18:57:39.310676 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:57:39.309959 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/2907ab4e-a813-4e8a-9358-2021a20445a4-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"2907ab4e-a813-4e8a-9358-2021a20445a4\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 18:57:39.310676 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:57:39.310151 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/2907ab4e-a813-4e8a-9358-2021a20445a4-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"2907ab4e-a813-4e8a-9358-2021a20445a4\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 18:57:39.310676 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:57:39.310349 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/2907ab4e-a813-4e8a-9358-2021a20445a4-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"2907ab4e-a813-4e8a-9358-2021a20445a4\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 18:57:39.310676 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:57:39.310659 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/2907ab4e-a813-4e8a-9358-2021a20445a4-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"2907ab4e-a813-4e8a-9358-2021a20445a4\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 18:57:39.310986 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:57:39.310809 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/2907ab4e-a813-4e8a-9358-2021a20445a4-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"2907ab4e-a813-4e8a-9358-2021a20445a4\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 18:57:39.311075 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:57:39.311044 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/2907ab4e-a813-4e8a-9358-2021a20445a4-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"2907ab4e-a813-4e8a-9358-2021a20445a4\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 18:57:39.311180 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:57:39.311152 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/2907ab4e-a813-4e8a-9358-2021a20445a4-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"2907ab4e-a813-4e8a-9358-2021a20445a4\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 18:57:39.311896 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:57:39.311836 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/2907ab4e-a813-4e8a-9358-2021a20445a4-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"2907ab4e-a813-4e8a-9358-2021a20445a4\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 18:57:39.312147 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:57:39.312125 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/2907ab4e-a813-4e8a-9358-2021a20445a4-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"2907ab4e-a813-4e8a-9358-2021a20445a4\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 18:57:39.312735 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:57:39.312716 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/2907ab4e-a813-4e8a-9358-2021a20445a4-web-config\") pod \"prometheus-k8s-0\" (UID: \"2907ab4e-a813-4e8a-9358-2021a20445a4\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 18:57:39.312900 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:57:39.312874 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/2907ab4e-a813-4e8a-9358-2021a20445a4-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"2907ab4e-a813-4e8a-9358-2021a20445a4\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 18:57:39.313051 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:57:39.313034 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/secret/2907ab4e-a813-4e8a-9358-2021a20445a4-config\") pod \"prometheus-k8s-0\" (UID: \"2907ab4e-a813-4e8a-9358-2021a20445a4\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 18:57:39.315124 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:57:39.315105 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-h9zj8\" (UniqueName: \"kubernetes.io/projected/2907ab4e-a813-4e8a-9358-2021a20445a4-kube-api-access-h9zj8\") pod \"prometheus-k8s-0\" (UID: \"2907ab4e-a813-4e8a-9358-2021a20445a4\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 18:57:39.460950 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:57:39.460909 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" Jul 06 18:57:39.615055 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:57:39.615022 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jul 06 18:57:39.615983 ip-10-0-130-59 kubenswrapper[2570]: W0706 18:57:39.615960 2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2907ab4e_a813_4e8a_9358_2021a20445a4.slice/crio-fe841180cba63a510af2334eb91e0988ec4a8498606d47cf152f91d014310ac6 WatchSource:0}: Error finding container fe841180cba63a510af2334eb91e0988ec4a8498606d47cf152f91d014310ac6: Status 404 returned error can't find the container with id fe841180cba63a510af2334eb91e0988ec4a8498606d47cf152f91d014310ac6 Jul 06 18:57:39.788079 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:57:39.788034 2570 generic.go:356] "Generic (PLEG): container finished" podID="2907ab4e-a813-4e8a-9358-2021a20445a4" containerID="2be0c376e9c3a8dcfa7bc084965bf641d3fce006b17dcf65bf4a13a3fc3a7c46" exitCode=0 Jul 06 18:57:39.788541 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:57:39.788113 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"2907ab4e-a813-4e8a-9358-2021a20445a4","Type":"ContainerDied","Data":"2be0c376e9c3a8dcfa7bc084965bf641d3fce006b17dcf65bf4a13a3fc3a7c46"} Jul 06 18:57:39.788541 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:57:39.788223 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"2907ab4e-a813-4e8a-9358-2021a20445a4","Type":"ContainerStarted","Data":"fe841180cba63a510af2334eb91e0988ec4a8498606d47cf152f91d014310ac6"} Jul 06 18:57:40.796424 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:57:40.796393 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"2907ab4e-a813-4e8a-9358-2021a20445a4","Type":"ContainerStarted","Data":"d5ad6ce4cab606bfaf4f67e181b5810ed138f2de88487a9c7c7660945782c163"} Jul 06 18:57:40.796786 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:57:40.796474 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"2907ab4e-a813-4e8a-9358-2021a20445a4","Type":"ContainerStarted","Data":"dc41162c8c3d52bfcec5d5685278174ae2e58fff1663283d9fc0bdcf1d35537d"} Jul 06 18:57:40.796786 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:57:40.796519 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"2907ab4e-a813-4e8a-9358-2021a20445a4","Type":"ContainerStarted","Data":"5e7c41099602f8ae023f768bad01f89541cd8ee14cc90d84ef63e914bd919d31"} Jul 06 18:57:40.796786 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:57:40.796552 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"2907ab4e-a813-4e8a-9358-2021a20445a4","Type":"ContainerStarted","Data":"7e9d6ecb9dba17053184e48030e06ce47d371c4696bddb159f8a9733e465d5bf"} Jul 06 18:57:40.796786 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:57:40.796576 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"2907ab4e-a813-4e8a-9358-2021a20445a4","Type":"ContainerStarted","Data":"a3127d28094044268baeaa0b4780d71dd7caf7fad34041e5a76c1cae3af61233"} Jul 06 18:57:40.796786 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:57:40.796636 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"2907ab4e-a813-4e8a-9358-2021a20445a4","Type":"ContainerStarted","Data":"d690f23ae9006935cc6cdac73ca4225d21644fd8a28a984f7d499f094cb83c89"} Jul 06 18:57:40.798596 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:57:40.798564 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-7c47bc74cb-xrsz6" event={"ID":"564f0d46-248e-4a95-9416-2a6d23478fb0","Type":"ContainerStarted","Data":"3c025e0029b9237ef0925f86f218a407386680da0f9e046c98e4af4060cfaf00"} Jul 06 18:57:40.798684 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:57:40.798634 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-7c47bc74cb-xrsz6" event={"ID":"564f0d46-248e-4a95-9416-2a6d23478fb0","Type":"ContainerStarted","Data":"afad3a2e8ce1240f9fd7c76b92e56368f40aa17c25a795bb4cee6190d866b7d6"} Jul 06 18:57:40.830563 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:57:40.830261 2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-k8s-0" podStartSLOduration=1.814636267 podStartE2EDuration="1.830239779s" totalImagesPullingTime="0s" totalInitContainerRuntime="15.603512ms" isStatefulPod=false podCreationTimestamp="2026-07-06 18:57:39 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 18:57:40.828426402 +0000 UTC m=+140.383447597" watchObservedRunningTime="2026-07-06 18:57:40.830239779 +0000 UTC m=+140.385260952" Jul 06 18:57:41.027736 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:57:41.027703 2570 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="45a3639f-79ab-4bb6-8a64-f2b4458f3375" path="/var/lib/kubelet/pods/45a3639f-79ab-4bb6-8a64-f2b4458f3375/volumes" Jul 06 18:57:41.805135 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:57:41.805093 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-7c47bc74cb-xrsz6" event={"ID":"564f0d46-248e-4a95-9416-2a6d23478fb0","Type":"ContainerStarted","Data":"cfa5ff8405b3a74c6bdc955b2e59a30b420489c755053d95a4a94f4b3a95b6fb"} Jul 06 18:57:41.832399 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:57:41.832348 2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/telemeter-client-7c47bc74cb-xrsz6" podStartSLOduration=1.850548426 podStartE2EDuration="3.832332555s" totalImagesPullingTime="1.981784129s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 18:57:38 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 18:57:41.829826542 +0000 UTC m=+141.384847713" watchObservedRunningTime="2026-07-06 18:57:41.832332555 +0000 UTC m=+141.387353727" Jul 06 18:57:42.660442 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:57:42.660395 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k8fbq_bc7057bd-88da-48af-ba32-e6efdc6e8362/konnectivity-agent/0.log" limit=1024 Jul 06 18:57:42.751861 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:57:42.751836 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k8fbq_bc7057bd-88da-48af-ba32-e6efdc6e8362/konnectivity-agent/0.log" limit=1024 Jul 06 18:57:42.809792 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:57:42.809763 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k8fbq_bc7057bd-88da-48af-ba32-e6efdc6e8362/konnectivity-agent/0.log" limit=1024 Jul 06 18:57:43.372104 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:57:43.372069 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k8fbq_bc7057bd-88da-48af-ba32-e6efdc6e8362/konnectivity-agent/0.log" limit=1024 Jul 06 18:57:43.472497 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:57:43.472468 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k8fbq_bc7057bd-88da-48af-ba32-e6efdc6e8362/konnectivity-agent/0.log" limit=1024 Jul 06 18:57:44.461844 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:57:44.461808 2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-k8s-0" Jul 06 18:57:53.696356 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:57:53.696318 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k8fbq_bc7057bd-88da-48af-ba32-e6efdc6e8362/konnectivity-agent/0.log" limit=1024 Jul 06 18:57:53.801527 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:57:53.801495 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k8fbq_bc7057bd-88da-48af-ba32-e6efdc6e8362/konnectivity-agent/0.log" limit=1024 Jul 06 18:57:53.900141 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:57:53.900106 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k8fbq_bc7057bd-88da-48af-ba32-e6efdc6e8362/konnectivity-agent/0.log" limit=1024 Jul 06 18:58:29.145631 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:58:29.145574 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k8fbq_bc7057bd-88da-48af-ba32-e6efdc6e8362/konnectivity-agent/0.log" limit=1024 Jul 06 18:58:29.228750 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:58:29.228719 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k8fbq_bc7057bd-88da-48af-ba32-e6efdc6e8362/konnectivity-agent/0.log" limit=1024 Jul 06 18:58:39.461876 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:58:39.461835 2570 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/prometheus-k8s-0" Jul 06 18:58:39.477271 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:58:39.477246 2570 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/prometheus-k8s-0" Jul 06 18:58:40.059355 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:58:40.059328 2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-k8s-0" Jul 06 18:58:48.385475 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:58:48.385438 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k8fbq_bc7057bd-88da-48af-ba32-e6efdc6e8362/konnectivity-agent/0.log" limit=1024 Jul 06 18:58:48.443318 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:58:48.443287 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k8fbq_bc7057bd-88da-48af-ba32-e6efdc6e8362/konnectivity-agent/0.log" limit=1024 Jul 06 18:58:58.656995 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:58:58.656963 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k8fbq_bc7057bd-88da-48af-ba32-e6efdc6e8362/konnectivity-agent/0.log" limit=1024 Jul 06 18:58:58.735828 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:58:58.735795 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k8fbq_bc7057bd-88da-48af-ba32-e6efdc6e8362/konnectivity-agent/0.log" limit=1024 Jul 06 18:59:00.393230 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:59:00.393190 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k8fbq_bc7057bd-88da-48af-ba32-e6efdc6e8362/konnectivity-agent/0.log" limit=1024 Jul 06 18:59:00.469224 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:59:00.469193 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k8fbq_bc7057bd-88da-48af-ba32-e6efdc6e8362/konnectivity-agent/0.log" limit=1024 Jul 06 18:59:09.945122 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:59:09.945091 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k8fbq_bc7057bd-88da-48af-ba32-e6efdc6e8362/konnectivity-agent/0.log" limit=1024 Jul 06 18:59:10.044801 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:59:10.044760 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k8fbq_bc7057bd-88da-48af-ba32-e6efdc6e8362/konnectivity-agent/0.log" limit=1024 Jul 06 18:59:10.099139 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:59:10.099070 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k8fbq_bc7057bd-88da-48af-ba32-e6efdc6e8362/konnectivity-agent/0.log" limit=1024 Jul 06 18:59:16.241017 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:59:16.240979 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k8fbq_bc7057bd-88da-48af-ba32-e6efdc6e8362/konnectivity-agent/0.log" limit=1024 Jul 06 18:59:16.332425 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:59:16.332389 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k8fbq_bc7057bd-88da-48af-ba32-e6efdc6e8362/konnectivity-agent/0.log" limit=1024 Jul 06 18:59:16.415706 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:59:16.415674 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k8fbq_bc7057bd-88da-48af-ba32-e6efdc6e8362/konnectivity-agent/0.log" limit=1024 Jul 06 18:59:55.641337 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:59:55.641251 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k8fbq_bc7057bd-88da-48af-ba32-e6efdc6e8362/konnectivity-agent/0.log" limit=1024 Jul 06 18:59:56.725320 ip-10-0-130-59 kubenswrapper[2570]: I0706 18:59:56.725283 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k8fbq_bc7057bd-88da-48af-ba32-e6efdc6e8362/konnectivity-agent/0.log" limit=1024 Jul 06 19:00:22.679315 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:00:22.679261 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k8fbq_bc7057bd-88da-48af-ba32-e6efdc6e8362/konnectivity-agent/0.log" limit=1024 Jul 06 19:00:23.599516 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:00:23.599483 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k8fbq_bc7057bd-88da-48af-ba32-e6efdc6e8362/konnectivity-agent/0.log" limit=1024 Jul 06 19:00:26.536286 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:00:26.536251 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k8fbq_bc7057bd-88da-48af-ba32-e6efdc6e8362/konnectivity-agent/0.log" limit=1024 Jul 06 19:00:26.645951 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:00:26.645926 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k8fbq_bc7057bd-88da-48af-ba32-e6efdc6e8362/konnectivity-agent/0.log" limit=1024 Jul 06 19:00:27.706210 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:00:27.706175 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k8fbq_bc7057bd-88da-48af-ba32-e6efdc6e8362/konnectivity-agent/0.log" limit=1024 Jul 06 19:00:27.833119 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:00:27.833084 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k8fbq_bc7057bd-88da-48af-ba32-e6efdc6e8362/konnectivity-agent/0.log" limit=1024 Jul 06 19:00:28.485355 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:00:28.485321 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k8fbq_bc7057bd-88da-48af-ba32-e6efdc6e8362/konnectivity-agent/0.log" limit=1024 Jul 06 19:00:28.674417 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:00:28.674383 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k8fbq_bc7057bd-88da-48af-ba32-e6efdc6e8362/konnectivity-agent/0.log" limit=1024 Jul 06 19:00:28.803720 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:00:28.803638 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k8fbq_bc7057bd-88da-48af-ba32-e6efdc6e8362/konnectivity-agent/0.log" limit=1024 Jul 06 19:00:30.832264 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:00:30.832220 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k8fbq_bc7057bd-88da-48af-ba32-e6efdc6e8362/konnectivity-agent/0.log" limit=1024 Jul 06 19:00:31.063401 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:00:31.063369 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k8fbq_bc7057bd-88da-48af-ba32-e6efdc6e8362/konnectivity-agent/0.log" limit=1024 Jul 06 19:00:31.753923 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:00:31.753892 2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-cainjector-559cdd8c87-qhrbc"] Jul 06 19:00:31.757098 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:00:31.757080 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-cainjector-559cdd8c87-qhrbc" Jul 06 19:00:31.762449 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:00:31.762425 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"kube-root-ca.crt\"" Jul 06 19:00:31.763382 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:00:31.763360 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"openshift-service-ca.crt\"" Jul 06 19:00:31.763539 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:00:31.763403 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-cainjector-dockercfg-mjnzh\"" Jul 06 19:00:31.772886 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:00:31.772861 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-cainjector-559cdd8c87-qhrbc"] Jul 06 19:00:31.878288 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:00:31.878250 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/5d40c2bf-adb8-4444-832c-0f86f384bc78-bound-sa-token\") pod \"cert-manager-cainjector-559cdd8c87-qhrbc\" (UID: \"5d40c2bf-adb8-4444-832c-0f86f384bc78\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-qhrbc" Jul 06 19:00:31.878697 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:00:31.878365 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xrfxz\" (UniqueName: \"kubernetes.io/projected/5d40c2bf-adb8-4444-832c-0f86f384bc78-kube-api-access-xrfxz\") pod \"cert-manager-cainjector-559cdd8c87-qhrbc\" (UID: \"5d40c2bf-adb8-4444-832c-0f86f384bc78\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-qhrbc" Jul 06 19:00:31.979717 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:00:31.979679 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/5d40c2bf-adb8-4444-832c-0f86f384bc78-bound-sa-token\") pod \"cert-manager-cainjector-559cdd8c87-qhrbc\" (UID: \"5d40c2bf-adb8-4444-832c-0f86f384bc78\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-qhrbc" Jul 06 19:00:31.979894 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:00:31.979772 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xrfxz\" (UniqueName: \"kubernetes.io/projected/5d40c2bf-adb8-4444-832c-0f86f384bc78-kube-api-access-xrfxz\") pod \"cert-manager-cainjector-559cdd8c87-qhrbc\" (UID: \"5d40c2bf-adb8-4444-832c-0f86f384bc78\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-qhrbc" Jul 06 19:00:31.995266 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:00:31.995236 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/5d40c2bf-adb8-4444-832c-0f86f384bc78-bound-sa-token\") pod \"cert-manager-cainjector-559cdd8c87-qhrbc\" (UID: \"5d40c2bf-adb8-4444-832c-0f86f384bc78\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-qhrbc" Jul 06 19:00:31.995454 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:00:31.995433 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-xrfxz\" (UniqueName: \"kubernetes.io/projected/5d40c2bf-adb8-4444-832c-0f86f384bc78-kube-api-access-xrfxz\") pod \"cert-manager-cainjector-559cdd8c87-qhrbc\" (UID: \"5d40c2bf-adb8-4444-832c-0f86f384bc78\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-qhrbc" Jul 06 19:00:32.077776 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:00:32.077684 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-cainjector-559cdd8c87-qhrbc" Jul 06 19:00:32.213184 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:00:32.213155 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-cainjector-559cdd8c87-qhrbc"] Jul 06 19:00:32.216730 ip-10-0-130-59 kubenswrapper[2570]: W0706 19:00:32.216706 2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod5d40c2bf_adb8_4444_832c_0f86f384bc78.slice/crio-17329090be40a35e89a7b016ce4c00d04f5305d2872d589ee8eff3cb664af576 WatchSource:0}: Error finding container 17329090be40a35e89a7b016ce4c00d04f5305d2872d589ee8eff3cb664af576: Status 404 returned error can't find the container with id 17329090be40a35e89a7b016ce4c00d04f5305d2872d589ee8eff3cb664af576 Jul 06 19:00:32.218785 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:00:32.218766 2570 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 06 19:00:32.465405 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:00:32.465358 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-cainjector-559cdd8c87-qhrbc" event={"ID":"5d40c2bf-adb8-4444-832c-0f86f384bc78","Type":"ContainerStarted","Data":"17329090be40a35e89a7b016ce4c00d04f5305d2872d589ee8eff3cb664af576"} Jul 06 19:00:33.295312 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:00:33.295276 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k8fbq_bc7057bd-88da-48af-ba32-e6efdc6e8362/konnectivity-agent/0.log" limit=1024 Jul 06 19:00:33.395090 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:00:33.395046 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k8fbq_bc7057bd-88da-48af-ba32-e6efdc6e8362/konnectivity-agent/0.log" limit=1024 Jul 06 19:00:35.481475 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:00:35.481440 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-cainjector-559cdd8c87-qhrbc" event={"ID":"5d40c2bf-adb8-4444-832c-0f86f384bc78","Type":"ContainerStarted","Data":"c8315e19d6ff7783dce8c2390f658dcb739e204171270fa51021f952eb5b5908"} Jul 06 19:00:35.498126 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:00:35.498081 2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager/cert-manager-cainjector-559cdd8c87-qhrbc" podStartSLOduration=1.338281624 podStartE2EDuration="4.498065705s" totalImagesPullingTime="3.159784081s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 19:00:31 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 19:00:35.497234902 +0000 UTC m=+315.052256076" watchObservedRunningTime="2026-07-06 19:00:35.498065705 +0000 UTC m=+315.053086877" Jul 06 19:00:36.717249 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:00:36.717213 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k8fbq_bc7057bd-88da-48af-ba32-e6efdc6e8362/konnectivity-agent/0.log" limit=1024 Jul 06 19:00:36.815721 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:00:36.815688 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k8fbq_bc7057bd-88da-48af-ba32-e6efdc6e8362/konnectivity-agent/0.log" limit=1024 Jul 06 19:00:38.429908 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:00:38.429879 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k8fbq_bc7057bd-88da-48af-ba32-e6efdc6e8362/konnectivity-agent/0.log" limit=1024 Jul 06 19:00:39.477102 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:00:39.477063 2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-55b4d954d5-bqwss"] Jul 06 19:00:39.480428 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:00:39.480404 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-55b4d954d5-bqwss" Jul 06 19:00:39.483160 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:00:39.483142 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-dockercfg-lk24m\"" Jul 06 19:00:39.491489 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:00:39.491464 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-55b4d954d5-bqwss"] Jul 06 19:00:39.549256 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:00:39.549210 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/22d32b1c-3352-4d8d-b0ad-c24c95a4ac5b-bound-sa-token\") pod \"cert-manager-55b4d954d5-bqwss\" (UID: \"22d32b1c-3352-4d8d-b0ad-c24c95a4ac5b\") " pod="cert-manager/cert-manager-55b4d954d5-bqwss" Jul 06 19:00:39.549256 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:00:39.549254 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-55vg2\" (UniqueName: \"kubernetes.io/projected/22d32b1c-3352-4d8d-b0ad-c24c95a4ac5b-kube-api-access-55vg2\") pod \"cert-manager-55b4d954d5-bqwss\" (UID: \"22d32b1c-3352-4d8d-b0ad-c24c95a4ac5b\") " pod="cert-manager/cert-manager-55b4d954d5-bqwss" Jul 06 19:00:39.650151 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:00:39.650105 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/22d32b1c-3352-4d8d-b0ad-c24c95a4ac5b-bound-sa-token\") pod \"cert-manager-55b4d954d5-bqwss\" (UID: \"22d32b1c-3352-4d8d-b0ad-c24c95a4ac5b\") " pod="cert-manager/cert-manager-55b4d954d5-bqwss" Jul 06 19:00:39.650151 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:00:39.650156 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-55vg2\" (UniqueName: \"kubernetes.io/projected/22d32b1c-3352-4d8d-b0ad-c24c95a4ac5b-kube-api-access-55vg2\") pod \"cert-manager-55b4d954d5-bqwss\" (UID: \"22d32b1c-3352-4d8d-b0ad-c24c95a4ac5b\") " pod="cert-manager/cert-manager-55b4d954d5-bqwss" Jul 06 19:00:39.659637 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:00:39.659590 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/22d32b1c-3352-4d8d-b0ad-c24c95a4ac5b-bound-sa-token\") pod \"cert-manager-55b4d954d5-bqwss\" (UID: \"22d32b1c-3352-4d8d-b0ad-c24c95a4ac5b\") " pod="cert-manager/cert-manager-55b4d954d5-bqwss" Jul 06 19:00:39.659908 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:00:39.659887 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-55vg2\" (UniqueName: \"kubernetes.io/projected/22d32b1c-3352-4d8d-b0ad-c24c95a4ac5b-kube-api-access-55vg2\") pod \"cert-manager-55b4d954d5-bqwss\" (UID: \"22d32b1c-3352-4d8d-b0ad-c24c95a4ac5b\") " pod="cert-manager/cert-manager-55b4d954d5-bqwss" Jul 06 19:00:39.789865 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:00:39.789775 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-55b4d954d5-bqwss" Jul 06 19:00:39.924124 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:00:39.924067 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-55b4d954d5-bqwss"] Jul 06 19:00:39.926352 ip-10-0-130-59 kubenswrapper[2570]: W0706 19:00:39.926327 2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod22d32b1c_3352_4d8d_b0ad_c24c95a4ac5b.slice/crio-2e091472113b75136aa958e2af95ec647a560af46eab764358f06ac9584119d9 WatchSource:0}: Error finding container 2e091472113b75136aa958e2af95ec647a560af46eab764358f06ac9584119d9: Status 404 returned error can't find the container with id 2e091472113b75136aa958e2af95ec647a560af46eab764358f06ac9584119d9 Jul 06 19:00:40.504350 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:00:40.504320 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-55b4d954d5-bqwss" event={"ID":"22d32b1c-3352-4d8d-b0ad-c24c95a4ac5b","Type":"ContainerStarted","Data":"a7959d7985f98677b7f1cb23c1cbaecb5493c41885d94757bad18970a58ebc92"} Jul 06 19:00:40.504751 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:00:40.504359 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-55b4d954d5-bqwss" event={"ID":"22d32b1c-3352-4d8d-b0ad-c24c95a4ac5b","Type":"ContainerStarted","Data":"2e091472113b75136aa958e2af95ec647a560af46eab764358f06ac9584119d9"} Jul 06 19:00:40.530443 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:00:40.530388 2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager/cert-manager-55b4d954d5-bqwss" podStartSLOduration=1.530369876 podStartE2EDuration="1.530369876s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 19:00:39 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 19:00:40.528888612 +0000 UTC m=+320.083909783" watchObservedRunningTime="2026-07-06 19:00:40.530369876 +0000 UTC m=+320.085391047" Jul 06 19:00:42.005104 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:00:42.005076 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k8fbq_bc7057bd-88da-48af-ba32-e6efdc6e8362/konnectivity-agent/0.log" limit=1024 Jul 06 19:00:42.100777 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:00:42.100748 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k8fbq_bc7057bd-88da-48af-ba32-e6efdc6e8362/konnectivity-agent/0.log" limit=1024 Jul 06 19:00:44.488093 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:00:44.488059 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k8fbq_bc7057bd-88da-48af-ba32-e6efdc6e8362/konnectivity-agent/0.log" limit=1024 Jul 06 19:00:44.629219 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:00:44.629182 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k8fbq_bc7057bd-88da-48af-ba32-e6efdc6e8362/konnectivity-agent/0.log" limit=1024 Jul 06 19:00:59.798502 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:00:59.798451 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k8fbq_bc7057bd-88da-48af-ba32-e6efdc6e8362/konnectivity-agent/0.log" limit=1024 Jul 06 19:00:59.985097 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:00:59.985033 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k8fbq_bc7057bd-88da-48af-ba32-e6efdc6e8362/konnectivity-agent/0.log" limit=1024 Jul 06 19:01:00.359522 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:01:00.359481 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k8fbq_bc7057bd-88da-48af-ba32-e6efdc6e8362/konnectivity-agent/0.log" limit=1024 Jul 06 19:01:01.062369 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:01:01.062315 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k8fbq_bc7057bd-88da-48af-ba32-e6efdc6e8362/konnectivity-agent/0.log" limit=1024 Jul 06 19:01:01.194175 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:01:01.194135 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k8fbq_bc7057bd-88da-48af-ba32-e6efdc6e8362/konnectivity-agent/0.log" limit=1024 Jul 06 19:01:01.404401 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:01:01.404312 2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-546c6c6d78-g8lh9"] Jul 06 19:01:01.407728 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:01:01.407701 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-546c6c6d78-g8lh9" Jul 06 19:01:01.419971 ip-10-0-130-59 kubenswrapper[2570]: E0706 19:01:01.419944 2570 reflector.go:205] "Failed to watch" err="failed to list *v1.Secret: secrets \"opendatahub-operator-controller-webhook-cert\" is forbidden: User \"system:node:ip-10-0-130-59.ec2.internal\" cannot list resource \"secrets\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-130-59.ec2.internal' and this object" logger="UnhandledError" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-webhook-cert\"" type="*v1.Secret" Jul 06 19:01:01.419971 ip-10-0-130-59 kubenswrapper[2570]: E0706 19:01:01.419945 2570 reflector.go:205] "Failed to watch" err="failed to list *v1.Secret: secrets \"opendatahub-operator-controller-manager-service-cert\" is forbidden: User \"system:node:ip-10-0-130-59.ec2.internal\" cannot list resource \"secrets\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-130-59.ec2.internal' and this object" logger="UnhandledError" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-service-cert\"" type="*v1.Secret" Jul 06 19:01:01.420134 ip-10-0-130-59 kubenswrapper[2570]: E0706 19:01:01.420009 2570 status_manager.go:1018] "Failed to get status for pod" err="pods \"opendatahub-operator-controller-manager-546c6c6d78-g8lh9\" is forbidden: User \"system:node:ip-10-0-130-59.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-130-59.ec2.internal' and this object" podUID="583dec04-8578-41f5-92c2-b716d8a86b76" pod="opendatahub/opendatahub-operator-controller-manager-546c6c6d78-g8lh9" Jul 06 19:01:01.420476 ip-10-0-130-59 kubenswrapper[2570]: E0706 19:01:01.420454 2570 reflector.go:205] "Failed to watch" err="failed to list *v1.ConfigMap: configmaps \"kube-root-ca.crt\" is forbidden: User \"system:node:ip-10-0-130-59.ec2.internal\" cannot list resource \"configmaps\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-130-59.ec2.internal' and this object" logger="UnhandledError" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\"" type="*v1.ConfigMap" Jul 06 19:01:01.420655 ip-10-0-130-59 kubenswrapper[2570]: E0706 19:01:01.420637 2570 reflector.go:205] "Failed to watch" err="failed to list *v1.Secret: secrets \"opendatahub-operator-controller-manager-dockercfg-65wfm\" is forbidden: User \"system:node:ip-10-0-130-59.ec2.internal\" cannot list resource \"secrets\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-130-59.ec2.internal' and this object" logger="UnhandledError" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-dockercfg-65wfm\"" type="*v1.Secret" Jul 06 19:01:01.428262 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:01:01.428237 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/583dec04-8578-41f5-92c2-b716d8a86b76-webhook-cert\") pod \"opendatahub-operator-controller-manager-546c6c6d78-g8lh9\" (UID: \"583dec04-8578-41f5-92c2-b716d8a86b76\") " pod="opendatahub/opendatahub-operator-controller-manager-546c6c6d78-g8lh9" Jul 06 19:01:01.428411 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:01:01.428284 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zfbrb\" (UniqueName: \"kubernetes.io/projected/583dec04-8578-41f5-92c2-b716d8a86b76-kube-api-access-zfbrb\") pod \"opendatahub-operator-controller-manager-546c6c6d78-g8lh9\" (UID: \"583dec04-8578-41f5-92c2-b716d8a86b76\") " pod="opendatahub/opendatahub-operator-controller-manager-546c6c6d78-g8lh9" Jul 06 19:01:01.428411 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:01:01.428327 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/583dec04-8578-41f5-92c2-b716d8a86b76-manifests\") pod \"opendatahub-operator-controller-manager-546c6c6d78-g8lh9\" (UID: \"583dec04-8578-41f5-92c2-b716d8a86b76\") " pod="opendatahub/opendatahub-operator-controller-manager-546c6c6d78-g8lh9" Jul 06 19:01:01.428411 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:01:01.428399 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/583dec04-8578-41f5-92c2-b716d8a86b76-tmp\") pod \"opendatahub-operator-controller-manager-546c6c6d78-g8lh9\" (UID: \"583dec04-8578-41f5-92c2-b716d8a86b76\") " pod="opendatahub/opendatahub-operator-controller-manager-546c6c6d78-g8lh9" Jul 06 19:01:01.428578 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:01:01.428462 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/583dec04-8578-41f5-92c2-b716d8a86b76-apiservice-cert\") pod \"opendatahub-operator-controller-manager-546c6c6d78-g8lh9\" (UID: \"583dec04-8578-41f5-92c2-b716d8a86b76\") " pod="opendatahub/opendatahub-operator-controller-manager-546c6c6d78-g8lh9" Jul 06 19:01:01.436186 ip-10-0-130-59 kubenswrapper[2570]: E0706 19:01:01.436158 2570 reflector.go:205] "Failed to watch" err="failed to list *v1.ConfigMap: configmaps \"openshift-service-ca.crt\" is forbidden: User \"system:node:ip-10-0-130-59.ec2.internal\" cannot list resource \"configmaps\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-130-59.ec2.internal' and this object" logger="UnhandledError" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\"" type="*v1.ConfigMap" Jul 06 19:01:01.529677 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:01:01.529641 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/583dec04-8578-41f5-92c2-b716d8a86b76-webhook-cert\") pod \"opendatahub-operator-controller-manager-546c6c6d78-g8lh9\" (UID: \"583dec04-8578-41f5-92c2-b716d8a86b76\") " pod="opendatahub/opendatahub-operator-controller-manager-546c6c6d78-g8lh9" Jul 06 19:01:01.529880 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:01:01.529688 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zfbrb\" (UniqueName: \"kubernetes.io/projected/583dec04-8578-41f5-92c2-b716d8a86b76-kube-api-access-zfbrb\") pod \"opendatahub-operator-controller-manager-546c6c6d78-g8lh9\" (UID: \"583dec04-8578-41f5-92c2-b716d8a86b76\") " pod="opendatahub/opendatahub-operator-controller-manager-546c6c6d78-g8lh9" Jul 06 19:01:01.529880 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:01:01.529719 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/583dec04-8578-41f5-92c2-b716d8a86b76-manifests\") pod \"opendatahub-operator-controller-manager-546c6c6d78-g8lh9\" (UID: \"583dec04-8578-41f5-92c2-b716d8a86b76\") " pod="opendatahub/opendatahub-operator-controller-manager-546c6c6d78-g8lh9" Jul 06 19:01:01.529880 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:01:01.529754 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/583dec04-8578-41f5-92c2-b716d8a86b76-tmp\") pod \"opendatahub-operator-controller-manager-546c6c6d78-g8lh9\" (UID: \"583dec04-8578-41f5-92c2-b716d8a86b76\") " pod="opendatahub/opendatahub-operator-controller-manager-546c6c6d78-g8lh9" Jul 06 19:01:01.529880 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:01:01.529802 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/583dec04-8578-41f5-92c2-b716d8a86b76-apiservice-cert\") pod \"opendatahub-operator-controller-manager-546c6c6d78-g8lh9\" (UID: \"583dec04-8578-41f5-92c2-b716d8a86b76\") " pod="opendatahub/opendatahub-operator-controller-manager-546c6c6d78-g8lh9" Jul 06 19:01:01.530125 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:01:01.530105 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/583dec04-8578-41f5-92c2-b716d8a86b76-manifests\") pod \"opendatahub-operator-controller-manager-546c6c6d78-g8lh9\" (UID: \"583dec04-8578-41f5-92c2-b716d8a86b76\") " pod="opendatahub/opendatahub-operator-controller-manager-546c6c6d78-g8lh9" Jul 06 19:01:01.531979 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:01:01.531952 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/583dec04-8578-41f5-92c2-b716d8a86b76-tmp\") pod \"opendatahub-operator-controller-manager-546c6c6d78-g8lh9\" (UID: \"583dec04-8578-41f5-92c2-b716d8a86b76\") " pod="opendatahub/opendatahub-operator-controller-manager-546c6c6d78-g8lh9" Jul 06 19:01:01.563378 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:01:01.563344 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-546c6c6d78-g8lh9"] Jul 06 19:01:01.844308 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:01:01.844276 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k8fbq_bc7057bd-88da-48af-ba32-e6efdc6e8362/konnectivity-agent/0.log" limit=1024 Jul 06 19:01:01.973335 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:01:01.973304 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k8fbq_bc7057bd-88da-48af-ba32-e6efdc6e8362/konnectivity-agent/0.log" limit=1024 Jul 06 19:01:02.100870 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:01:02.100787 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k8fbq_bc7057bd-88da-48af-ba32-e6efdc6e8362/konnectivity-agent/0.log" limit=1024 Jul 06 19:01:02.196123 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:01:02.196078 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k8fbq_bc7057bd-88da-48af-ba32-e6efdc6e8362/konnectivity-agent/0.log" limit=1024 Jul 06 19:01:02.291069 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:01:02.291035 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\"" Jul 06 19:01:02.306204 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:01:02.306169 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k8fbq_bc7057bd-88da-48af-ba32-e6efdc6e8362/konnectivity-agent/0.log" limit=1024 Jul 06 19:01:02.422806 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:01:02.422780 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k8fbq_bc7057bd-88da-48af-ba32-e6efdc6e8362/konnectivity-agent/0.log" limit=1024 Jul 06 19:01:02.519520 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:01:02.519484 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-webhook-cert\"" Jul 06 19:01:02.530750 ip-10-0-130-59 kubenswrapper[2570]: E0706 19:01:02.530715 2570 secret.go:189] Couldn't get secret opendatahub/opendatahub-operator-controller-manager-service-cert: failed to sync secret cache: timed out waiting for the condition Jul 06 19:01:02.530943 ip-10-0-130-59 kubenswrapper[2570]: E0706 19:01:02.530721 2570 secret.go:189] Couldn't get secret opendatahub/opendatahub-operator-controller-manager-service-cert: failed to sync secret cache: timed out waiting for the condition Jul 06 19:01:02.530943 ip-10-0-130-59 kubenswrapper[2570]: E0706 19:01:02.530817 2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/583dec04-8578-41f5-92c2-b716d8a86b76-apiservice-cert podName:583dec04-8578-41f5-92c2-b716d8a86b76 nodeName:}" failed. No retries permitted until 2026-07-06 19:01:03.030790265 +0000 UTC m=+342.585811436 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "apiservice-cert" (UniqueName: "kubernetes.io/secret/583dec04-8578-41f5-92c2-b716d8a86b76-apiservice-cert") pod "opendatahub-operator-controller-manager-546c6c6d78-g8lh9" (UID: "583dec04-8578-41f5-92c2-b716d8a86b76") : failed to sync secret cache: timed out waiting for the condition Jul 06 19:01:02.530943 ip-10-0-130-59 kubenswrapper[2570]: E0706 19:01:02.530894 2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/583dec04-8578-41f5-92c2-b716d8a86b76-webhook-cert podName:583dec04-8578-41f5-92c2-b716d8a86b76 nodeName:}" failed. No retries permitted until 2026-07-06 19:01:03.030878328 +0000 UTC m=+342.585899478 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "webhook-cert" (UniqueName: "kubernetes.io/secret/583dec04-8578-41f5-92c2-b716d8a86b76-webhook-cert") pod "opendatahub-operator-controller-manager-546c6c6d78-g8lh9" (UID: "583dec04-8578-41f5-92c2-b716d8a86b76") : failed to sync secret cache: timed out waiting for the condition Jul 06 19:01:02.833550 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:01:02.833467 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-service-cert\"" Jul 06 19:01:02.910377 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:01:02.910345 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\"" Jul 06 19:01:02.918048 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:01:02.918020 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-dockercfg-65wfm\"" Jul 06 19:01:02.918962 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:01:02.918941 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-zfbrb\" (UniqueName: \"kubernetes.io/projected/583dec04-8578-41f5-92c2-b716d8a86b76-kube-api-access-zfbrb\") pod \"opendatahub-operator-controller-manager-546c6c6d78-g8lh9\" (UID: \"583dec04-8578-41f5-92c2-b716d8a86b76\") " pod="opendatahub/opendatahub-operator-controller-manager-546c6c6d78-g8lh9" Jul 06 19:01:03.042165 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:01:03.042114 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/583dec04-8578-41f5-92c2-b716d8a86b76-apiservice-cert\") pod \"opendatahub-operator-controller-manager-546c6c6d78-g8lh9\" (UID: \"583dec04-8578-41f5-92c2-b716d8a86b76\") " pod="opendatahub/opendatahub-operator-controller-manager-546c6c6d78-g8lh9" Jul 06 19:01:03.042165 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:01:03.042169 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/583dec04-8578-41f5-92c2-b716d8a86b76-webhook-cert\") pod \"opendatahub-operator-controller-manager-546c6c6d78-g8lh9\" (UID: \"583dec04-8578-41f5-92c2-b716d8a86b76\") " pod="opendatahub/opendatahub-operator-controller-manager-546c6c6d78-g8lh9" Jul 06 19:01:03.044545 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:01:03.044518 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/583dec04-8578-41f5-92c2-b716d8a86b76-webhook-cert\") pod \"opendatahub-operator-controller-manager-546c6c6d78-g8lh9\" (UID: \"583dec04-8578-41f5-92c2-b716d8a86b76\") " pod="opendatahub/opendatahub-operator-controller-manager-546c6c6d78-g8lh9" Jul 06 19:01:03.044694 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:01:03.044552 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/583dec04-8578-41f5-92c2-b716d8a86b76-apiservice-cert\") pod \"opendatahub-operator-controller-manager-546c6c6d78-g8lh9\" (UID: \"583dec04-8578-41f5-92c2-b716d8a86b76\") " pod="opendatahub/opendatahub-operator-controller-manager-546c6c6d78-g8lh9" Jul 06 19:01:03.218346 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:01:03.218309 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-546c6c6d78-g8lh9" Jul 06 19:01:03.359210 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:01:03.359187 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-546c6c6d78-g8lh9"] Jul 06 19:01:03.361701 ip-10-0-130-59 kubenswrapper[2570]: W0706 19:01:03.361671 2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod583dec04_8578_41f5_92c2_b716d8a86b76.slice/crio-a9bdd996e3496ce7475559dc60b7925721b9d60447da63ce78ba355839b4a2bd WatchSource:0}: Error finding container a9bdd996e3496ce7475559dc60b7925721b9d60447da63ce78ba355839b4a2bd: Status 404 returned error can't find the container with id a9bdd996e3496ce7475559dc60b7925721b9d60447da63ce78ba355839b4a2bd Jul 06 19:01:03.598801 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:01:03.598715 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-546c6c6d78-g8lh9" event={"ID":"583dec04-8578-41f5-92c2-b716d8a86b76","Type":"ContainerStarted","Data":"a9bdd996e3496ce7475559dc60b7925721b9d60447da63ce78ba355839b4a2bd"} Jul 06 19:01:04.765690 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:01:04.765647 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k8fbq_bc7057bd-88da-48af-ba32-e6efdc6e8362/konnectivity-agent/0.log" limit=1024 Jul 06 19:01:04.952031 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:01:04.951952 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k8fbq_bc7057bd-88da-48af-ba32-e6efdc6e8362/konnectivity-agent/0.log" limit=1024 Jul 06 19:01:06.614897 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:01:06.614809 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-546c6c6d78-g8lh9" event={"ID":"583dec04-8578-41f5-92c2-b716d8a86b76","Type":"ContainerStarted","Data":"8261612e371db60042c57fe8a15b0d3d6dc007ceb0072fc35331c3574f1c7bbe"} Jul 06 19:01:07.620993 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:01:07.620903 2570 generic.go:356] "Generic (PLEG): container finished" podID="583dec04-8578-41f5-92c2-b716d8a86b76" containerID="8261612e371db60042c57fe8a15b0d3d6dc007ceb0072fc35331c3574f1c7bbe" exitCode=0 Jul 06 19:01:07.620993 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:01:07.620950 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-546c6c6d78-g8lh9" event={"ID":"583dec04-8578-41f5-92c2-b716d8a86b76","Type":"ContainerDied","Data":"8261612e371db60042c57fe8a15b0d3d6dc007ceb0072fc35331c3574f1c7bbe"} Jul 06 19:01:08.627116 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:01:08.627086 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-546c6c6d78-g8lh9" event={"ID":"583dec04-8578-41f5-92c2-b716d8a86b76","Type":"ContainerStarted","Data":"807247fdcb802ecfafa0f4a732b078e203ff67ffbab3a93491d39f13104bf9b4"} Jul 06 19:01:08.627506 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:01:08.627210 2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/opendatahub-operator-controller-manager-546c6c6d78-g8lh9" Jul 06 19:01:08.658387 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:01:08.658335 2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/opendatahub-operator-controller-manager-546c6c6d78-g8lh9" podStartSLOduration=3.417807131 podStartE2EDuration="7.658316136s" totalImagesPullingTime="3.238091522s" totalInitContainerRuntime="1.002417483s" isStatefulPod=false podCreationTimestamp="2026-07-06 19:01:01 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 19:01:08.655944479 +0000 UTC m=+348.210965673" watchObservedRunningTime="2026-07-06 19:01:08.658316136 +0000 UTC m=+348.213337308" Jul 06 19:01:18.161570 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:01:18.161499 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k8fbq_bc7057bd-88da-48af-ba32-e6efdc6e8362/konnectivity-agent/0.log" limit=1024 Jul 06 19:01:18.266712 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:01:18.266680 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k8fbq_bc7057bd-88da-48af-ba32-e6efdc6e8362/konnectivity-agent/0.log" limit=1024 Jul 06 19:01:18.589246 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:01:18.589212 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k8fbq_bc7057bd-88da-48af-ba32-e6efdc6e8362/konnectivity-agent/0.log" limit=1024 Jul 06 19:01:18.690144 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:01:18.690111 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k8fbq_bc7057bd-88da-48af-ba32-e6efdc6e8362/konnectivity-agent/0.log" limit=1024 Jul 06 19:01:19.633788 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:01:19.633753 2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/opendatahub-operator-controller-manager-546c6c6d78-g8lh9" Jul 06 19:01:19.943768 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:01:19.943734 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k8fbq_bc7057bd-88da-48af-ba32-e6efdc6e8362/konnectivity-agent/0.log" limit=1024 Jul 06 19:01:21.595740 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:01:21.595704 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k8fbq_bc7057bd-88da-48af-ba32-e6efdc6e8362/konnectivity-agent/0.log" limit=1024 Jul 06 19:01:21.898131 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:01:21.898056 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k8fbq_bc7057bd-88da-48af-ba32-e6efdc6e8362/konnectivity-agent/0.log" limit=1024 Jul 06 19:01:22.383643 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:01:22.383591 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k8fbq_bc7057bd-88da-48af-ba32-e6efdc6e8362/konnectivity-agent/0.log" limit=1024 Jul 06 19:01:22.547094 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:01:22.547061 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k8fbq_bc7057bd-88da-48af-ba32-e6efdc6e8362/konnectivity-agent/0.log" limit=1024 Jul 06 19:01:22.811461 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:01:22.811430 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k8fbq_bc7057bd-88da-48af-ba32-e6efdc6e8362/konnectivity-agent/0.log" limit=1024 Jul 06 19:01:22.981053 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:01:22.981024 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k8fbq_bc7057bd-88da-48af-ba32-e6efdc6e8362/konnectivity-agent/0.log" limit=1024 Jul 06 19:01:23.077568 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:01:23.077465 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k8fbq_bc7057bd-88da-48af-ba32-e6efdc6e8362/konnectivity-agent/0.log" limit=1024 Jul 06 19:01:23.308790 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:01:23.308759 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k8fbq_bc7057bd-88da-48af-ba32-e6efdc6e8362/konnectivity-agent/0.log" limit=1024 Jul 06 19:01:23.446759 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:01:23.446727 2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/model-serving-api-54d89bd5fc-4f7jh"] Jul 06 19:01:23.450499 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:01:23.450476 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/model-serving-api-54d89bd5fc-4f7jh" Jul 06 19:01:23.452671 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:01:23.452648 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"model-serving-api-tls\"" Jul 06 19:01:23.478444 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:01:23.478415 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/model-serving-api-54d89bd5fc-4f7jh"] Jul 06 19:01:23.500973 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:01:23.500940 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k8fbq_bc7057bd-88da-48af-ba32-e6efdc6e8362/konnectivity-agent/0.log" limit=1024 Jul 06 19:01:23.530926 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:01:23.530891 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/bc0ca806-1da4-4fd0-a454-451382a07ddb-tls-certs\") pod \"model-serving-api-54d89bd5fc-4f7jh\" (UID: \"bc0ca806-1da4-4fd0-a454-451382a07ddb\") " pod="opendatahub/model-serving-api-54d89bd5fc-4f7jh" Jul 06 19:01:23.531107 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:01:23.530976 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2w7z9\" (UniqueName: \"kubernetes.io/projected/bc0ca806-1da4-4fd0-a454-451382a07ddb-kube-api-access-2w7z9\") pod \"model-serving-api-54d89bd5fc-4f7jh\" (UID: \"bc0ca806-1da4-4fd0-a454-451382a07ddb\") " pod="opendatahub/model-serving-api-54d89bd5fc-4f7jh" Jul 06 19:01:23.632417 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:01:23.632380 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/bc0ca806-1da4-4fd0-a454-451382a07ddb-tls-certs\") pod \"model-serving-api-54d89bd5fc-4f7jh\" (UID: \"bc0ca806-1da4-4fd0-a454-451382a07ddb\") " pod="opendatahub/model-serving-api-54d89bd5fc-4f7jh" Jul 06 19:01:23.632592 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:01:23.632434 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2w7z9\" (UniqueName: \"kubernetes.io/projected/bc0ca806-1da4-4fd0-a454-451382a07ddb-kube-api-access-2w7z9\") pod \"model-serving-api-54d89bd5fc-4f7jh\" (UID: \"bc0ca806-1da4-4fd0-a454-451382a07ddb\") " pod="opendatahub/model-serving-api-54d89bd5fc-4f7jh" Jul 06 19:01:23.632592 ip-10-0-130-59 kubenswrapper[2570]: E0706 19:01:23.632533 2570 secret.go:189] Couldn't get secret opendatahub/model-serving-api-tls: secret "model-serving-api-tls" not found Jul 06 19:01:23.632707 ip-10-0-130-59 kubenswrapper[2570]: E0706 19:01:23.632595 2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/bc0ca806-1da4-4fd0-a454-451382a07ddb-tls-certs podName:bc0ca806-1da4-4fd0-a454-451382a07ddb nodeName:}" failed. No retries permitted until 2026-07-06 19:01:24.132579068 +0000 UTC m=+363.687600218 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "tls-certs" (UniqueName: "kubernetes.io/secret/bc0ca806-1da4-4fd0-a454-451382a07ddb-tls-certs") pod "model-serving-api-54d89bd5fc-4f7jh" (UID: "bc0ca806-1da4-4fd0-a454-451382a07ddb") : secret "model-serving-api-tls" not found Jul 06 19:01:23.652300 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:01:23.647497 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-2w7z9\" (UniqueName: \"kubernetes.io/projected/bc0ca806-1da4-4fd0-a454-451382a07ddb-kube-api-access-2w7z9\") pod \"model-serving-api-54d89bd5fc-4f7jh\" (UID: \"bc0ca806-1da4-4fd0-a454-451382a07ddb\") " pod="opendatahub/model-serving-api-54d89bd5fc-4f7jh" Jul 06 19:01:23.923718 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:01:23.923683 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k8fbq_bc7057bd-88da-48af-ba32-e6efdc6e8362/konnectivity-agent/0.log" limit=1024 Jul 06 19:01:24.136428 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:01:24.136383 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/bc0ca806-1da4-4fd0-a454-451382a07ddb-tls-certs\") pod \"model-serving-api-54d89bd5fc-4f7jh\" (UID: \"bc0ca806-1da4-4fd0-a454-451382a07ddb\") " pod="opendatahub/model-serving-api-54d89bd5fc-4f7jh" Jul 06 19:01:24.138813 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:01:24.138793 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/bc0ca806-1da4-4fd0-a454-451382a07ddb-tls-certs\") pod \"model-serving-api-54d89bd5fc-4f7jh\" (UID: \"bc0ca806-1da4-4fd0-a454-451382a07ddb\") " pod="opendatahub/model-serving-api-54d89bd5fc-4f7jh" Jul 06 19:01:24.366089 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:01:24.365999 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/model-serving-api-54d89bd5fc-4f7jh" Jul 06 19:01:24.503140 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:01:24.502991 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/model-serving-api-54d89bd5fc-4f7jh"] Jul 06 19:01:24.505693 ip-10-0-130-59 kubenswrapper[2570]: W0706 19:01:24.505660 2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbc0ca806_1da4_4fd0_a454_451382a07ddb.slice/crio-40b86df6ad45e10207f793f5a6df5c91920df1983d5c4b0cbde674b66919f588 WatchSource:0}: Error finding container 40b86df6ad45e10207f793f5a6df5c91920df1983d5c4b0cbde674b66919f588: Status 404 returned error can't find the container with id 40b86df6ad45e10207f793f5a6df5c91920df1983d5c4b0cbde674b66919f588 Jul 06 19:01:24.700006 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:01:24.699976 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/model-serving-api-54d89bd5fc-4f7jh" event={"ID":"bc0ca806-1da4-4fd0-a454-451382a07ddb","Type":"ContainerStarted","Data":"40b86df6ad45e10207f793f5a6df5c91920df1983d5c4b0cbde674b66919f588"} Jul 06 19:01:26.711179 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:01:26.711154 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/model-serving-api-54d89bd5fc-4f7jh" event={"ID":"bc0ca806-1da4-4fd0-a454-451382a07ddb","Type":"ContainerStarted","Data":"2d42e12a7ccabb5943616a1efe4ca1748003d169dd1a9fb3eaa706b0cda8fd48"} Jul 06 19:01:26.711548 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:01:26.711228 2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/model-serving-api-54d89bd5fc-4f7jh" Jul 06 19:01:26.731267 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:01:26.731207 2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/model-serving-api-54d89bd5fc-4f7jh" podStartSLOduration=1.5852488249999999 podStartE2EDuration="3.731187245s" totalImagesPullingTime="2.14593842s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 19:01:23 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 19:01:26.730358287 +0000 UTC m=+366.285379484" watchObservedRunningTime="2026-07-06 19:01:26.731187245 +0000 UTC m=+366.286208418" Jul 06 19:01:28.276900 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:01:28.276865 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k8fbq_bc7057bd-88da-48af-ba32-e6efdc6e8362/konnectivity-agent/0.log" limit=1024 Jul 06 19:01:28.428443 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:01:28.428413 2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/kube-auth-proxy-6c488d768d-s2rqp"] Jul 06 19:01:28.431086 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:01:28.431069 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-6c488d768d-s2rqp" Jul 06 19:01:28.435343 ip-10-0-130-59 kubenswrapper[2570]: E0706 19:01:28.435308 2570 status_manager.go:1018] "Failed to get status for pod" err="pods \"kube-auth-proxy-6c488d768d-s2rqp\" is forbidden: User \"system:node:ip-10-0-130-59.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"openshift-ingress\": no relationship found between node 'ip-10-0-130-59.ec2.internal' and this object" podUID="8daf052f-a13f-4a56-9b8d-36c12e50d1b4" pod="openshift-ingress/kube-auth-proxy-6c488d768d-s2rqp" Jul 06 19:01:28.435343 ip-10-0-130-59 kubenswrapper[2570]: E0706 19:01:28.435327 2570 reflector.go:205] "Failed to watch" err="failed to list *v1.Secret: secrets \"kube-auth-proxy-dockercfg-j5v6g\" is forbidden: User \"system:node:ip-10-0-130-59.ec2.internal\" cannot list resource \"secrets\" in API group \"\" in the namespace \"openshift-ingress\": no relationship found between node 'ip-10-0-130-59.ec2.internal' and this object" logger="UnhandledError" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-dockercfg-j5v6g\"" type="*v1.Secret" Jul 06 19:01:28.435529 ip-10-0-130-59 kubenswrapper[2570]: E0706 19:01:28.435346 2570 reflector.go:205] "Failed to watch" err="failed to list *v1.Secret: secrets \"kube-auth-proxy-creds\" is forbidden: User \"system:node:ip-10-0-130-59.ec2.internal\" cannot list resource \"secrets\" in API group \"\" in the namespace \"openshift-ingress\": no relationship found between node 'ip-10-0-130-59.ec2.internal' and this object" logger="UnhandledError" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-creds\"" type="*v1.Secret" Jul 06 19:01:28.435529 ip-10-0-130-59 kubenswrapper[2570]: E0706 19:01:28.435349 2570 reflector.go:205] "Failed to watch" err="failed to list *v1.Secret: secrets \"kube-auth-proxy-tls\" is forbidden: User \"system:node:ip-10-0-130-59.ec2.internal\" cannot list resource \"secrets\" in API group \"\" in the namespace \"openshift-ingress\": no relationship found between node 'ip-10-0-130-59.ec2.internal' and this object" logger="UnhandledError" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-tls\"" type="*v1.Secret" Jul 06 19:01:28.436494 ip-10-0-130-59 kubenswrapper[2570]: E0706 19:01:28.436471 2570 reflector.go:205] "Failed to watch" err="failed to list *v1.ConfigMap: configmaps \"openshift-service-ca.crt\" is forbidden: User \"system:node:ip-10-0-130-59.ec2.internal\" cannot list resource \"configmaps\" in API group \"\" in the namespace \"openshift-ingress\": no relationship found between node 'ip-10-0-130-59.ec2.internal' and this object" logger="UnhandledError" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\"" type="*v1.ConfigMap" Jul 06 19:01:28.438170 ip-10-0-130-59 kubenswrapper[2570]: E0706 19:01:28.438142 2570 reflector.go:205] "Failed to watch" err="failed to list *v1.ConfigMap: configmaps \"kube-root-ca.crt\" is forbidden: User \"system:node:ip-10-0-130-59.ec2.internal\" cannot list resource \"configmaps\" in API group \"\" in the namespace \"openshift-ingress\": no relationship found between node 'ip-10-0-130-59.ec2.internal' and this object" logger="UnhandledError" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\"" type="*v1.ConfigMap" Jul 06 19:01:28.468563 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:01:28.468528 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-6c488d768d-s2rqp"] Jul 06 19:01:28.576915 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:01:28.576836 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/8daf052f-a13f-4a56-9b8d-36c12e50d1b4-tls-certs\") pod \"kube-auth-proxy-6c488d768d-s2rqp\" (UID: \"8daf052f-a13f-4a56-9b8d-36c12e50d1b4\") " pod="openshift-ingress/kube-auth-proxy-6c488d768d-s2rqp" Jul 06 19:01:28.576915 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:01:28.576868 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tnvl2\" (UniqueName: \"kubernetes.io/projected/8daf052f-a13f-4a56-9b8d-36c12e50d1b4-kube-api-access-tnvl2\") pod \"kube-auth-proxy-6c488d768d-s2rqp\" (UID: \"8daf052f-a13f-4a56-9b8d-36c12e50d1b4\") " pod="openshift-ingress/kube-auth-proxy-6c488d768d-s2rqp" Jul 06 19:01:28.576915 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:01:28.576902 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/8daf052f-a13f-4a56-9b8d-36c12e50d1b4-tmp\") pod \"kube-auth-proxy-6c488d768d-s2rqp\" (UID: \"8daf052f-a13f-4a56-9b8d-36c12e50d1b4\") " pod="openshift-ingress/kube-auth-proxy-6c488d768d-s2rqp" Jul 06 19:01:28.638677 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:01:28.638640 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k8fbq_bc7057bd-88da-48af-ba32-e6efdc6e8362/konnectivity-agent/0.log" limit=1024 Jul 06 19:01:28.677683 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:01:28.677648 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/8daf052f-a13f-4a56-9b8d-36c12e50d1b4-tls-certs\") pod \"kube-auth-proxy-6c488d768d-s2rqp\" (UID: \"8daf052f-a13f-4a56-9b8d-36c12e50d1b4\") " pod="openshift-ingress/kube-auth-proxy-6c488d768d-s2rqp" Jul 06 19:01:28.677815 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:01:28.677696 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tnvl2\" (UniqueName: \"kubernetes.io/projected/8daf052f-a13f-4a56-9b8d-36c12e50d1b4-kube-api-access-tnvl2\") pod \"kube-auth-proxy-6c488d768d-s2rqp\" (UID: \"8daf052f-a13f-4a56-9b8d-36c12e50d1b4\") " pod="openshift-ingress/kube-auth-proxy-6c488d768d-s2rqp" Jul 06 19:01:28.677815 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:01:28.677759 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/8daf052f-a13f-4a56-9b8d-36c12e50d1b4-tmp\") pod \"kube-auth-proxy-6c488d768d-s2rqp\" (UID: \"8daf052f-a13f-4a56-9b8d-36c12e50d1b4\") " pod="openshift-ingress/kube-auth-proxy-6c488d768d-s2rqp" Jul 06 19:01:28.679987 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:01:28.679965 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/8daf052f-a13f-4a56-9b8d-36c12e50d1b4-tmp\") pod \"kube-auth-proxy-6c488d768d-s2rqp\" (UID: \"8daf052f-a13f-4a56-9b8d-36c12e50d1b4\") " pod="openshift-ingress/kube-auth-proxy-6c488d768d-s2rqp" Jul 06 19:01:28.731777 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:01:28.731744 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k8fbq_bc7057bd-88da-48af-ba32-e6efdc6e8362/konnectivity-agent/0.log" limit=1024 Jul 06 19:01:29.519226 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:01:29.519189 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-dockercfg-j5v6g\"" Jul 06 19:01:29.620884 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:01:29.620853 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-creds\"" Jul 06 19:01:29.678768 ip-10-0-130-59 kubenswrapper[2570]: E0706 19:01:29.678374 2570 secret.go:189] Couldn't get secret openshift-ingress/kube-auth-proxy-tls: failed to sync secret cache: timed out waiting for the condition Jul 06 19:01:29.678768 ip-10-0-130-59 kubenswrapper[2570]: E0706 19:01:29.678464 2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8daf052f-a13f-4a56-9b8d-36c12e50d1b4-tls-certs podName:8daf052f-a13f-4a56-9b8d-36c12e50d1b4 nodeName:}" failed. No retries permitted until 2026-07-06 19:01:30.1784421 +0000 UTC m=+369.733463265 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "tls-certs" (UniqueName: "kubernetes.io/secret/8daf052f-a13f-4a56-9b8d-36c12e50d1b4-tls-certs") pod "kube-auth-proxy-6c488d768d-s2rqp" (UID: "8daf052f-a13f-4a56-9b8d-36c12e50d1b4") : failed to sync secret cache: timed out waiting for the condition Jul 06 19:01:29.684390 ip-10-0-130-59 kubenswrapper[2570]: E0706 19:01:29.684292 2570 projected.go:291] Couldn't get configMap openshift-ingress/kube-root-ca.crt: failed to sync configmap cache: timed out waiting for the condition Jul 06 19:01:29.783756 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:01:29.783674 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-tls\"" Jul 06 19:01:29.786379 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:01:29.786361 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\"" Jul 06 19:01:29.890156 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:01:29.890122 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\"" Jul 06 19:01:29.895428 ip-10-0-130-59 kubenswrapper[2570]: E0706 19:01:29.895411 2570 projected.go:196] Error preparing data for projected volume kube-api-access-tnvl2 for pod openshift-ingress/kube-auth-proxy-6c488d768d-s2rqp: failed to sync configmap cache: timed out waiting for the condition Jul 06 19:01:29.895497 ip-10-0-130-59 kubenswrapper[2570]: E0706 19:01:29.895469 2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/8daf052f-a13f-4a56-9b8d-36c12e50d1b4-kube-api-access-tnvl2 podName:8daf052f-a13f-4a56-9b8d-36c12e50d1b4 nodeName:}" failed. No retries permitted until 2026-07-06 19:01:30.395453582 +0000 UTC m=+369.950474732 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-tnvl2" (UniqueName: "kubernetes.io/projected/8daf052f-a13f-4a56-9b8d-36c12e50d1b4-kube-api-access-tnvl2") pod "kube-auth-proxy-6c488d768d-s2rqp" (UID: "8daf052f-a13f-4a56-9b8d-36c12e50d1b4") : failed to sync configmap cache: timed out waiting for the condition Jul 06 19:01:30.190850 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:01:30.190814 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/8daf052f-a13f-4a56-9b8d-36c12e50d1b4-tls-certs\") pod \"kube-auth-proxy-6c488d768d-s2rqp\" (UID: \"8daf052f-a13f-4a56-9b8d-36c12e50d1b4\") " pod="openshift-ingress/kube-auth-proxy-6c488d768d-s2rqp" Jul 06 19:01:30.193266 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:01:30.193247 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/8daf052f-a13f-4a56-9b8d-36c12e50d1b4-tls-certs\") pod \"kube-auth-proxy-6c488d768d-s2rqp\" (UID: \"8daf052f-a13f-4a56-9b8d-36c12e50d1b4\") " pod="openshift-ingress/kube-auth-proxy-6c488d768d-s2rqp" Jul 06 19:01:30.389953 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:01:30.389919 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k8fbq_bc7057bd-88da-48af-ba32-e6efdc6e8362/konnectivity-agent/0.log" limit=1024 Jul 06 19:01:30.494556 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:01:30.494448 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tnvl2\" (UniqueName: \"kubernetes.io/projected/8daf052f-a13f-4a56-9b8d-36c12e50d1b4-kube-api-access-tnvl2\") pod \"kube-auth-proxy-6c488d768d-s2rqp\" (UID: \"8daf052f-a13f-4a56-9b8d-36c12e50d1b4\") " pod="openshift-ingress/kube-auth-proxy-6c488d768d-s2rqp" Jul 06 19:01:30.497106 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:01:30.497076 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-tnvl2\" (UniqueName: \"kubernetes.io/projected/8daf052f-a13f-4a56-9b8d-36c12e50d1b4-kube-api-access-tnvl2\") pod \"kube-auth-proxy-6c488d768d-s2rqp\" (UID: \"8daf052f-a13f-4a56-9b8d-36c12e50d1b4\") " pod="openshift-ingress/kube-auth-proxy-6c488d768d-s2rqp" Jul 06 19:01:30.540736 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:01:30.540704 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-6c488d768d-s2rqp" Jul 06 19:01:30.602569 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:01:30.602541 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k8fbq_bc7057bd-88da-48af-ba32-e6efdc6e8362/konnectivity-agent/0.log" limit=1024 Jul 06 19:01:30.735033 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:01:30.735010 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-6c488d768d-s2rqp"] Jul 06 19:01:30.737533 ip-10-0-130-59 kubenswrapper[2570]: W0706 19:01:30.737509 2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8daf052f_a13f_4a56_9b8d_36c12e50d1b4.slice/crio-971cd43d987444a066ae7fab496350b15ff30d562883decbeaeb82453a1a6903 WatchSource:0}: Error finding container 971cd43d987444a066ae7fab496350b15ff30d562883decbeaeb82453a1a6903: Status 404 returned error can't find the container with id 971cd43d987444a066ae7fab496350b15ff30d562883decbeaeb82453a1a6903 Jul 06 19:01:31.519300 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:01:31.519265 2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-lws-operator/lws-controller-manager-5d5c698c7c-vvttd"] Jul 06 19:01:31.522158 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:01:31.522138 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-5d5c698c7c-vvttd" Jul 06 19:01:31.537956 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:01:31.537935 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"lws-manager-config\"" Jul 06 19:01:31.538292 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:01:31.538275 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"webhook-server-cert\"" Jul 06 19:01:31.539404 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:01:31.539384 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"kube-root-ca.crt\"" Jul 06 19:01:31.539518 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:01:31.539453 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"metrics-server-cert\"" Jul 06 19:01:31.539518 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:01:31.539457 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"openshift-service-ca.crt\"" Jul 06 19:01:31.539518 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:01:31.539487 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"lws-controller-manager-dockercfg-nmr7s\"" Jul 06 19:01:31.545900 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:01:31.545875 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-5d5c698c7c-vvttd"] Jul 06 19:01:31.605528 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:01:31.605486 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-m9666\" (UniqueName: \"kubernetes.io/projected/bf643fd2-138d-4a1d-8fa7-939d1c3ae3a1-kube-api-access-m9666\") pod \"lws-controller-manager-5d5c698c7c-vvttd\" (UID: \"bf643fd2-138d-4a1d-8fa7-939d1c3ae3a1\") " pod="openshift-lws-operator/lws-controller-manager-5d5c698c7c-vvttd" Jul 06 19:01:31.605713 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:01:31.605591 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/bf643fd2-138d-4a1d-8fa7-939d1c3ae3a1-manager-config\") pod \"lws-controller-manager-5d5c698c7c-vvttd\" (UID: \"bf643fd2-138d-4a1d-8fa7-939d1c3ae3a1\") " pod="openshift-lws-operator/lws-controller-manager-5d5c698c7c-vvttd" Jul 06 19:01:31.605713 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:01:31.605694 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/bf643fd2-138d-4a1d-8fa7-939d1c3ae3a1-cert\") pod \"lws-controller-manager-5d5c698c7c-vvttd\" (UID: \"bf643fd2-138d-4a1d-8fa7-939d1c3ae3a1\") " pod="openshift-lws-operator/lws-controller-manager-5d5c698c7c-vvttd" Jul 06 19:01:31.605803 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:01:31.605764 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/bf643fd2-138d-4a1d-8fa7-939d1c3ae3a1-metrics-cert\") pod \"lws-controller-manager-5d5c698c7c-vvttd\" (UID: \"bf643fd2-138d-4a1d-8fa7-939d1c3ae3a1\") " pod="openshift-lws-operator/lws-controller-manager-5d5c698c7c-vvttd" Jul 06 19:01:31.707148 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:01:31.707115 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/bf643fd2-138d-4a1d-8fa7-939d1c3ae3a1-metrics-cert\") pod \"lws-controller-manager-5d5c698c7c-vvttd\" (UID: \"bf643fd2-138d-4a1d-8fa7-939d1c3ae3a1\") " pod="openshift-lws-operator/lws-controller-manager-5d5c698c7c-vvttd" Jul 06 19:01:31.707328 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:01:31.707169 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-m9666\" (UniqueName: \"kubernetes.io/projected/bf643fd2-138d-4a1d-8fa7-939d1c3ae3a1-kube-api-access-m9666\") pod \"lws-controller-manager-5d5c698c7c-vvttd\" (UID: \"bf643fd2-138d-4a1d-8fa7-939d1c3ae3a1\") " pod="openshift-lws-operator/lws-controller-manager-5d5c698c7c-vvttd" Jul 06 19:01:31.707328 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:01:31.707216 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/bf643fd2-138d-4a1d-8fa7-939d1c3ae3a1-manager-config\") pod \"lws-controller-manager-5d5c698c7c-vvttd\" (UID: \"bf643fd2-138d-4a1d-8fa7-939d1c3ae3a1\") " pod="openshift-lws-operator/lws-controller-manager-5d5c698c7c-vvttd" Jul 06 19:01:31.707328 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:01:31.707282 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/bf643fd2-138d-4a1d-8fa7-939d1c3ae3a1-cert\") pod \"lws-controller-manager-5d5c698c7c-vvttd\" (UID: \"bf643fd2-138d-4a1d-8fa7-939d1c3ae3a1\") " pod="openshift-lws-operator/lws-controller-manager-5d5c698c7c-vvttd" Jul 06 19:01:31.708061 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:01:31.708033 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/bf643fd2-138d-4a1d-8fa7-939d1c3ae3a1-manager-config\") pod \"lws-controller-manager-5d5c698c7c-vvttd\" (UID: \"bf643fd2-138d-4a1d-8fa7-939d1c3ae3a1\") " pod="openshift-lws-operator/lws-controller-manager-5d5c698c7c-vvttd" Jul 06 19:01:31.710077 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:01:31.710051 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/bf643fd2-138d-4a1d-8fa7-939d1c3ae3a1-metrics-cert\") pod \"lws-controller-manager-5d5c698c7c-vvttd\" (UID: \"bf643fd2-138d-4a1d-8fa7-939d1c3ae3a1\") " pod="openshift-lws-operator/lws-controller-manager-5d5c698c7c-vvttd" Jul 06 19:01:31.710240 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:01:31.710218 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/bf643fd2-138d-4a1d-8fa7-939d1c3ae3a1-cert\") pod \"lws-controller-manager-5d5c698c7c-vvttd\" (UID: \"bf643fd2-138d-4a1d-8fa7-939d1c3ae3a1\") " pod="openshift-lws-operator/lws-controller-manager-5d5c698c7c-vvttd" Jul 06 19:01:31.727554 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:01:31.727525 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-m9666\" (UniqueName: \"kubernetes.io/projected/bf643fd2-138d-4a1d-8fa7-939d1c3ae3a1-kube-api-access-m9666\") pod \"lws-controller-manager-5d5c698c7c-vvttd\" (UID: \"bf643fd2-138d-4a1d-8fa7-939d1c3ae3a1\") " pod="openshift-lws-operator/lws-controller-manager-5d5c698c7c-vvttd" Jul 06 19:01:31.734365 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:01:31.734267 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-6c488d768d-s2rqp" event={"ID":"8daf052f-a13f-4a56-9b8d-36c12e50d1b4","Type":"ContainerStarted","Data":"971cd43d987444a066ae7fab496350b15ff30d562883decbeaeb82453a1a6903"} Jul 06 19:01:31.833174 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:01:31.833091 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-5d5c698c7c-vvttd" Jul 06 19:01:32.053803 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:01:32.053770 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-5d5c698c7c-vvttd"] Jul 06 19:01:32.057180 ip-10-0-130-59 kubenswrapper[2570]: W0706 19:01:32.057150 2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbf643fd2_138d_4a1d_8fa7_939d1c3ae3a1.slice/crio-ce827344d390bd47c5113aa124ba5eb0f7303b01c0753667cafcc10e234e6169 WatchSource:0}: Error finding container ce827344d390bd47c5113aa124ba5eb0f7303b01c0753667cafcc10e234e6169: Status 404 returned error can't find the container with id ce827344d390bd47c5113aa124ba5eb0f7303b01c0753667cafcc10e234e6169 Jul 06 19:01:32.741169 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:01:32.741133 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-5d5c698c7c-vvttd" event={"ID":"bf643fd2-138d-4a1d-8fa7-939d1c3ae3a1","Type":"ContainerStarted","Data":"ce827344d390bd47c5113aa124ba5eb0f7303b01c0753667cafcc10e234e6169"} Jul 06 19:01:33.387947 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:01:33.387903 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k8fbq_bc7057bd-88da-48af-ba32-e6efdc6e8362/konnectivity-agent/0.log" limit=1024 Jul 06 19:01:33.747540 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:01:33.747503 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-6c488d768d-s2rqp" event={"ID":"8daf052f-a13f-4a56-9b8d-36c12e50d1b4","Type":"ContainerStarted","Data":"9ddcfb3e0f089dbfbfa7c75b8156f19a1e6a235961663c2dfbfa6507b9c59660"} Jul 06 19:01:33.836778 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:01:33.836723 2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/kube-auth-proxy-6c488d768d-s2rqp" podStartSLOduration=3.787968766 podStartE2EDuration="5.8367067s" totalImagesPullingTime="2.048737934s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 19:01:28 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 19:01:33.81030602 +0000 UTC m=+373.365327192" watchObservedRunningTime="2026-07-06 19:01:33.8367067 +0000 UTC m=+373.391727868" Jul 06 19:01:35.295748 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:01:35.295711 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k8fbq_bc7057bd-88da-48af-ba32-e6efdc6e8362/konnectivity-agent/0.log" limit=1024 Jul 06 19:01:35.656087 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:01:35.656034 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k8fbq_bc7057bd-88da-48af-ba32-e6efdc6e8362/konnectivity-agent/0.log" limit=1024 Jul 06 19:01:35.758671 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:01:35.758632 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-5d5c698c7c-vvttd" event={"ID":"bf643fd2-138d-4a1d-8fa7-939d1c3ae3a1","Type":"ContainerStarted","Data":"0b41bee440909e939e891f1f1906707082381a49793fef29afdf5a92ebf709df"} Jul 06 19:01:35.758882 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:01:35.758690 2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-lws-operator/lws-controller-manager-5d5c698c7c-vvttd" Jul 06 19:01:35.794342 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:01:35.794276 2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-lws-operator/lws-controller-manager-5d5c698c7c-vvttd" podStartSLOduration=1.244912153 podStartE2EDuration="4.794252776s" totalImagesPullingTime="3.549340623s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 19:01:31 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 19:01:35.791890062 +0000 UTC m=+375.346911245" watchObservedRunningTime="2026-07-06 19:01:35.794252776 +0000 UTC m=+375.349273951" Jul 06 19:01:35.956447 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:01:35.956407 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k8fbq_bc7057bd-88da-48af-ba32-e6efdc6e8362/konnectivity-agent/0.log" limit=1024 Jul 06 19:01:36.281430 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:01:36.281350 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k8fbq_bc7057bd-88da-48af-ba32-e6efdc6e8362/konnectivity-agent/0.log" limit=1024 Jul 06 19:01:36.528849 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:01:36.528816 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k8fbq_bc7057bd-88da-48af-ba32-e6efdc6e8362/konnectivity-agent/0.log" limit=1024 Jul 06 19:01:36.658090 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:01:36.658010 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k8fbq_bc7057bd-88da-48af-ba32-e6efdc6e8362/konnectivity-agent/0.log" limit=1024 Jul 06 19:01:37.720565 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:01:37.720540 2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/model-serving-api-54d89bd5fc-4f7jh" Jul 06 19:01:37.763688 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:01:37.763653 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k8fbq_bc7057bd-88da-48af-ba32-e6efdc6e8362/konnectivity-agent/0.log" limit=1024 Jul 06 19:01:37.914801 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:01:37.914770 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k8fbq_bc7057bd-88da-48af-ba32-e6efdc6e8362/konnectivity-agent/0.log" limit=1024 Jul 06 19:01:38.274838 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:01:38.274804 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k8fbq_bc7057bd-88da-48af-ba32-e6efdc6e8362/konnectivity-agent/0.log" limit=1024 Jul 06 19:01:38.690813 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:01:38.690781 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k8fbq_bc7057bd-88da-48af-ba32-e6efdc6e8362/konnectivity-agent/0.log" limit=1024 Jul 06 19:01:38.817804 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:01:38.817772 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k8fbq_bc7057bd-88da-48af-ba32-e6efdc6e8362/konnectivity-agent/0.log" limit=1024 Jul 06 19:01:46.765326 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:01:46.765298 2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-lws-operator/lws-controller-manager-5d5c698c7c-vvttd" Jul 06 19:02:01.422250 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:02:01.422214 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k8fbq_bc7057bd-88da-48af-ba32-e6efdc6e8362/konnectivity-agent/0.log" limit=1024 Jul 06 19:02:08.152976 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:02:08.152937 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k8fbq_bc7057bd-88da-48af-ba32-e6efdc6e8362/konnectivity-agent/0.log" limit=1024 Jul 06 19:02:08.249756 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:02:08.249727 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k8fbq_bc7057bd-88da-48af-ba32-e6efdc6e8362/konnectivity-agent/0.log" limit=1024 Jul 06 19:02:22.906703 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:02:22.906667 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k8fbq_bc7057bd-88da-48af-ba32-e6efdc6e8362/konnectivity-agent/0.log" limit=1024 Jul 06 19:02:23.003103 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:02:23.003067 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k8fbq_bc7057bd-88da-48af-ba32-e6efdc6e8362/konnectivity-agent/0.log" limit=1024 Jul 06 19:02:23.103543 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:02:23.103513 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k8fbq_bc7057bd-88da-48af-ba32-e6efdc6e8362/konnectivity-agent/0.log" limit=1024 Jul 06 19:02:23.835275 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:02:23.835243 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k8fbq_bc7057bd-88da-48af-ba32-e6efdc6e8362/konnectivity-agent/0.log" limit=1024 Jul 06 19:02:24.059954 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:02:24.059917 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k8fbq_bc7057bd-88da-48af-ba32-e6efdc6e8362/konnectivity-agent/0.log" limit=1024 Jul 06 19:02:49.615215 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:02:49.615139 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k8fbq_bc7057bd-88da-48af-ba32-e6efdc6e8362/konnectivity-agent/0.log" limit=1024 Jul 06 19:02:49.702853 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:02:49.702819 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k8fbq_bc7057bd-88da-48af-ba32-e6efdc6e8362/konnectivity-agent/0.log" limit=1024 Jul 06 19:02:53.147485 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:02:53.147446 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k8fbq_bc7057bd-88da-48af-ba32-e6efdc6e8362/konnectivity-agent/0.log" limit=1024 Jul 06 19:02:57.155096 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:02:57.155064 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k8fbq_bc7057bd-88da-48af-ba32-e6efdc6e8362/konnectivity-agent/0.log" limit=1024 Jul 06 19:03:00.240583 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:03:00.240544 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k8fbq_bc7057bd-88da-48af-ba32-e6efdc6e8362/konnectivity-agent/0.log" limit=1024 Jul 06 19:03:00.349004 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:03:00.348969 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k8fbq_bc7057bd-88da-48af-ba32-e6efdc6e8362/konnectivity-agent/0.log" limit=1024 Jul 06 19:03:00.454141 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:03:00.454105 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k8fbq_bc7057bd-88da-48af-ba32-e6efdc6e8362/konnectivity-agent/0.log" limit=1024 Jul 06 19:03:01.512120 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:03:01.512089 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k8fbq_bc7057bd-88da-48af-ba32-e6efdc6e8362/konnectivity-agent/0.log" limit=1024 Jul 06 19:03:02.064252 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:03:02.064203 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k8fbq_bc7057bd-88da-48af-ba32-e6efdc6e8362/konnectivity-agent/0.log" limit=1024 Jul 06 19:03:02.235460 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:03:02.235424 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k8fbq_bc7057bd-88da-48af-ba32-e6efdc6e8362/konnectivity-agent/0.log" limit=1024 Jul 06 19:03:03.759072 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:03:03.759042 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k8fbq_bc7057bd-88da-48af-ba32-e6efdc6e8362/konnectivity-agent/0.log" limit=1024 Jul 06 19:03:06.202064 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:03:06.202030 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k8fbq_bc7057bd-88da-48af-ba32-e6efdc6e8362/konnectivity-agent/0.log" limit=1024 Jul 06 19:03:06.449082 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:03:06.449053 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k8fbq_bc7057bd-88da-48af-ba32-e6efdc6e8362/konnectivity-agent/0.log" limit=1024 Jul 06 19:03:06.543415 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:03:06.543336 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k8fbq_bc7057bd-88da-48af-ba32-e6efdc6e8362/konnectivity-agent/0.log" limit=1024 Jul 06 19:03:08.317419 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:03:08.317387 2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/dns-operator-controller-manager-864dd78c4c-bs6mw"] Jul 06 19:03:08.319809 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:03:08.319793 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-bs6mw" Jul 06 19:03:08.324266 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:03:08.324245 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"dns-operator-controller-env\"" Jul 06 19:03:08.324579 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:03:08.324562 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\"" Jul 06 19:03:08.324699 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:03:08.324565 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\"" Jul 06 19:03:08.326754 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:03:08.326558 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"dns-operator-controller-manager-dockercfg-lkd99\"" Jul 06 19:03:08.341527 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:03:08.341502 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/dns-operator-controller-manager-864dd78c4c-bs6mw"] Jul 06 19:03:08.405147 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:03:08.405111 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k8fbq_bc7057bd-88da-48af-ba32-e6efdc6e8362/konnectivity-agent/0.log" limit=1024 Jul 06 19:03:08.460704 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:03:08.460659 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ckqfg\" (UniqueName: \"kubernetes.io/projected/88de4830-cf1c-4169-8494-e6669a05ad3d-kube-api-access-ckqfg\") pod \"dns-operator-controller-manager-864dd78c4c-bs6mw\" (UID: \"88de4830-cf1c-4169-8494-e6669a05ad3d\") " pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-bs6mw" Jul 06 19:03:08.562385 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:03:08.562336 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ckqfg\" (UniqueName: \"kubernetes.io/projected/88de4830-cf1c-4169-8494-e6669a05ad3d-kube-api-access-ckqfg\") pod \"dns-operator-controller-manager-864dd78c4c-bs6mw\" (UID: \"88de4830-cf1c-4169-8494-e6669a05ad3d\") " pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-bs6mw" Jul 06 19:03:08.572093 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:03:08.572034 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-ckqfg\" (UniqueName: \"kubernetes.io/projected/88de4830-cf1c-4169-8494-e6669a05ad3d-kube-api-access-ckqfg\") pod \"dns-operator-controller-manager-864dd78c4c-bs6mw\" (UID: \"88de4830-cf1c-4169-8494-e6669a05ad3d\") " pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-bs6mw" Jul 06 19:03:08.630152 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:03:08.630122 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-bs6mw" Jul 06 19:03:08.777694 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:03:08.777672 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/dns-operator-controller-manager-864dd78c4c-bs6mw"] Jul 06 19:03:08.779821 ip-10-0-130-59 kubenswrapper[2570]: W0706 19:03:08.779783 2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod88de4830_cf1c_4169_8494_e6669a05ad3d.slice/crio-4c77778ec8822894f74b918ef7254c85ab1e8e18ca4c08735eebe0196f535836 WatchSource:0}: Error finding container 4c77778ec8822894f74b918ef7254c85ab1e8e18ca4c08735eebe0196f535836: Status 404 returned error can't find the container with id 4c77778ec8822894f74b918ef7254c85ab1e8e18ca4c08735eebe0196f535836 Jul 06 19:03:09.168038 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:03:09.168007 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-bs6mw" event={"ID":"88de4830-cf1c-4169-8494-e6669a05ad3d","Type":"ContainerStarted","Data":"4c77778ec8822894f74b918ef7254c85ab1e8e18ca4c08735eebe0196f535836"} Jul 06 19:03:10.199937 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:03:10.199900 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k8fbq_bc7057bd-88da-48af-ba32-e6efdc6e8362/konnectivity-agent/0.log" limit=1024 Jul 06 19:03:10.296363 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:03:10.296333 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k8fbq_bc7057bd-88da-48af-ba32-e6efdc6e8362/konnectivity-agent/0.log" limit=1024 Jul 06 19:03:10.914858 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:03:10.914824 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k8fbq_bc7057bd-88da-48af-ba32-e6efdc6e8362/konnectivity-agent/0.log" limit=1024 Jul 06 19:03:11.070005 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:03:11.069955 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k8fbq_bc7057bd-88da-48af-ba32-e6efdc6e8362/konnectivity-agent/0.log" limit=1024 Jul 06 19:03:12.190788 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:03:12.190753 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-bs6mw" event={"ID":"88de4830-cf1c-4169-8494-e6669a05ad3d","Type":"ContainerStarted","Data":"c6632368da74025bb81bf0179c7df2c7fd9e59a0199586a1381ba8dc4d7ac56a"} Jul 06 19:03:12.191175 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:03:12.190872 2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-bs6mw" Jul 06 19:03:12.224247 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:03:12.224191 2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-bs6mw" podStartSLOduration=1.44665945 podStartE2EDuration="4.224175772s" totalImagesPullingTime="2.777516322s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 19:03:08 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 19:03:12.222135703 +0000 UTC m=+471.777156875" watchObservedRunningTime="2026-07-06 19:03:12.224175772 +0000 UTC m=+471.779196944" Jul 06 19:03:13.521940 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:03:13.521902 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k8fbq_bc7057bd-88da-48af-ba32-e6efdc6e8362/konnectivity-agent/0.log" limit=1024 Jul 06 19:03:13.604536 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:03:13.604502 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k8fbq_bc7057bd-88da-48af-ba32-e6efdc6e8362/konnectivity-agent/0.log" limit=1024 Jul 06 19:03:21.996492 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:03:21.996457 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k8fbq_bc7057bd-88da-48af-ba32-e6efdc6e8362/konnectivity-agent/0.log" limit=1024 Jul 06 19:03:22.164469 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:03:22.164431 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k8fbq_bc7057bd-88da-48af-ba32-e6efdc6e8362/konnectivity-agent/0.log" limit=1024 Jul 06 19:03:22.344070 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:03:22.343959 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k8fbq_bc7057bd-88da-48af-ba32-e6efdc6e8362/konnectivity-agent/0.log" limit=1024 Jul 06 19:03:23.197338 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:03:23.197311 2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-bs6mw" Jul 06 19:03:24.615843 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:03:24.615809 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k8fbq_bc7057bd-88da-48af-ba32-e6efdc6e8362/konnectivity-agent/0.log" limit=1024 Jul 06 19:03:24.763288 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:03:24.763252 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k8fbq_bc7057bd-88da-48af-ba32-e6efdc6e8362/konnectivity-agent/0.log" limit=1024 Jul 06 19:03:28.775824 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:03:28.775792 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k8fbq_bc7057bd-88da-48af-ba32-e6efdc6e8362/konnectivity-agent/0.log" limit=1024 Jul 06 19:03:28.909238 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:03:28.909208 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k8fbq_bc7057bd-88da-48af-ba32-e6efdc6e8362/konnectivity-agent/0.log" limit=1024 Jul 06 19:03:31.758669 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:03:31.758635 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k8fbq_bc7057bd-88da-48af-ba32-e6efdc6e8362/konnectivity-agent/0.log" limit=1024 Jul 06 19:03:31.855343 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:03:31.855308 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k8fbq_bc7057bd-88da-48af-ba32-e6efdc6e8362/konnectivity-agent/0.log" limit=1024 Jul 06 19:03:41.333258 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:03:41.333220 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k8fbq_bc7057bd-88da-48af-ba32-e6efdc6e8362/konnectivity-agent/0.log" limit=1024 Jul 06 19:03:41.433909 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:03:41.433881 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k8fbq_bc7057bd-88da-48af-ba32-e6efdc6e8362/konnectivity-agent/0.log" limit=1024 Jul 06 19:03:41.947167 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:03:41.947130 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k8fbq_bc7057bd-88da-48af-ba32-e6efdc6e8362/konnectivity-agent/0.log" limit=1024 Jul 06 19:03:42.042334 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:03:42.042302 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k8fbq_bc7057bd-88da-48af-ba32-e6efdc6e8362/konnectivity-agent/0.log" limit=1024 Jul 06 19:03:43.404793 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:03:43.404763 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k8fbq_bc7057bd-88da-48af-ba32-e6efdc6e8362/konnectivity-agent/0.log" limit=1024 Jul 06 19:03:43.505657 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:03:43.505598 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k8fbq_bc7057bd-88da-48af-ba32-e6efdc6e8362/konnectivity-agent/0.log" limit=1024 Jul 06 19:03:48.277631 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:03:48.277575 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k8fbq_bc7057bd-88da-48af-ba32-e6efdc6e8362/konnectivity-agent/0.log" limit=1024 Jul 06 19:03:48.378000 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:03:48.377966 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k8fbq_bc7057bd-88da-48af-ba32-e6efdc6e8362/konnectivity-agent/0.log" limit=1024 Jul 06 19:04:02.356248 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:04:02.356208 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k8fbq_bc7057bd-88da-48af-ba32-e6efdc6e8362/konnectivity-agent/0.log" limit=1024 Jul 06 19:04:02.426744 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:04:02.426692 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k8fbq_bc7057bd-88da-48af-ba32-e6efdc6e8362/konnectivity-agent/0.log" limit=1024 Jul 06 19:04:02.832050 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:04:02.832015 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k8fbq_bc7057bd-88da-48af-ba32-e6efdc6e8362/konnectivity-agent/0.log" limit=1024 Jul 06 19:04:03.536956 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:04:03.536925 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k8fbq_bc7057bd-88da-48af-ba32-e6efdc6e8362/konnectivity-agent/0.log" limit=1024 Jul 06 19:04:04.438938 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:04:04.438902 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k8fbq_bc7057bd-88da-48af-ba32-e6efdc6e8362/konnectivity-agent/0.log" limit=1024 Jul 06 19:04:04.542912 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:04:04.542884 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k8fbq_bc7057bd-88da-48af-ba32-e6efdc6e8362/konnectivity-agent/0.log" limit=1024 Jul 06 19:04:04.633455 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:04:04.633430 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k8fbq_bc7057bd-88da-48af-ba32-e6efdc6e8362/konnectivity-agent/0.log" limit=1024 Jul 06 19:04:04.871087 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:04:04.871009 2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-flg9z"] Jul 06 19:04:04.873945 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:04:04.873929 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-54d6d988d5-flg9z" Jul 06 19:04:04.878497 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:04:04.878477 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"limitador-limits-config-limitador\"" Jul 06 19:04:04.878711 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:04:04.878694 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"default-dockercfg-nqzkc\"" Jul 06 19:04:04.893800 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:04:04.893778 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-flg9z"] Jul 06 19:04:04.946209 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:04:04.946181 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/9bc5684c-2f3b-40be-91f7-a749ff3ff4c3-config-file\") pod \"limitador-limitador-54d6d988d5-flg9z\" (UID: \"9bc5684c-2f3b-40be-91f7-a749ff3ff4c3\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-flg9z" Jul 06 19:04:04.946367 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:04:04.946219 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-g49jl\" (UniqueName: \"kubernetes.io/projected/9bc5684c-2f3b-40be-91f7-a749ff3ff4c3-kube-api-access-g49jl\") pod \"limitador-limitador-54d6d988d5-flg9z\" (UID: \"9bc5684c-2f3b-40be-91f7-a749ff3ff4c3\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-flg9z" Jul 06 19:04:04.951218 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:04:04.951197 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-flg9z"] Jul 06 19:04:05.047020 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:04:05.046979 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/9bc5684c-2f3b-40be-91f7-a749ff3ff4c3-config-file\") pod \"limitador-limitador-54d6d988d5-flg9z\" (UID: \"9bc5684c-2f3b-40be-91f7-a749ff3ff4c3\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-flg9z" Jul 06 19:04:05.047198 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:04:05.047057 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-g49jl\" (UniqueName: \"kubernetes.io/projected/9bc5684c-2f3b-40be-91f7-a749ff3ff4c3-kube-api-access-g49jl\") pod \"limitador-limitador-54d6d988d5-flg9z\" (UID: \"9bc5684c-2f3b-40be-91f7-a749ff3ff4c3\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-flg9z" Jul 06 19:04:05.047786 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:04:05.047727 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/9bc5684c-2f3b-40be-91f7-a749ff3ff4c3-config-file\") pod \"limitador-limitador-54d6d988d5-flg9z\" (UID: \"9bc5684c-2f3b-40be-91f7-a749ff3ff4c3\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-flg9z" Jul 06 19:04:05.058059 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:04:05.058036 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-g49jl\" (UniqueName: \"kubernetes.io/projected/9bc5684c-2f3b-40be-91f7-a749ff3ff4c3-kube-api-access-g49jl\") pod \"limitador-limitador-54d6d988d5-flg9z\" (UID: \"9bc5684c-2f3b-40be-91f7-a749ff3ff4c3\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-flg9z" Jul 06 19:04:05.184347 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:04:05.184319 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-54d6d988d5-flg9z" Jul 06 19:04:05.325586 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:04:05.325562 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-flg9z"] Jul 06 19:04:05.327940 ip-10-0-130-59 kubenswrapper[2570]: W0706 19:04:05.327913 2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9bc5684c_2f3b_40be_91f7_a749ff3ff4c3.slice/crio-347351a702fad72f5f1abd7b85f02e1f409808f9043af2f929890a8149be49f1 WatchSource:0}: Error finding container 347351a702fad72f5f1abd7b85f02e1f409808f9043af2f929890a8149be49f1: Status 404 returned error can't find the container with id 347351a702fad72f5f1abd7b85f02e1f409808f9043af2f929890a8149be49f1 Jul 06 19:04:05.367749 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:04:05.367720 2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-58cd955675-p8dcp"] Jul 06 19:04:05.373575 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:04:05.373541 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-58cd955675-p8dcp" Jul 06 19:04:05.380122 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:04:05.380095 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-authorino-dockercfg-b9zxp\"" Jul 06 19:04:05.380427 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:04:05.380403 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-58cd955675-p8dcp"] Jul 06 19:04:05.424504 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:04:05.424467 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k8fbq_bc7057bd-88da-48af-ba32-e6efdc6e8362/konnectivity-agent/0.log" limit=1024 Jul 06 19:04:05.433806 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:04:05.433771 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-54d6d988d5-flg9z" event={"ID":"9bc5684c-2f3b-40be-91f7-a749ff3ff4c3","Type":"ContainerStarted","Data":"347351a702fad72f5f1abd7b85f02e1f409808f9043af2f929890a8149be49f1"} Jul 06 19:04:05.451426 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:04:05.451358 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-k2zfv\" (UniqueName: \"kubernetes.io/projected/8be8ccee-f673-45d1-858d-133e3c1ae6ea-kube-api-access-k2zfv\") pod \"authorino-58cd955675-p8dcp\" (UID: \"8be8ccee-f673-45d1-858d-133e3c1ae6ea\") " pod="kuadrant-system/authorino-58cd955675-p8dcp" Jul 06 19:04:05.542007 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:04:05.541973 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k8fbq_bc7057bd-88da-48af-ba32-e6efdc6e8362/konnectivity-agent/0.log" limit=1024 Jul 06 19:04:05.552226 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:04:05.552198 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-k2zfv\" (UniqueName: \"kubernetes.io/projected/8be8ccee-f673-45d1-858d-133e3c1ae6ea-kube-api-access-k2zfv\") pod \"authorino-58cd955675-p8dcp\" (UID: \"8be8ccee-f673-45d1-858d-133e3c1ae6ea\") " pod="kuadrant-system/authorino-58cd955675-p8dcp" Jul 06 19:04:05.562607 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:04:05.562577 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-k2zfv\" (UniqueName: \"kubernetes.io/projected/8be8ccee-f673-45d1-858d-133e3c1ae6ea-kube-api-access-k2zfv\") pod \"authorino-58cd955675-p8dcp\" (UID: \"8be8ccee-f673-45d1-858d-133e3c1ae6ea\") " pod="kuadrant-system/authorino-58cd955675-p8dcp" Jul 06 19:04:05.684377 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:04:05.684340 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-58cd955675-p8dcp" Jul 06 19:04:05.843604 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:04:05.843568 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-58cd955675-p8dcp"] Jul 06 19:04:05.851669 ip-10-0-130-59 kubenswrapper[2570]: W0706 19:04:05.851591 2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod8be8ccee_f673_45d1_858d_133e3c1ae6ea.slice/crio-9e4eddc9d5f920e5ec9c197ffd175c408177c03e9550adeb5eec640c2826023b WatchSource:0}: Error finding container 9e4eddc9d5f920e5ec9c197ffd175c408177c03e9550adeb5eec640c2826023b: Status 404 returned error can't find the container with id 9e4eddc9d5f920e5ec9c197ffd175c408177c03e9550adeb5eec640c2826023b Jul 06 19:04:06.440122 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:04:06.440086 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-58cd955675-p8dcp" event={"ID":"8be8ccee-f673-45d1-858d-133e3c1ae6ea","Type":"ContainerStarted","Data":"9e4eddc9d5f920e5ec9c197ffd175c408177c03e9550adeb5eec640c2826023b"} Jul 06 19:04:08.454088 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:04:08.454051 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-54d6d988d5-flg9z" event={"ID":"9bc5684c-2f3b-40be-91f7-a749ff3ff4c3","Type":"ContainerStarted","Data":"6823613233b6a5a30e32c02ce71b4f53aa4389be9441d0c9aff06aec3a97d1d6"} Jul 06 19:04:08.454551 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:04:08.454243 2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-limitador-54d6d988d5-flg9z" Jul 06 19:04:08.472179 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:04:08.472135 2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-limitador-54d6d988d5-flg9z" podStartSLOduration=1.855131433 podStartE2EDuration="4.472116198s" totalImagesPullingTime="2.616984765s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 19:04:04 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 19:04:08.470440239 +0000 UTC m=+528.025461422" watchObservedRunningTime="2026-07-06 19:04:08.472116198 +0000 UTC m=+528.027137373" Jul 06 19:04:10.465682 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:04:10.465644 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-58cd955675-p8dcp" event={"ID":"8be8ccee-f673-45d1-858d-133e3c1ae6ea","Type":"ContainerStarted","Data":"869824ce01e984c3af7e55536ca558817d04933704872058adfc8528b35eddd7"} Jul 06 19:04:10.489941 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:04:10.489895 2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-58cd955675-p8dcp" podStartSLOduration=1.171507334 podStartE2EDuration="5.489879928s" totalImagesPullingTime="4.318372594s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 19:04:05 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 19:04:10.489841618 +0000 UTC m=+530.044862793" watchObservedRunningTime="2026-07-06 19:04:10.489879928 +0000 UTC m=+530.044901100" Jul 06 19:04:11.209694 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:04:11.209658 2570 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-58cd955675-p8dcp"] Jul 06 19:04:12.474878 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:04:12.474784 2570 kuberuntime_container.go:872] "Killing container with a grace period" pod="kuadrant-system/authorino-58cd955675-p8dcp" podUID="8be8ccee-f673-45d1-858d-133e3c1ae6ea" containerName="authorino" containerID="cri-o://869824ce01e984c3af7e55536ca558817d04933704872058adfc8528b35eddd7" gracePeriod=30 Jul 06 19:04:12.721899 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:04:12.721874 2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-58cd955675-p8dcp" Jul 06 19:04:12.817988 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:04:12.817907 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-k2zfv\" (UniqueName: \"kubernetes.io/projected/8be8ccee-f673-45d1-858d-133e3c1ae6ea-kube-api-access-k2zfv\") pod \"8be8ccee-f673-45d1-858d-133e3c1ae6ea\" (UID: \"8be8ccee-f673-45d1-858d-133e3c1ae6ea\") " Jul 06 19:04:12.819994 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:04:12.819972 2570 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8be8ccee-f673-45d1-858d-133e3c1ae6ea-kube-api-access-k2zfv" (OuterVolumeSpecName: "kube-api-access-k2zfv") pod "8be8ccee-f673-45d1-858d-133e3c1ae6ea" (UID: "8be8ccee-f673-45d1-858d-133e3c1ae6ea"). InnerVolumeSpecName "kube-api-access-k2zfv". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 06 19:04:12.919137 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:04:12.919108 2570 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-k2zfv\" (UniqueName: \"kubernetes.io/projected/8be8ccee-f673-45d1-858d-133e3c1ae6ea-kube-api-access-k2zfv\") on node \"ip-10-0-130-59.ec2.internal\" DevicePath \"\"" Jul 06 19:04:13.480988 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:04:13.480946 2570 generic.go:356] "Generic (PLEG): container finished" podID="8be8ccee-f673-45d1-858d-133e3c1ae6ea" containerID="869824ce01e984c3af7e55536ca558817d04933704872058adfc8528b35eddd7" exitCode=0 Jul 06 19:04:13.480988 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:04:13.480992 2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-58cd955675-p8dcp" Jul 06 19:04:13.481484 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:04:13.480993 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-58cd955675-p8dcp" event={"ID":"8be8ccee-f673-45d1-858d-133e3c1ae6ea","Type":"ContainerDied","Data":"869824ce01e984c3af7e55536ca558817d04933704872058adfc8528b35eddd7"} Jul 06 19:04:13.481484 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:04:13.481033 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-58cd955675-p8dcp" event={"ID":"8be8ccee-f673-45d1-858d-133e3c1ae6ea","Type":"ContainerDied","Data":"9e4eddc9d5f920e5ec9c197ffd175c408177c03e9550adeb5eec640c2826023b"} Jul 06 19:04:13.481484 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:04:13.481054 2570 scope.go:117] "RemoveContainer" containerID="869824ce01e984c3af7e55536ca558817d04933704872058adfc8528b35eddd7" Jul 06 19:04:13.489389 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:04:13.489370 2570 scope.go:117] "RemoveContainer" containerID="869824ce01e984c3af7e55536ca558817d04933704872058adfc8528b35eddd7" Jul 06 19:04:13.489673 ip-10-0-130-59 kubenswrapper[2570]: E0706 19:04:13.489652 2570 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"869824ce01e984c3af7e55536ca558817d04933704872058adfc8528b35eddd7\": container with ID starting with 869824ce01e984c3af7e55536ca558817d04933704872058adfc8528b35eddd7 not found: ID does not exist" containerID="869824ce01e984c3af7e55536ca558817d04933704872058adfc8528b35eddd7" Jul 06 19:04:13.489723 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:04:13.489684 2570 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"869824ce01e984c3af7e55536ca558817d04933704872058adfc8528b35eddd7"} err="failed to get container status \"869824ce01e984c3af7e55536ca558817d04933704872058adfc8528b35eddd7\": rpc error: code = NotFound desc = could not find container \"869824ce01e984c3af7e55536ca558817d04933704872058adfc8528b35eddd7\": container with ID starting with 869824ce01e984c3af7e55536ca558817d04933704872058adfc8528b35eddd7 not found: ID does not exist" Jul 06 19:04:13.501631 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:04:13.501597 2570 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-58cd955675-p8dcp"] Jul 06 19:04:13.506887 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:04:13.506867 2570 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-58cd955675-p8dcp"] Jul 06 19:04:15.027728 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:04:15.027693 2570 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="8be8ccee-f673-45d1-858d-133e3c1ae6ea" path="/var/lib/kubelet/pods/8be8ccee-f673-45d1-858d-133e3c1ae6ea/volumes" Jul 06 19:04:19.460515 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:04:19.460485 2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-limitador-54d6d988d5-flg9z" Jul 06 19:04:20.939256 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:04:20.939226 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k8fbq_bc7057bd-88da-48af-ba32-e6efdc6e8362/konnectivity-agent/0.log" limit=1024 Jul 06 19:04:33.632486 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:04:33.632452 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k8fbq_bc7057bd-88da-48af-ba32-e6efdc6e8362/konnectivity-agent/0.log" limit=1024 Jul 06 19:04:33.824306 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:04:33.824273 2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-5d4b67c446-9g76h"] Jul 06 19:04:33.824753 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:04:33.824735 2570 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8be8ccee-f673-45d1-858d-133e3c1ae6ea" containerName="authorino" Jul 06 19:04:33.824753 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:04:33.824755 2570 state_mem.go:107] "Deleted CPUSet assignment" podUID="8be8ccee-f673-45d1-858d-133e3c1ae6ea" containerName="authorino" Jul 06 19:04:33.824901 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:04:33.824860 2570 memory_manager.go:368] "RemoveStaleState removing state" podUID="8be8ccee-f673-45d1-858d-133e3c1ae6ea" containerName="authorino" Jul 06 19:04:33.831839 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:04:33.831818 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5d4b67c446-9g76h" Jul 06 19:04:33.835070 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:04:33.835050 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-authorino-dockercfg-b9zxp\"" Jul 06 19:04:33.847426 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:04:33.847404 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-5d4b67c446-9g76h"] Jul 06 19:04:34.007880 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:04:34.007852 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-t79xb\" (UniqueName: \"kubernetes.io/projected/7a277419-7d1b-426f-90cd-63d5a11bd07b-kube-api-access-t79xb\") pod \"authorino-5d4b67c446-9g76h\" (UID: \"7a277419-7d1b-426f-90cd-63d5a11bd07b\") " pod="kuadrant-system/authorino-5d4b67c446-9g76h" Jul 06 19:04:34.073382 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:04:34.073348 2570 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-5d4b67c446-9g76h"] Jul 06 19:04:34.073630 ip-10-0-130-59 kubenswrapper[2570]: E0706 19:04:34.073598 2570 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[kube-api-access-t79xb], unattached volumes=[], failed to process volumes=[]: context canceled" pod="kuadrant-system/authorino-5d4b67c446-9g76h" podUID="7a277419-7d1b-426f-90cd-63d5a11bd07b" Jul 06 19:04:34.108889 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:04:34.108859 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-t79xb\" (UniqueName: \"kubernetes.io/projected/7a277419-7d1b-426f-90cd-63d5a11bd07b-kube-api-access-t79xb\") pod \"authorino-5d4b67c446-9g76h\" (UID: \"7a277419-7d1b-426f-90cd-63d5a11bd07b\") " pod="kuadrant-system/authorino-5d4b67c446-9g76h" Jul 06 19:04:34.125976 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:04:34.125950 2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-77d5fc7699-sf7rg"] Jul 06 19:04:34.128273 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:04:34.128256 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-77d5fc7699-sf7rg" Jul 06 19:04:34.128556 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:04:34.128536 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-t79xb\" (UniqueName: \"kubernetes.io/projected/7a277419-7d1b-426f-90cd-63d5a11bd07b-kube-api-access-t79xb\") pod \"authorino-5d4b67c446-9g76h\" (UID: \"7a277419-7d1b-426f-90cd-63d5a11bd07b\") " pod="kuadrant-system/authorino-5d4b67c446-9g76h" Jul 06 19:04:34.139659 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:04:34.139638 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-77d5fc7699-sf7rg"] Jul 06 19:04:34.149835 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:04:34.149813 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k8fbq_bc7057bd-88da-48af-ba32-e6efdc6e8362/konnectivity-agent/0.log" limit=1024 Jul 06 19:04:34.209942 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:04:34.209909 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6m9s4\" (UniqueName: \"kubernetes.io/projected/45b95465-8868-4d3c-a57d-ed0dd828b8da-kube-api-access-6m9s4\") pod \"authorino-77d5fc7699-sf7rg\" (UID: \"45b95465-8868-4d3c-a57d-ed0dd828b8da\") " pod="kuadrant-system/authorino-77d5fc7699-sf7rg" Jul 06 19:04:34.310986 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:04:34.310917 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6m9s4\" (UniqueName: \"kubernetes.io/projected/45b95465-8868-4d3c-a57d-ed0dd828b8da-kube-api-access-6m9s4\") pod \"authorino-77d5fc7699-sf7rg\" (UID: \"45b95465-8868-4d3c-a57d-ed0dd828b8da\") " pod="kuadrant-system/authorino-77d5fc7699-sf7rg" Jul 06 19:04:34.327699 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:04:34.327677 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-6m9s4\" (UniqueName: \"kubernetes.io/projected/45b95465-8868-4d3c-a57d-ed0dd828b8da-kube-api-access-6m9s4\") pod \"authorino-77d5fc7699-sf7rg\" (UID: \"45b95465-8868-4d3c-a57d-ed0dd828b8da\") " pod="kuadrant-system/authorino-77d5fc7699-sf7rg" Jul 06 19:04:34.400744 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:04:34.400713 2570 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-77d5fc7699-sf7rg"] Jul 06 19:04:34.400972 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:04:34.400960 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-77d5fc7699-sf7rg" Jul 06 19:04:34.430363 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:04:34.430338 2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-7b6544547-bmbw7"] Jul 06 19:04:34.432763 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:04:34.432742 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-7b6544547-bmbw7" Jul 06 19:04:34.435519 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:04:34.435500 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-server-cert\"" Jul 06 19:04:34.449385 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:04:34.449336 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-7b6544547-bmbw7"] Jul 06 19:04:34.513574 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:04:34.513542 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/519d32f5-21dd-4c66-8c71-67d6d724c33a-tls-cert\") pod \"authorino-7b6544547-bmbw7\" (UID: \"519d32f5-21dd-4c66-8c71-67d6d724c33a\") " pod="kuadrant-system/authorino-7b6544547-bmbw7" Jul 06 19:04:34.513806 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:04:34.513784 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-h8tpn\" (UniqueName: \"kubernetes.io/projected/519d32f5-21dd-4c66-8c71-67d6d724c33a-kube-api-access-h8tpn\") pod \"authorino-7b6544547-bmbw7\" (UID: \"519d32f5-21dd-4c66-8c71-67d6d724c33a\") " pod="kuadrant-system/authorino-7b6544547-bmbw7" Jul 06 19:04:34.536070 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:04:34.536048 2570 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-77d5fc7699-sf7rg"] Jul 06 19:04:34.538735 ip-10-0-130-59 kubenswrapper[2570]: W0706 19:04:34.538710 2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod45b95465_8868_4d3c_a57d_ed0dd828b8da.slice/crio-27b24ab7ce1c63086bb116c6ebe7aa76482625d3187a8d3602018b652f9eb243 WatchSource:0}: Error finding container 27b24ab7ce1c63086bb116c6ebe7aa76482625d3187a8d3602018b652f9eb243: Status 404 returned error can't find the container with id 27b24ab7ce1c63086bb116c6ebe7aa76482625d3187a8d3602018b652f9eb243 Jul 06 19:04:34.577283 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:04:34.577224 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-77d5fc7699-sf7rg" event={"ID":"45b95465-8868-4d3c-a57d-ed0dd828b8da","Type":"ContainerStarted","Data":"27b24ab7ce1c63086bb116c6ebe7aa76482625d3187a8d3602018b652f9eb243"} Jul 06 19:04:34.577283 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:04:34.577240 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5d4b67c446-9g76h" Jul 06 19:04:34.583177 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:04:34.583156 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5d4b67c446-9g76h" Jul 06 19:04:34.614497 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:04:34.614470 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-h8tpn\" (UniqueName: \"kubernetes.io/projected/519d32f5-21dd-4c66-8c71-67d6d724c33a-kube-api-access-h8tpn\") pod \"authorino-7b6544547-bmbw7\" (UID: \"519d32f5-21dd-4c66-8c71-67d6d724c33a\") " pod="kuadrant-system/authorino-7b6544547-bmbw7" Jul 06 19:04:34.614633 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:04:34.614593 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/519d32f5-21dd-4c66-8c71-67d6d724c33a-tls-cert\") pod \"authorino-7b6544547-bmbw7\" (UID: \"519d32f5-21dd-4c66-8c71-67d6d724c33a\") " pod="kuadrant-system/authorino-7b6544547-bmbw7" Jul 06 19:04:34.616889 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:04:34.616872 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/519d32f5-21dd-4c66-8c71-67d6d724c33a-tls-cert\") pod \"authorino-7b6544547-bmbw7\" (UID: \"519d32f5-21dd-4c66-8c71-67d6d724c33a\") " pod="kuadrant-system/authorino-7b6544547-bmbw7" Jul 06 19:04:34.623976 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:04:34.623961 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-h8tpn\" (UniqueName: \"kubernetes.io/projected/519d32f5-21dd-4c66-8c71-67d6d724c33a-kube-api-access-h8tpn\") pod \"authorino-7b6544547-bmbw7\" (UID: \"519d32f5-21dd-4c66-8c71-67d6d724c33a\") " pod="kuadrant-system/authorino-7b6544547-bmbw7" Jul 06 19:04:34.715204 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:04:34.715175 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-t79xb\" (UniqueName: \"kubernetes.io/projected/7a277419-7d1b-426f-90cd-63d5a11bd07b-kube-api-access-t79xb\") pod \"7a277419-7d1b-426f-90cd-63d5a11bd07b\" (UID: \"7a277419-7d1b-426f-90cd-63d5a11bd07b\") " Jul 06 19:04:34.717107 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:04:34.717084 2570 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7a277419-7d1b-426f-90cd-63d5a11bd07b-kube-api-access-t79xb" (OuterVolumeSpecName: "kube-api-access-t79xb") pod "7a277419-7d1b-426f-90cd-63d5a11bd07b" (UID: "7a277419-7d1b-426f-90cd-63d5a11bd07b"). InnerVolumeSpecName "kube-api-access-t79xb". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 06 19:04:34.744944 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:04:34.744921 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-7b6544547-bmbw7" Jul 06 19:04:34.816453 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:04:34.816424 2570 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-t79xb\" (UniqueName: \"kubernetes.io/projected/7a277419-7d1b-426f-90cd-63d5a11bd07b-kube-api-access-t79xb\") on node \"ip-10-0-130-59.ec2.internal\" DevicePath \"\"" Jul 06 19:04:34.872897 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:04:34.872821 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-7b6544547-bmbw7"] Jul 06 19:04:34.876176 ip-10-0-130-59 kubenswrapper[2570]: W0706 19:04:34.876150 2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod519d32f5_21dd_4c66_8c71_67d6d724c33a.slice/crio-457f8a23aafe2fefdf073e614e591e37e83a291f58c592db01e9a58689185598 WatchSource:0}: Error finding container 457f8a23aafe2fefdf073e614e591e37e83a291f58c592db01e9a58689185598: Status 404 returned error can't find the container with id 457f8a23aafe2fefdf073e614e591e37e83a291f58c592db01e9a58689185598 Jul 06 19:04:35.583090 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:04:35.583051 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-7b6544547-bmbw7" event={"ID":"519d32f5-21dd-4c66-8c71-67d6d724c33a","Type":"ContainerStarted","Data":"d843720d8e05448abc3e96737e809100bd5bad722bfa6763c4c0096b16e3f47f"} Jul 06 19:04:35.583090 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:04:35.583095 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-7b6544547-bmbw7" event={"ID":"519d32f5-21dd-4c66-8c71-67d6d724c33a","Type":"ContainerStarted","Data":"457f8a23aafe2fefdf073e614e591e37e83a291f58c592db01e9a58689185598"} Jul 06 19:04:35.584498 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:04:35.584477 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5d4b67c446-9g76h" Jul 06 19:04:35.584649 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:04:35.584502 2570 kuberuntime_container.go:872] "Killing container with a grace period" pod="kuadrant-system/authorino-77d5fc7699-sf7rg" podUID="45b95465-8868-4d3c-a57d-ed0dd828b8da" containerName="authorino" containerID="cri-o://09d26680c5ef7759f4d056397c40383dd4d0a7ebc748007d782d998f4929c138" gracePeriod=30 Jul 06 19:04:35.584649 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:04:35.584481 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-77d5fc7699-sf7rg" event={"ID":"45b95465-8868-4d3c-a57d-ed0dd828b8da","Type":"ContainerStarted","Data":"09d26680c5ef7759f4d056397c40383dd4d0a7ebc748007d782d998f4929c138"} Jul 06 19:04:35.599953 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:04:35.599910 2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-7b6544547-bmbw7" podStartSLOduration=1.189044917 podStartE2EDuration="1.599895784s" totalImagesPullingTime="410.850867ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 19:04:34 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 19:04:35.598170947 +0000 UTC m=+555.153192119" watchObservedRunningTime="2026-07-06 19:04:35.599895784 +0000 UTC m=+555.154916956" Jul 06 19:04:35.625453 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:04:35.625304 2570 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-5d4b67c446-9g76h"] Jul 06 19:04:35.628023 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:04:35.628003 2570 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-5d4b67c446-9g76h"] Jul 06 19:04:35.643521 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:04:35.643466 2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-77d5fc7699-sf7rg" podStartSLOduration=0.971426356 podStartE2EDuration="1.643444726s" totalImagesPullingTime="672.01837ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 19:04:34 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 19:04:35.642641522 +0000 UTC m=+555.197662692" watchObservedRunningTime="2026-07-06 19:04:35.643444726 +0000 UTC m=+555.198465900" Jul 06 19:04:35.818638 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:04:35.818598 2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-77d5fc7699-sf7rg" Jul 06 19:04:35.928738 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:04:35.928697 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-6m9s4\" (UniqueName: \"kubernetes.io/projected/45b95465-8868-4d3c-a57d-ed0dd828b8da-kube-api-access-6m9s4\") pod \"45b95465-8868-4d3c-a57d-ed0dd828b8da\" (UID: \"45b95465-8868-4d3c-a57d-ed0dd828b8da\") " Jul 06 19:04:35.930718 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:04:35.930695 2570 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/45b95465-8868-4d3c-a57d-ed0dd828b8da-kube-api-access-6m9s4" (OuterVolumeSpecName: "kube-api-access-6m9s4") pod "45b95465-8868-4d3c-a57d-ed0dd828b8da" (UID: "45b95465-8868-4d3c-a57d-ed0dd828b8da"). InnerVolumeSpecName "kube-api-access-6m9s4". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 06 19:04:36.030036 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:04:36.029994 2570 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-6m9s4\" (UniqueName: \"kubernetes.io/projected/45b95465-8868-4d3c-a57d-ed0dd828b8da-kube-api-access-6m9s4\") on node \"ip-10-0-130-59.ec2.internal\" DevicePath \"\"" Jul 06 19:04:36.590546 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:04:36.590511 2570 generic.go:356] "Generic (PLEG): container finished" podID="45b95465-8868-4d3c-a57d-ed0dd828b8da" containerID="09d26680c5ef7759f4d056397c40383dd4d0a7ebc748007d782d998f4929c138" exitCode=0 Jul 06 19:04:36.590765 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:04:36.590553 2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-77d5fc7699-sf7rg" Jul 06 19:04:36.590765 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:04:36.590592 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-77d5fc7699-sf7rg" event={"ID":"45b95465-8868-4d3c-a57d-ed0dd828b8da","Type":"ContainerDied","Data":"09d26680c5ef7759f4d056397c40383dd4d0a7ebc748007d782d998f4929c138"} Jul 06 19:04:36.590765 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:04:36.590649 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-77d5fc7699-sf7rg" event={"ID":"45b95465-8868-4d3c-a57d-ed0dd828b8da","Type":"ContainerDied","Data":"27b24ab7ce1c63086bb116c6ebe7aa76482625d3187a8d3602018b652f9eb243"} Jul 06 19:04:36.590765 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:04:36.590668 2570 scope.go:117] "RemoveContainer" containerID="09d26680c5ef7759f4d056397c40383dd4d0a7ebc748007d782d998f4929c138" Jul 06 19:04:36.599144 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:04:36.599128 2570 scope.go:117] "RemoveContainer" containerID="09d26680c5ef7759f4d056397c40383dd4d0a7ebc748007d782d998f4929c138" Jul 06 19:04:36.599395 ip-10-0-130-59 kubenswrapper[2570]: E0706 19:04:36.599379 2570 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"09d26680c5ef7759f4d056397c40383dd4d0a7ebc748007d782d998f4929c138\": container with ID starting with 09d26680c5ef7759f4d056397c40383dd4d0a7ebc748007d782d998f4929c138 not found: ID does not exist" containerID="09d26680c5ef7759f4d056397c40383dd4d0a7ebc748007d782d998f4929c138" Jul 06 19:04:36.599442 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:04:36.599403 2570 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"09d26680c5ef7759f4d056397c40383dd4d0a7ebc748007d782d998f4929c138"} err="failed to get container status \"09d26680c5ef7759f4d056397c40383dd4d0a7ebc748007d782d998f4929c138\": rpc error: code = NotFound desc = could not find container \"09d26680c5ef7759f4d056397c40383dd4d0a7ebc748007d782d998f4929c138\": container with ID starting with 09d26680c5ef7759f4d056397c40383dd4d0a7ebc748007d782d998f4929c138 not found: ID does not exist" Jul 06 19:04:36.613424 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:04:36.613400 2570 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-77d5fc7699-sf7rg"] Jul 06 19:04:36.614903 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:04:36.614886 2570 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-77d5fc7699-sf7rg"] Jul 06 19:04:37.028090 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:04:37.028061 2570 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="45b95465-8868-4d3c-a57d-ed0dd828b8da" path="/var/lib/kubelet/pods/45b95465-8868-4d3c-a57d-ed0dd828b8da/volumes" Jul 06 19:04:37.028446 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:04:37.028378 2570 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="7a277419-7d1b-426f-90cd-63d5a11bd07b" path="/var/lib/kubelet/pods/7a277419-7d1b-426f-90cd-63d5a11bd07b/volumes" Jul 06 19:04:37.435756 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:04:37.435728 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k8fbq_bc7057bd-88da-48af-ba32-e6efdc6e8362/konnectivity-agent/0.log" limit=1024 Jul 06 19:04:37.536149 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:04:37.536119 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k8fbq_bc7057bd-88da-48af-ba32-e6efdc6e8362/konnectivity-agent/0.log" limit=1024 Jul 06 19:04:39.925512 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:04:39.925470 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k8fbq_bc7057bd-88da-48af-ba32-e6efdc6e8362/konnectivity-agent/0.log" limit=1024 Jul 06 19:04:40.021245 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:04:40.021209 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k8fbq_bc7057bd-88da-48af-ba32-e6efdc6e8362/konnectivity-agent/0.log" limit=1024 Jul 06 19:04:40.123329 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:04:40.123293 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k8fbq_bc7057bd-88da-48af-ba32-e6efdc6e8362/konnectivity-agent/0.log" limit=1024 Jul 06 19:04:40.239801 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:04:40.239718 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k8fbq_bc7057bd-88da-48af-ba32-e6efdc6e8362/konnectivity-agent/0.log" limit=1024 Jul 06 19:04:43.927756 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:04:43.927720 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k8fbq_bc7057bd-88da-48af-ba32-e6efdc6e8362/konnectivity-agent/0.log" limit=1024 Jul 06 19:04:44.193527 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:04:44.193446 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k8fbq_bc7057bd-88da-48af-ba32-e6efdc6e8362/konnectivity-agent/0.log" limit=1024 Jul 06 19:05:06.971943 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:05:06.971902 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k8fbq_bc7057bd-88da-48af-ba32-e6efdc6e8362/konnectivity-agent/0.log" limit=1024 Jul 06 19:05:15.527340 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:05:15.527303 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k8fbq_bc7057bd-88da-48af-ba32-e6efdc6e8362/konnectivity-agent/0.log" limit=1024 Jul 06 19:05:16.039089 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:05:16.039057 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k8fbq_bc7057bd-88da-48af-ba32-e6efdc6e8362/konnectivity-agent/0.log" limit=1024 Jul 06 19:05:16.255893 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:05:16.255861 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k8fbq_bc7057bd-88da-48af-ba32-e6efdc6e8362/konnectivity-agent/0.log" limit=1024 Jul 06 19:05:16.801550 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:05:16.801519 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k8fbq_bc7057bd-88da-48af-ba32-e6efdc6e8362/konnectivity-agent/0.log" limit=1024 Jul 06 19:05:17.066120 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:05:17.066030 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k8fbq_bc7057bd-88da-48af-ba32-e6efdc6e8362/konnectivity-agent/0.log" limit=1024 Jul 06 19:05:17.199841 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:05:17.199805 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-flg9z"] Jul 06 19:05:27.082465 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:05:27.082428 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k8fbq_bc7057bd-88da-48af-ba32-e6efdc6e8362/konnectivity-agent/0.log" limit=1024 Jul 06 19:05:27.219767 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:05:27.219732 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k8fbq_bc7057bd-88da-48af-ba32-e6efdc6e8362/konnectivity-agent/0.log" limit=1024 Jul 06 19:05:28.663843 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:05:28.663788 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k8fbq_bc7057bd-88da-48af-ba32-e6efdc6e8362/konnectivity-agent/0.log" limit=1024 Jul 06 19:05:28.822387 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:05:28.822352 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k8fbq_bc7057bd-88da-48af-ba32-e6efdc6e8362/konnectivity-agent/0.log" limit=1024 Jul 06 19:05:31.909134 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:05:31.909100 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k8fbq_bc7057bd-88da-48af-ba32-e6efdc6e8362/konnectivity-agent/0.log" limit=1024 Jul 06 19:05:32.292102 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:05:32.292068 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-flg9z"] Jul 06 19:05:36.326104 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:05:36.326073 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k8fbq_bc7057bd-88da-48af-ba32-e6efdc6e8362/konnectivity-agent/0.log" limit=1024 Jul 06 19:05:37.105841 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:05:37.105797 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-flg9z"] Jul 06 19:05:48.885214 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:05:48.885145 2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-598466548d-kcrlh"] Jul 06 19:05:48.885665 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:05:48.885503 2570 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="45b95465-8868-4d3c-a57d-ed0dd828b8da" containerName="authorino" Jul 06 19:05:48.885665 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:05:48.885516 2570 state_mem.go:107] "Deleted CPUSet assignment" podUID="45b95465-8868-4d3c-a57d-ed0dd828b8da" containerName="authorino" Jul 06 19:05:48.885665 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:05:48.885583 2570 memory_manager.go:368] "RemoveStaleState removing state" podUID="45b95465-8868-4d3c-a57d-ed0dd828b8da" containerName="authorino" Jul 06 19:05:48.887932 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:05:48.887916 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-kcrlh" Jul 06 19:05:48.890524 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:05:48.890500 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"kube-root-ca.crt\"" Jul 06 19:05:48.891524 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:05:48.891507 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"openshift-service-ca.crt\"" Jul 06 19:05:48.891657 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:05:48.891507 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-trlp-test-simulated-kserve-self-signed-certs\"" Jul 06 19:05:48.891657 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:05:48.891637 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"default-dockercfg-crqlx\"" Jul 06 19:05:48.900962 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:05:48.900939 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-598466548d-kcrlh"] Jul 06 19:05:48.925072 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:05:48.925046 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k8fbq_bc7057bd-88da-48af-ba32-e6efdc6e8362/konnectivity-agent/0.log" limit=1024 Jul 06 19:05:48.962987 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:05:48.962960 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lmj27\" (UniqueName: \"kubernetes.io/projected/ddd50824-42b6-4a8a-a2bc-6f840eb356cb-kube-api-access-lmj27\") pod \"e2e-trlp-test-simulated-kserve-598466548d-kcrlh\" (UID: \"ddd50824-42b6-4a8a-a2bc-6f840eb356cb\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-kcrlh" Jul 06 19:05:48.963104 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:05:48.962996 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ddd50824-42b6-4a8a-a2bc-6f840eb356cb-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-598466548d-kcrlh\" (UID: \"ddd50824-42b6-4a8a-a2bc-6f840eb356cb\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-kcrlh" Jul 06 19:05:48.963104 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:05:48.963017 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/ddd50824-42b6-4a8a-a2bc-6f840eb356cb-dshm\") pod \"e2e-trlp-test-simulated-kserve-598466548d-kcrlh\" (UID: \"ddd50824-42b6-4a8a-a2bc-6f840eb356cb\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-kcrlh" Jul 06 19:05:48.963104 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:05:48.963039 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ddd50824-42b6-4a8a-a2bc-6f840eb356cb-model-cache\") pod \"e2e-trlp-test-simulated-kserve-598466548d-kcrlh\" (UID: \"ddd50824-42b6-4a8a-a2bc-6f840eb356cb\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-kcrlh" Jul 06 19:05:48.963104 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:05:48.963059 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ddd50824-42b6-4a8a-a2bc-6f840eb356cb-tmp-dir\") pod \"e2e-trlp-test-simulated-kserve-598466548d-kcrlh\" (UID: \"ddd50824-42b6-4a8a-a2bc-6f840eb356cb\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-kcrlh" Jul 06 19:05:48.963104 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:05:48.963102 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/ddd50824-42b6-4a8a-a2bc-6f840eb356cb-home\") pod \"e2e-trlp-test-simulated-kserve-598466548d-kcrlh\" (UID: \"ddd50824-42b6-4a8a-a2bc-6f840eb356cb\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-kcrlh" Jul 06 19:05:49.063660 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:05:49.063626 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lmj27\" (UniqueName: \"kubernetes.io/projected/ddd50824-42b6-4a8a-a2bc-6f840eb356cb-kube-api-access-lmj27\") pod \"e2e-trlp-test-simulated-kserve-598466548d-kcrlh\" (UID: \"ddd50824-42b6-4a8a-a2bc-6f840eb356cb\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-kcrlh" Jul 06 19:05:49.063845 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:05:49.063675 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ddd50824-42b6-4a8a-a2bc-6f840eb356cb-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-598466548d-kcrlh\" (UID: \"ddd50824-42b6-4a8a-a2bc-6f840eb356cb\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-kcrlh" Jul 06 19:05:49.063845 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:05:49.063704 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/ddd50824-42b6-4a8a-a2bc-6f840eb356cb-dshm\") pod \"e2e-trlp-test-simulated-kserve-598466548d-kcrlh\" (UID: \"ddd50824-42b6-4a8a-a2bc-6f840eb356cb\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-kcrlh" Jul 06 19:05:49.063845 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:05:49.063731 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ddd50824-42b6-4a8a-a2bc-6f840eb356cb-model-cache\") pod \"e2e-trlp-test-simulated-kserve-598466548d-kcrlh\" (UID: \"ddd50824-42b6-4a8a-a2bc-6f840eb356cb\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-kcrlh" Jul 06 19:05:49.063845 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:05:49.063752 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ddd50824-42b6-4a8a-a2bc-6f840eb356cb-tmp-dir\") pod \"e2e-trlp-test-simulated-kserve-598466548d-kcrlh\" (UID: \"ddd50824-42b6-4a8a-a2bc-6f840eb356cb\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-kcrlh" Jul 06 19:05:49.064062 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:05:49.063857 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/ddd50824-42b6-4a8a-a2bc-6f840eb356cb-home\") pod \"e2e-trlp-test-simulated-kserve-598466548d-kcrlh\" (UID: \"ddd50824-42b6-4a8a-a2bc-6f840eb356cb\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-kcrlh" Jul 06 19:05:49.064122 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:05:49.064093 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ddd50824-42b6-4a8a-a2bc-6f840eb356cb-model-cache\") pod \"e2e-trlp-test-simulated-kserve-598466548d-kcrlh\" (UID: \"ddd50824-42b6-4a8a-a2bc-6f840eb356cb\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-kcrlh" Jul 06 19:05:49.064220 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:05:49.064200 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ddd50824-42b6-4a8a-a2bc-6f840eb356cb-tmp-dir\") pod \"e2e-trlp-test-simulated-kserve-598466548d-kcrlh\" (UID: \"ddd50824-42b6-4a8a-a2bc-6f840eb356cb\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-kcrlh" Jul 06 19:05:49.064280 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:05:49.064237 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/ddd50824-42b6-4a8a-a2bc-6f840eb356cb-home\") pod \"e2e-trlp-test-simulated-kserve-598466548d-kcrlh\" (UID: \"ddd50824-42b6-4a8a-a2bc-6f840eb356cb\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-kcrlh" Jul 06 19:05:49.066045 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:05:49.066022 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/ddd50824-42b6-4a8a-a2bc-6f840eb356cb-dshm\") pod \"e2e-trlp-test-simulated-kserve-598466548d-kcrlh\" (UID: \"ddd50824-42b6-4a8a-a2bc-6f840eb356cb\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-kcrlh" Jul 06 19:05:49.066226 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:05:49.066211 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ddd50824-42b6-4a8a-a2bc-6f840eb356cb-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-598466548d-kcrlh\" (UID: \"ddd50824-42b6-4a8a-a2bc-6f840eb356cb\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-kcrlh" Jul 06 19:05:49.071729 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:05:49.071705 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-lmj27\" (UniqueName: \"kubernetes.io/projected/ddd50824-42b6-4a8a-a2bc-6f840eb356cb-kube-api-access-lmj27\") pod \"e2e-trlp-test-simulated-kserve-598466548d-kcrlh\" (UID: \"ddd50824-42b6-4a8a-a2bc-6f840eb356cb\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-kcrlh" Jul 06 19:05:49.197693 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:05:49.197666 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-kcrlh" Jul 06 19:05:49.326891 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:05:49.326862 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-598466548d-kcrlh"] Jul 06 19:05:49.328079 ip-10-0-130-59 kubenswrapper[2570]: W0706 19:05:49.328053 2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podddd50824_42b6_4a8a_a2bc_6f840eb356cb.slice/crio-d72ce06a0b47149cc5db96bba20d2b8646b59b0950e6e7709c1659c11b275dee WatchSource:0}: Error finding container d72ce06a0b47149cc5db96bba20d2b8646b59b0950e6e7709c1659c11b275dee: Status 404 returned error can't find the container with id d72ce06a0b47149cc5db96bba20d2b8646b59b0950e6e7709c1659c11b275dee Jul 06 19:05:49.329348 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:05:49.329331 2570 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 06 19:05:49.934724 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:05:49.934644 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-kcrlh" event={"ID":"ddd50824-42b6-4a8a-a2bc-6f840eb356cb","Type":"ContainerStarted","Data":"d72ce06a0b47149cc5db96bba20d2b8646b59b0950e6e7709c1659c11b275dee"} Jul 06 19:05:49.996848 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:05:49.996806 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-flg9z"] Jul 06 19:05:50.940347 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:05:50.940312 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-kcrlh" event={"ID":"ddd50824-42b6-4a8a-a2bc-6f840eb356cb","Type":"ContainerStarted","Data":"1b66893a53db2b8dce973330a842d2a4327b5ea1f0b88c32e7fe06c7766db79d"} Jul 06 19:05:50.959044 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:05:50.959000 2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-kcrlh" podStartSLOduration=1.845055901 podStartE2EDuration="2.95898527s" totalImagesPullingTime="1.113929369s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 19:05:48 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 19:05:50.957878753 +0000 UTC m=+630.512899943" watchObservedRunningTime="2026-07-06 19:05:50.95898527 +0000 UTC m=+630.514006442" Jul 06 19:05:59.198153 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:05:59.198119 2570 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-kcrlh" Jul 06 19:05:59.198634 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:05:59.198187 2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-kcrlh" Jul 06 19:05:59.210596 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:05:59.210572 2570 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-kcrlh" Jul 06 19:05:59.993328 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:05:59.993297 2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-kcrlh" Jul 06 19:06:04.363633 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:06:04.363582 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k8fbq_bc7057bd-88da-48af-ba32-e6efdc6e8362/konnectivity-agent/0.log" limit=1024 Jul 06 19:06:05.707809 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:06:05.707772 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-flg9z"] Jul 06 19:06:18.284179 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:06:18.284147 2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-74b68447df-dw68w"] Jul 06 19:06:18.286971 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:06:18.286952 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-dw68w" Jul 06 19:06:18.294994 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:06:18.294970 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"facebook-opt-125m-simulated-kserve-self-signed-certs\"" Jul 06 19:06:18.304686 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:06:18.304653 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-74b68447df-dw68w"] Jul 06 19:06:18.331274 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:06:18.331242 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k8fbq_bc7057bd-88da-48af-ba32-e6efdc6e8362/konnectivity-agent/0.log" limit=1024 Jul 06 19:06:18.426971 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:06:18.426942 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b7b15020-8cd7-420b-88f6-7658a405a697-dshm\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-dw68w\" (UID: \"b7b15020-8cd7-420b-88f6-7658a405a697\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-dw68w" Jul 06 19:06:18.427132 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:06:18.426984 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b7b15020-8cd7-420b-88f6-7658a405a697-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-dw68w\" (UID: \"b7b15020-8cd7-420b-88f6-7658a405a697\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-dw68w" Jul 06 19:06:18.427132 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:06:18.427056 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b7b15020-8cd7-420b-88f6-7658a405a697-tmp-dir\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-dw68w\" (UID: \"b7b15020-8cd7-420b-88f6-7658a405a697\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-dw68w" Jul 06 19:06:18.427132 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:06:18.427093 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b7b15020-8cd7-420b-88f6-7658a405a697-home\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-dw68w\" (UID: \"b7b15020-8cd7-420b-88f6-7658a405a697\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-dw68w" Jul 06 19:06:18.427287 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:06:18.427148 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b7b15020-8cd7-420b-88f6-7658a405a697-model-cache\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-dw68w\" (UID: \"b7b15020-8cd7-420b-88f6-7658a405a697\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-dw68w" Jul 06 19:06:18.427287 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:06:18.427173 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5dc79\" (UniqueName: \"kubernetes.io/projected/b7b15020-8cd7-420b-88f6-7658a405a697-kube-api-access-5dc79\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-dw68w\" (UID: \"b7b15020-8cd7-420b-88f6-7658a405a697\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-dw68w" Jul 06 19:06:18.528538 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:06:18.528504 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b7b15020-8cd7-420b-88f6-7658a405a697-tmp-dir\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-dw68w\" (UID: \"b7b15020-8cd7-420b-88f6-7658a405a697\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-dw68w" Jul 06 19:06:18.528538 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:06:18.528543 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b7b15020-8cd7-420b-88f6-7658a405a697-home\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-dw68w\" (UID: \"b7b15020-8cd7-420b-88f6-7658a405a697\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-dw68w" Jul 06 19:06:18.528813 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:06:18.528594 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b7b15020-8cd7-420b-88f6-7658a405a697-model-cache\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-dw68w\" (UID: \"b7b15020-8cd7-420b-88f6-7658a405a697\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-dw68w" Jul 06 19:06:18.528813 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:06:18.528643 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5dc79\" (UniqueName: \"kubernetes.io/projected/b7b15020-8cd7-420b-88f6-7658a405a697-kube-api-access-5dc79\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-dw68w\" (UID: \"b7b15020-8cd7-420b-88f6-7658a405a697\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-dw68w" Jul 06 19:06:18.528813 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:06:18.528726 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b7b15020-8cd7-420b-88f6-7658a405a697-dshm\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-dw68w\" (UID: \"b7b15020-8cd7-420b-88f6-7658a405a697\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-dw68w" Jul 06 19:06:18.528813 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:06:18.528764 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b7b15020-8cd7-420b-88f6-7658a405a697-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-dw68w\" (UID: \"b7b15020-8cd7-420b-88f6-7658a405a697\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-dw68w" Jul 06 19:06:18.529009 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:06:18.528994 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b7b15020-8cd7-420b-88f6-7658a405a697-tmp-dir\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-dw68w\" (UID: \"b7b15020-8cd7-420b-88f6-7658a405a697\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-dw68w" Jul 06 19:06:18.529072 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:06:18.529042 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b7b15020-8cd7-420b-88f6-7658a405a697-model-cache\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-dw68w\" (UID: \"b7b15020-8cd7-420b-88f6-7658a405a697\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-dw68w" Jul 06 19:06:18.529135 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:06:18.529111 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b7b15020-8cd7-420b-88f6-7658a405a697-home\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-dw68w\" (UID: \"b7b15020-8cd7-420b-88f6-7658a405a697\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-dw68w" Jul 06 19:06:18.530978 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:06:18.530957 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b7b15020-8cd7-420b-88f6-7658a405a697-dshm\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-dw68w\" (UID: \"b7b15020-8cd7-420b-88f6-7658a405a697\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-dw68w" Jul 06 19:06:18.531143 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:06:18.531124 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b7b15020-8cd7-420b-88f6-7658a405a697-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-dw68w\" (UID: \"b7b15020-8cd7-420b-88f6-7658a405a697\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-dw68w" Jul 06 19:06:18.537876 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:06:18.537820 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-5dc79\" (UniqueName: \"kubernetes.io/projected/b7b15020-8cd7-420b-88f6-7658a405a697-kube-api-access-5dc79\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-dw68w\" (UID: \"b7b15020-8cd7-420b-88f6-7658a405a697\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-dw68w" Jul 06 19:06:18.597880 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:06:18.597851 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-dw68w" Jul 06 19:06:18.729156 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:06:18.729123 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-74b68447df-dw68w"] Jul 06 19:06:18.732284 ip-10-0-130-59 kubenswrapper[2570]: W0706 19:06:18.732259 2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb7b15020_8cd7_420b_88f6_7658a405a697.slice/crio-61de00a2ec0e824727bbc4f0e8533801cec2ed89fce6969bad23e327b1cf23b5 WatchSource:0}: Error finding container 61de00a2ec0e824727bbc4f0e8533801cec2ed89fce6969bad23e327b1cf23b5: Status 404 returned error can't find the container with id 61de00a2ec0e824727bbc4f0e8533801cec2ed89fce6969bad23e327b1cf23b5 Jul 06 19:06:19.068062 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:06:19.068020 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-dw68w" event={"ID":"b7b15020-8cd7-420b-88f6-7658a405a697","Type":"ContainerStarted","Data":"62f34796bcadd30b51701e1b97c731a6d20c6e8caccbcfe0fd448bdad05061ae"} Jul 06 19:06:19.068248 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:06:19.068085 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-dw68w" event={"ID":"b7b15020-8cd7-420b-88f6-7658a405a697","Type":"ContainerStarted","Data":"61de00a2ec0e824727bbc4f0e8533801cec2ed89fce6969bad23e327b1cf23b5"} Jul 06 19:06:19.086604 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:06:19.086557 2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-dw68w" podStartSLOduration=0.897784014 podStartE2EDuration="1.086542321s" totalImagesPullingTime="188.758307ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 19:06:18 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 19:06:19.085064751 +0000 UTC m=+658.640085946" watchObservedRunningTime="2026-07-06 19:06:19.086542321 +0000 UTC m=+658.641563534" Jul 06 19:06:20.503701 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:06:20.503664 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-flg9z"] Jul 06 19:06:26.672155 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:06:26.672124 2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-rjgms"] Jul 06 19:06:26.676129 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:06:26.676108 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-rjgms" Jul 06 19:06:26.678678 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:06:26.678654 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"premium-simulated-simulated-premium-kserve-self-signed-certs\"" Jul 06 19:06:26.684697 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:06:26.684668 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-rjgms"] Jul 06 19:06:26.700290 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:06:26.700260 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-65t2v\" (UniqueName: \"kubernetes.io/projected/b40f5752-a1af-4e5c-a266-d1d863dbed05-kube-api-access-65t2v\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-rjgms\" (UID: \"b40f5752-a1af-4e5c-a266-d1d863dbed05\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-rjgms" Jul 06 19:06:26.700406 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:06:26.700296 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b40f5752-a1af-4e5c-a266-d1d863dbed05-tmp-dir\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-rjgms\" (UID: \"b40f5752-a1af-4e5c-a266-d1d863dbed05\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-rjgms" Jul 06 19:06:26.700459 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:06:26.700440 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b40f5752-a1af-4e5c-a266-d1d863dbed05-dshm\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-rjgms\" (UID: \"b40f5752-a1af-4e5c-a266-d1d863dbed05\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-rjgms" Jul 06 19:06:26.700505 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:06:26.700466 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b40f5752-a1af-4e5c-a266-d1d863dbed05-model-cache\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-rjgms\" (UID: \"b40f5752-a1af-4e5c-a266-d1d863dbed05\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-rjgms" Jul 06 19:06:26.700505 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:06:26.700482 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b40f5752-a1af-4e5c-a266-d1d863dbed05-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-rjgms\" (UID: \"b40f5752-a1af-4e5c-a266-d1d863dbed05\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-rjgms" Jul 06 19:06:26.700505 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:06:26.700501 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b40f5752-a1af-4e5c-a266-d1d863dbed05-home\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-rjgms\" (UID: \"b40f5752-a1af-4e5c-a266-d1d863dbed05\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-rjgms" Jul 06 19:06:26.706879 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:06:26.706852 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k8fbq_bc7057bd-88da-48af-ba32-e6efdc6e8362/konnectivity-agent/0.log" limit=1024 Jul 06 19:06:26.801190 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:06:26.801154 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b40f5752-a1af-4e5c-a266-d1d863dbed05-dshm\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-rjgms\" (UID: \"b40f5752-a1af-4e5c-a266-d1d863dbed05\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-rjgms" Jul 06 19:06:26.801190 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:06:26.801190 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b40f5752-a1af-4e5c-a266-d1d863dbed05-model-cache\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-rjgms\" (UID: \"b40f5752-a1af-4e5c-a266-d1d863dbed05\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-rjgms" Jul 06 19:06:26.801440 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:06:26.801210 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b40f5752-a1af-4e5c-a266-d1d863dbed05-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-rjgms\" (UID: \"b40f5752-a1af-4e5c-a266-d1d863dbed05\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-rjgms" Jul 06 19:06:26.801440 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:06:26.801228 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b40f5752-a1af-4e5c-a266-d1d863dbed05-home\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-rjgms\" (UID: \"b40f5752-a1af-4e5c-a266-d1d863dbed05\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-rjgms" Jul 06 19:06:26.801440 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:06:26.801288 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-65t2v\" (UniqueName: \"kubernetes.io/projected/b40f5752-a1af-4e5c-a266-d1d863dbed05-kube-api-access-65t2v\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-rjgms\" (UID: \"b40f5752-a1af-4e5c-a266-d1d863dbed05\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-rjgms" Jul 06 19:06:26.801440 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:06:26.801309 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b40f5752-a1af-4e5c-a266-d1d863dbed05-tmp-dir\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-rjgms\" (UID: \"b40f5752-a1af-4e5c-a266-d1d863dbed05\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-rjgms" Jul 06 19:06:26.801725 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:06:26.801706 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b40f5752-a1af-4e5c-a266-d1d863dbed05-home\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-rjgms\" (UID: \"b40f5752-a1af-4e5c-a266-d1d863dbed05\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-rjgms" Jul 06 19:06:26.801788 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:06:26.801741 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b40f5752-a1af-4e5c-a266-d1d863dbed05-tmp-dir\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-rjgms\" (UID: \"b40f5752-a1af-4e5c-a266-d1d863dbed05\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-rjgms" Jul 06 19:06:26.801788 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:06:26.801774 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b40f5752-a1af-4e5c-a266-d1d863dbed05-model-cache\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-rjgms\" (UID: \"b40f5752-a1af-4e5c-a266-d1d863dbed05\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-rjgms" Jul 06 19:06:26.803473 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:06:26.803454 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b40f5752-a1af-4e5c-a266-d1d863dbed05-dshm\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-rjgms\" (UID: \"b40f5752-a1af-4e5c-a266-d1d863dbed05\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-rjgms" Jul 06 19:06:26.803742 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:06:26.803725 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b40f5752-a1af-4e5c-a266-d1d863dbed05-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-rjgms\" (UID: \"b40f5752-a1af-4e5c-a266-d1d863dbed05\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-rjgms" Jul 06 19:06:26.809534 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:06:26.809511 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-65t2v\" (UniqueName: \"kubernetes.io/projected/b40f5752-a1af-4e5c-a266-d1d863dbed05-kube-api-access-65t2v\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-rjgms\" (UID: \"b40f5752-a1af-4e5c-a266-d1d863dbed05\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-rjgms" Jul 06 19:06:26.986079 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:06:26.985981 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-rjgms" Jul 06 19:06:27.114533 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:06:27.114486 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-rjgms"] Jul 06 19:06:27.117015 ip-10-0-130-59 kubenswrapper[2570]: W0706 19:06:27.116990 2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podb40f5752_a1af_4e5c_a266_d1d863dbed05.slice/crio-f4b41601370325cc44776c840d64cd53ecdbe823707608b2fb5aea180cdd6cd7 WatchSource:0}: Error finding container f4b41601370325cc44776c840d64cd53ecdbe823707608b2fb5aea180cdd6cd7: Status 404 returned error can't find the container with id f4b41601370325cc44776c840d64cd53ecdbe823707608b2fb5aea180cdd6cd7 Jul 06 19:06:28.109602 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:06:28.109559 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-rjgms" event={"ID":"b40f5752-a1af-4e5c-a266-d1d863dbed05","Type":"ContainerStarted","Data":"5d44bc5fe922579ab72ab7a040300356ce0935dc5dad479bcff158206d13f026"} Jul 06 19:06:28.109995 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:06:28.109628 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-rjgms" event={"ID":"b40f5752-a1af-4e5c-a266-d1d863dbed05","Type":"ContainerStarted","Data":"f4b41601370325cc44776c840d64cd53ecdbe823707608b2fb5aea180cdd6cd7"} Jul 06 19:06:28.129666 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:06:28.129624 2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-rjgms" podStartSLOduration=1.890543005 podStartE2EDuration="2.129591743s" totalImagesPullingTime="239.048738ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 19:06:26 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 19:06:28.127926962 +0000 UTC m=+667.682948160" watchObservedRunningTime="2026-07-06 19:06:28.129591743 +0000 UTC m=+667.684612916" Jul 06 19:06:28.598198 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:06:28.598166 2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-dw68w" Jul 06 19:06:28.598372 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:06:28.598243 2570 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-dw68w" Jul 06 19:06:28.610404 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:06:28.610383 2570 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-dw68w" Jul 06 19:06:29.124833 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:06:29.124806 2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-dw68w" Jul 06 19:06:29.409417 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:06:29.409324 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-flg9z"] Jul 06 19:06:36.986317 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:06:36.986277 2570 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-rjgms" Jul 06 19:06:36.986680 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:06:36.986337 2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-rjgms" Jul 06 19:06:36.998556 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:06:36.998532 2570 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-rjgms" Jul 06 19:06:37.167158 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:06:37.167127 2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-rjgms" Jul 06 19:07:15.753771 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:07:15.753691 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k8fbq_bc7057bd-88da-48af-ba32-e6efdc6e8362/konnectivity-agent/0.log" limit=1024 Jul 06 19:07:30.743169 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:07:30.743134 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k8fbq_bc7057bd-88da-48af-ba32-e6efdc6e8362/konnectivity-agent/0.log" limit=1024 Jul 06 19:07:48.250829 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:07:48.250790 2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-97768fdb-g9kx6"] Jul 06 19:07:48.254329 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:07:48.254307 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-97768fdb-g9kx6" Jul 06 19:07:48.262429 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:07:48.262404 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-97768fdb-g9kx6"] Jul 06 19:07:48.351597 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:07:48.351565 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-w7p8r\" (UniqueName: \"kubernetes.io/projected/642a7ce9-6401-4aef-b8ef-1d70a16ce7bc-kube-api-access-w7p8r\") pod \"authorino-97768fdb-g9kx6\" (UID: \"642a7ce9-6401-4aef-b8ef-1d70a16ce7bc\") " pod="kuadrant-system/authorino-97768fdb-g9kx6" Jul 06 19:07:48.351768 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:07:48.351694 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/642a7ce9-6401-4aef-b8ef-1d70a16ce7bc-tls-cert\") pod \"authorino-97768fdb-g9kx6\" (UID: \"642a7ce9-6401-4aef-b8ef-1d70a16ce7bc\") " pod="kuadrant-system/authorino-97768fdb-g9kx6" Jul 06 19:07:48.452708 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:07:48.452666 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/642a7ce9-6401-4aef-b8ef-1d70a16ce7bc-tls-cert\") pod \"authorino-97768fdb-g9kx6\" (UID: \"642a7ce9-6401-4aef-b8ef-1d70a16ce7bc\") " pod="kuadrant-system/authorino-97768fdb-g9kx6" Jul 06 19:07:48.452915 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:07:48.452738 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-w7p8r\" (UniqueName: \"kubernetes.io/projected/642a7ce9-6401-4aef-b8ef-1d70a16ce7bc-kube-api-access-w7p8r\") pod \"authorino-97768fdb-g9kx6\" (UID: \"642a7ce9-6401-4aef-b8ef-1d70a16ce7bc\") " pod="kuadrant-system/authorino-97768fdb-g9kx6" Jul 06 19:07:48.455192 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:07:48.455169 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/642a7ce9-6401-4aef-b8ef-1d70a16ce7bc-tls-cert\") pod \"authorino-97768fdb-g9kx6\" (UID: \"642a7ce9-6401-4aef-b8ef-1d70a16ce7bc\") " pod="kuadrant-system/authorino-97768fdb-g9kx6" Jul 06 19:07:48.461606 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:07:48.461581 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-w7p8r\" (UniqueName: \"kubernetes.io/projected/642a7ce9-6401-4aef-b8ef-1d70a16ce7bc-kube-api-access-w7p8r\") pod \"authorino-97768fdb-g9kx6\" (UID: \"642a7ce9-6401-4aef-b8ef-1d70a16ce7bc\") " pod="kuadrant-system/authorino-97768fdb-g9kx6" Jul 06 19:07:48.563765 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:07:48.563689 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-97768fdb-g9kx6" Jul 06 19:07:48.694905 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:07:48.694884 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-97768fdb-g9kx6"] Jul 06 19:07:48.697598 ip-10-0-130-59 kubenswrapper[2570]: W0706 19:07:48.697569 2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod642a7ce9_6401_4aef_b8ef_1d70a16ce7bc.slice/crio-2291b6b23657af7919ccab47cb6b06f5e19861c9815bca69584524604ab979ae WatchSource:0}: Error finding container 2291b6b23657af7919ccab47cb6b06f5e19861c9815bca69584524604ab979ae: Status 404 returned error can't find the container with id 2291b6b23657af7919ccab47cb6b06f5e19861c9815bca69584524604ab979ae Jul 06 19:07:49.493516 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:07:49.493429 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-97768fdb-g9kx6" event={"ID":"642a7ce9-6401-4aef-b8ef-1d70a16ce7bc","Type":"ContainerStarted","Data":"e444edae7eec3e504e972e30cd2ae37591c643d17419b8e6eaa0220950cf3e27"} Jul 06 19:07:49.493516 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:07:49.493477 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-97768fdb-g9kx6" event={"ID":"642a7ce9-6401-4aef-b8ef-1d70a16ce7bc","Type":"ContainerStarted","Data":"2291b6b23657af7919ccab47cb6b06f5e19861c9815bca69584524604ab979ae"} Jul 06 19:07:49.523359 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:07:49.523294 2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-97768fdb-g9kx6" podStartSLOduration=1.045412743 podStartE2EDuration="1.523270271s" totalImagesPullingTime="477.857528ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 19:07:48 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 19:07:49.520113839 +0000 UTC m=+749.075135010" watchObservedRunningTime="2026-07-06 19:07:49.523270271 +0000 UTC m=+749.078291447" Jul 06 19:07:49.573634 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:07:49.572888 2570 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-7b6544547-bmbw7"] Jul 06 19:07:49.573634 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:07:49.573083 2570 kuberuntime_container.go:872] "Killing container with a grace period" pod="kuadrant-system/authorino-7b6544547-bmbw7" podUID="519d32f5-21dd-4c66-8c71-67d6d724c33a" containerName="authorino" containerID="cri-o://d843720d8e05448abc3e96737e809100bd5bad722bfa6763c4c0096b16e3f47f" gracePeriod=30 Jul 06 19:07:49.841228 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:07:49.841203 2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-7b6544547-bmbw7" Jul 06 19:07:49.873590 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:07:49.873565 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/519d32f5-21dd-4c66-8c71-67d6d724c33a-tls-cert\") pod \"519d32f5-21dd-4c66-8c71-67d6d724c33a\" (UID: \"519d32f5-21dd-4c66-8c71-67d6d724c33a\") " Jul 06 19:07:49.873788 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:07:49.873699 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-h8tpn\" (UniqueName: \"kubernetes.io/projected/519d32f5-21dd-4c66-8c71-67d6d724c33a-kube-api-access-h8tpn\") pod \"519d32f5-21dd-4c66-8c71-67d6d724c33a\" (UID: \"519d32f5-21dd-4c66-8c71-67d6d724c33a\") " Jul 06 19:07:49.875734 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:07:49.875696 2570 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/519d32f5-21dd-4c66-8c71-67d6d724c33a-kube-api-access-h8tpn" (OuterVolumeSpecName: "kube-api-access-h8tpn") pod "519d32f5-21dd-4c66-8c71-67d6d724c33a" (UID: "519d32f5-21dd-4c66-8c71-67d6d724c33a"). InnerVolumeSpecName "kube-api-access-h8tpn". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 06 19:07:49.885476 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:07:49.885450 2570 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/519d32f5-21dd-4c66-8c71-67d6d724c33a-tls-cert" (OuterVolumeSpecName: "tls-cert") pod "519d32f5-21dd-4c66-8c71-67d6d724c33a" (UID: "519d32f5-21dd-4c66-8c71-67d6d724c33a"). InnerVolumeSpecName "tls-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 06 19:07:49.974772 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:07:49.974735 2570 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-h8tpn\" (UniqueName: \"kubernetes.io/projected/519d32f5-21dd-4c66-8c71-67d6d724c33a-kube-api-access-h8tpn\") on node \"ip-10-0-130-59.ec2.internal\" DevicePath \"\"" Jul 06 19:07:49.974772 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:07:49.974764 2570 reconciler_common.go:299] "Volume detached for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/519d32f5-21dd-4c66-8c71-67d6d724c33a-tls-cert\") on node \"ip-10-0-130-59.ec2.internal\" DevicePath \"\"" Jul 06 19:07:50.437681 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:07:50.437628 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k8fbq_bc7057bd-88da-48af-ba32-e6efdc6e8362/konnectivity-agent/0.log" limit=1024 Jul 06 19:07:50.499443 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:07:50.499409 2570 generic.go:356] "Generic (PLEG): container finished" podID="519d32f5-21dd-4c66-8c71-67d6d724c33a" containerID="d843720d8e05448abc3e96737e809100bd5bad722bfa6763c4c0096b16e3f47f" exitCode=0 Jul 06 19:07:50.499443 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:07:50.499447 2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-7b6544547-bmbw7" Jul 06 19:07:50.499990 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:07:50.499475 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-7b6544547-bmbw7" event={"ID":"519d32f5-21dd-4c66-8c71-67d6d724c33a","Type":"ContainerDied","Data":"d843720d8e05448abc3e96737e809100bd5bad722bfa6763c4c0096b16e3f47f"} Jul 06 19:07:50.499990 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:07:50.499519 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-7b6544547-bmbw7" event={"ID":"519d32f5-21dd-4c66-8c71-67d6d724c33a","Type":"ContainerDied","Data":"457f8a23aafe2fefdf073e614e591e37e83a291f58c592db01e9a58689185598"} Jul 06 19:07:50.499990 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:07:50.499542 2570 scope.go:117] "RemoveContainer" containerID="d843720d8e05448abc3e96737e809100bd5bad722bfa6763c4c0096b16e3f47f" Jul 06 19:07:50.509302 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:07:50.509280 2570 scope.go:117] "RemoveContainer" containerID="d843720d8e05448abc3e96737e809100bd5bad722bfa6763c4c0096b16e3f47f" Jul 06 19:07:50.509640 ip-10-0-130-59 kubenswrapper[2570]: E0706 19:07:50.509594 2570 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d843720d8e05448abc3e96737e809100bd5bad722bfa6763c4c0096b16e3f47f\": container with ID starting with d843720d8e05448abc3e96737e809100bd5bad722bfa6763c4c0096b16e3f47f not found: ID does not exist" containerID="d843720d8e05448abc3e96737e809100bd5bad722bfa6763c4c0096b16e3f47f" Jul 06 19:07:50.509742 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:07:50.509650 2570 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d843720d8e05448abc3e96737e809100bd5bad722bfa6763c4c0096b16e3f47f"} err="failed to get container status \"d843720d8e05448abc3e96737e809100bd5bad722bfa6763c4c0096b16e3f47f\": rpc error: code = NotFound desc = could not find container \"d843720d8e05448abc3e96737e809100bd5bad722bfa6763c4c0096b16e3f47f\": container with ID starting with d843720d8e05448abc3e96737e809100bd5bad722bfa6763c4c0096b16e3f47f not found: ID does not exist" Jul 06 19:07:50.534896 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:07:50.534867 2570 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-7b6544547-bmbw7"] Jul 06 19:07:50.544851 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:07:50.544826 2570 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-7b6544547-bmbw7"] Jul 06 19:07:50.546467 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:07:50.546439 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k8fbq_bc7057bd-88da-48af-ba32-e6efdc6e8362/konnectivity-agent/0.log" limit=1024 Jul 06 19:07:51.029014 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:07:51.028977 2570 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="519d32f5-21dd-4c66-8c71-67d6d724c33a" path="/var/lib/kubelet/pods/519d32f5-21dd-4c66-8c71-67d6d724c33a/volumes" Jul 06 19:07:51.831928 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:07:51.831896 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k8fbq_bc7057bd-88da-48af-ba32-e6efdc6e8362/konnectivity-agent/0.log" limit=1024 Jul 06 19:07:51.926180 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:07:51.926146 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k8fbq_bc7057bd-88da-48af-ba32-e6efdc6e8362/konnectivity-agent/0.log" limit=1024 Jul 06 19:08:21.493681 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:08:21.493646 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-flg9z"] Jul 06 19:08:32.297938 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:08:32.297906 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-flg9z"] Jul 06 19:08:38.427377 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:08:38.427333 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k8fbq_bc7057bd-88da-48af-ba32-e6efdc6e8362/konnectivity-agent/0.log" limit=1024 Jul 06 19:08:38.527953 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:08:38.527917 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k8fbq_bc7057bd-88da-48af-ba32-e6efdc6e8362/konnectivity-agent/0.log" limit=1024 Jul 06 19:08:39.582571 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:08:39.582535 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k8fbq_bc7057bd-88da-48af-ba32-e6efdc6e8362/konnectivity-agent/0.log" limit=1024 Jul 06 19:08:42.305731 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:08:42.305660 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-flg9z"] Jul 06 19:08:52.292535 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:08:52.292498 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-flg9z"] Jul 06 19:09:02.593187 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:09:02.593152 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-flg9z"] Jul 06 19:09:05.355261 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:09:05.355229 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k8fbq_bc7057bd-88da-48af-ba32-e6efdc6e8362/konnectivity-agent/0.log" limit=1024 Jul 06 19:09:07.157103 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:09:07.157072 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k8fbq_bc7057bd-88da-48af-ba32-e6efdc6e8362/konnectivity-agent/0.log" limit=1024 Jul 06 19:09:31.196386 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:09:31.196349 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-flg9z"] Jul 06 19:09:42.198379 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:09:42.198343 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-flg9z"] Jul 06 19:09:50.179569 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:09:50.179538 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k8fbq_bc7057bd-88da-48af-ba32-e6efdc6e8362/konnectivity-agent/0.log" limit=1024 Jul 06 19:09:50.320328 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:09:50.320293 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k8fbq_bc7057bd-88da-48af-ba32-e6efdc6e8362/konnectivity-agent/0.log" limit=1024 Jul 06 19:09:50.454995 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:09:50.454912 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k8fbq_bc7057bd-88da-48af-ba32-e6efdc6e8362/konnectivity-agent/0.log" limit=1024 Jul 06 19:09:57.011862 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:09:57.011823 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k8fbq_bc7057bd-88da-48af-ba32-e6efdc6e8362/konnectivity-agent/0.log" limit=1024 Jul 06 19:09:57.129739 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:09:57.129709 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k8fbq_bc7057bd-88da-48af-ba32-e6efdc6e8362/konnectivity-agent/0.log" limit=1024 Jul 06 19:09:57.233411 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:09:57.233368 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k8fbq_bc7057bd-88da-48af-ba32-e6efdc6e8362/konnectivity-agent/0.log" limit=1024 Jul 06 19:09:57.420750 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:09:57.420724 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k8fbq_bc7057bd-88da-48af-ba32-e6efdc6e8362/konnectivity-agent/0.log" limit=1024 Jul 06 19:10:03.325579 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:10:03.325546 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k8fbq_bc7057bd-88da-48af-ba32-e6efdc6e8362/konnectivity-agent/0.log" limit=1024 Jul 06 19:10:03.435402 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:10:03.435370 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k8fbq_bc7057bd-88da-48af-ba32-e6efdc6e8362/konnectivity-agent/0.log" limit=1024 Jul 06 19:10:20.192911 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:10:20.192821 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-flg9z"] Jul 06 19:10:30.335027 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:10:30.334994 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-flg9z"] Jul 06 19:10:39.111272 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:10:39.111229 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-flg9z"] Jul 06 19:10:48.112192 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:10:48.112147 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-flg9z"] Jul 06 19:10:58.639430 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:10:58.639395 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-flg9z"] Jul 06 19:11:07.810651 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:11:07.810599 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-flg9z"] Jul 06 19:11:19.610197 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:11:19.610162 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-flg9z"] Jul 06 19:11:29.309061 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:11:29.309024 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-flg9z"] Jul 06 19:11:52.011577 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:11:52.011499 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-flg9z"] Jul 06 19:12:01.500829 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:12:01.500792 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-flg9z"] Jul 06 19:12:07.780879 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:12:07.780843 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k8fbq_bc7057bd-88da-48af-ba32-e6efdc6e8362/konnectivity-agent/0.log" limit=1024 Jul 06 19:12:24.990417 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:12:24.990375 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k8fbq_bc7057bd-88da-48af-ba32-e6efdc6e8362/konnectivity-agent/0.log" limit=1024 Jul 06 19:12:26.902639 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:12:26.902595 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-flg9z"] Jul 06 19:12:39.518476 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:12:39.518442 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-flg9z"] Jul 06 19:12:55.323753 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:12:55.323717 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-flg9z"] Jul 06 19:13:05.727450 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:13:05.723825 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-flg9z"] Jul 06 19:13:22.307606 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:13:22.307513 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-flg9z"] Jul 06 19:13:55.270989 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:13:55.270953 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k8fbq_bc7057bd-88da-48af-ba32-e6efdc6e8362/konnectivity-agent/0.log" limit=1024 Jul 06 19:15:24.086767 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:15:24.086732 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-flg9z"] Jul 06 19:15:34.444372 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:15:34.444338 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-flg9z"] Jul 06 19:15:47.032937 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:15:47.032903 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-flg9z"] Jul 06 19:15:58.310063 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:15:58.310026 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-flg9z"] Jul 06 19:16:10.327089 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:16:10.327051 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-flg9z"] Jul 06 19:16:19.533255 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:16:19.533180 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-flg9z"] Jul 06 19:16:30.167582 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:16:30.166529 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-flg9z"] Jul 06 19:16:40.013104 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:16:40.013066 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-flg9z"] Jul 06 19:16:52.392838 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:16:52.392802 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-flg9z"] Jul 06 19:17:01.338718 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:17:01.338680 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-flg9z"] Jul 06 19:17:14.921419 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:17:14.921377 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-flg9z"] Jul 06 19:17:33.324519 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:17:33.324486 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-flg9z"] Jul 06 19:17:38.257424 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:17:38.257391 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k8fbq_bc7057bd-88da-48af-ba32-e6efdc6e8362/konnectivity-agent/0.log" limit=1024 Jul 06 19:17:39.803421 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:17:39.803388 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k8fbq_bc7057bd-88da-48af-ba32-e6efdc6e8362/konnectivity-agent/0.log" limit=1024 Jul 06 19:17:42.239987 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:17:42.239905 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-flg9z"] Jul 06 19:17:52.302803 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:17:52.302762 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-flg9z"] Jul 06 19:18:04.302368 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:18:04.302327 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-flg9z"] Jul 06 19:18:21.141002 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:18:21.140974 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-flg9z"] Jul 06 19:18:31.328111 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:18:31.328080 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-flg9z"] Jul 06 19:18:41.007600 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:18:41.007571 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-flg9z"] Jul 06 19:18:50.926218 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:18:50.926184 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-flg9z"] Jul 06 19:19:00.422530 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:19:00.422492 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-flg9z"] Jul 06 19:19:10.517202 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:19:10.517172 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-flg9z"] Jul 06 19:19:16.738130 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:19:16.738049 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-flg9z"] Jul 06 19:19:31.617391 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:19:31.617356 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-flg9z"] Jul 06 19:19:32.304430 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:19:32.304396 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k8fbq_bc7057bd-88da-48af-ba32-e6efdc6e8362/konnectivity-agent/0.log" limit=1024 Jul 06 19:19:45.310300 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:19:45.310265 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-flg9z"] Jul 06 19:19:55.723570 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:19:55.723536 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-flg9z"] Jul 06 19:20:06.845005 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:20:06.844957 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k8fbq_bc7057bd-88da-48af-ba32-e6efdc6e8362/konnectivity-agent/0.log" limit=1024 Jul 06 19:20:06.946075 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:20:06.946035 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k8fbq_bc7057bd-88da-48af-ba32-e6efdc6e8362/konnectivity-agent/0.log" limit=1024 Jul 06 19:20:09.296512 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:20:09.296474 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-flg9z"] Jul 06 19:20:16.941220 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:20:16.941189 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-flg9z"] Jul 06 19:20:28.440438 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:20:28.440397 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-flg9z"] Jul 06 19:20:40.339735 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:20:40.339699 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-flg9z"] Jul 06 19:20:50.773381 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:20:50.773306 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-flg9z"] Jul 06 19:20:53.119640 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:20:53.119586 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k8fbq_bc7057bd-88da-48af-ba32-e6efdc6e8362/konnectivity-agent/0.log" limit=1024 Jul 06 19:21:00.697672 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:21:00.697634 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k8fbq_bc7057bd-88da-48af-ba32-e6efdc6e8362/konnectivity-agent/0.log" limit=1024 Jul 06 19:21:04.507120 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:21:04.507090 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-flg9z"] Jul 06 19:21:09.018084 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:21:09.018054 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k8fbq_bc7057bd-88da-48af-ba32-e6efdc6e8362/konnectivity-agent/0.log" limit=1024 Jul 06 19:21:13.211881 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:21:13.211844 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-flg9z"] Jul 06 19:21:14.262896 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:21:14.262858 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k8fbq_bc7057bd-88da-48af-ba32-e6efdc6e8362/konnectivity-agent/0.log" limit=1024 Jul 06 19:21:17.280147 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:21:17.280114 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k8fbq_bc7057bd-88da-48af-ba32-e6efdc6e8362/konnectivity-agent/0.log" limit=1024 Jul 06 19:21:17.474656 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:21:17.474600 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k8fbq_bc7057bd-88da-48af-ba32-e6efdc6e8362/konnectivity-agent/0.log" limit=1024 Jul 06 19:21:17.803577 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:21:17.803539 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k8fbq_bc7057bd-88da-48af-ba32-e6efdc6e8362/konnectivity-agent/0.log" limit=1024 Jul 06 19:21:18.003985 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:21:18.003945 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k8fbq_bc7057bd-88da-48af-ba32-e6efdc6e8362/konnectivity-agent/0.log" limit=1024 Jul 06 19:21:22.622814 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:21:22.622781 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-flg9z"] Jul 06 19:21:24.588544 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:21:24.588511 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k8fbq_bc7057bd-88da-48af-ba32-e6efdc6e8362/konnectivity-agent/0.log" limit=1024 Jul 06 19:21:24.714788 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:21:24.714752 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k8fbq_bc7057bd-88da-48af-ba32-e6efdc6e8362/konnectivity-agent/0.log" limit=1024 Jul 06 19:21:24.945547 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:21:24.945513 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k8fbq_bc7057bd-88da-48af-ba32-e6efdc6e8362/konnectivity-agent/0.log" limit=1024 Jul 06 19:21:25.106033 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:21:25.105949 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k8fbq_bc7057bd-88da-48af-ba32-e6efdc6e8362/konnectivity-agent/0.log" limit=1024 Jul 06 19:21:26.158558 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:21:26.158526 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k8fbq_bc7057bd-88da-48af-ba32-e6efdc6e8362/konnectivity-agent/0.log" limit=1024 Jul 06 19:21:26.487046 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:21:26.487014 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k8fbq_bc7057bd-88da-48af-ba32-e6efdc6e8362/konnectivity-agent/0.log" limit=1024 Jul 06 19:21:26.731335 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:21:26.731303 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k8fbq_bc7057bd-88da-48af-ba32-e6efdc6e8362/konnectivity-agent/0.log" limit=1024 Jul 06 19:21:26.940151 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:21:26.940123 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k8fbq_bc7057bd-88da-48af-ba32-e6efdc6e8362/konnectivity-agent/0.log" limit=1024 Jul 06 19:21:27.326998 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:21:27.326895 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k8fbq_bc7057bd-88da-48af-ba32-e6efdc6e8362/konnectivity-agent/0.log" limit=1024 Jul 06 19:21:27.444662 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:21:27.444602 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k8fbq_bc7057bd-88da-48af-ba32-e6efdc6e8362/konnectivity-agent/0.log" limit=1024 Jul 06 19:21:28.830494 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:21:28.830464 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k8fbq_bc7057bd-88da-48af-ba32-e6efdc6e8362/konnectivity-agent/0.log" limit=1024 Jul 06 19:21:28.966071 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:21:28.966022 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k8fbq_bc7057bd-88da-48af-ba32-e6efdc6e8362/konnectivity-agent/0.log" limit=1024 Jul 06 19:21:31.228950 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:21:31.228903 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k8fbq_bc7057bd-88da-48af-ba32-e6efdc6e8362/konnectivity-agent/0.log" limit=1024 Jul 06 19:21:31.451361 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:21:31.451332 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k8fbq_bc7057bd-88da-48af-ba32-e6efdc6e8362/konnectivity-agent/0.log" limit=1024 Jul 06 19:21:31.565945 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:21:31.565873 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k8fbq_bc7057bd-88da-48af-ba32-e6efdc6e8362/konnectivity-agent/0.log" limit=1024 Jul 06 19:21:31.784040 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:21:31.784002 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k8fbq_bc7057bd-88da-48af-ba32-e6efdc6e8362/konnectivity-agent/0.log" limit=1024 Jul 06 19:21:33.395673 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:21:33.395603 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k8fbq_bc7057bd-88da-48af-ba32-e6efdc6e8362/konnectivity-agent/0.log" limit=1024 Jul 06 19:21:38.062582 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:21:38.062541 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k8fbq_bc7057bd-88da-48af-ba32-e6efdc6e8362/konnectivity-agent/0.log" limit=1024 Jul 06 19:21:38.170659 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:21:38.170623 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k8fbq_bc7057bd-88da-48af-ba32-e6efdc6e8362/konnectivity-agent/0.log" limit=1024 Jul 06 19:21:38.549702 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:21:38.549668 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k8fbq_bc7057bd-88da-48af-ba32-e6efdc6e8362/konnectivity-agent/0.log" limit=1024 Jul 06 19:21:38.892127 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:21:38.892097 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k8fbq_bc7057bd-88da-48af-ba32-e6efdc6e8362/konnectivity-agent/0.log" limit=1024 Jul 06 19:21:39.696947 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:21:39.696913 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k8fbq_bc7057bd-88da-48af-ba32-e6efdc6e8362/konnectivity-agent/0.log" limit=1024 Jul 06 19:21:39.878760 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:21:39.878727 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k8fbq_bc7057bd-88da-48af-ba32-e6efdc6e8362/konnectivity-agent/0.log" limit=1024 Jul 06 19:21:40.034471 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:21:40.034396 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k8fbq_bc7057bd-88da-48af-ba32-e6efdc6e8362/konnectivity-agent/0.log" limit=1024 Jul 06 19:21:40.362812 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:21:40.362729 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k8fbq_bc7057bd-88da-48af-ba32-e6efdc6e8362/konnectivity-agent/0.log" limit=1024 Jul 06 19:21:40.632677 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:21:40.632562 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k8fbq_bc7057bd-88da-48af-ba32-e6efdc6e8362/konnectivity-agent/0.log" limit=1024 Jul 06 19:21:40.740461 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:21:40.740427 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k8fbq_bc7057bd-88da-48af-ba32-e6efdc6e8362/konnectivity-agent/0.log" limit=1024 Jul 06 19:21:41.875983 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:21:41.875950 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k8fbq_bc7057bd-88da-48af-ba32-e6efdc6e8362/konnectivity-agent/0.log" limit=1024 Jul 06 19:21:42.718668 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:21:42.718632 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-flg9z"] Jul 06 19:21:44.755321 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:21:44.755288 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k8fbq_bc7057bd-88da-48af-ba32-e6efdc6e8362/konnectivity-agent/0.log" limit=1024 Jul 06 19:21:45.253245 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:21:45.253199 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k8fbq_bc7057bd-88da-48af-ba32-e6efdc6e8362/konnectivity-agent/0.log" limit=1024 Jul 06 19:21:50.916992 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:21:50.916959 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k8fbq_bc7057bd-88da-48af-ba32-e6efdc6e8362/konnectivity-agent/0.log" limit=1024 Jul 06 19:21:51.017919 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:21:51.017885 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k8fbq_bc7057bd-88da-48af-ba32-e6efdc6e8362/konnectivity-agent/0.log" limit=1024 Jul 06 19:21:53.007388 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:21:53.007353 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-flg9z"] Jul 06 19:21:54.159400 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:21:54.159363 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k8fbq_bc7057bd-88da-48af-ba32-e6efdc6e8362/konnectivity-agent/0.log" limit=1024 Jul 06 19:21:57.364663 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:21:57.364636 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k8fbq_bc7057bd-88da-48af-ba32-e6efdc6e8362/konnectivity-agent/0.log" limit=1024 Jul 06 19:21:57.463063 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:21:57.463026 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k8fbq_bc7057bd-88da-48af-ba32-e6efdc6e8362/konnectivity-agent/0.log" limit=1024 Jul 06 19:21:57.632714 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:21:57.632638 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k8fbq_bc7057bd-88da-48af-ba32-e6efdc6e8362/konnectivity-agent/0.log" limit=1024 Jul 06 19:21:57.733562 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:21:57.733529 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k8fbq_bc7057bd-88da-48af-ba32-e6efdc6e8362/konnectivity-agent/0.log" limit=1024 Jul 06 19:21:58.852171 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:21:58.852132 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k8fbq_bc7057bd-88da-48af-ba32-e6efdc6e8362/konnectivity-agent/0.log" limit=1024 Jul 06 19:21:58.953761 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:21:58.953729 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k8fbq_bc7057bd-88da-48af-ba32-e6efdc6e8362/konnectivity-agent/0.log" limit=1024 Jul 06 19:21:59.214831 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:21:59.214804 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k8fbq_bc7057bd-88da-48af-ba32-e6efdc6e8362/konnectivity-agent/0.log" limit=1024 Jul 06 19:21:59.386773 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:21:59.386731 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k8fbq_bc7057bd-88da-48af-ba32-e6efdc6e8362/konnectivity-agent/0.log" limit=1024 Jul 06 19:21:59.899935 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:21:59.899897 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k8fbq_bc7057bd-88da-48af-ba32-e6efdc6e8362/konnectivity-agent/0.log" limit=1024 Jul 06 19:22:00.002512 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:22:00.002472 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k8fbq_bc7057bd-88da-48af-ba32-e6efdc6e8362/konnectivity-agent/0.log" limit=1024 Jul 06 19:22:01.106737 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:22:01.106568 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k8fbq_bc7057bd-88da-48af-ba32-e6efdc6e8362/konnectivity-agent/0.log" limit=1024 Jul 06 19:22:02.104982 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:22:02.104942 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-flg9z"] Jul 06 19:22:03.442093 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:22:03.442056 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k8fbq_bc7057bd-88da-48af-ba32-e6efdc6e8362/konnectivity-agent/0.log" limit=1024 Jul 06 19:22:03.614466 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:22:03.614435 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k8fbq_bc7057bd-88da-48af-ba32-e6efdc6e8362/konnectivity-agent/0.log" limit=1024 Jul 06 19:22:03.707258 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:22:03.707178 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k8fbq_bc7057bd-88da-48af-ba32-e6efdc6e8362/konnectivity-agent/0.log" limit=1024 Jul 06 19:22:04.040486 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:22:04.040397 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k8fbq_bc7057bd-88da-48af-ba32-e6efdc6e8362/konnectivity-agent/0.log" limit=1024 Jul 06 19:22:05.579713 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:22:05.579680 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k8fbq_bc7057bd-88da-48af-ba32-e6efdc6e8362/konnectivity-agent/0.log" limit=1024 Jul 06 19:22:10.124836 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:22:10.124798 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k8fbq_bc7057bd-88da-48af-ba32-e6efdc6e8362/konnectivity-agent/0.log" limit=1024 Jul 06 19:22:10.215488 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:22:10.215459 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k8fbq_bc7057bd-88da-48af-ba32-e6efdc6e8362/konnectivity-agent/0.log" limit=1024 Jul 06 19:22:10.682097 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:22:10.682069 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k8fbq_bc7057bd-88da-48af-ba32-e6efdc6e8362/konnectivity-agent/0.log" limit=1024 Jul 06 19:22:10.855646 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:22:10.855605 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k8fbq_bc7057bd-88da-48af-ba32-e6efdc6e8362/konnectivity-agent/0.log" limit=1024 Jul 06 19:22:10.920998 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:22:10.920964 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-flg9z"] Jul 06 19:22:12.477836 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:22:12.477737 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k8fbq_bc7057bd-88da-48af-ba32-e6efdc6e8362/konnectivity-agent/0.log" limit=1024 Jul 06 19:22:18.738890 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:22:18.738854 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k8fbq_bc7057bd-88da-48af-ba32-e6efdc6e8362/konnectivity-agent/0.log" limit=1024 Jul 06 19:22:18.829023 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:22:18.828980 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k8fbq_bc7057bd-88da-48af-ba32-e6efdc6e8362/konnectivity-agent/0.log" limit=1024 Jul 06 19:22:18.940168 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:22:18.940130 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k8fbq_bc7057bd-88da-48af-ba32-e6efdc6e8362/konnectivity-agent/0.log" limit=1024 Jul 06 19:22:19.052882 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:22:19.052801 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k8fbq_bc7057bd-88da-48af-ba32-e6efdc6e8362/konnectivity-agent/0.log" limit=1024 Jul 06 19:22:19.541568 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:22:19.541530 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k8fbq_bc7057bd-88da-48af-ba32-e6efdc6e8362/konnectivity-agent/0.log" limit=1024 Jul 06 19:22:45.583176 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:22:45.583135 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k8fbq_bc7057bd-88da-48af-ba32-e6efdc6e8362/konnectivity-agent/0.log" limit=1024 Jul 06 19:22:52.085093 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:22:52.085052 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k8fbq_bc7057bd-88da-48af-ba32-e6efdc6e8362/konnectivity-agent/0.log" limit=1024 Jul 06 19:22:52.224845 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:22:52.224807 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k8fbq_bc7057bd-88da-48af-ba32-e6efdc6e8362/konnectivity-agent/0.log" limit=1024 Jul 06 19:22:52.397004 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:22:52.396913 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k8fbq_bc7057bd-88da-48af-ba32-e6efdc6e8362/konnectivity-agent/0.log" limit=1024 Jul 06 19:23:09.779473 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:23:09.779438 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k8fbq_bc7057bd-88da-48af-ba32-e6efdc6e8362/konnectivity-agent/0.log" limit=1024 Jul 06 19:23:16.186033 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:23:16.185988 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k8fbq_bc7057bd-88da-48af-ba32-e6efdc6e8362/konnectivity-agent/0.log" limit=1024 Jul 06 19:23:16.304539 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:23:16.304506 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k8fbq_bc7057bd-88da-48af-ba32-e6efdc6e8362/konnectivity-agent/0.log" limit=1024 Jul 06 19:23:16.452670 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:23:16.452549 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k8fbq_bc7057bd-88da-48af-ba32-e6efdc6e8362/konnectivity-agent/0.log" limit=1024 Jul 06 19:23:27.130546 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:23:27.130512 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k8fbq_bc7057bd-88da-48af-ba32-e6efdc6e8362/konnectivity-agent/0.log" limit=1024 Jul 06 19:23:28.415903 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:23:28.415867 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k8fbq_bc7057bd-88da-48af-ba32-e6efdc6e8362/konnectivity-agent/0.log" limit=1024 Jul 06 19:23:36.281033 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:23:36.280999 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k8fbq_bc7057bd-88da-48af-ba32-e6efdc6e8362/konnectivity-agent/0.log" limit=1024 Jul 06 19:23:36.495176 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:23:36.495138 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k8fbq_bc7057bd-88da-48af-ba32-e6efdc6e8362/konnectivity-agent/0.log" limit=1024 Jul 06 19:23:36.676804 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:23:36.676781 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k8fbq_bc7057bd-88da-48af-ba32-e6efdc6e8362/konnectivity-agent/0.log" limit=1024 Jul 06 19:23:37.566493 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:23:37.566457 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k8fbq_bc7057bd-88da-48af-ba32-e6efdc6e8362/konnectivity-agent/0.log" limit=1024 Jul 06 19:23:37.769875 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:23:37.769847 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k8fbq_bc7057bd-88da-48af-ba32-e6efdc6e8362/konnectivity-agent/0.log" limit=1024 Jul 06 19:23:38.823971 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:23:38.823936 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-flg9z"] Jul 06 19:23:39.696098 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:23:39.696067 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k8fbq_bc7057bd-88da-48af-ba32-e6efdc6e8362/konnectivity-agent/0.log" limit=1024 Jul 06 19:23:46.441243 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:23:46.441145 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k8fbq_bc7057bd-88da-48af-ba32-e6efdc6e8362/konnectivity-agent/0.log" limit=1024 Jul 06 19:23:46.523588 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:23:46.523557 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-flg9z"] Jul 06 19:23:46.550390 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:23:46.550365 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k8fbq_bc7057bd-88da-48af-ba32-e6efdc6e8362/konnectivity-agent/0.log" limit=1024 Jul 06 19:23:47.341220 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:23:47.341181 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k8fbq_bc7057bd-88da-48af-ba32-e6efdc6e8362/konnectivity-agent/0.log" limit=1024 Jul 06 19:23:54.197693 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:23:54.197659 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k8fbq_bc7057bd-88da-48af-ba32-e6efdc6e8362/konnectivity-agent/0.log" limit=1024 Jul 06 19:23:54.305998 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:23:54.305957 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k8fbq_bc7057bd-88da-48af-ba32-e6efdc6e8362/konnectivity-agent/0.log" limit=1024 Jul 06 19:23:54.312741 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:23:54.312706 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-flg9z"] Jul 06 19:23:54.581757 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:23:54.581662 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k8fbq_bc7057bd-88da-48af-ba32-e6efdc6e8362/konnectivity-agent/0.log" limit=1024 Jul 06 19:23:54.964273 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:23:54.964239 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k8fbq_bc7057bd-88da-48af-ba32-e6efdc6e8362/konnectivity-agent/0.log" limit=1024 Jul 06 19:23:55.083699 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:23:55.083668 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k8fbq_bc7057bd-88da-48af-ba32-e6efdc6e8362/konnectivity-agent/0.log" limit=1024 Jul 06 19:23:57.977579 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:23:57.977538 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k8fbq_bc7057bd-88da-48af-ba32-e6efdc6e8362/konnectivity-agent/0.log" limit=1024 Jul 06 19:23:58.170526 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:23:58.170488 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k8fbq_bc7057bd-88da-48af-ba32-e6efdc6e8362/konnectivity-agent/0.log" limit=1024 Jul 06 19:23:58.313869 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:23:58.313778 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k8fbq_bc7057bd-88da-48af-ba32-e6efdc6e8362/konnectivity-agent/0.log" limit=1024 Jul 06 19:23:58.517720 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:23:58.517681 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k8fbq_bc7057bd-88da-48af-ba32-e6efdc6e8362/konnectivity-agent/0.log" limit=1024 Jul 06 19:23:58.943956 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:23:58.943921 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k8fbq_bc7057bd-88da-48af-ba32-e6efdc6e8362/konnectivity-agent/0.log" limit=1024 Jul 06 19:23:59.100375 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:23:59.100346 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k8fbq_bc7057bd-88da-48af-ba32-e6efdc6e8362/konnectivity-agent/0.log" limit=1024 Jul 06 19:24:01.545168 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:24:01.545135 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-flg9z"] Jul 06 19:24:02.438409 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:24:02.438377 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k8fbq_bc7057bd-88da-48af-ba32-e6efdc6e8362/konnectivity-agent/0.log" limit=1024 Jul 06 19:24:03.214862 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:24:03.214830 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k8fbq_bc7057bd-88da-48af-ba32-e6efdc6e8362/konnectivity-agent/0.log" limit=1024 Jul 06 19:24:03.360080 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:24:03.360046 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k8fbq_bc7057bd-88da-48af-ba32-e6efdc6e8362/konnectivity-agent/0.log" limit=1024 Jul 06 19:24:03.691923 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:24:03.691894 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k8fbq_bc7057bd-88da-48af-ba32-e6efdc6e8362/konnectivity-agent/0.log" limit=1024 Jul 06 19:24:09.961154 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:24:09.961114 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k8fbq_bc7057bd-88da-48af-ba32-e6efdc6e8362/konnectivity-agent/0.log" limit=1024 Jul 06 19:24:10.059120 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:24:10.059082 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k8fbq_bc7057bd-88da-48af-ba32-e6efdc6e8362/konnectivity-agent/0.log" limit=1024 Jul 06 19:24:10.629182 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:24:10.629147 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k8fbq_bc7057bd-88da-48af-ba32-e6efdc6e8362/konnectivity-agent/0.log" limit=1024 Jul 06 19:24:10.941198 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:24:10.941166 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k8fbq_bc7057bd-88da-48af-ba32-e6efdc6e8362/konnectivity-agent/0.log" limit=1024 Jul 06 19:24:11.108116 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:24:11.108079 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k8fbq_bc7057bd-88da-48af-ba32-e6efdc6e8362/konnectivity-agent/0.log" limit=1024 Jul 06 19:24:14.329974 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:24:14.329934 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-flg9z"] Jul 06 19:24:17.445213 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:24:17.445182 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k8fbq_bc7057bd-88da-48af-ba32-e6efdc6e8362/konnectivity-agent/0.log" limit=1024 Jul 06 19:24:17.677323 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:24:17.677290 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k8fbq_bc7057bd-88da-48af-ba32-e6efdc6e8362/konnectivity-agent/0.log" limit=1024 Jul 06 19:24:17.809759 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:24:17.809675 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k8fbq_bc7057bd-88da-48af-ba32-e6efdc6e8362/konnectivity-agent/0.log" limit=1024 Jul 06 19:24:18.067262 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:24:18.067163 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k8fbq_bc7057bd-88da-48af-ba32-e6efdc6e8362/konnectivity-agent/0.log" limit=1024 Jul 06 19:24:18.293663 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:24:18.293624 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k8fbq_bc7057bd-88da-48af-ba32-e6efdc6e8362/konnectivity-agent/0.log" limit=1024 Jul 06 19:24:18.557045 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:24:18.557006 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k8fbq_bc7057bd-88da-48af-ba32-e6efdc6e8362/konnectivity-agent/0.log" limit=1024 Jul 06 19:24:18.703418 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:24:18.703388 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k8fbq_bc7057bd-88da-48af-ba32-e6efdc6e8362/konnectivity-agent/0.log" limit=1024 Jul 06 19:24:24.924145 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:24:24.924111 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k8fbq_bc7057bd-88da-48af-ba32-e6efdc6e8362/konnectivity-agent/0.log" limit=1024 Jul 06 19:24:25.061204 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:24:25.061164 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k8fbq_bc7057bd-88da-48af-ba32-e6efdc6e8362/konnectivity-agent/0.log" limit=1024 Jul 06 19:24:25.199040 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:24:25.198923 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k8fbq_bc7057bd-88da-48af-ba32-e6efdc6e8362/konnectivity-agent/0.log" limit=1024 Jul 06 19:24:25.307187 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:24:25.307151 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k8fbq_bc7057bd-88da-48af-ba32-e6efdc6e8362/konnectivity-agent/0.log" limit=1024 Jul 06 19:24:29.816688 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:24:29.816650 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k8fbq_bc7057bd-88da-48af-ba32-e6efdc6e8362/konnectivity-agent/0.log" limit=1024 Jul 06 19:24:29.915745 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:24:29.915713 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k8fbq_bc7057bd-88da-48af-ba32-e6efdc6e8362/konnectivity-agent/0.log" limit=1024 Jul 06 19:24:31.373318 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:24:31.373283 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k8fbq_bc7057bd-88da-48af-ba32-e6efdc6e8362/konnectivity-agent/0.log" limit=1024 Jul 06 19:24:31.550863 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:24:31.550829 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k8fbq_bc7057bd-88da-48af-ba32-e6efdc6e8362/konnectivity-agent/0.log" limit=1024 Jul 06 19:24:38.368702 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:24:38.368669 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k8fbq_bc7057bd-88da-48af-ba32-e6efdc6e8362/konnectivity-agent/0.log" limit=1024 Jul 06 19:24:38.876898 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:24:38.876856 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k8fbq_bc7057bd-88da-48af-ba32-e6efdc6e8362/konnectivity-agent/0.log" limit=1024 Jul 06 19:24:39.075018 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:24:39.074978 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k8fbq_bc7057bd-88da-48af-ba32-e6efdc6e8362/konnectivity-agent/0.log" limit=1024 Jul 06 19:24:42.081987 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:24:42.081945 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k8fbq_bc7057bd-88da-48af-ba32-e6efdc6e8362/konnectivity-agent/0.log" limit=1024 Jul 06 19:24:42.366767 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:24:42.366688 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k8fbq_bc7057bd-88da-48af-ba32-e6efdc6e8362/konnectivity-agent/0.log" limit=1024 Jul 06 19:24:42.518593 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:24:42.518549 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k8fbq_bc7057bd-88da-48af-ba32-e6efdc6e8362/konnectivity-agent/0.log" limit=1024 Jul 06 19:24:42.871158 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:24:42.871124 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k8fbq_bc7057bd-88da-48af-ba32-e6efdc6e8362/konnectivity-agent/0.log" limit=1024 Jul 06 19:24:43.303309 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:24:43.303271 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k8fbq_bc7057bd-88da-48af-ba32-e6efdc6e8362/konnectivity-agent/0.log" limit=1024 Jul 06 19:24:43.403177 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:24:43.403146 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k8fbq_bc7057bd-88da-48af-ba32-e6efdc6e8362/konnectivity-agent/0.log" limit=1024 Jul 06 19:24:46.457062 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:24:46.457028 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k8fbq_bc7057bd-88da-48af-ba32-e6efdc6e8362/konnectivity-agent/0.log" limit=1024 Jul 06 19:24:47.808019 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:24:47.807983 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-flg9z"] Jul 06 19:24:51.699635 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:24:51.699571 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k8fbq_bc7057bd-88da-48af-ba32-e6efdc6e8362/konnectivity-agent/0.log" limit=1024 Jul 06 19:24:51.801727 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:24:51.801690 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k8fbq_bc7057bd-88da-48af-ba32-e6efdc6e8362/konnectivity-agent/0.log" limit=1024 Jul 06 19:24:54.314121 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:24:54.314091 2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-mt-4f73e900/e2e-lifecycle-model-4f73e900-kserve-86b778d9c9-smg7b"] Jul 06 19:24:54.314485 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:24:54.314470 2570 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="519d32f5-21dd-4c66-8c71-67d6d724c33a" containerName="authorino" Jul 06 19:24:54.314485 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:24:54.314483 2570 state_mem.go:107] "Deleted CPUSet assignment" podUID="519d32f5-21dd-4c66-8c71-67d6d724c33a" containerName="authorino" Jul 06 19:24:54.314571 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:24:54.314560 2570 memory_manager.go:368] "RemoveStaleState removing state" podUID="519d32f5-21dd-4c66-8c71-67d6d724c33a" containerName="authorino" Jul 06 19:24:54.317422 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:24:54.317404 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-4f73e900/e2e-lifecycle-model-4f73e900-kserve-86b778d9c9-smg7b" Jul 06 19:24:54.320108 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:24:54.320085 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-mt-4f73e900\"/\"e2e-lifecycle-model-4f73e900-kserve-self-signed-certs\"" Jul 06 19:24:54.320991 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:24:54.320974 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-mt-4f73e900\"/\"kube-root-ca.crt\"" Jul 06 19:24:54.321056 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:24:54.320974 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-mt-4f73e900\"/\"default-dockercfg-dcb94\"" Jul 06 19:24:54.321056 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:24:54.321028 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-mt-4f73e900\"/\"openshift-service-ca.crt\"" Jul 06 19:24:54.329083 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:24:54.329063 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-mt-4f73e900/e2e-lifecycle-model-4f73e900-kserve-86b778d9c9-smg7b"] Jul 06 19:24:54.339475 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:24:54.339441 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k8fbq_bc7057bd-88da-48af-ba32-e6efdc6e8362/konnectivity-agent/0.log" limit=1024 Jul 06 19:24:54.444332 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:24:54.444300 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/fac9fc63-4edf-4fb2-acb4-42f811eb15aa-tmp-dir\") pod \"e2e-lifecycle-model-4f73e900-kserve-86b778d9c9-smg7b\" (UID: \"fac9fc63-4edf-4fb2-acb4-42f811eb15aa\") " pod="ai-tenant-e2e-mt-4f73e900/e2e-lifecycle-model-4f73e900-kserve-86b778d9c9-smg7b" Jul 06 19:24:54.444332 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:24:54.444344 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/fac9fc63-4edf-4fb2-acb4-42f811eb15aa-model-cache\") pod \"e2e-lifecycle-model-4f73e900-kserve-86b778d9c9-smg7b\" (UID: \"fac9fc63-4edf-4fb2-acb4-42f811eb15aa\") " pod="ai-tenant-e2e-mt-4f73e900/e2e-lifecycle-model-4f73e900-kserve-86b778d9c9-smg7b" Jul 06 19:24:54.444542 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:24:54.444361 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/fac9fc63-4edf-4fb2-acb4-42f811eb15aa-home\") pod \"e2e-lifecycle-model-4f73e900-kserve-86b778d9c9-smg7b\" (UID: \"fac9fc63-4edf-4fb2-acb4-42f811eb15aa\") " pod="ai-tenant-e2e-mt-4f73e900/e2e-lifecycle-model-4f73e900-kserve-86b778d9c9-smg7b" Jul 06 19:24:54.444542 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:24:54.444431 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/fac9fc63-4edf-4fb2-acb4-42f811eb15aa-tls-certs\") pod \"e2e-lifecycle-model-4f73e900-kserve-86b778d9c9-smg7b\" (UID: \"fac9fc63-4edf-4fb2-acb4-42f811eb15aa\") " pod="ai-tenant-e2e-mt-4f73e900/e2e-lifecycle-model-4f73e900-kserve-86b778d9c9-smg7b" Jul 06 19:24:54.444542 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:24:54.444490 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-q6gcl\" (UniqueName: \"kubernetes.io/projected/fac9fc63-4edf-4fb2-acb4-42f811eb15aa-kube-api-access-q6gcl\") pod \"e2e-lifecycle-model-4f73e900-kserve-86b778d9c9-smg7b\" (UID: \"fac9fc63-4edf-4fb2-acb4-42f811eb15aa\") " pod="ai-tenant-e2e-mt-4f73e900/e2e-lifecycle-model-4f73e900-kserve-86b778d9c9-smg7b" Jul 06 19:24:54.444671 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:24:54.444541 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/fac9fc63-4edf-4fb2-acb4-42f811eb15aa-dshm\") pod \"e2e-lifecycle-model-4f73e900-kserve-86b778d9c9-smg7b\" (UID: \"fac9fc63-4edf-4fb2-acb4-42f811eb15aa\") " pod="ai-tenant-e2e-mt-4f73e900/e2e-lifecycle-model-4f73e900-kserve-86b778d9c9-smg7b" Jul 06 19:24:54.545576 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:24:54.545538 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/fac9fc63-4edf-4fb2-acb4-42f811eb15aa-tmp-dir\") pod \"e2e-lifecycle-model-4f73e900-kserve-86b778d9c9-smg7b\" (UID: \"fac9fc63-4edf-4fb2-acb4-42f811eb15aa\") " pod="ai-tenant-e2e-mt-4f73e900/e2e-lifecycle-model-4f73e900-kserve-86b778d9c9-smg7b" Jul 06 19:24:54.545787 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:24:54.545585 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/fac9fc63-4edf-4fb2-acb4-42f811eb15aa-model-cache\") pod \"e2e-lifecycle-model-4f73e900-kserve-86b778d9c9-smg7b\" (UID: \"fac9fc63-4edf-4fb2-acb4-42f811eb15aa\") " pod="ai-tenant-e2e-mt-4f73e900/e2e-lifecycle-model-4f73e900-kserve-86b778d9c9-smg7b" Jul 06 19:24:54.545787 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:24:54.545604 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/fac9fc63-4edf-4fb2-acb4-42f811eb15aa-home\") pod \"e2e-lifecycle-model-4f73e900-kserve-86b778d9c9-smg7b\" (UID: \"fac9fc63-4edf-4fb2-acb4-42f811eb15aa\") " pod="ai-tenant-e2e-mt-4f73e900/e2e-lifecycle-model-4f73e900-kserve-86b778d9c9-smg7b" Jul 06 19:24:54.545787 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:24:54.545671 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/fac9fc63-4edf-4fb2-acb4-42f811eb15aa-tls-certs\") pod \"e2e-lifecycle-model-4f73e900-kserve-86b778d9c9-smg7b\" (UID: \"fac9fc63-4edf-4fb2-acb4-42f811eb15aa\") " pod="ai-tenant-e2e-mt-4f73e900/e2e-lifecycle-model-4f73e900-kserve-86b778d9c9-smg7b" Jul 06 19:24:54.545787 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:24:54.545714 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-q6gcl\" (UniqueName: \"kubernetes.io/projected/fac9fc63-4edf-4fb2-acb4-42f811eb15aa-kube-api-access-q6gcl\") pod \"e2e-lifecycle-model-4f73e900-kserve-86b778d9c9-smg7b\" (UID: \"fac9fc63-4edf-4fb2-acb4-42f811eb15aa\") " pod="ai-tenant-e2e-mt-4f73e900/e2e-lifecycle-model-4f73e900-kserve-86b778d9c9-smg7b" Jul 06 19:24:54.545787 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:24:54.545766 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/fac9fc63-4edf-4fb2-acb4-42f811eb15aa-dshm\") pod \"e2e-lifecycle-model-4f73e900-kserve-86b778d9c9-smg7b\" (UID: \"fac9fc63-4edf-4fb2-acb4-42f811eb15aa\") " pod="ai-tenant-e2e-mt-4f73e900/e2e-lifecycle-model-4f73e900-kserve-86b778d9c9-smg7b" Jul 06 19:24:54.546059 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:24:54.545977 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/fac9fc63-4edf-4fb2-acb4-42f811eb15aa-tmp-dir\") pod \"e2e-lifecycle-model-4f73e900-kserve-86b778d9c9-smg7b\" (UID: \"fac9fc63-4edf-4fb2-acb4-42f811eb15aa\") " pod="ai-tenant-e2e-mt-4f73e900/e2e-lifecycle-model-4f73e900-kserve-86b778d9c9-smg7b" Jul 06 19:24:54.546059 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:24:54.546031 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/fac9fc63-4edf-4fb2-acb4-42f811eb15aa-model-cache\") pod \"e2e-lifecycle-model-4f73e900-kserve-86b778d9c9-smg7b\" (UID: \"fac9fc63-4edf-4fb2-acb4-42f811eb15aa\") " pod="ai-tenant-e2e-mt-4f73e900/e2e-lifecycle-model-4f73e900-kserve-86b778d9c9-smg7b" Jul 06 19:24:54.546166 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:24:54.546079 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/fac9fc63-4edf-4fb2-acb4-42f811eb15aa-home\") pod \"e2e-lifecycle-model-4f73e900-kserve-86b778d9c9-smg7b\" (UID: \"fac9fc63-4edf-4fb2-acb4-42f811eb15aa\") " pod="ai-tenant-e2e-mt-4f73e900/e2e-lifecycle-model-4f73e900-kserve-86b778d9c9-smg7b" Jul 06 19:24:54.547978 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:24:54.547954 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/fac9fc63-4edf-4fb2-acb4-42f811eb15aa-dshm\") pod \"e2e-lifecycle-model-4f73e900-kserve-86b778d9c9-smg7b\" (UID: \"fac9fc63-4edf-4fb2-acb4-42f811eb15aa\") " pod="ai-tenant-e2e-mt-4f73e900/e2e-lifecycle-model-4f73e900-kserve-86b778d9c9-smg7b" Jul 06 19:24:54.548214 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:24:54.548197 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/fac9fc63-4edf-4fb2-acb4-42f811eb15aa-tls-certs\") pod \"e2e-lifecycle-model-4f73e900-kserve-86b778d9c9-smg7b\" (UID: \"fac9fc63-4edf-4fb2-acb4-42f811eb15aa\") " pod="ai-tenant-e2e-mt-4f73e900/e2e-lifecycle-model-4f73e900-kserve-86b778d9c9-smg7b" Jul 06 19:24:54.556216 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:24:54.556195 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-q6gcl\" (UniqueName: \"kubernetes.io/projected/fac9fc63-4edf-4fb2-acb4-42f811eb15aa-kube-api-access-q6gcl\") pod \"e2e-lifecycle-model-4f73e900-kserve-86b778d9c9-smg7b\" (UID: \"fac9fc63-4edf-4fb2-acb4-42f811eb15aa\") " pod="ai-tenant-e2e-mt-4f73e900/e2e-lifecycle-model-4f73e900-kserve-86b778d9c9-smg7b" Jul 06 19:24:54.627285 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:24:54.627194 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-4f73e900/e2e-lifecycle-model-4f73e900-kserve-86b778d9c9-smg7b" Jul 06 19:24:54.760533 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:24:54.760508 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-mt-4f73e900/e2e-lifecycle-model-4f73e900-kserve-86b778d9c9-smg7b"] Jul 06 19:24:54.762834 ip-10-0-130-59 kubenswrapper[2570]: W0706 19:24:54.762803 2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podfac9fc63_4edf_4fb2_acb4_42f811eb15aa.slice/crio-b5c01c2c88e48bfa92f130a1fd0b53a16088dcdb575fe624188d57eee9eafcfd WatchSource:0}: Error finding container b5c01c2c88e48bfa92f130a1fd0b53a16088dcdb575fe624188d57eee9eafcfd: Status 404 returned error can't find the container with id b5c01c2c88e48bfa92f130a1fd0b53a16088dcdb575fe624188d57eee9eafcfd Jul 06 19:24:55.396864 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:24:55.396827 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-4f73e900/e2e-lifecycle-model-4f73e900-kserve-86b778d9c9-smg7b" event={"ID":"fac9fc63-4edf-4fb2-acb4-42f811eb15aa","Type":"ContainerStarted","Data":"44989b8792744142c41eba5514c79e7fe654f169edc6769fc04e0992639c010d"} Jul 06 19:24:55.397237 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:24:55.396873 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-4f73e900/e2e-lifecycle-model-4f73e900-kserve-86b778d9c9-smg7b" event={"ID":"fac9fc63-4edf-4fb2-acb4-42f811eb15aa","Type":"ContainerStarted","Data":"b5c01c2c88e48bfa92f130a1fd0b53a16088dcdb575fe624188d57eee9eafcfd"} Jul 06 19:24:55.422870 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:24:55.422819 2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-mt-4f73e900/e2e-lifecycle-model-4f73e900-kserve-86b778d9c9-smg7b" podStartSLOduration=1.422802403 podStartE2EDuration="1.422802403s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 19:24:54 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 19:24:55.422638656 +0000 UTC m=+1774.977659820" watchObservedRunningTime="2026-07-06 19:24:55.422802403 +0000 UTC m=+1774.977823603" Jul 06 19:24:59.603388 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:24:59.603352 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-flg9z"] Jul 06 19:25:04.627893 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:25:04.627845 2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-mt-4f73e900/e2e-lifecycle-model-4f73e900-kserve-86b778d9c9-smg7b" Jul 06 19:25:04.627893 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:25:04.627912 2570 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-mt-4f73e900/e2e-lifecycle-model-4f73e900-kserve-86b778d9c9-smg7b" Jul 06 19:25:04.640390 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:25:04.640359 2570 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-mt-4f73e900/e2e-lifecycle-model-4f73e900-kserve-86b778d9c9-smg7b" Jul 06 19:25:05.453847 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:25:05.453815 2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-mt-4f73e900/e2e-lifecycle-model-4f73e900-kserve-86b778d9c9-smg7b" Jul 06 19:25:07.981266 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:25:07.981237 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k8fbq_bc7057bd-88da-48af-ba32-e6efdc6e8362/konnectivity-agent/0.log" limit=1024 Jul 06 19:25:08.166883 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:25:08.166844 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k8fbq_bc7057bd-88da-48af-ba32-e6efdc6e8362/konnectivity-agent/0.log" limit=1024 Jul 06 19:25:11.422953 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:25:11.422920 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-flg9z"] Jul 06 19:25:14.188386 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:25:14.188299 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k8fbq_bc7057bd-88da-48af-ba32-e6efdc6e8362/konnectivity-agent/0.log" limit=1024 Jul 06 19:25:15.142090 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:25:15.142050 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k8fbq_bc7057bd-88da-48af-ba32-e6efdc6e8362/konnectivity-agent/0.log" limit=1024 Jul 06 19:25:20.358196 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:25:20.358159 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k8fbq_bc7057bd-88da-48af-ba32-e6efdc6e8362/konnectivity-agent/0.log" limit=1024 Jul 06 19:25:25.287197 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:25:25.287162 2570 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-mt-4f73e900/e2e-lifecycle-model-4f73e900-kserve-86b778d9c9-smg7b"] Jul 06 19:25:25.385826 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:25:25.385785 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k8fbq_bc7057bd-88da-48af-ba32-e6efdc6e8362/konnectivity-agent/0.log" limit=1024 Jul 06 19:25:26.232604 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:25:26.232565 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-flg9z"] Jul 06 19:25:40.307722 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:25:40.307663 2570 kuberuntime_container.go:872] "Killing container with a grace period" pod="ai-tenant-e2e-mt-4f73e900/e2e-lifecycle-model-4f73e900-kserve-86b778d9c9-smg7b" podUID="fac9fc63-4edf-4fb2-acb4-42f811eb15aa" containerName="main" containerID="cri-o://44989b8792744142c41eba5514c79e7fe654f169edc6769fc04e0992639c010d" gracePeriod=45 Jul 06 19:25:40.549068 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:25:40.549045 2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-4f73e900/e2e-lifecycle-model-4f73e900-kserve-86b778d9c9-smg7b" Jul 06 19:25:40.606544 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:25:40.606450 2570 generic.go:356] "Generic (PLEG): container finished" podID="fac9fc63-4edf-4fb2-acb4-42f811eb15aa" containerID="44989b8792744142c41eba5514c79e7fe654f169edc6769fc04e0992639c010d" exitCode=0 Jul 06 19:25:40.606544 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:25:40.606518 2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-4f73e900/e2e-lifecycle-model-4f73e900-kserve-86b778d9c9-smg7b" Jul 06 19:25:40.606759 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:25:40.606532 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-4f73e900/e2e-lifecycle-model-4f73e900-kserve-86b778d9c9-smg7b" event={"ID":"fac9fc63-4edf-4fb2-acb4-42f811eb15aa","Type":"ContainerDied","Data":"44989b8792744142c41eba5514c79e7fe654f169edc6769fc04e0992639c010d"} Jul 06 19:25:40.606759 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:25:40.606599 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-4f73e900/e2e-lifecycle-model-4f73e900-kserve-86b778d9c9-smg7b" event={"ID":"fac9fc63-4edf-4fb2-acb4-42f811eb15aa","Type":"ContainerDied","Data":"b5c01c2c88e48bfa92f130a1fd0b53a16088dcdb575fe624188d57eee9eafcfd"} Jul 06 19:25:40.606759 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:25:40.606648 2570 scope.go:117] "RemoveContainer" containerID="44989b8792744142c41eba5514c79e7fe654f169edc6769fc04e0992639c010d" Jul 06 19:25:40.616640 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:25:40.616605 2570 scope.go:117] "RemoveContainer" containerID="44989b8792744142c41eba5514c79e7fe654f169edc6769fc04e0992639c010d" Jul 06 19:25:40.616928 ip-10-0-130-59 kubenswrapper[2570]: E0706 19:25:40.616900 2570 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"44989b8792744142c41eba5514c79e7fe654f169edc6769fc04e0992639c010d\": container with ID starting with 44989b8792744142c41eba5514c79e7fe654f169edc6769fc04e0992639c010d not found: ID does not exist" containerID="44989b8792744142c41eba5514c79e7fe654f169edc6769fc04e0992639c010d" Jul 06 19:25:40.617000 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:25:40.616936 2570 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"44989b8792744142c41eba5514c79e7fe654f169edc6769fc04e0992639c010d"} err="failed to get container status \"44989b8792744142c41eba5514c79e7fe654f169edc6769fc04e0992639c010d\": rpc error: code = NotFound desc = could not find container \"44989b8792744142c41eba5514c79e7fe654f169edc6769fc04e0992639c010d\": container with ID starting with 44989b8792744142c41eba5514c79e7fe654f169edc6769fc04e0992639c010d not found: ID does not exist" Jul 06 19:25:40.668155 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:25:40.668131 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/fac9fc63-4edf-4fb2-acb4-42f811eb15aa-home\") pod \"fac9fc63-4edf-4fb2-acb4-42f811eb15aa\" (UID: \"fac9fc63-4edf-4fb2-acb4-42f811eb15aa\") " Jul 06 19:25:40.668155 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:25:40.668159 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/fac9fc63-4edf-4fb2-acb4-42f811eb15aa-dshm\") pod \"fac9fc63-4edf-4fb2-acb4-42f811eb15aa\" (UID: \"fac9fc63-4edf-4fb2-acb4-42f811eb15aa\") " Jul 06 19:25:40.668344 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:25:40.668217 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-q6gcl\" (UniqueName: \"kubernetes.io/projected/fac9fc63-4edf-4fb2-acb4-42f811eb15aa-kube-api-access-q6gcl\") pod \"fac9fc63-4edf-4fb2-acb4-42f811eb15aa\" (UID: \"fac9fc63-4edf-4fb2-acb4-42f811eb15aa\") " Jul 06 19:25:40.668344 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:25:40.668243 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/fac9fc63-4edf-4fb2-acb4-42f811eb15aa-model-cache\") pod \"fac9fc63-4edf-4fb2-acb4-42f811eb15aa\" (UID: \"fac9fc63-4edf-4fb2-acb4-42f811eb15aa\") " Jul 06 19:25:40.668344 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:25:40.668297 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/fac9fc63-4edf-4fb2-acb4-42f811eb15aa-tmp-dir\") pod \"fac9fc63-4edf-4fb2-acb4-42f811eb15aa\" (UID: \"fac9fc63-4edf-4fb2-acb4-42f811eb15aa\") " Jul 06 19:25:40.668492 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:25:40.668343 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/fac9fc63-4edf-4fb2-acb4-42f811eb15aa-tls-certs\") pod \"fac9fc63-4edf-4fb2-acb4-42f811eb15aa\" (UID: \"fac9fc63-4edf-4fb2-acb4-42f811eb15aa\") " Jul 06 19:25:40.668492 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:25:40.668439 2570 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fac9fc63-4edf-4fb2-acb4-42f811eb15aa-home" (OuterVolumeSpecName: "home") pod "fac9fc63-4edf-4fb2-acb4-42f811eb15aa" (UID: "fac9fc63-4edf-4fb2-acb4-42f811eb15aa"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 19:25:40.668600 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:25:40.668523 2570 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fac9fc63-4edf-4fb2-acb4-42f811eb15aa-model-cache" (OuterVolumeSpecName: "model-cache") pod "fac9fc63-4edf-4fb2-acb4-42f811eb15aa" (UID: "fac9fc63-4edf-4fb2-acb4-42f811eb15aa"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 19:25:40.668697 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:25:40.668605 2570 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fac9fc63-4edf-4fb2-acb4-42f811eb15aa-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "fac9fc63-4edf-4fb2-acb4-42f811eb15aa" (UID: "fac9fc63-4edf-4fb2-acb4-42f811eb15aa"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 19:25:40.668760 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:25:40.668738 2570 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/fac9fc63-4edf-4fb2-acb4-42f811eb15aa-home\") on node \"ip-10-0-130-59.ec2.internal\" DevicePath \"\"" Jul 06 19:25:40.668807 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:25:40.668756 2570 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/fac9fc63-4edf-4fb2-acb4-42f811eb15aa-model-cache\") on node \"ip-10-0-130-59.ec2.internal\" DevicePath \"\"" Jul 06 19:25:40.668807 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:25:40.668773 2570 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/fac9fc63-4edf-4fb2-acb4-42f811eb15aa-tmp-dir\") on node \"ip-10-0-130-59.ec2.internal\" DevicePath \"\"" Jul 06 19:25:40.670483 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:25:40.670457 2570 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fac9fc63-4edf-4fb2-acb4-42f811eb15aa-dshm" (OuterVolumeSpecName: "dshm") pod "fac9fc63-4edf-4fb2-acb4-42f811eb15aa" (UID: "fac9fc63-4edf-4fb2-acb4-42f811eb15aa"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 19:25:40.670585 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:25:40.670478 2570 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/fac9fc63-4edf-4fb2-acb4-42f811eb15aa-kube-api-access-q6gcl" (OuterVolumeSpecName: "kube-api-access-q6gcl") pod "fac9fc63-4edf-4fb2-acb4-42f811eb15aa" (UID: "fac9fc63-4edf-4fb2-acb4-42f811eb15aa"). InnerVolumeSpecName "kube-api-access-q6gcl". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 06 19:25:40.670585 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:25:40.670493 2570 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/fac9fc63-4edf-4fb2-acb4-42f811eb15aa-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "fac9fc63-4edf-4fb2-acb4-42f811eb15aa" (UID: "fac9fc63-4edf-4fb2-acb4-42f811eb15aa"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 06 19:25:40.770178 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:25:40.770144 2570 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/fac9fc63-4edf-4fb2-acb4-42f811eb15aa-dshm\") on node \"ip-10-0-130-59.ec2.internal\" DevicePath \"\"" Jul 06 19:25:40.770178 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:25:40.770172 2570 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-q6gcl\" (UniqueName: \"kubernetes.io/projected/fac9fc63-4edf-4fb2-acb4-42f811eb15aa-kube-api-access-q6gcl\") on node \"ip-10-0-130-59.ec2.internal\" DevicePath \"\"" Jul 06 19:25:40.770178 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:25:40.770182 2570 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/fac9fc63-4edf-4fb2-acb4-42f811eb15aa-tls-certs\") on node \"ip-10-0-130-59.ec2.internal\" DevicePath \"\"" Jul 06 19:25:40.941984 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:25:40.941947 2570 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-mt-4f73e900/e2e-lifecycle-model-4f73e900-kserve-86b778d9c9-smg7b"] Jul 06 19:25:40.948875 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:25:40.948848 2570 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-mt-4f73e900/e2e-lifecycle-model-4f73e900-kserve-86b778d9c9-smg7b"] Jul 06 19:25:41.028493 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:25:41.028458 2570 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="fac9fc63-4edf-4fb2-acb4-42f811eb15aa" path="/var/lib/kubelet/pods/fac9fc63-4edf-4fb2-acb4-42f811eb15aa/volumes" Jul 06 19:26:02.392300 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:26:02.392261 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k8fbq_bc7057bd-88da-48af-ba32-e6efdc6e8362/konnectivity-agent/0.log" limit=1024 Jul 06 19:26:04.582042 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:26:04.582007 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k8fbq_bc7057bd-88da-48af-ba32-e6efdc6e8362/konnectivity-agent/0.log" limit=1024 Jul 06 19:26:05.227128 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:26:05.227095 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-flg9z"] Jul 06 19:26:12.172977 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:26:12.172938 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k8fbq_bc7057bd-88da-48af-ba32-e6efdc6e8362/konnectivity-agent/0.log" limit=1024 Jul 06 19:26:30.743203 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:26:30.743168 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k8fbq_bc7057bd-88da-48af-ba32-e6efdc6e8362/konnectivity-agent/0.log" limit=1024 Jul 06 19:26:30.933316 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:26:30.933278 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k8fbq_bc7057bd-88da-48af-ba32-e6efdc6e8362/konnectivity-agent/0.log" limit=1024 Jul 06 19:26:31.276087 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:26:31.276049 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k8fbq_bc7057bd-88da-48af-ba32-e6efdc6e8362/konnectivity-agent/0.log" limit=1024 Jul 06 19:26:31.566096 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:26:31.566014 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k8fbq_bc7057bd-88da-48af-ba32-e6efdc6e8362/konnectivity-agent/0.log" limit=1024 Jul 06 19:26:32.942601 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:26:32.942567 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k8fbq_bc7057bd-88da-48af-ba32-e6efdc6e8362/konnectivity-agent/0.log" limit=1024 Jul 06 19:26:33.390046 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:26:33.389926 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k8fbq_bc7057bd-88da-48af-ba32-e6efdc6e8362/konnectivity-agent/0.log" limit=1024 Jul 06 19:26:34.419524 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:26:34.419486 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k8fbq_bc7057bd-88da-48af-ba32-e6efdc6e8362/konnectivity-agent/0.log" limit=1024 Jul 06 19:26:34.841726 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:26:34.841686 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k8fbq_bc7057bd-88da-48af-ba32-e6efdc6e8362/konnectivity-agent/0.log" limit=1024 Jul 06 19:26:37.457242 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:26:37.457210 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-flg9z"] Jul 06 19:26:38.596954 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:26:38.596923 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k8fbq_bc7057bd-88da-48af-ba32-e6efdc6e8362/konnectivity-agent/0.log" limit=1024 Jul 06 19:26:46.232389 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:26:46.232280 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k8fbq_bc7057bd-88da-48af-ba32-e6efdc6e8362/konnectivity-agent/0.log" limit=1024 Jul 06 19:26:46.440186 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:26:46.440141 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k8fbq_bc7057bd-88da-48af-ba32-e6efdc6e8362/konnectivity-agent/0.log" limit=1024 Jul 06 19:26:46.840894 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:26:46.840854 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k8fbq_bc7057bd-88da-48af-ba32-e6efdc6e8362/konnectivity-agent/0.log" limit=1024 Jul 06 19:26:49.493356 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:26:49.493328 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-flg9z"] Jul 06 19:26:57.343938 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:26:57.343896 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k8fbq_bc7057bd-88da-48af-ba32-e6efdc6e8362/konnectivity-agent/0.log" limit=1024 Jul 06 19:27:00.823302 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:27:00.823269 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k8fbq_bc7057bd-88da-48af-ba32-e6efdc6e8362/konnectivity-agent/0.log" limit=1024 Jul 06 19:27:00.968591 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:27:00.968549 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k8fbq_bc7057bd-88da-48af-ba32-e6efdc6e8362/konnectivity-agent/0.log" limit=1024 Jul 06 19:27:01.131155 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:27:01.131049 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k8fbq_bc7057bd-88da-48af-ba32-e6efdc6e8362/konnectivity-agent/0.log" limit=1024 Jul 06 19:27:06.757468 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:27:06.757430 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k8fbq_bc7057bd-88da-48af-ba32-e6efdc6e8362/konnectivity-agent/0.log" limit=1024 Jul 06 19:27:11.750695 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:27:11.750651 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k8fbq_bc7057bd-88da-48af-ba32-e6efdc6e8362/konnectivity-agent/0.log" limit=1024 Jul 06 19:27:12.085537 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:27:12.085441 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k8fbq_bc7057bd-88da-48af-ba32-e6efdc6e8362/konnectivity-agent/0.log" limit=1024 Jul 06 19:27:33.095545 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:27:33.095510 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k8fbq_bc7057bd-88da-48af-ba32-e6efdc6e8362/konnectivity-agent/0.log" limit=1024 Jul 06 19:27:39.838856 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:27:39.838816 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k8fbq_bc7057bd-88da-48af-ba32-e6efdc6e8362/konnectivity-agent/0.log" limit=1024 Jul 06 19:27:39.988284 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:27:39.988245 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k8fbq_bc7057bd-88da-48af-ba32-e6efdc6e8362/konnectivity-agent/0.log" limit=1024 Jul 06 19:27:40.247831 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:27:40.247789 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k8fbq_bc7057bd-88da-48af-ba32-e6efdc6e8362/konnectivity-agent/0.log" limit=1024 Jul 06 19:27:40.889175 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:27:40.889132 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k8fbq_bc7057bd-88da-48af-ba32-e6efdc6e8362/konnectivity-agent/0.log" limit=1024 Jul 06 19:27:41.159862 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:27:41.159771 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k8fbq_bc7057bd-88da-48af-ba32-e6efdc6e8362/konnectivity-agent/0.log" limit=1024 Jul 06 19:27:44.089417 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:27:44.089380 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k8fbq_bc7057bd-88da-48af-ba32-e6efdc6e8362/konnectivity-agent/0.log" limit=1024 Jul 06 19:27:44.359994 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:27:44.359906 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k8fbq_bc7057bd-88da-48af-ba32-e6efdc6e8362/konnectivity-agent/0.log" limit=1024 Jul 06 19:27:45.068550 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:27:45.068513 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k8fbq_bc7057bd-88da-48af-ba32-e6efdc6e8362/konnectivity-agent/0.log" limit=1024 Jul 06 19:27:45.440053 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:27:45.440021 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k8fbq_bc7057bd-88da-48af-ba32-e6efdc6e8362/konnectivity-agent/0.log" limit=1024 Jul 06 19:27:45.694693 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:27:45.694600 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k8fbq_bc7057bd-88da-48af-ba32-e6efdc6e8362/konnectivity-agent/0.log" limit=1024 Jul 06 19:27:48.657920 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:27:48.657885 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k8fbq_bc7057bd-88da-48af-ba32-e6efdc6e8362/konnectivity-agent/0.log" limit=1024 Jul 06 19:27:50.081402 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:27:50.081369 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k8fbq_bc7057bd-88da-48af-ba32-e6efdc6e8362/konnectivity-agent/0.log" limit=1024 Jul 06 19:27:51.825894 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:27:51.825859 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-flg9z"] Jul 06 19:27:58.681341 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:27:58.681306 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k8fbq_bc7057bd-88da-48af-ba32-e6efdc6e8362/konnectivity-agent/0.log" limit=1024 Jul 06 19:28:05.032318 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:28:05.032283 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k8fbq_bc7057bd-88da-48af-ba32-e6efdc6e8362/konnectivity-agent/0.log" limit=1024 Jul 06 19:28:22.036026 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:28:22.035940 2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-inf-a-88c5f6/test-model-88c5f6-kserve-549bf8b49-8fk7k"] Jul 06 19:28:22.036577 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:28:22.036506 2570 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="fac9fc63-4edf-4fb2-acb4-42f811eb15aa" containerName="main" Jul 06 19:28:22.036577 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:28:22.036528 2570 state_mem.go:107] "Deleted CPUSet assignment" podUID="fac9fc63-4edf-4fb2-acb4-42f811eb15aa" containerName="main" Jul 06 19:28:22.036762 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:28:22.036741 2570 memory_manager.go:368] "RemoveStaleState removing state" podUID="fac9fc63-4edf-4fb2-acb4-42f811eb15aa" containerName="main" Jul 06 19:28:22.039809 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:28:22.039788 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-88c5f6/test-model-88c5f6-kserve-549bf8b49-8fk7k" Jul 06 19:28:22.054433 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:28:22.054416 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-a-88c5f6\"/\"kube-root-ca.crt\"" Jul 06 19:28:22.055160 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:28:22.055141 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-a-88c5f6\"/\"openshift-service-ca.crt\"" Jul 06 19:28:22.055951 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:28:22.055932 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-a-88c5f6\"/\"test-model-88c5f6-kserve-self-signed-certs\"" Jul 06 19:28:22.056047 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:28:22.055956 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-a-88c5f6\"/\"default-dockercfg-n99f5\"" Jul 06 19:28:22.114245 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:28:22.114222 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mv9tp\" (UniqueName: \"kubernetes.io/projected/cad66095-2da5-4270-87e3-7a84ae09f7be-kube-api-access-mv9tp\") pod \"test-model-88c5f6-kserve-549bf8b49-8fk7k\" (UID: \"cad66095-2da5-4270-87e3-7a84ae09f7be\") " pod="ai-tenant-e2e-inf-a-88c5f6/test-model-88c5f6-kserve-549bf8b49-8fk7k" Jul 06 19:28:22.114338 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:28:22.114252 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/cad66095-2da5-4270-87e3-7a84ae09f7be-home\") pod \"test-model-88c5f6-kserve-549bf8b49-8fk7k\" (UID: \"cad66095-2da5-4270-87e3-7a84ae09f7be\") " pod="ai-tenant-e2e-inf-a-88c5f6/test-model-88c5f6-kserve-549bf8b49-8fk7k" Jul 06 19:28:22.114385 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:28:22.114341 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/cad66095-2da5-4270-87e3-7a84ae09f7be-model-cache\") pod \"test-model-88c5f6-kserve-549bf8b49-8fk7k\" (UID: \"cad66095-2da5-4270-87e3-7a84ae09f7be\") " pod="ai-tenant-e2e-inf-a-88c5f6/test-model-88c5f6-kserve-549bf8b49-8fk7k" Jul 06 19:28:22.114421 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:28:22.114392 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/cad66095-2da5-4270-87e3-7a84ae09f7be-tmp-dir\") pod \"test-model-88c5f6-kserve-549bf8b49-8fk7k\" (UID: \"cad66095-2da5-4270-87e3-7a84ae09f7be\") " pod="ai-tenant-e2e-inf-a-88c5f6/test-model-88c5f6-kserve-549bf8b49-8fk7k" Jul 06 19:28:22.114511 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:28:22.114494 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/cad66095-2da5-4270-87e3-7a84ae09f7be-tls-certs\") pod \"test-model-88c5f6-kserve-549bf8b49-8fk7k\" (UID: \"cad66095-2da5-4270-87e3-7a84ae09f7be\") " pod="ai-tenant-e2e-inf-a-88c5f6/test-model-88c5f6-kserve-549bf8b49-8fk7k" Jul 06 19:28:22.114657 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:28:22.114631 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/cad66095-2da5-4270-87e3-7a84ae09f7be-dshm\") pod \"test-model-88c5f6-kserve-549bf8b49-8fk7k\" (UID: \"cad66095-2da5-4270-87e3-7a84ae09f7be\") " pod="ai-tenant-e2e-inf-a-88c5f6/test-model-88c5f6-kserve-549bf8b49-8fk7k" Jul 06 19:28:22.215368 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:28:22.215345 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/cad66095-2da5-4270-87e3-7a84ae09f7be-model-cache\") pod \"test-model-88c5f6-kserve-549bf8b49-8fk7k\" (UID: \"cad66095-2da5-4270-87e3-7a84ae09f7be\") " pod="ai-tenant-e2e-inf-a-88c5f6/test-model-88c5f6-kserve-549bf8b49-8fk7k" Jul 06 19:28:22.215477 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:28:22.215393 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/cad66095-2da5-4270-87e3-7a84ae09f7be-tmp-dir\") pod \"test-model-88c5f6-kserve-549bf8b49-8fk7k\" (UID: \"cad66095-2da5-4270-87e3-7a84ae09f7be\") " pod="ai-tenant-e2e-inf-a-88c5f6/test-model-88c5f6-kserve-549bf8b49-8fk7k" Jul 06 19:28:22.215477 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:28:22.215458 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/cad66095-2da5-4270-87e3-7a84ae09f7be-tls-certs\") pod \"test-model-88c5f6-kserve-549bf8b49-8fk7k\" (UID: \"cad66095-2da5-4270-87e3-7a84ae09f7be\") " pod="ai-tenant-e2e-inf-a-88c5f6/test-model-88c5f6-kserve-549bf8b49-8fk7k" Jul 06 19:28:22.215634 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:28:22.215494 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/cad66095-2da5-4270-87e3-7a84ae09f7be-dshm\") pod \"test-model-88c5f6-kserve-549bf8b49-8fk7k\" (UID: \"cad66095-2da5-4270-87e3-7a84ae09f7be\") " pod="ai-tenant-e2e-inf-a-88c5f6/test-model-88c5f6-kserve-549bf8b49-8fk7k" Jul 06 19:28:22.215634 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:28:22.215529 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mv9tp\" (UniqueName: \"kubernetes.io/projected/cad66095-2da5-4270-87e3-7a84ae09f7be-kube-api-access-mv9tp\") pod \"test-model-88c5f6-kserve-549bf8b49-8fk7k\" (UID: \"cad66095-2da5-4270-87e3-7a84ae09f7be\") " pod="ai-tenant-e2e-inf-a-88c5f6/test-model-88c5f6-kserve-549bf8b49-8fk7k" Jul 06 19:28:22.215634 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:28:22.215570 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/cad66095-2da5-4270-87e3-7a84ae09f7be-home\") pod \"test-model-88c5f6-kserve-549bf8b49-8fk7k\" (UID: \"cad66095-2da5-4270-87e3-7a84ae09f7be\") " pod="ai-tenant-e2e-inf-a-88c5f6/test-model-88c5f6-kserve-549bf8b49-8fk7k" Jul 06 19:28:22.215986 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:28:22.215965 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/cad66095-2da5-4270-87e3-7a84ae09f7be-home\") pod \"test-model-88c5f6-kserve-549bf8b49-8fk7k\" (UID: \"cad66095-2da5-4270-87e3-7a84ae09f7be\") " pod="ai-tenant-e2e-inf-a-88c5f6/test-model-88c5f6-kserve-549bf8b49-8fk7k" Jul 06 19:28:22.216092 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:28:22.216030 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/cad66095-2da5-4270-87e3-7a84ae09f7be-tmp-dir\") pod \"test-model-88c5f6-kserve-549bf8b49-8fk7k\" (UID: \"cad66095-2da5-4270-87e3-7a84ae09f7be\") " pod="ai-tenant-e2e-inf-a-88c5f6/test-model-88c5f6-kserve-549bf8b49-8fk7k" Jul 06 19:28:22.216161 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:28:22.216125 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/cad66095-2da5-4270-87e3-7a84ae09f7be-model-cache\") pod \"test-model-88c5f6-kserve-549bf8b49-8fk7k\" (UID: \"cad66095-2da5-4270-87e3-7a84ae09f7be\") " pod="ai-tenant-e2e-inf-a-88c5f6/test-model-88c5f6-kserve-549bf8b49-8fk7k" Jul 06 19:28:22.217659 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:28:22.217641 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/cad66095-2da5-4270-87e3-7a84ae09f7be-dshm\") pod \"test-model-88c5f6-kserve-549bf8b49-8fk7k\" (UID: \"cad66095-2da5-4270-87e3-7a84ae09f7be\") " pod="ai-tenant-e2e-inf-a-88c5f6/test-model-88c5f6-kserve-549bf8b49-8fk7k" Jul 06 19:28:22.217985 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:28:22.217969 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/cad66095-2da5-4270-87e3-7a84ae09f7be-tls-certs\") pod \"test-model-88c5f6-kserve-549bf8b49-8fk7k\" (UID: \"cad66095-2da5-4270-87e3-7a84ae09f7be\") " pod="ai-tenant-e2e-inf-a-88c5f6/test-model-88c5f6-kserve-549bf8b49-8fk7k" Jul 06 19:28:23.723905 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:28:23.723871 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-a-88c5f6/test-model-88c5f6-kserve-549bf8b49-8fk7k"] Jul 06 19:28:23.724954 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:28:23.724925 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k8fbq_bc7057bd-88da-48af-ba32-e6efdc6e8362/konnectivity-agent/0.log" limit=1024 Jul 06 19:28:23.744480 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:28:23.744455 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-mv9tp\" (UniqueName: \"kubernetes.io/projected/cad66095-2da5-4270-87e3-7a84ae09f7be-kube-api-access-mv9tp\") pod \"test-model-88c5f6-kserve-549bf8b49-8fk7k\" (UID: \"cad66095-2da5-4270-87e3-7a84ae09f7be\") " pod="ai-tenant-e2e-inf-a-88c5f6/test-model-88c5f6-kserve-549bf8b49-8fk7k" Jul 06 19:28:23.848656 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:28:23.848621 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-88c5f6/test-model-88c5f6-kserve-549bf8b49-8fk7k" Jul 06 19:28:23.870817 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:28:23.870775 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k8fbq_bc7057bd-88da-48af-ba32-e6efdc6e8362/konnectivity-agent/0.log" limit=1024 Jul 06 19:28:23.970209 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:28:23.970169 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k8fbq_bc7057bd-88da-48af-ba32-e6efdc6e8362/konnectivity-agent/0.log" limit=1024 Jul 06 19:28:23.984068 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:28:23.984047 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-a-88c5f6/test-model-88c5f6-kserve-549bf8b49-8fk7k"] Jul 06 19:28:23.985750 ip-10-0-130-59 kubenswrapper[2570]: W0706 19:28:23.985725 2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podcad66095_2da5_4270_87e3_7a84ae09f7be.slice/crio-93762b8cfcb70f12bbd5fa5e4bf93ba566b345fcdfe86bb1c302e1d3c29a2ba2 WatchSource:0}: Error finding container 93762b8cfcb70f12bbd5fa5e4bf93ba566b345fcdfe86bb1c302e1d3c29a2ba2: Status 404 returned error can't find the container with id 93762b8cfcb70f12bbd5fa5e4bf93ba566b345fcdfe86bb1c302e1d3c29a2ba2 Jul 06 19:28:24.385510 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:28:24.385418 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-88c5f6/test-model-88c5f6-kserve-549bf8b49-8fk7k" event={"ID":"cad66095-2da5-4270-87e3-7a84ae09f7be","Type":"ContainerStarted","Data":"4e5b2cd67002731ab93795e6420ae70f8adccbea4443c50bddb64191a554ce22"} Jul 06 19:28:24.385510 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:28:24.385468 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-88c5f6/test-model-88c5f6-kserve-549bf8b49-8fk7k" event={"ID":"cad66095-2da5-4270-87e3-7a84ae09f7be","Type":"ContainerStarted","Data":"93762b8cfcb70f12bbd5fa5e4bf93ba566b345fcdfe86bb1c302e1d3c29a2ba2"} Jul 06 19:28:25.421916 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:28:25.421882 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k8fbq_bc7057bd-88da-48af-ba32-e6efdc6e8362/konnectivity-agent/0.log" limit=1024 Jul 06 19:28:25.561980 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:28:25.561955 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k8fbq_bc7057bd-88da-48af-ba32-e6efdc6e8362/konnectivity-agent/0.log" limit=1024 Jul 06 19:28:26.709695 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:28:26.709660 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k8fbq_bc7057bd-88da-48af-ba32-e6efdc6e8362/konnectivity-agent/0.log" limit=1024 Jul 06 19:28:26.833003 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:28:26.832972 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k8fbq_bc7057bd-88da-48af-ba32-e6efdc6e8362/konnectivity-agent/0.log" limit=1024 Jul 06 19:28:27.960104 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:28:27.960066 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k8fbq_bc7057bd-88da-48af-ba32-e6efdc6e8362/konnectivity-agent/0.log" limit=1024 Jul 06 19:28:28.091254 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:28:28.091201 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k8fbq_bc7057bd-88da-48af-ba32-e6efdc6e8362/konnectivity-agent/0.log" limit=1024 Jul 06 19:28:32.732743 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:28:32.732686 2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-inf-a-88c5f6/test-model-88c5f6-kserve-549bf8b49-8fk7k" podStartSLOduration=16.732669198 podStartE2EDuration="16.732669198s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 19:28:16 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 19:28:24.438755988 +0000 UTC m=+1983.993777181" watchObservedRunningTime="2026-07-06 19:28:32.732669198 +0000 UTC m=+1992.287690370" Jul 06 19:28:32.733159 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:28:32.733108 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-flg9z"] Jul 06 19:28:33.849195 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:28:33.849169 2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-inf-a-88c5f6/test-model-88c5f6-kserve-549bf8b49-8fk7k" Jul 06 19:28:33.849577 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:28:33.849221 2570 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-inf-a-88c5f6/test-model-88c5f6-kserve-549bf8b49-8fk7k" Jul 06 19:28:33.861549 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:28:33.861528 2570 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-inf-a-88c5f6/test-model-88c5f6-kserve-549bf8b49-8fk7k" Jul 06 19:28:34.452762 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:28:34.452736 2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-inf-a-88c5f6/test-model-88c5f6-kserve-549bf8b49-8fk7k" Jul 06 19:28:42.489413 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:28:42.489378 2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-inf-b-cc274f/test-model-cc274f-kserve-f767b9755-6hq9k"] Jul 06 19:28:42.492960 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:28:42.492937 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-cc274f/test-model-cc274f-kserve-f767b9755-6hq9k" Jul 06 19:28:42.499032 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:28:42.499016 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-b-cc274f\"/\"default-dockercfg-w4q4w\"" Jul 06 19:28:42.499874 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:28:42.499845 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-b-cc274f\"/\"test-model-cc274f-kserve-self-signed-certs\"" Jul 06 19:28:42.499975 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:28:42.499886 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-b-cc274f\"/\"openshift-service-ca.crt\"" Jul 06 19:28:42.499975 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:28:42.499853 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-b-cc274f\"/\"kube-root-ca.crt\"" Jul 06 19:28:42.506354 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:28:42.506336 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-b-cc274f/test-model-cc274f-kserve-f767b9755-6hq9k"] Jul 06 19:28:42.526920 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:28:42.526896 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k8fbq_bc7057bd-88da-48af-ba32-e6efdc6e8362/konnectivity-agent/0.log" limit=1024 Jul 06 19:28:42.587915 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:28:42.587892 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-w9n2f\" (UniqueName: \"kubernetes.io/projected/6f1a2a91-a390-48f4-b99f-a677f779a7c6-kube-api-access-w9n2f\") pod \"test-model-cc274f-kserve-f767b9755-6hq9k\" (UID: \"6f1a2a91-a390-48f4-b99f-a677f779a7c6\") " pod="ai-tenant-e2e-inf-b-cc274f/test-model-cc274f-kserve-f767b9755-6hq9k" Jul 06 19:28:42.588004 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:28:42.587929 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/6f1a2a91-a390-48f4-b99f-a677f779a7c6-dshm\") pod \"test-model-cc274f-kserve-f767b9755-6hq9k\" (UID: \"6f1a2a91-a390-48f4-b99f-a677f779a7c6\") " pod="ai-tenant-e2e-inf-b-cc274f/test-model-cc274f-kserve-f767b9755-6hq9k" Jul 06 19:28:42.588004 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:28:42.587952 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/6f1a2a91-a390-48f4-b99f-a677f779a7c6-tmp-dir\") pod \"test-model-cc274f-kserve-f767b9755-6hq9k\" (UID: \"6f1a2a91-a390-48f4-b99f-a677f779a7c6\") " pod="ai-tenant-e2e-inf-b-cc274f/test-model-cc274f-kserve-f767b9755-6hq9k" Jul 06 19:28:42.588093 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:28:42.588004 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/6f1a2a91-a390-48f4-b99f-a677f779a7c6-model-cache\") pod \"test-model-cc274f-kserve-f767b9755-6hq9k\" (UID: \"6f1a2a91-a390-48f4-b99f-a677f779a7c6\") " pod="ai-tenant-e2e-inf-b-cc274f/test-model-cc274f-kserve-f767b9755-6hq9k" Jul 06 19:28:42.588093 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:28:42.588029 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/6f1a2a91-a390-48f4-b99f-a677f779a7c6-home\") pod \"test-model-cc274f-kserve-f767b9755-6hq9k\" (UID: \"6f1a2a91-a390-48f4-b99f-a677f779a7c6\") " pod="ai-tenant-e2e-inf-b-cc274f/test-model-cc274f-kserve-f767b9755-6hq9k" Jul 06 19:28:42.588093 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:28:42.588076 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/6f1a2a91-a390-48f4-b99f-a677f779a7c6-tls-certs\") pod \"test-model-cc274f-kserve-f767b9755-6hq9k\" (UID: \"6f1a2a91-a390-48f4-b99f-a677f779a7c6\") " pod="ai-tenant-e2e-inf-b-cc274f/test-model-cc274f-kserve-f767b9755-6hq9k" Jul 06 19:28:42.688831 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:28:42.688797 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/6f1a2a91-a390-48f4-b99f-a677f779a7c6-tmp-dir\") pod \"test-model-cc274f-kserve-f767b9755-6hq9k\" (UID: \"6f1a2a91-a390-48f4-b99f-a677f779a7c6\") " pod="ai-tenant-e2e-inf-b-cc274f/test-model-cc274f-kserve-f767b9755-6hq9k" Jul 06 19:28:42.688831 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:28:42.688836 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/6f1a2a91-a390-48f4-b99f-a677f779a7c6-model-cache\") pod \"test-model-cc274f-kserve-f767b9755-6hq9k\" (UID: \"6f1a2a91-a390-48f4-b99f-a677f779a7c6\") " pod="ai-tenant-e2e-inf-b-cc274f/test-model-cc274f-kserve-f767b9755-6hq9k" Jul 06 19:28:42.688965 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:28:42.688860 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/6f1a2a91-a390-48f4-b99f-a677f779a7c6-home\") pod \"test-model-cc274f-kserve-f767b9755-6hq9k\" (UID: \"6f1a2a91-a390-48f4-b99f-a677f779a7c6\") " pod="ai-tenant-e2e-inf-b-cc274f/test-model-cc274f-kserve-f767b9755-6hq9k" Jul 06 19:28:42.688965 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:28:42.688884 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/6f1a2a91-a390-48f4-b99f-a677f779a7c6-tls-certs\") pod \"test-model-cc274f-kserve-f767b9755-6hq9k\" (UID: \"6f1a2a91-a390-48f4-b99f-a677f779a7c6\") " pod="ai-tenant-e2e-inf-b-cc274f/test-model-cc274f-kserve-f767b9755-6hq9k" Jul 06 19:28:42.688965 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:28:42.688927 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-w9n2f\" (UniqueName: \"kubernetes.io/projected/6f1a2a91-a390-48f4-b99f-a677f779a7c6-kube-api-access-w9n2f\") pod \"test-model-cc274f-kserve-f767b9755-6hq9k\" (UID: \"6f1a2a91-a390-48f4-b99f-a677f779a7c6\") " pod="ai-tenant-e2e-inf-b-cc274f/test-model-cc274f-kserve-f767b9755-6hq9k" Jul 06 19:28:42.688965 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:28:42.688956 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/6f1a2a91-a390-48f4-b99f-a677f779a7c6-dshm\") pod \"test-model-cc274f-kserve-f767b9755-6hq9k\" (UID: \"6f1a2a91-a390-48f4-b99f-a677f779a7c6\") " pod="ai-tenant-e2e-inf-b-cc274f/test-model-cc274f-kserve-f767b9755-6hq9k" Jul 06 19:28:42.689239 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:28:42.689216 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/6f1a2a91-a390-48f4-b99f-a677f779a7c6-home\") pod \"test-model-cc274f-kserve-f767b9755-6hq9k\" (UID: \"6f1a2a91-a390-48f4-b99f-a677f779a7c6\") " pod="ai-tenant-e2e-inf-b-cc274f/test-model-cc274f-kserve-f767b9755-6hq9k" Jul 06 19:28:42.689307 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:28:42.689250 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/6f1a2a91-a390-48f4-b99f-a677f779a7c6-model-cache\") pod \"test-model-cc274f-kserve-f767b9755-6hq9k\" (UID: \"6f1a2a91-a390-48f4-b99f-a677f779a7c6\") " pod="ai-tenant-e2e-inf-b-cc274f/test-model-cc274f-kserve-f767b9755-6hq9k" Jul 06 19:28:42.689307 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:28:42.689263 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/6f1a2a91-a390-48f4-b99f-a677f779a7c6-tmp-dir\") pod \"test-model-cc274f-kserve-f767b9755-6hq9k\" (UID: \"6f1a2a91-a390-48f4-b99f-a677f779a7c6\") " pod="ai-tenant-e2e-inf-b-cc274f/test-model-cc274f-kserve-f767b9755-6hq9k" Jul 06 19:28:42.691169 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:28:42.691152 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/6f1a2a91-a390-48f4-b99f-a677f779a7c6-dshm\") pod \"test-model-cc274f-kserve-f767b9755-6hq9k\" (UID: \"6f1a2a91-a390-48f4-b99f-a677f779a7c6\") " pod="ai-tenant-e2e-inf-b-cc274f/test-model-cc274f-kserve-f767b9755-6hq9k" Jul 06 19:28:42.691334 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:28:42.691318 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/6f1a2a91-a390-48f4-b99f-a677f779a7c6-tls-certs\") pod \"test-model-cc274f-kserve-f767b9755-6hq9k\" (UID: \"6f1a2a91-a390-48f4-b99f-a677f779a7c6\") " pod="ai-tenant-e2e-inf-b-cc274f/test-model-cc274f-kserve-f767b9755-6hq9k" Jul 06 19:28:42.699415 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:28:42.699392 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-w9n2f\" (UniqueName: \"kubernetes.io/projected/6f1a2a91-a390-48f4-b99f-a677f779a7c6-kube-api-access-w9n2f\") pod \"test-model-cc274f-kserve-f767b9755-6hq9k\" (UID: \"6f1a2a91-a390-48f4-b99f-a677f779a7c6\") " pod="ai-tenant-e2e-inf-b-cc274f/test-model-cc274f-kserve-f767b9755-6hq9k" Jul 06 19:28:42.802121 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:28:42.802070 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-cc274f/test-model-cc274f-kserve-f767b9755-6hq9k" Jul 06 19:28:42.941786 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:28:42.941764 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-b-cc274f/test-model-cc274f-kserve-f767b9755-6hq9k"] Jul 06 19:28:42.944015 ip-10-0-130-59 kubenswrapper[2570]: W0706 19:28:42.943986 2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod6f1a2a91_a390_48f4_b99f_a677f779a7c6.slice/crio-4e08b46d855962db913e4fa76ccc04e235aa03a1438051ad9ec10ba3031babb9 WatchSource:0}: Error finding container 4e08b46d855962db913e4fa76ccc04e235aa03a1438051ad9ec10ba3031babb9: Status 404 returned error can't find the container with id 4e08b46d855962db913e4fa76ccc04e235aa03a1438051ad9ec10ba3031babb9 Jul 06 19:28:43.486877 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:28:43.486847 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-cc274f/test-model-cc274f-kserve-f767b9755-6hq9k" event={"ID":"6f1a2a91-a390-48f4-b99f-a677f779a7c6","Type":"ContainerStarted","Data":"14b791c6a02282276f3832b71fc1d5ce5ba3a5f388cee9e7458fb1d36fadd184"} Jul 06 19:28:43.487035 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:28:43.486884 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-cc274f/test-model-cc274f-kserve-f767b9755-6hq9k" event={"ID":"6f1a2a91-a390-48f4-b99f-a677f779a7c6","Type":"ContainerStarted","Data":"4e08b46d855962db913e4fa76ccc04e235aa03a1438051ad9ec10ba3031babb9"} Jul 06 19:28:49.041377 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:28:49.041326 2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-inf-b-cc274f/test-model-cc274f-kserve-f767b9755-6hq9k" podStartSLOduration=7.041310521 podStartE2EDuration="7.041310521s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 19:28:42 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 19:28:43.529099196 +0000 UTC m=+2003.084120368" watchObservedRunningTime="2026-07-06 19:28:49.041310521 +0000 UTC m=+2008.596331731" Jul 06 19:28:49.041792 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:28:49.041575 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-flg9z"] Jul 06 19:28:52.802355 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:28:52.802316 2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-inf-b-cc274f/test-model-cc274f-kserve-f767b9755-6hq9k" Jul 06 19:28:52.802853 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:28:52.802370 2570 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-inf-b-cc274f/test-model-cc274f-kserve-f767b9755-6hq9k" Jul 06 19:28:52.814655 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:28:52.814634 2570 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-inf-b-cc274f/test-model-cc274f-kserve-f767b9755-6hq9k" Jul 06 19:28:53.548769 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:28:53.548740 2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-inf-b-cc274f/test-model-cc274f-kserve-f767b9755-6hq9k" Jul 06 19:29:00.160021 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:29:00.159996 2570 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-a-88c5f6/test-model-88c5f6-kserve-549bf8b49-8fk7k"] Jul 06 19:29:00.260889 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:29:00.260859 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k8fbq_bc7057bd-88da-48af-ba32-e6efdc6e8362/konnectivity-agent/0.log" limit=1024 Jul 06 19:29:01.339115 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:29:01.339081 2570 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-b-cc274f/test-model-cc274f-kserve-f767b9755-6hq9k"] Jul 06 19:29:01.437081 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:29:01.437054 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k8fbq_bc7057bd-88da-48af-ba32-e6efdc6e8362/konnectivity-agent/0.log" limit=1024 Jul 06 19:29:01.816882 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:29:01.816848 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k8fbq_bc7057bd-88da-48af-ba32-e6efdc6e8362/konnectivity-agent/0.log" limit=1024 Jul 06 19:29:02.166057 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:29:02.165978 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k8fbq_bc7057bd-88da-48af-ba32-e6efdc6e8362/konnectivity-agent/0.log" limit=1024 Jul 06 19:29:02.634604 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:29:02.634572 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k8fbq_bc7057bd-88da-48af-ba32-e6efdc6e8362/konnectivity-agent/0.log" limit=1024 Jul 06 19:29:06.939332 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:29:06.939298 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-flg9z"] Jul 06 19:29:07.333186 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:29:07.333110 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k8fbq_bc7057bd-88da-48af-ba32-e6efdc6e8362/konnectivity-agent/0.log" limit=1024 Jul 06 19:29:15.179459 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:29:15.179423 2570 kuberuntime_container.go:872] "Killing container with a grace period" pod="ai-tenant-e2e-inf-a-88c5f6/test-model-88c5f6-kserve-549bf8b49-8fk7k" podUID="cad66095-2da5-4270-87e3-7a84ae09f7be" containerName="main" containerID="cri-o://4e5b2cd67002731ab93795e6420ae70f8adccbea4443c50bddb64191a554ce22" gracePeriod=45 Jul 06 19:29:15.428360 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:29:15.428336 2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-88c5f6/test-model-88c5f6-kserve-549bf8b49-8fk7k" Jul 06 19:29:15.590865 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:29:15.590799 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/cad66095-2da5-4270-87e3-7a84ae09f7be-home\") pod \"cad66095-2da5-4270-87e3-7a84ae09f7be\" (UID: \"cad66095-2da5-4270-87e3-7a84ae09f7be\") " Jul 06 19:29:15.590865 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:29:15.590847 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-mv9tp\" (UniqueName: \"kubernetes.io/projected/cad66095-2da5-4270-87e3-7a84ae09f7be-kube-api-access-mv9tp\") pod \"cad66095-2da5-4270-87e3-7a84ae09f7be\" (UID: \"cad66095-2da5-4270-87e3-7a84ae09f7be\") " Jul 06 19:29:15.591072 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:29:15.590882 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/cad66095-2da5-4270-87e3-7a84ae09f7be-model-cache\") pod \"cad66095-2da5-4270-87e3-7a84ae09f7be\" (UID: \"cad66095-2da5-4270-87e3-7a84ae09f7be\") " Jul 06 19:29:15.591072 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:29:15.590953 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/cad66095-2da5-4270-87e3-7a84ae09f7be-tls-certs\") pod \"cad66095-2da5-4270-87e3-7a84ae09f7be\" (UID: \"cad66095-2da5-4270-87e3-7a84ae09f7be\") " Jul 06 19:29:15.591072 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:29:15.590978 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/cad66095-2da5-4270-87e3-7a84ae09f7be-dshm\") pod \"cad66095-2da5-4270-87e3-7a84ae09f7be\" (UID: \"cad66095-2da5-4270-87e3-7a84ae09f7be\") " Jul 06 19:29:15.591072 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:29:15.591007 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/cad66095-2da5-4270-87e3-7a84ae09f7be-tmp-dir\") pod \"cad66095-2da5-4270-87e3-7a84ae09f7be\" (UID: \"cad66095-2da5-4270-87e3-7a84ae09f7be\") " Jul 06 19:29:15.591072 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:29:15.591034 2570 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/cad66095-2da5-4270-87e3-7a84ae09f7be-home" (OuterVolumeSpecName: "home") pod "cad66095-2da5-4270-87e3-7a84ae09f7be" (UID: "cad66095-2da5-4270-87e3-7a84ae09f7be"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 19:29:15.591327 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:29:15.591163 2570 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/cad66095-2da5-4270-87e3-7a84ae09f7be-model-cache" (OuterVolumeSpecName: "model-cache") pod "cad66095-2da5-4270-87e3-7a84ae09f7be" (UID: "cad66095-2da5-4270-87e3-7a84ae09f7be"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 19:29:15.591384 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:29:15.591324 2570 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/cad66095-2da5-4270-87e3-7a84ae09f7be-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "cad66095-2da5-4270-87e3-7a84ae09f7be" (UID: "cad66095-2da5-4270-87e3-7a84ae09f7be"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 19:29:15.591384 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:29:15.591344 2570 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/cad66095-2da5-4270-87e3-7a84ae09f7be-home\") on node \"ip-10-0-130-59.ec2.internal\" DevicePath \"\"" Jul 06 19:29:15.591384 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:29:15.591363 2570 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/cad66095-2da5-4270-87e3-7a84ae09f7be-model-cache\") on node \"ip-10-0-130-59.ec2.internal\" DevicePath \"\"" Jul 06 19:29:15.593062 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:29:15.593032 2570 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/cad66095-2da5-4270-87e3-7a84ae09f7be-dshm" (OuterVolumeSpecName: "dshm") pod "cad66095-2da5-4270-87e3-7a84ae09f7be" (UID: "cad66095-2da5-4270-87e3-7a84ae09f7be"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 19:29:15.593163 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:29:15.593072 2570 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/cad66095-2da5-4270-87e3-7a84ae09f7be-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "cad66095-2da5-4270-87e3-7a84ae09f7be" (UID: "cad66095-2da5-4270-87e3-7a84ae09f7be"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 06 19:29:15.593163 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:29:15.593079 2570 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/cad66095-2da5-4270-87e3-7a84ae09f7be-kube-api-access-mv9tp" (OuterVolumeSpecName: "kube-api-access-mv9tp") pod "cad66095-2da5-4270-87e3-7a84ae09f7be" (UID: "cad66095-2da5-4270-87e3-7a84ae09f7be"). InnerVolumeSpecName "kube-api-access-mv9tp". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 06 19:29:15.646702 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:29:15.646664 2570 generic.go:356] "Generic (PLEG): container finished" podID="cad66095-2da5-4270-87e3-7a84ae09f7be" containerID="4e5b2cd67002731ab93795e6420ae70f8adccbea4443c50bddb64191a554ce22" exitCode=0 Jul 06 19:29:15.646873 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:29:15.646706 2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-88c5f6/test-model-88c5f6-kserve-549bf8b49-8fk7k" Jul 06 19:29:15.646873 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:29:15.646724 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-88c5f6/test-model-88c5f6-kserve-549bf8b49-8fk7k" event={"ID":"cad66095-2da5-4270-87e3-7a84ae09f7be","Type":"ContainerDied","Data":"4e5b2cd67002731ab93795e6420ae70f8adccbea4443c50bddb64191a554ce22"} Jul 06 19:29:15.646873 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:29:15.646765 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-88c5f6/test-model-88c5f6-kserve-549bf8b49-8fk7k" event={"ID":"cad66095-2da5-4270-87e3-7a84ae09f7be","Type":"ContainerDied","Data":"93762b8cfcb70f12bbd5fa5e4bf93ba566b345fcdfe86bb1c302e1d3c29a2ba2"} Jul 06 19:29:15.646873 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:29:15.646786 2570 scope.go:117] "RemoveContainer" containerID="4e5b2cd67002731ab93795e6420ae70f8adccbea4443c50bddb64191a554ce22" Jul 06 19:29:15.655405 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:29:15.655388 2570 scope.go:117] "RemoveContainer" containerID="4e5b2cd67002731ab93795e6420ae70f8adccbea4443c50bddb64191a554ce22" Jul 06 19:29:15.655673 ip-10-0-130-59 kubenswrapper[2570]: E0706 19:29:15.655653 2570 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4e5b2cd67002731ab93795e6420ae70f8adccbea4443c50bddb64191a554ce22\": container with ID starting with 4e5b2cd67002731ab93795e6420ae70f8adccbea4443c50bddb64191a554ce22 not found: ID does not exist" containerID="4e5b2cd67002731ab93795e6420ae70f8adccbea4443c50bddb64191a554ce22" Jul 06 19:29:15.655723 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:29:15.655685 2570 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4e5b2cd67002731ab93795e6420ae70f8adccbea4443c50bddb64191a554ce22"} err="failed to get container status \"4e5b2cd67002731ab93795e6420ae70f8adccbea4443c50bddb64191a554ce22\": rpc error: code = NotFound desc = could not find container \"4e5b2cd67002731ab93795e6420ae70f8adccbea4443c50bddb64191a554ce22\": container with ID starting with 4e5b2cd67002731ab93795e6420ae70f8adccbea4443c50bddb64191a554ce22 not found: ID does not exist" Jul 06 19:29:15.687662 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:29:15.687640 2570 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-a-88c5f6/test-model-88c5f6-kserve-549bf8b49-8fk7k"] Jul 06 19:29:15.690466 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:29:15.690445 2570 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-inf-a-88c5f6/test-model-88c5f6-kserve-549bf8b49-8fk7k"] Jul 06 19:29:15.692069 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:29:15.692045 2570 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/cad66095-2da5-4270-87e3-7a84ae09f7be-tls-certs\") on node \"ip-10-0-130-59.ec2.internal\" DevicePath \"\"" Jul 06 19:29:15.692069 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:29:15.692065 2570 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/cad66095-2da5-4270-87e3-7a84ae09f7be-dshm\") on node \"ip-10-0-130-59.ec2.internal\" DevicePath \"\"" Jul 06 19:29:15.692069 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:29:15.692074 2570 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/cad66095-2da5-4270-87e3-7a84ae09f7be-tmp-dir\") on node \"ip-10-0-130-59.ec2.internal\" DevicePath \"\"" Jul 06 19:29:15.692226 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:29:15.692082 2570 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-mv9tp\" (UniqueName: \"kubernetes.io/projected/cad66095-2da5-4270-87e3-7a84ae09f7be-kube-api-access-mv9tp\") on node \"ip-10-0-130-59.ec2.internal\" DevicePath \"\"" Jul 06 19:29:16.359100 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:29:16.359039 2570 kuberuntime_container.go:872] "Killing container with a grace period" pod="ai-tenant-e2e-inf-b-cc274f/test-model-cc274f-kserve-f767b9755-6hq9k" podUID="6f1a2a91-a390-48f4-b99f-a677f779a7c6" containerName="main" containerID="cri-o://14b791c6a02282276f3832b71fc1d5ce5ba3a5f388cee9e7458fb1d36fadd184" gracePeriod=45 Jul 06 19:29:16.617788 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:29:16.617731 2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-cc274f/test-model-cc274f-kserve-f767b9755-6hq9k" Jul 06 19:29:16.652221 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:29:16.652195 2570 generic.go:356] "Generic (PLEG): container finished" podID="6f1a2a91-a390-48f4-b99f-a677f779a7c6" containerID="14b791c6a02282276f3832b71fc1d5ce5ba3a5f388cee9e7458fb1d36fadd184" exitCode=0 Jul 06 19:29:16.652340 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:29:16.652231 2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-cc274f/test-model-cc274f-kserve-f767b9755-6hq9k" Jul 06 19:29:16.652340 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:29:16.652280 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-cc274f/test-model-cc274f-kserve-f767b9755-6hq9k" event={"ID":"6f1a2a91-a390-48f4-b99f-a677f779a7c6","Type":"ContainerDied","Data":"14b791c6a02282276f3832b71fc1d5ce5ba3a5f388cee9e7458fb1d36fadd184"} Jul 06 19:29:16.652340 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:29:16.652316 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-cc274f/test-model-cc274f-kserve-f767b9755-6hq9k" event={"ID":"6f1a2a91-a390-48f4-b99f-a677f779a7c6","Type":"ContainerDied","Data":"4e08b46d855962db913e4fa76ccc04e235aa03a1438051ad9ec10ba3031babb9"} Jul 06 19:29:16.652501 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:29:16.652340 2570 scope.go:117] "RemoveContainer" containerID="14b791c6a02282276f3832b71fc1d5ce5ba3a5f388cee9e7458fb1d36fadd184" Jul 06 19:29:16.660810 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:29:16.660781 2570 scope.go:117] "RemoveContainer" containerID="14b791c6a02282276f3832b71fc1d5ce5ba3a5f388cee9e7458fb1d36fadd184" Jul 06 19:29:16.661053 ip-10-0-130-59 kubenswrapper[2570]: E0706 19:29:16.661035 2570 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"14b791c6a02282276f3832b71fc1d5ce5ba3a5f388cee9e7458fb1d36fadd184\": container with ID starting with 14b791c6a02282276f3832b71fc1d5ce5ba3a5f388cee9e7458fb1d36fadd184 not found: ID does not exist" containerID="14b791c6a02282276f3832b71fc1d5ce5ba3a5f388cee9e7458fb1d36fadd184" Jul 06 19:29:16.661102 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:29:16.661061 2570 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"14b791c6a02282276f3832b71fc1d5ce5ba3a5f388cee9e7458fb1d36fadd184"} err="failed to get container status \"14b791c6a02282276f3832b71fc1d5ce5ba3a5f388cee9e7458fb1d36fadd184\": rpc error: code = NotFound desc = could not find container \"14b791c6a02282276f3832b71fc1d5ce5ba3a5f388cee9e7458fb1d36fadd184\": container with ID starting with 14b791c6a02282276f3832b71fc1d5ce5ba3a5f388cee9e7458fb1d36fadd184 not found: ID does not exist" Jul 06 19:29:16.802244 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:29:16.802198 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-w9n2f\" (UniqueName: \"kubernetes.io/projected/6f1a2a91-a390-48f4-b99f-a677f779a7c6-kube-api-access-w9n2f\") pod \"6f1a2a91-a390-48f4-b99f-a677f779a7c6\" (UID: \"6f1a2a91-a390-48f4-b99f-a677f779a7c6\") " Jul 06 19:29:16.802438 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:29:16.802299 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/6f1a2a91-a390-48f4-b99f-a677f779a7c6-home\") pod \"6f1a2a91-a390-48f4-b99f-a677f779a7c6\" (UID: \"6f1a2a91-a390-48f4-b99f-a677f779a7c6\") " Jul 06 19:29:16.802438 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:29:16.802338 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/6f1a2a91-a390-48f4-b99f-a677f779a7c6-dshm\") pod \"6f1a2a91-a390-48f4-b99f-a677f779a7c6\" (UID: \"6f1a2a91-a390-48f4-b99f-a677f779a7c6\") " Jul 06 19:29:16.802438 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:29:16.802355 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/6f1a2a91-a390-48f4-b99f-a677f779a7c6-tmp-dir\") pod \"6f1a2a91-a390-48f4-b99f-a677f779a7c6\" (UID: \"6f1a2a91-a390-48f4-b99f-a677f779a7c6\") " Jul 06 19:29:16.802438 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:29:16.802378 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/6f1a2a91-a390-48f4-b99f-a677f779a7c6-tls-certs\") pod \"6f1a2a91-a390-48f4-b99f-a677f779a7c6\" (UID: \"6f1a2a91-a390-48f4-b99f-a677f779a7c6\") " Jul 06 19:29:16.802438 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:29:16.802409 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/6f1a2a91-a390-48f4-b99f-a677f779a7c6-model-cache\") pod \"6f1a2a91-a390-48f4-b99f-a677f779a7c6\" (UID: \"6f1a2a91-a390-48f4-b99f-a677f779a7c6\") " Jul 06 19:29:16.802745 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:29:16.802605 2570 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6f1a2a91-a390-48f4-b99f-a677f779a7c6-home" (OuterVolumeSpecName: "home") pod "6f1a2a91-a390-48f4-b99f-a677f779a7c6" (UID: "6f1a2a91-a390-48f4-b99f-a677f779a7c6"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 19:29:16.802745 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:29:16.802637 2570 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6f1a2a91-a390-48f4-b99f-a677f779a7c6-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "6f1a2a91-a390-48f4-b99f-a677f779a7c6" (UID: "6f1a2a91-a390-48f4-b99f-a677f779a7c6"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 19:29:16.802847 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:29:16.802780 2570 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6f1a2a91-a390-48f4-b99f-a677f779a7c6-model-cache" (OuterVolumeSpecName: "model-cache") pod "6f1a2a91-a390-48f4-b99f-a677f779a7c6" (UID: "6f1a2a91-a390-48f4-b99f-a677f779a7c6"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 19:29:16.802847 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:29:16.802801 2570 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/6f1a2a91-a390-48f4-b99f-a677f779a7c6-home\") on node \"ip-10-0-130-59.ec2.internal\" DevicePath \"\"" Jul 06 19:29:16.802847 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:29:16.802814 2570 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/6f1a2a91-a390-48f4-b99f-a677f779a7c6-tmp-dir\") on node \"ip-10-0-130-59.ec2.internal\" DevicePath \"\"" Jul 06 19:29:16.804495 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:29:16.804468 2570 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6f1a2a91-a390-48f4-b99f-a677f779a7c6-dshm" (OuterVolumeSpecName: "dshm") pod "6f1a2a91-a390-48f4-b99f-a677f779a7c6" (UID: "6f1a2a91-a390-48f4-b99f-a677f779a7c6"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 19:29:16.804591 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:29:16.804504 2570 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6f1a2a91-a390-48f4-b99f-a677f779a7c6-kube-api-access-w9n2f" (OuterVolumeSpecName: "kube-api-access-w9n2f") pod "6f1a2a91-a390-48f4-b99f-a677f779a7c6" (UID: "6f1a2a91-a390-48f4-b99f-a677f779a7c6"). InnerVolumeSpecName "kube-api-access-w9n2f". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 06 19:29:16.804591 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:29:16.804523 2570 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/6f1a2a91-a390-48f4-b99f-a677f779a7c6-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "6f1a2a91-a390-48f4-b99f-a677f779a7c6" (UID: "6f1a2a91-a390-48f4-b99f-a677f779a7c6"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 06 19:29:16.903606 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:29:16.903538 2570 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-w9n2f\" (UniqueName: \"kubernetes.io/projected/6f1a2a91-a390-48f4-b99f-a677f779a7c6-kube-api-access-w9n2f\") on node \"ip-10-0-130-59.ec2.internal\" DevicePath \"\"" Jul 06 19:29:16.903606 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:29:16.903570 2570 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/6f1a2a91-a390-48f4-b99f-a677f779a7c6-dshm\") on node \"ip-10-0-130-59.ec2.internal\" DevicePath \"\"" Jul 06 19:29:16.903606 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:29:16.903579 2570 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/6f1a2a91-a390-48f4-b99f-a677f779a7c6-tls-certs\") on node \"ip-10-0-130-59.ec2.internal\" DevicePath \"\"" Jul 06 19:29:16.903606 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:29:16.903588 2570 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/6f1a2a91-a390-48f4-b99f-a677f779a7c6-model-cache\") on node \"ip-10-0-130-59.ec2.internal\" DevicePath \"\"" Jul 06 19:29:16.982738 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:29:16.982713 2570 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-b-cc274f/test-model-cc274f-kserve-f767b9755-6hq9k"] Jul 06 19:29:16.985652 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:29:16.985631 2570 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-inf-b-cc274f/test-model-cc274f-kserve-f767b9755-6hq9k"] Jul 06 19:29:17.028204 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:29:17.028180 2570 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="6f1a2a91-a390-48f4-b99f-a677f779a7c6" path="/var/lib/kubelet/pods/6f1a2a91-a390-48f4-b99f-a677f779a7c6/volumes" Jul 06 19:29:17.028548 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:29:17.028536 2570 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="cad66095-2da5-4270-87e3-7a84ae09f7be" path="/var/lib/kubelet/pods/cad66095-2da5-4270-87e3-7a84ae09f7be/volumes" Jul 06 19:29:30.846006 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:29:30.845971 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k8fbq_bc7057bd-88da-48af-ba32-e6efdc6e8362/konnectivity-agent/0.log" limit=1024 Jul 06 19:29:30.975969 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:29:30.975937 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k8fbq_bc7057bd-88da-48af-ba32-e6efdc6e8362/konnectivity-agent/0.log" limit=1024 Jul 06 19:29:44.358867 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:29:44.358777 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k8fbq_bc7057bd-88da-48af-ba32-e6efdc6e8362/konnectivity-agent/0.log" limit=1024 Jul 06 19:29:44.720882 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:29:44.720854 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k8fbq_bc7057bd-88da-48af-ba32-e6efdc6e8362/konnectivity-agent/0.log" limit=1024 Jul 06 19:29:44.911031 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:29:44.910992 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k8fbq_bc7057bd-88da-48af-ba32-e6efdc6e8362/konnectivity-agent/0.log" limit=1024 Jul 06 19:29:45.082819 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:29:45.082735 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k8fbq_bc7057bd-88da-48af-ba32-e6efdc6e8362/konnectivity-agent/0.log" limit=1024 Jul 06 19:29:45.601689 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:29:45.601652 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k8fbq_bc7057bd-88da-48af-ba32-e6efdc6e8362/konnectivity-agent/0.log" limit=1024 Jul 06 19:29:45.827895 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:29:45.827866 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k8fbq_bc7057bd-88da-48af-ba32-e6efdc6e8362/konnectivity-agent/0.log" limit=1024 Jul 06 19:29:46.037845 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:29:46.037816 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k8fbq_bc7057bd-88da-48af-ba32-e6efdc6e8362/konnectivity-agent/0.log" limit=1024 Jul 06 19:29:46.197425 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:29:46.197394 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k8fbq_bc7057bd-88da-48af-ba32-e6efdc6e8362/konnectivity-agent/0.log" limit=1024 Jul 06 19:29:49.250540 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:29:49.250506 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-flg9z"] Jul 06 19:29:54.130283 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:29:54.130241 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k8fbq_bc7057bd-88da-48af-ba32-e6efdc6e8362/konnectivity-agent/0.log" limit=1024 Jul 06 19:29:54.284818 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:29:54.284786 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k8fbq_bc7057bd-88da-48af-ba32-e6efdc6e8362/konnectivity-agent/0.log" limit=1024 Jul 06 19:29:54.525040 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:29:54.525011 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k8fbq_bc7057bd-88da-48af-ba32-e6efdc6e8362/konnectivity-agent/0.log" limit=1024 Jul 06 19:29:54.810246 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:29:54.810162 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k8fbq_bc7057bd-88da-48af-ba32-e6efdc6e8362/konnectivity-agent/0.log" limit=1024 Jul 06 19:29:55.255460 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:29:55.255429 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k8fbq_bc7057bd-88da-48af-ba32-e6efdc6e8362/konnectivity-agent/0.log" limit=1024 Jul 06 19:30:09.082714 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:30:09.082676 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k8fbq_bc7057bd-88da-48af-ba32-e6efdc6e8362/konnectivity-agent/0.log" limit=1024 Jul 06 19:30:09.242408 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:30:09.242371 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k8fbq_bc7057bd-88da-48af-ba32-e6efdc6e8362/konnectivity-agent/0.log" limit=1024 Jul 06 19:30:11.201254 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:30:11.201218 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k8fbq_bc7057bd-88da-48af-ba32-e6efdc6e8362/konnectivity-agent/0.log" limit=1024 Jul 06 19:30:11.773929 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:30:11.773898 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k8fbq_bc7057bd-88da-48af-ba32-e6efdc6e8362/konnectivity-agent/0.log" limit=1024 Jul 06 19:30:12.352549 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:30:12.352516 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k8fbq_bc7057bd-88da-48af-ba32-e6efdc6e8362/konnectivity-agent/0.log" limit=1024 Jul 06 19:30:13.278330 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:30:13.278293 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k8fbq_bc7057bd-88da-48af-ba32-e6efdc6e8362/konnectivity-agent/0.log" limit=1024 Jul 06 19:30:13.675944 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:30:13.675910 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k8fbq_bc7057bd-88da-48af-ba32-e6efdc6e8362/konnectivity-agent/0.log" limit=1024 Jul 06 19:30:15.847466 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:30:15.847429 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k8fbq_bc7057bd-88da-48af-ba32-e6efdc6e8362/konnectivity-agent/0.log" limit=1024 Jul 06 19:30:16.376891 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:30:16.376853 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k8fbq_bc7057bd-88da-48af-ba32-e6efdc6e8362/konnectivity-agent/0.log" limit=1024 Jul 06 19:30:16.903895 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:30:16.903855 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k8fbq_bc7057bd-88da-48af-ba32-e6efdc6e8362/konnectivity-agent/0.log" limit=1024 Jul 06 19:30:17.099793 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:30:17.099761 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k8fbq_bc7057bd-88da-48af-ba32-e6efdc6e8362/konnectivity-agent/0.log" limit=1024 Jul 06 19:30:17.275073 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:30:17.275039 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-flg9z"] Jul 06 19:30:20.332249 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:30:20.332207 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k8fbq_bc7057bd-88da-48af-ba32-e6efdc6e8362/konnectivity-agent/0.log" limit=1024 Jul 06 19:30:20.514056 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:30:20.514022 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k8fbq_bc7057bd-88da-48af-ba32-e6efdc6e8362/konnectivity-agent/0.log" limit=1024 Jul 06 19:30:20.718020 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:30:20.717990 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k8fbq_bc7057bd-88da-48af-ba32-e6efdc6e8362/konnectivity-agent/0.log" limit=1024 Jul 06 19:30:21.557047 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:30:21.557005 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k8fbq_bc7057bd-88da-48af-ba32-e6efdc6e8362/konnectivity-agent/0.log" limit=1024 Jul 06 19:30:21.724091 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:30:21.724051 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k8fbq_bc7057bd-88da-48af-ba32-e6efdc6e8362/konnectivity-agent/0.log" limit=1024 Jul 06 19:30:24.085431 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:30:24.085377 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k8fbq_bc7057bd-88da-48af-ba32-e6efdc6e8362/konnectivity-agent/0.log" limit=1024 Jul 06 19:30:24.181731 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:30:24.181698 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k8fbq_bc7057bd-88da-48af-ba32-e6efdc6e8362/konnectivity-agent/0.log" limit=1024 Jul 06 19:30:24.343238 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:30:24.343159 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k8fbq_bc7057bd-88da-48af-ba32-e6efdc6e8362/konnectivity-agent/0.log" limit=1024 Jul 06 19:30:24.538247 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:30:24.538213 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k8fbq_bc7057bd-88da-48af-ba32-e6efdc6e8362/konnectivity-agent/0.log" limit=1024 Jul 06 19:30:25.067419 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:30:25.067377 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k8fbq_bc7057bd-88da-48af-ba32-e6efdc6e8362/konnectivity-agent/0.log" limit=1024 Jul 06 19:30:31.622698 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:30:31.622657 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-flg9z"] Jul 06 19:30:32.911306 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:30:32.911266 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k8fbq_bc7057bd-88da-48af-ba32-e6efdc6e8362/konnectivity-agent/0.log" limit=1024 Jul 06 19:30:33.295964 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:30:33.295919 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k8fbq_bc7057bd-88da-48af-ba32-e6efdc6e8362/konnectivity-agent/0.log" limit=1024 Jul 06 19:30:33.398101 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:30:33.398073 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k8fbq_bc7057bd-88da-48af-ba32-e6efdc6e8362/konnectivity-agent/0.log" limit=1024 Jul 06 19:30:33.520433 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:30:33.520401 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k8fbq_bc7057bd-88da-48af-ba32-e6efdc6e8362/konnectivity-agent/0.log" limit=1024 Jul 06 19:30:33.748372 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:30:33.748334 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k8fbq_bc7057bd-88da-48af-ba32-e6efdc6e8362/konnectivity-agent/0.log" limit=1024 Jul 06 19:30:39.678506 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:30:39.678473 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k8fbq_bc7057bd-88da-48af-ba32-e6efdc6e8362/konnectivity-agent/0.log" limit=1024 Jul 06 19:30:39.780788 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:30:39.780717 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k8fbq_bc7057bd-88da-48af-ba32-e6efdc6e8362/konnectivity-agent/0.log" limit=1024 Jul 06 19:30:40.016697 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:30:40.016595 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k8fbq_bc7057bd-88da-48af-ba32-e6efdc6e8362/konnectivity-agent/0.log" limit=1024 Jul 06 19:30:40.216283 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:30:40.216249 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k8fbq_bc7057bd-88da-48af-ba32-e6efdc6e8362/konnectivity-agent/0.log" limit=1024 Jul 06 19:30:40.376213 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:30:40.376139 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k8fbq_bc7057bd-88da-48af-ba32-e6efdc6e8362/konnectivity-agent/0.log" limit=1024 Jul 06 19:30:40.623174 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:30:40.623066 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k8fbq_bc7057bd-88da-48af-ba32-e6efdc6e8362/konnectivity-agent/0.log" limit=1024 Jul 06 19:30:40.792928 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:30:40.792893 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k8fbq_bc7057bd-88da-48af-ba32-e6efdc6e8362/konnectivity-agent/0.log" limit=1024 Jul 06 19:30:40.894142 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:30:40.894115 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k8fbq_bc7057bd-88da-48af-ba32-e6efdc6e8362/konnectivity-agent/0.log" limit=1024 Jul 06 19:30:40.995885 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:30:40.995846 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k8fbq_bc7057bd-88da-48af-ba32-e6efdc6e8362/konnectivity-agent/0.log" limit=1024 Jul 06 19:30:41.181338 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:30:41.181300 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k8fbq_bc7057bd-88da-48af-ba32-e6efdc6e8362/konnectivity-agent/0.log" limit=1024 Jul 06 19:30:42.804126 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:30:42.804095 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-flg9z"] Jul 06 19:30:46.714331 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:30:46.714290 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k8fbq_bc7057bd-88da-48af-ba32-e6efdc6e8362/konnectivity-agent/0.log" limit=1024 Jul 06 19:30:46.817942 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:30:46.817910 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k8fbq_bc7057bd-88da-48af-ba32-e6efdc6e8362/konnectivity-agent/0.log" limit=1024 Jul 06 19:30:46.997657 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:30:46.997551 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k8fbq_bc7057bd-88da-48af-ba32-e6efdc6e8362/konnectivity-agent/0.log" limit=1024 Jul 06 19:30:47.263345 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:30:47.263264 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k8fbq_bc7057bd-88da-48af-ba32-e6efdc6e8362/konnectivity-agent/0.log" limit=1024 Jul 06 19:30:47.563119 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:30:47.563046 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k8fbq_bc7057bd-88da-48af-ba32-e6efdc6e8362/konnectivity-agent/0.log" limit=1024 Jul 06 19:30:48.034318 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:30:48.034281 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k8fbq_bc7057bd-88da-48af-ba32-e6efdc6e8362/konnectivity-agent/0.log" limit=1024 Jul 06 19:30:48.387325 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:30:48.387220 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k8fbq_bc7057bd-88da-48af-ba32-e6efdc6e8362/konnectivity-agent/0.log" limit=1024 Jul 06 19:30:52.413955 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:30:52.413918 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k8fbq_bc7057bd-88da-48af-ba32-e6efdc6e8362/konnectivity-agent/0.log" limit=1024 Jul 06 19:30:52.589676 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:30:52.589638 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k8fbq_bc7057bd-88da-48af-ba32-e6efdc6e8362/konnectivity-agent/0.log" limit=1024 Jul 06 19:30:52.785042 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:30:52.784980 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k8fbq_bc7057bd-88da-48af-ba32-e6efdc6e8362/konnectivity-agent/0.log" limit=1024 Jul 06 19:30:53.621174 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:30:53.621111 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k8fbq_bc7057bd-88da-48af-ba32-e6efdc6e8362/konnectivity-agent/0.log" limit=1024 Jul 06 19:30:53.742518 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:30:53.742481 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k8fbq_bc7057bd-88da-48af-ba32-e6efdc6e8362/konnectivity-agent/0.log" limit=1024 Jul 06 19:30:56.001397 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:30:56.001362 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-flg9z"] Jul 06 19:30:56.751834 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:30:56.751800 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k8fbq_bc7057bd-88da-48af-ba32-e6efdc6e8362/konnectivity-agent/0.log" limit=1024 Jul 06 19:30:57.093142 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:30:57.093052 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k8fbq_bc7057bd-88da-48af-ba32-e6efdc6e8362/konnectivity-agent/0.log" limit=1024 Jul 06 19:30:57.536645 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:30:57.536586 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k8fbq_bc7057bd-88da-48af-ba32-e6efdc6e8362/konnectivity-agent/0.log" limit=1024 Jul 06 19:30:57.898765 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:30:57.898688 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k8fbq_bc7057bd-88da-48af-ba32-e6efdc6e8362/konnectivity-agent/0.log" limit=1024 Jul 06 19:31:00.360877 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:31:00.360840 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k8fbq_bc7057bd-88da-48af-ba32-e6efdc6e8362/konnectivity-agent/0.log" limit=1024 Jul 06 19:31:00.504354 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:31:00.504320 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k8fbq_bc7057bd-88da-48af-ba32-e6efdc6e8362/konnectivity-agent/0.log" limit=1024 Jul 06 19:31:00.734408 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:31:00.734367 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k8fbq_bc7057bd-88da-48af-ba32-e6efdc6e8362/konnectivity-agent/0.log" limit=1024 Jul 06 19:31:00.945962 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:31:00.945925 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k8fbq_bc7057bd-88da-48af-ba32-e6efdc6e8362/konnectivity-agent/0.log" limit=1024 Jul 06 19:31:01.780731 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:31:01.780683 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k8fbq_bc7057bd-88da-48af-ba32-e6efdc6e8362/konnectivity-agent/0.log" limit=1024 Jul 06 19:31:01.950818 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:31:01.950783 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k8fbq_bc7057bd-88da-48af-ba32-e6efdc6e8362/konnectivity-agent/0.log" limit=1024 Jul 06 19:31:04.743245 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:31:04.743193 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k8fbq_bc7057bd-88da-48af-ba32-e6efdc6e8362/konnectivity-agent/0.log" limit=1024 Jul 06 19:31:06.072274 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:31:06.072236 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-flg9z"] Jul 06 19:31:13.705200 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:31:13.705120 2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-a-3550d9/auth-test-model-3550d9-kserve-6bfb6bcc6c-nmvgm"] Jul 06 19:31:13.705675 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:31:13.705482 2570 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="cad66095-2da5-4270-87e3-7a84ae09f7be" containerName="main" Jul 06 19:31:13.705675 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:31:13.705492 2570 state_mem.go:107] "Deleted CPUSet assignment" podUID="cad66095-2da5-4270-87e3-7a84ae09f7be" containerName="main" Jul 06 19:31:13.705675 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:31:13.705500 2570 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6f1a2a91-a390-48f4-b99f-a677f779a7c6" containerName="main" Jul 06 19:31:13.705675 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:31:13.705505 2570 state_mem.go:107] "Deleted CPUSet assignment" podUID="6f1a2a91-a390-48f4-b99f-a677f779a7c6" containerName="main" Jul 06 19:31:13.705675 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:31:13.705577 2570 memory_manager.go:368] "RemoveStaleState removing state" podUID="6f1a2a91-a390-48f4-b99f-a677f779a7c6" containerName="main" Jul 06 19:31:13.705675 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:31:13.705585 2570 memory_manager.go:368] "RemoveStaleState removing state" podUID="cad66095-2da5-4270-87e3-7a84ae09f7be" containerName="main" Jul 06 19:31:13.708515 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:31:13.708500 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-3550d9/auth-test-model-3550d9-kserve-6bfb6bcc6c-nmvgm" Jul 06 19:31:13.712740 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:31:13.712719 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-3550d9\"/\"kube-root-ca.crt\"" Jul 06 19:31:13.712740 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:31:13.712730 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-3550d9\"/\"auth-test-model-3550d9-kserve-self-signed-certs\"" Jul 06 19:31:13.712916 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:31:13.712746 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-3550d9\"/\"openshift-service-ca.crt\"" Jul 06 19:31:13.712916 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:31:13.712722 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-3550d9\"/\"default-dockercfg-cv6ch\"" Jul 06 19:31:13.725989 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:31:13.725969 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-3550d9/auth-test-model-3550d9-kserve-6bfb6bcc6c-nmvgm"] Jul 06 19:31:13.768190 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:31:13.768135 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k8fbq_bc7057bd-88da-48af-ba32-e6efdc6e8362/konnectivity-agent/0.log" limit=1024 Jul 06 19:31:13.805138 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:31:13.805118 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7zq24\" (UniqueName: \"kubernetes.io/projected/6c86ba36-39a2-4e94-ba85-f195dfd115bd-kube-api-access-7zq24\") pod \"auth-test-model-3550d9-kserve-6bfb6bcc6c-nmvgm\" (UID: \"6c86ba36-39a2-4e94-ba85-f195dfd115bd\") " pod="ai-tenant-e2e-shared-a-3550d9/auth-test-model-3550d9-kserve-6bfb6bcc6c-nmvgm" Jul 06 19:31:13.805239 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:31:13.805168 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/6c86ba36-39a2-4e94-ba85-f195dfd115bd-home\") pod \"auth-test-model-3550d9-kserve-6bfb6bcc6c-nmvgm\" (UID: \"6c86ba36-39a2-4e94-ba85-f195dfd115bd\") " pod="ai-tenant-e2e-shared-a-3550d9/auth-test-model-3550d9-kserve-6bfb6bcc6c-nmvgm" Jul 06 19:31:13.805239 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:31:13.805200 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/6c86ba36-39a2-4e94-ba85-f195dfd115bd-tls-certs\") pod \"auth-test-model-3550d9-kserve-6bfb6bcc6c-nmvgm\" (UID: \"6c86ba36-39a2-4e94-ba85-f195dfd115bd\") " pod="ai-tenant-e2e-shared-a-3550d9/auth-test-model-3550d9-kserve-6bfb6bcc6c-nmvgm" Jul 06 19:31:13.805239 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:31:13.805216 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/6c86ba36-39a2-4e94-ba85-f195dfd115bd-dshm\") pod \"auth-test-model-3550d9-kserve-6bfb6bcc6c-nmvgm\" (UID: \"6c86ba36-39a2-4e94-ba85-f195dfd115bd\") " pod="ai-tenant-e2e-shared-a-3550d9/auth-test-model-3550d9-kserve-6bfb6bcc6c-nmvgm" Jul 06 19:31:13.805239 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:31:13.805234 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/6c86ba36-39a2-4e94-ba85-f195dfd115bd-model-cache\") pod \"auth-test-model-3550d9-kserve-6bfb6bcc6c-nmvgm\" (UID: \"6c86ba36-39a2-4e94-ba85-f195dfd115bd\") " pod="ai-tenant-e2e-shared-a-3550d9/auth-test-model-3550d9-kserve-6bfb6bcc6c-nmvgm" Jul 06 19:31:13.805373 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:31:13.805256 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/6c86ba36-39a2-4e94-ba85-f195dfd115bd-tmp-dir\") pod \"auth-test-model-3550d9-kserve-6bfb6bcc6c-nmvgm\" (UID: \"6c86ba36-39a2-4e94-ba85-f195dfd115bd\") " pod="ai-tenant-e2e-shared-a-3550d9/auth-test-model-3550d9-kserve-6bfb6bcc6c-nmvgm" Jul 06 19:31:13.906405 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:31:13.906380 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/6c86ba36-39a2-4e94-ba85-f195dfd115bd-home\") pod \"auth-test-model-3550d9-kserve-6bfb6bcc6c-nmvgm\" (UID: \"6c86ba36-39a2-4e94-ba85-f195dfd115bd\") " pod="ai-tenant-e2e-shared-a-3550d9/auth-test-model-3550d9-kserve-6bfb6bcc6c-nmvgm" Jul 06 19:31:13.906494 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:31:13.906414 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/6c86ba36-39a2-4e94-ba85-f195dfd115bd-tls-certs\") pod \"auth-test-model-3550d9-kserve-6bfb6bcc6c-nmvgm\" (UID: \"6c86ba36-39a2-4e94-ba85-f195dfd115bd\") " pod="ai-tenant-e2e-shared-a-3550d9/auth-test-model-3550d9-kserve-6bfb6bcc6c-nmvgm" Jul 06 19:31:13.906494 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:31:13.906432 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/6c86ba36-39a2-4e94-ba85-f195dfd115bd-dshm\") pod \"auth-test-model-3550d9-kserve-6bfb6bcc6c-nmvgm\" (UID: \"6c86ba36-39a2-4e94-ba85-f195dfd115bd\") " pod="ai-tenant-e2e-shared-a-3550d9/auth-test-model-3550d9-kserve-6bfb6bcc6c-nmvgm" Jul 06 19:31:13.906563 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:31:13.906541 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/6c86ba36-39a2-4e94-ba85-f195dfd115bd-model-cache\") pod \"auth-test-model-3550d9-kserve-6bfb6bcc6c-nmvgm\" (UID: \"6c86ba36-39a2-4e94-ba85-f195dfd115bd\") " pod="ai-tenant-e2e-shared-a-3550d9/auth-test-model-3550d9-kserve-6bfb6bcc6c-nmvgm" Jul 06 19:31:13.906597 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:31:13.906578 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/6c86ba36-39a2-4e94-ba85-f195dfd115bd-tmp-dir\") pod \"auth-test-model-3550d9-kserve-6bfb6bcc6c-nmvgm\" (UID: \"6c86ba36-39a2-4e94-ba85-f195dfd115bd\") " pod="ai-tenant-e2e-shared-a-3550d9/auth-test-model-3550d9-kserve-6bfb6bcc6c-nmvgm" Jul 06 19:31:13.906708 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:31:13.906670 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7zq24\" (UniqueName: \"kubernetes.io/projected/6c86ba36-39a2-4e94-ba85-f195dfd115bd-kube-api-access-7zq24\") pod \"auth-test-model-3550d9-kserve-6bfb6bcc6c-nmvgm\" (UID: \"6c86ba36-39a2-4e94-ba85-f195dfd115bd\") " pod="ai-tenant-e2e-shared-a-3550d9/auth-test-model-3550d9-kserve-6bfb6bcc6c-nmvgm" Jul 06 19:31:13.906817 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:31:13.906796 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/6c86ba36-39a2-4e94-ba85-f195dfd115bd-home\") pod \"auth-test-model-3550d9-kserve-6bfb6bcc6c-nmvgm\" (UID: \"6c86ba36-39a2-4e94-ba85-f195dfd115bd\") " pod="ai-tenant-e2e-shared-a-3550d9/auth-test-model-3550d9-kserve-6bfb6bcc6c-nmvgm" Jul 06 19:31:13.906903 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:31:13.906883 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/6c86ba36-39a2-4e94-ba85-f195dfd115bd-model-cache\") pod \"auth-test-model-3550d9-kserve-6bfb6bcc6c-nmvgm\" (UID: \"6c86ba36-39a2-4e94-ba85-f195dfd115bd\") " pod="ai-tenant-e2e-shared-a-3550d9/auth-test-model-3550d9-kserve-6bfb6bcc6c-nmvgm" Jul 06 19:31:13.906987 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:31:13.906976 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/6c86ba36-39a2-4e94-ba85-f195dfd115bd-tmp-dir\") pod \"auth-test-model-3550d9-kserve-6bfb6bcc6c-nmvgm\" (UID: \"6c86ba36-39a2-4e94-ba85-f195dfd115bd\") " pod="ai-tenant-e2e-shared-a-3550d9/auth-test-model-3550d9-kserve-6bfb6bcc6c-nmvgm" Jul 06 19:31:13.908691 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:31:13.908659 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/6c86ba36-39a2-4e94-ba85-f195dfd115bd-dshm\") pod \"auth-test-model-3550d9-kserve-6bfb6bcc6c-nmvgm\" (UID: \"6c86ba36-39a2-4e94-ba85-f195dfd115bd\") " pod="ai-tenant-e2e-shared-a-3550d9/auth-test-model-3550d9-kserve-6bfb6bcc6c-nmvgm" Jul 06 19:31:13.908922 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:31:13.908907 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/6c86ba36-39a2-4e94-ba85-f195dfd115bd-tls-certs\") pod \"auth-test-model-3550d9-kserve-6bfb6bcc6c-nmvgm\" (UID: \"6c86ba36-39a2-4e94-ba85-f195dfd115bd\") " pod="ai-tenant-e2e-shared-a-3550d9/auth-test-model-3550d9-kserve-6bfb6bcc6c-nmvgm" Jul 06 19:31:13.916834 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:31:13.916814 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-7zq24\" (UniqueName: \"kubernetes.io/projected/6c86ba36-39a2-4e94-ba85-f195dfd115bd-kube-api-access-7zq24\") pod \"auth-test-model-3550d9-kserve-6bfb6bcc6c-nmvgm\" (UID: \"6c86ba36-39a2-4e94-ba85-f195dfd115bd\") " pod="ai-tenant-e2e-shared-a-3550d9/auth-test-model-3550d9-kserve-6bfb6bcc6c-nmvgm" Jul 06 19:31:14.017513 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:31:14.017461 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-3550d9/auth-test-model-3550d9-kserve-6bfb6bcc6c-nmvgm" Jul 06 19:31:14.226398 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:31:14.226373 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-3550d9/auth-test-model-3550d9-kserve-6bfb6bcc6c-nmvgm"] Jul 06 19:31:14.228415 ip-10-0-130-59 kubenswrapper[2570]: W0706 19:31:14.228386 2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod6c86ba36_39a2_4e94_ba85_f195dfd115bd.slice/crio-31c9ae2d4595bfc0f6e1c9b7bbdafd10c2a2eba952a6f65194224e0dcf6c618c WatchSource:0}: Error finding container 31c9ae2d4595bfc0f6e1c9b7bbdafd10c2a2eba952a6f65194224e0dcf6c618c: Status 404 returned error can't find the container with id 31c9ae2d4595bfc0f6e1c9b7bbdafd10c2a2eba952a6f65194224e0dcf6c618c Jul 06 19:31:15.104851 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:31:15.104814 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k8fbq_bc7057bd-88da-48af-ba32-e6efdc6e8362/konnectivity-agent/0.log" limit=1024 Jul 06 19:31:15.207426 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:31:15.207397 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k8fbq_bc7057bd-88da-48af-ba32-e6efdc6e8362/konnectivity-agent/0.log" limit=1024 Jul 06 19:31:15.217472 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:31:15.217449 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-3550d9/auth-test-model-3550d9-kserve-6bfb6bcc6c-nmvgm" event={"ID":"6c86ba36-39a2-4e94-ba85-f195dfd115bd","Type":"ContainerStarted","Data":"2af72a18c61a05676fca770b766183cfbd743d5e290a3d820ff9b36b335dc5c5"} Jul 06 19:31:15.217592 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:31:15.217483 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-3550d9/auth-test-model-3550d9-kserve-6bfb6bcc6c-nmvgm" event={"ID":"6c86ba36-39a2-4e94-ba85-f195dfd115bd","Type":"ContainerStarted","Data":"31c9ae2d4595bfc0f6e1c9b7bbdafd10c2a2eba952a6f65194224e0dcf6c618c"} Jul 06 19:31:15.245425 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:31:15.245360 2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-a-3550d9/auth-test-model-3550d9-kserve-6bfb6bcc6c-nmvgm" podStartSLOduration=2.245342364 podStartE2EDuration="2.245342364s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 19:31:13 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 19:31:15.244800722 +0000 UTC m=+2154.799821896" watchObservedRunningTime="2026-07-06 19:31:15.245342364 +0000 UTC m=+2154.800363537" Jul 06 19:31:18.702032 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:31:18.701999 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-flg9z"] Jul 06 19:31:22.884390 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:31:22.884360 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k8fbq_bc7057bd-88da-48af-ba32-e6efdc6e8362/konnectivity-agent/0.log" limit=1024 Jul 06 19:31:22.978128 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:31:22.978093 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k8fbq_bc7057bd-88da-48af-ba32-e6efdc6e8362/konnectivity-agent/0.log" limit=1024 Jul 06 19:31:24.017858 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:31:24.017822 2570 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-a-3550d9/auth-test-model-3550d9-kserve-6bfb6bcc6c-nmvgm" Jul 06 19:31:24.018220 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:31:24.017878 2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-a-3550d9/auth-test-model-3550d9-kserve-6bfb6bcc6c-nmvgm" Jul 06 19:31:24.029988 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:31:24.029967 2570 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-a-3550d9/auth-test-model-3550d9-kserve-6bfb6bcc6c-nmvgm" Jul 06 19:31:24.270379 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:31:24.270314 2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-a-3550d9/auth-test-model-3550d9-kserve-6bfb6bcc6c-nmvgm" Jul 06 19:31:38.147641 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:31:38.147589 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k8fbq_bc7057bd-88da-48af-ba32-e6efdc6e8362/konnectivity-agent/0.log" limit=1024 Jul 06 19:31:41.820261 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:31:41.820221 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-flg9z"] Jul 06 19:31:46.791832 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:31:46.791799 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k8fbq_bc7057bd-88da-48af-ba32-e6efdc6e8362/konnectivity-agent/0.log" limit=1024 Jul 06 19:31:50.726769 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:31:50.726728 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k8fbq_bc7057bd-88da-48af-ba32-e6efdc6e8362/konnectivity-agent/0.log" limit=1024 Jul 06 19:32:11.918195 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:32:11.918158 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-flg9z"] Jul 06 19:32:21.173778 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:32:21.173739 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k8fbq_bc7057bd-88da-48af-ba32-e6efdc6e8362/konnectivity-agent/0.log" limit=1024 Jul 06 19:32:32.113496 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:32:32.113465 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-flg9z"] Jul 06 19:32:37.522401 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:32:37.522360 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k8fbq_bc7057bd-88da-48af-ba32-e6efdc6e8362/konnectivity-agent/0.log" limit=1024 Jul 06 19:32:37.613269 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:32:37.613244 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k8fbq_bc7057bd-88da-48af-ba32-e6efdc6e8362/konnectivity-agent/0.log" limit=1024 Jul 06 19:32:47.159073 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:32:47.158987 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-flg9z"] Jul 06 19:33:15.802383 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:33:15.802351 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-flg9z"] Jul 06 19:33:33.699429 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:33:33.699398 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-flg9z"] Jul 06 19:33:49.506319 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:33:49.506291 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-flg9z"] Jul 06 19:34:01.035802 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:34:01.035774 2570 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-3550d9/auth-test-model-3550d9-kserve-6bfb6bcc6c-nmvgm"] Jul 06 19:34:01.050076 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:34:01.050050 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k8fbq_bc7057bd-88da-48af-ba32-e6efdc6e8362/konnectivity-agent/0.log" limit=1024 Jul 06 19:34:03.112581 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:34:03.112542 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k8fbq_bc7057bd-88da-48af-ba32-e6efdc6e8362/konnectivity-agent/0.log" limit=1024 Jul 06 19:34:04.496836 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:34:04.496807 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-flg9z"] Jul 06 19:34:16.055757 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:34:16.055646 2570 kuberuntime_container.go:872] "Killing container with a grace period" pod="ai-tenant-e2e-shared-a-3550d9/auth-test-model-3550d9-kserve-6bfb6bcc6c-nmvgm" podUID="6c86ba36-39a2-4e94-ba85-f195dfd115bd" containerName="main" containerID="cri-o://2af72a18c61a05676fca770b766183cfbd743d5e290a3d820ff9b36b335dc5c5" gracePeriod=45 Jul 06 19:34:16.299129 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:34:16.299105 2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-3550d9/auth-test-model-3550d9-kserve-6bfb6bcc6c-nmvgm" Jul 06 19:34:16.374998 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:34:16.374925 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/6c86ba36-39a2-4e94-ba85-f195dfd115bd-dshm\") pod \"6c86ba36-39a2-4e94-ba85-f195dfd115bd\" (UID: \"6c86ba36-39a2-4e94-ba85-f195dfd115bd\") " Jul 06 19:34:16.374998 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:34:16.374965 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-7zq24\" (UniqueName: \"kubernetes.io/projected/6c86ba36-39a2-4e94-ba85-f195dfd115bd-kube-api-access-7zq24\") pod \"6c86ba36-39a2-4e94-ba85-f195dfd115bd\" (UID: \"6c86ba36-39a2-4e94-ba85-f195dfd115bd\") " Jul 06 19:34:16.375286 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:34:16.375064 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/6c86ba36-39a2-4e94-ba85-f195dfd115bd-home\") pod \"6c86ba36-39a2-4e94-ba85-f195dfd115bd\" (UID: \"6c86ba36-39a2-4e94-ba85-f195dfd115bd\") " Jul 06 19:34:16.375286 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:34:16.375181 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/6c86ba36-39a2-4e94-ba85-f195dfd115bd-model-cache\") pod \"6c86ba36-39a2-4e94-ba85-f195dfd115bd\" (UID: \"6c86ba36-39a2-4e94-ba85-f195dfd115bd\") " Jul 06 19:34:16.375286 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:34:16.375218 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/6c86ba36-39a2-4e94-ba85-f195dfd115bd-tmp-dir\") pod \"6c86ba36-39a2-4e94-ba85-f195dfd115bd\" (UID: \"6c86ba36-39a2-4e94-ba85-f195dfd115bd\") " Jul 06 19:34:16.375286 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:34:16.375245 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/6c86ba36-39a2-4e94-ba85-f195dfd115bd-tls-certs\") pod \"6c86ba36-39a2-4e94-ba85-f195dfd115bd\" (UID: \"6c86ba36-39a2-4e94-ba85-f195dfd115bd\") " Jul 06 19:34:16.375286 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:34:16.375270 2570 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6c86ba36-39a2-4e94-ba85-f195dfd115bd-home" (OuterVolumeSpecName: "home") pod "6c86ba36-39a2-4e94-ba85-f195dfd115bd" (UID: "6c86ba36-39a2-4e94-ba85-f195dfd115bd"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 19:34:16.375514 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:34:16.375417 2570 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6c86ba36-39a2-4e94-ba85-f195dfd115bd-model-cache" (OuterVolumeSpecName: "model-cache") pod "6c86ba36-39a2-4e94-ba85-f195dfd115bd" (UID: "6c86ba36-39a2-4e94-ba85-f195dfd115bd"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 19:34:16.375514 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:34:16.375461 2570 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6c86ba36-39a2-4e94-ba85-f195dfd115bd-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "6c86ba36-39a2-4e94-ba85-f195dfd115bd" (UID: "6c86ba36-39a2-4e94-ba85-f195dfd115bd"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 19:34:16.375588 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:34:16.375577 2570 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/6c86ba36-39a2-4e94-ba85-f195dfd115bd-tmp-dir\") on node \"ip-10-0-130-59.ec2.internal\" DevicePath \"\"" Jul 06 19:34:16.375660 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:34:16.375596 2570 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/6c86ba36-39a2-4e94-ba85-f195dfd115bd-home\") on node \"ip-10-0-130-59.ec2.internal\" DevicePath \"\"" Jul 06 19:34:16.375660 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:34:16.375631 2570 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/6c86ba36-39a2-4e94-ba85-f195dfd115bd-model-cache\") on node \"ip-10-0-130-59.ec2.internal\" DevicePath \"\"" Jul 06 19:34:16.377130 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:34:16.377108 2570 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6c86ba36-39a2-4e94-ba85-f195dfd115bd-dshm" (OuterVolumeSpecName: "dshm") pod "6c86ba36-39a2-4e94-ba85-f195dfd115bd" (UID: "6c86ba36-39a2-4e94-ba85-f195dfd115bd"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 19:34:16.377183 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:34:16.377151 2570 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6c86ba36-39a2-4e94-ba85-f195dfd115bd-kube-api-access-7zq24" (OuterVolumeSpecName: "kube-api-access-7zq24") pod "6c86ba36-39a2-4e94-ba85-f195dfd115bd" (UID: "6c86ba36-39a2-4e94-ba85-f195dfd115bd"). InnerVolumeSpecName "kube-api-access-7zq24". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 06 19:34:16.377283 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:34:16.377265 2570 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/6c86ba36-39a2-4e94-ba85-f195dfd115bd-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "6c86ba36-39a2-4e94-ba85-f195dfd115bd" (UID: "6c86ba36-39a2-4e94-ba85-f195dfd115bd"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 06 19:34:16.476557 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:34:16.476528 2570 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/6c86ba36-39a2-4e94-ba85-f195dfd115bd-tls-certs\") on node \"ip-10-0-130-59.ec2.internal\" DevicePath \"\"" Jul 06 19:34:16.476557 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:34:16.476551 2570 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/6c86ba36-39a2-4e94-ba85-f195dfd115bd-dshm\") on node \"ip-10-0-130-59.ec2.internal\" DevicePath \"\"" Jul 06 19:34:16.476744 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:34:16.476565 2570 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-7zq24\" (UniqueName: \"kubernetes.io/projected/6c86ba36-39a2-4e94-ba85-f195dfd115bd-kube-api-access-7zq24\") on node \"ip-10-0-130-59.ec2.internal\" DevicePath \"\"" Jul 06 19:34:17.091786 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:34:17.091756 2570 generic.go:356] "Generic (PLEG): container finished" podID="6c86ba36-39a2-4e94-ba85-f195dfd115bd" containerID="2af72a18c61a05676fca770b766183cfbd743d5e290a3d820ff9b36b335dc5c5" exitCode=0 Jul 06 19:34:17.092227 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:34:17.091795 2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-3550d9/auth-test-model-3550d9-kserve-6bfb6bcc6c-nmvgm" Jul 06 19:34:17.092227 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:34:17.091839 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-3550d9/auth-test-model-3550d9-kserve-6bfb6bcc6c-nmvgm" event={"ID":"6c86ba36-39a2-4e94-ba85-f195dfd115bd","Type":"ContainerDied","Data":"2af72a18c61a05676fca770b766183cfbd743d5e290a3d820ff9b36b335dc5c5"} Jul 06 19:34:17.092227 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:34:17.091876 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-3550d9/auth-test-model-3550d9-kserve-6bfb6bcc6c-nmvgm" event={"ID":"6c86ba36-39a2-4e94-ba85-f195dfd115bd","Type":"ContainerDied","Data":"31c9ae2d4595bfc0f6e1c9b7bbdafd10c2a2eba952a6f65194224e0dcf6c618c"} Jul 06 19:34:17.092227 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:34:17.091894 2570 scope.go:117] "RemoveContainer" containerID="2af72a18c61a05676fca770b766183cfbd743d5e290a3d820ff9b36b335dc5c5" Jul 06 19:34:17.100046 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:34:17.100020 2570 scope.go:117] "RemoveContainer" containerID="2af72a18c61a05676fca770b766183cfbd743d5e290a3d820ff9b36b335dc5c5" Jul 06 19:34:17.100307 ip-10-0-130-59 kubenswrapper[2570]: E0706 19:34:17.100288 2570 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2af72a18c61a05676fca770b766183cfbd743d5e290a3d820ff9b36b335dc5c5\": container with ID starting with 2af72a18c61a05676fca770b766183cfbd743d5e290a3d820ff9b36b335dc5c5 not found: ID does not exist" containerID="2af72a18c61a05676fca770b766183cfbd743d5e290a3d820ff9b36b335dc5c5" Jul 06 19:34:17.100361 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:34:17.100314 2570 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2af72a18c61a05676fca770b766183cfbd743d5e290a3d820ff9b36b335dc5c5"} err="failed to get container status \"2af72a18c61a05676fca770b766183cfbd743d5e290a3d820ff9b36b335dc5c5\": rpc error: code = NotFound desc = could not find container \"2af72a18c61a05676fca770b766183cfbd743d5e290a3d820ff9b36b335dc5c5\": container with ID starting with 2af72a18c61a05676fca770b766183cfbd743d5e290a3d820ff9b36b335dc5c5 not found: ID does not exist" Jul 06 19:34:17.118813 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:34:17.118788 2570 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-3550d9/auth-test-model-3550d9-kserve-6bfb6bcc6c-nmvgm"] Jul 06 19:34:17.128008 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:34:17.127986 2570 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-a-3550d9/auth-test-model-3550d9-kserve-6bfb6bcc6c-nmvgm"] Jul 06 19:34:19.033734 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:34:19.033701 2570 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="6c86ba36-39a2-4e94-ba85-f195dfd115bd" path="/var/lib/kubelet/pods/6c86ba36-39a2-4e94-ba85-f195dfd115bd/volumes" Jul 06 19:34:29.902924 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:34:29.902895 2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-a-3550d9/sub-test-model-3550d9-kserve-f7579f8dc-tp46f"] Jul 06 19:34:29.903293 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:34:29.903253 2570 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6c86ba36-39a2-4e94-ba85-f195dfd115bd" containerName="main" Jul 06 19:34:29.903293 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:34:29.903264 2570 state_mem.go:107] "Deleted CPUSet assignment" podUID="6c86ba36-39a2-4e94-ba85-f195dfd115bd" containerName="main" Jul 06 19:34:29.903406 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:34:29.903334 2570 memory_manager.go:368] "RemoveStaleState removing state" podUID="6c86ba36-39a2-4e94-ba85-f195dfd115bd" containerName="main" Jul 06 19:34:29.906287 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:34:29.906271 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-3550d9/sub-test-model-3550d9-kserve-f7579f8dc-tp46f" Jul 06 19:34:29.908814 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:34:29.908794 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-3550d9\"/\"kube-root-ca.crt\"" Jul 06 19:34:29.910343 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:34:29.910320 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-3550d9\"/\"sub-test-model-3550d9-kserve-self-signed-certs\"" Jul 06 19:34:29.910482 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:34:29.910377 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-3550d9\"/\"default-dockercfg-cv6ch\"" Jul 06 19:34:29.910560 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:34:29.910545 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-3550d9\"/\"openshift-service-ca.crt\"" Jul 06 19:34:29.923791 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:34:29.923751 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k8fbq_bc7057bd-88da-48af-ba32-e6efdc6e8362/konnectivity-agent/0.log" limit=1024 Jul 06 19:34:29.925259 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:34:29.925236 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-3550d9/sub-test-model-3550d9-kserve-f7579f8dc-tp46f"] Jul 06 19:34:29.990925 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:34:29.990896 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3812a9b5-6073-4dcc-9ba6-95a4cb5821ca-tmp-dir\") pod \"sub-test-model-3550d9-kserve-f7579f8dc-tp46f\" (UID: \"3812a9b5-6073-4dcc-9ba6-95a4cb5821ca\") " pod="ai-tenant-e2e-shared-a-3550d9/sub-test-model-3550d9-kserve-f7579f8dc-tp46f" Jul 06 19:34:29.991036 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:34:29.990954 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/3812a9b5-6073-4dcc-9ba6-95a4cb5821ca-dshm\") pod \"sub-test-model-3550d9-kserve-f7579f8dc-tp46f\" (UID: \"3812a9b5-6073-4dcc-9ba6-95a4cb5821ca\") " pod="ai-tenant-e2e-shared-a-3550d9/sub-test-model-3550d9-kserve-f7579f8dc-tp46f" Jul 06 19:34:29.991036 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:34:29.991005 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/3812a9b5-6073-4dcc-9ba6-95a4cb5821ca-model-cache\") pod \"sub-test-model-3550d9-kserve-f7579f8dc-tp46f\" (UID: \"3812a9b5-6073-4dcc-9ba6-95a4cb5821ca\") " pod="ai-tenant-e2e-shared-a-3550d9/sub-test-model-3550d9-kserve-f7579f8dc-tp46f" Jul 06 19:34:29.991125 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:34:29.991036 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/3812a9b5-6073-4dcc-9ba6-95a4cb5821ca-tls-certs\") pod \"sub-test-model-3550d9-kserve-f7579f8dc-tp46f\" (UID: \"3812a9b5-6073-4dcc-9ba6-95a4cb5821ca\") " pod="ai-tenant-e2e-shared-a-3550d9/sub-test-model-3550d9-kserve-f7579f8dc-tp46f" Jul 06 19:34:29.991164 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:34:29.991124 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/3812a9b5-6073-4dcc-9ba6-95a4cb5821ca-home\") pod \"sub-test-model-3550d9-kserve-f7579f8dc-tp46f\" (UID: \"3812a9b5-6073-4dcc-9ba6-95a4cb5821ca\") " pod="ai-tenant-e2e-shared-a-3550d9/sub-test-model-3550d9-kserve-f7579f8dc-tp46f" Jul 06 19:34:29.991199 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:34:29.991171 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-t2mpb\" (UniqueName: \"kubernetes.io/projected/3812a9b5-6073-4dcc-9ba6-95a4cb5821ca-kube-api-access-t2mpb\") pod \"sub-test-model-3550d9-kserve-f7579f8dc-tp46f\" (UID: \"3812a9b5-6073-4dcc-9ba6-95a4cb5821ca\") " pod="ai-tenant-e2e-shared-a-3550d9/sub-test-model-3550d9-kserve-f7579f8dc-tp46f" Jul 06 19:34:30.092376 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:34:30.092344 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-t2mpb\" (UniqueName: \"kubernetes.io/projected/3812a9b5-6073-4dcc-9ba6-95a4cb5821ca-kube-api-access-t2mpb\") pod \"sub-test-model-3550d9-kserve-f7579f8dc-tp46f\" (UID: \"3812a9b5-6073-4dcc-9ba6-95a4cb5821ca\") " pod="ai-tenant-e2e-shared-a-3550d9/sub-test-model-3550d9-kserve-f7579f8dc-tp46f" Jul 06 19:34:30.092493 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:34:30.092393 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3812a9b5-6073-4dcc-9ba6-95a4cb5821ca-tmp-dir\") pod \"sub-test-model-3550d9-kserve-f7579f8dc-tp46f\" (UID: \"3812a9b5-6073-4dcc-9ba6-95a4cb5821ca\") " pod="ai-tenant-e2e-shared-a-3550d9/sub-test-model-3550d9-kserve-f7579f8dc-tp46f" Jul 06 19:34:30.092802 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:34:30.092780 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3812a9b5-6073-4dcc-9ba6-95a4cb5821ca-tmp-dir\") pod \"sub-test-model-3550d9-kserve-f7579f8dc-tp46f\" (UID: \"3812a9b5-6073-4dcc-9ba6-95a4cb5821ca\") " pod="ai-tenant-e2e-shared-a-3550d9/sub-test-model-3550d9-kserve-f7579f8dc-tp46f" Jul 06 19:34:30.092915 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:34:30.092842 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/3812a9b5-6073-4dcc-9ba6-95a4cb5821ca-dshm\") pod \"sub-test-model-3550d9-kserve-f7579f8dc-tp46f\" (UID: \"3812a9b5-6073-4dcc-9ba6-95a4cb5821ca\") " pod="ai-tenant-e2e-shared-a-3550d9/sub-test-model-3550d9-kserve-f7579f8dc-tp46f" Jul 06 19:34:30.092915 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:34:30.092884 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/3812a9b5-6073-4dcc-9ba6-95a4cb5821ca-model-cache\") pod \"sub-test-model-3550d9-kserve-f7579f8dc-tp46f\" (UID: \"3812a9b5-6073-4dcc-9ba6-95a4cb5821ca\") " pod="ai-tenant-e2e-shared-a-3550d9/sub-test-model-3550d9-kserve-f7579f8dc-tp46f" Jul 06 19:34:30.093060 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:34:30.092929 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/3812a9b5-6073-4dcc-9ba6-95a4cb5821ca-tls-certs\") pod \"sub-test-model-3550d9-kserve-f7579f8dc-tp46f\" (UID: \"3812a9b5-6073-4dcc-9ba6-95a4cb5821ca\") " pod="ai-tenant-e2e-shared-a-3550d9/sub-test-model-3550d9-kserve-f7579f8dc-tp46f" Jul 06 19:34:30.093060 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:34:30.093029 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/3812a9b5-6073-4dcc-9ba6-95a4cb5821ca-home\") pod \"sub-test-model-3550d9-kserve-f7579f8dc-tp46f\" (UID: \"3812a9b5-6073-4dcc-9ba6-95a4cb5821ca\") " pod="ai-tenant-e2e-shared-a-3550d9/sub-test-model-3550d9-kserve-f7579f8dc-tp46f" Jul 06 19:34:30.093232 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:34:30.093191 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/3812a9b5-6073-4dcc-9ba6-95a4cb5821ca-model-cache\") pod \"sub-test-model-3550d9-kserve-f7579f8dc-tp46f\" (UID: \"3812a9b5-6073-4dcc-9ba6-95a4cb5821ca\") " pod="ai-tenant-e2e-shared-a-3550d9/sub-test-model-3550d9-kserve-f7579f8dc-tp46f" Jul 06 19:34:30.093360 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:34:30.093294 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/3812a9b5-6073-4dcc-9ba6-95a4cb5821ca-home\") pod \"sub-test-model-3550d9-kserve-f7579f8dc-tp46f\" (UID: \"3812a9b5-6073-4dcc-9ba6-95a4cb5821ca\") " pod="ai-tenant-e2e-shared-a-3550d9/sub-test-model-3550d9-kserve-f7579f8dc-tp46f" Jul 06 19:34:30.094895 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:34:30.094872 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/3812a9b5-6073-4dcc-9ba6-95a4cb5821ca-dshm\") pod \"sub-test-model-3550d9-kserve-f7579f8dc-tp46f\" (UID: \"3812a9b5-6073-4dcc-9ba6-95a4cb5821ca\") " pod="ai-tenant-e2e-shared-a-3550d9/sub-test-model-3550d9-kserve-f7579f8dc-tp46f" Jul 06 19:34:30.095279 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:34:30.095258 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/3812a9b5-6073-4dcc-9ba6-95a4cb5821ca-tls-certs\") pod \"sub-test-model-3550d9-kserve-f7579f8dc-tp46f\" (UID: \"3812a9b5-6073-4dcc-9ba6-95a4cb5821ca\") " pod="ai-tenant-e2e-shared-a-3550d9/sub-test-model-3550d9-kserve-f7579f8dc-tp46f" Jul 06 19:34:30.100662 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:34:30.100644 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-t2mpb\" (UniqueName: \"kubernetes.io/projected/3812a9b5-6073-4dcc-9ba6-95a4cb5821ca-kube-api-access-t2mpb\") pod \"sub-test-model-3550d9-kserve-f7579f8dc-tp46f\" (UID: \"3812a9b5-6073-4dcc-9ba6-95a4cb5821ca\") " pod="ai-tenant-e2e-shared-a-3550d9/sub-test-model-3550d9-kserve-f7579f8dc-tp46f" Jul 06 19:34:30.222932 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:34:30.222913 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-3550d9/sub-test-model-3550d9-kserve-f7579f8dc-tp46f" Jul 06 19:34:30.359665 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:34:30.359637 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-3550d9/sub-test-model-3550d9-kserve-f7579f8dc-tp46f"] Jul 06 19:34:30.362004 ip-10-0-130-59 kubenswrapper[2570]: W0706 19:34:30.361981 2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod3812a9b5_6073_4dcc_9ba6_95a4cb5821ca.slice/crio-1535ae591d58536a98fdb91485ddf17221755c427d4f7a02ad4ef6a226b37c4d WatchSource:0}: Error finding container 1535ae591d58536a98fdb91485ddf17221755c427d4f7a02ad4ef6a226b37c4d: Status 404 returned error can't find the container with id 1535ae591d58536a98fdb91485ddf17221755c427d4f7a02ad4ef6a226b37c4d Jul 06 19:34:31.160528 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:34:31.160485 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-3550d9/sub-test-model-3550d9-kserve-f7579f8dc-tp46f" event={"ID":"3812a9b5-6073-4dcc-9ba6-95a4cb5821ca","Type":"ContainerStarted","Data":"cecbace026bfaead8bc68636d425dd2699f8e39b2f162deaa5cde49abccec28d"} Jul 06 19:34:31.160528 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:34:31.160529 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-3550d9/sub-test-model-3550d9-kserve-f7579f8dc-tp46f" event={"ID":"3812a9b5-6073-4dcc-9ba6-95a4cb5821ca","Type":"ContainerStarted","Data":"1535ae591d58536a98fdb91485ddf17221755c427d4f7a02ad4ef6a226b37c4d"} Jul 06 19:34:31.178861 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:34:31.178820 2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-a-3550d9/sub-test-model-3550d9-kserve-f7579f8dc-tp46f" podStartSLOduration=2.178805365 podStartE2EDuration="2.178805365s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 19:34:29 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 19:34:31.177268078 +0000 UTC m=+2350.732289270" watchObservedRunningTime="2026-07-06 19:34:31.178805365 +0000 UTC m=+2350.733826598" Jul 06 19:34:31.651465 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:34:31.651432 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k8fbq_bc7057bd-88da-48af-ba32-e6efdc6e8362/konnectivity-agent/0.log" limit=1024 Jul 06 19:34:31.753935 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:34:31.753909 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k8fbq_bc7057bd-88da-48af-ba32-e6efdc6e8362/konnectivity-agent/0.log" limit=1024 Jul 06 19:34:33.893377 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:34:33.893330 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-flg9z"] Jul 06 19:34:40.224070 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:34:40.224037 2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-a-3550d9/sub-test-model-3550d9-kserve-f7579f8dc-tp46f" Jul 06 19:34:40.224454 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:34:40.224091 2570 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-a-3550d9/sub-test-model-3550d9-kserve-f7579f8dc-tp46f" Jul 06 19:34:40.236325 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:34:40.236301 2570 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-a-3550d9/sub-test-model-3550d9-kserve-f7579f8dc-tp46f" Jul 06 19:34:41.216669 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:34:41.216641 2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-a-3550d9/sub-test-model-3550d9-kserve-f7579f8dc-tp46f" Jul 06 19:34:54.249782 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:34:54.249745 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k8fbq_bc7057bd-88da-48af-ba32-e6efdc6e8362/konnectivity-agent/0.log" limit=1024 Jul 06 19:34:58.704268 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:34:58.704231 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-flg9z"] Jul 06 19:35:19.894842 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:35:19.894800 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-flg9z"] Jul 06 19:35:37.743030 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:35:37.742991 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-flg9z"] Jul 06 19:35:42.872074 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:35:42.871997 2570 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-3550d9/sub-test-model-3550d9-kserve-f7579f8dc-tp46f"] Jul 06 19:35:42.899800 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:35:42.899770 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k8fbq_bc7057bd-88da-48af-ba32-e6efdc6e8362/konnectivity-agent/0.log" limit=1024 Jul 06 19:35:43.808104 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:35:43.808064 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k8fbq_bc7057bd-88da-48af-ba32-e6efdc6e8362/konnectivity-agent/0.log" limit=1024 Jul 06 19:35:48.203854 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:35:48.203820 2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-a-3550d9/rate-test-model-3550d9-kserve-6578856955-5gct2"] Jul 06 19:35:48.207328 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:35:48.207309 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-3550d9/rate-test-model-3550d9-kserve-6578856955-5gct2" Jul 06 19:35:48.211386 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:35:48.211362 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-3550d9\"/\"rate-test-model-3550d9-kserve-self-signed-certs\"" Jul 06 19:35:48.216242 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:35:48.216220 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-3550d9/rate-test-model-3550d9-kserve-6578856955-5gct2"] Jul 06 19:35:48.237126 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:35:48.237104 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k8fbq_bc7057bd-88da-48af-ba32-e6efdc6e8362/konnectivity-agent/0.log" limit=1024 Jul 06 19:35:48.304370 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:35:48.304342 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/c58818ef-10a8-4370-ad2b-1fbc23a2c531-dshm\") pod \"rate-test-model-3550d9-kserve-6578856955-5gct2\" (UID: \"c58818ef-10a8-4370-ad2b-1fbc23a2c531\") " pod="ai-tenant-e2e-shared-a-3550d9/rate-test-model-3550d9-kserve-6578856955-5gct2" Jul 06 19:35:48.304489 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:35:48.304395 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dkljp\" (UniqueName: \"kubernetes.io/projected/c58818ef-10a8-4370-ad2b-1fbc23a2c531-kube-api-access-dkljp\") pod \"rate-test-model-3550d9-kserve-6578856955-5gct2\" (UID: \"c58818ef-10a8-4370-ad2b-1fbc23a2c531\") " pod="ai-tenant-e2e-shared-a-3550d9/rate-test-model-3550d9-kserve-6578856955-5gct2" Jul 06 19:35:48.304489 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:35:48.304476 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/c58818ef-10a8-4370-ad2b-1fbc23a2c531-model-cache\") pod \"rate-test-model-3550d9-kserve-6578856955-5gct2\" (UID: \"c58818ef-10a8-4370-ad2b-1fbc23a2c531\") " pod="ai-tenant-e2e-shared-a-3550d9/rate-test-model-3550d9-kserve-6578856955-5gct2" Jul 06 19:35:48.304562 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:35:48.304506 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/c58818ef-10a8-4370-ad2b-1fbc23a2c531-home\") pod \"rate-test-model-3550d9-kserve-6578856955-5gct2\" (UID: \"c58818ef-10a8-4370-ad2b-1fbc23a2c531\") " pod="ai-tenant-e2e-shared-a-3550d9/rate-test-model-3550d9-kserve-6578856955-5gct2" Jul 06 19:35:48.304562 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:35:48.304528 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c58818ef-10a8-4370-ad2b-1fbc23a2c531-tls-certs\") pod \"rate-test-model-3550d9-kserve-6578856955-5gct2\" (UID: \"c58818ef-10a8-4370-ad2b-1fbc23a2c531\") " pod="ai-tenant-e2e-shared-a-3550d9/rate-test-model-3550d9-kserve-6578856955-5gct2" Jul 06 19:35:48.304653 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:35:48.304592 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c58818ef-10a8-4370-ad2b-1fbc23a2c531-tmp-dir\") pod \"rate-test-model-3550d9-kserve-6578856955-5gct2\" (UID: \"c58818ef-10a8-4370-ad2b-1fbc23a2c531\") " pod="ai-tenant-e2e-shared-a-3550d9/rate-test-model-3550d9-kserve-6578856955-5gct2" Jul 06 19:35:48.405193 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:35:48.405161 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/c58818ef-10a8-4370-ad2b-1fbc23a2c531-model-cache\") pod \"rate-test-model-3550d9-kserve-6578856955-5gct2\" (UID: \"c58818ef-10a8-4370-ad2b-1fbc23a2c531\") " pod="ai-tenant-e2e-shared-a-3550d9/rate-test-model-3550d9-kserve-6578856955-5gct2" Jul 06 19:35:48.405193 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:35:48.405197 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/c58818ef-10a8-4370-ad2b-1fbc23a2c531-home\") pod \"rate-test-model-3550d9-kserve-6578856955-5gct2\" (UID: \"c58818ef-10a8-4370-ad2b-1fbc23a2c531\") " pod="ai-tenant-e2e-shared-a-3550d9/rate-test-model-3550d9-kserve-6578856955-5gct2" Jul 06 19:35:48.405400 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:35:48.405217 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c58818ef-10a8-4370-ad2b-1fbc23a2c531-tls-certs\") pod \"rate-test-model-3550d9-kserve-6578856955-5gct2\" (UID: \"c58818ef-10a8-4370-ad2b-1fbc23a2c531\") " pod="ai-tenant-e2e-shared-a-3550d9/rate-test-model-3550d9-kserve-6578856955-5gct2" Jul 06 19:35:48.405400 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:35:48.405261 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c58818ef-10a8-4370-ad2b-1fbc23a2c531-tmp-dir\") pod \"rate-test-model-3550d9-kserve-6578856955-5gct2\" (UID: \"c58818ef-10a8-4370-ad2b-1fbc23a2c531\") " pod="ai-tenant-e2e-shared-a-3550d9/rate-test-model-3550d9-kserve-6578856955-5gct2" Jul 06 19:35:48.405400 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:35:48.405291 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/c58818ef-10a8-4370-ad2b-1fbc23a2c531-dshm\") pod \"rate-test-model-3550d9-kserve-6578856955-5gct2\" (UID: \"c58818ef-10a8-4370-ad2b-1fbc23a2c531\") " pod="ai-tenant-e2e-shared-a-3550d9/rate-test-model-3550d9-kserve-6578856955-5gct2" Jul 06 19:35:48.405400 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:35:48.405336 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dkljp\" (UniqueName: \"kubernetes.io/projected/c58818ef-10a8-4370-ad2b-1fbc23a2c531-kube-api-access-dkljp\") pod \"rate-test-model-3550d9-kserve-6578856955-5gct2\" (UID: \"c58818ef-10a8-4370-ad2b-1fbc23a2c531\") " pod="ai-tenant-e2e-shared-a-3550d9/rate-test-model-3550d9-kserve-6578856955-5gct2" Jul 06 19:35:48.405644 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:35:48.405554 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/c58818ef-10a8-4370-ad2b-1fbc23a2c531-home\") pod \"rate-test-model-3550d9-kserve-6578856955-5gct2\" (UID: \"c58818ef-10a8-4370-ad2b-1fbc23a2c531\") " pod="ai-tenant-e2e-shared-a-3550d9/rate-test-model-3550d9-kserve-6578856955-5gct2" Jul 06 19:35:48.405711 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:35:48.405657 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/c58818ef-10a8-4370-ad2b-1fbc23a2c531-model-cache\") pod \"rate-test-model-3550d9-kserve-6578856955-5gct2\" (UID: \"c58818ef-10a8-4370-ad2b-1fbc23a2c531\") " pod="ai-tenant-e2e-shared-a-3550d9/rate-test-model-3550d9-kserve-6578856955-5gct2" Jul 06 19:35:48.405750 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:35:48.405735 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c58818ef-10a8-4370-ad2b-1fbc23a2c531-tmp-dir\") pod \"rate-test-model-3550d9-kserve-6578856955-5gct2\" (UID: \"c58818ef-10a8-4370-ad2b-1fbc23a2c531\") " pod="ai-tenant-e2e-shared-a-3550d9/rate-test-model-3550d9-kserve-6578856955-5gct2" Jul 06 19:35:48.407531 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:35:48.407511 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/c58818ef-10a8-4370-ad2b-1fbc23a2c531-dshm\") pod \"rate-test-model-3550d9-kserve-6578856955-5gct2\" (UID: \"c58818ef-10a8-4370-ad2b-1fbc23a2c531\") " pod="ai-tenant-e2e-shared-a-3550d9/rate-test-model-3550d9-kserve-6578856955-5gct2" Jul 06 19:35:48.407849 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:35:48.407833 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c58818ef-10a8-4370-ad2b-1fbc23a2c531-tls-certs\") pod \"rate-test-model-3550d9-kserve-6578856955-5gct2\" (UID: \"c58818ef-10a8-4370-ad2b-1fbc23a2c531\") " pod="ai-tenant-e2e-shared-a-3550d9/rate-test-model-3550d9-kserve-6578856955-5gct2" Jul 06 19:35:48.414126 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:35:48.414101 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-dkljp\" (UniqueName: \"kubernetes.io/projected/c58818ef-10a8-4370-ad2b-1fbc23a2c531-kube-api-access-dkljp\") pod \"rate-test-model-3550d9-kserve-6578856955-5gct2\" (UID: \"c58818ef-10a8-4370-ad2b-1fbc23a2c531\") " pod="ai-tenant-e2e-shared-a-3550d9/rate-test-model-3550d9-kserve-6578856955-5gct2" Jul 06 19:35:48.518393 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:35:48.518313 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-3550d9/rate-test-model-3550d9-kserve-6578856955-5gct2" Jul 06 19:35:48.660756 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:35:48.660734 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-3550d9/rate-test-model-3550d9-kserve-6578856955-5gct2"] Jul 06 19:35:48.662603 ip-10-0-130-59 kubenswrapper[2570]: W0706 19:35:48.662574 2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podc58818ef_10a8_4370_ad2b_1fbc23a2c531.slice/crio-1e01160fae84dc8b03eae9dc3e47a893cc00be1245d32e2780e585b3c1269c9a WatchSource:0}: Error finding container 1e01160fae84dc8b03eae9dc3e47a893cc00be1245d32e2780e585b3c1269c9a: Status 404 returned error can't find the container with id 1e01160fae84dc8b03eae9dc3e47a893cc00be1245d32e2780e585b3c1269c9a Jul 06 19:35:49.530019 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:35:49.529985 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-3550d9/rate-test-model-3550d9-kserve-6578856955-5gct2" event={"ID":"c58818ef-10a8-4370-ad2b-1fbc23a2c531","Type":"ContainerStarted","Data":"5412479f9366fb2b984361ed5a2ceda9232ded06120d80f76f40bb615aa71ac2"} Jul 06 19:35:49.530019 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:35:49.530029 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-3550d9/rate-test-model-3550d9-kserve-6578856955-5gct2" event={"ID":"c58818ef-10a8-4370-ad2b-1fbc23a2c531","Type":"ContainerStarted","Data":"1e01160fae84dc8b03eae9dc3e47a893cc00be1245d32e2780e585b3c1269c9a"} Jul 06 19:35:49.551093 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:35:49.551042 2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-a-3550d9/rate-test-model-3550d9-kserve-6578856955-5gct2" podStartSLOduration=1.551026091 podStartE2EDuration="1.551026091s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 19:35:48 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 19:35:49.549878506 +0000 UTC m=+2429.104899680" watchObservedRunningTime="2026-07-06 19:35:49.551026091 +0000 UTC m=+2429.106047263" Jul 06 19:35:55.699977 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:35:55.699938 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-flg9z"] Jul 06 19:35:57.891661 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:35:57.891628 2570 kuberuntime_container.go:872] "Killing container with a grace period" pod="ai-tenant-e2e-shared-a-3550d9/sub-test-model-3550d9-kserve-f7579f8dc-tp46f" podUID="3812a9b5-6073-4dcc-9ba6-95a4cb5821ca" containerName="main" containerID="cri-o://cecbace026bfaead8bc68636d425dd2699f8e39b2f162deaa5cde49abccec28d" gracePeriod=45 Jul 06 19:35:58.141470 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:35:58.141447 2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-3550d9/sub-test-model-3550d9-kserve-f7579f8dc-tp46f" Jul 06 19:35:58.201628 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:35:58.201589 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/3812a9b5-6073-4dcc-9ba6-95a4cb5821ca-tls-certs\") pod \"3812a9b5-6073-4dcc-9ba6-95a4cb5821ca\" (UID: \"3812a9b5-6073-4dcc-9ba6-95a4cb5821ca\") " Jul 06 19:35:58.201830 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:35:58.201666 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/3812a9b5-6073-4dcc-9ba6-95a4cb5821ca-dshm\") pod \"3812a9b5-6073-4dcc-9ba6-95a4cb5821ca\" (UID: \"3812a9b5-6073-4dcc-9ba6-95a4cb5821ca\") " Jul 06 19:35:58.201830 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:35:58.201700 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3812a9b5-6073-4dcc-9ba6-95a4cb5821ca-tmp-dir\") pod \"3812a9b5-6073-4dcc-9ba6-95a4cb5821ca\" (UID: \"3812a9b5-6073-4dcc-9ba6-95a4cb5821ca\") " Jul 06 19:35:58.201830 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:35:58.201723 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/3812a9b5-6073-4dcc-9ba6-95a4cb5821ca-home\") pod \"3812a9b5-6073-4dcc-9ba6-95a4cb5821ca\" (UID: \"3812a9b5-6073-4dcc-9ba6-95a4cb5821ca\") " Jul 06 19:35:58.201830 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:35:58.201806 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-t2mpb\" (UniqueName: \"kubernetes.io/projected/3812a9b5-6073-4dcc-9ba6-95a4cb5821ca-kube-api-access-t2mpb\") pod \"3812a9b5-6073-4dcc-9ba6-95a4cb5821ca\" (UID: \"3812a9b5-6073-4dcc-9ba6-95a4cb5821ca\") " Jul 06 19:35:58.202040 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:35:58.201874 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/3812a9b5-6073-4dcc-9ba6-95a4cb5821ca-model-cache\") pod \"3812a9b5-6073-4dcc-9ba6-95a4cb5821ca\" (UID: \"3812a9b5-6073-4dcc-9ba6-95a4cb5821ca\") " Jul 06 19:35:58.202040 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:35:58.201956 2570 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3812a9b5-6073-4dcc-9ba6-95a4cb5821ca-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "3812a9b5-6073-4dcc-9ba6-95a4cb5821ca" (UID: "3812a9b5-6073-4dcc-9ba6-95a4cb5821ca"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 19:35:58.202150 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:35:58.202049 2570 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3812a9b5-6073-4dcc-9ba6-95a4cb5821ca-home" (OuterVolumeSpecName: "home") pod "3812a9b5-6073-4dcc-9ba6-95a4cb5821ca" (UID: "3812a9b5-6073-4dcc-9ba6-95a4cb5821ca"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 19:35:58.202226 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:35:58.202201 2570 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3812a9b5-6073-4dcc-9ba6-95a4cb5821ca-model-cache" (OuterVolumeSpecName: "model-cache") pod "3812a9b5-6073-4dcc-9ba6-95a4cb5821ca" (UID: "3812a9b5-6073-4dcc-9ba6-95a4cb5821ca"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 19:35:58.202226 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:35:58.202218 2570 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3812a9b5-6073-4dcc-9ba6-95a4cb5821ca-tmp-dir\") on node \"ip-10-0-130-59.ec2.internal\" DevicePath \"\"" Jul 06 19:35:58.202330 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:35:58.202232 2570 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/3812a9b5-6073-4dcc-9ba6-95a4cb5821ca-home\") on node \"ip-10-0-130-59.ec2.internal\" DevicePath \"\"" Jul 06 19:35:58.203791 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:35:58.203760 2570 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3812a9b5-6073-4dcc-9ba6-95a4cb5821ca-dshm" (OuterVolumeSpecName: "dshm") pod "3812a9b5-6073-4dcc-9ba6-95a4cb5821ca" (UID: "3812a9b5-6073-4dcc-9ba6-95a4cb5821ca"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 19:35:58.203905 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:35:58.203841 2570 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/3812a9b5-6073-4dcc-9ba6-95a4cb5821ca-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "3812a9b5-6073-4dcc-9ba6-95a4cb5821ca" (UID: "3812a9b5-6073-4dcc-9ba6-95a4cb5821ca"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 06 19:35:58.203905 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:35:58.203863 2570 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3812a9b5-6073-4dcc-9ba6-95a4cb5821ca-kube-api-access-t2mpb" (OuterVolumeSpecName: "kube-api-access-t2mpb") pod "3812a9b5-6073-4dcc-9ba6-95a4cb5821ca" (UID: "3812a9b5-6073-4dcc-9ba6-95a4cb5821ca"). InnerVolumeSpecName "kube-api-access-t2mpb". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 06 19:35:58.302897 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:35:58.302865 2570 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/3812a9b5-6073-4dcc-9ba6-95a4cb5821ca-model-cache\") on node \"ip-10-0-130-59.ec2.internal\" DevicePath \"\"" Jul 06 19:35:58.302897 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:35:58.302892 2570 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/3812a9b5-6073-4dcc-9ba6-95a4cb5821ca-tls-certs\") on node \"ip-10-0-130-59.ec2.internal\" DevicePath \"\"" Jul 06 19:35:58.302897 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:35:58.302900 2570 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/3812a9b5-6073-4dcc-9ba6-95a4cb5821ca-dshm\") on node \"ip-10-0-130-59.ec2.internal\" DevicePath \"\"" Jul 06 19:35:58.303130 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:35:58.302908 2570 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-t2mpb\" (UniqueName: \"kubernetes.io/projected/3812a9b5-6073-4dcc-9ba6-95a4cb5821ca-kube-api-access-t2mpb\") on node \"ip-10-0-130-59.ec2.internal\" DevicePath \"\"" Jul 06 19:35:58.519531 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:35:58.519441 2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-a-3550d9/rate-test-model-3550d9-kserve-6578856955-5gct2" Jul 06 19:35:58.519531 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:35:58.519493 2570 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-a-3550d9/rate-test-model-3550d9-kserve-6578856955-5gct2" Jul 06 19:35:58.532407 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:35:58.532386 2570 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-a-3550d9/rate-test-model-3550d9-kserve-6578856955-5gct2" Jul 06 19:35:58.574863 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:35:58.574831 2570 generic.go:356] "Generic (PLEG): container finished" podID="3812a9b5-6073-4dcc-9ba6-95a4cb5821ca" containerID="cecbace026bfaead8bc68636d425dd2699f8e39b2f162deaa5cde49abccec28d" exitCode=0 Jul 06 19:35:58.575058 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:35:58.574880 2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-3550d9/sub-test-model-3550d9-kserve-f7579f8dc-tp46f" Jul 06 19:35:58.575058 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:35:58.574929 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-3550d9/sub-test-model-3550d9-kserve-f7579f8dc-tp46f" event={"ID":"3812a9b5-6073-4dcc-9ba6-95a4cb5821ca","Type":"ContainerDied","Data":"cecbace026bfaead8bc68636d425dd2699f8e39b2f162deaa5cde49abccec28d"} Jul 06 19:35:58.575058 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:35:58.574980 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-3550d9/sub-test-model-3550d9-kserve-f7579f8dc-tp46f" event={"ID":"3812a9b5-6073-4dcc-9ba6-95a4cb5821ca","Type":"ContainerDied","Data":"1535ae591d58536a98fdb91485ddf17221755c427d4f7a02ad4ef6a226b37c4d"} Jul 06 19:35:58.575058 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:35:58.575003 2570 scope.go:117] "RemoveContainer" containerID="cecbace026bfaead8bc68636d425dd2699f8e39b2f162deaa5cde49abccec28d" Jul 06 19:35:58.584522 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:35:58.584508 2570 scope.go:117] "RemoveContainer" containerID="cecbace026bfaead8bc68636d425dd2699f8e39b2f162deaa5cde49abccec28d" Jul 06 19:35:58.584842 ip-10-0-130-59 kubenswrapper[2570]: E0706 19:35:58.584823 2570 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"cecbace026bfaead8bc68636d425dd2699f8e39b2f162deaa5cde49abccec28d\": container with ID starting with cecbace026bfaead8bc68636d425dd2699f8e39b2f162deaa5cde49abccec28d not found: ID does not exist" containerID="cecbace026bfaead8bc68636d425dd2699f8e39b2f162deaa5cde49abccec28d" Jul 06 19:35:58.584922 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:35:58.584851 2570 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"cecbace026bfaead8bc68636d425dd2699f8e39b2f162deaa5cde49abccec28d"} err="failed to get container status \"cecbace026bfaead8bc68636d425dd2699f8e39b2f162deaa5cde49abccec28d\": rpc error: code = NotFound desc = could not find container \"cecbace026bfaead8bc68636d425dd2699f8e39b2f162deaa5cde49abccec28d\": container with ID starting with cecbace026bfaead8bc68636d425dd2699f8e39b2f162deaa5cde49abccec28d not found: ID does not exist" Jul 06 19:35:58.588224 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:35:58.588208 2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-a-3550d9/rate-test-model-3550d9-kserve-6578856955-5gct2" Jul 06 19:35:58.597041 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:35:58.597021 2570 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-3550d9/sub-test-model-3550d9-kserve-f7579f8dc-tp46f"] Jul 06 19:35:58.601221 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:35:58.601196 2570 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-a-3550d9/sub-test-model-3550d9-kserve-f7579f8dc-tp46f"] Jul 06 19:35:59.034045 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:35:59.034012 2570 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="3812a9b5-6073-4dcc-9ba6-95a4cb5821ca" path="/var/lib/kubelet/pods/3812a9b5-6073-4dcc-9ba6-95a4cb5821ca/volumes" Jul 06 19:36:15.982503 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:36:15.982462 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k8fbq_bc7057bd-88da-48af-ba32-e6efdc6e8362/konnectivity-agent/0.log" limit=1024 Jul 06 19:36:16.093753 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:36:16.093719 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k8fbq_bc7057bd-88da-48af-ba32-e6efdc6e8362/konnectivity-agent/0.log" limit=1024 Jul 06 19:36:23.636143 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:36:23.636106 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k8fbq_bc7057bd-88da-48af-ba32-e6efdc6e8362/konnectivity-agent/0.log" limit=1024 Jul 06 19:36:23.735798 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:36:23.735763 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k8fbq_bc7057bd-88da-48af-ba32-e6efdc6e8362/konnectivity-agent/0.log" limit=1024 Jul 06 19:36:25.922890 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:36:25.922848 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k8fbq_bc7057bd-88da-48af-ba32-e6efdc6e8362/konnectivity-agent/0.log" limit=1024 Jul 06 19:36:30.392951 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:36:30.392916 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-flg9z"] Jul 06 19:36:48.701759 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:36:48.701718 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-flg9z"] Jul 06 19:37:02.810205 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:37:02.810168 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-flg9z"] Jul 06 19:37:10.348244 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:37:10.348203 2570 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-3550d9/rate-test-model-3550d9-kserve-6578856955-5gct2"] Jul 06 19:37:10.374883 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:37:10.374850 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k8fbq_bc7057bd-88da-48af-ba32-e6efdc6e8362/konnectivity-agent/0.log" limit=1024 Jul 06 19:37:11.324508 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:37:11.324398 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k8fbq_bc7057bd-88da-48af-ba32-e6efdc6e8362/konnectivity-agent/0.log" limit=1024 Jul 06 19:37:12.413468 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:37:12.413387 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k8fbq_bc7057bd-88da-48af-ba32-e6efdc6e8362/konnectivity-agent/0.log" limit=1024 Jul 06 19:37:12.805880 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:37:12.805849 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k8fbq_bc7057bd-88da-48af-ba32-e6efdc6e8362/konnectivity-agent/0.log" limit=1024 Jul 06 19:37:12.935878 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:37:12.935844 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k8fbq_bc7057bd-88da-48af-ba32-e6efdc6e8362/konnectivity-agent/0.log" limit=1024 Jul 06 19:37:13.036427 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:37:13.036399 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k8fbq_bc7057bd-88da-48af-ba32-e6efdc6e8362/konnectivity-agent/0.log" limit=1024 Jul 06 19:37:13.232562 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:37:13.232528 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k8fbq_bc7057bd-88da-48af-ba32-e6efdc6e8362/konnectivity-agent/0.log" limit=1024 Jul 06 19:37:16.115689 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:37:16.115656 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-flg9z"] Jul 06 19:37:17.874154 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:37:17.874105 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k8fbq_bc7057bd-88da-48af-ba32-e6efdc6e8362/konnectivity-agent/0.log" limit=1024 Jul 06 19:37:25.367992 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:37:25.367950 2570 kuberuntime_container.go:872] "Killing container with a grace period" pod="ai-tenant-e2e-shared-a-3550d9/rate-test-model-3550d9-kserve-6578856955-5gct2" podUID="c58818ef-10a8-4370-ad2b-1fbc23a2c531" containerName="main" containerID="cri-o://5412479f9366fb2b984361ed5a2ceda9232ded06120d80f76f40bb615aa71ac2" gracePeriod=45 Jul 06 19:37:25.619393 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:37:25.619337 2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-3550d9/rate-test-model-3550d9-kserve-6578856955-5gct2" Jul 06 19:37:25.727161 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:37:25.727128 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c58818ef-10a8-4370-ad2b-1fbc23a2c531-tls-certs\") pod \"c58818ef-10a8-4370-ad2b-1fbc23a2c531\" (UID: \"c58818ef-10a8-4370-ad2b-1fbc23a2c531\") " Jul 06 19:37:25.727161 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:37:25.727158 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c58818ef-10a8-4370-ad2b-1fbc23a2c531-tmp-dir\") pod \"c58818ef-10a8-4370-ad2b-1fbc23a2c531\" (UID: \"c58818ef-10a8-4370-ad2b-1fbc23a2c531\") " Jul 06 19:37:25.727357 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:37:25.727304 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-dkljp\" (UniqueName: \"kubernetes.io/projected/c58818ef-10a8-4370-ad2b-1fbc23a2c531-kube-api-access-dkljp\") pod \"c58818ef-10a8-4370-ad2b-1fbc23a2c531\" (UID: \"c58818ef-10a8-4370-ad2b-1fbc23a2c531\") " Jul 06 19:37:25.727400 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:37:25.727359 2570 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c58818ef-10a8-4370-ad2b-1fbc23a2c531-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "c58818ef-10a8-4370-ad2b-1fbc23a2c531" (UID: "c58818ef-10a8-4370-ad2b-1fbc23a2c531"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 19:37:25.727400 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:37:25.727361 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/c58818ef-10a8-4370-ad2b-1fbc23a2c531-home\") pod \"c58818ef-10a8-4370-ad2b-1fbc23a2c531\" (UID: \"c58818ef-10a8-4370-ad2b-1fbc23a2c531\") " Jul 06 19:37:25.727477 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:37:25.727450 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/c58818ef-10a8-4370-ad2b-1fbc23a2c531-model-cache\") pod \"c58818ef-10a8-4370-ad2b-1fbc23a2c531\" (UID: \"c58818ef-10a8-4370-ad2b-1fbc23a2c531\") " Jul 06 19:37:25.727525 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:37:25.727489 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/c58818ef-10a8-4370-ad2b-1fbc23a2c531-dshm\") pod \"c58818ef-10a8-4370-ad2b-1fbc23a2c531\" (UID: \"c58818ef-10a8-4370-ad2b-1fbc23a2c531\") " Jul 06 19:37:25.727604 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:37:25.727562 2570 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c58818ef-10a8-4370-ad2b-1fbc23a2c531-home" (OuterVolumeSpecName: "home") pod "c58818ef-10a8-4370-ad2b-1fbc23a2c531" (UID: "c58818ef-10a8-4370-ad2b-1fbc23a2c531"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 19:37:25.727694 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:37:25.727636 2570 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c58818ef-10a8-4370-ad2b-1fbc23a2c531-model-cache" (OuterVolumeSpecName: "model-cache") pod "c58818ef-10a8-4370-ad2b-1fbc23a2c531" (UID: "c58818ef-10a8-4370-ad2b-1fbc23a2c531"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 19:37:25.727862 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:37:25.727844 2570 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/c58818ef-10a8-4370-ad2b-1fbc23a2c531-home\") on node \"ip-10-0-130-59.ec2.internal\" DevicePath \"\"" Jul 06 19:37:25.727923 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:37:25.727864 2570 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/c58818ef-10a8-4370-ad2b-1fbc23a2c531-model-cache\") on node \"ip-10-0-130-59.ec2.internal\" DevicePath \"\"" Jul 06 19:37:25.727923 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:37:25.727877 2570 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c58818ef-10a8-4370-ad2b-1fbc23a2c531-tmp-dir\") on node \"ip-10-0-130-59.ec2.internal\" DevicePath \"\"" Jul 06 19:37:25.729343 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:37:25.729311 2570 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c58818ef-10a8-4370-ad2b-1fbc23a2c531-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "c58818ef-10a8-4370-ad2b-1fbc23a2c531" (UID: "c58818ef-10a8-4370-ad2b-1fbc23a2c531"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 06 19:37:25.729343 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:37:25.729316 2570 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c58818ef-10a8-4370-ad2b-1fbc23a2c531-kube-api-access-dkljp" (OuterVolumeSpecName: "kube-api-access-dkljp") pod "c58818ef-10a8-4370-ad2b-1fbc23a2c531" (UID: "c58818ef-10a8-4370-ad2b-1fbc23a2c531"). InnerVolumeSpecName "kube-api-access-dkljp". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 06 19:37:25.729490 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:37:25.729410 2570 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c58818ef-10a8-4370-ad2b-1fbc23a2c531-dshm" (OuterVolumeSpecName: "dshm") pod "c58818ef-10a8-4370-ad2b-1fbc23a2c531" (UID: "c58818ef-10a8-4370-ad2b-1fbc23a2c531"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 19:37:25.829095 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:37:25.829065 2570 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/c58818ef-10a8-4370-ad2b-1fbc23a2c531-dshm\") on node \"ip-10-0-130-59.ec2.internal\" DevicePath \"\"" Jul 06 19:37:25.829095 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:37:25.829089 2570 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c58818ef-10a8-4370-ad2b-1fbc23a2c531-tls-certs\") on node \"ip-10-0-130-59.ec2.internal\" DevicePath \"\"" Jul 06 19:37:25.829095 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:37:25.829099 2570 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-dkljp\" (UniqueName: \"kubernetes.io/projected/c58818ef-10a8-4370-ad2b-1fbc23a2c531-kube-api-access-dkljp\") on node \"ip-10-0-130-59.ec2.internal\" DevicePath \"\"" Jul 06 19:37:26.009662 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:37:26.009598 2570 generic.go:356] "Generic (PLEG): container finished" podID="c58818ef-10a8-4370-ad2b-1fbc23a2c531" containerID="5412479f9366fb2b984361ed5a2ceda9232ded06120d80f76f40bb615aa71ac2" exitCode=0 Jul 06 19:37:26.009662 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:37:26.009660 2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-3550d9/rate-test-model-3550d9-kserve-6578856955-5gct2" Jul 06 19:37:26.009877 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:37:26.009679 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-3550d9/rate-test-model-3550d9-kserve-6578856955-5gct2" event={"ID":"c58818ef-10a8-4370-ad2b-1fbc23a2c531","Type":"ContainerDied","Data":"5412479f9366fb2b984361ed5a2ceda9232ded06120d80f76f40bb615aa71ac2"} Jul 06 19:37:26.009877 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:37:26.009724 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-3550d9/rate-test-model-3550d9-kserve-6578856955-5gct2" event={"ID":"c58818ef-10a8-4370-ad2b-1fbc23a2c531","Type":"ContainerDied","Data":"1e01160fae84dc8b03eae9dc3e47a893cc00be1245d32e2780e585b3c1269c9a"} Jul 06 19:37:26.009877 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:37:26.009742 2570 scope.go:117] "RemoveContainer" containerID="5412479f9366fb2b984361ed5a2ceda9232ded06120d80f76f40bb615aa71ac2" Jul 06 19:37:26.018695 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:37:26.018679 2570 scope.go:117] "RemoveContainer" containerID="5412479f9366fb2b984361ed5a2ceda9232ded06120d80f76f40bb615aa71ac2" Jul 06 19:37:26.018941 ip-10-0-130-59 kubenswrapper[2570]: E0706 19:37:26.018923 2570 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5412479f9366fb2b984361ed5a2ceda9232ded06120d80f76f40bb615aa71ac2\": container with ID starting with 5412479f9366fb2b984361ed5a2ceda9232ded06120d80f76f40bb615aa71ac2 not found: ID does not exist" containerID="5412479f9366fb2b984361ed5a2ceda9232ded06120d80f76f40bb615aa71ac2" Jul 06 19:37:26.019010 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:37:26.018948 2570 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5412479f9366fb2b984361ed5a2ceda9232ded06120d80f76f40bb615aa71ac2"} err="failed to get container status \"5412479f9366fb2b984361ed5a2ceda9232ded06120d80f76f40bb615aa71ac2\": rpc error: code = NotFound desc = could not find container \"5412479f9366fb2b984361ed5a2ceda9232ded06120d80f76f40bb615aa71ac2\": container with ID starting with 5412479f9366fb2b984361ed5a2ceda9232ded06120d80f76f40bb615aa71ac2 not found: ID does not exist" Jul 06 19:37:26.030242 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:37:26.030221 2570 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-3550d9/rate-test-model-3550d9-kserve-6578856955-5gct2"] Jul 06 19:37:26.037282 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:37:26.037262 2570 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-a-3550d9/rate-test-model-3550d9-kserve-6578856955-5gct2"] Jul 06 19:37:27.028437 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:37:27.028406 2570 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c58818ef-10a8-4370-ad2b-1fbc23a2c531" path="/var/lib/kubelet/pods/c58818ef-10a8-4370-ad2b-1fbc23a2c531/volumes" Jul 06 19:37:54.908704 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:37:54.908640 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k8fbq_bc7057bd-88da-48af-ba32-e6efdc6e8362/konnectivity-agent/0.log" limit=1024 Jul 06 19:37:55.209184 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:37:55.209151 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k8fbq_bc7057bd-88da-48af-ba32-e6efdc6e8362/konnectivity-agent/0.log" limit=1024 Jul 06 19:37:55.435039 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:37:55.435008 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k8fbq_bc7057bd-88da-48af-ba32-e6efdc6e8362/konnectivity-agent/0.log" limit=1024 Jul 06 19:37:55.607288 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:37:55.607197 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k8fbq_bc7057bd-88da-48af-ba32-e6efdc6e8362/konnectivity-agent/0.log" limit=1024 Jul 06 19:37:56.070604 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:37:56.070560 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k8fbq_bc7057bd-88da-48af-ba32-e6efdc6e8362/konnectivity-agent/0.log" limit=1024 Jul 06 19:37:56.180360 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:37:56.180328 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k8fbq_bc7057bd-88da-48af-ba32-e6efdc6e8362/konnectivity-agent/0.log" limit=1024 Jul 06 19:37:56.352671 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:37:56.352569 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k8fbq_bc7057bd-88da-48af-ba32-e6efdc6e8362/konnectivity-agent/0.log" limit=1024 Jul 06 19:38:04.892735 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:38:04.892705 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-flg9z"] Jul 06 19:38:05.450713 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:38:05.450670 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k8fbq_bc7057bd-88da-48af-ba32-e6efdc6e8362/konnectivity-agent/0.log" limit=1024 Jul 06 19:38:05.593491 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:38:05.593446 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k8fbq_bc7057bd-88da-48af-ba32-e6efdc6e8362/konnectivity-agent/0.log" limit=1024 Jul 06 19:38:05.750927 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:38:05.750846 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k8fbq_bc7057bd-88da-48af-ba32-e6efdc6e8362/konnectivity-agent/0.log" limit=1024 Jul 06 19:38:05.975089 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:38:05.975042 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k8fbq_bc7057bd-88da-48af-ba32-e6efdc6e8362/konnectivity-agent/0.log" limit=1024 Jul 06 19:38:06.377488 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:38:06.377452 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-97768fdb-g9kx6_642a7ce9-6401-4aef-b8ef-1d70a16ce7bc/authorino/0.log" Jul 06 19:38:11.998505 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:38:11.998474 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_model-serving-api-54d89bd5fc-4f7jh_bc0ca806-1da4-4fd0-a454-451382a07ddb/server/0.log" Jul 06 19:38:12.250116 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:38:12.250042 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-546c6c6d78-g8lh9_583dec04-8578-41f5-92c2-b716d8a86b76/copy-manifests/0.log" Jul 06 19:38:12.257251 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:38:12.257229 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-546c6c6d78-g8lh9_583dec04-8578-41f5-92c2-b716d8a86b76/manager/0.log" Jul 06 19:38:14.018440 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:38:14.018408 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-97768fdb-g9kx6_642a7ce9-6401-4aef-b8ef-1d70a16ce7bc/authorino/0.log" Jul 06 19:38:14.266364 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:38:14.266333 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_dns-operator-controller-manager-864dd78c4c-bs6mw_88de4830-cf1c-4169-8494-e6669a05ad3d/manager/0.log" Jul 06 19:38:14.755081 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:38:14.755058 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-limitador-54d6d988d5-flg9z_9bc5684c-2f3b-40be-91f7-a749ff3ff4c3/limitador/0.log" Jul 06 19:38:15.613425 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:38:15.613395 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-6c488d768d-s2rqp_8daf052f-a13f-4a56-9b8d-36c12e50d1b4/kube-auth-proxy/0.log" Jul 06 19:38:16.722102 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:38:16.722049 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-trlp-test-simulated-kserve-598466548d-kcrlh_ddd50824-42b6-4a8a-a2bc-6f840eb356cb/main/0.log" Jul 06 19:38:16.948949 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:38:16.948917 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_facebook-opt-125m-simulated-kserve-74b68447df-dw68w_b7b15020-8cd7-420b-88f6-7658a405a697/main/0.log" Jul 06 19:38:17.071233 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:38:17.071145 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_premium-simulated-simulated-premium-kserve-5fc6fb456f-rjgms_b40f5752-a1af-4e5c-a266-d1d863dbed05/main/0.log" Jul 06 19:38:21.123273 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:38:21.123241 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k8fbq_bc7057bd-88da-48af-ba32-e6efdc6e8362/konnectivity-agent/0.log" limit=1024 Jul 06 19:38:21.221548 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:38:21.221518 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k8fbq_bc7057bd-88da-48af-ba32-e6efdc6e8362/konnectivity-agent/0.log" limit=1024 Jul 06 19:38:24.204459 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:38:24.204421 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-ln98s_4beea8e8-7750-4a7a-bc49-249b3d7e3ab3/global-pull-secret-syncer/0.log" Jul 06 19:38:24.305650 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:38:24.305599 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-k8fbq_bc7057bd-88da-48af-ba32-e6efdc6e8362/konnectivity-agent/0.log" Jul 06 19:38:24.408806 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:38:24.408771 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-130-59.ec2.internal_d57d2f4948f6aacb3452bada80a2e242/haproxy/0.log" Jul 06 19:38:29.964338 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:38:29.964303 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-97768fdb-g9kx6_642a7ce9-6401-4aef-b8ef-1d70a16ce7bc/authorino/0.log" Jul 06 19:38:30.025810 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:38:30.025784 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_dns-operator-controller-manager-864dd78c4c-bs6mw_88de4830-cf1c-4169-8494-e6669a05ad3d/manager/0.log" Jul 06 19:38:30.247262 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:38:30.247183 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-limitador-54d6d988d5-flg9z_9bc5684c-2f3b-40be-91f7-a749ff3ff4c3/limitador/0.log" Jul 06 19:38:31.910221 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:38:31.910188 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_0b59ed61-5ebf-4244-8ae9-1bd151f4673f/alertmanager/0.log" Jul 06 19:38:31.934024 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:38:31.933999 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_0b59ed61-5ebf-4244-8ae9-1bd151f4673f/config-reloader/0.log" Jul 06 19:38:31.957412 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:38:31.957382 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_0b59ed61-5ebf-4244-8ae9-1bd151f4673f/kube-rbac-proxy-web/0.log" Jul 06 19:38:31.980090 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:38:31.980061 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_0b59ed61-5ebf-4244-8ae9-1bd151f4673f/kube-rbac-proxy/0.log" Jul 06 19:38:32.006313 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:38:32.006281 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_0b59ed61-5ebf-4244-8ae9-1bd151f4673f/kube-rbac-proxy-metric/0.log" Jul 06 19:38:32.034680 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:38:32.034649 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_0b59ed61-5ebf-4244-8ae9-1bd151f4673f/prom-label-proxy/0.log" Jul 06 19:38:32.060819 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:38:32.060729 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_0b59ed61-5ebf-4244-8ae9-1bd151f4673f/init-config-reloader/0.log" Jul 06 19:38:32.151954 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:38:32.151924 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-7c9dbb8b7d-lrhkk_30d171a0-0417-4444-8af2-154f0d6c82f9/kube-state-metrics/0.log" Jul 06 19:38:32.170999 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:38:32.170966 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-7c9dbb8b7d-lrhkk_30d171a0-0417-4444-8af2-154f0d6c82f9/kube-rbac-proxy-main/0.log" Jul 06 19:38:32.196400 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:38:32.196365 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-7c9dbb8b7d-lrhkk_30d171a0-0417-4444-8af2-154f0d6c82f9/kube-rbac-proxy-self/0.log" Jul 06 19:38:32.227823 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:38:32.227794 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_metrics-server-59d88b459-jqgzb_f575cfd3-dd2e-4ecf-ab0f-1860884038a3/metrics-server/0.log" Jul 06 19:38:32.257659 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:38:32.257634 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_monitoring-plugin-d5dcff7cf-c9prf_8c828f23-ece7-439d-9ca2-ef0642e73506/monitoring-plugin/0.log" Jul 06 19:38:32.292677 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:38:32.292647 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-2fd54_b3ab8883-a662-4f63-9429-13b7f1c3a3b3/node-exporter/0.log" Jul 06 19:38:32.319947 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:38:32.319857 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-2fd54_b3ab8883-a662-4f63-9429-13b7f1c3a3b3/kube-rbac-proxy/0.log" Jul 06 19:38:32.348033 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:38:32.348009 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-2fd54_b3ab8883-a662-4f63-9429-13b7f1c3a3b3/init-textfile/0.log" Jul 06 19:38:32.563141 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:38:32.563107 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-54cf96dfcf-h62zc_910d0e65-b069-41b4-a8d5-2881341454e7/kube-rbac-proxy-main/0.log" Jul 06 19:38:32.597114 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:38:32.597038 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-54cf96dfcf-h62zc_910d0e65-b069-41b4-a8d5-2881341454e7/kube-rbac-proxy-self/0.log" Jul 06 19:38:32.623320 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:38:32.623283 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-54cf96dfcf-h62zc_910d0e65-b069-41b4-a8d5-2881341454e7/openshift-state-metrics/0.log" Jul 06 19:38:32.663269 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:38:32.663242 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_2907ab4e-a813-4e8a-9358-2021a20445a4/prometheus/0.log" Jul 06 19:38:32.690312 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:38:32.690263 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_2907ab4e-a813-4e8a-9358-2021a20445a4/config-reloader/0.log" Jul 06 19:38:32.714468 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:38:32.714443 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_2907ab4e-a813-4e8a-9358-2021a20445a4/thanos-sidecar/0.log" Jul 06 19:38:32.740958 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:38:32.740933 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_2907ab4e-a813-4e8a-9358-2021a20445a4/kube-rbac-proxy-web/0.log" Jul 06 19:38:32.765080 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:38:32.765051 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_2907ab4e-a813-4e8a-9358-2021a20445a4/kube-rbac-proxy/0.log" Jul 06 19:38:32.787331 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:38:32.787307 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_2907ab4e-a813-4e8a-9358-2021a20445a4/kube-rbac-proxy-thanos/0.log" Jul 06 19:38:32.809803 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:38:32.809775 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_2907ab4e-a813-4e8a-9358-2021a20445a4/init-config-reloader/0.log" Jul 06 19:38:32.918926 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:38:32.918832 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-7c47bc74cb-xrsz6_564f0d46-248e-4a95-9416-2a6d23478fb0/telemeter-client/0.log" Jul 06 19:38:32.946727 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:38:32.946693 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-7c47bc74cb-xrsz6_564f0d46-248e-4a95-9416-2a6d23478fb0/reload/0.log" Jul 06 19:38:32.972628 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:38:32.972597 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-7c47bc74cb-xrsz6_564f0d46-248e-4a95-9416-2a6d23478fb0/kube-rbac-proxy/0.log" Jul 06 19:38:33.018730 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:38:33.018701 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-7d85fbb9b-8c6ms_3c3710b4-98b2-42cf-9425-5a1b35e7f794/thanos-query/0.log" Jul 06 19:38:33.042188 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:38:33.042165 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-7d85fbb9b-8c6ms_3c3710b4-98b2-42cf-9425-5a1b35e7f794/kube-rbac-proxy-web/0.log" Jul 06 19:38:33.066711 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:38:33.066682 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-7d85fbb9b-8c6ms_3c3710b4-98b2-42cf-9425-5a1b35e7f794/kube-rbac-proxy/0.log" Jul 06 19:38:33.091078 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:38:33.091054 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-7d85fbb9b-8c6ms_3c3710b4-98b2-42cf-9425-5a1b35e7f794/prom-label-proxy/0.log" Jul 06 19:38:33.115934 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:38:33.115909 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-7d85fbb9b-8c6ms_3c3710b4-98b2-42cf-9425-5a1b35e7f794/kube-rbac-proxy-rules/0.log" Jul 06 19:38:33.139154 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:38:33.139127 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-7d85fbb9b-8c6ms_3c3710b4-98b2-42cf-9425-5a1b35e7f794/kube-rbac-proxy-metrics/0.log" Jul 06 19:38:34.179527 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:38:34.179492 2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-446hm/perf-node-gather-daemonset-6bbkw"] Jul 06 19:38:34.179921 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:38:34.179896 2570 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c58818ef-10a8-4370-ad2b-1fbc23a2c531" containerName="main" Jul 06 19:38:34.179921 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:38:34.179909 2570 state_mem.go:107] "Deleted CPUSet assignment" podUID="c58818ef-10a8-4370-ad2b-1fbc23a2c531" containerName="main" Jul 06 19:38:34.180001 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:38:34.179930 2570 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3812a9b5-6073-4dcc-9ba6-95a4cb5821ca" containerName="main" Jul 06 19:38:34.180001 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:38:34.179936 2570 state_mem.go:107] "Deleted CPUSet assignment" podUID="3812a9b5-6073-4dcc-9ba6-95a4cb5821ca" containerName="main" Jul 06 19:38:34.180001 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:38:34.179994 2570 memory_manager.go:368] "RemoveStaleState removing state" podUID="3812a9b5-6073-4dcc-9ba6-95a4cb5821ca" containerName="main" Jul 06 19:38:34.180001 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:38:34.180001 2570 memory_manager.go:368] "RemoveStaleState removing state" podUID="c58818ef-10a8-4370-ad2b-1fbc23a2c531" containerName="main" Jul 06 19:38:34.183211 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:38:34.183189 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-446hm/perf-node-gather-daemonset-6bbkw" Jul 06 19:38:34.185869 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:38:34.185848 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-446hm\"/\"kube-root-ca.crt\"" Jul 06 19:38:34.185869 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:38:34.185860 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-446hm\"/\"openshift-service-ca.crt\"" Jul 06 19:38:34.186035 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:38:34.185906 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-446hm\"/\"default-dockercfg-vqxjx\"" Jul 06 19:38:34.197091 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:38:34.197064 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-446hm/perf-node-gather-daemonset-6bbkw"] Jul 06 19:38:34.323048 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:38:34.323022 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/f1bcb049-b731-428b-82dc-36a24c00154a-lib-modules\") pod \"perf-node-gather-daemonset-6bbkw\" (UID: \"f1bcb049-b731-428b-82dc-36a24c00154a\") " pod="openshift-must-gather-446hm/perf-node-gather-daemonset-6bbkw" Jul 06 19:38:34.323235 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:38:34.323084 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/f1bcb049-b731-428b-82dc-36a24c00154a-proc\") pod \"perf-node-gather-daemonset-6bbkw\" (UID: \"f1bcb049-b731-428b-82dc-36a24c00154a\") " pod="openshift-must-gather-446hm/perf-node-gather-daemonset-6bbkw" Jul 06 19:38:34.323235 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:38:34.323102 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/f1bcb049-b731-428b-82dc-36a24c00154a-podres\") pod \"perf-node-gather-daemonset-6bbkw\" (UID: \"f1bcb049-b731-428b-82dc-36a24c00154a\") " pod="openshift-must-gather-446hm/perf-node-gather-daemonset-6bbkw" Jul 06 19:38:34.323235 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:38:34.323139 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/f1bcb049-b731-428b-82dc-36a24c00154a-sys\") pod \"perf-node-gather-daemonset-6bbkw\" (UID: \"f1bcb049-b731-428b-82dc-36a24c00154a\") " pod="openshift-must-gather-446hm/perf-node-gather-daemonset-6bbkw" Jul 06 19:38:34.323356 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:38:34.323229 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2bvsf\" (UniqueName: \"kubernetes.io/projected/f1bcb049-b731-428b-82dc-36a24c00154a-kube-api-access-2bvsf\") pod \"perf-node-gather-daemonset-6bbkw\" (UID: \"f1bcb049-b731-428b-82dc-36a24c00154a\") " pod="openshift-must-gather-446hm/perf-node-gather-daemonset-6bbkw" Jul 06 19:38:34.424236 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:38:34.424199 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/f1bcb049-b731-428b-82dc-36a24c00154a-sys\") pod \"perf-node-gather-daemonset-6bbkw\" (UID: \"f1bcb049-b731-428b-82dc-36a24c00154a\") " pod="openshift-must-gather-446hm/perf-node-gather-daemonset-6bbkw" Jul 06 19:38:34.424418 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:38:34.424262 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2bvsf\" (UniqueName: \"kubernetes.io/projected/f1bcb049-b731-428b-82dc-36a24c00154a-kube-api-access-2bvsf\") pod \"perf-node-gather-daemonset-6bbkw\" (UID: \"f1bcb049-b731-428b-82dc-36a24c00154a\") " pod="openshift-must-gather-446hm/perf-node-gather-daemonset-6bbkw" Jul 06 19:38:34.424418 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:38:34.424305 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/f1bcb049-b731-428b-82dc-36a24c00154a-lib-modules\") pod \"perf-node-gather-daemonset-6bbkw\" (UID: \"f1bcb049-b731-428b-82dc-36a24c00154a\") " pod="openshift-must-gather-446hm/perf-node-gather-daemonset-6bbkw" Jul 06 19:38:34.424418 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:38:34.424353 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/f1bcb049-b731-428b-82dc-36a24c00154a-sys\") pod \"perf-node-gather-daemonset-6bbkw\" (UID: \"f1bcb049-b731-428b-82dc-36a24c00154a\") " pod="openshift-must-gather-446hm/perf-node-gather-daemonset-6bbkw" Jul 06 19:38:34.424418 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:38:34.424370 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/f1bcb049-b731-428b-82dc-36a24c00154a-proc\") pod \"perf-node-gather-daemonset-6bbkw\" (UID: \"f1bcb049-b731-428b-82dc-36a24c00154a\") " pod="openshift-must-gather-446hm/perf-node-gather-daemonset-6bbkw" Jul 06 19:38:34.424418 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:38:34.424388 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/f1bcb049-b731-428b-82dc-36a24c00154a-podres\") pod \"perf-node-gather-daemonset-6bbkw\" (UID: \"f1bcb049-b731-428b-82dc-36a24c00154a\") " pod="openshift-must-gather-446hm/perf-node-gather-daemonset-6bbkw" Jul 06 19:38:34.424648 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:38:34.424428 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/f1bcb049-b731-428b-82dc-36a24c00154a-proc\") pod \"perf-node-gather-daemonset-6bbkw\" (UID: \"f1bcb049-b731-428b-82dc-36a24c00154a\") " pod="openshift-must-gather-446hm/perf-node-gather-daemonset-6bbkw" Jul 06 19:38:34.424648 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:38:34.424502 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/f1bcb049-b731-428b-82dc-36a24c00154a-podres\") pod \"perf-node-gather-daemonset-6bbkw\" (UID: \"f1bcb049-b731-428b-82dc-36a24c00154a\") " pod="openshift-must-gather-446hm/perf-node-gather-daemonset-6bbkw" Jul 06 19:38:34.424648 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:38:34.424508 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/f1bcb049-b731-428b-82dc-36a24c00154a-lib-modules\") pod \"perf-node-gather-daemonset-6bbkw\" (UID: \"f1bcb049-b731-428b-82dc-36a24c00154a\") " pod="openshift-must-gather-446hm/perf-node-gather-daemonset-6bbkw" Jul 06 19:38:34.433712 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:38:34.433661 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-2bvsf\" (UniqueName: \"kubernetes.io/projected/f1bcb049-b731-428b-82dc-36a24c00154a-kube-api-access-2bvsf\") pod \"perf-node-gather-daemonset-6bbkw\" (UID: \"f1bcb049-b731-428b-82dc-36a24c00154a\") " pod="openshift-must-gather-446hm/perf-node-gather-daemonset-6bbkw" Jul 06 19:38:34.493539 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:38:34.493507 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-446hm/perf-node-gather-daemonset-6bbkw" Jul 06 19:38:34.629533 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:38:34.629503 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-446hm/perf-node-gather-daemonset-6bbkw"] Jul 06 19:38:34.633016 ip-10-0-130-59 kubenswrapper[2570]: W0706 19:38:34.632988 2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-podf1bcb049_b731_428b_82dc_36a24c00154a.slice/crio-6845776ebcd5ac894e7f64e3b84b2f7c63fa566d5d6e0670f931e3c82bc17c59 WatchSource:0}: Error finding container 6845776ebcd5ac894e7f64e3b84b2f7c63fa566d5d6e0670f931e3c82bc17c59: Status 404 returned error can't find the container with id 6845776ebcd5ac894e7f64e3b84b2f7c63fa566d5d6e0670f931e3c82bc17c59 Jul 06 19:38:35.331688 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:38:35.331652 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-446hm/perf-node-gather-daemonset-6bbkw" event={"ID":"f1bcb049-b731-428b-82dc-36a24c00154a","Type":"ContainerStarted","Data":"c1e7a0bb948a0aabcc4f146b7071112c1e87b6238aa6fa0f339e9392e3f6356a"} Jul 06 19:38:35.332084 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:38:35.331708 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-446hm/perf-node-gather-daemonset-6bbkw" event={"ID":"f1bcb049-b731-428b-82dc-36a24c00154a","Type":"ContainerStarted","Data":"6845776ebcd5ac894e7f64e3b84b2f7c63fa566d5d6e0670f931e3c82bc17c59"} Jul 06 19:38:35.332084 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:38:35.331786 2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-446hm/perf-node-gather-daemonset-6bbkw" Jul 06 19:38:35.349077 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:38:35.349032 2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-446hm/perf-node-gather-daemonset-6bbkw" podStartSLOduration=1.3490158349999999 podStartE2EDuration="1.349015835s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-06 19:38:34 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 19:38:35.347534173 +0000 UTC m=+2594.902555362" watchObservedRunningTime="2026-07-06 19:38:35.349015835 +0000 UTC m=+2594.904037010" Jul 06 19:38:37.229695 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:38:37.229672 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-dk4j7_c68e3e9c-2b68-4ed9-99c5-f76eb52b09c3/dns/0.log" Jul 06 19:38:37.264425 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:38:37.264396 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-dk4j7_c68e3e9c-2b68-4ed9-99c5-f76eb52b09c3/kube-rbac-proxy/0.log" Jul 06 19:38:37.366665 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:38:37.366636 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-4qk4s_a88da320-d446-42e3-b155-418a20f35893/dns-node-resolver/0.log" Jul 06 19:38:38.109065 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:38:38.109039 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-sxf4k_da433510-33a8-4e5e-9a3d-29d8d0ec0502/node-ca/0.log" Jul 06 19:38:39.418177 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:38:39.418139 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-6c488d768d-s2rqp_8daf052f-a13f-4a56-9b8d-36c12e50d1b4/kube-auth-proxy/0.log" Jul 06 19:38:40.360360 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:38:40.360331 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-22lv8_60870efe-6ea8-49ed-a879-7a1d7dee12d0/serve-healthcheck-canary/0.log" Jul 06 19:38:41.003360 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:38:41.003330 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-mkrnp_5f316223-4aea-420d-be1b-c2921c1f994f/kube-rbac-proxy/0.log" Jul 06 19:38:41.026206 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:38:41.026179 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-mkrnp_5f316223-4aea-420d-be1b-c2921c1f994f/exporter/0.log" Jul 06 19:38:41.054687 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:38:41.054644 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-mkrnp_5f316223-4aea-420d-be1b-c2921c1f994f/extractor/0.log" Jul 06 19:38:41.347751 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:38:41.347675 2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-446hm/perf-node-gather-daemonset-6bbkw" Jul 06 19:38:43.843879 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:38:43.843775 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_model-serving-api-54d89bd5fc-4f7jh_bc0ca806-1da4-4fd0-a454-451382a07ddb/server/0.log" Jul 06 19:38:43.980326 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:38:43.980292 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-546c6c6d78-g8lh9_583dec04-8578-41f5-92c2-b716d8a86b76/manager/0.log" Jul 06 19:38:44.006313 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:38:44.006285 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-546c6c6d78-g8lh9_583dec04-8578-41f5-92c2-b716d8a86b76/copy-manifests/0.log" Jul 06 19:38:44.953385 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:38:44.953354 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-6c488d768d-s2rqp_8daf052f-a13f-4a56-9b8d-36c12e50d1b4/kube-auth-proxy/0.log" Jul 06 19:38:45.732760 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:38:45.732726 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-lws-operator_lws-controller-manager-5d5c698c7c-vvttd_bf643fd2-138d-4a1d-8fa7-939d1c3ae3a1/manager/0.log" Jul 06 19:38:46.553886 ip-10-0-130-59 kubenswrapper[2570]: I0706 19:38:46.553858 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-6c488d768d-s2rqp_8daf052f-a13f-4a56-9b8d-36c12e50d1b4/kube-auth-proxy/0.log"