Jul 30 15:08:08.351827 ip-10-0-133-210 systemd[1]: kubelet.service: Failed to load environment files: No such file or directory Jul 30 15:08:08.351836 ip-10-0-133-210 systemd[1]: kubelet.service: Failed to run 'start-pre' task: No such file or directory Jul 30 15:08:08.351843 ip-10-0-133-210 systemd[1]: kubelet.service: Failed with result 'resources'. Jul 30 15:08:08.352066 ip-10-0-133-210 systemd[1]: Failed to start Kubernetes Kubelet. Jul 30 15:08:18.600952 ip-10-0-133-210 systemd[1]: kubelet.service: Failed to schedule restart job: Unit crio.service not found. Jul 30 15:08:18.600970 ip-10-0-133-210 systemd[1]: kubelet.service: Failed with result 'resources'. -- Boot 9ebb31c6818542e5b8093ab1ca895ec8 -- Jul 30 15:10:16.680252 ip-10-0-133-210 systemd[1]: Starting Kubernetes Kubelet... Jul 30 15:10:17.033349 ip-10-0-133-210 kubenswrapper[2576]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 30 15:10:17.033349 ip-10-0-133-210 kubenswrapper[2576]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version. Jul 30 15:10:17.033349 ip-10-0-133-210 kubenswrapper[2576]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 30 15:10:17.033349 ip-10-0-133-210 kubenswrapper[2576]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI. Jul 30 15:10:17.033349 ip-10-0-133-210 kubenswrapper[2576]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 30 15:10:17.035525 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:10:17.035438 2576 server.go:213] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime" Jul 30 15:10:17.040225 ip-10-0-133-210 kubenswrapper[2576]: W0730 15:10:17.040210 2576 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 30 15:10:17.040225 ip-10-0-133-210 kubenswrapper[2576]: W0730 15:10:17.040225 2576 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 30 15:10:17.040294 ip-10-0-133-210 kubenswrapper[2576]: W0730 15:10:17.040230 2576 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 30 15:10:17.040294 ip-10-0-133-210 kubenswrapper[2576]: W0730 15:10:17.040234 2576 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 30 15:10:17.040294 ip-10-0-133-210 kubenswrapper[2576]: W0730 15:10:17.040237 2576 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 30 15:10:17.040294 ip-10-0-133-210 kubenswrapper[2576]: W0730 15:10:17.040240 2576 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 30 15:10:17.040294 ip-10-0-133-210 kubenswrapper[2576]: W0730 15:10:17.040243 2576 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 30 15:10:17.040294 ip-10-0-133-210 kubenswrapper[2576]: W0730 15:10:17.040245 2576 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 30 15:10:17.040294 ip-10-0-133-210 kubenswrapper[2576]: W0730 15:10:17.040249 2576 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 30 15:10:17.040294 ip-10-0-133-210 kubenswrapper[2576]: W0730 15:10:17.040252 2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 30 15:10:17.040294 ip-10-0-133-210 kubenswrapper[2576]: W0730 15:10:17.040254 2576 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 30 15:10:17.040294 ip-10-0-133-210 kubenswrapper[2576]: W0730 15:10:17.040258 2576 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 30 15:10:17.040294 ip-10-0-133-210 kubenswrapper[2576]: W0730 15:10:17.040262 2576 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 30 15:10:17.040294 ip-10-0-133-210 kubenswrapper[2576]: W0730 15:10:17.040265 2576 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 30 15:10:17.040294 ip-10-0-133-210 kubenswrapper[2576]: W0730 15:10:17.040267 2576 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 30 15:10:17.040294 ip-10-0-133-210 kubenswrapper[2576]: W0730 15:10:17.040270 2576 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 30 15:10:17.040294 ip-10-0-133-210 kubenswrapper[2576]: W0730 15:10:17.040273 2576 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 30 15:10:17.040294 ip-10-0-133-210 kubenswrapper[2576]: W0730 15:10:17.040276 2576 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 30 15:10:17.040294 ip-10-0-133-210 kubenswrapper[2576]: W0730 15:10:17.040278 2576 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 30 15:10:17.040294 ip-10-0-133-210 kubenswrapper[2576]: W0730 15:10:17.040282 2576 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 30 15:10:17.040294 ip-10-0-133-210 kubenswrapper[2576]: W0730 15:10:17.040286 2576 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 30 15:10:17.040294 ip-10-0-133-210 kubenswrapper[2576]: W0730 15:10:17.040290 2576 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 30 15:10:17.040806 ip-10-0-133-210 kubenswrapper[2576]: W0730 15:10:17.040293 2576 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 30 15:10:17.040806 ip-10-0-133-210 kubenswrapper[2576]: W0730 15:10:17.040296 2576 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 30 15:10:17.040806 ip-10-0-133-210 kubenswrapper[2576]: W0730 15:10:17.040300 2576 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 30 15:10:17.040806 ip-10-0-133-210 kubenswrapper[2576]: W0730 15:10:17.040303 2576 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 30 15:10:17.040806 ip-10-0-133-210 kubenswrapper[2576]: W0730 15:10:17.040306 2576 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 30 15:10:17.040806 ip-10-0-133-210 kubenswrapper[2576]: W0730 15:10:17.040309 2576 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 30 15:10:17.040806 ip-10-0-133-210 kubenswrapper[2576]: W0730 15:10:17.040311 2576 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 30 15:10:17.040806 ip-10-0-133-210 kubenswrapper[2576]: W0730 15:10:17.040314 2576 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 30 15:10:17.040806 ip-10-0-133-210 kubenswrapper[2576]: W0730 15:10:17.040316 2576 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 30 15:10:17.040806 ip-10-0-133-210 kubenswrapper[2576]: W0730 15:10:17.040319 2576 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 30 15:10:17.040806 ip-10-0-133-210 kubenswrapper[2576]: W0730 15:10:17.040322 2576 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 30 15:10:17.040806 ip-10-0-133-210 kubenswrapper[2576]: W0730 15:10:17.040326 2576 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 30 15:10:17.040806 ip-10-0-133-210 kubenswrapper[2576]: W0730 15:10:17.040329 2576 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 30 15:10:17.040806 ip-10-0-133-210 kubenswrapper[2576]: W0730 15:10:17.040331 2576 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 30 15:10:17.040806 ip-10-0-133-210 kubenswrapper[2576]: W0730 15:10:17.040334 2576 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 30 15:10:17.040806 ip-10-0-133-210 kubenswrapper[2576]: W0730 15:10:17.040336 2576 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 30 15:10:17.040806 ip-10-0-133-210 kubenswrapper[2576]: W0730 15:10:17.040339 2576 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 30 15:10:17.040806 ip-10-0-133-210 kubenswrapper[2576]: W0730 15:10:17.040342 2576 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 30 15:10:17.040806 ip-10-0-133-210 kubenswrapper[2576]: W0730 15:10:17.040345 2576 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 30 15:10:17.041291 ip-10-0-133-210 kubenswrapper[2576]: W0730 15:10:17.040347 2576 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 30 15:10:17.041291 ip-10-0-133-210 kubenswrapper[2576]: W0730 15:10:17.040350 2576 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 30 15:10:17.041291 ip-10-0-133-210 kubenswrapper[2576]: W0730 15:10:17.040352 2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 30 15:10:17.041291 ip-10-0-133-210 kubenswrapper[2576]: W0730 15:10:17.040355 2576 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 30 15:10:17.041291 ip-10-0-133-210 kubenswrapper[2576]: W0730 15:10:17.040357 2576 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 30 15:10:17.041291 ip-10-0-133-210 kubenswrapper[2576]: W0730 15:10:17.040360 2576 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 30 15:10:17.041291 ip-10-0-133-210 kubenswrapper[2576]: W0730 15:10:17.040362 2576 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 30 15:10:17.041291 ip-10-0-133-210 kubenswrapper[2576]: W0730 15:10:17.040365 2576 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 30 15:10:17.041291 ip-10-0-133-210 kubenswrapper[2576]: W0730 15:10:17.040368 2576 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 30 15:10:17.041291 ip-10-0-133-210 kubenswrapper[2576]: W0730 15:10:17.040372 2576 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 30 15:10:17.041291 ip-10-0-133-210 kubenswrapper[2576]: W0730 15:10:17.040375 2576 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 30 15:10:17.041291 ip-10-0-133-210 kubenswrapper[2576]: W0730 15:10:17.040378 2576 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 30 15:10:17.041291 ip-10-0-133-210 kubenswrapper[2576]: W0730 15:10:17.040432 2576 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 30 15:10:17.041291 ip-10-0-133-210 kubenswrapper[2576]: W0730 15:10:17.040437 2576 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 30 15:10:17.041291 ip-10-0-133-210 kubenswrapper[2576]: W0730 15:10:17.040441 2576 feature_gate.go:328] unrecognized feature gate: Example Jul 30 15:10:17.041291 ip-10-0-133-210 kubenswrapper[2576]: W0730 15:10:17.040444 2576 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 30 15:10:17.041291 ip-10-0-133-210 kubenswrapper[2576]: W0730 15:10:17.040447 2576 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 30 15:10:17.041291 ip-10-0-133-210 kubenswrapper[2576]: W0730 15:10:17.040450 2576 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 30 15:10:17.041291 ip-10-0-133-210 kubenswrapper[2576]: W0730 15:10:17.040453 2576 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 30 15:10:17.041291 ip-10-0-133-210 kubenswrapper[2576]: W0730 15:10:17.040456 2576 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 30 15:10:17.041778 ip-10-0-133-210 kubenswrapper[2576]: W0730 15:10:17.040458 2576 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 30 15:10:17.041778 ip-10-0-133-210 kubenswrapper[2576]: W0730 15:10:17.040461 2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 30 15:10:17.041778 ip-10-0-133-210 kubenswrapper[2576]: W0730 15:10:17.040465 2576 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 30 15:10:17.041778 ip-10-0-133-210 kubenswrapper[2576]: W0730 15:10:17.040469 2576 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 30 15:10:17.041778 ip-10-0-133-210 kubenswrapper[2576]: W0730 15:10:17.040472 2576 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 30 15:10:17.041778 ip-10-0-133-210 kubenswrapper[2576]: W0730 15:10:17.040475 2576 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 30 15:10:17.041778 ip-10-0-133-210 kubenswrapper[2576]: W0730 15:10:17.040478 2576 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 30 15:10:17.041778 ip-10-0-133-210 kubenswrapper[2576]: W0730 15:10:17.040481 2576 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 30 15:10:17.041778 ip-10-0-133-210 kubenswrapper[2576]: W0730 15:10:17.040484 2576 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 30 15:10:17.041778 ip-10-0-133-210 kubenswrapper[2576]: W0730 15:10:17.040487 2576 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 30 15:10:17.041778 ip-10-0-133-210 kubenswrapper[2576]: W0730 15:10:17.040490 2576 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 30 15:10:17.041778 ip-10-0-133-210 kubenswrapper[2576]: W0730 15:10:17.040493 2576 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 30 15:10:17.041778 ip-10-0-133-210 kubenswrapper[2576]: W0730 15:10:17.040495 2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 30 15:10:17.041778 ip-10-0-133-210 kubenswrapper[2576]: W0730 15:10:17.040498 2576 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 30 15:10:17.041778 ip-10-0-133-210 kubenswrapper[2576]: W0730 15:10:17.040501 2576 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 30 15:10:17.041778 ip-10-0-133-210 kubenswrapper[2576]: W0730 15:10:17.040504 2576 feature_gate.go:328] unrecognized feature gate: Example2 Jul 30 15:10:17.041778 ip-10-0-133-210 kubenswrapper[2576]: W0730 15:10:17.040506 2576 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 30 15:10:17.041778 ip-10-0-133-210 kubenswrapper[2576]: W0730 15:10:17.040509 2576 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 30 15:10:17.041778 ip-10-0-133-210 kubenswrapper[2576]: W0730 15:10:17.040511 2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 30 15:10:17.041778 ip-10-0-133-210 kubenswrapper[2576]: W0730 15:10:17.040514 2576 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 30 15:10:17.042278 ip-10-0-133-210 kubenswrapper[2576]: W0730 15:10:17.040517 2576 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 30 15:10:17.042278 ip-10-0-133-210 kubenswrapper[2576]: W0730 15:10:17.040519 2576 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 30 15:10:17.042278 ip-10-0-133-210 kubenswrapper[2576]: W0730 15:10:17.040522 2576 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 30 15:10:17.042278 ip-10-0-133-210 kubenswrapper[2576]: W0730 15:10:17.040525 2576 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 30 15:10:17.042278 ip-10-0-133-210 kubenswrapper[2576]: W0730 15:10:17.040528 2576 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 30 15:10:17.042278 ip-10-0-133-210 kubenswrapper[2576]: W0730 15:10:17.040532 2576 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 30 15:10:17.042278 ip-10-0-133-210 kubenswrapper[2576]: W0730 15:10:17.040534 2576 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 30 15:10:17.042278 ip-10-0-133-210 kubenswrapper[2576]: W0730 15:10:17.040537 2576 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 30 15:10:17.042278 ip-10-0-133-210 kubenswrapper[2576]: W0730 15:10:17.040539 2576 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 30 15:10:17.042278 ip-10-0-133-210 kubenswrapper[2576]: W0730 15:10:17.040542 2576 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 30 15:10:17.042278 ip-10-0-133-210 kubenswrapper[2576]: W0730 15:10:17.040545 2576 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 30 15:10:17.042278 ip-10-0-133-210 kubenswrapper[2576]: W0730 15:10:17.040547 2576 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 30 15:10:17.042278 ip-10-0-133-210 kubenswrapper[2576]: W0730 15:10:17.040550 2576 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 30 15:10:17.042278 ip-10-0-133-210 kubenswrapper[2576]: W0730 15:10:17.040554 2576 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 30 15:10:17.042278 ip-10-0-133-210 kubenswrapper[2576]: W0730 15:10:17.040557 2576 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 30 15:10:17.042278 ip-10-0-133-210 kubenswrapper[2576]: W0730 15:10:17.040560 2576 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 30 15:10:17.042278 ip-10-0-133-210 kubenswrapper[2576]: W0730 15:10:17.040562 2576 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 30 15:10:17.042278 ip-10-0-133-210 kubenswrapper[2576]: W0730 15:10:17.040565 2576 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 30 15:10:17.042278 ip-10-0-133-210 kubenswrapper[2576]: W0730 15:10:17.041300 2576 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 30 15:10:17.042278 ip-10-0-133-210 kubenswrapper[2576]: W0730 15:10:17.041306 2576 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 30 15:10:17.042752 ip-10-0-133-210 kubenswrapper[2576]: W0730 15:10:17.041309 2576 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 30 15:10:17.042752 ip-10-0-133-210 kubenswrapper[2576]: W0730 15:10:17.041312 2576 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 30 15:10:17.042752 ip-10-0-133-210 kubenswrapper[2576]: W0730 15:10:17.041315 2576 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 30 15:10:17.042752 ip-10-0-133-210 kubenswrapper[2576]: W0730 15:10:17.041318 2576 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 30 15:10:17.042752 ip-10-0-133-210 kubenswrapper[2576]: W0730 15:10:17.041321 2576 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 30 15:10:17.042752 ip-10-0-133-210 kubenswrapper[2576]: W0730 15:10:17.041324 2576 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 30 15:10:17.042752 ip-10-0-133-210 kubenswrapper[2576]: W0730 15:10:17.041327 2576 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 30 15:10:17.042752 ip-10-0-133-210 kubenswrapper[2576]: W0730 15:10:17.041330 2576 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 30 15:10:17.042752 ip-10-0-133-210 kubenswrapper[2576]: W0730 15:10:17.041333 2576 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 30 15:10:17.042752 ip-10-0-133-210 kubenswrapper[2576]: W0730 15:10:17.041335 2576 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 30 15:10:17.042752 ip-10-0-133-210 kubenswrapper[2576]: W0730 15:10:17.041338 2576 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 30 15:10:17.042752 ip-10-0-133-210 kubenswrapper[2576]: W0730 15:10:17.041341 2576 feature_gate.go:328] unrecognized feature gate: Example2 Jul 30 15:10:17.042752 ip-10-0-133-210 kubenswrapper[2576]: W0730 15:10:17.041344 2576 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 30 15:10:17.042752 ip-10-0-133-210 kubenswrapper[2576]: W0730 15:10:17.041347 2576 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 30 15:10:17.042752 ip-10-0-133-210 kubenswrapper[2576]: W0730 15:10:17.041350 2576 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 30 15:10:17.042752 ip-10-0-133-210 kubenswrapper[2576]: W0730 15:10:17.041353 2576 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 30 15:10:17.042752 ip-10-0-133-210 kubenswrapper[2576]: W0730 15:10:17.041355 2576 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 30 15:10:17.042752 ip-10-0-133-210 kubenswrapper[2576]: W0730 15:10:17.041358 2576 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 30 15:10:17.042752 ip-10-0-133-210 kubenswrapper[2576]: W0730 15:10:17.041361 2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 30 15:10:17.042752 ip-10-0-133-210 kubenswrapper[2576]: W0730 15:10:17.041364 2576 feature_gate.go:328] unrecognized feature gate: Example Jul 30 15:10:17.043270 ip-10-0-133-210 kubenswrapper[2576]: W0730 15:10:17.041367 2576 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 30 15:10:17.043270 ip-10-0-133-210 kubenswrapper[2576]: W0730 15:10:17.041370 2576 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 30 15:10:17.043270 ip-10-0-133-210 kubenswrapper[2576]: W0730 15:10:17.041372 2576 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 30 15:10:17.043270 ip-10-0-133-210 kubenswrapper[2576]: W0730 15:10:17.041375 2576 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 30 15:10:17.043270 ip-10-0-133-210 kubenswrapper[2576]: W0730 15:10:17.041378 2576 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 30 15:10:17.043270 ip-10-0-133-210 kubenswrapper[2576]: W0730 15:10:17.041380 2576 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 30 15:10:17.043270 ip-10-0-133-210 kubenswrapper[2576]: W0730 15:10:17.041383 2576 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 30 15:10:17.043270 ip-10-0-133-210 kubenswrapper[2576]: W0730 15:10:17.041385 2576 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 30 15:10:17.043270 ip-10-0-133-210 kubenswrapper[2576]: W0730 15:10:17.041388 2576 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 30 15:10:17.043270 ip-10-0-133-210 kubenswrapper[2576]: W0730 15:10:17.041390 2576 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 30 15:10:17.043270 ip-10-0-133-210 kubenswrapper[2576]: W0730 15:10:17.041393 2576 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 30 15:10:17.043270 ip-10-0-133-210 kubenswrapper[2576]: W0730 15:10:17.041396 2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 30 15:10:17.043270 ip-10-0-133-210 kubenswrapper[2576]: W0730 15:10:17.041398 2576 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 30 15:10:17.043270 ip-10-0-133-210 kubenswrapper[2576]: W0730 15:10:17.041401 2576 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 30 15:10:17.043270 ip-10-0-133-210 kubenswrapper[2576]: W0730 15:10:17.041404 2576 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 30 15:10:17.043270 ip-10-0-133-210 kubenswrapper[2576]: W0730 15:10:17.041406 2576 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 30 15:10:17.043270 ip-10-0-133-210 kubenswrapper[2576]: W0730 15:10:17.041409 2576 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 30 15:10:17.043270 ip-10-0-133-210 kubenswrapper[2576]: W0730 15:10:17.041413 2576 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 30 15:10:17.043270 ip-10-0-133-210 kubenswrapper[2576]: W0730 15:10:17.041418 2576 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 30 15:10:17.043744 ip-10-0-133-210 kubenswrapper[2576]: W0730 15:10:17.041420 2576 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 30 15:10:17.043744 ip-10-0-133-210 kubenswrapper[2576]: W0730 15:10:17.041423 2576 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 30 15:10:17.043744 ip-10-0-133-210 kubenswrapper[2576]: W0730 15:10:17.041426 2576 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 30 15:10:17.043744 ip-10-0-133-210 kubenswrapper[2576]: W0730 15:10:17.041428 2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 30 15:10:17.043744 ip-10-0-133-210 kubenswrapper[2576]: W0730 15:10:17.041431 2576 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 30 15:10:17.043744 ip-10-0-133-210 kubenswrapper[2576]: W0730 15:10:17.041434 2576 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 30 15:10:17.043744 ip-10-0-133-210 kubenswrapper[2576]: W0730 15:10:17.041437 2576 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 30 15:10:17.043744 ip-10-0-133-210 kubenswrapper[2576]: W0730 15:10:17.041439 2576 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 30 15:10:17.043744 ip-10-0-133-210 kubenswrapper[2576]: W0730 15:10:17.041442 2576 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 30 15:10:17.043744 ip-10-0-133-210 kubenswrapper[2576]: W0730 15:10:17.041444 2576 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 30 15:10:17.043744 ip-10-0-133-210 kubenswrapper[2576]: W0730 15:10:17.041448 2576 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 30 15:10:17.043744 ip-10-0-133-210 kubenswrapper[2576]: W0730 15:10:17.041451 2576 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 30 15:10:17.043744 ip-10-0-133-210 kubenswrapper[2576]: W0730 15:10:17.041454 2576 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 30 15:10:17.043744 ip-10-0-133-210 kubenswrapper[2576]: W0730 15:10:17.041456 2576 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 30 15:10:17.043744 ip-10-0-133-210 kubenswrapper[2576]: W0730 15:10:17.041459 2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 30 15:10:17.043744 ip-10-0-133-210 kubenswrapper[2576]: W0730 15:10:17.041462 2576 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 30 15:10:17.043744 ip-10-0-133-210 kubenswrapper[2576]: W0730 15:10:17.041464 2576 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 30 15:10:17.043744 ip-10-0-133-210 kubenswrapper[2576]: W0730 15:10:17.041467 2576 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 30 15:10:17.043744 ip-10-0-133-210 kubenswrapper[2576]: W0730 15:10:17.041469 2576 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 30 15:10:17.043744 ip-10-0-133-210 kubenswrapper[2576]: W0730 15:10:17.041472 2576 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 30 15:10:17.044256 ip-10-0-133-210 kubenswrapper[2576]: W0730 15:10:17.041474 2576 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 30 15:10:17.044256 ip-10-0-133-210 kubenswrapper[2576]: W0730 15:10:17.041477 2576 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 30 15:10:17.044256 ip-10-0-133-210 kubenswrapper[2576]: W0730 15:10:17.041479 2576 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 30 15:10:17.044256 ip-10-0-133-210 kubenswrapper[2576]: W0730 15:10:17.041482 2576 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 30 15:10:17.044256 ip-10-0-133-210 kubenswrapper[2576]: W0730 15:10:17.041484 2576 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 30 15:10:17.044256 ip-10-0-133-210 kubenswrapper[2576]: W0730 15:10:17.041487 2576 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 30 15:10:17.044256 ip-10-0-133-210 kubenswrapper[2576]: W0730 15:10:17.041489 2576 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 30 15:10:17.044256 ip-10-0-133-210 kubenswrapper[2576]: W0730 15:10:17.041492 2576 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 30 15:10:17.044256 ip-10-0-133-210 kubenswrapper[2576]: W0730 15:10:17.041495 2576 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 30 15:10:17.044256 ip-10-0-133-210 kubenswrapper[2576]: W0730 15:10:17.041497 2576 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 30 15:10:17.044256 ip-10-0-133-210 kubenswrapper[2576]: W0730 15:10:17.041500 2576 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 30 15:10:17.044256 ip-10-0-133-210 kubenswrapper[2576]: W0730 15:10:17.041503 2576 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 30 15:10:17.044256 ip-10-0-133-210 kubenswrapper[2576]: W0730 15:10:17.041505 2576 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 30 15:10:17.044256 ip-10-0-133-210 kubenswrapper[2576]: W0730 15:10:17.041508 2576 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 30 15:10:17.044256 ip-10-0-133-210 kubenswrapper[2576]: W0730 15:10:17.041510 2576 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 30 15:10:17.044256 ip-10-0-133-210 kubenswrapper[2576]: W0730 15:10:17.041515 2576 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 30 15:10:17.044256 ip-10-0-133-210 kubenswrapper[2576]: W0730 15:10:17.041519 2576 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 30 15:10:17.044256 ip-10-0-133-210 kubenswrapper[2576]: W0730 15:10:17.041523 2576 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 30 15:10:17.044256 ip-10-0-133-210 kubenswrapper[2576]: W0730 15:10:17.041526 2576 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 30 15:10:17.044737 ip-10-0-133-210 kubenswrapper[2576]: W0730 15:10:17.041530 2576 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 30 15:10:17.044737 ip-10-0-133-210 kubenswrapper[2576]: W0730 15:10:17.041533 2576 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 30 15:10:17.044737 ip-10-0-133-210 kubenswrapper[2576]: W0730 15:10:17.041537 2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 30 15:10:17.044737 ip-10-0-133-210 kubenswrapper[2576]: W0730 15:10:17.041540 2576 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 30 15:10:17.044737 ip-10-0-133-210 kubenswrapper[2576]: W0730 15:10:17.041543 2576 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 30 15:10:17.044737 ip-10-0-133-210 kubenswrapper[2576]: W0730 15:10:17.041546 2576 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 30 15:10:17.044737 ip-10-0-133-210 kubenswrapper[2576]: W0730 15:10:17.041549 2576 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 30 15:10:17.044737 ip-10-0-133-210 kubenswrapper[2576]: W0730 15:10:17.041552 2576 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 30 15:10:17.044737 ip-10-0-133-210 kubenswrapper[2576]: W0730 15:10:17.041555 2576 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 30 15:10:17.044737 ip-10-0-133-210 kubenswrapper[2576]: W0730 15:10:17.041557 2576 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 30 15:10:17.044737 ip-10-0-133-210 kubenswrapper[2576]: W0730 15:10:17.041559 2576 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 30 15:10:17.044737 ip-10-0-133-210 kubenswrapper[2576]: W0730 15:10:17.041562 2576 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 30 15:10:17.044737 ip-10-0-133-210 kubenswrapper[2576]: W0730 15:10:17.041564 2576 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 30 15:10:17.044737 ip-10-0-133-210 kubenswrapper[2576]: W0730 15:10:17.041567 2576 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 30 15:10:17.044737 ip-10-0-133-210 kubenswrapper[2576]: W0730 15:10:17.041569 2576 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 30 15:10:17.044737 ip-10-0-133-210 kubenswrapper[2576]: W0730 15:10:17.041572 2576 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 30 15:10:17.044737 ip-10-0-133-210 kubenswrapper[2576]: W0730 15:10:17.041575 2576 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 30 15:10:17.044737 ip-10-0-133-210 kubenswrapper[2576]: W0730 15:10:17.041577 2576 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 30 15:10:17.044737 ip-10-0-133-210 kubenswrapper[2576]: W0730 15:10:17.041580 2576 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 30 15:10:17.044737 ip-10-0-133-210 kubenswrapper[2576]: W0730 15:10:17.041966 2576 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 30 15:10:17.045236 ip-10-0-133-210 kubenswrapper[2576]: W0730 15:10:17.041971 2576 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 30 15:10:17.045236 ip-10-0-133-210 kubenswrapper[2576]: W0730 15:10:17.041975 2576 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 30 15:10:17.045236 ip-10-0-133-210 kubenswrapper[2576]: W0730 15:10:17.041978 2576 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 30 15:10:17.045236 ip-10-0-133-210 kubenswrapper[2576]: W0730 15:10:17.041981 2576 feature_gate.go:328] unrecognized feature gate: Example2 Jul 30 15:10:17.045236 ip-10-0-133-210 kubenswrapper[2576]: W0730 15:10:17.041984 2576 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 30 15:10:17.045236 ip-10-0-133-210 kubenswrapper[2576]: W0730 15:10:17.041987 2576 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 30 15:10:17.045236 ip-10-0-133-210 kubenswrapper[2576]: W0730 15:10:17.041989 2576 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 30 15:10:17.045236 ip-10-0-133-210 kubenswrapper[2576]: W0730 15:10:17.041992 2576 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 30 15:10:17.045236 ip-10-0-133-210 kubenswrapper[2576]: W0730 15:10:17.041995 2576 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 30 15:10:17.045236 ip-10-0-133-210 kubenswrapper[2576]: W0730 15:10:17.042000 2576 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 30 15:10:17.045236 ip-10-0-133-210 kubenswrapper[2576]: W0730 15:10:17.042005 2576 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 30 15:10:17.045236 ip-10-0-133-210 kubenswrapper[2576]: W0730 15:10:17.042009 2576 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 30 15:10:17.045236 ip-10-0-133-210 kubenswrapper[2576]: W0730 15:10:17.042012 2576 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 30 15:10:17.045236 ip-10-0-133-210 kubenswrapper[2576]: W0730 15:10:17.042016 2576 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 30 15:10:17.045236 ip-10-0-133-210 kubenswrapper[2576]: W0730 15:10:17.042022 2576 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 30 15:10:17.045236 ip-10-0-133-210 kubenswrapper[2576]: W0730 15:10:17.042025 2576 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 30 15:10:17.045236 ip-10-0-133-210 kubenswrapper[2576]: W0730 15:10:17.042028 2576 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 30 15:10:17.045236 ip-10-0-133-210 kubenswrapper[2576]: W0730 15:10:17.042031 2576 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 30 15:10:17.045236 ip-10-0-133-210 kubenswrapper[2576]: W0730 15:10:17.042033 2576 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 30 15:10:17.045695 ip-10-0-133-210 kubenswrapper[2576]: W0730 15:10:17.042036 2576 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 30 15:10:17.045695 ip-10-0-133-210 kubenswrapper[2576]: W0730 15:10:17.042039 2576 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 30 15:10:17.045695 ip-10-0-133-210 kubenswrapper[2576]: W0730 15:10:17.042042 2576 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 30 15:10:17.045695 ip-10-0-133-210 kubenswrapper[2576]: W0730 15:10:17.042045 2576 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 30 15:10:17.045695 ip-10-0-133-210 kubenswrapper[2576]: W0730 15:10:17.042047 2576 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 30 15:10:17.045695 ip-10-0-133-210 kubenswrapper[2576]: W0730 15:10:17.042050 2576 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 30 15:10:17.045695 ip-10-0-133-210 kubenswrapper[2576]: W0730 15:10:17.042053 2576 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 30 15:10:17.045695 ip-10-0-133-210 kubenswrapper[2576]: W0730 15:10:17.042055 2576 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 30 15:10:17.045695 ip-10-0-133-210 kubenswrapper[2576]: W0730 15:10:17.042058 2576 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 30 15:10:17.045695 ip-10-0-133-210 kubenswrapper[2576]: W0730 15:10:17.042060 2576 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 30 15:10:17.045695 ip-10-0-133-210 kubenswrapper[2576]: W0730 15:10:17.042063 2576 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 30 15:10:17.045695 ip-10-0-133-210 kubenswrapper[2576]: W0730 15:10:17.042065 2576 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 30 15:10:17.045695 ip-10-0-133-210 kubenswrapper[2576]: W0730 15:10:17.042068 2576 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 30 15:10:17.045695 ip-10-0-133-210 kubenswrapper[2576]: W0730 15:10:17.042071 2576 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 30 15:10:17.045695 ip-10-0-133-210 kubenswrapper[2576]: W0730 15:10:17.042073 2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 30 15:10:17.045695 ip-10-0-133-210 kubenswrapper[2576]: W0730 15:10:17.042076 2576 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 30 15:10:17.045695 ip-10-0-133-210 kubenswrapper[2576]: W0730 15:10:17.042078 2576 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 30 15:10:17.045695 ip-10-0-133-210 kubenswrapper[2576]: W0730 15:10:17.042081 2576 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 30 15:10:17.045695 ip-10-0-133-210 kubenswrapper[2576]: W0730 15:10:17.042084 2576 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 30 15:10:17.045695 ip-10-0-133-210 kubenswrapper[2576]: W0730 15:10:17.042086 2576 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 30 15:10:17.046240 ip-10-0-133-210 kubenswrapper[2576]: W0730 15:10:17.042089 2576 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 30 15:10:17.046240 ip-10-0-133-210 kubenswrapper[2576]: W0730 15:10:17.042092 2576 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 30 15:10:17.046240 ip-10-0-133-210 kubenswrapper[2576]: W0730 15:10:17.042095 2576 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 30 15:10:17.046240 ip-10-0-133-210 kubenswrapper[2576]: W0730 15:10:17.042098 2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 30 15:10:17.046240 ip-10-0-133-210 kubenswrapper[2576]: W0730 15:10:17.042100 2576 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 30 15:10:17.046240 ip-10-0-133-210 kubenswrapper[2576]: W0730 15:10:17.042103 2576 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 30 15:10:17.046240 ip-10-0-133-210 kubenswrapper[2576]: W0730 15:10:17.042105 2576 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 30 15:10:17.046240 ip-10-0-133-210 kubenswrapper[2576]: W0730 15:10:17.042109 2576 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 30 15:10:17.046240 ip-10-0-133-210 kubenswrapper[2576]: W0730 15:10:17.042112 2576 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 30 15:10:17.046240 ip-10-0-133-210 kubenswrapper[2576]: W0730 15:10:17.042114 2576 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 30 15:10:17.046240 ip-10-0-133-210 kubenswrapper[2576]: W0730 15:10:17.042117 2576 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 30 15:10:17.046240 ip-10-0-133-210 kubenswrapper[2576]: W0730 15:10:17.042120 2576 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 30 15:10:17.046240 ip-10-0-133-210 kubenswrapper[2576]: W0730 15:10:17.042122 2576 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 30 15:10:17.046240 ip-10-0-133-210 kubenswrapper[2576]: W0730 15:10:17.042124 2576 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 30 15:10:17.046240 ip-10-0-133-210 kubenswrapper[2576]: W0730 15:10:17.042127 2576 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 30 15:10:17.046240 ip-10-0-133-210 kubenswrapper[2576]: W0730 15:10:17.042130 2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 30 15:10:17.046240 ip-10-0-133-210 kubenswrapper[2576]: W0730 15:10:17.042132 2576 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 30 15:10:17.046240 ip-10-0-133-210 kubenswrapper[2576]: W0730 15:10:17.042135 2576 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 30 15:10:17.046240 ip-10-0-133-210 kubenswrapper[2576]: W0730 15:10:17.042137 2576 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 30 15:10:17.046702 ip-10-0-133-210 kubenswrapper[2576]: W0730 15:10:17.042140 2576 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 30 15:10:17.046702 ip-10-0-133-210 kubenswrapper[2576]: W0730 15:10:17.042143 2576 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 30 15:10:17.046702 ip-10-0-133-210 kubenswrapper[2576]: W0730 15:10:17.042146 2576 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 30 15:10:17.046702 ip-10-0-133-210 kubenswrapper[2576]: W0730 15:10:17.042149 2576 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 30 15:10:17.046702 ip-10-0-133-210 kubenswrapper[2576]: W0730 15:10:17.042151 2576 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 30 15:10:17.046702 ip-10-0-133-210 kubenswrapper[2576]: W0730 15:10:17.042154 2576 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 30 15:10:17.046702 ip-10-0-133-210 kubenswrapper[2576]: W0730 15:10:17.042156 2576 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 30 15:10:17.046702 ip-10-0-133-210 kubenswrapper[2576]: W0730 15:10:17.042161 2576 feature_gate.go:328] unrecognized feature gate: Example Jul 30 15:10:17.046702 ip-10-0-133-210 kubenswrapper[2576]: W0730 15:10:17.042163 2576 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 30 15:10:17.046702 ip-10-0-133-210 kubenswrapper[2576]: W0730 15:10:17.042166 2576 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 30 15:10:17.046702 ip-10-0-133-210 kubenswrapper[2576]: W0730 15:10:17.042169 2576 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 30 15:10:17.046702 ip-10-0-133-210 kubenswrapper[2576]: W0730 15:10:17.042172 2576 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 30 15:10:17.046702 ip-10-0-133-210 kubenswrapper[2576]: W0730 15:10:17.042174 2576 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 30 15:10:17.046702 ip-10-0-133-210 kubenswrapper[2576]: W0730 15:10:17.042177 2576 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 30 15:10:17.046702 ip-10-0-133-210 kubenswrapper[2576]: W0730 15:10:17.042179 2576 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 30 15:10:17.046702 ip-10-0-133-210 kubenswrapper[2576]: W0730 15:10:17.042182 2576 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 30 15:10:17.046702 ip-10-0-133-210 kubenswrapper[2576]: W0730 15:10:17.042185 2576 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 30 15:10:17.046702 ip-10-0-133-210 kubenswrapper[2576]: W0730 15:10:17.042187 2576 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 30 15:10:17.046702 ip-10-0-133-210 kubenswrapper[2576]: W0730 15:10:17.042189 2576 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 30 15:10:17.046702 ip-10-0-133-210 kubenswrapper[2576]: W0730 15:10:17.042193 2576 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 30 15:10:17.047242 ip-10-0-133-210 kubenswrapper[2576]: W0730 15:10:17.042198 2576 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 30 15:10:17.047242 ip-10-0-133-210 kubenswrapper[2576]: W0730 15:10:17.042201 2576 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 30 15:10:17.047242 ip-10-0-133-210 kubenswrapper[2576]: W0730 15:10:17.042204 2576 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 30 15:10:17.047242 ip-10-0-133-210 kubenswrapper[2576]: W0730 15:10:17.042207 2576 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 30 15:10:17.047242 ip-10-0-133-210 kubenswrapper[2576]: W0730 15:10:17.042209 2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 30 15:10:17.047242 ip-10-0-133-210 kubenswrapper[2576]: W0730 15:10:17.042212 2576 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 30 15:10:17.047242 ip-10-0-133-210 kubenswrapper[2576]: W0730 15:10:17.042214 2576 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 30 15:10:17.047242 ip-10-0-133-210 kubenswrapper[2576]: W0730 15:10:17.042217 2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 30 15:10:17.047242 ip-10-0-133-210 kubenswrapper[2576]: W0730 15:10:17.042220 2576 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 30 15:10:17.047242 ip-10-0-133-210 kubenswrapper[2576]: W0730 15:10:17.042223 2576 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 30 15:10:17.047242 ip-10-0-133-210 kubenswrapper[2576]: W0730 15:10:17.042225 2576 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 30 15:10:17.047242 ip-10-0-133-210 kubenswrapper[2576]: W0730 15:10:17.042228 2576 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 30 15:10:17.047242 ip-10-0-133-210 kubenswrapper[2576]: W0730 15:10:17.042230 2576 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 30 15:10:17.047242 ip-10-0-133-210 kubenswrapper[2576]: W0730 15:10:17.042233 2576 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 30 15:10:17.047242 ip-10-0-133-210 kubenswrapper[2576]: W0730 15:10:17.042235 2576 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 30 15:10:17.047242 ip-10-0-133-210 kubenswrapper[2576]: W0730 15:10:17.042238 2576 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 30 15:10:17.047242 ip-10-0-133-210 kubenswrapper[2576]: W0730 15:10:17.042240 2576 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 30 15:10:17.047242 ip-10-0-133-210 kubenswrapper[2576]: W0730 15:10:17.042243 2576 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 30 15:10:17.047242 ip-10-0-133-210 kubenswrapper[2576]: W0730 15:10:17.042245 2576 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 30 15:10:17.047242 ip-10-0-133-210 kubenswrapper[2576]: W0730 15:10:17.042249 2576 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 30 15:10:17.047739 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:10:17.042318 2576 flags.go:64] FLAG: --address="0.0.0.0" Jul 30 15:10:17.047739 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:10:17.042326 2576 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]" Jul 30 15:10:17.047739 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:10:17.042333 2576 flags.go:64] FLAG: --anonymous-auth="true" Jul 30 15:10:17.047739 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:10:17.042337 2576 flags.go:64] FLAG: --application-metrics-count-limit="100" Jul 30 15:10:17.047739 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:10:17.042342 2576 flags.go:64] FLAG: --authentication-token-webhook="false" Jul 30 15:10:17.047739 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:10:17.042346 2576 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s" Jul 30 15:10:17.047739 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:10:17.042350 2576 flags.go:64] FLAG: --authorization-mode="AlwaysAllow" Jul 30 15:10:17.047739 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:10:17.042355 2576 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s" Jul 30 15:10:17.047739 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:10:17.042358 2576 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s" Jul 30 15:10:17.047739 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:10:17.042361 2576 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id" Jul 30 15:10:17.047739 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:10:17.042365 2576 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig" Jul 30 15:10:17.047739 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:10:17.042368 2576 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki" Jul 30 15:10:17.047739 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:10:17.042372 2576 flags.go:64] FLAG: --cgroup-driver="cgroupfs" Jul 30 15:10:17.047739 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:10:17.042375 2576 flags.go:64] FLAG: --cgroup-root="" Jul 30 15:10:17.047739 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:10:17.042378 2576 flags.go:64] FLAG: --cgroups-per-qos="true" Jul 30 15:10:17.047739 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:10:17.042381 2576 flags.go:64] FLAG: --client-ca-file="" Jul 30 15:10:17.047739 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:10:17.042385 2576 flags.go:64] FLAG: --cloud-provider="external" Jul 30 15:10:17.047739 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:10:17.042388 2576 flags.go:64] FLAG: --cluster-dns="[]" Jul 30 15:10:17.047739 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:10:17.042393 2576 flags.go:64] FLAG: --cluster-domain="" Jul 30 15:10:17.047739 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:10:17.042396 2576 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf" Jul 30 15:10:17.047739 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:10:17.042399 2576 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d" Jul 30 15:10:17.047739 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:10:17.042402 2576 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json" Jul 30 15:10:17.047739 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:10:17.042406 2576 flags.go:64] FLAG: --container-log-max-files="5" Jul 30 15:10:17.047739 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:10:17.042410 2576 flags.go:64] FLAG: --container-log-max-size="10Mi" Jul 30 15:10:17.048325 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:10:17.042413 2576 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock" Jul 30 15:10:17.048325 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:10:17.042416 2576 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock" Jul 30 15:10:17.048325 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:10:17.042419 2576 flags.go:64] FLAG: --containerd-namespace="k8s.io" Jul 30 15:10:17.048325 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:10:17.042422 2576 flags.go:64] FLAG: --contention-profiling="false" Jul 30 15:10:17.048325 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:10:17.042425 2576 flags.go:64] FLAG: --cpu-cfs-quota="true" Jul 30 15:10:17.048325 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:10:17.042428 2576 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms" Jul 30 15:10:17.048325 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:10:17.042431 2576 flags.go:64] FLAG: --cpu-manager-policy="none" Jul 30 15:10:17.048325 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:10:17.042436 2576 flags.go:64] FLAG: --cpu-manager-policy-options="" Jul 30 15:10:17.048325 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:10:17.042441 2576 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s" Jul 30 15:10:17.048325 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:10:17.042444 2576 flags.go:64] FLAG: --enable-controller-attach-detach="true" Jul 30 15:10:17.048325 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:10:17.042447 2576 flags.go:64] FLAG: --enable-debugging-handlers="true" Jul 30 15:10:17.048325 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:10:17.042450 2576 flags.go:64] FLAG: --enable-load-reader="false" Jul 30 15:10:17.048325 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:10:17.042453 2576 flags.go:64] FLAG: --enable-server="true" Jul 30 15:10:17.048325 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:10:17.042457 2576 flags.go:64] FLAG: --enforce-node-allocatable="[pods]" Jul 30 15:10:17.048325 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:10:17.042461 2576 flags.go:64] FLAG: --event-burst="100" Jul 30 15:10:17.048325 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:10:17.042464 2576 flags.go:64] FLAG: --event-qps="50" Jul 30 15:10:17.048325 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:10:17.042467 2576 flags.go:64] FLAG: --event-storage-age-limit="default=0" Jul 30 15:10:17.048325 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:10:17.042470 2576 flags.go:64] FLAG: --event-storage-event-limit="default=0" Jul 30 15:10:17.048325 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:10:17.042473 2576 flags.go:64] FLAG: --eviction-hard="" Jul 30 15:10:17.048325 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:10:17.042477 2576 flags.go:64] FLAG: --eviction-max-pod-grace-period="0" Jul 30 15:10:17.048325 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:10:17.042480 2576 flags.go:64] FLAG: --eviction-minimum-reclaim="" Jul 30 15:10:17.048325 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:10:17.042484 2576 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s" Jul 30 15:10:17.048325 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:10:17.042487 2576 flags.go:64] FLAG: --eviction-soft="" Jul 30 15:10:17.048325 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:10:17.042490 2576 flags.go:64] FLAG: --eviction-soft-grace-period="" Jul 30 15:10:17.048325 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:10:17.042493 2576 flags.go:64] FLAG: --exit-on-lock-contention="false" Jul 30 15:10:17.048963 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:10:17.042496 2576 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false" Jul 30 15:10:17.048963 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:10:17.042499 2576 flags.go:64] FLAG: --experimental-mounter-path="" Jul 30 15:10:17.048963 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:10:17.042502 2576 flags.go:64] FLAG: --fail-cgroupv1="false" Jul 30 15:10:17.048963 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:10:17.042505 2576 flags.go:64] FLAG: --fail-swap-on="true" Jul 30 15:10:17.048963 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:10:17.042507 2576 flags.go:64] FLAG: --feature-gates="" Jul 30 15:10:17.048963 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:10:17.042511 2576 flags.go:64] FLAG: --file-check-frequency="20s" Jul 30 15:10:17.048963 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:10:17.042514 2576 flags.go:64] FLAG: --global-housekeeping-interval="1m0s" Jul 30 15:10:17.048963 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:10:17.042517 2576 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge" Jul 30 15:10:17.048963 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:10:17.042520 2576 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1" Jul 30 15:10:17.048963 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:10:17.042523 2576 flags.go:64] FLAG: --healthz-port="10248" Jul 30 15:10:17.048963 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:10:17.042527 2576 flags.go:64] FLAG: --help="false" Jul 30 15:10:17.048963 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:10:17.042529 2576 flags.go:64] FLAG: --hostname-override="ip-10-0-133-210.ec2.internal" Jul 30 15:10:17.048963 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:10:17.042533 2576 flags.go:64] FLAG: --housekeeping-interval="10s" Jul 30 15:10:17.048963 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:10:17.042536 2576 flags.go:64] FLAG: --http-check-frequency="20s" Jul 30 15:10:17.048963 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:10:17.042540 2576 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins" Jul 30 15:10:17.048963 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:10:17.042543 2576 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml" Jul 30 15:10:17.048963 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:10:17.042546 2576 flags.go:64] FLAG: --image-gc-high-threshold="85" Jul 30 15:10:17.048963 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:10:17.042549 2576 flags.go:64] FLAG: --image-gc-low-threshold="80" Jul 30 15:10:17.048963 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:10:17.042552 2576 flags.go:64] FLAG: --image-service-endpoint="" Jul 30 15:10:17.048963 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:10:17.042556 2576 flags.go:64] FLAG: --kernel-memcg-notification="false" Jul 30 15:10:17.048963 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:10:17.042559 2576 flags.go:64] FLAG: --kube-api-burst="100" Jul 30 15:10:17.048963 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:10:17.042562 2576 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf" Jul 30 15:10:17.048963 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:10:17.042566 2576 flags.go:64] FLAG: --kube-api-qps="50" Jul 30 15:10:17.048963 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:10:17.042568 2576 flags.go:64] FLAG: --kube-reserved="" Jul 30 15:10:17.049722 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:10:17.042571 2576 flags.go:64] FLAG: --kube-reserved-cgroup="" Jul 30 15:10:17.049722 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:10:17.042574 2576 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig" Jul 30 15:10:17.049722 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:10:17.042577 2576 flags.go:64] FLAG: --kubelet-cgroups="" Jul 30 15:10:17.049722 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:10:17.042580 2576 flags.go:64] FLAG: --local-storage-capacity-isolation="true" Jul 30 15:10:17.049722 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:10:17.042583 2576 flags.go:64] FLAG: --lock-file="" Jul 30 15:10:17.049722 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:10:17.042586 2576 flags.go:64] FLAG: --log-cadvisor-usage="false" Jul 30 15:10:17.049722 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:10:17.042589 2576 flags.go:64] FLAG: --log-flush-frequency="5s" Jul 30 15:10:17.049722 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:10:17.042592 2576 flags.go:64] FLAG: --log-json-info-buffer-size="0" Jul 30 15:10:17.049722 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:10:17.042598 2576 flags.go:64] FLAG: --log-json-split-stream="false" Jul 30 15:10:17.049722 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:10:17.042600 2576 flags.go:64] FLAG: --log-text-info-buffer-size="0" Jul 30 15:10:17.049722 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:10:17.042603 2576 flags.go:64] FLAG: --log-text-split-stream="false" Jul 30 15:10:17.049722 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:10:17.042606 2576 flags.go:64] FLAG: --logging-format="text" Jul 30 15:10:17.049722 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:10:17.042609 2576 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id" Jul 30 15:10:17.049722 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:10:17.042612 2576 flags.go:64] FLAG: --make-iptables-util-chains="true" Jul 30 15:10:17.049722 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:10:17.042615 2576 flags.go:64] FLAG: --manifest-url="" Jul 30 15:10:17.049722 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:10:17.042618 2576 flags.go:64] FLAG: --manifest-url-header="" Jul 30 15:10:17.049722 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:10:17.042622 2576 flags.go:64] FLAG: --max-housekeeping-interval="15s" Jul 30 15:10:17.049722 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:10:17.042625 2576 flags.go:64] FLAG: --max-open-files="1000000" Jul 30 15:10:17.049722 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:10:17.042629 2576 flags.go:64] FLAG: --max-pods="110" Jul 30 15:10:17.049722 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:10:17.042632 2576 flags.go:64] FLAG: --maximum-dead-containers="-1" Jul 30 15:10:17.049722 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:10:17.042635 2576 flags.go:64] FLAG: --maximum-dead-containers-per-container="1" Jul 30 15:10:17.049722 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:10:17.042638 2576 flags.go:64] FLAG: --memory-manager-policy="None" Jul 30 15:10:17.049722 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:10:17.042642 2576 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s" Jul 30 15:10:17.049722 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:10:17.042646 2576 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s" Jul 30 15:10:17.049722 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:10:17.042649 2576 flags.go:64] FLAG: --node-ip="0.0.0.0" Jul 30 15:10:17.050763 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:10:17.042652 2576 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel" Jul 30 15:10:17.050763 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:10:17.042659 2576 flags.go:64] FLAG: --node-status-max-images="50" Jul 30 15:10:17.050763 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:10:17.042662 2576 flags.go:64] FLAG: --node-status-update-frequency="10s" Jul 30 15:10:17.050763 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:10:17.042665 2576 flags.go:64] FLAG: --oom-score-adj="-999" Jul 30 15:10:17.050763 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:10:17.042669 2576 flags.go:64] FLAG: --pod-cidr="" Jul 30 15:10:17.050763 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:10:17.042672 2576 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:7505336a406394b8c1c77d14d890a9f628aa4cfff172312179cb9af8c28c30bb" Jul 30 15:10:17.050763 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:10:17.042677 2576 flags.go:64] FLAG: --pod-manifest-path="" Jul 30 15:10:17.050763 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:10:17.042680 2576 flags.go:64] FLAG: --pod-max-pids="-1" Jul 30 15:10:17.050763 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:10:17.042684 2576 flags.go:64] FLAG: --pods-per-core="0" Jul 30 15:10:17.050763 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:10:17.042686 2576 flags.go:64] FLAG: --port="10250" Jul 30 15:10:17.050763 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:10:17.042689 2576 flags.go:64] FLAG: --protect-kernel-defaults="false" Jul 30 15:10:17.050763 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:10:17.042692 2576 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-0cde2cd904d216a3f" Jul 30 15:10:17.050763 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:10:17.042695 2576 flags.go:64] FLAG: --qos-reserved="" Jul 30 15:10:17.050763 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:10:17.042698 2576 flags.go:64] FLAG: --read-only-port="10255" Jul 30 15:10:17.050763 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:10:17.042701 2576 flags.go:64] FLAG: --register-node="true" Jul 30 15:10:17.050763 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:10:17.042704 2576 flags.go:64] FLAG: --register-with-taints="" Jul 30 15:10:17.050763 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:10:17.042708 2576 flags.go:64] FLAG: --registry-burst="10" Jul 30 15:10:17.050763 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:10:17.042711 2576 flags.go:64] FLAG: --registry-qps="5" Jul 30 15:10:17.050763 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:10:17.042714 2576 flags.go:64] FLAG: --reserved-cpus="" Jul 30 15:10:17.050763 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:10:17.042718 2576 flags.go:64] FLAG: --reserved-memory="" Jul 30 15:10:17.050763 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:10:17.042721 2576 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf" Jul 30 15:10:17.050763 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:10:17.042724 2576 flags.go:64] FLAG: --root-dir="/var/lib/kubelet" Jul 30 15:10:17.050763 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:10:17.042728 2576 flags.go:64] FLAG: --rotate-certificates="false" Jul 30 15:10:17.050763 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:10:17.042730 2576 flags.go:64] FLAG: --rotate-server-certificates="false" Jul 30 15:10:17.050763 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:10:17.042733 2576 flags.go:64] FLAG: --runonce="false" Jul 30 15:10:17.051449 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:10:17.042736 2576 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service" Jul 30 15:10:17.051449 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:10:17.042739 2576 flags.go:64] FLAG: --runtime-request-timeout="2m0s" Jul 30 15:10:17.051449 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:10:17.042742 2576 flags.go:64] FLAG: --seccomp-default="false" Jul 30 15:10:17.051449 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:10:17.042745 2576 flags.go:64] FLAG: --serialize-image-pulls="true" Jul 30 15:10:17.051449 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:10:17.042748 2576 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s" Jul 30 15:10:17.051449 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:10:17.042752 2576 flags.go:64] FLAG: --storage-driver-db="cadvisor" Jul 30 15:10:17.051449 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:10:17.042756 2576 flags.go:64] FLAG: --storage-driver-host="localhost:8086" Jul 30 15:10:17.051449 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:10:17.042759 2576 flags.go:64] FLAG: --storage-driver-password="root" Jul 30 15:10:17.051449 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:10:17.042764 2576 flags.go:64] FLAG: --storage-driver-secure="false" Jul 30 15:10:17.051449 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:10:17.042767 2576 flags.go:64] FLAG: --storage-driver-table="stats" Jul 30 15:10:17.051449 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:10:17.042770 2576 flags.go:64] FLAG: --storage-driver-user="root" Jul 30 15:10:17.051449 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:10:17.042773 2576 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s" Jul 30 15:10:17.051449 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:10:17.042776 2576 flags.go:64] FLAG: --sync-frequency="1m0s" Jul 30 15:10:17.051449 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:10:17.042780 2576 flags.go:64] FLAG: --system-cgroups="" Jul 30 15:10:17.051449 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:10:17.042783 2576 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi" Jul 30 15:10:17.051449 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:10:17.042788 2576 flags.go:64] FLAG: --system-reserved-cgroup="" Jul 30 15:10:17.051449 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:10:17.042791 2576 flags.go:64] FLAG: --tls-cert-file="" Jul 30 15:10:17.051449 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:10:17.042794 2576 flags.go:64] FLAG: --tls-cipher-suites="[]" Jul 30 15:10:17.051449 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:10:17.042799 2576 flags.go:64] FLAG: --tls-min-version="" Jul 30 15:10:17.051449 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:10:17.042802 2576 flags.go:64] FLAG: --tls-private-key-file="" Jul 30 15:10:17.051449 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:10:17.042805 2576 flags.go:64] FLAG: --topology-manager-policy="none" Jul 30 15:10:17.051449 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:10:17.042808 2576 flags.go:64] FLAG: --topology-manager-policy-options="" Jul 30 15:10:17.051449 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:10:17.042811 2576 flags.go:64] FLAG: --topology-manager-scope="container" Jul 30 15:10:17.051449 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:10:17.042814 2576 flags.go:64] FLAG: --v="2" Jul 30 15:10:17.051449 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:10:17.042818 2576 flags.go:64] FLAG: --version="false" Jul 30 15:10:17.052109 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:10:17.042822 2576 flags.go:64] FLAG: --vmodule="" Jul 30 15:10:17.052109 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:10:17.042826 2576 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec" Jul 30 15:10:17.052109 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:10:17.042829 2576 flags.go:64] FLAG: --volume-stats-agg-period="1m0s" Jul 30 15:10:17.052109 ip-10-0-133-210 kubenswrapper[2576]: W0730 15:10:17.042956 2576 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 30 15:10:17.052109 ip-10-0-133-210 kubenswrapper[2576]: W0730 15:10:17.042960 2576 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 30 15:10:17.052109 ip-10-0-133-210 kubenswrapper[2576]: W0730 15:10:17.042963 2576 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 30 15:10:17.052109 ip-10-0-133-210 kubenswrapper[2576]: W0730 15:10:17.042966 2576 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 30 15:10:17.052109 ip-10-0-133-210 kubenswrapper[2576]: W0730 15:10:17.042969 2576 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 30 15:10:17.052109 ip-10-0-133-210 kubenswrapper[2576]: W0730 15:10:17.042972 2576 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 30 15:10:17.052109 ip-10-0-133-210 kubenswrapper[2576]: W0730 15:10:17.042975 2576 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 30 15:10:17.052109 ip-10-0-133-210 kubenswrapper[2576]: W0730 15:10:17.042977 2576 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 30 15:10:17.052109 ip-10-0-133-210 kubenswrapper[2576]: W0730 15:10:17.042980 2576 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 30 15:10:17.052109 ip-10-0-133-210 kubenswrapper[2576]: W0730 15:10:17.042984 2576 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 30 15:10:17.052109 ip-10-0-133-210 kubenswrapper[2576]: W0730 15:10:17.042987 2576 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 30 15:10:17.052109 ip-10-0-133-210 kubenswrapper[2576]: W0730 15:10:17.042990 2576 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 30 15:10:17.052109 ip-10-0-133-210 kubenswrapper[2576]: W0730 15:10:17.042993 2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 30 15:10:17.052109 ip-10-0-133-210 kubenswrapper[2576]: W0730 15:10:17.042996 2576 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 30 15:10:17.052109 ip-10-0-133-210 kubenswrapper[2576]: W0730 15:10:17.042999 2576 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 30 15:10:17.052109 ip-10-0-133-210 kubenswrapper[2576]: W0730 15:10:17.043001 2576 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 30 15:10:17.052109 ip-10-0-133-210 kubenswrapper[2576]: W0730 15:10:17.043004 2576 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 30 15:10:17.052109 ip-10-0-133-210 kubenswrapper[2576]: W0730 15:10:17.043007 2576 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 30 15:10:17.052622 ip-10-0-133-210 kubenswrapper[2576]: W0730 15:10:17.043011 2576 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 30 15:10:17.052622 ip-10-0-133-210 kubenswrapper[2576]: W0730 15:10:17.043013 2576 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 30 15:10:17.052622 ip-10-0-133-210 kubenswrapper[2576]: W0730 15:10:17.043016 2576 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 30 15:10:17.052622 ip-10-0-133-210 kubenswrapper[2576]: W0730 15:10:17.043018 2576 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 30 15:10:17.052622 ip-10-0-133-210 kubenswrapper[2576]: W0730 15:10:17.043022 2576 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 30 15:10:17.052622 ip-10-0-133-210 kubenswrapper[2576]: W0730 15:10:17.043025 2576 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 30 15:10:17.052622 ip-10-0-133-210 kubenswrapper[2576]: W0730 15:10:17.043028 2576 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 30 15:10:17.052622 ip-10-0-133-210 kubenswrapper[2576]: W0730 15:10:17.043030 2576 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 30 15:10:17.052622 ip-10-0-133-210 kubenswrapper[2576]: W0730 15:10:17.043033 2576 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 30 15:10:17.052622 ip-10-0-133-210 kubenswrapper[2576]: W0730 15:10:17.043036 2576 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 30 15:10:17.052622 ip-10-0-133-210 kubenswrapper[2576]: W0730 15:10:17.043038 2576 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 30 15:10:17.052622 ip-10-0-133-210 kubenswrapper[2576]: W0730 15:10:17.043041 2576 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 30 15:10:17.052622 ip-10-0-133-210 kubenswrapper[2576]: W0730 15:10:17.043044 2576 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 30 15:10:17.052622 ip-10-0-133-210 kubenswrapper[2576]: W0730 15:10:17.043046 2576 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 30 15:10:17.052622 ip-10-0-133-210 kubenswrapper[2576]: W0730 15:10:17.043049 2576 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 30 15:10:17.052622 ip-10-0-133-210 kubenswrapper[2576]: W0730 15:10:17.043051 2576 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 30 15:10:17.052622 ip-10-0-133-210 kubenswrapper[2576]: W0730 15:10:17.043054 2576 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 30 15:10:17.052622 ip-10-0-133-210 kubenswrapper[2576]: W0730 15:10:17.043057 2576 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 30 15:10:17.052622 ip-10-0-133-210 kubenswrapper[2576]: W0730 15:10:17.043059 2576 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 30 15:10:17.052622 ip-10-0-133-210 kubenswrapper[2576]: W0730 15:10:17.043062 2576 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 30 15:10:17.053141 ip-10-0-133-210 kubenswrapper[2576]: W0730 15:10:17.043065 2576 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 30 15:10:17.053141 ip-10-0-133-210 kubenswrapper[2576]: W0730 15:10:17.043075 2576 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 30 15:10:17.053141 ip-10-0-133-210 kubenswrapper[2576]: W0730 15:10:17.043078 2576 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 30 15:10:17.053141 ip-10-0-133-210 kubenswrapper[2576]: W0730 15:10:17.043082 2576 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 30 15:10:17.053141 ip-10-0-133-210 kubenswrapper[2576]: W0730 15:10:17.043085 2576 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 30 15:10:17.053141 ip-10-0-133-210 kubenswrapper[2576]: W0730 15:10:17.043088 2576 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 30 15:10:17.053141 ip-10-0-133-210 kubenswrapper[2576]: W0730 15:10:17.043093 2576 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 30 15:10:17.053141 ip-10-0-133-210 kubenswrapper[2576]: W0730 15:10:17.043095 2576 feature_gate.go:328] unrecognized feature gate: Example Jul 30 15:10:17.053141 ip-10-0-133-210 kubenswrapper[2576]: W0730 15:10:17.043098 2576 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 30 15:10:17.053141 ip-10-0-133-210 kubenswrapper[2576]: W0730 15:10:17.043101 2576 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 30 15:10:17.053141 ip-10-0-133-210 kubenswrapper[2576]: W0730 15:10:17.043103 2576 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 30 15:10:17.053141 ip-10-0-133-210 kubenswrapper[2576]: W0730 15:10:17.043106 2576 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 30 15:10:17.053141 ip-10-0-133-210 kubenswrapper[2576]: W0730 15:10:17.043108 2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 30 15:10:17.053141 ip-10-0-133-210 kubenswrapper[2576]: W0730 15:10:17.043112 2576 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 30 15:10:17.053141 ip-10-0-133-210 kubenswrapper[2576]: W0730 15:10:17.043114 2576 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 30 15:10:17.053141 ip-10-0-133-210 kubenswrapper[2576]: W0730 15:10:17.043117 2576 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 30 15:10:17.053141 ip-10-0-133-210 kubenswrapper[2576]: W0730 15:10:17.043120 2576 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 30 15:10:17.053141 ip-10-0-133-210 kubenswrapper[2576]: W0730 15:10:17.043122 2576 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 30 15:10:17.053141 ip-10-0-133-210 kubenswrapper[2576]: W0730 15:10:17.043125 2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 30 15:10:17.053141 ip-10-0-133-210 kubenswrapper[2576]: W0730 15:10:17.043127 2576 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 30 15:10:17.053654 ip-10-0-133-210 kubenswrapper[2576]: W0730 15:10:17.043130 2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 30 15:10:17.053654 ip-10-0-133-210 kubenswrapper[2576]: W0730 15:10:17.043132 2576 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 30 15:10:17.053654 ip-10-0-133-210 kubenswrapper[2576]: W0730 15:10:17.043136 2576 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 30 15:10:17.053654 ip-10-0-133-210 kubenswrapper[2576]: W0730 15:10:17.043140 2576 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 30 15:10:17.053654 ip-10-0-133-210 kubenswrapper[2576]: W0730 15:10:17.043143 2576 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 30 15:10:17.053654 ip-10-0-133-210 kubenswrapper[2576]: W0730 15:10:17.043146 2576 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 30 15:10:17.053654 ip-10-0-133-210 kubenswrapper[2576]: W0730 15:10:17.043149 2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 30 15:10:17.053654 ip-10-0-133-210 kubenswrapper[2576]: W0730 15:10:17.043152 2576 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 30 15:10:17.053654 ip-10-0-133-210 kubenswrapper[2576]: W0730 15:10:17.043155 2576 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 30 15:10:17.053654 ip-10-0-133-210 kubenswrapper[2576]: W0730 15:10:17.043157 2576 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 30 15:10:17.053654 ip-10-0-133-210 kubenswrapper[2576]: W0730 15:10:17.043160 2576 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 30 15:10:17.053654 ip-10-0-133-210 kubenswrapper[2576]: W0730 15:10:17.043163 2576 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 30 15:10:17.053654 ip-10-0-133-210 kubenswrapper[2576]: W0730 15:10:17.043165 2576 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 30 15:10:17.053654 ip-10-0-133-210 kubenswrapper[2576]: W0730 15:10:17.043168 2576 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 30 15:10:17.053654 ip-10-0-133-210 kubenswrapper[2576]: W0730 15:10:17.043170 2576 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 30 15:10:17.053654 ip-10-0-133-210 kubenswrapper[2576]: W0730 15:10:17.043174 2576 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 30 15:10:17.053654 ip-10-0-133-210 kubenswrapper[2576]: W0730 15:10:17.043177 2576 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 30 15:10:17.053654 ip-10-0-133-210 kubenswrapper[2576]: W0730 15:10:17.043180 2576 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 30 15:10:17.053654 ip-10-0-133-210 kubenswrapper[2576]: W0730 15:10:17.043183 2576 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 30 15:10:17.054147 ip-10-0-133-210 kubenswrapper[2576]: W0730 15:10:17.043186 2576 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 30 15:10:17.054147 ip-10-0-133-210 kubenswrapper[2576]: W0730 15:10:17.043189 2576 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 30 15:10:17.054147 ip-10-0-133-210 kubenswrapper[2576]: W0730 15:10:17.043191 2576 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 30 15:10:17.054147 ip-10-0-133-210 kubenswrapper[2576]: W0730 15:10:17.043194 2576 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 30 15:10:17.054147 ip-10-0-133-210 kubenswrapper[2576]: W0730 15:10:17.043197 2576 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 30 15:10:17.054147 ip-10-0-133-210 kubenswrapper[2576]: W0730 15:10:17.043200 2576 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 30 15:10:17.054147 ip-10-0-133-210 kubenswrapper[2576]: W0730 15:10:17.043202 2576 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 30 15:10:17.054147 ip-10-0-133-210 kubenswrapper[2576]: W0730 15:10:17.043206 2576 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 30 15:10:17.054147 ip-10-0-133-210 kubenswrapper[2576]: W0730 15:10:17.043210 2576 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 30 15:10:17.054147 ip-10-0-133-210 kubenswrapper[2576]: W0730 15:10:17.043213 2576 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 30 15:10:17.054147 ip-10-0-133-210 kubenswrapper[2576]: W0730 15:10:17.043216 2576 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 30 15:10:17.054147 ip-10-0-133-210 kubenswrapper[2576]: W0730 15:10:17.043218 2576 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 30 15:10:17.054147 ip-10-0-133-210 kubenswrapper[2576]: W0730 15:10:17.043221 2576 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 30 15:10:17.054147 ip-10-0-133-210 kubenswrapper[2576]: W0730 15:10:17.043224 2576 feature_gate.go:328] unrecognized feature gate: Example2 Jul 30 15:10:17.054147 ip-10-0-133-210 kubenswrapper[2576]: W0730 15:10:17.043228 2576 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 30 15:10:17.054147 ip-10-0-133-210 kubenswrapper[2576]: W0730 15:10:17.043231 2576 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 30 15:10:17.054147 ip-10-0-133-210 kubenswrapper[2576]: W0730 15:10:17.043235 2576 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 30 15:10:17.054147 ip-10-0-133-210 kubenswrapper[2576]: W0730 15:10:17.043237 2576 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 30 15:10:17.054584 ip-10-0-133-210 kubenswrapper[2576]: W0730 15:10:17.043240 2576 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 30 15:10:17.054584 ip-10-0-133-210 kubenswrapper[2576]: W0730 15:10:17.043243 2576 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 30 15:10:17.054584 ip-10-0-133-210 kubenswrapper[2576]: W0730 15:10:17.043245 2576 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 30 15:10:17.054584 ip-10-0-133-210 kubenswrapper[2576]: W0730 15:10:17.043248 2576 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 30 15:10:17.054584 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:10:17.043254 2576 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 30 15:10:17.054584 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:10:17.051571 2576 server.go:529] "Kubelet version" kubeletVersion="v1.34.9" Jul 30 15:10:17.054584 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:10:17.051587 2576 server.go:531] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK="" Jul 30 15:10:17.054584 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:10:17.052400 2576 watchdog_linux.go:95] "Systemd watchdog is not enabled" Jul 30 15:10:17.054584 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:10:17.052409 2576 watchdog_linux.go:137] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started." Jul 30 15:10:17.054584 ip-10-0-133-210 kubenswrapper[2576]: W0730 15:10:17.052470 2576 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 30 15:10:17.054584 ip-10-0-133-210 kubenswrapper[2576]: W0730 15:10:17.052475 2576 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 30 15:10:17.054584 ip-10-0-133-210 kubenswrapper[2576]: W0730 15:10:17.052480 2576 feature_gate.go:328] unrecognized feature gate: Example Jul 30 15:10:17.054584 ip-10-0-133-210 kubenswrapper[2576]: W0730 15:10:17.052483 2576 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 30 15:10:17.054584 ip-10-0-133-210 kubenswrapper[2576]: W0730 15:10:17.052487 2576 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 30 15:10:17.054584 ip-10-0-133-210 kubenswrapper[2576]: W0730 15:10:17.052490 2576 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 30 15:10:17.054969 ip-10-0-133-210 kubenswrapper[2576]: W0730 15:10:17.052494 2576 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 30 15:10:17.054969 ip-10-0-133-210 kubenswrapper[2576]: W0730 15:10:17.052499 2576 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 30 15:10:17.054969 ip-10-0-133-210 kubenswrapper[2576]: W0730 15:10:17.052502 2576 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 30 15:10:17.054969 ip-10-0-133-210 kubenswrapper[2576]: W0730 15:10:17.052505 2576 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 30 15:10:17.054969 ip-10-0-133-210 kubenswrapper[2576]: W0730 15:10:17.052508 2576 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 30 15:10:17.054969 ip-10-0-133-210 kubenswrapper[2576]: W0730 15:10:17.052511 2576 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 30 15:10:17.054969 ip-10-0-133-210 kubenswrapper[2576]: W0730 15:10:17.052514 2576 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 30 15:10:17.054969 ip-10-0-133-210 kubenswrapper[2576]: W0730 15:10:17.052518 2576 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 30 15:10:17.054969 ip-10-0-133-210 kubenswrapper[2576]: W0730 15:10:17.052520 2576 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 30 15:10:17.054969 ip-10-0-133-210 kubenswrapper[2576]: W0730 15:10:17.052523 2576 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 30 15:10:17.054969 ip-10-0-133-210 kubenswrapper[2576]: W0730 15:10:17.052526 2576 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 30 15:10:17.054969 ip-10-0-133-210 kubenswrapper[2576]: W0730 15:10:17.052528 2576 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 30 15:10:17.054969 ip-10-0-133-210 kubenswrapper[2576]: W0730 15:10:17.052531 2576 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 30 15:10:17.054969 ip-10-0-133-210 kubenswrapper[2576]: W0730 15:10:17.052534 2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 30 15:10:17.054969 ip-10-0-133-210 kubenswrapper[2576]: W0730 15:10:17.052539 2576 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 30 15:10:17.054969 ip-10-0-133-210 kubenswrapper[2576]: W0730 15:10:17.052542 2576 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 30 15:10:17.054969 ip-10-0-133-210 kubenswrapper[2576]: W0730 15:10:17.052545 2576 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 30 15:10:17.054969 ip-10-0-133-210 kubenswrapper[2576]: W0730 15:10:17.052548 2576 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 30 15:10:17.054969 ip-10-0-133-210 kubenswrapper[2576]: W0730 15:10:17.052550 2576 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 30 15:10:17.055428 ip-10-0-133-210 kubenswrapper[2576]: W0730 15:10:17.052553 2576 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 30 15:10:17.055428 ip-10-0-133-210 kubenswrapper[2576]: W0730 15:10:17.052555 2576 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 30 15:10:17.055428 ip-10-0-133-210 kubenswrapper[2576]: W0730 15:10:17.052558 2576 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 30 15:10:17.055428 ip-10-0-133-210 kubenswrapper[2576]: W0730 15:10:17.052560 2576 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 30 15:10:17.055428 ip-10-0-133-210 kubenswrapper[2576]: W0730 15:10:17.052563 2576 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 30 15:10:17.055428 ip-10-0-133-210 kubenswrapper[2576]: W0730 15:10:17.052566 2576 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 30 15:10:17.055428 ip-10-0-133-210 kubenswrapper[2576]: W0730 15:10:17.052568 2576 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 30 15:10:17.055428 ip-10-0-133-210 kubenswrapper[2576]: W0730 15:10:17.052571 2576 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 30 15:10:17.055428 ip-10-0-133-210 kubenswrapper[2576]: W0730 15:10:17.052573 2576 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 30 15:10:17.055428 ip-10-0-133-210 kubenswrapper[2576]: W0730 15:10:17.052576 2576 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 30 15:10:17.055428 ip-10-0-133-210 kubenswrapper[2576]: W0730 15:10:17.052579 2576 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 30 15:10:17.055428 ip-10-0-133-210 kubenswrapper[2576]: W0730 15:10:17.052581 2576 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 30 15:10:17.055428 ip-10-0-133-210 kubenswrapper[2576]: W0730 15:10:17.052584 2576 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 30 15:10:17.055428 ip-10-0-133-210 kubenswrapper[2576]: W0730 15:10:17.052587 2576 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 30 15:10:17.055428 ip-10-0-133-210 kubenswrapper[2576]: W0730 15:10:17.052589 2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 30 15:10:17.055428 ip-10-0-133-210 kubenswrapper[2576]: W0730 15:10:17.052592 2576 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 30 15:10:17.055428 ip-10-0-133-210 kubenswrapper[2576]: W0730 15:10:17.052594 2576 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 30 15:10:17.055428 ip-10-0-133-210 kubenswrapper[2576]: W0730 15:10:17.052597 2576 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 30 15:10:17.055428 ip-10-0-133-210 kubenswrapper[2576]: W0730 15:10:17.052600 2576 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 30 15:10:17.055428 ip-10-0-133-210 kubenswrapper[2576]: W0730 15:10:17.052602 2576 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 30 15:10:17.055993 ip-10-0-133-210 kubenswrapper[2576]: W0730 15:10:17.052605 2576 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 30 15:10:17.055993 ip-10-0-133-210 kubenswrapper[2576]: W0730 15:10:17.052611 2576 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 30 15:10:17.055993 ip-10-0-133-210 kubenswrapper[2576]: W0730 15:10:17.052614 2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 30 15:10:17.055993 ip-10-0-133-210 kubenswrapper[2576]: W0730 15:10:17.052616 2576 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 30 15:10:17.055993 ip-10-0-133-210 kubenswrapper[2576]: W0730 15:10:17.052619 2576 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 30 15:10:17.055993 ip-10-0-133-210 kubenswrapper[2576]: W0730 15:10:17.052621 2576 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 30 15:10:17.055993 ip-10-0-133-210 kubenswrapper[2576]: W0730 15:10:17.052624 2576 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 30 15:10:17.055993 ip-10-0-133-210 kubenswrapper[2576]: W0730 15:10:17.052627 2576 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 30 15:10:17.055993 ip-10-0-133-210 kubenswrapper[2576]: W0730 15:10:17.052630 2576 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 30 15:10:17.055993 ip-10-0-133-210 kubenswrapper[2576]: W0730 15:10:17.052632 2576 feature_gate.go:328] unrecognized feature gate: Example2 Jul 30 15:10:17.055993 ip-10-0-133-210 kubenswrapper[2576]: W0730 15:10:17.052635 2576 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 30 15:10:17.055993 ip-10-0-133-210 kubenswrapper[2576]: W0730 15:10:17.052638 2576 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 30 15:10:17.055993 ip-10-0-133-210 kubenswrapper[2576]: W0730 15:10:17.052640 2576 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 30 15:10:17.055993 ip-10-0-133-210 kubenswrapper[2576]: W0730 15:10:17.052643 2576 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 30 15:10:17.055993 ip-10-0-133-210 kubenswrapper[2576]: W0730 15:10:17.052645 2576 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 30 15:10:17.055993 ip-10-0-133-210 kubenswrapper[2576]: W0730 15:10:17.052648 2576 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 30 15:10:17.055993 ip-10-0-133-210 kubenswrapper[2576]: W0730 15:10:17.052651 2576 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 30 15:10:17.055993 ip-10-0-133-210 kubenswrapper[2576]: W0730 15:10:17.052653 2576 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 30 15:10:17.055993 ip-10-0-133-210 kubenswrapper[2576]: W0730 15:10:17.052656 2576 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 30 15:10:17.055993 ip-10-0-133-210 kubenswrapper[2576]: W0730 15:10:17.052658 2576 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 30 15:10:17.056535 ip-10-0-133-210 kubenswrapper[2576]: W0730 15:10:17.052661 2576 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 30 15:10:17.056535 ip-10-0-133-210 kubenswrapper[2576]: W0730 15:10:17.052663 2576 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 30 15:10:17.056535 ip-10-0-133-210 kubenswrapper[2576]: W0730 15:10:17.052666 2576 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 30 15:10:17.056535 ip-10-0-133-210 kubenswrapper[2576]: W0730 15:10:17.052669 2576 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 30 15:10:17.056535 ip-10-0-133-210 kubenswrapper[2576]: W0730 15:10:17.052671 2576 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 30 15:10:17.056535 ip-10-0-133-210 kubenswrapper[2576]: W0730 15:10:17.052675 2576 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 30 15:10:17.056535 ip-10-0-133-210 kubenswrapper[2576]: W0730 15:10:17.052677 2576 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 30 15:10:17.056535 ip-10-0-133-210 kubenswrapper[2576]: W0730 15:10:17.052681 2576 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 30 15:10:17.056535 ip-10-0-133-210 kubenswrapper[2576]: W0730 15:10:17.052685 2576 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 30 15:10:17.056535 ip-10-0-133-210 kubenswrapper[2576]: W0730 15:10:17.052688 2576 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 30 15:10:17.056535 ip-10-0-133-210 kubenswrapper[2576]: W0730 15:10:17.052690 2576 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 30 15:10:17.056535 ip-10-0-133-210 kubenswrapper[2576]: W0730 15:10:17.052693 2576 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 30 15:10:17.056535 ip-10-0-133-210 kubenswrapper[2576]: W0730 15:10:17.052695 2576 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 30 15:10:17.056535 ip-10-0-133-210 kubenswrapper[2576]: W0730 15:10:17.052698 2576 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 30 15:10:17.056535 ip-10-0-133-210 kubenswrapper[2576]: W0730 15:10:17.052701 2576 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 30 15:10:17.056535 ip-10-0-133-210 kubenswrapper[2576]: W0730 15:10:17.052704 2576 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 30 15:10:17.056535 ip-10-0-133-210 kubenswrapper[2576]: W0730 15:10:17.052706 2576 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 30 15:10:17.056535 ip-10-0-133-210 kubenswrapper[2576]: W0730 15:10:17.052708 2576 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 30 15:10:17.056535 ip-10-0-133-210 kubenswrapper[2576]: W0730 15:10:17.052711 2576 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 30 15:10:17.057018 ip-10-0-133-210 kubenswrapper[2576]: W0730 15:10:17.052713 2576 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 30 15:10:17.057018 ip-10-0-133-210 kubenswrapper[2576]: W0730 15:10:17.052716 2576 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 30 15:10:17.057018 ip-10-0-133-210 kubenswrapper[2576]: W0730 15:10:17.052719 2576 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 30 15:10:17.057018 ip-10-0-133-210 kubenswrapper[2576]: W0730 15:10:17.052721 2576 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 30 15:10:17.057018 ip-10-0-133-210 kubenswrapper[2576]: W0730 15:10:17.052723 2576 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 30 15:10:17.057018 ip-10-0-133-210 kubenswrapper[2576]: W0730 15:10:17.052726 2576 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 30 15:10:17.057018 ip-10-0-133-210 kubenswrapper[2576]: W0730 15:10:17.052729 2576 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 30 15:10:17.057018 ip-10-0-133-210 kubenswrapper[2576]: W0730 15:10:17.052731 2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 30 15:10:17.057018 ip-10-0-133-210 kubenswrapper[2576]: W0730 15:10:17.052734 2576 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 30 15:10:17.057018 ip-10-0-133-210 kubenswrapper[2576]: W0730 15:10:17.052736 2576 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 30 15:10:17.057018 ip-10-0-133-210 kubenswrapper[2576]: W0730 15:10:17.052739 2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 30 15:10:17.057018 ip-10-0-133-210 kubenswrapper[2576]: W0730 15:10:17.052741 2576 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 30 15:10:17.057018 ip-10-0-133-210 kubenswrapper[2576]: W0730 15:10:17.052744 2576 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 30 15:10:17.057018 ip-10-0-133-210 kubenswrapper[2576]: W0730 15:10:17.052747 2576 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 30 15:10:17.057018 ip-10-0-133-210 kubenswrapper[2576]: W0730 15:10:17.052750 2576 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 30 15:10:17.057384 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:10:17.052756 2576 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 30 15:10:17.057384 ip-10-0-133-210 kubenswrapper[2576]: W0730 15:10:17.052867 2576 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 30 15:10:17.057384 ip-10-0-133-210 kubenswrapper[2576]: W0730 15:10:17.052873 2576 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 30 15:10:17.057384 ip-10-0-133-210 kubenswrapper[2576]: W0730 15:10:17.052876 2576 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 30 15:10:17.057384 ip-10-0-133-210 kubenswrapper[2576]: W0730 15:10:17.052878 2576 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 30 15:10:17.057384 ip-10-0-133-210 kubenswrapper[2576]: W0730 15:10:17.052881 2576 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 30 15:10:17.057384 ip-10-0-133-210 kubenswrapper[2576]: W0730 15:10:17.052884 2576 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 30 15:10:17.057384 ip-10-0-133-210 kubenswrapper[2576]: W0730 15:10:17.052887 2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 30 15:10:17.057384 ip-10-0-133-210 kubenswrapper[2576]: W0730 15:10:17.052891 2576 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 30 15:10:17.057384 ip-10-0-133-210 kubenswrapper[2576]: W0730 15:10:17.052895 2576 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 30 15:10:17.057384 ip-10-0-133-210 kubenswrapper[2576]: W0730 15:10:17.052898 2576 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 30 15:10:17.057384 ip-10-0-133-210 kubenswrapper[2576]: W0730 15:10:17.052901 2576 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 30 15:10:17.057384 ip-10-0-133-210 kubenswrapper[2576]: W0730 15:10:17.052918 2576 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 30 15:10:17.057384 ip-10-0-133-210 kubenswrapper[2576]: W0730 15:10:17.052923 2576 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 30 15:10:17.057729 ip-10-0-133-210 kubenswrapper[2576]: W0730 15:10:17.052927 2576 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 30 15:10:17.057729 ip-10-0-133-210 kubenswrapper[2576]: W0730 15:10:17.052929 2576 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 30 15:10:17.057729 ip-10-0-133-210 kubenswrapper[2576]: W0730 15:10:17.052932 2576 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 30 15:10:17.057729 ip-10-0-133-210 kubenswrapper[2576]: W0730 15:10:17.052935 2576 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 30 15:10:17.057729 ip-10-0-133-210 kubenswrapper[2576]: W0730 15:10:17.052937 2576 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 30 15:10:17.057729 ip-10-0-133-210 kubenswrapper[2576]: W0730 15:10:17.052940 2576 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 30 15:10:17.057729 ip-10-0-133-210 kubenswrapper[2576]: W0730 15:10:17.052942 2576 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 30 15:10:17.057729 ip-10-0-133-210 kubenswrapper[2576]: W0730 15:10:17.052945 2576 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 30 15:10:17.057729 ip-10-0-133-210 kubenswrapper[2576]: W0730 15:10:17.052947 2576 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 30 15:10:17.057729 ip-10-0-133-210 kubenswrapper[2576]: W0730 15:10:17.052950 2576 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 30 15:10:17.057729 ip-10-0-133-210 kubenswrapper[2576]: W0730 15:10:17.052953 2576 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 30 15:10:17.057729 ip-10-0-133-210 kubenswrapper[2576]: W0730 15:10:17.052955 2576 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 30 15:10:17.057729 ip-10-0-133-210 kubenswrapper[2576]: W0730 15:10:17.052958 2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 30 15:10:17.057729 ip-10-0-133-210 kubenswrapper[2576]: W0730 15:10:17.052960 2576 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 30 15:10:17.057729 ip-10-0-133-210 kubenswrapper[2576]: W0730 15:10:17.052963 2576 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 30 15:10:17.057729 ip-10-0-133-210 kubenswrapper[2576]: W0730 15:10:17.052965 2576 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 30 15:10:17.057729 ip-10-0-133-210 kubenswrapper[2576]: W0730 15:10:17.052968 2576 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 30 15:10:17.057729 ip-10-0-133-210 kubenswrapper[2576]: W0730 15:10:17.052970 2576 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 30 15:10:17.057729 ip-10-0-133-210 kubenswrapper[2576]: W0730 15:10:17.052974 2576 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 30 15:10:17.057729 ip-10-0-133-210 kubenswrapper[2576]: W0730 15:10:17.052977 2576 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 30 15:10:17.058241 ip-10-0-133-210 kubenswrapper[2576]: W0730 15:10:17.052980 2576 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 30 15:10:17.058241 ip-10-0-133-210 kubenswrapper[2576]: W0730 15:10:17.052982 2576 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 30 15:10:17.058241 ip-10-0-133-210 kubenswrapper[2576]: W0730 15:10:17.052985 2576 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 30 15:10:17.058241 ip-10-0-133-210 kubenswrapper[2576]: W0730 15:10:17.052988 2576 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 30 15:10:17.058241 ip-10-0-133-210 kubenswrapper[2576]: W0730 15:10:17.052990 2576 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 30 15:10:17.058241 ip-10-0-133-210 kubenswrapper[2576]: W0730 15:10:17.052993 2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 30 15:10:17.058241 ip-10-0-133-210 kubenswrapper[2576]: W0730 15:10:17.052995 2576 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 30 15:10:17.058241 ip-10-0-133-210 kubenswrapper[2576]: W0730 15:10:17.052998 2576 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 30 15:10:17.058241 ip-10-0-133-210 kubenswrapper[2576]: W0730 15:10:17.053000 2576 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 30 15:10:17.058241 ip-10-0-133-210 kubenswrapper[2576]: W0730 15:10:17.053003 2576 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 30 15:10:17.058241 ip-10-0-133-210 kubenswrapper[2576]: W0730 15:10:17.053005 2576 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 30 15:10:17.058241 ip-10-0-133-210 kubenswrapper[2576]: W0730 15:10:17.053008 2576 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 30 15:10:17.058241 ip-10-0-133-210 kubenswrapper[2576]: W0730 15:10:17.053011 2576 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 30 15:10:17.058241 ip-10-0-133-210 kubenswrapper[2576]: W0730 15:10:17.053014 2576 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 30 15:10:17.058241 ip-10-0-133-210 kubenswrapper[2576]: W0730 15:10:17.053016 2576 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 30 15:10:17.058241 ip-10-0-133-210 kubenswrapper[2576]: W0730 15:10:17.053019 2576 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 30 15:10:17.058241 ip-10-0-133-210 kubenswrapper[2576]: W0730 15:10:17.053021 2576 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 30 15:10:17.058241 ip-10-0-133-210 kubenswrapper[2576]: W0730 15:10:17.053024 2576 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 30 15:10:17.058241 ip-10-0-133-210 kubenswrapper[2576]: W0730 15:10:17.053026 2576 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 30 15:10:17.058241 ip-10-0-133-210 kubenswrapper[2576]: W0730 15:10:17.053029 2576 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 30 15:10:17.058733 ip-10-0-133-210 kubenswrapper[2576]: W0730 15:10:17.053031 2576 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 30 15:10:17.058733 ip-10-0-133-210 kubenswrapper[2576]: W0730 15:10:17.053034 2576 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 30 15:10:17.058733 ip-10-0-133-210 kubenswrapper[2576]: W0730 15:10:17.053036 2576 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 30 15:10:17.058733 ip-10-0-133-210 kubenswrapper[2576]: W0730 15:10:17.053039 2576 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 30 15:10:17.058733 ip-10-0-133-210 kubenswrapper[2576]: W0730 15:10:17.053041 2576 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 30 15:10:17.058733 ip-10-0-133-210 kubenswrapper[2576]: W0730 15:10:17.053044 2576 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 30 15:10:17.058733 ip-10-0-133-210 kubenswrapper[2576]: W0730 15:10:17.053047 2576 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 30 15:10:17.058733 ip-10-0-133-210 kubenswrapper[2576]: W0730 15:10:17.053049 2576 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 30 15:10:17.058733 ip-10-0-133-210 kubenswrapper[2576]: W0730 15:10:17.053052 2576 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 30 15:10:17.058733 ip-10-0-133-210 kubenswrapper[2576]: W0730 15:10:17.053054 2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 30 15:10:17.058733 ip-10-0-133-210 kubenswrapper[2576]: W0730 15:10:17.053058 2576 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 30 15:10:17.058733 ip-10-0-133-210 kubenswrapper[2576]: W0730 15:10:17.053062 2576 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 30 15:10:17.058733 ip-10-0-133-210 kubenswrapper[2576]: W0730 15:10:17.053066 2576 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 30 15:10:17.058733 ip-10-0-133-210 kubenswrapper[2576]: W0730 15:10:17.053069 2576 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 30 15:10:17.058733 ip-10-0-133-210 kubenswrapper[2576]: W0730 15:10:17.053072 2576 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 30 15:10:17.058733 ip-10-0-133-210 kubenswrapper[2576]: W0730 15:10:17.053074 2576 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 30 15:10:17.058733 ip-10-0-133-210 kubenswrapper[2576]: W0730 15:10:17.053077 2576 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 30 15:10:17.058733 ip-10-0-133-210 kubenswrapper[2576]: W0730 15:10:17.053079 2576 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 30 15:10:17.058733 ip-10-0-133-210 kubenswrapper[2576]: W0730 15:10:17.053082 2576 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 30 15:10:17.059203 ip-10-0-133-210 kubenswrapper[2576]: W0730 15:10:17.053084 2576 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 30 15:10:17.059203 ip-10-0-133-210 kubenswrapper[2576]: W0730 15:10:17.053087 2576 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 30 15:10:17.059203 ip-10-0-133-210 kubenswrapper[2576]: W0730 15:10:17.053089 2576 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 30 15:10:17.059203 ip-10-0-133-210 kubenswrapper[2576]: W0730 15:10:17.053092 2576 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 30 15:10:17.059203 ip-10-0-133-210 kubenswrapper[2576]: W0730 15:10:17.053094 2576 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 30 15:10:17.059203 ip-10-0-133-210 kubenswrapper[2576]: W0730 15:10:17.053097 2576 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 30 15:10:17.059203 ip-10-0-133-210 kubenswrapper[2576]: W0730 15:10:17.053100 2576 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 30 15:10:17.059203 ip-10-0-133-210 kubenswrapper[2576]: W0730 15:10:17.053103 2576 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 30 15:10:17.059203 ip-10-0-133-210 kubenswrapper[2576]: W0730 15:10:17.053105 2576 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 30 15:10:17.059203 ip-10-0-133-210 kubenswrapper[2576]: W0730 15:10:17.053108 2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 30 15:10:17.059203 ip-10-0-133-210 kubenswrapper[2576]: W0730 15:10:17.053112 2576 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 30 15:10:17.059203 ip-10-0-133-210 kubenswrapper[2576]: W0730 15:10:17.053115 2576 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 30 15:10:17.059203 ip-10-0-133-210 kubenswrapper[2576]: W0730 15:10:17.053118 2576 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 30 15:10:17.059203 ip-10-0-133-210 kubenswrapper[2576]: W0730 15:10:17.053121 2576 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 30 15:10:17.059203 ip-10-0-133-210 kubenswrapper[2576]: W0730 15:10:17.053123 2576 feature_gate.go:328] unrecognized feature gate: Example2 Jul 30 15:10:17.059203 ip-10-0-133-210 kubenswrapper[2576]: W0730 15:10:17.053126 2576 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 30 15:10:17.059203 ip-10-0-133-210 kubenswrapper[2576]: W0730 15:10:17.053129 2576 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 30 15:10:17.059203 ip-10-0-133-210 kubenswrapper[2576]: W0730 15:10:17.053131 2576 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 30 15:10:17.059203 ip-10-0-133-210 kubenswrapper[2576]: W0730 15:10:17.053134 2576 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 30 15:10:17.059203 ip-10-0-133-210 kubenswrapper[2576]: W0730 15:10:17.053136 2576 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 30 15:10:17.059698 ip-10-0-133-210 kubenswrapper[2576]: W0730 15:10:17.053139 2576 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 30 15:10:17.059698 ip-10-0-133-210 kubenswrapper[2576]: W0730 15:10:17.053142 2576 feature_gate.go:328] unrecognized feature gate: Example Jul 30 15:10:17.059698 ip-10-0-133-210 kubenswrapper[2576]: W0730 15:10:17.053145 2576 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 30 15:10:17.059698 ip-10-0-133-210 kubenswrapper[2576]: W0730 15:10:17.053147 2576 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 30 15:10:17.059698 ip-10-0-133-210 kubenswrapper[2576]: W0730 15:10:17.053150 2576 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 30 15:10:17.059698 ip-10-0-133-210 kubenswrapper[2576]: W0730 15:10:17.053153 2576 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 30 15:10:17.059698 ip-10-0-133-210 kubenswrapper[2576]: W0730 15:10:17.053156 2576 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 30 15:10:17.059698 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:10:17.053162 2576 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 30 15:10:17.059698 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:10:17.053279 2576 server.go:962] "Client rotation is on, will bootstrap in background" Jul 30 15:10:17.059698 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:10:17.056172 2576 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir" Jul 30 15:10:17.059698 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:10:17.057689 2576 server.go:1019] "Starting client certificate rotation" Jul 30 15:10:17.059698 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:10:17.057790 2576 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet" Jul 30 15:10:17.059698 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:10:17.057823 2576 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet" Jul 30 15:10:17.060386 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:10:17.060373 2576 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Jul 30 15:10:17.062347 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:10:17.062334 2576 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Jul 30 15:10:17.074574 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:10:17.074558 2576 log.go:25] "Validated CRI v1 runtime API" Jul 30 15:10:17.077053 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:10:17.077040 2576 log.go:25] "Validated CRI v1 image API" Jul 30 15:10:17.078307 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:10:17.078288 2576 server.go:1429] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd" Jul 30 15:10:17.082768 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:10:17.082739 2576 fs.go:135] Filesystem UUIDs: map[12bd7acc-ba4c-436e-916b-45edce31ba4e:/dev/nvme0n1p4 4e79491c-bc3c-482d-add9-9d7713556703:/dev/nvme0n1p3 7B77-95E7:/dev/nvme0n1p2] Jul 30 15:10:17.082768 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:10:17.082759 2576 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}] Jul 30 15:10:17.084845 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:10:17.084828 2576 reflector.go:439] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Jul 30 15:10:17.088418 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:10:17.088311 2576 manager.go:216] Machine: {Timestamp:2026-07-30 15:10:17.087113383 +0000 UTC m=+0.301067910 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3106539 MemoryCapacity:33164496896 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec265c73c2b3adab39fb6f9c38a12fd0 SystemUUID:ec265c73-c2b3-adab-39fb-6f9c38a12fd0 BootID:9ebb31c6-8185-42e5-b809-3ab1ca895ec8 Filesystems:[{Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16582250496 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true} {Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16582246400 Type:vfs Inodes:4048400 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6632902656 Type:vfs Inodes:819200 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6111232 Type:vfs Inodes:18446744073709551615 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:89:0f:85:2f:a7 Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:89:0f:85:2f:a7 Speed:0 Mtu:9001} {Name:ovs-system MacAddress:de:85:68:8c:c6:df Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:33164496896 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:37486592 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None} Jul 30 15:10:17.088418 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:10:17.088414 2576 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available. Jul 30 15:10:17.088551 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:10:17.088490 2576 manager.go:232] Version: {KernelVersion:5.14.0-570.129.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260717-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:} Jul 30 15:10:17.092043 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:10:17.091828 2576 container_manager_linux.go:273] "Container manager verified user specified cgroup-root exists" cgroupRoot=[] Jul 30 15:10:17.092178 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:10:17.092048 2576 container_manager_linux.go:278] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-133-210.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2} Jul 30 15:10:17.092745 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:10:17.092734 2576 topology_manager.go:138] "Creating topology manager with none policy" Jul 30 15:10:17.092779 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:10:17.092748 2576 container_manager_linux.go:309] "Creating device plugin manager" Jul 30 15:10:17.092779 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:10:17.092756 2576 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock" Jul 30 15:10:17.092779 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:10:17.092770 2576 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock" Jul 30 15:10:17.094349 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:10:17.094340 2576 container_manager_linux.go:318] "Creating Dynamic Resource Allocation (DRA) manager" Jul 30 15:10:17.097270 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:10:17.097259 2576 state_mem.go:36] "Initialized new in-memory state store" Jul 30 15:10:17.097392 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:10:17.097382 2576 server.go:1240] "Using root directory" path="/var/lib/kubelet" Jul 30 15:10:17.097421 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:10:17.097414 2576 kubelet.go:486] "Attempting to sync node with API server" Jul 30 15:10:17.097448 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:10:17.097440 2576 kubelet.go:387] "Adding static pod path" path="/etc/kubernetes/manifests" Jul 30 15:10:17.097478 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:10:17.097450 2576 file.go:69] "Watching path" path="/etc/kubernetes/manifests" Jul 30 15:10:17.097478 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:10:17.097459 2576 kubelet.go:398] "Adding apiserver pod source" Jul 30 15:10:17.097478 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:10:17.097467 2576 apiserver.go:42] "Waiting for node sync before watching apiserver pods" Jul 30 15:10:17.097846 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:10:17.097833 2576 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Jul 30 15:10:17.097890 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:10:17.097862 2576 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Jul 30 15:10:17.099760 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:10:17.099741 2576 kuberuntime_manager.go:301] "Container runtime initialized" containerRuntime="cri-o" version="1.34.10-2.rhaos4.21.git83ee25b.el9" apiVersion="v1" Jul 30 15:10:17.101314 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:10:17.101289 2576 kubelet.go:960] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled" Jul 30 15:10:17.101417 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:10:17.101328 2576 kubelet.go:984] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled" Jul 30 15:10:17.102569 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:10:17.102556 2576 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume" Jul 30 15:10:17.102644 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:10:17.102577 2576 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir" Jul 30 15:10:17.102644 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:10:17.102586 2576 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo" Jul 30 15:10:17.102644 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:10:17.102595 2576 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path" Jul 30 15:10:17.102644 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:10:17.102607 2576 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs" Jul 30 15:10:17.102644 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:10:17.102616 2576 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret" Jul 30 15:10:17.102644 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:10:17.102627 2576 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi" Jul 30 15:10:17.102644 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:10:17.102635 2576 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api" Jul 30 15:10:17.102644 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:10:17.102645 2576 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc" Jul 30 15:10:17.102882 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:10:17.102655 2576 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap" Jul 30 15:10:17.102882 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:10:17.102668 2576 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected" Jul 30 15:10:17.102882 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:10:17.102677 2576 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume" Jul 30 15:10:17.103366 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:10:17.103354 2576 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi" Jul 30 15:10:17.103421 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:10:17.103367 2576 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image" Jul 30 15:10:17.106309 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:10:17.106290 2576 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-133-210.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope Jul 30 15:10:17.106508 ip-10-0-133-210 kubenswrapper[2576]: E0730 15:10:17.106487 2576 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service" Jul 30 15:10:17.106572 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:10:17.106506 2576 server.go:1268] "Started kubelet" Jul 30 15:10:17.106618 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:10:17.106588 2576 server.go:180] "Starting to listen" address="0.0.0.0" port=10250 Jul 30 15:10:17.106660 ip-10-0-133-210 kubenswrapper[2576]: E0730 15:10:17.106642 2576 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-133-210.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node" Jul 30 15:10:17.107376 ip-10-0-133-210 systemd[1]: Started Kubernetes Kubelet. Jul 30 15:10:17.107544 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:10:17.107369 2576 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10 Jul 30 15:10:17.107544 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:10:17.107389 2576 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-l72vm" Jul 30 15:10:17.107544 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:10:17.107413 2576 server_v1.go:49] "podresources" method="list" useActivePods=true Jul 30 15:10:17.108960 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:10:17.108940 2576 server.go:310] "Adding debug handlers to kubelet server" Jul 30 15:10:17.109477 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:10:17.109459 2576 server.go:249] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock" Jul 30 15:10:17.113205 ip-10-0-133-210 kubenswrapper[2576]: E0730 15:10:17.113173 2576 kubelet.go:1638] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache" Jul 30 15:10:17.113792 ip-10-0-133-210 kubenswrapper[2576]: E0730 15:10:17.112942 2576 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-133-210.ec2.internal.18c719ddaa02fe4d default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-133-210.ec2.internal,UID:ip-10-0-133-210.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-133-210.ec2.internal,},FirstTimestamp:2026-07-30 15:10:17.106480717 +0000 UTC m=+0.320435244,LastTimestamp:2026-07-30 15:10:17.106480717 +0000 UTC m=+0.320435244,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-133-210.ec2.internal,}" Jul 30 15:10:17.115375 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:10:17.115361 2576 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer" Jul 30 15:10:17.115429 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:10:17.115361 2576 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving" Jul 30 15:10:17.115491 ip-10-0-133-210 kubenswrapper[2576]: E0730 15:10:17.115481 2576 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-133-210.ec2.internal\" not found" Jul 30 15:10:17.116161 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:10:17.116143 2576 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-l72vm" Jul 30 15:10:17.116251 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:10:17.116216 2576 desired_state_of_world_populator.go:146] "Desired state populator starts to run" Jul 30 15:10:17.116251 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:10:17.116218 2576 volume_manager.go:311] "The desired_state_of_world populator starts" Jul 30 15:10:17.116251 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:10:17.116236 2576 volume_manager.go:313] "Starting Kubelet Volume Manager" Jul 30 15:10:17.116408 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:10:17.116316 2576 reconstruct.go:97] "Volume reconstruction finished" Jul 30 15:10:17.116408 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:10:17.116327 2576 reconciler.go:29] "Reconciler: start to sync state" Jul 30 15:10:17.116408 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:10:17.116329 2576 factory.go:55] Registering systemd factory Jul 30 15:10:17.116408 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:10:17.116343 2576 factory.go:223] Registration of the systemd container factory successfully Jul 30 15:10:17.116617 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:10:17.116516 2576 factory.go:153] Registering CRI-O factory Jul 30 15:10:17.116617 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:10:17.116529 2576 factory.go:223] Registration of the crio container factory successfully Jul 30 15:10:17.116617 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:10:17.116594 2576 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory Jul 30 15:10:17.116617 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:10:17.116616 2576 factory.go:103] Registering Raw factory Jul 30 15:10:17.116844 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:10:17.116630 2576 manager.go:1196] Started watching for new ooms in manager Jul 30 15:10:17.117030 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:10:17.117015 2576 manager.go:349] Starting recovery of all containers Jul 30 15:10:17.118102 ip-10-0-133-210 kubenswrapper[2576]: E0730 15:10:17.118061 2576 reflector.go:205] "Failed to watch" err="failed to list *v1.CSIDriver: csidrivers.storage.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"csidrivers\" in API group \"storage.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver" Jul 30 15:10:17.118393 ip-10-0-133-210 kubenswrapper[2576]: E0730 15:10:17.118175 2576 controller.go:145] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-133-210.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="200ms" Jul 30 15:10:17.129114 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:10:17.129099 2576 manager.go:354] Recovery completed Jul 30 15:10:17.133170 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:10:17.133159 2576 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 30 15:10:17.134864 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:10:17.134849 2576 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-210.ec2.internal" event="NodeHasSufficientMemory" Jul 30 15:10:17.134925 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:10:17.134878 2576 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-210.ec2.internal" event="NodeHasNoDiskPressure" Jul 30 15:10:17.134925 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:10:17.134888 2576 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-210.ec2.internal" event="NodeHasSufficientPID" Jul 30 15:10:17.135430 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:10:17.135417 2576 cpu_manager.go:222] "Starting CPU manager" policy="none" Jul 30 15:10:17.135430 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:10:17.135429 2576 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s" Jul 30 15:10:17.135507 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:10:17.135431 2576 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 30 15:10:17.135507 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:10:17.135443 2576 state_mem.go:36] "Initialized new in-memory state store" Jul 30 15:10:17.136070 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:10:17.136054 2576 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-210.ec2.internal" event="NodeHasSufficientMemory" Jul 30 15:10:17.136143 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:10:17.136078 2576 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-210.ec2.internal" event="NodeHasNoDiskPressure" Jul 30 15:10:17.136143 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:10:17.136089 2576 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-210.ec2.internal" event="NodeHasSufficientPID" Jul 30 15:10:17.136915 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:10:17.136892 2576 policy_none.go:49] "None policy: Start" Jul 30 15:10:17.136968 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:10:17.136919 2576 memory_manager.go:187] "Starting memorymanager" policy="None" Jul 30 15:10:17.136968 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:10:17.136933 2576 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint" Jul 30 15:10:17.137878 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:10:17.137866 2576 policy_none.go:47] "Start" Jul 30 15:10:17.177609 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:10:17.177455 2576 manager.go:337] "Starting Device Plugin manager" Jul 30 15:10:17.177770 ip-10-0-133-210 kubenswrapper[2576]: E0730 15:10:17.177632 2576 manager.go:513] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint" Jul 30 15:10:17.177770 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:10:17.177643 2576 server.go:87] "Starting device plugin registration server" Jul 30 15:10:17.177859 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:10:17.177840 2576 eviction_manager.go:189] "Eviction manager: starting control loop" Jul 30 15:10:17.177901 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:10:17.177849 2576 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s" Jul 30 15:10:17.177994 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:10:17.177973 2576 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry" Jul 30 15:10:17.178083 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:10:17.178065 2576 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts" Jul 30 15:10:17.178083 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:10:17.178079 2576 plugin_manager.go:118] "Starting Kubelet Plugin Manager" Jul 30 15:10:17.178530 ip-10-0-133-210 kubenswrapper[2576]: E0730 15:10:17.178497 2576 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\"" Jul 30 15:10:17.178627 ip-10-0-133-210 kubenswrapper[2576]: E0730 15:10:17.178551 2576 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-133-210.ec2.internal\" not found" Jul 30 15:10:18.566851 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:10:18.566312 2576 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 30 15:10:18.567247 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:10:18.567231 2576 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-210.ec2.internal" event="NodeHasSufficientMemory" Jul 30 15:10:18.567319 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:10:18.567262 2576 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-210.ec2.internal" event="NodeHasNoDiskPressure" Jul 30 15:10:18.567319 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:10:18.567278 2576 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-210.ec2.internal" event="NodeHasSufficientPID" Jul 30 15:10:18.567319 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:10:18.567310 2576 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-133-210.ec2.internal" Jul 30 15:10:18.575896 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:10:18.575843 2576 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-133-210.ec2.internal" Jul 30 15:10:18.575896 ip-10-0-133-210 kubenswrapper[2576]: E0730 15:10:18.575869 2576 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-133-210.ec2.internal\": node \"ip-10-0-133-210.ec2.internal\" not found" Jul 30 15:10:18.588760 ip-10-0-133-210 kubenswrapper[2576]: E0730 15:10:18.588741 2576 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-133-210.ec2.internal\" not found" Jul 30 15:10:18.605060 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:10:18.605031 2576 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4" Jul 30 15:10:18.606223 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:10:18.606207 2576 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6" Jul 30 15:10:18.606287 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:10:18.606241 2576 status_manager.go:244] "Starting to sync pod status with apiserver" Jul 30 15:10:18.606287 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:10:18.606261 2576 kubelet.go:2448] "Starting kubelet main sync loop" Jul 30 15:10:18.606372 ip-10-0-133-210 kubenswrapper[2576]: E0730 15:10:18.606292 2576 kubelet.go:2472] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful" Jul 30 15:10:18.608625 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:10:18.608602 2576 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Jul 30 15:10:18.688914 ip-10-0-133-210 kubenswrapper[2576]: E0730 15:10:18.688877 2576 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-133-210.ec2.internal\" not found" Jul 30 15:10:18.707297 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:10:18.707264 2576 kubelet.go:2535] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-210.ec2.internal","kube-system/kube-apiserver-proxy-ip-10-0-133-210.ec2.internal"] Jul 30 15:10:18.707371 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:10:18.707361 2576 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 30 15:10:18.709266 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:10:18.709249 2576 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-210.ec2.internal" event="NodeHasSufficientMemory" Jul 30 15:10:18.709328 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:10:18.709285 2576 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-210.ec2.internal" event="NodeHasNoDiskPressure" Jul 30 15:10:18.709328 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:10:18.709298 2576 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-210.ec2.internal" event="NodeHasSufficientPID" Jul 30 15:10:18.711573 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:10:18.711560 2576 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 30 15:10:18.711701 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:10:18.711687 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-210.ec2.internal" Jul 30 15:10:18.711743 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:10:18.711733 2576 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 30 15:10:18.712416 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:10:18.712396 2576 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-210.ec2.internal" event="NodeHasSufficientMemory" Jul 30 15:10:18.712515 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:10:18.712424 2576 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-210.ec2.internal" event="NodeHasNoDiskPressure" Jul 30 15:10:18.712515 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:10:18.712434 2576 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-210.ec2.internal" event="NodeHasSufficientPID" Jul 30 15:10:18.712515 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:10:18.712395 2576 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-210.ec2.internal" event="NodeHasSufficientMemory" Jul 30 15:10:18.712515 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:10:18.712488 2576 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-210.ec2.internal" event="NodeHasNoDiskPressure" Jul 30 15:10:18.712515 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:10:18.712498 2576 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-210.ec2.internal" event="NodeHasSufficientPID" Jul 30 15:10:18.714788 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:10:18.714773 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-133-210.ec2.internal" Jul 30 15:10:18.714856 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:10:18.714804 2576 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 30 15:10:18.715473 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:10:18.715450 2576 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-210.ec2.internal" event="NodeHasSufficientMemory" Jul 30 15:10:18.715565 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:10:18.715514 2576 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-210.ec2.internal" event="NodeHasNoDiskPressure" Jul 30 15:10:18.715565 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:10:18.715529 2576 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-210.ec2.internal" event="NodeHasSufficientPID" Jul 30 15:10:18.728660 ip-10-0-133-210 kubenswrapper[2576]: E0730 15:10:18.728640 2576 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-133-210.ec2.internal\" not found" node="ip-10-0-133-210.ec2.internal" Jul 30 15:10:18.732038 ip-10-0-133-210 kubenswrapper[2576]: E0730 15:10:18.732021 2576 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-133-210.ec2.internal\" not found" node="ip-10-0-133-210.ec2.internal" Jul 30 15:10:18.789740 ip-10-0-133-210 kubenswrapper[2576]: E0730 15:10:18.789705 2576 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-133-210.ec2.internal\" not found" Jul 30 15:10:18.807005 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:10:18.806984 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/f80b605a8a24b316000fbccb170e3adc-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-133-210.ec2.internal\" (UID: \"f80b605a8a24b316000fbccb170e3adc\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-210.ec2.internal" Jul 30 15:10:18.807097 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:10:18.807015 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/f80b605a8a24b316000fbccb170e3adc-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-133-210.ec2.internal\" (UID: \"f80b605a8a24b316000fbccb170e3adc\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-210.ec2.internal" Jul 30 15:10:18.807097 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:10:18.807035 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/2da2e1115617abcb95d9273e2e0aad34-config\") pod \"kube-apiserver-proxy-ip-10-0-133-210.ec2.internal\" (UID: \"2da2e1115617abcb95d9273e2e0aad34\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-133-210.ec2.internal" Jul 30 15:10:18.890562 ip-10-0-133-210 kubenswrapper[2576]: E0730 15:10:18.890495 2576 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-133-210.ec2.internal\" not found" Jul 30 15:10:18.907882 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:10:18.907856 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/f80b605a8a24b316000fbccb170e3adc-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-133-210.ec2.internal\" (UID: \"f80b605a8a24b316000fbccb170e3adc\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-210.ec2.internal" Jul 30 15:10:18.907940 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:10:18.907917 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/f80b605a8a24b316000fbccb170e3adc-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-133-210.ec2.internal\" (UID: \"f80b605a8a24b316000fbccb170e3adc\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-210.ec2.internal" Jul 30 15:10:18.907992 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:10:18.907977 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/f80b605a8a24b316000fbccb170e3adc-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-133-210.ec2.internal\" (UID: \"f80b605a8a24b316000fbccb170e3adc\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-210.ec2.internal" Jul 30 15:10:18.908026 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:10:18.908003 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/2da2e1115617abcb95d9273e2e0aad34-config\") pod \"kube-apiserver-proxy-ip-10-0-133-210.ec2.internal\" (UID: \"2da2e1115617abcb95d9273e2e0aad34\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-133-210.ec2.internal" Jul 30 15:10:18.908064 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:10:18.908025 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/2da2e1115617abcb95d9273e2e0aad34-config\") pod \"kube-apiserver-proxy-ip-10-0-133-210.ec2.internal\" (UID: \"2da2e1115617abcb95d9273e2e0aad34\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-133-210.ec2.internal" Jul 30 15:10:18.908101 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:10:18.908059 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/f80b605a8a24b316000fbccb170e3adc-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-133-210.ec2.internal\" (UID: \"f80b605a8a24b316000fbccb170e3adc\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-210.ec2.internal" Jul 30 15:10:18.991058 ip-10-0-133-210 kubenswrapper[2576]: E0730 15:10:18.991028 2576 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-133-210.ec2.internal\" not found" Jul 30 15:10:19.032610 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:10:19.032581 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-210.ec2.internal" Jul 30 15:10:19.035734 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:10:19.035716 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-133-210.ec2.internal" Jul 30 15:10:19.091758 ip-10-0-133-210 kubenswrapper[2576]: E0730 15:10:19.091719 2576 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-133-210.ec2.internal\" not found" Jul 30 15:10:19.192337 ip-10-0-133-210 kubenswrapper[2576]: E0730 15:10:19.192264 2576 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-133-210.ec2.internal\" not found" Jul 30 15:10:19.256259 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:10:19.256227 2576 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160" Jul 30 15:10:19.292619 ip-10-0-133-210 kubenswrapper[2576]: E0730 15:10:19.292591 2576 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-133-210.ec2.internal\" not found" Jul 30 15:10:19.345067 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:10:19.345034 2576 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials" Jul 30 15:10:19.345520 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:10:19.345204 2576 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received" Jul 30 15:10:19.345520 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:10:19.345253 2576 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received" Jul 30 15:10:19.393544 ip-10-0-133-210 kubenswrapper[2576]: E0730 15:10:19.393505 2576 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-133-210.ec2.internal\" not found" Jul 30 15:10:19.406348 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:10:19.406313 2576 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-07-29 15:05:18 +0000 UTC" deadline="2028-04-15 22:22:33.237612812 +0000 UTC" Jul 30 15:10:19.406348 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:10:19.406339 2576 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="15007h12m13.831277522s" Jul 30 15:10:19.406791 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:10:19.406777 2576 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160" Jul 30 15:10:19.504152 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:10:19.504087 2576 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-210.ec2.internal" Jul 30 15:10:19.518158 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:10:19.518135 2576 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Jul 30 15:10:19.519895 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:10:19.519880 2576 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-133-210.ec2.internal" Jul 30 15:10:19.526776 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:10:19.526762 2576 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Jul 30 15:10:19.639859 ip-10-0-133-210 kubenswrapper[2576]: W0730 15:10:19.639819 2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2da2e1115617abcb95d9273e2e0aad34.slice/crio-8f37fedadf8bac02dfe1c3e3d94d19c53c6303283e4f7741729ea340837a335f WatchSource:0}: Error finding container 8f37fedadf8bac02dfe1c3e3d94d19c53c6303283e4f7741729ea340837a335f: Status 404 returned error can't find the container with id 8f37fedadf8bac02dfe1c3e3d94d19c53c6303283e4f7741729ea340837a335f Jul 30 15:10:19.644315 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:10:19.644301 2576 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 30 15:10:19.694016 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:10:19.693985 2576 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160" Jul 30 15:10:20.386838 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:10:20.386807 2576 apiserver.go:52] "Watching apiserver" Jul 30 15:10:20.396830 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:10:20.396805 2576 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66" Jul 30 15:10:20.397205 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:10:20.397183 2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/node-ca-ktmgn","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-210.ec2.internal","openshift-multus/multus-additional-cni-plugins-m2j4z","openshift-multus/multus-gb69c","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-47m5x","openshift-dns/node-resolver-j74qd","openshift-multus/network-metrics-daemon-dbmt8","openshift-network-diagnostics/network-check-target-q858z","openshift-network-operator/iptables-alerter-4g9qs","openshift-ovn-kubernetes/ovnkube-node-6mb29","kube-system/global-pull-secret-syncer-qvn96","kube-system/konnectivity-agent-7j468","kube-system/kube-apiserver-proxy-ip-10-0-133-210.ec2.internal","openshift-cluster-node-tuning-operator/tuned-fxr7d"] Jul 30 15:10:20.399578 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:10:20.399560 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-dbmt8" Jul 30 15:10:20.399646 ip-10-0-133-210 kubenswrapper[2576]: E0730 15:10:20.399629 2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-dbmt8" podUID="32b9046b-bd5c-4f23-b832-6fd0bad1c8d6" Jul 30 15:10:20.401660 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:10:20.401640 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-qvn96" Jul 30 15:10:20.401749 ip-10-0-133-210 kubenswrapper[2576]: E0730 15:10:20.401705 2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-qvn96" podUID="e76c844a-a34c-4587-93a7-023d511a0b9f" Jul 30 15:10:20.403919 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:10:20.403894 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-m2j4z" Jul 30 15:10:20.404334 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:10:20.404293 2576 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving" Jul 30 15:10:20.406103 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:10:20.406085 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-gb69c" Jul 30 15:10:20.406390 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:10:20.406366 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-b9fwp\"" Jul 30 15:10:20.406390 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:10:20.406379 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\"" Jul 30 15:10:20.406596 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:10:20.406562 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\"" Jul 30 15:10:20.406685 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:10:20.406624 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\"" Jul 30 15:10:20.406685 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:10:20.406675 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\"" Jul 30 15:10:20.406811 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:10:20.406795 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\"" Jul 30 15:10:20.408780 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:10:20.408458 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\"" Jul 30 15:10:20.408780 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:10:20.408532 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-667p6\"" Jul 30 15:10:20.408780 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:10:20.408656 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-47m5x" Jul 30 15:10:20.410800 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:10:20.410781 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-j74qd" Jul 30 15:10:20.411249 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:10:20.410933 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\"" Jul 30 15:10:20.411249 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:10:20.410934 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\"" Jul 30 15:10:20.411249 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:10:20.411083 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-phm5d\"" Jul 30 15:10:20.411249 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:10:20.411162 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\"" Jul 30 15:10:20.413102 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:10:20.413080 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-zxdsg\"" Jul 30 15:10:20.413102 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:10:20.413103 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\"" Jul 30 15:10:20.413282 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:10:20.413269 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-4g9qs" Jul 30 15:10:20.413332 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:10:20.413303 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\"" Jul 30 15:10:20.414158 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:10:20.413755 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/23fd99c2-79a8-41eb-bb0f-92a363e32553-host-var-lib-cni-bin\") pod \"multus-gb69c\" (UID: \"23fd99c2-79a8-41eb-bb0f-92a363e32553\") " pod="openshift-multus/multus-gb69c" Jul 30 15:10:20.414158 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:10:20.413802 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/23fd99c2-79a8-41eb-bb0f-92a363e32553-hostroot\") pod \"multus-gb69c\" (UID: \"23fd99c2-79a8-41eb-bb0f-92a363e32553\") " pod="openshift-multus/multus-gb69c" Jul 30 15:10:20.414158 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:10:20.413972 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/23fd99c2-79a8-41eb-bb0f-92a363e32553-host-run-multus-certs\") pod \"multus-gb69c\" (UID: \"23fd99c2-79a8-41eb-bb0f-92a363e32553\") " pod="openshift-multus/multus-gb69c" Jul 30 15:10:20.414158 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:10:20.414029 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/e76c844a-a34c-4587-93a7-023d511a0b9f-kubelet-config\") pod \"global-pull-secret-syncer-qvn96\" (UID: \"e76c844a-a34c-4587-93a7-023d511a0b9f\") " pod="kube-system/global-pull-secret-syncer-qvn96" Jul 30 15:10:20.414158 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:10:20.414057 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/23fd99c2-79a8-41eb-bb0f-92a363e32553-multus-cni-dir\") pod \"multus-gb69c\" (UID: \"23fd99c2-79a8-41eb-bb0f-92a363e32553\") " pod="openshift-multus/multus-gb69c" Jul 30 15:10:20.414158 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:10:20.414090 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/23fd99c2-79a8-41eb-bb0f-92a363e32553-host-var-lib-cni-multus\") pod \"multus-gb69c\" (UID: \"23fd99c2-79a8-41eb-bb0f-92a363e32553\") " pod="openshift-multus/multus-gb69c" Jul 30 15:10:20.415775 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:10:20.414596 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/23fd99c2-79a8-41eb-bb0f-92a363e32553-multus-daemon-config\") pod \"multus-gb69c\" (UID: \"23fd99c2-79a8-41eb-bb0f-92a363e32553\") " pod="openshift-multus/multus-gb69c" Jul 30 15:10:20.415775 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:10:20.414640 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/23fd99c2-79a8-41eb-bb0f-92a363e32553-cnibin\") pod \"multus-gb69c\" (UID: \"23fd99c2-79a8-41eb-bb0f-92a363e32553\") " pod="openshift-multus/multus-gb69c" Jul 30 15:10:20.415775 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:10:20.414672 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/f62011d0-f5be-413a-93a2-8a7a98b67130-cnibin\") pod \"multus-additional-cni-plugins-m2j4z\" (UID: \"f62011d0-f5be-413a-93a2-8a7a98b67130\") " pod="openshift-multus/multus-additional-cni-plugins-m2j4z" Jul 30 15:10:20.415775 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:10:20.414701 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/f62011d0-f5be-413a-93a2-8a7a98b67130-tuning-conf-dir\") pod \"multus-additional-cni-plugins-m2j4z\" (UID: \"f62011d0-f5be-413a-93a2-8a7a98b67130\") " pod="openshift-multus/multus-additional-cni-plugins-m2j4z" Jul 30 15:10:20.415775 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:10:20.414729 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/23fd99c2-79a8-41eb-bb0f-92a363e32553-system-cni-dir\") pod \"multus-gb69c\" (UID: \"23fd99c2-79a8-41eb-bb0f-92a363e32553\") " pod="openshift-multus/multus-gb69c" Jul 30 15:10:20.415775 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:10:20.414758 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/f62011d0-f5be-413a-93a2-8a7a98b67130-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-m2j4z\" (UID: \"f62011d0-f5be-413a-93a2-8a7a98b67130\") " pod="openshift-multus/multus-additional-cni-plugins-m2j4z" Jul 30 15:10:20.415775 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:10:20.414786 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/e76c844a-a34c-4587-93a7-023d511a0b9f-dbus\") pod \"global-pull-secret-syncer-qvn96\" (UID: \"e76c844a-a34c-4587-93a7-023d511a0b9f\") " pod="kube-system/global-pull-secret-syncer-qvn96" Jul 30 15:10:20.415775 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:10:20.414820 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/f62011d0-f5be-413a-93a2-8a7a98b67130-system-cni-dir\") pod \"multus-additional-cni-plugins-m2j4z\" (UID: \"f62011d0-f5be-413a-93a2-8a7a98b67130\") " pod="openshift-multus/multus-additional-cni-plugins-m2j4z" Jul 30 15:10:20.415775 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:10:20.414849 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/f62011d0-f5be-413a-93a2-8a7a98b67130-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-m2j4z\" (UID: \"f62011d0-f5be-413a-93a2-8a7a98b67130\") " pod="openshift-multus/multus-additional-cni-plugins-m2j4z" Jul 30 15:10:20.415775 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:10:20.414876 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/23fd99c2-79a8-41eb-bb0f-92a363e32553-multus-socket-dir-parent\") pod \"multus-gb69c\" (UID: \"23fd99c2-79a8-41eb-bb0f-92a363e32553\") " pod="openshift-multus/multus-gb69c" Jul 30 15:10:20.415775 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:10:20.414903 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/23fd99c2-79a8-41eb-bb0f-92a363e32553-multus-conf-dir\") pod \"multus-gb69c\" (UID: \"23fd99c2-79a8-41eb-bb0f-92a363e32553\") " pod="openshift-multus/multus-gb69c" Jul 30 15:10:20.415775 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:10:20.414951 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/23fd99c2-79a8-41eb-bb0f-92a363e32553-etc-kubernetes\") pod \"multus-gb69c\" (UID: \"23fd99c2-79a8-41eb-bb0f-92a363e32553\") " pod="openshift-multus/multus-gb69c" Jul 30 15:10:20.415775 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:10:20.414983 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/e76c844a-a34c-4587-93a7-023d511a0b9f-original-pull-secret\") pod \"global-pull-secret-syncer-qvn96\" (UID: \"e76c844a-a34c-4587-93a7-023d511a0b9f\") " pod="kube-system/global-pull-secret-syncer-qvn96" Jul 30 15:10:20.415775 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:10:20.415027 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/23fd99c2-79a8-41eb-bb0f-92a363e32553-cni-binary-copy\") pod \"multus-gb69c\" (UID: \"23fd99c2-79a8-41eb-bb0f-92a363e32553\") " pod="openshift-multus/multus-gb69c" Jul 30 15:10:20.415775 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:10:20.415052 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/23fd99c2-79a8-41eb-bb0f-92a363e32553-host-run-netns\") pod \"multus-gb69c\" (UID: \"23fd99c2-79a8-41eb-bb0f-92a363e32553\") " pod="openshift-multus/multus-gb69c" Jul 30 15:10:20.415775 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:10:20.415124 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-b88wn\" (UniqueName: \"kubernetes.io/projected/23fd99c2-79a8-41eb-bb0f-92a363e32553-kube-api-access-b88wn\") pod \"multus-gb69c\" (UID: \"23fd99c2-79a8-41eb-bb0f-92a363e32553\") " pod="openshift-multus/multus-gb69c" Jul 30 15:10:20.416748 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:10:20.415219 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/23fd99c2-79a8-41eb-bb0f-92a363e32553-os-release\") pod \"multus-gb69c\" (UID: \"23fd99c2-79a8-41eb-bb0f-92a363e32553\") " pod="openshift-multus/multus-gb69c" Jul 30 15:10:20.416748 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:10:20.415245 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/32b9046b-bd5c-4f23-b832-6fd0bad1c8d6-metrics-certs\") pod \"network-metrics-daemon-dbmt8\" (UID: \"32b9046b-bd5c-4f23-b832-6fd0bad1c8d6\") " pod="openshift-multus/network-metrics-daemon-dbmt8" Jul 30 15:10:20.416748 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:10:20.415281 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/f62011d0-f5be-413a-93a2-8a7a98b67130-os-release\") pod \"multus-additional-cni-plugins-m2j4z\" (UID: \"f62011d0-f5be-413a-93a2-8a7a98b67130\") " pod="openshift-multus/multus-additional-cni-plugins-m2j4z" Jul 30 15:10:20.416748 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:10:20.415311 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/f62011d0-f5be-413a-93a2-8a7a98b67130-cni-binary-copy\") pod \"multus-additional-cni-plugins-m2j4z\" (UID: \"f62011d0-f5be-413a-93a2-8a7a98b67130\") " pod="openshift-multus/multus-additional-cni-plugins-m2j4z" Jul 30 15:10:20.416748 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:10:20.415344 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2ppz7\" (UniqueName: \"kubernetes.io/projected/f62011d0-f5be-413a-93a2-8a7a98b67130-kube-api-access-2ppz7\") pod \"multus-additional-cni-plugins-m2j4z\" (UID: \"f62011d0-f5be-413a-93a2-8a7a98b67130\") " pod="openshift-multus/multus-additional-cni-plugins-m2j4z" Jul 30 15:10:20.416748 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:10:20.415370 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-x8hjl\" (UniqueName: \"kubernetes.io/projected/32b9046b-bd5c-4f23-b832-6fd0bad1c8d6-kube-api-access-x8hjl\") pod \"network-metrics-daemon-dbmt8\" (UID: \"32b9046b-bd5c-4f23-b832-6fd0bad1c8d6\") " pod="openshift-multus/network-metrics-daemon-dbmt8" Jul 30 15:10:20.416748 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:10:20.415628 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/23fd99c2-79a8-41eb-bb0f-92a363e32553-host-run-k8s-cni-cncf-io\") pod \"multus-gb69c\" (UID: \"23fd99c2-79a8-41eb-bb0f-92a363e32553\") " pod="openshift-multus/multus-gb69c" Jul 30 15:10:20.416748 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:10:20.415662 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/23fd99c2-79a8-41eb-bb0f-92a363e32553-host-var-lib-kubelet\") pod \"multus-gb69c\" (UID: \"23fd99c2-79a8-41eb-bb0f-92a363e32553\") " pod="openshift-multus/multus-gb69c" Jul 30 15:10:20.416748 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:10:20.415904 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-fxr7d" Jul 30 15:10:20.417337 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:10:20.417316 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\"" Jul 30 15:10:20.417480 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:10:20.417444 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\"" Jul 30 15:10:20.417590 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:10:20.417531 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-j75hw\"" Jul 30 15:10:20.417991 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:10:20.417968 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-q858z" Jul 30 15:10:20.418088 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:10:20.417969 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\"" Jul 30 15:10:20.418088 ip-10-0-133-210 kubenswrapper[2576]: E0730 15:10:20.418049 2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-q858z" podUID="1cd62f00-3530-42c3-9aab-499796a7c906" Jul 30 15:10:20.418613 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:10:20.418593 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-p4j4q\"" Jul 30 15:10:20.419496 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:10:20.419178 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\"" Jul 30 15:10:20.419496 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:10:20.419222 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\"" Jul 30 15:10:20.420652 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:10:20.420633 2576 reflector.go:439] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Jul 30 15:10:20.420750 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:10:20.420723 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-6mb29" Jul 30 15:10:20.422997 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:10:20.422938 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-ktmgn" Jul 30 15:10:20.423384 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:10:20.423366 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\"" Jul 30 15:10:20.424579 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:10:20.424306 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\"" Jul 30 15:10:20.424579 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:10:20.424339 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\"" Jul 30 15:10:20.424579 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:10:20.424409 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\"" Jul 30 15:10:20.424579 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:10:20.424504 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\"" Jul 30 15:10:20.424579 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:10:20.424544 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\"" Jul 30 15:10:20.424814 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:10:20.424642 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-9wgz9\"" Jul 30 15:10:20.425323 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:10:20.425304 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\"" Jul 30 15:10:20.425425 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:10:20.425348 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-7j468" Jul 30 15:10:20.425513 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:10:20.425497 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\"" Jul 30 15:10:20.425513 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:10:20.425508 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-kpcjs\"" Jul 30 15:10:20.425685 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:10:20.425670 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\"" Jul 30 15:10:20.427718 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:10:20.427700 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\"" Jul 30 15:10:20.427921 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:10:20.427902 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\"" Jul 30 15:10:20.428026 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:10:20.427972 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-cvkbc\"" Jul 30 15:10:20.448123 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:10:20.448092 2576 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-mtg5h" Jul 30 15:10:20.455735 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:10:20.455714 2576 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-mtg5h" Jul 30 15:10:20.505617 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:10:20.505592 2576 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world" Jul 30 15:10:20.515781 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:10:20.515751 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/f62011d0-f5be-413a-93a2-8a7a98b67130-system-cni-dir\") pod \"multus-additional-cni-plugins-m2j4z\" (UID: \"f62011d0-f5be-413a-93a2-8a7a98b67130\") " pod="openshift-multus/multus-additional-cni-plugins-m2j4z" Jul 30 15:10:20.515937 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:10:20.515792 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/f62011d0-f5be-413a-93a2-8a7a98b67130-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-m2j4z\" (UID: \"f62011d0-f5be-413a-93a2-8a7a98b67130\") " pod="openshift-multus/multus-additional-cni-plugins-m2j4z" Jul 30 15:10:20.515937 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:10:20.515843 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/23fd99c2-79a8-41eb-bb0f-92a363e32553-multus-conf-dir\") pod \"multus-gb69c\" (UID: \"23fd99c2-79a8-41eb-bb0f-92a363e32553\") " pod="openshift-multus/multus-gb69c" Jul 30 15:10:20.515937 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:10:20.515872 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/7fb7dc92-4394-4433-9f1d-0f2c237136fe-host-run-ovn-kubernetes\") pod \"ovnkube-node-6mb29\" (UID: \"7fb7dc92-4394-4433-9f1d-0f2c237136fe\") " pod="openshift-ovn-kubernetes/ovnkube-node-6mb29" Jul 30 15:10:20.515937 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:10:20.515893 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/7fb7dc92-4394-4433-9f1d-0f2c237136fe-ovn-node-metrics-cert\") pod \"ovnkube-node-6mb29\" (UID: \"7fb7dc92-4394-4433-9f1d-0f2c237136fe\") " pod="openshift-ovn-kubernetes/ovnkube-node-6mb29" Jul 30 15:10:20.515937 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:10:20.515917 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/4b006058-0163-4e82-9a1f-979b182ddbf1-host\") pod \"node-ca-ktmgn\" (UID: \"4b006058-0163-4e82-9a1f-979b182ddbf1\") " pod="openshift-image-registry/node-ca-ktmgn" Jul 30 15:10:20.516182 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:10:20.515941 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5mt5l\" (UniqueName: \"kubernetes.io/projected/4b006058-0163-4e82-9a1f-979b182ddbf1-kube-api-access-5mt5l\") pod \"node-ca-ktmgn\" (UID: \"4b006058-0163-4e82-9a1f-979b182ddbf1\") " pod="openshift-image-registry/node-ca-ktmgn" Jul 30 15:10:20.516182 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:10:20.515982 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/e76c844a-a34c-4587-93a7-023d511a0b9f-original-pull-secret\") pod \"global-pull-secret-syncer-qvn96\" (UID: \"e76c844a-a34c-4587-93a7-023d511a0b9f\") " pod="kube-system/global-pull-secret-syncer-qvn96" Jul 30 15:10:20.516182 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:10:20.516004 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zc7nr\" (UniqueName: \"kubernetes.io/projected/57353464-4671-4d1f-b2b5-473d618d461d-kube-api-access-zc7nr\") pod \"aws-ebs-csi-driver-node-47m5x\" (UID: \"57353464-4671-4d1f-b2b5-473d618d461d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-47m5x" Jul 30 15:10:20.516182 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:10:20.516079 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/f62011d0-f5be-413a-93a2-8a7a98b67130-system-cni-dir\") pod \"multus-additional-cni-plugins-m2j4z\" (UID: \"f62011d0-f5be-413a-93a2-8a7a98b67130\") " pod="openshift-multus/multus-additional-cni-plugins-m2j4z" Jul 30 15:10:20.516477 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:10:20.516448 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/3caae76d-69ba-4860-96fc-cca51b9ba7db-var-lib-kubelet\") pod \"tuned-fxr7d\" (UID: \"3caae76d-69ba-4860-96fc-cca51b9ba7db\") " pod="openshift-cluster-node-tuning-operator/tuned-fxr7d" Jul 30 15:10:20.516477 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:10:20.516460 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/23fd99c2-79a8-41eb-bb0f-92a363e32553-multus-conf-dir\") pod \"multus-gb69c\" (UID: \"23fd99c2-79a8-41eb-bb0f-92a363e32553\") " pod="openshift-multus/multus-gb69c" Jul 30 15:10:20.516616 ip-10-0-133-210 kubenswrapper[2576]: E0730 15:10:20.516464 2576 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 30 15:10:20.516616 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:10:20.516479 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/7fb7dc92-4394-4433-9f1d-0f2c237136fe-run-ovn\") pod \"ovnkube-node-6mb29\" (UID: \"7fb7dc92-4394-4433-9f1d-0f2c237136fe\") " pod="openshift-ovn-kubernetes/ovnkube-node-6mb29" Jul 30 15:10:20.516856 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:10:20.516822 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/3caae76d-69ba-4860-96fc-cca51b9ba7db-host\") pod \"tuned-fxr7d\" (UID: \"3caae76d-69ba-4860-96fc-cca51b9ba7db\") " pod="openshift-cluster-node-tuning-operator/tuned-fxr7d" Jul 30 15:10:20.516928 ip-10-0-133-210 kubenswrapper[2576]: E0730 15:10:20.516889 2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e76c844a-a34c-4587-93a7-023d511a0b9f-original-pull-secret podName:e76c844a-a34c-4587-93a7-023d511a0b9f nodeName:}" failed. No retries permitted until 2026-07-30 15:10:21.016862893 +0000 UTC m=+2.942615532 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/e76c844a-a34c-4587-93a7-023d511a0b9f-original-pull-secret") pod "global-pull-secret-syncer-qvn96" (UID: "e76c844a-a34c-4587-93a7-023d511a0b9f") : object "kube-system"/"original-pull-secret" not registered Jul 30 15:10:20.517052 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:10:20.516927 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/32b9046b-bd5c-4f23-b832-6fd0bad1c8d6-metrics-certs\") pod \"network-metrics-daemon-dbmt8\" (UID: \"32b9046b-bd5c-4f23-b832-6fd0bad1c8d6\") " pod="openshift-multus/network-metrics-daemon-dbmt8" Jul 30 15:10:20.517052 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:10:20.516959 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/f62011d0-f5be-413a-93a2-8a7a98b67130-os-release\") pod \"multus-additional-cni-plugins-m2j4z\" (UID: \"f62011d0-f5be-413a-93a2-8a7a98b67130\") " pod="openshift-multus/multus-additional-cni-plugins-m2j4z" Jul 30 15:10:20.517052 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:10:20.516992 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2ppz7\" (UniqueName: \"kubernetes.io/projected/f62011d0-f5be-413a-93a2-8a7a98b67130-kube-api-access-2ppz7\") pod \"multus-additional-cni-plugins-m2j4z\" (UID: \"f62011d0-f5be-413a-93a2-8a7a98b67130\") " pod="openshift-multus/multus-additional-cni-plugins-m2j4z" Jul 30 15:10:20.517052 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:10:20.517027 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/3caae76d-69ba-4860-96fc-cca51b9ba7db-etc-sysctl-d\") pod \"tuned-fxr7d\" (UID: \"3caae76d-69ba-4860-96fc-cca51b9ba7db\") " pod="openshift-cluster-node-tuning-operator/tuned-fxr7d" Jul 30 15:10:20.517268 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:10:20.517056 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/7fb7dc92-4394-4433-9f1d-0f2c237136fe-host-run-netns\") pod \"ovnkube-node-6mb29\" (UID: \"7fb7dc92-4394-4433-9f1d-0f2c237136fe\") " pod="openshift-ovn-kubernetes/ovnkube-node-6mb29" Jul 30 15:10:20.517268 ip-10-0-133-210 kubenswrapper[2576]: E0730 15:10:20.517065 2576 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 30 15:10:20.517268 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:10:20.517081 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/7fb7dc92-4394-4433-9f1d-0f2c237136fe-ovnkube-config\") pod \"ovnkube-node-6mb29\" (UID: \"7fb7dc92-4394-4433-9f1d-0f2c237136fe\") " pod="openshift-ovn-kubernetes/ovnkube-node-6mb29" Jul 30 15:10:20.517268 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:10:20.517078 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/f62011d0-f5be-413a-93a2-8a7a98b67130-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-m2j4z\" (UID: \"f62011d0-f5be-413a-93a2-8a7a98b67130\") " pod="openshift-multus/multus-additional-cni-plugins-m2j4z" Jul 30 15:10:20.517268 ip-10-0-133-210 kubenswrapper[2576]: E0730 15:10:20.517164 2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/32b9046b-bd5c-4f23-b832-6fd0bad1c8d6-metrics-certs podName:32b9046b-bd5c-4f23-b832-6fd0bad1c8d6 nodeName:}" failed. No retries permitted until 2026-07-30 15:10:21.017137196 +0000 UTC m=+2.942889831 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/32b9046b-bd5c-4f23-b832-6fd0bad1c8d6-metrics-certs") pod "network-metrics-daemon-dbmt8" (UID: "32b9046b-bd5c-4f23-b832-6fd0bad1c8d6") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 30 15:10:20.517268 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:10:20.517186 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/f62011d0-f5be-413a-93a2-8a7a98b67130-os-release\") pod \"multus-additional-cni-plugins-m2j4z\" (UID: \"f62011d0-f5be-413a-93a2-8a7a98b67130\") " pod="openshift-multus/multus-additional-cni-plugins-m2j4z" Jul 30 15:10:20.517268 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:10:20.517239 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-x8hjl\" (UniqueName: \"kubernetes.io/projected/32b9046b-bd5c-4f23-b832-6fd0bad1c8d6-kube-api-access-x8hjl\") pod \"network-metrics-daemon-dbmt8\" (UID: \"32b9046b-bd5c-4f23-b832-6fd0bad1c8d6\") " pod="openshift-multus/network-metrics-daemon-dbmt8" Jul 30 15:10:20.517585 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:10:20.517282 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/23fd99c2-79a8-41eb-bb0f-92a363e32553-host-run-k8s-cni-cncf-io\") pod \"multus-gb69c\" (UID: \"23fd99c2-79a8-41eb-bb0f-92a363e32553\") " pod="openshift-multus/multus-gb69c" Jul 30 15:10:20.517585 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:10:20.517343 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/23fd99c2-79a8-41eb-bb0f-92a363e32553-host-run-k8s-cni-cncf-io\") pod \"multus-gb69c\" (UID: \"23fd99c2-79a8-41eb-bb0f-92a363e32553\") " pod="openshift-multus/multus-gb69c" Jul 30 15:10:20.517585 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:10:20.517429 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/23fd99c2-79a8-41eb-bb0f-92a363e32553-host-var-lib-kubelet\") pod \"multus-gb69c\" (UID: \"23fd99c2-79a8-41eb-bb0f-92a363e32553\") " pod="openshift-multus/multus-gb69c" Jul 30 15:10:20.517585 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:10:20.517489 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/3caae76d-69ba-4860-96fc-cca51b9ba7db-run\") pod \"tuned-fxr7d\" (UID: \"3caae76d-69ba-4860-96fc-cca51b9ba7db\") " pod="openshift-cluster-node-tuning-operator/tuned-fxr7d" Jul 30 15:10:20.517585 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:10:20.517545 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/23fd99c2-79a8-41eb-bb0f-92a363e32553-host-var-lib-kubelet\") pod \"multus-gb69c\" (UID: \"23fd99c2-79a8-41eb-bb0f-92a363e32553\") " pod="openshift-multus/multus-gb69c" Jul 30 15:10:20.517806 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:10:20.517590 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/7fb7dc92-4394-4433-9f1d-0f2c237136fe-var-lib-openvswitch\") pod \"ovnkube-node-6mb29\" (UID: \"7fb7dc92-4394-4433-9f1d-0f2c237136fe\") " pod="openshift-ovn-kubernetes/ovnkube-node-6mb29" Jul 30 15:10:20.517806 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:10:20.517628 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/7fb7dc92-4394-4433-9f1d-0f2c237136fe-host-cni-bin\") pod \"ovnkube-node-6mb29\" (UID: \"7fb7dc92-4394-4433-9f1d-0f2c237136fe\") " pod="openshift-ovn-kubernetes/ovnkube-node-6mb29" Jul 30 15:10:20.517806 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:10:20.517658 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/7fb7dc92-4394-4433-9f1d-0f2c237136fe-ovnkube-script-lib\") pod \"ovnkube-node-6mb29\" (UID: \"7fb7dc92-4394-4433-9f1d-0f2c237136fe\") " pod="openshift-ovn-kubernetes/ovnkube-node-6mb29" Jul 30 15:10:20.517806 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:10:20.517705 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/23fd99c2-79a8-41eb-bb0f-92a363e32553-hostroot\") pod \"multus-gb69c\" (UID: \"23fd99c2-79a8-41eb-bb0f-92a363e32553\") " pod="openshift-multus/multus-gb69c" Jul 30 15:10:20.517806 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:10:20.517750 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/23fd99c2-79a8-41eb-bb0f-92a363e32553-hostroot\") pod \"multus-gb69c\" (UID: \"23fd99c2-79a8-41eb-bb0f-92a363e32553\") " pod="openshift-multus/multus-gb69c" Jul 30 15:10:20.517806 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:10:20.517759 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/23fd99c2-79a8-41eb-bb0f-92a363e32553-host-run-multus-certs\") pod \"multus-gb69c\" (UID: \"23fd99c2-79a8-41eb-bb0f-92a363e32553\") " pod="openshift-multus/multus-gb69c" Jul 30 15:10:20.517806 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:10:20.517805 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7pzx7\" (UniqueName: \"kubernetes.io/projected/1cd62f00-3530-42c3-9aab-499796a7c906-kube-api-access-7pzx7\") pod \"network-check-target-q858z\" (UID: \"1cd62f00-3530-42c3-9aab-499796a7c906\") " pod="openshift-network-diagnostics/network-check-target-q858z" Jul 30 15:10:20.518180 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:10:20.517813 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/23fd99c2-79a8-41eb-bb0f-92a363e32553-host-run-multus-certs\") pod \"multus-gb69c\" (UID: \"23fd99c2-79a8-41eb-bb0f-92a363e32553\") " pod="openshift-multus/multus-gb69c" Jul 30 15:10:20.518180 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:10:20.517860 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/e76c844a-a34c-4587-93a7-023d511a0b9f-kubelet-config\") pod \"global-pull-secret-syncer-qvn96\" (UID: \"e76c844a-a34c-4587-93a7-023d511a0b9f\") " pod="kube-system/global-pull-secret-syncer-qvn96" Jul 30 15:10:20.518180 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:10:20.517892 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/23fd99c2-79a8-41eb-bb0f-92a363e32553-multus-cni-dir\") pod \"multus-gb69c\" (UID: \"23fd99c2-79a8-41eb-bb0f-92a363e32553\") " pod="openshift-multus/multus-gb69c" Jul 30 15:10:20.518180 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:10:20.517950 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/e76c844a-a34c-4587-93a7-023d511a0b9f-kubelet-config\") pod \"global-pull-secret-syncer-qvn96\" (UID: \"e76c844a-a34c-4587-93a7-023d511a0b9f\") " pod="kube-system/global-pull-secret-syncer-qvn96" Jul 30 15:10:20.518180 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:10:20.517985 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/23fd99c2-79a8-41eb-bb0f-92a363e32553-host-var-lib-cni-multus\") pod \"multus-gb69c\" (UID: \"23fd99c2-79a8-41eb-bb0f-92a363e32553\") " pod="openshift-multus/multus-gb69c" Jul 30 15:10:20.518180 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:10:20.518018 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/7fb7dc92-4394-4433-9f1d-0f2c237136fe-run-systemd\") pod \"ovnkube-node-6mb29\" (UID: \"7fb7dc92-4394-4433-9f1d-0f2c237136fe\") " pod="openshift-ovn-kubernetes/ovnkube-node-6mb29" Jul 30 15:10:20.518180 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:10:20.518046 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/7fb7dc92-4394-4433-9f1d-0f2c237136fe-run-openvswitch\") pod \"ovnkube-node-6mb29\" (UID: \"7fb7dc92-4394-4433-9f1d-0f2c237136fe\") " pod="openshift-ovn-kubernetes/ovnkube-node-6mb29" Jul 30 15:10:20.518180 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:10:20.518078 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/f62011d0-f5be-413a-93a2-8a7a98b67130-cnibin\") pod \"multus-additional-cni-plugins-m2j4z\" (UID: \"f62011d0-f5be-413a-93a2-8a7a98b67130\") " pod="openshift-multus/multus-additional-cni-plugins-m2j4z" Jul 30 15:10:20.518180 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:10:20.518117 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/f62011d0-f5be-413a-93a2-8a7a98b67130-tuning-conf-dir\") pod \"multus-additional-cni-plugins-m2j4z\" (UID: \"f62011d0-f5be-413a-93a2-8a7a98b67130\") " pod="openshift-multus/multus-additional-cni-plugins-m2j4z" Jul 30 15:10:20.518180 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:10:20.518151 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/57353464-4671-4d1f-b2b5-473d618d461d-socket-dir\") pod \"aws-ebs-csi-driver-node-47m5x\" (UID: \"57353464-4671-4d1f-b2b5-473d618d461d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-47m5x" Jul 30 15:10:20.518180 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:10:20.518182 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/5be52bf9-46b0-4e79-9d34-9b32b8b891b4-host-slash\") pod \"iptables-alerter-4g9qs\" (UID: \"5be52bf9-46b0-4e79-9d34-9b32b8b891b4\") " pod="openshift-network-operator/iptables-alerter-4g9qs" Jul 30 15:10:20.518704 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:10:20.518211 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cdcss\" (UniqueName: \"kubernetes.io/projected/5be52bf9-46b0-4e79-9d34-9b32b8b891b4-kube-api-access-cdcss\") pod \"iptables-alerter-4g9qs\" (UID: \"5be52bf9-46b0-4e79-9d34-9b32b8b891b4\") " pod="openshift-network-operator/iptables-alerter-4g9qs" Jul 30 15:10:20.518704 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:10:20.518236 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/3caae76d-69ba-4860-96fc-cca51b9ba7db-etc-modprobe-d\") pod \"tuned-fxr7d\" (UID: \"3caae76d-69ba-4860-96fc-cca51b9ba7db\") " pod="openshift-cluster-node-tuning-operator/tuned-fxr7d" Jul 30 15:10:20.518704 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:10:20.518263 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/7fb7dc92-4394-4433-9f1d-0f2c237136fe-systemd-units\") pod \"ovnkube-node-6mb29\" (UID: \"7fb7dc92-4394-4433-9f1d-0f2c237136fe\") " pod="openshift-ovn-kubernetes/ovnkube-node-6mb29" Jul 30 15:10:20.518704 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:10:20.518290 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-d6f4c\" (UniqueName: \"kubernetes.io/projected/7fb7dc92-4394-4433-9f1d-0f2c237136fe-kube-api-access-d6f4c\") pod \"ovnkube-node-6mb29\" (UID: \"7fb7dc92-4394-4433-9f1d-0f2c237136fe\") " pod="openshift-ovn-kubernetes/ovnkube-node-6mb29" Jul 30 15:10:20.518704 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:10:20.518317 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/3caae76d-69ba-4860-96fc-cca51b9ba7db-lib-modules\") pod \"tuned-fxr7d\" (UID: \"3caae76d-69ba-4860-96fc-cca51b9ba7db\") " pod="openshift-cluster-node-tuning-operator/tuned-fxr7d" Jul 30 15:10:20.518704 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:10:20.518346 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-c9j5g\" (UniqueName: \"kubernetes.io/projected/3caae76d-69ba-4860-96fc-cca51b9ba7db-kube-api-access-c9j5g\") pod \"tuned-fxr7d\" (UID: \"3caae76d-69ba-4860-96fc-cca51b9ba7db\") " pod="openshift-cluster-node-tuning-operator/tuned-fxr7d" Jul 30 15:10:20.518704 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:10:20.518378 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/7fb7dc92-4394-4433-9f1d-0f2c237136fe-log-socket\") pod \"ovnkube-node-6mb29\" (UID: \"7fb7dc92-4394-4433-9f1d-0f2c237136fe\") " pod="openshift-ovn-kubernetes/ovnkube-node-6mb29" Jul 30 15:10:20.518704 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:10:20.518391 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/23fd99c2-79a8-41eb-bb0f-92a363e32553-host-var-lib-cni-multus\") pod \"multus-gb69c\" (UID: \"23fd99c2-79a8-41eb-bb0f-92a363e32553\") " pod="openshift-multus/multus-gb69c" Jul 30 15:10:20.518704 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:10:20.518423 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/347bd19f-7728-47be-986d-337d0f0df246-hosts-file\") pod \"node-resolver-j74qd\" (UID: \"347bd19f-7728-47be-986d-337d0f0df246\") " pod="openshift-dns/node-resolver-j74qd" Jul 30 15:10:20.518704 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:10:20.518460 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/347bd19f-7728-47be-986d-337d0f0df246-tmp-dir\") pod \"node-resolver-j74qd\" (UID: \"347bd19f-7728-47be-986d-337d0f0df246\") " pod="openshift-dns/node-resolver-j74qd" Jul 30 15:10:20.518704 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:10:20.518489 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/f62011d0-f5be-413a-93a2-8a7a98b67130-cnibin\") pod \"multus-additional-cni-plugins-m2j4z\" (UID: \"f62011d0-f5be-413a-93a2-8a7a98b67130\") " pod="openshift-multus/multus-additional-cni-plugins-m2j4z" Jul 30 15:10:20.518704 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:10:20.518490 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/e76c844a-a34c-4587-93a7-023d511a0b9f-dbus\") pod \"global-pull-secret-syncer-qvn96\" (UID: \"e76c844a-a34c-4587-93a7-023d511a0b9f\") " pod="kube-system/global-pull-secret-syncer-qvn96" Jul 30 15:10:20.518704 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:10:20.518538 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/23fd99c2-79a8-41eb-bb0f-92a363e32553-multus-socket-dir-parent\") pod \"multus-gb69c\" (UID: \"23fd99c2-79a8-41eb-bb0f-92a363e32553\") " pod="openshift-multus/multus-gb69c" Jul 30 15:10:20.518704 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:10:20.518551 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/e76c844a-a34c-4587-93a7-023d511a0b9f-dbus\") pod \"global-pull-secret-syncer-qvn96\" (UID: \"e76c844a-a34c-4587-93a7-023d511a0b9f\") " pod="kube-system/global-pull-secret-syncer-qvn96" Jul 30 15:10:20.518704 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:10:20.518575 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/23fd99c2-79a8-41eb-bb0f-92a363e32553-etc-kubernetes\") pod \"multus-gb69c\" (UID: \"23fd99c2-79a8-41eb-bb0f-92a363e32553\") " pod="openshift-multus/multus-gb69c" Jul 30 15:10:20.518704 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:10:20.518342 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/23fd99c2-79a8-41eb-bb0f-92a363e32553-multus-cni-dir\") pod \"multus-gb69c\" (UID: \"23fd99c2-79a8-41eb-bb0f-92a363e32553\") " pod="openshift-multus/multus-gb69c" Jul 30 15:10:20.519426 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:10:20.518881 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/f62011d0-f5be-413a-93a2-8a7a98b67130-tuning-conf-dir\") pod \"multus-additional-cni-plugins-m2j4z\" (UID: \"f62011d0-f5be-413a-93a2-8a7a98b67130\") " pod="openshift-multus/multus-additional-cni-plugins-m2j4z" Jul 30 15:10:20.519426 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:10:20.518962 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/23fd99c2-79a8-41eb-bb0f-92a363e32553-multus-socket-dir-parent\") pod \"multus-gb69c\" (UID: \"23fd99c2-79a8-41eb-bb0f-92a363e32553\") " pod="openshift-multus/multus-gb69c" Jul 30 15:10:20.519426 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:10:20.519005 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/23fd99c2-79a8-41eb-bb0f-92a363e32553-etc-kubernetes\") pod \"multus-gb69c\" (UID: \"23fd99c2-79a8-41eb-bb0f-92a363e32553\") " pod="openshift-multus/multus-gb69c" Jul 30 15:10:20.519617 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:10:20.519576 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/4b006058-0163-4e82-9a1f-979b182ddbf1-serviceca\") pod \"node-ca-ktmgn\" (UID: \"4b006058-0163-4e82-9a1f-979b182ddbf1\") " pod="openshift-image-registry/node-ca-ktmgn" Jul 30 15:10:20.519698 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:10:20.519647 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/23fd99c2-79a8-41eb-bb0f-92a363e32553-cni-binary-copy\") pod \"multus-gb69c\" (UID: \"23fd99c2-79a8-41eb-bb0f-92a363e32553\") " pod="openshift-multus/multus-gb69c" Jul 30 15:10:20.519698 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:10:20.519690 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/23fd99c2-79a8-41eb-bb0f-92a363e32553-host-run-netns\") pod \"multus-gb69c\" (UID: \"23fd99c2-79a8-41eb-bb0f-92a363e32553\") " pod="openshift-multus/multus-gb69c" Jul 30 15:10:20.519799 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:10:20.519717 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-b88wn\" (UniqueName: \"kubernetes.io/projected/23fd99c2-79a8-41eb-bb0f-92a363e32553-kube-api-access-b88wn\") pod \"multus-gb69c\" (UID: \"23fd99c2-79a8-41eb-bb0f-92a363e32553\") " pod="openshift-multus/multus-gb69c" Jul 30 15:10:20.519799 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:10:20.519759 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/3caae76d-69ba-4860-96fc-cca51b9ba7db-etc-systemd\") pod \"tuned-fxr7d\" (UID: \"3caae76d-69ba-4860-96fc-cca51b9ba7db\") " pod="openshift-cluster-node-tuning-operator/tuned-fxr7d" Jul 30 15:10:20.519799 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:10:20.519792 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/7fb7dc92-4394-4433-9f1d-0f2c237136fe-env-overrides\") pod \"ovnkube-node-6mb29\" (UID: \"7fb7dc92-4394-4433-9f1d-0f2c237136fe\") " pod="openshift-ovn-kubernetes/ovnkube-node-6mb29" Jul 30 15:10:20.519931 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:10:20.519821 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/3caae76d-69ba-4860-96fc-cca51b9ba7db-tmp\") pod \"tuned-fxr7d\" (UID: \"3caae76d-69ba-4860-96fc-cca51b9ba7db\") " pod="openshift-cluster-node-tuning-operator/tuned-fxr7d" Jul 30 15:10:20.519931 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:10:20.519851 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/23fd99c2-79a8-41eb-bb0f-92a363e32553-os-release\") pod \"multus-gb69c\" (UID: \"23fd99c2-79a8-41eb-bb0f-92a363e32553\") " pod="openshift-multus/multus-gb69c" Jul 30 15:10:20.519931 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:10:20.519880 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/f62011d0-f5be-413a-93a2-8a7a98b67130-cni-binary-copy\") pod \"multus-additional-cni-plugins-m2j4z\" (UID: \"f62011d0-f5be-413a-93a2-8a7a98b67130\") " pod="openshift-multus/multus-additional-cni-plugins-m2j4z" Jul 30 15:10:20.519931 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:10:20.519915 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/3caae76d-69ba-4860-96fc-cca51b9ba7db-etc-sysctl-conf\") pod \"tuned-fxr7d\" (UID: \"3caae76d-69ba-4860-96fc-cca51b9ba7db\") " pod="openshift-cluster-node-tuning-operator/tuned-fxr7d" Jul 30 15:10:20.520122 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:10:20.519942 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/7fb7dc92-4394-4433-9f1d-0f2c237136fe-host-cni-netd\") pod \"ovnkube-node-6mb29\" (UID: \"7fb7dc92-4394-4433-9f1d-0f2c237136fe\") " pod="openshift-ovn-kubernetes/ovnkube-node-6mb29" Jul 30 15:10:20.520122 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:10:20.519962 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/57353464-4671-4d1f-b2b5-473d618d461d-device-dir\") pod \"aws-ebs-csi-driver-node-47m5x\" (UID: \"57353464-4671-4d1f-b2b5-473d618d461d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-47m5x" Jul 30 15:10:20.520122 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:10:20.519981 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/5be52bf9-46b0-4e79-9d34-9b32b8b891b4-iptables-alerter-script\") pod \"iptables-alerter-4g9qs\" (UID: \"5be52bf9-46b0-4e79-9d34-9b32b8b891b4\") " pod="openshift-network-operator/iptables-alerter-4g9qs" Jul 30 15:10:20.520122 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:10:20.520005 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/3caae76d-69ba-4860-96fc-cca51b9ba7db-etc-tuned\") pod \"tuned-fxr7d\" (UID: \"3caae76d-69ba-4860-96fc-cca51b9ba7db\") " pod="openshift-cluster-node-tuning-operator/tuned-fxr7d" Jul 30 15:10:20.520122 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:10:20.520035 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/7fb7dc92-4394-4433-9f1d-0f2c237136fe-etc-openvswitch\") pod \"ovnkube-node-6mb29\" (UID: \"7fb7dc92-4394-4433-9f1d-0f2c237136fe\") " pod="openshift-ovn-kubernetes/ovnkube-node-6mb29" Jul 30 15:10:20.520349 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:10:20.520124 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/a2197c57-2dc6-410e-9910-a045dedcb8d9-konnectivity-ca\") pod \"konnectivity-agent-7j468\" (UID: \"a2197c57-2dc6-410e-9910-a045dedcb8d9\") " pod="kube-system/konnectivity-agent-7j468" Jul 30 15:10:20.520349 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:10:20.520151 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/23fd99c2-79a8-41eb-bb0f-92a363e32553-os-release\") pod \"multus-gb69c\" (UID: \"23fd99c2-79a8-41eb-bb0f-92a363e32553\") " pod="openshift-multus/multus-gb69c" Jul 30 15:10:20.520349 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:10:20.520165 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/23fd99c2-79a8-41eb-bb0f-92a363e32553-host-run-netns\") pod \"multus-gb69c\" (UID: \"23fd99c2-79a8-41eb-bb0f-92a363e32553\") " pod="openshift-multus/multus-gb69c" Jul 30 15:10:20.520349 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:10:20.520170 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/23fd99c2-79a8-41eb-bb0f-92a363e32553-host-var-lib-cni-bin\") pod \"multus-gb69c\" (UID: \"23fd99c2-79a8-41eb-bb0f-92a363e32553\") " pod="openshift-multus/multus-gb69c" Jul 30 15:10:20.520349 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:10:20.520241 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/23fd99c2-79a8-41eb-bb0f-92a363e32553-host-var-lib-cni-bin\") pod \"multus-gb69c\" (UID: \"23fd99c2-79a8-41eb-bb0f-92a363e32553\") " pod="openshift-multus/multus-gb69c" Jul 30 15:10:20.520349 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:10:20.520245 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/57353464-4671-4d1f-b2b5-473d618d461d-kubelet-dir\") pod \"aws-ebs-csi-driver-node-47m5x\" (UID: \"57353464-4671-4d1f-b2b5-473d618d461d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-47m5x" Jul 30 15:10:20.520621 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:10:20.520392 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/23fd99c2-79a8-41eb-bb0f-92a363e32553-multus-daemon-config\") pod \"multus-gb69c\" (UID: \"23fd99c2-79a8-41eb-bb0f-92a363e32553\") " pod="openshift-multus/multus-gb69c" Jul 30 15:10:20.520621 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:10:20.520445 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/57353464-4671-4d1f-b2b5-473d618d461d-etc-selinux\") pod \"aws-ebs-csi-driver-node-47m5x\" (UID: \"57353464-4671-4d1f-b2b5-473d618d461d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-47m5x" Jul 30 15:10:20.520621 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:10:20.520494 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/23fd99c2-79a8-41eb-bb0f-92a363e32553-cni-binary-copy\") pod \"multus-gb69c\" (UID: \"23fd99c2-79a8-41eb-bb0f-92a363e32553\") " pod="openshift-multus/multus-gb69c" Jul 30 15:10:20.520621 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:10:20.520516 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/57353464-4671-4d1f-b2b5-473d618d461d-sys-fs\") pod \"aws-ebs-csi-driver-node-47m5x\" (UID: \"57353464-4671-4d1f-b2b5-473d618d461d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-47m5x" Jul 30 15:10:20.520621 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:10:20.520529 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/f62011d0-f5be-413a-93a2-8a7a98b67130-cni-binary-copy\") pod \"multus-additional-cni-plugins-m2j4z\" (UID: \"f62011d0-f5be-413a-93a2-8a7a98b67130\") " pod="openshift-multus/multus-additional-cni-plugins-m2j4z" Jul 30 15:10:20.520621 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:10:20.520565 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/3caae76d-69ba-4860-96fc-cca51b9ba7db-etc-kubernetes\") pod \"tuned-fxr7d\" (UID: \"3caae76d-69ba-4860-96fc-cca51b9ba7db\") " pod="openshift-cluster-node-tuning-operator/tuned-fxr7d" Jul 30 15:10:20.520621 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:10:20.520602 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/3caae76d-69ba-4860-96fc-cca51b9ba7db-sys\") pod \"tuned-fxr7d\" (UID: \"3caae76d-69ba-4860-96fc-cca51b9ba7db\") " pod="openshift-cluster-node-tuning-operator/tuned-fxr7d" Jul 30 15:10:20.520911 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:10:20.520639 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/7fb7dc92-4394-4433-9f1d-0f2c237136fe-host-kubelet\") pod \"ovnkube-node-6mb29\" (UID: \"7fb7dc92-4394-4433-9f1d-0f2c237136fe\") " pod="openshift-ovn-kubernetes/ovnkube-node-6mb29" Jul 30 15:10:20.520911 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:10:20.520706 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/7fb7dc92-4394-4433-9f1d-0f2c237136fe-host-slash\") pod \"ovnkube-node-6mb29\" (UID: \"7fb7dc92-4394-4433-9f1d-0f2c237136fe\") " pod="openshift-ovn-kubernetes/ovnkube-node-6mb29" Jul 30 15:10:20.521143 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:10:20.521119 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/7fb7dc92-4394-4433-9f1d-0f2c237136fe-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-6mb29\" (UID: \"7fb7dc92-4394-4433-9f1d-0f2c237136fe\") " pod="openshift-ovn-kubernetes/ovnkube-node-6mb29" Jul 30 15:10:20.521255 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:10:20.521152 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/23fd99c2-79a8-41eb-bb0f-92a363e32553-cnibin\") pod \"multus-gb69c\" (UID: \"23fd99c2-79a8-41eb-bb0f-92a363e32553\") " pod="openshift-multus/multus-gb69c" Jul 30 15:10:20.521255 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:10:20.521169 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/23fd99c2-79a8-41eb-bb0f-92a363e32553-system-cni-dir\") pod \"multus-gb69c\" (UID: \"23fd99c2-79a8-41eb-bb0f-92a363e32553\") " pod="openshift-multus/multus-gb69c" Jul 30 15:10:20.521255 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:10:20.521193 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/57353464-4671-4d1f-b2b5-473d618d461d-registration-dir\") pod \"aws-ebs-csi-driver-node-47m5x\" (UID: \"57353464-4671-4d1f-b2b5-473d618d461d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-47m5x" Jul 30 15:10:20.521255 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:10:20.521217 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/a2197c57-2dc6-410e-9910-a045dedcb8d9-agent-certs\") pod \"konnectivity-agent-7j468\" (UID: \"a2197c57-2dc6-410e-9910-a045dedcb8d9\") " pod="kube-system/konnectivity-agent-7j468" Jul 30 15:10:20.521255 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:10:20.521234 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/23fd99c2-79a8-41eb-bb0f-92a363e32553-cnibin\") pod \"multus-gb69c\" (UID: \"23fd99c2-79a8-41eb-bb0f-92a363e32553\") " pod="openshift-multus/multus-gb69c" Jul 30 15:10:20.521255 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:10:20.521248 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/23fd99c2-79a8-41eb-bb0f-92a363e32553-system-cni-dir\") pod \"multus-gb69c\" (UID: \"23fd99c2-79a8-41eb-bb0f-92a363e32553\") " pod="openshift-multus/multus-gb69c" Jul 30 15:10:20.521493 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:10:20.521239 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/3caae76d-69ba-4860-96fc-cca51b9ba7db-etc-sysconfig\") pod \"tuned-fxr7d\" (UID: \"3caae76d-69ba-4860-96fc-cca51b9ba7db\") " pod="openshift-cluster-node-tuning-operator/tuned-fxr7d" Jul 30 15:10:20.521493 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:10:20.521296 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/23fd99c2-79a8-41eb-bb0f-92a363e32553-multus-daemon-config\") pod \"multus-gb69c\" (UID: \"23fd99c2-79a8-41eb-bb0f-92a363e32553\") " pod="openshift-multus/multus-gb69c" Jul 30 15:10:20.521493 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:10:20.521298 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/7fb7dc92-4394-4433-9f1d-0f2c237136fe-node-log\") pod \"ovnkube-node-6mb29\" (UID: \"7fb7dc92-4394-4433-9f1d-0f2c237136fe\") " pod="openshift-ovn-kubernetes/ovnkube-node-6mb29" Jul 30 15:10:20.521493 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:10:20.521339 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-phfqm\" (UniqueName: \"kubernetes.io/projected/347bd19f-7728-47be-986d-337d0f0df246-kube-api-access-phfqm\") pod \"node-resolver-j74qd\" (UID: \"347bd19f-7728-47be-986d-337d0f0df246\") " pod="openshift-dns/node-resolver-j74qd" Jul 30 15:10:20.521493 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:10:20.521360 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/f62011d0-f5be-413a-93a2-8a7a98b67130-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-m2j4z\" (UID: \"f62011d0-f5be-413a-93a2-8a7a98b67130\") " pod="openshift-multus/multus-additional-cni-plugins-m2j4z" Jul 30 15:10:20.521723 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:10:20.521707 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/f62011d0-f5be-413a-93a2-8a7a98b67130-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-m2j4z\" (UID: \"f62011d0-f5be-413a-93a2-8a7a98b67130\") " pod="openshift-multus/multus-additional-cni-plugins-m2j4z" Jul 30 15:10:20.527272 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:10:20.527236 2576 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory" Jul 30 15:10:20.530663 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:10:20.530644 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-b88wn\" (UniqueName: \"kubernetes.io/projected/23fd99c2-79a8-41eb-bb0f-92a363e32553-kube-api-access-b88wn\") pod \"multus-gb69c\" (UID: \"23fd99c2-79a8-41eb-bb0f-92a363e32553\") " pod="openshift-multus/multus-gb69c" Jul 30 15:10:20.530736 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:10:20.530663 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2ppz7\" (UniqueName: \"kubernetes.io/projected/f62011d0-f5be-413a-93a2-8a7a98b67130-kube-api-access-2ppz7\") pod \"multus-additional-cni-plugins-m2j4z\" (UID: \"f62011d0-f5be-413a-93a2-8a7a98b67130\") " pod="openshift-multus/multus-additional-cni-plugins-m2j4z" Jul 30 15:10:20.530736 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:10:20.530728 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-x8hjl\" (UniqueName: \"kubernetes.io/projected/32b9046b-bd5c-4f23-b832-6fd0bad1c8d6-kube-api-access-x8hjl\") pod \"network-metrics-daemon-dbmt8\" (UID: \"32b9046b-bd5c-4f23-b832-6fd0bad1c8d6\") " pod="openshift-multus/network-metrics-daemon-dbmt8" Jul 30 15:10:20.579490 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:10:20.579456 2576 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Jul 30 15:10:20.610587 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:10:20.610538 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-133-210.ec2.internal" event={"ID":"2da2e1115617abcb95d9273e2e0aad34","Type":"ContainerStarted","Data":"8f37fedadf8bac02dfe1c3e3d94d19c53c6303283e4f7741729ea340837a335f"} Jul 30 15:10:20.622147 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:10:20.622098 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/57353464-4671-4d1f-b2b5-473d618d461d-etc-selinux\") pod \"aws-ebs-csi-driver-node-47m5x\" (UID: \"57353464-4671-4d1f-b2b5-473d618d461d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-47m5x" Jul 30 15:10:20.622306 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:10:20.622156 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/57353464-4671-4d1f-b2b5-473d618d461d-sys-fs\") pod \"aws-ebs-csi-driver-node-47m5x\" (UID: \"57353464-4671-4d1f-b2b5-473d618d461d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-47m5x" Jul 30 15:10:20.622306 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:10:20.622177 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/3caae76d-69ba-4860-96fc-cca51b9ba7db-etc-kubernetes\") pod \"tuned-fxr7d\" (UID: \"3caae76d-69ba-4860-96fc-cca51b9ba7db\") " pod="openshift-cluster-node-tuning-operator/tuned-fxr7d" Jul 30 15:10:20.622306 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:10:20.622197 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/3caae76d-69ba-4860-96fc-cca51b9ba7db-sys\") pod \"tuned-fxr7d\" (UID: \"3caae76d-69ba-4860-96fc-cca51b9ba7db\") " pod="openshift-cluster-node-tuning-operator/tuned-fxr7d" Jul 30 15:10:20.622306 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:10:20.622218 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/7fb7dc92-4394-4433-9f1d-0f2c237136fe-host-kubelet\") pod \"ovnkube-node-6mb29\" (UID: \"7fb7dc92-4394-4433-9f1d-0f2c237136fe\") " pod="openshift-ovn-kubernetes/ovnkube-node-6mb29" Jul 30 15:10:20.622306 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:10:20.622237 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/57353464-4671-4d1f-b2b5-473d618d461d-etc-selinux\") pod \"aws-ebs-csi-driver-node-47m5x\" (UID: \"57353464-4671-4d1f-b2b5-473d618d461d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-47m5x" Jul 30 15:10:20.622543 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:10:20.622310 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/57353464-4671-4d1f-b2b5-473d618d461d-sys-fs\") pod \"aws-ebs-csi-driver-node-47m5x\" (UID: \"57353464-4671-4d1f-b2b5-473d618d461d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-47m5x" Jul 30 15:10:20.622543 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:10:20.622353 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/3caae76d-69ba-4860-96fc-cca51b9ba7db-sys\") pod \"tuned-fxr7d\" (UID: \"3caae76d-69ba-4860-96fc-cca51b9ba7db\") " pod="openshift-cluster-node-tuning-operator/tuned-fxr7d" Jul 30 15:10:20.622543 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:10:20.622396 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/7fb7dc92-4394-4433-9f1d-0f2c237136fe-host-kubelet\") pod \"ovnkube-node-6mb29\" (UID: \"7fb7dc92-4394-4433-9f1d-0f2c237136fe\") " pod="openshift-ovn-kubernetes/ovnkube-node-6mb29" Jul 30 15:10:20.622543 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:10:20.622426 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/7fb7dc92-4394-4433-9f1d-0f2c237136fe-host-slash\") pod \"ovnkube-node-6mb29\" (UID: \"7fb7dc92-4394-4433-9f1d-0f2c237136fe\") " pod="openshift-ovn-kubernetes/ovnkube-node-6mb29" Jul 30 15:10:20.622543 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:10:20.622462 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/7fb7dc92-4394-4433-9f1d-0f2c237136fe-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-6mb29\" (UID: \"7fb7dc92-4394-4433-9f1d-0f2c237136fe\") " pod="openshift-ovn-kubernetes/ovnkube-node-6mb29" Jul 30 15:10:20.622543 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:10:20.622490 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/57353464-4671-4d1f-b2b5-473d618d461d-registration-dir\") pod \"aws-ebs-csi-driver-node-47m5x\" (UID: \"57353464-4671-4d1f-b2b5-473d618d461d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-47m5x" Jul 30 15:10:20.622543 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:10:20.622511 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/3caae76d-69ba-4860-96fc-cca51b9ba7db-etc-kubernetes\") pod \"tuned-fxr7d\" (UID: \"3caae76d-69ba-4860-96fc-cca51b9ba7db\") " pod="openshift-cluster-node-tuning-operator/tuned-fxr7d" Jul 30 15:10:20.622543 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:10:20.622524 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/7fb7dc92-4394-4433-9f1d-0f2c237136fe-host-slash\") pod \"ovnkube-node-6mb29\" (UID: \"7fb7dc92-4394-4433-9f1d-0f2c237136fe\") " pod="openshift-ovn-kubernetes/ovnkube-node-6mb29" Jul 30 15:10:20.622931 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:10:20.622546 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/7fb7dc92-4394-4433-9f1d-0f2c237136fe-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-6mb29\" (UID: \"7fb7dc92-4394-4433-9f1d-0f2c237136fe\") " pod="openshift-ovn-kubernetes/ovnkube-node-6mb29" Jul 30 15:10:20.622931 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:10:20.622550 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/57353464-4671-4d1f-b2b5-473d618d461d-registration-dir\") pod \"aws-ebs-csi-driver-node-47m5x\" (UID: \"57353464-4671-4d1f-b2b5-473d618d461d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-47m5x" Jul 30 15:10:20.622931 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:10:20.622597 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/a2197c57-2dc6-410e-9910-a045dedcb8d9-agent-certs\") pod \"konnectivity-agent-7j468\" (UID: \"a2197c57-2dc6-410e-9910-a045dedcb8d9\") " pod="kube-system/konnectivity-agent-7j468" Jul 30 15:10:20.622931 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:10:20.622625 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/3caae76d-69ba-4860-96fc-cca51b9ba7db-etc-sysconfig\") pod \"tuned-fxr7d\" (UID: \"3caae76d-69ba-4860-96fc-cca51b9ba7db\") " pod="openshift-cluster-node-tuning-operator/tuned-fxr7d" Jul 30 15:10:20.622931 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:10:20.622650 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/7fb7dc92-4394-4433-9f1d-0f2c237136fe-node-log\") pod \"ovnkube-node-6mb29\" (UID: \"7fb7dc92-4394-4433-9f1d-0f2c237136fe\") " pod="openshift-ovn-kubernetes/ovnkube-node-6mb29" Jul 30 15:10:20.622931 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:10:20.622675 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-phfqm\" (UniqueName: \"kubernetes.io/projected/347bd19f-7728-47be-986d-337d0f0df246-kube-api-access-phfqm\") pod \"node-resolver-j74qd\" (UID: \"347bd19f-7728-47be-986d-337d0f0df246\") " pod="openshift-dns/node-resolver-j74qd" Jul 30 15:10:20.622931 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:10:20.622706 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/7fb7dc92-4394-4433-9f1d-0f2c237136fe-host-run-ovn-kubernetes\") pod \"ovnkube-node-6mb29\" (UID: \"7fb7dc92-4394-4433-9f1d-0f2c237136fe\") " pod="openshift-ovn-kubernetes/ovnkube-node-6mb29" Jul 30 15:10:20.622931 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:10:20.622716 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/3caae76d-69ba-4860-96fc-cca51b9ba7db-etc-sysconfig\") pod \"tuned-fxr7d\" (UID: \"3caae76d-69ba-4860-96fc-cca51b9ba7db\") " pod="openshift-cluster-node-tuning-operator/tuned-fxr7d" Jul 30 15:10:20.622931 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:10:20.622733 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/7fb7dc92-4394-4433-9f1d-0f2c237136fe-ovn-node-metrics-cert\") pod \"ovnkube-node-6mb29\" (UID: \"7fb7dc92-4394-4433-9f1d-0f2c237136fe\") " pod="openshift-ovn-kubernetes/ovnkube-node-6mb29" Jul 30 15:10:20.622931 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:10:20.622757 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/4b006058-0163-4e82-9a1f-979b182ddbf1-host\") pod \"node-ca-ktmgn\" (UID: \"4b006058-0163-4e82-9a1f-979b182ddbf1\") " pod="openshift-image-registry/node-ca-ktmgn" Jul 30 15:10:20.622931 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:10:20.622782 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5mt5l\" (UniqueName: \"kubernetes.io/projected/4b006058-0163-4e82-9a1f-979b182ddbf1-kube-api-access-5mt5l\") pod \"node-ca-ktmgn\" (UID: \"4b006058-0163-4e82-9a1f-979b182ddbf1\") " pod="openshift-image-registry/node-ca-ktmgn" Jul 30 15:10:20.622931 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:10:20.622839 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zc7nr\" (UniqueName: \"kubernetes.io/projected/57353464-4671-4d1f-b2b5-473d618d461d-kube-api-access-zc7nr\") pod \"aws-ebs-csi-driver-node-47m5x\" (UID: \"57353464-4671-4d1f-b2b5-473d618d461d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-47m5x" Jul 30 15:10:20.622931 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:10:20.622865 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/3caae76d-69ba-4860-96fc-cca51b9ba7db-var-lib-kubelet\") pod \"tuned-fxr7d\" (UID: \"3caae76d-69ba-4860-96fc-cca51b9ba7db\") " pod="openshift-cluster-node-tuning-operator/tuned-fxr7d" Jul 30 15:10:20.622931 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:10:20.622888 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/7fb7dc92-4394-4433-9f1d-0f2c237136fe-run-ovn\") pod \"ovnkube-node-6mb29\" (UID: \"7fb7dc92-4394-4433-9f1d-0f2c237136fe\") " pod="openshift-ovn-kubernetes/ovnkube-node-6mb29" Jul 30 15:10:20.622931 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:10:20.622911 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/3caae76d-69ba-4860-96fc-cca51b9ba7db-host\") pod \"tuned-fxr7d\" (UID: \"3caae76d-69ba-4860-96fc-cca51b9ba7db\") " pod="openshift-cluster-node-tuning-operator/tuned-fxr7d" Jul 30 15:10:20.622931 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:10:20.622922 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/7fb7dc92-4394-4433-9f1d-0f2c237136fe-node-log\") pod \"ovnkube-node-6mb29\" (UID: \"7fb7dc92-4394-4433-9f1d-0f2c237136fe\") " pod="openshift-ovn-kubernetes/ovnkube-node-6mb29" Jul 30 15:10:20.623757 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:10:20.622949 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/3caae76d-69ba-4860-96fc-cca51b9ba7db-etc-sysctl-d\") pod \"tuned-fxr7d\" (UID: \"3caae76d-69ba-4860-96fc-cca51b9ba7db\") " pod="openshift-cluster-node-tuning-operator/tuned-fxr7d" Jul 30 15:10:20.623757 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:10:20.622969 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/3caae76d-69ba-4860-96fc-cca51b9ba7db-var-lib-kubelet\") pod \"tuned-fxr7d\" (UID: \"3caae76d-69ba-4860-96fc-cca51b9ba7db\") " pod="openshift-cluster-node-tuning-operator/tuned-fxr7d" Jul 30 15:10:20.623757 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:10:20.622987 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/7fb7dc92-4394-4433-9f1d-0f2c237136fe-host-run-netns\") pod \"ovnkube-node-6mb29\" (UID: \"7fb7dc92-4394-4433-9f1d-0f2c237136fe\") " pod="openshift-ovn-kubernetes/ovnkube-node-6mb29" Jul 30 15:10:20.623757 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:10:20.623013 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/7fb7dc92-4394-4433-9f1d-0f2c237136fe-ovnkube-config\") pod \"ovnkube-node-6mb29\" (UID: \"7fb7dc92-4394-4433-9f1d-0f2c237136fe\") " pod="openshift-ovn-kubernetes/ovnkube-node-6mb29" Jul 30 15:10:20.623757 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:10:20.623039 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/3caae76d-69ba-4860-96fc-cca51b9ba7db-run\") pod \"tuned-fxr7d\" (UID: \"3caae76d-69ba-4860-96fc-cca51b9ba7db\") " pod="openshift-cluster-node-tuning-operator/tuned-fxr7d" Jul 30 15:10:20.623757 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:10:20.623063 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/7fb7dc92-4394-4433-9f1d-0f2c237136fe-var-lib-openvswitch\") pod \"ovnkube-node-6mb29\" (UID: \"7fb7dc92-4394-4433-9f1d-0f2c237136fe\") " pod="openshift-ovn-kubernetes/ovnkube-node-6mb29" Jul 30 15:10:20.623757 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:10:20.623088 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/7fb7dc92-4394-4433-9f1d-0f2c237136fe-host-cni-bin\") pod \"ovnkube-node-6mb29\" (UID: \"7fb7dc92-4394-4433-9f1d-0f2c237136fe\") " pod="openshift-ovn-kubernetes/ovnkube-node-6mb29" Jul 30 15:10:20.623757 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:10:20.623123 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/7fb7dc92-4394-4433-9f1d-0f2c237136fe-host-run-ovn-kubernetes\") pod \"ovnkube-node-6mb29\" (UID: \"7fb7dc92-4394-4433-9f1d-0f2c237136fe\") " pod="openshift-ovn-kubernetes/ovnkube-node-6mb29" Jul 30 15:10:20.623757 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:10:20.623132 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/7fb7dc92-4394-4433-9f1d-0f2c237136fe-ovnkube-script-lib\") pod \"ovnkube-node-6mb29\" (UID: \"7fb7dc92-4394-4433-9f1d-0f2c237136fe\") " pod="openshift-ovn-kubernetes/ovnkube-node-6mb29" Jul 30 15:10:20.623757 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:10:20.623174 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7pzx7\" (UniqueName: \"kubernetes.io/projected/1cd62f00-3530-42c3-9aab-499796a7c906-kube-api-access-7pzx7\") pod \"network-check-target-q858z\" (UID: \"1cd62f00-3530-42c3-9aab-499796a7c906\") " pod="openshift-network-diagnostics/network-check-target-q858z" Jul 30 15:10:20.623757 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:10:20.623193 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/7fb7dc92-4394-4433-9f1d-0f2c237136fe-run-systemd\") pod \"ovnkube-node-6mb29\" (UID: \"7fb7dc92-4394-4433-9f1d-0f2c237136fe\") " pod="openshift-ovn-kubernetes/ovnkube-node-6mb29" Jul 30 15:10:20.623757 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:10:20.623209 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/7fb7dc92-4394-4433-9f1d-0f2c237136fe-host-run-netns\") pod \"ovnkube-node-6mb29\" (UID: \"7fb7dc92-4394-4433-9f1d-0f2c237136fe\") " pod="openshift-ovn-kubernetes/ovnkube-node-6mb29" Jul 30 15:10:20.623757 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:10:20.623240 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/3caae76d-69ba-4860-96fc-cca51b9ba7db-run\") pod \"tuned-fxr7d\" (UID: \"3caae76d-69ba-4860-96fc-cca51b9ba7db\") " pod="openshift-cluster-node-tuning-operator/tuned-fxr7d" Jul 30 15:10:20.623757 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:10:20.623263 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/7fb7dc92-4394-4433-9f1d-0f2c237136fe-run-ovn\") pod \"ovnkube-node-6mb29\" (UID: \"7fb7dc92-4394-4433-9f1d-0f2c237136fe\") " pod="openshift-ovn-kubernetes/ovnkube-node-6mb29" Jul 30 15:10:20.623757 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:10:20.623287 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/7fb7dc92-4394-4433-9f1d-0f2c237136fe-run-systemd\") pod \"ovnkube-node-6mb29\" (UID: \"7fb7dc92-4394-4433-9f1d-0f2c237136fe\") " pod="openshift-ovn-kubernetes/ovnkube-node-6mb29" Jul 30 15:10:20.623757 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:10:20.623277 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/4b006058-0163-4e82-9a1f-979b182ddbf1-host\") pod \"node-ca-ktmgn\" (UID: \"4b006058-0163-4e82-9a1f-979b182ddbf1\") " pod="openshift-image-registry/node-ca-ktmgn" Jul 30 15:10:20.623757 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:10:20.623378 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/3caae76d-69ba-4860-96fc-cca51b9ba7db-host\") pod \"tuned-fxr7d\" (UID: \"3caae76d-69ba-4860-96fc-cca51b9ba7db\") " pod="openshift-cluster-node-tuning-operator/tuned-fxr7d" Jul 30 15:10:20.623757 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:10:20.623416 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/7fb7dc92-4394-4433-9f1d-0f2c237136fe-run-openvswitch\") pod \"ovnkube-node-6mb29\" (UID: \"7fb7dc92-4394-4433-9f1d-0f2c237136fe\") " pod="openshift-ovn-kubernetes/ovnkube-node-6mb29" Jul 30 15:10:20.624626 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:10:20.623444 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/57353464-4671-4d1f-b2b5-473d618d461d-socket-dir\") pod \"aws-ebs-csi-driver-node-47m5x\" (UID: \"57353464-4671-4d1f-b2b5-473d618d461d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-47m5x" Jul 30 15:10:20.624626 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:10:20.623469 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/5be52bf9-46b0-4e79-9d34-9b32b8b891b4-host-slash\") pod \"iptables-alerter-4g9qs\" (UID: \"5be52bf9-46b0-4e79-9d34-9b32b8b891b4\") " pod="openshift-network-operator/iptables-alerter-4g9qs" Jul 30 15:10:20.624626 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:10:20.623484 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/3caae76d-69ba-4860-96fc-cca51b9ba7db-etc-sysctl-d\") pod \"tuned-fxr7d\" (UID: \"3caae76d-69ba-4860-96fc-cca51b9ba7db\") " pod="openshift-cluster-node-tuning-operator/tuned-fxr7d" Jul 30 15:10:20.624626 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:10:20.623494 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/7fb7dc92-4394-4433-9f1d-0f2c237136fe-run-openvswitch\") pod \"ovnkube-node-6mb29\" (UID: \"7fb7dc92-4394-4433-9f1d-0f2c237136fe\") " pod="openshift-ovn-kubernetes/ovnkube-node-6mb29" Jul 30 15:10:20.624626 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:10:20.623493 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cdcss\" (UniqueName: \"kubernetes.io/projected/5be52bf9-46b0-4e79-9d34-9b32b8b891b4-kube-api-access-cdcss\") pod \"iptables-alerter-4g9qs\" (UID: \"5be52bf9-46b0-4e79-9d34-9b32b8b891b4\") " pod="openshift-network-operator/iptables-alerter-4g9qs" Jul 30 15:10:20.624626 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:10:20.623549 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/3caae76d-69ba-4860-96fc-cca51b9ba7db-etc-modprobe-d\") pod \"tuned-fxr7d\" (UID: \"3caae76d-69ba-4860-96fc-cca51b9ba7db\") " pod="openshift-cluster-node-tuning-operator/tuned-fxr7d" Jul 30 15:10:20.624626 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:10:20.623574 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/7fb7dc92-4394-4433-9f1d-0f2c237136fe-systemd-units\") pod \"ovnkube-node-6mb29\" (UID: \"7fb7dc92-4394-4433-9f1d-0f2c237136fe\") " pod="openshift-ovn-kubernetes/ovnkube-node-6mb29" Jul 30 15:10:20.624626 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:10:20.623598 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-d6f4c\" (UniqueName: \"kubernetes.io/projected/7fb7dc92-4394-4433-9f1d-0f2c237136fe-kube-api-access-d6f4c\") pod \"ovnkube-node-6mb29\" (UID: \"7fb7dc92-4394-4433-9f1d-0f2c237136fe\") " pod="openshift-ovn-kubernetes/ovnkube-node-6mb29" Jul 30 15:10:20.624626 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:10:20.623615 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/3caae76d-69ba-4860-96fc-cca51b9ba7db-lib-modules\") pod \"tuned-fxr7d\" (UID: \"3caae76d-69ba-4860-96fc-cca51b9ba7db\") " pod="openshift-cluster-node-tuning-operator/tuned-fxr7d" Jul 30 15:10:20.624626 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:10:20.623630 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-c9j5g\" (UniqueName: \"kubernetes.io/projected/3caae76d-69ba-4860-96fc-cca51b9ba7db-kube-api-access-c9j5g\") pod \"tuned-fxr7d\" (UID: \"3caae76d-69ba-4860-96fc-cca51b9ba7db\") " pod="openshift-cluster-node-tuning-operator/tuned-fxr7d" Jul 30 15:10:20.624626 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:10:20.623659 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/7fb7dc92-4394-4433-9f1d-0f2c237136fe-log-socket\") pod \"ovnkube-node-6mb29\" (UID: \"7fb7dc92-4394-4433-9f1d-0f2c237136fe\") " pod="openshift-ovn-kubernetes/ovnkube-node-6mb29" Jul 30 15:10:20.624626 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:10:20.623674 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/347bd19f-7728-47be-986d-337d0f0df246-hosts-file\") pod \"node-resolver-j74qd\" (UID: \"347bd19f-7728-47be-986d-337d0f0df246\") " pod="openshift-dns/node-resolver-j74qd" Jul 30 15:10:20.624626 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:10:20.623689 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/347bd19f-7728-47be-986d-337d0f0df246-tmp-dir\") pod \"node-resolver-j74qd\" (UID: \"347bd19f-7728-47be-986d-337d0f0df246\") " pod="openshift-dns/node-resolver-j74qd" Jul 30 15:10:20.624626 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:10:20.623709 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/4b006058-0163-4e82-9a1f-979b182ddbf1-serviceca\") pod \"node-ca-ktmgn\" (UID: \"4b006058-0163-4e82-9a1f-979b182ddbf1\") " pod="openshift-image-registry/node-ca-ktmgn" Jul 30 15:10:20.624626 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:10:20.623718 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/5be52bf9-46b0-4e79-9d34-9b32b8b891b4-host-slash\") pod \"iptables-alerter-4g9qs\" (UID: \"5be52bf9-46b0-4e79-9d34-9b32b8b891b4\") " pod="openshift-network-operator/iptables-alerter-4g9qs" Jul 30 15:10:20.624626 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:10:20.623735 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/3caae76d-69ba-4860-96fc-cca51b9ba7db-etc-systemd\") pod \"tuned-fxr7d\" (UID: \"3caae76d-69ba-4860-96fc-cca51b9ba7db\") " pod="openshift-cluster-node-tuning-operator/tuned-fxr7d" Jul 30 15:10:20.624626 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:10:20.623754 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/7fb7dc92-4394-4433-9f1d-0f2c237136fe-env-overrides\") pod \"ovnkube-node-6mb29\" (UID: \"7fb7dc92-4394-4433-9f1d-0f2c237136fe\") " pod="openshift-ovn-kubernetes/ovnkube-node-6mb29" Jul 30 15:10:20.625394 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:10:20.623770 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/3caae76d-69ba-4860-96fc-cca51b9ba7db-tmp\") pod \"tuned-fxr7d\" (UID: \"3caae76d-69ba-4860-96fc-cca51b9ba7db\") " pod="openshift-cluster-node-tuning-operator/tuned-fxr7d" Jul 30 15:10:20.625394 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:10:20.623774 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/7fb7dc92-4394-4433-9f1d-0f2c237136fe-var-lib-openvswitch\") pod \"ovnkube-node-6mb29\" (UID: \"7fb7dc92-4394-4433-9f1d-0f2c237136fe\") " pod="openshift-ovn-kubernetes/ovnkube-node-6mb29" Jul 30 15:10:20.625394 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:10:20.623774 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/7fb7dc92-4394-4433-9f1d-0f2c237136fe-ovnkube-script-lib\") pod \"ovnkube-node-6mb29\" (UID: \"7fb7dc92-4394-4433-9f1d-0f2c237136fe\") " pod="openshift-ovn-kubernetes/ovnkube-node-6mb29" Jul 30 15:10:20.625394 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:10:20.623786 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/3caae76d-69ba-4860-96fc-cca51b9ba7db-etc-sysctl-conf\") pod \"tuned-fxr7d\" (UID: \"3caae76d-69ba-4860-96fc-cca51b9ba7db\") " pod="openshift-cluster-node-tuning-operator/tuned-fxr7d" Jul 30 15:10:20.625394 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:10:20.623827 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/7fb7dc92-4394-4433-9f1d-0f2c237136fe-host-cni-bin\") pod \"ovnkube-node-6mb29\" (UID: \"7fb7dc92-4394-4433-9f1d-0f2c237136fe\") " pod="openshift-ovn-kubernetes/ovnkube-node-6mb29" Jul 30 15:10:20.625394 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:10:20.623828 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/7fb7dc92-4394-4433-9f1d-0f2c237136fe-host-cni-netd\") pod \"ovnkube-node-6mb29\" (UID: \"7fb7dc92-4394-4433-9f1d-0f2c237136fe\") " pod="openshift-ovn-kubernetes/ovnkube-node-6mb29" Jul 30 15:10:20.625394 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:10:20.623857 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/7fb7dc92-4394-4433-9f1d-0f2c237136fe-host-cni-netd\") pod \"ovnkube-node-6mb29\" (UID: \"7fb7dc92-4394-4433-9f1d-0f2c237136fe\") " pod="openshift-ovn-kubernetes/ovnkube-node-6mb29" Jul 30 15:10:20.625394 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:10:20.623867 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/3caae76d-69ba-4860-96fc-cca51b9ba7db-etc-sysctl-conf\") pod \"tuned-fxr7d\" (UID: \"3caae76d-69ba-4860-96fc-cca51b9ba7db\") " pod="openshift-cluster-node-tuning-operator/tuned-fxr7d" Jul 30 15:10:20.625394 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:10:20.623869 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/7fb7dc92-4394-4433-9f1d-0f2c237136fe-ovnkube-config\") pod \"ovnkube-node-6mb29\" (UID: \"7fb7dc92-4394-4433-9f1d-0f2c237136fe\") " pod="openshift-ovn-kubernetes/ovnkube-node-6mb29" Jul 30 15:10:20.625394 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:10:20.623894 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/7fb7dc92-4394-4433-9f1d-0f2c237136fe-log-socket\") pod \"ovnkube-node-6mb29\" (UID: \"7fb7dc92-4394-4433-9f1d-0f2c237136fe\") " pod="openshift-ovn-kubernetes/ovnkube-node-6mb29" Jul 30 15:10:20.625394 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:10:20.623912 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/57353464-4671-4d1f-b2b5-473d618d461d-device-dir\") pod \"aws-ebs-csi-driver-node-47m5x\" (UID: \"57353464-4671-4d1f-b2b5-473d618d461d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-47m5x" Jul 30 15:10:20.625394 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:10:20.623899 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/347bd19f-7728-47be-986d-337d0f0df246-hosts-file\") pod \"node-resolver-j74qd\" (UID: \"347bd19f-7728-47be-986d-337d0f0df246\") " pod="openshift-dns/node-resolver-j74qd" Jul 30 15:10:20.625394 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:10:20.623874 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/57353464-4671-4d1f-b2b5-473d618d461d-device-dir\") pod \"aws-ebs-csi-driver-node-47m5x\" (UID: \"57353464-4671-4d1f-b2b5-473d618d461d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-47m5x" Jul 30 15:10:20.625394 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:10:20.624147 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/3caae76d-69ba-4860-96fc-cca51b9ba7db-etc-modprobe-d\") pod \"tuned-fxr7d\" (UID: \"3caae76d-69ba-4860-96fc-cca51b9ba7db\") " pod="openshift-cluster-node-tuning-operator/tuned-fxr7d" Jul 30 15:10:20.625394 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:10:20.624184 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/5be52bf9-46b0-4e79-9d34-9b32b8b891b4-iptables-alerter-script\") pod \"iptables-alerter-4g9qs\" (UID: \"5be52bf9-46b0-4e79-9d34-9b32b8b891b4\") " pod="openshift-network-operator/iptables-alerter-4g9qs" Jul 30 15:10:20.625394 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:10:20.624208 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/3caae76d-69ba-4860-96fc-cca51b9ba7db-etc-tuned\") pod \"tuned-fxr7d\" (UID: \"3caae76d-69ba-4860-96fc-cca51b9ba7db\") " pod="openshift-cluster-node-tuning-operator/tuned-fxr7d" Jul 30 15:10:20.625394 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:10:20.624216 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/7fb7dc92-4394-4433-9f1d-0f2c237136fe-env-overrides\") pod \"ovnkube-node-6mb29\" (UID: \"7fb7dc92-4394-4433-9f1d-0f2c237136fe\") " pod="openshift-ovn-kubernetes/ovnkube-node-6mb29" Jul 30 15:10:20.626217 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:10:20.624235 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/57353464-4671-4d1f-b2b5-473d618d461d-socket-dir\") pod \"aws-ebs-csi-driver-node-47m5x\" (UID: \"57353464-4671-4d1f-b2b5-473d618d461d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-47m5x" Jul 30 15:10:20.626217 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:10:20.624247 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/7fb7dc92-4394-4433-9f1d-0f2c237136fe-etc-openvswitch\") pod \"ovnkube-node-6mb29\" (UID: \"7fb7dc92-4394-4433-9f1d-0f2c237136fe\") " pod="openshift-ovn-kubernetes/ovnkube-node-6mb29" Jul 30 15:10:20.626217 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:10:20.624205 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/3caae76d-69ba-4860-96fc-cca51b9ba7db-lib-modules\") pod \"tuned-fxr7d\" (UID: \"3caae76d-69ba-4860-96fc-cca51b9ba7db\") " pod="openshift-cluster-node-tuning-operator/tuned-fxr7d" Jul 30 15:10:20.626217 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:10:20.624261 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/7fb7dc92-4394-4433-9f1d-0f2c237136fe-systemd-units\") pod \"ovnkube-node-6mb29\" (UID: \"7fb7dc92-4394-4433-9f1d-0f2c237136fe\") " pod="openshift-ovn-kubernetes/ovnkube-node-6mb29" Jul 30 15:10:20.626217 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:10:20.624294 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/a2197c57-2dc6-410e-9910-a045dedcb8d9-konnectivity-ca\") pod \"konnectivity-agent-7j468\" (UID: \"a2197c57-2dc6-410e-9910-a045dedcb8d9\") " pod="kube-system/konnectivity-agent-7j468" Jul 30 15:10:20.626217 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:10:20.624321 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/4b006058-0163-4e82-9a1f-979b182ddbf1-serviceca\") pod \"node-ca-ktmgn\" (UID: \"4b006058-0163-4e82-9a1f-979b182ddbf1\") " pod="openshift-image-registry/node-ca-ktmgn" Jul 30 15:10:20.626217 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:10:20.624413 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/3caae76d-69ba-4860-96fc-cca51b9ba7db-etc-systemd\") pod \"tuned-fxr7d\" (UID: \"3caae76d-69ba-4860-96fc-cca51b9ba7db\") " pod="openshift-cluster-node-tuning-operator/tuned-fxr7d" Jul 30 15:10:20.626217 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:10:20.624474 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/57353464-4671-4d1f-b2b5-473d618d461d-kubelet-dir\") pod \"aws-ebs-csi-driver-node-47m5x\" (UID: \"57353464-4671-4d1f-b2b5-473d618d461d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-47m5x" Jul 30 15:10:20.626217 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:10:20.624478 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/347bd19f-7728-47be-986d-337d0f0df246-tmp-dir\") pod \"node-resolver-j74qd\" (UID: \"347bd19f-7728-47be-986d-337d0f0df246\") " pod="openshift-dns/node-resolver-j74qd" Jul 30 15:10:20.626217 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:10:20.624605 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/57353464-4671-4d1f-b2b5-473d618d461d-kubelet-dir\") pod \"aws-ebs-csi-driver-node-47m5x\" (UID: \"57353464-4671-4d1f-b2b5-473d618d461d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-47m5x" Jul 30 15:10:20.626217 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:10:20.624642 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/7fb7dc92-4394-4433-9f1d-0f2c237136fe-etc-openvswitch\") pod \"ovnkube-node-6mb29\" (UID: \"7fb7dc92-4394-4433-9f1d-0f2c237136fe\") " pod="openshift-ovn-kubernetes/ovnkube-node-6mb29" Jul 30 15:10:20.626217 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:10:20.624973 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/5be52bf9-46b0-4e79-9d34-9b32b8b891b4-iptables-alerter-script\") pod \"iptables-alerter-4g9qs\" (UID: \"5be52bf9-46b0-4e79-9d34-9b32b8b891b4\") " pod="openshift-network-operator/iptables-alerter-4g9qs" Jul 30 15:10:20.626217 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:10:20.624987 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/a2197c57-2dc6-410e-9910-a045dedcb8d9-konnectivity-ca\") pod \"konnectivity-agent-7j468\" (UID: \"a2197c57-2dc6-410e-9910-a045dedcb8d9\") " pod="kube-system/konnectivity-agent-7j468" Jul 30 15:10:20.626217 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:10:20.625619 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/a2197c57-2dc6-410e-9910-a045dedcb8d9-agent-certs\") pod \"konnectivity-agent-7j468\" (UID: \"a2197c57-2dc6-410e-9910-a045dedcb8d9\") " pod="kube-system/konnectivity-agent-7j468" Jul 30 15:10:20.626217 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:10:20.625795 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/7fb7dc92-4394-4433-9f1d-0f2c237136fe-ovn-node-metrics-cert\") pod \"ovnkube-node-6mb29\" (UID: \"7fb7dc92-4394-4433-9f1d-0f2c237136fe\") " pod="openshift-ovn-kubernetes/ovnkube-node-6mb29" Jul 30 15:10:20.626831 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:10:20.626423 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/3caae76d-69ba-4860-96fc-cca51b9ba7db-etc-tuned\") pod \"tuned-fxr7d\" (UID: \"3caae76d-69ba-4860-96fc-cca51b9ba7db\") " pod="openshift-cluster-node-tuning-operator/tuned-fxr7d" Jul 30 15:10:20.626831 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:10:20.626804 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/3caae76d-69ba-4860-96fc-cca51b9ba7db-tmp\") pod \"tuned-fxr7d\" (UID: \"3caae76d-69ba-4860-96fc-cca51b9ba7db\") " pod="openshift-cluster-node-tuning-operator/tuned-fxr7d" Jul 30 15:10:20.634234 ip-10-0-133-210 kubenswrapper[2576]: E0730 15:10:20.634209 2576 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 30 15:10:20.634382 ip-10-0-133-210 kubenswrapper[2576]: E0730 15:10:20.634237 2576 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 30 15:10:20.634382 ip-10-0-133-210 kubenswrapper[2576]: E0730 15:10:20.634250 2576 projected.go:196] Error preparing data for projected volume kube-api-access-7pzx7 for pod openshift-network-diagnostics/network-check-target-q858z: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 30 15:10:20.634382 ip-10-0-133-210 kubenswrapper[2576]: E0730 15:10:20.634307 2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/1cd62f00-3530-42c3-9aab-499796a7c906-kube-api-access-7pzx7 podName:1cd62f00-3530-42c3-9aab-499796a7c906 nodeName:}" failed. No retries permitted until 2026-07-30 15:10:21.134289233 +0000 UTC m=+3.060041855 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-7pzx7" (UniqueName: "kubernetes.io/projected/1cd62f00-3530-42c3-9aab-499796a7c906-kube-api-access-7pzx7") pod "network-check-target-q858z" (UID: "1cd62f00-3530-42c3-9aab-499796a7c906") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 30 15:10:20.635569 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:10:20.635543 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-phfqm\" (UniqueName: \"kubernetes.io/projected/347bd19f-7728-47be-986d-337d0f0df246-kube-api-access-phfqm\") pod \"node-resolver-j74qd\" (UID: \"347bd19f-7728-47be-986d-337d0f0df246\") " pod="openshift-dns/node-resolver-j74qd" Jul 30 15:10:20.636242 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:10:20.636222 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-cdcss\" (UniqueName: \"kubernetes.io/projected/5be52bf9-46b0-4e79-9d34-9b32b8b891b4-kube-api-access-cdcss\") pod \"iptables-alerter-4g9qs\" (UID: \"5be52bf9-46b0-4e79-9d34-9b32b8b891b4\") " pod="openshift-network-operator/iptables-alerter-4g9qs" Jul 30 15:10:20.636833 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:10:20.636811 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zc7nr\" (UniqueName: \"kubernetes.io/projected/57353464-4671-4d1f-b2b5-473d618d461d-kube-api-access-zc7nr\") pod \"aws-ebs-csi-driver-node-47m5x\" (UID: \"57353464-4671-4d1f-b2b5-473d618d461d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-47m5x" Jul 30 15:10:20.637244 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:10:20.637196 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-d6f4c\" (UniqueName: \"kubernetes.io/projected/7fb7dc92-4394-4433-9f1d-0f2c237136fe-kube-api-access-d6f4c\") pod \"ovnkube-node-6mb29\" (UID: \"7fb7dc92-4394-4433-9f1d-0f2c237136fe\") " pod="openshift-ovn-kubernetes/ovnkube-node-6mb29" Jul 30 15:10:20.637310 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:10:20.637285 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-c9j5g\" (UniqueName: \"kubernetes.io/projected/3caae76d-69ba-4860-96fc-cca51b9ba7db-kube-api-access-c9j5g\") pod \"tuned-fxr7d\" (UID: \"3caae76d-69ba-4860-96fc-cca51b9ba7db\") " pod="openshift-cluster-node-tuning-operator/tuned-fxr7d" Jul 30 15:10:20.637410 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:10:20.637386 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5mt5l\" (UniqueName: \"kubernetes.io/projected/4b006058-0163-4e82-9a1f-979b182ddbf1-kube-api-access-5mt5l\") pod \"node-ca-ktmgn\" (UID: \"4b006058-0163-4e82-9a1f-979b182ddbf1\") " pod="openshift-image-registry/node-ca-ktmgn" Jul 30 15:10:20.717849 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:10:20.717819 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-m2j4z" Jul 30 15:10:20.723963 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:10:20.723942 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-gb69c" Jul 30 15:10:20.733588 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:10:20.733567 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-47m5x" Jul 30 15:10:20.739202 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:10:20.739174 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-j74qd" Jul 30 15:10:20.745769 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:10:20.745751 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-4g9qs" Jul 30 15:10:20.752276 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:10:20.752258 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-fxr7d" Jul 30 15:10:20.758855 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:10:20.758837 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-6mb29" Jul 30 15:10:20.765460 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:10:20.765439 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-ktmgn" Jul 30 15:10:20.771055 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:10:20.771033 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-7j468" Jul 30 15:10:21.028133 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:10:21.028034 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/e76c844a-a34c-4587-93a7-023d511a0b9f-original-pull-secret\") pod \"global-pull-secret-syncer-qvn96\" (UID: \"e76c844a-a34c-4587-93a7-023d511a0b9f\") " pod="kube-system/global-pull-secret-syncer-qvn96" Jul 30 15:10:21.028133 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:10:21.028075 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/32b9046b-bd5c-4f23-b832-6fd0bad1c8d6-metrics-certs\") pod \"network-metrics-daemon-dbmt8\" (UID: \"32b9046b-bd5c-4f23-b832-6fd0bad1c8d6\") " pod="openshift-multus/network-metrics-daemon-dbmt8" Jul 30 15:10:21.028311 ip-10-0-133-210 kubenswrapper[2576]: E0730 15:10:21.028215 2576 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 30 15:10:21.028311 ip-10-0-133-210 kubenswrapper[2576]: E0730 15:10:21.028282 2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e76c844a-a34c-4587-93a7-023d511a0b9f-original-pull-secret podName:e76c844a-a34c-4587-93a7-023d511a0b9f nodeName:}" failed. No retries permitted until 2026-07-30 15:10:22.028266596 +0000 UTC m=+3.954019220 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/e76c844a-a34c-4587-93a7-023d511a0b9f-original-pull-secret") pod "global-pull-secret-syncer-qvn96" (UID: "e76c844a-a34c-4587-93a7-023d511a0b9f") : object "kube-system"/"original-pull-secret" not registered Jul 30 15:10:21.028403 ip-10-0-133-210 kubenswrapper[2576]: E0730 15:10:21.028215 2576 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 30 15:10:21.028403 ip-10-0-133-210 kubenswrapper[2576]: E0730 15:10:21.028387 2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/32b9046b-bd5c-4f23-b832-6fd0bad1c8d6-metrics-certs podName:32b9046b-bd5c-4f23-b832-6fd0bad1c8d6 nodeName:}" failed. No retries permitted until 2026-07-30 15:10:22.028367142 +0000 UTC m=+3.954119780 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/32b9046b-bd5c-4f23-b832-6fd0bad1c8d6-metrics-certs") pod "network-metrics-daemon-dbmt8" (UID: "32b9046b-bd5c-4f23-b832-6fd0bad1c8d6") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 30 15:10:21.229138 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:10:21.229097 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7pzx7\" (UniqueName: \"kubernetes.io/projected/1cd62f00-3530-42c3-9aab-499796a7c906-kube-api-access-7pzx7\") pod \"network-check-target-q858z\" (UID: \"1cd62f00-3530-42c3-9aab-499796a7c906\") " pod="openshift-network-diagnostics/network-check-target-q858z" Jul 30 15:10:21.229323 ip-10-0-133-210 kubenswrapper[2576]: E0730 15:10:21.229234 2576 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 30 15:10:21.229323 ip-10-0-133-210 kubenswrapper[2576]: E0730 15:10:21.229248 2576 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 30 15:10:21.229323 ip-10-0-133-210 kubenswrapper[2576]: E0730 15:10:21.229256 2576 projected.go:196] Error preparing data for projected volume kube-api-access-7pzx7 for pod openshift-network-diagnostics/network-check-target-q858z: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 30 15:10:21.229323 ip-10-0-133-210 kubenswrapper[2576]: E0730 15:10:21.229309 2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/1cd62f00-3530-42c3-9aab-499796a7c906-kube-api-access-7pzx7 podName:1cd62f00-3530-42c3-9aab-499796a7c906 nodeName:}" failed. No retries permitted until 2026-07-30 15:10:22.229291434 +0000 UTC m=+4.155044077 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-7pzx7" (UniqueName: "kubernetes.io/projected/1cd62f00-3530-42c3-9aab-499796a7c906-kube-api-access-7pzx7") pod "network-check-target-q858z" (UID: "1cd62f00-3530-42c3-9aab-499796a7c906") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 30 15:10:21.300932 ip-10-0-133-210 kubenswrapper[2576]: W0730 15:10:21.300887 2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3caae76d_69ba_4860_96fc_cca51b9ba7db.slice/crio-367a3902c88e96218a4e7cbc471fe47034805f086389ae1305acb8c47ff8e611 WatchSource:0}: Error finding container 367a3902c88e96218a4e7cbc471fe47034805f086389ae1305acb8c47ff8e611: Status 404 returned error can't find the container with id 367a3902c88e96218a4e7cbc471fe47034805f086389ae1305acb8c47ff8e611 Jul 30 15:10:21.305194 ip-10-0-133-210 kubenswrapper[2576]: W0730 15:10:21.305168 2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4b006058_0163_4e82_9a1f_979b182ddbf1.slice/crio-ad5d43c299f09ff8341168697385b117ac77fefecef0ec6159e1fe0729606661 WatchSource:0}: Error finding container ad5d43c299f09ff8341168697385b117ac77fefecef0ec6159e1fe0729606661: Status 404 returned error can't find the container with id ad5d43c299f09ff8341168697385b117ac77fefecef0ec6159e1fe0729606661 Jul 30 15:10:21.306343 ip-10-0-133-210 kubenswrapper[2576]: W0730 15:10:21.306318 2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5be52bf9_46b0_4e79_9d34_9b32b8b891b4.slice/crio-c36f0766610e93eac6649125bc368c25978ef5f9104bee23c56ea6e370a2e5dd WatchSource:0}: Error finding container c36f0766610e93eac6649125bc368c25978ef5f9104bee23c56ea6e370a2e5dd: Status 404 returned error can't find the container with id c36f0766610e93eac6649125bc368c25978ef5f9104bee23c56ea6e370a2e5dd Jul 30 15:10:21.307845 ip-10-0-133-210 kubenswrapper[2576]: W0730 15:10:21.307697 2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7fb7dc92_4394_4433_9f1d_0f2c237136fe.slice/crio-25a618c0ed4b5c9152c9628accec5775d2817cbcc99e5d692613cf0f940ebcbb WatchSource:0}: Error finding container 25a618c0ed4b5c9152c9628accec5775d2817cbcc99e5d692613cf0f940ebcbb: Status 404 returned error can't find the container with id 25a618c0ed4b5c9152c9628accec5775d2817cbcc99e5d692613cf0f940ebcbb Jul 30 15:10:21.309544 ip-10-0-133-210 kubenswrapper[2576]: W0730 15:10:21.309521 2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod57353464_4671_4d1f_b2b5_473d618d461d.slice/crio-9159c5c9f91a5f969603aedb2132173abf71fd89b1c4b854d19ba0a882ba4a27 WatchSource:0}: Error finding container 9159c5c9f91a5f969603aedb2132173abf71fd89b1c4b854d19ba0a882ba4a27: Status 404 returned error can't find the container with id 9159c5c9f91a5f969603aedb2132173abf71fd89b1c4b854d19ba0a882ba4a27 Jul 30 15:10:21.310577 ip-10-0-133-210 kubenswrapper[2576]: W0730 15:10:21.310534 2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf62011d0_f5be_413a_93a2_8a7a98b67130.slice/crio-6ec10db325265f9b98361daa2aaa64e9e054ee851009a4506637aa451e26b42c WatchSource:0}: Error finding container 6ec10db325265f9b98361daa2aaa64e9e054ee851009a4506637aa451e26b42c: Status 404 returned error can't find the container with id 6ec10db325265f9b98361daa2aaa64e9e054ee851009a4506637aa451e26b42c Jul 30 15:10:21.310937 ip-10-0-133-210 kubenswrapper[2576]: W0730 15:10:21.310834 2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod347bd19f_7728_47be_986d_337d0f0df246.slice/crio-5f4693ca6fa93cf342362ab102763022cfb0a55e6628fbe2df8096e595d20e6c WatchSource:0}: Error finding container 5f4693ca6fa93cf342362ab102763022cfb0a55e6628fbe2df8096e595d20e6c: Status 404 returned error can't find the container with id 5f4693ca6fa93cf342362ab102763022cfb0a55e6628fbe2df8096e595d20e6c Jul 30 15:10:21.311910 ip-10-0-133-210 kubenswrapper[2576]: W0730 15:10:21.311704 2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod23fd99c2_79a8_41eb_bb0f_92a363e32553.slice/crio-c1b8a8325d8eb0ff1044b0010863e6aa8c1429ed558c1a6ecf10a9a0954ee9bf WatchSource:0}: Error finding container c1b8a8325d8eb0ff1044b0010863e6aa8c1429ed558c1a6ecf10a9a0954ee9bf: Status 404 returned error can't find the container with id c1b8a8325d8eb0ff1044b0010863e6aa8c1429ed558c1a6ecf10a9a0954ee9bf Jul 30 15:10:21.312764 ip-10-0-133-210 kubenswrapper[2576]: W0730 15:10:21.312670 2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda2197c57_2dc6_410e_9910_a045dedcb8d9.slice/crio-2196988375c76ef4bc44ceb22130c773b7359da6a37daa06d4c1c4ba8741d6d3 WatchSource:0}: Error finding container 2196988375c76ef4bc44ceb22130c773b7359da6a37daa06d4c1c4ba8741d6d3: Status 404 returned error can't find the container with id 2196988375c76ef4bc44ceb22130c773b7359da6a37daa06d4c1c4ba8741d6d3 Jul 30 15:10:21.459441 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:10:21.459383 2576 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-29 15:05:20 +0000 UTC" deadline="2028-03-11 04:04:38.329545448 +0000 UTC" Jul 30 15:10:21.459441 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:10:21.459431 2576 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="14148h54m16.870116977s" Jul 30 15:10:21.607244 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:10:21.607158 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-dbmt8" Jul 30 15:10:21.607244 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:10:21.607203 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-qvn96" Jul 30 15:10:21.607429 ip-10-0-133-210 kubenswrapper[2576]: E0730 15:10:21.607283 2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-dbmt8" podUID="32b9046b-bd5c-4f23-b832-6fd0bad1c8d6" Jul 30 15:10:21.607429 ip-10-0-133-210 kubenswrapper[2576]: E0730 15:10:21.607397 2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-qvn96" podUID="e76c844a-a34c-4587-93a7-023d511a0b9f" Jul 30 15:10:21.613296 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:10:21.613273 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-47m5x" event={"ID":"57353464-4671-4d1f-b2b5-473d618d461d","Type":"ContainerStarted","Data":"9159c5c9f91a5f969603aedb2132173abf71fd89b1c4b854d19ba0a882ba4a27"} Jul 30 15:10:21.614246 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:10:21.614223 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-4g9qs" event={"ID":"5be52bf9-46b0-4e79-9d34-9b32b8b891b4","Type":"ContainerStarted","Data":"c36f0766610e93eac6649125bc368c25978ef5f9104bee23c56ea6e370a2e5dd"} Jul 30 15:10:21.615180 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:10:21.615161 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-ktmgn" event={"ID":"4b006058-0163-4e82-9a1f-979b182ddbf1","Type":"ContainerStarted","Data":"ad5d43c299f09ff8341168697385b117ac77fefecef0ec6159e1fe0729606661"} Jul 30 15:10:21.616312 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:10:21.616292 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-6mb29" event={"ID":"7fb7dc92-4394-4433-9f1d-0f2c237136fe","Type":"ContainerStarted","Data":"25a618c0ed4b5c9152c9628accec5775d2817cbcc99e5d692613cf0f940ebcbb"} Jul 30 15:10:21.617322 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:10:21.617303 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-fxr7d" event={"ID":"3caae76d-69ba-4860-96fc-cca51b9ba7db","Type":"ContainerStarted","Data":"367a3902c88e96218a4e7cbc471fe47034805f086389ae1305acb8c47ff8e611"} Jul 30 15:10:21.619011 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:10:21.618988 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-133-210.ec2.internal" event={"ID":"2da2e1115617abcb95d9273e2e0aad34","Type":"ContainerStarted","Data":"93689f04764b213648dbbd0f3c1b987560c07960a377d0c6e591135412a2125c"} Jul 30 15:10:21.620038 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:10:21.620020 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-7j468" event={"ID":"a2197c57-2dc6-410e-9910-a045dedcb8d9","Type":"ContainerStarted","Data":"2196988375c76ef4bc44ceb22130c773b7359da6a37daa06d4c1c4ba8741d6d3"} Jul 30 15:10:21.620935 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:10:21.620917 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-gb69c" event={"ID":"23fd99c2-79a8-41eb-bb0f-92a363e32553","Type":"ContainerStarted","Data":"c1b8a8325d8eb0ff1044b0010863e6aa8c1429ed558c1a6ecf10a9a0954ee9bf"} Jul 30 15:10:21.621969 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:10:21.621950 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-j74qd" event={"ID":"347bd19f-7728-47be-986d-337d0f0df246","Type":"ContainerStarted","Data":"5f4693ca6fa93cf342362ab102763022cfb0a55e6628fbe2df8096e595d20e6c"} Jul 30 15:10:21.622921 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:10:21.622903 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-m2j4z" event={"ID":"f62011d0-f5be-413a-93a2-8a7a98b67130","Type":"ContainerStarted","Data":"6ec10db325265f9b98361daa2aaa64e9e054ee851009a4506637aa451e26b42c"} Jul 30 15:10:22.015072 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:10:22.014990 2576 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160" Jul 30 15:10:22.034938 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:10:22.034897 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/e76c844a-a34c-4587-93a7-023d511a0b9f-original-pull-secret\") pod \"global-pull-secret-syncer-qvn96\" (UID: \"e76c844a-a34c-4587-93a7-023d511a0b9f\") " pod="kube-system/global-pull-secret-syncer-qvn96" Jul 30 15:10:22.035133 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:10:22.034954 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/32b9046b-bd5c-4f23-b832-6fd0bad1c8d6-metrics-certs\") pod \"network-metrics-daemon-dbmt8\" (UID: \"32b9046b-bd5c-4f23-b832-6fd0bad1c8d6\") " pod="openshift-multus/network-metrics-daemon-dbmt8" Jul 30 15:10:22.035206 ip-10-0-133-210 kubenswrapper[2576]: E0730 15:10:22.035133 2576 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 30 15:10:22.035206 ip-10-0-133-210 kubenswrapper[2576]: E0730 15:10:22.035196 2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/32b9046b-bd5c-4f23-b832-6fd0bad1c8d6-metrics-certs podName:32b9046b-bd5c-4f23-b832-6fd0bad1c8d6 nodeName:}" failed. No retries permitted until 2026-07-30 15:10:24.035177203 +0000 UTC m=+5.960929833 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/32b9046b-bd5c-4f23-b832-6fd0bad1c8d6-metrics-certs") pod "network-metrics-daemon-dbmt8" (UID: "32b9046b-bd5c-4f23-b832-6fd0bad1c8d6") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 30 15:10:22.035699 ip-10-0-133-210 kubenswrapper[2576]: E0730 15:10:22.035656 2576 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 30 15:10:22.035763 ip-10-0-133-210 kubenswrapper[2576]: E0730 15:10:22.035739 2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e76c844a-a34c-4587-93a7-023d511a0b9f-original-pull-secret podName:e76c844a-a34c-4587-93a7-023d511a0b9f nodeName:}" failed. No retries permitted until 2026-07-30 15:10:24.035722754 +0000 UTC m=+5.961475380 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/e76c844a-a34c-4587-93a7-023d511a0b9f-original-pull-secret") pod "global-pull-secret-syncer-qvn96" (UID: "e76c844a-a34c-4587-93a7-023d511a0b9f") : object "kube-system"/"original-pull-secret" not registered Jul 30 15:10:22.239616 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:10:22.236426 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7pzx7\" (UniqueName: \"kubernetes.io/projected/1cd62f00-3530-42c3-9aab-499796a7c906-kube-api-access-7pzx7\") pod \"network-check-target-q858z\" (UID: \"1cd62f00-3530-42c3-9aab-499796a7c906\") " pod="openshift-network-diagnostics/network-check-target-q858z" Jul 30 15:10:22.239616 ip-10-0-133-210 kubenswrapper[2576]: E0730 15:10:22.236573 2576 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 30 15:10:22.239616 ip-10-0-133-210 kubenswrapper[2576]: E0730 15:10:22.236587 2576 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 30 15:10:22.239616 ip-10-0-133-210 kubenswrapper[2576]: E0730 15:10:22.236597 2576 projected.go:196] Error preparing data for projected volume kube-api-access-7pzx7 for pod openshift-network-diagnostics/network-check-target-q858z: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 30 15:10:22.239616 ip-10-0-133-210 kubenswrapper[2576]: E0730 15:10:22.236643 2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/1cd62f00-3530-42c3-9aab-499796a7c906-kube-api-access-7pzx7 podName:1cd62f00-3530-42c3-9aab-499796a7c906 nodeName:}" failed. No retries permitted until 2026-07-30 15:10:24.236626519 +0000 UTC m=+6.162379145 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-7pzx7" (UniqueName: "kubernetes.io/projected/1cd62f00-3530-42c3-9aab-499796a7c906-kube-api-access-7pzx7") pod "network-check-target-q858z" (UID: "1cd62f00-3530-42c3-9aab-499796a7c906") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 30 15:10:22.460518 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:10:22.460420 2576 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-29 15:05:20 +0000 UTC" deadline="2028-03-30 23:27:04.19160136 +0000 UTC" Jul 30 15:10:22.460518 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:10:22.460466 2576 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="14624h16m41.731139864s" Jul 30 15:10:22.609058 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:10:22.608553 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-q858z" Jul 30 15:10:22.609058 ip-10-0-133-210 kubenswrapper[2576]: E0730 15:10:22.608700 2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-q858z" podUID="1cd62f00-3530-42c3-9aab-499796a7c906" Jul 30 15:10:22.635857 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:10:22.635821 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-210.ec2.internal" event={"ID":"f80b605a8a24b316000fbccb170e3adc","Type":"ContainerStarted","Data":"97777f0018640c93f5f6428969a28873eb7b3101679d7bb54ba441a4c849c1c4"} Jul 30 15:10:23.607431 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:10:23.607343 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-qvn96" Jul 30 15:10:23.607916 ip-10-0-133-210 kubenswrapper[2576]: E0730 15:10:23.607481 2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-qvn96" podUID="e76c844a-a34c-4587-93a7-023d511a0b9f" Jul 30 15:10:23.607916 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:10:23.607667 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-dbmt8" Jul 30 15:10:23.607916 ip-10-0-133-210 kubenswrapper[2576]: E0730 15:10:23.607783 2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-dbmt8" podUID="32b9046b-bd5c-4f23-b832-6fd0bad1c8d6" Jul 30 15:10:24.052738 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:10:24.052649 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/e76c844a-a34c-4587-93a7-023d511a0b9f-original-pull-secret\") pod \"global-pull-secret-syncer-qvn96\" (UID: \"e76c844a-a34c-4587-93a7-023d511a0b9f\") " pod="kube-system/global-pull-secret-syncer-qvn96" Jul 30 15:10:24.052738 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:10:24.052703 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/32b9046b-bd5c-4f23-b832-6fd0bad1c8d6-metrics-certs\") pod \"network-metrics-daemon-dbmt8\" (UID: \"32b9046b-bd5c-4f23-b832-6fd0bad1c8d6\") " pod="openshift-multus/network-metrics-daemon-dbmt8" Jul 30 15:10:24.052969 ip-10-0-133-210 kubenswrapper[2576]: E0730 15:10:24.052845 2576 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 30 15:10:24.052969 ip-10-0-133-210 kubenswrapper[2576]: E0730 15:10:24.052907 2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/32b9046b-bd5c-4f23-b832-6fd0bad1c8d6-metrics-certs podName:32b9046b-bd5c-4f23-b832-6fd0bad1c8d6 nodeName:}" failed. No retries permitted until 2026-07-30 15:10:28.052888751 +0000 UTC m=+9.978641376 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/32b9046b-bd5c-4f23-b832-6fd0bad1c8d6-metrics-certs") pod "network-metrics-daemon-dbmt8" (UID: "32b9046b-bd5c-4f23-b832-6fd0bad1c8d6") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 30 15:10:24.053318 ip-10-0-133-210 kubenswrapper[2576]: E0730 15:10:24.053296 2576 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 30 15:10:24.053418 ip-10-0-133-210 kubenswrapper[2576]: E0730 15:10:24.053355 2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e76c844a-a34c-4587-93a7-023d511a0b9f-original-pull-secret podName:e76c844a-a34c-4587-93a7-023d511a0b9f nodeName:}" failed. No retries permitted until 2026-07-30 15:10:28.053339653 +0000 UTC m=+9.979092278 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/e76c844a-a34c-4587-93a7-023d511a0b9f-original-pull-secret") pod "global-pull-secret-syncer-qvn96" (UID: "e76c844a-a34c-4587-93a7-023d511a0b9f") : object "kube-system"/"original-pull-secret" not registered Jul 30 15:10:24.254542 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:10:24.254501 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7pzx7\" (UniqueName: \"kubernetes.io/projected/1cd62f00-3530-42c3-9aab-499796a7c906-kube-api-access-7pzx7\") pod \"network-check-target-q858z\" (UID: \"1cd62f00-3530-42c3-9aab-499796a7c906\") " pod="openshift-network-diagnostics/network-check-target-q858z" Jul 30 15:10:24.254728 ip-10-0-133-210 kubenswrapper[2576]: E0730 15:10:24.254675 2576 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 30 15:10:24.254728 ip-10-0-133-210 kubenswrapper[2576]: E0730 15:10:24.254701 2576 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 30 15:10:24.254728 ip-10-0-133-210 kubenswrapper[2576]: E0730 15:10:24.254714 2576 projected.go:196] Error preparing data for projected volume kube-api-access-7pzx7 for pod openshift-network-diagnostics/network-check-target-q858z: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 30 15:10:24.254895 ip-10-0-133-210 kubenswrapper[2576]: E0730 15:10:24.254778 2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/1cd62f00-3530-42c3-9aab-499796a7c906-kube-api-access-7pzx7 podName:1cd62f00-3530-42c3-9aab-499796a7c906 nodeName:}" failed. No retries permitted until 2026-07-30 15:10:28.254757969 +0000 UTC m=+10.180510610 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-7pzx7" (UniqueName: "kubernetes.io/projected/1cd62f00-3530-42c3-9aab-499796a7c906-kube-api-access-7pzx7") pod "network-check-target-q858z" (UID: "1cd62f00-3530-42c3-9aab-499796a7c906") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 30 15:10:24.607849 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:10:24.606963 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-q858z" Jul 30 15:10:24.607849 ip-10-0-133-210 kubenswrapper[2576]: E0730 15:10:24.607096 2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-q858z" podUID="1cd62f00-3530-42c3-9aab-499796a7c906" Jul 30 15:10:25.606492 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:10:25.606453 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-qvn96" Jul 30 15:10:25.606692 ip-10-0-133-210 kubenswrapper[2576]: E0730 15:10:25.606591 2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-qvn96" podUID="e76c844a-a34c-4587-93a7-023d511a0b9f" Jul 30 15:10:25.606999 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:10:25.606980 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-dbmt8" Jul 30 15:10:25.607129 ip-10-0-133-210 kubenswrapper[2576]: E0730 15:10:25.607089 2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-dbmt8" podUID="32b9046b-bd5c-4f23-b832-6fd0bad1c8d6" Jul 30 15:10:26.607252 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:10:26.607211 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-q858z" Jul 30 15:10:26.607662 ip-10-0-133-210 kubenswrapper[2576]: E0730 15:10:26.607380 2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-q858z" podUID="1cd62f00-3530-42c3-9aab-499796a7c906" Jul 30 15:10:27.606841 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:10:27.606806 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-dbmt8" Jul 30 15:10:27.607015 ip-10-0-133-210 kubenswrapper[2576]: E0730 15:10:27.606928 2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-dbmt8" podUID="32b9046b-bd5c-4f23-b832-6fd0bad1c8d6" Jul 30 15:10:27.607356 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:10:27.607233 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-qvn96" Jul 30 15:10:27.607356 ip-10-0-133-210 kubenswrapper[2576]: E0730 15:10:27.607344 2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-qvn96" podUID="e76c844a-a34c-4587-93a7-023d511a0b9f" Jul 30 15:10:28.087638 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:10:28.087548 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/e76c844a-a34c-4587-93a7-023d511a0b9f-original-pull-secret\") pod \"global-pull-secret-syncer-qvn96\" (UID: \"e76c844a-a34c-4587-93a7-023d511a0b9f\") " pod="kube-system/global-pull-secret-syncer-qvn96" Jul 30 15:10:28.087638 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:10:28.087601 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/32b9046b-bd5c-4f23-b832-6fd0bad1c8d6-metrics-certs\") pod \"network-metrics-daemon-dbmt8\" (UID: \"32b9046b-bd5c-4f23-b832-6fd0bad1c8d6\") " pod="openshift-multus/network-metrics-daemon-dbmt8" Jul 30 15:10:28.087856 ip-10-0-133-210 kubenswrapper[2576]: E0730 15:10:28.087693 2576 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 30 15:10:28.087856 ip-10-0-133-210 kubenswrapper[2576]: E0730 15:10:28.087741 2576 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 30 15:10:28.087856 ip-10-0-133-210 kubenswrapper[2576]: E0730 15:10:28.087757 2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e76c844a-a34c-4587-93a7-023d511a0b9f-original-pull-secret podName:e76c844a-a34c-4587-93a7-023d511a0b9f nodeName:}" failed. No retries permitted until 2026-07-30 15:10:36.087737246 +0000 UTC m=+18.013489886 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/e76c844a-a34c-4587-93a7-023d511a0b9f-original-pull-secret") pod "global-pull-secret-syncer-qvn96" (UID: "e76c844a-a34c-4587-93a7-023d511a0b9f") : object "kube-system"/"original-pull-secret" not registered Jul 30 15:10:28.087856 ip-10-0-133-210 kubenswrapper[2576]: E0730 15:10:28.087775 2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/32b9046b-bd5c-4f23-b832-6fd0bad1c8d6-metrics-certs podName:32b9046b-bd5c-4f23-b832-6fd0bad1c8d6 nodeName:}" failed. No retries permitted until 2026-07-30 15:10:36.08776382 +0000 UTC m=+18.013516443 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/32b9046b-bd5c-4f23-b832-6fd0bad1c8d6-metrics-certs") pod "network-metrics-daemon-dbmt8" (UID: "32b9046b-bd5c-4f23-b832-6fd0bad1c8d6") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 30 15:10:28.289355 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:10:28.289271 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7pzx7\" (UniqueName: \"kubernetes.io/projected/1cd62f00-3530-42c3-9aab-499796a7c906-kube-api-access-7pzx7\") pod \"network-check-target-q858z\" (UID: \"1cd62f00-3530-42c3-9aab-499796a7c906\") " pod="openshift-network-diagnostics/network-check-target-q858z" Jul 30 15:10:28.289510 ip-10-0-133-210 kubenswrapper[2576]: E0730 15:10:28.289421 2576 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 30 15:10:28.289510 ip-10-0-133-210 kubenswrapper[2576]: E0730 15:10:28.289447 2576 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 30 15:10:28.289510 ip-10-0-133-210 kubenswrapper[2576]: E0730 15:10:28.289460 2576 projected.go:196] Error preparing data for projected volume kube-api-access-7pzx7 for pod openshift-network-diagnostics/network-check-target-q858z: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 30 15:10:28.289645 ip-10-0-133-210 kubenswrapper[2576]: E0730 15:10:28.289521 2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/1cd62f00-3530-42c3-9aab-499796a7c906-kube-api-access-7pzx7 podName:1cd62f00-3530-42c3-9aab-499796a7c906 nodeName:}" failed. No retries permitted until 2026-07-30 15:10:36.289499682 +0000 UTC m=+18.215252320 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-7pzx7" (UniqueName: "kubernetes.io/projected/1cd62f00-3530-42c3-9aab-499796a7c906-kube-api-access-7pzx7") pod "network-check-target-q858z" (UID: "1cd62f00-3530-42c3-9aab-499796a7c906") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 30 15:10:28.610753 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:10:28.610324 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-q858z" Jul 30 15:10:28.610753 ip-10-0-133-210 kubenswrapper[2576]: E0730 15:10:28.610434 2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-q858z" podUID="1cd62f00-3530-42c3-9aab-499796a7c906" Jul 30 15:10:29.606915 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:10:29.606471 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-dbmt8" Jul 30 15:10:29.606915 ip-10-0-133-210 kubenswrapper[2576]: E0730 15:10:29.606581 2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-dbmt8" podUID="32b9046b-bd5c-4f23-b832-6fd0bad1c8d6" Jul 30 15:10:29.606915 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:10:29.606636 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-qvn96" Jul 30 15:10:29.606915 ip-10-0-133-210 kubenswrapper[2576]: E0730 15:10:29.606743 2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-qvn96" podUID="e76c844a-a34c-4587-93a7-023d511a0b9f" Jul 30 15:10:30.607616 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:10:30.607579 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-q858z" Jul 30 15:10:30.608074 ip-10-0-133-210 kubenswrapper[2576]: E0730 15:10:30.607720 2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-q858z" podUID="1cd62f00-3530-42c3-9aab-499796a7c906" Jul 30 15:10:31.606563 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:10:31.606522 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-dbmt8" Jul 30 15:10:31.606563 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:10:31.606556 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-qvn96" Jul 30 15:10:31.606817 ip-10-0-133-210 kubenswrapper[2576]: E0730 15:10:31.606665 2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-dbmt8" podUID="32b9046b-bd5c-4f23-b832-6fd0bad1c8d6" Jul 30 15:10:31.606817 ip-10-0-133-210 kubenswrapper[2576]: E0730 15:10:31.606790 2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-qvn96" podUID="e76c844a-a34c-4587-93a7-023d511a0b9f" Jul 30 15:10:32.607211 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:10:32.607127 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-q858z" Jul 30 15:10:32.607653 ip-10-0-133-210 kubenswrapper[2576]: E0730 15:10:32.607295 2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-q858z" podUID="1cd62f00-3530-42c3-9aab-499796a7c906" Jul 30 15:10:33.606618 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:10:33.606584 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-dbmt8" Jul 30 15:10:33.606798 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:10:33.606584 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-qvn96" Jul 30 15:10:33.606798 ip-10-0-133-210 kubenswrapper[2576]: E0730 15:10:33.606745 2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-dbmt8" podUID="32b9046b-bd5c-4f23-b832-6fd0bad1c8d6" Jul 30 15:10:33.606903 ip-10-0-133-210 kubenswrapper[2576]: E0730 15:10:33.606790 2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-qvn96" podUID="e76c844a-a34c-4587-93a7-023d511a0b9f" Jul 30 15:10:34.606795 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:10:34.606755 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-q858z" Jul 30 15:10:34.607229 ip-10-0-133-210 kubenswrapper[2576]: E0730 15:10:34.606903 2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-q858z" podUID="1cd62f00-3530-42c3-9aab-499796a7c906" Jul 30 15:10:35.606707 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:10:35.606675 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-qvn96" Jul 30 15:10:35.606953 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:10:35.606677 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-dbmt8" Jul 30 15:10:35.606953 ip-10-0-133-210 kubenswrapper[2576]: E0730 15:10:35.606771 2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-qvn96" podUID="e76c844a-a34c-4587-93a7-023d511a0b9f" Jul 30 15:10:35.606953 ip-10-0-133-210 kubenswrapper[2576]: E0730 15:10:35.606897 2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-dbmt8" podUID="32b9046b-bd5c-4f23-b832-6fd0bad1c8d6" Jul 30 15:10:36.152121 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:10:36.152069 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/e76c844a-a34c-4587-93a7-023d511a0b9f-original-pull-secret\") pod \"global-pull-secret-syncer-qvn96\" (UID: \"e76c844a-a34c-4587-93a7-023d511a0b9f\") " pod="kube-system/global-pull-secret-syncer-qvn96" Jul 30 15:10:36.152302 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:10:36.152130 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/32b9046b-bd5c-4f23-b832-6fd0bad1c8d6-metrics-certs\") pod \"network-metrics-daemon-dbmt8\" (UID: \"32b9046b-bd5c-4f23-b832-6fd0bad1c8d6\") " pod="openshift-multus/network-metrics-daemon-dbmt8" Jul 30 15:10:36.152302 ip-10-0-133-210 kubenswrapper[2576]: E0730 15:10:36.152228 2576 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 30 15:10:36.152302 ip-10-0-133-210 kubenswrapper[2576]: E0730 15:10:36.152301 2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e76c844a-a34c-4587-93a7-023d511a0b9f-original-pull-secret podName:e76c844a-a34c-4587-93a7-023d511a0b9f nodeName:}" failed. No retries permitted until 2026-07-30 15:10:52.152282159 +0000 UTC m=+34.078034784 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/e76c844a-a34c-4587-93a7-023d511a0b9f-original-pull-secret") pod "global-pull-secret-syncer-qvn96" (UID: "e76c844a-a34c-4587-93a7-023d511a0b9f") : object "kube-system"/"original-pull-secret" not registered Jul 30 15:10:36.152487 ip-10-0-133-210 kubenswrapper[2576]: E0730 15:10:36.152238 2576 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 30 15:10:36.152487 ip-10-0-133-210 kubenswrapper[2576]: E0730 15:10:36.152384 2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/32b9046b-bd5c-4f23-b832-6fd0bad1c8d6-metrics-certs podName:32b9046b-bd5c-4f23-b832-6fd0bad1c8d6 nodeName:}" failed. No retries permitted until 2026-07-30 15:10:52.152369681 +0000 UTC m=+34.078122317 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/32b9046b-bd5c-4f23-b832-6fd0bad1c8d6-metrics-certs") pod "network-metrics-daemon-dbmt8" (UID: "32b9046b-bd5c-4f23-b832-6fd0bad1c8d6") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 30 15:10:36.353814 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:10:36.353770 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7pzx7\" (UniqueName: \"kubernetes.io/projected/1cd62f00-3530-42c3-9aab-499796a7c906-kube-api-access-7pzx7\") pod \"network-check-target-q858z\" (UID: \"1cd62f00-3530-42c3-9aab-499796a7c906\") " pod="openshift-network-diagnostics/network-check-target-q858z" Jul 30 15:10:36.353980 ip-10-0-133-210 kubenswrapper[2576]: E0730 15:10:36.353946 2576 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 30 15:10:36.353980 ip-10-0-133-210 kubenswrapper[2576]: E0730 15:10:36.353974 2576 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 30 15:10:36.354166 ip-10-0-133-210 kubenswrapper[2576]: E0730 15:10:36.353987 2576 projected.go:196] Error preparing data for projected volume kube-api-access-7pzx7 for pod openshift-network-diagnostics/network-check-target-q858z: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 30 15:10:36.354166 ip-10-0-133-210 kubenswrapper[2576]: E0730 15:10:36.354052 2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/1cd62f00-3530-42c3-9aab-499796a7c906-kube-api-access-7pzx7 podName:1cd62f00-3530-42c3-9aab-499796a7c906 nodeName:}" failed. No retries permitted until 2026-07-30 15:10:52.354032625 +0000 UTC m=+34.279785248 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-7pzx7" (UniqueName: "kubernetes.io/projected/1cd62f00-3530-42c3-9aab-499796a7c906-kube-api-access-7pzx7") pod "network-check-target-q858z" (UID: "1cd62f00-3530-42c3-9aab-499796a7c906") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 30 15:10:36.607243 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:10:36.607162 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-q858z" Jul 30 15:10:36.607647 ip-10-0-133-210 kubenswrapper[2576]: E0730 15:10:36.607291 2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-q858z" podUID="1cd62f00-3530-42c3-9aab-499796a7c906" Jul 30 15:10:37.607244 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:10:37.607205 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-qvn96" Jul 30 15:10:37.607696 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:10:37.607205 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-dbmt8" Jul 30 15:10:37.607696 ip-10-0-133-210 kubenswrapper[2576]: E0730 15:10:37.607352 2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-qvn96" podUID="e76c844a-a34c-4587-93a7-023d511a0b9f" Jul 30 15:10:37.607696 ip-10-0-133-210 kubenswrapper[2576]: E0730 15:10:37.607390 2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-dbmt8" podUID="32b9046b-bd5c-4f23-b832-6fd0bad1c8d6" Jul 30 15:10:38.607229 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:10:38.607189 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-q858z" Jul 30 15:10:38.607430 ip-10-0-133-210 kubenswrapper[2576]: E0730 15:10:38.607318 2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-q858z" podUID="1cd62f00-3530-42c3-9aab-499796a7c906" Jul 30 15:10:39.607413 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:10:39.607378 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-dbmt8" Jul 30 15:10:39.607579 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:10:39.607415 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-qvn96" Jul 30 15:10:39.607579 ip-10-0-133-210 kubenswrapper[2576]: E0730 15:10:39.607512 2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-dbmt8" podUID="32b9046b-bd5c-4f23-b832-6fd0bad1c8d6" Jul 30 15:10:39.607980 ip-10-0-133-210 kubenswrapper[2576]: E0730 15:10:39.607649 2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-qvn96" podUID="e76c844a-a34c-4587-93a7-023d511a0b9f" Jul 30 15:10:40.606806 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:10:40.606650 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-q858z" Jul 30 15:10:40.606906 ip-10-0-133-210 kubenswrapper[2576]: E0730 15:10:40.606889 2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-q858z" podUID="1cd62f00-3530-42c3-9aab-499796a7c906" Jul 30 15:10:40.671642 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:10:40.671613 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-47m5x" event={"ID":"57353464-4671-4d1f-b2b5-473d618d461d","Type":"ContainerStarted","Data":"3a35151f4856e481ade3613f0e8b418f7129763e5f0ff73d53c80f40bbbdd431"} Jul 30 15:10:40.672980 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:10:40.672953 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-ktmgn" event={"ID":"4b006058-0163-4e82-9a1f-979b182ddbf1","Type":"ContainerStarted","Data":"8d7df10c7b2b355e1273a4e8e9677d175a96d7c5292bdfeada618a710920c1ad"} Jul 30 15:10:40.675574 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:10:40.675552 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-6mb29" event={"ID":"7fb7dc92-4394-4433-9f1d-0f2c237136fe","Type":"ContainerStarted","Data":"2fcbf10ce665c21b6986459c855873037e182e23f71df496ea9a3c650c7f3ef9"} Jul 30 15:10:40.675663 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:10:40.675607 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-6mb29" event={"ID":"7fb7dc92-4394-4433-9f1d-0f2c237136fe","Type":"ContainerStarted","Data":"f2d15b99ddd780eacef4362480bf06f0f3c4c92cb531956f1cfc95584817593f"} Jul 30 15:10:40.675663 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:10:40.675638 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-6mb29" event={"ID":"7fb7dc92-4394-4433-9f1d-0f2c237136fe","Type":"ContainerStarted","Data":"2a0c38e4e57ecf74d8f8af4d37cae7cc4efceacaa4063068e4f174de3ef24716"} Jul 30 15:10:40.675759 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:10:40.675665 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-6mb29" event={"ID":"7fb7dc92-4394-4433-9f1d-0f2c237136fe","Type":"ContainerStarted","Data":"e5aab103904383d53406a2d5cab579f70d7c32ee0d56931061c915947fcad1b5"} Jul 30 15:10:40.675759 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:10:40.675692 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-6mb29" event={"ID":"7fb7dc92-4394-4433-9f1d-0f2c237136fe","Type":"ContainerStarted","Data":"3b69020cbfcf2764b6a6c94d24efbe9cbf44bd4b82c87ae14a315146aed9b7be"} Jul 30 15:10:40.676990 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:10:40.676968 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-fxr7d" event={"ID":"3caae76d-69ba-4860-96fc-cca51b9ba7db","Type":"ContainerStarted","Data":"97d82ffb22fb37f3213e5db43d2aec110fc30227141b03cc2d9f63c1edbfe837"} Jul 30 15:10:40.678367 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:10:40.678345 2576 generic.go:356] "Generic (PLEG): container finished" podID="f80b605a8a24b316000fbccb170e3adc" containerID="55f95c2e0104ea12c3cee7e5b0b5c84690c26757dc31d9b3e1580703c6e8cf3a" exitCode=0 Jul 30 15:10:40.678460 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:10:40.678406 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-210.ec2.internal" event={"ID":"f80b605a8a24b316000fbccb170e3adc","Type":"ContainerDied","Data":"55f95c2e0104ea12c3cee7e5b0b5c84690c26757dc31d9b3e1580703c6e8cf3a"} Jul 30 15:10:40.679677 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:10:40.679658 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-7j468" event={"ID":"a2197c57-2dc6-410e-9910-a045dedcb8d9","Type":"ContainerStarted","Data":"60c00b9e3a6655597cfa0b909ff5466670022735bdf46c19eaf04de2aae99529"} Jul 30 15:10:40.681521 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:10:40.681501 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-gb69c" event={"ID":"23fd99c2-79a8-41eb-bb0f-92a363e32553","Type":"ContainerStarted","Data":"b770ce55657fb8998757a10ee0e1664dfbb4bdc7c094b67ee9aa86d882839241"} Jul 30 15:10:40.682646 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:10:40.682618 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-j74qd" event={"ID":"347bd19f-7728-47be-986d-337d0f0df246","Type":"ContainerStarted","Data":"07915eb06e3210de8ee7d12fd7ac80f6fcea50ac3c18fbdd27b1bdabe8a2bdc7"} Jul 30 15:10:40.684246 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:10:40.684213 2576 generic.go:356] "Generic (PLEG): container finished" podID="f62011d0-f5be-413a-93a2-8a7a98b67130" containerID="ddc009a7ef1a809119e52e5871241f8cd354dc7c4a261299a5151dd02a19a751" exitCode=0 Jul 30 15:10:40.684347 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:10:40.684259 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-m2j4z" event={"ID":"f62011d0-f5be-413a-93a2-8a7a98b67130","Type":"ContainerDied","Data":"ddc009a7ef1a809119e52e5871241f8cd354dc7c4a261299a5151dd02a19a751"} Jul 30 15:10:40.688556 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:10:40.688517 2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-133-210.ec2.internal" podStartSLOduration=21.688504996 podStartE2EDuration="21.688504996s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-30 15:10:19 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 15:10:21.634212735 +0000 UTC m=+3.559965380" watchObservedRunningTime="2026-07-30 15:10:40.688504996 +0000 UTC m=+22.614257641" Jul 30 15:10:40.688664 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:10:40.688643 2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/node-ca-ktmgn" podStartSLOduration=3.941731446 podStartE2EDuration="22.688639155s" totalImagesPullingTime="18.746907709s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-30 15:10:18 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 15:10:40.688307231 +0000 UTC m=+22.614059875" watchObservedRunningTime="2026-07-30 15:10:40.688639155 +0000 UTC m=+22.614391799" Jul 30 15:10:40.742428 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:10:40.742366 2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-fxr7d" podStartSLOduration=3.989664972 podStartE2EDuration="22.742345779s" totalImagesPullingTime="18.752680807s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-30 15:10:18 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 15:10:40.741892608 +0000 UTC m=+22.667645266" watchObservedRunningTime="2026-07-30 15:10:40.742345779 +0000 UTC m=+22.668098425" Jul 30 15:10:40.761736 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:10:40.761687 2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-gb69c" podStartSLOduration=3.978698284 podStartE2EDuration="22.761665753s" totalImagesPullingTime="18.782967469s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-30 15:10:18 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 15:10:40.760723714 +0000 UTC m=+22.686476388" watchObservedRunningTime="2026-07-30 15:10:40.761665753 +0000 UTC m=+22.687418401" Jul 30 15:10:40.792920 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:10:40.792870 2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/node-resolver-j74qd" podStartSLOduration=4.073027613 podStartE2EDuration="22.792853747s" totalImagesPullingTime="18.719826134s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-30 15:10:18 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 15:10:40.792531387 +0000 UTC m=+22.718284034" watchObservedRunningTime="2026-07-30 15:10:40.792853747 +0000 UTC m=+22.718606391" Jul 30 15:10:40.793252 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:10:40.793217 2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/konnectivity-agent-7j468" podStartSLOduration=11.931318031 podStartE2EDuration="22.793205142s" totalImagesPullingTime="10.861887111s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 15:10:18 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 15:10:40.776834305 +0000 UTC m=+22.702586974" watchObservedRunningTime="2026-07-30 15:10:40.793205142 +0000 UTC m=+22.718957787" Jul 30 15:10:40.879478 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:10:40.879385 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7j468_a2197c57-2dc6-410e-9910-a045dedcb8d9/konnectivity-agent/0.log" limit=1024 Jul 30 15:10:40.959236 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:10:40.959208 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7j468_a2197c57-2dc6-410e-9910-a045dedcb8d9/konnectivity-agent/0.log" limit=1024 Jul 30 15:10:41.157850 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:10:41.157735 2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-7j468" Jul 30 15:10:41.158398 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:10:41.158374 2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-7j468" Jul 30 15:10:41.607220 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:10:41.607189 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-qvn96" Jul 30 15:10:41.607466 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:10:41.607193 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-dbmt8" Jul 30 15:10:41.607466 ip-10-0-133-210 kubenswrapper[2576]: E0730 15:10:41.607312 2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-qvn96" podUID="e76c844a-a34c-4587-93a7-023d511a0b9f" Jul 30 15:10:41.607466 ip-10-0-133-210 kubenswrapper[2576]: E0730 15:10:41.607373 2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-dbmt8" podUID="32b9046b-bd5c-4f23-b832-6fd0bad1c8d6" Jul 30 15:10:41.688293 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:10:41.688255 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-4g9qs" event={"ID":"5be52bf9-46b0-4e79-9d34-9b32b8b891b4","Type":"ContainerStarted","Data":"ecc992b33ff2aaadd5d17a1bd71022e57d09e122a94deeec4076601b1b6d6bbd"} Jul 30 15:10:41.691733 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:10:41.691700 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-6mb29" event={"ID":"7fb7dc92-4394-4433-9f1d-0f2c237136fe","Type":"ContainerStarted","Data":"a69160790efdb040d0600f015fe8aee36b7946a7ae815088db667779dbe9517a"} Jul 30 15:10:41.693908 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:10:41.693879 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-210.ec2.internal" event={"ID":"f80b605a8a24b316000fbccb170e3adc","Type":"ContainerStarted","Data":"14d232834bc229aaf7abc63ed7b1affd60d2d57eb7a205d688dd610329b55b00"} Jul 30 15:10:41.694924 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:10:41.694902 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-7j468" Jul 30 15:10:41.695034 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:10:41.694965 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-7j468" Jul 30 15:10:41.707926 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:10:41.707844 2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-4g9qs" podStartSLOduration=4.9620895350000005 podStartE2EDuration="23.707823241s" totalImagesPullingTime="18.745733706s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-30 15:10:18 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 15:10:41.707281214 +0000 UTC m=+23.633033850" watchObservedRunningTime="2026-07-30 15:10:41.707823241 +0000 UTC m=+23.633575887" Jul 30 15:10:41.738783 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:10:41.738731 2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-210.ec2.internal" podStartSLOduration=22.738714652 podStartE2EDuration="22.738714652s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-30 15:10:19 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 15:10:41.738282025 +0000 UTC m=+23.664034671" watchObservedRunningTime="2026-07-30 15:10:41.738714652 +0000 UTC m=+23.664467297" Jul 30 15:10:41.762188 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:10:41.762145 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7j468_a2197c57-2dc6-410e-9910-a045dedcb8d9/konnectivity-agent/0.log" limit=1024 Jul 30 15:10:41.863070 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:10:41.863036 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7j468_a2197c57-2dc6-410e-9910-a045dedcb8d9/konnectivity-agent/0.log" limit=1024 Jul 30 15:10:41.934739 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:10:41.934709 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7j468_a2197c57-2dc6-410e-9910-a045dedcb8d9/konnectivity-agent/0.log" limit=1024 Jul 30 15:10:42.607174 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:10:42.607126 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-q858z" Jul 30 15:10:42.607346 ip-10-0-133-210 kubenswrapper[2576]: E0730 15:10:42.607257 2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-q858z" podUID="1cd62f00-3530-42c3-9aab-499796a7c906" Jul 30 15:10:42.626232 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:10:42.626200 2576 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock" Jul 30 15:10:42.698275 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:10:42.698230 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-47m5x" event={"ID":"57353464-4671-4d1f-b2b5-473d618d461d","Type":"ContainerStarted","Data":"a6a705939d87fa303b367152aaa89e9fc2b20b2a309b516979a6a57c4086d429"} Jul 30 15:10:43.478416 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:10:43.478327 2576 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-07-30T15:10:42.626225389Z","UUID":"35af1a64-cb7b-45cc-bb6a-c9c02e8f7053","Handler":null,"Name":"","Endpoint":""} Jul 30 15:10:43.480282 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:10:43.480136 2576 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0 Jul 30 15:10:43.480282 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:10:43.480184 2576 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock Jul 30 15:10:43.607385 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:10:43.607353 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-dbmt8" Jul 30 15:10:43.607385 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:10:43.607397 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-qvn96" Jul 30 15:10:43.607619 ip-10-0-133-210 kubenswrapper[2576]: E0730 15:10:43.607499 2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-dbmt8" podUID="32b9046b-bd5c-4f23-b832-6fd0bad1c8d6" Jul 30 15:10:43.607678 ip-10-0-133-210 kubenswrapper[2576]: E0730 15:10:43.607643 2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-qvn96" podUID="e76c844a-a34c-4587-93a7-023d511a0b9f" Jul 30 15:10:43.704101 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:10:43.704063 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-6mb29" event={"ID":"7fb7dc92-4394-4433-9f1d-0f2c237136fe","Type":"ContainerStarted","Data":"1b1a45a976a4f43c05d35f94789521ff2f520ff9472188d00bc3366e7f8be92a"} Jul 30 15:10:44.606805 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:10:44.606766 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-q858z" Jul 30 15:10:44.607007 ip-10-0-133-210 kubenswrapper[2576]: E0730 15:10:44.606914 2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-q858z" podUID="1cd62f00-3530-42c3-9aab-499796a7c906" Jul 30 15:10:44.651757 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:10:44.651723 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7j468_a2197c57-2dc6-410e-9910-a045dedcb8d9/konnectivity-agent/0.log" limit=1024 Jul 30 15:10:44.722438 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:10:44.722404 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7j468_a2197c57-2dc6-410e-9910-a045dedcb8d9/konnectivity-agent/0.log" limit=1024 Jul 30 15:10:45.607135 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:10:45.606897 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-qvn96" Jul 30 15:10:45.607135 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:10:45.606945 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-dbmt8" Jul 30 15:10:45.607361 ip-10-0-133-210 kubenswrapper[2576]: E0730 15:10:45.607184 2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-qvn96" podUID="e76c844a-a34c-4587-93a7-023d511a0b9f" Jul 30 15:10:45.607361 ip-10-0-133-210 kubenswrapper[2576]: E0730 15:10:45.607213 2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-dbmt8" podUID="32b9046b-bd5c-4f23-b832-6fd0bad1c8d6" Jul 30 15:10:45.710076 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:10:45.710039 2576 generic.go:356] "Generic (PLEG): container finished" podID="f62011d0-f5be-413a-93a2-8a7a98b67130" containerID="ebcb10e8cf4deb39dbdcfffe216d63bcf39ec1bd645c8d7fa49b63bd46a5a099" exitCode=0 Jul 30 15:10:45.710252 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:10:45.710081 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-m2j4z" event={"ID":"f62011d0-f5be-413a-93a2-8a7a98b67130","Type":"ContainerDied","Data":"ebcb10e8cf4deb39dbdcfffe216d63bcf39ec1bd645c8d7fa49b63bd46a5a099"} Jul 30 15:10:45.712326 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:10:45.712304 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-47m5x" event={"ID":"57353464-4671-4d1f-b2b5-473d618d461d","Type":"ContainerStarted","Data":"bcab4bac492f0f9f6e43f256caa00888c287d37beb03a2bc7502c4b08d0361bb"} Jul 30 15:10:45.715794 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:10:45.715774 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-6mb29" event={"ID":"7fb7dc92-4394-4433-9f1d-0f2c237136fe","Type":"ContainerStarted","Data":"0e5b26558a6162305d79d41d0902f83b95728b0f2282fd41bc16363b7bcee787"} Jul 30 15:10:45.715988 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:10:45.715964 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-6mb29" Jul 30 15:10:45.716088 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:10:45.716036 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-6mb29" Jul 30 15:10:45.716241 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:10:45.716148 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-6mb29" Jul 30 15:10:45.731306 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:10:45.731280 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-6mb29" Jul 30 15:10:45.731904 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:10:45.731376 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-6mb29" Jul 30 15:10:45.758806 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:10:45.758757 2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-47m5x" podStartSLOduration=4.309862421 podStartE2EDuration="27.758741542s" totalImagesPullingTime="23.448879121s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-30 15:10:18 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 15:10:45.758542693 +0000 UTC m=+27.684295337" watchObservedRunningTime="2026-07-30 15:10:45.758741542 +0000 UTC m=+27.684494202" Jul 30 15:10:45.796921 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:10:45.796859 2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-6mb29" podStartSLOduration=8.986579014 podStartE2EDuration="27.796841848s" totalImagesPullingTime="18.810262834s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-30 15:10:18 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 15:10:45.795081385 +0000 UTC m=+27.720834029" watchObservedRunningTime="2026-07-30 15:10:45.796841848 +0000 UTC m=+27.722594547" Jul 30 15:10:46.607347 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:10:46.607310 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-q858z" Jul 30 15:10:46.607508 ip-10-0-133-210 kubenswrapper[2576]: E0730 15:10:46.607450 2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-q858z" podUID="1cd62f00-3530-42c3-9aab-499796a7c906" Jul 30 15:10:47.249538 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:10:47.249298 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7j468_a2197c57-2dc6-410e-9910-a045dedcb8d9/konnectivity-agent/0.log" limit=1024 Jul 30 15:10:47.352529 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:10:47.352504 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7j468_a2197c57-2dc6-410e-9910-a045dedcb8d9/konnectivity-agent/0.log" limit=1024 Jul 30 15:10:47.566306 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:10:47.566238 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-qvn96"] Jul 30 15:10:47.566450 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:10:47.566372 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-qvn96" Jul 30 15:10:47.566500 ip-10-0-133-210 kubenswrapper[2576]: E0730 15:10:47.566450 2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-qvn96" podUID="e76c844a-a34c-4587-93a7-023d511a0b9f" Jul 30 15:10:47.569536 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:10:47.569512 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-q858z"] Jul 30 15:10:47.569643 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:10:47.569604 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-q858z" Jul 30 15:10:47.569684 ip-10-0-133-210 kubenswrapper[2576]: E0730 15:10:47.569670 2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-q858z" podUID="1cd62f00-3530-42c3-9aab-499796a7c906" Jul 30 15:10:47.580895 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:10:47.580874 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-dbmt8"] Jul 30 15:10:47.581010 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:10:47.580960 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-dbmt8" Jul 30 15:10:47.581061 ip-10-0-133-210 kubenswrapper[2576]: E0730 15:10:47.581033 2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-dbmt8" podUID="32b9046b-bd5c-4f23-b832-6fd0bad1c8d6" Jul 30 15:10:47.722281 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:10:47.722242 2576 generic.go:356] "Generic (PLEG): container finished" podID="f62011d0-f5be-413a-93a2-8a7a98b67130" containerID="46ede5703daef556ae20dd2fd09b79b2400eb8e8df796a56261ac61e2cd66a5c" exitCode=0 Jul 30 15:10:47.722432 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:10:47.722329 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-m2j4z" event={"ID":"f62011d0-f5be-413a-93a2-8a7a98b67130","Type":"ContainerDied","Data":"46ede5703daef556ae20dd2fd09b79b2400eb8e8df796a56261ac61e2cd66a5c"} Jul 30 15:10:49.040305 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:10:49.040277 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7j468_a2197c57-2dc6-410e-9910-a045dedcb8d9/konnectivity-agent/0.log" limit=1024 Jul 30 15:10:49.132406 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:10:49.132367 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7j468_a2197c57-2dc6-410e-9910-a045dedcb8d9/konnectivity-agent/0.log" limit=1024 Jul 30 15:10:49.243353 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:10:49.243311 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7j468_a2197c57-2dc6-410e-9910-a045dedcb8d9/konnectivity-agent/0.log" limit=1024 Jul 30 15:10:49.606900 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:10:49.606869 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-qvn96" Jul 30 15:10:49.607046 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:10:49.606967 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-dbmt8" Jul 30 15:10:49.607046 ip-10-0-133-210 kubenswrapper[2576]: E0730 15:10:49.606974 2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-qvn96" podUID="e76c844a-a34c-4587-93a7-023d511a0b9f" Jul 30 15:10:49.607046 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:10:49.606997 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-q858z" Jul 30 15:10:49.607177 ip-10-0-133-210 kubenswrapper[2576]: E0730 15:10:49.607093 2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-dbmt8" podUID="32b9046b-bd5c-4f23-b832-6fd0bad1c8d6" Jul 30 15:10:49.607216 ip-10-0-133-210 kubenswrapper[2576]: E0730 15:10:49.607175 2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-q858z" podUID="1cd62f00-3530-42c3-9aab-499796a7c906" Jul 30 15:10:49.729128 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:10:49.729026 2576 generic.go:356] "Generic (PLEG): container finished" podID="f62011d0-f5be-413a-93a2-8a7a98b67130" containerID="a9dbd3cafbebd0965e31e259cf08802dbc1d330a51ade682623ad2fd5b365622" exitCode=0 Jul 30 15:10:49.729263 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:10:49.729095 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-m2j4z" event={"ID":"f62011d0-f5be-413a-93a2-8a7a98b67130","Type":"ContainerDied","Data":"a9dbd3cafbebd0965e31e259cf08802dbc1d330a51ade682623ad2fd5b365622"} Jul 30 15:10:50.916454 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:10:50.916252 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7j468_a2197c57-2dc6-410e-9910-a045dedcb8d9/konnectivity-agent/0.log" limit=1024 Jul 30 15:10:51.201681 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:10:51.201596 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7j468_a2197c57-2dc6-410e-9910-a045dedcb8d9/konnectivity-agent/0.log" limit=1024 Jul 30 15:10:51.584058 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:10:51.583979 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7j468_a2197c57-2dc6-410e-9910-a045dedcb8d9/konnectivity-agent/0.log" limit=1024 Jul 30 15:10:51.606629 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:10:51.606601 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-qvn96" Jul 30 15:10:51.606781 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:10:51.606641 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-dbmt8" Jul 30 15:10:51.606781 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:10:51.606640 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-q858z" Jul 30 15:10:51.606781 ip-10-0-133-210 kubenswrapper[2576]: E0730 15:10:51.606726 2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-qvn96" podUID="e76c844a-a34c-4587-93a7-023d511a0b9f" Jul 30 15:10:51.606905 ip-10-0-133-210 kubenswrapper[2576]: E0730 15:10:51.606812 2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-dbmt8" podUID="32b9046b-bd5c-4f23-b832-6fd0bad1c8d6" Jul 30 15:10:51.606905 ip-10-0-133-210 kubenswrapper[2576]: E0730 15:10:51.606889 2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-q858z" podUID="1cd62f00-3530-42c3-9aab-499796a7c906" Jul 30 15:10:51.687907 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:10:51.687866 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7j468_a2197c57-2dc6-410e-9910-a045dedcb8d9/konnectivity-agent/0.log" limit=1024 Jul 30 15:10:52.167768 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:10:52.167730 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/e76c844a-a34c-4587-93a7-023d511a0b9f-original-pull-secret\") pod \"global-pull-secret-syncer-qvn96\" (UID: \"e76c844a-a34c-4587-93a7-023d511a0b9f\") " pod="kube-system/global-pull-secret-syncer-qvn96" Jul 30 15:10:52.168207 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:10:52.167772 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/32b9046b-bd5c-4f23-b832-6fd0bad1c8d6-metrics-certs\") pod \"network-metrics-daemon-dbmt8\" (UID: \"32b9046b-bd5c-4f23-b832-6fd0bad1c8d6\") " pod="openshift-multus/network-metrics-daemon-dbmt8" Jul 30 15:10:52.168207 ip-10-0-133-210 kubenswrapper[2576]: E0730 15:10:52.167877 2576 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 30 15:10:52.168207 ip-10-0-133-210 kubenswrapper[2576]: E0730 15:10:52.167901 2576 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 30 15:10:52.168207 ip-10-0-133-210 kubenswrapper[2576]: E0730 15:10:52.167944 2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/32b9046b-bd5c-4f23-b832-6fd0bad1c8d6-metrics-certs podName:32b9046b-bd5c-4f23-b832-6fd0bad1c8d6 nodeName:}" failed. No retries permitted until 2026-07-30 15:11:24.167926391 +0000 UTC m=+66.093679022 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/32b9046b-bd5c-4f23-b832-6fd0bad1c8d6-metrics-certs") pod "network-metrics-daemon-dbmt8" (UID: "32b9046b-bd5c-4f23-b832-6fd0bad1c8d6") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 30 15:10:52.168207 ip-10-0-133-210 kubenswrapper[2576]: E0730 15:10:52.167963 2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e76c844a-a34c-4587-93a7-023d511a0b9f-original-pull-secret podName:e76c844a-a34c-4587-93a7-023d511a0b9f nodeName:}" failed. No retries permitted until 2026-07-30 15:11:24.167954748 +0000 UTC m=+66.093707370 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/e76c844a-a34c-4587-93a7-023d511a0b9f-original-pull-secret") pod "global-pull-secret-syncer-qvn96" (UID: "e76c844a-a34c-4587-93a7-023d511a0b9f") : object "kube-system"/"original-pull-secret" not registered Jul 30 15:10:52.369727 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:10:52.369696 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7pzx7\" (UniqueName: \"kubernetes.io/projected/1cd62f00-3530-42c3-9aab-499796a7c906-kube-api-access-7pzx7\") pod \"network-check-target-q858z\" (UID: \"1cd62f00-3530-42c3-9aab-499796a7c906\") " pod="openshift-network-diagnostics/network-check-target-q858z" Jul 30 15:10:52.369896 ip-10-0-133-210 kubenswrapper[2576]: E0730 15:10:52.369855 2576 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 30 15:10:52.369896 ip-10-0-133-210 kubenswrapper[2576]: E0730 15:10:52.369881 2576 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 30 15:10:52.369896 ip-10-0-133-210 kubenswrapper[2576]: E0730 15:10:52.369894 2576 projected.go:196] Error preparing data for projected volume kube-api-access-7pzx7 for pod openshift-network-diagnostics/network-check-target-q858z: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 30 15:10:52.370028 ip-10-0-133-210 kubenswrapper[2576]: E0730 15:10:52.369957 2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/1cd62f00-3530-42c3-9aab-499796a7c906-kube-api-access-7pzx7 podName:1cd62f00-3530-42c3-9aab-499796a7c906 nodeName:}" failed. No retries permitted until 2026-07-30 15:11:24.369937874 +0000 UTC m=+66.295690512 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-7pzx7" (UniqueName: "kubernetes.io/projected/1cd62f00-3530-42c3-9aab-499796a7c906-kube-api-access-7pzx7") pod "network-check-target-q858z" (UID: "1cd62f00-3530-42c3-9aab-499796a7c906") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 30 15:10:52.876027 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:10:52.875956 2576 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-210.ec2.internal" event="NodeReady" Jul 30 15:10:52.876207 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:10:52.876093 2576 kubelet_node_status.go:489] "Fast updating node status as it just became ready" Jul 30 15:10:52.924924 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:10:52.924889 2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-jwbls"] Jul 30 15:10:52.930710 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:10:52.930680 2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-qtv7l"] Jul 30 15:10:52.930856 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:10:52.930815 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-jwbls" Jul 30 15:10:52.932540 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:10:52.932515 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-qtv7l" Jul 30 15:10:52.933998 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:10:52.933962 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\"" Jul 30 15:10:52.934128 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:10:52.933999 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-7hzbg\"" Jul 30 15:10:52.934128 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:10:52.934027 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\"" Jul 30 15:10:52.934861 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:10:52.934840 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-hpfnt\"" Jul 30 15:10:52.935167 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:10:52.935144 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\"" Jul 30 15:10:52.935272 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:10:52.935212 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\"" Jul 30 15:10:52.935621 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:10:52.935603 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\"" Jul 30 15:10:52.936921 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:10:52.936896 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-jwbls"] Jul 30 15:10:52.943290 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:10:52.943268 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-qtv7l"] Jul 30 15:10:52.954418 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:10:52.954399 2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-dvkw7"] Jul 30 15:10:52.956332 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:10:52.956314 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-dvkw7" Jul 30 15:10:52.961132 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:10:52.961086 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\"" Jul 30 15:10:52.961132 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:10:52.961103 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\"" Jul 30 15:10:52.961385 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:10:52.961368 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-6v78w\"" Jul 30 15:10:52.961385 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:10:52.961383 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\"" Jul 30 15:10:52.961603 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:10:52.961588 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\"" Jul 30 15:10:52.969464 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:10:52.969436 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-dvkw7"] Jul 30 15:10:53.074163 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:10:53.074125 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/9f3ebd63-d1cc-4920-94dc-65d6a62b96fa-crio-socket\") pod \"insights-runtime-extractor-dvkw7\" (UID: \"9f3ebd63-d1cc-4920-94dc-65d6a62b96fa\") " pod="openshift-insights/insights-runtime-extractor-dvkw7" Jul 30 15:10:53.074350 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:10:53.074218 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c460206a-fa1f-4c09-b498-467313191c13-tmp-dir\") pod \"dns-default-jwbls\" (UID: \"c460206a-fa1f-4c09-b498-467313191c13\") " pod="openshift-dns/dns-default-jwbls" Jul 30 15:10:53.074350 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:10:53.074250 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/9f3ebd63-d1cc-4920-94dc-65d6a62b96fa-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-dvkw7\" (UID: \"9f3ebd63-d1cc-4920-94dc-65d6a62b96fa\") " pod="openshift-insights/insights-runtime-extractor-dvkw7" Jul 30 15:10:53.074350 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:10:53.074271 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9m2th\" (UniqueName: \"kubernetes.io/projected/647c67b8-98e9-4905-9e04-35cc6cc3bd33-kube-api-access-9m2th\") pod \"ingress-canary-qtv7l\" (UID: \"647c67b8-98e9-4905-9e04-35cc6cc3bd33\") " pod="openshift-ingress-canary/ingress-canary-qtv7l" Jul 30 15:10:53.074522 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:10:53.074365 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/9f3ebd63-d1cc-4920-94dc-65d6a62b96fa-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-dvkw7\" (UID: \"9f3ebd63-d1cc-4920-94dc-65d6a62b96fa\") " pod="openshift-insights/insights-runtime-extractor-dvkw7" Jul 30 15:10:53.074522 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:10:53.074434 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/9f3ebd63-d1cc-4920-94dc-65d6a62b96fa-data-volume\") pod \"insights-runtime-extractor-dvkw7\" (UID: \"9f3ebd63-d1cc-4920-94dc-65d6a62b96fa\") " pod="openshift-insights/insights-runtime-extractor-dvkw7" Jul 30 15:10:53.074522 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:10:53.074460 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pb9xv\" (UniqueName: \"kubernetes.io/projected/c460206a-fa1f-4c09-b498-467313191c13-kube-api-access-pb9xv\") pod \"dns-default-jwbls\" (UID: \"c460206a-fa1f-4c09-b498-467313191c13\") " pod="openshift-dns/dns-default-jwbls" Jul 30 15:10:53.074522 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:10:53.074485 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/647c67b8-98e9-4905-9e04-35cc6cc3bd33-cert\") pod \"ingress-canary-qtv7l\" (UID: \"647c67b8-98e9-4905-9e04-35cc6cc3bd33\") " pod="openshift-ingress-canary/ingress-canary-qtv7l" Jul 30 15:10:53.074680 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:10:53.074556 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/c460206a-fa1f-4c09-b498-467313191c13-metrics-tls\") pod \"dns-default-jwbls\" (UID: \"c460206a-fa1f-4c09-b498-467313191c13\") " pod="openshift-dns/dns-default-jwbls" Jul 30 15:10:53.074680 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:10:53.074598 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/c460206a-fa1f-4c09-b498-467313191c13-config-volume\") pod \"dns-default-jwbls\" (UID: \"c460206a-fa1f-4c09-b498-467313191c13\") " pod="openshift-dns/dns-default-jwbls" Jul 30 15:10:53.074680 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:10:53.074629 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2snzx\" (UniqueName: \"kubernetes.io/projected/9f3ebd63-d1cc-4920-94dc-65d6a62b96fa-kube-api-access-2snzx\") pod \"insights-runtime-extractor-dvkw7\" (UID: \"9f3ebd63-d1cc-4920-94dc-65d6a62b96fa\") " pod="openshift-insights/insights-runtime-extractor-dvkw7" Jul 30 15:10:53.110380 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:10:53.110345 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7j468_a2197c57-2dc6-410e-9910-a045dedcb8d9/konnectivity-agent/0.log" limit=1024 Jul 30 15:10:53.175235 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:10:53.175139 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/9f3ebd63-d1cc-4920-94dc-65d6a62b96fa-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-dvkw7\" (UID: \"9f3ebd63-d1cc-4920-94dc-65d6a62b96fa\") " pod="openshift-insights/insights-runtime-extractor-dvkw7" Jul 30 15:10:53.175235 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:10:53.175210 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/9f3ebd63-d1cc-4920-94dc-65d6a62b96fa-data-volume\") pod \"insights-runtime-extractor-dvkw7\" (UID: \"9f3ebd63-d1cc-4920-94dc-65d6a62b96fa\") " pod="openshift-insights/insights-runtime-extractor-dvkw7" Jul 30 15:10:53.175235 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:10:53.175233 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pb9xv\" (UniqueName: \"kubernetes.io/projected/c460206a-fa1f-4c09-b498-467313191c13-kube-api-access-pb9xv\") pod \"dns-default-jwbls\" (UID: \"c460206a-fa1f-4c09-b498-467313191c13\") " pod="openshift-dns/dns-default-jwbls" Jul 30 15:10:53.175817 ip-10-0-133-210 kubenswrapper[2576]: E0730 15:10:53.175303 2576 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 30 15:10:53.175817 ip-10-0-133-210 kubenswrapper[2576]: E0730 15:10:53.175388 2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/9f3ebd63-d1cc-4920-94dc-65d6a62b96fa-kube-rbac-proxy-cm podName:9f3ebd63-d1cc-4920-94dc-65d6a62b96fa nodeName:}" failed. No retries permitted until 2026-07-30 15:10:53.67536342 +0000 UTC m=+35.601116281 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/9f3ebd63-d1cc-4920-94dc-65d6a62b96fa-kube-rbac-proxy-cm") pod "insights-runtime-extractor-dvkw7" (UID: "9f3ebd63-d1cc-4920-94dc-65d6a62b96fa") : configmap "kube-rbac-proxy" not found Jul 30 15:10:53.175817 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:10:53.175414 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/647c67b8-98e9-4905-9e04-35cc6cc3bd33-cert\") pod \"ingress-canary-qtv7l\" (UID: \"647c67b8-98e9-4905-9e04-35cc6cc3bd33\") " pod="openshift-ingress-canary/ingress-canary-qtv7l" Jul 30 15:10:53.175817 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:10:53.175459 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/c460206a-fa1f-4c09-b498-467313191c13-metrics-tls\") pod \"dns-default-jwbls\" (UID: \"c460206a-fa1f-4c09-b498-467313191c13\") " pod="openshift-dns/dns-default-jwbls" Jul 30 15:10:53.175817 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:10:53.175489 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/c460206a-fa1f-4c09-b498-467313191c13-config-volume\") pod \"dns-default-jwbls\" (UID: \"c460206a-fa1f-4c09-b498-467313191c13\") " pod="openshift-dns/dns-default-jwbls" Jul 30 15:10:53.175817 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:10:53.175531 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2snzx\" (UniqueName: \"kubernetes.io/projected/9f3ebd63-d1cc-4920-94dc-65d6a62b96fa-kube-api-access-2snzx\") pod \"insights-runtime-extractor-dvkw7\" (UID: \"9f3ebd63-d1cc-4920-94dc-65d6a62b96fa\") " pod="openshift-insights/insights-runtime-extractor-dvkw7" Jul 30 15:10:53.175817 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:10:53.175560 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/9f3ebd63-d1cc-4920-94dc-65d6a62b96fa-data-volume\") pod \"insights-runtime-extractor-dvkw7\" (UID: \"9f3ebd63-d1cc-4920-94dc-65d6a62b96fa\") " pod="openshift-insights/insights-runtime-extractor-dvkw7" Jul 30 15:10:53.175817 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:10:53.175568 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/9f3ebd63-d1cc-4920-94dc-65d6a62b96fa-crio-socket\") pod \"insights-runtime-extractor-dvkw7\" (UID: \"9f3ebd63-d1cc-4920-94dc-65d6a62b96fa\") " pod="openshift-insights/insights-runtime-extractor-dvkw7" Jul 30 15:10:53.175817 ip-10-0-133-210 kubenswrapper[2576]: E0730 15:10:53.175593 2576 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 30 15:10:53.175817 ip-10-0-133-210 kubenswrapper[2576]: E0730 15:10:53.175643 2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c460206a-fa1f-4c09-b498-467313191c13-metrics-tls podName:c460206a-fa1f-4c09-b498-467313191c13 nodeName:}" failed. No retries permitted until 2026-07-30 15:10:53.675625187 +0000 UTC m=+35.601377814 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/c460206a-fa1f-4c09-b498-467313191c13-metrics-tls") pod "dns-default-jwbls" (UID: "c460206a-fa1f-4c09-b498-467313191c13") : secret "dns-default-metrics-tls" not found Jul 30 15:10:53.175817 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:10:53.175596 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c460206a-fa1f-4c09-b498-467313191c13-tmp-dir\") pod \"dns-default-jwbls\" (UID: \"c460206a-fa1f-4c09-b498-467313191c13\") " pod="openshift-dns/dns-default-jwbls" Jul 30 15:10:53.175817 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:10:53.175687 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/9f3ebd63-d1cc-4920-94dc-65d6a62b96fa-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-dvkw7\" (UID: \"9f3ebd63-d1cc-4920-94dc-65d6a62b96fa\") " pod="openshift-insights/insights-runtime-extractor-dvkw7" Jul 30 15:10:53.175817 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:10:53.175712 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9m2th\" (UniqueName: \"kubernetes.io/projected/647c67b8-98e9-4905-9e04-35cc6cc3bd33-kube-api-access-9m2th\") pod \"ingress-canary-qtv7l\" (UID: \"647c67b8-98e9-4905-9e04-35cc6cc3bd33\") " pod="openshift-ingress-canary/ingress-canary-qtv7l" Jul 30 15:10:53.175817 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:10:53.175825 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/9f3ebd63-d1cc-4920-94dc-65d6a62b96fa-crio-socket\") pod \"insights-runtime-extractor-dvkw7\" (UID: \"9f3ebd63-d1cc-4920-94dc-65d6a62b96fa\") " pod="openshift-insights/insights-runtime-extractor-dvkw7" Jul 30 15:10:53.176427 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:10:53.175876 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c460206a-fa1f-4c09-b498-467313191c13-tmp-dir\") pod \"dns-default-jwbls\" (UID: \"c460206a-fa1f-4c09-b498-467313191c13\") " pod="openshift-dns/dns-default-jwbls" Jul 30 15:10:53.176427 ip-10-0-133-210 kubenswrapper[2576]: E0730 15:10:53.175899 2576 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 30 15:10:53.176427 ip-10-0-133-210 kubenswrapper[2576]: E0730 15:10:53.175959 2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9f3ebd63-d1cc-4920-94dc-65d6a62b96fa-insights-runtime-extractor-tls podName:9f3ebd63-d1cc-4920-94dc-65d6a62b96fa nodeName:}" failed. No retries permitted until 2026-07-30 15:10:53.675946494 +0000 UTC m=+35.601699124 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/9f3ebd63-d1cc-4920-94dc-65d6a62b96fa-insights-runtime-extractor-tls") pod "insights-runtime-extractor-dvkw7" (UID: "9f3ebd63-d1cc-4920-94dc-65d6a62b96fa") : secret "insights-runtime-extractor-tls" not found Jul 30 15:10:53.176427 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:10:53.176251 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/c460206a-fa1f-4c09-b498-467313191c13-config-volume\") pod \"dns-default-jwbls\" (UID: \"c460206a-fa1f-4c09-b498-467313191c13\") " pod="openshift-dns/dns-default-jwbls" Jul 30 15:10:53.180216 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:10:53.180191 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/647c67b8-98e9-4905-9e04-35cc6cc3bd33-cert\") pod \"ingress-canary-qtv7l\" (UID: \"647c67b8-98e9-4905-9e04-35cc6cc3bd33\") " pod="openshift-ingress-canary/ingress-canary-qtv7l" Jul 30 15:10:53.192420 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:10:53.192394 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2snzx\" (UniqueName: \"kubernetes.io/projected/9f3ebd63-d1cc-4920-94dc-65d6a62b96fa-kube-api-access-2snzx\") pod \"insights-runtime-extractor-dvkw7\" (UID: \"9f3ebd63-d1cc-4920-94dc-65d6a62b96fa\") " pod="openshift-insights/insights-runtime-extractor-dvkw7" Jul 30 15:10:53.193354 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:10:53.193326 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9m2th\" (UniqueName: \"kubernetes.io/projected/647c67b8-98e9-4905-9e04-35cc6cc3bd33-kube-api-access-9m2th\") pod \"ingress-canary-qtv7l\" (UID: \"647c67b8-98e9-4905-9e04-35cc6cc3bd33\") " pod="openshift-ingress-canary/ingress-canary-qtv7l" Jul 30 15:10:53.193494 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:10:53.193458 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-pb9xv\" (UniqueName: \"kubernetes.io/projected/c460206a-fa1f-4c09-b498-467313191c13-kube-api-access-pb9xv\") pod \"dns-default-jwbls\" (UID: \"c460206a-fa1f-4c09-b498-467313191c13\") " pod="openshift-dns/dns-default-jwbls" Jul 30 15:10:53.218622 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:10:53.218585 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7j468_a2197c57-2dc6-410e-9910-a045dedcb8d9/konnectivity-agent/0.log" limit=1024 Jul 30 15:10:53.250344 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:10:53.250303 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-qtv7l" Jul 30 15:10:53.253326 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:10:53.253292 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-j74qd_347bd19f-7728-47be-986d-337d0f0df246/dns-node-resolver/0.log" Jul 30 15:10:53.607428 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:10:53.607356 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-q858z" Jul 30 15:10:53.607582 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:10:53.607470 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-qvn96" Jul 30 15:10:53.607649 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:10:53.607598 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-dbmt8" Jul 30 15:10:53.611597 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:10:53.611575 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-kvhp8\"" Jul 30 15:10:53.612776 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:10:53.612695 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-52dkp\"" Jul 30 15:10:53.612776 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:10:53.612704 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" Jul 30 15:10:53.612776 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:10:53.612731 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Jul 30 15:10:53.612776 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:10:53.612758 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" Jul 30 15:10:53.613034 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:10:53.612920 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Jul 30 15:10:53.680720 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:10:53.680687 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/9f3ebd63-d1cc-4920-94dc-65d6a62b96fa-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-dvkw7\" (UID: \"9f3ebd63-d1cc-4920-94dc-65d6a62b96fa\") " pod="openshift-insights/insights-runtime-extractor-dvkw7" Jul 30 15:10:53.680895 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:10:53.680748 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/c460206a-fa1f-4c09-b498-467313191c13-metrics-tls\") pod \"dns-default-jwbls\" (UID: \"c460206a-fa1f-4c09-b498-467313191c13\") " pod="openshift-dns/dns-default-jwbls" Jul 30 15:10:53.680895 ip-10-0-133-210 kubenswrapper[2576]: E0730 15:10:53.680815 2576 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 30 15:10:53.680895 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:10:53.680822 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/9f3ebd63-d1cc-4920-94dc-65d6a62b96fa-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-dvkw7\" (UID: \"9f3ebd63-d1cc-4920-94dc-65d6a62b96fa\") " pod="openshift-insights/insights-runtime-extractor-dvkw7" Jul 30 15:10:53.680895 ip-10-0-133-210 kubenswrapper[2576]: E0730 15:10:53.680855 2576 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 30 15:10:53.680895 ip-10-0-133-210 kubenswrapper[2576]: E0730 15:10:53.680879 2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/9f3ebd63-d1cc-4920-94dc-65d6a62b96fa-kube-rbac-proxy-cm podName:9f3ebd63-d1cc-4920-94dc-65d6a62b96fa nodeName:}" failed. No retries permitted until 2026-07-30 15:10:54.680858627 +0000 UTC m=+36.606611250 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/9f3ebd63-d1cc-4920-94dc-65d6a62b96fa-kube-rbac-proxy-cm") pod "insights-runtime-extractor-dvkw7" (UID: "9f3ebd63-d1cc-4920-94dc-65d6a62b96fa") : configmap "kube-rbac-proxy" not found Jul 30 15:10:53.681105 ip-10-0-133-210 kubenswrapper[2576]: E0730 15:10:53.680914 2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c460206a-fa1f-4c09-b498-467313191c13-metrics-tls podName:c460206a-fa1f-4c09-b498-467313191c13 nodeName:}" failed. No retries permitted until 2026-07-30 15:10:54.680897943 +0000 UTC m=+36.606650583 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/c460206a-fa1f-4c09-b498-467313191c13-metrics-tls") pod "dns-default-jwbls" (UID: "c460206a-fa1f-4c09-b498-467313191c13") : secret "dns-default-metrics-tls" not found Jul 30 15:10:53.681105 ip-10-0-133-210 kubenswrapper[2576]: E0730 15:10:53.680919 2576 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 30 15:10:53.681105 ip-10-0-133-210 kubenswrapper[2576]: E0730 15:10:53.680956 2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9f3ebd63-d1cc-4920-94dc-65d6a62b96fa-insights-runtime-extractor-tls podName:9f3ebd63-d1cc-4920-94dc-65d6a62b96fa nodeName:}" failed. No retries permitted until 2026-07-30 15:10:54.680944257 +0000 UTC m=+36.606696880 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/9f3ebd63-d1cc-4920-94dc-65d6a62b96fa-insights-runtime-extractor-tls") pod "insights-runtime-extractor-dvkw7" (UID: "9f3ebd63-d1cc-4920-94dc-65d6a62b96fa") : secret "insights-runtime-extractor-tls" not found Jul 30 15:10:53.913221 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:10:53.912981 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-qtv7l"] Jul 30 15:10:53.915919 ip-10-0-133-210 kubenswrapper[2576]: W0730 15:10:53.915895 2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod647c67b8_98e9_4905_9e04_35cc6cc3bd33.slice/crio-72a7b614d75ac759a4e690add9d6a3f72113cda40440bf95f5748fc70a61653e WatchSource:0}: Error finding container 72a7b614d75ac759a4e690add9d6a3f72113cda40440bf95f5748fc70a61653e: Status 404 returned error can't find the container with id 72a7b614d75ac759a4e690add9d6a3f72113cda40440bf95f5748fc70a61653e Jul 30 15:10:54.448632 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:10:54.448607 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-ktmgn_4b006058-0163-4e82-9a1f-979b182ddbf1/node-ca/0.log" Jul 30 15:10:54.690593 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:10:54.690536 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/c460206a-fa1f-4c09-b498-467313191c13-metrics-tls\") pod \"dns-default-jwbls\" (UID: \"c460206a-fa1f-4c09-b498-467313191c13\") " pod="openshift-dns/dns-default-jwbls" Jul 30 15:10:54.690802 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:10:54.690638 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/9f3ebd63-d1cc-4920-94dc-65d6a62b96fa-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-dvkw7\" (UID: \"9f3ebd63-d1cc-4920-94dc-65d6a62b96fa\") " pod="openshift-insights/insights-runtime-extractor-dvkw7" Jul 30 15:10:54.690802 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:10:54.690679 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/9f3ebd63-d1cc-4920-94dc-65d6a62b96fa-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-dvkw7\" (UID: \"9f3ebd63-d1cc-4920-94dc-65d6a62b96fa\") " pod="openshift-insights/insights-runtime-extractor-dvkw7" Jul 30 15:10:54.690802 ip-10-0-133-210 kubenswrapper[2576]: E0730 15:10:54.690705 2576 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 30 15:10:54.690802 ip-10-0-133-210 kubenswrapper[2576]: E0730 15:10:54.690775 2576 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 30 15:10:54.690802 ip-10-0-133-210 kubenswrapper[2576]: E0730 15:10:54.690798 2576 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 30 15:10:54.691013 ip-10-0-133-210 kubenswrapper[2576]: E0730 15:10:54.690779 2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c460206a-fa1f-4c09-b498-467313191c13-metrics-tls podName:c460206a-fa1f-4c09-b498-467313191c13 nodeName:}" failed. No retries permitted until 2026-07-30 15:10:56.690763865 +0000 UTC m=+38.616516488 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/c460206a-fa1f-4c09-b498-467313191c13-metrics-tls") pod "dns-default-jwbls" (UID: "c460206a-fa1f-4c09-b498-467313191c13") : secret "dns-default-metrics-tls" not found Jul 30 15:10:54.691013 ip-10-0-133-210 kubenswrapper[2576]: E0730 15:10:54.690849 2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/9f3ebd63-d1cc-4920-94dc-65d6a62b96fa-kube-rbac-proxy-cm podName:9f3ebd63-d1cc-4920-94dc-65d6a62b96fa nodeName:}" failed. No retries permitted until 2026-07-30 15:10:56.690833989 +0000 UTC m=+38.616586611 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/9f3ebd63-d1cc-4920-94dc-65d6a62b96fa-kube-rbac-proxy-cm") pod "insights-runtime-extractor-dvkw7" (UID: "9f3ebd63-d1cc-4920-94dc-65d6a62b96fa") : configmap "kube-rbac-proxy" not found Jul 30 15:10:54.691013 ip-10-0-133-210 kubenswrapper[2576]: E0730 15:10:54.690899 2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9f3ebd63-d1cc-4920-94dc-65d6a62b96fa-insights-runtime-extractor-tls podName:9f3ebd63-d1cc-4920-94dc-65d6a62b96fa nodeName:}" failed. No retries permitted until 2026-07-30 15:10:56.690886394 +0000 UTC m=+38.616639021 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/9f3ebd63-d1cc-4920-94dc-65d6a62b96fa-insights-runtime-extractor-tls") pod "insights-runtime-extractor-dvkw7" (UID: "9f3ebd63-d1cc-4920-94dc-65d6a62b96fa") : secret "insights-runtime-extractor-tls" not found Jul 30 15:10:54.742872 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:10:54.742790 2576 generic.go:356] "Generic (PLEG): container finished" podID="f62011d0-f5be-413a-93a2-8a7a98b67130" containerID="7032ba6da25aa8d739a42b417a82ec4358f1f06ea12bf086ea9cc307631ff18c" exitCode=0 Jul 30 15:10:54.742872 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:10:54.742847 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-m2j4z" event={"ID":"f62011d0-f5be-413a-93a2-8a7a98b67130","Type":"ContainerDied","Data":"7032ba6da25aa8d739a42b417a82ec4358f1f06ea12bf086ea9cc307631ff18c"} Jul 30 15:10:54.744042 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:10:54.744018 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-qtv7l" event={"ID":"647c67b8-98e9-4905-9e04-35cc6cc3bd33","Type":"ContainerStarted","Data":"72a7b614d75ac759a4e690add9d6a3f72113cda40440bf95f5748fc70a61653e"} Jul 30 15:10:55.750711 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:10:55.750673 2576 generic.go:356] "Generic (PLEG): container finished" podID="f62011d0-f5be-413a-93a2-8a7a98b67130" containerID="df1e79c9630f05ba5041ddb76e0734bd8df73a7c94ee15368f34fae349dcc0fb" exitCode=0 Jul 30 15:10:55.751127 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:10:55.750750 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-m2j4z" event={"ID":"f62011d0-f5be-413a-93a2-8a7a98b67130","Type":"ContainerDied","Data":"df1e79c9630f05ba5041ddb76e0734bd8df73a7c94ee15368f34fae349dcc0fb"} Jul 30 15:10:56.708591 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:10:56.708535 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/9f3ebd63-d1cc-4920-94dc-65d6a62b96fa-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-dvkw7\" (UID: \"9f3ebd63-d1cc-4920-94dc-65d6a62b96fa\") " pod="openshift-insights/insights-runtime-extractor-dvkw7" Jul 30 15:10:56.708772 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:10:56.708617 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/9f3ebd63-d1cc-4920-94dc-65d6a62b96fa-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-dvkw7\" (UID: \"9f3ebd63-d1cc-4920-94dc-65d6a62b96fa\") " pod="openshift-insights/insights-runtime-extractor-dvkw7" Jul 30 15:10:56.708772 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:10:56.708678 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/c460206a-fa1f-4c09-b498-467313191c13-metrics-tls\") pod \"dns-default-jwbls\" (UID: \"c460206a-fa1f-4c09-b498-467313191c13\") " pod="openshift-dns/dns-default-jwbls" Jul 30 15:10:56.708772 ip-10-0-133-210 kubenswrapper[2576]: E0730 15:10:56.708695 2576 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 30 15:10:56.708772 ip-10-0-133-210 kubenswrapper[2576]: E0730 15:10:56.708754 2576 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 30 15:10:56.708772 ip-10-0-133-210 kubenswrapper[2576]: E0730 15:10:56.708757 2576 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 30 15:10:56.709034 ip-10-0-133-210 kubenswrapper[2576]: E0730 15:10:56.708758 2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9f3ebd63-d1cc-4920-94dc-65d6a62b96fa-insights-runtime-extractor-tls podName:9f3ebd63-d1cc-4920-94dc-65d6a62b96fa nodeName:}" failed. No retries permitted until 2026-07-30 15:11:00.708739111 +0000 UTC m=+42.634491755 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/9f3ebd63-d1cc-4920-94dc-65d6a62b96fa-insights-runtime-extractor-tls") pod "insights-runtime-extractor-dvkw7" (UID: "9f3ebd63-d1cc-4920-94dc-65d6a62b96fa") : secret "insights-runtime-extractor-tls" not found Jul 30 15:10:56.709034 ip-10-0-133-210 kubenswrapper[2576]: E0730 15:10:56.708796 2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c460206a-fa1f-4c09-b498-467313191c13-metrics-tls podName:c460206a-fa1f-4c09-b498-467313191c13 nodeName:}" failed. No retries permitted until 2026-07-30 15:11:00.708786683 +0000 UTC m=+42.634539312 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/c460206a-fa1f-4c09-b498-467313191c13-metrics-tls") pod "dns-default-jwbls" (UID: "c460206a-fa1f-4c09-b498-467313191c13") : secret "dns-default-metrics-tls" not found Jul 30 15:10:56.709034 ip-10-0-133-210 kubenswrapper[2576]: E0730 15:10:56.708832 2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/9f3ebd63-d1cc-4920-94dc-65d6a62b96fa-kube-rbac-proxy-cm podName:9f3ebd63-d1cc-4920-94dc-65d6a62b96fa nodeName:}" failed. No retries permitted until 2026-07-30 15:11:00.708809736 +0000 UTC m=+42.634562376 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/9f3ebd63-d1cc-4920-94dc-65d6a62b96fa-kube-rbac-proxy-cm") pod "insights-runtime-extractor-dvkw7" (UID: "9f3ebd63-d1cc-4920-94dc-65d6a62b96fa") : configmap "kube-rbac-proxy" not found Jul 30 15:10:56.757361 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:10:56.757323 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-m2j4z" event={"ID":"f62011d0-f5be-413a-93a2-8a7a98b67130","Type":"ContainerStarted","Data":"8cf15cd772473c6cec8113c1086811a0fd36e8c9bd85df43850fd42c13e05a19"} Jul 30 15:10:56.781696 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:10:56.781633 2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-m2j4z" podStartSLOduration=9.278754616 podStartE2EDuration="38.781612461s" totalImagesPullingTime="29.502857845s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-30 15:10:18 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 15:10:56.7796118 +0000 UTC m=+38.705364446" watchObservedRunningTime="2026-07-30 15:10:56.781612461 +0000 UTC m=+38.707365105" Jul 30 15:10:57.217917 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:10:57.217879 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7j468_a2197c57-2dc6-410e-9910-a045dedcb8d9/konnectivity-agent/0.log" limit=1024 Jul 30 15:10:57.321454 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:10:57.321420 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7j468_a2197c57-2dc6-410e-9910-a045dedcb8d9/konnectivity-agent/0.log" limit=1024 Jul 30 15:10:57.764537 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:10:57.764499 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-qtv7l" event={"ID":"647c67b8-98e9-4905-9e04-35cc6cc3bd33","Type":"ContainerStarted","Data":"f5359d1ab968c55d12d94f346e142b6475eec1e127720128a90a7d58890d1a38"} Jul 30 15:10:57.782317 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:10:57.782261 2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-qtv7l" podStartSLOduration=2.501868607 podStartE2EDuration="5.782244788s" totalImagesPullingTime="3.280376181s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 15:10:52 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 15:10:57.781481164 +0000 UTC m=+39.707233811" watchObservedRunningTime="2026-07-30 15:10:57.782244788 +0000 UTC m=+39.707997410" Jul 30 15:11:00.740987 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:11:00.740945 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/9f3ebd63-d1cc-4920-94dc-65d6a62b96fa-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-dvkw7\" (UID: \"9f3ebd63-d1cc-4920-94dc-65d6a62b96fa\") " pod="openshift-insights/insights-runtime-extractor-dvkw7" Jul 30 15:11:00.741459 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:11:00.741010 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/c460206a-fa1f-4c09-b498-467313191c13-metrics-tls\") pod \"dns-default-jwbls\" (UID: \"c460206a-fa1f-4c09-b498-467313191c13\") " pod="openshift-dns/dns-default-jwbls" Jul 30 15:11:00.741459 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:11:00.741052 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/9f3ebd63-d1cc-4920-94dc-65d6a62b96fa-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-dvkw7\" (UID: \"9f3ebd63-d1cc-4920-94dc-65d6a62b96fa\") " pod="openshift-insights/insights-runtime-extractor-dvkw7" Jul 30 15:11:00.741459 ip-10-0-133-210 kubenswrapper[2576]: E0730 15:11:00.741071 2576 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 30 15:11:00.741459 ip-10-0-133-210 kubenswrapper[2576]: E0730 15:11:00.741162 2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/9f3ebd63-d1cc-4920-94dc-65d6a62b96fa-kube-rbac-proxy-cm podName:9f3ebd63-d1cc-4920-94dc-65d6a62b96fa nodeName:}" failed. No retries permitted until 2026-07-30 15:11:08.741143436 +0000 UTC m=+50.666896060 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/9f3ebd63-d1cc-4920-94dc-65d6a62b96fa-kube-rbac-proxy-cm") pod "insights-runtime-extractor-dvkw7" (UID: "9f3ebd63-d1cc-4920-94dc-65d6a62b96fa") : configmap "kube-rbac-proxy" not found Jul 30 15:11:00.741459 ip-10-0-133-210 kubenswrapper[2576]: E0730 15:11:00.741205 2576 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 30 15:11:00.741459 ip-10-0-133-210 kubenswrapper[2576]: E0730 15:11:00.741209 2576 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 30 15:11:00.741459 ip-10-0-133-210 kubenswrapper[2576]: E0730 15:11:00.741243 2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9f3ebd63-d1cc-4920-94dc-65d6a62b96fa-insights-runtime-extractor-tls podName:9f3ebd63-d1cc-4920-94dc-65d6a62b96fa nodeName:}" failed. No retries permitted until 2026-07-30 15:11:08.741232831 +0000 UTC m=+50.666985455 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/9f3ebd63-d1cc-4920-94dc-65d6a62b96fa-insights-runtime-extractor-tls") pod "insights-runtime-extractor-dvkw7" (UID: "9f3ebd63-d1cc-4920-94dc-65d6a62b96fa") : secret "insights-runtime-extractor-tls" not found Jul 30 15:11:00.741459 ip-10-0-133-210 kubenswrapper[2576]: E0730 15:11:00.741256 2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c460206a-fa1f-4c09-b498-467313191c13-metrics-tls podName:c460206a-fa1f-4c09-b498-467313191c13 nodeName:}" failed. No retries permitted until 2026-07-30 15:11:08.741249813 +0000 UTC m=+50.667002436 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/c460206a-fa1f-4c09-b498-467313191c13-metrics-tls") pod "dns-default-jwbls" (UID: "c460206a-fa1f-4c09-b498-467313191c13") : secret "dns-default-metrics-tls" not found Jul 30 15:11:02.173023 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:11:02.172982 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7j468_a2197c57-2dc6-410e-9910-a045dedcb8d9/konnectivity-agent/0.log" limit=1024 Jul 30 15:11:04.258096 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:11:04.258052 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7j468_a2197c57-2dc6-410e-9910-a045dedcb8d9/konnectivity-agent/0.log" limit=1024 Jul 30 15:11:04.352455 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:11:04.352410 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7j468_a2197c57-2dc6-410e-9910-a045dedcb8d9/konnectivity-agent/0.log" limit=1024 Jul 30 15:11:06.457056 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:11:06.457024 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7j468_a2197c57-2dc6-410e-9910-a045dedcb8d9/konnectivity-agent/0.log" limit=1024 Jul 30 15:11:06.548297 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:11:06.548265 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7j468_a2197c57-2dc6-410e-9910-a045dedcb8d9/konnectivity-agent/0.log" limit=1024 Jul 30 15:11:08.800755 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:11:08.800724 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/9f3ebd63-d1cc-4920-94dc-65d6a62b96fa-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-dvkw7\" (UID: \"9f3ebd63-d1cc-4920-94dc-65d6a62b96fa\") " pod="openshift-insights/insights-runtime-extractor-dvkw7" Jul 30 15:11:08.801163 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:11:08.800766 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/9f3ebd63-d1cc-4920-94dc-65d6a62b96fa-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-dvkw7\" (UID: \"9f3ebd63-d1cc-4920-94dc-65d6a62b96fa\") " pod="openshift-insights/insights-runtime-extractor-dvkw7" Jul 30 15:11:08.801163 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:11:08.800808 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/c460206a-fa1f-4c09-b498-467313191c13-metrics-tls\") pod \"dns-default-jwbls\" (UID: \"c460206a-fa1f-4c09-b498-467313191c13\") " pod="openshift-dns/dns-default-jwbls" Jul 30 15:11:08.801163 ip-10-0-133-210 kubenswrapper[2576]: E0730 15:11:08.800882 2576 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 30 15:11:08.801163 ip-10-0-133-210 kubenswrapper[2576]: E0730 15:11:08.800884 2576 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 30 15:11:08.801163 ip-10-0-133-210 kubenswrapper[2576]: E0730 15:11:08.800930 2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c460206a-fa1f-4c09-b498-467313191c13-metrics-tls podName:c460206a-fa1f-4c09-b498-467313191c13 nodeName:}" failed. No retries permitted until 2026-07-30 15:11:24.800917894 +0000 UTC m=+66.726670516 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/c460206a-fa1f-4c09-b498-467313191c13-metrics-tls") pod "dns-default-jwbls" (UID: "c460206a-fa1f-4c09-b498-467313191c13") : secret "dns-default-metrics-tls" not found Jul 30 15:11:08.801163 ip-10-0-133-210 kubenswrapper[2576]: E0730 15:11:08.800942 2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9f3ebd63-d1cc-4920-94dc-65d6a62b96fa-insights-runtime-extractor-tls podName:9f3ebd63-d1cc-4920-94dc-65d6a62b96fa nodeName:}" failed. No retries permitted until 2026-07-30 15:11:24.80093658 +0000 UTC m=+66.726689202 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/9f3ebd63-d1cc-4920-94dc-65d6a62b96fa-insights-runtime-extractor-tls") pod "insights-runtime-extractor-dvkw7" (UID: "9f3ebd63-d1cc-4920-94dc-65d6a62b96fa") : secret "insights-runtime-extractor-tls" not found Jul 30 15:11:08.801386 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:11:08.801337 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/9f3ebd63-d1cc-4920-94dc-65d6a62b96fa-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-dvkw7\" (UID: \"9f3ebd63-d1cc-4920-94dc-65d6a62b96fa\") " pod="openshift-insights/insights-runtime-extractor-dvkw7" Jul 30 15:11:10.400637 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:11:10.400601 2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-network-console/networking-console-plugin-74c9f884b7-lcg2q"] Jul 30 15:11:10.418082 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:11:10.418049 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-console/networking-console-plugin-74c9f884b7-lcg2q"] Jul 30 15:11:10.418254 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:11:10.418185 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-console/networking-console-plugin-74c9f884b7-lcg2q" Jul 30 15:11:10.421123 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:11:10.421085 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-console\"/\"networking-console-plugin\"" Jul 30 15:11:10.421417 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:11:10.421399 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"networking-console-plugin-cert\"" Jul 30 15:11:10.421606 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:11:10.421594 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"default-dockercfg-6ftrj\"" Jul 30 15:11:10.515979 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:11:10.515939 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/4ce0180e-5f58-4c5c-bffd-be53eb55a430-nginx-conf\") pod \"networking-console-plugin-74c9f884b7-lcg2q\" (UID: \"4ce0180e-5f58-4c5c-bffd-be53eb55a430\") " pod="openshift-network-console/networking-console-plugin-74c9f884b7-lcg2q" Jul 30 15:11:10.515979 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:11:10.515977 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/4ce0180e-5f58-4c5c-bffd-be53eb55a430-networking-console-plugin-cert\") pod \"networking-console-plugin-74c9f884b7-lcg2q\" (UID: \"4ce0180e-5f58-4c5c-bffd-be53eb55a430\") " pod="openshift-network-console/networking-console-plugin-74c9f884b7-lcg2q" Jul 30 15:11:10.617319 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:11:10.617291 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/4ce0180e-5f58-4c5c-bffd-be53eb55a430-nginx-conf\") pod \"networking-console-plugin-74c9f884b7-lcg2q\" (UID: \"4ce0180e-5f58-4c5c-bffd-be53eb55a430\") " pod="openshift-network-console/networking-console-plugin-74c9f884b7-lcg2q" Jul 30 15:11:10.617319 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:11:10.617321 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/4ce0180e-5f58-4c5c-bffd-be53eb55a430-networking-console-plugin-cert\") pod \"networking-console-plugin-74c9f884b7-lcg2q\" (UID: \"4ce0180e-5f58-4c5c-bffd-be53eb55a430\") " pod="openshift-network-console/networking-console-plugin-74c9f884b7-lcg2q" Jul 30 15:11:10.617570 ip-10-0-133-210 kubenswrapper[2576]: E0730 15:11:10.617512 2576 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found Jul 30 15:11:10.617630 ip-10-0-133-210 kubenswrapper[2576]: E0730 15:11:10.617581 2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4ce0180e-5f58-4c5c-bffd-be53eb55a430-networking-console-plugin-cert podName:4ce0180e-5f58-4c5c-bffd-be53eb55a430 nodeName:}" failed. No retries permitted until 2026-07-30 15:11:11.117566069 +0000 UTC m=+53.043318692 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/4ce0180e-5f58-4c5c-bffd-be53eb55a430-networking-console-plugin-cert") pod "networking-console-plugin-74c9f884b7-lcg2q" (UID: "4ce0180e-5f58-4c5c-bffd-be53eb55a430") : secret "networking-console-plugin-cert" not found Jul 30 15:11:10.617980 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:11:10.617956 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/4ce0180e-5f58-4c5c-bffd-be53eb55a430-nginx-conf\") pod \"networking-console-plugin-74c9f884b7-lcg2q\" (UID: \"4ce0180e-5f58-4c5c-bffd-be53eb55a430\") " pod="openshift-network-console/networking-console-plugin-74c9f884b7-lcg2q" Jul 30 15:11:10.627774 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:11:10.627746 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7j468_a2197c57-2dc6-410e-9910-a045dedcb8d9/konnectivity-agent/0.log" limit=1024 Jul 30 15:11:10.718797 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:11:10.718766 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7j468_a2197c57-2dc6-410e-9910-a045dedcb8d9/konnectivity-agent/0.log" limit=1024 Jul 30 15:11:11.120572 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:11:11.120476 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/4ce0180e-5f58-4c5c-bffd-be53eb55a430-networking-console-plugin-cert\") pod \"networking-console-plugin-74c9f884b7-lcg2q\" (UID: \"4ce0180e-5f58-4c5c-bffd-be53eb55a430\") " pod="openshift-network-console/networking-console-plugin-74c9f884b7-lcg2q" Jul 30 15:11:11.120731 ip-10-0-133-210 kubenswrapper[2576]: E0730 15:11:11.120625 2576 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found Jul 30 15:11:11.120731 ip-10-0-133-210 kubenswrapper[2576]: E0730 15:11:11.120708 2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4ce0180e-5f58-4c5c-bffd-be53eb55a430-networking-console-plugin-cert podName:4ce0180e-5f58-4c5c-bffd-be53eb55a430 nodeName:}" failed. No retries permitted until 2026-07-30 15:11:12.120676537 +0000 UTC m=+54.046429160 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/4ce0180e-5f58-4c5c-bffd-be53eb55a430-networking-console-plugin-cert") pod "networking-console-plugin-74c9f884b7-lcg2q" (UID: "4ce0180e-5f58-4c5c-bffd-be53eb55a430") : secret "networking-console-plugin-cert" not found Jul 30 15:11:12.127281 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:11:12.127233 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/4ce0180e-5f58-4c5c-bffd-be53eb55a430-networking-console-plugin-cert\") pod \"networking-console-plugin-74c9f884b7-lcg2q\" (UID: \"4ce0180e-5f58-4c5c-bffd-be53eb55a430\") " pod="openshift-network-console/networking-console-plugin-74c9f884b7-lcg2q" Jul 30 15:11:12.127866 ip-10-0-133-210 kubenswrapper[2576]: E0730 15:11:12.127412 2576 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found Jul 30 15:11:12.127866 ip-10-0-133-210 kubenswrapper[2576]: E0730 15:11:12.127500 2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4ce0180e-5f58-4c5c-bffd-be53eb55a430-networking-console-plugin-cert podName:4ce0180e-5f58-4c5c-bffd-be53eb55a430 nodeName:}" failed. No retries permitted until 2026-07-30 15:11:14.127478072 +0000 UTC m=+56.053230695 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/4ce0180e-5f58-4c5c-bffd-be53eb55a430-networking-console-plugin-cert") pod "networking-console-plugin-74c9f884b7-lcg2q" (UID: "4ce0180e-5f58-4c5c-bffd-be53eb55a430") : secret "networking-console-plugin-cert" not found Jul 30 15:11:14.141305 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:11:14.141251 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/4ce0180e-5f58-4c5c-bffd-be53eb55a430-networking-console-plugin-cert\") pod \"networking-console-plugin-74c9f884b7-lcg2q\" (UID: \"4ce0180e-5f58-4c5c-bffd-be53eb55a430\") " pod="openshift-network-console/networking-console-plugin-74c9f884b7-lcg2q" Jul 30 15:11:14.141779 ip-10-0-133-210 kubenswrapper[2576]: E0730 15:11:14.141378 2576 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found Jul 30 15:11:14.141779 ip-10-0-133-210 kubenswrapper[2576]: E0730 15:11:14.141434 2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4ce0180e-5f58-4c5c-bffd-be53eb55a430-networking-console-plugin-cert podName:4ce0180e-5f58-4c5c-bffd-be53eb55a430 nodeName:}" failed. No retries permitted until 2026-07-30 15:11:18.141421945 +0000 UTC m=+60.067174568 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/4ce0180e-5f58-4c5c-bffd-be53eb55a430-networking-console-plugin-cert") pod "networking-console-plugin-74c9f884b7-lcg2q" (UID: "4ce0180e-5f58-4c5c-bffd-be53eb55a430") : secret "networking-console-plugin-cert" not found Jul 30 15:11:14.852077 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:11:14.852023 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7j468_a2197c57-2dc6-410e-9910-a045dedcb8d9/konnectivity-agent/0.log" limit=1024 Jul 30 15:11:14.950433 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:11:14.950401 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7j468_a2197c57-2dc6-410e-9910-a045dedcb8d9/konnectivity-agent/0.log" limit=1024 Jul 30 15:11:15.275255 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:11:15.275220 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7j468_a2197c57-2dc6-410e-9910-a045dedcb8d9/konnectivity-agent/0.log" limit=1024 Jul 30 15:11:15.339149 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:11:15.339102 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7j468_a2197c57-2dc6-410e-9910-a045dedcb8d9/konnectivity-agent/0.log" limit=1024 Jul 30 15:11:15.583301 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:11:15.583198 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7j468_a2197c57-2dc6-410e-9910-a045dedcb8d9/konnectivity-agent/0.log" limit=1024 Jul 30 15:11:15.676613 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:11:15.676580 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7j468_a2197c57-2dc6-410e-9910-a045dedcb8d9/konnectivity-agent/0.log" limit=1024 Jul 30 15:11:17.198557 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:11:17.198524 2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-service-ca/service-ca-658d7b7c78-vdhkt"] Jul 30 15:11:17.200771 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:11:17.200754 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca/service-ca-658d7b7c78-vdhkt" Jul 30 15:11:17.204355 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:11:17.204334 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"kube-root-ca.crt\"" Jul 30 15:11:17.205035 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:11:17.205012 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca\"/\"service-ca-dockercfg-s8cgw\"" Jul 30 15:11:17.205366 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:11:17.205350 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca\"/\"signing-key\"" Jul 30 15:11:17.205435 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:11:17.205384 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"signing-cabundle\"" Jul 30 15:11:17.205510 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:11:17.205489 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"openshift-service-ca.crt\"" Jul 30 15:11:17.214072 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:11:17.214045 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca/service-ca-658d7b7c78-vdhkt"] Jul 30 15:11:17.263624 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:11:17.263584 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/f2e567f3-cbda-4dcc-9d78-2ea3a6344ad2-signing-cabundle\") pod \"service-ca-658d7b7c78-vdhkt\" (UID: \"f2e567f3-cbda-4dcc-9d78-2ea3a6344ad2\") " pod="openshift-service-ca/service-ca-658d7b7c78-vdhkt" Jul 30 15:11:17.263809 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:11:17.263630 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bsbdg\" (UniqueName: \"kubernetes.io/projected/f2e567f3-cbda-4dcc-9d78-2ea3a6344ad2-kube-api-access-bsbdg\") pod \"service-ca-658d7b7c78-vdhkt\" (UID: \"f2e567f3-cbda-4dcc-9d78-2ea3a6344ad2\") " pod="openshift-service-ca/service-ca-658d7b7c78-vdhkt" Jul 30 15:11:17.263809 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:11:17.263698 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/f2e567f3-cbda-4dcc-9d78-2ea3a6344ad2-signing-key\") pod \"service-ca-658d7b7c78-vdhkt\" (UID: \"f2e567f3-cbda-4dcc-9d78-2ea3a6344ad2\") " pod="openshift-service-ca/service-ca-658d7b7c78-vdhkt" Jul 30 15:11:17.364847 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:11:17.364794 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/f2e567f3-cbda-4dcc-9d78-2ea3a6344ad2-signing-key\") pod \"service-ca-658d7b7c78-vdhkt\" (UID: \"f2e567f3-cbda-4dcc-9d78-2ea3a6344ad2\") " pod="openshift-service-ca/service-ca-658d7b7c78-vdhkt" Jul 30 15:11:17.365034 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:11:17.364924 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/f2e567f3-cbda-4dcc-9d78-2ea3a6344ad2-signing-cabundle\") pod \"service-ca-658d7b7c78-vdhkt\" (UID: \"f2e567f3-cbda-4dcc-9d78-2ea3a6344ad2\") " pod="openshift-service-ca/service-ca-658d7b7c78-vdhkt" Jul 30 15:11:17.365034 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:11:17.364951 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bsbdg\" (UniqueName: \"kubernetes.io/projected/f2e567f3-cbda-4dcc-9d78-2ea3a6344ad2-kube-api-access-bsbdg\") pod \"service-ca-658d7b7c78-vdhkt\" (UID: \"f2e567f3-cbda-4dcc-9d78-2ea3a6344ad2\") " pod="openshift-service-ca/service-ca-658d7b7c78-vdhkt" Jul 30 15:11:17.365626 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:11:17.365606 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/f2e567f3-cbda-4dcc-9d78-2ea3a6344ad2-signing-cabundle\") pod \"service-ca-658d7b7c78-vdhkt\" (UID: \"f2e567f3-cbda-4dcc-9d78-2ea3a6344ad2\") " pod="openshift-service-ca/service-ca-658d7b7c78-vdhkt" Jul 30 15:11:17.368379 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:11:17.368350 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/f2e567f3-cbda-4dcc-9d78-2ea3a6344ad2-signing-key\") pod \"service-ca-658d7b7c78-vdhkt\" (UID: \"f2e567f3-cbda-4dcc-9d78-2ea3a6344ad2\") " pod="openshift-service-ca/service-ca-658d7b7c78-vdhkt" Jul 30 15:11:17.377686 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:11:17.377656 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bsbdg\" (UniqueName: \"kubernetes.io/projected/f2e567f3-cbda-4dcc-9d78-2ea3a6344ad2-kube-api-access-bsbdg\") pod \"service-ca-658d7b7c78-vdhkt\" (UID: \"f2e567f3-cbda-4dcc-9d78-2ea3a6344ad2\") " pod="openshift-service-ca/service-ca-658d7b7c78-vdhkt" Jul 30 15:11:17.509206 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:11:17.509173 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca/service-ca-658d7b7c78-vdhkt" Jul 30 15:11:17.635143 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:11:17.635068 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca/service-ca-658d7b7c78-vdhkt"] Jul 30 15:11:17.637664 ip-10-0-133-210 kubenswrapper[2576]: W0730 15:11:17.637623 2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf2e567f3_cbda_4dcc_9d78_2ea3a6344ad2.slice/crio-f13a6c27489489cf7b51e923c082a2778559138efba367e0a49d4a55e947e28f WatchSource:0}: Error finding container f13a6c27489489cf7b51e923c082a2778559138efba367e0a49d4a55e947e28f: Status 404 returned error can't find the container with id f13a6c27489489cf7b51e923c082a2778559138efba367e0a49d4a55e947e28f Jul 30 15:11:17.732655 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:11:17.732627 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-6mb29" Jul 30 15:11:17.812733 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:11:17.812646 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca/service-ca-658d7b7c78-vdhkt" event={"ID":"f2e567f3-cbda-4dcc-9d78-2ea3a6344ad2","Type":"ContainerStarted","Data":"f13a6c27489489cf7b51e923c082a2778559138efba367e0a49d4a55e947e28f"} Jul 30 15:11:17.972961 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:11:17.972926 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7j468_a2197c57-2dc6-410e-9910-a045dedcb8d9/konnectivity-agent/0.log" limit=1024 Jul 30 15:11:18.071134 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:11:18.071044 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7j468_a2197c57-2dc6-410e-9910-a045dedcb8d9/konnectivity-agent/0.log" limit=1024 Jul 30 15:11:18.171912 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:11:18.171871 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/4ce0180e-5f58-4c5c-bffd-be53eb55a430-networking-console-plugin-cert\") pod \"networking-console-plugin-74c9f884b7-lcg2q\" (UID: \"4ce0180e-5f58-4c5c-bffd-be53eb55a430\") " pod="openshift-network-console/networking-console-plugin-74c9f884b7-lcg2q" Jul 30 15:11:18.172089 ip-10-0-133-210 kubenswrapper[2576]: E0730 15:11:18.171996 2576 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found Jul 30 15:11:18.172089 ip-10-0-133-210 kubenswrapper[2576]: E0730 15:11:18.172057 2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4ce0180e-5f58-4c5c-bffd-be53eb55a430-networking-console-plugin-cert podName:4ce0180e-5f58-4c5c-bffd-be53eb55a430 nodeName:}" failed. No retries permitted until 2026-07-30 15:11:26.17204051 +0000 UTC m=+68.097793135 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/4ce0180e-5f58-4c5c-bffd-be53eb55a430-networking-console-plugin-cert") pod "networking-console-plugin-74c9f884b7-lcg2q" (UID: "4ce0180e-5f58-4c5c-bffd-be53eb55a430") : secret "networking-console-plugin-cert" not found Jul 30 15:11:19.030272 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:11:19.030227 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7j468_a2197c57-2dc6-410e-9910-a045dedcb8d9/konnectivity-agent/0.log" limit=1024 Jul 30 15:11:20.822538 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:11:20.822498 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca/service-ca-658d7b7c78-vdhkt" event={"ID":"f2e567f3-cbda-4dcc-9d78-2ea3a6344ad2","Type":"ContainerStarted","Data":"9be03671c18a416e6e633349e3930df9579ff2e66a513ba59cfb906ce5b063f5"} Jul 30 15:11:20.859888 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:11:20.859829 2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-service-ca/service-ca-658d7b7c78-vdhkt" podStartSLOduration=1.2601767609999999 podStartE2EDuration="3.859809018s" totalImagesPullingTime="2.599632257s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 15:11:17 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 15:11:20.858559303 +0000 UTC m=+62.784311949" watchObservedRunningTime="2026-07-30 15:11:20.859809018 +0000 UTC m=+62.785561666" Jul 30 15:11:22.185740 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:11:22.185694 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7j468_a2197c57-2dc6-410e-9910-a045dedcb8d9/konnectivity-agent/0.log" limit=1024 Jul 30 15:11:22.466267 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:11:22.466227 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7j468_a2197c57-2dc6-410e-9910-a045dedcb8d9/konnectivity-agent/0.log" limit=1024 Jul 30 15:11:22.563503 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:11:22.563466 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7j468_a2197c57-2dc6-410e-9910-a045dedcb8d9/konnectivity-agent/0.log" limit=1024 Jul 30 15:11:22.674657 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:11:22.674621 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7j468_a2197c57-2dc6-410e-9910-a045dedcb8d9/konnectivity-agent/0.log" limit=1024 Jul 30 15:11:22.879883 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:11:22.879801 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7j468_a2197c57-2dc6-410e-9910-a045dedcb8d9/konnectivity-agent/0.log" limit=1024 Jul 30 15:11:23.684361 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:11:23.684330 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7j468_a2197c57-2dc6-410e-9910-a045dedcb8d9/konnectivity-agent/0.log" limit=1024 Jul 30 15:11:24.100842 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:11:24.100805 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7j468_a2197c57-2dc6-410e-9910-a045dedcb8d9/konnectivity-agent/0.log" limit=1024 Jul 30 15:11:24.220432 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:11:24.220401 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/e76c844a-a34c-4587-93a7-023d511a0b9f-original-pull-secret\") pod \"global-pull-secret-syncer-qvn96\" (UID: \"e76c844a-a34c-4587-93a7-023d511a0b9f\") " pod="kube-system/global-pull-secret-syncer-qvn96" Jul 30 15:11:24.220432 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:11:24.220436 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/32b9046b-bd5c-4f23-b832-6fd0bad1c8d6-metrics-certs\") pod \"network-metrics-daemon-dbmt8\" (UID: \"32b9046b-bd5c-4f23-b832-6fd0bad1c8d6\") " pod="openshift-multus/network-metrics-daemon-dbmt8" Jul 30 15:11:24.223414 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:11:24.223390 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Jul 30 15:11:24.223728 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:11:24.223709 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Jul 30 15:11:24.231301 ip-10-0-133-210 kubenswrapper[2576]: E0730 15:11:24.231285 2576 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found Jul 30 15:11:24.231387 ip-10-0-133-210 kubenswrapper[2576]: E0730 15:11:24.231340 2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/32b9046b-bd5c-4f23-b832-6fd0bad1c8d6-metrics-certs podName:32b9046b-bd5c-4f23-b832-6fd0bad1c8d6 nodeName:}" failed. No retries permitted until 2026-07-30 15:12:28.231326094 +0000 UTC m=+130.157078717 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/32b9046b-bd5c-4f23-b832-6fd0bad1c8d6-metrics-certs") pod "network-metrics-daemon-dbmt8" (UID: "32b9046b-bd5c-4f23-b832-6fd0bad1c8d6") : secret "metrics-daemon-secret" not found Jul 30 15:11:24.233482 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:11:24.233459 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/e76c844a-a34c-4587-93a7-023d511a0b9f-original-pull-secret\") pod \"global-pull-secret-syncer-qvn96\" (UID: \"e76c844a-a34c-4587-93a7-023d511a0b9f\") " pod="kube-system/global-pull-secret-syncer-qvn96" Jul 30 15:11:24.421696 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:11:24.421614 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7pzx7\" (UniqueName: \"kubernetes.io/projected/1cd62f00-3530-42c3-9aab-499796a7c906-kube-api-access-7pzx7\") pod \"network-check-target-q858z\" (UID: \"1cd62f00-3530-42c3-9aab-499796a7c906\") " pod="openshift-network-diagnostics/network-check-target-q858z" Jul 30 15:11:24.424874 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:11:24.424855 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" Jul 30 15:11:24.435642 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:11:24.435616 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" Jul 30 15:11:24.445379 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:11:24.445351 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7pzx7\" (UniqueName: \"kubernetes.io/projected/1cd62f00-3530-42c3-9aab-499796a7c906-kube-api-access-7pzx7\") pod \"network-check-target-q858z\" (UID: \"1cd62f00-3530-42c3-9aab-499796a7c906\") " pod="openshift-network-diagnostics/network-check-target-q858z" Jul 30 15:11:24.485475 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:11:24.485439 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7j468_a2197c57-2dc6-410e-9910-a045dedcb8d9/konnectivity-agent/0.log" limit=1024 Jul 30 15:11:24.522033 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:11:24.522003 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-52dkp\"" Jul 30 15:11:24.524386 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:11:24.524370 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-qvn96" Jul 30 15:11:24.529733 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:11:24.529716 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-q858z" Jul 30 15:11:24.694313 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:11:24.694235 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-q858z"] Jul 30 15:11:24.696771 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:11:24.696749 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-qvn96"] Jul 30 15:11:24.698172 ip-10-0-133-210 kubenswrapper[2576]: W0730 15:11:24.698147 2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1cd62f00_3530_42c3_9aab_499796a7c906.slice/crio-31952ee092212420fca9c429a5f95bb4bd4f475e544e4aa1d73a169acc2c7988 WatchSource:0}: Error finding container 31952ee092212420fca9c429a5f95bb4bd4f475e544e4aa1d73a169acc2c7988: Status 404 returned error can't find the container with id 31952ee092212420fca9c429a5f95bb4bd4f475e544e4aa1d73a169acc2c7988 Jul 30 15:11:24.698628 ip-10-0-133-210 kubenswrapper[2576]: W0730 15:11:24.698612 2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode76c844a_a34c_4587_93a7_023d511a0b9f.slice/crio-356030e87aed909b626cc1c5ad2ebb144d57be1679d6a4486852b4b286c2855e WatchSource:0}: Error finding container 356030e87aed909b626cc1c5ad2ebb144d57be1679d6a4486852b4b286c2855e: Status 404 returned error can't find the container with id 356030e87aed909b626cc1c5ad2ebb144d57be1679d6a4486852b4b286c2855e Jul 30 15:11:24.823785 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:11:24.823748 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/c460206a-fa1f-4c09-b498-467313191c13-metrics-tls\") pod \"dns-default-jwbls\" (UID: \"c460206a-fa1f-4c09-b498-467313191c13\") " pod="openshift-dns/dns-default-jwbls" Jul 30 15:11:24.823937 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:11:24.823813 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/9f3ebd63-d1cc-4920-94dc-65d6a62b96fa-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-dvkw7\" (UID: \"9f3ebd63-d1cc-4920-94dc-65d6a62b96fa\") " pod="openshift-insights/insights-runtime-extractor-dvkw7" Jul 30 15:11:24.823937 ip-10-0-133-210 kubenswrapper[2576]: E0730 15:11:24.823913 2576 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 30 15:11:24.824037 ip-10-0-133-210 kubenswrapper[2576]: E0730 15:11:24.823983 2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c460206a-fa1f-4c09-b498-467313191c13-metrics-tls podName:c460206a-fa1f-4c09-b498-467313191c13 nodeName:}" failed. No retries permitted until 2026-07-30 15:11:56.823967977 +0000 UTC m=+98.749720603 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/c460206a-fa1f-4c09-b498-467313191c13-metrics-tls") pod "dns-default-jwbls" (UID: "c460206a-fa1f-4c09-b498-467313191c13") : secret "dns-default-metrics-tls" not found Jul 30 15:11:24.824037 ip-10-0-133-210 kubenswrapper[2576]: E0730 15:11:24.823913 2576 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 30 15:11:24.824139 ip-10-0-133-210 kubenswrapper[2576]: E0730 15:11:24.824063 2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9f3ebd63-d1cc-4920-94dc-65d6a62b96fa-insights-runtime-extractor-tls podName:9f3ebd63-d1cc-4920-94dc-65d6a62b96fa nodeName:}" failed. No retries permitted until 2026-07-30 15:11:56.824050084 +0000 UTC m=+98.749802721 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/9f3ebd63-d1cc-4920-94dc-65d6a62b96fa-insights-runtime-extractor-tls") pod "insights-runtime-extractor-dvkw7" (UID: "9f3ebd63-d1cc-4920-94dc-65d6a62b96fa") : secret "insights-runtime-extractor-tls" not found Jul 30 15:11:24.834012 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:11:24.833970 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-q858z" event={"ID":"1cd62f00-3530-42c3-9aab-499796a7c906","Type":"ContainerStarted","Data":"31952ee092212420fca9c429a5f95bb4bd4f475e544e4aa1d73a169acc2c7988"} Jul 30 15:11:24.835149 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:11:24.835126 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-qvn96" event={"ID":"e76c844a-a34c-4587-93a7-023d511a0b9f","Type":"ContainerStarted","Data":"356030e87aed909b626cc1c5ad2ebb144d57be1679d6a4486852b4b286c2855e"} Jul 30 15:11:24.897738 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:11:24.897700 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7j468_a2197c57-2dc6-410e-9910-a045dedcb8d9/konnectivity-agent/0.log" limit=1024 Jul 30 15:11:25.088444 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:11:25.088407 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7j468_a2197c57-2dc6-410e-9910-a045dedcb8d9/konnectivity-agent/0.log" limit=1024 Jul 30 15:11:25.668600 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:11:25.668559 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7j468_a2197c57-2dc6-410e-9910-a045dedcb8d9/konnectivity-agent/0.log" limit=1024 Jul 30 15:11:25.862034 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:11:25.861969 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7j468_a2197c57-2dc6-410e-9910-a045dedcb8d9/konnectivity-agent/0.log" limit=1024 Jul 30 15:11:25.980216 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:11:25.980162 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7j468_a2197c57-2dc6-410e-9910-a045dedcb8d9/konnectivity-agent/0.log" limit=1024 Jul 30 15:11:26.082091 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:11:26.082052 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7j468_a2197c57-2dc6-410e-9910-a045dedcb8d9/konnectivity-agent/0.log" limit=1024 Jul 30 15:11:26.188043 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:11:26.188010 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7j468_a2197c57-2dc6-410e-9910-a045dedcb8d9/konnectivity-agent/0.log" limit=1024 Jul 30 15:11:26.235213 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:11:26.235050 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/4ce0180e-5f58-4c5c-bffd-be53eb55a430-networking-console-plugin-cert\") pod \"networking-console-plugin-74c9f884b7-lcg2q\" (UID: \"4ce0180e-5f58-4c5c-bffd-be53eb55a430\") " pod="openshift-network-console/networking-console-plugin-74c9f884b7-lcg2q" Jul 30 15:11:26.235395 ip-10-0-133-210 kubenswrapper[2576]: E0730 15:11:26.235253 2576 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found Jul 30 15:11:26.235395 ip-10-0-133-210 kubenswrapper[2576]: E0730 15:11:26.235324 2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4ce0180e-5f58-4c5c-bffd-be53eb55a430-networking-console-plugin-cert podName:4ce0180e-5f58-4c5c-bffd-be53eb55a430 nodeName:}" failed. No retries permitted until 2026-07-30 15:11:42.23530439 +0000 UTC m=+84.161057018 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/4ce0180e-5f58-4c5c-bffd-be53eb55a430-networking-console-plugin-cert") pod "networking-console-plugin-74c9f884b7-lcg2q" (UID: "4ce0180e-5f58-4c5c-bffd-be53eb55a430") : secret "networking-console-plugin-cert" not found Jul 30 15:11:26.289547 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:11:26.289496 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7j468_a2197c57-2dc6-410e-9910-a045dedcb8d9/konnectivity-agent/0.log" limit=1024 Jul 30 15:11:26.694254 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:11:26.694163 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7j468_a2197c57-2dc6-410e-9910-a045dedcb8d9/konnectivity-agent/0.log" limit=1024 Jul 30 15:11:26.784636 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:11:26.784597 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7j468_a2197c57-2dc6-410e-9910-a045dedcb8d9/konnectivity-agent/0.log" limit=1024 Jul 30 15:11:27.293723 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:11:27.293684 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7j468_a2197c57-2dc6-410e-9910-a045dedcb8d9/konnectivity-agent/0.log" limit=1024 Jul 30 15:11:27.915868 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:11:27.915832 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7j468_a2197c57-2dc6-410e-9910-a045dedcb8d9/konnectivity-agent/0.log" limit=1024 Jul 30 15:11:28.081997 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:11:28.081918 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7j468_a2197c57-2dc6-410e-9910-a045dedcb8d9/konnectivity-agent/0.log" limit=1024 Jul 30 15:11:28.287525 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:11:28.287471 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7j468_a2197c57-2dc6-410e-9910-a045dedcb8d9/konnectivity-agent/0.log" limit=1024 Jul 30 15:11:28.441633 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:11:28.441595 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7j468_a2197c57-2dc6-410e-9910-a045dedcb8d9/konnectivity-agent/0.log" limit=1024 Jul 30 15:11:28.694826 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:11:28.694739 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7j468_a2197c57-2dc6-410e-9910-a045dedcb8d9/konnectivity-agent/0.log" limit=1024 Jul 30 15:11:29.287370 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:11:29.287325 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7j468_a2197c57-2dc6-410e-9910-a045dedcb8d9/konnectivity-agent/0.log" limit=1024 Jul 30 15:11:29.732343 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:11:29.732312 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7j468_a2197c57-2dc6-410e-9910-a045dedcb8d9/konnectivity-agent/0.log" limit=1024 Jul 30 15:11:29.850826 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:11:29.850723 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-qvn96" event={"ID":"e76c844a-a34c-4587-93a7-023d511a0b9f","Type":"ContainerStarted","Data":"b94bc005a9820448ebc9b350dc00b28796bd69263b2eb608eba3a08d2fe68571"} Jul 30 15:11:29.852244 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:11:29.852219 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-q858z" event={"ID":"1cd62f00-3530-42c3-9aab-499796a7c906","Type":"ContainerStarted","Data":"7f6cbaf899bd7aa4607c74e3f00e073683e37faf1a9cbf410fe38d470fde675d"} Jul 30 15:11:29.852365 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:11:29.852350 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-q858z" Jul 30 15:11:29.897430 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:11:29.897382 2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-qvn96" podStartSLOduration=67.123553963 podStartE2EDuration="1m11.897365944s" totalImagesPullingTime="4.773811981s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-30 15:10:18 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 15:11:29.896216426 +0000 UTC m=+71.821969071" watchObservedRunningTime="2026-07-30 15:11:29.897365944 +0000 UTC m=+71.823118634" Jul 30 15:11:29.928118 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:11:29.928066 2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-q858z" podStartSLOduration=67.15753721 podStartE2EDuration="1m11.928050524s" totalImagesPullingTime="4.770513314s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 15:10:18 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 15:11:29.926535431 +0000 UTC m=+71.852288077" watchObservedRunningTime="2026-07-30 15:11:29.928050524 +0000 UTC m=+71.853803147" Jul 30 15:11:29.930977 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:11:29.930954 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7j468_a2197c57-2dc6-410e-9910-a045dedcb8d9/konnectivity-agent/0.log" limit=1024 Jul 30 15:11:30.311164 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:11:30.311129 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7j468_a2197c57-2dc6-410e-9910-a045dedcb8d9/konnectivity-agent/0.log" limit=1024 Jul 30 15:11:31.098643 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:11:31.098607 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7j468_a2197c57-2dc6-410e-9910-a045dedcb8d9/konnectivity-agent/0.log" limit=1024 Jul 30 15:11:31.300984 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:11:31.300943 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7j468_a2197c57-2dc6-410e-9910-a045dedcb8d9/konnectivity-agent/0.log" limit=1024 Jul 30 15:11:31.702760 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:11:31.702729 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7j468_a2197c57-2dc6-410e-9910-a045dedcb8d9/konnectivity-agent/0.log" limit=1024 Jul 30 15:11:31.888332 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:11:31.888298 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7j468_a2197c57-2dc6-410e-9910-a045dedcb8d9/konnectivity-agent/0.log" limit=1024 Jul 30 15:11:32.283371 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:11:32.283333 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7j468_a2197c57-2dc6-410e-9910-a045dedcb8d9/konnectivity-agent/0.log" limit=1024 Jul 30 15:11:32.675902 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:11:32.675828 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7j468_a2197c57-2dc6-410e-9910-a045dedcb8d9/konnectivity-agent/0.log" limit=1024 Jul 30 15:11:32.882490 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:11:32.882458 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7j468_a2197c57-2dc6-410e-9910-a045dedcb8d9/konnectivity-agent/0.log" limit=1024 Jul 30 15:11:36.243491 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:11:36.243457 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7j468_a2197c57-2dc6-410e-9910-a045dedcb8d9/konnectivity-agent/0.log" limit=1024 Jul 30 15:11:36.478885 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:11:36.478845 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7j468_a2197c57-2dc6-410e-9910-a045dedcb8d9/konnectivity-agent/0.log" limit=1024 Jul 30 15:11:36.597529 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:11:36.597450 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7j468_a2197c57-2dc6-410e-9910-a045dedcb8d9/konnectivity-agent/0.log" limit=1024 Jul 30 15:11:41.880924 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:11:41.880800 2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-745984f67b-qznjc"] Jul 30 15:11:41.883219 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:11:41.883200 2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-665996d579-ghsqw"] Jul 30 15:11:41.883384 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:11:41.883314 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-745984f67b-qznjc" Jul 30 15:11:41.884953 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:11:41.884933 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-665996d579-ghsqw" Jul 30 15:11:41.887249 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:11:41.887232 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"openshift-service-ca.crt\"" Jul 30 15:11:41.888101 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:11:41.888086 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"kube-root-ca.crt\"" Jul 30 15:11:41.888621 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:11:41.888605 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"managed-serviceaccount-dockercfg-lzpbp\"" Jul 30 15:11:41.889055 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:11:41.889032 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"open-cluster-management-image-pull-credentials\"" Jul 30 15:11:41.889380 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:11:41.889156 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"managed-serviceaccount-hub-kubeconfig\"" Jul 30 15:11:41.894861 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:11:41.894841 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"work-manager-hub-kubeconfig\"" Jul 30 15:11:41.901355 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:11:41.901334 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-745984f67b-qznjc"] Jul 30 15:11:41.910729 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:11:41.910699 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-665996d579-ghsqw"] Jul 30 15:11:41.964330 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:11:41.964295 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/121b9359-bbf9-4882-b5c3-6a71b37b5a67-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-745984f67b-qznjc\" (UID: \"121b9359-bbf9-4882-b5c3-6a71b37b5a67\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-745984f67b-qznjc" Jul 30 15:11:41.964521 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:11:41.964359 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rqgp4\" (UniqueName: \"kubernetes.io/projected/121b9359-bbf9-4882-b5c3-6a71b37b5a67-kube-api-access-rqgp4\") pod \"managed-serviceaccount-addon-agent-745984f67b-qznjc\" (UID: \"121b9359-bbf9-4882-b5c3-6a71b37b5a67\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-745984f67b-qznjc" Jul 30 15:11:41.964521 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:11:41.964388 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6p8t5\" (UniqueName: \"kubernetes.io/projected/36da7179-c230-4348-864e-56401ba6a722-kube-api-access-6p8t5\") pod \"klusterlet-addon-workmgr-665996d579-ghsqw\" (UID: \"36da7179-c230-4348-864e-56401ba6a722\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-665996d579-ghsqw" Jul 30 15:11:41.964521 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:11:41.964415 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/36da7179-c230-4348-864e-56401ba6a722-tmp\") pod \"klusterlet-addon-workmgr-665996d579-ghsqw\" (UID: \"36da7179-c230-4348-864e-56401ba6a722\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-665996d579-ghsqw" Jul 30 15:11:41.964521 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:11:41.964450 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/36da7179-c230-4348-864e-56401ba6a722-klusterlet-config\") pod \"klusterlet-addon-workmgr-665996d579-ghsqw\" (UID: \"36da7179-c230-4348-864e-56401ba6a722\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-665996d579-ghsqw" Jul 30 15:11:42.043000 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:11:42.042969 2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-5c455b4d8b-ns98l"] Jul 30 15:11:42.044744 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:11:42.044728 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-5c455b4d8b-ns98l" Jul 30 15:11:42.051465 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:11:42.051442 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-private-configuration\"" Jul 30 15:11:42.051586 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:11:42.051486 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-tls\"" Jul 30 15:11:42.051708 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:11:42.051695 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-9t9xr\"" Jul 30 15:11:42.054563 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:11:42.054545 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"installation-pull-secrets\"" Jul 30 15:11:42.058663 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:11:42.058644 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"trusted-ca\"" Jul 30 15:11:42.064910 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:11:42.064890 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/121b9359-bbf9-4882-b5c3-6a71b37b5a67-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-745984f67b-qznjc\" (UID: \"121b9359-bbf9-4882-b5c3-6a71b37b5a67\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-745984f67b-qznjc" Jul 30 15:11:42.064985 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:11:42.064929 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rqgp4\" (UniqueName: \"kubernetes.io/projected/121b9359-bbf9-4882-b5c3-6a71b37b5a67-kube-api-access-rqgp4\") pod \"managed-serviceaccount-addon-agent-745984f67b-qznjc\" (UID: \"121b9359-bbf9-4882-b5c3-6a71b37b5a67\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-745984f67b-qznjc" Jul 30 15:11:42.064985 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:11:42.064949 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6p8t5\" (UniqueName: \"kubernetes.io/projected/36da7179-c230-4348-864e-56401ba6a722-kube-api-access-6p8t5\") pod \"klusterlet-addon-workmgr-665996d579-ghsqw\" (UID: \"36da7179-c230-4348-864e-56401ba6a722\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-665996d579-ghsqw" Jul 30 15:11:42.064985 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:11:42.064968 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/36da7179-c230-4348-864e-56401ba6a722-tmp\") pod \"klusterlet-addon-workmgr-665996d579-ghsqw\" (UID: \"36da7179-c230-4348-864e-56401ba6a722\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-665996d579-ghsqw" Jul 30 15:11:42.064985 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:11:42.064984 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/36da7179-c230-4348-864e-56401ba6a722-klusterlet-config\") pod \"klusterlet-addon-workmgr-665996d579-ghsqw\" (UID: \"36da7179-c230-4348-864e-56401ba6a722\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-665996d579-ghsqw" Jul 30 15:11:42.065367 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:11:42.065347 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/36da7179-c230-4348-864e-56401ba6a722-tmp\") pod \"klusterlet-addon-workmgr-665996d579-ghsqw\" (UID: \"36da7179-c230-4348-864e-56401ba6a722\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-665996d579-ghsqw" Jul 30 15:11:42.067415 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:11:42.067389 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/36da7179-c230-4348-864e-56401ba6a722-klusterlet-config\") pod \"klusterlet-addon-workmgr-665996d579-ghsqw\" (UID: \"36da7179-c230-4348-864e-56401ba6a722\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-665996d579-ghsqw" Jul 30 15:11:42.067502 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:11:42.067437 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/121b9359-bbf9-4882-b5c3-6a71b37b5a67-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-745984f67b-qznjc\" (UID: \"121b9359-bbf9-4882-b5c3-6a71b37b5a67\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-745984f67b-qznjc" Jul 30 15:11:42.077059 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:11:42.077034 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-5c455b4d8b-ns98l"] Jul 30 15:11:42.095711 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:11:42.095686 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rqgp4\" (UniqueName: \"kubernetes.io/projected/121b9359-bbf9-4882-b5c3-6a71b37b5a67-kube-api-access-rqgp4\") pod \"managed-serviceaccount-addon-agent-745984f67b-qznjc\" (UID: \"121b9359-bbf9-4882-b5c3-6a71b37b5a67\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-745984f67b-qznjc" Jul 30 15:11:42.097467 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:11:42.097450 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6p8t5\" (UniqueName: \"kubernetes.io/projected/36da7179-c230-4348-864e-56401ba6a722-kube-api-access-6p8t5\") pod \"klusterlet-addon-workmgr-665996d579-ghsqw\" (UID: \"36da7179-c230-4348-864e-56401ba6a722\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-665996d579-ghsqw" Jul 30 15:11:42.123562 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:11:42.123528 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7j468_a2197c57-2dc6-410e-9910-a045dedcb8d9/konnectivity-agent/0.log" limit=1024 Jul 30 15:11:42.165514 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:11:42.165421 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/c11dde2f-dda0-4c65-818a-661b94d8799f-bound-sa-token\") pod \"image-registry-5c455b4d8b-ns98l\" (UID: \"c11dde2f-dda0-4c65-818a-661b94d8799f\") " pod="openshift-image-registry/image-registry-5c455b4d8b-ns98l" Jul 30 15:11:42.165514 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:11:42.165459 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/c11dde2f-dda0-4c65-818a-661b94d8799f-registry-tls\") pod \"image-registry-5c455b4d8b-ns98l\" (UID: \"c11dde2f-dda0-4c65-818a-661b94d8799f\") " pod="openshift-image-registry/image-registry-5c455b4d8b-ns98l" Jul 30 15:11:42.165514 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:11:42.165476 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/c11dde2f-dda0-4c65-818a-661b94d8799f-trusted-ca\") pod \"image-registry-5c455b4d8b-ns98l\" (UID: \"c11dde2f-dda0-4c65-818a-661b94d8799f\") " pod="openshift-image-registry/image-registry-5c455b4d8b-ns98l" Jul 30 15:11:42.165809 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:11:42.165624 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/c11dde2f-dda0-4c65-818a-661b94d8799f-image-registry-private-configuration\") pod \"image-registry-5c455b4d8b-ns98l\" (UID: \"c11dde2f-dda0-4c65-818a-661b94d8799f\") " pod="openshift-image-registry/image-registry-5c455b4d8b-ns98l" Jul 30 15:11:42.165809 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:11:42.165662 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/c11dde2f-dda0-4c65-818a-661b94d8799f-installation-pull-secrets\") pod \"image-registry-5c455b4d8b-ns98l\" (UID: \"c11dde2f-dda0-4c65-818a-661b94d8799f\") " pod="openshift-image-registry/image-registry-5c455b4d8b-ns98l" Jul 30 15:11:42.165809 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:11:42.165690 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8pr9g\" (UniqueName: \"kubernetes.io/projected/c11dde2f-dda0-4c65-818a-661b94d8799f-kube-api-access-8pr9g\") pod \"image-registry-5c455b4d8b-ns98l\" (UID: \"c11dde2f-dda0-4c65-818a-661b94d8799f\") " pod="openshift-image-registry/image-registry-5c455b4d8b-ns98l" Jul 30 15:11:42.165809 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:11:42.165735 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/c11dde2f-dda0-4c65-818a-661b94d8799f-registry-certificates\") pod \"image-registry-5c455b4d8b-ns98l\" (UID: \"c11dde2f-dda0-4c65-818a-661b94d8799f\") " pod="openshift-image-registry/image-registry-5c455b4d8b-ns98l" Jul 30 15:11:42.165809 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:11:42.165768 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/c11dde2f-dda0-4c65-818a-661b94d8799f-ca-trust-extracted\") pod \"image-registry-5c455b4d8b-ns98l\" (UID: \"c11dde2f-dda0-4c65-818a-661b94d8799f\") " pod="openshift-image-registry/image-registry-5c455b4d8b-ns98l" Jul 30 15:11:42.204555 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:11:42.204518 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-745984f67b-qznjc" Jul 30 15:11:42.211162 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:11:42.211135 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-665996d579-ghsqw" Jul 30 15:11:42.267628 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:11:42.266583 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/c11dde2f-dda0-4c65-818a-661b94d8799f-image-registry-private-configuration\") pod \"image-registry-5c455b4d8b-ns98l\" (UID: \"c11dde2f-dda0-4c65-818a-661b94d8799f\") " pod="openshift-image-registry/image-registry-5c455b4d8b-ns98l" Jul 30 15:11:42.267628 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:11:42.266633 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/c11dde2f-dda0-4c65-818a-661b94d8799f-installation-pull-secrets\") pod \"image-registry-5c455b4d8b-ns98l\" (UID: \"c11dde2f-dda0-4c65-818a-661b94d8799f\") " pod="openshift-image-registry/image-registry-5c455b4d8b-ns98l" Jul 30 15:11:42.267628 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:11:42.266665 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8pr9g\" (UniqueName: \"kubernetes.io/projected/c11dde2f-dda0-4c65-818a-661b94d8799f-kube-api-access-8pr9g\") pod \"image-registry-5c455b4d8b-ns98l\" (UID: \"c11dde2f-dda0-4c65-818a-661b94d8799f\") " pod="openshift-image-registry/image-registry-5c455b4d8b-ns98l" Jul 30 15:11:42.267628 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:11:42.266713 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/c11dde2f-dda0-4c65-818a-661b94d8799f-registry-certificates\") pod \"image-registry-5c455b4d8b-ns98l\" (UID: \"c11dde2f-dda0-4c65-818a-661b94d8799f\") " pod="openshift-image-registry/image-registry-5c455b4d8b-ns98l" Jul 30 15:11:42.267628 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:11:42.266744 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/c11dde2f-dda0-4c65-818a-661b94d8799f-ca-trust-extracted\") pod \"image-registry-5c455b4d8b-ns98l\" (UID: \"c11dde2f-dda0-4c65-818a-661b94d8799f\") " pod="openshift-image-registry/image-registry-5c455b4d8b-ns98l" Jul 30 15:11:42.267628 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:11:42.266796 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/c11dde2f-dda0-4c65-818a-661b94d8799f-bound-sa-token\") pod \"image-registry-5c455b4d8b-ns98l\" (UID: \"c11dde2f-dda0-4c65-818a-661b94d8799f\") " pod="openshift-image-registry/image-registry-5c455b4d8b-ns98l" Jul 30 15:11:42.267628 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:11:42.266828 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/c11dde2f-dda0-4c65-818a-661b94d8799f-registry-tls\") pod \"image-registry-5c455b4d8b-ns98l\" (UID: \"c11dde2f-dda0-4c65-818a-661b94d8799f\") " pod="openshift-image-registry/image-registry-5c455b4d8b-ns98l" Jul 30 15:11:42.267628 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:11:42.266855 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/c11dde2f-dda0-4c65-818a-661b94d8799f-trusted-ca\") pod \"image-registry-5c455b4d8b-ns98l\" (UID: \"c11dde2f-dda0-4c65-818a-661b94d8799f\") " pod="openshift-image-registry/image-registry-5c455b4d8b-ns98l" Jul 30 15:11:42.267628 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:11:42.266924 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/4ce0180e-5f58-4c5c-bffd-be53eb55a430-networking-console-plugin-cert\") pod \"networking-console-plugin-74c9f884b7-lcg2q\" (UID: \"4ce0180e-5f58-4c5c-bffd-be53eb55a430\") " pod="openshift-network-console/networking-console-plugin-74c9f884b7-lcg2q" Jul 30 15:11:42.267628 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:11:42.267455 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7j468_a2197c57-2dc6-410e-9910-a045dedcb8d9/konnectivity-agent/0.log" limit=1024 Jul 30 15:11:42.268710 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:11:42.268549 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/c11dde2f-dda0-4c65-818a-661b94d8799f-registry-certificates\") pod \"image-registry-5c455b4d8b-ns98l\" (UID: \"c11dde2f-dda0-4c65-818a-661b94d8799f\") " pod="openshift-image-registry/image-registry-5c455b4d8b-ns98l" Jul 30 15:11:42.268710 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:11:42.268592 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/c11dde2f-dda0-4c65-818a-661b94d8799f-trusted-ca\") pod \"image-registry-5c455b4d8b-ns98l\" (UID: \"c11dde2f-dda0-4c65-818a-661b94d8799f\") " pod="openshift-image-registry/image-registry-5c455b4d8b-ns98l" Jul 30 15:11:42.268710 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:11:42.268671 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/c11dde2f-dda0-4c65-818a-661b94d8799f-ca-trust-extracted\") pod \"image-registry-5c455b4d8b-ns98l\" (UID: \"c11dde2f-dda0-4c65-818a-661b94d8799f\") " pod="openshift-image-registry/image-registry-5c455b4d8b-ns98l" Jul 30 15:11:42.271071 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:11:42.271041 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/c11dde2f-dda0-4c65-818a-661b94d8799f-image-registry-private-configuration\") pod \"image-registry-5c455b4d8b-ns98l\" (UID: \"c11dde2f-dda0-4c65-818a-661b94d8799f\") " pod="openshift-image-registry/image-registry-5c455b4d8b-ns98l" Jul 30 15:11:42.271240 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:11:42.271220 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/c11dde2f-dda0-4c65-818a-661b94d8799f-registry-tls\") pod \"image-registry-5c455b4d8b-ns98l\" (UID: \"c11dde2f-dda0-4c65-818a-661b94d8799f\") " pod="openshift-image-registry/image-registry-5c455b4d8b-ns98l" Jul 30 15:11:42.272138 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:11:42.272094 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/4ce0180e-5f58-4c5c-bffd-be53eb55a430-networking-console-plugin-cert\") pod \"networking-console-plugin-74c9f884b7-lcg2q\" (UID: \"4ce0180e-5f58-4c5c-bffd-be53eb55a430\") " pod="openshift-network-console/networking-console-plugin-74c9f884b7-lcg2q" Jul 30 15:11:42.274423 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:11:42.274395 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/c11dde2f-dda0-4c65-818a-661b94d8799f-installation-pull-secrets\") pod \"image-registry-5c455b4d8b-ns98l\" (UID: \"c11dde2f-dda0-4c65-818a-661b94d8799f\") " pod="openshift-image-registry/image-registry-5c455b4d8b-ns98l" Jul 30 15:11:42.280457 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:11:42.280436 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8pr9g\" (UniqueName: \"kubernetes.io/projected/c11dde2f-dda0-4c65-818a-661b94d8799f-kube-api-access-8pr9g\") pod \"image-registry-5c455b4d8b-ns98l\" (UID: \"c11dde2f-dda0-4c65-818a-661b94d8799f\") " pod="openshift-image-registry/image-registry-5c455b4d8b-ns98l" Jul 30 15:11:42.280599 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:11:42.280583 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/c11dde2f-dda0-4c65-818a-661b94d8799f-bound-sa-token\") pod \"image-registry-5c455b4d8b-ns98l\" (UID: \"c11dde2f-dda0-4c65-818a-661b94d8799f\") " pod="openshift-image-registry/image-registry-5c455b4d8b-ns98l" Jul 30 15:11:42.348663 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:11:42.348638 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-745984f67b-qznjc"] Jul 30 15:11:42.350325 ip-10-0-133-210 kubenswrapper[2576]: W0730 15:11:42.350300 2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod121b9359_bbf9_4882_b5c3_6a71b37b5a67.slice/crio-8759ee20e87278b79e13daee8d65528913b41940119666183744657944efaf38 WatchSource:0}: Error finding container 8759ee20e87278b79e13daee8d65528913b41940119666183744657944efaf38: Status 404 returned error can't find the container with id 8759ee20e87278b79e13daee8d65528913b41940119666183744657944efaf38 Jul 30 15:11:42.352747 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:11:42.352729 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-5c455b4d8b-ns98l" Jul 30 15:11:42.373266 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:11:42.373239 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-665996d579-ghsqw"] Jul 30 15:11:42.383593 ip-10-0-133-210 kubenswrapper[2576]: W0730 15:11:42.383458 2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod36da7179_c230_4348_864e_56401ba6a722.slice/crio-50774537662d41a906ccf2758e0c7e73f7d5cf44b4d9dab3a1c240eb30788693 WatchSource:0}: Error finding container 50774537662d41a906ccf2758e0c7e73f7d5cf44b4d9dab3a1c240eb30788693: Status 404 returned error can't find the container with id 50774537662d41a906ccf2758e0c7e73f7d5cf44b4d9dab3a1c240eb30788693 Jul 30 15:11:42.494842 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:11:42.494816 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-5c455b4d8b-ns98l"] Jul 30 15:11:42.496242 ip-10-0-133-210 kubenswrapper[2576]: W0730 15:11:42.496221 2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc11dde2f_dda0_4c65_818a_661b94d8799f.slice/crio-21b248feedf4af02460f7289c4db3a0fc3672fb17a726737cc8548b8040d0e06 WatchSource:0}: Error finding container 21b248feedf4af02460f7289c4db3a0fc3672fb17a726737cc8548b8040d0e06: Status 404 returned error can't find the container with id 21b248feedf4af02460f7289c4db3a0fc3672fb17a726737cc8548b8040d0e06 Jul 30 15:11:42.529228 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:11:42.529195 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"default-dockercfg-6ftrj\"" Jul 30 15:11:42.537594 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:11:42.537563 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-console/networking-console-plugin-74c9f884b7-lcg2q" Jul 30 15:11:42.679182 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:11:42.679150 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-console/networking-console-plugin-74c9f884b7-lcg2q"] Jul 30 15:11:42.682023 ip-10-0-133-210 kubenswrapper[2576]: W0730 15:11:42.681957 2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4ce0180e_5f58_4c5c_bffd_be53eb55a430.slice/crio-00561ca23a4f55403e1ae71cf8dca336b314b8e5b3caddafbc0707234519c9de WatchSource:0}: Error finding container 00561ca23a4f55403e1ae71cf8dca336b314b8e5b3caddafbc0707234519c9de: Status 404 returned error can't find the container with id 00561ca23a4f55403e1ae71cf8dca336b314b8e5b3caddafbc0707234519c9de Jul 30 15:11:42.887695 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:11:42.887657 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-5c455b4d8b-ns98l" event={"ID":"c11dde2f-dda0-4c65-818a-661b94d8799f","Type":"ContainerStarted","Data":"439aad368e720e5bb901e68b3dc0b1dc44b8f71e0fb27c2f41b5e3c1bcf9a9ec"} Jul 30 15:11:42.888087 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:11:42.887740 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-5c455b4d8b-ns98l" event={"ID":"c11dde2f-dda0-4c65-818a-661b94d8799f","Type":"ContainerStarted","Data":"21b248feedf4af02460f7289c4db3a0fc3672fb17a726737cc8548b8040d0e06"} Jul 30 15:11:42.888087 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:11:42.887768 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-5c455b4d8b-ns98l" Jul 30 15:11:42.888938 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:11:42.888915 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-745984f67b-qznjc" event={"ID":"121b9359-bbf9-4882-b5c3-6a71b37b5a67","Type":"ContainerStarted","Data":"8759ee20e87278b79e13daee8d65528913b41940119666183744657944efaf38"} Jul 30 15:11:42.890118 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:11:42.890080 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-console/networking-console-plugin-74c9f884b7-lcg2q" event={"ID":"4ce0180e-5f58-4c5c-bffd-be53eb55a430","Type":"ContainerStarted","Data":"00561ca23a4f55403e1ae71cf8dca336b314b8e5b3caddafbc0707234519c9de"} Jul 30 15:11:42.891120 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:11:42.891092 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-665996d579-ghsqw" event={"ID":"36da7179-c230-4348-864e-56401ba6a722","Type":"ContainerStarted","Data":"50774537662d41a906ccf2758e0c7e73f7d5cf44b4d9dab3a1c240eb30788693"} Jul 30 15:11:42.925625 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:11:42.925575 2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/image-registry-5c455b4d8b-ns98l" podStartSLOduration=0.925559731 podStartE2EDuration="925.559731ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 15:11:42 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 15:11:42.925244654 +0000 UTC m=+84.850997300" watchObservedRunningTime="2026-07-30 15:11:42.925559731 +0000 UTC m=+84.851312355" Jul 30 15:11:44.727630 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:11:44.727578 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7j468_a2197c57-2dc6-410e-9910-a045dedcb8d9/konnectivity-agent/0.log" limit=1024 Jul 30 15:11:44.822313 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:11:44.822278 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7j468_a2197c57-2dc6-410e-9910-a045dedcb8d9/konnectivity-agent/0.log" limit=1024 Jul 30 15:11:45.188310 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:11:45.188182 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7j468_a2197c57-2dc6-410e-9910-a045dedcb8d9/konnectivity-agent/0.log" limit=1024 Jul 30 15:11:46.905222 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:11:46.905104 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-745984f67b-qznjc" event={"ID":"121b9359-bbf9-4882-b5c3-6a71b37b5a67","Type":"ContainerStarted","Data":"f37b777875df1ad7435304f625214d6619daa554b861b313fc82be5466bf87f9"} Jul 30 15:11:46.906681 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:11:46.906653 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-console/networking-console-plugin-74c9f884b7-lcg2q" event={"ID":"4ce0180e-5f58-4c5c-bffd-be53eb55a430","Type":"ContainerStarted","Data":"a1e66fec1bdabfeead88cedd7c16f1a89dab3a3e3ad47910a0e965428b889fb2"} Jul 30 15:11:46.908015 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:11:46.907991 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-665996d579-ghsqw" event={"ID":"36da7179-c230-4348-864e-56401ba6a722","Type":"ContainerStarted","Data":"d0a6771bd6b473f7397b5be1b194701aefa4546058a622b9d940243116e35882"} Jul 30 15:11:46.908145 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:11:46.908131 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-665996d579-ghsqw" Jul 30 15:11:46.909779 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:11:46.909763 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-665996d579-ghsqw" Jul 30 15:11:46.944767 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:11:46.944713 2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-745984f67b-qznjc" podStartSLOduration=1.696794202 podStartE2EDuration="5.944698732s" totalImagesPullingTime="4.24790453s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 15:11:41 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 15:11:46.942974517 +0000 UTC m=+88.868727161" watchObservedRunningTime="2026-07-30 15:11:46.944698732 +0000 UTC m=+88.870451376" Jul 30 15:11:46.970991 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:11:46.968526 2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-console/networking-console-plugin-74c9f884b7-lcg2q" podStartSLOduration=33.061874411 podStartE2EDuration="36.968500736s" totalImagesPullingTime="3.906626325s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 15:11:10 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 15:11:46.967172549 +0000 UTC m=+88.892925194" watchObservedRunningTime="2026-07-30 15:11:46.968500736 +0000 UTC m=+88.894253382" Jul 30 15:11:46.990565 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:11:46.990512 2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-665996d579-ghsqw" podStartSLOduration=1.7612168750000001 podStartE2EDuration="5.990495048s" totalImagesPullingTime="4.229278173s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 15:11:41 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 15:11:46.989691275 +0000 UTC m=+88.915443921" watchObservedRunningTime="2026-07-30 15:11:46.990495048 +0000 UTC m=+88.916247692" Jul 30 15:11:47.108603 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:11:47.108571 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7j468_a2197c57-2dc6-410e-9910-a045dedcb8d9/konnectivity-agent/0.log" limit=1024 Jul 30 15:11:47.210219 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:11:47.210134 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7j468_a2197c57-2dc6-410e-9910-a045dedcb8d9/konnectivity-agent/0.log" limit=1024 Jul 30 15:11:47.304124 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:11:47.304078 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7j468_a2197c57-2dc6-410e-9910-a045dedcb8d9/konnectivity-agent/0.log" limit=1024 Jul 30 15:11:51.525171 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:11:51.525127 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7j468_a2197c57-2dc6-410e-9910-a045dedcb8d9/konnectivity-agent/0.log" limit=1024 Jul 30 15:11:51.625388 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:11:51.625354 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7j468_a2197c57-2dc6-410e-9910-a045dedcb8d9/konnectivity-agent/0.log" limit=1024 Jul 30 15:11:51.708071 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:11:51.708031 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7j468_a2197c57-2dc6-410e-9910-a045dedcb8d9/konnectivity-agent/0.log" limit=1024 Jul 30 15:11:51.794371 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:11:51.794275 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7j468_a2197c57-2dc6-410e-9910-a045dedcb8d9/konnectivity-agent/0.log" limit=1024 Jul 30 15:11:54.727513 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:11:54.727475 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7j468_a2197c57-2dc6-410e-9910-a045dedcb8d9/konnectivity-agent/0.log" limit=1024 Jul 30 15:11:54.890954 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:11:54.890913 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7j468_a2197c57-2dc6-410e-9910-a045dedcb8d9/konnectivity-agent/0.log" limit=1024 Jul 30 15:11:54.991831 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:11:54.991737 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7j468_a2197c57-2dc6-410e-9910-a045dedcb8d9/konnectivity-agent/0.log" limit=1024 Jul 30 15:11:56.884747 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:11:56.884712 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/c460206a-fa1f-4c09-b498-467313191c13-metrics-tls\") pod \"dns-default-jwbls\" (UID: \"c460206a-fa1f-4c09-b498-467313191c13\") " pod="openshift-dns/dns-default-jwbls" Jul 30 15:11:56.885148 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:11:56.884785 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/9f3ebd63-d1cc-4920-94dc-65d6a62b96fa-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-dvkw7\" (UID: \"9f3ebd63-d1cc-4920-94dc-65d6a62b96fa\") " pod="openshift-insights/insights-runtime-extractor-dvkw7" Jul 30 15:11:56.887150 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:11:56.887127 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/c460206a-fa1f-4c09-b498-467313191c13-metrics-tls\") pod \"dns-default-jwbls\" (UID: \"c460206a-fa1f-4c09-b498-467313191c13\") " pod="openshift-dns/dns-default-jwbls" Jul 30 15:11:56.887223 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:11:56.887169 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/9f3ebd63-d1cc-4920-94dc-65d6a62b96fa-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-dvkw7\" (UID: \"9f3ebd63-d1cc-4920-94dc-65d6a62b96fa\") " pod="openshift-insights/insights-runtime-extractor-dvkw7" Jul 30 15:11:56.890144 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:11:56.890099 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7j468_a2197c57-2dc6-410e-9910-a045dedcb8d9/konnectivity-agent/0.log" limit=1024 Jul 30 15:11:57.147615 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:11:57.147534 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-7hzbg\"" Jul 30 15:11:57.154181 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:11:57.154160 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-jwbls" Jul 30 15:11:57.169714 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:11:57.169689 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-6v78w\"" Jul 30 15:11:57.176984 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:11:57.176961 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-dvkw7" Jul 30 15:11:57.322902 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:11:57.322819 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-jwbls"] Jul 30 15:11:57.325589 ip-10-0-133-210 kubenswrapper[2576]: W0730 15:11:57.325567 2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc460206a_fa1f_4c09_b498_467313191c13.slice/crio-43a37660e44de9729eb77fd7086101e574a830173b3cf68c424b0d2eca16e384 WatchSource:0}: Error finding container 43a37660e44de9729eb77fd7086101e574a830173b3cf68c424b0d2eca16e384: Status 404 returned error can't find the container with id 43a37660e44de9729eb77fd7086101e574a830173b3cf68c424b0d2eca16e384 Jul 30 15:11:57.351780 ip-10-0-133-210 kubenswrapper[2576]: W0730 15:11:57.351752 2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9f3ebd63_d1cc_4920_94dc_65d6a62b96fa.slice/crio-e2b72a2a34b17fb4d8e0d9f60810ae20f297313507ea10133db0b6ae23bf4729 WatchSource:0}: Error finding container e2b72a2a34b17fb4d8e0d9f60810ae20f297313507ea10133db0b6ae23bf4729: Status 404 returned error can't find the container with id e2b72a2a34b17fb4d8e0d9f60810ae20f297313507ea10133db0b6ae23bf4729 Jul 30 15:11:57.351944 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:11:57.351923 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-dvkw7"] Jul 30 15:11:57.947610 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:11:57.947474 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-dvkw7" event={"ID":"9f3ebd63-d1cc-4920-94dc-65d6a62b96fa","Type":"ContainerStarted","Data":"e2b72a2a34b17fb4d8e0d9f60810ae20f297313507ea10133db0b6ae23bf4729"} Jul 30 15:11:57.949321 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:11:57.949293 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-jwbls" event={"ID":"c460206a-fa1f-4c09-b498-467313191c13","Type":"ContainerStarted","Data":"43a37660e44de9729eb77fd7086101e574a830173b3cf68c424b0d2eca16e384"} Jul 30 15:11:59.073723 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:11:59.073687 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7j468_a2197c57-2dc6-410e-9910-a045dedcb8d9/konnectivity-agent/0.log" limit=1024 Jul 30 15:11:59.350277 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:11:59.350166 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7j468_a2197c57-2dc6-410e-9910-a045dedcb8d9/konnectivity-agent/0.log" limit=1024 Jul 30 15:11:59.636731 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:11:59.636602 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7j468_a2197c57-2dc6-410e-9910-a045dedcb8d9/konnectivity-agent/0.log" limit=1024 Jul 30 15:11:59.647316 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:11:59.647288 2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-sv88g"] Jul 30 15:11:59.650347 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:11:59.649394 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-sv88g" Jul 30 15:11:59.666529 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:11:59.666508 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\"" Jul 30 15:11:59.666652 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:11:59.666574 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\"" Jul 30 15:11:59.666652 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:11:59.666619 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\"" Jul 30 15:11:59.668353 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:11:59.668335 2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/openshift-state-metrics-b8f49f5c6-7dcnj"] Jul 30 15:11:59.670525 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:11:59.670504 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-7dcnj" Jul 30 15:11:59.675775 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:11:59.675749 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\"" Jul 30 15:11:59.675880 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:11:59.675794 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\"" Jul 30 15:11:59.675880 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:11:59.675804 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\"" Jul 30 15:11:59.676342 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:11:59.676312 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-tpj8t\"" Jul 30 15:11:59.699964 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:11:59.699939 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-dockercfg-65js9\"" Jul 30 15:11:59.703036 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:11:59.703015 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-tls\"" Jul 30 15:11:59.715841 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:11:59.715817 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-kube-rbac-proxy-config\"" Jul 30 15:11:59.729562 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:11:59.729533 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/openshift-state-metrics-b8f49f5c6-7dcnj"] Jul 30 15:11:59.811993 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:11:59.811957 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/16cf6c6b-a33a-445c-8a97-26226073093c-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-b8f49f5c6-7dcnj\" (UID: \"16cf6c6b-a33a-445c-8a97-26226073093c\") " pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-7dcnj" Jul 30 15:11:59.812178 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:11:59.812054 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/16cf6c6b-a33a-445c-8a97-26226073093c-metrics-client-ca\") pod \"openshift-state-metrics-b8f49f5c6-7dcnj\" (UID: \"16cf6c6b-a33a-445c-8a97-26226073093c\") " pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-7dcnj" Jul 30 15:11:59.812178 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:11:59.812082 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/fb41dae1-7bc8-46a1-a05e-73f26d7c0391-node-exporter-tls\") pod \"node-exporter-sv88g\" (UID: \"fb41dae1-7bc8-46a1-a05e-73f26d7c0391\") " pod="openshift-monitoring/node-exporter-sv88g" Jul 30 15:11:59.812178 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:11:59.812130 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/fb41dae1-7bc8-46a1-a05e-73f26d7c0391-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-sv88g\" (UID: \"fb41dae1-7bc8-46a1-a05e-73f26d7c0391\") " pod="openshift-monitoring/node-exporter-sv88g" Jul 30 15:11:59.812178 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:11:59.812167 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/fb41dae1-7bc8-46a1-a05e-73f26d7c0391-node-exporter-textfile\") pod \"node-exporter-sv88g\" (UID: \"fb41dae1-7bc8-46a1-a05e-73f26d7c0391\") " pod="openshift-monitoring/node-exporter-sv88g" Jul 30 15:11:59.812400 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:11:59.812184 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-58gjd\" (UniqueName: \"kubernetes.io/projected/fb41dae1-7bc8-46a1-a05e-73f26d7c0391-kube-api-access-58gjd\") pod \"node-exporter-sv88g\" (UID: \"fb41dae1-7bc8-46a1-a05e-73f26d7c0391\") " pod="openshift-monitoring/node-exporter-sv88g" Jul 30 15:11:59.812400 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:11:59.812205 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/fb41dae1-7bc8-46a1-a05e-73f26d7c0391-metrics-client-ca\") pod \"node-exporter-sv88g\" (UID: \"fb41dae1-7bc8-46a1-a05e-73f26d7c0391\") " pod="openshift-monitoring/node-exporter-sv88g" Jul 30 15:11:59.812400 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:11:59.812285 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8v7vw\" (UniqueName: \"kubernetes.io/projected/16cf6c6b-a33a-445c-8a97-26226073093c-kube-api-access-8v7vw\") pod \"openshift-state-metrics-b8f49f5c6-7dcnj\" (UID: \"16cf6c6b-a33a-445c-8a97-26226073093c\") " pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-7dcnj" Jul 30 15:11:59.812400 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:11:59.812322 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/fb41dae1-7bc8-46a1-a05e-73f26d7c0391-sys\") pod \"node-exporter-sv88g\" (UID: \"fb41dae1-7bc8-46a1-a05e-73f26d7c0391\") " pod="openshift-monitoring/node-exporter-sv88g" Jul 30 15:11:59.812400 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:11:59.812351 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/16cf6c6b-a33a-445c-8a97-26226073093c-openshift-state-metrics-tls\") pod \"openshift-state-metrics-b8f49f5c6-7dcnj\" (UID: \"16cf6c6b-a33a-445c-8a97-26226073093c\") " pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-7dcnj" Jul 30 15:11:59.812400 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:11:59.812383 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/fb41dae1-7bc8-46a1-a05e-73f26d7c0391-node-exporter-accelerators-collector-config\") pod \"node-exporter-sv88g\" (UID: \"fb41dae1-7bc8-46a1-a05e-73f26d7c0391\") " pod="openshift-monitoring/node-exporter-sv88g" Jul 30 15:11:59.812640 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:11:59.812412 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/fb41dae1-7bc8-46a1-a05e-73f26d7c0391-node-exporter-wtmp\") pod \"node-exporter-sv88g\" (UID: \"fb41dae1-7bc8-46a1-a05e-73f26d7c0391\") " pod="openshift-monitoring/node-exporter-sv88g" Jul 30 15:11:59.812640 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:11:59.812438 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/fb41dae1-7bc8-46a1-a05e-73f26d7c0391-root\") pod \"node-exporter-sv88g\" (UID: \"fb41dae1-7bc8-46a1-a05e-73f26d7c0391\") " pod="openshift-monitoring/node-exporter-sv88g" Jul 30 15:11:59.873445 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:11:59.873413 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7j468_a2197c57-2dc6-410e-9910-a045dedcb8d9/konnectivity-agent/0.log" limit=1024 Jul 30 15:11:59.913511 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:11:59.913427 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/fb41dae1-7bc8-46a1-a05e-73f26d7c0391-node-exporter-textfile\") pod \"node-exporter-sv88g\" (UID: \"fb41dae1-7bc8-46a1-a05e-73f26d7c0391\") " pod="openshift-monitoring/node-exporter-sv88g" Jul 30 15:11:59.913511 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:11:59.913470 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-58gjd\" (UniqueName: \"kubernetes.io/projected/fb41dae1-7bc8-46a1-a05e-73f26d7c0391-kube-api-access-58gjd\") pod \"node-exporter-sv88g\" (UID: \"fb41dae1-7bc8-46a1-a05e-73f26d7c0391\") " pod="openshift-monitoring/node-exporter-sv88g" Jul 30 15:11:59.913731 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:11:59.913598 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/fb41dae1-7bc8-46a1-a05e-73f26d7c0391-metrics-client-ca\") pod \"node-exporter-sv88g\" (UID: \"fb41dae1-7bc8-46a1-a05e-73f26d7c0391\") " pod="openshift-monitoring/node-exporter-sv88g" Jul 30 15:11:59.913731 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:11:59.913659 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8v7vw\" (UniqueName: \"kubernetes.io/projected/16cf6c6b-a33a-445c-8a97-26226073093c-kube-api-access-8v7vw\") pod \"openshift-state-metrics-b8f49f5c6-7dcnj\" (UID: \"16cf6c6b-a33a-445c-8a97-26226073093c\") " pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-7dcnj" Jul 30 15:11:59.913731 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:11:59.913692 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/fb41dae1-7bc8-46a1-a05e-73f26d7c0391-sys\") pod \"node-exporter-sv88g\" (UID: \"fb41dae1-7bc8-46a1-a05e-73f26d7c0391\") " pod="openshift-monitoring/node-exporter-sv88g" Jul 30 15:11:59.913731 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:11:59.913721 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/16cf6c6b-a33a-445c-8a97-26226073093c-openshift-state-metrics-tls\") pod \"openshift-state-metrics-b8f49f5c6-7dcnj\" (UID: \"16cf6c6b-a33a-445c-8a97-26226073093c\") " pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-7dcnj" Jul 30 15:11:59.913929 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:11:59.913752 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/fb41dae1-7bc8-46a1-a05e-73f26d7c0391-node-exporter-accelerators-collector-config\") pod \"node-exporter-sv88g\" (UID: \"fb41dae1-7bc8-46a1-a05e-73f26d7c0391\") " pod="openshift-monitoring/node-exporter-sv88g" Jul 30 15:11:59.913929 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:11:59.913800 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/fb41dae1-7bc8-46a1-a05e-73f26d7c0391-node-exporter-wtmp\") pod \"node-exporter-sv88g\" (UID: \"fb41dae1-7bc8-46a1-a05e-73f26d7c0391\") " pod="openshift-monitoring/node-exporter-sv88g" Jul 30 15:11:59.913929 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:11:59.913807 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/fb41dae1-7bc8-46a1-a05e-73f26d7c0391-node-exporter-textfile\") pod \"node-exporter-sv88g\" (UID: \"fb41dae1-7bc8-46a1-a05e-73f26d7c0391\") " pod="openshift-monitoring/node-exporter-sv88g" Jul 30 15:11:59.913929 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:11:59.913829 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/fb41dae1-7bc8-46a1-a05e-73f26d7c0391-root\") pod \"node-exporter-sv88g\" (UID: \"fb41dae1-7bc8-46a1-a05e-73f26d7c0391\") " pod="openshift-monitoring/node-exporter-sv88g" Jul 30 15:11:59.913929 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:11:59.913876 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/16cf6c6b-a33a-445c-8a97-26226073093c-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-b8f49f5c6-7dcnj\" (UID: \"16cf6c6b-a33a-445c-8a97-26226073093c\") " pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-7dcnj" Jul 30 15:11:59.914202 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:11:59.913942 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/16cf6c6b-a33a-445c-8a97-26226073093c-metrics-client-ca\") pod \"openshift-state-metrics-b8f49f5c6-7dcnj\" (UID: \"16cf6c6b-a33a-445c-8a97-26226073093c\") " pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-7dcnj" Jul 30 15:11:59.914202 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:11:59.913970 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/fb41dae1-7bc8-46a1-a05e-73f26d7c0391-node-exporter-tls\") pod \"node-exporter-sv88g\" (UID: \"fb41dae1-7bc8-46a1-a05e-73f26d7c0391\") " pod="openshift-monitoring/node-exporter-sv88g" Jul 30 15:11:59.914202 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:11:59.914016 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/fb41dae1-7bc8-46a1-a05e-73f26d7c0391-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-sv88g\" (UID: \"fb41dae1-7bc8-46a1-a05e-73f26d7c0391\") " pod="openshift-monitoring/node-exporter-sv88g" Jul 30 15:11:59.914202 ip-10-0-133-210 kubenswrapper[2576]: E0730 15:11:59.914077 2576 secret.go:189] Couldn't get secret openshift-monitoring/openshift-state-metrics-tls: secret "openshift-state-metrics-tls" not found Jul 30 15:11:59.914202 ip-10-0-133-210 kubenswrapper[2576]: E0730 15:11:59.914162 2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/16cf6c6b-a33a-445c-8a97-26226073093c-openshift-state-metrics-tls podName:16cf6c6b-a33a-445c-8a97-26226073093c nodeName:}" failed. No retries permitted until 2026-07-30 15:12:00.414142213 +0000 UTC m=+102.339894849 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "openshift-state-metrics-tls" (UniqueName: "kubernetes.io/secret/16cf6c6b-a33a-445c-8a97-26226073093c-openshift-state-metrics-tls") pod "openshift-state-metrics-b8f49f5c6-7dcnj" (UID: "16cf6c6b-a33a-445c-8a97-26226073093c") : secret "openshift-state-metrics-tls" not found Jul 30 15:11:59.914449 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:11:59.914280 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/fb41dae1-7bc8-46a1-a05e-73f26d7c0391-metrics-client-ca\") pod \"node-exporter-sv88g\" (UID: \"fb41dae1-7bc8-46a1-a05e-73f26d7c0391\") " pod="openshift-monitoring/node-exporter-sv88g" Jul 30 15:11:59.914449 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:11:59.914353 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/fb41dae1-7bc8-46a1-a05e-73f26d7c0391-root\") pod \"node-exporter-sv88g\" (UID: \"fb41dae1-7bc8-46a1-a05e-73f26d7c0391\") " pod="openshift-monitoring/node-exporter-sv88g" Jul 30 15:11:59.914449 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:11:59.914355 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/fb41dae1-7bc8-46a1-a05e-73f26d7c0391-node-exporter-accelerators-collector-config\") pod \"node-exporter-sv88g\" (UID: \"fb41dae1-7bc8-46a1-a05e-73f26d7c0391\") " pod="openshift-monitoring/node-exporter-sv88g" Jul 30 15:11:59.914612 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:11:59.914488 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/fb41dae1-7bc8-46a1-a05e-73f26d7c0391-node-exporter-wtmp\") pod \"node-exporter-sv88g\" (UID: \"fb41dae1-7bc8-46a1-a05e-73f26d7c0391\") " pod="openshift-monitoring/node-exporter-sv88g" Jul 30 15:11:59.914821 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:11:59.914778 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/fb41dae1-7bc8-46a1-a05e-73f26d7c0391-sys\") pod \"node-exporter-sv88g\" (UID: \"fb41dae1-7bc8-46a1-a05e-73f26d7c0391\") " pod="openshift-monitoring/node-exporter-sv88g" Jul 30 15:11:59.915025 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:11:59.914996 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/16cf6c6b-a33a-445c-8a97-26226073093c-metrics-client-ca\") pod \"openshift-state-metrics-b8f49f5c6-7dcnj\" (UID: \"16cf6c6b-a33a-445c-8a97-26226073093c\") " pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-7dcnj" Jul 30 15:11:59.917268 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:11:59.917240 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/fb41dae1-7bc8-46a1-a05e-73f26d7c0391-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-sv88g\" (UID: \"fb41dae1-7bc8-46a1-a05e-73f26d7c0391\") " pod="openshift-monitoring/node-exporter-sv88g" Jul 30 15:11:59.917369 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:11:59.917313 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/fb41dae1-7bc8-46a1-a05e-73f26d7c0391-node-exporter-tls\") pod \"node-exporter-sv88g\" (UID: \"fb41dae1-7bc8-46a1-a05e-73f26d7c0391\") " pod="openshift-monitoring/node-exporter-sv88g" Jul 30 15:11:59.917434 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:11:59.917411 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/16cf6c6b-a33a-445c-8a97-26226073093c-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-b8f49f5c6-7dcnj\" (UID: \"16cf6c6b-a33a-445c-8a97-26226073093c\") " pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-7dcnj" Jul 30 15:11:59.935608 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:11:59.935579 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8v7vw\" (UniqueName: \"kubernetes.io/projected/16cf6c6b-a33a-445c-8a97-26226073093c-kube-api-access-8v7vw\") pod \"openshift-state-metrics-b8f49f5c6-7dcnj\" (UID: \"16cf6c6b-a33a-445c-8a97-26226073093c\") " pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-7dcnj" Jul 30 15:11:59.944770 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:11:59.944750 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-58gjd\" (UniqueName: \"kubernetes.io/projected/fb41dae1-7bc8-46a1-a05e-73f26d7c0391-kube-api-access-58gjd\") pod \"node-exporter-sv88g\" (UID: \"fb41dae1-7bc8-46a1-a05e-73f26d7c0391\") " pod="openshift-monitoring/node-exporter-sv88g" Jul 30 15:11:59.959682 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:11:59.959656 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-sv88g" Jul 30 15:11:59.964853 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:11:59.964532 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-jwbls" event={"ID":"c460206a-fa1f-4c09-b498-467313191c13","Type":"ContainerStarted","Data":"ecc8684a20374c6d50a75a0908b5ca23375398c85193af23e0d37455a7ee3e9a"} Jul 30 15:11:59.964983 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:11:59.964964 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-jwbls" event={"ID":"c460206a-fa1f-4c09-b498-467313191c13","Type":"ContainerStarted","Data":"5c0cea084c59f610075e44c3880dff143b7995a01a450e9b089c71639190da40"} Jul 30 15:11:59.965046 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:11:59.965019 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-jwbls" Jul 30 15:12:00.002601 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:12:00.002555 2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/dns-default-jwbls" podStartSLOduration=66.078995366 podStartE2EDuration="1m8.002539895s" totalImagesPullingTime="1.923544529s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 15:10:52 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 15:12:00.001463074 +0000 UTC m=+101.927215761" watchObservedRunningTime="2026-07-30 15:12:00.002539895 +0000 UTC m=+101.928292540" Jul 30 15:12:00.204096 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:12:00.203955 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7j468_a2197c57-2dc6-410e-9910-a045dedcb8d9/konnectivity-agent/0.log" limit=1024 Jul 30 15:12:00.418928 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:12:00.418892 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/16cf6c6b-a33a-445c-8a97-26226073093c-openshift-state-metrics-tls\") pod \"openshift-state-metrics-b8f49f5c6-7dcnj\" (UID: \"16cf6c6b-a33a-445c-8a97-26226073093c\") " pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-7dcnj" Jul 30 15:12:00.421884 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:12:00.421857 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/16cf6c6b-a33a-445c-8a97-26226073093c-openshift-state-metrics-tls\") pod \"openshift-state-metrics-b8f49f5c6-7dcnj\" (UID: \"16cf6c6b-a33a-445c-8a97-26226073093c\") " pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-7dcnj" Jul 30 15:12:00.436452 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:12:00.436418 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7j468_a2197c57-2dc6-410e-9910-a045dedcb8d9/konnectivity-agent/0.log" limit=1024 Jul 30 15:12:00.580879 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:12:00.580836 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-7dcnj" Jul 30 15:12:00.628207 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:12:00.628156 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7j468_a2197c57-2dc6-410e-9910-a045dedcb8d9/konnectivity-agent/0.log" limit=1024 Jul 30 15:12:00.789207 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:12:00.789143 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7j468_a2197c57-2dc6-410e-9910-a045dedcb8d9/konnectivity-agent/0.log" limit=1024 Jul 30 15:12:00.857304 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:12:00.857208 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-q858z" Jul 30 15:12:01.019242 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:12:01.019203 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7j468_a2197c57-2dc6-410e-9910-a045dedcb8d9/konnectivity-agent/0.log" limit=1024 Jul 30 15:12:01.250507 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:12:01.250465 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7j468_a2197c57-2dc6-410e-9910-a045dedcb8d9/konnectivity-agent/0.log" limit=1024 Jul 30 15:12:02.357955 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:12:02.357900 2576 patch_prober.go:28] interesting pod/image-registry-5c455b4d8b-ns98l container/registry namespace/openshift-image-registry: Liveness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body={"errors":[{"code":"UNAVAILABLE","message":"service unavailable","detail":"health check failed: please see /debug/health"}]} Jul 30 15:12:02.358411 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:12:02.357993 2576 prober.go:120] "Probe failed" probeType="Liveness" pod="openshift-image-registry/image-registry-5c455b4d8b-ns98l" podUID="c11dde2f-dda0-4c65-818a-661b94d8799f" containerName="registry" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 30 15:12:02.571935 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:12:02.571905 2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/thanos-querier-6ff66dc4cf-kl2x6"] Jul 30 15:12:02.594437 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:12:02.594406 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/thanos-querier-6ff66dc4cf-kl2x6" Jul 30 15:12:02.601142 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:12:02.601097 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-grpc-tls-dvpkcmhe86cfc\"" Jul 30 15:12:02.601343 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:12:02.601097 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-web\"" Jul 30 15:12:02.601343 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:12:02.601206 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-rules\"" Jul 30 15:12:02.601343 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:12:02.601209 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-dockercfg-rtstr\"" Jul 30 15:12:02.601343 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:12:02.601294 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-metrics\"" Jul 30 15:12:02.601653 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:12:02.601625 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy\"" Jul 30 15:12:02.604775 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:12:02.604749 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7j468_a2197c57-2dc6-410e-9910-a045dedcb8d9/konnectivity-agent/0.log" limit=1024 Jul 30 15:12:02.610388 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:12:02.610310 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-tls\"" Jul 30 15:12:02.617978 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:12:02.617947 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/thanos-querier-6ff66dc4cf-kl2x6"] Jul 30 15:12:02.739325 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:12:02.739293 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/0af6b334-8a41-459d-9e05-4d3769d69a4c-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-6ff66dc4cf-kl2x6\" (UID: \"0af6b334-8a41-459d-9e05-4d3769d69a4c\") " pod="openshift-monitoring/thanos-querier-6ff66dc4cf-kl2x6" Jul 30 15:12:02.739325 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:12:02.739334 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/0af6b334-8a41-459d-9e05-4d3769d69a4c-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-6ff66dc4cf-kl2x6\" (UID: \"0af6b334-8a41-459d-9e05-4d3769d69a4c\") " pod="openshift-monitoring/thanos-querier-6ff66dc4cf-kl2x6" Jul 30 15:12:02.739552 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:12:02.739365 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pr6kp\" (UniqueName: \"kubernetes.io/projected/0af6b334-8a41-459d-9e05-4d3769d69a4c-kube-api-access-pr6kp\") pod \"thanos-querier-6ff66dc4cf-kl2x6\" (UID: \"0af6b334-8a41-459d-9e05-4d3769d69a4c\") " pod="openshift-monitoring/thanos-querier-6ff66dc4cf-kl2x6" Jul 30 15:12:02.739552 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:12:02.739450 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/0af6b334-8a41-459d-9e05-4d3769d69a4c-secret-grpc-tls\") pod \"thanos-querier-6ff66dc4cf-kl2x6\" (UID: \"0af6b334-8a41-459d-9e05-4d3769d69a4c\") " pod="openshift-monitoring/thanos-querier-6ff66dc4cf-kl2x6" Jul 30 15:12:02.739647 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:12:02.739556 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/0af6b334-8a41-459d-9e05-4d3769d69a4c-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-6ff66dc4cf-kl2x6\" (UID: \"0af6b334-8a41-459d-9e05-4d3769d69a4c\") " pod="openshift-monitoring/thanos-querier-6ff66dc4cf-kl2x6" Jul 30 15:12:02.739647 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:12:02.739601 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/0af6b334-8a41-459d-9e05-4d3769d69a4c-metrics-client-ca\") pod \"thanos-querier-6ff66dc4cf-kl2x6\" (UID: \"0af6b334-8a41-459d-9e05-4d3769d69a4c\") " pod="openshift-monitoring/thanos-querier-6ff66dc4cf-kl2x6" Jul 30 15:12:02.739724 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:12:02.739680 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/0af6b334-8a41-459d-9e05-4d3769d69a4c-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-6ff66dc4cf-kl2x6\" (UID: \"0af6b334-8a41-459d-9e05-4d3769d69a4c\") " pod="openshift-monitoring/thanos-querier-6ff66dc4cf-kl2x6" Jul 30 15:12:02.739761 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:12:02.739740 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/0af6b334-8a41-459d-9e05-4d3769d69a4c-secret-thanos-querier-tls\") pod \"thanos-querier-6ff66dc4cf-kl2x6\" (UID: \"0af6b334-8a41-459d-9e05-4d3769d69a4c\") " pod="openshift-monitoring/thanos-querier-6ff66dc4cf-kl2x6" Jul 30 15:12:02.840397 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:12:02.840357 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/0af6b334-8a41-459d-9e05-4d3769d69a4c-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-6ff66dc4cf-kl2x6\" (UID: \"0af6b334-8a41-459d-9e05-4d3769d69a4c\") " pod="openshift-monitoring/thanos-querier-6ff66dc4cf-kl2x6" Jul 30 15:12:02.840612 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:12:02.840420 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/0af6b334-8a41-459d-9e05-4d3769d69a4c-secret-thanos-querier-tls\") pod \"thanos-querier-6ff66dc4cf-kl2x6\" (UID: \"0af6b334-8a41-459d-9e05-4d3769d69a4c\") " pod="openshift-monitoring/thanos-querier-6ff66dc4cf-kl2x6" Jul 30 15:12:02.840612 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:12:02.840445 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/0af6b334-8a41-459d-9e05-4d3769d69a4c-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-6ff66dc4cf-kl2x6\" (UID: \"0af6b334-8a41-459d-9e05-4d3769d69a4c\") " pod="openshift-monitoring/thanos-querier-6ff66dc4cf-kl2x6" Jul 30 15:12:02.840612 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:12:02.840471 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/0af6b334-8a41-459d-9e05-4d3769d69a4c-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-6ff66dc4cf-kl2x6\" (UID: \"0af6b334-8a41-459d-9e05-4d3769d69a4c\") " pod="openshift-monitoring/thanos-querier-6ff66dc4cf-kl2x6" Jul 30 15:12:02.840612 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:12:02.840493 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pr6kp\" (UniqueName: \"kubernetes.io/projected/0af6b334-8a41-459d-9e05-4d3769d69a4c-kube-api-access-pr6kp\") pod \"thanos-querier-6ff66dc4cf-kl2x6\" (UID: \"0af6b334-8a41-459d-9e05-4d3769d69a4c\") " pod="openshift-monitoring/thanos-querier-6ff66dc4cf-kl2x6" Jul 30 15:12:02.840612 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:12:02.840552 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/0af6b334-8a41-459d-9e05-4d3769d69a4c-secret-grpc-tls\") pod \"thanos-querier-6ff66dc4cf-kl2x6\" (UID: \"0af6b334-8a41-459d-9e05-4d3769d69a4c\") " pod="openshift-monitoring/thanos-querier-6ff66dc4cf-kl2x6" Jul 30 15:12:02.840612 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:12:02.840596 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/0af6b334-8a41-459d-9e05-4d3769d69a4c-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-6ff66dc4cf-kl2x6\" (UID: \"0af6b334-8a41-459d-9e05-4d3769d69a4c\") " pod="openshift-monitoring/thanos-querier-6ff66dc4cf-kl2x6" Jul 30 15:12:02.840918 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:12:02.840631 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/0af6b334-8a41-459d-9e05-4d3769d69a4c-metrics-client-ca\") pod \"thanos-querier-6ff66dc4cf-kl2x6\" (UID: \"0af6b334-8a41-459d-9e05-4d3769d69a4c\") " pod="openshift-monitoring/thanos-querier-6ff66dc4cf-kl2x6" Jul 30 15:12:02.841442 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:12:02.841370 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/0af6b334-8a41-459d-9e05-4d3769d69a4c-metrics-client-ca\") pod \"thanos-querier-6ff66dc4cf-kl2x6\" (UID: \"0af6b334-8a41-459d-9e05-4d3769d69a4c\") " pod="openshift-monitoring/thanos-querier-6ff66dc4cf-kl2x6" Jul 30 15:12:02.841930 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:12:02.841837 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7j468_a2197c57-2dc6-410e-9910-a045dedcb8d9/konnectivity-agent/0.log" limit=1024 Jul 30 15:12:02.843372 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:12:02.843335 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/0af6b334-8a41-459d-9e05-4d3769d69a4c-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-6ff66dc4cf-kl2x6\" (UID: \"0af6b334-8a41-459d-9e05-4d3769d69a4c\") " pod="openshift-monitoring/thanos-querier-6ff66dc4cf-kl2x6" Jul 30 15:12:02.843842 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:12:02.843821 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/0af6b334-8a41-459d-9e05-4d3769d69a4c-secret-thanos-querier-tls\") pod \"thanos-querier-6ff66dc4cf-kl2x6\" (UID: \"0af6b334-8a41-459d-9e05-4d3769d69a4c\") " pod="openshift-monitoring/thanos-querier-6ff66dc4cf-kl2x6" Jul 30 15:12:02.843983 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:12:02.843960 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/0af6b334-8a41-459d-9e05-4d3769d69a4c-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-6ff66dc4cf-kl2x6\" (UID: \"0af6b334-8a41-459d-9e05-4d3769d69a4c\") " pod="openshift-monitoring/thanos-querier-6ff66dc4cf-kl2x6" Jul 30 15:12:02.844265 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:12:02.844219 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/0af6b334-8a41-459d-9e05-4d3769d69a4c-secret-grpc-tls\") pod \"thanos-querier-6ff66dc4cf-kl2x6\" (UID: \"0af6b334-8a41-459d-9e05-4d3769d69a4c\") " pod="openshift-monitoring/thanos-querier-6ff66dc4cf-kl2x6" Jul 30 15:12:02.844350 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:12:02.844261 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/0af6b334-8a41-459d-9e05-4d3769d69a4c-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-6ff66dc4cf-kl2x6\" (UID: \"0af6b334-8a41-459d-9e05-4d3769d69a4c\") " pod="openshift-monitoring/thanos-querier-6ff66dc4cf-kl2x6" Jul 30 15:12:02.845426 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:12:02.845403 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/0af6b334-8a41-459d-9e05-4d3769d69a4c-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-6ff66dc4cf-kl2x6\" (UID: \"0af6b334-8a41-459d-9e05-4d3769d69a4c\") " pod="openshift-monitoring/thanos-querier-6ff66dc4cf-kl2x6" Jul 30 15:12:02.864873 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:12:02.864796 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-pr6kp\" (UniqueName: \"kubernetes.io/projected/0af6b334-8a41-459d-9e05-4d3769d69a4c-kube-api-access-pr6kp\") pod \"thanos-querier-6ff66dc4cf-kl2x6\" (UID: \"0af6b334-8a41-459d-9e05-4d3769d69a4c\") " pod="openshift-monitoring/thanos-querier-6ff66dc4cf-kl2x6" Jul 30 15:12:02.906332 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:12:02.906292 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/thanos-querier-6ff66dc4cf-kl2x6" Jul 30 15:12:02.966027 ip-10-0-133-210 kubenswrapper[2576]: W0730 15:12:02.965981 2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfb41dae1_7bc8_46a1_a05e_73f26d7c0391.slice/crio-702803e25e9cf362a3e40fe0af2a3a6b1b4449ef39a7e45446f15a2a5b32cd2b WatchSource:0}: Error finding container 702803e25e9cf362a3e40fe0af2a3a6b1b4449ef39a7e45446f15a2a5b32cd2b: Status 404 returned error can't find the container with id 702803e25e9cf362a3e40fe0af2a3a6b1b4449ef39a7e45446f15a2a5b32cd2b Jul 30 15:12:02.976233 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:12:02.976203 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-sv88g" event={"ID":"fb41dae1-7bc8-46a1-a05e-73f26d7c0391","Type":"ContainerStarted","Data":"702803e25e9cf362a3e40fe0af2a3a6b1b4449ef39a7e45446f15a2a5b32cd2b"} Jul 30 15:12:02.989928 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:12:02.989873 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7j468_a2197c57-2dc6-410e-9910-a045dedcb8d9/konnectivity-agent/0.log" limit=1024 Jul 30 15:12:03.106066 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:12:03.106033 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/openshift-state-metrics-b8f49f5c6-7dcnj"] Jul 30 15:12:03.109681 ip-10-0-133-210 kubenswrapper[2576]: W0730 15:12:03.109653 2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod16cf6c6b_a33a_445c_8a97_26226073093c.slice/crio-22a96c162cb9add90c5e23b433b6c6a21aeb1a5d6ac6fe73385dde9fb679803b WatchSource:0}: Error finding container 22a96c162cb9add90c5e23b433b6c6a21aeb1a5d6ac6fe73385dde9fb679803b: Status 404 returned error can't find the container with id 22a96c162cb9add90c5e23b433b6c6a21aeb1a5d6ac6fe73385dde9fb679803b Jul 30 15:12:03.137477 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:12:03.137453 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/thanos-querier-6ff66dc4cf-kl2x6"] Jul 30 15:12:03.299622 ip-10-0-133-210 kubenswrapper[2576]: W0730 15:12:03.299586 2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0af6b334_8a41_459d_9e05_4d3769d69a4c.slice/crio-9c750de421b290c39bd523e76846319800aa0ca68406eeb2b8e63eb2afcbd6af WatchSource:0}: Error finding container 9c750de421b290c39bd523e76846319800aa0ca68406eeb2b8e63eb2afcbd6af: Status 404 returned error can't find the container with id 9c750de421b290c39bd523e76846319800aa0ca68406eeb2b8e63eb2afcbd6af Jul 30 15:12:03.898681 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:12:03.898650 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-image-registry/image-registry-5c455b4d8b-ns98l" Jul 30 15:12:03.981351 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:12:03.981314 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-dvkw7" event={"ID":"9f3ebd63-d1cc-4920-94dc-65d6a62b96fa","Type":"ContainerStarted","Data":"6e18ef6c4a27b67c28fecaa1f5372c916357a51d55229cc706ad44a133efd03b"} Jul 30 15:12:03.982543 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:12:03.982516 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-6ff66dc4cf-kl2x6" event={"ID":"0af6b334-8a41-459d-9e05-4d3769d69a4c","Type":"ContainerStarted","Data":"9c750de421b290c39bd523e76846319800aa0ca68406eeb2b8e63eb2afcbd6af"} Jul 30 15:12:03.984370 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:12:03.984346 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-7dcnj" event={"ID":"16cf6c6b-a33a-445c-8a97-26226073093c","Type":"ContainerStarted","Data":"2c5875f57078a930bca26e225753f0192588398b0ba3b84422499f497a67d400"} Jul 30 15:12:03.984475 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:12:03.984390 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-7dcnj" event={"ID":"16cf6c6b-a33a-445c-8a97-26226073093c","Type":"ContainerStarted","Data":"da59835fe3bd03286ce65b1a4db744e47d4121d7dbf8135684f433e00e67d344"} Jul 30 15:12:03.984475 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:12:03.984409 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-7dcnj" event={"ID":"16cf6c6b-a33a-445c-8a97-26226073093c","Type":"ContainerStarted","Data":"22a96c162cb9add90c5e23b433b6c6a21aeb1a5d6ac6fe73385dde9fb679803b"} Jul 30 15:12:04.102450 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:12:04.102412 2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/monitoring-plugin-68b7fc9574-9h5mk"] Jul 30 15:12:04.114923 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:12:04.114884 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/monitoring-plugin-68b7fc9574-9h5mk" Jul 30 15:12:04.116564 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:12:04.116532 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7j468_a2197c57-2dc6-410e-9910-a045dedcb8d9/konnectivity-agent/0.log" limit=1024 Jul 30 15:12:04.117922 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:12:04.117899 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"monitoring-plugin-cert\"" Jul 30 15:12:04.118326 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:12:04.118300 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"monitoring-plugin-dockercfg-n6jvn\"" Jul 30 15:12:04.143138 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:12:04.143096 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/monitoring-plugin-68b7fc9574-9h5mk"] Jul 30 15:12:04.250909 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:12:04.250865 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/f29662a6-58a3-4a59-bc19-4e8bfc47de63-monitoring-plugin-cert\") pod \"monitoring-plugin-68b7fc9574-9h5mk\" (UID: \"f29662a6-58a3-4a59-bc19-4e8bfc47de63\") " pod="openshift-monitoring/monitoring-plugin-68b7fc9574-9h5mk" Jul 30 15:12:04.251150 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:12:04.250932 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9p49l\" (UniqueName: \"kubernetes.io/projected/f29662a6-58a3-4a59-bc19-4e8bfc47de63-kube-api-access-9p49l\") pod \"monitoring-plugin-68b7fc9574-9h5mk\" (UID: \"f29662a6-58a3-4a59-bc19-4e8bfc47de63\") " pod="openshift-monitoring/monitoring-plugin-68b7fc9574-9h5mk" Jul 30 15:12:04.268420 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:12:04.268386 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7j468_a2197c57-2dc6-410e-9910-a045dedcb8d9/konnectivity-agent/0.log" limit=1024 Jul 30 15:12:04.352184 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:12:04.352018 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/f29662a6-58a3-4a59-bc19-4e8bfc47de63-monitoring-plugin-cert\") pod \"monitoring-plugin-68b7fc9574-9h5mk\" (UID: \"f29662a6-58a3-4a59-bc19-4e8bfc47de63\") " pod="openshift-monitoring/monitoring-plugin-68b7fc9574-9h5mk" Jul 30 15:12:04.352184 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:12:04.352081 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9p49l\" (UniqueName: \"kubernetes.io/projected/f29662a6-58a3-4a59-bc19-4e8bfc47de63-kube-api-access-9p49l\") pod \"monitoring-plugin-68b7fc9574-9h5mk\" (UID: \"f29662a6-58a3-4a59-bc19-4e8bfc47de63\") " pod="openshift-monitoring/monitoring-plugin-68b7fc9574-9h5mk" Jul 30 15:12:04.356194 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:12:04.356163 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/f29662a6-58a3-4a59-bc19-4e8bfc47de63-monitoring-plugin-cert\") pod \"monitoring-plugin-68b7fc9574-9h5mk\" (UID: \"f29662a6-58a3-4a59-bc19-4e8bfc47de63\") " pod="openshift-monitoring/monitoring-plugin-68b7fc9574-9h5mk" Jul 30 15:12:04.361256 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:12:04.361230 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9p49l\" (UniqueName: \"kubernetes.io/projected/f29662a6-58a3-4a59-bc19-4e8bfc47de63-kube-api-access-9p49l\") pod \"monitoring-plugin-68b7fc9574-9h5mk\" (UID: \"f29662a6-58a3-4a59-bc19-4e8bfc47de63\") " pod="openshift-monitoring/monitoring-plugin-68b7fc9574-9h5mk" Jul 30 15:12:04.408449 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:12:04.408411 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7j468_a2197c57-2dc6-410e-9910-a045dedcb8d9/konnectivity-agent/0.log" limit=1024 Jul 30 15:12:04.426300 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:12:04.426266 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/monitoring-plugin-68b7fc9574-9h5mk" Jul 30 15:12:04.583256 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:12:04.583181 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/monitoring-plugin-68b7fc9574-9h5mk"] Jul 30 15:12:04.587077 ip-10-0-133-210 kubenswrapper[2576]: W0730 15:12:04.587044 2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf29662a6_58a3_4a59_bc19_4e8bfc47de63.slice/crio-62de42a879799980b70b4c177a183ba8be24857214b99e2b111a5e040554922f WatchSource:0}: Error finding container 62de42a879799980b70b4c177a183ba8be24857214b99e2b111a5e040554922f: Status 404 returned error can't find the container with id 62de42a879799980b70b4c177a183ba8be24857214b99e2b111a5e040554922f Jul 30 15:12:04.694167 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:12:04.694103 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7j468_a2197c57-2dc6-410e-9910-a045dedcb8d9/konnectivity-agent/0.log" limit=1024 Jul 30 15:12:04.992169 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:12:04.992134 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/monitoring-plugin-68b7fc9574-9h5mk" event={"ID":"f29662a6-58a3-4a59-bc19-4e8bfc47de63","Type":"ContainerStarted","Data":"62de42a879799980b70b4c177a183ba8be24857214b99e2b111a5e040554922f"} Jul 30 15:12:05.835969 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:12:05.835896 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7j468_a2197c57-2dc6-410e-9910-a045dedcb8d9/konnectivity-agent/0.log" limit=1024 Jul 30 15:12:06.723739 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:12:06.723700 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7j468_a2197c57-2dc6-410e-9910-a045dedcb8d9/konnectivity-agent/0.log" limit=1024 Jul 30 15:12:06.920075 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:12:06.920041 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7j468_a2197c57-2dc6-410e-9910-a045dedcb8d9/konnectivity-agent/0.log" limit=1024 Jul 30 15:12:09.970256 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:12:09.970220 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-jwbls" Jul 30 15:12:10.015513 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:12:10.014964 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-dvkw7" event={"ID":"9f3ebd63-d1cc-4920-94dc-65d6a62b96fa","Type":"ContainerStarted","Data":"661c6ba0721a7ccae24c602aa955673d560be4fc34d657a45454d5e83169ce12"} Jul 30 15:12:10.016946 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:12:10.016908 2576 generic.go:356] "Generic (PLEG): container finished" podID="fb41dae1-7bc8-46a1-a05e-73f26d7c0391" containerID="81517e59213ebf8aca12d31a5d0325c2ca776bba213d5143f37eb089900df234" exitCode=0 Jul 30 15:12:10.017083 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:12:10.016980 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-sv88g" event={"ID":"fb41dae1-7bc8-46a1-a05e-73f26d7c0391","Type":"ContainerDied","Data":"81517e59213ebf8aca12d31a5d0325c2ca776bba213d5143f37eb089900df234"} Jul 30 15:12:10.019658 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:12:10.019635 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-6ff66dc4cf-kl2x6" event={"ID":"0af6b334-8a41-459d-9e05-4d3769d69a4c","Type":"ContainerStarted","Data":"04dcaf6edf1858c6efbaa5d96299f02e40cb2e3253d87a64ab1a16347bdf8b3c"} Jul 30 15:12:10.019752 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:12:10.019691 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-6ff66dc4cf-kl2x6" event={"ID":"0af6b334-8a41-459d-9e05-4d3769d69a4c","Type":"ContainerStarted","Data":"ed27484dc0d1cc7da0b7bdc1852882234c0edfdfc2469443d4363ed0363f7b9c"} Jul 30 15:12:10.019752 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:12:10.019721 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-6ff66dc4cf-kl2x6" event={"ID":"0af6b334-8a41-459d-9e05-4d3769d69a4c","Type":"ContainerStarted","Data":"240acd165227a365b1f346902c292201bf2fc3fafaa15d9f0612bcbd22fb1948"} Jul 30 15:12:10.022294 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:12:10.022252 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-7dcnj" event={"ID":"16cf6c6b-a33a-445c-8a97-26226073093c","Type":"ContainerStarted","Data":"9168140b4e7404a11ad3eaabcb114a217297ce5760e593d943c20b13069b4aa1"} Jul 30 15:12:10.024699 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:12:10.024674 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/monitoring-plugin-68b7fc9574-9h5mk" event={"ID":"f29662a6-58a3-4a59-bc19-4e8bfc47de63","Type":"ContainerStarted","Data":"f31fb434b203588f2b2081f588f2aba8557f85ea82425b1f85d2576057dfd649"} Jul 30 15:12:10.025070 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:12:10.025050 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/monitoring-plugin-68b7fc9574-9h5mk" Jul 30 15:12:10.030872 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:12:10.030829 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/monitoring-plugin-68b7fc9574-9h5mk" Jul 30 15:12:10.084647 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:12:10.084605 2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-7dcnj" podStartSLOduration=5.784636692 podStartE2EDuration="11.084588716s" totalImagesPullingTime="5.299952024s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 15:11:59 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 15:12:10.083419473 +0000 UTC m=+112.009172120" watchObservedRunningTime="2026-07-30 15:12:10.084588716 +0000 UTC m=+112.010341362" Jul 30 15:12:10.116365 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:12:10.115564 2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/monitoring-plugin-68b7fc9574-9h5mk" podStartSLOduration=1.57537635 podStartE2EDuration="6.1155453s" totalImagesPullingTime="4.54016895s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 15:12:04 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 15:12:10.111202585 +0000 UTC m=+112.036955254" watchObservedRunningTime="2026-07-30 15:12:10.1155453 +0000 UTC m=+112.041297945" Jul 30 15:12:11.030962 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:12:11.030922 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-sv88g" event={"ID":"fb41dae1-7bc8-46a1-a05e-73f26d7c0391","Type":"ContainerStarted","Data":"afc12311fd9cb70deff92474bd2b495af8f387dff9fffc7ec4cf50ddbdda9371"} Jul 30 15:12:11.031435 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:12:11.030990 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-sv88g" event={"ID":"fb41dae1-7bc8-46a1-a05e-73f26d7c0391","Type":"ContainerStarted","Data":"cb24c731531d0ac8b3a978caf52d11ded8fba8e9a61b64fdd2817e5bbe119f82"} Jul 30 15:12:11.034487 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:12:11.034455 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-6ff66dc4cf-kl2x6" event={"ID":"0af6b334-8a41-459d-9e05-4d3769d69a4c","Type":"ContainerStarted","Data":"495d5497c2cc9f34e4af8e87cdb6dd9fcf1c3663319f10fd6332171125d7b600"} Jul 30 15:12:11.034690 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:12:11.034663 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-6ff66dc4cf-kl2x6" event={"ID":"0af6b334-8a41-459d-9e05-4d3769d69a4c","Type":"ContainerStarted","Data":"a85e581e48547c6537ec941e1045dccfefc16f9e7a166f231393148cfcaea421"} Jul 30 15:12:11.034787 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:12:11.034722 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/thanos-querier-6ff66dc4cf-kl2x6" Jul 30 15:12:11.034787 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:12:11.034763 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-6ff66dc4cf-kl2x6" event={"ID":"0af6b334-8a41-459d-9e05-4d3769d69a4c","Type":"ContainerStarted","Data":"6df2ba61eab30c539acb24e46e0718272148b030bfdceed0a5e49cd7e1b2623f"} Jul 30 15:12:11.056792 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:12:11.056741 2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-sv88g" podStartSLOduration=6.128752303 podStartE2EDuration="12.056726696s" totalImagesPullingTime="5.884523057s" totalInitContainerRuntime="43.451336ms" isStatefulPod=true podCreationTimestamp="2026-07-30 15:11:59 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 15:12:11.054760121 +0000 UTC m=+112.980512768" watchObservedRunningTime="2026-07-30 15:12:11.056726696 +0000 UTC m=+112.982479342" Jul 30 15:12:11.087560 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:12:11.087519 2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/thanos-querier-6ff66dc4cf-kl2x6" podStartSLOduration=2.3595190329999998 podStartE2EDuration="9.08750551s" totalImagesPullingTime="6.727986477s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 15:12:02 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 15:12:11.085862601 +0000 UTC m=+113.011615283" watchObservedRunningTime="2026-07-30 15:12:11.08750551 +0000 UTC m=+113.013258157" Jul 30 15:12:13.606021 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:12:13.605975 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7j468_a2197c57-2dc6-410e-9910-a045dedcb8d9/konnectivity-agent/0.log" limit=1024 Jul 30 15:12:13.702688 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:12:13.702645 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7j468_a2197c57-2dc6-410e-9910-a045dedcb8d9/konnectivity-agent/0.log" limit=1024 Jul 30 15:12:14.050879 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:12:14.050846 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-dvkw7" event={"ID":"9f3ebd63-d1cc-4920-94dc-65d6a62b96fa","Type":"ContainerStarted","Data":"d6d7b0d46d8793c1d2baeffacdfa0fd787af4c3cb0eba580679d5108c8d9df6f"} Jul 30 15:12:14.075732 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:12:14.073141 2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-dvkw7" podStartSLOduration=66.73288973 podStartE2EDuration="1m22.073097481s" totalImagesPullingTime="15.340207751s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-30 15:10:52 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 15:12:14.072275025 +0000 UTC m=+115.998027670" watchObservedRunningTime="2026-07-30 15:12:14.073097481 +0000 UTC m=+115.998850127" Jul 30 15:12:14.586541 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:12:14.586499 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7j468_a2197c57-2dc6-410e-9910-a045dedcb8d9/konnectivity-agent/0.log" limit=1024 Jul 30 15:12:14.690402 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:12:14.690372 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7j468_a2197c57-2dc6-410e-9910-a045dedcb8d9/konnectivity-agent/0.log" limit=1024 Jul 30 15:12:16.651229 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:12:16.651184 2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/downloads-7c97dc4555-w82pr"] Jul 30 15:12:16.673516 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:12:16.673487 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7j468_a2197c57-2dc6-410e-9910-a045dedcb8d9/konnectivity-agent/0.log" limit=1024 Jul 30 15:12:16.718672 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:12:16.718642 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/downloads-7c97dc4555-w82pr"] Jul 30 15:12:16.718793 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:12:16.718731 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/downloads-7c97dc4555-w82pr" Jul 30 15:12:16.727552 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:12:16.727533 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"default-dockercfg-g7hph\"" Jul 30 15:12:16.727670 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:12:16.727538 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"kube-root-ca.crt\"" Jul 30 15:12:16.727670 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:12:16.727632 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"openshift-service-ca.crt\"" Jul 30 15:12:16.773394 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:12:16.773361 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-d772l\" (UniqueName: \"kubernetes.io/projected/2ddfacc9-629f-447c-badb-ae14f1ed1abf-kube-api-access-d772l\") pod \"downloads-7c97dc4555-w82pr\" (UID: \"2ddfacc9-629f-447c-badb-ae14f1ed1abf\") " pod="openshift-console/downloads-7c97dc4555-w82pr" Jul 30 15:12:16.826627 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:12:16.826589 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7j468_a2197c57-2dc6-410e-9910-a045dedcb8d9/konnectivity-agent/0.log" limit=1024 Jul 30 15:12:16.874640 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:12:16.874603 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-d772l\" (UniqueName: \"kubernetes.io/projected/2ddfacc9-629f-447c-badb-ae14f1ed1abf-kube-api-access-d772l\") pod \"downloads-7c97dc4555-w82pr\" (UID: \"2ddfacc9-629f-447c-badb-ae14f1ed1abf\") " pod="openshift-console/downloads-7c97dc4555-w82pr" Jul 30 15:12:16.888561 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:12:16.888524 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-d772l\" (UniqueName: \"kubernetes.io/projected/2ddfacc9-629f-447c-badb-ae14f1ed1abf-kube-api-access-d772l\") pod \"downloads-7c97dc4555-w82pr\" (UID: \"2ddfacc9-629f-447c-badb-ae14f1ed1abf\") " pod="openshift-console/downloads-7c97dc4555-w82pr" Jul 30 15:12:17.004530 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:12:17.004495 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7j468_a2197c57-2dc6-410e-9910-a045dedcb8d9/konnectivity-agent/0.log" limit=1024 Jul 30 15:12:17.027332 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:12:17.027303 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/downloads-7c97dc4555-w82pr" Jul 30 15:12:17.044909 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:12:17.044884 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/thanos-querier-6ff66dc4cf-kl2x6" Jul 30 15:12:17.197176 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:12:17.197145 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/downloads-7c97dc4555-w82pr"] Jul 30 15:12:17.200228 ip-10-0-133-210 kubenswrapper[2576]: W0730 15:12:17.200197 2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2ddfacc9_629f_447c_badb_ae14f1ed1abf.slice/crio-69ae65243d38a1dd12a662cf489e981716eec2ef3c376fc97f53a81557d0f43a WatchSource:0}: Error finding container 69ae65243d38a1dd12a662cf489e981716eec2ef3c376fc97f53a81557d0f43a: Status 404 returned error can't find the container with id 69ae65243d38a1dd12a662cf489e981716eec2ef3c376fc97f53a81557d0f43a Jul 30 15:12:17.278783 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:12:17.278708 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7j468_a2197c57-2dc6-410e-9910-a045dedcb8d9/konnectivity-agent/0.log" limit=1024 Jul 30 15:12:17.390652 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:12:17.390608 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7j468_a2197c57-2dc6-410e-9910-a045dedcb8d9/konnectivity-agent/0.log" limit=1024 Jul 30 15:12:17.709553 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:12:17.709470 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7j468_a2197c57-2dc6-410e-9910-a045dedcb8d9/konnectivity-agent/0.log" limit=1024 Jul 30 15:12:18.067370 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:12:18.067326 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/downloads-7c97dc4555-w82pr" event={"ID":"2ddfacc9-629f-447c-badb-ae14f1ed1abf","Type":"ContainerStarted","Data":"69ae65243d38a1dd12a662cf489e981716eec2ef3c376fc97f53a81557d0f43a"} Jul 30 15:12:18.303605 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:12:18.303567 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7j468_a2197c57-2dc6-410e-9910-a045dedcb8d9/konnectivity-agent/0.log" limit=1024 Jul 30 15:12:18.513252 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:12:18.513211 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7j468_a2197c57-2dc6-410e-9910-a045dedcb8d9/konnectivity-agent/0.log" limit=1024 Jul 30 15:12:18.580585 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:12:18.580549 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-jwbls_c460206a-fa1f-4c09-b498-467313191c13/dns/0.log" Jul 30 15:12:18.802546 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:12:18.802414 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-jwbls_c460206a-fa1f-4c09-b498-467313191c13/kube-rbac-proxy/0.log" Jul 30 15:12:18.943742 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:12:18.943713 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7j468_a2197c57-2dc6-410e-9910-a045dedcb8d9/konnectivity-agent/0.log" limit=1024 Jul 30 15:12:19.166235 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:12:19.166169 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-j74qd_347bd19f-7728-47be-986d-337d0f0df246/dns-node-resolver/0.log" Jul 30 15:12:19.280415 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:12:19.280382 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7j468_a2197c57-2dc6-410e-9910-a045dedcb8d9/konnectivity-agent/0.log" limit=1024 Jul 30 15:12:19.433977 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:12:19.433878 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7j468_a2197c57-2dc6-410e-9910-a045dedcb8d9/konnectivity-agent/0.log" limit=1024 Jul 30 15:12:20.170354 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:12:20.170323 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-qtv7l_647c67b8-98e9-4905-9e04-35cc6cc3bd33/serve-healthcheck-canary/0.log" Jul 30 15:12:20.734300 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:12:20.734262 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7j468_a2197c57-2dc6-410e-9910-a045dedcb8d9/konnectivity-agent/0.log" limit=1024 Jul 30 15:12:20.949931 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:12:20.949893 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7j468_a2197c57-2dc6-410e-9910-a045dedcb8d9/konnectivity-agent/0.log" limit=1024 Jul 30 15:12:21.562392 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:12:21.562350 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7j468_a2197c57-2dc6-410e-9910-a045dedcb8d9/konnectivity-agent/0.log" limit=1024 Jul 30 15:12:21.727417 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:12:21.727335 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7j468_a2197c57-2dc6-410e-9910-a045dedcb8d9/konnectivity-agent/0.log" limit=1024 Jul 30 15:12:24.311298 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:12:24.311261 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7j468_a2197c57-2dc6-410e-9910-a045dedcb8d9/konnectivity-agent/0.log" limit=1024 Jul 30 15:12:24.421231 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:12:24.421198 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7j468_a2197c57-2dc6-410e-9910-a045dedcb8d9/konnectivity-agent/0.log" limit=1024 Jul 30 15:12:25.187244 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:12:25.187206 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7j468_a2197c57-2dc6-410e-9910-a045dedcb8d9/konnectivity-agent/0.log" limit=1024 Jul 30 15:12:25.293956 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:12:25.293920 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7j468_a2197c57-2dc6-410e-9910-a045dedcb8d9/konnectivity-agent/0.log" limit=1024 Jul 30 15:12:26.966946 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:12:26.966912 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7j468_a2197c57-2dc6-410e-9910-a045dedcb8d9/konnectivity-agent/0.log" limit=1024 Jul 30 15:12:27.127449 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:12:27.127413 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7j468_a2197c57-2dc6-410e-9910-a045dedcb8d9/konnectivity-agent/0.log" limit=1024 Jul 30 15:12:27.309051 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:12:27.308974 2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-674d5c6d68-bhkxd"] Jul 30 15:12:27.311616 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:12:27.311592 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-674d5c6d68-bhkxd" Jul 30 15:12:27.319013 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:12:27.318986 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"service-ca\"" Jul 30 15:12:27.319151 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:12:27.318987 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-oauth-config\"" Jul 30 15:12:27.319664 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:12:27.319432 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-dockercfg-zcgz2\"" Jul 30 15:12:27.319664 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:12:27.319560 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"oauth-serving-cert\"" Jul 30 15:12:27.320188 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:12:27.320166 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"console-config\"" Jul 30 15:12:27.320279 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:12:27.320172 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-serving-cert\"" Jul 30 15:12:27.334900 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:12:27.334847 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-674d5c6d68-bhkxd"] Jul 30 15:12:27.362922 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:12:27.362897 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-djlb6\" (UniqueName: \"kubernetes.io/projected/7649d750-27d4-4ac6-be2e-65c667e58ff9-kube-api-access-djlb6\") pod \"console-674d5c6d68-bhkxd\" (UID: \"7649d750-27d4-4ac6-be2e-65c667e58ff9\") " pod="openshift-console/console-674d5c6d68-bhkxd" Jul 30 15:12:27.363051 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:12:27.362937 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/7649d750-27d4-4ac6-be2e-65c667e58ff9-console-serving-cert\") pod \"console-674d5c6d68-bhkxd\" (UID: \"7649d750-27d4-4ac6-be2e-65c667e58ff9\") " pod="openshift-console/console-674d5c6d68-bhkxd" Jul 30 15:12:27.363051 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:12:27.362975 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/7649d750-27d4-4ac6-be2e-65c667e58ff9-console-config\") pod \"console-674d5c6d68-bhkxd\" (UID: \"7649d750-27d4-4ac6-be2e-65c667e58ff9\") " pod="openshift-console/console-674d5c6d68-bhkxd" Jul 30 15:12:27.363051 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:12:27.363032 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/7649d750-27d4-4ac6-be2e-65c667e58ff9-oauth-serving-cert\") pod \"console-674d5c6d68-bhkxd\" (UID: \"7649d750-27d4-4ac6-be2e-65c667e58ff9\") " pod="openshift-console/console-674d5c6d68-bhkxd" Jul 30 15:12:27.363201 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:12:27.363063 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/7649d750-27d4-4ac6-be2e-65c667e58ff9-console-oauth-config\") pod \"console-674d5c6d68-bhkxd\" (UID: \"7649d750-27d4-4ac6-be2e-65c667e58ff9\") " pod="openshift-console/console-674d5c6d68-bhkxd" Jul 30 15:12:27.363201 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:12:27.363088 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/7649d750-27d4-4ac6-be2e-65c667e58ff9-service-ca\") pod \"console-674d5c6d68-bhkxd\" (UID: \"7649d750-27d4-4ac6-be2e-65c667e58ff9\") " pod="openshift-console/console-674d5c6d68-bhkxd" Jul 30 15:12:27.463734 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:12:27.463687 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/7649d750-27d4-4ac6-be2e-65c667e58ff9-console-serving-cert\") pod \"console-674d5c6d68-bhkxd\" (UID: \"7649d750-27d4-4ac6-be2e-65c667e58ff9\") " pod="openshift-console/console-674d5c6d68-bhkxd" Jul 30 15:12:27.463903 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:12:27.463758 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/7649d750-27d4-4ac6-be2e-65c667e58ff9-console-config\") pod \"console-674d5c6d68-bhkxd\" (UID: \"7649d750-27d4-4ac6-be2e-65c667e58ff9\") " pod="openshift-console/console-674d5c6d68-bhkxd" Jul 30 15:12:27.463903 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:12:27.463787 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/7649d750-27d4-4ac6-be2e-65c667e58ff9-oauth-serving-cert\") pod \"console-674d5c6d68-bhkxd\" (UID: \"7649d750-27d4-4ac6-be2e-65c667e58ff9\") " pod="openshift-console/console-674d5c6d68-bhkxd" Jul 30 15:12:27.463903 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:12:27.463811 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/7649d750-27d4-4ac6-be2e-65c667e58ff9-console-oauth-config\") pod \"console-674d5c6d68-bhkxd\" (UID: \"7649d750-27d4-4ac6-be2e-65c667e58ff9\") " pod="openshift-console/console-674d5c6d68-bhkxd" Jul 30 15:12:27.463903 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:12:27.463827 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/7649d750-27d4-4ac6-be2e-65c667e58ff9-service-ca\") pod \"console-674d5c6d68-bhkxd\" (UID: \"7649d750-27d4-4ac6-be2e-65c667e58ff9\") " pod="openshift-console/console-674d5c6d68-bhkxd" Jul 30 15:12:27.464141 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:12:27.463907 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-djlb6\" (UniqueName: \"kubernetes.io/projected/7649d750-27d4-4ac6-be2e-65c667e58ff9-kube-api-access-djlb6\") pod \"console-674d5c6d68-bhkxd\" (UID: \"7649d750-27d4-4ac6-be2e-65c667e58ff9\") " pod="openshift-console/console-674d5c6d68-bhkxd" Jul 30 15:12:27.464851 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:12:27.464799 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/7649d750-27d4-4ac6-be2e-65c667e58ff9-oauth-serving-cert\") pod \"console-674d5c6d68-bhkxd\" (UID: \"7649d750-27d4-4ac6-be2e-65c667e58ff9\") " pod="openshift-console/console-674d5c6d68-bhkxd" Jul 30 15:12:27.464851 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:12:27.464841 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/7649d750-27d4-4ac6-be2e-65c667e58ff9-console-config\") pod \"console-674d5c6d68-bhkxd\" (UID: \"7649d750-27d4-4ac6-be2e-65c667e58ff9\") " pod="openshift-console/console-674d5c6d68-bhkxd" Jul 30 15:12:27.465213 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:12:27.465190 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/7649d750-27d4-4ac6-be2e-65c667e58ff9-service-ca\") pod \"console-674d5c6d68-bhkxd\" (UID: \"7649d750-27d4-4ac6-be2e-65c667e58ff9\") " pod="openshift-console/console-674d5c6d68-bhkxd" Jul 30 15:12:27.466887 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:12:27.466866 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/7649d750-27d4-4ac6-be2e-65c667e58ff9-console-oauth-config\") pod \"console-674d5c6d68-bhkxd\" (UID: \"7649d750-27d4-4ac6-be2e-65c667e58ff9\") " pod="openshift-console/console-674d5c6d68-bhkxd" Jul 30 15:12:27.467192 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:12:27.467165 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/7649d750-27d4-4ac6-be2e-65c667e58ff9-console-serving-cert\") pod \"console-674d5c6d68-bhkxd\" (UID: \"7649d750-27d4-4ac6-be2e-65c667e58ff9\") " pod="openshift-console/console-674d5c6d68-bhkxd" Jul 30 15:12:27.477907 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:12:27.477880 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-djlb6\" (UniqueName: \"kubernetes.io/projected/7649d750-27d4-4ac6-be2e-65c667e58ff9-kube-api-access-djlb6\") pod \"console-674d5c6d68-bhkxd\" (UID: \"7649d750-27d4-4ac6-be2e-65c667e58ff9\") " pod="openshift-console/console-674d5c6d68-bhkxd" Jul 30 15:12:27.623342 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:12:27.623236 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-674d5c6d68-bhkxd" Jul 30 15:12:27.776720 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:12:27.776687 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-674d5c6d68-bhkxd"] Jul 30 15:12:27.779612 ip-10-0-133-210 kubenswrapper[2576]: W0730 15:12:27.779567 2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7649d750_27d4_4ac6_be2e_65c667e58ff9.slice/crio-7ac66f900beb2e4c014d0bc2e751893ff7918f105ee3f5708a11111986f4f158 WatchSource:0}: Error finding container 7ac66f900beb2e4c014d0bc2e751893ff7918f105ee3f5708a11111986f4f158: Status 404 returned error can't find the container with id 7ac66f900beb2e4c014d0bc2e751893ff7918f105ee3f5708a11111986f4f158 Jul 30 15:12:28.110441 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:12:28.110405 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-674d5c6d68-bhkxd" event={"ID":"7649d750-27d4-4ac6-be2e-65c667e58ff9","Type":"ContainerStarted","Data":"7ac66f900beb2e4c014d0bc2e751893ff7918f105ee3f5708a11111986f4f158"} Jul 30 15:12:28.270878 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:12:28.270794 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/32b9046b-bd5c-4f23-b832-6fd0bad1c8d6-metrics-certs\") pod \"network-metrics-daemon-dbmt8\" (UID: \"32b9046b-bd5c-4f23-b832-6fd0bad1c8d6\") " pod="openshift-multus/network-metrics-daemon-dbmt8" Jul 30 15:12:28.273693 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:12:28.273667 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/32b9046b-bd5c-4f23-b832-6fd0bad1c8d6-metrics-certs\") pod \"network-metrics-daemon-dbmt8\" (UID: \"32b9046b-bd5c-4f23-b832-6fd0bad1c8d6\") " pod="openshift-multus/network-metrics-daemon-dbmt8" Jul 30 15:12:28.435429 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:12:28.435347 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-kvhp8\"" Jul 30 15:12:28.440930 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:12:28.440890 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-dbmt8" Jul 30 15:12:28.624754 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:12:28.624711 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-dbmt8"] Jul 30 15:12:33.567772 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:12:33.567734 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7j468_a2197c57-2dc6-410e-9910-a045dedcb8d9/konnectivity-agent/0.log" limit=1024 Jul 30 15:12:36.072086 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:12:36.072052 2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-8dc8b79db-6qpxs"] Jul 30 15:12:36.086238 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:12:36.086213 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-8dc8b79db-6qpxs" Jul 30 15:12:36.097422 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:12:36.097388 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-8dc8b79db-6qpxs"] Jul 30 15:12:36.099267 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:12:36.099148 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"trusted-ca-bundle\"" Jul 30 15:12:36.241434 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:12:36.241394 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/9347fded-d414-44c0-8e6e-4870e3f59c73-service-ca\") pod \"console-8dc8b79db-6qpxs\" (UID: \"9347fded-d414-44c0-8e6e-4870e3f59c73\") " pod="openshift-console/console-8dc8b79db-6qpxs" Jul 30 15:12:36.241620 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:12:36.241445 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/9347fded-d414-44c0-8e6e-4870e3f59c73-console-serving-cert\") pod \"console-8dc8b79db-6qpxs\" (UID: \"9347fded-d414-44c0-8e6e-4870e3f59c73\") " pod="openshift-console/console-8dc8b79db-6qpxs" Jul 30 15:12:36.241620 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:12:36.241501 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-92mgz\" (UniqueName: \"kubernetes.io/projected/9347fded-d414-44c0-8e6e-4870e3f59c73-kube-api-access-92mgz\") pod \"console-8dc8b79db-6qpxs\" (UID: \"9347fded-d414-44c0-8e6e-4870e3f59c73\") " pod="openshift-console/console-8dc8b79db-6qpxs" Jul 30 15:12:36.241620 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:12:36.241531 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/9347fded-d414-44c0-8e6e-4870e3f59c73-console-config\") pod \"console-8dc8b79db-6qpxs\" (UID: \"9347fded-d414-44c0-8e6e-4870e3f59c73\") " pod="openshift-console/console-8dc8b79db-6qpxs" Jul 30 15:12:36.241620 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:12:36.241554 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/9347fded-d414-44c0-8e6e-4870e3f59c73-console-oauth-config\") pod \"console-8dc8b79db-6qpxs\" (UID: \"9347fded-d414-44c0-8e6e-4870e3f59c73\") " pod="openshift-console/console-8dc8b79db-6qpxs" Jul 30 15:12:36.241620 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:12:36.241592 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/9347fded-d414-44c0-8e6e-4870e3f59c73-trusted-ca-bundle\") pod \"console-8dc8b79db-6qpxs\" (UID: \"9347fded-d414-44c0-8e6e-4870e3f59c73\") " pod="openshift-console/console-8dc8b79db-6qpxs" Jul 30 15:12:36.241794 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:12:36.241634 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/9347fded-d414-44c0-8e6e-4870e3f59c73-oauth-serving-cert\") pod \"console-8dc8b79db-6qpxs\" (UID: \"9347fded-d414-44c0-8e6e-4870e3f59c73\") " pod="openshift-console/console-8dc8b79db-6qpxs" Jul 30 15:12:36.342797 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:12:36.342706 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/9347fded-d414-44c0-8e6e-4870e3f59c73-console-serving-cert\") pod \"console-8dc8b79db-6qpxs\" (UID: \"9347fded-d414-44c0-8e6e-4870e3f59c73\") " pod="openshift-console/console-8dc8b79db-6qpxs" Jul 30 15:12:36.342972 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:12:36.342858 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-92mgz\" (UniqueName: \"kubernetes.io/projected/9347fded-d414-44c0-8e6e-4870e3f59c73-kube-api-access-92mgz\") pod \"console-8dc8b79db-6qpxs\" (UID: \"9347fded-d414-44c0-8e6e-4870e3f59c73\") " pod="openshift-console/console-8dc8b79db-6qpxs" Jul 30 15:12:36.342972 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:12:36.342899 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/9347fded-d414-44c0-8e6e-4870e3f59c73-console-config\") pod \"console-8dc8b79db-6qpxs\" (UID: \"9347fded-d414-44c0-8e6e-4870e3f59c73\") " pod="openshift-console/console-8dc8b79db-6qpxs" Jul 30 15:12:36.342972 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:12:36.342929 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/9347fded-d414-44c0-8e6e-4870e3f59c73-console-oauth-config\") pod \"console-8dc8b79db-6qpxs\" (UID: \"9347fded-d414-44c0-8e6e-4870e3f59c73\") " pod="openshift-console/console-8dc8b79db-6qpxs" Jul 30 15:12:36.342972 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:12:36.342969 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/9347fded-d414-44c0-8e6e-4870e3f59c73-trusted-ca-bundle\") pod \"console-8dc8b79db-6qpxs\" (UID: \"9347fded-d414-44c0-8e6e-4870e3f59c73\") " pod="openshift-console/console-8dc8b79db-6qpxs" Jul 30 15:12:36.343219 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:12:36.342994 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/9347fded-d414-44c0-8e6e-4870e3f59c73-oauth-serving-cert\") pod \"console-8dc8b79db-6qpxs\" (UID: \"9347fded-d414-44c0-8e6e-4870e3f59c73\") " pod="openshift-console/console-8dc8b79db-6qpxs" Jul 30 15:12:36.343219 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:12:36.343033 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/9347fded-d414-44c0-8e6e-4870e3f59c73-service-ca\") pod \"console-8dc8b79db-6qpxs\" (UID: \"9347fded-d414-44c0-8e6e-4870e3f59c73\") " pod="openshift-console/console-8dc8b79db-6qpxs" Jul 30 15:12:36.343749 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:12:36.343699 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/9347fded-d414-44c0-8e6e-4870e3f59c73-console-config\") pod \"console-8dc8b79db-6qpxs\" (UID: \"9347fded-d414-44c0-8e6e-4870e3f59c73\") " pod="openshift-console/console-8dc8b79db-6qpxs" Jul 30 15:12:36.343749 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:12:36.343717 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/9347fded-d414-44c0-8e6e-4870e3f59c73-service-ca\") pod \"console-8dc8b79db-6qpxs\" (UID: \"9347fded-d414-44c0-8e6e-4870e3f59c73\") " pod="openshift-console/console-8dc8b79db-6qpxs" Jul 30 15:12:36.343936 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:12:36.343870 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/9347fded-d414-44c0-8e6e-4870e3f59c73-oauth-serving-cert\") pod \"console-8dc8b79db-6qpxs\" (UID: \"9347fded-d414-44c0-8e6e-4870e3f59c73\") " pod="openshift-console/console-8dc8b79db-6qpxs" Jul 30 15:12:36.344207 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:12:36.344184 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/9347fded-d414-44c0-8e6e-4870e3f59c73-trusted-ca-bundle\") pod \"console-8dc8b79db-6qpxs\" (UID: \"9347fded-d414-44c0-8e6e-4870e3f59c73\") " pod="openshift-console/console-8dc8b79db-6qpxs" Jul 30 15:12:36.345571 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:12:36.345549 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/9347fded-d414-44c0-8e6e-4870e3f59c73-console-serving-cert\") pod \"console-8dc8b79db-6qpxs\" (UID: \"9347fded-d414-44c0-8e6e-4870e3f59c73\") " pod="openshift-console/console-8dc8b79db-6qpxs" Jul 30 15:12:36.346050 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:12:36.346028 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/9347fded-d414-44c0-8e6e-4870e3f59c73-console-oauth-config\") pod \"console-8dc8b79db-6qpxs\" (UID: \"9347fded-d414-44c0-8e6e-4870e3f59c73\") " pod="openshift-console/console-8dc8b79db-6qpxs" Jul 30 15:12:36.354441 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:12:36.354393 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-92mgz\" (UniqueName: \"kubernetes.io/projected/9347fded-d414-44c0-8e6e-4870e3f59c73-kube-api-access-92mgz\") pod \"console-8dc8b79db-6qpxs\" (UID: \"9347fded-d414-44c0-8e6e-4870e3f59c73\") " pod="openshift-console/console-8dc8b79db-6qpxs" Jul 30 15:12:36.398502 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:12:36.398459 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-8dc8b79db-6qpxs" Jul 30 15:12:36.511136 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:12:36.511066 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7j468_a2197c57-2dc6-410e-9910-a045dedcb8d9/konnectivity-agent/0.log" limit=1024 Jul 30 15:12:36.670172 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:12:36.670077 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7j468_a2197c57-2dc6-410e-9910-a045dedcb8d9/konnectivity-agent/0.log" limit=1024 Jul 30 15:12:37.006696 ip-10-0-133-210 kubenswrapper[2576]: W0730 15:12:37.006665 2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod32b9046b_bd5c_4f23_b832_6fd0bad1c8d6.slice/crio-5fc0bd785b612dc89cfa9b63216137dd52c1c04e425d7ba69604684dc1f7e6ee WatchSource:0}: Error finding container 5fc0bd785b612dc89cfa9b63216137dd52c1c04e425d7ba69604684dc1f7e6ee: Status 404 returned error can't find the container with id 5fc0bd785b612dc89cfa9b63216137dd52c1c04e425d7ba69604684dc1f7e6ee Jul 30 15:12:37.148615 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:12:37.148577 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-dbmt8" event={"ID":"32b9046b-bd5c-4f23-b832-6fd0bad1c8d6","Type":"ContainerStarted","Data":"5fc0bd785b612dc89cfa9b63216137dd52c1c04e425d7ba69604684dc1f7e6ee"} Jul 30 15:12:37.354572 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:12:37.350989 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-8dc8b79db-6qpxs"] Jul 30 15:12:37.359810 ip-10-0-133-210 kubenswrapper[2576]: W0730 15:12:37.359779 2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9347fded_d414_44c0_8e6e_4870e3f59c73.slice/crio-c4f8539ab62597a2782e918c854b92ba1609b8d7f307269f100947dfa05a8ded WatchSource:0}: Error finding container c4f8539ab62597a2782e918c854b92ba1609b8d7f307269f100947dfa05a8ded: Status 404 returned error can't find the container with id c4f8539ab62597a2782e918c854b92ba1609b8d7f307269f100947dfa05a8ded Jul 30 15:12:38.156677 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:12:38.156561 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-8dc8b79db-6qpxs" event={"ID":"9347fded-d414-44c0-8e6e-4870e3f59c73","Type":"ContainerStarted","Data":"3184e8fcc0f21d1aeacb654e1a0f8b64604756285a78bde910e7ede5efe09515"} Jul 30 15:12:38.156677 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:12:38.156628 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-8dc8b79db-6qpxs" event={"ID":"9347fded-d414-44c0-8e6e-4870e3f59c73","Type":"ContainerStarted","Data":"c4f8539ab62597a2782e918c854b92ba1609b8d7f307269f100947dfa05a8ded"} Jul 30 15:12:38.159897 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:12:38.159589 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-674d5c6d68-bhkxd" event={"ID":"7649d750-27d4-4ac6-be2e-65c667e58ff9","Type":"ContainerStarted","Data":"84f4e0b86222fa9a9d5addd181d79180b7d012276c6755b27b665b8b73658f4c"} Jul 30 15:12:38.163162 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:12:38.163131 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/downloads-7c97dc4555-w82pr" event={"ID":"2ddfacc9-629f-447c-badb-ae14f1ed1abf","Type":"ContainerStarted","Data":"aa36c6674a68923c7ca162fac49f425983984b6bfb7d6d36acd8425684d6dabe"} Jul 30 15:12:38.163664 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:12:38.163619 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/downloads-7c97dc4555-w82pr" Jul 30 15:12:38.179813 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:12:38.179780 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/downloads-7c97dc4555-w82pr" Jul 30 15:12:38.197159 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:12:38.196690 2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-8dc8b79db-6qpxs" podStartSLOduration=2.196668797 podStartE2EDuration="2.196668797s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 15:12:36 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 15:12:38.196290003 +0000 UTC m=+140.122042677" watchObservedRunningTime="2026-07-30 15:12:38.196668797 +0000 UTC m=+140.122421435" Jul 30 15:12:38.265627 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:12:38.265564 2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-674d5c6d68-bhkxd" podStartSLOduration=1.8580571959999999 podStartE2EDuration="11.265544226s" totalImagesPullingTime="9.40748703s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 15:12:27 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 15:12:38.265442984 +0000 UTC m=+140.191195631" watchObservedRunningTime="2026-07-30 15:12:38.265544226 +0000 UTC m=+140.191296873" Jul 30 15:12:38.321053 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:12:38.320988 2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/downloads-7c97dc4555-w82pr" podStartSLOduration=2.287575142 podStartE2EDuration="22.320963696s" totalImagesPullingTime="20.033388554s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 15:12:16 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 15:12:38.320500368 +0000 UTC m=+140.246253034" watchObservedRunningTime="2026-07-30 15:12:38.320963696 +0000 UTC m=+140.246716342" Jul 30 15:12:39.171772 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:12:39.171680 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-dbmt8" event={"ID":"32b9046b-bd5c-4f23-b832-6fd0bad1c8d6","Type":"ContainerStarted","Data":"8ce3e5a40e951320a4656ee5a6711c355d69d52d8c2c56c82412f652a8bee829"} Jul 30 15:12:39.171772 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:12:39.171742 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-dbmt8" event={"ID":"32b9046b-bd5c-4f23-b832-6fd0bad1c8d6","Type":"ContainerStarted","Data":"8e5687b2245649050843b739eca8f9a8b74929526892512a672afc1eaf2f40c5"} Jul 30 15:12:39.183621 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:12:39.183587 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7j468_a2197c57-2dc6-410e-9910-a045dedcb8d9/konnectivity-agent/0.log" limit=1024 Jul 30 15:12:39.335467 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:12:39.335424 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7j468_a2197c57-2dc6-410e-9910-a045dedcb8d9/konnectivity-agent/0.log" limit=1024 Jul 30 15:12:39.507443 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:12:39.507400 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7j468_a2197c57-2dc6-410e-9910-a045dedcb8d9/konnectivity-agent/0.log" limit=1024 Jul 30 15:12:39.675813 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:12:39.675773 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7j468_a2197c57-2dc6-410e-9910-a045dedcb8d9/konnectivity-agent/0.log" limit=1024 Jul 30 15:12:41.280771 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:12:41.280734 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7j468_a2197c57-2dc6-410e-9910-a045dedcb8d9/konnectivity-agent/0.log" limit=1024 Jul 30 15:12:41.450045 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:12:41.450002 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7j468_a2197c57-2dc6-410e-9910-a045dedcb8d9/konnectivity-agent/0.log" limit=1024 Jul 30 15:12:43.905897 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:12:43.905867 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7j468_a2197c57-2dc6-410e-9910-a045dedcb8d9/konnectivity-agent/0.log" limit=1024 Jul 30 15:12:43.997913 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:12:43.997869 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7j468_a2197c57-2dc6-410e-9910-a045dedcb8d9/konnectivity-agent/0.log" limit=1024 Jul 30 15:12:46.398595 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:12:46.398556 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-8dc8b79db-6qpxs" Jul 30 15:12:46.399088 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:12:46.398671 2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-8dc8b79db-6qpxs" Jul 30 15:12:46.403753 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:12:46.403727 2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-8dc8b79db-6qpxs" Jul 30 15:12:46.428783 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:12:46.426370 2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-dbmt8" podStartSLOduration=146.986123013 podStartE2EDuration="2m28.426351345s" totalImagesPullingTime="1.440228332s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 15:10:18 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 15:12:39.199741831 +0000 UTC m=+141.125494477" watchObservedRunningTime="2026-07-30 15:12:46.426351345 +0000 UTC m=+148.352103991" Jul 30 15:12:47.210425 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:12:47.210398 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-8dc8b79db-6qpxs" Jul 30 15:12:47.280359 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:12:47.280327 2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-674d5c6d68-bhkxd"] Jul 30 15:12:47.623985 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:12:47.623948 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-674d5c6d68-bhkxd" Jul 30 15:12:48.478899 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:12:48.478867 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7j468_a2197c57-2dc6-410e-9910-a045dedcb8d9/konnectivity-agent/0.log" limit=1024 Jul 30 15:12:48.552045 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:12:48.552008 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7j468_a2197c57-2dc6-410e-9910-a045dedcb8d9/konnectivity-agent/0.log" limit=1024 Jul 30 15:12:49.092463 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:12:49.092431 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7j468_a2197c57-2dc6-410e-9910-a045dedcb8d9/konnectivity-agent/0.log" limit=1024 Jul 30 15:12:49.189352 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:12:49.189311 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7j468_a2197c57-2dc6-410e-9910-a045dedcb8d9/konnectivity-agent/0.log" limit=1024 Jul 30 15:12:50.105593 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:12:50.105556 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7j468_a2197c57-2dc6-410e-9910-a045dedcb8d9/konnectivity-agent/0.log" limit=1024 Jul 30 15:12:50.202955 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:12:50.202919 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7j468_a2197c57-2dc6-410e-9910-a045dedcb8d9/konnectivity-agent/0.log" limit=1024 Jul 30 15:13:06.451064 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:13:06.451026 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7j468_a2197c57-2dc6-410e-9910-a045dedcb8d9/konnectivity-agent/0.log" limit=1024 Jul 30 15:13:06.589747 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:13:06.589706 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7j468_a2197c57-2dc6-410e-9910-a045dedcb8d9/konnectivity-agent/0.log" limit=1024 Jul 30 15:13:12.298877 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:13:12.298793 2576 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-674d5c6d68-bhkxd" podUID="7649d750-27d4-4ac6-be2e-65c667e58ff9" containerName="console" containerID="cri-o://84f4e0b86222fa9a9d5addd181d79180b7d012276c6755b27b665b8b73658f4c" gracePeriod=15 Jul 30 15:13:12.572006 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:13:12.571941 2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-674d5c6d68-bhkxd" Jul 30 15:13:12.649404 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:13:12.649369 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-djlb6\" (UniqueName: \"kubernetes.io/projected/7649d750-27d4-4ac6-be2e-65c667e58ff9-kube-api-access-djlb6\") pod \"7649d750-27d4-4ac6-be2e-65c667e58ff9\" (UID: \"7649d750-27d4-4ac6-be2e-65c667e58ff9\") " Jul 30 15:13:12.649587 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:13:12.649411 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/7649d750-27d4-4ac6-be2e-65c667e58ff9-oauth-serving-cert\") pod \"7649d750-27d4-4ac6-be2e-65c667e58ff9\" (UID: \"7649d750-27d4-4ac6-be2e-65c667e58ff9\") " Jul 30 15:13:12.649587 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:13:12.649436 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/7649d750-27d4-4ac6-be2e-65c667e58ff9-service-ca\") pod \"7649d750-27d4-4ac6-be2e-65c667e58ff9\" (UID: \"7649d750-27d4-4ac6-be2e-65c667e58ff9\") " Jul 30 15:13:12.649587 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:13:12.649462 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/7649d750-27d4-4ac6-be2e-65c667e58ff9-console-serving-cert\") pod \"7649d750-27d4-4ac6-be2e-65c667e58ff9\" (UID: \"7649d750-27d4-4ac6-be2e-65c667e58ff9\") " Jul 30 15:13:12.649587 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:13:12.649540 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/7649d750-27d4-4ac6-be2e-65c667e58ff9-console-oauth-config\") pod \"7649d750-27d4-4ac6-be2e-65c667e58ff9\" (UID: \"7649d750-27d4-4ac6-be2e-65c667e58ff9\") " Jul 30 15:13:12.649587 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:13:12.649566 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/7649d750-27d4-4ac6-be2e-65c667e58ff9-console-config\") pod \"7649d750-27d4-4ac6-be2e-65c667e58ff9\" (UID: \"7649d750-27d4-4ac6-be2e-65c667e58ff9\") " Jul 30 15:13:12.649893 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:13:12.649815 2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7649d750-27d4-4ac6-be2e-65c667e58ff9-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "7649d750-27d4-4ac6-be2e-65c667e58ff9" (UID: "7649d750-27d4-4ac6-be2e-65c667e58ff9"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 30 15:13:12.649972 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:13:12.649945 2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7649d750-27d4-4ac6-be2e-65c667e58ff9-service-ca" (OuterVolumeSpecName: "service-ca") pod "7649d750-27d4-4ac6-be2e-65c667e58ff9" (UID: "7649d750-27d4-4ac6-be2e-65c667e58ff9"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 30 15:13:12.650043 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:13:12.650016 2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7649d750-27d4-4ac6-be2e-65c667e58ff9-console-config" (OuterVolumeSpecName: "console-config") pod "7649d750-27d4-4ac6-be2e-65c667e58ff9" (UID: "7649d750-27d4-4ac6-be2e-65c667e58ff9"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 30 15:13:12.651776 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:13:12.651753 2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/7649d750-27d4-4ac6-be2e-65c667e58ff9-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "7649d750-27d4-4ac6-be2e-65c667e58ff9" (UID: "7649d750-27d4-4ac6-be2e-65c667e58ff9"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 30 15:13:12.651891 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:13:12.651815 2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7649d750-27d4-4ac6-be2e-65c667e58ff9-kube-api-access-djlb6" (OuterVolumeSpecName: "kube-api-access-djlb6") pod "7649d750-27d4-4ac6-be2e-65c667e58ff9" (UID: "7649d750-27d4-4ac6-be2e-65c667e58ff9"). InnerVolumeSpecName "kube-api-access-djlb6". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 30 15:13:12.651891 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:13:12.651839 2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/7649d750-27d4-4ac6-be2e-65c667e58ff9-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "7649d750-27d4-4ac6-be2e-65c667e58ff9" (UID: "7649d750-27d4-4ac6-be2e-65c667e58ff9"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 30 15:13:12.751127 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:13:12.751083 2576 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/7649d750-27d4-4ac6-be2e-65c667e58ff9-console-oauth-config\") on node \"ip-10-0-133-210.ec2.internal\" DevicePath \"\"" Jul 30 15:13:12.751271 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:13:12.751139 2576 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/7649d750-27d4-4ac6-be2e-65c667e58ff9-console-config\") on node \"ip-10-0-133-210.ec2.internal\" DevicePath \"\"" Jul 30 15:13:12.751271 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:13:12.751149 2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-djlb6\" (UniqueName: \"kubernetes.io/projected/7649d750-27d4-4ac6-be2e-65c667e58ff9-kube-api-access-djlb6\") on node \"ip-10-0-133-210.ec2.internal\" DevicePath \"\"" Jul 30 15:13:12.751271 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:13:12.751158 2576 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/7649d750-27d4-4ac6-be2e-65c667e58ff9-oauth-serving-cert\") on node \"ip-10-0-133-210.ec2.internal\" DevicePath \"\"" Jul 30 15:13:12.751271 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:13:12.751166 2576 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/7649d750-27d4-4ac6-be2e-65c667e58ff9-service-ca\") on node \"ip-10-0-133-210.ec2.internal\" DevicePath \"\"" Jul 30 15:13:12.751271 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:13:12.751175 2576 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/7649d750-27d4-4ac6-be2e-65c667e58ff9-console-serving-cert\") on node \"ip-10-0-133-210.ec2.internal\" DevicePath \"\"" Jul 30 15:13:13.303085 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:13:13.303050 2576 generic.go:356] "Generic (PLEG): container finished" podID="7649d750-27d4-4ac6-be2e-65c667e58ff9" containerID="84f4e0b86222fa9a9d5addd181d79180b7d012276c6755b27b665b8b73658f4c" exitCode=0 Jul 30 15:13:13.303510 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:13:13.303100 2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-674d5c6d68-bhkxd" Jul 30 15:13:13.303510 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:13:13.303142 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-674d5c6d68-bhkxd" event={"ID":"7649d750-27d4-4ac6-be2e-65c667e58ff9","Type":"ContainerDied","Data":"84f4e0b86222fa9a9d5addd181d79180b7d012276c6755b27b665b8b73658f4c"} Jul 30 15:13:13.303510 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:13:13.303202 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-674d5c6d68-bhkxd" event={"ID":"7649d750-27d4-4ac6-be2e-65c667e58ff9","Type":"ContainerDied","Data":"7ac66f900beb2e4c014d0bc2e751893ff7918f105ee3f5708a11111986f4f158"} Jul 30 15:13:13.303510 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:13:13.303226 2576 scope.go:117] "RemoveContainer" containerID="84f4e0b86222fa9a9d5addd181d79180b7d012276c6755b27b665b8b73658f4c" Jul 30 15:13:13.313485 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:13:13.312394 2576 scope.go:117] "RemoveContainer" containerID="84f4e0b86222fa9a9d5addd181d79180b7d012276c6755b27b665b8b73658f4c" Jul 30 15:13:13.316480 ip-10-0-133-210 kubenswrapper[2576]: E0730 15:13:13.316447 2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"84f4e0b86222fa9a9d5addd181d79180b7d012276c6755b27b665b8b73658f4c\": container with ID starting with 84f4e0b86222fa9a9d5addd181d79180b7d012276c6755b27b665b8b73658f4c not found: ID does not exist" containerID="84f4e0b86222fa9a9d5addd181d79180b7d012276c6755b27b665b8b73658f4c" Jul 30 15:13:13.316587 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:13:13.316491 2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"84f4e0b86222fa9a9d5addd181d79180b7d012276c6755b27b665b8b73658f4c"} err="failed to get container status \"84f4e0b86222fa9a9d5addd181d79180b7d012276c6755b27b665b8b73658f4c\": rpc error: code = NotFound desc = could not find container \"84f4e0b86222fa9a9d5addd181d79180b7d012276c6755b27b665b8b73658f4c\": container with ID starting with 84f4e0b86222fa9a9d5addd181d79180b7d012276c6755b27b665b8b73658f4c not found: ID does not exist" Jul 30 15:13:13.326583 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:13:13.326553 2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-674d5c6d68-bhkxd"] Jul 30 15:13:13.330606 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:13:13.330583 2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-674d5c6d68-bhkxd"] Jul 30 15:13:14.611844 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:13:14.611812 2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="7649d750-27d4-4ac6-be2e-65c667e58ff9" path="/var/lib/kubelet/pods/7649d750-27d4-4ac6-be2e-65c667e58ff9/volumes" Jul 30 15:13:18.967618 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:13:18.967565 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7j468_a2197c57-2dc6-410e-9910-a045dedcb8d9/konnectivity-agent/0.log" limit=1024 Jul 30 15:13:19.093934 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:13:19.093878 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7j468_a2197c57-2dc6-410e-9910-a045dedcb8d9/konnectivity-agent/0.log" limit=1024 Jul 30 15:13:21.472510 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:13:21.472451 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7j468_a2197c57-2dc6-410e-9910-a045dedcb8d9/konnectivity-agent/0.log" limit=1024 Jul 30 15:13:22.900667 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:13:22.900635 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7j468_a2197c57-2dc6-410e-9910-a045dedcb8d9/konnectivity-agent/0.log" limit=1024 Jul 30 15:13:28.117971 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:13:28.117938 2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-84957b45b9-gfkpb"] Jul 30 15:13:28.118381 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:13:28.118234 2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7649d750-27d4-4ac6-be2e-65c667e58ff9" containerName="console" Jul 30 15:13:28.118381 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:13:28.118245 2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="7649d750-27d4-4ac6-be2e-65c667e58ff9" containerName="console" Jul 30 15:13:28.118381 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:13:28.118300 2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="7649d750-27d4-4ac6-be2e-65c667e58ff9" containerName="console" Jul 30 15:13:28.120841 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:13:28.120821 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-84957b45b9-gfkpb" Jul 30 15:13:28.136227 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:13:28.136194 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-84957b45b9-gfkpb"] Jul 30 15:13:28.169981 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:13:28.169943 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/0b57ea84-6127-4db8-aced-5ac87eb8185d-oauth-serving-cert\") pod \"console-84957b45b9-gfkpb\" (UID: \"0b57ea84-6127-4db8-aced-5ac87eb8185d\") " pod="openshift-console/console-84957b45b9-gfkpb" Jul 30 15:13:28.169981 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:13:28.169982 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5sh6t\" (UniqueName: \"kubernetes.io/projected/0b57ea84-6127-4db8-aced-5ac87eb8185d-kube-api-access-5sh6t\") pod \"console-84957b45b9-gfkpb\" (UID: \"0b57ea84-6127-4db8-aced-5ac87eb8185d\") " pod="openshift-console/console-84957b45b9-gfkpb" Jul 30 15:13:28.170253 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:13:28.170002 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/0b57ea84-6127-4db8-aced-5ac87eb8185d-console-serving-cert\") pod \"console-84957b45b9-gfkpb\" (UID: \"0b57ea84-6127-4db8-aced-5ac87eb8185d\") " pod="openshift-console/console-84957b45b9-gfkpb" Jul 30 15:13:28.170253 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:13:28.170079 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/0b57ea84-6127-4db8-aced-5ac87eb8185d-service-ca\") pod \"console-84957b45b9-gfkpb\" (UID: \"0b57ea84-6127-4db8-aced-5ac87eb8185d\") " pod="openshift-console/console-84957b45b9-gfkpb" Jul 30 15:13:28.170253 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:13:28.170160 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/0b57ea84-6127-4db8-aced-5ac87eb8185d-trusted-ca-bundle\") pod \"console-84957b45b9-gfkpb\" (UID: \"0b57ea84-6127-4db8-aced-5ac87eb8185d\") " pod="openshift-console/console-84957b45b9-gfkpb" Jul 30 15:13:28.170253 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:13:28.170205 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/0b57ea84-6127-4db8-aced-5ac87eb8185d-console-config\") pod \"console-84957b45b9-gfkpb\" (UID: \"0b57ea84-6127-4db8-aced-5ac87eb8185d\") " pod="openshift-console/console-84957b45b9-gfkpb" Jul 30 15:13:28.170408 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:13:28.170262 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/0b57ea84-6127-4db8-aced-5ac87eb8185d-console-oauth-config\") pod \"console-84957b45b9-gfkpb\" (UID: \"0b57ea84-6127-4db8-aced-5ac87eb8185d\") " pod="openshift-console/console-84957b45b9-gfkpb" Jul 30 15:13:28.189003 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:13:28.188960 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7j468_a2197c57-2dc6-410e-9910-a045dedcb8d9/konnectivity-agent/0.log" limit=1024 Jul 30 15:13:28.270978 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:13:28.270943 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/0b57ea84-6127-4db8-aced-5ac87eb8185d-console-oauth-config\") pod \"console-84957b45b9-gfkpb\" (UID: \"0b57ea84-6127-4db8-aced-5ac87eb8185d\") " pod="openshift-console/console-84957b45b9-gfkpb" Jul 30 15:13:28.271176 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:13:28.270984 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/0b57ea84-6127-4db8-aced-5ac87eb8185d-oauth-serving-cert\") pod \"console-84957b45b9-gfkpb\" (UID: \"0b57ea84-6127-4db8-aced-5ac87eb8185d\") " pod="openshift-console/console-84957b45b9-gfkpb" Jul 30 15:13:28.271176 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:13:28.271020 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5sh6t\" (UniqueName: \"kubernetes.io/projected/0b57ea84-6127-4db8-aced-5ac87eb8185d-kube-api-access-5sh6t\") pod \"console-84957b45b9-gfkpb\" (UID: \"0b57ea84-6127-4db8-aced-5ac87eb8185d\") " pod="openshift-console/console-84957b45b9-gfkpb" Jul 30 15:13:28.271176 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:13:28.271048 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/0b57ea84-6127-4db8-aced-5ac87eb8185d-console-serving-cert\") pod \"console-84957b45b9-gfkpb\" (UID: \"0b57ea84-6127-4db8-aced-5ac87eb8185d\") " pod="openshift-console/console-84957b45b9-gfkpb" Jul 30 15:13:28.271176 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:13:28.271076 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/0b57ea84-6127-4db8-aced-5ac87eb8185d-service-ca\") pod \"console-84957b45b9-gfkpb\" (UID: \"0b57ea84-6127-4db8-aced-5ac87eb8185d\") " pod="openshift-console/console-84957b45b9-gfkpb" Jul 30 15:13:28.271176 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:13:28.271140 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/0b57ea84-6127-4db8-aced-5ac87eb8185d-trusted-ca-bundle\") pod \"console-84957b45b9-gfkpb\" (UID: \"0b57ea84-6127-4db8-aced-5ac87eb8185d\") " pod="openshift-console/console-84957b45b9-gfkpb" Jul 30 15:13:28.271401 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:13:28.271183 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/0b57ea84-6127-4db8-aced-5ac87eb8185d-console-config\") pod \"console-84957b45b9-gfkpb\" (UID: \"0b57ea84-6127-4db8-aced-5ac87eb8185d\") " pod="openshift-console/console-84957b45b9-gfkpb" Jul 30 15:13:28.271857 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:13:28.271827 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/0b57ea84-6127-4db8-aced-5ac87eb8185d-oauth-serving-cert\") pod \"console-84957b45b9-gfkpb\" (UID: \"0b57ea84-6127-4db8-aced-5ac87eb8185d\") " pod="openshift-console/console-84957b45b9-gfkpb" Jul 30 15:13:28.271969 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:13:28.271827 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/0b57ea84-6127-4db8-aced-5ac87eb8185d-service-ca\") pod \"console-84957b45b9-gfkpb\" (UID: \"0b57ea84-6127-4db8-aced-5ac87eb8185d\") " pod="openshift-console/console-84957b45b9-gfkpb" Jul 30 15:13:28.271969 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:13:28.271940 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/0b57ea84-6127-4db8-aced-5ac87eb8185d-trusted-ca-bundle\") pod \"console-84957b45b9-gfkpb\" (UID: \"0b57ea84-6127-4db8-aced-5ac87eb8185d\") " pod="openshift-console/console-84957b45b9-gfkpb" Jul 30 15:13:28.271969 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:13:28.271959 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/0b57ea84-6127-4db8-aced-5ac87eb8185d-console-config\") pod \"console-84957b45b9-gfkpb\" (UID: \"0b57ea84-6127-4db8-aced-5ac87eb8185d\") " pod="openshift-console/console-84957b45b9-gfkpb" Jul 30 15:13:28.273566 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:13:28.273542 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/0b57ea84-6127-4db8-aced-5ac87eb8185d-console-serving-cert\") pod \"console-84957b45b9-gfkpb\" (UID: \"0b57ea84-6127-4db8-aced-5ac87eb8185d\") " pod="openshift-console/console-84957b45b9-gfkpb" Jul 30 15:13:28.273681 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:13:28.273586 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/0b57ea84-6127-4db8-aced-5ac87eb8185d-console-oauth-config\") pod \"console-84957b45b9-gfkpb\" (UID: \"0b57ea84-6127-4db8-aced-5ac87eb8185d\") " pod="openshift-console/console-84957b45b9-gfkpb" Jul 30 15:13:28.281140 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:13:28.281056 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5sh6t\" (UniqueName: \"kubernetes.io/projected/0b57ea84-6127-4db8-aced-5ac87eb8185d-kube-api-access-5sh6t\") pod \"console-84957b45b9-gfkpb\" (UID: \"0b57ea84-6127-4db8-aced-5ac87eb8185d\") " pod="openshift-console/console-84957b45b9-gfkpb" Jul 30 15:13:28.307060 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:13:28.303977 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7j468_a2197c57-2dc6-410e-9910-a045dedcb8d9/konnectivity-agent/0.log" limit=1024 Jul 30 15:13:28.430463 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:13:28.430367 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-84957b45b9-gfkpb" Jul 30 15:13:28.575980 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:13:28.575950 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-84957b45b9-gfkpb"] Jul 30 15:13:28.577715 ip-10-0-133-210 kubenswrapper[2576]: W0730 15:13:28.577690 2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0b57ea84_6127_4db8_aced_5ac87eb8185d.slice/crio-21de9d059b3bec4d61bba2ec558dbddb7bee4cc5a27ce3dbf73c84111b6dcc2b WatchSource:0}: Error finding container 21de9d059b3bec4d61bba2ec558dbddb7bee4cc5a27ce3dbf73c84111b6dcc2b: Status 404 returned error can't find the container with id 21de9d059b3bec4d61bba2ec558dbddb7bee4cc5a27ce3dbf73c84111b6dcc2b Jul 30 15:13:29.364793 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:13:29.364756 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-84957b45b9-gfkpb" event={"ID":"0b57ea84-6127-4db8-aced-5ac87eb8185d","Type":"ContainerStarted","Data":"f5511121b6f5a6f727e76d38ea5fb1fee76a12b93bba30f410c3707b6914f0eb"} Jul 30 15:13:29.365308 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:13:29.364806 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-84957b45b9-gfkpb" event={"ID":"0b57ea84-6127-4db8-aced-5ac87eb8185d","Type":"ContainerStarted","Data":"21de9d059b3bec4d61bba2ec558dbddb7bee4cc5a27ce3dbf73c84111b6dcc2b"} Jul 30 15:13:29.385641 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:13:29.385590 2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-84957b45b9-gfkpb" podStartSLOduration=1.3855747090000001 podStartE2EDuration="1.385574709s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 15:13:28 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 15:13:29.385041659 +0000 UTC m=+191.310794338" watchObservedRunningTime="2026-07-30 15:13:29.385574709 +0000 UTC m=+191.311327417" Jul 30 15:13:38.430842 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:13:38.430800 2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-84957b45b9-gfkpb" Jul 30 15:13:38.431274 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:13:38.430880 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-84957b45b9-gfkpb" Jul 30 15:13:38.435563 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:13:38.435539 2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-84957b45b9-gfkpb" Jul 30 15:13:39.404730 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:13:39.404702 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-84957b45b9-gfkpb" Jul 30 15:13:39.465353 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:13:39.465323 2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-8dc8b79db-6qpxs"] Jul 30 15:13:39.579081 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:13:39.579036 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7j468_a2197c57-2dc6-410e-9910-a045dedcb8d9/konnectivity-agent/0.log" limit=1024 Jul 30 15:13:39.680368 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:13:39.680281 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7j468_a2197c57-2dc6-410e-9910-a045dedcb8d9/konnectivity-agent/0.log" limit=1024 Jul 30 15:13:39.749318 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:13:39.749284 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7j468_a2197c57-2dc6-410e-9910-a045dedcb8d9/konnectivity-agent/0.log" limit=1024 Jul 30 15:13:46.064460 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:13:46.064428 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7j468_a2197c57-2dc6-410e-9910-a045dedcb8d9/konnectivity-agent/0.log" limit=1024 Jul 30 15:13:47.425849 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:13:47.425801 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7j468_a2197c57-2dc6-410e-9910-a045dedcb8d9/konnectivity-agent/0.log" limit=1024 Jul 30 15:13:47.497474 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:13:47.497434 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7j468_a2197c57-2dc6-410e-9910-a045dedcb8d9/konnectivity-agent/0.log" limit=1024 Jul 30 15:14:04.484065 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:14:04.484006 2576 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-8dc8b79db-6qpxs" podUID="9347fded-d414-44c0-8e6e-4870e3f59c73" containerName="console" containerID="cri-o://3184e8fcc0f21d1aeacb654e1a0f8b64604756285a78bde910e7ede5efe09515" gracePeriod=15 Jul 30 15:14:04.733244 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:14:04.733219 2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-8dc8b79db-6qpxs" Jul 30 15:14:04.762724 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:14:04.762304 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/9347fded-d414-44c0-8e6e-4870e3f59c73-console-oauth-config\") pod \"9347fded-d414-44c0-8e6e-4870e3f59c73\" (UID: \"9347fded-d414-44c0-8e6e-4870e3f59c73\") " Jul 30 15:14:04.762724 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:14:04.762362 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/9347fded-d414-44c0-8e6e-4870e3f59c73-service-ca\") pod \"9347fded-d414-44c0-8e6e-4870e3f59c73\" (UID: \"9347fded-d414-44c0-8e6e-4870e3f59c73\") " Jul 30 15:14:04.762724 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:14:04.762581 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-92mgz\" (UniqueName: \"kubernetes.io/projected/9347fded-d414-44c0-8e6e-4870e3f59c73-kube-api-access-92mgz\") pod \"9347fded-d414-44c0-8e6e-4870e3f59c73\" (UID: \"9347fded-d414-44c0-8e6e-4870e3f59c73\") " Jul 30 15:14:04.763017 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:14:04.762761 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/9347fded-d414-44c0-8e6e-4870e3f59c73-console-serving-cert\") pod \"9347fded-d414-44c0-8e6e-4870e3f59c73\" (UID: \"9347fded-d414-44c0-8e6e-4870e3f59c73\") " Jul 30 15:14:04.763017 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:14:04.762813 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/9347fded-d414-44c0-8e6e-4870e3f59c73-oauth-serving-cert\") pod \"9347fded-d414-44c0-8e6e-4870e3f59c73\" (UID: \"9347fded-d414-44c0-8e6e-4870e3f59c73\") " Jul 30 15:14:04.763017 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:14:04.762860 2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9347fded-d414-44c0-8e6e-4870e3f59c73-service-ca" (OuterVolumeSpecName: "service-ca") pod "9347fded-d414-44c0-8e6e-4870e3f59c73" (UID: "9347fded-d414-44c0-8e6e-4870e3f59c73"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 30 15:14:04.763017 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:14:04.762877 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/9347fded-d414-44c0-8e6e-4870e3f59c73-console-config\") pod \"9347fded-d414-44c0-8e6e-4870e3f59c73\" (UID: \"9347fded-d414-44c0-8e6e-4870e3f59c73\") " Jul 30 15:14:04.763017 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:14:04.762940 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/9347fded-d414-44c0-8e6e-4870e3f59c73-trusted-ca-bundle\") pod \"9347fded-d414-44c0-8e6e-4870e3f59c73\" (UID: \"9347fded-d414-44c0-8e6e-4870e3f59c73\") " Jul 30 15:14:04.763567 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:14:04.763545 2576 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/9347fded-d414-44c0-8e6e-4870e3f59c73-service-ca\") on node \"ip-10-0-133-210.ec2.internal\" DevicePath \"\"" Jul 30 15:14:04.763787 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:14:04.763747 2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9347fded-d414-44c0-8e6e-4870e3f59c73-console-config" (OuterVolumeSpecName: "console-config") pod "9347fded-d414-44c0-8e6e-4870e3f59c73" (UID: "9347fded-d414-44c0-8e6e-4870e3f59c73"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 30 15:14:04.763947 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:14:04.763817 2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9347fded-d414-44c0-8e6e-4870e3f59c73-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "9347fded-d414-44c0-8e6e-4870e3f59c73" (UID: "9347fded-d414-44c0-8e6e-4870e3f59c73"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 30 15:14:04.763947 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:14:04.763853 2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9347fded-d414-44c0-8e6e-4870e3f59c73-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "9347fded-d414-44c0-8e6e-4870e3f59c73" (UID: "9347fded-d414-44c0-8e6e-4870e3f59c73"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 30 15:14:04.764934 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:14:04.764902 2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9347fded-d414-44c0-8e6e-4870e3f59c73-kube-api-access-92mgz" (OuterVolumeSpecName: "kube-api-access-92mgz") pod "9347fded-d414-44c0-8e6e-4870e3f59c73" (UID: "9347fded-d414-44c0-8e6e-4870e3f59c73"). InnerVolumeSpecName "kube-api-access-92mgz". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 30 15:14:04.765046 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:14:04.765021 2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/9347fded-d414-44c0-8e6e-4870e3f59c73-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "9347fded-d414-44c0-8e6e-4870e3f59c73" (UID: "9347fded-d414-44c0-8e6e-4870e3f59c73"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 30 15:14:04.765692 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:14:04.765666 2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/9347fded-d414-44c0-8e6e-4870e3f59c73-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "9347fded-d414-44c0-8e6e-4870e3f59c73" (UID: "9347fded-d414-44c0-8e6e-4870e3f59c73"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 30 15:14:04.864977 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:14:04.864944 2576 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/9347fded-d414-44c0-8e6e-4870e3f59c73-oauth-serving-cert\") on node \"ip-10-0-133-210.ec2.internal\" DevicePath \"\"" Jul 30 15:14:04.864977 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:14:04.864970 2576 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/9347fded-d414-44c0-8e6e-4870e3f59c73-console-config\") on node \"ip-10-0-133-210.ec2.internal\" DevicePath \"\"" Jul 30 15:14:04.864977 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:14:04.864981 2576 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/9347fded-d414-44c0-8e6e-4870e3f59c73-trusted-ca-bundle\") on node \"ip-10-0-133-210.ec2.internal\" DevicePath \"\"" Jul 30 15:14:04.865237 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:14:04.864990 2576 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/9347fded-d414-44c0-8e6e-4870e3f59c73-console-oauth-config\") on node \"ip-10-0-133-210.ec2.internal\" DevicePath \"\"" Jul 30 15:14:04.865237 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:14:04.864999 2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-92mgz\" (UniqueName: \"kubernetes.io/projected/9347fded-d414-44c0-8e6e-4870e3f59c73-kube-api-access-92mgz\") on node \"ip-10-0-133-210.ec2.internal\" DevicePath \"\"" Jul 30 15:14:04.865237 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:14:04.865008 2576 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/9347fded-d414-44c0-8e6e-4870e3f59c73-console-serving-cert\") on node \"ip-10-0-133-210.ec2.internal\" DevicePath \"\"" Jul 30 15:14:05.495039 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:14:05.495007 2576 generic.go:356] "Generic (PLEG): container finished" podID="9347fded-d414-44c0-8e6e-4870e3f59c73" containerID="3184e8fcc0f21d1aeacb654e1a0f8b64604756285a78bde910e7ede5efe09515" exitCode=0 Jul 30 15:14:05.495433 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:14:05.495062 2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-8dc8b79db-6qpxs" Jul 30 15:14:05.495433 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:14:05.495067 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-8dc8b79db-6qpxs" event={"ID":"9347fded-d414-44c0-8e6e-4870e3f59c73","Type":"ContainerDied","Data":"3184e8fcc0f21d1aeacb654e1a0f8b64604756285a78bde910e7ede5efe09515"} Jul 30 15:14:05.495433 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:14:05.495193 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-8dc8b79db-6qpxs" event={"ID":"9347fded-d414-44c0-8e6e-4870e3f59c73","Type":"ContainerDied","Data":"c4f8539ab62597a2782e918c854b92ba1609b8d7f307269f100947dfa05a8ded"} Jul 30 15:14:05.495433 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:14:05.495215 2576 scope.go:117] "RemoveContainer" containerID="3184e8fcc0f21d1aeacb654e1a0f8b64604756285a78bde910e7ede5efe09515" Jul 30 15:14:05.503983 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:14:05.503966 2576 scope.go:117] "RemoveContainer" containerID="3184e8fcc0f21d1aeacb654e1a0f8b64604756285a78bde910e7ede5efe09515" Jul 30 15:14:05.504270 ip-10-0-133-210 kubenswrapper[2576]: E0730 15:14:05.504249 2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3184e8fcc0f21d1aeacb654e1a0f8b64604756285a78bde910e7ede5efe09515\": container with ID starting with 3184e8fcc0f21d1aeacb654e1a0f8b64604756285a78bde910e7ede5efe09515 not found: ID does not exist" containerID="3184e8fcc0f21d1aeacb654e1a0f8b64604756285a78bde910e7ede5efe09515" Jul 30 15:14:05.504327 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:14:05.504279 2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3184e8fcc0f21d1aeacb654e1a0f8b64604756285a78bde910e7ede5efe09515"} err="failed to get container status \"3184e8fcc0f21d1aeacb654e1a0f8b64604756285a78bde910e7ede5efe09515\": rpc error: code = NotFound desc = could not find container \"3184e8fcc0f21d1aeacb654e1a0f8b64604756285a78bde910e7ede5efe09515\": container with ID starting with 3184e8fcc0f21d1aeacb654e1a0f8b64604756285a78bde910e7ede5efe09515 not found: ID does not exist" Jul 30 15:14:05.516100 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:14:05.516076 2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-8dc8b79db-6qpxs"] Jul 30 15:14:05.522384 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:14:05.522364 2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-8dc8b79db-6qpxs"] Jul 30 15:14:06.611190 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:14:06.611159 2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="9347fded-d414-44c0-8e6e-4870e3f59c73" path="/var/lib/kubelet/pods/9347fded-d414-44c0-8e6e-4870e3f59c73/volumes" Jul 30 15:14:11.114230 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:14:11.114177 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7j468_a2197c57-2dc6-410e-9910-a045dedcb8d9/konnectivity-agent/0.log" limit=1024 Jul 30 15:14:11.464706 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:14:11.464668 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7j468_a2197c57-2dc6-410e-9910-a045dedcb8d9/konnectivity-agent/0.log" limit=1024 Jul 30 15:14:11.525998 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:14:11.525964 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7j468_a2197c57-2dc6-410e-9910-a045dedcb8d9/konnectivity-agent/0.log" limit=1024 Jul 30 15:14:34.129098 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:14:34.129052 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7j468_a2197c57-2dc6-410e-9910-a045dedcb8d9/konnectivity-agent/0.log" limit=1024 Jul 30 15:14:34.226974 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:14:34.226927 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7j468_a2197c57-2dc6-410e-9910-a045dedcb8d9/konnectivity-agent/0.log" limit=1024 Jul 30 15:14:39.793975 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:14:39.793936 2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-74d998c855-zjnm7"] Jul 30 15:14:39.794389 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:14:39.794300 2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9347fded-d414-44c0-8e6e-4870e3f59c73" containerName="console" Jul 30 15:14:39.794389 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:14:39.794314 2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="9347fded-d414-44c0-8e6e-4870e3f59c73" containerName="console" Jul 30 15:14:39.794389 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:14:39.794367 2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="9347fded-d414-44c0-8e6e-4870e3f59c73" containerName="console" Jul 30 15:14:39.796230 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:14:39.796212 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-74d998c855-zjnm7" Jul 30 15:14:39.810882 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:14:39.810852 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-74d998c855-zjnm7"] Jul 30 15:14:39.859783 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:14:39.859744 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7j468_a2197c57-2dc6-410e-9910-a045dedcb8d9/konnectivity-agent/0.log" limit=1024 Jul 30 15:14:39.934054 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:14:39.934016 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/0a8d4006-7747-4036-9b25-16ade9760013-oauth-serving-cert\") pod \"console-74d998c855-zjnm7\" (UID: \"0a8d4006-7747-4036-9b25-16ade9760013\") " pod="openshift-console/console-74d998c855-zjnm7" Jul 30 15:14:39.934241 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:14:39.934064 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/0a8d4006-7747-4036-9b25-16ade9760013-trusted-ca-bundle\") pod \"console-74d998c855-zjnm7\" (UID: \"0a8d4006-7747-4036-9b25-16ade9760013\") " pod="openshift-console/console-74d998c855-zjnm7" Jul 30 15:14:39.934241 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:14:39.934136 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/0a8d4006-7747-4036-9b25-16ade9760013-console-oauth-config\") pod \"console-74d998c855-zjnm7\" (UID: \"0a8d4006-7747-4036-9b25-16ade9760013\") " pod="openshift-console/console-74d998c855-zjnm7" Jul 30 15:14:39.934241 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:14:39.934182 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/0a8d4006-7747-4036-9b25-16ade9760013-console-config\") pod \"console-74d998c855-zjnm7\" (UID: \"0a8d4006-7747-4036-9b25-16ade9760013\") " pod="openshift-console/console-74d998c855-zjnm7" Jul 30 15:14:39.934241 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:14:39.934202 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/0a8d4006-7747-4036-9b25-16ade9760013-service-ca\") pod \"console-74d998c855-zjnm7\" (UID: \"0a8d4006-7747-4036-9b25-16ade9760013\") " pod="openshift-console/console-74d998c855-zjnm7" Jul 30 15:14:39.934241 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:14:39.934233 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4qh47\" (UniqueName: \"kubernetes.io/projected/0a8d4006-7747-4036-9b25-16ade9760013-kube-api-access-4qh47\") pod \"console-74d998c855-zjnm7\" (UID: \"0a8d4006-7747-4036-9b25-16ade9760013\") " pod="openshift-console/console-74d998c855-zjnm7" Jul 30 15:14:39.934405 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:14:39.934257 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/0a8d4006-7747-4036-9b25-16ade9760013-console-serving-cert\") pod \"console-74d998c855-zjnm7\" (UID: \"0a8d4006-7747-4036-9b25-16ade9760013\") " pod="openshift-console/console-74d998c855-zjnm7" Jul 30 15:14:39.982590 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:14:39.982558 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7j468_a2197c57-2dc6-410e-9910-a045dedcb8d9/konnectivity-agent/0.log" limit=1024 Jul 30 15:14:40.035666 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:14:40.035582 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/0a8d4006-7747-4036-9b25-16ade9760013-oauth-serving-cert\") pod \"console-74d998c855-zjnm7\" (UID: \"0a8d4006-7747-4036-9b25-16ade9760013\") " pod="openshift-console/console-74d998c855-zjnm7" Jul 30 15:14:40.035666 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:14:40.035627 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/0a8d4006-7747-4036-9b25-16ade9760013-trusted-ca-bundle\") pod \"console-74d998c855-zjnm7\" (UID: \"0a8d4006-7747-4036-9b25-16ade9760013\") " pod="openshift-console/console-74d998c855-zjnm7" Jul 30 15:14:40.035666 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:14:40.035646 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/0a8d4006-7747-4036-9b25-16ade9760013-console-oauth-config\") pod \"console-74d998c855-zjnm7\" (UID: \"0a8d4006-7747-4036-9b25-16ade9760013\") " pod="openshift-console/console-74d998c855-zjnm7" Jul 30 15:14:40.035946 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:14:40.035703 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/0a8d4006-7747-4036-9b25-16ade9760013-console-config\") pod \"console-74d998c855-zjnm7\" (UID: \"0a8d4006-7747-4036-9b25-16ade9760013\") " pod="openshift-console/console-74d998c855-zjnm7" Jul 30 15:14:40.035946 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:14:40.035746 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/0a8d4006-7747-4036-9b25-16ade9760013-service-ca\") pod \"console-74d998c855-zjnm7\" (UID: \"0a8d4006-7747-4036-9b25-16ade9760013\") " pod="openshift-console/console-74d998c855-zjnm7" Jul 30 15:14:40.035946 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:14:40.035768 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4qh47\" (UniqueName: \"kubernetes.io/projected/0a8d4006-7747-4036-9b25-16ade9760013-kube-api-access-4qh47\") pod \"console-74d998c855-zjnm7\" (UID: \"0a8d4006-7747-4036-9b25-16ade9760013\") " pod="openshift-console/console-74d998c855-zjnm7" Jul 30 15:14:40.035946 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:14:40.035800 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/0a8d4006-7747-4036-9b25-16ade9760013-console-serving-cert\") pod \"console-74d998c855-zjnm7\" (UID: \"0a8d4006-7747-4036-9b25-16ade9760013\") " pod="openshift-console/console-74d998c855-zjnm7" Jul 30 15:14:40.036588 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:14:40.036560 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/0a8d4006-7747-4036-9b25-16ade9760013-console-config\") pod \"console-74d998c855-zjnm7\" (UID: \"0a8d4006-7747-4036-9b25-16ade9760013\") " pod="openshift-console/console-74d998c855-zjnm7" Jul 30 15:14:40.036712 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:14:40.036594 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/0a8d4006-7747-4036-9b25-16ade9760013-service-ca\") pod \"console-74d998c855-zjnm7\" (UID: \"0a8d4006-7747-4036-9b25-16ade9760013\") " pod="openshift-console/console-74d998c855-zjnm7" Jul 30 15:14:40.036712 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:14:40.036596 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/0a8d4006-7747-4036-9b25-16ade9760013-trusted-ca-bundle\") pod \"console-74d998c855-zjnm7\" (UID: \"0a8d4006-7747-4036-9b25-16ade9760013\") " pod="openshift-console/console-74d998c855-zjnm7" Jul 30 15:14:40.036712 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:14:40.036690 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/0a8d4006-7747-4036-9b25-16ade9760013-oauth-serving-cert\") pod \"console-74d998c855-zjnm7\" (UID: \"0a8d4006-7747-4036-9b25-16ade9760013\") " pod="openshift-console/console-74d998c855-zjnm7" Jul 30 15:14:40.038194 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:14:40.038167 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/0a8d4006-7747-4036-9b25-16ade9760013-console-serving-cert\") pod \"console-74d998c855-zjnm7\" (UID: \"0a8d4006-7747-4036-9b25-16ade9760013\") " pod="openshift-console/console-74d998c855-zjnm7" Jul 30 15:14:40.038283 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:14:40.038222 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/0a8d4006-7747-4036-9b25-16ade9760013-console-oauth-config\") pod \"console-74d998c855-zjnm7\" (UID: \"0a8d4006-7747-4036-9b25-16ade9760013\") " pod="openshift-console/console-74d998c855-zjnm7" Jul 30 15:14:40.046616 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:14:40.046560 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4qh47\" (UniqueName: \"kubernetes.io/projected/0a8d4006-7747-4036-9b25-16ade9760013-kube-api-access-4qh47\") pod \"console-74d998c855-zjnm7\" (UID: \"0a8d4006-7747-4036-9b25-16ade9760013\") " pod="openshift-console/console-74d998c855-zjnm7" Jul 30 15:14:40.066627 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:14:40.066589 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7j468_a2197c57-2dc6-410e-9910-a045dedcb8d9/konnectivity-agent/0.log" limit=1024 Jul 30 15:14:40.105362 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:14:40.105316 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-74d998c855-zjnm7" Jul 30 15:14:40.240445 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:14:40.240412 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-74d998c855-zjnm7"] Jul 30 15:14:40.243338 ip-10-0-133-210 kubenswrapper[2576]: W0730 15:14:40.243308 2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0a8d4006_7747_4036_9b25_16ade9760013.slice/crio-dbf8ec740e6f455f1a899244e50a7dd450a6728c8703702c2d76a7c0de97d872 WatchSource:0}: Error finding container dbf8ec740e6f455f1a899244e50a7dd450a6728c8703702c2d76a7c0de97d872: Status 404 returned error can't find the container with id dbf8ec740e6f455f1a899244e50a7dd450a6728c8703702c2d76a7c0de97d872 Jul 30 15:14:40.593316 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:14:40.593271 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7j468_a2197c57-2dc6-410e-9910-a045dedcb8d9/konnectivity-agent/0.log" limit=1024 Jul 30 15:14:40.619790 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:14:40.619756 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-74d998c855-zjnm7" event={"ID":"0a8d4006-7747-4036-9b25-16ade9760013","Type":"ContainerStarted","Data":"7087f395b29883f3ea21c15c9ac631139d0af8d94af44a42edb245c38817c80f"} Jul 30 15:14:40.619954 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:14:40.619802 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-74d998c855-zjnm7" event={"ID":"0a8d4006-7747-4036-9b25-16ade9760013","Type":"ContainerStarted","Data":"dbf8ec740e6f455f1a899244e50a7dd450a6728c8703702c2d76a7c0de97d872"} Jul 30 15:14:40.645559 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:14:40.645509 2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-74d998c855-zjnm7" podStartSLOduration=1.6454935069999999 podStartE2EDuration="1.645493507s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 15:14:39 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 15:14:40.644055772 +0000 UTC m=+262.569808418" watchObservedRunningTime="2026-07-30 15:14:40.645493507 +0000 UTC m=+262.571246152" Jul 30 15:14:40.694307 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:14:40.694278 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7j468_a2197c57-2dc6-410e-9910-a045dedcb8d9/konnectivity-agent/0.log" limit=1024 Jul 30 15:14:46.126380 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:14:46.126342 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7j468_a2197c57-2dc6-410e-9910-a045dedcb8d9/konnectivity-agent/0.log" limit=1024 Jul 30 15:14:46.226942 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:14:46.226903 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7j468_a2197c57-2dc6-410e-9910-a045dedcb8d9/konnectivity-agent/0.log" limit=1024 Jul 30 15:14:50.105885 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:14:50.105841 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-74d998c855-zjnm7" Jul 30 15:14:50.106440 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:14:50.105947 2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-74d998c855-zjnm7" Jul 30 15:14:50.110626 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:14:50.110602 2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-74d998c855-zjnm7" Jul 30 15:14:50.658001 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:14:50.657970 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-74d998c855-zjnm7" Jul 30 15:14:50.754820 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:14:50.754780 2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-84957b45b9-gfkpb"] Jul 30 15:14:51.888644 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:14:51.888613 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7j468_a2197c57-2dc6-410e-9910-a045dedcb8d9/konnectivity-agent/0.log" limit=1024 Jul 30 15:14:51.994463 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:14:51.994426 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7j468_a2197c57-2dc6-410e-9910-a045dedcb8d9/konnectivity-agent/0.log" limit=1024 Jul 30 15:14:54.524233 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:14:54.524198 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7j468_a2197c57-2dc6-410e-9910-a045dedcb8d9/konnectivity-agent/0.log" limit=1024 Jul 30 15:14:54.624677 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:14:54.624642 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7j468_a2197c57-2dc6-410e-9910-a045dedcb8d9/konnectivity-agent/0.log" limit=1024 Jul 30 15:15:15.773708 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:15:15.773650 2576 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-84957b45b9-gfkpb" podUID="0b57ea84-6127-4db8-aced-5ac87eb8185d" containerName="console" containerID="cri-o://f5511121b6f5a6f727e76d38ea5fb1fee76a12b93bba30f410c3707b6914f0eb" gracePeriod=15 Jul 30 15:15:16.016642 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:15:16.016611 2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-84957b45b9-gfkpb" Jul 30 15:15:16.129386 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:15:16.129297 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/0b57ea84-6127-4db8-aced-5ac87eb8185d-console-serving-cert\") pod \"0b57ea84-6127-4db8-aced-5ac87eb8185d\" (UID: \"0b57ea84-6127-4db8-aced-5ac87eb8185d\") " Jul 30 15:15:16.129386 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:15:16.129347 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/0b57ea84-6127-4db8-aced-5ac87eb8185d-service-ca\") pod \"0b57ea84-6127-4db8-aced-5ac87eb8185d\" (UID: \"0b57ea84-6127-4db8-aced-5ac87eb8185d\") " Jul 30 15:15:16.129386 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:15:16.129365 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/0b57ea84-6127-4db8-aced-5ac87eb8185d-trusted-ca-bundle\") pod \"0b57ea84-6127-4db8-aced-5ac87eb8185d\" (UID: \"0b57ea84-6127-4db8-aced-5ac87eb8185d\") " Jul 30 15:15:16.129643 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:15:16.129416 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/0b57ea84-6127-4db8-aced-5ac87eb8185d-console-oauth-config\") pod \"0b57ea84-6127-4db8-aced-5ac87eb8185d\" (UID: \"0b57ea84-6127-4db8-aced-5ac87eb8185d\") " Jul 30 15:15:16.129643 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:15:16.129439 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/0b57ea84-6127-4db8-aced-5ac87eb8185d-oauth-serving-cert\") pod \"0b57ea84-6127-4db8-aced-5ac87eb8185d\" (UID: \"0b57ea84-6127-4db8-aced-5ac87eb8185d\") " Jul 30 15:15:16.129643 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:15:16.129454 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-5sh6t\" (UniqueName: \"kubernetes.io/projected/0b57ea84-6127-4db8-aced-5ac87eb8185d-kube-api-access-5sh6t\") pod \"0b57ea84-6127-4db8-aced-5ac87eb8185d\" (UID: \"0b57ea84-6127-4db8-aced-5ac87eb8185d\") " Jul 30 15:15:16.129643 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:15:16.129486 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/0b57ea84-6127-4db8-aced-5ac87eb8185d-console-config\") pod \"0b57ea84-6127-4db8-aced-5ac87eb8185d\" (UID: \"0b57ea84-6127-4db8-aced-5ac87eb8185d\") " Jul 30 15:15:16.129889 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:15:16.129835 2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0b57ea84-6127-4db8-aced-5ac87eb8185d-service-ca" (OuterVolumeSpecName: "service-ca") pod "0b57ea84-6127-4db8-aced-5ac87eb8185d" (UID: "0b57ea84-6127-4db8-aced-5ac87eb8185d"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 30 15:15:16.130010 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:15:16.129960 2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0b57ea84-6127-4db8-aced-5ac87eb8185d-console-config" (OuterVolumeSpecName: "console-config") pod "0b57ea84-6127-4db8-aced-5ac87eb8185d" (UID: "0b57ea84-6127-4db8-aced-5ac87eb8185d"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 30 15:15:16.130010 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:15:16.129966 2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0b57ea84-6127-4db8-aced-5ac87eb8185d-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "0b57ea84-6127-4db8-aced-5ac87eb8185d" (UID: "0b57ea84-6127-4db8-aced-5ac87eb8185d"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 30 15:15:16.130143 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:15:16.130009 2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0b57ea84-6127-4db8-aced-5ac87eb8185d-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "0b57ea84-6127-4db8-aced-5ac87eb8185d" (UID: "0b57ea84-6127-4db8-aced-5ac87eb8185d"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 30 15:15:16.131651 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:15:16.131626 2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/0b57ea84-6127-4db8-aced-5ac87eb8185d-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "0b57ea84-6127-4db8-aced-5ac87eb8185d" (UID: "0b57ea84-6127-4db8-aced-5ac87eb8185d"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 30 15:15:16.131730 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:15:16.131664 2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/0b57ea84-6127-4db8-aced-5ac87eb8185d-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "0b57ea84-6127-4db8-aced-5ac87eb8185d" (UID: "0b57ea84-6127-4db8-aced-5ac87eb8185d"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 30 15:15:16.131730 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:15:16.131681 2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0b57ea84-6127-4db8-aced-5ac87eb8185d-kube-api-access-5sh6t" (OuterVolumeSpecName: "kube-api-access-5sh6t") pod "0b57ea84-6127-4db8-aced-5ac87eb8185d" (UID: "0b57ea84-6127-4db8-aced-5ac87eb8185d"). InnerVolumeSpecName "kube-api-access-5sh6t". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 30 15:15:16.230769 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:15:16.230731 2576 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/0b57ea84-6127-4db8-aced-5ac87eb8185d-console-oauth-config\") on node \"ip-10-0-133-210.ec2.internal\" DevicePath \"\"" Jul 30 15:15:16.230769 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:15:16.230763 2576 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/0b57ea84-6127-4db8-aced-5ac87eb8185d-oauth-serving-cert\") on node \"ip-10-0-133-210.ec2.internal\" DevicePath \"\"" Jul 30 15:15:16.230769 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:15:16.230772 2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-5sh6t\" (UniqueName: \"kubernetes.io/projected/0b57ea84-6127-4db8-aced-5ac87eb8185d-kube-api-access-5sh6t\") on node \"ip-10-0-133-210.ec2.internal\" DevicePath \"\"" Jul 30 15:15:16.230769 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:15:16.230782 2576 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/0b57ea84-6127-4db8-aced-5ac87eb8185d-console-config\") on node \"ip-10-0-133-210.ec2.internal\" DevicePath \"\"" Jul 30 15:15:16.231035 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:15:16.230791 2576 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/0b57ea84-6127-4db8-aced-5ac87eb8185d-console-serving-cert\") on node \"ip-10-0-133-210.ec2.internal\" DevicePath \"\"" Jul 30 15:15:16.231035 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:15:16.230799 2576 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/0b57ea84-6127-4db8-aced-5ac87eb8185d-service-ca\") on node \"ip-10-0-133-210.ec2.internal\" DevicePath \"\"" Jul 30 15:15:16.231035 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:15:16.230809 2576 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/0b57ea84-6127-4db8-aced-5ac87eb8185d-trusted-ca-bundle\") on node \"ip-10-0-133-210.ec2.internal\" DevicePath \"\"" Jul 30 15:15:16.750199 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:15:16.750164 2576 generic.go:356] "Generic (PLEG): container finished" podID="0b57ea84-6127-4db8-aced-5ac87eb8185d" containerID="f5511121b6f5a6f727e76d38ea5fb1fee76a12b93bba30f410c3707b6914f0eb" exitCode=0 Jul 30 15:15:16.750387 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:15:16.750209 2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-84957b45b9-gfkpb" Jul 30 15:15:16.750387 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:15:16.750252 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-84957b45b9-gfkpb" event={"ID":"0b57ea84-6127-4db8-aced-5ac87eb8185d","Type":"ContainerDied","Data":"f5511121b6f5a6f727e76d38ea5fb1fee76a12b93bba30f410c3707b6914f0eb"} Jul 30 15:15:16.750387 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:15:16.750311 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-84957b45b9-gfkpb" event={"ID":"0b57ea84-6127-4db8-aced-5ac87eb8185d","Type":"ContainerDied","Data":"21de9d059b3bec4d61bba2ec558dbddb7bee4cc5a27ce3dbf73c84111b6dcc2b"} Jul 30 15:15:16.750387 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:15:16.750342 2576 scope.go:117] "RemoveContainer" containerID="f5511121b6f5a6f727e76d38ea5fb1fee76a12b93bba30f410c3707b6914f0eb" Jul 30 15:15:16.758227 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:15:16.758210 2576 scope.go:117] "RemoveContainer" containerID="f5511121b6f5a6f727e76d38ea5fb1fee76a12b93bba30f410c3707b6914f0eb" Jul 30 15:15:16.758463 ip-10-0-133-210 kubenswrapper[2576]: E0730 15:15:16.758446 2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f5511121b6f5a6f727e76d38ea5fb1fee76a12b93bba30f410c3707b6914f0eb\": container with ID starting with f5511121b6f5a6f727e76d38ea5fb1fee76a12b93bba30f410c3707b6914f0eb not found: ID does not exist" containerID="f5511121b6f5a6f727e76d38ea5fb1fee76a12b93bba30f410c3707b6914f0eb" Jul 30 15:15:16.758523 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:15:16.758471 2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f5511121b6f5a6f727e76d38ea5fb1fee76a12b93bba30f410c3707b6914f0eb"} err="failed to get container status \"f5511121b6f5a6f727e76d38ea5fb1fee76a12b93bba30f410c3707b6914f0eb\": rpc error: code = NotFound desc = could not find container \"f5511121b6f5a6f727e76d38ea5fb1fee76a12b93bba30f410c3707b6914f0eb\": container with ID starting with f5511121b6f5a6f727e76d38ea5fb1fee76a12b93bba30f410c3707b6914f0eb not found: ID does not exist" Jul 30 15:15:16.769630 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:15:16.769611 2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-84957b45b9-gfkpb"] Jul 30 15:15:16.773170 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:15:16.773150 2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-84957b45b9-gfkpb"] Jul 30 15:15:18.427704 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:15:18.427676 2576 kubelet.go:1648] "Image garbage collection succeeded" Jul 30 15:15:18.612357 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:15:18.612239 2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="0b57ea84-6127-4db8-aced-5ac87eb8185d" path="/var/lib/kubelet/pods/0b57ea84-6127-4db8-aced-5ac87eb8185d/volumes" Jul 30 15:15:37.368174 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:15:37.368137 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7j468_a2197c57-2dc6-410e-9910-a045dedcb8d9/konnectivity-agent/0.log" limit=1024 Jul 30 15:15:38.477300 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:15:38.477236 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7j468_a2197c57-2dc6-410e-9910-a045dedcb8d9/konnectivity-agent/0.log" limit=1024 Jul 30 15:16:04.426412 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:16:04.426379 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7j468_a2197c57-2dc6-410e-9910-a045dedcb8d9/konnectivity-agent/0.log" limit=1024 Jul 30 15:16:05.834607 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:16:05.834566 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7j468_a2197c57-2dc6-410e-9910-a045dedcb8d9/konnectivity-agent/0.log" limit=1024 Jul 30 15:16:08.226248 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:16:08.226208 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7j468_a2197c57-2dc6-410e-9910-a045dedcb8d9/konnectivity-agent/0.log" limit=1024 Jul 30 15:16:08.295805 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:16:08.295770 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7j468_a2197c57-2dc6-410e-9910-a045dedcb8d9/konnectivity-agent/0.log" limit=1024 Jul 30 15:16:10.159995 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:16:10.159918 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7j468_a2197c57-2dc6-410e-9910-a045dedcb8d9/konnectivity-agent/0.log" limit=1024 Jul 30 15:16:10.265904 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:16:10.265866 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7j468_a2197c57-2dc6-410e-9910-a045dedcb8d9/konnectivity-agent/0.log" limit=1024 Jul 30 15:16:10.716490 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:16:10.716453 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7j468_a2197c57-2dc6-410e-9910-a045dedcb8d9/konnectivity-agent/0.log" limit=1024 Jul 30 15:16:10.859517 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:16:10.859482 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7j468_a2197c57-2dc6-410e-9910-a045dedcb8d9/konnectivity-agent/0.log" limit=1024 Jul 30 15:16:12.375582 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:16:12.375553 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7j468_a2197c57-2dc6-410e-9910-a045dedcb8d9/konnectivity-agent/0.log" limit=1024 Jul 30 15:16:12.570129 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:16:12.570075 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7j468_a2197c57-2dc6-410e-9910-a045dedcb8d9/konnectivity-agent/0.log" limit=1024 Jul 30 15:16:13.447979 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:16:13.447947 2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-cainjector-559cdd8c87-db66h"] Jul 30 15:16:13.448373 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:16:13.448303 2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0b57ea84-6127-4db8-aced-5ac87eb8185d" containerName="console" Jul 30 15:16:13.448373 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:16:13.448316 2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="0b57ea84-6127-4db8-aced-5ac87eb8185d" containerName="console" Jul 30 15:16:13.448446 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:16:13.448377 2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="0b57ea84-6127-4db8-aced-5ac87eb8185d" containerName="console" Jul 30 15:16:13.450063 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:16:13.450046 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-cainjector-559cdd8c87-db66h" Jul 30 15:16:13.453016 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:16:13.452992 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"kube-root-ca.crt\"" Jul 30 15:16:13.453187 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:16:13.453035 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"openshift-service-ca.crt\"" Jul 30 15:16:13.454204 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:16:13.454184 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-cainjector-dockercfg-jl54s\"" Jul 30 15:16:13.464036 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:16:13.464006 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-cainjector-559cdd8c87-db66h"] Jul 30 15:16:13.572864 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:16:13.572806 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6s5tz\" (UniqueName: \"kubernetes.io/projected/72ed145f-cc09-409c-9643-b1692cf58417-kube-api-access-6s5tz\") pod \"cert-manager-cainjector-559cdd8c87-db66h\" (UID: \"72ed145f-cc09-409c-9643-b1692cf58417\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-db66h" Jul 30 15:16:13.573053 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:16:13.572879 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/72ed145f-cc09-409c-9643-b1692cf58417-bound-sa-token\") pod \"cert-manager-cainjector-559cdd8c87-db66h\" (UID: \"72ed145f-cc09-409c-9643-b1692cf58417\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-db66h" Jul 30 15:16:13.673678 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:16:13.673629 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6s5tz\" (UniqueName: \"kubernetes.io/projected/72ed145f-cc09-409c-9643-b1692cf58417-kube-api-access-6s5tz\") pod \"cert-manager-cainjector-559cdd8c87-db66h\" (UID: \"72ed145f-cc09-409c-9643-b1692cf58417\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-db66h" Jul 30 15:16:13.673819 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:16:13.673695 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/72ed145f-cc09-409c-9643-b1692cf58417-bound-sa-token\") pod \"cert-manager-cainjector-559cdd8c87-db66h\" (UID: \"72ed145f-cc09-409c-9643-b1692cf58417\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-db66h" Jul 30 15:16:13.683213 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:16:13.683181 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/72ed145f-cc09-409c-9643-b1692cf58417-bound-sa-token\") pod \"cert-manager-cainjector-559cdd8c87-db66h\" (UID: \"72ed145f-cc09-409c-9643-b1692cf58417\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-db66h" Jul 30 15:16:13.683614 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:16:13.683591 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6s5tz\" (UniqueName: \"kubernetes.io/projected/72ed145f-cc09-409c-9643-b1692cf58417-kube-api-access-6s5tz\") pod \"cert-manager-cainjector-559cdd8c87-db66h\" (UID: \"72ed145f-cc09-409c-9643-b1692cf58417\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-db66h" Jul 30 15:16:13.759305 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:16:13.759257 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-cainjector-559cdd8c87-db66h" Jul 30 15:16:13.886715 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:16:13.886661 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-cainjector-559cdd8c87-db66h"] Jul 30 15:16:13.889723 ip-10-0-133-210 kubenswrapper[2576]: W0730 15:16:13.889697 2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod72ed145f_cc09_409c_9643_b1692cf58417.slice/crio-05b8da69aeef0a78131d4c8c4e27fe42a7044a9abd37264ee2b5e924510399a3 WatchSource:0}: Error finding container 05b8da69aeef0a78131d4c8c4e27fe42a7044a9abd37264ee2b5e924510399a3: Status 404 returned error can't find the container with id 05b8da69aeef0a78131d4c8c4e27fe42a7044a9abd37264ee2b5e924510399a3 Jul 30 15:16:13.891576 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:16:13.891559 2576 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 30 15:16:13.962319 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:16:13.962279 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-cainjector-559cdd8c87-db66h" event={"ID":"72ed145f-cc09-409c-9643-b1692cf58417","Type":"ContainerStarted","Data":"05b8da69aeef0a78131d4c8c4e27fe42a7044a9abd37264ee2b5e924510399a3"} Jul 30 15:16:15.221226 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:16:15.221186 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7j468_a2197c57-2dc6-410e-9910-a045dedcb8d9/konnectivity-agent/0.log" limit=1024 Jul 30 15:16:15.321583 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:16:15.321547 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7j468_a2197c57-2dc6-410e-9910-a045dedcb8d9/konnectivity-agent/0.log" limit=1024 Jul 30 15:16:17.980521 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:16:17.980480 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-cainjector-559cdd8c87-db66h" event={"ID":"72ed145f-cc09-409c-9643-b1692cf58417","Type":"ContainerStarted","Data":"48a60c8ff1473af121c11ff510a8a811881da044a77716472762d22ba7e519da"} Jul 30 15:16:18.000913 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:16:18.000857 2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager/cert-manager-cainjector-559cdd8c87-db66h" podStartSLOduration=1.452823346 podStartE2EDuration="5.000838653s" totalImagesPullingTime="3.548015307s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 15:16:13 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 15:16:17.998397462 +0000 UTC m=+359.924150107" watchObservedRunningTime="2026-07-30 15:16:18.000838653 +0000 UTC m=+359.926591298" Jul 30 15:16:19.448263 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:16:19.448224 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7j468_a2197c57-2dc6-410e-9910-a045dedcb8d9/konnectivity-agent/0.log" limit=1024 Jul 30 15:16:19.554133 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:16:19.554080 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7j468_a2197c57-2dc6-410e-9910-a045dedcb8d9/konnectivity-agent/0.log" limit=1024 Jul 30 15:16:20.176969 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:16:20.176932 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7j468_a2197c57-2dc6-410e-9910-a045dedcb8d9/konnectivity-agent/0.log" limit=1024 Jul 30 15:16:20.325789 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:16:20.325749 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7j468_a2197c57-2dc6-410e-9910-a045dedcb8d9/konnectivity-agent/0.log" limit=1024 Jul 30 15:16:20.445671 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:16:20.445578 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7j468_a2197c57-2dc6-410e-9910-a045dedcb8d9/konnectivity-agent/0.log" limit=1024 Jul 30 15:16:21.460820 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:16:21.460787 2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-lws-operator/openshift-lws-operator-fd8ccff4c-tpzh9"] Jul 30 15:16:21.462872 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:16:21.462856 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-tpzh9" Jul 30 15:16:21.465470 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:16:21.465447 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"kube-root-ca.crt\"" Jul 30 15:16:21.465592 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:16:21.465453 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"openshift-service-ca.crt\"" Jul 30 15:16:21.465592 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:16:21.465557 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"openshift-lws-operator-dockercfg-s86cw\"" Jul 30 15:16:21.472395 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:16:21.472373 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/openshift-lws-operator-fd8ccff4c-tpzh9"] Jul 30 15:16:21.538701 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:16:21.538662 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5jmbr\" (UniqueName: \"kubernetes.io/projected/265f471f-8da2-4164-8dc4-38d8b46a1090-kube-api-access-5jmbr\") pod \"openshift-lws-operator-fd8ccff4c-tpzh9\" (UID: \"265f471f-8da2-4164-8dc4-38d8b46a1090\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-tpzh9" Jul 30 15:16:21.538880 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:16:21.538740 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/265f471f-8da2-4164-8dc4-38d8b46a1090-tmp\") pod \"openshift-lws-operator-fd8ccff4c-tpzh9\" (UID: \"265f471f-8da2-4164-8dc4-38d8b46a1090\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-tpzh9" Jul 30 15:16:21.639806 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:16:21.639766 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5jmbr\" (UniqueName: \"kubernetes.io/projected/265f471f-8da2-4164-8dc4-38d8b46a1090-kube-api-access-5jmbr\") pod \"openshift-lws-operator-fd8ccff4c-tpzh9\" (UID: \"265f471f-8da2-4164-8dc4-38d8b46a1090\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-tpzh9" Jul 30 15:16:21.639969 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:16:21.639829 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/265f471f-8da2-4164-8dc4-38d8b46a1090-tmp\") pod \"openshift-lws-operator-fd8ccff4c-tpzh9\" (UID: \"265f471f-8da2-4164-8dc4-38d8b46a1090\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-tpzh9" Jul 30 15:16:21.640264 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:16:21.640247 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/265f471f-8da2-4164-8dc4-38d8b46a1090-tmp\") pod \"openshift-lws-operator-fd8ccff4c-tpzh9\" (UID: \"265f471f-8da2-4164-8dc4-38d8b46a1090\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-tpzh9" Jul 30 15:16:21.649332 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:16:21.649310 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5jmbr\" (UniqueName: \"kubernetes.io/projected/265f471f-8da2-4164-8dc4-38d8b46a1090-kube-api-access-5jmbr\") pod \"openshift-lws-operator-fd8ccff4c-tpzh9\" (UID: \"265f471f-8da2-4164-8dc4-38d8b46a1090\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-tpzh9" Jul 30 15:16:21.772220 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:16:21.772170 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-tpzh9" Jul 30 15:16:21.905614 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:16:21.905590 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/openshift-lws-operator-fd8ccff4c-tpzh9"] Jul 30 15:16:21.907806 ip-10-0-133-210 kubenswrapper[2576]: W0730 15:16:21.907774 2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod265f471f_8da2_4164_8dc4_38d8b46a1090.slice/crio-32ffe7752e7905a926459715fc6c558d9b2c2bdddfdd62a5f1de5842c2a79d56 WatchSource:0}: Error finding container 32ffe7752e7905a926459715fc6c558d9b2c2bdddfdd62a5f1de5842c2a79d56: Status 404 returned error can't find the container with id 32ffe7752e7905a926459715fc6c558d9b2c2bdddfdd62a5f1de5842c2a79d56 Jul 30 15:16:21.996852 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:16:21.996820 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-tpzh9" event={"ID":"265f471f-8da2-4164-8dc4-38d8b46a1090","Type":"ContainerStarted","Data":"32ffe7752e7905a926459715fc6c558d9b2c2bdddfdd62a5f1de5842c2a79d56"} Jul 30 15:16:25.011313 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:16:25.011279 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-tpzh9" event={"ID":"265f471f-8da2-4164-8dc4-38d8b46a1090","Type":"ContainerStarted","Data":"662d5c63db01a8a8b07c6faee1bd0d32485f8feb7b76ddc447bb3d131876faa5"} Jul 30 15:16:25.027417 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:16:25.027370 2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-tpzh9" podStartSLOduration=1.898677573 podStartE2EDuration="4.027356495s" totalImagesPullingTime="2.128678922s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 15:16:21 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 15:16:25.026910051 +0000 UTC m=+366.952662715" watchObservedRunningTime="2026-07-30 15:16:25.027356495 +0000 UTC m=+366.953109139" Jul 30 15:16:25.732379 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:16:25.732332 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7j468_a2197c57-2dc6-410e-9910-a045dedcb8d9/konnectivity-agent/0.log" limit=1024 Jul 30 15:16:25.831666 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:16:25.831622 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7j468_a2197c57-2dc6-410e-9910-a045dedcb8d9/konnectivity-agent/0.log" limit=1024 Jul 30 15:16:27.846707 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:16:27.846667 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7j468_a2197c57-2dc6-410e-9910-a045dedcb8d9/konnectivity-agent/0.log" limit=1024 Jul 30 15:16:46.090668 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:16:46.090625 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7j468_a2197c57-2dc6-410e-9910-a045dedcb8d9/konnectivity-agent/0.log" limit=1024 Jul 30 15:16:46.195579 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:16:46.195540 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7j468_a2197c57-2dc6-410e-9910-a045dedcb8d9/konnectivity-agent/0.log" limit=1024 Jul 30 15:16:46.707798 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:16:46.707763 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7j468_a2197c57-2dc6-410e-9910-a045dedcb8d9/konnectivity-agent/0.log" limit=1024 Jul 30 15:16:47.317990 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:16:47.317952 2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-56444d5c7b-jdfll"] Jul 30 15:16:47.321503 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:16:47.321476 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-56444d5c7b-jdfll" Jul 30 15:16:47.326991 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:16:47.326964 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\"" Jul 30 15:16:47.328094 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:16:47.328069 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-service-cert\"" Jul 30 15:16:47.328595 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:16:47.328335 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-webhook-cert\"" Jul 30 15:16:47.328595 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:16:47.328471 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\"" Jul 30 15:16:47.328595 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:16:47.328561 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-dockercfg-t7drl\"" Jul 30 15:16:47.362701 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:16:47.362669 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-56444d5c7b-jdfll"] Jul 30 15:16:47.451441 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:16:47.451403 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/2fcf0d71-64a1-4a7d-812b-ef92d1e5c927-webhook-cert\") pod \"opendatahub-operator-controller-manager-56444d5c7b-jdfll\" (UID: \"2fcf0d71-64a1-4a7d-812b-ef92d1e5c927\") " pod="opendatahub/opendatahub-operator-controller-manager-56444d5c7b-jdfll" Jul 30 15:16:47.451441 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:16:47.451438 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/2fcf0d71-64a1-4a7d-812b-ef92d1e5c927-tmp\") pod \"opendatahub-operator-controller-manager-56444d5c7b-jdfll\" (UID: \"2fcf0d71-64a1-4a7d-812b-ef92d1e5c927\") " pod="opendatahub/opendatahub-operator-controller-manager-56444d5c7b-jdfll" Jul 30 15:16:47.451663 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:16:47.451456 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/2fcf0d71-64a1-4a7d-812b-ef92d1e5c927-apiservice-cert\") pod \"opendatahub-operator-controller-manager-56444d5c7b-jdfll\" (UID: \"2fcf0d71-64a1-4a7d-812b-ef92d1e5c927\") " pod="opendatahub/opendatahub-operator-controller-manager-56444d5c7b-jdfll" Jul 30 15:16:47.451663 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:16:47.451476 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/2fcf0d71-64a1-4a7d-812b-ef92d1e5c927-manifests\") pod \"opendatahub-operator-controller-manager-56444d5c7b-jdfll\" (UID: \"2fcf0d71-64a1-4a7d-812b-ef92d1e5c927\") " pod="opendatahub/opendatahub-operator-controller-manager-56444d5c7b-jdfll" Jul 30 15:16:47.451663 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:16:47.451603 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4nlfh\" (UniqueName: \"kubernetes.io/projected/2fcf0d71-64a1-4a7d-812b-ef92d1e5c927-kube-api-access-4nlfh\") pod \"opendatahub-operator-controller-manager-56444d5c7b-jdfll\" (UID: \"2fcf0d71-64a1-4a7d-812b-ef92d1e5c927\") " pod="opendatahub/opendatahub-operator-controller-manager-56444d5c7b-jdfll" Jul 30 15:16:47.552723 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:16:47.552683 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4nlfh\" (UniqueName: \"kubernetes.io/projected/2fcf0d71-64a1-4a7d-812b-ef92d1e5c927-kube-api-access-4nlfh\") pod \"opendatahub-operator-controller-manager-56444d5c7b-jdfll\" (UID: \"2fcf0d71-64a1-4a7d-812b-ef92d1e5c927\") " pod="opendatahub/opendatahub-operator-controller-manager-56444d5c7b-jdfll" Jul 30 15:16:47.552895 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:16:47.552752 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/2fcf0d71-64a1-4a7d-812b-ef92d1e5c927-webhook-cert\") pod \"opendatahub-operator-controller-manager-56444d5c7b-jdfll\" (UID: \"2fcf0d71-64a1-4a7d-812b-ef92d1e5c927\") " pod="opendatahub/opendatahub-operator-controller-manager-56444d5c7b-jdfll" Jul 30 15:16:47.552895 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:16:47.552771 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/2fcf0d71-64a1-4a7d-812b-ef92d1e5c927-tmp\") pod \"opendatahub-operator-controller-manager-56444d5c7b-jdfll\" (UID: \"2fcf0d71-64a1-4a7d-812b-ef92d1e5c927\") " pod="opendatahub/opendatahub-operator-controller-manager-56444d5c7b-jdfll" Jul 30 15:16:47.552895 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:16:47.552789 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/2fcf0d71-64a1-4a7d-812b-ef92d1e5c927-apiservice-cert\") pod \"opendatahub-operator-controller-manager-56444d5c7b-jdfll\" (UID: \"2fcf0d71-64a1-4a7d-812b-ef92d1e5c927\") " pod="opendatahub/opendatahub-operator-controller-manager-56444d5c7b-jdfll" Jul 30 15:16:47.552895 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:16:47.552809 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/2fcf0d71-64a1-4a7d-812b-ef92d1e5c927-manifests\") pod \"opendatahub-operator-controller-manager-56444d5c7b-jdfll\" (UID: \"2fcf0d71-64a1-4a7d-812b-ef92d1e5c927\") " pod="opendatahub/opendatahub-operator-controller-manager-56444d5c7b-jdfll" Jul 30 15:16:47.553235 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:16:47.553214 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/2fcf0d71-64a1-4a7d-812b-ef92d1e5c927-manifests\") pod \"opendatahub-operator-controller-manager-56444d5c7b-jdfll\" (UID: \"2fcf0d71-64a1-4a7d-812b-ef92d1e5c927\") " pod="opendatahub/opendatahub-operator-controller-manager-56444d5c7b-jdfll" Jul 30 15:16:47.555283 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:16:47.555257 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/2fcf0d71-64a1-4a7d-812b-ef92d1e5c927-tmp\") pod \"opendatahub-operator-controller-manager-56444d5c7b-jdfll\" (UID: \"2fcf0d71-64a1-4a7d-812b-ef92d1e5c927\") " pod="opendatahub/opendatahub-operator-controller-manager-56444d5c7b-jdfll" Jul 30 15:16:47.555369 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:16:47.555307 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/2fcf0d71-64a1-4a7d-812b-ef92d1e5c927-apiservice-cert\") pod \"opendatahub-operator-controller-manager-56444d5c7b-jdfll\" (UID: \"2fcf0d71-64a1-4a7d-812b-ef92d1e5c927\") " pod="opendatahub/opendatahub-operator-controller-manager-56444d5c7b-jdfll" Jul 30 15:16:47.555441 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:16:47.555425 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/2fcf0d71-64a1-4a7d-812b-ef92d1e5c927-webhook-cert\") pod \"opendatahub-operator-controller-manager-56444d5c7b-jdfll\" (UID: \"2fcf0d71-64a1-4a7d-812b-ef92d1e5c927\") " pod="opendatahub/opendatahub-operator-controller-manager-56444d5c7b-jdfll" Jul 30 15:16:47.565577 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:16:47.565547 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4nlfh\" (UniqueName: \"kubernetes.io/projected/2fcf0d71-64a1-4a7d-812b-ef92d1e5c927-kube-api-access-4nlfh\") pod \"opendatahub-operator-controller-manager-56444d5c7b-jdfll\" (UID: \"2fcf0d71-64a1-4a7d-812b-ef92d1e5c927\") " pod="opendatahub/opendatahub-operator-controller-manager-56444d5c7b-jdfll" Jul 30 15:16:47.631920 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:16:47.631830 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-56444d5c7b-jdfll" Jul 30 15:16:47.784490 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:16:47.784466 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-56444d5c7b-jdfll"] Jul 30 15:16:47.786534 ip-10-0-133-210 kubenswrapper[2576]: W0730 15:16:47.786502 2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2fcf0d71_64a1_4a7d_812b_ef92d1e5c927.slice/crio-2c0a8a9ad1778d221351c59b2473726e1e8eac52e0c4d6b84382b0ead09b0d84 WatchSource:0}: Error finding container 2c0a8a9ad1778d221351c59b2473726e1e8eac52e0c4d6b84382b0ead09b0d84: Status 404 returned error can't find the container with id 2c0a8a9ad1778d221351c59b2473726e1e8eac52e0c4d6b84382b0ead09b0d84 Jul 30 15:16:48.103602 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:16:48.103564 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-56444d5c7b-jdfll" event={"ID":"2fcf0d71-64a1-4a7d-812b-ef92d1e5c927","Type":"ContainerStarted","Data":"2c0a8a9ad1778d221351c59b2473726e1e8eac52e0c4d6b84382b0ead09b0d84"} Jul 30 15:16:48.406891 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:16:48.406795 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7j468_a2197c57-2dc6-410e-9910-a045dedcb8d9/konnectivity-agent/0.log" limit=1024 Jul 30 15:16:48.512128 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:16:48.512079 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7j468_a2197c57-2dc6-410e-9910-a045dedcb8d9/konnectivity-agent/0.log" limit=1024 Jul 30 15:16:52.123205 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:16:52.123166 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-56444d5c7b-jdfll" event={"ID":"2fcf0d71-64a1-4a7d-812b-ef92d1e5c927","Type":"ContainerStarted","Data":"20796cda487b6a375f29d216f660d908b01548edaa8a2d0500a86771fb424d84"} Jul 30 15:16:52.329691 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:16:52.329657 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7j468_a2197c57-2dc6-410e-9910-a045dedcb8d9/konnectivity-agent/0.log" limit=1024 Jul 30 15:16:52.424526 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:16:52.424428 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7j468_a2197c57-2dc6-410e-9910-a045dedcb8d9/konnectivity-agent/0.log" limit=1024 Jul 30 15:16:53.129161 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:16:53.129100 2576 generic.go:356] "Generic (PLEG): container finished" podID="2fcf0d71-64a1-4a7d-812b-ef92d1e5c927" containerID="20796cda487b6a375f29d216f660d908b01548edaa8a2d0500a86771fb424d84" exitCode=0 Jul 30 15:16:53.129534 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:16:53.129181 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-56444d5c7b-jdfll" event={"ID":"2fcf0d71-64a1-4a7d-812b-ef92d1e5c927","Type":"ContainerDied","Data":"20796cda487b6a375f29d216f660d908b01548edaa8a2d0500a86771fb424d84"} Jul 30 15:16:54.134343 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:16:54.134310 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-56444d5c7b-jdfll" event={"ID":"2fcf0d71-64a1-4a7d-812b-ef92d1e5c927","Type":"ContainerStarted","Data":"d1cdf0b8122e6c20816e8d62e78b1b563375a0c0727425e8f90dd8634e01d7aa"} Jul 30 15:16:54.134760 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:16:54.134449 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/opendatahub-operator-controller-manager-56444d5c7b-jdfll" Jul 30 15:17:05.141669 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:17:05.141638 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/opendatahub-operator-controller-manager-56444d5c7b-jdfll" Jul 30 15:17:05.166414 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:17:05.166351 2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/opendatahub-operator-controller-manager-56444d5c7b-jdfll" podStartSLOduration=13.402048127 podStartE2EDuration="18.166332948s" totalImagesPullingTime="3.690332229s" totalInitContainerRuntime="1.073952592s" isStatefulPod=false podCreationTimestamp="2026-07-30 15:16:47 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 15:16:54.167424018 +0000 UTC m=+396.093176663" watchObservedRunningTime="2026-07-30 15:17:05.166332948 +0000 UTC m=+407.092085593" Jul 30 15:17:07.876952 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:17:07.876917 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7j468_a2197c57-2dc6-410e-9910-a045dedcb8d9/konnectivity-agent/0.log" limit=1024 Jul 30 15:17:08.100406 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:17:08.100365 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7j468_a2197c57-2dc6-410e-9910-a045dedcb8d9/konnectivity-agent/0.log" limit=1024 Jul 30 15:17:08.623123 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:17:08.623076 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7j468_a2197c57-2dc6-410e-9910-a045dedcb8d9/konnectivity-agent/0.log" limit=1024 Jul 30 15:17:08.795726 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:17:08.795685 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7j468_a2197c57-2dc6-410e-9910-a045dedcb8d9/konnectivity-agent/0.log" limit=1024 Jul 30 15:17:08.931261 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:17:08.931170 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7j468_a2197c57-2dc6-410e-9910-a045dedcb8d9/konnectivity-agent/0.log" limit=1024 Jul 30 15:17:09.059155 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:17:09.059126 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7j468_a2197c57-2dc6-410e-9910-a045dedcb8d9/konnectivity-agent/0.log" limit=1024 Jul 30 15:17:09.445726 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:17:09.445691 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7j468_a2197c57-2dc6-410e-9910-a045dedcb8d9/konnectivity-agent/0.log" limit=1024 Jul 30 15:17:09.843730 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:17:09.843691 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7j468_a2197c57-2dc6-410e-9910-a045dedcb8d9/konnectivity-agent/0.log" limit=1024 Jul 30 15:17:10.034878 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:17:10.034847 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7j468_a2197c57-2dc6-410e-9910-a045dedcb8d9/konnectivity-agent/0.log" limit=1024 Jul 30 15:17:10.136913 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:17:10.136831 2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/kube-auth-proxy-6976f6d894-6ckvp"] Jul 30 15:17:10.147153 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:17:10.147122 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-6976f6d894-6ckvp" Jul 30 15:17:10.147397 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:17:10.147369 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/09e1e2ea-ba7d-45e6-94eb-4ef6726590d3-tmp\") pod \"kube-auth-proxy-6976f6d894-6ckvp\" (UID: \"09e1e2ea-ba7d-45e6-94eb-4ef6726590d3\") " pod="openshift-ingress/kube-auth-proxy-6976f6d894-6ckvp" Jul 30 15:17:10.147645 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:17:10.147574 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2d9wx\" (UniqueName: \"kubernetes.io/projected/09e1e2ea-ba7d-45e6-94eb-4ef6726590d3-kube-api-access-2d9wx\") pod \"kube-auth-proxy-6976f6d894-6ckvp\" (UID: \"09e1e2ea-ba7d-45e6-94eb-4ef6726590d3\") " pod="openshift-ingress/kube-auth-proxy-6976f6d894-6ckvp" Jul 30 15:17:10.147645 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:17:10.147614 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/09e1e2ea-ba7d-45e6-94eb-4ef6726590d3-tls-certs\") pod \"kube-auth-proxy-6976f6d894-6ckvp\" (UID: \"09e1e2ea-ba7d-45e6-94eb-4ef6726590d3\") " pod="openshift-ingress/kube-auth-proxy-6976f6d894-6ckvp" Jul 30 15:17:10.150647 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:17:10.150608 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-creds\"" Jul 30 15:17:10.150945 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:17:10.150926 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-dockercfg-mkqrq\"" Jul 30 15:17:10.151301 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:17:10.151284 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\"" Jul 30 15:17:10.151550 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:17:10.151533 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-6976f6d894-6ckvp"] Jul 30 15:17:10.151673 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:17:10.151656 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-tls\"" Jul 30 15:17:10.151735 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:17:10.151684 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\"" Jul 30 15:17:10.248211 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:17:10.248170 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/09e1e2ea-ba7d-45e6-94eb-4ef6726590d3-tmp\") pod \"kube-auth-proxy-6976f6d894-6ckvp\" (UID: \"09e1e2ea-ba7d-45e6-94eb-4ef6726590d3\") " pod="openshift-ingress/kube-auth-proxy-6976f6d894-6ckvp" Jul 30 15:17:10.248404 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:17:10.248217 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2d9wx\" (UniqueName: \"kubernetes.io/projected/09e1e2ea-ba7d-45e6-94eb-4ef6726590d3-kube-api-access-2d9wx\") pod \"kube-auth-proxy-6976f6d894-6ckvp\" (UID: \"09e1e2ea-ba7d-45e6-94eb-4ef6726590d3\") " pod="openshift-ingress/kube-auth-proxy-6976f6d894-6ckvp" Jul 30 15:17:10.248404 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:17:10.248258 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/09e1e2ea-ba7d-45e6-94eb-4ef6726590d3-tls-certs\") pod \"kube-auth-proxy-6976f6d894-6ckvp\" (UID: \"09e1e2ea-ba7d-45e6-94eb-4ef6726590d3\") " pod="openshift-ingress/kube-auth-proxy-6976f6d894-6ckvp" Jul 30 15:17:10.248404 ip-10-0-133-210 kubenswrapper[2576]: E0730 15:17:10.248366 2576 secret.go:189] Couldn't get secret openshift-ingress/kube-auth-proxy-tls: secret "kube-auth-proxy-tls" not found Jul 30 15:17:10.248568 ip-10-0-133-210 kubenswrapper[2576]: E0730 15:17:10.248425 2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/09e1e2ea-ba7d-45e6-94eb-4ef6726590d3-tls-certs podName:09e1e2ea-ba7d-45e6-94eb-4ef6726590d3 nodeName:}" failed. No retries permitted until 2026-07-30 15:17:10.748406655 +0000 UTC m=+412.674159292 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "tls-certs" (UniqueName: "kubernetes.io/secret/09e1e2ea-ba7d-45e6-94eb-4ef6726590d3-tls-certs") pod "kube-auth-proxy-6976f6d894-6ckvp" (UID: "09e1e2ea-ba7d-45e6-94eb-4ef6726590d3") : secret "kube-auth-proxy-tls" not found Jul 30 15:17:10.250564 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:17:10.250542 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/09e1e2ea-ba7d-45e6-94eb-4ef6726590d3-tmp\") pod \"kube-auth-proxy-6976f6d894-6ckvp\" (UID: \"09e1e2ea-ba7d-45e6-94eb-4ef6726590d3\") " pod="openshift-ingress/kube-auth-proxy-6976f6d894-6ckvp" Jul 30 15:17:10.260135 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:17:10.260082 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2d9wx\" (UniqueName: \"kubernetes.io/projected/09e1e2ea-ba7d-45e6-94eb-4ef6726590d3-kube-api-access-2d9wx\") pod \"kube-auth-proxy-6976f6d894-6ckvp\" (UID: \"09e1e2ea-ba7d-45e6-94eb-4ef6726590d3\") " pod="openshift-ingress/kube-auth-proxy-6976f6d894-6ckvp" Jul 30 15:17:10.324902 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:17:10.324858 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7j468_a2197c57-2dc6-410e-9910-a045dedcb8d9/konnectivity-agent/0.log" limit=1024 Jul 30 15:17:10.750865 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:17:10.750823 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/09e1e2ea-ba7d-45e6-94eb-4ef6726590d3-tls-certs\") pod \"kube-auth-proxy-6976f6d894-6ckvp\" (UID: \"09e1e2ea-ba7d-45e6-94eb-4ef6726590d3\") " pod="openshift-ingress/kube-auth-proxy-6976f6d894-6ckvp" Jul 30 15:17:10.753246 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:17:10.753226 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/09e1e2ea-ba7d-45e6-94eb-4ef6726590d3-tls-certs\") pod \"kube-auth-proxy-6976f6d894-6ckvp\" (UID: \"09e1e2ea-ba7d-45e6-94eb-4ef6726590d3\") " pod="openshift-ingress/kube-auth-proxy-6976f6d894-6ckvp" Jul 30 15:17:10.758102 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:17:10.758080 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-6976f6d894-6ckvp" Jul 30 15:17:10.902307 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:17:10.902247 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-6976f6d894-6ckvp"] Jul 30 15:17:10.904506 ip-10-0-133-210 kubenswrapper[2576]: W0730 15:17:10.904475 2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod09e1e2ea_ba7d_45e6_94eb_4ef6726590d3.slice/crio-4a75ce4bea6a87fc1b1c936c3d5d29be0059d20bbbabcea0dfa54481b35f7cc5 WatchSource:0}: Error finding container 4a75ce4bea6a87fc1b1c936c3d5d29be0059d20bbbabcea0dfa54481b35f7cc5: Status 404 returned error can't find the container with id 4a75ce4bea6a87fc1b1c936c3d5d29be0059d20bbbabcea0dfa54481b35f7cc5 Jul 30 15:17:11.209258 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:17:11.209168 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-6976f6d894-6ckvp" event={"ID":"09e1e2ea-ba7d-45e6-94eb-4ef6726590d3","Type":"ContainerStarted","Data":"4a75ce4bea6a87fc1b1c936c3d5d29be0059d20bbbabcea0dfa54481b35f7cc5"} Jul 30 15:17:11.403306 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:17:11.403276 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7j468_a2197c57-2dc6-410e-9910-a045dedcb8d9/konnectivity-agent/0.log" limit=1024 Jul 30 15:17:11.536570 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:17:11.536530 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7j468_a2197c57-2dc6-410e-9910-a045dedcb8d9/konnectivity-agent/0.log" limit=1024 Jul 30 15:17:12.614302 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:17:12.614272 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7j468_a2197c57-2dc6-410e-9910-a045dedcb8d9/konnectivity-agent/0.log" limit=1024 Jul 30 15:17:14.224014 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:17:14.223980 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-6976f6d894-6ckvp" event={"ID":"09e1e2ea-ba7d-45e6-94eb-4ef6726590d3","Type":"ContainerStarted","Data":"e3b8d8ccf3876d0b5d6e980a124e62b4f1284dc7ffcaa34e39f74610f9b63c23"} Jul 30 15:17:14.248742 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:17:14.248680 2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/kube-auth-proxy-6976f6d894-6ckvp" podStartSLOduration=1.270561096 podStartE2EDuration="4.248659324s" totalImagesPullingTime="2.978098228s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 15:17:10 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 15:17:14.248300025 +0000 UTC m=+416.174052698" watchObservedRunningTime="2026-07-30 15:17:14.248659324 +0000 UTC m=+416.174411971" Jul 30 15:17:15.624152 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:17:15.624092 2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/istiod-openshift-gateway-5677ccb49b-htlrm"] Jul 30 15:17:15.627230 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:17:15.627206 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-htlrm" Jul 30 15:17:15.631366 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:17:15.631347 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-tls\"" Jul 30 15:17:15.632017 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:17:15.632000 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"cacerts\"" Jul 30 15:17:15.632184 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:17:15.632163 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istio-kubeconfig\"" Jul 30 15:17:15.633763 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:17:15.633499 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-openshift-gateway-dockercfg-45ptg\"" Jul 30 15:17:15.633763 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:17:15.633590 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\"" Jul 30 15:17:15.654628 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:17:15.654591 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-5677ccb49b-htlrm"] Jul 30 15:17:15.684180 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:17:15.684136 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/291e793a-4cbe-4576-a16f-eef021410b23-cacerts\") pod \"istiod-openshift-gateway-5677ccb49b-htlrm\" (UID: \"291e793a-4cbe-4576-a16f-eef021410b23\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-htlrm" Jul 30 15:17:15.684354 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:17:15.684190 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/291e793a-4cbe-4576-a16f-eef021410b23-istio-token\") pod \"istiod-openshift-gateway-5677ccb49b-htlrm\" (UID: \"291e793a-4cbe-4576-a16f-eef021410b23\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-htlrm" Jul 30 15:17:15.684354 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:17:15.684212 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/291e793a-4cbe-4576-a16f-eef021410b23-local-certs\") pod \"istiod-openshift-gateway-5677ccb49b-htlrm\" (UID: \"291e793a-4cbe-4576-a16f-eef021410b23\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-htlrm" Jul 30 15:17:15.684354 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:17:15.684262 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lczw8\" (UniqueName: \"kubernetes.io/projected/291e793a-4cbe-4576-a16f-eef021410b23-kube-api-access-lczw8\") pod \"istiod-openshift-gateway-5677ccb49b-htlrm\" (UID: \"291e793a-4cbe-4576-a16f-eef021410b23\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-htlrm" Jul 30 15:17:15.684506 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:17:15.684365 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/291e793a-4cbe-4576-a16f-eef021410b23-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-5677ccb49b-htlrm\" (UID: \"291e793a-4cbe-4576-a16f-eef021410b23\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-htlrm" Jul 30 15:17:15.684506 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:17:15.684411 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/291e793a-4cbe-4576-a16f-eef021410b23-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-5677ccb49b-htlrm\" (UID: \"291e793a-4cbe-4576-a16f-eef021410b23\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-htlrm" Jul 30 15:17:15.684506 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:17:15.684492 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/291e793a-4cbe-4576-a16f-eef021410b23-istio-kubeconfig\") pod \"istiod-openshift-gateway-5677ccb49b-htlrm\" (UID: \"291e793a-4cbe-4576-a16f-eef021410b23\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-htlrm" Jul 30 15:17:15.785290 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:17:15.785254 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/291e793a-4cbe-4576-a16f-eef021410b23-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-5677ccb49b-htlrm\" (UID: \"291e793a-4cbe-4576-a16f-eef021410b23\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-htlrm" Jul 30 15:17:15.785290 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:17:15.785293 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/291e793a-4cbe-4576-a16f-eef021410b23-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-5677ccb49b-htlrm\" (UID: \"291e793a-4cbe-4576-a16f-eef021410b23\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-htlrm" Jul 30 15:17:15.785526 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:17:15.785313 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/291e793a-4cbe-4576-a16f-eef021410b23-istio-kubeconfig\") pod \"istiod-openshift-gateway-5677ccb49b-htlrm\" (UID: \"291e793a-4cbe-4576-a16f-eef021410b23\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-htlrm" Jul 30 15:17:15.785526 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:17:15.785335 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/291e793a-4cbe-4576-a16f-eef021410b23-cacerts\") pod \"istiod-openshift-gateway-5677ccb49b-htlrm\" (UID: \"291e793a-4cbe-4576-a16f-eef021410b23\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-htlrm" Jul 30 15:17:15.785526 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:17:15.785356 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/291e793a-4cbe-4576-a16f-eef021410b23-istio-token\") pod \"istiod-openshift-gateway-5677ccb49b-htlrm\" (UID: \"291e793a-4cbe-4576-a16f-eef021410b23\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-htlrm" Jul 30 15:17:15.785526 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:17:15.785373 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/291e793a-4cbe-4576-a16f-eef021410b23-local-certs\") pod \"istiod-openshift-gateway-5677ccb49b-htlrm\" (UID: \"291e793a-4cbe-4576-a16f-eef021410b23\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-htlrm" Jul 30 15:17:15.785526 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:17:15.785393 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lczw8\" (UniqueName: \"kubernetes.io/projected/291e793a-4cbe-4576-a16f-eef021410b23-kube-api-access-lczw8\") pod \"istiod-openshift-gateway-5677ccb49b-htlrm\" (UID: \"291e793a-4cbe-4576-a16f-eef021410b23\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-htlrm" Jul 30 15:17:15.786090 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:17:15.786060 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/291e793a-4cbe-4576-a16f-eef021410b23-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-5677ccb49b-htlrm\" (UID: \"291e793a-4cbe-4576-a16f-eef021410b23\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-htlrm" Jul 30 15:17:15.787661 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:17:15.787638 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/291e793a-4cbe-4576-a16f-eef021410b23-local-certs\") pod \"istiod-openshift-gateway-5677ccb49b-htlrm\" (UID: \"291e793a-4cbe-4576-a16f-eef021410b23\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-htlrm" Jul 30 15:17:15.787788 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:17:15.787761 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/291e793a-4cbe-4576-a16f-eef021410b23-istio-kubeconfig\") pod \"istiod-openshift-gateway-5677ccb49b-htlrm\" (UID: \"291e793a-4cbe-4576-a16f-eef021410b23\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-htlrm" Jul 30 15:17:15.788010 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:17:15.787992 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/291e793a-4cbe-4576-a16f-eef021410b23-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-5677ccb49b-htlrm\" (UID: \"291e793a-4cbe-4576-a16f-eef021410b23\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-htlrm" Jul 30 15:17:15.788151 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:17:15.788100 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/291e793a-4cbe-4576-a16f-eef021410b23-cacerts\") pod \"istiod-openshift-gateway-5677ccb49b-htlrm\" (UID: \"291e793a-4cbe-4576-a16f-eef021410b23\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-htlrm" Jul 30 15:17:15.800303 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:17:15.800283 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/291e793a-4cbe-4576-a16f-eef021410b23-istio-token\") pod \"istiod-openshift-gateway-5677ccb49b-htlrm\" (UID: \"291e793a-4cbe-4576-a16f-eef021410b23\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-htlrm" Jul 30 15:17:15.805538 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:17:15.805515 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lczw8\" (UniqueName: \"kubernetes.io/projected/291e793a-4cbe-4576-a16f-eef021410b23-kube-api-access-lczw8\") pod \"istiod-openshift-gateway-5677ccb49b-htlrm\" (UID: \"291e793a-4cbe-4576-a16f-eef021410b23\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-htlrm" Jul 30 15:17:15.936172 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:17:15.936089 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-htlrm" Jul 30 15:17:16.081755 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:17:16.081644 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-5677ccb49b-htlrm"] Jul 30 15:17:16.083905 ip-10-0-133-210 kubenswrapper[2576]: W0730 15:17:16.083871 2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod291e793a_4cbe_4576_a16f_eef021410b23.slice/crio-d31ec6225674c911924450bb9cbfee4a0d439d4b99b471e3b658ca590c1c3666 WatchSource:0}: Error finding container d31ec6225674c911924450bb9cbfee4a0d439d4b99b471e3b658ca590c1c3666: Status 404 returned error can't find the container with id d31ec6225674c911924450bb9cbfee4a0d439d4b99b471e3b658ca590c1c3666 Jul 30 15:17:16.234158 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:17:16.234123 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-htlrm" event={"ID":"291e793a-4cbe-4576-a16f-eef021410b23","Type":"ContainerStarted","Data":"d31ec6225674c911924450bb9cbfee4a0d439d4b99b471e3b658ca590c1c3666"} Jul 30 15:17:16.935401 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:17:16.935367 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7j468_a2197c57-2dc6-410e-9910-a045dedcb8d9/konnectivity-agent/0.log" limit=1024 Jul 30 15:17:17.082608 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:17:17.082572 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7j468_a2197c57-2dc6-410e-9910-a045dedcb8d9/konnectivity-agent/0.log" limit=1024 Jul 30 15:17:17.463503 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:17:17.463458 2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-lws-operator/lws-controller-manager-556f467f75-v29c7"] Jul 30 15:17:17.468066 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:17:17.468038 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-556f467f75-v29c7" Jul 30 15:17:17.474399 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:17:17.474373 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"metrics-server-cert\"" Jul 30 15:17:17.474644 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:17:17.474374 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"lws-manager-config\"" Jul 30 15:17:17.474743 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:17:17.474417 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"lws-controller-manager-dockercfg-vwb5q\"" Jul 30 15:17:17.474826 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:17:17.474807 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"webhook-server-cert\"" Jul 30 15:17:17.501852 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:17:17.501825 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/fb4933d3-a2b3-49b0-bcdd-70b6fbbdfea0-manager-config\") pod \"lws-controller-manager-556f467f75-v29c7\" (UID: \"fb4933d3-a2b3-49b0-bcdd-70b6fbbdfea0\") " pod="openshift-lws-operator/lws-controller-manager-556f467f75-v29c7" Jul 30 15:17:17.502008 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:17:17.501883 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-d9ft2\" (UniqueName: \"kubernetes.io/projected/fb4933d3-a2b3-49b0-bcdd-70b6fbbdfea0-kube-api-access-d9ft2\") pod \"lws-controller-manager-556f467f75-v29c7\" (UID: \"fb4933d3-a2b3-49b0-bcdd-70b6fbbdfea0\") " pod="openshift-lws-operator/lws-controller-manager-556f467f75-v29c7" Jul 30 15:17:17.502008 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:17:17.501969 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/fb4933d3-a2b3-49b0-bcdd-70b6fbbdfea0-cert\") pod \"lws-controller-manager-556f467f75-v29c7\" (UID: \"fb4933d3-a2b3-49b0-bcdd-70b6fbbdfea0\") " pod="openshift-lws-operator/lws-controller-manager-556f467f75-v29c7" Jul 30 15:17:17.502134 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:17:17.502007 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/fb4933d3-a2b3-49b0-bcdd-70b6fbbdfea0-metrics-cert\") pod \"lws-controller-manager-556f467f75-v29c7\" (UID: \"fb4933d3-a2b3-49b0-bcdd-70b6fbbdfea0\") " pod="openshift-lws-operator/lws-controller-manager-556f467f75-v29c7" Jul 30 15:17:17.508068 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:17:17.508042 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-556f467f75-v29c7"] Jul 30 15:17:17.569520 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:17:17.569488 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7j468_a2197c57-2dc6-410e-9910-a045dedcb8d9/konnectivity-agent/0.log" limit=1024 Jul 30 15:17:17.602764 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:17:17.602739 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/fb4933d3-a2b3-49b0-bcdd-70b6fbbdfea0-cert\") pod \"lws-controller-manager-556f467f75-v29c7\" (UID: \"fb4933d3-a2b3-49b0-bcdd-70b6fbbdfea0\") " pod="openshift-lws-operator/lws-controller-manager-556f467f75-v29c7" Jul 30 15:17:17.602884 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:17:17.602777 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/fb4933d3-a2b3-49b0-bcdd-70b6fbbdfea0-metrics-cert\") pod \"lws-controller-manager-556f467f75-v29c7\" (UID: \"fb4933d3-a2b3-49b0-bcdd-70b6fbbdfea0\") " pod="openshift-lws-operator/lws-controller-manager-556f467f75-v29c7" Jul 30 15:17:17.602884 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:17:17.602812 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/fb4933d3-a2b3-49b0-bcdd-70b6fbbdfea0-manager-config\") pod \"lws-controller-manager-556f467f75-v29c7\" (UID: \"fb4933d3-a2b3-49b0-bcdd-70b6fbbdfea0\") " pod="openshift-lws-operator/lws-controller-manager-556f467f75-v29c7" Jul 30 15:17:17.602884 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:17:17.602859 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-d9ft2\" (UniqueName: \"kubernetes.io/projected/fb4933d3-a2b3-49b0-bcdd-70b6fbbdfea0-kube-api-access-d9ft2\") pod \"lws-controller-manager-556f467f75-v29c7\" (UID: \"fb4933d3-a2b3-49b0-bcdd-70b6fbbdfea0\") " pod="openshift-lws-operator/lws-controller-manager-556f467f75-v29c7" Jul 30 15:17:17.603591 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:17:17.603564 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/fb4933d3-a2b3-49b0-bcdd-70b6fbbdfea0-manager-config\") pod \"lws-controller-manager-556f467f75-v29c7\" (UID: \"fb4933d3-a2b3-49b0-bcdd-70b6fbbdfea0\") " pod="openshift-lws-operator/lws-controller-manager-556f467f75-v29c7" Jul 30 15:17:17.605365 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:17:17.605331 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/fb4933d3-a2b3-49b0-bcdd-70b6fbbdfea0-metrics-cert\") pod \"lws-controller-manager-556f467f75-v29c7\" (UID: \"fb4933d3-a2b3-49b0-bcdd-70b6fbbdfea0\") " pod="openshift-lws-operator/lws-controller-manager-556f467f75-v29c7" Jul 30 15:17:17.605456 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:17:17.605374 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/fb4933d3-a2b3-49b0-bcdd-70b6fbbdfea0-cert\") pod \"lws-controller-manager-556f467f75-v29c7\" (UID: \"fb4933d3-a2b3-49b0-bcdd-70b6fbbdfea0\") " pod="openshift-lws-operator/lws-controller-manager-556f467f75-v29c7" Jul 30 15:17:17.618456 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:17:17.618430 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-d9ft2\" (UniqueName: \"kubernetes.io/projected/fb4933d3-a2b3-49b0-bcdd-70b6fbbdfea0-kube-api-access-d9ft2\") pod \"lws-controller-manager-556f467f75-v29c7\" (UID: \"fb4933d3-a2b3-49b0-bcdd-70b6fbbdfea0\") " pod="openshift-lws-operator/lws-controller-manager-556f467f75-v29c7" Jul 30 15:17:17.763598 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:17:17.763507 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7j468_a2197c57-2dc6-410e-9910-a045dedcb8d9/konnectivity-agent/0.log" limit=1024 Jul 30 15:17:17.779426 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:17:17.779401 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-556f467f75-v29c7" Jul 30 15:17:17.928800 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:17:17.928777 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-556f467f75-v29c7"] Jul 30 15:17:17.931390 ip-10-0-133-210 kubenswrapper[2576]: W0730 15:17:17.931361 2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfb4933d3_a2b3_49b0_bcdd_70b6fbbdfea0.slice/crio-e97fa133463ccbc4d772ee5663dfe5d0d790359c1dd3264c0456bafa561b4171 WatchSource:0}: Error finding container e97fa133463ccbc4d772ee5663dfe5d0d790359c1dd3264c0456bafa561b4171: Status 404 returned error can't find the container with id e97fa133463ccbc4d772ee5663dfe5d0d790359c1dd3264c0456bafa561b4171 Jul 30 15:17:17.947977 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:17:17.947950 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7j468_a2197c57-2dc6-410e-9910-a045dedcb8d9/konnectivity-agent/0.log" limit=1024 Jul 30 15:17:18.245174 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:17:18.245135 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-556f467f75-v29c7" event={"ID":"fb4933d3-a2b3-49b0-bcdd-70b6fbbdfea0","Type":"ContainerStarted","Data":"e97fa133463ccbc4d772ee5663dfe5d0d790359c1dd3264c0456bafa561b4171"} Jul 30 15:17:19.037191 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:17:19.037151 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7j468_a2197c57-2dc6-410e-9910-a045dedcb8d9/konnectivity-agent/0.log" limit=1024 Jul 30 15:17:19.221311 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:17:19.221217 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7j468_a2197c57-2dc6-410e-9910-a045dedcb8d9/konnectivity-agent/0.log" limit=1024 Jul 30 15:17:19.350663 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:17:19.350572 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7j468_a2197c57-2dc6-410e-9910-a045dedcb8d9/konnectivity-agent/0.log" limit=1024 Jul 30 15:17:19.461688 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:17:19.461641 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7j468_a2197c57-2dc6-410e-9910-a045dedcb8d9/konnectivity-agent/0.log" limit=1024 Jul 30 15:17:19.598163 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:17:19.598087 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7j468_a2197c57-2dc6-410e-9910-a045dedcb8d9/konnectivity-agent/0.log" limit=1024 Jul 30 15:17:19.728010 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:17:19.727968 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7j468_a2197c57-2dc6-410e-9910-a045dedcb8d9/konnectivity-agent/0.log" limit=1024 Jul 30 15:17:20.334305 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:17:20.334268 2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 30 15:17:20.334647 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:17:20.334332 2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 30 15:17:21.262144 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:17:21.262082 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-556f467f75-v29c7" event={"ID":"fb4933d3-a2b3-49b0-bcdd-70b6fbbdfea0","Type":"ContainerStarted","Data":"5459e9fd6240576e4304f05927c7538698f23edd89ac526dd01caae7e637d00e"} Jul 30 15:17:21.262336 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:17:21.262210 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-lws-operator/lws-controller-manager-556f467f75-v29c7" Jul 30 15:17:21.263684 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:17:21.263656 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-htlrm" event={"ID":"291e793a-4cbe-4576-a16f-eef021410b23","Type":"ContainerStarted","Data":"20c1943fca0c847a6e670746d4a607f8cf4a3a90f879c638498a4374d08b3cf9"} Jul 30 15:17:21.263880 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:17:21.263861 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-htlrm" Jul 30 15:17:21.265398 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:17:21.265367 2576 patch_prober.go:28] interesting pod/istiod-openshift-gateway-5677ccb49b-htlrm container/discovery namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body= Jul 30 15:17:21.265508 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:17:21.265413 2576 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-htlrm" podUID="291e793a-4cbe-4576-a16f-eef021410b23" containerName="discovery" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 30 15:17:21.289404 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:17:21.289351 2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-lws-operator/lws-controller-manager-556f467f75-v29c7" podStartSLOduration=1.88780482 podStartE2EDuration="4.289330468s" totalImagesPullingTime="2.401525648s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 15:17:17 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 15:17:21.28688222 +0000 UTC m=+423.212634865" watchObservedRunningTime="2026-07-30 15:17:21.289330468 +0000 UTC m=+423.215083118" Jul 30 15:17:21.344141 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:17:21.344028 2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-htlrm" podStartSLOduration=2.095909446 podStartE2EDuration="6.344006138s" totalImagesPullingTime="4.248096692s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 15:17:15 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 15:17:21.342771695 +0000 UTC m=+423.268524344" watchObservedRunningTime="2026-07-30 15:17:21.344006138 +0000 UTC m=+423.269758784" Jul 30 15:17:22.268197 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:17:22.268170 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-htlrm" Jul 30 15:17:32.269994 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:17:32.269962 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-lws-operator/lws-controller-manager-556f467f75-v29c7" Jul 30 15:18:08.947363 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:18:08.947322 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7j468_a2197c57-2dc6-410e-9910-a045dedcb8d9/konnectivity-agent/0.log" limit=1024 Jul 30 15:18:09.051374 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:18:09.051344 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7j468_a2197c57-2dc6-410e-9910-a045dedcb8d9/konnectivity-agent/0.log" limit=1024 Jul 30 15:18:09.398045 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:18:09.398009 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7j468_a2197c57-2dc6-410e-9910-a045dedcb8d9/konnectivity-agent/0.log" limit=1024 Jul 30 15:18:27.430940 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:18:27.430902 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7j468_a2197c57-2dc6-410e-9910-a045dedcb8d9/konnectivity-agent/0.log" limit=1024 Jul 30 15:18:27.598082 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:18:27.598042 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7j468_a2197c57-2dc6-410e-9910-a045dedcb8d9/konnectivity-agent/0.log" limit=1024 Jul 30 15:18:33.201997 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:18:33.201961 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7j468_a2197c57-2dc6-410e-9910-a045dedcb8d9/konnectivity-agent/0.log" limit=1024 Jul 30 15:18:33.268564 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:18:33.268528 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7j468_a2197c57-2dc6-410e-9910-a045dedcb8d9/konnectivity-agent/0.log" limit=1024 Jul 30 15:18:34.408912 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:18:34.408878 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7j468_a2197c57-2dc6-410e-9910-a045dedcb8d9/konnectivity-agent/0.log" limit=1024 Jul 30 15:18:34.466302 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:18:34.466266 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7j468_a2197c57-2dc6-410e-9910-a045dedcb8d9/konnectivity-agent/0.log" limit=1024 Jul 30 15:18:38.631420 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:18:38.631382 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7j468_a2197c57-2dc6-410e-9910-a045dedcb8d9/konnectivity-agent/0.log" limit=1024 Jul 30 15:18:42.025398 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:18:42.025364 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7j468_a2197c57-2dc6-410e-9910-a045dedcb8d9/konnectivity-agent/0.log" limit=1024 Jul 30 15:18:42.829820 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:18:42.829775 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7j468_a2197c57-2dc6-410e-9910-a045dedcb8d9/konnectivity-agent/0.log" limit=1024 Jul 30 15:18:44.217160 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:18:44.217124 2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-74d998c855-zjnm7"] Jul 30 15:18:44.332432 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:18:44.332400 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7j468_a2197c57-2dc6-410e-9910-a045dedcb8d9/konnectivity-agent/0.log" limit=1024 Jul 30 15:18:44.432994 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:18:44.432957 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7j468_a2197c57-2dc6-410e-9910-a045dedcb8d9/konnectivity-agent/0.log" limit=1024 Jul 30 15:18:44.503780 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:18:44.503690 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7j468_a2197c57-2dc6-410e-9910-a045dedcb8d9/konnectivity-agent/0.log" limit=1024 Jul 30 15:18:45.934456 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:18:45.934422 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7j468_a2197c57-2dc6-410e-9910-a045dedcb8d9/konnectivity-agent/0.log" limit=1024 Jul 30 15:18:46.086580 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:18:46.086545 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7j468_a2197c57-2dc6-410e-9910-a045dedcb8d9/konnectivity-agent/0.log" limit=1024 Jul 30 15:18:46.191066 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:18:46.190975 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7j468_a2197c57-2dc6-410e-9910-a045dedcb8d9/konnectivity-agent/0.log" limit=1024 Jul 30 15:18:48.528707 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:18:48.528670 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7j468_a2197c57-2dc6-410e-9910-a045dedcb8d9/konnectivity-agent/0.log" limit=1024 Jul 30 15:18:50.285645 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:18:50.285599 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7j468_a2197c57-2dc6-410e-9910-a045dedcb8d9/konnectivity-agent/0.log" limit=1024 Jul 30 15:18:51.493584 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:18:51.493546 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7j468_a2197c57-2dc6-410e-9910-a045dedcb8d9/konnectivity-agent/0.log" limit=1024 Jul 30 15:18:52.153466 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:18:52.153431 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7j468_a2197c57-2dc6-410e-9910-a045dedcb8d9/konnectivity-agent/0.log" limit=1024 Jul 30 15:18:53.649793 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:18:53.649744 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7j468_a2197c57-2dc6-410e-9910-a045dedcb8d9/konnectivity-agent/0.log" limit=1024 Jul 30 15:18:55.050824 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:18:55.050789 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7j468_a2197c57-2dc6-410e-9910-a045dedcb8d9/konnectivity-agent/0.log" limit=1024 Jul 30 15:18:55.148035 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:18:55.148000 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7j468_a2197c57-2dc6-410e-9910-a045dedcb8d9/konnectivity-agent/0.log" limit=1024 Jul 30 15:18:59.034816 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:18:59.034773 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7j468_a2197c57-2dc6-410e-9910-a045dedcb8d9/konnectivity-agent/0.log" limit=1024 Jul 30 15:18:59.126490 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:18:59.126449 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7j468_a2197c57-2dc6-410e-9910-a045dedcb8d9/konnectivity-agent/0.log" limit=1024 Jul 30 15:18:59.210562 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:18:59.210530 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7j468_a2197c57-2dc6-410e-9910-a045dedcb8d9/konnectivity-agent/0.log" limit=1024 Jul 30 15:18:59.431084 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:18:59.430991 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7j468_a2197c57-2dc6-410e-9910-a045dedcb8d9/konnectivity-agent/0.log" limit=1024 Jul 30 15:18:59.506850 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:18:59.506811 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7j468_a2197c57-2dc6-410e-9910-a045dedcb8d9/konnectivity-agent/0.log" limit=1024 Jul 30 15:19:02.639223 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:19:02.639186 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7j468_a2197c57-2dc6-410e-9910-a045dedcb8d9/konnectivity-agent/0.log" limit=1024 Jul 30 15:19:02.739259 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:19:02.739221 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7j468_a2197c57-2dc6-410e-9910-a045dedcb8d9/konnectivity-agent/0.log" limit=1024 Jul 30 15:19:09.236442 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:19:09.236391 2576 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-74d998c855-zjnm7" podUID="0a8d4006-7747-4036-9b25-16ade9760013" containerName="console" containerID="cri-o://7087f395b29883f3ea21c15c9ac631139d0af8d94af44a42edb245c38817c80f" gracePeriod=15 Jul 30 15:19:09.243578 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:19:09.243151 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7j468_a2197c57-2dc6-410e-9910-a045dedcb8d9/konnectivity-agent/0.log" limit=1024 Jul 30 15:19:09.340455 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:19:09.340418 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7j468_a2197c57-2dc6-410e-9910-a045dedcb8d9/konnectivity-agent/0.log" limit=1024 Jul 30 15:19:09.490195 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:19:09.490131 2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-74d998c855-zjnm7" Jul 30 15:19:09.639997 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:19:09.639960 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/0a8d4006-7747-4036-9b25-16ade9760013-oauth-serving-cert\") pod \"0a8d4006-7747-4036-9b25-16ade9760013\" (UID: \"0a8d4006-7747-4036-9b25-16ade9760013\") " Jul 30 15:19:09.639997 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:19:09.639997 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/0a8d4006-7747-4036-9b25-16ade9760013-console-serving-cert\") pod \"0a8d4006-7747-4036-9b25-16ade9760013\" (UID: \"0a8d4006-7747-4036-9b25-16ade9760013\") " Jul 30 15:19:09.640294 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:19:09.640028 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/0a8d4006-7747-4036-9b25-16ade9760013-trusted-ca-bundle\") pod \"0a8d4006-7747-4036-9b25-16ade9760013\" (UID: \"0a8d4006-7747-4036-9b25-16ade9760013\") " Jul 30 15:19:09.640294 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:19:09.640070 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-4qh47\" (UniqueName: \"kubernetes.io/projected/0a8d4006-7747-4036-9b25-16ade9760013-kube-api-access-4qh47\") pod \"0a8d4006-7747-4036-9b25-16ade9760013\" (UID: \"0a8d4006-7747-4036-9b25-16ade9760013\") " Jul 30 15:19:09.640294 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:19:09.640093 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/0a8d4006-7747-4036-9b25-16ade9760013-service-ca\") pod \"0a8d4006-7747-4036-9b25-16ade9760013\" (UID: \"0a8d4006-7747-4036-9b25-16ade9760013\") " Jul 30 15:19:09.640294 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:19:09.640145 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/0a8d4006-7747-4036-9b25-16ade9760013-console-oauth-config\") pod \"0a8d4006-7747-4036-9b25-16ade9760013\" (UID: \"0a8d4006-7747-4036-9b25-16ade9760013\") " Jul 30 15:19:09.640294 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:19:09.640191 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/0a8d4006-7747-4036-9b25-16ade9760013-console-config\") pod \"0a8d4006-7747-4036-9b25-16ade9760013\" (UID: \"0a8d4006-7747-4036-9b25-16ade9760013\") " Jul 30 15:19:09.640615 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:19:09.640585 2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0a8d4006-7747-4036-9b25-16ade9760013-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "0a8d4006-7747-4036-9b25-16ade9760013" (UID: "0a8d4006-7747-4036-9b25-16ade9760013"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 30 15:19:09.640696 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:19:09.640593 2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0a8d4006-7747-4036-9b25-16ade9760013-service-ca" (OuterVolumeSpecName: "service-ca") pod "0a8d4006-7747-4036-9b25-16ade9760013" (UID: "0a8d4006-7747-4036-9b25-16ade9760013"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 30 15:19:09.640696 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:19:09.640619 2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0a8d4006-7747-4036-9b25-16ade9760013-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "0a8d4006-7747-4036-9b25-16ade9760013" (UID: "0a8d4006-7747-4036-9b25-16ade9760013"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 30 15:19:09.640766 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:19:09.640733 2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0a8d4006-7747-4036-9b25-16ade9760013-console-config" (OuterVolumeSpecName: "console-config") pod "0a8d4006-7747-4036-9b25-16ade9760013" (UID: "0a8d4006-7747-4036-9b25-16ade9760013"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 30 15:19:09.642400 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:19:09.642360 2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0a8d4006-7747-4036-9b25-16ade9760013-kube-api-access-4qh47" (OuterVolumeSpecName: "kube-api-access-4qh47") pod "0a8d4006-7747-4036-9b25-16ade9760013" (UID: "0a8d4006-7747-4036-9b25-16ade9760013"). InnerVolumeSpecName "kube-api-access-4qh47". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 30 15:19:09.642400 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:19:09.642383 2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/0a8d4006-7747-4036-9b25-16ade9760013-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "0a8d4006-7747-4036-9b25-16ade9760013" (UID: "0a8d4006-7747-4036-9b25-16ade9760013"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 30 15:19:09.642555 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:19:09.642447 2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/0a8d4006-7747-4036-9b25-16ade9760013-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "0a8d4006-7747-4036-9b25-16ade9760013" (UID: "0a8d4006-7747-4036-9b25-16ade9760013"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 30 15:19:09.719980 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:19:09.719942 2576 generic.go:356] "Generic (PLEG): container finished" podID="0a8d4006-7747-4036-9b25-16ade9760013" containerID="7087f395b29883f3ea21c15c9ac631139d0af8d94af44a42edb245c38817c80f" exitCode=0 Jul 30 15:19:09.720183 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:19:09.719990 2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-74d998c855-zjnm7" Jul 30 15:19:09.720183 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:19:09.720001 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-74d998c855-zjnm7" event={"ID":"0a8d4006-7747-4036-9b25-16ade9760013","Type":"ContainerDied","Data":"7087f395b29883f3ea21c15c9ac631139d0af8d94af44a42edb245c38817c80f"} Jul 30 15:19:09.720183 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:19:09.720052 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-74d998c855-zjnm7" event={"ID":"0a8d4006-7747-4036-9b25-16ade9760013","Type":"ContainerDied","Data":"dbf8ec740e6f455f1a899244e50a7dd450a6728c8703702c2d76a7c0de97d872"} Jul 30 15:19:09.720183 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:19:09.720078 2576 scope.go:117] "RemoveContainer" containerID="7087f395b29883f3ea21c15c9ac631139d0af8d94af44a42edb245c38817c80f" Jul 30 15:19:09.731484 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:19:09.731455 2576 scope.go:117] "RemoveContainer" containerID="7087f395b29883f3ea21c15c9ac631139d0af8d94af44a42edb245c38817c80f" Jul 30 15:19:09.731973 ip-10-0-133-210 kubenswrapper[2576]: E0730 15:19:09.731950 2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7087f395b29883f3ea21c15c9ac631139d0af8d94af44a42edb245c38817c80f\": container with ID starting with 7087f395b29883f3ea21c15c9ac631139d0af8d94af44a42edb245c38817c80f not found: ID does not exist" containerID="7087f395b29883f3ea21c15c9ac631139d0af8d94af44a42edb245c38817c80f" Jul 30 15:19:09.732028 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:19:09.731983 2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7087f395b29883f3ea21c15c9ac631139d0af8d94af44a42edb245c38817c80f"} err="failed to get container status \"7087f395b29883f3ea21c15c9ac631139d0af8d94af44a42edb245c38817c80f\": rpc error: code = NotFound desc = could not find container \"7087f395b29883f3ea21c15c9ac631139d0af8d94af44a42edb245c38817c80f\": container with ID starting with 7087f395b29883f3ea21c15c9ac631139d0af8d94af44a42edb245c38817c80f not found: ID does not exist" Jul 30 15:19:09.741034 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:19:09.740973 2576 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/0a8d4006-7747-4036-9b25-16ade9760013-trusted-ca-bundle\") on node \"ip-10-0-133-210.ec2.internal\" DevicePath \"\"" Jul 30 15:19:09.741034 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:19:09.740994 2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-4qh47\" (UniqueName: \"kubernetes.io/projected/0a8d4006-7747-4036-9b25-16ade9760013-kube-api-access-4qh47\") on node \"ip-10-0-133-210.ec2.internal\" DevicePath \"\"" Jul 30 15:19:09.741034 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:19:09.741006 2576 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/0a8d4006-7747-4036-9b25-16ade9760013-service-ca\") on node \"ip-10-0-133-210.ec2.internal\" DevicePath \"\"" Jul 30 15:19:09.741034 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:19:09.741015 2576 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/0a8d4006-7747-4036-9b25-16ade9760013-console-oauth-config\") on node \"ip-10-0-133-210.ec2.internal\" DevicePath \"\"" Jul 30 15:19:09.741034 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:19:09.741023 2576 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/0a8d4006-7747-4036-9b25-16ade9760013-console-config\") on node \"ip-10-0-133-210.ec2.internal\" DevicePath \"\"" Jul 30 15:19:09.741034 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:19:09.741031 2576 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/0a8d4006-7747-4036-9b25-16ade9760013-oauth-serving-cert\") on node \"ip-10-0-133-210.ec2.internal\" DevicePath \"\"" Jul 30 15:19:09.741308 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:19:09.741039 2576 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/0a8d4006-7747-4036-9b25-16ade9760013-console-serving-cert\") on node \"ip-10-0-133-210.ec2.internal\" DevicePath \"\"" Jul 30 15:19:09.750739 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:19:09.750702 2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-74d998c855-zjnm7"] Jul 30 15:19:09.753770 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:19:09.753741 2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-74d998c855-zjnm7"] Jul 30 15:19:10.612613 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:19:10.612573 2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="0a8d4006-7747-4036-9b25-16ade9760013" path="/var/lib/kubelet/pods/0a8d4006-7747-4036-9b25-16ade9760013/volumes" Jul 30 15:19:13.542480 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:19:13.542442 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7j468_a2197c57-2dc6-410e-9910-a045dedcb8d9/konnectivity-agent/0.log" limit=1024 Jul 30 15:19:13.643916 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:19:13.643881 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7j468_a2197c57-2dc6-410e-9910-a045dedcb8d9/konnectivity-agent/0.log" limit=1024 Jul 30 15:19:15.393690 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:19:15.393655 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7j468_a2197c57-2dc6-410e-9910-a045dedcb8d9/konnectivity-agent/0.log" limit=1024 Jul 30 15:19:15.468584 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:19:15.468549 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7j468_a2197c57-2dc6-410e-9910-a045dedcb8d9/konnectivity-agent/0.log" limit=1024 Jul 30 15:19:16.567560 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:19:16.567491 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7j468_a2197c57-2dc6-410e-9910-a045dedcb8d9/konnectivity-agent/0.log" limit=1024 Jul 30 15:19:16.666852 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:19:16.666812 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7j468_a2197c57-2dc6-410e-9910-a045dedcb8d9/konnectivity-agent/0.log" limit=1024 Jul 30 15:19:17.328672 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:19:17.328635 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7j468_a2197c57-2dc6-410e-9910-a045dedcb8d9/konnectivity-agent/0.log" limit=1024 Jul 30 15:19:17.428221 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:19:17.428181 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7j468_a2197c57-2dc6-410e-9910-a045dedcb8d9/konnectivity-agent/0.log" limit=1024 Jul 30 15:19:17.527618 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:19:17.527566 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7j468_a2197c57-2dc6-410e-9910-a045dedcb8d9/konnectivity-agent/0.log" limit=1024 Jul 30 15:19:17.599583 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:19:17.599483 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7j468_a2197c57-2dc6-410e-9910-a045dedcb8d9/konnectivity-agent/0.log" limit=1024 Jul 30 15:19:22.725729 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:19:22.725688 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7j468_a2197c57-2dc6-410e-9910-a045dedcb8d9/konnectivity-agent/0.log" limit=1024 Jul 30 15:19:22.822327 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:19:22.822293 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7j468_a2197c57-2dc6-410e-9910-a045dedcb8d9/konnectivity-agent/0.log" limit=1024 Jul 30 15:19:29.184283 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:19:29.184246 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7j468_a2197c57-2dc6-410e-9910-a045dedcb8d9/konnectivity-agent/0.log" limit=1024 Jul 30 15:19:29.284787 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:19:29.284754 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7j468_a2197c57-2dc6-410e-9910-a045dedcb8d9/konnectivity-agent/0.log" limit=1024 Jul 30 15:19:49.956706 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:19:49.956661 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7j468_a2197c57-2dc6-410e-9910-a045dedcb8d9/konnectivity-agent/0.log" limit=1024 Jul 30 15:19:50.590425 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:19:50.590390 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7j468_a2197c57-2dc6-410e-9910-a045dedcb8d9/konnectivity-agent/0.log" limit=1024 Jul 30 15:20:02.630835 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:20:02.630795 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7j468_a2197c57-2dc6-410e-9910-a045dedcb8d9/konnectivity-agent/0.log" limit=1024 Jul 30 15:20:02.731095 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:20:02.731060 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7j468_a2197c57-2dc6-410e-9910-a045dedcb8d9/konnectivity-agent/0.log" limit=1024 Jul 30 15:20:03.301527 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:20:03.301484 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7j468_a2197c57-2dc6-410e-9910-a045dedcb8d9/konnectivity-agent/0.log" limit=1024 Jul 30 15:20:03.402278 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:20:03.402249 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7j468_a2197c57-2dc6-410e-9910-a045dedcb8d9/konnectivity-agent/0.log" limit=1024 Jul 30 15:20:04.946321 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:20:04.946285 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7j468_a2197c57-2dc6-410e-9910-a045dedcb8d9/konnectivity-agent/0.log" limit=1024 Jul 30 15:20:23.308842 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:20:23.308803 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7j468_a2197c57-2dc6-410e-9910-a045dedcb8d9/konnectivity-agent/0.log" limit=1024 Jul 30 15:20:23.400455 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:20:23.400417 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7j468_a2197c57-2dc6-410e-9910-a045dedcb8d9/konnectivity-agent/0.log" limit=1024 Jul 30 15:20:23.770564 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:20:23.770530 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7j468_a2197c57-2dc6-410e-9910-a045dedcb8d9/konnectivity-agent/0.log" limit=1024 Jul 30 15:20:36.189286 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:20:36.189245 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7j468_a2197c57-2dc6-410e-9910-a045dedcb8d9/konnectivity-agent/0.log" limit=1024 Jul 30 15:20:36.289331 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:20:36.289271 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7j468_a2197c57-2dc6-410e-9910-a045dedcb8d9/konnectivity-agent/0.log" limit=1024 Jul 30 15:20:43.968920 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:20:43.968823 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7j468_a2197c57-2dc6-410e-9910-a045dedcb8d9/konnectivity-agent/0.log" limit=1024 Jul 30 15:20:45.676653 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:20:45.676616 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7j468_a2197c57-2dc6-410e-9910-a045dedcb8d9/konnectivity-agent/0.log" limit=1024 Jul 30 15:20:47.192234 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:20:47.192195 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7j468_a2197c57-2dc6-410e-9910-a045dedcb8d9/konnectivity-agent/0.log" limit=1024 Jul 30 15:20:47.291635 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:20:47.291593 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7j468_a2197c57-2dc6-410e-9910-a045dedcb8d9/konnectivity-agent/0.log" limit=1024 Jul 30 15:20:48.397977 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:20:48.397932 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7j468_a2197c57-2dc6-410e-9910-a045dedcb8d9/konnectivity-agent/0.log" limit=1024 Jul 30 15:20:48.499547 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:20:48.499503 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7j468_a2197c57-2dc6-410e-9910-a045dedcb8d9/konnectivity-agent/0.log" limit=1024 Jul 30 15:21:21.906760 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:21:21.906716 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7j468_a2197c57-2dc6-410e-9910-a045dedcb8d9/konnectivity-agent/0.log" limit=1024 Jul 30 15:21:22.024399 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:21:22.024360 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7j468_a2197c57-2dc6-410e-9910-a045dedcb8d9/konnectivity-agent/0.log" limit=1024 Jul 30 15:21:22.874971 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:21:22.874936 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7j468_a2197c57-2dc6-410e-9910-a045dedcb8d9/konnectivity-agent/0.log" limit=1024 Jul 30 15:21:23.188011 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:21:23.187893 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7j468_a2197c57-2dc6-410e-9910-a045dedcb8d9/konnectivity-agent/0.log" limit=1024 Jul 30 15:21:23.456263 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:21:23.456168 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7j468_a2197c57-2dc6-410e-9910-a045dedcb8d9/konnectivity-agent/0.log" limit=1024 Jul 30 15:21:23.620248 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:21:23.620200 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7j468_a2197c57-2dc6-410e-9910-a045dedcb8d9/konnectivity-agent/0.log" limit=1024 Jul 30 15:21:33.316315 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:21:33.316278 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7j468_a2197c57-2dc6-410e-9910-a045dedcb8d9/konnectivity-agent/0.log" limit=1024 Jul 30 15:21:33.415881 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:21:33.415845 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7j468_a2197c57-2dc6-410e-9910-a045dedcb8d9/konnectivity-agent/0.log" limit=1024 Jul 30 15:21:36.787315 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:21:36.787277 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7j468_a2197c57-2dc6-410e-9910-a045dedcb8d9/konnectivity-agent/0.log" limit=1024 Jul 30 15:21:36.888970 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:21:36.888937 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7j468_a2197c57-2dc6-410e-9910-a045dedcb8d9/konnectivity-agent/0.log" limit=1024 Jul 30 15:21:49.578541 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:21:49.578504 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7j468_a2197c57-2dc6-410e-9910-a045dedcb8d9/konnectivity-agent/0.log" limit=1024 Jul 30 15:22:02.028520 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:22:02.028475 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7j468_a2197c57-2dc6-410e-9910-a045dedcb8d9/konnectivity-agent/0.log" limit=1024 Jul 30 15:22:06.369699 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:22:06.369651 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7j468_a2197c57-2dc6-410e-9910-a045dedcb8d9/konnectivity-agent/0.log" limit=1024 Jul 30 15:22:25.290055 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:22:25.289957 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7j468_a2197c57-2dc6-410e-9910-a045dedcb8d9/konnectivity-agent/0.log" limit=1024 Jul 30 15:22:30.381545 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:22:30.381507 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7j468_a2197c57-2dc6-410e-9910-a045dedcb8d9/konnectivity-agent/0.log" limit=1024 Jul 30 15:22:41.948976 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:22:41.948933 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7j468_a2197c57-2dc6-410e-9910-a045dedcb8d9/konnectivity-agent/0.log" limit=1024 Jul 30 15:23:35.363784 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:23:35.363740 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7j468_a2197c57-2dc6-410e-9910-a045dedcb8d9/konnectivity-agent/0.log" limit=1024 Jul 30 15:23:35.464800 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:23:35.464767 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7j468_a2197c57-2dc6-410e-9910-a045dedcb8d9/konnectivity-agent/0.log" limit=1024 Jul 30 15:23:52.854972 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:23:52.854892 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7j468_a2197c57-2dc6-410e-9910-a045dedcb8d9/konnectivity-agent/0.log" limit=1024 Jul 30 15:23:52.942949 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:23:52.942912 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7j468_a2197c57-2dc6-410e-9910-a045dedcb8d9/konnectivity-agent/0.log" limit=1024 Jul 30 15:23:54.072281 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:23:54.072248 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7j468_a2197c57-2dc6-410e-9910-a045dedcb8d9/konnectivity-agent/0.log" limit=1024 Jul 30 15:23:54.160774 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:23:54.160738 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7j468_a2197c57-2dc6-410e-9910-a045dedcb8d9/konnectivity-agent/0.log" limit=1024 Jul 30 15:24:44.789923 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:24:44.789883 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7j468_a2197c57-2dc6-410e-9910-a045dedcb8d9/konnectivity-agent/0.log" limit=1024 Jul 30 15:25:59.302398 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:25:59.302357 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7j468_a2197c57-2dc6-410e-9910-a045dedcb8d9/konnectivity-agent/0.log" limit=1024 Jul 30 15:26:02.274707 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:26:02.274669 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7j468_a2197c57-2dc6-410e-9910-a045dedcb8d9/konnectivity-agent/0.log" limit=1024 Jul 30 15:27:15.475491 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:27:15.475448 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7j468_a2197c57-2dc6-410e-9910-a045dedcb8d9/konnectivity-agent/0.log" limit=1024 Jul 30 15:27:15.558616 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:27:15.558575 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7j468_a2197c57-2dc6-410e-9910-a045dedcb8d9/konnectivity-agent/0.log" limit=1024 Jul 30 15:27:15.679942 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:27:15.679903 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7j468_a2197c57-2dc6-410e-9910-a045dedcb8d9/konnectivity-agent/0.log" limit=1024 Jul 30 15:27:21.694876 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:27:21.694841 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7j468_a2197c57-2dc6-410e-9910-a045dedcb8d9/konnectivity-agent/0.log" limit=1024 Jul 30 15:27:21.801580 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:27:21.801538 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7j468_a2197c57-2dc6-410e-9910-a045dedcb8d9/konnectivity-agent/0.log" limit=1024 Jul 30 15:27:22.012041 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:27:22.011998 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7j468_a2197c57-2dc6-410e-9910-a045dedcb8d9/konnectivity-agent/0.log" limit=1024 Jul 30 15:27:22.110199 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:27:22.110160 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7j468_a2197c57-2dc6-410e-9910-a045dedcb8d9/konnectivity-agent/0.log" limit=1024 Jul 30 15:27:28.000444 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:27:28.000398 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7j468_a2197c57-2dc6-410e-9910-a045dedcb8d9/konnectivity-agent/0.log" limit=1024 Jul 30 15:27:28.087801 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:27:28.087759 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7j468_a2197c57-2dc6-410e-9910-a045dedcb8d9/konnectivity-agent/0.log" limit=1024 Jul 30 15:27:29.397780 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:27:29.397745 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7j468_a2197c57-2dc6-410e-9910-a045dedcb8d9/konnectivity-agent/0.log" limit=1024 Jul 30 15:28:47.321057 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:28:47.321022 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7j468_a2197c57-2dc6-410e-9910-a045dedcb8d9/konnectivity-agent/0.log" limit=1024 Jul 30 15:30:28.615798 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:30:28.615764 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7j468_a2197c57-2dc6-410e-9910-a045dedcb8d9/konnectivity-agent/0.log" limit=1024 Jul 30 15:30:28.672708 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:30:28.672656 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7j468_a2197c57-2dc6-410e-9910-a045dedcb8d9/konnectivity-agent/0.log" limit=1024 Jul 30 15:31:39.791086 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:31:39.791049 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7j468_a2197c57-2dc6-410e-9910-a045dedcb8d9/konnectivity-agent/0.log" limit=1024 Jul 30 15:31:40.927140 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:31:40.927088 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7j468_a2197c57-2dc6-410e-9910-a045dedcb8d9/konnectivity-agent/0.log" limit=1024 Jul 30 15:32:32.896570 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:32:32.896533 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7j468_a2197c57-2dc6-410e-9910-a045dedcb8d9/konnectivity-agent/0.log" limit=1024 Jul 30 15:34:51.653519 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:34:51.653485 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7j468_a2197c57-2dc6-410e-9910-a045dedcb8d9/konnectivity-agent/0.log" limit=1024 Jul 30 15:36:16.334716 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:36:16.334687 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7j468_a2197c57-2dc6-410e-9910-a045dedcb8d9/konnectivity-agent/0.log" limit=1024 Jul 30 15:37:54.520311 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:37:54.520274 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7j468_a2197c57-2dc6-410e-9910-a045dedcb8d9/konnectivity-agent/0.log" limit=1024 Jul 30 15:38:07.467806 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:38:07.467764 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7j468_a2197c57-2dc6-410e-9910-a045dedcb8d9/konnectivity-agent/0.log" limit=1024 Jul 30 15:38:35.273082 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:38:35.273050 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7j468_a2197c57-2dc6-410e-9910-a045dedcb8d9/konnectivity-agent/0.log" limit=1024 Jul 30 15:38:36.866550 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:38:36.866497 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7j468_a2197c57-2dc6-410e-9910-a045dedcb8d9/konnectivity-agent/0.log" limit=1024 Jul 30 15:41:18.989908 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:41:18.989858 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7j468_a2197c57-2dc6-410e-9910-a045dedcb8d9/konnectivity-agent/0.log" limit=1024 Jul 30 15:41:19.055878 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:41:19.055842 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7j468_a2197c57-2dc6-410e-9910-a045dedcb8d9/konnectivity-agent/0.log" limit=1024 Jul 30 15:42:43.052885 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:42:43.052844 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7j468_a2197c57-2dc6-410e-9910-a045dedcb8d9/konnectivity-agent/0.log" limit=1024 Jul 30 15:42:58.582307 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:42:58.582273 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7j468_a2197c57-2dc6-410e-9910-a045dedcb8d9/konnectivity-agent/0.log" limit=1024 Jul 30 15:43:19.858975 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:43:19.858879 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7j468_a2197c57-2dc6-410e-9910-a045dedcb8d9/konnectivity-agent/0.log" limit=1024 Jul 30 15:43:23.185090 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:43:23.185048 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7j468_a2197c57-2dc6-410e-9910-a045dedcb8d9/konnectivity-agent/0.log" limit=1024 Jul 30 15:43:23.277417 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:43:23.277379 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7j468_a2197c57-2dc6-410e-9910-a045dedcb8d9/konnectivity-agent/0.log" limit=1024 Jul 30 15:43:23.691092 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:43:23.691057 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7j468_a2197c57-2dc6-410e-9910-a045dedcb8d9/konnectivity-agent/0.log" limit=1024 Jul 30 15:43:30.330383 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:43:30.330338 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7j468_a2197c57-2dc6-410e-9910-a045dedcb8d9/konnectivity-agent/0.log" limit=1024 Jul 30 15:43:30.431167 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:43:30.431128 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7j468_a2197c57-2dc6-410e-9910-a045dedcb8d9/konnectivity-agent/0.log" limit=1024 Jul 30 15:43:30.581504 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:43:30.581405 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7j468_a2197c57-2dc6-410e-9910-a045dedcb8d9/konnectivity-agent/0.log" limit=1024 Jul 30 15:43:30.934622 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:43:30.934536 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7j468_a2197c57-2dc6-410e-9910-a045dedcb8d9/konnectivity-agent/0.log" limit=1024 Jul 30 15:43:31.043812 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:43:31.043774 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7j468_a2197c57-2dc6-410e-9910-a045dedcb8d9/konnectivity-agent/0.log" limit=1024 Jul 30 15:43:33.815929 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:43:33.815888 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7j468_a2197c57-2dc6-410e-9910-a045dedcb8d9/konnectivity-agent/0.log" limit=1024 Jul 30 15:43:33.915880 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:43:33.915842 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7j468_a2197c57-2dc6-410e-9910-a045dedcb8d9/konnectivity-agent/0.log" limit=1024 Jul 30 15:43:35.022190 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:43:35.022153 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7j468_a2197c57-2dc6-410e-9910-a045dedcb8d9/konnectivity-agent/0.log" limit=1024 Jul 30 15:43:35.498845 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:43:35.498790 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7j468_a2197c57-2dc6-410e-9910-a045dedcb8d9/konnectivity-agent/0.log" limit=1024 Jul 30 15:43:35.609029 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:43:35.608996 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7j468_a2197c57-2dc6-410e-9910-a045dedcb8d9/konnectivity-agent/0.log" limit=1024 Jul 30 15:43:35.697758 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:43:35.697721 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7j468_a2197c57-2dc6-410e-9910-a045dedcb8d9/konnectivity-agent/0.log" limit=1024 Jul 30 15:44:15.674596 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:44:15.674555 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7j468_a2197c57-2dc6-410e-9910-a045dedcb8d9/konnectivity-agent/0.log" limit=1024 Jul 30 15:44:15.784466 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:44:15.784429 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7j468_a2197c57-2dc6-410e-9910-a045dedcb8d9/konnectivity-agent/0.log" limit=1024 Jul 30 15:44:15.925932 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:44:15.925842 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7j468_a2197c57-2dc6-410e-9910-a045dedcb8d9/konnectivity-agent/0.log" limit=1024 Jul 30 15:44:16.013465 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:44:16.013424 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7j468_a2197c57-2dc6-410e-9910-a045dedcb8d9/konnectivity-agent/0.log" limit=1024 Jul 30 15:44:16.145188 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:44:16.145152 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7j468_a2197c57-2dc6-410e-9910-a045dedcb8d9/konnectivity-agent/0.log" limit=1024 Jul 30 15:44:16.399302 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:44:16.399265 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7j468_a2197c57-2dc6-410e-9910-a045dedcb8d9/konnectivity-agent/0.log" limit=1024 Jul 30 15:44:16.654419 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:44:16.654308 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7j468_a2197c57-2dc6-410e-9910-a045dedcb8d9/konnectivity-agent/0.log" limit=1024 Jul 30 15:44:16.734512 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:44:16.734474 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7j468_a2197c57-2dc6-410e-9910-a045dedcb8d9/konnectivity-agent/0.log" limit=1024 Jul 30 15:44:16.890780 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:44:16.890741 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7j468_a2197c57-2dc6-410e-9910-a045dedcb8d9/konnectivity-agent/0.log" limit=1024 Jul 30 15:44:17.279589 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:44:17.279559 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7j468_a2197c57-2dc6-410e-9910-a045dedcb8d9/konnectivity-agent/0.log" limit=1024 Jul 30 15:44:17.539368 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:44:17.539276 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7j468_a2197c57-2dc6-410e-9910-a045dedcb8d9/konnectivity-agent/0.log" limit=1024 Jul 30 15:44:20.247586 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:44:20.247529 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7j468_a2197c57-2dc6-410e-9910-a045dedcb8d9/konnectivity-agent/0.log" limit=1024 Jul 30 15:44:20.352056 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:44:20.352012 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7j468_a2197c57-2dc6-410e-9910-a045dedcb8d9/konnectivity-agent/0.log" limit=1024 Jul 30 15:44:20.577840 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:44:20.577742 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7j468_a2197c57-2dc6-410e-9910-a045dedcb8d9/konnectivity-agent/0.log" limit=1024 Jul 30 15:44:20.879407 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:44:20.879306 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7j468_a2197c57-2dc6-410e-9910-a045dedcb8d9/konnectivity-agent/0.log" limit=1024 Jul 30 15:44:20.978138 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:44:20.978094 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7j468_a2197c57-2dc6-410e-9910-a045dedcb8d9/konnectivity-agent/0.log" limit=1024 Jul 30 15:44:32.450334 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:44:32.450300 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7j468_a2197c57-2dc6-410e-9910-a045dedcb8d9/konnectivity-agent/0.log" limit=1024 Jul 30 15:44:51.887249 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:44:51.887211 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7j468_a2197c57-2dc6-410e-9910-a045dedcb8d9/konnectivity-agent/0.log" limit=1024 Jul 30 15:44:51.962288 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:44:51.962246 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7j468_a2197c57-2dc6-410e-9910-a045dedcb8d9/konnectivity-agent/0.log" limit=1024 Jul 30 15:44:52.060684 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:44:52.060645 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7j468_a2197c57-2dc6-410e-9910-a045dedcb8d9/konnectivity-agent/0.log" limit=1024 Jul 30 15:44:52.200717 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:44:52.200630 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7j468_a2197c57-2dc6-410e-9910-a045dedcb8d9/konnectivity-agent/0.log" limit=1024 Jul 30 15:44:52.359710 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:44:52.359676 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7j468_a2197c57-2dc6-410e-9910-a045dedcb8d9/konnectivity-agent/0.log" limit=1024 Jul 30 15:44:52.653746 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:44:52.653713 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7j468_a2197c57-2dc6-410e-9910-a045dedcb8d9/konnectivity-agent/0.log" limit=1024 Jul 30 15:44:52.850466 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:44:52.850422 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7j468_a2197c57-2dc6-410e-9910-a045dedcb8d9/konnectivity-agent/0.log" limit=1024 Jul 30 15:44:53.302938 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:44:53.302897 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7j468_a2197c57-2dc6-410e-9910-a045dedcb8d9/konnectivity-agent/0.log" limit=1024 Jul 30 15:44:53.488450 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:44:53.488415 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7j468_a2197c57-2dc6-410e-9910-a045dedcb8d9/konnectivity-agent/0.log" limit=1024 Jul 30 15:44:55.282835 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:44:55.282800 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7j468_a2197c57-2dc6-410e-9910-a045dedcb8d9/konnectivity-agent/0.log" limit=1024 Jul 30 15:44:55.388306 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:44:55.388272 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7j468_a2197c57-2dc6-410e-9910-a045dedcb8d9/konnectivity-agent/0.log" limit=1024 Jul 30 15:44:56.410300 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:44:56.410259 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7j468_a2197c57-2dc6-410e-9910-a045dedcb8d9/konnectivity-agent/0.log" limit=1024 Jul 30 15:44:56.835584 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:44:56.835552 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7j468_a2197c57-2dc6-410e-9910-a045dedcb8d9/konnectivity-agent/0.log" limit=1024 Jul 30 15:44:57.033835 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:44:57.033801 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7j468_a2197c57-2dc6-410e-9910-a045dedcb8d9/konnectivity-agent/0.log" limit=1024 Jul 30 15:44:57.134181 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:44:57.134082 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7j468_a2197c57-2dc6-410e-9910-a045dedcb8d9/konnectivity-agent/0.log" limit=1024 Jul 30 15:45:27.044257 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:45:27.044206 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7j468_a2197c57-2dc6-410e-9910-a045dedcb8d9/konnectivity-agent/0.log" limit=1024 Jul 30 15:45:27.124443 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:45:27.124395 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7j468_a2197c57-2dc6-410e-9910-a045dedcb8d9/konnectivity-agent/0.log" limit=1024 Jul 30 15:45:27.196872 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:45:27.196843 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7j468_a2197c57-2dc6-410e-9910-a045dedcb8d9/konnectivity-agent/0.log" limit=1024 Jul 30 15:45:27.275762 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:45:27.275724 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7j468_a2197c57-2dc6-410e-9910-a045dedcb8d9/konnectivity-agent/0.log" limit=1024 Jul 30 15:45:28.541588 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:45:28.541559 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7j468_a2197c57-2dc6-410e-9910-a045dedcb8d9/konnectivity-agent/0.log" limit=1024 Jul 30 15:45:28.795860 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:45:28.795782 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7j468_a2197c57-2dc6-410e-9910-a045dedcb8d9/konnectivity-agent/0.log" limit=1024 Jul 30 15:45:29.037053 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:45:29.037005 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7j468_a2197c57-2dc6-410e-9910-a045dedcb8d9/konnectivity-agent/0.log" limit=1024 Jul 30 15:45:29.425510 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:45:29.425471 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7j468_a2197c57-2dc6-410e-9910-a045dedcb8d9/konnectivity-agent/0.log" limit=1024 Jul 30 15:45:29.663859 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:45:29.663822 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7j468_a2197c57-2dc6-410e-9910-a045dedcb8d9/konnectivity-agent/0.log" limit=1024 Jul 30 15:45:31.498505 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:45:31.498468 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7j468_a2197c57-2dc6-410e-9910-a045dedcb8d9/konnectivity-agent/0.log" limit=1024 Jul 30 15:45:31.604446 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:45:31.604416 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7j468_a2197c57-2dc6-410e-9910-a045dedcb8d9/konnectivity-agent/0.log" limit=1024 Jul 30 15:45:32.652774 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:45:32.652737 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7j468_a2197c57-2dc6-410e-9910-a045dedcb8d9/konnectivity-agent/0.log" limit=1024 Jul 30 15:45:32.771551 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:45:32.771493 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7j468_a2197c57-2dc6-410e-9910-a045dedcb8d9/konnectivity-agent/0.log" limit=1024 Jul 30 15:45:33.236920 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:45:33.236883 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7j468_a2197c57-2dc6-410e-9910-a045dedcb8d9/konnectivity-agent/0.log" limit=1024 Jul 30 15:45:33.335031 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:45:33.334994 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7j468_a2197c57-2dc6-410e-9910-a045dedcb8d9/konnectivity-agent/0.log" limit=1024 Jul 30 15:46:03.484437 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:46:03.484394 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7j468_a2197c57-2dc6-410e-9910-a045dedcb8d9/konnectivity-agent/0.log" limit=1024 Jul 30 15:46:03.649879 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:46:03.649840 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7j468_a2197c57-2dc6-410e-9910-a045dedcb8d9/konnectivity-agent/0.log" limit=1024 Jul 30 15:46:03.787909 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:46:03.787830 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7j468_a2197c57-2dc6-410e-9910-a045dedcb8d9/konnectivity-agent/0.log" limit=1024 Jul 30 15:46:03.916310 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:46:03.916279 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7j468_a2197c57-2dc6-410e-9910-a045dedcb8d9/konnectivity-agent/0.log" limit=1024 Jul 30 15:46:04.025501 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:46:04.025451 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7j468_a2197c57-2dc6-410e-9910-a045dedcb8d9/konnectivity-agent/0.log" limit=1024 Jul 30 15:46:04.281644 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:46:04.281607 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7j468_a2197c57-2dc6-410e-9910-a045dedcb8d9/konnectivity-agent/0.log" limit=1024 Jul 30 15:46:04.804807 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:46:04.804769 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7j468_a2197c57-2dc6-410e-9910-a045dedcb8d9/konnectivity-agent/0.log" limit=1024 Jul 30 15:46:04.967291 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:46:04.967257 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7j468_a2197c57-2dc6-410e-9910-a045dedcb8d9/konnectivity-agent/0.log" limit=1024 Jul 30 15:46:05.209568 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:46:05.209476 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7j468_a2197c57-2dc6-410e-9910-a045dedcb8d9/konnectivity-agent/0.log" limit=1024 Jul 30 15:46:27.247404 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:46:27.247362 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7j468_a2197c57-2dc6-410e-9910-a045dedcb8d9/konnectivity-agent/0.log" limit=1024 Jul 30 15:46:34.807505 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:46:34.807471 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7j468_a2197c57-2dc6-410e-9910-a045dedcb8d9/konnectivity-agent/0.log" limit=1024 Jul 30 15:46:34.903559 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:46:34.903520 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7j468_a2197c57-2dc6-410e-9910-a045dedcb8d9/konnectivity-agent/0.log" limit=1024 Jul 30 15:46:34.999495 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:46:34.999451 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7j468_a2197c57-2dc6-410e-9910-a045dedcb8d9/konnectivity-agent/0.log" limit=1024 Jul 30 15:46:35.102332 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:46:35.102246 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7j468_a2197c57-2dc6-410e-9910-a045dedcb8d9/konnectivity-agent/0.log" limit=1024 Jul 30 15:46:40.934374 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:46:40.934329 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7j468_a2197c57-2dc6-410e-9910-a045dedcb8d9/konnectivity-agent/0.log" limit=1024 Jul 30 15:47:06.840558 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:47:06.840523 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7j468_a2197c57-2dc6-410e-9910-a045dedcb8d9/konnectivity-agent/0.log" limit=1024 Jul 30 15:47:14.029528 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:47:14.029487 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7j468_a2197c57-2dc6-410e-9910-a045dedcb8d9/konnectivity-agent/0.log" limit=1024 Jul 30 15:47:14.119424 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:47:14.119387 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7j468_a2197c57-2dc6-410e-9910-a045dedcb8d9/konnectivity-agent/0.log" limit=1024 Jul 30 15:47:14.212672 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:47:14.212636 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7j468_a2197c57-2dc6-410e-9910-a045dedcb8d9/konnectivity-agent/0.log" limit=1024 Jul 30 15:47:14.314184 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:47:14.314069 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7j468_a2197c57-2dc6-410e-9910-a045dedcb8d9/konnectivity-agent/0.log" limit=1024 Jul 30 15:47:30.258278 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:47:30.258226 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7j468_a2197c57-2dc6-410e-9910-a045dedcb8d9/konnectivity-agent/0.log" limit=1024 Jul 30 15:47:36.903169 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:47:36.903128 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7j468_a2197c57-2dc6-410e-9910-a045dedcb8d9/konnectivity-agent/0.log" limit=1024 Jul 30 15:47:37.006080 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:47:37.006040 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7j468_a2197c57-2dc6-410e-9910-a045dedcb8d9/konnectivity-agent/0.log" limit=1024 Jul 30 15:47:37.120991 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:47:37.120953 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7j468_a2197c57-2dc6-410e-9910-a045dedcb8d9/konnectivity-agent/0.log" limit=1024 Jul 30 15:47:37.220530 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:47:37.220500 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7j468_a2197c57-2dc6-410e-9910-a045dedcb8d9/konnectivity-agent/0.log" limit=1024 Jul 30 15:47:48.046646 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:47:48.046551 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7j468_a2197c57-2dc6-410e-9910-a045dedcb8d9/konnectivity-agent/0.log" limit=1024 Jul 30 15:47:58.997521 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:47:58.997484 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7j468_a2197c57-2dc6-410e-9910-a045dedcb8d9/konnectivity-agent/0.log" limit=1024 Jul 30 15:48:06.378775 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:48:06.378732 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7j468_a2197c57-2dc6-410e-9910-a045dedcb8d9/konnectivity-agent/0.log" limit=1024 Jul 30 15:48:07.821697 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:48:07.821656 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7j468_a2197c57-2dc6-410e-9910-a045dedcb8d9/konnectivity-agent/0.log" limit=1024 Jul 30 15:48:09.393423 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:48:09.393386 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7j468_a2197c57-2dc6-410e-9910-a045dedcb8d9/konnectivity-agent/0.log" limit=1024 Jul 30 15:48:15.732820 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:48:15.732787 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7j468_a2197c57-2dc6-410e-9910-a045dedcb8d9/konnectivity-agent/0.log" limit=1024 Jul 30 15:48:15.833749 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:48:15.833713 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7j468_a2197c57-2dc6-410e-9910-a045dedcb8d9/konnectivity-agent/0.log" limit=1024 Jul 30 15:48:16.474523 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:48:16.474462 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7j468_a2197c57-2dc6-410e-9910-a045dedcb8d9/konnectivity-agent/0.log" limit=1024 Jul 30 15:48:24.150261 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:48:24.150212 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7j468_a2197c57-2dc6-410e-9910-a045dedcb8d9/konnectivity-agent/0.log" limit=1024 Jul 30 15:48:24.273057 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:48:24.273020 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7j468_a2197c57-2dc6-410e-9910-a045dedcb8d9/konnectivity-agent/0.log" limit=1024 Jul 30 15:48:24.372726 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:48:24.372693 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7j468_a2197c57-2dc6-410e-9910-a045dedcb8d9/konnectivity-agent/0.log" limit=1024 Jul 30 15:48:24.868399 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:48:24.868348 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7j468_a2197c57-2dc6-410e-9910-a045dedcb8d9/konnectivity-agent/0.log" limit=1024 Jul 30 15:48:25.189927 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:48:25.189836 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7j468_a2197c57-2dc6-410e-9910-a045dedcb8d9/konnectivity-agent/0.log" limit=1024 Jul 30 15:48:27.954778 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:48:27.954741 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7j468_a2197c57-2dc6-410e-9910-a045dedcb8d9/konnectivity-agent/0.log" limit=1024 Jul 30 15:48:29.709373 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:48:29.709334 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7j468_a2197c57-2dc6-410e-9910-a045dedcb8d9/konnectivity-agent/0.log" limit=1024 Jul 30 15:48:29.808724 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:48:29.808684 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7j468_a2197c57-2dc6-410e-9910-a045dedcb8d9/konnectivity-agent/0.log" limit=1024 Jul 30 15:48:53.065597 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:48:53.065560 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7j468_a2197c57-2dc6-410e-9910-a045dedcb8d9/konnectivity-agent/0.log" limit=1024 Jul 30 15:48:53.843743 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:48:53.843706 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7j468_a2197c57-2dc6-410e-9910-a045dedcb8d9/konnectivity-agent/0.log" limit=1024 Jul 30 15:48:53.951820 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:48:53.951778 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7j468_a2197c57-2dc6-410e-9910-a045dedcb8d9/konnectivity-agent/0.log" limit=1024 Jul 30 15:48:54.313558 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:48:54.313512 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7j468_a2197c57-2dc6-410e-9910-a045dedcb8d9/konnectivity-agent/0.log" limit=1024 Jul 30 15:49:00.563005 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:49:00.562967 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7j468_a2197c57-2dc6-410e-9910-a045dedcb8d9/konnectivity-agent/0.log" limit=1024 Jul 30 15:49:00.659397 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:49:00.659362 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7j468_a2197c57-2dc6-410e-9910-a045dedcb8d9/konnectivity-agent/0.log" limit=1024 Jul 30 15:49:11.483571 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:49:11.483492 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7j468_a2197c57-2dc6-410e-9910-a045dedcb8d9/konnectivity-agent/0.log" limit=1024 Jul 30 15:49:11.574898 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:49:11.574855 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7j468_a2197c57-2dc6-410e-9910-a045dedcb8d9/konnectivity-agent/0.log" limit=1024 Jul 30 15:49:11.677557 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:49:11.677520 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7j468_a2197c57-2dc6-410e-9910-a045dedcb8d9/konnectivity-agent/0.log" limit=1024 Jul 30 15:49:12.094421 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:49:12.094385 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7j468_a2197c57-2dc6-410e-9910-a045dedcb8d9/konnectivity-agent/0.log" limit=1024 Jul 30 15:49:19.121872 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:49:19.121815 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7j468_a2197c57-2dc6-410e-9910-a045dedcb8d9/konnectivity-agent/0.log" limit=1024 Jul 30 15:49:19.230445 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:49:19.230407 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7j468_a2197c57-2dc6-410e-9910-a045dedcb8d9/konnectivity-agent/0.log" limit=1024 Jul 30 15:49:19.461544 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:49:19.461458 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7j468_a2197c57-2dc6-410e-9910-a045dedcb8d9/konnectivity-agent/0.log" limit=1024 Jul 30 15:49:19.686213 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:49:19.686163 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7j468_a2197c57-2dc6-410e-9910-a045dedcb8d9/konnectivity-agent/0.log" limit=1024 Jul 30 15:49:19.818892 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:49:19.818854 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7j468_a2197c57-2dc6-410e-9910-a045dedcb8d9/konnectivity-agent/0.log" limit=1024 Jul 30 15:49:19.924994 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:49:19.924959 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7j468_a2197c57-2dc6-410e-9910-a045dedcb8d9/konnectivity-agent/0.log" limit=1024 Jul 30 15:49:20.079645 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:49:20.079557 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7j468_a2197c57-2dc6-410e-9910-a045dedcb8d9/konnectivity-agent/0.log" limit=1024 Jul 30 15:49:26.813065 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:49:26.813029 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7j468_a2197c57-2dc6-410e-9910-a045dedcb8d9/konnectivity-agent/0.log" limit=1024 Jul 30 15:49:26.925731 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:49:26.925695 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7j468_a2197c57-2dc6-410e-9910-a045dedcb8d9/konnectivity-agent/0.log" limit=1024 Jul 30 15:49:27.018506 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:49:27.018466 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7j468_a2197c57-2dc6-410e-9910-a045dedcb8d9/konnectivity-agent/0.log" limit=1024 Jul 30 15:49:27.206156 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:49:27.206063 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7j468_a2197c57-2dc6-410e-9910-a045dedcb8d9/konnectivity-agent/0.log" limit=1024 Jul 30 15:49:33.554892 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:49:33.554853 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7j468_a2197c57-2dc6-410e-9910-a045dedcb8d9/konnectivity-agent/0.log" limit=1024 Jul 30 15:49:33.689487 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:49:33.689443 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7j468_a2197c57-2dc6-410e-9910-a045dedcb8d9/konnectivity-agent/0.log" limit=1024 Jul 30 15:49:45.708477 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:49:45.708439 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7j468_a2197c57-2dc6-410e-9910-a045dedcb8d9/konnectivity-agent/0.log" limit=1024 Jul 30 15:49:45.806739 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:49:45.806699 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7j468_a2197c57-2dc6-410e-9910-a045dedcb8d9/konnectivity-agent/0.log" limit=1024 Jul 30 15:49:47.776646 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:49:47.776608 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7j468_a2197c57-2dc6-410e-9910-a045dedcb8d9/konnectivity-agent/0.log" limit=1024 Jul 30 15:49:49.778202 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:49:49.778158 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7j468_a2197c57-2dc6-410e-9910-a045dedcb8d9/konnectivity-agent/0.log" limit=1024 Jul 30 15:50:21.386103 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:50:21.386066 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7j468_a2197c57-2dc6-410e-9910-a045dedcb8d9/konnectivity-agent/0.log" limit=1024 Jul 30 15:50:32.524825 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:50:32.524783 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7j468_a2197c57-2dc6-410e-9910-a045dedcb8d9/konnectivity-agent/0.log" limit=1024 Jul 30 15:50:32.624668 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:50:32.624637 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7j468_a2197c57-2dc6-410e-9910-a045dedcb8d9/konnectivity-agent/0.log" limit=1024 Jul 30 15:51:47.506444 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:51:47.506406 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7j468_a2197c57-2dc6-410e-9910-a045dedcb8d9/konnectivity-agent/0.log" limit=1024 Jul 30 15:51:47.924176 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:51:47.924070 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7j468_a2197c57-2dc6-410e-9910-a045dedcb8d9/konnectivity-agent/0.log" limit=1024 Jul 30 15:51:48.366082 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:51:48.366046 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7j468_a2197c57-2dc6-410e-9910-a045dedcb8d9/konnectivity-agent/0.log" limit=1024 Jul 30 15:51:51.067275 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:51:51.067228 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7j468_a2197c57-2dc6-410e-9910-a045dedcb8d9/konnectivity-agent/0.log" limit=1024 Jul 30 15:51:51.160606 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:51:51.160564 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7j468_a2197c57-2dc6-410e-9910-a045dedcb8d9/konnectivity-agent/0.log" limit=1024 Jul 30 15:51:52.248568 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:51:52.248533 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7j468_a2197c57-2dc6-410e-9910-a045dedcb8d9/konnectivity-agent/0.log" limit=1024 Jul 30 15:51:52.502315 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:51:52.502225 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7j468_a2197c57-2dc6-410e-9910-a045dedcb8d9/konnectivity-agent/0.log" limit=1024 Jul 30 15:51:52.847377 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:51:52.847286 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7j468_a2197c57-2dc6-410e-9910-a045dedcb8d9/konnectivity-agent/0.log" limit=1024 Jul 30 15:51:52.949797 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:51:52.949760 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7j468_a2197c57-2dc6-410e-9910-a045dedcb8d9/konnectivity-agent/0.log" limit=1024 Jul 30 15:52:16.146761 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:52:16.146668 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7j468_a2197c57-2dc6-410e-9910-a045dedcb8d9/konnectivity-agent/0.log" limit=1024 Jul 30 15:52:24.884440 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:52:24.884405 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7j468_a2197c57-2dc6-410e-9910-a045dedcb8d9/konnectivity-agent/0.log" limit=1024 Jul 30 15:52:25.990945 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:52:25.990908 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7j468_a2197c57-2dc6-410e-9910-a045dedcb8d9/konnectivity-agent/0.log" limit=1024 Jul 30 15:53:22.151983 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:53:22.151937 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7j468_a2197c57-2dc6-410e-9910-a045dedcb8d9/konnectivity-agent/0.log" limit=1024 Jul 30 15:53:23.767104 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:53:23.767064 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7j468_a2197c57-2dc6-410e-9910-a045dedcb8d9/konnectivity-agent/0.log" limit=1024 Jul 30 15:53:25.820215 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:53:25.820179 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7j468_a2197c57-2dc6-410e-9910-a045dedcb8d9/konnectivity-agent/0.log" limit=1024 Jul 30 15:53:30.862991 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:53:30.862961 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7j468_a2197c57-2dc6-410e-9910-a045dedcb8d9/konnectivity-agent/0.log" limit=1024 Jul 30 15:54:07.088538 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:54:07.088503 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7j468_a2197c57-2dc6-410e-9910-a045dedcb8d9/konnectivity-agent/0.log" limit=1024 Jul 30 15:54:08.607346 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:54:08.607309 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7j468_a2197c57-2dc6-410e-9910-a045dedcb8d9/konnectivity-agent/0.log" limit=1024 Jul 30 15:54:14.535433 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:54:14.535389 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7j468_a2197c57-2dc6-410e-9910-a045dedcb8d9/konnectivity-agent/0.log" limit=1024 Jul 30 15:54:14.674569 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:54:14.674529 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7j468_a2197c57-2dc6-410e-9910-a045dedcb8d9/konnectivity-agent/0.log" limit=1024 Jul 30 15:54:14.881512 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:54:14.881406 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7j468_a2197c57-2dc6-410e-9910-a045dedcb8d9/konnectivity-agent/0.log" limit=1024 Jul 30 15:54:15.080838 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:54:15.080801 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7j468_a2197c57-2dc6-410e-9910-a045dedcb8d9/konnectivity-agent/0.log" limit=1024 Jul 30 15:54:36.807641 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:54:36.807608 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7j468_a2197c57-2dc6-410e-9910-a045dedcb8d9/konnectivity-agent/0.log" limit=1024 Jul 30 15:54:36.901885 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:54:36.901839 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7j468_a2197c57-2dc6-410e-9910-a045dedcb8d9/konnectivity-agent/0.log" limit=1024 Jul 30 15:54:37.007095 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:54:37.007064 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7j468_a2197c57-2dc6-410e-9910-a045dedcb8d9/konnectivity-agent/0.log" limit=1024 Jul 30 15:54:37.736391 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:54:37.736357 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7j468_a2197c57-2dc6-410e-9910-a045dedcb8d9/konnectivity-agent/0.log" limit=1024 Jul 30 15:54:38.692278 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:54:38.692238 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7j468_a2197c57-2dc6-410e-9910-a045dedcb8d9/konnectivity-agent/0.log" limit=1024 Jul 30 15:54:38.810330 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:54:38.810296 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7j468_a2197c57-2dc6-410e-9910-a045dedcb8d9/konnectivity-agent/0.log" limit=1024 Jul 30 15:54:39.335838 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:54:39.335800 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7j468_a2197c57-2dc6-410e-9910-a045dedcb8d9/konnectivity-agent/0.log" limit=1024 Jul 30 15:54:47.472304 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:54:47.472264 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7j468_a2197c57-2dc6-410e-9910-a045dedcb8d9/konnectivity-agent/0.log" limit=1024 Jul 30 15:54:47.575732 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:54:47.575696 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7j468_a2197c57-2dc6-410e-9910-a045dedcb8d9/konnectivity-agent/0.log" limit=1024 Jul 30 15:54:54.491803 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:54:54.491741 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7j468_a2197c57-2dc6-410e-9910-a045dedcb8d9/konnectivity-agent/0.log" limit=1024 Jul 30 15:54:55.124098 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:54:55.124067 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7j468_a2197c57-2dc6-410e-9910-a045dedcb8d9/konnectivity-agent/0.log" limit=1024 Jul 30 15:54:55.280084 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:54:55.280037 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7j468_a2197c57-2dc6-410e-9910-a045dedcb8d9/konnectivity-agent/0.log" limit=1024 Jul 30 15:55:03.633789 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:55:03.633752 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7j468_a2197c57-2dc6-410e-9910-a045dedcb8d9/konnectivity-agent/0.log" limit=1024 Jul 30 15:55:03.734094 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:55:03.734059 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7j468_a2197c57-2dc6-410e-9910-a045dedcb8d9/konnectivity-agent/0.log" limit=1024 Jul 30 15:55:05.589222 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:55:05.589182 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7j468_a2197c57-2dc6-410e-9910-a045dedcb8d9/konnectivity-agent/0.log" limit=1024 Jul 30 15:55:05.691256 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:55:05.691224 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7j468_a2197c57-2dc6-410e-9910-a045dedcb8d9/konnectivity-agent/0.log" limit=1024 Jul 30 15:55:26.587839 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:55:26.587757 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7j468_a2197c57-2dc6-410e-9910-a045dedcb8d9/konnectivity-agent/0.log" limit=1024 Jul 30 15:55:34.229622 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:55:34.229584 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7j468_a2197c57-2dc6-410e-9910-a045dedcb8d9/konnectivity-agent/0.log" limit=1024 Jul 30 15:55:34.330509 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:55:34.330468 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7j468_a2197c57-2dc6-410e-9910-a045dedcb8d9/konnectivity-agent/0.log" limit=1024 Jul 30 15:55:34.872559 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:55:34.872519 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7j468_a2197c57-2dc6-410e-9910-a045dedcb8d9/konnectivity-agent/0.log" limit=1024 Jul 30 15:55:34.994695 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:55:34.994665 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7j468_a2197c57-2dc6-410e-9910-a045dedcb8d9/konnectivity-agent/0.log" limit=1024 Jul 30 15:55:35.503947 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:55:35.503912 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7j468_a2197c57-2dc6-410e-9910-a045dedcb8d9/konnectivity-agent/0.log" limit=1024 Jul 30 15:55:35.740288 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:55:35.740248 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7j468_a2197c57-2dc6-410e-9910-a045dedcb8d9/konnectivity-agent/0.log" limit=1024 Jul 30 15:55:38.224065 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:55:38.224035 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7j468_a2197c57-2dc6-410e-9910-a045dedcb8d9/konnectivity-agent/0.log" limit=1024 Jul 30 15:55:38.333575 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:55:38.333540 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7j468_a2197c57-2dc6-410e-9910-a045dedcb8d9/konnectivity-agent/0.log" limit=1024 Jul 30 15:55:39.406236 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:55:39.406202 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7j468_a2197c57-2dc6-410e-9910-a045dedcb8d9/konnectivity-agent/0.log" limit=1024 Jul 30 15:55:39.578237 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:55:39.578205 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7j468_a2197c57-2dc6-410e-9910-a045dedcb8d9/konnectivity-agent/0.log" limit=1024 Jul 30 15:55:40.178870 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:55:40.178837 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7j468_a2197c57-2dc6-410e-9910-a045dedcb8d9/konnectivity-agent/0.log" limit=1024 Jul 30 15:55:40.276606 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:55:40.276571 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7j468_a2197c57-2dc6-410e-9910-a045dedcb8d9/konnectivity-agent/0.log" limit=1024 Jul 30 15:56:03.299941 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:56:03.299906 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7j468_a2197c57-2dc6-410e-9910-a045dedcb8d9/konnectivity-agent/0.log" limit=1024 Jul 30 15:56:03.702002 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:56:03.701904 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7j468_a2197c57-2dc6-410e-9910-a045dedcb8d9/konnectivity-agent/0.log" limit=1024 Jul 30 15:56:04.098722 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:56:04.098686 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7j468_a2197c57-2dc6-410e-9910-a045dedcb8d9/konnectivity-agent/0.log" limit=1024 Jul 30 15:56:04.306517 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:56:04.306468 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7j468_a2197c57-2dc6-410e-9910-a045dedcb8d9/konnectivity-agent/0.log" limit=1024 Jul 30 15:56:07.361446 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:56:07.361398 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7j468_a2197c57-2dc6-410e-9910-a045dedcb8d9/konnectivity-agent/0.log" limit=1024 Jul 30 15:56:07.481326 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:56:07.481295 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7j468_a2197c57-2dc6-410e-9910-a045dedcb8d9/konnectivity-agent/0.log" limit=1024 Jul 30 15:56:08.485346 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:56:08.485294 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7j468_a2197c57-2dc6-410e-9910-a045dedcb8d9/konnectivity-agent/0.log" limit=1024 Jul 30 15:56:08.719485 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:56:08.719454 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7j468_a2197c57-2dc6-410e-9910-a045dedcb8d9/konnectivity-agent/0.log" limit=1024 Jul 30 15:56:09.046492 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:56:09.046445 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7j468_a2197c57-2dc6-410e-9910-a045dedcb8d9/konnectivity-agent/0.log" limit=1024 Jul 30 15:56:09.150209 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:56:09.150169 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7j468_a2197c57-2dc6-410e-9910-a045dedcb8d9/konnectivity-agent/0.log" limit=1024 Jul 30 15:56:37.568232 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:56:37.568194 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7j468_a2197c57-2dc6-410e-9910-a045dedcb8d9/konnectivity-agent/0.log" limit=1024 Jul 30 15:56:41.361427 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:56:41.361324 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7j468_a2197c57-2dc6-410e-9910-a045dedcb8d9/konnectivity-agent/0.log" limit=1024 Jul 30 15:57:25.542522 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:57:25.542486 2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-inf-b-762215/test-model-762215-kserve-5bb7d5677f-6x8dq"] Jul 30 15:57:25.543095 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:57:25.542917 2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0a8d4006-7747-4036-9b25-16ade9760013" containerName="console" Jul 30 15:57:25.543095 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:57:25.542936 2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="0a8d4006-7747-4036-9b25-16ade9760013" containerName="console" Jul 30 15:57:25.543095 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:57:25.543036 2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="0a8d4006-7747-4036-9b25-16ade9760013" containerName="console" Jul 30 15:57:25.545866 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:57:25.545845 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-762215/test-model-762215-kserve-5bb7d5677f-6x8dq" Jul 30 15:57:25.549658 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:57:25.549631 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-b-762215\"/\"default-dockercfg-49bcf\"" Jul 30 15:57:25.549768 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:57:25.549680 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-b-762215\"/\"test-model-762215-kserve-self-signed-certs\"" Jul 30 15:57:25.549768 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:57:25.549701 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-b-762215\"/\"openshift-service-ca.crt\"" Jul 30 15:57:25.549768 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:57:25.549680 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-b-762215\"/\"kube-root-ca.crt\"" Jul 30 15:57:25.563284 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:57:25.563262 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-b-762215/test-model-762215-kserve-5bb7d5677f-6x8dq"] Jul 30 15:57:25.592644 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:57:25.592610 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7j468_a2197c57-2dc6-410e-9910-a045dedcb8d9/konnectivity-agent/0.log" limit=1024 Jul 30 15:57:25.680655 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:57:25.680612 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/07dfe956-68ef-41b6-ab18-cfd595b99705-home\") pod \"test-model-762215-kserve-5bb7d5677f-6x8dq\" (UID: \"07dfe956-68ef-41b6-ab18-cfd595b99705\") " pod="ai-tenant-e2e-inf-b-762215/test-model-762215-kserve-5bb7d5677f-6x8dq" Jul 30 15:57:25.680839 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:57:25.680674 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/07dfe956-68ef-41b6-ab18-cfd595b99705-model-cache\") pod \"test-model-762215-kserve-5bb7d5677f-6x8dq\" (UID: \"07dfe956-68ef-41b6-ab18-cfd595b99705\") " pod="ai-tenant-e2e-inf-b-762215/test-model-762215-kserve-5bb7d5677f-6x8dq" Jul 30 15:57:25.680839 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:57:25.680755 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/07dfe956-68ef-41b6-ab18-cfd595b99705-dshm\") pod \"test-model-762215-kserve-5bb7d5677f-6x8dq\" (UID: \"07dfe956-68ef-41b6-ab18-cfd595b99705\") " pod="ai-tenant-e2e-inf-b-762215/test-model-762215-kserve-5bb7d5677f-6x8dq" Jul 30 15:57:25.680839 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:57:25.680829 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/07dfe956-68ef-41b6-ab18-cfd595b99705-tmp-dir\") pod \"test-model-762215-kserve-5bb7d5677f-6x8dq\" (UID: \"07dfe956-68ef-41b6-ab18-cfd595b99705\") " pod="ai-tenant-e2e-inf-b-762215/test-model-762215-kserve-5bb7d5677f-6x8dq" Jul 30 15:57:25.680945 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:57:25.680854 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/07dfe956-68ef-41b6-ab18-cfd595b99705-tls-certs\") pod \"test-model-762215-kserve-5bb7d5677f-6x8dq\" (UID: \"07dfe956-68ef-41b6-ab18-cfd595b99705\") " pod="ai-tenant-e2e-inf-b-762215/test-model-762215-kserve-5bb7d5677f-6x8dq" Jul 30 15:57:25.680945 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:57:25.680883 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tth9g\" (UniqueName: \"kubernetes.io/projected/07dfe956-68ef-41b6-ab18-cfd595b99705-kube-api-access-tth9g\") pod \"test-model-762215-kserve-5bb7d5677f-6x8dq\" (UID: \"07dfe956-68ef-41b6-ab18-cfd595b99705\") " pod="ai-tenant-e2e-inf-b-762215/test-model-762215-kserve-5bb7d5677f-6x8dq" Jul 30 15:57:25.781658 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:57:25.781617 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/07dfe956-68ef-41b6-ab18-cfd595b99705-dshm\") pod \"test-model-762215-kserve-5bb7d5677f-6x8dq\" (UID: \"07dfe956-68ef-41b6-ab18-cfd595b99705\") " pod="ai-tenant-e2e-inf-b-762215/test-model-762215-kserve-5bb7d5677f-6x8dq" Jul 30 15:57:25.781658 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:57:25.781665 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/07dfe956-68ef-41b6-ab18-cfd595b99705-tmp-dir\") pod \"test-model-762215-kserve-5bb7d5677f-6x8dq\" (UID: \"07dfe956-68ef-41b6-ab18-cfd595b99705\") " pod="ai-tenant-e2e-inf-b-762215/test-model-762215-kserve-5bb7d5677f-6x8dq" Jul 30 15:57:25.781913 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:57:25.781682 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/07dfe956-68ef-41b6-ab18-cfd595b99705-tls-certs\") pod \"test-model-762215-kserve-5bb7d5677f-6x8dq\" (UID: \"07dfe956-68ef-41b6-ab18-cfd595b99705\") " pod="ai-tenant-e2e-inf-b-762215/test-model-762215-kserve-5bb7d5677f-6x8dq" Jul 30 15:57:25.781913 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:57:25.781710 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tth9g\" (UniqueName: \"kubernetes.io/projected/07dfe956-68ef-41b6-ab18-cfd595b99705-kube-api-access-tth9g\") pod \"test-model-762215-kserve-5bb7d5677f-6x8dq\" (UID: \"07dfe956-68ef-41b6-ab18-cfd595b99705\") " pod="ai-tenant-e2e-inf-b-762215/test-model-762215-kserve-5bb7d5677f-6x8dq" Jul 30 15:57:25.781913 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:57:25.781763 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/07dfe956-68ef-41b6-ab18-cfd595b99705-home\") pod \"test-model-762215-kserve-5bb7d5677f-6x8dq\" (UID: \"07dfe956-68ef-41b6-ab18-cfd595b99705\") " pod="ai-tenant-e2e-inf-b-762215/test-model-762215-kserve-5bb7d5677f-6x8dq" Jul 30 15:57:25.781913 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:57:25.781812 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/07dfe956-68ef-41b6-ab18-cfd595b99705-model-cache\") pod \"test-model-762215-kserve-5bb7d5677f-6x8dq\" (UID: \"07dfe956-68ef-41b6-ab18-cfd595b99705\") " pod="ai-tenant-e2e-inf-b-762215/test-model-762215-kserve-5bb7d5677f-6x8dq" Jul 30 15:57:25.782230 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:57:25.782094 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/07dfe956-68ef-41b6-ab18-cfd595b99705-tmp-dir\") pod \"test-model-762215-kserve-5bb7d5677f-6x8dq\" (UID: \"07dfe956-68ef-41b6-ab18-cfd595b99705\") " pod="ai-tenant-e2e-inf-b-762215/test-model-762215-kserve-5bb7d5677f-6x8dq" Jul 30 15:57:25.782230 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:57:25.782192 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/07dfe956-68ef-41b6-ab18-cfd595b99705-model-cache\") pod \"test-model-762215-kserve-5bb7d5677f-6x8dq\" (UID: \"07dfe956-68ef-41b6-ab18-cfd595b99705\") " pod="ai-tenant-e2e-inf-b-762215/test-model-762215-kserve-5bb7d5677f-6x8dq" Jul 30 15:57:25.782315 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:57:25.782248 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/07dfe956-68ef-41b6-ab18-cfd595b99705-home\") pod \"test-model-762215-kserve-5bb7d5677f-6x8dq\" (UID: \"07dfe956-68ef-41b6-ab18-cfd595b99705\") " pod="ai-tenant-e2e-inf-b-762215/test-model-762215-kserve-5bb7d5677f-6x8dq" Jul 30 15:57:25.784014 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:57:25.783994 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/07dfe956-68ef-41b6-ab18-cfd595b99705-dshm\") pod \"test-model-762215-kserve-5bb7d5677f-6x8dq\" (UID: \"07dfe956-68ef-41b6-ab18-cfd595b99705\") " pod="ai-tenant-e2e-inf-b-762215/test-model-762215-kserve-5bb7d5677f-6x8dq" Jul 30 15:57:25.784205 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:57:25.784188 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/07dfe956-68ef-41b6-ab18-cfd595b99705-tls-certs\") pod \"test-model-762215-kserve-5bb7d5677f-6x8dq\" (UID: \"07dfe956-68ef-41b6-ab18-cfd595b99705\") " pod="ai-tenant-e2e-inf-b-762215/test-model-762215-kserve-5bb7d5677f-6x8dq" Jul 30 15:57:25.790472 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:57:25.790451 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tth9g\" (UniqueName: \"kubernetes.io/projected/07dfe956-68ef-41b6-ab18-cfd595b99705-kube-api-access-tth9g\") pod \"test-model-762215-kserve-5bb7d5677f-6x8dq\" (UID: \"07dfe956-68ef-41b6-ab18-cfd595b99705\") " pod="ai-tenant-e2e-inf-b-762215/test-model-762215-kserve-5bb7d5677f-6x8dq" Jul 30 15:57:25.855718 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:57:25.855618 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-762215/test-model-762215-kserve-5bb7d5677f-6x8dq" Jul 30 15:57:25.986844 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:57:25.986818 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-b-762215/test-model-762215-kserve-5bb7d5677f-6x8dq"] Jul 30 15:57:25.989253 ip-10-0-133-210 kubenswrapper[2576]: W0730 15:57:25.989224 2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod07dfe956_68ef_41b6_ab18_cfd595b99705.slice/crio-30d3dcf48c6e598cc51d9a78d0bc5e175ea75fc8f582aad6ac18d4537d771f3d WatchSource:0}: Error finding container 30d3dcf48c6e598cc51d9a78d0bc5e175ea75fc8f582aad6ac18d4537d771f3d: Status 404 returned error can't find the container with id 30d3dcf48c6e598cc51d9a78d0bc5e175ea75fc8f582aad6ac18d4537d771f3d Jul 30 15:57:25.991287 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:57:25.991263 2576 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 30 15:57:26.224036 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:57:26.224007 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-762215/test-model-762215-kserve-5bb7d5677f-6x8dq" event={"ID":"07dfe956-68ef-41b6-ab18-cfd595b99705","Type":"ContainerStarted","Data":"30d3dcf48c6e598cc51d9a78d0bc5e175ea75fc8f582aad6ac18d4537d771f3d"} Jul 30 15:57:27.230477 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:57:27.230435 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-762215/test-model-762215-kserve-5bb7d5677f-6x8dq" event={"ID":"07dfe956-68ef-41b6-ab18-cfd595b99705","Type":"ContainerStarted","Data":"5c3733fae99d81026de1d81b2ee6af5979aff553add44bf4af1b5f2d99fb5cf9"} Jul 30 15:57:27.250413 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:57:27.250361 2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-inf-b-762215/test-model-762215-kserve-5bb7d5677f-6x8dq" podStartSLOduration=1.148177814 podStartE2EDuration="2.250345887s" totalImagesPullingTime="1.102168073s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 15:57:25 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 15:57:27.24929316 +0000 UTC m=+2829.175046010" watchObservedRunningTime="2026-07-30 15:57:27.250345887 +0000 UTC m=+2829.176098532" Jul 30 15:57:35.855844 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:57:35.855808 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-inf-b-762215/test-model-762215-kserve-5bb7d5677f-6x8dq" Jul 30 15:57:35.856274 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:57:35.856120 2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-inf-b-762215/test-model-762215-kserve-5bb7d5677f-6x8dq" Jul 30 15:57:35.868244 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:57:35.868224 2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-inf-b-762215/test-model-762215-kserve-5bb7d5677f-6x8dq" Jul 30 15:57:36.278733 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:57:36.278708 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-inf-b-762215/test-model-762215-kserve-5bb7d5677f-6x8dq" Jul 30 15:58:11.980755 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:58:11.980661 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7j468_a2197c57-2dc6-410e-9910-a045dedcb8d9/konnectivity-agent/0.log" limit=1024 Jul 30 15:58:12.048119 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:58:12.048077 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7j468_a2197c57-2dc6-410e-9910-a045dedcb8d9/konnectivity-agent/0.log" limit=1024 Jul 30 15:58:23.729370 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:58:23.729339 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7j468_a2197c57-2dc6-410e-9910-a045dedcb8d9/konnectivity-agent/0.log" limit=1024 Jul 30 15:58:24.660690 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:58:24.660654 2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-b-762215/test-model-762215-kserve-5bb7d5677f-6x8dq"] Jul 30 15:58:25.217772 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:58:25.217735 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7j468_a2197c57-2dc6-410e-9910-a045dedcb8d9/konnectivity-agent/0.log" limit=1024 Jul 30 15:58:35.275981 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:58:35.275937 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7j468_a2197c57-2dc6-410e-9910-a045dedcb8d9/konnectivity-agent/0.log" limit=1024 Jul 30 15:58:36.090004 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:58:36.089959 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7j468_a2197c57-2dc6-410e-9910-a045dedcb8d9/konnectivity-agent/0.log" limit=1024 Jul 30 15:58:36.184734 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:58:36.184695 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7j468_a2197c57-2dc6-410e-9910-a045dedcb8d9/konnectivity-agent/0.log" limit=1024 Jul 30 15:58:39.679263 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:58:39.679223 2576 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-inf-b-762215/test-model-762215-kserve-5bb7d5677f-6x8dq" podUID="07dfe956-68ef-41b6-ab18-cfd595b99705" containerName="main" containerID="cri-o://5c3733fae99d81026de1d81b2ee6af5979aff553add44bf4af1b5f2d99fb5cf9" gracePeriod=45 Jul 30 15:58:39.921345 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:58:39.921320 2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-762215/test-model-762215-kserve-5bb7d5677f-6x8dq" Jul 30 15:58:40.006797 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:58:40.006763 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-tth9g\" (UniqueName: \"kubernetes.io/projected/07dfe956-68ef-41b6-ab18-cfd595b99705-kube-api-access-tth9g\") pod \"07dfe956-68ef-41b6-ab18-cfd595b99705\" (UID: \"07dfe956-68ef-41b6-ab18-cfd595b99705\") " Jul 30 15:58:40.006797 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:58:40.006811 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/07dfe956-68ef-41b6-ab18-cfd595b99705-dshm\") pod \"07dfe956-68ef-41b6-ab18-cfd595b99705\" (UID: \"07dfe956-68ef-41b6-ab18-cfd595b99705\") " Jul 30 15:58:40.007068 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:58:40.006859 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/07dfe956-68ef-41b6-ab18-cfd595b99705-model-cache\") pod \"07dfe956-68ef-41b6-ab18-cfd595b99705\" (UID: \"07dfe956-68ef-41b6-ab18-cfd595b99705\") " Jul 30 15:58:40.007068 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:58:40.007021 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/07dfe956-68ef-41b6-ab18-cfd595b99705-tmp-dir\") pod \"07dfe956-68ef-41b6-ab18-cfd595b99705\" (UID: \"07dfe956-68ef-41b6-ab18-cfd595b99705\") " Jul 30 15:58:40.007068 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:58:40.007062 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/07dfe956-68ef-41b6-ab18-cfd595b99705-home\") pod \"07dfe956-68ef-41b6-ab18-cfd595b99705\" (UID: \"07dfe956-68ef-41b6-ab18-cfd595b99705\") " Jul 30 15:58:40.007262 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:58:40.007087 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/07dfe956-68ef-41b6-ab18-cfd595b99705-tls-certs\") pod \"07dfe956-68ef-41b6-ab18-cfd595b99705\" (UID: \"07dfe956-68ef-41b6-ab18-cfd595b99705\") " Jul 30 15:58:40.007262 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:58:40.007084 2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/07dfe956-68ef-41b6-ab18-cfd595b99705-model-cache" (OuterVolumeSpecName: "model-cache") pod "07dfe956-68ef-41b6-ab18-cfd595b99705" (UID: "07dfe956-68ef-41b6-ab18-cfd595b99705"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 30 15:58:40.007386 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:58:40.007321 2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/07dfe956-68ef-41b6-ab18-cfd595b99705-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "07dfe956-68ef-41b6-ab18-cfd595b99705" (UID: "07dfe956-68ef-41b6-ab18-cfd595b99705"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 30 15:58:40.007386 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:58:40.007332 2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/07dfe956-68ef-41b6-ab18-cfd595b99705-home" (OuterVolumeSpecName: "home") pod "07dfe956-68ef-41b6-ab18-cfd595b99705" (UID: "07dfe956-68ef-41b6-ab18-cfd595b99705"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 30 15:58:40.007493 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:58:40.007423 2576 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/07dfe956-68ef-41b6-ab18-cfd595b99705-home\") on node \"ip-10-0-133-210.ec2.internal\" DevicePath \"\"" Jul 30 15:58:40.007493 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:58:40.007442 2576 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/07dfe956-68ef-41b6-ab18-cfd595b99705-model-cache\") on node \"ip-10-0-133-210.ec2.internal\" DevicePath \"\"" Jul 30 15:58:40.007493 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:58:40.007475 2576 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/07dfe956-68ef-41b6-ab18-cfd595b99705-tmp-dir\") on node \"ip-10-0-133-210.ec2.internal\" DevicePath \"\"" Jul 30 15:58:40.009073 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:58:40.009038 2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/07dfe956-68ef-41b6-ab18-cfd595b99705-dshm" (OuterVolumeSpecName: "dshm") pod "07dfe956-68ef-41b6-ab18-cfd595b99705" (UID: "07dfe956-68ef-41b6-ab18-cfd595b99705"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 30 15:58:40.009193 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:58:40.009102 2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/07dfe956-68ef-41b6-ab18-cfd595b99705-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "07dfe956-68ef-41b6-ab18-cfd595b99705" (UID: "07dfe956-68ef-41b6-ab18-cfd595b99705"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 30 15:58:40.009193 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:58:40.009139 2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/07dfe956-68ef-41b6-ab18-cfd595b99705-kube-api-access-tth9g" (OuterVolumeSpecName: "kube-api-access-tth9g") pod "07dfe956-68ef-41b6-ab18-cfd595b99705" (UID: "07dfe956-68ef-41b6-ab18-cfd595b99705"). InnerVolumeSpecName "kube-api-access-tth9g". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 30 15:58:40.107934 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:58:40.107898 2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-tth9g\" (UniqueName: \"kubernetes.io/projected/07dfe956-68ef-41b6-ab18-cfd595b99705-kube-api-access-tth9g\") on node \"ip-10-0-133-210.ec2.internal\" DevicePath \"\"" Jul 30 15:58:40.107934 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:58:40.107926 2576 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/07dfe956-68ef-41b6-ab18-cfd595b99705-dshm\") on node \"ip-10-0-133-210.ec2.internal\" DevicePath \"\"" Jul 30 15:58:40.107934 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:58:40.107937 2576 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/07dfe956-68ef-41b6-ab18-cfd595b99705-tls-certs\") on node \"ip-10-0-133-210.ec2.internal\" DevicePath \"\"" Jul 30 15:58:40.533539 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:58:40.533504 2576 generic.go:356] "Generic (PLEG): container finished" podID="07dfe956-68ef-41b6-ab18-cfd595b99705" containerID="5c3733fae99d81026de1d81b2ee6af5979aff553add44bf4af1b5f2d99fb5cf9" exitCode=0 Jul 30 15:58:40.533743 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:58:40.533551 2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-762215/test-model-762215-kserve-5bb7d5677f-6x8dq" Jul 30 15:58:40.533743 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:58:40.533585 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-762215/test-model-762215-kserve-5bb7d5677f-6x8dq" event={"ID":"07dfe956-68ef-41b6-ab18-cfd595b99705","Type":"ContainerDied","Data":"5c3733fae99d81026de1d81b2ee6af5979aff553add44bf4af1b5f2d99fb5cf9"} Jul 30 15:58:40.533743 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:58:40.533630 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-762215/test-model-762215-kserve-5bb7d5677f-6x8dq" event={"ID":"07dfe956-68ef-41b6-ab18-cfd595b99705","Type":"ContainerDied","Data":"30d3dcf48c6e598cc51d9a78d0bc5e175ea75fc8f582aad6ac18d4537d771f3d"} Jul 30 15:58:40.533743 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:58:40.533647 2576 scope.go:117] "RemoveContainer" containerID="5c3733fae99d81026de1d81b2ee6af5979aff553add44bf4af1b5f2d99fb5cf9" Jul 30 15:58:40.542700 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:58:40.542678 2576 scope.go:117] "RemoveContainer" containerID="5c3733fae99d81026de1d81b2ee6af5979aff553add44bf4af1b5f2d99fb5cf9" Jul 30 15:58:40.542985 ip-10-0-133-210 kubenswrapper[2576]: E0730 15:58:40.542967 2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5c3733fae99d81026de1d81b2ee6af5979aff553add44bf4af1b5f2d99fb5cf9\": container with ID starting with 5c3733fae99d81026de1d81b2ee6af5979aff553add44bf4af1b5f2d99fb5cf9 not found: ID does not exist" containerID="5c3733fae99d81026de1d81b2ee6af5979aff553add44bf4af1b5f2d99fb5cf9" Jul 30 15:58:40.543022 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:58:40.542996 2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5c3733fae99d81026de1d81b2ee6af5979aff553add44bf4af1b5f2d99fb5cf9"} err="failed to get container status \"5c3733fae99d81026de1d81b2ee6af5979aff553add44bf4af1b5f2d99fb5cf9\": rpc error: code = NotFound desc = could not find container \"5c3733fae99d81026de1d81b2ee6af5979aff553add44bf4af1b5f2d99fb5cf9\": container with ID starting with 5c3733fae99d81026de1d81b2ee6af5979aff553add44bf4af1b5f2d99fb5cf9 not found: ID does not exist" Jul 30 15:58:40.556071 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:58:40.556043 2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-b-762215/test-model-762215-kserve-5bb7d5677f-6x8dq"] Jul 30 15:58:40.560743 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:58:40.560720 2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-inf-b-762215/test-model-762215-kserve-5bb7d5677f-6x8dq"] Jul 30 15:58:40.613023 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:58:40.612991 2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="07dfe956-68ef-41b6-ab18-cfd595b99705" path="/var/lib/kubelet/pods/07dfe956-68ef-41b6-ab18-cfd595b99705/volumes" Jul 30 15:58:41.660435 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:58:41.660394 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7j468_a2197c57-2dc6-410e-9910-a045dedcb8d9/konnectivity-agent/0.log" limit=1024 Jul 30 15:58:41.761301 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:58:41.761255 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7j468_a2197c57-2dc6-410e-9910-a045dedcb8d9/konnectivity-agent/0.log" limit=1024 Jul 30 15:58:41.939045 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:58:41.938956 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7j468_a2197c57-2dc6-410e-9910-a045dedcb8d9/konnectivity-agent/0.log" limit=1024 Jul 30 15:58:42.174552 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:58:42.174510 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7j468_a2197c57-2dc6-410e-9910-a045dedcb8d9/konnectivity-agent/0.log" limit=1024 Jul 30 15:58:52.806081 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:58:52.806037 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7j468_a2197c57-2dc6-410e-9910-a045dedcb8d9/konnectivity-agent/0.log" limit=1024 Jul 30 15:58:52.906636 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:58:52.906601 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7j468_a2197c57-2dc6-410e-9910-a045dedcb8d9/konnectivity-agent/0.log" limit=1024 Jul 30 15:58:52.996509 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:58:52.996479 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7j468_a2197c57-2dc6-410e-9910-a045dedcb8d9/konnectivity-agent/0.log" limit=1024 Jul 30 15:58:53.073008 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:58:53.072918 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7j468_a2197c57-2dc6-410e-9910-a045dedcb8d9/konnectivity-agent/0.log" limit=1024 Jul 30 15:58:53.392078 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:58:53.391991 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7j468_a2197c57-2dc6-410e-9910-a045dedcb8d9/konnectivity-agent/0.log" limit=1024 Jul 30 15:58:59.339599 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:58:59.339555 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7j468_a2197c57-2dc6-410e-9910-a045dedcb8d9/konnectivity-agent/0.log" limit=1024 Jul 30 15:58:59.436755 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:58:59.436720 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7j468_a2197c57-2dc6-410e-9910-a045dedcb8d9/konnectivity-agent/0.log" limit=1024 Jul 30 15:58:59.609753 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:58:59.609644 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7j468_a2197c57-2dc6-410e-9910-a045dedcb8d9/konnectivity-agent/0.log" limit=1024 Jul 30 15:58:59.856688 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:58:59.856651 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7j468_a2197c57-2dc6-410e-9910-a045dedcb8d9/konnectivity-agent/0.log" limit=1024 Jul 30 15:59:00.208843 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:59:00.208807 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7j468_a2197c57-2dc6-410e-9910-a045dedcb8d9/konnectivity-agent/0.log" limit=1024 Jul 30 15:59:00.549546 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:59:00.549509 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7j468_a2197c57-2dc6-410e-9910-a045dedcb8d9/konnectivity-agent/0.log" limit=1024 Jul 30 15:59:00.907563 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:59:00.907477 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7j468_a2197c57-2dc6-410e-9910-a045dedcb8d9/konnectivity-agent/0.log" limit=1024 Jul 30 15:59:03.229807 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:59:03.229765 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7j468_a2197c57-2dc6-410e-9910-a045dedcb8d9/konnectivity-agent/0.log" limit=1024 Jul 30 15:59:03.326187 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:59:03.326150 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7j468_a2197c57-2dc6-410e-9910-a045dedcb8d9/konnectivity-agent/0.log" limit=1024 Jul 30 15:59:04.329620 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:59:04.329585 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7j468_a2197c57-2dc6-410e-9910-a045dedcb8d9/konnectivity-agent/0.log" limit=1024 Jul 30 15:59:04.466368 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:59:04.466329 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7j468_a2197c57-2dc6-410e-9910-a045dedcb8d9/konnectivity-agent/0.log" limit=1024 Jul 30 15:59:04.939780 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:59:04.939741 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7j468_a2197c57-2dc6-410e-9910-a045dedcb8d9/konnectivity-agent/0.log" limit=1024 Jul 30 15:59:05.103421 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:59:05.103383 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7j468_a2197c57-2dc6-410e-9910-a045dedcb8d9/konnectivity-agent/0.log" limit=1024 Jul 30 15:59:18.290126 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:59:18.290075 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7j468_a2197c57-2dc6-410e-9910-a045dedcb8d9/konnectivity-agent/0.log" limit=1024 Jul 30 15:59:28.157093 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:59:28.157056 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7j468_a2197c57-2dc6-410e-9910-a045dedcb8d9/konnectivity-agent/0.log" limit=1024 Jul 30 15:59:28.721462 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:59:28.721428 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7j468_a2197c57-2dc6-410e-9910-a045dedcb8d9/konnectivity-agent/0.log" limit=1024 Jul 30 15:59:28.996561 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:59:28.996462 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7j468_a2197c57-2dc6-410e-9910-a045dedcb8d9/konnectivity-agent/0.log" limit=1024 Jul 30 15:59:29.279543 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:59:29.279448 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7j468_a2197c57-2dc6-410e-9910-a045dedcb8d9/konnectivity-agent/0.log" limit=1024 Jul 30 15:59:32.367965 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:59:32.367905 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7j468_a2197c57-2dc6-410e-9910-a045dedcb8d9/konnectivity-agent/0.log" limit=1024 Jul 30 15:59:32.487294 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:59:32.487257 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7j468_a2197c57-2dc6-410e-9910-a045dedcb8d9/konnectivity-agent/0.log" limit=1024 Jul 30 15:59:33.573496 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:59:33.573460 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7j468_a2197c57-2dc6-410e-9910-a045dedcb8d9/konnectivity-agent/0.log" limit=1024 Jul 30 15:59:33.960783 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:59:33.960688 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7j468_a2197c57-2dc6-410e-9910-a045dedcb8d9/konnectivity-agent/0.log" limit=1024 Jul 30 15:59:34.199847 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:59:34.199811 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7j468_a2197c57-2dc6-410e-9910-a045dedcb8d9/konnectivity-agent/0.log" limit=1024 Jul 30 15:59:34.300979 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:59:34.300934 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7j468_a2197c57-2dc6-410e-9910-a045dedcb8d9/konnectivity-agent/0.log" limit=1024 Jul 30 15:59:57.376856 ip-10-0-133-210 kubenswrapper[2576]: I0730 15:59:57.376796 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7j468_a2197c57-2dc6-410e-9910-a045dedcb8d9/konnectivity-agent/0.log" limit=1024 Jul 30 16:00:16.030967 ip-10-0-133-210 kubenswrapper[2576]: I0730 16:00:16.030925 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7j468_a2197c57-2dc6-410e-9910-a045dedcb8d9/konnectivity-agent/0.log" limit=1024 Jul 30 16:00:16.176561 ip-10-0-133-210 kubenswrapper[2576]: I0730 16:00:16.176492 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7j468_a2197c57-2dc6-410e-9910-a045dedcb8d9/konnectivity-agent/0.log" limit=1024 Jul 30 16:00:16.236000 ip-10-0-133-210 kubenswrapper[2576]: I0730 16:00:16.235969 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7j468_a2197c57-2dc6-410e-9910-a045dedcb8d9/konnectivity-agent/0.log" limit=1024 Jul 30 16:00:16.596671 ip-10-0-133-210 kubenswrapper[2576]: I0730 16:00:16.596630 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7j468_a2197c57-2dc6-410e-9910-a045dedcb8d9/konnectivity-agent/0.log" limit=1024 Jul 30 16:00:22.174537 ip-10-0-133-210 kubenswrapper[2576]: I0730 16:00:22.174485 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7j468_a2197c57-2dc6-410e-9910-a045dedcb8d9/konnectivity-agent/0.log" limit=1024 Jul 30 16:00:22.274786 ip-10-0-133-210 kubenswrapper[2576]: I0730 16:00:22.274751 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7j468_a2197c57-2dc6-410e-9910-a045dedcb8d9/konnectivity-agent/0.log" limit=1024 Jul 30 16:00:22.446920 ip-10-0-133-210 kubenswrapper[2576]: I0730 16:00:22.446823 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7j468_a2197c57-2dc6-410e-9910-a045dedcb8d9/konnectivity-agent/0.log" limit=1024 Jul 30 16:00:22.685590 ip-10-0-133-210 kubenswrapper[2576]: I0730 16:00:22.685550 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7j468_a2197c57-2dc6-410e-9910-a045dedcb8d9/konnectivity-agent/0.log" limit=1024 Jul 30 16:00:38.222567 ip-10-0-133-210 kubenswrapper[2576]: I0730 16:00:38.222533 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7j468_a2197c57-2dc6-410e-9910-a045dedcb8d9/konnectivity-agent/0.log" limit=1024 Jul 30 16:00:38.295036 ip-10-0-133-210 kubenswrapper[2576]: I0730 16:00:38.295000 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7j468_a2197c57-2dc6-410e-9910-a045dedcb8d9/konnectivity-agent/0.log" limit=1024 Jul 30 16:00:38.355757 ip-10-0-133-210 kubenswrapper[2576]: I0730 16:00:38.355715 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7j468_a2197c57-2dc6-410e-9910-a045dedcb8d9/konnectivity-agent/0.log" limit=1024 Jul 30 16:00:38.458630 ip-10-0-133-210 kubenswrapper[2576]: I0730 16:00:38.458578 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7j468_a2197c57-2dc6-410e-9910-a045dedcb8d9/konnectivity-agent/0.log" limit=1024 Jul 30 16:00:39.091591 ip-10-0-133-210 kubenswrapper[2576]: I0730 16:00:39.091556 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7j468_a2197c57-2dc6-410e-9910-a045dedcb8d9/konnectivity-agent/0.log" limit=1024 Jul 30 16:00:44.545699 ip-10-0-133-210 kubenswrapper[2576]: I0730 16:00:44.545664 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7j468_a2197c57-2dc6-410e-9910-a045dedcb8d9/konnectivity-agent/0.log" limit=1024 Jul 30 16:00:44.646469 ip-10-0-133-210 kubenswrapper[2576]: I0730 16:00:44.646437 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7j468_a2197c57-2dc6-410e-9910-a045dedcb8d9/konnectivity-agent/0.log" limit=1024 Jul 30 16:00:45.248031 ip-10-0-133-210 kubenswrapper[2576]: I0730 16:00:45.247998 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7j468_a2197c57-2dc6-410e-9910-a045dedcb8d9/konnectivity-agent/0.log" limit=1024 Jul 30 16:00:45.349139 ip-10-0-133-210 kubenswrapper[2576]: I0730 16:00:45.349078 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7j468_a2197c57-2dc6-410e-9910-a045dedcb8d9/konnectivity-agent/0.log" limit=1024 Jul 30 16:00:45.466542 ip-10-0-133-210 kubenswrapper[2576]: I0730 16:00:45.466508 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7j468_a2197c57-2dc6-410e-9910-a045dedcb8d9/konnectivity-agent/0.log" limit=1024 Jul 30 16:00:45.844315 ip-10-0-133-210 kubenswrapper[2576]: I0730 16:00:45.844276 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7j468_a2197c57-2dc6-410e-9910-a045dedcb8d9/konnectivity-agent/0.log" limit=1024 Jul 30 16:00:46.039002 ip-10-0-133-210 kubenswrapper[2576]: I0730 16:00:46.038960 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7j468_a2197c57-2dc6-410e-9910-a045dedcb8d9/konnectivity-agent/0.log" limit=1024 Jul 30 16:00:48.715037 ip-10-0-133-210 kubenswrapper[2576]: I0730 16:00:48.715005 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7j468_a2197c57-2dc6-410e-9910-a045dedcb8d9/konnectivity-agent/0.log" limit=1024 Jul 30 16:00:48.814416 ip-10-0-133-210 kubenswrapper[2576]: I0730 16:00:48.814378 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7j468_a2197c57-2dc6-410e-9910-a045dedcb8d9/konnectivity-agent/0.log" limit=1024 Jul 30 16:00:49.822611 ip-10-0-133-210 kubenswrapper[2576]: I0730 16:00:49.822567 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7j468_a2197c57-2dc6-410e-9910-a045dedcb8d9/konnectivity-agent/0.log" limit=1024 Jul 30 16:00:49.919897 ip-10-0-133-210 kubenswrapper[2576]: I0730 16:00:49.919858 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7j468_a2197c57-2dc6-410e-9910-a045dedcb8d9/konnectivity-agent/0.log" limit=1024 Jul 30 16:00:50.358075 ip-10-0-133-210 kubenswrapper[2576]: I0730 16:00:50.358023 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7j468_a2197c57-2dc6-410e-9910-a045dedcb8d9/konnectivity-agent/0.log" limit=1024 Jul 30 16:00:50.430442 ip-10-0-133-210 kubenswrapper[2576]: I0730 16:00:50.430411 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7j468_a2197c57-2dc6-410e-9910-a045dedcb8d9/konnectivity-agent/0.log" limit=1024 Jul 30 16:01:13.789949 ip-10-0-133-210 kubenswrapper[2576]: I0730 16:01:13.789860 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7j468_a2197c57-2dc6-410e-9910-a045dedcb8d9/konnectivity-agent/0.log" limit=1024 Jul 30 16:01:14.172371 ip-10-0-133-210 kubenswrapper[2576]: I0730 16:01:14.172257 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7j468_a2197c57-2dc6-410e-9910-a045dedcb8d9/konnectivity-agent/0.log" limit=1024 Jul 30 16:01:14.490444 ip-10-0-133-210 kubenswrapper[2576]: I0730 16:01:14.490406 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7j468_a2197c57-2dc6-410e-9910-a045dedcb8d9/konnectivity-agent/0.log" limit=1024 Jul 30 16:01:15.127473 ip-10-0-133-210 kubenswrapper[2576]: I0730 16:01:15.127439 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7j468_a2197c57-2dc6-410e-9910-a045dedcb8d9/konnectivity-agent/0.log" limit=1024 Jul 30 16:01:17.917279 ip-10-0-133-210 kubenswrapper[2576]: I0730 16:01:17.917241 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7j468_a2197c57-2dc6-410e-9910-a045dedcb8d9/konnectivity-agent/0.log" limit=1024 Jul 30 16:01:17.982605 ip-10-0-133-210 kubenswrapper[2576]: I0730 16:01:17.982565 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7j468_a2197c57-2dc6-410e-9910-a045dedcb8d9/konnectivity-agent/0.log" limit=1024 Jul 30 16:01:19.129001 ip-10-0-133-210 kubenswrapper[2576]: I0730 16:01:19.128971 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7j468_a2197c57-2dc6-410e-9910-a045dedcb8d9/konnectivity-agent/0.log" limit=1024 Jul 30 16:01:19.442145 ip-10-0-133-210 kubenswrapper[2576]: I0730 16:01:19.442053 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7j468_a2197c57-2dc6-410e-9910-a045dedcb8d9/konnectivity-agent/0.log" limit=1024 Jul 30 16:01:19.553013 ip-10-0-133-210 kubenswrapper[2576]: I0730 16:01:19.552979 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7j468_a2197c57-2dc6-410e-9910-a045dedcb8d9/konnectivity-agent/0.log" limit=1024 Jul 30 16:01:19.671343 ip-10-0-133-210 kubenswrapper[2576]: I0730 16:01:19.671312 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7j468_a2197c57-2dc6-410e-9910-a045dedcb8d9/konnectivity-agent/0.log" limit=1024 Jul 30 16:01:42.962744 ip-10-0-133-210 kubenswrapper[2576]: I0730 16:01:42.962706 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7j468_a2197c57-2dc6-410e-9910-a045dedcb8d9/konnectivity-agent/0.log" limit=1024 Jul 30 16:01:48.226341 ip-10-0-133-210 kubenswrapper[2576]: I0730 16:01:48.226300 2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-a-f2cf4d/auth-test-model-f2cf4d-kserve-59c6ccdbff-2p458"] Jul 30 16:01:48.226840 ip-10-0-133-210 kubenswrapper[2576]: I0730 16:01:48.226635 2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="07dfe956-68ef-41b6-ab18-cfd595b99705" containerName="main" Jul 30 16:01:48.226840 ip-10-0-133-210 kubenswrapper[2576]: I0730 16:01:48.226646 2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="07dfe956-68ef-41b6-ab18-cfd595b99705" containerName="main" Jul 30 16:01:48.226840 ip-10-0-133-210 kubenswrapper[2576]: I0730 16:01:48.226698 2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="07dfe956-68ef-41b6-ab18-cfd595b99705" containerName="main" Jul 30 16:01:48.228453 ip-10-0-133-210 kubenswrapper[2576]: I0730 16:01:48.228432 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-f2cf4d/auth-test-model-f2cf4d-kserve-59c6ccdbff-2p458" Jul 30 16:01:48.231151 ip-10-0-133-210 kubenswrapper[2576]: I0730 16:01:48.231128 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-f2cf4d\"/\"kube-root-ca.crt\"" Jul 30 16:01:48.232360 ip-10-0-133-210 kubenswrapper[2576]: I0730 16:01:48.232332 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-f2cf4d\"/\"openshift-service-ca.crt\"" Jul 30 16:01:48.232360 ip-10-0-133-210 kubenswrapper[2576]: I0730 16:01:48.232355 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-f2cf4d\"/\"default-dockercfg-lvkhp\"" Jul 30 16:01:48.232547 ip-10-0-133-210 kubenswrapper[2576]: I0730 16:01:48.232372 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-f2cf4d\"/\"auth-test-model-f2cf4d-kserve-self-signed-certs\"" Jul 30 16:01:48.239383 ip-10-0-133-210 kubenswrapper[2576]: I0730 16:01:48.239357 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-f2cf4d/auth-test-model-f2cf4d-kserve-59c6ccdbff-2p458"] Jul 30 16:01:48.244132 ip-10-0-133-210 kubenswrapper[2576]: I0730 16:01:48.244093 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b45cbb00-6502-4064-baed-ab959e1e592c-home\") pod \"auth-test-model-f2cf4d-kserve-59c6ccdbff-2p458\" (UID: \"b45cbb00-6502-4064-baed-ab959e1e592c\") " pod="ai-tenant-e2e-shared-a-f2cf4d/auth-test-model-f2cf4d-kserve-59c6ccdbff-2p458" Jul 30 16:01:48.244255 ip-10-0-133-210 kubenswrapper[2576]: I0730 16:01:48.244140 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b45cbb00-6502-4064-baed-ab959e1e592c-dshm\") pod \"auth-test-model-f2cf4d-kserve-59c6ccdbff-2p458\" (UID: \"b45cbb00-6502-4064-baed-ab959e1e592c\") " pod="ai-tenant-e2e-shared-a-f2cf4d/auth-test-model-f2cf4d-kserve-59c6ccdbff-2p458" Jul 30 16:01:48.244255 ip-10-0-133-210 kubenswrapper[2576]: I0730 16:01:48.244171 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b45cbb00-6502-4064-baed-ab959e1e592c-model-cache\") pod \"auth-test-model-f2cf4d-kserve-59c6ccdbff-2p458\" (UID: \"b45cbb00-6502-4064-baed-ab959e1e592c\") " pod="ai-tenant-e2e-shared-a-f2cf4d/auth-test-model-f2cf4d-kserve-59c6ccdbff-2p458" Jul 30 16:01:48.244255 ip-10-0-133-210 kubenswrapper[2576]: I0730 16:01:48.244187 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b45cbb00-6502-4064-baed-ab959e1e592c-tls-certs\") pod \"auth-test-model-f2cf4d-kserve-59c6ccdbff-2p458\" (UID: \"b45cbb00-6502-4064-baed-ab959e1e592c\") " pod="ai-tenant-e2e-shared-a-f2cf4d/auth-test-model-f2cf4d-kserve-59c6ccdbff-2p458" Jul 30 16:01:48.244255 ip-10-0-133-210 kubenswrapper[2576]: I0730 16:01:48.244224 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rnk7f\" (UniqueName: \"kubernetes.io/projected/b45cbb00-6502-4064-baed-ab959e1e592c-kube-api-access-rnk7f\") pod \"auth-test-model-f2cf4d-kserve-59c6ccdbff-2p458\" (UID: \"b45cbb00-6502-4064-baed-ab959e1e592c\") " pod="ai-tenant-e2e-shared-a-f2cf4d/auth-test-model-f2cf4d-kserve-59c6ccdbff-2p458" Jul 30 16:01:48.244391 ip-10-0-133-210 kubenswrapper[2576]: I0730 16:01:48.244292 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b45cbb00-6502-4064-baed-ab959e1e592c-tmp-dir\") pod \"auth-test-model-f2cf4d-kserve-59c6ccdbff-2p458\" (UID: \"b45cbb00-6502-4064-baed-ab959e1e592c\") " pod="ai-tenant-e2e-shared-a-f2cf4d/auth-test-model-f2cf4d-kserve-59c6ccdbff-2p458" Jul 30 16:01:48.250175 ip-10-0-133-210 kubenswrapper[2576]: I0730 16:01:48.250136 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7j468_a2197c57-2dc6-410e-9910-a045dedcb8d9/konnectivity-agent/0.log" limit=1024 Jul 30 16:01:48.345119 ip-10-0-133-210 kubenswrapper[2576]: I0730 16:01:48.345071 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rnk7f\" (UniqueName: \"kubernetes.io/projected/b45cbb00-6502-4064-baed-ab959e1e592c-kube-api-access-rnk7f\") pod \"auth-test-model-f2cf4d-kserve-59c6ccdbff-2p458\" (UID: \"b45cbb00-6502-4064-baed-ab959e1e592c\") " pod="ai-tenant-e2e-shared-a-f2cf4d/auth-test-model-f2cf4d-kserve-59c6ccdbff-2p458" Jul 30 16:01:48.345307 ip-10-0-133-210 kubenswrapper[2576]: I0730 16:01:48.345143 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b45cbb00-6502-4064-baed-ab959e1e592c-tmp-dir\") pod \"auth-test-model-f2cf4d-kserve-59c6ccdbff-2p458\" (UID: \"b45cbb00-6502-4064-baed-ab959e1e592c\") " pod="ai-tenant-e2e-shared-a-f2cf4d/auth-test-model-f2cf4d-kserve-59c6ccdbff-2p458" Jul 30 16:01:48.345307 ip-10-0-133-210 kubenswrapper[2576]: I0730 16:01:48.345165 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b45cbb00-6502-4064-baed-ab959e1e592c-home\") pod \"auth-test-model-f2cf4d-kserve-59c6ccdbff-2p458\" (UID: \"b45cbb00-6502-4064-baed-ab959e1e592c\") " pod="ai-tenant-e2e-shared-a-f2cf4d/auth-test-model-f2cf4d-kserve-59c6ccdbff-2p458" Jul 30 16:01:48.345562 ip-10-0-133-210 kubenswrapper[2576]: I0730 16:01:48.345331 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b45cbb00-6502-4064-baed-ab959e1e592c-dshm\") pod \"auth-test-model-f2cf4d-kserve-59c6ccdbff-2p458\" (UID: \"b45cbb00-6502-4064-baed-ab959e1e592c\") " pod="ai-tenant-e2e-shared-a-f2cf4d/auth-test-model-f2cf4d-kserve-59c6ccdbff-2p458" Jul 30 16:01:48.345562 ip-10-0-133-210 kubenswrapper[2576]: I0730 16:01:48.345401 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b45cbb00-6502-4064-baed-ab959e1e592c-model-cache\") pod \"auth-test-model-f2cf4d-kserve-59c6ccdbff-2p458\" (UID: \"b45cbb00-6502-4064-baed-ab959e1e592c\") " pod="ai-tenant-e2e-shared-a-f2cf4d/auth-test-model-f2cf4d-kserve-59c6ccdbff-2p458" Jul 30 16:01:48.345562 ip-10-0-133-210 kubenswrapper[2576]: I0730 16:01:48.345432 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b45cbb00-6502-4064-baed-ab959e1e592c-tls-certs\") pod \"auth-test-model-f2cf4d-kserve-59c6ccdbff-2p458\" (UID: \"b45cbb00-6502-4064-baed-ab959e1e592c\") " pod="ai-tenant-e2e-shared-a-f2cf4d/auth-test-model-f2cf4d-kserve-59c6ccdbff-2p458" Jul 30 16:01:48.345562 ip-10-0-133-210 kubenswrapper[2576]: I0730 16:01:48.345499 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b45cbb00-6502-4064-baed-ab959e1e592c-home\") pod \"auth-test-model-f2cf4d-kserve-59c6ccdbff-2p458\" (UID: \"b45cbb00-6502-4064-baed-ab959e1e592c\") " pod="ai-tenant-e2e-shared-a-f2cf4d/auth-test-model-f2cf4d-kserve-59c6ccdbff-2p458" Jul 30 16:01:48.345562 ip-10-0-133-210 kubenswrapper[2576]: I0730 16:01:48.345520 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b45cbb00-6502-4064-baed-ab959e1e592c-tmp-dir\") pod \"auth-test-model-f2cf4d-kserve-59c6ccdbff-2p458\" (UID: \"b45cbb00-6502-4064-baed-ab959e1e592c\") " pod="ai-tenant-e2e-shared-a-f2cf4d/auth-test-model-f2cf4d-kserve-59c6ccdbff-2p458" Jul 30 16:01:48.345886 ip-10-0-133-210 kubenswrapper[2576]: I0730 16:01:48.345709 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b45cbb00-6502-4064-baed-ab959e1e592c-model-cache\") pod \"auth-test-model-f2cf4d-kserve-59c6ccdbff-2p458\" (UID: \"b45cbb00-6502-4064-baed-ab959e1e592c\") " pod="ai-tenant-e2e-shared-a-f2cf4d/auth-test-model-f2cf4d-kserve-59c6ccdbff-2p458" Jul 30 16:01:48.347446 ip-10-0-133-210 kubenswrapper[2576]: I0730 16:01:48.347422 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b45cbb00-6502-4064-baed-ab959e1e592c-dshm\") pod \"auth-test-model-f2cf4d-kserve-59c6ccdbff-2p458\" (UID: \"b45cbb00-6502-4064-baed-ab959e1e592c\") " pod="ai-tenant-e2e-shared-a-f2cf4d/auth-test-model-f2cf4d-kserve-59c6ccdbff-2p458" Jul 30 16:01:48.347780 ip-10-0-133-210 kubenswrapper[2576]: I0730 16:01:48.347762 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b45cbb00-6502-4064-baed-ab959e1e592c-tls-certs\") pod \"auth-test-model-f2cf4d-kserve-59c6ccdbff-2p458\" (UID: \"b45cbb00-6502-4064-baed-ab959e1e592c\") " pod="ai-tenant-e2e-shared-a-f2cf4d/auth-test-model-f2cf4d-kserve-59c6ccdbff-2p458" Jul 30 16:01:48.352396 ip-10-0-133-210 kubenswrapper[2576]: I0730 16:01:48.352375 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rnk7f\" (UniqueName: \"kubernetes.io/projected/b45cbb00-6502-4064-baed-ab959e1e592c-kube-api-access-rnk7f\") pod \"auth-test-model-f2cf4d-kserve-59c6ccdbff-2p458\" (UID: \"b45cbb00-6502-4064-baed-ab959e1e592c\") " pod="ai-tenant-e2e-shared-a-f2cf4d/auth-test-model-f2cf4d-kserve-59c6ccdbff-2p458" Jul 30 16:01:48.537678 ip-10-0-133-210 kubenswrapper[2576]: I0730 16:01:48.537600 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-f2cf4d/auth-test-model-f2cf4d-kserve-59c6ccdbff-2p458" Jul 30 16:01:48.667728 ip-10-0-133-210 kubenswrapper[2576]: I0730 16:01:48.667699 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-f2cf4d/auth-test-model-f2cf4d-kserve-59c6ccdbff-2p458"] Jul 30 16:01:48.669378 ip-10-0-133-210 kubenswrapper[2576]: W0730 16:01:48.669351 2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podb45cbb00_6502_4064_baed_ab959e1e592c.slice/crio-ac3f97135863d19c7a835bb389f61f94b9893bd359a82ac037ee559fe9d48ff4 WatchSource:0}: Error finding container ac3f97135863d19c7a835bb389f61f94b9893bd359a82ac037ee559fe9d48ff4: Status 404 returned error can't find the container with id ac3f97135863d19c7a835bb389f61f94b9893bd359a82ac037ee559fe9d48ff4 Jul 30 16:01:49.317650 ip-10-0-133-210 kubenswrapper[2576]: I0730 16:01:49.317614 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-f2cf4d/auth-test-model-f2cf4d-kserve-59c6ccdbff-2p458" event={"ID":"b45cbb00-6502-4064-baed-ab959e1e592c","Type":"ContainerStarted","Data":"1f55f071f473b047c8d99db42aff50b5c0494decb34f34136a76719b6ff3f1bc"} Jul 30 16:01:49.317650 ip-10-0-133-210 kubenswrapper[2576]: I0730 16:01:49.317655 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-f2cf4d/auth-test-model-f2cf4d-kserve-59c6ccdbff-2p458" event={"ID":"b45cbb00-6502-4064-baed-ab959e1e592c","Type":"ContainerStarted","Data":"ac3f97135863d19c7a835bb389f61f94b9893bd359a82ac037ee559fe9d48ff4"} Jul 30 16:01:49.337480 ip-10-0-133-210 kubenswrapper[2576]: I0730 16:01:49.337438 2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-a-f2cf4d/auth-test-model-f2cf4d-kserve-59c6ccdbff-2p458" podStartSLOduration=1.337423993 podStartE2EDuration="1.337423993s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 16:01:48 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 16:01:49.336469221 +0000 UTC m=+3091.262221867" watchObservedRunningTime="2026-07-30 16:01:49.337423993 +0000 UTC m=+3091.263176637" Jul 30 16:01:58.537944 ip-10-0-133-210 kubenswrapper[2576]: I0730 16:01:58.537875 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-a-f2cf4d/auth-test-model-f2cf4d-kserve-59c6ccdbff-2p458" Jul 30 16:01:58.538446 ip-10-0-133-210 kubenswrapper[2576]: I0730 16:01:58.538200 2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-a-f2cf4d/auth-test-model-f2cf4d-kserve-59c6ccdbff-2p458" Jul 30 16:01:58.550717 ip-10-0-133-210 kubenswrapper[2576]: I0730 16:01:58.550694 2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-a-f2cf4d/auth-test-model-f2cf4d-kserve-59c6ccdbff-2p458" Jul 30 16:01:59.373969 ip-10-0-133-210 kubenswrapper[2576]: I0730 16:01:59.373941 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-a-f2cf4d/auth-test-model-f2cf4d-kserve-59c6ccdbff-2p458" Jul 30 16:02:10.966910 ip-10-0-133-210 kubenswrapper[2576]: I0730 16:02:10.966855 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7j468_a2197c57-2dc6-410e-9910-a045dedcb8d9/konnectivity-agent/0.log" limit=1024 Jul 30 16:02:11.015642 ip-10-0-133-210 kubenswrapper[2576]: I0730 16:02:11.015606 2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-b-5468b9/auth-test-model-5468b9-kserve-5bdbdc4d8c-cmg6d"] Jul 30 16:02:11.018885 ip-10-0-133-210 kubenswrapper[2576]: I0730 16:02:11.018862 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-5468b9/auth-test-model-5468b9-kserve-5bdbdc4d8c-cmg6d" Jul 30 16:02:11.021539 ip-10-0-133-210 kubenswrapper[2576]: I0730 16:02:11.021518 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-5468b9\"/\"openshift-service-ca.crt\"" Jul 30 16:02:11.022661 ip-10-0-133-210 kubenswrapper[2576]: I0730 16:02:11.022640 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-5468b9\"/\"default-dockercfg-swktg\"" Jul 30 16:02:11.022795 ip-10-0-133-210 kubenswrapper[2576]: I0730 16:02:11.022778 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-5468b9\"/\"kube-root-ca.crt\"" Jul 30 16:02:11.022875 ip-10-0-133-210 kubenswrapper[2576]: I0730 16:02:11.022780 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-5468b9\"/\"auth-test-model-5468b9-kserve-self-signed-certs\"" Jul 30 16:02:11.029202 ip-10-0-133-210 kubenswrapper[2576]: I0730 16:02:11.029182 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-5468b9/auth-test-model-5468b9-kserve-5bdbdc4d8c-cmg6d"] Jul 30 16:02:11.140236 ip-10-0-133-210 kubenswrapper[2576]: I0730 16:02:11.140197 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b027c9b2-b093-42d8-b692-876a5a4c3074-dshm\") pod \"auth-test-model-5468b9-kserve-5bdbdc4d8c-cmg6d\" (UID: \"b027c9b2-b093-42d8-b692-876a5a4c3074\") " pod="ai-tenant-e2e-shared-b-5468b9/auth-test-model-5468b9-kserve-5bdbdc4d8c-cmg6d" Jul 30 16:02:11.140236 ip-10-0-133-210 kubenswrapper[2576]: I0730 16:02:11.140234 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b027c9b2-b093-42d8-b692-876a5a4c3074-home\") pod \"auth-test-model-5468b9-kserve-5bdbdc4d8c-cmg6d\" (UID: \"b027c9b2-b093-42d8-b692-876a5a4c3074\") " pod="ai-tenant-e2e-shared-b-5468b9/auth-test-model-5468b9-kserve-5bdbdc4d8c-cmg6d" Jul 30 16:02:11.140466 ip-10-0-133-210 kubenswrapper[2576]: I0730 16:02:11.140274 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b027c9b2-b093-42d8-b692-876a5a4c3074-tls-certs\") pod \"auth-test-model-5468b9-kserve-5bdbdc4d8c-cmg6d\" (UID: \"b027c9b2-b093-42d8-b692-876a5a4c3074\") " pod="ai-tenant-e2e-shared-b-5468b9/auth-test-model-5468b9-kserve-5bdbdc4d8c-cmg6d" Jul 30 16:02:11.140466 ip-10-0-133-210 kubenswrapper[2576]: I0730 16:02:11.140319 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b027c9b2-b093-42d8-b692-876a5a4c3074-tmp-dir\") pod \"auth-test-model-5468b9-kserve-5bdbdc4d8c-cmg6d\" (UID: \"b027c9b2-b093-42d8-b692-876a5a4c3074\") " pod="ai-tenant-e2e-shared-b-5468b9/auth-test-model-5468b9-kserve-5bdbdc4d8c-cmg6d" Jul 30 16:02:11.140466 ip-10-0-133-210 kubenswrapper[2576]: I0730 16:02:11.140378 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b027c9b2-b093-42d8-b692-876a5a4c3074-model-cache\") pod \"auth-test-model-5468b9-kserve-5bdbdc4d8c-cmg6d\" (UID: \"b027c9b2-b093-42d8-b692-876a5a4c3074\") " pod="ai-tenant-e2e-shared-b-5468b9/auth-test-model-5468b9-kserve-5bdbdc4d8c-cmg6d" Jul 30 16:02:11.140466 ip-10-0-133-210 kubenswrapper[2576]: I0730 16:02:11.140430 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wkrsn\" (UniqueName: \"kubernetes.io/projected/b027c9b2-b093-42d8-b692-876a5a4c3074-kube-api-access-wkrsn\") pod \"auth-test-model-5468b9-kserve-5bdbdc4d8c-cmg6d\" (UID: \"b027c9b2-b093-42d8-b692-876a5a4c3074\") " pod="ai-tenant-e2e-shared-b-5468b9/auth-test-model-5468b9-kserve-5bdbdc4d8c-cmg6d" Jul 30 16:02:11.241509 ip-10-0-133-210 kubenswrapper[2576]: I0730 16:02:11.241404 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b027c9b2-b093-42d8-b692-876a5a4c3074-model-cache\") pod \"auth-test-model-5468b9-kserve-5bdbdc4d8c-cmg6d\" (UID: \"b027c9b2-b093-42d8-b692-876a5a4c3074\") " pod="ai-tenant-e2e-shared-b-5468b9/auth-test-model-5468b9-kserve-5bdbdc4d8c-cmg6d" Jul 30 16:02:11.241509 ip-10-0-133-210 kubenswrapper[2576]: I0730 16:02:11.241470 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wkrsn\" (UniqueName: \"kubernetes.io/projected/b027c9b2-b093-42d8-b692-876a5a4c3074-kube-api-access-wkrsn\") pod \"auth-test-model-5468b9-kserve-5bdbdc4d8c-cmg6d\" (UID: \"b027c9b2-b093-42d8-b692-876a5a4c3074\") " pod="ai-tenant-e2e-shared-b-5468b9/auth-test-model-5468b9-kserve-5bdbdc4d8c-cmg6d" Jul 30 16:02:11.241509 ip-10-0-133-210 kubenswrapper[2576]: I0730 16:02:11.241499 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b027c9b2-b093-42d8-b692-876a5a4c3074-dshm\") pod \"auth-test-model-5468b9-kserve-5bdbdc4d8c-cmg6d\" (UID: \"b027c9b2-b093-42d8-b692-876a5a4c3074\") " pod="ai-tenant-e2e-shared-b-5468b9/auth-test-model-5468b9-kserve-5bdbdc4d8c-cmg6d" Jul 30 16:02:11.241886 ip-10-0-133-210 kubenswrapper[2576]: I0730 16:02:11.241522 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b027c9b2-b093-42d8-b692-876a5a4c3074-home\") pod \"auth-test-model-5468b9-kserve-5bdbdc4d8c-cmg6d\" (UID: \"b027c9b2-b093-42d8-b692-876a5a4c3074\") " pod="ai-tenant-e2e-shared-b-5468b9/auth-test-model-5468b9-kserve-5bdbdc4d8c-cmg6d" Jul 30 16:02:11.241886 ip-10-0-133-210 kubenswrapper[2576]: I0730 16:02:11.241568 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b027c9b2-b093-42d8-b692-876a5a4c3074-tls-certs\") pod \"auth-test-model-5468b9-kserve-5bdbdc4d8c-cmg6d\" (UID: \"b027c9b2-b093-42d8-b692-876a5a4c3074\") " pod="ai-tenant-e2e-shared-b-5468b9/auth-test-model-5468b9-kserve-5bdbdc4d8c-cmg6d" Jul 30 16:02:11.241886 ip-10-0-133-210 kubenswrapper[2576]: I0730 16:02:11.241691 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b027c9b2-b093-42d8-b692-876a5a4c3074-tmp-dir\") pod \"auth-test-model-5468b9-kserve-5bdbdc4d8c-cmg6d\" (UID: \"b027c9b2-b093-42d8-b692-876a5a4c3074\") " pod="ai-tenant-e2e-shared-b-5468b9/auth-test-model-5468b9-kserve-5bdbdc4d8c-cmg6d" Jul 30 16:02:11.241886 ip-10-0-133-210 kubenswrapper[2576]: I0730 16:02:11.241839 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b027c9b2-b093-42d8-b692-876a5a4c3074-model-cache\") pod \"auth-test-model-5468b9-kserve-5bdbdc4d8c-cmg6d\" (UID: \"b027c9b2-b093-42d8-b692-876a5a4c3074\") " pod="ai-tenant-e2e-shared-b-5468b9/auth-test-model-5468b9-kserve-5bdbdc4d8c-cmg6d" Jul 30 16:02:11.242062 ip-10-0-133-210 kubenswrapper[2576]: I0730 16:02:11.241889 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b027c9b2-b093-42d8-b692-876a5a4c3074-home\") pod \"auth-test-model-5468b9-kserve-5bdbdc4d8c-cmg6d\" (UID: \"b027c9b2-b093-42d8-b692-876a5a4c3074\") " pod="ai-tenant-e2e-shared-b-5468b9/auth-test-model-5468b9-kserve-5bdbdc4d8c-cmg6d" Jul 30 16:02:11.242062 ip-10-0-133-210 kubenswrapper[2576]: I0730 16:02:11.242014 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b027c9b2-b093-42d8-b692-876a5a4c3074-tmp-dir\") pod \"auth-test-model-5468b9-kserve-5bdbdc4d8c-cmg6d\" (UID: \"b027c9b2-b093-42d8-b692-876a5a4c3074\") " pod="ai-tenant-e2e-shared-b-5468b9/auth-test-model-5468b9-kserve-5bdbdc4d8c-cmg6d" Jul 30 16:02:11.243818 ip-10-0-133-210 kubenswrapper[2576]: I0730 16:02:11.243792 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b027c9b2-b093-42d8-b692-876a5a4c3074-dshm\") pod \"auth-test-model-5468b9-kserve-5bdbdc4d8c-cmg6d\" (UID: \"b027c9b2-b093-42d8-b692-876a5a4c3074\") " pod="ai-tenant-e2e-shared-b-5468b9/auth-test-model-5468b9-kserve-5bdbdc4d8c-cmg6d" Jul 30 16:02:11.244033 ip-10-0-133-210 kubenswrapper[2576]: I0730 16:02:11.244015 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b027c9b2-b093-42d8-b692-876a5a4c3074-tls-certs\") pod \"auth-test-model-5468b9-kserve-5bdbdc4d8c-cmg6d\" (UID: \"b027c9b2-b093-42d8-b692-876a5a4c3074\") " pod="ai-tenant-e2e-shared-b-5468b9/auth-test-model-5468b9-kserve-5bdbdc4d8c-cmg6d" Jul 30 16:02:11.250186 ip-10-0-133-210 kubenswrapper[2576]: I0730 16:02:11.250162 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wkrsn\" (UniqueName: \"kubernetes.io/projected/b027c9b2-b093-42d8-b692-876a5a4c3074-kube-api-access-wkrsn\") pod \"auth-test-model-5468b9-kserve-5bdbdc4d8c-cmg6d\" (UID: \"b027c9b2-b093-42d8-b692-876a5a4c3074\") " pod="ai-tenant-e2e-shared-b-5468b9/auth-test-model-5468b9-kserve-5bdbdc4d8c-cmg6d" Jul 30 16:02:11.329535 ip-10-0-133-210 kubenswrapper[2576]: I0730 16:02:11.329493 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-5468b9/auth-test-model-5468b9-kserve-5bdbdc4d8c-cmg6d" Jul 30 16:02:11.465196 ip-10-0-133-210 kubenswrapper[2576]: I0730 16:02:11.465155 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-5468b9/auth-test-model-5468b9-kserve-5bdbdc4d8c-cmg6d"] Jul 30 16:02:11.467872 ip-10-0-133-210 kubenswrapper[2576]: W0730 16:02:11.467846 2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podb027c9b2_b093_42d8_b692_876a5a4c3074.slice/crio-6ceb989e5d35983ecd86d3caa064e101ab8166f1390ec4e381684e923987cc13 WatchSource:0}: Error finding container 6ceb989e5d35983ecd86d3caa064e101ab8166f1390ec4e381684e923987cc13: Status 404 returned error can't find the container with id 6ceb989e5d35983ecd86d3caa064e101ab8166f1390ec4e381684e923987cc13 Jul 30 16:02:12.423199 ip-10-0-133-210 kubenswrapper[2576]: I0730 16:02:12.423156 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-5468b9/auth-test-model-5468b9-kserve-5bdbdc4d8c-cmg6d" event={"ID":"b027c9b2-b093-42d8-b692-876a5a4c3074","Type":"ContainerStarted","Data":"ad0bc2095856886bae3b224a05c1916414dd6331715f339fe77a7495e93c5551"} Jul 30 16:02:12.423199 ip-10-0-133-210 kubenswrapper[2576]: I0730 16:02:12.423206 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-5468b9/auth-test-model-5468b9-kserve-5bdbdc4d8c-cmg6d" event={"ID":"b027c9b2-b093-42d8-b692-876a5a4c3074","Type":"ContainerStarted","Data":"6ceb989e5d35983ecd86d3caa064e101ab8166f1390ec4e381684e923987cc13"} Jul 30 16:02:12.443722 ip-10-0-133-210 kubenswrapper[2576]: I0730 16:02:12.443672 2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-b-5468b9/auth-test-model-5468b9-kserve-5bdbdc4d8c-cmg6d" podStartSLOduration=2.443655485 podStartE2EDuration="2.443655485s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 16:02:10 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 16:02:12.442795757 +0000 UTC m=+3114.368548404" watchObservedRunningTime="2026-07-30 16:02:12.443655485 +0000 UTC m=+3114.369408351" Jul 30 16:02:21.330222 ip-10-0-133-210 kubenswrapper[2576]: I0730 16:02:21.330165 2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-b-5468b9/auth-test-model-5468b9-kserve-5bdbdc4d8c-cmg6d" Jul 30 16:02:21.330222 ip-10-0-133-210 kubenswrapper[2576]: I0730 16:02:21.330232 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-b-5468b9/auth-test-model-5468b9-kserve-5bdbdc4d8c-cmg6d" Jul 30 16:02:21.343037 ip-10-0-133-210 kubenswrapper[2576]: I0730 16:02:21.343009 2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-b-5468b9/auth-test-model-5468b9-kserve-5bdbdc4d8c-cmg6d" Jul 30 16:02:21.472842 ip-10-0-133-210 kubenswrapper[2576]: I0730 16:02:21.472804 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-b-5468b9/auth-test-model-5468b9-kserve-5bdbdc4d8c-cmg6d" Jul 30 16:02:26.801683 ip-10-0-133-210 kubenswrapper[2576]: I0730 16:02:26.801643 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7j468_a2197c57-2dc6-410e-9910-a045dedcb8d9/konnectivity-agent/0.log" limit=1024 Jul 30 16:03:53.646919 ip-10-0-133-210 kubenswrapper[2576]: I0730 16:03:53.646870 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7j468_a2197c57-2dc6-410e-9910-a045dedcb8d9/konnectivity-agent/0.log" limit=1024 Jul 30 16:04:54.940104 ip-10-0-133-210 kubenswrapper[2576]: I0730 16:04:54.940062 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7j468_a2197c57-2dc6-410e-9910-a045dedcb8d9/konnectivity-agent/0.log" limit=1024 Jul 30 16:06:30.659335 ip-10-0-133-210 kubenswrapper[2576]: I0730 16:06:30.659289 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7j468_a2197c57-2dc6-410e-9910-a045dedcb8d9/konnectivity-agent/0.log" limit=1024 Jul 30 16:06:49.833389 ip-10-0-133-210 kubenswrapper[2576]: I0730 16:06:49.833355 2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-f2cf4d/auth-test-model-f2cf4d-kserve-59c6ccdbff-2p458"] Jul 30 16:06:49.858309 ip-10-0-133-210 kubenswrapper[2576]: I0730 16:06:49.858277 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7j468_a2197c57-2dc6-410e-9910-a045dedcb8d9/konnectivity-agent/0.log" limit=1024 Jul 30 16:06:50.721387 ip-10-0-133-210 kubenswrapper[2576]: I0730 16:06:50.721355 2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-5468b9/auth-test-model-5468b9-kserve-5bdbdc4d8c-cmg6d"] Jul 30 16:06:50.742948 ip-10-0-133-210 kubenswrapper[2576]: I0730 16:06:50.742916 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7j468_a2197c57-2dc6-410e-9910-a045dedcb8d9/konnectivity-agent/0.log" limit=1024 Jul 30 16:06:58.370612 ip-10-0-133-210 kubenswrapper[2576]: I0730 16:06:58.370574 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7j468_a2197c57-2dc6-410e-9910-a045dedcb8d9/konnectivity-agent/0.log" limit=1024 Jul 30 16:07:04.851868 ip-10-0-133-210 kubenswrapper[2576]: I0730 16:07:04.851814 2576 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-a-f2cf4d/auth-test-model-f2cf4d-kserve-59c6ccdbff-2p458" podUID="b45cbb00-6502-4064-baed-ab959e1e592c" containerName="main" containerID="cri-o://1f55f071f473b047c8d99db42aff50b5c0494decb34f34136a76719b6ff3f1bc" gracePeriod=45 Jul 30 16:07:05.105567 ip-10-0-133-210 kubenswrapper[2576]: I0730 16:07:05.105506 2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-f2cf4d/auth-test-model-f2cf4d-kserve-59c6ccdbff-2p458" Jul 30 16:07:05.214997 ip-10-0-133-210 kubenswrapper[2576]: I0730 16:07:05.214960 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b45cbb00-6502-4064-baed-ab959e1e592c-model-cache\") pod \"b45cbb00-6502-4064-baed-ab959e1e592c\" (UID: \"b45cbb00-6502-4064-baed-ab959e1e592c\") " Jul 30 16:07:05.215239 ip-10-0-133-210 kubenswrapper[2576]: I0730 16:07:05.215023 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b45cbb00-6502-4064-baed-ab959e1e592c-tls-certs\") pod \"b45cbb00-6502-4064-baed-ab959e1e592c\" (UID: \"b45cbb00-6502-4064-baed-ab959e1e592c\") " Jul 30 16:07:05.215239 ip-10-0-133-210 kubenswrapper[2576]: I0730 16:07:05.215047 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b45cbb00-6502-4064-baed-ab959e1e592c-tmp-dir\") pod \"b45cbb00-6502-4064-baed-ab959e1e592c\" (UID: \"b45cbb00-6502-4064-baed-ab959e1e592c\") " Jul 30 16:07:05.215239 ip-10-0-133-210 kubenswrapper[2576]: I0730 16:07:05.215080 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-rnk7f\" (UniqueName: \"kubernetes.io/projected/b45cbb00-6502-4064-baed-ab959e1e592c-kube-api-access-rnk7f\") pod \"b45cbb00-6502-4064-baed-ab959e1e592c\" (UID: \"b45cbb00-6502-4064-baed-ab959e1e592c\") " Jul 30 16:07:05.215239 ip-10-0-133-210 kubenswrapper[2576]: I0730 16:07:05.215168 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b45cbb00-6502-4064-baed-ab959e1e592c-home\") pod \"b45cbb00-6502-4064-baed-ab959e1e592c\" (UID: \"b45cbb00-6502-4064-baed-ab959e1e592c\") " Jul 30 16:07:05.215239 ip-10-0-133-210 kubenswrapper[2576]: I0730 16:07:05.215233 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b45cbb00-6502-4064-baed-ab959e1e592c-dshm\") pod \"b45cbb00-6502-4064-baed-ab959e1e592c\" (UID: \"b45cbb00-6502-4064-baed-ab959e1e592c\") " Jul 30 16:07:05.215495 ip-10-0-133-210 kubenswrapper[2576]: I0730 16:07:05.215330 2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b45cbb00-6502-4064-baed-ab959e1e592c-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "b45cbb00-6502-4064-baed-ab959e1e592c" (UID: "b45cbb00-6502-4064-baed-ab959e1e592c"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 30 16:07:05.215495 ip-10-0-133-210 kubenswrapper[2576]: I0730 16:07:05.215362 2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b45cbb00-6502-4064-baed-ab959e1e592c-model-cache" (OuterVolumeSpecName: "model-cache") pod "b45cbb00-6502-4064-baed-ab959e1e592c" (UID: "b45cbb00-6502-4064-baed-ab959e1e592c"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 30 16:07:05.215607 ip-10-0-133-210 kubenswrapper[2576]: I0730 16:07:05.215492 2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b45cbb00-6502-4064-baed-ab959e1e592c-home" (OuterVolumeSpecName: "home") pod "b45cbb00-6502-4064-baed-ab959e1e592c" (UID: "b45cbb00-6502-4064-baed-ab959e1e592c"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 30 16:07:05.215607 ip-10-0-133-210 kubenswrapper[2576]: I0730 16:07:05.215550 2576 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b45cbb00-6502-4064-baed-ab959e1e592c-model-cache\") on node \"ip-10-0-133-210.ec2.internal\" DevicePath \"\"" Jul 30 16:07:05.215607 ip-10-0-133-210 kubenswrapper[2576]: I0730 16:07:05.215597 2576 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b45cbb00-6502-4064-baed-ab959e1e592c-tmp-dir\") on node \"ip-10-0-133-210.ec2.internal\" DevicePath \"\"" Jul 30 16:07:05.217440 ip-10-0-133-210 kubenswrapper[2576]: I0730 16:07:05.217411 2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b45cbb00-6502-4064-baed-ab959e1e592c-dshm" (OuterVolumeSpecName: "dshm") pod "b45cbb00-6502-4064-baed-ab959e1e592c" (UID: "b45cbb00-6502-4064-baed-ab959e1e592c"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 30 16:07:05.217440 ip-10-0-133-210 kubenswrapper[2576]: I0730 16:07:05.217420 2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b45cbb00-6502-4064-baed-ab959e1e592c-kube-api-access-rnk7f" (OuterVolumeSpecName: "kube-api-access-rnk7f") pod "b45cbb00-6502-4064-baed-ab959e1e592c" (UID: "b45cbb00-6502-4064-baed-ab959e1e592c"). InnerVolumeSpecName "kube-api-access-rnk7f". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 30 16:07:05.217592 ip-10-0-133-210 kubenswrapper[2576]: I0730 16:07:05.217470 2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b45cbb00-6502-4064-baed-ab959e1e592c-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "b45cbb00-6502-4064-baed-ab959e1e592c" (UID: "b45cbb00-6502-4064-baed-ab959e1e592c"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 30 16:07:05.316158 ip-10-0-133-210 kubenswrapper[2576]: I0730 16:07:05.316096 2576 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b45cbb00-6502-4064-baed-ab959e1e592c-home\") on node \"ip-10-0-133-210.ec2.internal\" DevicePath \"\"" Jul 30 16:07:05.316158 ip-10-0-133-210 kubenswrapper[2576]: I0730 16:07:05.316147 2576 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b45cbb00-6502-4064-baed-ab959e1e592c-dshm\") on node \"ip-10-0-133-210.ec2.internal\" DevicePath \"\"" Jul 30 16:07:05.316158 ip-10-0-133-210 kubenswrapper[2576]: I0730 16:07:05.316156 2576 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b45cbb00-6502-4064-baed-ab959e1e592c-tls-certs\") on node \"ip-10-0-133-210.ec2.internal\" DevicePath \"\"" Jul 30 16:07:05.316158 ip-10-0-133-210 kubenswrapper[2576]: I0730 16:07:05.316167 2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-rnk7f\" (UniqueName: \"kubernetes.io/projected/b45cbb00-6502-4064-baed-ab959e1e592c-kube-api-access-rnk7f\") on node \"ip-10-0-133-210.ec2.internal\" DevicePath \"\"" Jul 30 16:07:05.675399 ip-10-0-133-210 kubenswrapper[2576]: I0730 16:07:05.675360 2576 generic.go:356] "Generic (PLEG): container finished" podID="b45cbb00-6502-4064-baed-ab959e1e592c" containerID="1f55f071f473b047c8d99db42aff50b5c0494decb34f34136a76719b6ff3f1bc" exitCode=0 Jul 30 16:07:05.675580 ip-10-0-133-210 kubenswrapper[2576]: I0730 16:07:05.675408 2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-f2cf4d/auth-test-model-f2cf4d-kserve-59c6ccdbff-2p458" Jul 30 16:07:05.675580 ip-10-0-133-210 kubenswrapper[2576]: I0730 16:07:05.675446 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-f2cf4d/auth-test-model-f2cf4d-kserve-59c6ccdbff-2p458" event={"ID":"b45cbb00-6502-4064-baed-ab959e1e592c","Type":"ContainerDied","Data":"1f55f071f473b047c8d99db42aff50b5c0494decb34f34136a76719b6ff3f1bc"} Jul 30 16:07:05.675580 ip-10-0-133-210 kubenswrapper[2576]: I0730 16:07:05.675492 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-f2cf4d/auth-test-model-f2cf4d-kserve-59c6ccdbff-2p458" event={"ID":"b45cbb00-6502-4064-baed-ab959e1e592c","Type":"ContainerDied","Data":"ac3f97135863d19c7a835bb389f61f94b9893bd359a82ac037ee559fe9d48ff4"} Jul 30 16:07:05.675580 ip-10-0-133-210 kubenswrapper[2576]: I0730 16:07:05.675509 2576 scope.go:117] "RemoveContainer" containerID="1f55f071f473b047c8d99db42aff50b5c0494decb34f34136a76719b6ff3f1bc" Jul 30 16:07:05.683778 ip-10-0-133-210 kubenswrapper[2576]: I0730 16:07:05.683760 2576 scope.go:117] "RemoveContainer" containerID="1f55f071f473b047c8d99db42aff50b5c0494decb34f34136a76719b6ff3f1bc" Jul 30 16:07:05.684021 ip-10-0-133-210 kubenswrapper[2576]: E0730 16:07:05.684002 2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1f55f071f473b047c8d99db42aff50b5c0494decb34f34136a76719b6ff3f1bc\": container with ID starting with 1f55f071f473b047c8d99db42aff50b5c0494decb34f34136a76719b6ff3f1bc not found: ID does not exist" containerID="1f55f071f473b047c8d99db42aff50b5c0494decb34f34136a76719b6ff3f1bc" Jul 30 16:07:05.684066 ip-10-0-133-210 kubenswrapper[2576]: I0730 16:07:05.684029 2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1f55f071f473b047c8d99db42aff50b5c0494decb34f34136a76719b6ff3f1bc"} err="failed to get container status \"1f55f071f473b047c8d99db42aff50b5c0494decb34f34136a76719b6ff3f1bc\": rpc error: code = NotFound desc = could not find container \"1f55f071f473b047c8d99db42aff50b5c0494decb34f34136a76719b6ff3f1bc\": container with ID starting with 1f55f071f473b047c8d99db42aff50b5c0494decb34f34136a76719b6ff3f1bc not found: ID does not exist" Jul 30 16:07:05.696226 ip-10-0-133-210 kubenswrapper[2576]: I0730 16:07:05.696202 2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-f2cf4d/auth-test-model-f2cf4d-kserve-59c6ccdbff-2p458"] Jul 30 16:07:05.698490 ip-10-0-133-210 kubenswrapper[2576]: I0730 16:07:05.698471 2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-a-f2cf4d/auth-test-model-f2cf4d-kserve-59c6ccdbff-2p458"] Jul 30 16:07:05.740077 ip-10-0-133-210 kubenswrapper[2576]: I0730 16:07:05.740039 2576 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-b-5468b9/auth-test-model-5468b9-kserve-5bdbdc4d8c-cmg6d" podUID="b027c9b2-b093-42d8-b692-876a5a4c3074" containerName="main" containerID="cri-o://ad0bc2095856886bae3b224a05c1916414dd6331715f339fe77a7495e93c5551" gracePeriod=45 Jul 30 16:07:05.979917 ip-10-0-133-210 kubenswrapper[2576]: I0730 16:07:05.979892 2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-5468b9/auth-test-model-5468b9-kserve-5bdbdc4d8c-cmg6d" Jul 30 16:07:06.123595 ip-10-0-133-210 kubenswrapper[2576]: I0730 16:07:06.123557 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b027c9b2-b093-42d8-b692-876a5a4c3074-tmp-dir\") pod \"b027c9b2-b093-42d8-b692-876a5a4c3074\" (UID: \"b027c9b2-b093-42d8-b692-876a5a4c3074\") " Jul 30 16:07:06.123789 ip-10-0-133-210 kubenswrapper[2576]: I0730 16:07:06.123605 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-wkrsn\" (UniqueName: \"kubernetes.io/projected/b027c9b2-b093-42d8-b692-876a5a4c3074-kube-api-access-wkrsn\") pod \"b027c9b2-b093-42d8-b692-876a5a4c3074\" (UID: \"b027c9b2-b093-42d8-b692-876a5a4c3074\") " Jul 30 16:07:06.123789 ip-10-0-133-210 kubenswrapper[2576]: I0730 16:07:06.123651 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b027c9b2-b093-42d8-b692-876a5a4c3074-dshm\") pod \"b027c9b2-b093-42d8-b692-876a5a4c3074\" (UID: \"b027c9b2-b093-42d8-b692-876a5a4c3074\") " Jul 30 16:07:06.123789 ip-10-0-133-210 kubenswrapper[2576]: I0730 16:07:06.123686 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b027c9b2-b093-42d8-b692-876a5a4c3074-home\") pod \"b027c9b2-b093-42d8-b692-876a5a4c3074\" (UID: \"b027c9b2-b093-42d8-b692-876a5a4c3074\") " Jul 30 16:07:06.123789 ip-10-0-133-210 kubenswrapper[2576]: I0730 16:07:06.123730 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b027c9b2-b093-42d8-b692-876a5a4c3074-model-cache\") pod \"b027c9b2-b093-42d8-b692-876a5a4c3074\" (UID: \"b027c9b2-b093-42d8-b692-876a5a4c3074\") " Jul 30 16:07:06.123789 ip-10-0-133-210 kubenswrapper[2576]: I0730 16:07:06.123770 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b027c9b2-b093-42d8-b692-876a5a4c3074-tls-certs\") pod \"b027c9b2-b093-42d8-b692-876a5a4c3074\" (UID: \"b027c9b2-b093-42d8-b692-876a5a4c3074\") " Jul 30 16:07:06.124072 ip-10-0-133-210 kubenswrapper[2576]: I0730 16:07:06.123914 2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b027c9b2-b093-42d8-b692-876a5a4c3074-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "b027c9b2-b093-42d8-b692-876a5a4c3074" (UID: "b027c9b2-b093-42d8-b692-876a5a4c3074"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 30 16:07:06.124072 ip-10-0-133-210 kubenswrapper[2576]: I0730 16:07:06.124005 2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b027c9b2-b093-42d8-b692-876a5a4c3074-home" (OuterVolumeSpecName: "home") pod "b027c9b2-b093-42d8-b692-876a5a4c3074" (UID: "b027c9b2-b093-42d8-b692-876a5a4c3074"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 30 16:07:06.124072 ip-10-0-133-210 kubenswrapper[2576]: I0730 16:07:06.124021 2576 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b027c9b2-b093-42d8-b692-876a5a4c3074-tmp-dir\") on node \"ip-10-0-133-210.ec2.internal\" DevicePath \"\"" Jul 30 16:07:06.124072 ip-10-0-133-210 kubenswrapper[2576]: I0730 16:07:06.124036 2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b027c9b2-b093-42d8-b692-876a5a4c3074-model-cache" (OuterVolumeSpecName: "model-cache") pod "b027c9b2-b093-42d8-b692-876a5a4c3074" (UID: "b027c9b2-b093-42d8-b692-876a5a4c3074"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 30 16:07:06.125916 ip-10-0-133-210 kubenswrapper[2576]: I0730 16:07:06.125891 2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b027c9b2-b093-42d8-b692-876a5a4c3074-dshm" (OuterVolumeSpecName: "dshm") pod "b027c9b2-b093-42d8-b692-876a5a4c3074" (UID: "b027c9b2-b093-42d8-b692-876a5a4c3074"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 30 16:07:06.125916 ip-10-0-133-210 kubenswrapper[2576]: I0730 16:07:06.125904 2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b027c9b2-b093-42d8-b692-876a5a4c3074-kube-api-access-wkrsn" (OuterVolumeSpecName: "kube-api-access-wkrsn") pod "b027c9b2-b093-42d8-b692-876a5a4c3074" (UID: "b027c9b2-b093-42d8-b692-876a5a4c3074"). InnerVolumeSpecName "kube-api-access-wkrsn". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 30 16:07:06.126039 ip-10-0-133-210 kubenswrapper[2576]: I0730 16:07:06.125972 2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b027c9b2-b093-42d8-b692-876a5a4c3074-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "b027c9b2-b093-42d8-b692-876a5a4c3074" (UID: "b027c9b2-b093-42d8-b692-876a5a4c3074"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 30 16:07:06.225275 ip-10-0-133-210 kubenswrapper[2576]: I0730 16:07:06.225244 2576 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b027c9b2-b093-42d8-b692-876a5a4c3074-tls-certs\") on node \"ip-10-0-133-210.ec2.internal\" DevicePath \"\"" Jul 30 16:07:06.225275 ip-10-0-133-210 kubenswrapper[2576]: I0730 16:07:06.225274 2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-wkrsn\" (UniqueName: \"kubernetes.io/projected/b027c9b2-b093-42d8-b692-876a5a4c3074-kube-api-access-wkrsn\") on node \"ip-10-0-133-210.ec2.internal\" DevicePath \"\"" Jul 30 16:07:06.225457 ip-10-0-133-210 kubenswrapper[2576]: I0730 16:07:06.225284 2576 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b027c9b2-b093-42d8-b692-876a5a4c3074-dshm\") on node \"ip-10-0-133-210.ec2.internal\" DevicePath \"\"" Jul 30 16:07:06.225457 ip-10-0-133-210 kubenswrapper[2576]: I0730 16:07:06.225294 2576 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b027c9b2-b093-42d8-b692-876a5a4c3074-home\") on node \"ip-10-0-133-210.ec2.internal\" DevicePath \"\"" Jul 30 16:07:06.225457 ip-10-0-133-210 kubenswrapper[2576]: I0730 16:07:06.225303 2576 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b027c9b2-b093-42d8-b692-876a5a4c3074-model-cache\") on node \"ip-10-0-133-210.ec2.internal\" DevicePath \"\"" Jul 30 16:07:06.612701 ip-10-0-133-210 kubenswrapper[2576]: I0730 16:07:06.612616 2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b45cbb00-6502-4064-baed-ab959e1e592c" path="/var/lib/kubelet/pods/b45cbb00-6502-4064-baed-ab959e1e592c/volumes" Jul 30 16:07:06.681198 ip-10-0-133-210 kubenswrapper[2576]: I0730 16:07:06.681162 2576 generic.go:356] "Generic (PLEG): container finished" podID="b027c9b2-b093-42d8-b692-876a5a4c3074" containerID="ad0bc2095856886bae3b224a05c1916414dd6331715f339fe77a7495e93c5551" exitCode=0 Jul 30 16:07:06.681396 ip-10-0-133-210 kubenswrapper[2576]: I0730 16:07:06.681212 2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-5468b9/auth-test-model-5468b9-kserve-5bdbdc4d8c-cmg6d" Jul 30 16:07:06.681396 ip-10-0-133-210 kubenswrapper[2576]: I0730 16:07:06.681241 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-5468b9/auth-test-model-5468b9-kserve-5bdbdc4d8c-cmg6d" event={"ID":"b027c9b2-b093-42d8-b692-876a5a4c3074","Type":"ContainerDied","Data":"ad0bc2095856886bae3b224a05c1916414dd6331715f339fe77a7495e93c5551"} Jul 30 16:07:06.681396 ip-10-0-133-210 kubenswrapper[2576]: I0730 16:07:06.681281 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-5468b9/auth-test-model-5468b9-kserve-5bdbdc4d8c-cmg6d" event={"ID":"b027c9b2-b093-42d8-b692-876a5a4c3074","Type":"ContainerDied","Data":"6ceb989e5d35983ecd86d3caa064e101ab8166f1390ec4e381684e923987cc13"} Jul 30 16:07:06.681396 ip-10-0-133-210 kubenswrapper[2576]: I0730 16:07:06.681304 2576 scope.go:117] "RemoveContainer" containerID="ad0bc2095856886bae3b224a05c1916414dd6331715f339fe77a7495e93c5551" Jul 30 16:07:06.689240 ip-10-0-133-210 kubenswrapper[2576]: I0730 16:07:06.689221 2576 scope.go:117] "RemoveContainer" containerID="ad0bc2095856886bae3b224a05c1916414dd6331715f339fe77a7495e93c5551" Jul 30 16:07:06.689505 ip-10-0-133-210 kubenswrapper[2576]: E0730 16:07:06.689485 2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ad0bc2095856886bae3b224a05c1916414dd6331715f339fe77a7495e93c5551\": container with ID starting with ad0bc2095856886bae3b224a05c1916414dd6331715f339fe77a7495e93c5551 not found: ID does not exist" containerID="ad0bc2095856886bae3b224a05c1916414dd6331715f339fe77a7495e93c5551" Jul 30 16:07:06.689569 ip-10-0-133-210 kubenswrapper[2576]: I0730 16:07:06.689517 2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ad0bc2095856886bae3b224a05c1916414dd6331715f339fe77a7495e93c5551"} err="failed to get container status \"ad0bc2095856886bae3b224a05c1916414dd6331715f339fe77a7495e93c5551\": rpc error: code = NotFound desc = could not find container \"ad0bc2095856886bae3b224a05c1916414dd6331715f339fe77a7495e93c5551\": container with ID starting with ad0bc2095856886bae3b224a05c1916414dd6331715f339fe77a7495e93c5551 not found: ID does not exist" Jul 30 16:07:06.698293 ip-10-0-133-210 kubenswrapper[2576]: I0730 16:07:06.698272 2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-5468b9/auth-test-model-5468b9-kserve-5bdbdc4d8c-cmg6d"] Jul 30 16:07:06.702175 ip-10-0-133-210 kubenswrapper[2576]: I0730 16:07:06.702154 2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-b-5468b9/auth-test-model-5468b9-kserve-5bdbdc4d8c-cmg6d"] Jul 30 16:07:08.612301 ip-10-0-133-210 kubenswrapper[2576]: I0730 16:07:08.612268 2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b027c9b2-b093-42d8-b692-876a5a4c3074" path="/var/lib/kubelet/pods/b027c9b2-b093-42d8-b692-876a5a4c3074/volumes" Jul 30 16:07:11.122556 ip-10-0-133-210 kubenswrapper[2576]: I0730 16:07:11.122433 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7j468_a2197c57-2dc6-410e-9910-a045dedcb8d9/konnectivity-agent/0.log" limit=1024 Jul 30 16:07:13.908685 ip-10-0-133-210 kubenswrapper[2576]: I0730 16:07:13.908653 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7j468_a2197c57-2dc6-410e-9910-a045dedcb8d9/konnectivity-agent/0.log" limit=1024 Jul 30 16:07:13.957815 ip-10-0-133-210 kubenswrapper[2576]: I0730 16:07:13.957787 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7j468_a2197c57-2dc6-410e-9910-a045dedcb8d9/konnectivity-agent/0.log" limit=1024 Jul 30 16:07:37.443760 ip-10-0-133-210 kubenswrapper[2576]: I0730 16:07:37.443720 2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-b-5468b9/sub-test-model-5468b9-kserve-67df7b77f5-4w2jk"] Jul 30 16:07:37.444372 ip-10-0-133-210 kubenswrapper[2576]: I0730 16:07:37.444210 2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b45cbb00-6502-4064-baed-ab959e1e592c" containerName="main" Jul 30 16:07:37.444372 ip-10-0-133-210 kubenswrapper[2576]: I0730 16:07:37.444229 2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="b45cbb00-6502-4064-baed-ab959e1e592c" containerName="main" Jul 30 16:07:37.444372 ip-10-0-133-210 kubenswrapper[2576]: I0730 16:07:37.444253 2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b027c9b2-b093-42d8-b692-876a5a4c3074" containerName="main" Jul 30 16:07:37.444372 ip-10-0-133-210 kubenswrapper[2576]: I0730 16:07:37.444262 2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="b027c9b2-b093-42d8-b692-876a5a4c3074" containerName="main" Jul 30 16:07:37.444372 ip-10-0-133-210 kubenswrapper[2576]: I0730 16:07:37.444340 2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="b027c9b2-b093-42d8-b692-876a5a4c3074" containerName="main" Jul 30 16:07:37.444372 ip-10-0-133-210 kubenswrapper[2576]: I0730 16:07:37.444355 2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="b45cbb00-6502-4064-baed-ab959e1e592c" containerName="main" Jul 30 16:07:37.447439 ip-10-0-133-210 kubenswrapper[2576]: I0730 16:07:37.447416 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-5468b9/sub-test-model-5468b9-kserve-67df7b77f5-4w2jk" Jul 30 16:07:37.450460 ip-10-0-133-210 kubenswrapper[2576]: I0730 16:07:37.450435 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-5468b9\"/\"kube-root-ca.crt\"" Jul 30 16:07:37.451766 ip-10-0-133-210 kubenswrapper[2576]: I0730 16:07:37.451747 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-5468b9\"/\"default-dockercfg-swktg\"" Jul 30 16:07:37.451868 ip-10-0-133-210 kubenswrapper[2576]: I0730 16:07:37.451773 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-5468b9\"/\"sub-test-model-5468b9-kserve-self-signed-certs\"" Jul 30 16:07:37.451868 ip-10-0-133-210 kubenswrapper[2576]: I0730 16:07:37.451837 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-5468b9\"/\"openshift-service-ca.crt\"" Jul 30 16:07:37.456599 ip-10-0-133-210 kubenswrapper[2576]: I0730 16:07:37.456576 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-5468b9/sub-test-model-5468b9-kserve-67df7b77f5-4w2jk"] Jul 30 16:07:37.485194 ip-10-0-133-210 kubenswrapper[2576]: I0730 16:07:37.485155 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/679a5c9a-a0ed-45f2-99ec-4436155f9f5b-tmp-dir\") pod \"sub-test-model-5468b9-kserve-67df7b77f5-4w2jk\" (UID: \"679a5c9a-a0ed-45f2-99ec-4436155f9f5b\") " pod="ai-tenant-e2e-shared-b-5468b9/sub-test-model-5468b9-kserve-67df7b77f5-4w2jk" Jul 30 16:07:37.485372 ip-10-0-133-210 kubenswrapper[2576]: I0730 16:07:37.485219 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/679a5c9a-a0ed-45f2-99ec-4436155f9f5b-model-cache\") pod \"sub-test-model-5468b9-kserve-67df7b77f5-4w2jk\" (UID: \"679a5c9a-a0ed-45f2-99ec-4436155f9f5b\") " pod="ai-tenant-e2e-shared-b-5468b9/sub-test-model-5468b9-kserve-67df7b77f5-4w2jk" Jul 30 16:07:37.485372 ip-10-0-133-210 kubenswrapper[2576]: I0730 16:07:37.485246 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/679a5c9a-a0ed-45f2-99ec-4436155f9f5b-tls-certs\") pod \"sub-test-model-5468b9-kserve-67df7b77f5-4w2jk\" (UID: \"679a5c9a-a0ed-45f2-99ec-4436155f9f5b\") " pod="ai-tenant-e2e-shared-b-5468b9/sub-test-model-5468b9-kserve-67df7b77f5-4w2jk" Jul 30 16:07:37.485372 ip-10-0-133-210 kubenswrapper[2576]: I0730 16:07:37.485277 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/679a5c9a-a0ed-45f2-99ec-4436155f9f5b-home\") pod \"sub-test-model-5468b9-kserve-67df7b77f5-4w2jk\" (UID: \"679a5c9a-a0ed-45f2-99ec-4436155f9f5b\") " pod="ai-tenant-e2e-shared-b-5468b9/sub-test-model-5468b9-kserve-67df7b77f5-4w2jk" Jul 30 16:07:37.485372 ip-10-0-133-210 kubenswrapper[2576]: I0730 16:07:37.485323 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xvhsw\" (UniqueName: \"kubernetes.io/projected/679a5c9a-a0ed-45f2-99ec-4436155f9f5b-kube-api-access-xvhsw\") pod \"sub-test-model-5468b9-kserve-67df7b77f5-4w2jk\" (UID: \"679a5c9a-a0ed-45f2-99ec-4436155f9f5b\") " pod="ai-tenant-e2e-shared-b-5468b9/sub-test-model-5468b9-kserve-67df7b77f5-4w2jk" Jul 30 16:07:37.485513 ip-10-0-133-210 kubenswrapper[2576]: I0730 16:07:37.485378 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/679a5c9a-a0ed-45f2-99ec-4436155f9f5b-dshm\") pod \"sub-test-model-5468b9-kserve-67df7b77f5-4w2jk\" (UID: \"679a5c9a-a0ed-45f2-99ec-4436155f9f5b\") " pod="ai-tenant-e2e-shared-b-5468b9/sub-test-model-5468b9-kserve-67df7b77f5-4w2jk" Jul 30 16:07:37.585812 ip-10-0-133-210 kubenswrapper[2576]: I0730 16:07:37.585771 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/679a5c9a-a0ed-45f2-99ec-4436155f9f5b-home\") pod \"sub-test-model-5468b9-kserve-67df7b77f5-4w2jk\" (UID: \"679a5c9a-a0ed-45f2-99ec-4436155f9f5b\") " pod="ai-tenant-e2e-shared-b-5468b9/sub-test-model-5468b9-kserve-67df7b77f5-4w2jk" Jul 30 16:07:37.585812 ip-10-0-133-210 kubenswrapper[2576]: I0730 16:07:37.585808 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xvhsw\" (UniqueName: \"kubernetes.io/projected/679a5c9a-a0ed-45f2-99ec-4436155f9f5b-kube-api-access-xvhsw\") pod \"sub-test-model-5468b9-kserve-67df7b77f5-4w2jk\" (UID: \"679a5c9a-a0ed-45f2-99ec-4436155f9f5b\") " pod="ai-tenant-e2e-shared-b-5468b9/sub-test-model-5468b9-kserve-67df7b77f5-4w2jk" Jul 30 16:07:37.586032 ip-10-0-133-210 kubenswrapper[2576]: I0730 16:07:37.585848 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/679a5c9a-a0ed-45f2-99ec-4436155f9f5b-dshm\") pod \"sub-test-model-5468b9-kserve-67df7b77f5-4w2jk\" (UID: \"679a5c9a-a0ed-45f2-99ec-4436155f9f5b\") " pod="ai-tenant-e2e-shared-b-5468b9/sub-test-model-5468b9-kserve-67df7b77f5-4w2jk" Jul 30 16:07:37.586032 ip-10-0-133-210 kubenswrapper[2576]: I0730 16:07:37.585896 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/679a5c9a-a0ed-45f2-99ec-4436155f9f5b-tmp-dir\") pod \"sub-test-model-5468b9-kserve-67df7b77f5-4w2jk\" (UID: \"679a5c9a-a0ed-45f2-99ec-4436155f9f5b\") " pod="ai-tenant-e2e-shared-b-5468b9/sub-test-model-5468b9-kserve-67df7b77f5-4w2jk" Jul 30 16:07:37.586032 ip-10-0-133-210 kubenswrapper[2576]: I0730 16:07:37.585946 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/679a5c9a-a0ed-45f2-99ec-4436155f9f5b-model-cache\") pod \"sub-test-model-5468b9-kserve-67df7b77f5-4w2jk\" (UID: \"679a5c9a-a0ed-45f2-99ec-4436155f9f5b\") " pod="ai-tenant-e2e-shared-b-5468b9/sub-test-model-5468b9-kserve-67df7b77f5-4w2jk" Jul 30 16:07:37.586032 ip-10-0-133-210 kubenswrapper[2576]: I0730 16:07:37.585970 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/679a5c9a-a0ed-45f2-99ec-4436155f9f5b-tls-certs\") pod \"sub-test-model-5468b9-kserve-67df7b77f5-4w2jk\" (UID: \"679a5c9a-a0ed-45f2-99ec-4436155f9f5b\") " pod="ai-tenant-e2e-shared-b-5468b9/sub-test-model-5468b9-kserve-67df7b77f5-4w2jk" Jul 30 16:07:37.586307 ip-10-0-133-210 kubenswrapper[2576]: I0730 16:07:37.586285 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/679a5c9a-a0ed-45f2-99ec-4436155f9f5b-home\") pod \"sub-test-model-5468b9-kserve-67df7b77f5-4w2jk\" (UID: \"679a5c9a-a0ed-45f2-99ec-4436155f9f5b\") " pod="ai-tenant-e2e-shared-b-5468b9/sub-test-model-5468b9-kserve-67df7b77f5-4w2jk" Jul 30 16:07:37.586374 ip-10-0-133-210 kubenswrapper[2576]: I0730 16:07:37.586321 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/679a5c9a-a0ed-45f2-99ec-4436155f9f5b-tmp-dir\") pod \"sub-test-model-5468b9-kserve-67df7b77f5-4w2jk\" (UID: \"679a5c9a-a0ed-45f2-99ec-4436155f9f5b\") " pod="ai-tenant-e2e-shared-b-5468b9/sub-test-model-5468b9-kserve-67df7b77f5-4w2jk" Jul 30 16:07:37.586432 ip-10-0-133-210 kubenswrapper[2576]: I0730 16:07:37.586380 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/679a5c9a-a0ed-45f2-99ec-4436155f9f5b-model-cache\") pod \"sub-test-model-5468b9-kserve-67df7b77f5-4w2jk\" (UID: \"679a5c9a-a0ed-45f2-99ec-4436155f9f5b\") " pod="ai-tenant-e2e-shared-b-5468b9/sub-test-model-5468b9-kserve-67df7b77f5-4w2jk" Jul 30 16:07:37.588262 ip-10-0-133-210 kubenswrapper[2576]: I0730 16:07:37.588238 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/679a5c9a-a0ed-45f2-99ec-4436155f9f5b-dshm\") pod \"sub-test-model-5468b9-kserve-67df7b77f5-4w2jk\" (UID: \"679a5c9a-a0ed-45f2-99ec-4436155f9f5b\") " pod="ai-tenant-e2e-shared-b-5468b9/sub-test-model-5468b9-kserve-67df7b77f5-4w2jk" Jul 30 16:07:37.588501 ip-10-0-133-210 kubenswrapper[2576]: I0730 16:07:37.588484 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/679a5c9a-a0ed-45f2-99ec-4436155f9f5b-tls-certs\") pod \"sub-test-model-5468b9-kserve-67df7b77f5-4w2jk\" (UID: \"679a5c9a-a0ed-45f2-99ec-4436155f9f5b\") " pod="ai-tenant-e2e-shared-b-5468b9/sub-test-model-5468b9-kserve-67df7b77f5-4w2jk" Jul 30 16:07:37.596194 ip-10-0-133-210 kubenswrapper[2576]: I0730 16:07:37.596170 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xvhsw\" (UniqueName: \"kubernetes.io/projected/679a5c9a-a0ed-45f2-99ec-4436155f9f5b-kube-api-access-xvhsw\") pod \"sub-test-model-5468b9-kserve-67df7b77f5-4w2jk\" (UID: \"679a5c9a-a0ed-45f2-99ec-4436155f9f5b\") " pod="ai-tenant-e2e-shared-b-5468b9/sub-test-model-5468b9-kserve-67df7b77f5-4w2jk" Jul 30 16:07:37.757434 ip-10-0-133-210 kubenswrapper[2576]: I0730 16:07:37.757393 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-5468b9/sub-test-model-5468b9-kserve-67df7b77f5-4w2jk" Jul 30 16:07:37.890122 ip-10-0-133-210 kubenswrapper[2576]: I0730 16:07:37.890081 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-5468b9/sub-test-model-5468b9-kserve-67df7b77f5-4w2jk"] Jul 30 16:07:37.892217 ip-10-0-133-210 kubenswrapper[2576]: W0730 16:07:37.892190 2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod679a5c9a_a0ed_45f2_99ec_4436155f9f5b.slice/crio-ec7ba19492958670476f72f99f2c90f373159f200343a8e7b2ee590c2f5d2ce6 WatchSource:0}: Error finding container ec7ba19492958670476f72f99f2c90f373159f200343a8e7b2ee590c2f5d2ce6: Status 404 returned error can't find the container with id ec7ba19492958670476f72f99f2c90f373159f200343a8e7b2ee590c2f5d2ce6 Jul 30 16:07:38.279554 ip-10-0-133-210 kubenswrapper[2576]: I0730 16:07:38.279519 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7j468_a2197c57-2dc6-410e-9910-a045dedcb8d9/konnectivity-agent/0.log" limit=1024 Jul 30 16:07:38.821978 ip-10-0-133-210 kubenswrapper[2576]: I0730 16:07:38.821941 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-5468b9/sub-test-model-5468b9-kserve-67df7b77f5-4w2jk" event={"ID":"679a5c9a-a0ed-45f2-99ec-4436155f9f5b","Type":"ContainerStarted","Data":"c7a7f099c36b51390b868dfd8b3757eec8dfd4ca55d2e2c9f3cbf4b452e1144c"} Jul 30 16:07:38.822404 ip-10-0-133-210 kubenswrapper[2576]: I0730 16:07:38.821988 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-5468b9/sub-test-model-5468b9-kserve-67df7b77f5-4w2jk" event={"ID":"679a5c9a-a0ed-45f2-99ec-4436155f9f5b","Type":"ContainerStarted","Data":"ec7ba19492958670476f72f99f2c90f373159f200343a8e7b2ee590c2f5d2ce6"} Jul 30 16:07:38.841963 ip-10-0-133-210 kubenswrapper[2576]: I0730 16:07:38.841909 2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-b-5468b9/sub-test-model-5468b9-kserve-67df7b77f5-4w2jk" podStartSLOduration=1.841892195 podStartE2EDuration="1.841892195s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 16:07:37 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 16:07:38.841157834 +0000 UTC m=+3440.766910480" watchObservedRunningTime="2026-07-30 16:07:38.841892195 +0000 UTC m=+3440.767644839" Jul 30 16:07:47.758081 ip-10-0-133-210 kubenswrapper[2576]: I0730 16:07:47.758042 2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-b-5468b9/sub-test-model-5468b9-kserve-67df7b77f5-4w2jk" Jul 30 16:07:47.758081 ip-10-0-133-210 kubenswrapper[2576]: I0730 16:07:47.758091 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-b-5468b9/sub-test-model-5468b9-kserve-67df7b77f5-4w2jk" Jul 30 16:07:47.770589 ip-10-0-133-210 kubenswrapper[2576]: I0730 16:07:47.770560 2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-b-5468b9/sub-test-model-5468b9-kserve-67df7b77f5-4w2jk" Jul 30 16:07:47.872913 ip-10-0-133-210 kubenswrapper[2576]: I0730 16:07:47.872885 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-b-5468b9/sub-test-model-5468b9-kserve-67df7b77f5-4w2jk" Jul 30 16:09:17.960848 ip-10-0-133-210 kubenswrapper[2576]: I0730 16:09:17.960810 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7j468_a2197c57-2dc6-410e-9910-a045dedcb8d9/konnectivity-agent/0.log" limit=1024 Jul 30 16:09:18.612893 ip-10-0-133-210 kubenswrapper[2576]: I0730 16:09:18.612866 2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-5468b9/sub-test-model-5468b9-kserve-67df7b77f5-4w2jk"] Jul 30 16:09:19.237283 ip-10-0-133-210 kubenswrapper[2576]: I0730 16:09:19.237244 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7j468_a2197c57-2dc6-410e-9910-a045dedcb8d9/konnectivity-agent/0.log" limit=1024 Jul 30 16:09:23.542435 ip-10-0-133-210 kubenswrapper[2576]: I0730 16:09:23.542390 2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-a-f2cf4d/rate-test-model-f2cf4d-kserve-b49889b6c-gpn8n"] Jul 30 16:09:23.544546 ip-10-0-133-210 kubenswrapper[2576]: I0730 16:09:23.544530 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-f2cf4d/rate-test-model-f2cf4d-kserve-b49889b6c-gpn8n" Jul 30 16:09:23.547229 ip-10-0-133-210 kubenswrapper[2576]: I0730 16:09:23.547205 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-f2cf4d\"/\"kube-root-ca.crt\"" Jul 30 16:09:23.548357 ip-10-0-133-210 kubenswrapper[2576]: I0730 16:09:23.548338 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-f2cf4d\"/\"rate-test-model-f2cf4d-kserve-self-signed-certs\"" Jul 30 16:09:23.548472 ip-10-0-133-210 kubenswrapper[2576]: I0730 16:09:23.548376 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-f2cf4d\"/\"openshift-service-ca.crt\"" Jul 30 16:09:23.548472 ip-10-0-133-210 kubenswrapper[2576]: I0730 16:09:23.548397 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-f2cf4d\"/\"default-dockercfg-lvkhp\"" Jul 30 16:09:23.555606 ip-10-0-133-210 kubenswrapper[2576]: I0730 16:09:23.555587 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-f2cf4d/rate-test-model-f2cf4d-kserve-b49889b6c-gpn8n"] Jul 30 16:09:23.633101 ip-10-0-133-210 kubenswrapper[2576]: I0730 16:09:23.633065 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7j468_a2197c57-2dc6-410e-9910-a045dedcb8d9/konnectivity-agent/0.log" limit=1024 Jul 30 16:09:23.663845 ip-10-0-133-210 kubenswrapper[2576]: I0730 16:09:23.663810 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zlnbl\" (UniqueName: \"kubernetes.io/projected/dcc9c9cc-d361-4df2-86c0-6d6401dcc13b-kube-api-access-zlnbl\") pod \"rate-test-model-f2cf4d-kserve-b49889b6c-gpn8n\" (UID: \"dcc9c9cc-d361-4df2-86c0-6d6401dcc13b\") " pod="ai-tenant-e2e-shared-a-f2cf4d/rate-test-model-f2cf4d-kserve-b49889b6c-gpn8n" Jul 30 16:09:23.663977 ip-10-0-133-210 kubenswrapper[2576]: I0730 16:09:23.663862 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/dcc9c9cc-d361-4df2-86c0-6d6401dcc13b-home\") pod \"rate-test-model-f2cf4d-kserve-b49889b6c-gpn8n\" (UID: \"dcc9c9cc-d361-4df2-86c0-6d6401dcc13b\") " pod="ai-tenant-e2e-shared-a-f2cf4d/rate-test-model-f2cf4d-kserve-b49889b6c-gpn8n" Jul 30 16:09:23.663977 ip-10-0-133-210 kubenswrapper[2576]: I0730 16:09:23.663882 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/dcc9c9cc-d361-4df2-86c0-6d6401dcc13b-model-cache\") pod \"rate-test-model-f2cf4d-kserve-b49889b6c-gpn8n\" (UID: \"dcc9c9cc-d361-4df2-86c0-6d6401dcc13b\") " pod="ai-tenant-e2e-shared-a-f2cf4d/rate-test-model-f2cf4d-kserve-b49889b6c-gpn8n" Jul 30 16:09:23.663977 ip-10-0-133-210 kubenswrapper[2576]: I0730 16:09:23.663950 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/dcc9c9cc-d361-4df2-86c0-6d6401dcc13b-tls-certs\") pod \"rate-test-model-f2cf4d-kserve-b49889b6c-gpn8n\" (UID: \"dcc9c9cc-d361-4df2-86c0-6d6401dcc13b\") " pod="ai-tenant-e2e-shared-a-f2cf4d/rate-test-model-f2cf4d-kserve-b49889b6c-gpn8n" Jul 30 16:09:23.664156 ip-10-0-133-210 kubenswrapper[2576]: I0730 16:09:23.663987 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/dcc9c9cc-d361-4df2-86c0-6d6401dcc13b-tmp-dir\") pod \"rate-test-model-f2cf4d-kserve-b49889b6c-gpn8n\" (UID: \"dcc9c9cc-d361-4df2-86c0-6d6401dcc13b\") " pod="ai-tenant-e2e-shared-a-f2cf4d/rate-test-model-f2cf4d-kserve-b49889b6c-gpn8n" Jul 30 16:09:23.664156 ip-10-0-133-210 kubenswrapper[2576]: I0730 16:09:23.664006 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/dcc9c9cc-d361-4df2-86c0-6d6401dcc13b-dshm\") pod \"rate-test-model-f2cf4d-kserve-b49889b6c-gpn8n\" (UID: \"dcc9c9cc-d361-4df2-86c0-6d6401dcc13b\") " pod="ai-tenant-e2e-shared-a-f2cf4d/rate-test-model-f2cf4d-kserve-b49889b6c-gpn8n" Jul 30 16:09:23.770128 ip-10-0-133-210 kubenswrapper[2576]: I0730 16:09:23.765322 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/dcc9c9cc-d361-4df2-86c0-6d6401dcc13b-home\") pod \"rate-test-model-f2cf4d-kserve-b49889b6c-gpn8n\" (UID: \"dcc9c9cc-d361-4df2-86c0-6d6401dcc13b\") " pod="ai-tenant-e2e-shared-a-f2cf4d/rate-test-model-f2cf4d-kserve-b49889b6c-gpn8n" Jul 30 16:09:23.770128 ip-10-0-133-210 kubenswrapper[2576]: I0730 16:09:23.765385 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/dcc9c9cc-d361-4df2-86c0-6d6401dcc13b-model-cache\") pod \"rate-test-model-f2cf4d-kserve-b49889b6c-gpn8n\" (UID: \"dcc9c9cc-d361-4df2-86c0-6d6401dcc13b\") " pod="ai-tenant-e2e-shared-a-f2cf4d/rate-test-model-f2cf4d-kserve-b49889b6c-gpn8n" Jul 30 16:09:23.770128 ip-10-0-133-210 kubenswrapper[2576]: I0730 16:09:23.765456 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/dcc9c9cc-d361-4df2-86c0-6d6401dcc13b-tls-certs\") pod \"rate-test-model-f2cf4d-kserve-b49889b6c-gpn8n\" (UID: \"dcc9c9cc-d361-4df2-86c0-6d6401dcc13b\") " pod="ai-tenant-e2e-shared-a-f2cf4d/rate-test-model-f2cf4d-kserve-b49889b6c-gpn8n" Jul 30 16:09:23.770128 ip-10-0-133-210 kubenswrapper[2576]: I0730 16:09:23.765491 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/dcc9c9cc-d361-4df2-86c0-6d6401dcc13b-tmp-dir\") pod \"rate-test-model-f2cf4d-kserve-b49889b6c-gpn8n\" (UID: \"dcc9c9cc-d361-4df2-86c0-6d6401dcc13b\") " pod="ai-tenant-e2e-shared-a-f2cf4d/rate-test-model-f2cf4d-kserve-b49889b6c-gpn8n" Jul 30 16:09:23.770128 ip-10-0-133-210 kubenswrapper[2576]: I0730 16:09:23.765519 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/dcc9c9cc-d361-4df2-86c0-6d6401dcc13b-dshm\") pod \"rate-test-model-f2cf4d-kserve-b49889b6c-gpn8n\" (UID: \"dcc9c9cc-d361-4df2-86c0-6d6401dcc13b\") " pod="ai-tenant-e2e-shared-a-f2cf4d/rate-test-model-f2cf4d-kserve-b49889b6c-gpn8n" Jul 30 16:09:23.770128 ip-10-0-133-210 kubenswrapper[2576]: I0730 16:09:23.765624 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zlnbl\" (UniqueName: \"kubernetes.io/projected/dcc9c9cc-d361-4df2-86c0-6d6401dcc13b-kube-api-access-zlnbl\") pod \"rate-test-model-f2cf4d-kserve-b49889b6c-gpn8n\" (UID: \"dcc9c9cc-d361-4df2-86c0-6d6401dcc13b\") " pod="ai-tenant-e2e-shared-a-f2cf4d/rate-test-model-f2cf4d-kserve-b49889b6c-gpn8n" Jul 30 16:09:23.770128 ip-10-0-133-210 kubenswrapper[2576]: I0730 16:09:23.766422 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/dcc9c9cc-d361-4df2-86c0-6d6401dcc13b-home\") pod \"rate-test-model-f2cf4d-kserve-b49889b6c-gpn8n\" (UID: \"dcc9c9cc-d361-4df2-86c0-6d6401dcc13b\") " pod="ai-tenant-e2e-shared-a-f2cf4d/rate-test-model-f2cf4d-kserve-b49889b6c-gpn8n" Jul 30 16:09:23.770128 ip-10-0-133-210 kubenswrapper[2576]: I0730 16:09:23.766665 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/dcc9c9cc-d361-4df2-86c0-6d6401dcc13b-model-cache\") pod \"rate-test-model-f2cf4d-kserve-b49889b6c-gpn8n\" (UID: \"dcc9c9cc-d361-4df2-86c0-6d6401dcc13b\") " pod="ai-tenant-e2e-shared-a-f2cf4d/rate-test-model-f2cf4d-kserve-b49889b6c-gpn8n" Jul 30 16:09:23.770128 ip-10-0-133-210 kubenswrapper[2576]: I0730 16:09:23.769779 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/dcc9c9cc-d361-4df2-86c0-6d6401dcc13b-tls-certs\") pod \"rate-test-model-f2cf4d-kserve-b49889b6c-gpn8n\" (UID: \"dcc9c9cc-d361-4df2-86c0-6d6401dcc13b\") " pod="ai-tenant-e2e-shared-a-f2cf4d/rate-test-model-f2cf4d-kserve-b49889b6c-gpn8n" Jul 30 16:09:23.770128 ip-10-0-133-210 kubenswrapper[2576]: I0730 16:09:23.770049 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/dcc9c9cc-d361-4df2-86c0-6d6401dcc13b-tmp-dir\") pod \"rate-test-model-f2cf4d-kserve-b49889b6c-gpn8n\" (UID: \"dcc9c9cc-d361-4df2-86c0-6d6401dcc13b\") " pod="ai-tenant-e2e-shared-a-f2cf4d/rate-test-model-f2cf4d-kserve-b49889b6c-gpn8n" Jul 30 16:09:23.772353 ip-10-0-133-210 kubenswrapper[2576]: I0730 16:09:23.772320 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/dcc9c9cc-d361-4df2-86c0-6d6401dcc13b-dshm\") pod \"rate-test-model-f2cf4d-kserve-b49889b6c-gpn8n\" (UID: \"dcc9c9cc-d361-4df2-86c0-6d6401dcc13b\") " pod="ai-tenant-e2e-shared-a-f2cf4d/rate-test-model-f2cf4d-kserve-b49889b6c-gpn8n" Jul 30 16:09:23.778116 ip-10-0-133-210 kubenswrapper[2576]: I0730 16:09:23.778081 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zlnbl\" (UniqueName: \"kubernetes.io/projected/dcc9c9cc-d361-4df2-86c0-6d6401dcc13b-kube-api-access-zlnbl\") pod \"rate-test-model-f2cf4d-kserve-b49889b6c-gpn8n\" (UID: \"dcc9c9cc-d361-4df2-86c0-6d6401dcc13b\") " pod="ai-tenant-e2e-shared-a-f2cf4d/rate-test-model-f2cf4d-kserve-b49889b6c-gpn8n" Jul 30 16:09:23.853448 ip-10-0-133-210 kubenswrapper[2576]: I0730 16:09:23.853366 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-f2cf4d/rate-test-model-f2cf4d-kserve-b49889b6c-gpn8n" Jul 30 16:09:23.983541 ip-10-0-133-210 kubenswrapper[2576]: I0730 16:09:23.983515 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-f2cf4d/rate-test-model-f2cf4d-kserve-b49889b6c-gpn8n"] Jul 30 16:09:23.985951 ip-10-0-133-210 kubenswrapper[2576]: W0730 16:09:23.985928 2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-poddcc9c9cc_d361_4df2_86c0_6d6401dcc13b.slice/crio-3284052f6aa4929bf30ddc50b4efffd0dbf8d52dc96c408756fe08121ef21504 WatchSource:0}: Error finding container 3284052f6aa4929bf30ddc50b4efffd0dbf8d52dc96c408756fe08121ef21504: Status 404 returned error can't find the container with id 3284052f6aa4929bf30ddc50b4efffd0dbf8d52dc96c408756fe08121ef21504 Jul 30 16:09:24.278177 ip-10-0-133-210 kubenswrapper[2576]: I0730 16:09:24.278144 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-f2cf4d/rate-test-model-f2cf4d-kserve-b49889b6c-gpn8n" event={"ID":"dcc9c9cc-d361-4df2-86c0-6d6401dcc13b","Type":"ContainerStarted","Data":"103abc91b8211893744fe2e97b5138415028c1842126869a05eaa3f437a01cd4"} Jul 30 16:09:24.278177 ip-10-0-133-210 kubenswrapper[2576]: I0730 16:09:24.278184 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-f2cf4d/rate-test-model-f2cf4d-kserve-b49889b6c-gpn8n" event={"ID":"dcc9c9cc-d361-4df2-86c0-6d6401dcc13b","Type":"ContainerStarted","Data":"3284052f6aa4929bf30ddc50b4efffd0dbf8d52dc96c408756fe08121ef21504"} Jul 30 16:09:24.295421 ip-10-0-133-210 kubenswrapper[2576]: I0730 16:09:24.295372 2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-a-f2cf4d/rate-test-model-f2cf4d-kserve-b49889b6c-gpn8n" podStartSLOduration=1.29535342 podStartE2EDuration="1.29535342s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 16:09:23 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 16:09:24.294257023 +0000 UTC m=+3546.220009672" watchObservedRunningTime="2026-07-30 16:09:24.29535342 +0000 UTC m=+3546.221106066" Jul 30 16:09:33.631521 ip-10-0-133-210 kubenswrapper[2576]: I0730 16:09:33.631460 2576 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-b-5468b9/sub-test-model-5468b9-kserve-67df7b77f5-4w2jk" podUID="679a5c9a-a0ed-45f2-99ec-4436155f9f5b" containerName="main" containerID="cri-o://c7a7f099c36b51390b868dfd8b3757eec8dfd4ca55d2e2c9f3cbf4b452e1144c" gracePeriod=45 Jul 30 16:09:33.853560 ip-10-0-133-210 kubenswrapper[2576]: I0730 16:09:33.853526 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-a-f2cf4d/rate-test-model-f2cf4d-kserve-b49889b6c-gpn8n" Jul 30 16:09:33.853737 ip-10-0-133-210 kubenswrapper[2576]: I0730 16:09:33.853581 2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-a-f2cf4d/rate-test-model-f2cf4d-kserve-b49889b6c-gpn8n" Jul 30 16:09:33.866827 ip-10-0-133-210 kubenswrapper[2576]: I0730 16:09:33.866806 2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-a-f2cf4d/rate-test-model-f2cf4d-kserve-b49889b6c-gpn8n" Jul 30 16:09:33.880769 ip-10-0-133-210 kubenswrapper[2576]: I0730 16:09:33.880741 2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-5468b9/sub-test-model-5468b9-kserve-67df7b77f5-4w2jk" Jul 30 16:09:34.053941 ip-10-0-133-210 kubenswrapper[2576]: I0730 16:09:34.053903 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/679a5c9a-a0ed-45f2-99ec-4436155f9f5b-tmp-dir\") pod \"679a5c9a-a0ed-45f2-99ec-4436155f9f5b\" (UID: \"679a5c9a-a0ed-45f2-99ec-4436155f9f5b\") " Jul 30 16:09:34.053941 ip-10-0-133-210 kubenswrapper[2576]: I0730 16:09:34.053948 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-xvhsw\" (UniqueName: \"kubernetes.io/projected/679a5c9a-a0ed-45f2-99ec-4436155f9f5b-kube-api-access-xvhsw\") pod \"679a5c9a-a0ed-45f2-99ec-4436155f9f5b\" (UID: \"679a5c9a-a0ed-45f2-99ec-4436155f9f5b\") " Jul 30 16:09:34.054242 ip-10-0-133-210 kubenswrapper[2576]: I0730 16:09:34.053987 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/679a5c9a-a0ed-45f2-99ec-4436155f9f5b-home\") pod \"679a5c9a-a0ed-45f2-99ec-4436155f9f5b\" (UID: \"679a5c9a-a0ed-45f2-99ec-4436155f9f5b\") " Jul 30 16:09:34.054242 ip-10-0-133-210 kubenswrapper[2576]: I0730 16:09:34.054022 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/679a5c9a-a0ed-45f2-99ec-4436155f9f5b-tls-certs\") pod \"679a5c9a-a0ed-45f2-99ec-4436155f9f5b\" (UID: \"679a5c9a-a0ed-45f2-99ec-4436155f9f5b\") " Jul 30 16:09:34.054242 ip-10-0-133-210 kubenswrapper[2576]: I0730 16:09:34.054051 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/679a5c9a-a0ed-45f2-99ec-4436155f9f5b-model-cache\") pod \"679a5c9a-a0ed-45f2-99ec-4436155f9f5b\" (UID: \"679a5c9a-a0ed-45f2-99ec-4436155f9f5b\") " Jul 30 16:09:34.054242 ip-10-0-133-210 kubenswrapper[2576]: I0730 16:09:34.054068 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/679a5c9a-a0ed-45f2-99ec-4436155f9f5b-dshm\") pod \"679a5c9a-a0ed-45f2-99ec-4436155f9f5b\" (UID: \"679a5c9a-a0ed-45f2-99ec-4436155f9f5b\") " Jul 30 16:09:34.054455 ip-10-0-133-210 kubenswrapper[2576]: I0730 16:09:34.054247 2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/679a5c9a-a0ed-45f2-99ec-4436155f9f5b-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "679a5c9a-a0ed-45f2-99ec-4436155f9f5b" (UID: "679a5c9a-a0ed-45f2-99ec-4436155f9f5b"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 30 16:09:34.054455 ip-10-0-133-210 kubenswrapper[2576]: I0730 16:09:34.054349 2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/679a5c9a-a0ed-45f2-99ec-4436155f9f5b-home" (OuterVolumeSpecName: "home") pod "679a5c9a-a0ed-45f2-99ec-4436155f9f5b" (UID: "679a5c9a-a0ed-45f2-99ec-4436155f9f5b"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 30 16:09:34.054455 ip-10-0-133-210 kubenswrapper[2576]: I0730 16:09:34.054367 2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/679a5c9a-a0ed-45f2-99ec-4436155f9f5b-model-cache" (OuterVolumeSpecName: "model-cache") pod "679a5c9a-a0ed-45f2-99ec-4436155f9f5b" (UID: "679a5c9a-a0ed-45f2-99ec-4436155f9f5b"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 30 16:09:34.054455 ip-10-0-133-210 kubenswrapper[2576]: I0730 16:09:34.054355 2576 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/679a5c9a-a0ed-45f2-99ec-4436155f9f5b-tmp-dir\") on node \"ip-10-0-133-210.ec2.internal\" DevicePath \"\"" Jul 30 16:09:34.056281 ip-10-0-133-210 kubenswrapper[2576]: I0730 16:09:34.056247 2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/679a5c9a-a0ed-45f2-99ec-4436155f9f5b-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "679a5c9a-a0ed-45f2-99ec-4436155f9f5b" (UID: "679a5c9a-a0ed-45f2-99ec-4436155f9f5b"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 30 16:09:34.056281 ip-10-0-133-210 kubenswrapper[2576]: I0730 16:09:34.056264 2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/679a5c9a-a0ed-45f2-99ec-4436155f9f5b-dshm" (OuterVolumeSpecName: "dshm") pod "679a5c9a-a0ed-45f2-99ec-4436155f9f5b" (UID: "679a5c9a-a0ed-45f2-99ec-4436155f9f5b"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 30 16:09:34.056426 ip-10-0-133-210 kubenswrapper[2576]: I0730 16:09:34.056287 2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/679a5c9a-a0ed-45f2-99ec-4436155f9f5b-kube-api-access-xvhsw" (OuterVolumeSpecName: "kube-api-access-xvhsw") pod "679a5c9a-a0ed-45f2-99ec-4436155f9f5b" (UID: "679a5c9a-a0ed-45f2-99ec-4436155f9f5b"). InnerVolumeSpecName "kube-api-access-xvhsw". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 30 16:09:34.155485 ip-10-0-133-210 kubenswrapper[2576]: I0730 16:09:34.155437 2576 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/679a5c9a-a0ed-45f2-99ec-4436155f9f5b-tls-certs\") on node \"ip-10-0-133-210.ec2.internal\" DevicePath \"\"" Jul 30 16:09:34.155485 ip-10-0-133-210 kubenswrapper[2576]: I0730 16:09:34.155478 2576 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/679a5c9a-a0ed-45f2-99ec-4436155f9f5b-model-cache\") on node \"ip-10-0-133-210.ec2.internal\" DevicePath \"\"" Jul 30 16:09:34.155485 ip-10-0-133-210 kubenswrapper[2576]: I0730 16:09:34.155487 2576 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/679a5c9a-a0ed-45f2-99ec-4436155f9f5b-dshm\") on node \"ip-10-0-133-210.ec2.internal\" DevicePath \"\"" Jul 30 16:09:34.155485 ip-10-0-133-210 kubenswrapper[2576]: I0730 16:09:34.155497 2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-xvhsw\" (UniqueName: \"kubernetes.io/projected/679a5c9a-a0ed-45f2-99ec-4436155f9f5b-kube-api-access-xvhsw\") on node \"ip-10-0-133-210.ec2.internal\" DevicePath \"\"" Jul 30 16:09:34.155485 ip-10-0-133-210 kubenswrapper[2576]: I0730 16:09:34.155507 2576 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/679a5c9a-a0ed-45f2-99ec-4436155f9f5b-home\") on node \"ip-10-0-133-210.ec2.internal\" DevicePath \"\"" Jul 30 16:09:34.321075 ip-10-0-133-210 kubenswrapper[2576]: I0730 16:09:34.320993 2576 generic.go:356] "Generic (PLEG): container finished" podID="679a5c9a-a0ed-45f2-99ec-4436155f9f5b" containerID="c7a7f099c36b51390b868dfd8b3757eec8dfd4ca55d2e2c9f3cbf4b452e1144c" exitCode=0 Jul 30 16:09:34.321075 ip-10-0-133-210 kubenswrapper[2576]: I0730 16:09:34.321035 2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-5468b9/sub-test-model-5468b9-kserve-67df7b77f5-4w2jk" Jul 30 16:09:34.321274 ip-10-0-133-210 kubenswrapper[2576]: I0730 16:09:34.321079 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-5468b9/sub-test-model-5468b9-kserve-67df7b77f5-4w2jk" event={"ID":"679a5c9a-a0ed-45f2-99ec-4436155f9f5b","Type":"ContainerDied","Data":"c7a7f099c36b51390b868dfd8b3757eec8dfd4ca55d2e2c9f3cbf4b452e1144c"} Jul 30 16:09:34.321274 ip-10-0-133-210 kubenswrapper[2576]: I0730 16:09:34.321151 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-5468b9/sub-test-model-5468b9-kserve-67df7b77f5-4w2jk" event={"ID":"679a5c9a-a0ed-45f2-99ec-4436155f9f5b","Type":"ContainerDied","Data":"ec7ba19492958670476f72f99f2c90f373159f200343a8e7b2ee590c2f5d2ce6"} Jul 30 16:09:34.321274 ip-10-0-133-210 kubenswrapper[2576]: I0730 16:09:34.321174 2576 scope.go:117] "RemoveContainer" containerID="c7a7f099c36b51390b868dfd8b3757eec8dfd4ca55d2e2c9f3cbf4b452e1144c" Jul 30 16:09:34.330728 ip-10-0-133-210 kubenswrapper[2576]: I0730 16:09:34.330708 2576 scope.go:117] "RemoveContainer" containerID="c7a7f099c36b51390b868dfd8b3757eec8dfd4ca55d2e2c9f3cbf4b452e1144c" Jul 30 16:09:34.330962 ip-10-0-133-210 kubenswrapper[2576]: E0730 16:09:34.330944 2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c7a7f099c36b51390b868dfd8b3757eec8dfd4ca55d2e2c9f3cbf4b452e1144c\": container with ID starting with c7a7f099c36b51390b868dfd8b3757eec8dfd4ca55d2e2c9f3cbf4b452e1144c not found: ID does not exist" containerID="c7a7f099c36b51390b868dfd8b3757eec8dfd4ca55d2e2c9f3cbf4b452e1144c" Jul 30 16:09:34.331024 ip-10-0-133-210 kubenswrapper[2576]: I0730 16:09:34.330973 2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c7a7f099c36b51390b868dfd8b3757eec8dfd4ca55d2e2c9f3cbf4b452e1144c"} err="failed to get container status \"c7a7f099c36b51390b868dfd8b3757eec8dfd4ca55d2e2c9f3cbf4b452e1144c\": rpc error: code = NotFound desc = could not find container \"c7a7f099c36b51390b868dfd8b3757eec8dfd4ca55d2e2c9f3cbf4b452e1144c\": container with ID starting with c7a7f099c36b51390b868dfd8b3757eec8dfd4ca55d2e2c9f3cbf4b452e1144c not found: ID does not exist" Jul 30 16:09:34.334752 ip-10-0-133-210 kubenswrapper[2576]: I0730 16:09:34.334731 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-a-f2cf4d/rate-test-model-f2cf4d-kserve-b49889b6c-gpn8n" Jul 30 16:09:34.341610 ip-10-0-133-210 kubenswrapper[2576]: I0730 16:09:34.341588 2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-5468b9/sub-test-model-5468b9-kserve-67df7b77f5-4w2jk"] Jul 30 16:09:34.344942 ip-10-0-133-210 kubenswrapper[2576]: I0730 16:09:34.344920 2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-b-5468b9/sub-test-model-5468b9-kserve-67df7b77f5-4w2jk"] Jul 30 16:09:34.612778 ip-10-0-133-210 kubenswrapper[2576]: I0730 16:09:34.612698 2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="679a5c9a-a0ed-45f2-99ec-4436155f9f5b" path="/var/lib/kubelet/pods/679a5c9a-a0ed-45f2-99ec-4436155f9f5b/volumes" Jul 30 16:09:41.421679 ip-10-0-133-210 kubenswrapper[2576]: I0730 16:09:41.421605 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7j468_a2197c57-2dc6-410e-9910-a045dedcb8d9/konnectivity-agent/0.log" limit=1024 Jul 30 16:11:10.958169 ip-10-0-133-210 kubenswrapper[2576]: I0730 16:11:10.958130 2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-f2cf4d/rate-test-model-f2cf4d-kserve-b49889b6c-gpn8n"] Jul 30 16:11:10.995095 ip-10-0-133-210 kubenswrapper[2576]: I0730 16:11:10.995060 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7j468_a2197c57-2dc6-410e-9910-a045dedcb8d9/konnectivity-agent/0.log" limit=1024 Jul 30 16:11:11.858250 ip-10-0-133-210 kubenswrapper[2576]: I0730 16:11:11.858218 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7j468_a2197c57-2dc6-410e-9910-a045dedcb8d9/konnectivity-agent/0.log" limit=1024 Jul 30 16:11:25.977284 ip-10-0-133-210 kubenswrapper[2576]: I0730 16:11:25.977240 2576 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-a-f2cf4d/rate-test-model-f2cf4d-kserve-b49889b6c-gpn8n" podUID="dcc9c9cc-d361-4df2-86c0-6d6401dcc13b" containerName="main" containerID="cri-o://103abc91b8211893744fe2e97b5138415028c1842126869a05eaa3f437a01cd4" gracePeriod=45 Jul 30 16:11:26.221705 ip-10-0-133-210 kubenswrapper[2576]: I0730 16:11:26.221680 2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-f2cf4d/rate-test-model-f2cf4d-kserve-b49889b6c-gpn8n" Jul 30 16:11:26.306888 ip-10-0-133-210 kubenswrapper[2576]: I0730 16:11:26.306782 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-zlnbl\" (UniqueName: \"kubernetes.io/projected/dcc9c9cc-d361-4df2-86c0-6d6401dcc13b-kube-api-access-zlnbl\") pod \"dcc9c9cc-d361-4df2-86c0-6d6401dcc13b\" (UID: \"dcc9c9cc-d361-4df2-86c0-6d6401dcc13b\") " Jul 30 16:11:26.306888 ip-10-0-133-210 kubenswrapper[2576]: I0730 16:11:26.306837 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/dcc9c9cc-d361-4df2-86c0-6d6401dcc13b-dshm\") pod \"dcc9c9cc-d361-4df2-86c0-6d6401dcc13b\" (UID: \"dcc9c9cc-d361-4df2-86c0-6d6401dcc13b\") " Jul 30 16:11:26.306888 ip-10-0-133-210 kubenswrapper[2576]: I0730 16:11:26.306865 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/dcc9c9cc-d361-4df2-86c0-6d6401dcc13b-tls-certs\") pod \"dcc9c9cc-d361-4df2-86c0-6d6401dcc13b\" (UID: \"dcc9c9cc-d361-4df2-86c0-6d6401dcc13b\") " Jul 30 16:11:26.307171 ip-10-0-133-210 kubenswrapper[2576]: I0730 16:11:26.307145 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/dcc9c9cc-d361-4df2-86c0-6d6401dcc13b-home\") pod \"dcc9c9cc-d361-4df2-86c0-6d6401dcc13b\" (UID: \"dcc9c9cc-d361-4df2-86c0-6d6401dcc13b\") " Jul 30 16:11:26.307228 ip-10-0-133-210 kubenswrapper[2576]: I0730 16:11:26.307200 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/dcc9c9cc-d361-4df2-86c0-6d6401dcc13b-tmp-dir\") pod \"dcc9c9cc-d361-4df2-86c0-6d6401dcc13b\" (UID: \"dcc9c9cc-d361-4df2-86c0-6d6401dcc13b\") " Jul 30 16:11:26.307285 ip-10-0-133-210 kubenswrapper[2576]: I0730 16:11:26.307241 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/dcc9c9cc-d361-4df2-86c0-6d6401dcc13b-model-cache\") pod \"dcc9c9cc-d361-4df2-86c0-6d6401dcc13b\" (UID: \"dcc9c9cc-d361-4df2-86c0-6d6401dcc13b\") " Jul 30 16:11:26.307510 ip-10-0-133-210 kubenswrapper[2576]: I0730 16:11:26.307479 2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/dcc9c9cc-d361-4df2-86c0-6d6401dcc13b-home" (OuterVolumeSpecName: "home") pod "dcc9c9cc-d361-4df2-86c0-6d6401dcc13b" (UID: "dcc9c9cc-d361-4df2-86c0-6d6401dcc13b"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 30 16:11:26.307656 ip-10-0-133-210 kubenswrapper[2576]: I0730 16:11:26.307631 2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/dcc9c9cc-d361-4df2-86c0-6d6401dcc13b-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "dcc9c9cc-d361-4df2-86c0-6d6401dcc13b" (UID: "dcc9c9cc-d361-4df2-86c0-6d6401dcc13b"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 30 16:11:26.307713 ip-10-0-133-210 kubenswrapper[2576]: I0730 16:11:26.307691 2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/dcc9c9cc-d361-4df2-86c0-6d6401dcc13b-model-cache" (OuterVolumeSpecName: "model-cache") pod "dcc9c9cc-d361-4df2-86c0-6d6401dcc13b" (UID: "dcc9c9cc-d361-4df2-86c0-6d6401dcc13b"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 30 16:11:26.309182 ip-10-0-133-210 kubenswrapper[2576]: I0730 16:11:26.309152 2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/dcc9c9cc-d361-4df2-86c0-6d6401dcc13b-kube-api-access-zlnbl" (OuterVolumeSpecName: "kube-api-access-zlnbl") pod "dcc9c9cc-d361-4df2-86c0-6d6401dcc13b" (UID: "dcc9c9cc-d361-4df2-86c0-6d6401dcc13b"). InnerVolumeSpecName "kube-api-access-zlnbl". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 30 16:11:26.309457 ip-10-0-133-210 kubenswrapper[2576]: I0730 16:11:26.309434 2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/dcc9c9cc-d361-4df2-86c0-6d6401dcc13b-dshm" (OuterVolumeSpecName: "dshm") pod "dcc9c9cc-d361-4df2-86c0-6d6401dcc13b" (UID: "dcc9c9cc-d361-4df2-86c0-6d6401dcc13b"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 30 16:11:26.309532 ip-10-0-133-210 kubenswrapper[2576]: I0730 16:11:26.309439 2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/dcc9c9cc-d361-4df2-86c0-6d6401dcc13b-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "dcc9c9cc-d361-4df2-86c0-6d6401dcc13b" (UID: "dcc9c9cc-d361-4df2-86c0-6d6401dcc13b"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 30 16:11:26.407882 ip-10-0-133-210 kubenswrapper[2576]: I0730 16:11:26.407840 2576 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/dcc9c9cc-d361-4df2-86c0-6d6401dcc13b-model-cache\") on node \"ip-10-0-133-210.ec2.internal\" DevicePath \"\"" Jul 30 16:11:26.407882 ip-10-0-133-210 kubenswrapper[2576]: I0730 16:11:26.407873 2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-zlnbl\" (UniqueName: \"kubernetes.io/projected/dcc9c9cc-d361-4df2-86c0-6d6401dcc13b-kube-api-access-zlnbl\") on node \"ip-10-0-133-210.ec2.internal\" DevicePath \"\"" Jul 30 16:11:26.407882 ip-10-0-133-210 kubenswrapper[2576]: I0730 16:11:26.407884 2576 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/dcc9c9cc-d361-4df2-86c0-6d6401dcc13b-dshm\") on node \"ip-10-0-133-210.ec2.internal\" DevicePath \"\"" Jul 30 16:11:26.407882 ip-10-0-133-210 kubenswrapper[2576]: I0730 16:11:26.407894 2576 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/dcc9c9cc-d361-4df2-86c0-6d6401dcc13b-tls-certs\") on node \"ip-10-0-133-210.ec2.internal\" DevicePath \"\"" Jul 30 16:11:26.408196 ip-10-0-133-210 kubenswrapper[2576]: I0730 16:11:26.407903 2576 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/dcc9c9cc-d361-4df2-86c0-6d6401dcc13b-home\") on node \"ip-10-0-133-210.ec2.internal\" DevicePath \"\"" Jul 30 16:11:26.408196 ip-10-0-133-210 kubenswrapper[2576]: I0730 16:11:26.407911 2576 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/dcc9c9cc-d361-4df2-86c0-6d6401dcc13b-tmp-dir\") on node \"ip-10-0-133-210.ec2.internal\" DevicePath \"\"" Jul 30 16:11:26.800090 ip-10-0-133-210 kubenswrapper[2576]: I0730 16:11:26.800056 2576 generic.go:356] "Generic (PLEG): container finished" podID="dcc9c9cc-d361-4df2-86c0-6d6401dcc13b" containerID="103abc91b8211893744fe2e97b5138415028c1842126869a05eaa3f437a01cd4" exitCode=0 Jul 30 16:11:26.800272 ip-10-0-133-210 kubenswrapper[2576]: I0730 16:11:26.800136 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-f2cf4d/rate-test-model-f2cf4d-kserve-b49889b6c-gpn8n" event={"ID":"dcc9c9cc-d361-4df2-86c0-6d6401dcc13b","Type":"ContainerDied","Data":"103abc91b8211893744fe2e97b5138415028c1842126869a05eaa3f437a01cd4"} Jul 30 16:11:26.800272 ip-10-0-133-210 kubenswrapper[2576]: I0730 16:11:26.800168 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-f2cf4d/rate-test-model-f2cf4d-kserve-b49889b6c-gpn8n" event={"ID":"dcc9c9cc-d361-4df2-86c0-6d6401dcc13b","Type":"ContainerDied","Data":"3284052f6aa4929bf30ddc50b4efffd0dbf8d52dc96c408756fe08121ef21504"} Jul 30 16:11:26.800272 ip-10-0-133-210 kubenswrapper[2576]: I0730 16:11:26.800185 2576 scope.go:117] "RemoveContainer" containerID="103abc91b8211893744fe2e97b5138415028c1842126869a05eaa3f437a01cd4" Jul 30 16:11:26.800272 ip-10-0-133-210 kubenswrapper[2576]: I0730 16:11:26.800139 2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-f2cf4d/rate-test-model-f2cf4d-kserve-b49889b6c-gpn8n" Jul 30 16:11:26.808668 ip-10-0-133-210 kubenswrapper[2576]: I0730 16:11:26.808650 2576 scope.go:117] "RemoveContainer" containerID="103abc91b8211893744fe2e97b5138415028c1842126869a05eaa3f437a01cd4" Jul 30 16:11:26.808927 ip-10-0-133-210 kubenswrapper[2576]: E0730 16:11:26.808906 2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"103abc91b8211893744fe2e97b5138415028c1842126869a05eaa3f437a01cd4\": container with ID starting with 103abc91b8211893744fe2e97b5138415028c1842126869a05eaa3f437a01cd4 not found: ID does not exist" containerID="103abc91b8211893744fe2e97b5138415028c1842126869a05eaa3f437a01cd4" Jul 30 16:11:26.808997 ip-10-0-133-210 kubenswrapper[2576]: I0730 16:11:26.808939 2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"103abc91b8211893744fe2e97b5138415028c1842126869a05eaa3f437a01cd4"} err="failed to get container status \"103abc91b8211893744fe2e97b5138415028c1842126869a05eaa3f437a01cd4\": rpc error: code = NotFound desc = could not find container \"103abc91b8211893744fe2e97b5138415028c1842126869a05eaa3f437a01cd4\": container with ID starting with 103abc91b8211893744fe2e97b5138415028c1842126869a05eaa3f437a01cd4 not found: ID does not exist" Jul 30 16:11:26.816040 ip-10-0-133-210 kubenswrapper[2576]: I0730 16:11:26.816015 2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-f2cf4d/rate-test-model-f2cf4d-kserve-b49889b6c-gpn8n"] Jul 30 16:11:26.819277 ip-10-0-133-210 kubenswrapper[2576]: I0730 16:11:26.819257 2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-a-f2cf4d/rate-test-model-f2cf4d-kserve-b49889b6c-gpn8n"] Jul 30 16:11:28.612951 ip-10-0-133-210 kubenswrapper[2576]: I0730 16:11:28.612916 2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="dcc9c9cc-d361-4df2-86c0-6d6401dcc13b" path="/var/lib/kubelet/pods/dcc9c9cc-d361-4df2-86c0-6d6401dcc13b/volumes" Jul 30 16:11:31.399662 ip-10-0-133-210 kubenswrapper[2576]: I0730 16:11:31.399623 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7j468_a2197c57-2dc6-410e-9910-a045dedcb8d9/konnectivity-agent/0.log" limit=1024 Jul 30 16:11:31.599912 ip-10-0-133-210 kubenswrapper[2576]: I0730 16:11:31.599878 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7j468_a2197c57-2dc6-410e-9910-a045dedcb8d9/konnectivity-agent/0.log" limit=1024 Jul 30 16:11:31.924593 ip-10-0-133-210 kubenswrapper[2576]: I0730 16:11:31.924560 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7j468_a2197c57-2dc6-410e-9910-a045dedcb8d9/konnectivity-agent/0.log" limit=1024 Jul 30 16:11:32.025336 ip-10-0-133-210 kubenswrapper[2576]: I0730 16:11:32.025288 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7j468_a2197c57-2dc6-410e-9910-a045dedcb8d9/konnectivity-agent/0.log" limit=1024 Jul 30 16:11:34.994658 ip-10-0-133-210 kubenswrapper[2576]: I0730 16:11:34.994630 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7j468_a2197c57-2dc6-410e-9910-a045dedcb8d9/konnectivity-agent/0.log" limit=1024 Jul 30 16:11:35.095971 ip-10-0-133-210 kubenswrapper[2576]: I0730 16:11:35.095931 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7j468_a2197c57-2dc6-410e-9910-a045dedcb8d9/konnectivity-agent/0.log" limit=1024 Jul 30 16:11:36.234738 ip-10-0-133-210 kubenswrapper[2576]: I0730 16:11:36.234703 2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-a-6bc161-585f798855-75nn6"] Jul 30 16:11:36.235179 ip-10-0-133-210 kubenswrapper[2576]: I0730 16:11:36.235158 2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="dcc9c9cc-d361-4df2-86c0-6d6401dcc13b" containerName="main" Jul 30 16:11:36.235251 ip-10-0-133-210 kubenswrapper[2576]: I0730 16:11:36.235183 2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="dcc9c9cc-d361-4df2-86c0-6d6401dcc13b" containerName="main" Jul 30 16:11:36.235251 ip-10-0-133-210 kubenswrapper[2576]: I0730 16:11:36.235205 2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="679a5c9a-a0ed-45f2-99ec-4436155f9f5b" containerName="main" Jul 30 16:11:36.235251 ip-10-0-133-210 kubenswrapper[2576]: I0730 16:11:36.235215 2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="679a5c9a-a0ed-45f2-99ec-4436155f9f5b" containerName="main" Jul 30 16:11:36.235365 ip-10-0-133-210 kubenswrapper[2576]: I0730 16:11:36.235300 2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="679a5c9a-a0ed-45f2-99ec-4436155f9f5b" containerName="main" Jul 30 16:11:36.235365 ip-10-0-133-210 kubenswrapper[2576]: I0730 16:11:36.235317 2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="dcc9c9cc-d361-4df2-86c0-6d6401dcc13b" containerName="main" Jul 30 16:11:36.238252 ip-10-0-133-210 kubenswrapper[2576]: I0730 16:11:36.238233 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-6bc161-585f798855-75nn6" Jul 30 16:11:36.240921 ip-10-0-133-210 kubenswrapper[2576]: I0730 16:11:36.240898 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\"" Jul 30 16:11:36.240921 ip-10-0-133-210 kubenswrapper[2576]: I0730 16:11:36.240923 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-ipp-a-6bc161-serving-cert\"" Jul 30 16:11:36.241089 ip-10-0-133-210 kubenswrapper[2576]: I0730 16:11:36.240945 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-725n9\"" Jul 30 16:11:36.241945 ip-10-0-133-210 kubenswrapper[2576]: I0730 16:11:36.241930 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\"" Jul 30 16:11:36.246308 ip-10-0-133-210 kubenswrapper[2576]: I0730 16:11:36.246287 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-a-6bc161-585f798855-75nn6"] Jul 30 16:11:36.263985 ip-10-0-133-210 kubenswrapper[2576]: I0730 16:11:36.263963 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7j468_a2197c57-2dc6-410e-9910-a045dedcb8d9/konnectivity-agent/0.log" limit=1024 Jul 30 16:11:36.285367 ip-10-0-133-210 kubenswrapper[2576]: I0730 16:11:36.285345 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6xt4l\" (UniqueName: \"kubernetes.io/projected/c9a5dede-30f0-453b-abd1-32879f3b4961-kube-api-access-6xt4l\") pod \"maas-api-e2e-ipp-a-6bc161-585f798855-75nn6\" (UID: \"c9a5dede-30f0-453b-abd1-32879f3b4961\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-6bc161-585f798855-75nn6" Jul 30 16:11:36.285465 ip-10-0-133-210 kubenswrapper[2576]: I0730 16:11:36.285382 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c9a5dede-30f0-453b-abd1-32879f3b4961-maas-api-tls\") pod \"maas-api-e2e-ipp-a-6bc161-585f798855-75nn6\" (UID: \"c9a5dede-30f0-453b-abd1-32879f3b4961\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-6bc161-585f798855-75nn6" Jul 30 16:11:36.285465 ip-10-0-133-210 kubenswrapper[2576]: I0730 16:11:36.285412 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/c9a5dede-30f0-453b-abd1-32879f3b4961-openshift-service-ca\") pod \"maas-api-e2e-ipp-a-6bc161-585f798855-75nn6\" (UID: \"c9a5dede-30f0-453b-abd1-32879f3b4961\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-6bc161-585f798855-75nn6" Jul 30 16:11:36.386267 ip-10-0-133-210 kubenswrapper[2576]: I0730 16:11:36.386231 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6xt4l\" (UniqueName: \"kubernetes.io/projected/c9a5dede-30f0-453b-abd1-32879f3b4961-kube-api-access-6xt4l\") pod \"maas-api-e2e-ipp-a-6bc161-585f798855-75nn6\" (UID: \"c9a5dede-30f0-453b-abd1-32879f3b4961\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-6bc161-585f798855-75nn6" Jul 30 16:11:36.386267 ip-10-0-133-210 kubenswrapper[2576]: I0730 16:11:36.386267 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c9a5dede-30f0-453b-abd1-32879f3b4961-maas-api-tls\") pod \"maas-api-e2e-ipp-a-6bc161-585f798855-75nn6\" (UID: \"c9a5dede-30f0-453b-abd1-32879f3b4961\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-6bc161-585f798855-75nn6" Jul 30 16:11:36.386480 ip-10-0-133-210 kubenswrapper[2576]: I0730 16:11:36.386296 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/c9a5dede-30f0-453b-abd1-32879f3b4961-openshift-service-ca\") pod \"maas-api-e2e-ipp-a-6bc161-585f798855-75nn6\" (UID: \"c9a5dede-30f0-453b-abd1-32879f3b4961\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-6bc161-585f798855-75nn6" Jul 30 16:11:36.386480 ip-10-0-133-210 kubenswrapper[2576]: E0730 16:11:36.386410 2576 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-ipp-a-6bc161-serving-cert: secret "maas-api-e2e-ipp-a-6bc161-serving-cert" not found Jul 30 16:11:36.386564 ip-10-0-133-210 kubenswrapper[2576]: E0730 16:11:36.386492 2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c9a5dede-30f0-453b-abd1-32879f3b4961-maas-api-tls podName:c9a5dede-30f0-453b-abd1-32879f3b4961 nodeName:}" failed. No retries permitted until 2026-07-30 16:11:36.886470875 +0000 UTC m=+3678.812223502 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/c9a5dede-30f0-453b-abd1-32879f3b4961-maas-api-tls") pod "maas-api-e2e-ipp-a-6bc161-585f798855-75nn6" (UID: "c9a5dede-30f0-453b-abd1-32879f3b4961") : secret "maas-api-e2e-ipp-a-6bc161-serving-cert" not found Jul 30 16:11:36.386954 ip-10-0-133-210 kubenswrapper[2576]: I0730 16:11:36.386937 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/c9a5dede-30f0-453b-abd1-32879f3b4961-openshift-service-ca\") pod \"maas-api-e2e-ipp-a-6bc161-585f798855-75nn6\" (UID: \"c9a5dede-30f0-453b-abd1-32879f3b4961\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-6bc161-585f798855-75nn6" Jul 30 16:11:36.394929 ip-10-0-133-210 kubenswrapper[2576]: I0730 16:11:36.394900 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6xt4l\" (UniqueName: \"kubernetes.io/projected/c9a5dede-30f0-453b-abd1-32879f3b4961-kube-api-access-6xt4l\") pod \"maas-api-e2e-ipp-a-6bc161-585f798855-75nn6\" (UID: \"c9a5dede-30f0-453b-abd1-32879f3b4961\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-6bc161-585f798855-75nn6" Jul 30 16:11:36.891149 ip-10-0-133-210 kubenswrapper[2576]: I0730 16:11:36.891065 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c9a5dede-30f0-453b-abd1-32879f3b4961-maas-api-tls\") pod \"maas-api-e2e-ipp-a-6bc161-585f798855-75nn6\" (UID: \"c9a5dede-30f0-453b-abd1-32879f3b4961\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-6bc161-585f798855-75nn6" Jul 30 16:11:36.893896 ip-10-0-133-210 kubenswrapper[2576]: I0730 16:11:36.893866 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c9a5dede-30f0-453b-abd1-32879f3b4961-maas-api-tls\") pod \"maas-api-e2e-ipp-a-6bc161-585f798855-75nn6\" (UID: \"c9a5dede-30f0-453b-abd1-32879f3b4961\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-6bc161-585f798855-75nn6" Jul 30 16:11:37.152242 ip-10-0-133-210 kubenswrapper[2576]: I0730 16:11:37.152144 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-6bc161-585f798855-75nn6" Jul 30 16:11:37.286532 ip-10-0-133-210 kubenswrapper[2576]: I0730 16:11:37.286510 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-a-6bc161-585f798855-75nn6"] Jul 30 16:11:37.289074 ip-10-0-133-210 kubenswrapper[2576]: W0730 16:11:37.289049 2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc9a5dede_30f0_453b_abd1_32879f3b4961.slice/crio-3c53bcc4d8feb9815a9f5bc916da73a25769fc8a1f47fa9f56cec2b1cd78fac3 WatchSource:0}: Error finding container 3c53bcc4d8feb9815a9f5bc916da73a25769fc8a1f47fa9f56cec2b1cd78fac3: Status 404 returned error can't find the container with id 3c53bcc4d8feb9815a9f5bc916da73a25769fc8a1f47fa9f56cec2b1cd78fac3 Jul 30 16:11:37.290411 ip-10-0-133-210 kubenswrapper[2576]: I0730 16:11:37.290391 2576 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 30 16:11:37.853049 ip-10-0-133-210 kubenswrapper[2576]: I0730 16:11:37.853010 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-6bc161-585f798855-75nn6" event={"ID":"c9a5dede-30f0-453b-abd1-32879f3b4961","Type":"ContainerStarted","Data":"3c53bcc4d8feb9815a9f5bc916da73a25769fc8a1f47fa9f56cec2b1cd78fac3"} Jul 30 16:11:38.964570 ip-10-0-133-210 kubenswrapper[2576]: I0730 16:11:38.964530 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7j468_a2197c57-2dc6-410e-9910-a045dedcb8d9/konnectivity-agent/0.log" limit=1024 Jul 30 16:11:39.065130 ip-10-0-133-210 kubenswrapper[2576]: I0730 16:11:39.065079 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7j468_a2197c57-2dc6-410e-9910-a045dedcb8d9/konnectivity-agent/0.log" limit=1024 Jul 30 16:11:39.863001 ip-10-0-133-210 kubenswrapper[2576]: I0730 16:11:39.862911 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-6bc161-585f798855-75nn6" event={"ID":"c9a5dede-30f0-453b-abd1-32879f3b4961","Type":"ContainerStarted","Data":"8b3d59ec83ba7e177f5b34d67c47a5d668f6f2daaabe8a1444105b47e20d5784"} Jul 30 16:11:39.863194 ip-10-0-133-210 kubenswrapper[2576]: I0730 16:11:39.863025 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-6bc161-585f798855-75nn6" Jul 30 16:11:39.879740 ip-10-0-133-210 kubenswrapper[2576]: I0730 16:11:39.879690 2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-6bc161-585f798855-75nn6" podStartSLOduration=1.646237221 podStartE2EDuration="3.87967614s" totalImagesPullingTime="2.233438919s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 16:11:36 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 16:11:39.877838222 +0000 UTC m=+3681.803590884" watchObservedRunningTime="2026-07-30 16:11:39.87967614 +0000 UTC m=+3681.805428785" Jul 30 16:11:45.873910 ip-10-0-133-210 kubenswrapper[2576]: I0730 16:11:45.873882 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-6bc161-585f798855-75nn6" Jul 30 16:12:00.134730 ip-10-0-133-210 kubenswrapper[2576]: I0730 16:12:00.134691 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7j468_a2197c57-2dc6-410e-9910-a045dedcb8d9/konnectivity-agent/0.log" limit=1024 Jul 30 16:12:00.895125 ip-10-0-133-210 kubenswrapper[2576]: I0730 16:12:00.895069 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7j468_a2197c57-2dc6-410e-9910-a045dedcb8d9/konnectivity-agent/0.log" limit=1024 Jul 30 16:12:01.226888 ip-10-0-133-210 kubenswrapper[2576]: I0730 16:12:01.226855 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7j468_a2197c57-2dc6-410e-9910-a045dedcb8d9/konnectivity-agent/0.log" limit=1024 Jul 30 16:12:01.834623 ip-10-0-133-210 kubenswrapper[2576]: I0730 16:12:01.834588 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7j468_a2197c57-2dc6-410e-9910-a045dedcb8d9/konnectivity-agent/0.log" limit=1024 Jul 30 16:12:08.086771 ip-10-0-133-210 kubenswrapper[2576]: I0730 16:12:08.086733 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7j468_a2197c57-2dc6-410e-9910-a045dedcb8d9/konnectivity-agent/0.log" limit=1024 Jul 30 16:12:08.181468 ip-10-0-133-210 kubenswrapper[2576]: I0730 16:12:08.181420 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7j468_a2197c57-2dc6-410e-9910-a045dedcb8d9/konnectivity-agent/0.log" limit=1024 Jul 30 16:12:09.177479 ip-10-0-133-210 kubenswrapper[2576]: I0730 16:12:09.177443 2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-b-ae9970-67ffb857c4-mzpmc"] Jul 30 16:12:09.180486 ip-10-0-133-210 kubenswrapper[2576]: I0730 16:12:09.180468 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-ae9970-67ffb857c4-mzpmc" Jul 30 16:12:09.182908 ip-10-0-133-210 kubenswrapper[2576]: I0730 16:12:09.182889 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-ipp-b-ae9970-serving-cert\"" Jul 30 16:12:09.191905 ip-10-0-133-210 kubenswrapper[2576]: I0730 16:12:09.191879 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-b-ae9970-67ffb857c4-mzpmc"] Jul 30 16:12:09.214003 ip-10-0-133-210 kubenswrapper[2576]: I0730 16:12:09.213965 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7j468_a2197c57-2dc6-410e-9910-a045dedcb8d9/konnectivity-agent/0.log" limit=1024 Jul 30 16:12:09.268341 ip-10-0-133-210 kubenswrapper[2576]: I0730 16:12:09.268303 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b5307962-931c-41f6-a862-942f9283dee4-openshift-service-ca\") pod \"maas-api-e2e-ipp-b-ae9970-67ffb857c4-mzpmc\" (UID: \"b5307962-931c-41f6-a862-942f9283dee4\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-ae9970-67ffb857c4-mzpmc" Jul 30 16:12:09.268341 ip-10-0-133-210 kubenswrapper[2576]: I0730 16:12:09.268348 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-668hn\" (UniqueName: \"kubernetes.io/projected/b5307962-931c-41f6-a862-942f9283dee4-kube-api-access-668hn\") pod \"maas-api-e2e-ipp-b-ae9970-67ffb857c4-mzpmc\" (UID: \"b5307962-931c-41f6-a862-942f9283dee4\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-ae9970-67ffb857c4-mzpmc" Jul 30 16:12:09.268630 ip-10-0-133-210 kubenswrapper[2576]: I0730 16:12:09.268437 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b5307962-931c-41f6-a862-942f9283dee4-maas-api-tls\") pod \"maas-api-e2e-ipp-b-ae9970-67ffb857c4-mzpmc\" (UID: \"b5307962-931c-41f6-a862-942f9283dee4\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-ae9970-67ffb857c4-mzpmc" Jul 30 16:12:09.369617 ip-10-0-133-210 kubenswrapper[2576]: I0730 16:12:09.369584 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b5307962-931c-41f6-a862-942f9283dee4-maas-api-tls\") pod \"maas-api-e2e-ipp-b-ae9970-67ffb857c4-mzpmc\" (UID: \"b5307962-931c-41f6-a862-942f9283dee4\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-ae9970-67ffb857c4-mzpmc" Jul 30 16:12:09.369790 ip-10-0-133-210 kubenswrapper[2576]: I0730 16:12:09.369641 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b5307962-931c-41f6-a862-942f9283dee4-openshift-service-ca\") pod \"maas-api-e2e-ipp-b-ae9970-67ffb857c4-mzpmc\" (UID: \"b5307962-931c-41f6-a862-942f9283dee4\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-ae9970-67ffb857c4-mzpmc" Jul 30 16:12:09.369790 ip-10-0-133-210 kubenswrapper[2576]: I0730 16:12:09.369664 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-668hn\" (UniqueName: \"kubernetes.io/projected/b5307962-931c-41f6-a862-942f9283dee4-kube-api-access-668hn\") pod \"maas-api-e2e-ipp-b-ae9970-67ffb857c4-mzpmc\" (UID: \"b5307962-931c-41f6-a862-942f9283dee4\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-ae9970-67ffb857c4-mzpmc" Jul 30 16:12:09.370394 ip-10-0-133-210 kubenswrapper[2576]: I0730 16:12:09.370371 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b5307962-931c-41f6-a862-942f9283dee4-openshift-service-ca\") pod \"maas-api-e2e-ipp-b-ae9970-67ffb857c4-mzpmc\" (UID: \"b5307962-931c-41f6-a862-942f9283dee4\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-ae9970-67ffb857c4-mzpmc" Jul 30 16:12:09.372058 ip-10-0-133-210 kubenswrapper[2576]: I0730 16:12:09.372040 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b5307962-931c-41f6-a862-942f9283dee4-maas-api-tls\") pod \"maas-api-e2e-ipp-b-ae9970-67ffb857c4-mzpmc\" (UID: \"b5307962-931c-41f6-a862-942f9283dee4\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-ae9970-67ffb857c4-mzpmc" Jul 30 16:12:09.377444 ip-10-0-133-210 kubenswrapper[2576]: I0730 16:12:09.377421 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-668hn\" (UniqueName: \"kubernetes.io/projected/b5307962-931c-41f6-a862-942f9283dee4-kube-api-access-668hn\") pod \"maas-api-e2e-ipp-b-ae9970-67ffb857c4-mzpmc\" (UID: \"b5307962-931c-41f6-a862-942f9283dee4\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-ae9970-67ffb857c4-mzpmc" Jul 30 16:12:09.413880 ip-10-0-133-210 kubenswrapper[2576]: I0730 16:12:09.413846 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7j468_a2197c57-2dc6-410e-9910-a045dedcb8d9/konnectivity-agent/0.log" limit=1024 Jul 30 16:12:09.490873 ip-10-0-133-210 kubenswrapper[2576]: I0730 16:12:09.490835 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-ae9970-67ffb857c4-mzpmc" Jul 30 16:12:09.664780 ip-10-0-133-210 kubenswrapper[2576]: I0730 16:12:09.664749 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7j468_a2197c57-2dc6-410e-9910-a045dedcb8d9/konnectivity-agent/0.log" limit=1024 Jul 30 16:12:09.757752 ip-10-0-133-210 kubenswrapper[2576]: I0730 16:12:09.757643 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7j468_a2197c57-2dc6-410e-9910-a045dedcb8d9/konnectivity-agent/0.log" limit=1024 Jul 30 16:12:09.832858 ip-10-0-133-210 kubenswrapper[2576]: I0730 16:12:09.832828 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-b-ae9970-67ffb857c4-mzpmc"] Jul 30 16:12:09.835884 ip-10-0-133-210 kubenswrapper[2576]: W0730 16:12:09.835856 2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb5307962_931c_41f6_a862_942f9283dee4.slice/crio-dee7b256ecc8dce98e40f32634cb027f987e2fe9b52828789944e8da07bc57e2 WatchSource:0}: Error finding container dee7b256ecc8dce98e40f32634cb027f987e2fe9b52828789944e8da07bc57e2: Status 404 returned error can't find the container with id dee7b256ecc8dce98e40f32634cb027f987e2fe9b52828789944e8da07bc57e2 Jul 30 16:12:09.994490 ip-10-0-133-210 kubenswrapper[2576]: I0730 16:12:09.994454 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-ae9970-67ffb857c4-mzpmc" event={"ID":"b5307962-931c-41f6-a862-942f9283dee4","Type":"ContainerStarted","Data":"dee7b256ecc8dce98e40f32634cb027f987e2fe9b52828789944e8da07bc57e2"} Jul 30 16:12:10.999966 ip-10-0-133-210 kubenswrapper[2576]: I0730 16:12:10.999930 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-ae9970-67ffb857c4-mzpmc" event={"ID":"b5307962-931c-41f6-a862-942f9283dee4","Type":"ContainerStarted","Data":"0ea1fa719fc7074d3490f908653a28431df172af48c020978448ad9d82b6bc09"} Jul 30 16:12:11.000364 ip-10-0-133-210 kubenswrapper[2576]: I0730 16:12:11.000045 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-ae9970-67ffb857c4-mzpmc" Jul 30 16:12:11.017195 ip-10-0-133-210 kubenswrapper[2576]: I0730 16:12:11.017137 2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-ae9970-67ffb857c4-mzpmc" podStartSLOduration=1.670860259 podStartE2EDuration="2.01710398s" totalImagesPullingTime="346.243721ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 16:12:09 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 16:12:11.015742211 +0000 UTC m=+3712.941494869" watchObservedRunningTime="2026-07-30 16:12:11.01710398 +0000 UTC m=+3712.942856624" Jul 30 16:12:17.009881 ip-10-0-133-210 kubenswrapper[2576]: I0730 16:12:17.009845 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-ae9970-67ffb857c4-mzpmc" Jul 30 16:12:33.173603 ip-10-0-133-210 kubenswrapper[2576]: I0730 16:12:33.173567 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7j468_a2197c57-2dc6-410e-9910-a045dedcb8d9/konnectivity-agent/0.log" limit=1024 Jul 30 16:12:42.829557 ip-10-0-133-210 kubenswrapper[2576]: I0730 16:12:42.829525 2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-ipp-a-6bc161/ipp-route-6bc161-kserve-65c4dc464c-rcppn"] Jul 30 16:12:42.833139 ip-10-0-133-210 kubenswrapper[2576]: I0730 16:12:42.833117 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-ipp-a-6bc161/ipp-route-6bc161-kserve-65c4dc464c-rcppn" Jul 30 16:12:42.835879 ip-10-0-133-210 kubenswrapper[2576]: I0730 16:12:42.835856 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-ipp-a-6bc161\"/\"openshift-service-ca.crt\"" Jul 30 16:12:42.835879 ip-10-0-133-210 kubenswrapper[2576]: I0730 16:12:42.835853 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-ipp-a-6bc161\"/\"kube-root-ca.crt\"" Jul 30 16:12:42.837014 ip-10-0-133-210 kubenswrapper[2576]: I0730 16:12:42.836997 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-ipp-a-6bc161\"/\"ipp-route-6bc161-kserve-self-signed-certs\"" Jul 30 16:12:42.837137 ip-10-0-133-210 kubenswrapper[2576]: I0730 16:12:42.837038 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-ipp-a-6bc161\"/\"default-dockercfg-llqdl\"" Jul 30 16:12:42.841395 ip-10-0-133-210 kubenswrapper[2576]: I0730 16:12:42.841375 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-ipp-a-6bc161/ipp-route-6bc161-kserve-65c4dc464c-rcppn"] Jul 30 16:12:42.857451 ip-10-0-133-210 kubenswrapper[2576]: I0730 16:12:42.857420 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7j468_a2197c57-2dc6-410e-9910-a045dedcb8d9/konnectivity-agent/0.log" limit=1024 Jul 30 16:12:42.928017 ip-10-0-133-210 kubenswrapper[2576]: I0730 16:12:42.927977 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/5bfaf12f-9b5f-475b-97d4-e0bbdd5de426-tmp-dir\") pod \"ipp-route-6bc161-kserve-65c4dc464c-rcppn\" (UID: \"5bfaf12f-9b5f-475b-97d4-e0bbdd5de426\") " pod="ai-tenant-e2e-ipp-a-6bc161/ipp-route-6bc161-kserve-65c4dc464c-rcppn" Jul 30 16:12:42.928017 ip-10-0-133-210 kubenswrapper[2576]: I0730 16:12:42.928013 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/5bfaf12f-9b5f-475b-97d4-e0bbdd5de426-dshm\") pod \"ipp-route-6bc161-kserve-65c4dc464c-rcppn\" (UID: \"5bfaf12f-9b5f-475b-97d4-e0bbdd5de426\") " pod="ai-tenant-e2e-ipp-a-6bc161/ipp-route-6bc161-kserve-65c4dc464c-rcppn" Jul 30 16:12:42.928261 ip-10-0-133-210 kubenswrapper[2576]: I0730 16:12:42.928088 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/5bfaf12f-9b5f-475b-97d4-e0bbdd5de426-model-cache\") pod \"ipp-route-6bc161-kserve-65c4dc464c-rcppn\" (UID: \"5bfaf12f-9b5f-475b-97d4-e0bbdd5de426\") " pod="ai-tenant-e2e-ipp-a-6bc161/ipp-route-6bc161-kserve-65c4dc464c-rcppn" Jul 30 16:12:42.928261 ip-10-0-133-210 kubenswrapper[2576]: I0730 16:12:42.928141 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nn5w8\" (UniqueName: \"kubernetes.io/projected/5bfaf12f-9b5f-475b-97d4-e0bbdd5de426-kube-api-access-nn5w8\") pod \"ipp-route-6bc161-kserve-65c4dc464c-rcppn\" (UID: \"5bfaf12f-9b5f-475b-97d4-e0bbdd5de426\") " pod="ai-tenant-e2e-ipp-a-6bc161/ipp-route-6bc161-kserve-65c4dc464c-rcppn" Jul 30 16:12:42.928261 ip-10-0-133-210 kubenswrapper[2576]: I0730 16:12:42.928172 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/5bfaf12f-9b5f-475b-97d4-e0bbdd5de426-home\") pod \"ipp-route-6bc161-kserve-65c4dc464c-rcppn\" (UID: \"5bfaf12f-9b5f-475b-97d4-e0bbdd5de426\") " pod="ai-tenant-e2e-ipp-a-6bc161/ipp-route-6bc161-kserve-65c4dc464c-rcppn" Jul 30 16:12:42.928261 ip-10-0-133-210 kubenswrapper[2576]: I0730 16:12:42.928188 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/5bfaf12f-9b5f-475b-97d4-e0bbdd5de426-tls-certs\") pod \"ipp-route-6bc161-kserve-65c4dc464c-rcppn\" (UID: \"5bfaf12f-9b5f-475b-97d4-e0bbdd5de426\") " pod="ai-tenant-e2e-ipp-a-6bc161/ipp-route-6bc161-kserve-65c4dc464c-rcppn" Jul 30 16:12:43.028838 ip-10-0-133-210 kubenswrapper[2576]: I0730 16:12:43.028795 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/5bfaf12f-9b5f-475b-97d4-e0bbdd5de426-model-cache\") pod \"ipp-route-6bc161-kserve-65c4dc464c-rcppn\" (UID: \"5bfaf12f-9b5f-475b-97d4-e0bbdd5de426\") " pod="ai-tenant-e2e-ipp-a-6bc161/ipp-route-6bc161-kserve-65c4dc464c-rcppn" Jul 30 16:12:43.028838 ip-10-0-133-210 kubenswrapper[2576]: I0730 16:12:43.028840 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nn5w8\" (UniqueName: \"kubernetes.io/projected/5bfaf12f-9b5f-475b-97d4-e0bbdd5de426-kube-api-access-nn5w8\") pod \"ipp-route-6bc161-kserve-65c4dc464c-rcppn\" (UID: \"5bfaf12f-9b5f-475b-97d4-e0bbdd5de426\") " pod="ai-tenant-e2e-ipp-a-6bc161/ipp-route-6bc161-kserve-65c4dc464c-rcppn" Jul 30 16:12:43.029147 ip-10-0-133-210 kubenswrapper[2576]: I0730 16:12:43.028879 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/5bfaf12f-9b5f-475b-97d4-e0bbdd5de426-home\") pod \"ipp-route-6bc161-kserve-65c4dc464c-rcppn\" (UID: \"5bfaf12f-9b5f-475b-97d4-e0bbdd5de426\") " pod="ai-tenant-e2e-ipp-a-6bc161/ipp-route-6bc161-kserve-65c4dc464c-rcppn" Jul 30 16:12:43.029147 ip-10-0-133-210 kubenswrapper[2576]: I0730 16:12:43.028899 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/5bfaf12f-9b5f-475b-97d4-e0bbdd5de426-tls-certs\") pod \"ipp-route-6bc161-kserve-65c4dc464c-rcppn\" (UID: \"5bfaf12f-9b5f-475b-97d4-e0bbdd5de426\") " pod="ai-tenant-e2e-ipp-a-6bc161/ipp-route-6bc161-kserve-65c4dc464c-rcppn" Jul 30 16:12:43.029147 ip-10-0-133-210 kubenswrapper[2576]: I0730 16:12:43.028937 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/5bfaf12f-9b5f-475b-97d4-e0bbdd5de426-tmp-dir\") pod \"ipp-route-6bc161-kserve-65c4dc464c-rcppn\" (UID: \"5bfaf12f-9b5f-475b-97d4-e0bbdd5de426\") " pod="ai-tenant-e2e-ipp-a-6bc161/ipp-route-6bc161-kserve-65c4dc464c-rcppn" Jul 30 16:12:43.029147 ip-10-0-133-210 kubenswrapper[2576]: I0730 16:12:43.028960 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/5bfaf12f-9b5f-475b-97d4-e0bbdd5de426-dshm\") pod \"ipp-route-6bc161-kserve-65c4dc464c-rcppn\" (UID: \"5bfaf12f-9b5f-475b-97d4-e0bbdd5de426\") " pod="ai-tenant-e2e-ipp-a-6bc161/ipp-route-6bc161-kserve-65c4dc464c-rcppn" Jul 30 16:12:43.029380 ip-10-0-133-210 kubenswrapper[2576]: I0730 16:12:43.029343 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/5bfaf12f-9b5f-475b-97d4-e0bbdd5de426-model-cache\") pod \"ipp-route-6bc161-kserve-65c4dc464c-rcppn\" (UID: \"5bfaf12f-9b5f-475b-97d4-e0bbdd5de426\") " pod="ai-tenant-e2e-ipp-a-6bc161/ipp-route-6bc161-kserve-65c4dc464c-rcppn" Jul 30 16:12:43.029380 ip-10-0-133-210 kubenswrapper[2576]: I0730 16:12:43.029368 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/5bfaf12f-9b5f-475b-97d4-e0bbdd5de426-home\") pod \"ipp-route-6bc161-kserve-65c4dc464c-rcppn\" (UID: \"5bfaf12f-9b5f-475b-97d4-e0bbdd5de426\") " pod="ai-tenant-e2e-ipp-a-6bc161/ipp-route-6bc161-kserve-65c4dc464c-rcppn" Jul 30 16:12:43.029464 ip-10-0-133-210 kubenswrapper[2576]: I0730 16:12:43.029434 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/5bfaf12f-9b5f-475b-97d4-e0bbdd5de426-tmp-dir\") pod \"ipp-route-6bc161-kserve-65c4dc464c-rcppn\" (UID: \"5bfaf12f-9b5f-475b-97d4-e0bbdd5de426\") " pod="ai-tenant-e2e-ipp-a-6bc161/ipp-route-6bc161-kserve-65c4dc464c-rcppn" Jul 30 16:12:43.031250 ip-10-0-133-210 kubenswrapper[2576]: I0730 16:12:43.031229 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/5bfaf12f-9b5f-475b-97d4-e0bbdd5de426-dshm\") pod \"ipp-route-6bc161-kserve-65c4dc464c-rcppn\" (UID: \"5bfaf12f-9b5f-475b-97d4-e0bbdd5de426\") " pod="ai-tenant-e2e-ipp-a-6bc161/ipp-route-6bc161-kserve-65c4dc464c-rcppn" Jul 30 16:12:43.031555 ip-10-0-133-210 kubenswrapper[2576]: I0730 16:12:43.031537 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/5bfaf12f-9b5f-475b-97d4-e0bbdd5de426-tls-certs\") pod \"ipp-route-6bc161-kserve-65c4dc464c-rcppn\" (UID: \"5bfaf12f-9b5f-475b-97d4-e0bbdd5de426\") " pod="ai-tenant-e2e-ipp-a-6bc161/ipp-route-6bc161-kserve-65c4dc464c-rcppn" Jul 30 16:12:43.036949 ip-10-0-133-210 kubenswrapper[2576]: I0730 16:12:43.036926 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nn5w8\" (UniqueName: \"kubernetes.io/projected/5bfaf12f-9b5f-475b-97d4-e0bbdd5de426-kube-api-access-nn5w8\") pod \"ipp-route-6bc161-kserve-65c4dc464c-rcppn\" (UID: \"5bfaf12f-9b5f-475b-97d4-e0bbdd5de426\") " pod="ai-tenant-e2e-ipp-a-6bc161/ipp-route-6bc161-kserve-65c4dc464c-rcppn" Jul 30 16:12:43.142971 ip-10-0-133-210 kubenswrapper[2576]: I0730 16:12:43.142873 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-ipp-a-6bc161/ipp-route-6bc161-kserve-65c4dc464c-rcppn" Jul 30 16:12:43.273515 ip-10-0-133-210 kubenswrapper[2576]: I0730 16:12:43.273486 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-ipp-a-6bc161/ipp-route-6bc161-kserve-65c4dc464c-rcppn"] Jul 30 16:12:43.276038 ip-10-0-133-210 kubenswrapper[2576]: W0730 16:12:43.276007 2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod5bfaf12f_9b5f_475b_97d4_e0bbdd5de426.slice/crio-ae11392baf4d0174c15bffb212b99b6a697fa72b9acd2f52e61c950b8e065dcb WatchSource:0}: Error finding container ae11392baf4d0174c15bffb212b99b6a697fa72b9acd2f52e61c950b8e065dcb: Status 404 returned error can't find the container with id ae11392baf4d0174c15bffb212b99b6a697fa72b9acd2f52e61c950b8e065dcb Jul 30 16:12:44.142104 ip-10-0-133-210 kubenswrapper[2576]: I0730 16:12:44.142067 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-ipp-a-6bc161/ipp-route-6bc161-kserve-65c4dc464c-rcppn" event={"ID":"5bfaf12f-9b5f-475b-97d4-e0bbdd5de426","Type":"ContainerStarted","Data":"289cd7c8807b433c79732a342433c002b7d12d51579cf13185553abdeebe8f84"} Jul 30 16:12:44.142104 ip-10-0-133-210 kubenswrapper[2576]: I0730 16:12:44.142121 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-ipp-a-6bc161/ipp-route-6bc161-kserve-65c4dc464c-rcppn" event={"ID":"5bfaf12f-9b5f-475b-97d4-e0bbdd5de426","Type":"ContainerStarted","Data":"ae11392baf4d0174c15bffb212b99b6a697fa72b9acd2f52e61c950b8e065dcb"} Jul 30 16:12:44.159394 ip-10-0-133-210 kubenswrapper[2576]: I0730 16:12:44.159326 2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-ipp-a-6bc161/ipp-route-6bc161-kserve-65c4dc464c-rcppn" podStartSLOduration=2.159309964 podStartE2EDuration="2.159309964s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 16:12:42 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 16:12:44.158225569 +0000 UTC m=+3746.083978213" watchObservedRunningTime="2026-07-30 16:12:44.159309964 +0000 UTC m=+3746.085062662" Jul 30 16:12:53.143358 ip-10-0-133-210 kubenswrapper[2576]: I0730 16:12:53.143307 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-ipp-a-6bc161/ipp-route-6bc161-kserve-65c4dc464c-rcppn" Jul 30 16:12:53.143832 ip-10-0-133-210 kubenswrapper[2576]: I0730 16:12:53.143374 2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-ipp-a-6bc161/ipp-route-6bc161-kserve-65c4dc464c-rcppn" Jul 30 16:12:53.156048 ip-10-0-133-210 kubenswrapper[2576]: I0730 16:12:53.156020 2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-ipp-a-6bc161/ipp-route-6bc161-kserve-65c4dc464c-rcppn" Jul 30 16:12:53.191995 ip-10-0-133-210 kubenswrapper[2576]: I0730 16:12:53.191966 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-ipp-a-6bc161/ipp-route-6bc161-kserve-65c4dc464c-rcppn" Jul 30 16:13:29.695508 ip-10-0-133-210 kubenswrapper[2576]: I0730 16:13:29.695468 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7j468_a2197c57-2dc6-410e-9910-a045dedcb8d9/konnectivity-agent/0.log" limit=1024 Jul 30 16:13:31.156216 ip-10-0-133-210 kubenswrapper[2576]: I0730 16:13:31.156178 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7j468_a2197c57-2dc6-410e-9910-a045dedcb8d9/konnectivity-agent/0.log" limit=1024 Jul 30 16:13:31.524705 ip-10-0-133-210 kubenswrapper[2576]: I0730 16:13:31.524665 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7j468_a2197c57-2dc6-410e-9910-a045dedcb8d9/konnectivity-agent/0.log" limit=1024 Jul 30 16:13:31.609567 ip-10-0-133-210 kubenswrapper[2576]: I0730 16:13:31.609533 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7j468_a2197c57-2dc6-410e-9910-a045dedcb8d9/konnectivity-agent/0.log" limit=1024 Jul 30 16:13:34.629171 ip-10-0-133-210 kubenswrapper[2576]: I0730 16:13:34.629135 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7j468_a2197c57-2dc6-410e-9910-a045dedcb8d9/konnectivity-agent/0.log" limit=1024 Jul 30 16:13:34.726615 ip-10-0-133-210 kubenswrapper[2576]: I0730 16:13:34.726571 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7j468_a2197c57-2dc6-410e-9910-a045dedcb8d9/konnectivity-agent/0.log" limit=1024 Jul 30 16:13:35.855341 ip-10-0-133-210 kubenswrapper[2576]: I0730 16:13:35.855311 2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-812b03-7786b95d85-h6skc"] Jul 30 16:13:35.858386 ip-10-0-133-210 kubenswrapper[2576]: I0730 16:13:35.858363 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-812b03-7786b95d85-h6skc" Jul 30 16:13:35.861589 ip-10-0-133-210 kubenswrapper[2576]: I0730 16:13:35.861568 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-ipp-cleanup-812b03-serving-cert\"" Jul 30 16:13:35.875223 ip-10-0-133-210 kubenswrapper[2576]: I0730 16:13:35.875201 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-812b03-7786b95d85-h6skc"] Jul 30 16:13:35.928362 ip-10-0-133-210 kubenswrapper[2576]: I0730 16:13:35.928330 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7j468_a2197c57-2dc6-410e-9910-a045dedcb8d9/konnectivity-agent/0.log" limit=1024 Jul 30 16:13:36.003170 ip-10-0-133-210 kubenswrapper[2576]: I0730 16:13:36.003130 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/245b87c4-7d34-4740-9fcd-6f946a720047-openshift-service-ca\") pod \"maas-api-e2e-ipp-cleanup-812b03-7786b95d85-h6skc\" (UID: \"245b87c4-7d34-4740-9fcd-6f946a720047\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-812b03-7786b95d85-h6skc" Jul 30 16:13:36.003361 ip-10-0-133-210 kubenswrapper[2576]: I0730 16:13:36.003197 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-krbl6\" (UniqueName: \"kubernetes.io/projected/245b87c4-7d34-4740-9fcd-6f946a720047-kube-api-access-krbl6\") pod \"maas-api-e2e-ipp-cleanup-812b03-7786b95d85-h6skc\" (UID: \"245b87c4-7d34-4740-9fcd-6f946a720047\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-812b03-7786b95d85-h6skc" Jul 30 16:13:36.003361 ip-10-0-133-210 kubenswrapper[2576]: I0730 16:13:36.003255 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/245b87c4-7d34-4740-9fcd-6f946a720047-maas-api-tls\") pod \"maas-api-e2e-ipp-cleanup-812b03-7786b95d85-h6skc\" (UID: \"245b87c4-7d34-4740-9fcd-6f946a720047\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-812b03-7786b95d85-h6skc" Jul 30 16:13:36.028965 ip-10-0-133-210 kubenswrapper[2576]: I0730 16:13:36.028931 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7j468_a2197c57-2dc6-410e-9910-a045dedcb8d9/konnectivity-agent/0.log" limit=1024 Jul 30 16:13:36.104465 ip-10-0-133-210 kubenswrapper[2576]: I0730 16:13:36.104428 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-krbl6\" (UniqueName: \"kubernetes.io/projected/245b87c4-7d34-4740-9fcd-6f946a720047-kube-api-access-krbl6\") pod \"maas-api-e2e-ipp-cleanup-812b03-7786b95d85-h6skc\" (UID: \"245b87c4-7d34-4740-9fcd-6f946a720047\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-812b03-7786b95d85-h6skc" Jul 30 16:13:36.104669 ip-10-0-133-210 kubenswrapper[2576]: I0730 16:13:36.104488 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/245b87c4-7d34-4740-9fcd-6f946a720047-maas-api-tls\") pod \"maas-api-e2e-ipp-cleanup-812b03-7786b95d85-h6skc\" (UID: \"245b87c4-7d34-4740-9fcd-6f946a720047\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-812b03-7786b95d85-h6skc" Jul 30 16:13:36.104669 ip-10-0-133-210 kubenswrapper[2576]: I0730 16:13:36.104522 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/245b87c4-7d34-4740-9fcd-6f946a720047-openshift-service-ca\") pod \"maas-api-e2e-ipp-cleanup-812b03-7786b95d85-h6skc\" (UID: \"245b87c4-7d34-4740-9fcd-6f946a720047\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-812b03-7786b95d85-h6skc" Jul 30 16:13:36.105229 ip-10-0-133-210 kubenswrapper[2576]: I0730 16:13:36.105207 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/245b87c4-7d34-4740-9fcd-6f946a720047-openshift-service-ca\") pod \"maas-api-e2e-ipp-cleanup-812b03-7786b95d85-h6skc\" (UID: \"245b87c4-7d34-4740-9fcd-6f946a720047\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-812b03-7786b95d85-h6skc" Jul 30 16:13:36.106899 ip-10-0-133-210 kubenswrapper[2576]: I0730 16:13:36.106850 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/245b87c4-7d34-4740-9fcd-6f946a720047-maas-api-tls\") pod \"maas-api-e2e-ipp-cleanup-812b03-7786b95d85-h6skc\" (UID: \"245b87c4-7d34-4740-9fcd-6f946a720047\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-812b03-7786b95d85-h6skc" Jul 30 16:13:36.112615 ip-10-0-133-210 kubenswrapper[2576]: I0730 16:13:36.112589 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-krbl6\" (UniqueName: \"kubernetes.io/projected/245b87c4-7d34-4740-9fcd-6f946a720047-kube-api-access-krbl6\") pod \"maas-api-e2e-ipp-cleanup-812b03-7786b95d85-h6skc\" (UID: \"245b87c4-7d34-4740-9fcd-6f946a720047\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-812b03-7786b95d85-h6skc" Jul 30 16:13:36.168812 ip-10-0-133-210 kubenswrapper[2576]: I0730 16:13:36.168750 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-812b03-7786b95d85-h6skc" Jul 30 16:13:36.298300 ip-10-0-133-210 kubenswrapper[2576]: I0730 16:13:36.298265 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-812b03-7786b95d85-h6skc"] Jul 30 16:13:36.302455 ip-10-0-133-210 kubenswrapper[2576]: W0730 16:13:36.302423 2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod245b87c4_7d34_4740_9fcd_6f946a720047.slice/crio-223fc8892754f570889d8c90d103c71d2272e29b887122462aa3b1d15ce8c56a WatchSource:0}: Error finding container 223fc8892754f570889d8c90d103c71d2272e29b887122462aa3b1d15ce8c56a: Status 404 returned error can't find the container with id 223fc8892754f570889d8c90d103c71d2272e29b887122462aa3b1d15ce8c56a Jul 30 16:13:36.329438 ip-10-0-133-210 kubenswrapper[2576]: I0730 16:13:36.329402 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7j468_a2197c57-2dc6-410e-9910-a045dedcb8d9/konnectivity-agent/0.log" limit=1024 Jul 30 16:13:36.370279 ip-10-0-133-210 kubenswrapper[2576]: I0730 16:13:36.370178 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-812b03-7786b95d85-h6skc" event={"ID":"245b87c4-7d34-4740-9fcd-6f946a720047","Type":"ContainerStarted","Data":"223fc8892754f570889d8c90d103c71d2272e29b887122462aa3b1d15ce8c56a"} Jul 30 16:13:36.428982 ip-10-0-133-210 kubenswrapper[2576]: I0730 16:13:36.428943 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7j468_a2197c57-2dc6-410e-9910-a045dedcb8d9/konnectivity-agent/0.log" limit=1024 Jul 30 16:13:37.375952 ip-10-0-133-210 kubenswrapper[2576]: I0730 16:13:37.375855 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-812b03-7786b95d85-h6skc" event={"ID":"245b87c4-7d34-4740-9fcd-6f946a720047","Type":"ContainerStarted","Data":"e46a585885b79c58b1b4130c8a30d72b74844957502e06d00241ff0c5378580a"} Jul 30 16:13:37.375952 ip-10-0-133-210 kubenswrapper[2576]: I0730 16:13:37.375924 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-812b03-7786b95d85-h6skc" Jul 30 16:13:37.392153 ip-10-0-133-210 kubenswrapper[2576]: I0730 16:13:37.392082 2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-812b03-7786b95d85-h6skc" podStartSLOduration=1.686101012 podStartE2EDuration="2.392066693s" totalImagesPullingTime="705.965681ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 16:13:35 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 16:13:37.391967056 +0000 UTC m=+3799.317719703" watchObservedRunningTime="2026-07-30 16:13:37.392066693 +0000 UTC m=+3799.317819340" Jul 30 16:13:43.386272 ip-10-0-133-210 kubenswrapper[2576]: I0730 16:13:43.386237 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-812b03-7786b95d85-h6skc" Jul 30 16:13:59.751535 ip-10-0-133-210 kubenswrapper[2576]: I0730 16:13:59.751495 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7j468_a2197c57-2dc6-410e-9910-a045dedcb8d9/konnectivity-agent/0.log" limit=1024 Jul 30 16:14:06.310888 ip-10-0-133-210 kubenswrapper[2576]: I0730 16:14:06.310850 2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-812b03-7786b95d85-h6skc"] Jul 30 16:14:06.311329 ip-10-0-133-210 kubenswrapper[2576]: I0730 16:14:06.311095 2576 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-812b03-7786b95d85-h6skc" podUID="245b87c4-7d34-4740-9fcd-6f946a720047" containerName="maas-api" containerID="cri-o://e46a585885b79c58b1b4130c8a30d72b74844957502e06d00241ff0c5378580a" gracePeriod=30 Jul 30 16:14:06.466637 ip-10-0-133-210 kubenswrapper[2576]: I0730 16:14:06.466600 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7j468_a2197c57-2dc6-410e-9910-a045dedcb8d9/konnectivity-agent/0.log" limit=1024 Jul 30 16:14:06.504606 ip-10-0-133-210 kubenswrapper[2576]: I0730 16:14:06.504568 2576 generic.go:356] "Generic (PLEG): container finished" podID="245b87c4-7d34-4740-9fcd-6f946a720047" containerID="e46a585885b79c58b1b4130c8a30d72b74844957502e06d00241ff0c5378580a" exitCode=0 Jul 30 16:14:06.504771 ip-10-0-133-210 kubenswrapper[2576]: I0730 16:14:06.504643 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-812b03-7786b95d85-h6skc" event={"ID":"245b87c4-7d34-4740-9fcd-6f946a720047","Type":"ContainerDied","Data":"e46a585885b79c58b1b4130c8a30d72b74844957502e06d00241ff0c5378580a"} Jul 30 16:14:06.559415 ip-10-0-133-210 kubenswrapper[2576]: I0730 16:14:06.559323 2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-812b03-7786b95d85-h6skc" Jul 30 16:14:06.561794 ip-10-0-133-210 kubenswrapper[2576]: I0730 16:14:06.561724 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7j468_a2197c57-2dc6-410e-9910-a045dedcb8d9/konnectivity-agent/0.log" limit=1024 Jul 30 16:14:06.632778 ip-10-0-133-210 kubenswrapper[2576]: I0730 16:14:06.632745 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7j468_a2197c57-2dc6-410e-9910-a045dedcb8d9/konnectivity-agent/0.log" limit=1024 Jul 30 16:14:06.674083 ip-10-0-133-210 kubenswrapper[2576]: I0730 16:14:06.674051 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-krbl6\" (UniqueName: \"kubernetes.io/projected/245b87c4-7d34-4740-9fcd-6f946a720047-kube-api-access-krbl6\") pod \"245b87c4-7d34-4740-9fcd-6f946a720047\" (UID: \"245b87c4-7d34-4740-9fcd-6f946a720047\") " Jul 30 16:14:06.674276 ip-10-0-133-210 kubenswrapper[2576]: I0730 16:14:06.674143 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/245b87c4-7d34-4740-9fcd-6f946a720047-maas-api-tls\") pod \"245b87c4-7d34-4740-9fcd-6f946a720047\" (UID: \"245b87c4-7d34-4740-9fcd-6f946a720047\") " Jul 30 16:14:06.674276 ip-10-0-133-210 kubenswrapper[2576]: I0730 16:14:06.674208 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/245b87c4-7d34-4740-9fcd-6f946a720047-openshift-service-ca\") pod \"245b87c4-7d34-4740-9fcd-6f946a720047\" (UID: \"245b87c4-7d34-4740-9fcd-6f946a720047\") " Jul 30 16:14:06.674572 ip-10-0-133-210 kubenswrapper[2576]: I0730 16:14:06.674537 2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/245b87c4-7d34-4740-9fcd-6f946a720047-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "245b87c4-7d34-4740-9fcd-6f946a720047" (UID: "245b87c4-7d34-4740-9fcd-6f946a720047"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 30 16:14:06.676415 ip-10-0-133-210 kubenswrapper[2576]: I0730 16:14:06.676390 2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/245b87c4-7d34-4740-9fcd-6f946a720047-kube-api-access-krbl6" (OuterVolumeSpecName: "kube-api-access-krbl6") pod "245b87c4-7d34-4740-9fcd-6f946a720047" (UID: "245b87c4-7d34-4740-9fcd-6f946a720047"). InnerVolumeSpecName "kube-api-access-krbl6". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 30 16:14:06.676513 ip-10-0-133-210 kubenswrapper[2576]: I0730 16:14:06.676442 2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/245b87c4-7d34-4740-9fcd-6f946a720047-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "245b87c4-7d34-4740-9fcd-6f946a720047" (UID: "245b87c4-7d34-4740-9fcd-6f946a720047"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 30 16:14:06.775210 ip-10-0-133-210 kubenswrapper[2576]: I0730 16:14:06.775178 2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-krbl6\" (UniqueName: \"kubernetes.io/projected/245b87c4-7d34-4740-9fcd-6f946a720047-kube-api-access-krbl6\") on node \"ip-10-0-133-210.ec2.internal\" DevicePath \"\"" Jul 30 16:14:06.775210 ip-10-0-133-210 kubenswrapper[2576]: I0730 16:14:06.775206 2576 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/245b87c4-7d34-4740-9fcd-6f946a720047-maas-api-tls\") on node \"ip-10-0-133-210.ec2.internal\" DevicePath \"\"" Jul 30 16:14:06.775210 ip-10-0-133-210 kubenswrapper[2576]: I0730 16:14:06.775216 2576 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/245b87c4-7d34-4740-9fcd-6f946a720047-openshift-service-ca\") on node \"ip-10-0-133-210.ec2.internal\" DevicePath \"\"" Jul 30 16:14:07.037992 ip-10-0-133-210 kubenswrapper[2576]: I0730 16:14:07.037953 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7j468_a2197c57-2dc6-410e-9910-a045dedcb8d9/konnectivity-agent/0.log" limit=1024 Jul 30 16:14:07.509896 ip-10-0-133-210 kubenswrapper[2576]: I0730 16:14:07.509865 2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-812b03-7786b95d85-h6skc" Jul 30 16:14:07.509896 ip-10-0-133-210 kubenswrapper[2576]: I0730 16:14:07.509875 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-812b03-7786b95d85-h6skc" event={"ID":"245b87c4-7d34-4740-9fcd-6f946a720047","Type":"ContainerDied","Data":"223fc8892754f570889d8c90d103c71d2272e29b887122462aa3b1d15ce8c56a"} Jul 30 16:14:07.510421 ip-10-0-133-210 kubenswrapper[2576]: I0730 16:14:07.509929 2576 scope.go:117] "RemoveContainer" containerID="e46a585885b79c58b1b4130c8a30d72b74844957502e06d00241ff0c5378580a" Jul 30 16:14:07.540138 ip-10-0-133-210 kubenswrapper[2576]: I0730 16:14:07.540091 2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-812b03-7786b95d85-h6skc"] Jul 30 16:14:07.546658 ip-10-0-133-210 kubenswrapper[2576]: I0730 16:14:07.546630 2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-812b03-7786b95d85-h6skc"] Jul 30 16:14:08.613282 ip-10-0-133-210 kubenswrapper[2576]: I0730 16:14:08.613251 2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="245b87c4-7d34-4740-9fcd-6f946a720047" path="/var/lib/kubelet/pods/245b87c4-7d34-4740-9fcd-6f946a720047/volumes" Jul 30 16:14:12.677334 ip-10-0-133-210 kubenswrapper[2576]: I0730 16:14:12.677287 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7j468_a2197c57-2dc6-410e-9910-a045dedcb8d9/konnectivity-agent/0.log" limit=1024 Jul 30 16:14:12.739246 ip-10-0-133-210 kubenswrapper[2576]: I0730 16:14:12.739212 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7j468_a2197c57-2dc6-410e-9910-a045dedcb8d9/konnectivity-agent/0.log" limit=1024 Jul 30 16:14:12.970970 ip-10-0-133-210 kubenswrapper[2576]: I0730 16:14:12.970940 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7j468_a2197c57-2dc6-410e-9910-a045dedcb8d9/konnectivity-agent/0.log" limit=1024 Jul 30 16:14:13.239392 ip-10-0-133-210 kubenswrapper[2576]: I0730 16:14:13.239302 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7j468_a2197c57-2dc6-410e-9910-a045dedcb8d9/konnectivity-agent/0.log" limit=1024 Jul 30 16:14:32.427169 ip-10-0-133-210 kubenswrapper[2576]: I0730 16:14:32.427132 2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-a-6bc161-585f798855-75nn6"] Jul 30 16:14:32.427694 ip-10-0-133-210 kubenswrapper[2576]: I0730 16:14:32.427366 2576 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-6bc161-585f798855-75nn6" podUID="c9a5dede-30f0-453b-abd1-32879f3b4961" containerName="maas-api" containerID="cri-o://8b3d59ec83ba7e177f5b34d67c47a5d668f6f2daaabe8a1444105b47e20d5784" gracePeriod=30 Jul 30 16:14:32.468334 ip-10-0-133-210 kubenswrapper[2576]: I0730 16:14:32.468301 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7j468_a2197c57-2dc6-410e-9910-a045dedcb8d9/konnectivity-agent/0.log" limit=1024 Jul 30 16:14:32.626001 ip-10-0-133-210 kubenswrapper[2576]: I0730 16:14:32.625962 2576 generic.go:356] "Generic (PLEG): container finished" podID="c9a5dede-30f0-453b-abd1-32879f3b4961" containerID="8b3d59ec83ba7e177f5b34d67c47a5d668f6f2daaabe8a1444105b47e20d5784" exitCode=0 Jul 30 16:14:32.626182 ip-10-0-133-210 kubenswrapper[2576]: I0730 16:14:32.626026 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-6bc161-585f798855-75nn6" event={"ID":"c9a5dede-30f0-453b-abd1-32879f3b4961","Type":"ContainerDied","Data":"8b3d59ec83ba7e177f5b34d67c47a5d668f6f2daaabe8a1444105b47e20d5784"} Jul 30 16:14:32.685012 ip-10-0-133-210 kubenswrapper[2576]: I0730 16:14:32.684950 2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-6bc161-585f798855-75nn6" Jul 30 16:14:32.804173 ip-10-0-133-210 kubenswrapper[2576]: I0730 16:14:32.804144 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c9a5dede-30f0-453b-abd1-32879f3b4961-maas-api-tls\") pod \"c9a5dede-30f0-453b-abd1-32879f3b4961\" (UID: \"c9a5dede-30f0-453b-abd1-32879f3b4961\") " Jul 30 16:14:32.804376 ip-10-0-133-210 kubenswrapper[2576]: I0730 16:14:32.804208 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-6xt4l\" (UniqueName: \"kubernetes.io/projected/c9a5dede-30f0-453b-abd1-32879f3b4961-kube-api-access-6xt4l\") pod \"c9a5dede-30f0-453b-abd1-32879f3b4961\" (UID: \"c9a5dede-30f0-453b-abd1-32879f3b4961\") " Jul 30 16:14:32.804376 ip-10-0-133-210 kubenswrapper[2576]: I0730 16:14:32.804276 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/c9a5dede-30f0-453b-abd1-32879f3b4961-openshift-service-ca\") pod \"c9a5dede-30f0-453b-abd1-32879f3b4961\" (UID: \"c9a5dede-30f0-453b-abd1-32879f3b4961\") " Jul 30 16:14:32.804640 ip-10-0-133-210 kubenswrapper[2576]: I0730 16:14:32.804609 2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c9a5dede-30f0-453b-abd1-32879f3b4961-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "c9a5dede-30f0-453b-abd1-32879f3b4961" (UID: "c9a5dede-30f0-453b-abd1-32879f3b4961"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 30 16:14:32.806530 ip-10-0-133-210 kubenswrapper[2576]: I0730 16:14:32.806495 2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c9a5dede-30f0-453b-abd1-32879f3b4961-kube-api-access-6xt4l" (OuterVolumeSpecName: "kube-api-access-6xt4l") pod "c9a5dede-30f0-453b-abd1-32879f3b4961" (UID: "c9a5dede-30f0-453b-abd1-32879f3b4961"). InnerVolumeSpecName "kube-api-access-6xt4l". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 30 16:14:32.806622 ip-10-0-133-210 kubenswrapper[2576]: I0730 16:14:32.806536 2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c9a5dede-30f0-453b-abd1-32879f3b4961-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "c9a5dede-30f0-453b-abd1-32879f3b4961" (UID: "c9a5dede-30f0-453b-abd1-32879f3b4961"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 30 16:14:32.904786 ip-10-0-133-210 kubenswrapper[2576]: I0730 16:14:32.904749 2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-6xt4l\" (UniqueName: \"kubernetes.io/projected/c9a5dede-30f0-453b-abd1-32879f3b4961-kube-api-access-6xt4l\") on node \"ip-10-0-133-210.ec2.internal\" DevicePath \"\"" Jul 30 16:14:32.904786 ip-10-0-133-210 kubenswrapper[2576]: I0730 16:14:32.904777 2576 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/c9a5dede-30f0-453b-abd1-32879f3b4961-openshift-service-ca\") on node \"ip-10-0-133-210.ec2.internal\" DevicePath \"\"" Jul 30 16:14:32.904786 ip-10-0-133-210 kubenswrapper[2576]: I0730 16:14:32.904787 2576 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c9a5dede-30f0-453b-abd1-32879f3b4961-maas-api-tls\") on node \"ip-10-0-133-210.ec2.internal\" DevicePath \"\"" Jul 30 16:14:33.630899 ip-10-0-133-210 kubenswrapper[2576]: I0730 16:14:33.630866 2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-6bc161-585f798855-75nn6" Jul 30 16:14:33.630899 ip-10-0-133-210 kubenswrapper[2576]: I0730 16:14:33.630877 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-6bc161-585f798855-75nn6" event={"ID":"c9a5dede-30f0-453b-abd1-32879f3b4961","Type":"ContainerDied","Data":"3c53bcc4d8feb9815a9f5bc916da73a25769fc8a1f47fa9f56cec2b1cd78fac3"} Jul 30 16:14:33.631442 ip-10-0-133-210 kubenswrapper[2576]: I0730 16:14:33.630956 2576 scope.go:117] "RemoveContainer" containerID="8b3d59ec83ba7e177f5b34d67c47a5d668f6f2daaabe8a1444105b47e20d5784" Jul 30 16:14:33.651187 ip-10-0-133-210 kubenswrapper[2576]: I0730 16:14:33.651154 2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-a-6bc161-585f798855-75nn6"] Jul 30 16:14:33.654706 ip-10-0-133-210 kubenswrapper[2576]: I0730 16:14:33.654679 2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-a-6bc161-585f798855-75nn6"] Jul 30 16:14:34.310939 ip-10-0-133-210 kubenswrapper[2576]: I0730 16:14:34.310897 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7j468_a2197c57-2dc6-410e-9910-a045dedcb8d9/konnectivity-agent/0.log" limit=1024 Jul 30 16:14:34.613424 ip-10-0-133-210 kubenswrapper[2576]: I0730 16:14:34.613340 2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c9a5dede-30f0-453b-abd1-32879f3b4961" path="/var/lib/kubelet/pods/c9a5dede-30f0-453b-abd1-32879f3b4961/volumes" Jul 30 16:14:38.025403 ip-10-0-133-210 kubenswrapper[2576]: I0730 16:14:38.025369 2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-ipp-a-6bc161/ipp-route-6bc161-kserve-65c4dc464c-rcppn"] Jul 30 16:14:38.133553 ip-10-0-133-210 kubenswrapper[2576]: I0730 16:14:38.133514 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7j468_a2197c57-2dc6-410e-9910-a045dedcb8d9/konnectivity-agent/0.log" limit=1024 Jul 30 16:14:53.044703 ip-10-0-133-210 kubenswrapper[2576]: I0730 16:14:53.044601 2576 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-ipp-a-6bc161/ipp-route-6bc161-kserve-65c4dc464c-rcppn" podUID="5bfaf12f-9b5f-475b-97d4-e0bbdd5de426" containerName="main" containerID="cri-o://289cd7c8807b433c79732a342433c002b7d12d51579cf13185553abdeebe8f84" gracePeriod=45 Jul 30 16:14:53.182161 ip-10-0-133-210 kubenswrapper[2576]: I0730 16:14:53.182104 2576 prober.go:120] "Probe failed" probeType="Readiness" pod="ai-tenant-e2e-ipp-a-6bc161/ipp-route-6bc161-kserve-65c4dc464c-rcppn" podUID="5bfaf12f-9b5f-475b-97d4-e0bbdd5de426" containerName="main" probeResult="failure" output="Get \"https://10.134.0.35:8000/ready\": dial tcp 10.134.0.35:8000: connect: connection refused" Jul 30 16:14:53.298683 ip-10-0-133-210 kubenswrapper[2576]: I0730 16:14:53.298622 2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-ipp-a-6bc161/ipp-route-6bc161-kserve-65c4dc464c-rcppn" Jul 30 16:14:53.371345 ip-10-0-133-210 kubenswrapper[2576]: I0730 16:14:53.371313 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/5bfaf12f-9b5f-475b-97d4-e0bbdd5de426-dshm\") pod \"5bfaf12f-9b5f-475b-97d4-e0bbdd5de426\" (UID: \"5bfaf12f-9b5f-475b-97d4-e0bbdd5de426\") " Jul 30 16:14:53.371545 ip-10-0-133-210 kubenswrapper[2576]: I0730 16:14:53.371376 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-nn5w8\" (UniqueName: \"kubernetes.io/projected/5bfaf12f-9b5f-475b-97d4-e0bbdd5de426-kube-api-access-nn5w8\") pod \"5bfaf12f-9b5f-475b-97d4-e0bbdd5de426\" (UID: \"5bfaf12f-9b5f-475b-97d4-e0bbdd5de426\") " Jul 30 16:14:53.371545 ip-10-0-133-210 kubenswrapper[2576]: I0730 16:14:53.371396 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/5bfaf12f-9b5f-475b-97d4-e0bbdd5de426-tls-certs\") pod \"5bfaf12f-9b5f-475b-97d4-e0bbdd5de426\" (UID: \"5bfaf12f-9b5f-475b-97d4-e0bbdd5de426\") " Jul 30 16:14:53.371545 ip-10-0-133-210 kubenswrapper[2576]: I0730 16:14:53.371422 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/5bfaf12f-9b5f-475b-97d4-e0bbdd5de426-tmp-dir\") pod \"5bfaf12f-9b5f-475b-97d4-e0bbdd5de426\" (UID: \"5bfaf12f-9b5f-475b-97d4-e0bbdd5de426\") " Jul 30 16:14:53.371545 ip-10-0-133-210 kubenswrapper[2576]: I0730 16:14:53.371460 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/5bfaf12f-9b5f-475b-97d4-e0bbdd5de426-model-cache\") pod \"5bfaf12f-9b5f-475b-97d4-e0bbdd5de426\" (UID: \"5bfaf12f-9b5f-475b-97d4-e0bbdd5de426\") " Jul 30 16:14:53.371545 ip-10-0-133-210 kubenswrapper[2576]: I0730 16:14:53.371531 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/5bfaf12f-9b5f-475b-97d4-e0bbdd5de426-home\") pod \"5bfaf12f-9b5f-475b-97d4-e0bbdd5de426\" (UID: \"5bfaf12f-9b5f-475b-97d4-e0bbdd5de426\") " Jul 30 16:14:53.371824 ip-10-0-133-210 kubenswrapper[2576]: I0730 16:14:53.371748 2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5bfaf12f-9b5f-475b-97d4-e0bbdd5de426-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "5bfaf12f-9b5f-475b-97d4-e0bbdd5de426" (UID: "5bfaf12f-9b5f-475b-97d4-e0bbdd5de426"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 30 16:14:53.371824 ip-10-0-133-210 kubenswrapper[2576]: I0730 16:14:53.371762 2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5bfaf12f-9b5f-475b-97d4-e0bbdd5de426-model-cache" (OuterVolumeSpecName: "model-cache") pod "5bfaf12f-9b5f-475b-97d4-e0bbdd5de426" (UID: "5bfaf12f-9b5f-475b-97d4-e0bbdd5de426"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 30 16:14:53.371918 ip-10-0-133-210 kubenswrapper[2576]: I0730 16:14:53.371874 2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5bfaf12f-9b5f-475b-97d4-e0bbdd5de426-home" (OuterVolumeSpecName: "home") pod "5bfaf12f-9b5f-475b-97d4-e0bbdd5de426" (UID: "5bfaf12f-9b5f-475b-97d4-e0bbdd5de426"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 30 16:14:53.373649 ip-10-0-133-210 kubenswrapper[2576]: I0730 16:14:53.373621 2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5bfaf12f-9b5f-475b-97d4-e0bbdd5de426-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "5bfaf12f-9b5f-475b-97d4-e0bbdd5de426" (UID: "5bfaf12f-9b5f-475b-97d4-e0bbdd5de426"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 30 16:14:53.373938 ip-10-0-133-210 kubenswrapper[2576]: I0730 16:14:53.373921 2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5bfaf12f-9b5f-475b-97d4-e0bbdd5de426-kube-api-access-nn5w8" (OuterVolumeSpecName: "kube-api-access-nn5w8") pod "5bfaf12f-9b5f-475b-97d4-e0bbdd5de426" (UID: "5bfaf12f-9b5f-475b-97d4-e0bbdd5de426"). InnerVolumeSpecName "kube-api-access-nn5w8". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 30 16:14:53.374007 ip-10-0-133-210 kubenswrapper[2576]: I0730 16:14:53.373923 2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5bfaf12f-9b5f-475b-97d4-e0bbdd5de426-dshm" (OuterVolumeSpecName: "dshm") pod "5bfaf12f-9b5f-475b-97d4-e0bbdd5de426" (UID: "5bfaf12f-9b5f-475b-97d4-e0bbdd5de426"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 30 16:14:53.472533 ip-10-0-133-210 kubenswrapper[2576]: I0730 16:14:53.472489 2576 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/5bfaf12f-9b5f-475b-97d4-e0bbdd5de426-tmp-dir\") on node \"ip-10-0-133-210.ec2.internal\" DevicePath \"\"" Jul 30 16:14:53.472533 ip-10-0-133-210 kubenswrapper[2576]: I0730 16:14:53.472529 2576 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/5bfaf12f-9b5f-475b-97d4-e0bbdd5de426-model-cache\") on node \"ip-10-0-133-210.ec2.internal\" DevicePath \"\"" Jul 30 16:14:53.472533 ip-10-0-133-210 kubenswrapper[2576]: I0730 16:14:53.472540 2576 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/5bfaf12f-9b5f-475b-97d4-e0bbdd5de426-home\") on node \"ip-10-0-133-210.ec2.internal\" DevicePath \"\"" Jul 30 16:14:53.472795 ip-10-0-133-210 kubenswrapper[2576]: I0730 16:14:53.472551 2576 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/5bfaf12f-9b5f-475b-97d4-e0bbdd5de426-dshm\") on node \"ip-10-0-133-210.ec2.internal\" DevicePath \"\"" Jul 30 16:14:53.472795 ip-10-0-133-210 kubenswrapper[2576]: I0730 16:14:53.472563 2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-nn5w8\" (UniqueName: \"kubernetes.io/projected/5bfaf12f-9b5f-475b-97d4-e0bbdd5de426-kube-api-access-nn5w8\") on node \"ip-10-0-133-210.ec2.internal\" DevicePath \"\"" Jul 30 16:14:53.472795 ip-10-0-133-210 kubenswrapper[2576]: I0730 16:14:53.472573 2576 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/5bfaf12f-9b5f-475b-97d4-e0bbdd5de426-tls-certs\") on node \"ip-10-0-133-210.ec2.internal\" DevicePath \"\"" Jul 30 16:14:53.716933 ip-10-0-133-210 kubenswrapper[2576]: I0730 16:14:53.716903 2576 generic.go:356] "Generic (PLEG): container finished" podID="5bfaf12f-9b5f-475b-97d4-e0bbdd5de426" containerID="289cd7c8807b433c79732a342433c002b7d12d51579cf13185553abdeebe8f84" exitCode=0 Jul 30 16:14:53.717082 ip-10-0-133-210 kubenswrapper[2576]: I0730 16:14:53.716949 2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-ipp-a-6bc161/ipp-route-6bc161-kserve-65c4dc464c-rcppn" Jul 30 16:14:53.717082 ip-10-0-133-210 kubenswrapper[2576]: I0730 16:14:53.716971 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-ipp-a-6bc161/ipp-route-6bc161-kserve-65c4dc464c-rcppn" event={"ID":"5bfaf12f-9b5f-475b-97d4-e0bbdd5de426","Type":"ContainerDied","Data":"289cd7c8807b433c79732a342433c002b7d12d51579cf13185553abdeebe8f84"} Jul 30 16:14:53.717082 ip-10-0-133-210 kubenswrapper[2576]: I0730 16:14:53.717003 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-ipp-a-6bc161/ipp-route-6bc161-kserve-65c4dc464c-rcppn" event={"ID":"5bfaf12f-9b5f-475b-97d4-e0bbdd5de426","Type":"ContainerDied","Data":"ae11392baf4d0174c15bffb212b99b6a697fa72b9acd2f52e61c950b8e065dcb"} Jul 30 16:14:53.717082 ip-10-0-133-210 kubenswrapper[2576]: I0730 16:14:53.717020 2576 scope.go:117] "RemoveContainer" containerID="289cd7c8807b433c79732a342433c002b7d12d51579cf13185553abdeebe8f84" Jul 30 16:14:53.725995 ip-10-0-133-210 kubenswrapper[2576]: I0730 16:14:53.725974 2576 scope.go:117] "RemoveContainer" containerID="289cd7c8807b433c79732a342433c002b7d12d51579cf13185553abdeebe8f84" Jul 30 16:14:53.726370 ip-10-0-133-210 kubenswrapper[2576]: E0730 16:14:53.726350 2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"289cd7c8807b433c79732a342433c002b7d12d51579cf13185553abdeebe8f84\": container with ID starting with 289cd7c8807b433c79732a342433c002b7d12d51579cf13185553abdeebe8f84 not found: ID does not exist" containerID="289cd7c8807b433c79732a342433c002b7d12d51579cf13185553abdeebe8f84" Jul 30 16:14:53.726437 ip-10-0-133-210 kubenswrapper[2576]: I0730 16:14:53.726378 2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"289cd7c8807b433c79732a342433c002b7d12d51579cf13185553abdeebe8f84"} err="failed to get container status \"289cd7c8807b433c79732a342433c002b7d12d51579cf13185553abdeebe8f84\": rpc error: code = NotFound desc = could not find container \"289cd7c8807b433c79732a342433c002b7d12d51579cf13185553abdeebe8f84\": container with ID starting with 289cd7c8807b433c79732a342433c002b7d12d51579cf13185553abdeebe8f84 not found: ID does not exist" Jul 30 16:14:53.738202 ip-10-0-133-210 kubenswrapper[2576]: I0730 16:14:53.738164 2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-ipp-a-6bc161/ipp-route-6bc161-kserve-65c4dc464c-rcppn"] Jul 30 16:14:53.741967 ip-10-0-133-210 kubenswrapper[2576]: I0730 16:14:53.741943 2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-ipp-a-6bc161/ipp-route-6bc161-kserve-65c4dc464c-rcppn"] Jul 30 16:14:54.613049 ip-10-0-133-210 kubenswrapper[2576]: I0730 16:14:54.613013 2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="5bfaf12f-9b5f-475b-97d4-e0bbdd5de426" path="/var/lib/kubelet/pods/5bfaf12f-9b5f-475b-97d4-e0bbdd5de426/volumes" Jul 30 16:15:00.138322 ip-10-0-133-210 kubenswrapper[2576]: I0730 16:15:00.138286 2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ipp-b-ae9970-29757135-l2gr6"] Jul 30 16:15:00.138722 ip-10-0-133-210 kubenswrapper[2576]: I0730 16:15:00.138631 2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="245b87c4-7d34-4740-9fcd-6f946a720047" containerName="maas-api" Jul 30 16:15:00.138722 ip-10-0-133-210 kubenswrapper[2576]: I0730 16:15:00.138644 2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="245b87c4-7d34-4740-9fcd-6f946a720047" containerName="maas-api" Jul 30 16:15:00.138722 ip-10-0-133-210 kubenswrapper[2576]: I0730 16:15:00.138657 2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5bfaf12f-9b5f-475b-97d4-e0bbdd5de426" containerName="main" Jul 30 16:15:00.138722 ip-10-0-133-210 kubenswrapper[2576]: I0730 16:15:00.138663 2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="5bfaf12f-9b5f-475b-97d4-e0bbdd5de426" containerName="main" Jul 30 16:15:00.138722 ip-10-0-133-210 kubenswrapper[2576]: I0730 16:15:00.138677 2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c9a5dede-30f0-453b-abd1-32879f3b4961" containerName="maas-api" Jul 30 16:15:00.138722 ip-10-0-133-210 kubenswrapper[2576]: I0730 16:15:00.138683 2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="c9a5dede-30f0-453b-abd1-32879f3b4961" containerName="maas-api" Jul 30 16:15:00.138910 ip-10-0-133-210 kubenswrapper[2576]: I0730 16:15:00.138732 2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="5bfaf12f-9b5f-475b-97d4-e0bbdd5de426" containerName="main" Jul 30 16:15:00.138910 ip-10-0-133-210 kubenswrapper[2576]: I0730 16:15:00.138740 2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="245b87c4-7d34-4740-9fcd-6f946a720047" containerName="maas-api" Jul 30 16:15:00.138910 ip-10-0-133-210 kubenswrapper[2576]: I0730 16:15:00.138746 2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="c9a5dede-30f0-453b-abd1-32879f3b4961" containerName="maas-api" Jul 30 16:15:00.142868 ip-10-0-133-210 kubenswrapper[2576]: I0730 16:15:00.142850 2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-5468b9-29757135-j6r55"] Jul 30 16:15:00.142990 ip-10-0-133-210 kubenswrapper[2576]: I0730 16:15:00.142966 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ipp-b-ae9970-29757135-l2gr6" Jul 30 16:15:00.145599 ip-10-0-133-210 kubenswrapper[2576]: I0730 16:15:00.145579 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-dg2n2\"" Jul 30 16:15:00.145828 ip-10-0-133-210 kubenswrapper[2576]: I0730 16:15:00.145812 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-5468b9-29757135-j6r55" Jul 30 16:15:00.149653 ip-10-0-133-210 kubenswrapper[2576]: I0730 16:15:00.149631 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-5468b9-29757135-j6r55"] Jul 30 16:15:00.153347 ip-10-0-133-210 kubenswrapper[2576]: I0730 16:15:00.153222 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ipp-b-ae9970-29757135-l2gr6"] Jul 30 16:15:00.155799 ip-10-0-133-210 kubenswrapper[2576]: I0730 16:15:00.155774 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ipp-b-ae9970-29757135-l2gr6" Jul 30 16:15:00.162633 ip-10-0-133-210 kubenswrapper[2576]: I0730 16:15:00.162615 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-5468b9-29757135-j6r55" Jul 30 16:15:00.325724 ip-10-0-133-210 kubenswrapper[2576]: I0730 16:15:00.325639 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-5468b9-29757135-j6r55"] Jul 30 16:15:00.327891 ip-10-0-133-210 kubenswrapper[2576]: W0730 16:15:00.327859 2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc41fc57c_0f86_4cc3_9d28_52a13b8d6e13.slice/crio-7765c158ac50efc8b03622d896ce01d0de4812f7e2b5dadff4866068b5972198 WatchSource:0}: Error finding container 7765c158ac50efc8b03622d896ce01d0de4812f7e2b5dadff4866068b5972198: Status 404 returned error can't find the container with id 7765c158ac50efc8b03622d896ce01d0de4812f7e2b5dadff4866068b5972198 Jul 30 16:15:00.510265 ip-10-0-133-210 kubenswrapper[2576]: I0730 16:15:00.510241 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ipp-b-ae9970-29757135-l2gr6"] Jul 30 16:15:00.512929 ip-10-0-133-210 kubenswrapper[2576]: W0730 16:15:00.512902 2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod418d5a0f_3a28_455f_850a_94c33d13b1f4.slice/crio-2423c821237c2b24aabdd821cebdba0f3826c92aa0ff3003f1fa2b179fa59f28 WatchSource:0}: Error finding container 2423c821237c2b24aabdd821cebdba0f3826c92aa0ff3003f1fa2b179fa59f28: Status 404 returned error can't find the container with id 2423c821237c2b24aabdd821cebdba0f3826c92aa0ff3003f1fa2b179fa59f28 Jul 30 16:15:00.752011 ip-10-0-133-210 kubenswrapper[2576]: I0730 16:15:00.751970 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-5468b9-29757135-j6r55" event={"ID":"c41fc57c-0f86-4cc3-9d28-52a13b8d6e13","Type":"ContainerStarted","Data":"7765c158ac50efc8b03622d896ce01d0de4812f7e2b5dadff4866068b5972198"} Jul 30 16:15:00.753068 ip-10-0-133-210 kubenswrapper[2576]: I0730 16:15:00.753035 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ipp-b-ae9970-29757135-l2gr6" event={"ID":"418d5a0f-3a28-455f-850a-94c33d13b1f4","Type":"ContainerStarted","Data":"2423c821237c2b24aabdd821cebdba0f3826c92aa0ff3003f1fa2b179fa59f28"} Jul 30 16:15:04.773955 ip-10-0-133-210 kubenswrapper[2576]: I0730 16:15:04.773918 2576 generic.go:356] "Generic (PLEG): container finished" podID="c41fc57c-0f86-4cc3-9d28-52a13b8d6e13" containerID="fcc2c7ee2c94665883eb9ae587df6b0e29aafb251c11e475de435e7002933417" exitCode=0 Jul 30 16:15:04.774516 ip-10-0-133-210 kubenswrapper[2576]: I0730 16:15:04.774003 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-5468b9-29757135-j6r55" event={"ID":"c41fc57c-0f86-4cc3-9d28-52a13b8d6e13","Type":"ContainerDied","Data":"fcc2c7ee2c94665883eb9ae587df6b0e29aafb251c11e475de435e7002933417"} Jul 30 16:15:04.775630 ip-10-0-133-210 kubenswrapper[2576]: I0730 16:15:04.775596 2576 generic.go:356] "Generic (PLEG): container finished" podID="418d5a0f-3a28-455f-850a-94c33d13b1f4" containerID="de67cd2434bbeea4977bbd0a77b55734a94e19bb445a491baff657e747a77b4c" exitCode=0 Jul 30 16:15:04.775743 ip-10-0-133-210 kubenswrapper[2576]: I0730 16:15:04.775666 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ipp-b-ae9970-29757135-l2gr6" event={"ID":"418d5a0f-3a28-455f-850a-94c33d13b1f4","Type":"ContainerDied","Data":"de67cd2434bbeea4977bbd0a77b55734a94e19bb445a491baff657e747a77b4c"} Jul 30 16:15:05.918137 ip-10-0-133-210 kubenswrapper[2576]: I0730 16:15:05.918097 2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-5468b9-29757135-j6r55" Jul 30 16:15:05.946615 ip-10-0-133-210 kubenswrapper[2576]: I0730 16:15:05.946589 2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ipp-b-ae9970-29757135-l2gr6" Jul 30 16:15:06.785859 ip-10-0-133-210 kubenswrapper[2576]: I0730 16:15:06.785821 2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ipp-b-ae9970-29757135-l2gr6" Jul 30 16:15:06.786047 ip-10-0-133-210 kubenswrapper[2576]: I0730 16:15:06.785859 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ipp-b-ae9970-29757135-l2gr6" event={"ID":"418d5a0f-3a28-455f-850a-94c33d13b1f4","Type":"ContainerDied","Data":"2423c821237c2b24aabdd821cebdba0f3826c92aa0ff3003f1fa2b179fa59f28"} Jul 30 16:15:06.786047 ip-10-0-133-210 kubenswrapper[2576]: I0730 16:15:06.785916 2576 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="2423c821237c2b24aabdd821cebdba0f3826c92aa0ff3003f1fa2b179fa59f28" Jul 30 16:15:06.787217 ip-10-0-133-210 kubenswrapper[2576]: I0730 16:15:06.787196 2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-5468b9-29757135-j6r55" Jul 30 16:15:06.787217 ip-10-0-133-210 kubenswrapper[2576]: I0730 16:15:06.787210 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-5468b9-29757135-j6r55" event={"ID":"c41fc57c-0f86-4cc3-9d28-52a13b8d6e13","Type":"ContainerDied","Data":"7765c158ac50efc8b03622d896ce01d0de4812f7e2b5dadff4866068b5972198"} Jul 30 16:15:06.787363 ip-10-0-133-210 kubenswrapper[2576]: I0730 16:15:06.787251 2576 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="7765c158ac50efc8b03622d896ce01d0de4812f7e2b5dadff4866068b5972198" Jul 30 16:15:14.555172 ip-10-0-133-210 kubenswrapper[2576]: I0730 16:15:14.555133 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7j468_a2197c57-2dc6-410e-9910-a045dedcb8d9/konnectivity-agent/0.log" limit=1024 Jul 30 16:15:15.563718 ip-10-0-133-210 kubenswrapper[2576]: I0730 16:15:15.563676 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7j468_a2197c57-2dc6-410e-9910-a045dedcb8d9/konnectivity-agent/0.log" limit=1024 Jul 30 16:15:18.482499 ip-10-0-133-210 kubenswrapper[2576]: I0730 16:15:18.482467 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7j468_a2197c57-2dc6-410e-9910-a045dedcb8d9/konnectivity-agent/0.log" limit=1024 Jul 30 16:15:21.230583 ip-10-0-133-210 kubenswrapper[2576]: I0730 16:15:21.230543 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7j468_a2197c57-2dc6-410e-9910-a045dedcb8d9/konnectivity-agent/0.log" limit=1024 Jul 30 16:15:21.304485 ip-10-0-133-210 kubenswrapper[2576]: I0730 16:15:21.304447 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7j468_a2197c57-2dc6-410e-9910-a045dedcb8d9/konnectivity-agent/0.log" limit=1024 Jul 30 16:15:21.584615 ip-10-0-133-210 kubenswrapper[2576]: I0730 16:15:21.584524 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7j468_a2197c57-2dc6-410e-9910-a045dedcb8d9/konnectivity-agent/0.log" limit=1024 Jul 30 16:15:21.871765 ip-10-0-133-210 kubenswrapper[2576]: I0730 16:15:21.871675 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7j468_a2197c57-2dc6-410e-9910-a045dedcb8d9/konnectivity-agent/0.log" limit=1024 Jul 30 16:15:32.617463 ip-10-0-133-210 kubenswrapper[2576]: I0730 16:15:32.617429 2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-b-ae9970-67ffb857c4-mzpmc"] Jul 30 16:15:32.617973 ip-10-0-133-210 kubenswrapper[2576]: I0730 16:15:32.617674 2576 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-ae9970-67ffb857c4-mzpmc" podUID="b5307962-931c-41f6-a862-942f9283dee4" containerName="maas-api" containerID="cri-o://0ea1fa719fc7074d3490f908653a28431df172af48c020978448ad9d82b6bc09" gracePeriod=30 Jul 30 16:15:32.666756 ip-10-0-133-210 kubenswrapper[2576]: I0730 16:15:32.666712 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7j468_a2197c57-2dc6-410e-9910-a045dedcb8d9/konnectivity-agent/0.log" limit=1024 Jul 30 16:15:32.685599 ip-10-0-133-210 kubenswrapper[2576]: I0730 16:15:32.685537 2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ipp-b-ae9970-29757135-l2gr6"] Jul 30 16:15:32.687364 ip-10-0-133-210 kubenswrapper[2576]: I0730 16:15:32.687333 2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ipp-b-ae9970-29757135-l2gr6"] Jul 30 16:15:32.798991 ip-10-0-133-210 kubenswrapper[2576]: I0730 16:15:32.798962 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7j468_a2197c57-2dc6-410e-9910-a045dedcb8d9/konnectivity-agent/0.log" limit=1024 Jul 30 16:15:32.864561 ip-10-0-133-210 kubenswrapper[2576]: I0730 16:15:32.864536 2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-ae9970-67ffb857c4-mzpmc" Jul 30 16:15:32.900885 ip-10-0-133-210 kubenswrapper[2576]: I0730 16:15:32.900797 2576 generic.go:356] "Generic (PLEG): container finished" podID="b5307962-931c-41f6-a862-942f9283dee4" containerID="0ea1fa719fc7074d3490f908653a28431df172af48c020978448ad9d82b6bc09" exitCode=0 Jul 30 16:15:32.900885 ip-10-0-133-210 kubenswrapper[2576]: I0730 16:15:32.900840 2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-ae9970-67ffb857c4-mzpmc" Jul 30 16:15:32.900885 ip-10-0-133-210 kubenswrapper[2576]: I0730 16:15:32.900866 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-ae9970-67ffb857c4-mzpmc" event={"ID":"b5307962-931c-41f6-a862-942f9283dee4","Type":"ContainerDied","Data":"0ea1fa719fc7074d3490f908653a28431df172af48c020978448ad9d82b6bc09"} Jul 30 16:15:32.901131 ip-10-0-133-210 kubenswrapper[2576]: I0730 16:15:32.900912 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-ae9970-67ffb857c4-mzpmc" event={"ID":"b5307962-931c-41f6-a862-942f9283dee4","Type":"ContainerDied","Data":"dee7b256ecc8dce98e40f32634cb027f987e2fe9b52828789944e8da07bc57e2"} Jul 30 16:15:32.901131 ip-10-0-133-210 kubenswrapper[2576]: I0730 16:15:32.900935 2576 scope.go:117] "RemoveContainer" containerID="0ea1fa719fc7074d3490f908653a28431df172af48c020978448ad9d82b6bc09" Jul 30 16:15:32.904851 ip-10-0-133-210 kubenswrapper[2576]: I0730 16:15:32.904822 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7j468_a2197c57-2dc6-410e-9910-a045dedcb8d9/konnectivity-agent/0.log" limit=1024 Jul 30 16:15:32.910809 ip-10-0-133-210 kubenswrapper[2576]: I0730 16:15:32.910789 2576 scope.go:117] "RemoveContainer" containerID="0ea1fa719fc7074d3490f908653a28431df172af48c020978448ad9d82b6bc09" Jul 30 16:15:32.911218 ip-10-0-133-210 kubenswrapper[2576]: E0730 16:15:32.911190 2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0ea1fa719fc7074d3490f908653a28431df172af48c020978448ad9d82b6bc09\": container with ID starting with 0ea1fa719fc7074d3490f908653a28431df172af48c020978448ad9d82b6bc09 not found: ID does not exist" containerID="0ea1fa719fc7074d3490f908653a28431df172af48c020978448ad9d82b6bc09" Jul 30 16:15:32.911318 ip-10-0-133-210 kubenswrapper[2576]: I0730 16:15:32.911228 2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0ea1fa719fc7074d3490f908653a28431df172af48c020978448ad9d82b6bc09"} err="failed to get container status \"0ea1fa719fc7074d3490f908653a28431df172af48c020978448ad9d82b6bc09\": rpc error: code = NotFound desc = could not find container \"0ea1fa719fc7074d3490f908653a28431df172af48c020978448ad9d82b6bc09\": container with ID starting with 0ea1fa719fc7074d3490f908653a28431df172af48c020978448ad9d82b6bc09 not found: ID does not exist" Jul 30 16:15:33.021022 ip-10-0-133-210 kubenswrapper[2576]: I0730 16:15:33.020986 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b5307962-931c-41f6-a862-942f9283dee4-openshift-service-ca\") pod \"b5307962-931c-41f6-a862-942f9283dee4\" (UID: \"b5307962-931c-41f6-a862-942f9283dee4\") " Jul 30 16:15:33.021255 ip-10-0-133-210 kubenswrapper[2576]: I0730 16:15:33.021076 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-668hn\" (UniqueName: \"kubernetes.io/projected/b5307962-931c-41f6-a862-942f9283dee4-kube-api-access-668hn\") pod \"b5307962-931c-41f6-a862-942f9283dee4\" (UID: \"b5307962-931c-41f6-a862-942f9283dee4\") " Jul 30 16:15:33.021255 ip-10-0-133-210 kubenswrapper[2576]: I0730 16:15:33.021215 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b5307962-931c-41f6-a862-942f9283dee4-maas-api-tls\") pod \"b5307962-931c-41f6-a862-942f9283dee4\" (UID: \"b5307962-931c-41f6-a862-942f9283dee4\") " Jul 30 16:15:33.021428 ip-10-0-133-210 kubenswrapper[2576]: I0730 16:15:33.021394 2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b5307962-931c-41f6-a862-942f9283dee4-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "b5307962-931c-41f6-a862-942f9283dee4" (UID: "b5307962-931c-41f6-a862-942f9283dee4"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 30 16:15:33.021549 ip-10-0-133-210 kubenswrapper[2576]: I0730 16:15:33.021481 2576 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b5307962-931c-41f6-a862-942f9283dee4-openshift-service-ca\") on node \"ip-10-0-133-210.ec2.internal\" DevicePath \"\"" Jul 30 16:15:33.023275 ip-10-0-133-210 kubenswrapper[2576]: I0730 16:15:33.023246 2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b5307962-931c-41f6-a862-942f9283dee4-kube-api-access-668hn" (OuterVolumeSpecName: "kube-api-access-668hn") pod "b5307962-931c-41f6-a862-942f9283dee4" (UID: "b5307962-931c-41f6-a862-942f9283dee4"). InnerVolumeSpecName "kube-api-access-668hn". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 30 16:15:33.023275 ip-10-0-133-210 kubenswrapper[2576]: I0730 16:15:33.023258 2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b5307962-931c-41f6-a862-942f9283dee4-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "b5307962-931c-41f6-a862-942f9283dee4" (UID: "b5307962-931c-41f6-a862-942f9283dee4"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 30 16:15:33.122379 ip-10-0-133-210 kubenswrapper[2576]: I0730 16:15:33.122344 2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-668hn\" (UniqueName: \"kubernetes.io/projected/b5307962-931c-41f6-a862-942f9283dee4-kube-api-access-668hn\") on node \"ip-10-0-133-210.ec2.internal\" DevicePath \"\"" Jul 30 16:15:33.122379 ip-10-0-133-210 kubenswrapper[2576]: I0730 16:15:33.122377 2576 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b5307962-931c-41f6-a862-942f9283dee4-maas-api-tls\") on node \"ip-10-0-133-210.ec2.internal\" DevicePath \"\"" Jul 30 16:15:33.179924 ip-10-0-133-210 kubenswrapper[2576]: I0730 16:15:33.179837 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7j468_a2197c57-2dc6-410e-9910-a045dedcb8d9/konnectivity-agent/0.log" limit=1024 Jul 30 16:15:33.257499 ip-10-0-133-210 kubenswrapper[2576]: I0730 16:15:33.257457 2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-b-ae9970-67ffb857c4-mzpmc"] Jul 30 16:15:33.265598 ip-10-0-133-210 kubenswrapper[2576]: I0730 16:15:33.265567 2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-b-ae9970-67ffb857c4-mzpmc"] Jul 30 16:15:33.554895 ip-10-0-133-210 kubenswrapper[2576]: I0730 16:15:33.554861 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7j468_a2197c57-2dc6-410e-9910-a045dedcb8d9/konnectivity-agent/0.log" limit=1024 Jul 30 16:15:34.612691 ip-10-0-133-210 kubenswrapper[2576]: I0730 16:15:34.612651 2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="418d5a0f-3a28-455f-850a-94c33d13b1f4" path="/var/lib/kubelet/pods/418d5a0f-3a28-455f-850a-94c33d13b1f4/volumes" Jul 30 16:15:34.613347 ip-10-0-133-210 kubenswrapper[2576]: I0730 16:15:34.613325 2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b5307962-931c-41f6-a862-942f9283dee4" path="/var/lib/kubelet/pods/b5307962-931c-41f6-a862-942f9283dee4/volumes" Jul 30 16:15:40.356152 ip-10-0-133-210 kubenswrapper[2576]: I0730 16:15:40.356092 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7j468_a2197c57-2dc6-410e-9910-a045dedcb8d9/konnectivity-agent/0.log" limit=1024 Jul 30 16:15:40.553676 ip-10-0-133-210 kubenswrapper[2576]: I0730 16:15:40.553636 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7j468_a2197c57-2dc6-410e-9910-a045dedcb8d9/konnectivity-agent/0.log" limit=1024 Jul 30 16:15:40.771165 ip-10-0-133-210 kubenswrapper[2576]: I0730 16:15:40.771045 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7j468_a2197c57-2dc6-410e-9910-a045dedcb8d9/konnectivity-agent/0.log" limit=1024 Jul 30 16:15:41.169490 ip-10-0-133-210 kubenswrapper[2576]: I0730 16:15:41.169405 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7j468_a2197c57-2dc6-410e-9910-a045dedcb8d9/konnectivity-agent/0.log" limit=1024 Jul 30 16:15:41.176561 ip-10-0-133-210 kubenswrapper[2576]: I0730 16:15:41.176536 2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-f2cf4d-4t2gb"] Jul 30 16:15:41.176841 ip-10-0-133-210 kubenswrapper[2576]: I0730 16:15:41.176830 2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b5307962-931c-41f6-a862-942f9283dee4" containerName="maas-api" Jul 30 16:15:41.176881 ip-10-0-133-210 kubenswrapper[2576]: I0730 16:15:41.176843 2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="b5307962-931c-41f6-a862-942f9283dee4" containerName="maas-api" Jul 30 16:15:41.176881 ip-10-0-133-210 kubenswrapper[2576]: I0730 16:15:41.176861 2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c41fc57c-0f86-4cc3-9d28-52a13b8d6e13" containerName="cleanup" Jul 30 16:15:41.176881 ip-10-0-133-210 kubenswrapper[2576]: I0730 16:15:41.176867 2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="c41fc57c-0f86-4cc3-9d28-52a13b8d6e13" containerName="cleanup" Jul 30 16:15:41.176881 ip-10-0-133-210 kubenswrapper[2576]: I0730 16:15:41.176874 2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="418d5a0f-3a28-455f-850a-94c33d13b1f4" containerName="cleanup" Jul 30 16:15:41.176881 ip-10-0-133-210 kubenswrapper[2576]: I0730 16:15:41.176879 2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="418d5a0f-3a28-455f-850a-94c33d13b1f4" containerName="cleanup" Jul 30 16:15:41.177035 ip-10-0-133-210 kubenswrapper[2576]: I0730 16:15:41.176928 2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="c41fc57c-0f86-4cc3-9d28-52a13b8d6e13" containerName="cleanup" Jul 30 16:15:41.177035 ip-10-0-133-210 kubenswrapper[2576]: I0730 16:15:41.176936 2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="b5307962-931c-41f6-a862-942f9283dee4" containerName="maas-api" Jul 30 16:15:41.177035 ip-10-0-133-210 kubenswrapper[2576]: I0730 16:15:41.176941 2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="418d5a0f-3a28-455f-850a-94c33d13b1f4" containerName="cleanup" Jul 30 16:15:41.179955 ip-10-0-133-210 kubenswrapper[2576]: I0730 16:15:41.179940 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-f2cf4d-4t2gb" Jul 30 16:15:41.204918 ip-10-0-133-210 kubenswrapper[2576]: E0730 16:15:41.204875 2576 status_manager.go:1018] "Failed to get status for pod" err="pods \"maas-api-revoke-keys-e2e-shared-a-f2cf4d-4t2gb\" is forbidden: User \"system:node:ip-10-0-133-210.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"odh-ai-gateway-infra\": no relationship found between node 'ip-10-0-133-210.ec2.internal' and this object" podUID="c57b645c-41e9-490d-a788-35e8153ca515" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-f2cf4d-4t2gb" Jul 30 16:15:41.204918 ip-10-0-133-210 kubenswrapper[2576]: E0730 16:15:41.204885 2576 reflector.go:205] "Failed to watch" err="failed to list *v1.ConfigMap: configmaps \"openshift-service-ca.crt\" is forbidden: User \"system:node:ip-10-0-133-210.ec2.internal\" cannot list resource \"configmaps\" in API group \"\" in the namespace \"odh-ai-gateway-infra\": no relationship found between node 'ip-10-0-133-210.ec2.internal' and this object" logger="UnhandledError" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\"" type="*v1.ConfigMap" Jul 30 16:15:41.205240 ip-10-0-133-210 kubenswrapper[2576]: I0730 16:15:41.205222 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-dg2n2\"" Jul 30 16:15:41.214263 ip-10-0-133-210 kubenswrapper[2576]: I0730 16:15:41.214233 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-f2cf4d-4t2gb"] Jul 30 16:15:41.288333 ip-10-0-133-210 kubenswrapper[2576]: I0730 16:15:41.288292 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/c57b645c-41e9-490d-a788-35e8153ca515-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-a-f2cf4d-4t2gb\" (UID: \"c57b645c-41e9-490d-a788-35e8153ca515\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-f2cf4d-4t2gb" Jul 30 16:15:41.388936 ip-10-0-133-210 kubenswrapper[2576]: I0730 16:15:41.388896 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/c57b645c-41e9-490d-a788-35e8153ca515-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-a-f2cf4d-4t2gb\" (UID: \"c57b645c-41e9-490d-a788-35e8153ca515\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-f2cf4d-4t2gb" Jul 30 16:15:42.389182 ip-10-0-133-210 kubenswrapper[2576]: E0730 16:15:42.389137 2576 configmap.go:193] Couldn't get configMap odh-ai-gateway-infra/openshift-service-ca.crt: failed to sync configmap cache: timed out waiting for the condition Jul 30 16:15:42.389652 ip-10-0-133-210 kubenswrapper[2576]: E0730 16:15:42.389240 2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/c57b645c-41e9-490d-a788-35e8153ca515-maas-api-service-ca podName:c57b645c-41e9-490d-a788-35e8153ca515 nodeName:}" failed. No retries permitted until 2026-07-30 16:15:42.889218981 +0000 UTC m=+3924.814971604 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-service-ca" (UniqueName: "kubernetes.io/configmap/c57b645c-41e9-490d-a788-35e8153ca515-maas-api-service-ca") pod "maas-api-revoke-keys-e2e-shared-a-f2cf4d-4t2gb" (UID: "c57b645c-41e9-490d-a788-35e8153ca515") : failed to sync configmap cache: timed out waiting for the condition Jul 30 16:15:42.561414 ip-10-0-133-210 kubenswrapper[2576]: I0730 16:15:42.561377 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\"" Jul 30 16:15:42.900377 ip-10-0-133-210 kubenswrapper[2576]: I0730 16:15:42.900338 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/c57b645c-41e9-490d-a788-35e8153ca515-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-a-f2cf4d-4t2gb\" (UID: \"c57b645c-41e9-490d-a788-35e8153ca515\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-f2cf4d-4t2gb" Jul 30 16:15:42.900977 ip-10-0-133-210 kubenswrapper[2576]: I0730 16:15:42.900957 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/c57b645c-41e9-490d-a788-35e8153ca515-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-a-f2cf4d-4t2gb\" (UID: \"c57b645c-41e9-490d-a788-35e8153ca515\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-f2cf4d-4t2gb" Jul 30 16:15:42.989568 ip-10-0-133-210 kubenswrapper[2576]: I0730 16:15:42.989526 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-f2cf4d-4t2gb" Jul 30 16:15:43.124852 ip-10-0-133-210 kubenswrapper[2576]: I0730 16:15:43.124827 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-f2cf4d-4t2gb"] Jul 30 16:15:43.127180 ip-10-0-133-210 kubenswrapper[2576]: W0730 16:15:43.127145 2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc57b645c_41e9_490d_a788_35e8153ca515.slice/crio-e2257ddeffc349bfd1ebd93b0e3d2d09e7712173e0cdc8ed9ed1701f7de36d8f WatchSource:0}: Error finding container e2257ddeffc349bfd1ebd93b0e3d2d09e7712173e0cdc8ed9ed1701f7de36d8f: Status 404 returned error can't find the container with id e2257ddeffc349bfd1ebd93b0e3d2d09e7712173e0cdc8ed9ed1701f7de36d8f Jul 30 16:15:43.961957 ip-10-0-133-210 kubenswrapper[2576]: I0730 16:15:43.961919 2576 generic.go:356] "Generic (PLEG): container finished" podID="c57b645c-41e9-490d-a788-35e8153ca515" containerID="12510d33bd8d76f5a08dd35a65c2556293aa2be18bf8016fd7891e3919e66cfd" exitCode=0 Jul 30 16:15:43.962379 ip-10-0-133-210 kubenswrapper[2576]: I0730 16:15:43.962009 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-f2cf4d-4t2gb" event={"ID":"c57b645c-41e9-490d-a788-35e8153ca515","Type":"ContainerDied","Data":"12510d33bd8d76f5a08dd35a65c2556293aa2be18bf8016fd7891e3919e66cfd"} Jul 30 16:15:43.962379 ip-10-0-133-210 kubenswrapper[2576]: I0730 16:15:43.962070 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-f2cf4d-4t2gb" event={"ID":"c57b645c-41e9-490d-a788-35e8153ca515","Type":"ContainerStarted","Data":"e2257ddeffc349bfd1ebd93b0e3d2d09e7712173e0cdc8ed9ed1701f7de36d8f"} Jul 30 16:15:45.092729 ip-10-0-133-210 kubenswrapper[2576]: I0730 16:15:45.092703 2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-f2cf4d-4t2gb" Jul 30 16:15:45.117375 ip-10-0-133-210 kubenswrapper[2576]: I0730 16:15:45.117345 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/c57b645c-41e9-490d-a788-35e8153ca515-maas-api-service-ca\") pod \"c57b645c-41e9-490d-a788-35e8153ca515\" (UID: \"c57b645c-41e9-490d-a788-35e8153ca515\") " Jul 30 16:15:45.117689 ip-10-0-133-210 kubenswrapper[2576]: I0730 16:15:45.117659 2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c57b645c-41e9-490d-a788-35e8153ca515-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "c57b645c-41e9-490d-a788-35e8153ca515" (UID: "c57b645c-41e9-490d-a788-35e8153ca515"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 30 16:15:45.218045 ip-10-0-133-210 kubenswrapper[2576]: I0730 16:15:45.217952 2576 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/c57b645c-41e9-490d-a788-35e8153ca515-maas-api-service-ca\") on node \"ip-10-0-133-210.ec2.internal\" DevicePath \"\"" Jul 30 16:15:45.973271 ip-10-0-133-210 kubenswrapper[2576]: I0730 16:15:45.973239 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-f2cf4d-4t2gb" event={"ID":"c57b645c-41e9-490d-a788-35e8153ca515","Type":"ContainerDied","Data":"e2257ddeffc349bfd1ebd93b0e3d2d09e7712173e0cdc8ed9ed1701f7de36d8f"} Jul 30 16:15:45.973271 ip-10-0-133-210 kubenswrapper[2576]: I0730 16:15:45.973273 2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-f2cf4d-4t2gb" Jul 30 16:15:45.973477 ip-10-0-133-210 kubenswrapper[2576]: I0730 16:15:45.973289 2576 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="e2257ddeffc349bfd1ebd93b0e3d2d09e7712173e0cdc8ed9ed1701f7de36d8f" Jul 30 16:15:51.541583 ip-10-0-133-210 kubenswrapper[2576]: I0730 16:15:51.541549 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7j468_a2197c57-2dc6-410e-9910-a045dedcb8d9/konnectivity-agent/0.log" limit=1024 Jul 30 16:15:51.638751 ip-10-0-133-210 kubenswrapper[2576]: I0730 16:15:51.638713 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7j468_a2197c57-2dc6-410e-9910-a045dedcb8d9/konnectivity-agent/0.log" limit=1024 Jul 30 16:15:51.703560 ip-10-0-133-210 kubenswrapper[2576]: I0730 16:15:51.703524 2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-f2cf4d-4t2gb"] Jul 30 16:15:51.710961 ip-10-0-133-210 kubenswrapper[2576]: I0730 16:15:51.710929 2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-f2cf4d-4t2gb"] Jul 30 16:15:51.850082 ip-10-0-133-210 kubenswrapper[2576]: I0730 16:15:51.849965 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7j468_a2197c57-2dc6-410e-9910-a045dedcb8d9/konnectivity-agent/0.log" limit=1024 Jul 30 16:15:52.382518 ip-10-0-133-210 kubenswrapper[2576]: I0730 16:15:52.382484 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7j468_a2197c57-2dc6-410e-9910-a045dedcb8d9/konnectivity-agent/0.log" limit=1024 Jul 30 16:15:52.447309 ip-10-0-133-210 kubenswrapper[2576]: I0730 16:15:52.447274 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7j468_a2197c57-2dc6-410e-9910-a045dedcb8d9/konnectivity-agent/0.log" limit=1024 Jul 30 16:15:52.612462 ip-10-0-133-210 kubenswrapper[2576]: I0730 16:15:52.612429 2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c57b645c-41e9-490d-a788-35e8153ca515" path="/var/lib/kubelet/pods/c57b645c-41e9-490d-a788-35e8153ca515/volumes" Jul 30 16:15:58.575259 ip-10-0-133-210 kubenswrapper[2576]: I0730 16:15:58.575226 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7j468_a2197c57-2dc6-410e-9910-a045dedcb8d9/konnectivity-agent/0.log" limit=1024 Jul 30 16:15:58.774940 ip-10-0-133-210 kubenswrapper[2576]: I0730 16:15:58.774904 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7j468_a2197c57-2dc6-410e-9910-a045dedcb8d9/konnectivity-agent/0.log" limit=1024 Jul 30 16:15:58.927160 ip-10-0-133-210 kubenswrapper[2576]: I0730 16:15:58.927013 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7j468_a2197c57-2dc6-410e-9910-a045dedcb8d9/konnectivity-agent/0.log" limit=1024 Jul 30 16:15:59.174155 ip-10-0-133-210 kubenswrapper[2576]: I0730 16:15:59.174091 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7j468_a2197c57-2dc6-410e-9910-a045dedcb8d9/konnectivity-agent/0.log" limit=1024 Jul 30 16:15:59.315549 ip-10-0-133-210 kubenswrapper[2576]: I0730 16:15:59.315514 2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-5468b9-kkprm"] Jul 30 16:15:59.315826 ip-10-0-133-210 kubenswrapper[2576]: I0730 16:15:59.315815 2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c57b645c-41e9-490d-a788-35e8153ca515" containerName="revoke-keys" Jul 30 16:15:59.315871 ip-10-0-133-210 kubenswrapper[2576]: I0730 16:15:59.315828 2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="c57b645c-41e9-490d-a788-35e8153ca515" containerName="revoke-keys" Jul 30 16:15:59.315906 ip-10-0-133-210 kubenswrapper[2576]: I0730 16:15:59.315880 2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="c57b645c-41e9-490d-a788-35e8153ca515" containerName="revoke-keys" Jul 30 16:15:59.317731 ip-10-0-133-210 kubenswrapper[2576]: I0730 16:15:59.317706 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-5468b9-kkprm" Jul 30 16:15:59.322485 ip-10-0-133-210 kubenswrapper[2576]: I0730 16:15:59.322466 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\"" Jul 30 16:15:59.322858 ip-10-0-133-210 kubenswrapper[2576]: I0730 16:15:59.322836 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-dg2n2\"" Jul 30 16:15:59.332271 ip-10-0-133-210 kubenswrapper[2576]: I0730 16:15:59.332238 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-5468b9-kkprm"] Jul 30 16:15:59.421409 ip-10-0-133-210 kubenswrapper[2576]: I0730 16:15:59.421373 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/b2b10631-e60b-4dd1-85e4-91c54d95e72a-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-b-5468b9-kkprm\" (UID: \"b2b10631-e60b-4dd1-85e4-91c54d95e72a\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-5468b9-kkprm" Jul 30 16:15:59.521983 ip-10-0-133-210 kubenswrapper[2576]: I0730 16:15:59.521942 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/b2b10631-e60b-4dd1-85e4-91c54d95e72a-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-b-5468b9-kkprm\" (UID: \"b2b10631-e60b-4dd1-85e4-91c54d95e72a\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-5468b9-kkprm" Jul 30 16:15:59.522596 ip-10-0-133-210 kubenswrapper[2576]: I0730 16:15:59.522575 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/b2b10631-e60b-4dd1-85e4-91c54d95e72a-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-b-5468b9-kkprm\" (UID: \"b2b10631-e60b-4dd1-85e4-91c54d95e72a\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-5468b9-kkprm" Jul 30 16:15:59.627257 ip-10-0-133-210 kubenswrapper[2576]: I0730 16:15:59.627172 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-5468b9-kkprm" Jul 30 16:15:59.768719 ip-10-0-133-210 kubenswrapper[2576]: I0730 16:15:59.768684 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-5468b9-kkprm"] Jul 30 16:15:59.770480 ip-10-0-133-210 kubenswrapper[2576]: W0730 16:15:59.770439 2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb2b10631_e60b_4dd1_85e4_91c54d95e72a.slice/crio-21502aadcb8d40211c5fc2fb0496b1dda2024a756e5ac14b7483c86c0a7e51ef WatchSource:0}: Error finding container 21502aadcb8d40211c5fc2fb0496b1dda2024a756e5ac14b7483c86c0a7e51ef: Status 404 returned error can't find the container with id 21502aadcb8d40211c5fc2fb0496b1dda2024a756e5ac14b7483c86c0a7e51ef Jul 30 16:16:00.036096 ip-10-0-133-210 kubenswrapper[2576]: I0730 16:16:00.036067 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-5468b9-kkprm" event={"ID":"b2b10631-e60b-4dd1-85e4-91c54d95e72a","Type":"ContainerStarted","Data":"21502aadcb8d40211c5fc2fb0496b1dda2024a756e5ac14b7483c86c0a7e51ef"} Jul 30 16:16:01.041418 ip-10-0-133-210 kubenswrapper[2576]: I0730 16:16:01.041383 2576 generic.go:356] "Generic (PLEG): container finished" podID="b2b10631-e60b-4dd1-85e4-91c54d95e72a" containerID="69eca81d6ab660657e1254b9851ba983d015012d7769c62d5999d8ebfdd330f5" exitCode=0 Jul 30 16:16:01.041813 ip-10-0-133-210 kubenswrapper[2576]: I0730 16:16:01.041449 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-5468b9-kkprm" event={"ID":"b2b10631-e60b-4dd1-85e4-91c54d95e72a","Type":"ContainerDied","Data":"69eca81d6ab660657e1254b9851ba983d015012d7769c62d5999d8ebfdd330f5"} Jul 30 16:16:02.178810 ip-10-0-133-210 kubenswrapper[2576]: I0730 16:16:02.178780 2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-5468b9-kkprm" Jul 30 16:16:02.243290 ip-10-0-133-210 kubenswrapper[2576]: I0730 16:16:02.243253 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/b2b10631-e60b-4dd1-85e4-91c54d95e72a-maas-api-service-ca\") pod \"b2b10631-e60b-4dd1-85e4-91c54d95e72a\" (UID: \"b2b10631-e60b-4dd1-85e4-91c54d95e72a\") " Jul 30 16:16:02.243654 ip-10-0-133-210 kubenswrapper[2576]: I0730 16:16:02.243630 2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b2b10631-e60b-4dd1-85e4-91c54d95e72a-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "b2b10631-e60b-4dd1-85e4-91c54d95e72a" (UID: "b2b10631-e60b-4dd1-85e4-91c54d95e72a"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 30 16:16:02.344916 ip-10-0-133-210 kubenswrapper[2576]: I0730 16:16:02.344823 2576 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/b2b10631-e60b-4dd1-85e4-91c54d95e72a-maas-api-service-ca\") on node \"ip-10-0-133-210.ec2.internal\" DevicePath \"\"" Jul 30 16:16:03.053260 ip-10-0-133-210 kubenswrapper[2576]: I0730 16:16:03.053229 2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-5468b9-kkprm" Jul 30 16:16:03.053260 ip-10-0-133-210 kubenswrapper[2576]: I0730 16:16:03.053246 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-5468b9-kkprm" event={"ID":"b2b10631-e60b-4dd1-85e4-91c54d95e72a","Type":"ContainerDied","Data":"21502aadcb8d40211c5fc2fb0496b1dda2024a756e5ac14b7483c86c0a7e51ef"} Jul 30 16:16:03.053476 ip-10-0-133-210 kubenswrapper[2576]: I0730 16:16:03.053289 2576 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="21502aadcb8d40211c5fc2fb0496b1dda2024a756e5ac14b7483c86c0a7e51ef" Jul 30 16:16:09.872199 ip-10-0-133-210 kubenswrapper[2576]: I0730 16:16:09.870988 2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-5468b9-29757135-j6r55"] Jul 30 16:16:09.876383 ip-10-0-133-210 kubenswrapper[2576]: I0730 16:16:09.876335 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7j468_a2197c57-2dc6-410e-9910-a045dedcb8d9/konnectivity-agent/0.log" limit=1024 Jul 30 16:16:09.876661 ip-10-0-133-210 kubenswrapper[2576]: I0730 16:16:09.876639 2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-5468b9-29757135-j6r55"] Jul 30 16:16:09.973543 ip-10-0-133-210 kubenswrapper[2576]: I0730 16:16:09.973504 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7j468_a2197c57-2dc6-410e-9910-a045dedcb8d9/konnectivity-agent/0.log" limit=1024 Jul 30 16:16:10.042130 ip-10-0-133-210 kubenswrapper[2576]: I0730 16:16:10.042041 2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-5468b9-kkprm"] Jul 30 16:16:10.045393 ip-10-0-133-210 kubenswrapper[2576]: I0730 16:16:10.045363 2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-5468b9-kkprm"] Jul 30 16:16:10.046303 ip-10-0-133-210 kubenswrapper[2576]: I0730 16:16:10.046281 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7j468_a2197c57-2dc6-410e-9910-a045dedcb8d9/konnectivity-agent/0.log" limit=1024 Jul 30 16:16:10.105550 ip-10-0-133-210 kubenswrapper[2576]: I0730 16:16:10.105515 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7j468_a2197c57-2dc6-410e-9910-a045dedcb8d9/konnectivity-agent/0.log" limit=1024 Jul 30 16:16:10.612393 ip-10-0-133-210 kubenswrapper[2576]: I0730 16:16:10.612364 2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b2b10631-e60b-4dd1-85e4-91c54d95e72a" path="/var/lib/kubelet/pods/b2b10631-e60b-4dd1-85e4-91c54d95e72a/volumes" Jul 30 16:16:10.612691 ip-10-0-133-210 kubenswrapper[2576]: I0730 16:16:10.612675 2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c41fc57c-0f86-4cc3-9d28-52a13b8d6e13" path="/var/lib/kubelet/pods/c41fc57c-0f86-4cc3-9d28-52a13b8d6e13/volumes" Jul 30 16:16:10.637709 ip-10-0-133-210 kubenswrapper[2576]: I0730 16:16:10.637666 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7j468_a2197c57-2dc6-410e-9910-a045dedcb8d9/konnectivity-agent/0.log" limit=1024 Jul 30 16:16:16.690982 ip-10-0-133-210 kubenswrapper[2576]: I0730 16:16:16.690948 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7j468_a2197c57-2dc6-410e-9910-a045dedcb8d9/konnectivity-agent/0.log" limit=1024 Jul 30 16:16:16.780358 ip-10-0-133-210 kubenswrapper[2576]: I0730 16:16:16.780329 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7j468_a2197c57-2dc6-410e-9910-a045dedcb8d9/konnectivity-agent/0.log" limit=1024 Jul 30 16:16:16.975392 ip-10-0-133-210 kubenswrapper[2576]: I0730 16:16:16.975350 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7j468_a2197c57-2dc6-410e-9910-a045dedcb8d9/konnectivity-agent/0.log" limit=1024 Jul 30 16:16:17.223624 ip-10-0-133-210 kubenswrapper[2576]: I0730 16:16:17.223593 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7j468_a2197c57-2dc6-410e-9910-a045dedcb8d9/konnectivity-agent/0.log" limit=1024 Jul 30 16:16:24.494815 ip-10-0-133-210 kubenswrapper[2576]: I0730 16:16:24.494786 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-56444d5c7b-jdfll_2fcf0d71-64a1-4a7d-812b-ef92d1e5c927/copy-manifests/0.log" Jul 30 16:16:24.503056 ip-10-0-133-210 kubenswrapper[2576]: I0730 16:16:24.503026 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-56444d5c7b-jdfll_2fcf0d71-64a1-4a7d-812b-ef92d1e5c927/manager/0.log" Jul 30 16:16:28.485744 ip-10-0-133-210 kubenswrapper[2576]: I0730 16:16:28.485708 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-5677ccb49b-htlrm_291e793a-4cbe-4576-a16f-eef021410b23/discovery/0.log" Jul 30 16:16:28.613282 ip-10-0-133-210 kubenswrapper[2576]: I0730 16:16:28.613248 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-6976f6d894-6ckvp_09e1e2ea-ba7d-45e6-94eb-4ef6726590d3/kube-auth-proxy/0.log" Jul 30 16:16:35.426382 ip-10-0-133-210 kubenswrapper[2576]: I0730 16:16:35.426350 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7j468_a2197c57-2dc6-410e-9910-a045dedcb8d9/konnectivity-agent/0.log" limit=1024 Jul 30 16:16:35.481548 ip-10-0-133-210 kubenswrapper[2576]: I0730 16:16:35.481512 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7j468_a2197c57-2dc6-410e-9910-a045dedcb8d9/konnectivity-agent/0.log" limit=1024 Jul 30 16:16:36.551365 ip-10-0-133-210 kubenswrapper[2576]: I0730 16:16:36.551303 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7j468_a2197c57-2dc6-410e-9910-a045dedcb8d9/konnectivity-agent/0.log" limit=1024 Jul 30 16:16:36.647216 ip-10-0-133-210 kubenswrapper[2576]: I0730 16:16:36.647183 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7j468_a2197c57-2dc6-410e-9910-a045dedcb8d9/konnectivity-agent/0.log" limit=1024 Jul 30 16:16:40.604644 ip-10-0-133-210 kubenswrapper[2576]: I0730 16:16:40.604607 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-qvn96_e76c844a-a34c-4587-93a7-023d511a0b9f/global-pull-secret-syncer/0.log" Jul 30 16:16:40.659786 ip-10-0-133-210 kubenswrapper[2576]: I0730 16:16:40.659754 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-7j468_a2197c57-2dc6-410e-9910-a045dedcb8d9/konnectivity-agent/0.log" Jul 30 16:16:40.762851 ip-10-0-133-210 kubenswrapper[2576]: I0730 16:16:40.762818 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-133-210.ec2.internal_2da2e1115617abcb95d9273e2e0aad34/haproxy/0.log" Jul 30 16:16:48.336679 ip-10-0-133-210 kubenswrapper[2576]: I0730 16:16:48.336644 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_monitoring-plugin-68b7fc9574-9h5mk_f29662a6-58a3-4a59-bc19-4e8bfc47de63/monitoring-plugin/0.log" Jul 30 16:16:48.520356 ip-10-0-133-210 kubenswrapper[2576]: I0730 16:16:48.520328 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-sv88g_fb41dae1-7bc8-46a1-a05e-73f26d7c0391/node-exporter/0.log" Jul 30 16:16:48.541508 ip-10-0-133-210 kubenswrapper[2576]: I0730 16:16:48.541474 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-sv88g_fb41dae1-7bc8-46a1-a05e-73f26d7c0391/kube-rbac-proxy/0.log" Jul 30 16:16:48.568941 ip-10-0-133-210 kubenswrapper[2576]: I0730 16:16:48.568912 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-sv88g_fb41dae1-7bc8-46a1-a05e-73f26d7c0391/init-textfile/0.log" Jul 30 16:16:48.607099 ip-10-0-133-210 kubenswrapper[2576]: I0730 16:16:48.607007 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-b8f49f5c6-7dcnj_16cf6c6b-a33a-445c-8a97-26226073093c/kube-rbac-proxy-main/0.log" Jul 30 16:16:48.632976 ip-10-0-133-210 kubenswrapper[2576]: I0730 16:16:48.632952 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-b8f49f5c6-7dcnj_16cf6c6b-a33a-445c-8a97-26226073093c/kube-rbac-proxy-self/0.log" Jul 30 16:16:48.658758 ip-10-0-133-210 kubenswrapper[2576]: I0730 16:16:48.658733 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-b8f49f5c6-7dcnj_16cf6c6b-a33a-445c-8a97-26226073093c/openshift-state-metrics/0.log" Jul 30 16:16:49.191431 ip-10-0-133-210 kubenswrapper[2576]: I0730 16:16:49.191399 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-6ff66dc4cf-kl2x6_0af6b334-8a41-459d-9e05-4d3769d69a4c/thanos-query/0.log" Jul 30 16:16:49.234875 ip-10-0-133-210 kubenswrapper[2576]: I0730 16:16:49.234838 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-6ff66dc4cf-kl2x6_0af6b334-8a41-459d-9e05-4d3769d69a4c/kube-rbac-proxy-web/0.log" Jul 30 16:16:49.267856 ip-10-0-133-210 kubenswrapper[2576]: I0730 16:16:49.267825 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-6ff66dc4cf-kl2x6_0af6b334-8a41-459d-9e05-4d3769d69a4c/kube-rbac-proxy/0.log" Jul 30 16:16:49.297152 ip-10-0-133-210 kubenswrapper[2576]: I0730 16:16:49.297103 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-6ff66dc4cf-kl2x6_0af6b334-8a41-459d-9e05-4d3769d69a4c/prom-label-proxy/0.log" Jul 30 16:16:49.319663 ip-10-0-133-210 kubenswrapper[2576]: I0730 16:16:49.319629 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-6ff66dc4cf-kl2x6_0af6b334-8a41-459d-9e05-4d3769d69a4c/kube-rbac-proxy-rules/0.log" Jul 30 16:16:49.347217 ip-10-0-133-210 kubenswrapper[2576]: I0730 16:16:49.347181 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-6ff66dc4cf-kl2x6_0af6b334-8a41-459d-9e05-4d3769d69a4c/kube-rbac-proxy-metrics/0.log" Jul 30 16:16:50.584520 ip-10-0-133-210 kubenswrapper[2576]: I0730 16:16:50.584485 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-network-console_networking-console-plugin-74c9f884b7-lcg2q_4ce0180e-5f58-4c5c-bffd-be53eb55a430/networking-console-plugin/0.log" Jul 30 16:16:51.176440 ip-10-0-133-210 kubenswrapper[2576]: I0730 16:16:51.176404 2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-cv7vc/perf-node-gather-daemonset-zj64k"] Jul 30 16:16:51.176726 ip-10-0-133-210 kubenswrapper[2576]: I0730 16:16:51.176713 2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b2b10631-e60b-4dd1-85e4-91c54d95e72a" containerName="revoke-keys" Jul 30 16:16:51.176771 ip-10-0-133-210 kubenswrapper[2576]: I0730 16:16:51.176727 2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="b2b10631-e60b-4dd1-85e4-91c54d95e72a" containerName="revoke-keys" Jul 30 16:16:51.176806 ip-10-0-133-210 kubenswrapper[2576]: I0730 16:16:51.176779 2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="b2b10631-e60b-4dd1-85e4-91c54d95e72a" containerName="revoke-keys" Jul 30 16:16:51.178578 ip-10-0-133-210 kubenswrapper[2576]: I0730 16:16:51.178563 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-cv7vc/perf-node-gather-daemonset-zj64k" Jul 30 16:16:51.181557 ip-10-0-133-210 kubenswrapper[2576]: I0730 16:16:51.181530 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-cv7vc\"/\"kube-root-ca.crt\"" Jul 30 16:16:51.181691 ip-10-0-133-210 kubenswrapper[2576]: I0730 16:16:51.181568 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-cv7vc\"/\"openshift-service-ca.crt\"" Jul 30 16:16:51.182947 ip-10-0-133-210 kubenswrapper[2576]: I0730 16:16:51.182927 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-cv7vc\"/\"default-dockercfg-jszsr\"" Jul 30 16:16:51.189126 ip-10-0-133-210 kubenswrapper[2576]: I0730 16:16:51.189089 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-cv7vc/perf-node-gather-daemonset-zj64k"] Jul 30 16:16:51.253588 ip-10-0-133-210 kubenswrapper[2576]: I0730 16:16:51.253537 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8rqc8\" (UniqueName: \"kubernetes.io/projected/a3a8d054-7253-4ef6-bb08-7198a4573077-kube-api-access-8rqc8\") pod \"perf-node-gather-daemonset-zj64k\" (UID: \"a3a8d054-7253-4ef6-bb08-7198a4573077\") " pod="openshift-must-gather-cv7vc/perf-node-gather-daemonset-zj64k" Jul 30 16:16:51.253860 ip-10-0-133-210 kubenswrapper[2576]: I0730 16:16:51.253599 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/a3a8d054-7253-4ef6-bb08-7198a4573077-lib-modules\") pod \"perf-node-gather-daemonset-zj64k\" (UID: \"a3a8d054-7253-4ef6-bb08-7198a4573077\") " pod="openshift-must-gather-cv7vc/perf-node-gather-daemonset-zj64k" Jul 30 16:16:51.253860 ip-10-0-133-210 kubenswrapper[2576]: I0730 16:16:51.253703 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/a3a8d054-7253-4ef6-bb08-7198a4573077-proc\") pod \"perf-node-gather-daemonset-zj64k\" (UID: \"a3a8d054-7253-4ef6-bb08-7198a4573077\") " pod="openshift-must-gather-cv7vc/perf-node-gather-daemonset-zj64k" Jul 30 16:16:51.253860 ip-10-0-133-210 kubenswrapper[2576]: I0730 16:16:51.253748 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/a3a8d054-7253-4ef6-bb08-7198a4573077-podres\") pod \"perf-node-gather-daemonset-zj64k\" (UID: \"a3a8d054-7253-4ef6-bb08-7198a4573077\") " pod="openshift-must-gather-cv7vc/perf-node-gather-daemonset-zj64k" Jul 30 16:16:51.253860 ip-10-0-133-210 kubenswrapper[2576]: I0730 16:16:51.253777 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/a3a8d054-7253-4ef6-bb08-7198a4573077-sys\") pod \"perf-node-gather-daemonset-zj64k\" (UID: \"a3a8d054-7253-4ef6-bb08-7198a4573077\") " pod="openshift-must-gather-cv7vc/perf-node-gather-daemonset-zj64k" Jul 30 16:16:51.355036 ip-10-0-133-210 kubenswrapper[2576]: I0730 16:16:51.354991 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8rqc8\" (UniqueName: \"kubernetes.io/projected/a3a8d054-7253-4ef6-bb08-7198a4573077-kube-api-access-8rqc8\") pod \"perf-node-gather-daemonset-zj64k\" (UID: \"a3a8d054-7253-4ef6-bb08-7198a4573077\") " pod="openshift-must-gather-cv7vc/perf-node-gather-daemonset-zj64k" Jul 30 16:16:51.355036 ip-10-0-133-210 kubenswrapper[2576]: I0730 16:16:51.355041 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/a3a8d054-7253-4ef6-bb08-7198a4573077-lib-modules\") pod \"perf-node-gather-daemonset-zj64k\" (UID: \"a3a8d054-7253-4ef6-bb08-7198a4573077\") " pod="openshift-must-gather-cv7vc/perf-node-gather-daemonset-zj64k" Jul 30 16:16:51.355338 ip-10-0-133-210 kubenswrapper[2576]: I0730 16:16:51.355085 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/a3a8d054-7253-4ef6-bb08-7198a4573077-proc\") pod \"perf-node-gather-daemonset-zj64k\" (UID: \"a3a8d054-7253-4ef6-bb08-7198a4573077\") " pod="openshift-must-gather-cv7vc/perf-node-gather-daemonset-zj64k" Jul 30 16:16:51.355338 ip-10-0-133-210 kubenswrapper[2576]: I0730 16:16:51.355138 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/a3a8d054-7253-4ef6-bb08-7198a4573077-podres\") pod \"perf-node-gather-daemonset-zj64k\" (UID: \"a3a8d054-7253-4ef6-bb08-7198a4573077\") " pod="openshift-must-gather-cv7vc/perf-node-gather-daemonset-zj64k" Jul 30 16:16:51.355338 ip-10-0-133-210 kubenswrapper[2576]: I0730 16:16:51.355165 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/a3a8d054-7253-4ef6-bb08-7198a4573077-sys\") pod \"perf-node-gather-daemonset-zj64k\" (UID: \"a3a8d054-7253-4ef6-bb08-7198a4573077\") " pod="openshift-must-gather-cv7vc/perf-node-gather-daemonset-zj64k" Jul 30 16:16:51.355338 ip-10-0-133-210 kubenswrapper[2576]: I0730 16:16:51.355205 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/a3a8d054-7253-4ef6-bb08-7198a4573077-proc\") pod \"perf-node-gather-daemonset-zj64k\" (UID: \"a3a8d054-7253-4ef6-bb08-7198a4573077\") " pod="openshift-must-gather-cv7vc/perf-node-gather-daemonset-zj64k" Jul 30 16:16:51.355338 ip-10-0-133-210 kubenswrapper[2576]: I0730 16:16:51.355250 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/a3a8d054-7253-4ef6-bb08-7198a4573077-lib-modules\") pod \"perf-node-gather-daemonset-zj64k\" (UID: \"a3a8d054-7253-4ef6-bb08-7198a4573077\") " pod="openshift-must-gather-cv7vc/perf-node-gather-daemonset-zj64k" Jul 30 16:16:51.355338 ip-10-0-133-210 kubenswrapper[2576]: I0730 16:16:51.355284 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/a3a8d054-7253-4ef6-bb08-7198a4573077-sys\") pod \"perf-node-gather-daemonset-zj64k\" (UID: \"a3a8d054-7253-4ef6-bb08-7198a4573077\") " pod="openshift-must-gather-cv7vc/perf-node-gather-daemonset-zj64k" Jul 30 16:16:51.355338 ip-10-0-133-210 kubenswrapper[2576]: I0730 16:16:51.355320 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/a3a8d054-7253-4ef6-bb08-7198a4573077-podres\") pod \"perf-node-gather-daemonset-zj64k\" (UID: \"a3a8d054-7253-4ef6-bb08-7198a4573077\") " pod="openshift-must-gather-cv7vc/perf-node-gather-daemonset-zj64k" Jul 30 16:16:51.366159 ip-10-0-133-210 kubenswrapper[2576]: I0730 16:16:51.366130 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8rqc8\" (UniqueName: \"kubernetes.io/projected/a3a8d054-7253-4ef6-bb08-7198a4573077-kube-api-access-8rqc8\") pod \"perf-node-gather-daemonset-zj64k\" (UID: \"a3a8d054-7253-4ef6-bb08-7198a4573077\") " pod="openshift-must-gather-cv7vc/perf-node-gather-daemonset-zj64k" Jul 30 16:16:51.489233 ip-10-0-133-210 kubenswrapper[2576]: I0730 16:16:51.489199 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-cv7vc/perf-node-gather-daemonset-zj64k" Jul 30 16:16:51.619042 ip-10-0-133-210 kubenswrapper[2576]: I0730 16:16:51.619010 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-cv7vc/perf-node-gather-daemonset-zj64k"] Jul 30 16:16:51.622347 ip-10-0-133-210 kubenswrapper[2576]: W0730 16:16:51.622322 2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-poda3a8d054_7253_4ef6_bb08_7198a4573077.slice/crio-8f3988f1cd4e21870cc78b2e4ae86b7b496be8e6a046a46114d2b42b62e702f0 WatchSource:0}: Error finding container 8f3988f1cd4e21870cc78b2e4ae86b7b496be8e6a046a46114d2b42b62e702f0: Status 404 returned error can't find the container with id 8f3988f1cd4e21870cc78b2e4ae86b7b496be8e6a046a46114d2b42b62e702f0 Jul 30 16:16:51.811016 ip-10-0-133-210 kubenswrapper[2576]: I0730 16:16:51.810934 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_downloads-7c97dc4555-w82pr_2ddfacc9-629f-447c-badb-ae14f1ed1abf/download-server/0.log" Jul 30 16:16:52.260269 ip-10-0-133-210 kubenswrapper[2576]: I0730 16:16:52.260230 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-cv7vc/perf-node-gather-daemonset-zj64k" event={"ID":"a3a8d054-7253-4ef6-bb08-7198a4573077","Type":"ContainerStarted","Data":"89380e61e806abb8a447d0a501e4d3c1f96e342530ac831216575b953bb933b3"} Jul 30 16:16:52.260436 ip-10-0-133-210 kubenswrapper[2576]: I0730 16:16:52.260293 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-cv7vc/perf-node-gather-daemonset-zj64k" event={"ID":"a3a8d054-7253-4ef6-bb08-7198a4573077","Type":"ContainerStarted","Data":"8f3988f1cd4e21870cc78b2e4ae86b7b496be8e6a046a46114d2b42b62e702f0"} Jul 30 16:16:52.260436 ip-10-0-133-210 kubenswrapper[2576]: I0730 16:16:52.260352 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-cv7vc/perf-node-gather-daemonset-zj64k" Jul 30 16:16:52.284644 ip-10-0-133-210 kubenswrapper[2576]: I0730 16:16:52.284591 2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-cv7vc/perf-node-gather-daemonset-zj64k" podStartSLOduration=1.284575939 podStartE2EDuration="1.284575939s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-30 16:16:51 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 16:16:52.282437211 +0000 UTC m=+3994.208189858" watchObservedRunningTime="2026-07-30 16:16:52.284575939 +0000 UTC m=+3994.210328583" Jul 30 16:16:53.654241 ip-10-0-133-210 kubenswrapper[2576]: I0730 16:16:53.654206 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-jwbls_c460206a-fa1f-4c09-b498-467313191c13/dns/0.log" Jul 30 16:16:53.677313 ip-10-0-133-210 kubenswrapper[2576]: I0730 16:16:53.677288 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-jwbls_c460206a-fa1f-4c09-b498-467313191c13/kube-rbac-proxy/0.log" Jul 30 16:16:53.729260 ip-10-0-133-210 kubenswrapper[2576]: I0730 16:16:53.729230 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-j74qd_347bd19f-7728-47be-986d-337d0f0df246/dns-node-resolver/0.log" Jul 30 16:16:54.296070 ip-10-0-133-210 kubenswrapper[2576]: I0730 16:16:54.296034 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_image-registry-5c455b4d8b-ns98l_c11dde2f-dda0-4c65-818a-661b94d8799f/registry/0.log" Jul 30 16:16:54.387828 ip-10-0-133-210 kubenswrapper[2576]: I0730 16:16:54.387798 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-ktmgn_4b006058-0163-4e82-9a1f-979b182ddbf1/node-ca/0.log" Jul 30 16:16:55.509361 ip-10-0-133-210 kubenswrapper[2576]: I0730 16:16:55.509326 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-5677ccb49b-htlrm_291e793a-4cbe-4576-a16f-eef021410b23/discovery/0.log" Jul 30 16:16:55.538967 ip-10-0-133-210 kubenswrapper[2576]: I0730 16:16:55.538940 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-6976f6d894-6ckvp_09e1e2ea-ba7d-45e6-94eb-4ef6726590d3/kube-auth-proxy/0.log" Jul 30 16:16:56.579811 ip-10-0-133-210 kubenswrapper[2576]: I0730 16:16:56.579779 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-qtv7l_647c67b8-98e9-4905-9e04-35cc6cc3bd33/serve-healthcheck-canary/0.log" Jul 30 16:16:57.158856 ip-10-0-133-210 kubenswrapper[2576]: I0730 16:16:57.158825 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-dvkw7_9f3ebd63-d1cc-4920-94dc-65d6a62b96fa/kube-rbac-proxy/0.log" Jul 30 16:16:57.181401 ip-10-0-133-210 kubenswrapper[2576]: I0730 16:16:57.181375 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-dvkw7_9f3ebd63-d1cc-4920-94dc-65d6a62b96fa/exporter/0.log" Jul 30 16:16:57.207956 ip-10-0-133-210 kubenswrapper[2576]: I0730 16:16:57.207926 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-dvkw7_9f3ebd63-d1cc-4920-94dc-65d6a62b96fa/extractor/0.log" Jul 30 16:16:58.274509 ip-10-0-133-210 kubenswrapper[2576]: I0730 16:16:58.274481 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-cv7vc/perf-node-gather-daemonset-zj64k" Jul 30 16:17:00.051321 ip-10-0-133-210 kubenswrapper[2576]: I0730 16:17:00.051294 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-56444d5c7b-jdfll_2fcf0d71-64a1-4a7d-812b-ef92d1e5c927/manager/0.log" Jul 30 16:17:00.073163 ip-10-0-133-210 kubenswrapper[2576]: I0730 16:17:00.073136 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-56444d5c7b-jdfll_2fcf0d71-64a1-4a7d-812b-ef92d1e5c927/copy-manifests/0.log" Jul 30 16:17:00.740859 ip-10-0-133-210 kubenswrapper[2576]: I0730 16:17:00.740824 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-5677ccb49b-htlrm_291e793a-4cbe-4576-a16f-eef021410b23/discovery/0.log" Jul 30 16:17:00.751779 ip-10-0-133-210 kubenswrapper[2576]: I0730 16:17:00.751748 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-6976f6d894-6ckvp_09e1e2ea-ba7d-45e6-94eb-4ef6726590d3/kube-auth-proxy/0.log" Jul 30 16:17:01.613774 ip-10-0-133-210 kubenswrapper[2576]: I0730 16:17:01.613748 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-lws-operator_lws-controller-manager-556f467f75-v29c7_fb4933d3-a2b3-49b0-bcdd-70b6fbbdfea0/manager/0.log" Jul 30 16:17:01.640121 ip-10-0-133-210 kubenswrapper[2576]: I0730 16:17:01.640079 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-lws-operator_openshift-lws-operator-fd8ccff4c-tpzh9_265f471f-8da2-4164-8dc4-38d8b46a1090/openshift-lws-operator/0.log" Jul 30 16:17:02.457041 ip-10-0-133-210 kubenswrapper[2576]: I0730 16:17:02.457004 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-5677ccb49b-htlrm_291e793a-4cbe-4576-a16f-eef021410b23/discovery/0.log" Jul 30 16:17:02.470942 ip-10-0-133-210 kubenswrapper[2576]: I0730 16:17:02.470915 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-6976f6d894-6ckvp_09e1e2ea-ba7d-45e6-94eb-4ef6726590d3/kube-auth-proxy/0.log" Jul 30 16:17:07.801541 ip-10-0-133-210 kubenswrapper[2576]: I0730 16:17:07.801512 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-m2j4z_f62011d0-f5be-413a-93a2-8a7a98b67130/kube-multus-additional-cni-plugins/0.log" Jul 30 16:17:07.837223 ip-10-0-133-210 kubenswrapper[2576]: I0730 16:17:07.837196 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-m2j4z_f62011d0-f5be-413a-93a2-8a7a98b67130/egress-router-binary-copy/0.log" Jul 30 16:17:07.873233 ip-10-0-133-210 kubenswrapper[2576]: I0730 16:17:07.873201 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-m2j4z_f62011d0-f5be-413a-93a2-8a7a98b67130/cni-plugins/0.log" Jul 30 16:17:07.909180 ip-10-0-133-210 kubenswrapper[2576]: I0730 16:17:07.909148 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-m2j4z_f62011d0-f5be-413a-93a2-8a7a98b67130/bond-cni-plugin/0.log" Jul 30 16:17:07.945061 ip-10-0-133-210 kubenswrapper[2576]: I0730 16:17:07.945029 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-m2j4z_f62011d0-f5be-413a-93a2-8a7a98b67130/routeoverride-cni/0.log" Jul 30 16:17:07.981227 ip-10-0-133-210 kubenswrapper[2576]: I0730 16:17:07.981200 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-m2j4z_f62011d0-f5be-413a-93a2-8a7a98b67130/whereabouts-cni-bincopy/0.log" Jul 30 16:17:08.017051 ip-10-0-133-210 kubenswrapper[2576]: I0730 16:17:08.017021 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-m2j4z_f62011d0-f5be-413a-93a2-8a7a98b67130/whereabouts-cni/0.log" Jul 30 16:17:08.377727 ip-10-0-133-210 kubenswrapper[2576]: I0730 16:17:08.377699 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-gb69c_23fd99c2-79a8-41eb-bb0f-92a363e32553/kube-multus/0.log"