Jul 21 12:37:44.906680 ip-10-0-130-157 systemd[1]: kubelet.service: Failed to load environment files: No such file or directory Jul 21 12:37:44.906691 ip-10-0-130-157 systemd[1]: kubelet.service: Failed to run 'start-pre' task: No such file or directory Jul 21 12:37:44.906700 ip-10-0-130-157 systemd[1]: kubelet.service: Failed with result 'resources'. Jul 21 12:37:44.907005 ip-10-0-130-157 systemd[1]: Failed to start Kubernetes Kubelet. Jul 21 12:37:54.958954 ip-10-0-130-157 systemd[1]: kubelet.service: Failed to schedule restart job: Unit crio.service not found. Jul 21 12:37:54.958972 ip-10-0-130-157 systemd[1]: kubelet.service: Failed with result 'resources'. -- Boot 721d54dee9a448c4ae0efb686919bf15 -- Jul 21 12:39:50.676961 ip-10-0-130-157 systemd[1]: Starting Kubernetes Kubelet... Jul 21 12:39:51.202684 ip-10-0-130-157 kubenswrapper[2576]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 21 12:39:51.202684 ip-10-0-130-157 kubenswrapper[2576]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version. Jul 21 12:39:51.202684 ip-10-0-130-157 kubenswrapper[2576]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 21 12:39:51.202684 ip-10-0-130-157 kubenswrapper[2576]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI. Jul 21 12:39:51.202684 ip-10-0-130-157 kubenswrapper[2576]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 21 12:39:51.202684 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:39:51.104171 2576 server.go:213] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime" Jul 21 12:39:53.604915 ip-10-0-130-157 systemd[1]: Started Kubernetes Kubelet. Jul 21 12:39:53.655271 ip-10-0-130-157 kubenswrapper[2576]: W0721 12:39:53.538555 2576 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 21 12:39:53.655271 ip-10-0-130-157 kubenswrapper[2576]: W0721 12:39:53.538576 2576 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 21 12:39:53.655271 ip-10-0-130-157 kubenswrapper[2576]: W0721 12:39:53.538581 2576 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 21 12:39:53.655271 ip-10-0-130-157 kubenswrapper[2576]: W0721 12:39:53.538584 2576 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 21 12:39:53.655271 ip-10-0-130-157 kubenswrapper[2576]: W0721 12:39:53.538586 2576 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 21 12:39:53.655271 ip-10-0-130-157 kubenswrapper[2576]: W0721 12:39:53.538589 2576 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 21 12:39:53.655271 ip-10-0-130-157 kubenswrapper[2576]: W0721 12:39:53.538592 2576 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 21 12:39:53.655271 ip-10-0-130-157 kubenswrapper[2576]: W0721 12:39:53.538595 2576 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 21 12:39:53.655271 ip-10-0-130-157 kubenswrapper[2576]: W0721 12:39:53.538598 2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 21 12:39:53.655271 ip-10-0-130-157 kubenswrapper[2576]: W0721 12:39:53.538600 2576 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 21 12:39:53.655271 ip-10-0-130-157 kubenswrapper[2576]: W0721 12:39:53.538603 2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 21 12:39:53.655271 ip-10-0-130-157 kubenswrapper[2576]: W0721 12:39:53.538605 2576 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 21 12:39:53.655271 ip-10-0-130-157 kubenswrapper[2576]: W0721 12:39:53.538608 2576 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 21 12:39:53.655271 ip-10-0-130-157 kubenswrapper[2576]: W0721 12:39:53.538611 2576 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 21 12:39:53.655271 ip-10-0-130-157 kubenswrapper[2576]: W0721 12:39:53.538614 2576 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 21 12:39:53.655271 ip-10-0-130-157 kubenswrapper[2576]: W0721 12:39:53.538616 2576 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 21 12:39:53.655271 ip-10-0-130-157 kubenswrapper[2576]: W0721 12:39:53.538619 2576 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 21 12:39:53.655271 ip-10-0-130-157 kubenswrapper[2576]: W0721 12:39:53.538622 2576 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 21 12:39:53.655271 ip-10-0-130-157 kubenswrapper[2576]: W0721 12:39:53.538624 2576 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 21 12:39:53.655271 ip-10-0-130-157 kubenswrapper[2576]: W0721 12:39:53.538627 2576 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 21 12:39:53.657526 ip-10-0-130-157 kubenswrapper[2576]: W0721 12:39:53.538629 2576 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 21 12:39:53.657526 ip-10-0-130-157 kubenswrapper[2576]: W0721 12:39:53.538632 2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 21 12:39:53.657526 ip-10-0-130-157 kubenswrapper[2576]: W0721 12:39:53.538635 2576 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 21 12:39:53.657526 ip-10-0-130-157 kubenswrapper[2576]: W0721 12:39:53.538637 2576 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 21 12:39:53.657526 ip-10-0-130-157 kubenswrapper[2576]: W0721 12:39:53.538640 2576 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 21 12:39:53.657526 ip-10-0-130-157 kubenswrapper[2576]: W0721 12:39:53.538642 2576 feature_gate.go:328] unrecognized feature gate: Example Jul 21 12:39:53.657526 ip-10-0-130-157 kubenswrapper[2576]: W0721 12:39:53.538644 2576 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 21 12:39:53.657526 ip-10-0-130-157 kubenswrapper[2576]: W0721 12:39:53.538647 2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 21 12:39:53.657526 ip-10-0-130-157 kubenswrapper[2576]: W0721 12:39:53.538650 2576 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 21 12:39:53.657526 ip-10-0-130-157 kubenswrapper[2576]: W0721 12:39:53.538652 2576 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 21 12:39:53.657526 ip-10-0-130-157 kubenswrapper[2576]: W0721 12:39:53.538655 2576 feature_gate.go:328] unrecognized feature gate: Example2 Jul 21 12:39:53.657526 ip-10-0-130-157 kubenswrapper[2576]: W0721 12:39:53.538662 2576 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 21 12:39:53.657526 ip-10-0-130-157 kubenswrapper[2576]: W0721 12:39:53.538665 2576 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 21 12:39:53.657526 ip-10-0-130-157 kubenswrapper[2576]: W0721 12:39:53.538668 2576 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 21 12:39:53.657526 ip-10-0-130-157 kubenswrapper[2576]: W0721 12:39:53.538671 2576 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 21 12:39:53.657526 ip-10-0-130-157 kubenswrapper[2576]: W0721 12:39:53.538673 2576 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 21 12:39:53.657526 ip-10-0-130-157 kubenswrapper[2576]: W0721 12:39:53.538676 2576 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 21 12:39:53.657526 ip-10-0-130-157 kubenswrapper[2576]: W0721 12:39:53.538679 2576 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 21 12:39:53.657526 ip-10-0-130-157 kubenswrapper[2576]: W0721 12:39:53.538681 2576 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 21 12:39:53.657526 ip-10-0-130-157 kubenswrapper[2576]: W0721 12:39:53.538684 2576 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 21 12:39:53.658314 ip-10-0-130-157 kubenswrapper[2576]: W0721 12:39:53.538686 2576 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 21 12:39:53.658314 ip-10-0-130-157 kubenswrapper[2576]: W0721 12:39:53.538689 2576 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 21 12:39:53.658314 ip-10-0-130-157 kubenswrapper[2576]: W0721 12:39:53.538691 2576 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 21 12:39:53.658314 ip-10-0-130-157 kubenswrapper[2576]: W0721 12:39:53.538694 2576 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 21 12:39:53.658314 ip-10-0-130-157 kubenswrapper[2576]: W0721 12:39:53.538696 2576 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 21 12:39:53.658314 ip-10-0-130-157 kubenswrapper[2576]: W0721 12:39:53.538700 2576 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 21 12:39:53.658314 ip-10-0-130-157 kubenswrapper[2576]: W0721 12:39:53.538703 2576 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 21 12:39:53.658314 ip-10-0-130-157 kubenswrapper[2576]: W0721 12:39:53.538705 2576 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 21 12:39:53.658314 ip-10-0-130-157 kubenswrapper[2576]: W0721 12:39:53.538708 2576 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 21 12:39:53.658314 ip-10-0-130-157 kubenswrapper[2576]: W0721 12:39:53.538711 2576 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 21 12:39:53.658314 ip-10-0-130-157 kubenswrapper[2576]: W0721 12:39:53.538713 2576 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 21 12:39:53.658314 ip-10-0-130-157 kubenswrapper[2576]: W0721 12:39:53.538716 2576 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 21 12:39:53.658314 ip-10-0-130-157 kubenswrapper[2576]: W0721 12:39:53.538719 2576 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 21 12:39:53.658314 ip-10-0-130-157 kubenswrapper[2576]: W0721 12:39:53.538721 2576 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 21 12:39:53.658314 ip-10-0-130-157 kubenswrapper[2576]: W0721 12:39:53.538724 2576 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 21 12:39:53.658314 ip-10-0-130-157 kubenswrapper[2576]: W0721 12:39:53.538726 2576 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 21 12:39:53.658314 ip-10-0-130-157 kubenswrapper[2576]: W0721 12:39:53.538728 2576 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 21 12:39:53.658314 ip-10-0-130-157 kubenswrapper[2576]: W0721 12:39:53.538734 2576 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 21 12:39:53.658314 ip-10-0-130-157 kubenswrapper[2576]: W0721 12:39:53.538736 2576 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 21 12:39:53.658314 ip-10-0-130-157 kubenswrapper[2576]: W0721 12:39:53.538739 2576 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 21 12:39:53.660385 ip-10-0-130-157 kubenswrapper[2576]: W0721 12:39:53.538741 2576 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 21 12:39:53.660385 ip-10-0-130-157 kubenswrapper[2576]: W0721 12:39:53.538745 2576 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 21 12:39:53.660385 ip-10-0-130-157 kubenswrapper[2576]: W0721 12:39:53.538748 2576 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 21 12:39:53.660385 ip-10-0-130-157 kubenswrapper[2576]: W0721 12:39:53.538750 2576 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 21 12:39:53.660385 ip-10-0-130-157 kubenswrapper[2576]: W0721 12:39:53.538753 2576 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 21 12:39:53.660385 ip-10-0-130-157 kubenswrapper[2576]: W0721 12:39:53.538756 2576 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 21 12:39:53.660385 ip-10-0-130-157 kubenswrapper[2576]: W0721 12:39:53.538759 2576 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 21 12:39:53.660385 ip-10-0-130-157 kubenswrapper[2576]: W0721 12:39:53.538762 2576 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 21 12:39:53.660385 ip-10-0-130-157 kubenswrapper[2576]: W0721 12:39:53.538764 2576 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 21 12:39:53.660385 ip-10-0-130-157 kubenswrapper[2576]: W0721 12:39:53.538767 2576 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 21 12:39:53.660385 ip-10-0-130-157 kubenswrapper[2576]: W0721 12:39:53.538769 2576 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 21 12:39:53.660385 ip-10-0-130-157 kubenswrapper[2576]: W0721 12:39:53.538772 2576 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 21 12:39:53.660385 ip-10-0-130-157 kubenswrapper[2576]: W0721 12:39:53.538775 2576 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 21 12:39:53.660385 ip-10-0-130-157 kubenswrapper[2576]: W0721 12:39:53.538778 2576 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 21 12:39:53.660385 ip-10-0-130-157 kubenswrapper[2576]: W0721 12:39:53.538781 2576 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 21 12:39:53.660385 ip-10-0-130-157 kubenswrapper[2576]: W0721 12:39:53.538783 2576 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 21 12:39:53.660385 ip-10-0-130-157 kubenswrapper[2576]: W0721 12:39:53.538786 2576 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 21 12:39:53.660385 ip-10-0-130-157 kubenswrapper[2576]: W0721 12:39:53.538788 2576 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 21 12:39:53.660385 ip-10-0-130-157 kubenswrapper[2576]: W0721 12:39:53.538791 2576 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 21 12:39:53.661199 ip-10-0-130-157 kubenswrapper[2576]: W0721 12:39:53.538793 2576 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 21 12:39:53.661199 ip-10-0-130-157 kubenswrapper[2576]: W0721 12:39:53.538797 2576 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 21 12:39:53.661199 ip-10-0-130-157 kubenswrapper[2576]: W0721 12:39:53.538800 2576 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 21 12:39:53.661199 ip-10-0-130-157 kubenswrapper[2576]: W0721 12:39:53.538803 2576 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 21 12:39:53.661199 ip-10-0-130-157 kubenswrapper[2576]: W0721 12:39:53.538809 2576 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 21 12:39:53.661199 ip-10-0-130-157 kubenswrapper[2576]: W0721 12:39:53.538814 2576 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 21 12:39:53.661199 ip-10-0-130-157 kubenswrapper[2576]: W0721 12:39:53.538816 2576 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 21 12:39:53.661199 ip-10-0-130-157 kubenswrapper[2576]: W0721 12:39:53.538821 2576 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 21 12:39:53.661199 ip-10-0-130-157 kubenswrapper[2576]: W0721 12:39:53.538826 2576 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 21 12:39:53.661199 ip-10-0-130-157 kubenswrapper[2576]: W0721 12:39:53.538830 2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 21 12:39:53.661199 ip-10-0-130-157 kubenswrapper[2576]: W0721 12:39:53.538834 2576 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 21 12:39:53.661199 ip-10-0-130-157 kubenswrapper[2576]: W0721 12:39:53.538837 2576 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 21 12:39:53.661199 ip-10-0-130-157 kubenswrapper[2576]: W0721 12:39:53.538841 2576 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 21 12:39:53.661199 ip-10-0-130-157 kubenswrapper[2576]: W0721 12:39:53.538844 2576 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 21 12:39:53.661199 ip-10-0-130-157 kubenswrapper[2576]: W0721 12:39:53.538846 2576 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 21 12:39:53.661199 ip-10-0-130-157 kubenswrapper[2576]: W0721 12:39:53.538849 2576 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 21 12:39:53.661199 ip-10-0-130-157 kubenswrapper[2576]: W0721 12:39:53.538852 2576 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 21 12:39:53.661199 ip-10-0-130-157 kubenswrapper[2576]: W0721 12:39:53.538854 2576 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 21 12:39:53.661199 ip-10-0-130-157 kubenswrapper[2576]: W0721 12:39:53.538857 2576 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 21 12:39:53.663615 ip-10-0-130-157 kubenswrapper[2576]: W0721 12:39:53.538860 2576 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 21 12:39:53.663615 ip-10-0-130-157 kubenswrapper[2576]: W0721 12:39:53.539659 2576 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 21 12:39:53.663615 ip-10-0-130-157 kubenswrapper[2576]: W0721 12:39:53.539667 2576 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 21 12:39:53.663615 ip-10-0-130-157 kubenswrapper[2576]: W0721 12:39:53.539670 2576 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 21 12:39:53.663615 ip-10-0-130-157 kubenswrapper[2576]: W0721 12:39:53.539674 2576 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 21 12:39:53.663615 ip-10-0-130-157 kubenswrapper[2576]: W0721 12:39:53.539677 2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 21 12:39:53.663615 ip-10-0-130-157 kubenswrapper[2576]: W0721 12:39:53.539679 2576 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 21 12:39:53.663615 ip-10-0-130-157 kubenswrapper[2576]: W0721 12:39:53.539682 2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 21 12:39:53.663615 ip-10-0-130-157 kubenswrapper[2576]: W0721 12:39:53.539685 2576 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 21 12:39:53.663615 ip-10-0-130-157 kubenswrapper[2576]: W0721 12:39:53.539688 2576 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 21 12:39:53.663615 ip-10-0-130-157 kubenswrapper[2576]: W0721 12:39:53.539691 2576 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 21 12:39:53.663615 ip-10-0-130-157 kubenswrapper[2576]: W0721 12:39:53.539694 2576 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 21 12:39:53.663615 ip-10-0-130-157 kubenswrapper[2576]: W0721 12:39:53.539697 2576 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 21 12:39:53.663615 ip-10-0-130-157 kubenswrapper[2576]: W0721 12:39:53.539699 2576 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 21 12:39:53.663615 ip-10-0-130-157 kubenswrapper[2576]: W0721 12:39:53.539703 2576 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 21 12:39:53.663615 ip-10-0-130-157 kubenswrapper[2576]: W0721 12:39:53.539707 2576 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 21 12:39:53.663615 ip-10-0-130-157 kubenswrapper[2576]: W0721 12:39:53.539710 2576 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 21 12:39:53.663615 ip-10-0-130-157 kubenswrapper[2576]: W0721 12:39:53.539713 2576 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 21 12:39:53.663615 ip-10-0-130-157 kubenswrapper[2576]: W0721 12:39:53.539720 2576 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 21 12:39:53.664422 ip-10-0-130-157 kubenswrapper[2576]: W0721 12:39:53.539723 2576 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 21 12:39:53.664422 ip-10-0-130-157 kubenswrapper[2576]: W0721 12:39:53.539726 2576 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 21 12:39:53.664422 ip-10-0-130-157 kubenswrapper[2576]: W0721 12:39:53.539728 2576 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 21 12:39:53.664422 ip-10-0-130-157 kubenswrapper[2576]: W0721 12:39:53.539731 2576 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 21 12:39:53.664422 ip-10-0-130-157 kubenswrapper[2576]: W0721 12:39:53.539733 2576 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 21 12:39:53.664422 ip-10-0-130-157 kubenswrapper[2576]: W0721 12:39:53.539736 2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 21 12:39:53.664422 ip-10-0-130-157 kubenswrapper[2576]: W0721 12:39:53.539739 2576 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 21 12:39:53.664422 ip-10-0-130-157 kubenswrapper[2576]: W0721 12:39:53.539742 2576 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 21 12:39:53.664422 ip-10-0-130-157 kubenswrapper[2576]: W0721 12:39:53.539746 2576 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 21 12:39:53.664422 ip-10-0-130-157 kubenswrapper[2576]: W0721 12:39:53.539749 2576 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 21 12:39:53.664422 ip-10-0-130-157 kubenswrapper[2576]: W0721 12:39:53.539753 2576 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 21 12:39:53.664422 ip-10-0-130-157 kubenswrapper[2576]: W0721 12:39:53.539755 2576 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 21 12:39:53.664422 ip-10-0-130-157 kubenswrapper[2576]: W0721 12:39:53.539758 2576 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 21 12:39:53.664422 ip-10-0-130-157 kubenswrapper[2576]: W0721 12:39:53.539761 2576 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 21 12:39:53.664422 ip-10-0-130-157 kubenswrapper[2576]: W0721 12:39:53.539764 2576 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 21 12:39:53.664422 ip-10-0-130-157 kubenswrapper[2576]: W0721 12:39:53.539766 2576 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 21 12:39:53.664422 ip-10-0-130-157 kubenswrapper[2576]: W0721 12:39:53.539769 2576 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 21 12:39:53.664422 ip-10-0-130-157 kubenswrapper[2576]: W0721 12:39:53.539771 2576 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 21 12:39:53.664422 ip-10-0-130-157 kubenswrapper[2576]: W0721 12:39:53.539774 2576 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 21 12:39:53.667746 ip-10-0-130-157 kubenswrapper[2576]: W0721 12:39:53.539777 2576 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 21 12:39:53.667746 ip-10-0-130-157 kubenswrapper[2576]: W0721 12:39:53.539780 2576 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 21 12:39:53.667746 ip-10-0-130-157 kubenswrapper[2576]: W0721 12:39:53.539783 2576 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 21 12:39:53.667746 ip-10-0-130-157 kubenswrapper[2576]: W0721 12:39:53.539786 2576 feature_gate.go:328] unrecognized feature gate: Example Jul 21 12:39:53.667746 ip-10-0-130-157 kubenswrapper[2576]: W0721 12:39:53.539788 2576 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 21 12:39:53.667746 ip-10-0-130-157 kubenswrapper[2576]: W0721 12:39:53.539791 2576 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 21 12:39:53.667746 ip-10-0-130-157 kubenswrapper[2576]: W0721 12:39:53.539794 2576 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 21 12:39:53.667746 ip-10-0-130-157 kubenswrapper[2576]: W0721 12:39:53.539797 2576 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 21 12:39:53.667746 ip-10-0-130-157 kubenswrapper[2576]: W0721 12:39:53.539800 2576 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 21 12:39:53.667746 ip-10-0-130-157 kubenswrapper[2576]: W0721 12:39:53.539802 2576 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 21 12:39:53.667746 ip-10-0-130-157 kubenswrapper[2576]: W0721 12:39:53.539805 2576 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 21 12:39:53.667746 ip-10-0-130-157 kubenswrapper[2576]: W0721 12:39:53.539807 2576 feature_gate.go:328] unrecognized feature gate: Example2 Jul 21 12:39:53.667746 ip-10-0-130-157 kubenswrapper[2576]: W0721 12:39:53.539810 2576 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 21 12:39:53.667746 ip-10-0-130-157 kubenswrapper[2576]: W0721 12:39:53.539813 2576 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 21 12:39:53.667746 ip-10-0-130-157 kubenswrapper[2576]: W0721 12:39:53.539815 2576 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 21 12:39:53.667746 ip-10-0-130-157 kubenswrapper[2576]: W0721 12:39:53.539817 2576 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 21 12:39:53.667746 ip-10-0-130-157 kubenswrapper[2576]: W0721 12:39:53.539820 2576 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 21 12:39:53.667746 ip-10-0-130-157 kubenswrapper[2576]: W0721 12:39:53.539822 2576 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 21 12:39:53.667746 ip-10-0-130-157 kubenswrapper[2576]: W0721 12:39:53.539825 2576 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 21 12:39:53.667746 ip-10-0-130-157 kubenswrapper[2576]: W0721 12:39:53.539828 2576 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 21 12:39:53.668948 ip-10-0-130-157 kubenswrapper[2576]: W0721 12:39:53.539831 2576 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 21 12:39:53.668948 ip-10-0-130-157 kubenswrapper[2576]: W0721 12:39:53.539833 2576 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 21 12:39:53.668948 ip-10-0-130-157 kubenswrapper[2576]: W0721 12:39:53.539836 2576 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 21 12:39:53.668948 ip-10-0-130-157 kubenswrapper[2576]: W0721 12:39:53.539838 2576 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 21 12:39:53.668948 ip-10-0-130-157 kubenswrapper[2576]: W0721 12:39:53.539841 2576 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 21 12:39:53.668948 ip-10-0-130-157 kubenswrapper[2576]: W0721 12:39:53.539843 2576 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 21 12:39:53.668948 ip-10-0-130-157 kubenswrapper[2576]: W0721 12:39:53.539846 2576 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 21 12:39:53.668948 ip-10-0-130-157 kubenswrapper[2576]: W0721 12:39:53.539848 2576 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 21 12:39:53.668948 ip-10-0-130-157 kubenswrapper[2576]: W0721 12:39:53.539851 2576 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 21 12:39:53.668948 ip-10-0-130-157 kubenswrapper[2576]: W0721 12:39:53.539854 2576 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 21 12:39:53.668948 ip-10-0-130-157 kubenswrapper[2576]: W0721 12:39:53.539857 2576 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 21 12:39:53.668948 ip-10-0-130-157 kubenswrapper[2576]: W0721 12:39:53.539860 2576 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 21 12:39:53.668948 ip-10-0-130-157 kubenswrapper[2576]: W0721 12:39:53.539862 2576 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 21 12:39:53.668948 ip-10-0-130-157 kubenswrapper[2576]: W0721 12:39:53.539865 2576 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 21 12:39:53.668948 ip-10-0-130-157 kubenswrapper[2576]: W0721 12:39:53.539867 2576 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 21 12:39:53.668948 ip-10-0-130-157 kubenswrapper[2576]: W0721 12:39:53.539870 2576 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 21 12:39:53.668948 ip-10-0-130-157 kubenswrapper[2576]: W0721 12:39:53.539872 2576 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 21 12:39:53.668948 ip-10-0-130-157 kubenswrapper[2576]: W0721 12:39:53.539875 2576 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 21 12:39:53.668948 ip-10-0-130-157 kubenswrapper[2576]: W0721 12:39:53.539878 2576 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 21 12:39:53.668948 ip-10-0-130-157 kubenswrapper[2576]: W0721 12:39:53.539880 2576 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 21 12:39:53.671803 ip-10-0-130-157 kubenswrapper[2576]: W0721 12:39:53.539882 2576 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 21 12:39:53.671803 ip-10-0-130-157 kubenswrapper[2576]: W0721 12:39:53.539885 2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 21 12:39:53.671803 ip-10-0-130-157 kubenswrapper[2576]: W0721 12:39:53.539888 2576 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 21 12:39:53.671803 ip-10-0-130-157 kubenswrapper[2576]: W0721 12:39:53.539890 2576 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 21 12:39:53.671803 ip-10-0-130-157 kubenswrapper[2576]: W0721 12:39:53.539893 2576 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 21 12:39:53.671803 ip-10-0-130-157 kubenswrapper[2576]: W0721 12:39:53.539896 2576 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 21 12:39:53.671803 ip-10-0-130-157 kubenswrapper[2576]: W0721 12:39:53.539900 2576 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 21 12:39:53.671803 ip-10-0-130-157 kubenswrapper[2576]: W0721 12:39:53.539902 2576 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 21 12:39:53.671803 ip-10-0-130-157 kubenswrapper[2576]: W0721 12:39:53.539906 2576 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 21 12:39:53.671803 ip-10-0-130-157 kubenswrapper[2576]: W0721 12:39:53.539910 2576 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 21 12:39:53.671803 ip-10-0-130-157 kubenswrapper[2576]: W0721 12:39:53.539913 2576 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 21 12:39:53.671803 ip-10-0-130-157 kubenswrapper[2576]: W0721 12:39:53.539916 2576 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 21 12:39:53.671803 ip-10-0-130-157 kubenswrapper[2576]: W0721 12:39:53.539919 2576 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 21 12:39:53.671803 ip-10-0-130-157 kubenswrapper[2576]: W0721 12:39:53.539921 2576 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 21 12:39:53.671803 ip-10-0-130-157 kubenswrapper[2576]: W0721 12:39:53.539924 2576 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 21 12:39:53.671803 ip-10-0-130-157 kubenswrapper[2576]: W0721 12:39:53.539927 2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 21 12:39:53.671803 ip-10-0-130-157 kubenswrapper[2576]: W0721 12:39:53.539930 2576 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 21 12:39:53.671803 ip-10-0-130-157 kubenswrapper[2576]: W0721 12:39:53.539932 2576 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 21 12:39:53.671803 ip-10-0-130-157 kubenswrapper[2576]: W0721 12:39:53.539935 2576 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 21 12:39:53.671803 ip-10-0-130-157 kubenswrapper[2576]: W0721 12:39:53.539938 2576 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 21 12:39:53.672623 ip-10-0-130-157 kubenswrapper[2576]: W0721 12:39:53.539940 2576 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 21 12:39:53.672623 ip-10-0-130-157 kubenswrapper[2576]: W0721 12:39:53.539943 2576 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 21 12:39:53.672623 ip-10-0-130-157 kubenswrapper[2576]: W0721 12:39:53.540382 2576 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 21 12:39:53.672623 ip-10-0-130-157 kubenswrapper[2576]: W0721 12:39:53.540388 2576 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 21 12:39:53.672623 ip-10-0-130-157 kubenswrapper[2576]: W0721 12:39:53.540391 2576 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 21 12:39:53.672623 ip-10-0-130-157 kubenswrapper[2576]: W0721 12:39:53.540394 2576 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 21 12:39:53.672623 ip-10-0-130-157 kubenswrapper[2576]: W0721 12:39:53.540396 2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 21 12:39:53.672623 ip-10-0-130-157 kubenswrapper[2576]: W0721 12:39:53.540399 2576 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 21 12:39:53.672623 ip-10-0-130-157 kubenswrapper[2576]: W0721 12:39:53.540402 2576 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 21 12:39:53.672623 ip-10-0-130-157 kubenswrapper[2576]: W0721 12:39:53.540405 2576 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 21 12:39:53.672623 ip-10-0-130-157 kubenswrapper[2576]: W0721 12:39:53.540408 2576 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 21 12:39:53.672623 ip-10-0-130-157 kubenswrapper[2576]: W0721 12:39:53.540410 2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 21 12:39:53.672623 ip-10-0-130-157 kubenswrapper[2576]: W0721 12:39:53.540413 2576 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 21 12:39:53.672623 ip-10-0-130-157 kubenswrapper[2576]: W0721 12:39:53.540415 2576 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 21 12:39:53.672623 ip-10-0-130-157 kubenswrapper[2576]: W0721 12:39:53.540418 2576 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 21 12:39:53.672623 ip-10-0-130-157 kubenswrapper[2576]: W0721 12:39:53.540420 2576 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 21 12:39:53.672623 ip-10-0-130-157 kubenswrapper[2576]: W0721 12:39:53.540423 2576 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 21 12:39:53.672623 ip-10-0-130-157 kubenswrapper[2576]: W0721 12:39:53.540426 2576 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 21 12:39:53.672623 ip-10-0-130-157 kubenswrapper[2576]: W0721 12:39:53.540430 2576 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 21 12:39:53.677537 ip-10-0-130-157 kubenswrapper[2576]: W0721 12:39:53.540433 2576 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 21 12:39:53.677537 ip-10-0-130-157 kubenswrapper[2576]: W0721 12:39:53.540436 2576 feature_gate.go:328] unrecognized feature gate: Example2 Jul 21 12:39:53.677537 ip-10-0-130-157 kubenswrapper[2576]: W0721 12:39:53.540438 2576 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 21 12:39:53.677537 ip-10-0-130-157 kubenswrapper[2576]: W0721 12:39:53.540441 2576 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 21 12:39:53.677537 ip-10-0-130-157 kubenswrapper[2576]: W0721 12:39:53.540443 2576 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 21 12:39:53.677537 ip-10-0-130-157 kubenswrapper[2576]: W0721 12:39:53.540446 2576 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 21 12:39:53.677537 ip-10-0-130-157 kubenswrapper[2576]: W0721 12:39:53.540449 2576 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 21 12:39:53.677537 ip-10-0-130-157 kubenswrapper[2576]: W0721 12:39:53.540451 2576 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 21 12:39:53.677537 ip-10-0-130-157 kubenswrapper[2576]: W0721 12:39:53.540454 2576 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 21 12:39:53.677537 ip-10-0-130-157 kubenswrapper[2576]: W0721 12:39:53.540457 2576 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 21 12:39:53.677537 ip-10-0-130-157 kubenswrapper[2576]: W0721 12:39:53.540459 2576 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 21 12:39:53.677537 ip-10-0-130-157 kubenswrapper[2576]: W0721 12:39:53.540462 2576 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 21 12:39:53.677537 ip-10-0-130-157 kubenswrapper[2576]: W0721 12:39:53.540464 2576 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 21 12:39:53.677537 ip-10-0-130-157 kubenswrapper[2576]: W0721 12:39:53.540467 2576 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 21 12:39:53.677537 ip-10-0-130-157 kubenswrapper[2576]: W0721 12:39:53.540470 2576 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 21 12:39:53.677537 ip-10-0-130-157 kubenswrapper[2576]: W0721 12:39:53.540472 2576 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 21 12:39:53.677537 ip-10-0-130-157 kubenswrapper[2576]: W0721 12:39:53.540475 2576 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 21 12:39:53.677537 ip-10-0-130-157 kubenswrapper[2576]: W0721 12:39:53.540478 2576 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 21 12:39:53.677537 ip-10-0-130-157 kubenswrapper[2576]: W0721 12:39:53.540481 2576 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 21 12:39:53.677537 ip-10-0-130-157 kubenswrapper[2576]: W0721 12:39:53.540483 2576 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 21 12:39:53.678364 ip-10-0-130-157 kubenswrapper[2576]: W0721 12:39:53.540488 2576 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 21 12:39:53.678364 ip-10-0-130-157 kubenswrapper[2576]: W0721 12:39:53.540492 2576 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 21 12:39:53.678364 ip-10-0-130-157 kubenswrapper[2576]: W0721 12:39:53.540495 2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 21 12:39:53.678364 ip-10-0-130-157 kubenswrapper[2576]: W0721 12:39:53.540499 2576 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 21 12:39:53.678364 ip-10-0-130-157 kubenswrapper[2576]: W0721 12:39:53.540501 2576 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 21 12:39:53.678364 ip-10-0-130-157 kubenswrapper[2576]: W0721 12:39:53.540504 2576 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 21 12:39:53.678364 ip-10-0-130-157 kubenswrapper[2576]: W0721 12:39:53.540507 2576 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 21 12:39:53.678364 ip-10-0-130-157 kubenswrapper[2576]: W0721 12:39:53.540510 2576 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 21 12:39:53.678364 ip-10-0-130-157 kubenswrapper[2576]: W0721 12:39:53.540528 2576 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 21 12:39:53.678364 ip-10-0-130-157 kubenswrapper[2576]: W0721 12:39:53.540531 2576 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 21 12:39:53.678364 ip-10-0-130-157 kubenswrapper[2576]: W0721 12:39:53.540534 2576 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 21 12:39:53.678364 ip-10-0-130-157 kubenswrapper[2576]: W0721 12:39:53.540537 2576 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 21 12:39:53.678364 ip-10-0-130-157 kubenswrapper[2576]: W0721 12:39:53.540543 2576 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 21 12:39:53.678364 ip-10-0-130-157 kubenswrapper[2576]: W0721 12:39:53.540546 2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 21 12:39:53.678364 ip-10-0-130-157 kubenswrapper[2576]: W0721 12:39:53.540550 2576 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 21 12:39:53.678364 ip-10-0-130-157 kubenswrapper[2576]: W0721 12:39:53.540553 2576 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 21 12:39:53.678364 ip-10-0-130-157 kubenswrapper[2576]: W0721 12:39:53.540556 2576 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 21 12:39:53.678364 ip-10-0-130-157 kubenswrapper[2576]: W0721 12:39:53.540559 2576 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 21 12:39:53.678364 ip-10-0-130-157 kubenswrapper[2576]: W0721 12:39:53.540561 2576 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 21 12:39:53.682488 ip-10-0-130-157 kubenswrapper[2576]: W0721 12:39:53.540564 2576 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 21 12:39:53.682488 ip-10-0-130-157 kubenswrapper[2576]: W0721 12:39:53.540566 2576 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 21 12:39:53.682488 ip-10-0-130-157 kubenswrapper[2576]: W0721 12:39:53.540569 2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 21 12:39:53.682488 ip-10-0-130-157 kubenswrapper[2576]: W0721 12:39:53.540571 2576 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 21 12:39:53.682488 ip-10-0-130-157 kubenswrapper[2576]: W0721 12:39:53.540574 2576 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 21 12:39:53.682488 ip-10-0-130-157 kubenswrapper[2576]: W0721 12:39:53.540576 2576 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 21 12:39:53.682488 ip-10-0-130-157 kubenswrapper[2576]: W0721 12:39:53.540580 2576 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 21 12:39:53.682488 ip-10-0-130-157 kubenswrapper[2576]: W0721 12:39:53.540582 2576 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 21 12:39:53.682488 ip-10-0-130-157 kubenswrapper[2576]: W0721 12:39:53.540587 2576 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 21 12:39:53.682488 ip-10-0-130-157 kubenswrapper[2576]: W0721 12:39:53.540590 2576 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 21 12:39:53.682488 ip-10-0-130-157 kubenswrapper[2576]: W0721 12:39:53.540592 2576 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 21 12:39:53.682488 ip-10-0-130-157 kubenswrapper[2576]: W0721 12:39:53.540596 2576 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 21 12:39:53.682488 ip-10-0-130-157 kubenswrapper[2576]: W0721 12:39:53.540599 2576 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 21 12:39:53.682488 ip-10-0-130-157 kubenswrapper[2576]: W0721 12:39:53.540601 2576 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 21 12:39:53.682488 ip-10-0-130-157 kubenswrapper[2576]: W0721 12:39:53.540604 2576 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 21 12:39:53.682488 ip-10-0-130-157 kubenswrapper[2576]: W0721 12:39:53.540606 2576 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 21 12:39:53.682488 ip-10-0-130-157 kubenswrapper[2576]: W0721 12:39:53.540609 2576 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 21 12:39:53.682488 ip-10-0-130-157 kubenswrapper[2576]: W0721 12:39:53.540611 2576 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 21 12:39:53.682488 ip-10-0-130-157 kubenswrapper[2576]: W0721 12:39:53.540615 2576 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 21 12:39:53.682488 ip-10-0-130-157 kubenswrapper[2576]: W0721 12:39:53.540618 2576 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 21 12:39:53.683309 ip-10-0-130-157 kubenswrapper[2576]: W0721 12:39:53.540621 2576 feature_gate.go:328] unrecognized feature gate: Example Jul 21 12:39:53.683309 ip-10-0-130-157 kubenswrapper[2576]: W0721 12:39:53.540624 2576 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 21 12:39:53.683309 ip-10-0-130-157 kubenswrapper[2576]: W0721 12:39:53.540627 2576 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 21 12:39:53.683309 ip-10-0-130-157 kubenswrapper[2576]: W0721 12:39:53.540629 2576 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 21 12:39:53.683309 ip-10-0-130-157 kubenswrapper[2576]: W0721 12:39:53.540632 2576 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 21 12:39:53.683309 ip-10-0-130-157 kubenswrapper[2576]: W0721 12:39:53.540634 2576 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 21 12:39:53.683309 ip-10-0-130-157 kubenswrapper[2576]: W0721 12:39:53.540637 2576 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 21 12:39:53.683309 ip-10-0-130-157 kubenswrapper[2576]: W0721 12:39:53.540640 2576 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 21 12:39:53.683309 ip-10-0-130-157 kubenswrapper[2576]: W0721 12:39:53.540643 2576 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 21 12:39:53.683309 ip-10-0-130-157 kubenswrapper[2576]: W0721 12:39:53.540645 2576 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 21 12:39:53.683309 ip-10-0-130-157 kubenswrapper[2576]: W0721 12:39:53.540648 2576 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 21 12:39:53.683309 ip-10-0-130-157 kubenswrapper[2576]: W0721 12:39:53.540650 2576 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 21 12:39:53.683309 ip-10-0-130-157 kubenswrapper[2576]: W0721 12:39:53.540653 2576 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 21 12:39:53.683309 ip-10-0-130-157 kubenswrapper[2576]: W0721 12:39:53.540655 2576 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 21 12:39:53.683309 ip-10-0-130-157 kubenswrapper[2576]: W0721 12:39:53.540658 2576 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 21 12:39:53.683309 ip-10-0-130-157 kubenswrapper[2576]: W0721 12:39:53.540660 2576 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 21 12:39:53.683309 ip-10-0-130-157 kubenswrapper[2576]: W0721 12:39:53.540663 2576 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 21 12:39:53.683309 ip-10-0-130-157 kubenswrapper[2576]: W0721 12:39:53.540665 2576 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 21 12:39:53.683309 ip-10-0-130-157 kubenswrapper[2576]: W0721 12:39:53.540668 2576 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 21 12:39:53.683309 ip-10-0-130-157 kubenswrapper[2576]: W0721 12:39:53.540670 2576 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 21 12:39:53.687032 ip-10-0-130-157 kubenswrapper[2576]: W0721 12:39:53.540674 2576 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 21 12:39:53.687032 ip-10-0-130-157 kubenswrapper[2576]: W0721 12:39:53.540677 2576 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 21 12:39:53.687032 ip-10-0-130-157 kubenswrapper[2576]: W0721 12:39:53.540682 2576 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 21 12:39:53.687032 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:39:53.540761 2576 flags.go:64] FLAG: --address="0.0.0.0" Jul 21 12:39:53.687032 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:39:53.540769 2576 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]" Jul 21 12:39:53.687032 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:39:53.540778 2576 flags.go:64] FLAG: --anonymous-auth="true" Jul 21 12:39:53.687032 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:39:53.540785 2576 flags.go:64] FLAG: --application-metrics-count-limit="100" Jul 21 12:39:53.687032 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:39:53.540790 2576 flags.go:64] FLAG: --authentication-token-webhook="false" Jul 21 12:39:53.687032 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:39:53.540794 2576 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s" Jul 21 12:39:53.687032 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:39:53.540798 2576 flags.go:64] FLAG: --authorization-mode="AlwaysAllow" Jul 21 12:39:53.687032 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:39:53.540807 2576 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s" Jul 21 12:39:53.687032 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:39:53.540810 2576 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s" Jul 21 12:39:53.687032 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:39:53.540813 2576 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id" Jul 21 12:39:53.687032 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:39:53.540817 2576 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig" Jul 21 12:39:53.687032 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:39:53.540820 2576 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki" Jul 21 12:39:53.687032 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:39:53.540823 2576 flags.go:64] FLAG: --cgroup-driver="cgroupfs" Jul 21 12:39:53.687032 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:39:53.540826 2576 flags.go:64] FLAG: --cgroup-root="" Jul 21 12:39:53.687032 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:39:53.540829 2576 flags.go:64] FLAG: --cgroups-per-qos="true" Jul 21 12:39:53.687032 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:39:53.540832 2576 flags.go:64] FLAG: --client-ca-file="" Jul 21 12:39:53.687032 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:39:53.540836 2576 flags.go:64] FLAG: --cloud-provider="external" Jul 21 12:39:53.687032 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:39:53.540839 2576 flags.go:64] FLAG: --cluster-dns="[]" Jul 21 12:39:53.687032 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:39:53.540843 2576 flags.go:64] FLAG: --cluster-domain="" Jul 21 12:39:53.687032 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:39:53.540846 2576 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf" Jul 21 12:39:53.687032 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:39:53.540849 2576 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d" Jul 21 12:39:53.687860 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:39:53.540852 2576 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json" Jul 21 12:39:53.687860 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:39:53.540856 2576 flags.go:64] FLAG: --container-log-max-files="5" Jul 21 12:39:53.687860 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:39:53.540860 2576 flags.go:64] FLAG: --container-log-max-size="10Mi" Jul 21 12:39:53.687860 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:39:53.540863 2576 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock" Jul 21 12:39:53.687860 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:39:53.540866 2576 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock" Jul 21 12:39:53.687860 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:39:53.540869 2576 flags.go:64] FLAG: --containerd-namespace="k8s.io" Jul 21 12:39:53.687860 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:39:53.540872 2576 flags.go:64] FLAG: --contention-profiling="false" Jul 21 12:39:53.687860 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:39:53.540875 2576 flags.go:64] FLAG: --cpu-cfs-quota="true" Jul 21 12:39:53.687860 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:39:53.540880 2576 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms" Jul 21 12:39:53.687860 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:39:53.540883 2576 flags.go:64] FLAG: --cpu-manager-policy="none" Jul 21 12:39:53.687860 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:39:53.540887 2576 flags.go:64] FLAG: --cpu-manager-policy-options="" Jul 21 12:39:53.687860 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:39:53.540891 2576 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s" Jul 21 12:39:53.687860 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:39:53.540895 2576 flags.go:64] FLAG: --enable-controller-attach-detach="true" Jul 21 12:39:53.687860 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:39:53.540898 2576 flags.go:64] FLAG: --enable-debugging-handlers="true" Jul 21 12:39:53.687860 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:39:53.540901 2576 flags.go:64] FLAG: --enable-load-reader="false" Jul 21 12:39:53.687860 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:39:53.540904 2576 flags.go:64] FLAG: --enable-server="true" Jul 21 12:39:53.687860 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:39:53.540907 2576 flags.go:64] FLAG: --enforce-node-allocatable="[pods]" Jul 21 12:39:53.687860 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:39:53.540912 2576 flags.go:64] FLAG: --event-burst="100" Jul 21 12:39:53.687860 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:39:53.540915 2576 flags.go:64] FLAG: --event-qps="50" Jul 21 12:39:53.687860 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:39:53.540918 2576 flags.go:64] FLAG: --event-storage-age-limit="default=0" Jul 21 12:39:53.687860 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:39:53.540921 2576 flags.go:64] FLAG: --event-storage-event-limit="default=0" Jul 21 12:39:53.687860 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:39:53.540924 2576 flags.go:64] FLAG: --eviction-hard="" Jul 21 12:39:53.687860 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:39:53.540928 2576 flags.go:64] FLAG: --eviction-max-pod-grace-period="0" Jul 21 12:39:53.687860 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:39:53.540931 2576 flags.go:64] FLAG: --eviction-minimum-reclaim="" Jul 21 12:39:53.788901 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:39:53.540934 2576 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s" Jul 21 12:39:53.788901 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:39:53.540937 2576 flags.go:64] FLAG: --eviction-soft="" Jul 21 12:39:53.788901 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:39:53.540940 2576 flags.go:64] FLAG: --eviction-soft-grace-period="" Jul 21 12:39:53.788901 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:39:53.540943 2576 flags.go:64] FLAG: --exit-on-lock-contention="false" Jul 21 12:39:53.788901 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:39:53.540946 2576 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false" Jul 21 12:39:53.788901 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:39:53.540949 2576 flags.go:64] FLAG: --experimental-mounter-path="" Jul 21 12:39:53.788901 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:39:53.540952 2576 flags.go:64] FLAG: --fail-cgroupv1="false" Jul 21 12:39:53.788901 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:39:53.540955 2576 flags.go:64] FLAG: --fail-swap-on="true" Jul 21 12:39:53.788901 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:39:53.540958 2576 flags.go:64] FLAG: --feature-gates="" Jul 21 12:39:53.788901 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:39:53.540962 2576 flags.go:64] FLAG: --file-check-frequency="20s" Jul 21 12:39:53.788901 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:39:53.540965 2576 flags.go:64] FLAG: --global-housekeeping-interval="1m0s" Jul 21 12:39:53.788901 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:39:53.540968 2576 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge" Jul 21 12:39:53.788901 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:39:53.540972 2576 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1" Jul 21 12:39:53.788901 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:39:53.540975 2576 flags.go:64] FLAG: --healthz-port="10248" Jul 21 12:39:53.788901 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:39:53.540978 2576 flags.go:64] FLAG: --help="false" Jul 21 12:39:53.788901 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:39:53.540980 2576 flags.go:64] FLAG: --hostname-override="ip-10-0-130-157.ec2.internal" Jul 21 12:39:53.788901 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:39:53.540985 2576 flags.go:64] FLAG: --housekeeping-interval="10s" Jul 21 12:39:53.788901 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:39:53.540988 2576 flags.go:64] FLAG: --http-check-frequency="20s" Jul 21 12:39:53.788901 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:39:53.540992 2576 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins" Jul 21 12:39:53.788901 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:39:53.540996 2576 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml" Jul 21 12:39:53.788901 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:39:53.540999 2576 flags.go:64] FLAG: --image-gc-high-threshold="85" Jul 21 12:39:53.788901 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:39:53.541002 2576 flags.go:64] FLAG: --image-gc-low-threshold="80" Jul 21 12:39:53.788901 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:39:53.541005 2576 flags.go:64] FLAG: --image-service-endpoint="" Jul 21 12:39:53.789995 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:39:53.541008 2576 flags.go:64] FLAG: --kernel-memcg-notification="false" Jul 21 12:39:53.789995 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:39:53.541011 2576 flags.go:64] FLAG: --kube-api-burst="100" Jul 21 12:39:53.789995 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:39:53.541014 2576 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf" Jul 21 12:39:53.789995 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:39:53.541017 2576 flags.go:64] FLAG: --kube-api-qps="50" Jul 21 12:39:53.789995 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:39:53.541020 2576 flags.go:64] FLAG: --kube-reserved="" Jul 21 12:39:53.789995 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:39:53.541023 2576 flags.go:64] FLAG: --kube-reserved-cgroup="" Jul 21 12:39:53.789995 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:39:53.541026 2576 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig" Jul 21 12:39:53.789995 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:39:53.541029 2576 flags.go:64] FLAG: --kubelet-cgroups="" Jul 21 12:39:53.789995 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:39:53.541032 2576 flags.go:64] FLAG: --local-storage-capacity-isolation="true" Jul 21 12:39:53.789995 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:39:53.541034 2576 flags.go:64] FLAG: --lock-file="" Jul 21 12:39:53.789995 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:39:53.541037 2576 flags.go:64] FLAG: --log-cadvisor-usage="false" Jul 21 12:39:53.789995 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:39:53.541040 2576 flags.go:64] FLAG: --log-flush-frequency="5s" Jul 21 12:39:53.789995 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:39:53.541043 2576 flags.go:64] FLAG: --log-json-info-buffer-size="0" Jul 21 12:39:53.789995 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:39:53.541049 2576 flags.go:64] FLAG: --log-json-split-stream="false" Jul 21 12:39:53.789995 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:39:53.541052 2576 flags.go:64] FLAG: --log-text-info-buffer-size="0" Jul 21 12:39:53.789995 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:39:53.541055 2576 flags.go:64] FLAG: --log-text-split-stream="false" Jul 21 12:39:53.789995 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:39:53.541058 2576 flags.go:64] FLAG: --logging-format="text" Jul 21 12:39:53.789995 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:39:53.541061 2576 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id" Jul 21 12:39:53.789995 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:39:53.541064 2576 flags.go:64] FLAG: --make-iptables-util-chains="true" Jul 21 12:39:53.789995 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:39:53.541067 2576 flags.go:64] FLAG: --manifest-url="" Jul 21 12:39:53.789995 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:39:53.541069 2576 flags.go:64] FLAG: --manifest-url-header="" Jul 21 12:39:53.789995 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:39:53.541074 2576 flags.go:64] FLAG: --max-housekeeping-interval="15s" Jul 21 12:39:53.789995 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:39:53.541077 2576 flags.go:64] FLAG: --max-open-files="1000000" Jul 21 12:39:53.789995 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:39:53.541081 2576 flags.go:64] FLAG: --max-pods="110" Jul 21 12:39:53.789995 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:39:53.541084 2576 flags.go:64] FLAG: --maximum-dead-containers="-1" Jul 21 12:39:53.794010 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:39:53.541088 2576 flags.go:64] FLAG: --maximum-dead-containers-per-container="1" Jul 21 12:39:53.794010 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:39:53.541091 2576 flags.go:64] FLAG: --memory-manager-policy="None" Jul 21 12:39:53.794010 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:39:53.541096 2576 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s" Jul 21 12:39:53.794010 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:39:53.541099 2576 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s" Jul 21 12:39:53.794010 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:39:53.541102 2576 flags.go:64] FLAG: --node-ip="0.0.0.0" Jul 21 12:39:53.794010 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:39:53.541105 2576 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel" Jul 21 12:39:53.794010 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:39:53.541113 2576 flags.go:64] FLAG: --node-status-max-images="50" Jul 21 12:39:53.794010 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:39:53.541116 2576 flags.go:64] FLAG: --node-status-update-frequency="10s" Jul 21 12:39:53.794010 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:39:53.541119 2576 flags.go:64] FLAG: --oom-score-adj="-999" Jul 21 12:39:53.794010 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:39:53.541122 2576 flags.go:64] FLAG: --pod-cidr="" Jul 21 12:39:53.794010 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:39:53.541125 2576 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:4017e99d76d7b91598d3a3430cd0eba6eda8b33653addd3838fdfe3cca69c5a0" Jul 21 12:39:53.794010 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:39:53.541130 2576 flags.go:64] FLAG: --pod-manifest-path="" Jul 21 12:39:53.794010 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:39:53.541133 2576 flags.go:64] FLAG: --pod-max-pids="-1" Jul 21 12:39:53.794010 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:39:53.541136 2576 flags.go:64] FLAG: --pods-per-core="0" Jul 21 12:39:53.794010 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:39:53.541138 2576 flags.go:64] FLAG: --port="10250" Jul 21 12:39:53.794010 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:39:53.541141 2576 flags.go:64] FLAG: --protect-kernel-defaults="false" Jul 21 12:39:53.794010 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:39:53.541144 2576 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-0a025ae2b16d3f659" Jul 21 12:39:53.794010 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:39:53.541147 2576 flags.go:64] FLAG: --qos-reserved="" Jul 21 12:39:53.794010 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:39:53.541150 2576 flags.go:64] FLAG: --read-only-port="10255" Jul 21 12:39:53.794010 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:39:53.541153 2576 flags.go:64] FLAG: --register-node="true" Jul 21 12:39:53.794010 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:39:53.541156 2576 flags.go:64] FLAG: --register-with-taints="" Jul 21 12:39:53.794010 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:39:53.541160 2576 flags.go:64] FLAG: --registry-burst="10" Jul 21 12:39:53.794010 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:39:53.541164 2576 flags.go:64] FLAG: --registry-qps="5" Jul 21 12:39:53.794010 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:39:53.541167 2576 flags.go:64] FLAG: --reserved-cpus="" Jul 21 12:39:53.794832 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:39:53.541169 2576 flags.go:64] FLAG: --reserved-memory="" Jul 21 12:39:53.794832 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:39:53.541173 2576 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf" Jul 21 12:39:53.794832 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:39:53.541176 2576 flags.go:64] FLAG: --root-dir="/var/lib/kubelet" Jul 21 12:39:53.794832 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:39:53.541179 2576 flags.go:64] FLAG: --rotate-certificates="false" Jul 21 12:39:53.794832 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:39:53.541181 2576 flags.go:64] FLAG: --rotate-server-certificates="false" Jul 21 12:39:53.794832 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:39:53.541184 2576 flags.go:64] FLAG: --runonce="false" Jul 21 12:39:53.794832 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:39:53.541187 2576 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service" Jul 21 12:39:53.794832 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:39:53.541190 2576 flags.go:64] FLAG: --runtime-request-timeout="2m0s" Jul 21 12:39:53.794832 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:39:53.541193 2576 flags.go:64] FLAG: --seccomp-default="false" Jul 21 12:39:53.794832 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:39:53.541197 2576 flags.go:64] FLAG: --serialize-image-pulls="true" Jul 21 12:39:53.794832 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:39:53.541201 2576 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s" Jul 21 12:39:53.794832 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:39:53.541204 2576 flags.go:64] FLAG: --storage-driver-db="cadvisor" Jul 21 12:39:53.794832 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:39:53.541207 2576 flags.go:64] FLAG: --storage-driver-host="localhost:8086" Jul 21 12:39:53.794832 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:39:53.541210 2576 flags.go:64] FLAG: --storage-driver-password="root" Jul 21 12:39:53.794832 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:39:53.541213 2576 flags.go:64] FLAG: --storage-driver-secure="false" Jul 21 12:39:53.794832 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:39:53.541216 2576 flags.go:64] FLAG: --storage-driver-table="stats" Jul 21 12:39:53.794832 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:39:53.541219 2576 flags.go:64] FLAG: --storage-driver-user="root" Jul 21 12:39:53.794832 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:39:53.541221 2576 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s" Jul 21 12:39:53.794832 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:39:53.541224 2576 flags.go:64] FLAG: --sync-frequency="1m0s" Jul 21 12:39:53.794832 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:39:53.541227 2576 flags.go:64] FLAG: --system-cgroups="" Jul 21 12:39:53.794832 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:39:53.541230 2576 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi" Jul 21 12:39:53.794832 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:39:53.541236 2576 flags.go:64] FLAG: --system-reserved-cgroup="" Jul 21 12:39:53.794832 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:39:53.541238 2576 flags.go:64] FLAG: --tls-cert-file="" Jul 21 12:39:53.794832 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:39:53.541241 2576 flags.go:64] FLAG: --tls-cipher-suites="[]" Jul 21 12:39:53.794832 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:39:53.541245 2576 flags.go:64] FLAG: --tls-min-version="" Jul 21 12:39:53.799346 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:39:53.541248 2576 flags.go:64] FLAG: --tls-private-key-file="" Jul 21 12:39:53.799346 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:39:53.541251 2576 flags.go:64] FLAG: --topology-manager-policy="none" Jul 21 12:39:53.799346 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:39:53.541254 2576 flags.go:64] FLAG: --topology-manager-policy-options="" Jul 21 12:39:53.799346 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:39:53.541257 2576 flags.go:64] FLAG: --topology-manager-scope="container" Jul 21 12:39:53.799346 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:39:53.541260 2576 flags.go:64] FLAG: --v="2" Jul 21 12:39:53.799346 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:39:53.541264 2576 flags.go:64] FLAG: --version="false" Jul 21 12:39:53.799346 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:39:53.541269 2576 flags.go:64] FLAG: --vmodule="" Jul 21 12:39:53.799346 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:39:53.541272 2576 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec" Jul 21 12:39:53.799346 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:39:53.541276 2576 flags.go:64] FLAG: --volume-stats-agg-period="1m0s" Jul 21 12:39:53.799346 ip-10-0-130-157 kubenswrapper[2576]: W0721 12:39:53.541393 2576 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 21 12:39:53.799346 ip-10-0-130-157 kubenswrapper[2576]: W0721 12:39:53.541397 2576 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 21 12:39:53.799346 ip-10-0-130-157 kubenswrapper[2576]: W0721 12:39:53.541400 2576 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 21 12:39:53.799346 ip-10-0-130-157 kubenswrapper[2576]: W0721 12:39:53.541403 2576 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 21 12:39:53.799346 ip-10-0-130-157 kubenswrapper[2576]: W0721 12:39:53.541406 2576 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 21 12:39:53.799346 ip-10-0-130-157 kubenswrapper[2576]: W0721 12:39:53.541408 2576 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 21 12:39:53.799346 ip-10-0-130-157 kubenswrapper[2576]: W0721 12:39:53.541411 2576 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 21 12:39:53.799346 ip-10-0-130-157 kubenswrapper[2576]: W0721 12:39:53.541417 2576 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 21 12:39:53.799346 ip-10-0-130-157 kubenswrapper[2576]: W0721 12:39:53.541421 2576 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 21 12:39:53.799346 ip-10-0-130-157 kubenswrapper[2576]: W0721 12:39:53.541424 2576 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 21 12:39:53.799346 ip-10-0-130-157 kubenswrapper[2576]: W0721 12:39:53.541426 2576 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 21 12:39:53.799346 ip-10-0-130-157 kubenswrapper[2576]: W0721 12:39:53.541429 2576 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 21 12:39:53.800156 ip-10-0-130-157 kubenswrapper[2576]: W0721 12:39:53.541432 2576 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 21 12:39:53.800156 ip-10-0-130-157 kubenswrapper[2576]: W0721 12:39:53.541435 2576 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 21 12:39:53.800156 ip-10-0-130-157 kubenswrapper[2576]: W0721 12:39:53.541438 2576 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 21 12:39:53.800156 ip-10-0-130-157 kubenswrapper[2576]: W0721 12:39:53.541440 2576 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 21 12:39:53.800156 ip-10-0-130-157 kubenswrapper[2576]: W0721 12:39:53.541443 2576 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 21 12:39:53.800156 ip-10-0-130-157 kubenswrapper[2576]: W0721 12:39:53.541446 2576 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 21 12:39:53.800156 ip-10-0-130-157 kubenswrapper[2576]: W0721 12:39:53.541448 2576 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 21 12:39:53.800156 ip-10-0-130-157 kubenswrapper[2576]: W0721 12:39:53.541451 2576 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 21 12:39:53.800156 ip-10-0-130-157 kubenswrapper[2576]: W0721 12:39:53.541454 2576 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 21 12:39:53.800156 ip-10-0-130-157 kubenswrapper[2576]: W0721 12:39:53.541457 2576 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 21 12:39:53.800156 ip-10-0-130-157 kubenswrapper[2576]: W0721 12:39:53.541459 2576 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 21 12:39:53.800156 ip-10-0-130-157 kubenswrapper[2576]: W0721 12:39:53.541462 2576 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 21 12:39:53.800156 ip-10-0-130-157 kubenswrapper[2576]: W0721 12:39:53.541464 2576 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 21 12:39:53.800156 ip-10-0-130-157 kubenswrapper[2576]: W0721 12:39:53.541467 2576 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 21 12:39:53.800156 ip-10-0-130-157 kubenswrapper[2576]: W0721 12:39:53.541470 2576 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 21 12:39:53.800156 ip-10-0-130-157 kubenswrapper[2576]: W0721 12:39:53.541472 2576 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 21 12:39:53.800156 ip-10-0-130-157 kubenswrapper[2576]: W0721 12:39:53.541475 2576 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 21 12:39:53.800156 ip-10-0-130-157 kubenswrapper[2576]: W0721 12:39:53.541477 2576 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 21 12:39:53.800156 ip-10-0-130-157 kubenswrapper[2576]: W0721 12:39:53.541480 2576 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 21 12:39:53.800156 ip-10-0-130-157 kubenswrapper[2576]: W0721 12:39:53.541484 2576 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 21 12:39:53.803424 ip-10-0-130-157 kubenswrapper[2576]: W0721 12:39:53.541488 2576 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 21 12:39:53.803424 ip-10-0-130-157 kubenswrapper[2576]: W0721 12:39:53.541492 2576 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 21 12:39:53.803424 ip-10-0-130-157 kubenswrapper[2576]: W0721 12:39:53.541495 2576 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 21 12:39:53.803424 ip-10-0-130-157 kubenswrapper[2576]: W0721 12:39:53.541499 2576 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 21 12:39:53.803424 ip-10-0-130-157 kubenswrapper[2576]: W0721 12:39:53.541502 2576 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 21 12:39:53.803424 ip-10-0-130-157 kubenswrapper[2576]: W0721 12:39:53.541505 2576 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 21 12:39:53.803424 ip-10-0-130-157 kubenswrapper[2576]: W0721 12:39:53.541508 2576 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 21 12:39:53.803424 ip-10-0-130-157 kubenswrapper[2576]: W0721 12:39:53.541527 2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 21 12:39:53.803424 ip-10-0-130-157 kubenswrapper[2576]: W0721 12:39:53.541532 2576 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 21 12:39:53.803424 ip-10-0-130-157 kubenswrapper[2576]: W0721 12:39:53.541535 2576 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 21 12:39:53.803424 ip-10-0-130-157 kubenswrapper[2576]: W0721 12:39:53.541539 2576 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 21 12:39:53.803424 ip-10-0-130-157 kubenswrapper[2576]: W0721 12:39:53.541542 2576 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 21 12:39:53.803424 ip-10-0-130-157 kubenswrapper[2576]: W0721 12:39:53.541545 2576 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 21 12:39:53.803424 ip-10-0-130-157 kubenswrapper[2576]: W0721 12:39:53.541548 2576 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 21 12:39:53.803424 ip-10-0-130-157 kubenswrapper[2576]: W0721 12:39:53.541551 2576 feature_gate.go:328] unrecognized feature gate: Example Jul 21 12:39:53.803424 ip-10-0-130-157 kubenswrapper[2576]: W0721 12:39:53.541554 2576 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 21 12:39:53.803424 ip-10-0-130-157 kubenswrapper[2576]: W0721 12:39:53.541557 2576 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 21 12:39:53.803424 ip-10-0-130-157 kubenswrapper[2576]: W0721 12:39:53.541560 2576 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 21 12:39:53.803424 ip-10-0-130-157 kubenswrapper[2576]: W0721 12:39:53.541562 2576 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 21 12:39:53.804234 ip-10-0-130-157 kubenswrapper[2576]: W0721 12:39:53.541565 2576 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 21 12:39:53.804234 ip-10-0-130-157 kubenswrapper[2576]: W0721 12:39:53.541567 2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 21 12:39:53.804234 ip-10-0-130-157 kubenswrapper[2576]: W0721 12:39:53.541570 2576 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 21 12:39:53.804234 ip-10-0-130-157 kubenswrapper[2576]: W0721 12:39:53.541572 2576 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 21 12:39:53.804234 ip-10-0-130-157 kubenswrapper[2576]: W0721 12:39:53.541575 2576 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 21 12:39:53.804234 ip-10-0-130-157 kubenswrapper[2576]: W0721 12:39:53.541577 2576 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 21 12:39:53.804234 ip-10-0-130-157 kubenswrapper[2576]: W0721 12:39:53.541580 2576 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 21 12:39:53.804234 ip-10-0-130-157 kubenswrapper[2576]: W0721 12:39:53.541582 2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 21 12:39:53.804234 ip-10-0-130-157 kubenswrapper[2576]: W0721 12:39:53.541585 2576 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 21 12:39:53.804234 ip-10-0-130-157 kubenswrapper[2576]: W0721 12:39:53.541588 2576 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 21 12:39:53.804234 ip-10-0-130-157 kubenswrapper[2576]: W0721 12:39:53.541590 2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 21 12:39:53.804234 ip-10-0-130-157 kubenswrapper[2576]: W0721 12:39:53.541593 2576 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 21 12:39:53.804234 ip-10-0-130-157 kubenswrapper[2576]: W0721 12:39:53.541596 2576 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 21 12:39:53.804234 ip-10-0-130-157 kubenswrapper[2576]: W0721 12:39:53.541599 2576 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 21 12:39:53.804234 ip-10-0-130-157 kubenswrapper[2576]: W0721 12:39:53.541602 2576 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 21 12:39:53.804234 ip-10-0-130-157 kubenswrapper[2576]: W0721 12:39:53.541604 2576 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 21 12:39:53.804234 ip-10-0-130-157 kubenswrapper[2576]: W0721 12:39:53.541607 2576 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 21 12:39:53.804234 ip-10-0-130-157 kubenswrapper[2576]: W0721 12:39:53.541610 2576 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 21 12:39:53.804234 ip-10-0-130-157 kubenswrapper[2576]: W0721 12:39:53.541612 2576 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 21 12:39:53.804234 ip-10-0-130-157 kubenswrapper[2576]: W0721 12:39:53.541615 2576 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 21 12:39:53.807164 ip-10-0-130-157 kubenswrapper[2576]: W0721 12:39:53.541619 2576 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 21 12:39:53.807164 ip-10-0-130-157 kubenswrapper[2576]: W0721 12:39:53.541623 2576 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 21 12:39:53.807164 ip-10-0-130-157 kubenswrapper[2576]: W0721 12:39:53.541626 2576 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 21 12:39:53.807164 ip-10-0-130-157 kubenswrapper[2576]: W0721 12:39:53.541629 2576 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 21 12:39:53.807164 ip-10-0-130-157 kubenswrapper[2576]: W0721 12:39:53.541631 2576 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 21 12:39:53.807164 ip-10-0-130-157 kubenswrapper[2576]: W0721 12:39:53.541634 2576 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 21 12:39:53.807164 ip-10-0-130-157 kubenswrapper[2576]: W0721 12:39:53.541636 2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 21 12:39:53.807164 ip-10-0-130-157 kubenswrapper[2576]: W0721 12:39:53.541639 2576 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 21 12:39:53.807164 ip-10-0-130-157 kubenswrapper[2576]: W0721 12:39:53.541642 2576 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 21 12:39:53.807164 ip-10-0-130-157 kubenswrapper[2576]: W0721 12:39:53.541644 2576 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 21 12:39:53.807164 ip-10-0-130-157 kubenswrapper[2576]: W0721 12:39:53.541647 2576 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 21 12:39:53.807164 ip-10-0-130-157 kubenswrapper[2576]: W0721 12:39:53.541650 2576 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 21 12:39:53.807164 ip-10-0-130-157 kubenswrapper[2576]: W0721 12:39:53.541652 2576 feature_gate.go:328] unrecognized feature gate: Example2 Jul 21 12:39:53.807164 ip-10-0-130-157 kubenswrapper[2576]: W0721 12:39:53.541655 2576 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 21 12:39:53.807164 ip-10-0-130-157 kubenswrapper[2576]: W0721 12:39:53.541657 2576 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 21 12:39:53.807164 ip-10-0-130-157 kubenswrapper[2576]: W0721 12:39:53.541660 2576 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 21 12:39:53.807164 ip-10-0-130-157 kubenswrapper[2576]: W0721 12:39:53.541662 2576 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 21 12:39:53.807164 ip-10-0-130-157 kubenswrapper[2576]: W0721 12:39:53.541665 2576 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 21 12:39:53.807164 ip-10-0-130-157 kubenswrapper[2576]: W0721 12:39:53.541667 2576 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 21 12:39:53.807164 ip-10-0-130-157 kubenswrapper[2576]: W0721 12:39:53.541670 2576 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 21 12:39:53.807970 ip-10-0-130-157 kubenswrapper[2576]: W0721 12:39:53.541672 2576 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 21 12:39:53.807970 ip-10-0-130-157 kubenswrapper[2576]: W0721 12:39:53.541675 2576 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 21 12:39:53.807970 ip-10-0-130-157 kubenswrapper[2576]: W0721 12:39:53.541678 2576 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 21 12:39:53.807970 ip-10-0-130-157 kubenswrapper[2576]: W0721 12:39:53.541680 2576 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 21 12:39:53.807970 ip-10-0-130-157 kubenswrapper[2576]: W0721 12:39:53.541683 2576 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 21 12:39:53.807970 ip-10-0-130-157 kubenswrapper[2576]: W0721 12:39:53.541686 2576 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 21 12:39:53.807970 ip-10-0-130-157 kubenswrapper[2576]: W0721 12:39:53.541688 2576 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 21 12:39:53.807970 ip-10-0-130-157 kubenswrapper[2576]: W0721 12:39:53.541691 2576 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 21 12:39:53.807970 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:39:53.541697 2576 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 21 12:39:53.807970 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:39:53.548335 2576 server.go:529] "Kubelet version" kubeletVersion="v1.34.9" Jul 21 12:39:53.807970 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:39:53.548355 2576 server.go:531] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK="" Jul 21 12:39:53.807970 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:39:53.548375 2576 watchdog_linux.go:95] "Systemd watchdog is not enabled" Jul 21 12:39:53.807970 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:39:53.548384 2576 watchdog_linux.go:137] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started." Jul 21 12:39:53.807970 ip-10-0-130-157 kubenswrapper[2576]: W0721 12:39:53.548442 2576 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 21 12:39:53.807970 ip-10-0-130-157 kubenswrapper[2576]: W0721 12:39:53.548448 2576 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 21 12:39:53.812043 ip-10-0-130-157 kubenswrapper[2576]: W0721 12:39:53.548451 2576 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 21 12:39:53.812043 ip-10-0-130-157 kubenswrapper[2576]: W0721 12:39:53.548454 2576 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 21 12:39:53.812043 ip-10-0-130-157 kubenswrapper[2576]: W0721 12:39:53.548457 2576 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 21 12:39:53.812043 ip-10-0-130-157 kubenswrapper[2576]: W0721 12:39:53.548460 2576 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 21 12:39:53.812043 ip-10-0-130-157 kubenswrapper[2576]: W0721 12:39:53.548463 2576 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 21 12:39:53.812043 ip-10-0-130-157 kubenswrapper[2576]: W0721 12:39:53.548466 2576 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 21 12:39:53.812043 ip-10-0-130-157 kubenswrapper[2576]: W0721 12:39:53.548470 2576 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 21 12:39:53.812043 ip-10-0-130-157 kubenswrapper[2576]: W0721 12:39:53.548475 2576 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 21 12:39:53.812043 ip-10-0-130-157 kubenswrapper[2576]: W0721 12:39:53.548478 2576 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 21 12:39:53.812043 ip-10-0-130-157 kubenswrapper[2576]: W0721 12:39:53.548481 2576 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 21 12:39:53.812043 ip-10-0-130-157 kubenswrapper[2576]: W0721 12:39:53.548484 2576 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 21 12:39:53.812043 ip-10-0-130-157 kubenswrapper[2576]: W0721 12:39:53.548487 2576 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 21 12:39:53.812043 ip-10-0-130-157 kubenswrapper[2576]: W0721 12:39:53.548490 2576 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 21 12:39:53.812043 ip-10-0-130-157 kubenswrapper[2576]: W0721 12:39:53.548493 2576 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 21 12:39:53.812043 ip-10-0-130-157 kubenswrapper[2576]: W0721 12:39:53.548496 2576 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 21 12:39:53.812043 ip-10-0-130-157 kubenswrapper[2576]: W0721 12:39:53.548499 2576 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 21 12:39:53.812043 ip-10-0-130-157 kubenswrapper[2576]: W0721 12:39:53.548501 2576 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 21 12:39:53.812043 ip-10-0-130-157 kubenswrapper[2576]: W0721 12:39:53.548504 2576 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 21 12:39:53.812043 ip-10-0-130-157 kubenswrapper[2576]: W0721 12:39:53.548507 2576 feature_gate.go:328] unrecognized feature gate: Example2 Jul 21 12:39:53.812043 ip-10-0-130-157 kubenswrapper[2576]: W0721 12:39:53.548509 2576 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 21 12:39:53.812860 ip-10-0-130-157 kubenswrapper[2576]: W0721 12:39:53.548528 2576 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 21 12:39:53.812860 ip-10-0-130-157 kubenswrapper[2576]: W0721 12:39:53.548532 2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 21 12:39:53.812860 ip-10-0-130-157 kubenswrapper[2576]: W0721 12:39:53.548535 2576 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 21 12:39:53.812860 ip-10-0-130-157 kubenswrapper[2576]: W0721 12:39:53.548538 2576 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 21 12:39:53.812860 ip-10-0-130-157 kubenswrapper[2576]: W0721 12:39:53.548540 2576 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 21 12:39:53.812860 ip-10-0-130-157 kubenswrapper[2576]: W0721 12:39:53.548546 2576 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 21 12:39:53.812860 ip-10-0-130-157 kubenswrapper[2576]: W0721 12:39:53.548548 2576 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 21 12:39:53.812860 ip-10-0-130-157 kubenswrapper[2576]: W0721 12:39:53.548553 2576 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 21 12:39:53.812860 ip-10-0-130-157 kubenswrapper[2576]: W0721 12:39:53.548555 2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 21 12:39:53.812860 ip-10-0-130-157 kubenswrapper[2576]: W0721 12:39:53.548558 2576 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 21 12:39:53.812860 ip-10-0-130-157 kubenswrapper[2576]: W0721 12:39:53.548561 2576 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 21 12:39:53.812860 ip-10-0-130-157 kubenswrapper[2576]: W0721 12:39:53.548563 2576 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 21 12:39:53.812860 ip-10-0-130-157 kubenswrapper[2576]: W0721 12:39:53.548566 2576 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 21 12:39:53.812860 ip-10-0-130-157 kubenswrapper[2576]: W0721 12:39:53.548569 2576 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 21 12:39:53.812860 ip-10-0-130-157 kubenswrapper[2576]: W0721 12:39:53.548571 2576 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 21 12:39:53.812860 ip-10-0-130-157 kubenswrapper[2576]: W0721 12:39:53.548574 2576 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 21 12:39:53.812860 ip-10-0-130-157 kubenswrapper[2576]: W0721 12:39:53.548576 2576 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 21 12:39:53.812860 ip-10-0-130-157 kubenswrapper[2576]: W0721 12:39:53.548579 2576 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 21 12:39:53.812860 ip-10-0-130-157 kubenswrapper[2576]: W0721 12:39:53.548582 2576 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 21 12:39:53.812860 ip-10-0-130-157 kubenswrapper[2576]: W0721 12:39:53.548584 2576 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 21 12:39:53.817895 ip-10-0-130-157 kubenswrapper[2576]: W0721 12:39:53.548587 2576 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 21 12:39:53.817895 ip-10-0-130-157 kubenswrapper[2576]: W0721 12:39:53.548589 2576 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 21 12:39:53.817895 ip-10-0-130-157 kubenswrapper[2576]: W0721 12:39:53.548592 2576 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 21 12:39:53.817895 ip-10-0-130-157 kubenswrapper[2576]: W0721 12:39:53.548595 2576 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 21 12:39:53.817895 ip-10-0-130-157 kubenswrapper[2576]: W0721 12:39:53.548598 2576 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 21 12:39:53.817895 ip-10-0-130-157 kubenswrapper[2576]: W0721 12:39:53.548601 2576 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 21 12:39:53.817895 ip-10-0-130-157 kubenswrapper[2576]: W0721 12:39:53.548603 2576 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 21 12:39:53.817895 ip-10-0-130-157 kubenswrapper[2576]: W0721 12:39:53.548606 2576 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 21 12:39:53.817895 ip-10-0-130-157 kubenswrapper[2576]: W0721 12:39:53.548608 2576 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 21 12:39:53.817895 ip-10-0-130-157 kubenswrapper[2576]: W0721 12:39:53.548611 2576 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 21 12:39:53.817895 ip-10-0-130-157 kubenswrapper[2576]: W0721 12:39:53.548613 2576 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 21 12:39:53.817895 ip-10-0-130-157 kubenswrapper[2576]: W0721 12:39:53.548616 2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 21 12:39:53.817895 ip-10-0-130-157 kubenswrapper[2576]: W0721 12:39:53.548620 2576 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 21 12:39:53.817895 ip-10-0-130-157 kubenswrapper[2576]: W0721 12:39:53.548624 2576 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 21 12:39:53.817895 ip-10-0-130-157 kubenswrapper[2576]: W0721 12:39:53.548626 2576 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 21 12:39:53.817895 ip-10-0-130-157 kubenswrapper[2576]: W0721 12:39:53.548629 2576 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 21 12:39:53.817895 ip-10-0-130-157 kubenswrapper[2576]: W0721 12:39:53.548634 2576 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 21 12:39:53.817895 ip-10-0-130-157 kubenswrapper[2576]: W0721 12:39:53.548639 2576 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 21 12:39:53.817895 ip-10-0-130-157 kubenswrapper[2576]: W0721 12:39:53.548642 2576 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 21 12:39:53.818713 ip-10-0-130-157 kubenswrapper[2576]: W0721 12:39:53.548645 2576 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 21 12:39:53.818713 ip-10-0-130-157 kubenswrapper[2576]: W0721 12:39:53.548999 2576 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 21 12:39:53.818713 ip-10-0-130-157 kubenswrapper[2576]: W0721 12:39:53.549003 2576 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 21 12:39:53.818713 ip-10-0-130-157 kubenswrapper[2576]: W0721 12:39:53.549006 2576 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 21 12:39:53.818713 ip-10-0-130-157 kubenswrapper[2576]: W0721 12:39:53.549009 2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 21 12:39:53.818713 ip-10-0-130-157 kubenswrapper[2576]: W0721 12:39:53.549011 2576 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 21 12:39:53.818713 ip-10-0-130-157 kubenswrapper[2576]: W0721 12:39:53.549014 2576 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 21 12:39:53.818713 ip-10-0-130-157 kubenswrapper[2576]: W0721 12:39:53.549016 2576 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 21 12:39:53.818713 ip-10-0-130-157 kubenswrapper[2576]: W0721 12:39:53.549020 2576 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 21 12:39:53.818713 ip-10-0-130-157 kubenswrapper[2576]: W0721 12:39:53.549023 2576 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 21 12:39:53.818713 ip-10-0-130-157 kubenswrapper[2576]: W0721 12:39:53.549025 2576 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 21 12:39:53.818713 ip-10-0-130-157 kubenswrapper[2576]: W0721 12:39:53.549028 2576 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 21 12:39:53.818713 ip-10-0-130-157 kubenswrapper[2576]: W0721 12:39:53.549030 2576 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 21 12:39:53.818713 ip-10-0-130-157 kubenswrapper[2576]: W0721 12:39:53.549033 2576 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 21 12:39:53.818713 ip-10-0-130-157 kubenswrapper[2576]: W0721 12:39:53.549036 2576 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 21 12:39:53.818713 ip-10-0-130-157 kubenswrapper[2576]: W0721 12:39:53.549038 2576 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 21 12:39:53.818713 ip-10-0-130-157 kubenswrapper[2576]: W0721 12:39:53.549041 2576 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 21 12:39:53.818713 ip-10-0-130-157 kubenswrapper[2576]: W0721 12:39:53.549043 2576 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 21 12:39:53.818713 ip-10-0-130-157 kubenswrapper[2576]: W0721 12:39:53.549046 2576 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 21 12:39:53.818713 ip-10-0-130-157 kubenswrapper[2576]: W0721 12:39:53.549049 2576 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 21 12:39:53.819522 ip-10-0-130-157 kubenswrapper[2576]: W0721 12:39:53.549051 2576 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 21 12:39:53.819522 ip-10-0-130-157 kubenswrapper[2576]: W0721 12:39:53.549054 2576 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 21 12:39:53.819522 ip-10-0-130-157 kubenswrapper[2576]: W0721 12:39:53.549056 2576 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 21 12:39:53.819522 ip-10-0-130-157 kubenswrapper[2576]: W0721 12:39:53.549059 2576 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 21 12:39:53.819522 ip-10-0-130-157 kubenswrapper[2576]: W0721 12:39:53.549062 2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 21 12:39:53.819522 ip-10-0-130-157 kubenswrapper[2576]: W0721 12:39:53.549064 2576 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 21 12:39:53.819522 ip-10-0-130-157 kubenswrapper[2576]: W0721 12:39:53.549066 2576 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 21 12:39:53.819522 ip-10-0-130-157 kubenswrapper[2576]: W0721 12:39:53.549069 2576 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 21 12:39:53.819522 ip-10-0-130-157 kubenswrapper[2576]: W0721 12:39:53.549071 2576 feature_gate.go:328] unrecognized feature gate: Example Jul 21 12:39:53.819522 ip-10-0-130-157 kubenswrapper[2576]: W0721 12:39:53.549075 2576 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 21 12:39:53.819522 ip-10-0-130-157 kubenswrapper[2576]: W0721 12:39:53.549079 2576 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 21 12:39:53.819522 ip-10-0-130-157 kubenswrapper[2576]: W0721 12:39:53.549082 2576 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 21 12:39:53.819522 ip-10-0-130-157 kubenswrapper[2576]: W0721 12:39:53.549084 2576 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 21 12:39:53.819522 ip-10-0-130-157 kubenswrapper[2576]: W0721 12:39:53.549087 2576 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 21 12:39:53.819522 ip-10-0-130-157 kubenswrapper[2576]: W0721 12:39:53.549090 2576 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 21 12:39:53.819522 ip-10-0-130-157 kubenswrapper[2576]: W0721 12:39:53.549093 2576 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 21 12:39:53.819522 ip-10-0-130-157 kubenswrapper[2576]: W0721 12:39:53.549095 2576 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 21 12:39:53.819522 ip-10-0-130-157 kubenswrapper[2576]: W0721 12:39:53.549098 2576 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 21 12:39:53.921622 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:39:53.549103 2576 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 21 12:39:53.921622 ip-10-0-130-157 kubenswrapper[2576]: W0721 12:39:53.549215 2576 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 21 12:39:53.921622 ip-10-0-130-157 kubenswrapper[2576]: W0721 12:39:53.549219 2576 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 21 12:39:53.921622 ip-10-0-130-157 kubenswrapper[2576]: W0721 12:39:53.549222 2576 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 21 12:39:53.921622 ip-10-0-130-157 kubenswrapper[2576]: W0721 12:39:53.549225 2576 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 21 12:39:53.921622 ip-10-0-130-157 kubenswrapper[2576]: W0721 12:39:53.549228 2576 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 21 12:39:53.921622 ip-10-0-130-157 kubenswrapper[2576]: W0721 12:39:53.549231 2576 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 21 12:39:53.921622 ip-10-0-130-157 kubenswrapper[2576]: W0721 12:39:53.549234 2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 21 12:39:53.921622 ip-10-0-130-157 kubenswrapper[2576]: W0721 12:39:53.549237 2576 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 21 12:39:53.921622 ip-10-0-130-157 kubenswrapper[2576]: W0721 12:39:53.549239 2576 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 21 12:39:53.921622 ip-10-0-130-157 kubenswrapper[2576]: W0721 12:39:53.549242 2576 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 21 12:39:53.921622 ip-10-0-130-157 kubenswrapper[2576]: W0721 12:39:53.549245 2576 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 21 12:39:53.921622 ip-10-0-130-157 kubenswrapper[2576]: W0721 12:39:53.549248 2576 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 21 12:39:53.921622 ip-10-0-130-157 kubenswrapper[2576]: W0721 12:39:53.549250 2576 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 21 12:39:53.921622 ip-10-0-130-157 kubenswrapper[2576]: W0721 12:39:53.549252 2576 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 21 12:39:53.922396 ip-10-0-130-157 kubenswrapper[2576]: W0721 12:39:53.549255 2576 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 21 12:39:53.922396 ip-10-0-130-157 kubenswrapper[2576]: W0721 12:39:53.549257 2576 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 21 12:39:53.922396 ip-10-0-130-157 kubenswrapper[2576]: W0721 12:39:53.549261 2576 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 21 12:39:53.922396 ip-10-0-130-157 kubenswrapper[2576]: W0721 12:39:53.549265 2576 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 21 12:39:53.922396 ip-10-0-130-157 kubenswrapper[2576]: W0721 12:39:53.549268 2576 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 21 12:39:53.922396 ip-10-0-130-157 kubenswrapper[2576]: W0721 12:39:53.549271 2576 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 21 12:39:53.922396 ip-10-0-130-157 kubenswrapper[2576]: W0721 12:39:53.549274 2576 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 21 12:39:53.922396 ip-10-0-130-157 kubenswrapper[2576]: W0721 12:39:53.549277 2576 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 21 12:39:53.922396 ip-10-0-130-157 kubenswrapper[2576]: W0721 12:39:53.549280 2576 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 21 12:39:53.922396 ip-10-0-130-157 kubenswrapper[2576]: W0721 12:39:53.549285 2576 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 21 12:39:53.922396 ip-10-0-130-157 kubenswrapper[2576]: W0721 12:39:53.549288 2576 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 21 12:39:53.922396 ip-10-0-130-157 kubenswrapper[2576]: W0721 12:39:53.549290 2576 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 21 12:39:53.922396 ip-10-0-130-157 kubenswrapper[2576]: W0721 12:39:53.549293 2576 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 21 12:39:53.922396 ip-10-0-130-157 kubenswrapper[2576]: W0721 12:39:53.549296 2576 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 21 12:39:53.922396 ip-10-0-130-157 kubenswrapper[2576]: W0721 12:39:53.549298 2576 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 21 12:39:53.922396 ip-10-0-130-157 kubenswrapper[2576]: W0721 12:39:53.549301 2576 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 21 12:39:53.922396 ip-10-0-130-157 kubenswrapper[2576]: W0721 12:39:53.549304 2576 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 21 12:39:53.922396 ip-10-0-130-157 kubenswrapper[2576]: W0721 12:39:53.549306 2576 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 21 12:39:53.922396 ip-10-0-130-157 kubenswrapper[2576]: W0721 12:39:53.549309 2576 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 21 12:39:53.923216 ip-10-0-130-157 kubenswrapper[2576]: W0721 12:39:53.549311 2576 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 21 12:39:53.923216 ip-10-0-130-157 kubenswrapper[2576]: W0721 12:39:53.549314 2576 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 21 12:39:53.923216 ip-10-0-130-157 kubenswrapper[2576]: W0721 12:39:53.549317 2576 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 21 12:39:53.923216 ip-10-0-130-157 kubenswrapper[2576]: W0721 12:39:53.549319 2576 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 21 12:39:53.923216 ip-10-0-130-157 kubenswrapper[2576]: W0721 12:39:53.549322 2576 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 21 12:39:53.923216 ip-10-0-130-157 kubenswrapper[2576]: W0721 12:39:53.549325 2576 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 21 12:39:53.923216 ip-10-0-130-157 kubenswrapper[2576]: W0721 12:39:53.549327 2576 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 21 12:39:53.923216 ip-10-0-130-157 kubenswrapper[2576]: W0721 12:39:53.549330 2576 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 21 12:39:53.923216 ip-10-0-130-157 kubenswrapper[2576]: W0721 12:39:53.549332 2576 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 21 12:39:53.923216 ip-10-0-130-157 kubenswrapper[2576]: W0721 12:39:53.549335 2576 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 21 12:39:53.923216 ip-10-0-130-157 kubenswrapper[2576]: W0721 12:39:53.549337 2576 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 21 12:39:53.923216 ip-10-0-130-157 kubenswrapper[2576]: W0721 12:39:53.549340 2576 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 21 12:39:53.923216 ip-10-0-130-157 kubenswrapper[2576]: W0721 12:39:53.549342 2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 21 12:39:53.923216 ip-10-0-130-157 kubenswrapper[2576]: W0721 12:39:53.549344 2576 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 21 12:39:53.923216 ip-10-0-130-157 kubenswrapper[2576]: W0721 12:39:53.549347 2576 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 21 12:39:53.923216 ip-10-0-130-157 kubenswrapper[2576]: W0721 12:39:53.549350 2576 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 21 12:39:53.923216 ip-10-0-130-157 kubenswrapper[2576]: W0721 12:39:53.549352 2576 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 21 12:39:53.923216 ip-10-0-130-157 kubenswrapper[2576]: W0721 12:39:53.549355 2576 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 21 12:39:53.923216 ip-10-0-130-157 kubenswrapper[2576]: W0721 12:39:53.549357 2576 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 21 12:39:53.923216 ip-10-0-130-157 kubenswrapper[2576]: W0721 12:39:53.549359 2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 21 12:39:53.925673 ip-10-0-130-157 kubenswrapper[2576]: W0721 12:39:53.549363 2576 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 21 12:39:53.925673 ip-10-0-130-157 kubenswrapper[2576]: W0721 12:39:53.549366 2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 21 12:39:53.925673 ip-10-0-130-157 kubenswrapper[2576]: W0721 12:39:53.549374 2576 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 21 12:39:53.925673 ip-10-0-130-157 kubenswrapper[2576]: W0721 12:39:53.549377 2576 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 21 12:39:53.925673 ip-10-0-130-157 kubenswrapper[2576]: W0721 12:39:53.549380 2576 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 21 12:39:53.925673 ip-10-0-130-157 kubenswrapper[2576]: W0721 12:39:53.549383 2576 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 21 12:39:53.925673 ip-10-0-130-157 kubenswrapper[2576]: W0721 12:39:53.549386 2576 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 21 12:39:53.925673 ip-10-0-130-157 kubenswrapper[2576]: W0721 12:39:53.549389 2576 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 21 12:39:53.925673 ip-10-0-130-157 kubenswrapper[2576]: W0721 12:39:53.549392 2576 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 21 12:39:53.925673 ip-10-0-130-157 kubenswrapper[2576]: W0721 12:39:53.549394 2576 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 21 12:39:53.925673 ip-10-0-130-157 kubenswrapper[2576]: W0721 12:39:53.549397 2576 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 21 12:39:53.925673 ip-10-0-130-157 kubenswrapper[2576]: W0721 12:39:53.549401 2576 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 21 12:39:53.925673 ip-10-0-130-157 kubenswrapper[2576]: W0721 12:39:53.549405 2576 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 21 12:39:53.925673 ip-10-0-130-157 kubenswrapper[2576]: W0721 12:39:53.549407 2576 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 21 12:39:53.925673 ip-10-0-130-157 kubenswrapper[2576]: W0721 12:39:53.549410 2576 feature_gate.go:328] unrecognized feature gate: Example Jul 21 12:39:53.925673 ip-10-0-130-157 kubenswrapper[2576]: W0721 12:39:53.549413 2576 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 21 12:39:53.925673 ip-10-0-130-157 kubenswrapper[2576]: W0721 12:39:53.549415 2576 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 21 12:39:53.925673 ip-10-0-130-157 kubenswrapper[2576]: W0721 12:39:53.549418 2576 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 21 12:39:53.925673 ip-10-0-130-157 kubenswrapper[2576]: W0721 12:39:53.549420 2576 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 21 12:39:53.926470 ip-10-0-130-157 kubenswrapper[2576]: W0721 12:39:53.549423 2576 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 21 12:39:53.926470 ip-10-0-130-157 kubenswrapper[2576]: W0721 12:39:53.549426 2576 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 21 12:39:53.926470 ip-10-0-130-157 kubenswrapper[2576]: W0721 12:39:53.549428 2576 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 21 12:39:53.926470 ip-10-0-130-157 kubenswrapper[2576]: W0721 12:39:53.549431 2576 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 21 12:39:53.926470 ip-10-0-130-157 kubenswrapper[2576]: W0721 12:39:53.549433 2576 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 21 12:39:53.926470 ip-10-0-130-157 kubenswrapper[2576]: W0721 12:39:53.549435 2576 feature_gate.go:328] unrecognized feature gate: Example2 Jul 21 12:39:53.926470 ip-10-0-130-157 kubenswrapper[2576]: W0721 12:39:53.549438 2576 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 21 12:39:53.926470 ip-10-0-130-157 kubenswrapper[2576]: W0721 12:39:53.549440 2576 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 21 12:39:53.926470 ip-10-0-130-157 kubenswrapper[2576]: W0721 12:39:53.549443 2576 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 21 12:39:53.926470 ip-10-0-130-157 kubenswrapper[2576]: W0721 12:39:53.549446 2576 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 21 12:39:53.926470 ip-10-0-130-157 kubenswrapper[2576]: W0721 12:39:53.549448 2576 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 21 12:39:53.926470 ip-10-0-130-157 kubenswrapper[2576]: W0721 12:39:53.549451 2576 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 21 12:39:53.926470 ip-10-0-130-157 kubenswrapper[2576]: W0721 12:39:53.549453 2576 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 21 12:39:53.926470 ip-10-0-130-157 kubenswrapper[2576]: W0721 12:39:53.549455 2576 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 21 12:39:53.926470 ip-10-0-130-157 kubenswrapper[2576]: W0721 12:39:53.549458 2576 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 21 12:39:53.926470 ip-10-0-130-157 kubenswrapper[2576]: W0721 12:39:53.549462 2576 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 21 12:39:53.926470 ip-10-0-130-157 kubenswrapper[2576]: W0721 12:39:53.549465 2576 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 21 12:39:53.926470 ip-10-0-130-157 kubenswrapper[2576]: W0721 12:39:53.549468 2576 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 21 12:39:53.926470 ip-10-0-130-157 kubenswrapper[2576]: W0721 12:39:53.549471 2576 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 21 12:39:53.926470 ip-10-0-130-157 kubenswrapper[2576]: W0721 12:39:53.549473 2576 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 21 12:39:53.929747 ip-10-0-130-157 kubenswrapper[2576]: W0721 12:39:53.549476 2576 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 21 12:39:53.929747 ip-10-0-130-157 kubenswrapper[2576]: W0721 12:39:53.549480 2576 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 21 12:39:53.929747 ip-10-0-130-157 kubenswrapper[2576]: W0721 12:39:53.549485 2576 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 21 12:39:53.929747 ip-10-0-130-157 kubenswrapper[2576]: W0721 12:39:53.549489 2576 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 21 12:39:53.929747 ip-10-0-130-157 kubenswrapper[2576]: W0721 12:39:53.549492 2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 21 12:39:53.929747 ip-10-0-130-157 kubenswrapper[2576]: W0721 12:39:53.549494 2576 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 21 12:39:53.929747 ip-10-0-130-157 kubenswrapper[2576]: W0721 12:39:53.549497 2576 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 21 12:39:53.929747 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:39:53.549501 2576 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 21 12:39:53.929747 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:39:53.549641 2576 server.go:962] "Client rotation is on, will bootstrap in background" Jul 21 12:39:53.929747 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:39:53.551664 2576 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir" Jul 21 12:39:53.929747 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:39:53.553096 2576 server.go:1019] "Starting client certificate rotation" Jul 21 12:39:53.929747 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:39:53.553203 2576 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet" Jul 21 12:39:53.929747 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:39:53.553250 2576 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet" Jul 21 12:39:53.930162 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:39:53.556500 2576 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Jul 21 12:39:53.930162 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:39:53.558368 2576 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Jul 21 12:39:53.930162 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:39:53.569643 2576 log.go:25] "Validated CRI v1 runtime API" Jul 21 12:39:53.930162 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:39:53.571965 2576 log.go:25] "Validated CRI v1 image API" Jul 21 12:39:53.930162 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:39:53.572885 2576 server.go:1429] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd" Jul 21 12:39:53.930162 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:39:53.575109 2576 fs.go:135] Filesystem UUIDs: map[1a42ffa0-01a1-4140-94db-f2652d8a6ca0:/dev/nvme0n1p3 7B77-95E7:/dev/nvme0n1p2 a9ef9c65-c350-4088-9846-9ccd5b0f512d:/dev/nvme0n1p4] Jul 21 12:39:53.930162 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:39:53.575125 2576 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}] Jul 21 12:39:54.031588 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:39:53.580794 2576 manager.go:216] Machine: {Timestamp:2026-07-21 12:39:53.57957717 +0000 UTC m=+1.414841027 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3098634 MemoryCapacity:33164500992 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec23486b6f4eab79f5548c5c2ca54e4a SystemUUID:ec23486b-6f4e-ab79-f554-8c5c2ca54e4a BootID:721d54de-e9a4-48c4-ae0e-fb686919bf15 Filesystems:[{Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6111232 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16582250496 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true} {Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16582250496 Type:vfs Inodes:4048401 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6632902656 Type:vfs Inodes:819200 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:84:ce:15:18:11 Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:84:ce:15:18:11 Speed:0 Mtu:9001} {Name:ovs-system MacAddress:e2:30:f9:ae:fd:f0 Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:33164500992 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:37486592 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None} Jul 21 12:39:54.031588 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:39:53.580905 2576 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available. Jul 21 12:39:54.031588 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:39:53.581027 2576 manager.go:232] Version: {KernelVersion:5.14.0-570.128.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260715-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:} Jul 21 12:39:54.031588 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:39:53.584423 2576 container_manager_linux.go:273] "Container manager verified user specified cgroup-root exists" cgroupRoot=[] Jul 21 12:39:54.031588 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:39:53.584457 2576 container_manager_linux.go:278] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-130-157.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2} Jul 21 12:39:54.031588 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:39:53.585396 2576 topology_manager.go:138] "Creating topology manager with none policy" Jul 21 12:39:54.031588 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:39:53.585410 2576 container_manager_linux.go:309] "Creating device plugin manager" Jul 21 12:39:54.031588 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:39:53.585425 2576 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock" Jul 21 12:39:54.031588 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:39:53.586153 2576 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock" Jul 21 12:39:54.031588 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:39:53.587451 2576 container_manager_linux.go:318] "Creating Dynamic Resource Allocation (DRA) manager" Jul 21 12:39:54.031588 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:39:53.589157 2576 reflector.go:439] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Jul 21 12:39:54.031588 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:39:53.590526 2576 state_mem.go:36] "Initialized new in-memory state store" Jul 21 12:39:54.031588 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:39:53.590684 2576 server.go:1240] "Using root directory" path="/var/lib/kubelet" Jul 21 12:39:54.031588 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:39:53.590726 2576 kubelet.go:486] "Attempting to sync node with API server" Jul 21 12:39:54.031588 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:39:53.590745 2576 kubelet.go:387] "Adding static pod path" path="/etc/kubernetes/manifests" Jul 21 12:39:54.031588 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:39:53.590765 2576 file.go:69] "Watching path" path="/etc/kubernetes/manifests" Jul 21 12:39:54.031588 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:39:53.590778 2576 kubelet.go:398] "Adding apiserver pod source" Jul 21 12:39:54.039747 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:39:53.590791 2576 apiserver.go:42] "Waiting for node sync before watching apiserver pods" Jul 21 12:39:54.039747 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:39:53.591220 2576 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Jul 21 12:39:54.039747 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:39:53.591240 2576 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Jul 21 12:39:54.039747 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:39:53.593983 2576 kuberuntime_manager.go:301] "Container runtime initialized" containerRuntime="cri-o" version="1.34.10-2.rhaos4.21.git83ee25b.el9" apiVersion="v1" Jul 21 12:39:54.039747 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:39:53.596606 2576 kubelet.go:960] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled" Jul 21 12:39:54.039747 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:39:53.596645 2576 kubelet.go:984] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled" Jul 21 12:39:54.039747 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:39:53.597831 2576 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume" Jul 21 12:39:54.039747 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:39:53.597849 2576 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir" Jul 21 12:39:54.039747 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:39:53.597858 2576 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo" Jul 21 12:39:54.039747 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:39:53.597867 2576 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path" Jul 21 12:39:54.039747 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:39:53.597879 2576 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs" Jul 21 12:39:54.039747 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:39:53.597887 2576 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret" Jul 21 12:39:54.039747 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:39:53.597898 2576 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi" Jul 21 12:39:54.039747 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:39:53.597907 2576 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api" Jul 21 12:39:54.039747 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:39:53.597916 2576 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc" Jul 21 12:39:54.039747 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:39:53.597939 2576 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap" Jul 21 12:39:54.039747 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:39:53.597963 2576 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected" Jul 21 12:39:54.039747 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:39:53.597975 2576 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume" Jul 21 12:39:54.039747 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:39:53.600237 2576 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi" Jul 21 12:39:54.039747 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:39:53.600250 2576 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image" Jul 21 12:39:54.039747 ip-10-0-130-157 kubenswrapper[2576]: E0721 12:39:53.600456 2576 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-130-157.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node" Jul 21 12:39:54.039747 ip-10-0-130-157 kubenswrapper[2576]: E0721 12:39:53.600469 2576 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service" Jul 21 12:39:54.039747 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:39:53.602215 2576 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-130-157.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope Jul 21 12:39:54.039747 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:39:53.604121 2576 server.go:1268] "Started kubelet" Jul 21 12:39:54.039747 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:39:53.604209 2576 server.go:180] "Starting to listen" address="0.0.0.0" port=10250 Jul 21 12:39:54.039747 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:39:53.604215 2576 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10 Jul 21 12:39:54.039747 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:39:53.604283 2576 server_v1.go:49] "podresources" method="list" useActivePods=true Jul 21 12:39:54.039747 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:39:53.609959 2576 server.go:249] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock" Jul 21 12:39:54.039747 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:39:53.611156 2576 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-7kdjb" Jul 21 12:39:54.039747 ip-10-0-130-157 kubenswrapper[2576]: E0721 12:39:53.611267 2576 kubelet.go:1638] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache" Jul 21 12:39:54.039747 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:39:53.611899 2576 server.go:310] "Adding debug handlers to kubelet server" Jul 21 12:39:54.039747 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:39:53.616065 2576 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer" Jul 21 12:39:54.039747 ip-10-0-130-157 kubenswrapper[2576]: E0721 12:39:53.616167 2576 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-130-157.ec2.internal\" not found" Jul 21 12:39:54.039747 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:39:53.616064 2576 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving" Jul 21 12:39:54.039747 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:39:53.616658 2576 desired_state_of_world_populator.go:146] "Desired state populator starts to run" Jul 21 12:39:54.039747 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:39:53.616659 2576 volume_manager.go:311] "The desired_state_of_world populator starts" Jul 21 12:39:54.039747 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:39:53.616684 2576 volume_manager.go:313] "Starting Kubelet Volume Manager" Jul 21 12:39:54.039747 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:39:53.616818 2576 reconstruct.go:97] "Volume reconstruction finished" Jul 21 12:39:54.039747 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:39:53.616826 2576 reconciler.go:29] "Reconciler: start to sync state" Jul 21 12:39:54.039747 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:39:53.616961 2576 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory Jul 21 12:39:54.039747 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:39:53.616972 2576 factory.go:55] Registering systemd factory Jul 21 12:39:54.039747 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:39:53.616981 2576 factory.go:223] Registration of the systemd container factory successfully Jul 21 12:39:54.039747 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:39:53.617208 2576 factory.go:153] Registering CRI-O factory Jul 21 12:39:54.039747 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:39:53.617224 2576 factory.go:223] Registration of the crio container factory successfully Jul 21 12:39:54.039747 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:39:53.617249 2576 factory.go:103] Registering Raw factory Jul 21 12:39:54.039747 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:39:53.617264 2576 manager.go:1196] Started watching for new ooms in manager Jul 21 12:39:54.039747 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:39:53.617631 2576 manager.go:349] Starting recovery of all containers Jul 21 12:39:54.039747 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:39:53.619181 2576 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-7kdjb" Jul 21 12:39:54.041380 ip-10-0-130-157 kubenswrapper[2576]: E0721 12:39:53.624175 2576 controller.go:145] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-130-157.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="200ms" Jul 21 12:39:54.041380 ip-10-0-130-157 kubenswrapper[2576]: E0721 12:39:53.624245 2576 reflector.go:205] "Failed to watch" err="failed to list *v1.CSIDriver: csidrivers.storage.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"csidrivers\" in API group \"storage.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver" Jul 21 12:39:54.041380 ip-10-0-130-157 kubenswrapper[2576]: E0721 12:39:53.624349 2576 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-130-157.ec2.internal.18c44e6f9b94659a default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-130-157.ec2.internal,UID:ip-10-0-130-157.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-130-157.ec2.internal,},FirstTimestamp:2026-07-21 12:39:53.604089242 +0000 UTC m=+1.439353155,LastTimestamp:2026-07-21 12:39:53.604089242 +0000 UTC m=+1.439353155,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-130-157.ec2.internal,}" Jul 21 12:39:54.041380 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:39:53.626713 2576 manager.go:354] Recovery completed Jul 21 12:39:54.041380 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:39:53.631000 2576 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 21 12:39:54.041380 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:39:53.635047 2576 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-157.ec2.internal" event="NodeHasSufficientMemory" Jul 21 12:39:54.041380 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:39:53.635076 2576 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-157.ec2.internal" event="NodeHasNoDiskPressure" Jul 21 12:39:54.041380 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:39:53.635089 2576 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-157.ec2.internal" event="NodeHasSufficientPID" Jul 21 12:39:54.041380 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:39:53.635682 2576 cpu_manager.go:222] "Starting CPU manager" policy="none" Jul 21 12:39:54.041380 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:39:53.635691 2576 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s" Jul 21 12:39:54.041380 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:39:53.635709 2576 state_mem.go:36] "Initialized new in-memory state store" Jul 21 12:39:54.041380 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:39:53.635692 2576 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 21 12:39:54.041380 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:39:53.636529 2576 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-157.ec2.internal" event="NodeHasSufficientMemory" Jul 21 12:39:54.041380 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:39:53.636555 2576 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-157.ec2.internal" event="NodeHasNoDiskPressure" Jul 21 12:39:54.041380 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:39:53.636567 2576 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-157.ec2.internal" event="NodeHasSufficientPID" Jul 21 12:39:54.041380 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:39:53.636938 2576 policy_none.go:49] "None policy: Start" Jul 21 12:39:54.041380 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:39:53.636951 2576 memory_manager.go:187] "Starting memorymanager" policy="None" Jul 21 12:39:54.041380 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:39:53.636961 2576 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint" Jul 21 12:39:54.041380 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:39:53.638117 2576 policy_none.go:47] "Start" Jul 21 12:39:54.041380 ip-10-0-130-157 kubenswrapper[2576]: E0721 12:39:53.717212 2576 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-130-157.ec2.internal\" not found" Jul 21 12:39:54.041380 ip-10-0-130-157 kubenswrapper[2576]: E0721 12:39:53.817488 2576 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-130-157.ec2.internal\" not found" Jul 21 12:39:54.041380 ip-10-0-130-157 kubenswrapper[2576]: E0721 12:39:53.830988 2576 nodelease.go:49] "Failed to get node when trying to set owner ref to the node lease" err="nodes \"ip-10-0-130-157.ec2.internal\" not found" node="ip-10-0-130-157.ec2.internal" Jul 21 12:39:54.041380 ip-10-0-130-157 kubenswrapper[2576]: E0721 12:39:53.918213 2576 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-130-157.ec2.internal\" not found" Jul 21 12:39:54.041380 ip-10-0-130-157 kubenswrapper[2576]: E0721 12:39:54.018559 2576 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-130-157.ec2.internal\" not found" Jul 21 12:39:54.119182 ip-10-0-130-157 kubenswrapper[2576]: E0721 12:39:54.119157 2576 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-130-157.ec2.internal\" not found" Jul 21 12:39:54.202292 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:39:54.202156 2576 manager.go:337] "Starting Device Plugin manager" Jul 21 12:39:54.202404 ip-10-0-130-157 kubenswrapper[2576]: E0721 12:39:54.202314 2576 manager.go:513] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint" Jul 21 12:39:54.202404 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:39:54.202327 2576 server.go:87] "Starting device plugin registration server" Jul 21 12:39:54.202615 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:39:54.202602 2576 eviction_manager.go:189] "Eviction manager: starting control loop" Jul 21 12:39:54.202679 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:39:54.202617 2576 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s" Jul 21 12:39:54.202739 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:39:54.202724 2576 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry" Jul 21 12:39:54.202833 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:39:54.202815 2576 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts" Jul 21 12:39:54.202833 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:39:54.202828 2576 plugin_manager.go:118] "Starting Kubelet Plugin Manager" Jul 21 12:39:54.203316 ip-10-0-130-157 kubenswrapper[2576]: E0721 12:39:54.203295 2576 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\"" Jul 21 12:39:54.203387 ip-10-0-130-157 kubenswrapper[2576]: E0721 12:39:54.203352 2576 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-130-157.ec2.internal\" not found" Jul 21 12:39:54.302781 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:39:54.302696 2576 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 21 12:39:54.304032 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:39:54.304012 2576 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-157.ec2.internal" event="NodeHasSufficientMemory" Jul 21 12:39:54.304137 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:39:54.304047 2576 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-157.ec2.internal" event="NodeHasNoDiskPressure" Jul 21 12:39:54.304137 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:39:54.304062 2576 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-157.ec2.internal" event="NodeHasSufficientPID" Jul 21 12:39:54.304137 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:39:54.304096 2576 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-130-157.ec2.internal" Jul 21 12:39:54.309019 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:39:54.308991 2576 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4" Jul 21 12:39:54.310241 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:39:54.310220 2576 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6" Jul 21 12:39:54.310348 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:39:54.310268 2576 status_manager.go:244] "Starting to sync pod status with apiserver" Jul 21 12:39:54.310348 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:39:54.310292 2576 kubelet.go:2448] "Starting kubelet main sync loop" Jul 21 12:39:54.310348 ip-10-0-130-157 kubenswrapper[2576]: E0721 12:39:54.310333 2576 kubelet.go:2472] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful" Jul 21 12:39:54.312738 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:39:54.312717 2576 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Jul 21 12:39:54.313616 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:39:54.313600 2576 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-130-157.ec2.internal" Jul 21 12:39:54.313694 ip-10-0-130-157 kubenswrapper[2576]: E0721 12:39:54.313622 2576 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-130-157.ec2.internal\": node \"ip-10-0-130-157.ec2.internal\" not found" Jul 21 12:39:54.383169 ip-10-0-130-157 kubenswrapper[2576]: E0721 12:39:54.383136 2576 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-130-157.ec2.internal\" not found" Jul 21 12:39:54.411237 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:39:54.411188 2576 kubelet.go:2535] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-157.ec2.internal","kube-system/kube-apiserver-proxy-ip-10-0-130-157.ec2.internal"] Jul 21 12:39:54.411346 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:39:54.411297 2576 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 21 12:39:54.413929 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:39:54.413913 2576 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-157.ec2.internal" event="NodeHasSufficientMemory" Jul 21 12:39:54.413993 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:39:54.413943 2576 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-157.ec2.internal" event="NodeHasNoDiskPressure" Jul 21 12:39:54.413993 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:39:54.413956 2576 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-157.ec2.internal" event="NodeHasSufficientPID" Jul 21 12:39:54.415154 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:39:54.415142 2576 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 21 12:39:54.415260 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:39:54.415247 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-157.ec2.internal" Jul 21 12:39:54.415306 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:39:54.415279 2576 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 21 12:39:54.416767 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:39:54.416748 2576 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-157.ec2.internal" event="NodeHasSufficientMemory" Jul 21 12:39:54.416879 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:39:54.416780 2576 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-157.ec2.internal" event="NodeHasNoDiskPressure" Jul 21 12:39:54.416879 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:39:54.416796 2576 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-157.ec2.internal" event="NodeHasSufficientPID" Jul 21 12:39:54.416879 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:39:54.416799 2576 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-157.ec2.internal" event="NodeHasSufficientMemory" Jul 21 12:39:54.416879 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:39:54.416819 2576 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-157.ec2.internal" event="NodeHasNoDiskPressure" Jul 21 12:39:54.416879 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:39:54.416833 2576 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-157.ec2.internal" event="NodeHasSufficientPID" Jul 21 12:39:54.417996 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:39:54.417980 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-130-157.ec2.internal" Jul 21 12:39:54.418073 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:39:54.418010 2576 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 21 12:39:54.419096 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:39:54.419074 2576 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-157.ec2.internal" event="NodeHasSufficientMemory" Jul 21 12:39:54.419164 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:39:54.419106 2576 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-157.ec2.internal" event="NodeHasNoDiskPressure" Jul 21 12:39:54.419164 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:39:54.419118 2576 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-157.ec2.internal" event="NodeHasSufficientPID" Jul 21 12:39:54.421150 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:39:54.421130 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/e47f524b69dd5de36bff36ee415660e7-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-130-157.ec2.internal\" (UID: \"e47f524b69dd5de36bff36ee415660e7\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-157.ec2.internal" Jul 21 12:39:54.421252 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:39:54.421154 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/77735b552764c08668c42d1fa6bcbce3-config\") pod \"kube-apiserver-proxy-ip-10-0-130-157.ec2.internal\" (UID: \"77735b552764c08668c42d1fa6bcbce3\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-130-157.ec2.internal" Jul 21 12:39:54.421252 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:39:54.421172 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/e47f524b69dd5de36bff36ee415660e7-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-130-157.ec2.internal\" (UID: \"e47f524b69dd5de36bff36ee415660e7\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-157.ec2.internal" Jul 21 12:39:54.442431 ip-10-0-130-157 kubenswrapper[2576]: E0721 12:39:54.442410 2576 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-130-157.ec2.internal\" not found" node="ip-10-0-130-157.ec2.internal" Jul 21 12:39:54.446983 ip-10-0-130-157 kubenswrapper[2576]: E0721 12:39:54.446967 2576 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-130-157.ec2.internal\" not found" node="ip-10-0-130-157.ec2.internal" Jul 21 12:39:54.483705 ip-10-0-130-157 kubenswrapper[2576]: E0721 12:39:54.483667 2576 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-130-157.ec2.internal\" not found" Jul 21 12:39:54.522285 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:39:54.522265 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/e47f524b69dd5de36bff36ee415660e7-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-130-157.ec2.internal\" (UID: \"e47f524b69dd5de36bff36ee415660e7\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-157.ec2.internal" Jul 21 12:39:54.522374 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:39:54.522298 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/e47f524b69dd5de36bff36ee415660e7-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-130-157.ec2.internal\" (UID: \"e47f524b69dd5de36bff36ee415660e7\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-157.ec2.internal" Jul 21 12:39:54.522374 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:39:54.522316 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/77735b552764c08668c42d1fa6bcbce3-config\") pod \"kube-apiserver-proxy-ip-10-0-130-157.ec2.internal\" (UID: \"77735b552764c08668c42d1fa6bcbce3\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-130-157.ec2.internal" Jul 21 12:39:54.522374 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:39:54.522360 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/77735b552764c08668c42d1fa6bcbce3-config\") pod \"kube-apiserver-proxy-ip-10-0-130-157.ec2.internal\" (UID: \"77735b552764c08668c42d1fa6bcbce3\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-130-157.ec2.internal" Jul 21 12:39:54.522465 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:39:54.522367 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/e47f524b69dd5de36bff36ee415660e7-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-130-157.ec2.internal\" (UID: \"e47f524b69dd5de36bff36ee415660e7\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-157.ec2.internal" Jul 21 12:39:54.522465 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:39:54.522367 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/e47f524b69dd5de36bff36ee415660e7-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-130-157.ec2.internal\" (UID: \"e47f524b69dd5de36bff36ee415660e7\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-157.ec2.internal" Jul 21 12:39:54.528455 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:39:54.528429 2576 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160" Jul 21 12:39:54.552667 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:39:54.552636 2576 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials" Jul 21 12:39:54.552870 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:39:54.552761 2576 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received" Jul 21 12:39:54.552870 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:39:54.552818 2576 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received" Jul 21 12:39:54.552998 ip-10-0-130-157 kubenswrapper[2576]: E0721 12:39:54.552820 2576 event.go:368] "Unable to write event (may retry after sleeping)" err="Patch \"https://a979754ffbe0f401d954ce47bcc8f771-f9fe16ecbdc9a5d4.elb.us-east-1.amazonaws.com:6443/api/v1/namespaces/default/events/ip-10-0-130-157.ec2.internal.18c44e6f9d6d4cc1\": read tcp 10.0.130.157:33234->44.205.192.253:6443: use of closed network connection" event="&Event{ObjectMeta:{ip-10-0-130-157.ec2.internal.18c44e6f9d6d4cc1 default 6232 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-130-157.ec2.internal,UID:ip-10-0-130-157.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasNoDiskPressure,Message:Node ip-10-0-130-157.ec2.internal status is now: NodeHasNoDiskPressure,Source:EventSource{Component:kubelet,Host:ip-10-0-130-157.ec2.internal,},FirstTimestamp:2026-07-21 12:39:53 +0000 UTC,LastTimestamp:2026-07-21 12:39:54.419112273 +0000 UTC m=+2.254376129,Count:7,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-130-157.ec2.internal,}" Jul 21 12:39:54.583987 ip-10-0-130-157 kubenswrapper[2576]: E0721 12:39:54.583952 2576 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-130-157.ec2.internal\" not found" Jul 21 12:39:54.623890 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:39:54.623852 2576 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-07-20 12:34:53 +0000 UTC" deadline="2028-03-24 08:06:55.684377888 +0000 UTC" Jul 21 12:39:54.623890 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:39:54.623880 2576 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="14683h27m1.060501085s" Jul 21 12:39:54.685050 ip-10-0-130-157 kubenswrapper[2576]: E0721 12:39:54.685024 2576 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-130-157.ec2.internal\" not found" Jul 21 12:39:54.745428 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:39:54.745396 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-157.ec2.internal" Jul 21 12:39:54.747768 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:39:54.747747 2576 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160" Jul 21 12:39:54.750404 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:39:54.750386 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-130-157.ec2.internal" Jul 21 12:39:54.753376 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:39:54.753361 2576 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160" Jul 21 12:39:54.816676 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:39:54.816624 2576 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-157.ec2.internal" Jul 21 12:39:54.828156 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:39:54.828138 2576 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Jul 21 12:39:54.829216 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:39:54.829204 2576 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-130-157.ec2.internal" Jul 21 12:39:54.836010 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:39:54.835999 2576 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Jul 21 12:39:55.462774 ip-10-0-130-157 kubenswrapper[2576]: W0721 12:39:55.462731 2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode47f524b69dd5de36bff36ee415660e7.slice/crio-083f37808fb1cdc7a7a435396bbb2925740c7a3db9c6d9dd9afc208227cd8976 WatchSource:0}: Error finding container 083f37808fb1cdc7a7a435396bbb2925740c7a3db9c6d9dd9afc208227cd8976: Status 404 returned error can't find the container with id 083f37808fb1cdc7a7a435396bbb2925740c7a3db9c6d9dd9afc208227cd8976 Jul 21 12:39:55.466634 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:39:55.466617 2576 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 21 12:39:55.591776 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:39:55.591545 2576 apiserver.go:52] "Watching apiserver" Jul 21 12:39:55.598854 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:39:55.598827 2576 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66" Jul 21 12:39:55.599854 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:39:55.599814 2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kube-system/konnectivity-agent-skc26","kube-system/kube-apiserver-proxy-ip-10-0-130-157.ec2.internal","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-wb7vw","openshift-dns/node-resolver-6crjb","openshift-multus/multus-additional-cni-plugins-wnhr4","openshift-multus/multus-pgvgl","openshift-multus/network-metrics-daemon-l9rfj","openshift-cluster-node-tuning-operator/tuned-k7j9j","openshift-image-registry/node-ca-km4fw","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-157.ec2.internal","openshift-network-diagnostics/network-check-target-6f57m","openshift-network-operator/iptables-alerter-dr85z","openshift-ovn-kubernetes/ovnkube-node-6nf7d","kube-system/global-pull-secret-syncer-g2tkf"] Jul 21 12:39:55.601378 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:39:55.601322 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-skc26" Jul 21 12:39:55.602918 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:39:55.602899 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-wb7vw" Jul 21 12:39:55.603284 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:39:55.603267 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\"" Jul 21 12:39:55.603394 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:39:55.603372 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-xd589\"" Jul 21 12:39:55.603439 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:39:55.603403 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\"" Jul 21 12:39:55.604361 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:39:55.604343 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-6crjb" Jul 21 12:39:55.604646 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:39:55.604629 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-tvsqj\"" Jul 21 12:39:55.604831 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:39:55.604810 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\"" Jul 21 12:39:55.604916 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:39:55.604867 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\"" Jul 21 12:39:55.604916 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:39:55.604902 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\"" Jul 21 12:39:55.605731 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:39:55.605712 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-wnhr4" Jul 21 12:39:55.606228 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:39:55.606183 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\"" Jul 21 12:39:55.606318 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:39:55.606305 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\"" Jul 21 12:39:55.606375 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:39:55.606329 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-vm5ss\"" Jul 21 12:39:55.606846 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:39:55.606820 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-pgvgl" Jul 21 12:39:55.607440 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:39:55.607420 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\"" Jul 21 12:39:55.607546 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:39:55.607420 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-lw4cj\"" Jul 21 12:39:55.607546 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:39:55.607476 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\"" Jul 21 12:39:55.607696 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:39:55.607680 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\"" Jul 21 12:39:55.607927 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:39:55.607911 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\"" Jul 21 12:39:55.608003 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:39:55.607935 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\"" Jul 21 12:39:55.608003 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:39:55.607978 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-l9rfj" Jul 21 12:39:55.608108 ip-10-0-130-157 kubenswrapper[2576]: E0721 12:39:55.608035 2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-l9rfj" podUID="d932e6af-610e-4967-87b1-e489cead334d" Jul 21 12:39:55.608440 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:39:55.608426 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\"" Jul 21 12:39:55.608440 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:39:55.608434 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-stjnz\"" Jul 21 12:39:55.609156 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:39:55.609141 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-k7j9j" Jul 21 12:39:55.610375 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:39:55.610358 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-km4fw" Jul 21 12:39:55.610848 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:39:55.610828 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\"" Jul 21 12:39:55.610932 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:39:55.610885 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-wx662\"" Jul 21 12:39:55.611052 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:39:55.611040 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\"" Jul 21 12:39:55.611889 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:39:55.611874 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-cfl8l\"" Jul 21 12:39:55.611977 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:39:55.611935 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-6f57m" Jul 21 12:39:55.612039 ip-10-0-130-157 kubenswrapper[2576]: E0721 12:39:55.612013 2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-6f57m" podUID="ea2d55a6-d43a-4d57-b49c-b63904d8810e" Jul 21 12:39:55.612160 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:39:55.612134 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\"" Jul 21 12:39:55.612348 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:39:55.612330 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\"" Jul 21 12:39:55.612603 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:39:55.612582 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\"" Jul 21 12:39:55.613298 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:39:55.613277 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-dr85z" Jul 21 12:39:55.614740 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:39:55.614719 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-6nf7d" Jul 21 12:39:55.615336 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:39:55.615133 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\"" Jul 21 12:39:55.615336 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:39:55.615318 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-2jhhq\"" Jul 21 12:39:55.615480 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:39:55.615457 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\"" Jul 21 12:39:55.615578 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:39:55.615482 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\"" Jul 21 12:39:55.616508 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:39:55.616466 2576 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving" Jul 21 12:39:55.616508 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:39:55.616494 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\"" Jul 21 12:39:55.616740 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:39:55.616722 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-g2tkf" Jul 21 12:39:55.616874 ip-10-0-130-157 kubenswrapper[2576]: E0721 12:39:55.616816 2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-g2tkf" podUID="4dc9bfad-f2bb-4934-9f49-4cfd0f381ab2" Jul 21 12:39:55.616965 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:39:55.616876 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\"" Jul 21 12:39:55.616965 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:39:55.616880 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\"" Jul 21 12:39:55.617101 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:39:55.617084 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\"" Jul 21 12:39:55.617339 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:39:55.617319 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\"" Jul 21 12:39:55.617459 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:39:55.617442 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\"" Jul 21 12:39:55.618597 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:39:55.618373 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-jh5w6\"" Jul 21 12:39:55.618986 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:39:55.618697 2576 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world" Jul 21 12:39:55.626468 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:39:55.626426 2576 reflector.go:439] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Jul 21 12:39:55.627304 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:39:55.627287 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/c70252ec-a8bb-4dd0-9544-2ff28e109eb9-hosts-file\") pod \"node-resolver-6crjb\" (UID: \"c70252ec-a8bb-4dd0-9544-2ff28e109eb9\") " pod="openshift-dns/node-resolver-6crjb" Jul 21 12:39:55.627385 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:39:55.627316 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/a2b48106-9844-4431-9b6f-d4e1baacb15b-host-run-ovn-kubernetes\") pod \"ovnkube-node-6nf7d\" (UID: \"a2b48106-9844-4431-9b6f-d4e1baacb15b\") " pod="openshift-ovn-kubernetes/ovnkube-node-6nf7d" Jul 21 12:39:55.627385 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:39:55.627342 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/a2b48106-9844-4431-9b6f-d4e1baacb15b-ovnkube-config\") pod \"ovnkube-node-6nf7d\" (UID: \"a2b48106-9844-4431-9b6f-d4e1baacb15b\") " pod="openshift-ovn-kubernetes/ovnkube-node-6nf7d" Jul 21 12:39:55.627385 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:39:55.627373 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/1d597db9-aea4-46bd-a743-06fcb6bc24d2-etc-sysconfig\") pod \"tuned-k7j9j\" (UID: \"1d597db9-aea4-46bd-a743-06fcb6bc24d2\") " pod="openshift-cluster-node-tuning-operator/tuned-k7j9j" Jul 21 12:39:55.627566 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:39:55.627395 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/4dd651ed-4103-426d-83cc-ea465d4e3d7b-iptables-alerter-script\") pod \"iptables-alerter-dr85z\" (UID: \"4dd651ed-4103-426d-83cc-ea465d4e3d7b\") " pod="openshift-network-operator/iptables-alerter-dr85z" Jul 21 12:39:55.627566 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:39:55.627411 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/6713665c-3b71-4751-929e-16de48263766-agent-certs\") pod \"konnectivity-agent-skc26\" (UID: \"6713665c-3b71-4751-929e-16de48263766\") " pod="kube-system/konnectivity-agent-skc26" Jul 21 12:39:55.627566 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:39:55.627425 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/06df9492-bc31-44f9-9f87-1665786f3be4-etc-selinux\") pod \"aws-ebs-csi-driver-node-wb7vw\" (UID: \"06df9492-bc31-44f9-9f87-1665786f3be4\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-wb7vw" Jul 21 12:39:55.627566 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:39:55.627440 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/490c41c0-4938-41c5-a264-1cf80533b229-cnibin\") pod \"multus-pgvgl\" (UID: \"490c41c0-4938-41c5-a264-1cf80533b229\") " pod="openshift-multus/multus-pgvgl" Jul 21 12:39:55.627566 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:39:55.627478 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d932e6af-610e-4967-87b1-e489cead334d-metrics-certs\") pod \"network-metrics-daemon-l9rfj\" (UID: \"d932e6af-610e-4967-87b1-e489cead334d\") " pod="openshift-multus/network-metrics-daemon-l9rfj" Jul 21 12:39:55.627566 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:39:55.627543 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/1d597db9-aea4-46bd-a743-06fcb6bc24d2-etc-sysctl-conf\") pod \"tuned-k7j9j\" (UID: \"1d597db9-aea4-46bd-a743-06fcb6bc24d2\") " pod="openshift-cluster-node-tuning-operator/tuned-k7j9j" Jul 21 12:39:55.627866 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:39:55.627575 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hs9gs\" (UniqueName: \"kubernetes.io/projected/1d597db9-aea4-46bd-a743-06fcb6bc24d2-kube-api-access-hs9gs\") pod \"tuned-k7j9j\" (UID: \"1d597db9-aea4-46bd-a743-06fcb6bc24d2\") " pod="openshift-cluster-node-tuning-operator/tuned-k7j9j" Jul 21 12:39:55.627866 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:39:55.627625 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kvc8f\" (UniqueName: \"kubernetes.io/projected/ea2d55a6-d43a-4d57-b49c-b63904d8810e-kube-api-access-kvc8f\") pod \"network-check-target-6f57m\" (UID: \"ea2d55a6-d43a-4d57-b49c-b63904d8810e\") " pod="openshift-network-diagnostics/network-check-target-6f57m" Jul 21 12:39:55.627866 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:39:55.627640 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/64971bb0-35c9-4026-9680-2a7700efb382-system-cni-dir\") pod \"multus-additional-cni-plugins-wnhr4\" (UID: \"64971bb0-35c9-4026-9680-2a7700efb382\") " pod="openshift-multus/multus-additional-cni-plugins-wnhr4" Jul 21 12:39:55.627866 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:39:55.627661 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/a2b48106-9844-4431-9b6f-d4e1baacb15b-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-6nf7d\" (UID: \"a2b48106-9844-4431-9b6f-d4e1baacb15b\") " pod="openshift-ovn-kubernetes/ovnkube-node-6nf7d" Jul 21 12:39:55.627866 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:39:55.627687 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/490c41c0-4938-41c5-a264-1cf80533b229-host-var-lib-cni-bin\") pod \"multus-pgvgl\" (UID: \"490c41c0-4938-41c5-a264-1cf80533b229\") " pod="openshift-multus/multus-pgvgl" Jul 21 12:39:55.627866 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:39:55.627722 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/490c41c0-4938-41c5-a264-1cf80533b229-multus-conf-dir\") pod \"multus-pgvgl\" (UID: \"490c41c0-4938-41c5-a264-1cf80533b229\") " pod="openshift-multus/multus-pgvgl" Jul 21 12:39:55.627866 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:39:55.627742 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/1d597db9-aea4-46bd-a743-06fcb6bc24d2-lib-modules\") pod \"tuned-k7j9j\" (UID: \"1d597db9-aea4-46bd-a743-06fcb6bc24d2\") " pod="openshift-cluster-node-tuning-operator/tuned-k7j9j" Jul 21 12:39:55.627866 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:39:55.627778 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/1d597db9-aea4-46bd-a743-06fcb6bc24d2-host\") pod \"tuned-k7j9j\" (UID: \"1d597db9-aea4-46bd-a743-06fcb6bc24d2\") " pod="openshift-cluster-node-tuning-operator/tuned-k7j9j" Jul 21 12:39:55.627866 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:39:55.627823 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/4dc9bfad-f2bb-4934-9f49-4cfd0f381ab2-kubelet-config\") pod \"global-pull-secret-syncer-g2tkf\" (UID: \"4dc9bfad-f2bb-4934-9f49-4cfd0f381ab2\") " pod="kube-system/global-pull-secret-syncer-g2tkf" Jul 21 12:39:55.627866 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:39:55.627852 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/a2b48106-9844-4431-9b6f-d4e1baacb15b-host-run-netns\") pod \"ovnkube-node-6nf7d\" (UID: \"a2b48106-9844-4431-9b6f-d4e1baacb15b\") " pod="openshift-ovn-kubernetes/ovnkube-node-6nf7d" Jul 21 12:39:55.627866 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:39:55.627869 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/490c41c0-4938-41c5-a264-1cf80533b229-host-run-multus-certs\") pod \"multus-pgvgl\" (UID: \"490c41c0-4938-41c5-a264-1cf80533b229\") " pod="openshift-multus/multus-pgvgl" Jul 21 12:39:55.628478 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:39:55.627884 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/490c41c0-4938-41c5-a264-1cf80533b229-etc-kubernetes\") pod \"multus-pgvgl\" (UID: \"490c41c0-4938-41c5-a264-1cf80533b229\") " pod="openshift-multus/multus-pgvgl" Jul 21 12:39:55.628478 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:39:55.627899 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/1d597db9-aea4-46bd-a743-06fcb6bc24d2-tmp\") pod \"tuned-k7j9j\" (UID: \"1d597db9-aea4-46bd-a743-06fcb6bc24d2\") " pod="openshift-cluster-node-tuning-operator/tuned-k7j9j" Jul 21 12:39:55.628478 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:39:55.627935 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/4dc9bfad-f2bb-4934-9f49-4cfd0f381ab2-original-pull-secret\") pod \"global-pull-secret-syncer-g2tkf\" (UID: \"4dc9bfad-f2bb-4934-9f49-4cfd0f381ab2\") " pod="kube-system/global-pull-secret-syncer-g2tkf" Jul 21 12:39:55.628478 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:39:55.627961 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lwd2d\" (UniqueName: \"kubernetes.io/projected/c70252ec-a8bb-4dd0-9544-2ff28e109eb9-kube-api-access-lwd2d\") pod \"node-resolver-6crjb\" (UID: \"c70252ec-a8bb-4dd0-9544-2ff28e109eb9\") " pod="openshift-dns/node-resolver-6crjb" Jul 21 12:39:55.628478 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:39:55.628006 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/06df9492-bc31-44f9-9f87-1665786f3be4-registration-dir\") pod \"aws-ebs-csi-driver-node-wb7vw\" (UID: \"06df9492-bc31-44f9-9f87-1665786f3be4\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-wb7vw" Jul 21 12:39:55.628478 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:39:55.628028 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/06df9492-bc31-44f9-9f87-1665786f3be4-sys-fs\") pod \"aws-ebs-csi-driver-node-wb7vw\" (UID: \"06df9492-bc31-44f9-9f87-1665786f3be4\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-wb7vw" Jul 21 12:39:55.628478 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:39:55.628044 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/490c41c0-4938-41c5-a264-1cf80533b229-hostroot\") pod \"multus-pgvgl\" (UID: \"490c41c0-4938-41c5-a264-1cf80533b229\") " pod="openshift-multus/multus-pgvgl" Jul 21 12:39:55.628478 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:39:55.628057 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/1d597db9-aea4-46bd-a743-06fcb6bc24d2-run\") pod \"tuned-k7j9j\" (UID: \"1d597db9-aea4-46bd-a743-06fcb6bc24d2\") " pod="openshift-cluster-node-tuning-operator/tuned-k7j9j" Jul 21 12:39:55.628478 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:39:55.628081 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/4dc9bfad-f2bb-4934-9f49-4cfd0f381ab2-dbus\") pod \"global-pull-secret-syncer-g2tkf\" (UID: \"4dc9bfad-f2bb-4934-9f49-4cfd0f381ab2\") " pod="kube-system/global-pull-secret-syncer-g2tkf" Jul 21 12:39:55.628478 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:39:55.628100 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/64971bb0-35c9-4026-9680-2a7700efb382-cnibin\") pod \"multus-additional-cni-plugins-wnhr4\" (UID: \"64971bb0-35c9-4026-9680-2a7700efb382\") " pod="openshift-multus/multus-additional-cni-plugins-wnhr4" Jul 21 12:39:55.628478 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:39:55.628114 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/a2b48106-9844-4431-9b6f-d4e1baacb15b-host-kubelet\") pod \"ovnkube-node-6nf7d\" (UID: \"a2b48106-9844-4431-9b6f-d4e1baacb15b\") " pod="openshift-ovn-kubernetes/ovnkube-node-6nf7d" Jul 21 12:39:55.628478 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:39:55.628128 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/a2b48106-9844-4431-9b6f-d4e1baacb15b-run-systemd\") pod \"ovnkube-node-6nf7d\" (UID: \"a2b48106-9844-4431-9b6f-d4e1baacb15b\") " pod="openshift-ovn-kubernetes/ovnkube-node-6nf7d" Jul 21 12:39:55.628478 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:39:55.628149 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/1d597db9-aea4-46bd-a743-06fcb6bc24d2-etc-modprobe-d\") pod \"tuned-k7j9j\" (UID: \"1d597db9-aea4-46bd-a743-06fcb6bc24d2\") " pod="openshift-cluster-node-tuning-operator/tuned-k7j9j" Jul 21 12:39:55.628478 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:39:55.628178 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c70252ec-a8bb-4dd0-9544-2ff28e109eb9-tmp-dir\") pod \"node-resolver-6crjb\" (UID: \"c70252ec-a8bb-4dd0-9544-2ff28e109eb9\") " pod="openshift-dns/node-resolver-6crjb" Jul 21 12:39:55.628478 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:39:55.628208 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wc78j\" (UniqueName: \"kubernetes.io/projected/9cfd0727-8ed4-40bd-b1d5-66a531560bce-kube-api-access-wc78j\") pod \"node-ca-km4fw\" (UID: \"9cfd0727-8ed4-40bd-b1d5-66a531560bce\") " pod="openshift-image-registry/node-ca-km4fw" Jul 21 12:39:55.628478 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:39:55.628246 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/a2b48106-9844-4431-9b6f-d4e1baacb15b-env-overrides\") pod \"ovnkube-node-6nf7d\" (UID: \"a2b48106-9844-4431-9b6f-d4e1baacb15b\") " pod="openshift-ovn-kubernetes/ovnkube-node-6nf7d" Jul 21 12:39:55.628478 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:39:55.628261 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/6713665c-3b71-4751-929e-16de48263766-konnectivity-ca\") pod \"konnectivity-agent-skc26\" (UID: \"6713665c-3b71-4751-929e-16de48263766\") " pod="kube-system/konnectivity-agent-skc26" Jul 21 12:39:55.629135 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:39:55.628285 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/a2b48106-9844-4431-9b6f-d4e1baacb15b-systemd-units\") pod \"ovnkube-node-6nf7d\" (UID: \"a2b48106-9844-4431-9b6f-d4e1baacb15b\") " pod="openshift-ovn-kubernetes/ovnkube-node-6nf7d" Jul 21 12:39:55.629135 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:39:55.628334 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/a2b48106-9844-4431-9b6f-d4e1baacb15b-var-lib-openvswitch\") pod \"ovnkube-node-6nf7d\" (UID: \"a2b48106-9844-4431-9b6f-d4e1baacb15b\") " pod="openshift-ovn-kubernetes/ovnkube-node-6nf7d" Jul 21 12:39:55.629135 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:39:55.628357 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/a2b48106-9844-4431-9b6f-d4e1baacb15b-run-ovn\") pod \"ovnkube-node-6nf7d\" (UID: \"a2b48106-9844-4431-9b6f-d4e1baacb15b\") " pod="openshift-ovn-kubernetes/ovnkube-node-6nf7d" Jul 21 12:39:55.629135 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:39:55.628380 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ffzgq\" (UniqueName: \"kubernetes.io/projected/a2b48106-9844-4431-9b6f-d4e1baacb15b-kube-api-access-ffzgq\") pod \"ovnkube-node-6nf7d\" (UID: \"a2b48106-9844-4431-9b6f-d4e1baacb15b\") " pod="openshift-ovn-kubernetes/ovnkube-node-6nf7d" Jul 21 12:39:55.629135 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:39:55.628396 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/490c41c0-4938-41c5-a264-1cf80533b229-system-cni-dir\") pod \"multus-pgvgl\" (UID: \"490c41c0-4938-41c5-a264-1cf80533b229\") " pod="openshift-multus/multus-pgvgl" Jul 21 12:39:55.629135 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:39:55.628424 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zkg7v\" (UniqueName: \"kubernetes.io/projected/d932e6af-610e-4967-87b1-e489cead334d-kube-api-access-zkg7v\") pod \"network-metrics-daemon-l9rfj\" (UID: \"d932e6af-610e-4967-87b1-e489cead334d\") " pod="openshift-multus/network-metrics-daemon-l9rfj" Jul 21 12:39:55.629135 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:39:55.628445 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4d7vz\" (UniqueName: \"kubernetes.io/projected/64971bb0-35c9-4026-9680-2a7700efb382-kube-api-access-4d7vz\") pod \"multus-additional-cni-plugins-wnhr4\" (UID: \"64971bb0-35c9-4026-9680-2a7700efb382\") " pod="openshift-multus/multus-additional-cni-plugins-wnhr4" Jul 21 12:39:55.629135 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:39:55.628467 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/9cfd0727-8ed4-40bd-b1d5-66a531560bce-host\") pod \"node-ca-km4fw\" (UID: \"9cfd0727-8ed4-40bd-b1d5-66a531560bce\") " pod="openshift-image-registry/node-ca-km4fw" Jul 21 12:39:55.629135 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:39:55.628483 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/a2b48106-9844-4431-9b6f-d4e1baacb15b-node-log\") pod \"ovnkube-node-6nf7d\" (UID: \"a2b48106-9844-4431-9b6f-d4e1baacb15b\") " pod="openshift-ovn-kubernetes/ovnkube-node-6nf7d" Jul 21 12:39:55.629135 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:39:55.628499 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/490c41c0-4938-41c5-a264-1cf80533b229-multus-cni-dir\") pod \"multus-pgvgl\" (UID: \"490c41c0-4938-41c5-a264-1cf80533b229\") " pod="openshift-multus/multus-pgvgl" Jul 21 12:39:55.629135 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:39:55.628542 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/490c41c0-4938-41c5-a264-1cf80533b229-host-var-lib-cni-multus\") pod \"multus-pgvgl\" (UID: \"490c41c0-4938-41c5-a264-1cf80533b229\") " pod="openshift-multus/multus-pgvgl" Jul 21 12:39:55.629135 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:39:55.628583 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/1d597db9-aea4-46bd-a743-06fcb6bc24d2-var-lib-kubelet\") pod \"tuned-k7j9j\" (UID: \"1d597db9-aea4-46bd-a743-06fcb6bc24d2\") " pod="openshift-cluster-node-tuning-operator/tuned-k7j9j" Jul 21 12:39:55.629135 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:39:55.628617 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/a2b48106-9844-4431-9b6f-d4e1baacb15b-host-slash\") pod \"ovnkube-node-6nf7d\" (UID: \"a2b48106-9844-4431-9b6f-d4e1baacb15b\") " pod="openshift-ovn-kubernetes/ovnkube-node-6nf7d" Jul 21 12:39:55.629135 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:39:55.628646 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/06df9492-bc31-44f9-9f87-1665786f3be4-kubelet-dir\") pod \"aws-ebs-csi-driver-node-wb7vw\" (UID: \"06df9492-bc31-44f9-9f87-1665786f3be4\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-wb7vw" Jul 21 12:39:55.629135 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:39:55.628659 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/490c41c0-4938-41c5-a264-1cf80533b229-os-release\") pod \"multus-pgvgl\" (UID: \"490c41c0-4938-41c5-a264-1cf80533b229\") " pod="openshift-multus/multus-pgvgl" Jul 21 12:39:55.629135 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:39:55.628691 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5q68t\" (UniqueName: \"kubernetes.io/projected/490c41c0-4938-41c5-a264-1cf80533b229-kube-api-access-5q68t\") pod \"multus-pgvgl\" (UID: \"490c41c0-4938-41c5-a264-1cf80533b229\") " pod="openshift-multus/multus-pgvgl" Jul 21 12:39:55.629701 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:39:55.628719 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/a2b48106-9844-4431-9b6f-d4e1baacb15b-etc-openvswitch\") pod \"ovnkube-node-6nf7d\" (UID: \"a2b48106-9844-4431-9b6f-d4e1baacb15b\") " pod="openshift-ovn-kubernetes/ovnkube-node-6nf7d" Jul 21 12:39:55.629701 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:39:55.628742 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/490c41c0-4938-41c5-a264-1cf80533b229-host-run-k8s-cni-cncf-io\") pod \"multus-pgvgl\" (UID: \"490c41c0-4938-41c5-a264-1cf80533b229\") " pod="openshift-multus/multus-pgvgl" Jul 21 12:39:55.629701 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:39:55.628764 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/1d597db9-aea4-46bd-a743-06fcb6bc24d2-sys\") pod \"tuned-k7j9j\" (UID: \"1d597db9-aea4-46bd-a743-06fcb6bc24d2\") " pod="openshift-cluster-node-tuning-operator/tuned-k7j9j" Jul 21 12:39:55.629701 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:39:55.628795 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/64971bb0-35c9-4026-9680-2a7700efb382-cni-binary-copy\") pod \"multus-additional-cni-plugins-wnhr4\" (UID: \"64971bb0-35c9-4026-9680-2a7700efb382\") " pod="openshift-multus/multus-additional-cni-plugins-wnhr4" Jul 21 12:39:55.629701 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:39:55.628819 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/64971bb0-35c9-4026-9680-2a7700efb382-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-wnhr4\" (UID: \"64971bb0-35c9-4026-9680-2a7700efb382\") " pod="openshift-multus/multus-additional-cni-plugins-wnhr4" Jul 21 12:39:55.629701 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:39:55.628843 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/a2b48106-9844-4431-9b6f-d4e1baacb15b-host-cni-netd\") pod \"ovnkube-node-6nf7d\" (UID: \"a2b48106-9844-4431-9b6f-d4e1baacb15b\") " pod="openshift-ovn-kubernetes/ovnkube-node-6nf7d" Jul 21 12:39:55.629701 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:39:55.628864 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/490c41c0-4938-41c5-a264-1cf80533b229-host-run-netns\") pod \"multus-pgvgl\" (UID: \"490c41c0-4938-41c5-a264-1cf80533b229\") " pod="openshift-multus/multus-pgvgl" Jul 21 12:39:55.629701 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:39:55.628901 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/1d597db9-aea4-46bd-a743-06fcb6bc24d2-etc-sysctl-d\") pod \"tuned-k7j9j\" (UID: \"1d597db9-aea4-46bd-a743-06fcb6bc24d2\") " pod="openshift-cluster-node-tuning-operator/tuned-k7j9j" Jul 21 12:39:55.629701 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:39:55.628949 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/1d597db9-aea4-46bd-a743-06fcb6bc24d2-etc-systemd\") pod \"tuned-k7j9j\" (UID: \"1d597db9-aea4-46bd-a743-06fcb6bc24d2\") " pod="openshift-cluster-node-tuning-operator/tuned-k7j9j" Jul 21 12:39:55.629701 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:39:55.628972 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/64971bb0-35c9-4026-9680-2a7700efb382-os-release\") pod \"multus-additional-cni-plugins-wnhr4\" (UID: \"64971bb0-35c9-4026-9680-2a7700efb382\") " pod="openshift-multus/multus-additional-cni-plugins-wnhr4" Jul 21 12:39:55.629701 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:39:55.628996 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/a2b48106-9844-4431-9b6f-d4e1baacb15b-host-cni-bin\") pod \"ovnkube-node-6nf7d\" (UID: \"a2b48106-9844-4431-9b6f-d4e1baacb15b\") " pod="openshift-ovn-kubernetes/ovnkube-node-6nf7d" Jul 21 12:39:55.629701 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:39:55.629017 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/06df9492-bc31-44f9-9f87-1665786f3be4-device-dir\") pod \"aws-ebs-csi-driver-node-wb7vw\" (UID: \"06df9492-bc31-44f9-9f87-1665786f3be4\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-wb7vw" Jul 21 12:39:55.629701 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:39:55.629038 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/4dd651ed-4103-426d-83cc-ea465d4e3d7b-host-slash\") pod \"iptables-alerter-dr85z\" (UID: \"4dd651ed-4103-426d-83cc-ea465d4e3d7b\") " pod="openshift-network-operator/iptables-alerter-dr85z" Jul 21 12:39:55.629701 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:39:55.629062 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-klh55\" (UniqueName: \"kubernetes.io/projected/4dd651ed-4103-426d-83cc-ea465d4e3d7b-kube-api-access-klh55\") pod \"iptables-alerter-dr85z\" (UID: \"4dd651ed-4103-426d-83cc-ea465d4e3d7b\") " pod="openshift-network-operator/iptables-alerter-dr85z" Jul 21 12:39:55.629701 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:39:55.629088 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/64971bb0-35c9-4026-9680-2a7700efb382-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-wnhr4\" (UID: \"64971bb0-35c9-4026-9680-2a7700efb382\") " pod="openshift-multus/multus-additional-cni-plugins-wnhr4" Jul 21 12:39:55.629701 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:39:55.629114 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/a2b48106-9844-4431-9b6f-d4e1baacb15b-log-socket\") pod \"ovnkube-node-6nf7d\" (UID: \"a2b48106-9844-4431-9b6f-d4e1baacb15b\") " pod="openshift-ovn-kubernetes/ovnkube-node-6nf7d" Jul 21 12:39:55.630195 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:39:55.629137 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/06df9492-bc31-44f9-9f87-1665786f3be4-socket-dir\") pod \"aws-ebs-csi-driver-node-wb7vw\" (UID: \"06df9492-bc31-44f9-9f87-1665786f3be4\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-wb7vw" Jul 21 12:39:55.630195 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:39:55.629170 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2x47f\" (UniqueName: \"kubernetes.io/projected/06df9492-bc31-44f9-9f87-1665786f3be4-kube-api-access-2x47f\") pod \"aws-ebs-csi-driver-node-wb7vw\" (UID: \"06df9492-bc31-44f9-9f87-1665786f3be4\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-wb7vw" Jul 21 12:39:55.630195 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:39:55.629194 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/490c41c0-4938-41c5-a264-1cf80533b229-multus-daemon-config\") pod \"multus-pgvgl\" (UID: \"490c41c0-4938-41c5-a264-1cf80533b229\") " pod="openshift-multus/multus-pgvgl" Jul 21 12:39:55.630195 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:39:55.629218 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/1d597db9-aea4-46bd-a743-06fcb6bc24d2-etc-kubernetes\") pod \"tuned-k7j9j\" (UID: \"1d597db9-aea4-46bd-a743-06fcb6bc24d2\") " pod="openshift-cluster-node-tuning-operator/tuned-k7j9j" Jul 21 12:39:55.630195 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:39:55.629247 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/9cfd0727-8ed4-40bd-b1d5-66a531560bce-serviceca\") pod \"node-ca-km4fw\" (UID: \"9cfd0727-8ed4-40bd-b1d5-66a531560bce\") " pod="openshift-image-registry/node-ca-km4fw" Jul 21 12:39:55.630195 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:39:55.629281 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/a2b48106-9844-4431-9b6f-d4e1baacb15b-run-openvswitch\") pod \"ovnkube-node-6nf7d\" (UID: \"a2b48106-9844-4431-9b6f-d4e1baacb15b\") " pod="openshift-ovn-kubernetes/ovnkube-node-6nf7d" Jul 21 12:39:55.630195 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:39:55.629301 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/a2b48106-9844-4431-9b6f-d4e1baacb15b-ovn-node-metrics-cert\") pod \"ovnkube-node-6nf7d\" (UID: \"a2b48106-9844-4431-9b6f-d4e1baacb15b\") " pod="openshift-ovn-kubernetes/ovnkube-node-6nf7d" Jul 21 12:39:55.630195 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:39:55.629325 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/490c41c0-4938-41c5-a264-1cf80533b229-multus-socket-dir-parent\") pod \"multus-pgvgl\" (UID: \"490c41c0-4938-41c5-a264-1cf80533b229\") " pod="openshift-multus/multus-pgvgl" Jul 21 12:39:55.630195 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:39:55.629346 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/490c41c0-4938-41c5-a264-1cf80533b229-host-var-lib-kubelet\") pod \"multus-pgvgl\" (UID: \"490c41c0-4938-41c5-a264-1cf80533b229\") " pod="openshift-multus/multus-pgvgl" Jul 21 12:39:55.630195 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:39:55.629370 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/1d597db9-aea4-46bd-a743-06fcb6bc24d2-etc-tuned\") pod \"tuned-k7j9j\" (UID: \"1d597db9-aea4-46bd-a743-06fcb6bc24d2\") " pod="openshift-cluster-node-tuning-operator/tuned-k7j9j" Jul 21 12:39:55.630195 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:39:55.629399 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/490c41c0-4938-41c5-a264-1cf80533b229-cni-binary-copy\") pod \"multus-pgvgl\" (UID: \"490c41c0-4938-41c5-a264-1cf80533b229\") " pod="openshift-multus/multus-pgvgl" Jul 21 12:39:55.630195 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:39:55.629437 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/64971bb0-35c9-4026-9680-2a7700efb382-tuning-conf-dir\") pod \"multus-additional-cni-plugins-wnhr4\" (UID: \"64971bb0-35c9-4026-9680-2a7700efb382\") " pod="openshift-multus/multus-additional-cni-plugins-wnhr4" Jul 21 12:39:55.630195 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:39:55.629468 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/a2b48106-9844-4431-9b6f-d4e1baacb15b-ovnkube-script-lib\") pod \"ovnkube-node-6nf7d\" (UID: \"a2b48106-9844-4431-9b6f-d4e1baacb15b\") " pod="openshift-ovn-kubernetes/ovnkube-node-6nf7d" Jul 21 12:39:55.648583 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:39:55.648563 2576 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-t4c5b" Jul 21 12:39:55.663953 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:39:55.663934 2576 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-t4c5b" Jul 21 12:39:55.729812 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:39:55.729772 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c70252ec-a8bb-4dd0-9544-2ff28e109eb9-tmp-dir\") pod \"node-resolver-6crjb\" (UID: \"c70252ec-a8bb-4dd0-9544-2ff28e109eb9\") " pod="openshift-dns/node-resolver-6crjb" Jul 21 12:39:55.729812 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:39:55.729814 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wc78j\" (UniqueName: \"kubernetes.io/projected/9cfd0727-8ed4-40bd-b1d5-66a531560bce-kube-api-access-wc78j\") pod \"node-ca-km4fw\" (UID: \"9cfd0727-8ed4-40bd-b1d5-66a531560bce\") " pod="openshift-image-registry/node-ca-km4fw" Jul 21 12:39:55.730508 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:39:55.729842 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/a2b48106-9844-4431-9b6f-d4e1baacb15b-env-overrides\") pod \"ovnkube-node-6nf7d\" (UID: \"a2b48106-9844-4431-9b6f-d4e1baacb15b\") " pod="openshift-ovn-kubernetes/ovnkube-node-6nf7d" Jul 21 12:39:55.730508 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:39:55.729950 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/6713665c-3b71-4751-929e-16de48263766-konnectivity-ca\") pod \"konnectivity-agent-skc26\" (UID: \"6713665c-3b71-4751-929e-16de48263766\") " pod="kube-system/konnectivity-agent-skc26" Jul 21 12:39:55.730508 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:39:55.729982 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/a2b48106-9844-4431-9b6f-d4e1baacb15b-systemd-units\") pod \"ovnkube-node-6nf7d\" (UID: \"a2b48106-9844-4431-9b6f-d4e1baacb15b\") " pod="openshift-ovn-kubernetes/ovnkube-node-6nf7d" Jul 21 12:39:55.730508 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:39:55.730010 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/a2b48106-9844-4431-9b6f-d4e1baacb15b-var-lib-openvswitch\") pod \"ovnkube-node-6nf7d\" (UID: \"a2b48106-9844-4431-9b6f-d4e1baacb15b\") " pod="openshift-ovn-kubernetes/ovnkube-node-6nf7d" Jul 21 12:39:55.730508 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:39:55.730034 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/a2b48106-9844-4431-9b6f-d4e1baacb15b-run-ovn\") pod \"ovnkube-node-6nf7d\" (UID: \"a2b48106-9844-4431-9b6f-d4e1baacb15b\") " pod="openshift-ovn-kubernetes/ovnkube-node-6nf7d" Jul 21 12:39:55.730508 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:39:55.730058 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ffzgq\" (UniqueName: \"kubernetes.io/projected/a2b48106-9844-4431-9b6f-d4e1baacb15b-kube-api-access-ffzgq\") pod \"ovnkube-node-6nf7d\" (UID: \"a2b48106-9844-4431-9b6f-d4e1baacb15b\") " pod="openshift-ovn-kubernetes/ovnkube-node-6nf7d" Jul 21 12:39:55.730508 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:39:55.730083 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/490c41c0-4938-41c5-a264-1cf80533b229-system-cni-dir\") pod \"multus-pgvgl\" (UID: \"490c41c0-4938-41c5-a264-1cf80533b229\") " pod="openshift-multus/multus-pgvgl" Jul 21 12:39:55.730508 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:39:55.730111 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c70252ec-a8bb-4dd0-9544-2ff28e109eb9-tmp-dir\") pod \"node-resolver-6crjb\" (UID: \"c70252ec-a8bb-4dd0-9544-2ff28e109eb9\") " pod="openshift-dns/node-resolver-6crjb" Jul 21 12:39:55.730508 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:39:55.730120 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/a2b48106-9844-4431-9b6f-d4e1baacb15b-var-lib-openvswitch\") pod \"ovnkube-node-6nf7d\" (UID: \"a2b48106-9844-4431-9b6f-d4e1baacb15b\") " pod="openshift-ovn-kubernetes/ovnkube-node-6nf7d" Jul 21 12:39:55.730508 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:39:55.730148 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/490c41c0-4938-41c5-a264-1cf80533b229-system-cni-dir\") pod \"multus-pgvgl\" (UID: \"490c41c0-4938-41c5-a264-1cf80533b229\") " pod="openshift-multus/multus-pgvgl" Jul 21 12:39:55.730508 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:39:55.730152 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/a2b48106-9844-4431-9b6f-d4e1baacb15b-systemd-units\") pod \"ovnkube-node-6nf7d\" (UID: \"a2b48106-9844-4431-9b6f-d4e1baacb15b\") " pod="openshift-ovn-kubernetes/ovnkube-node-6nf7d" Jul 21 12:39:55.730508 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:39:55.730155 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/a2b48106-9844-4431-9b6f-d4e1baacb15b-run-ovn\") pod \"ovnkube-node-6nf7d\" (UID: \"a2b48106-9844-4431-9b6f-d4e1baacb15b\") " pod="openshift-ovn-kubernetes/ovnkube-node-6nf7d" Jul 21 12:39:55.730508 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:39:55.730176 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zkg7v\" (UniqueName: \"kubernetes.io/projected/d932e6af-610e-4967-87b1-e489cead334d-kube-api-access-zkg7v\") pod \"network-metrics-daemon-l9rfj\" (UID: \"d932e6af-610e-4967-87b1-e489cead334d\") " pod="openshift-multus/network-metrics-daemon-l9rfj" Jul 21 12:39:55.730508 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:39:55.730210 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4d7vz\" (UniqueName: \"kubernetes.io/projected/64971bb0-35c9-4026-9680-2a7700efb382-kube-api-access-4d7vz\") pod \"multus-additional-cni-plugins-wnhr4\" (UID: \"64971bb0-35c9-4026-9680-2a7700efb382\") " pod="openshift-multus/multus-additional-cni-plugins-wnhr4" Jul 21 12:39:55.730508 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:39:55.730247 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/9cfd0727-8ed4-40bd-b1d5-66a531560bce-host\") pod \"node-ca-km4fw\" (UID: \"9cfd0727-8ed4-40bd-b1d5-66a531560bce\") " pod="openshift-image-registry/node-ca-km4fw" Jul 21 12:39:55.730508 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:39:55.730270 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/a2b48106-9844-4431-9b6f-d4e1baacb15b-node-log\") pod \"ovnkube-node-6nf7d\" (UID: \"a2b48106-9844-4431-9b6f-d4e1baacb15b\") " pod="openshift-ovn-kubernetes/ovnkube-node-6nf7d" Jul 21 12:39:55.730508 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:39:55.730285 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/490c41c0-4938-41c5-a264-1cf80533b229-multus-cni-dir\") pod \"multus-pgvgl\" (UID: \"490c41c0-4938-41c5-a264-1cf80533b229\") " pod="openshift-multus/multus-pgvgl" Jul 21 12:39:55.730508 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:39:55.730303 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/490c41c0-4938-41c5-a264-1cf80533b229-host-var-lib-cni-multus\") pod \"multus-pgvgl\" (UID: \"490c41c0-4938-41c5-a264-1cf80533b229\") " pod="openshift-multus/multus-pgvgl" Jul 21 12:39:55.731355 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:39:55.730304 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/9cfd0727-8ed4-40bd-b1d5-66a531560bce-host\") pod \"node-ca-km4fw\" (UID: \"9cfd0727-8ed4-40bd-b1d5-66a531560bce\") " pod="openshift-image-registry/node-ca-km4fw" Jul 21 12:39:55.731355 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:39:55.730327 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/1d597db9-aea4-46bd-a743-06fcb6bc24d2-var-lib-kubelet\") pod \"tuned-k7j9j\" (UID: \"1d597db9-aea4-46bd-a743-06fcb6bc24d2\") " pod="openshift-cluster-node-tuning-operator/tuned-k7j9j" Jul 21 12:39:55.731355 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:39:55.730351 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/a2b48106-9844-4431-9b6f-d4e1baacb15b-host-slash\") pod \"ovnkube-node-6nf7d\" (UID: \"a2b48106-9844-4431-9b6f-d4e1baacb15b\") " pod="openshift-ovn-kubernetes/ovnkube-node-6nf7d" Jul 21 12:39:55.731355 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:39:55.730357 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/a2b48106-9844-4431-9b6f-d4e1baacb15b-env-overrides\") pod \"ovnkube-node-6nf7d\" (UID: \"a2b48106-9844-4431-9b6f-d4e1baacb15b\") " pod="openshift-ovn-kubernetes/ovnkube-node-6nf7d" Jul 21 12:39:55.731355 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:39:55.730368 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/490c41c0-4938-41c5-a264-1cf80533b229-host-var-lib-cni-multus\") pod \"multus-pgvgl\" (UID: \"490c41c0-4938-41c5-a264-1cf80533b229\") " pod="openshift-multus/multus-pgvgl" Jul 21 12:39:55.731355 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:39:55.730374 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/06df9492-bc31-44f9-9f87-1665786f3be4-kubelet-dir\") pod \"aws-ebs-csi-driver-node-wb7vw\" (UID: \"06df9492-bc31-44f9-9f87-1665786f3be4\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-wb7vw" Jul 21 12:39:55.731355 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:39:55.730407 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/a2b48106-9844-4431-9b6f-d4e1baacb15b-node-log\") pod \"ovnkube-node-6nf7d\" (UID: \"a2b48106-9844-4431-9b6f-d4e1baacb15b\") " pod="openshift-ovn-kubernetes/ovnkube-node-6nf7d" Jul 21 12:39:55.731355 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:39:55.730399 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/490c41c0-4938-41c5-a264-1cf80533b229-os-release\") pod \"multus-pgvgl\" (UID: \"490c41c0-4938-41c5-a264-1cf80533b229\") " pod="openshift-multus/multus-pgvgl" Jul 21 12:39:55.731355 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:39:55.730445 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5q68t\" (UniqueName: \"kubernetes.io/projected/490c41c0-4938-41c5-a264-1cf80533b229-kube-api-access-5q68t\") pod \"multus-pgvgl\" (UID: \"490c41c0-4938-41c5-a264-1cf80533b229\") " pod="openshift-multus/multus-pgvgl" Jul 21 12:39:55.731355 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:39:55.730450 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/1d597db9-aea4-46bd-a743-06fcb6bc24d2-var-lib-kubelet\") pod \"tuned-k7j9j\" (UID: \"1d597db9-aea4-46bd-a743-06fcb6bc24d2\") " pod="openshift-cluster-node-tuning-operator/tuned-k7j9j" Jul 21 12:39:55.731355 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:39:55.730458 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/a2b48106-9844-4431-9b6f-d4e1baacb15b-host-slash\") pod \"ovnkube-node-6nf7d\" (UID: \"a2b48106-9844-4431-9b6f-d4e1baacb15b\") " pod="openshift-ovn-kubernetes/ovnkube-node-6nf7d" Jul 21 12:39:55.731355 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:39:55.730472 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/a2b48106-9844-4431-9b6f-d4e1baacb15b-etc-openvswitch\") pod \"ovnkube-node-6nf7d\" (UID: \"a2b48106-9844-4431-9b6f-d4e1baacb15b\") " pod="openshift-ovn-kubernetes/ovnkube-node-6nf7d" Jul 21 12:39:55.731355 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:39:55.730498 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/490c41c0-4938-41c5-a264-1cf80533b229-host-run-k8s-cni-cncf-io\") pod \"multus-pgvgl\" (UID: \"490c41c0-4938-41c5-a264-1cf80533b229\") " pod="openshift-multus/multus-pgvgl" Jul 21 12:39:55.731355 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:39:55.730505 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/6713665c-3b71-4751-929e-16de48263766-konnectivity-ca\") pod \"konnectivity-agent-skc26\" (UID: \"6713665c-3b71-4751-929e-16de48263766\") " pod="kube-system/konnectivity-agent-skc26" Jul 21 12:39:55.731355 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:39:55.730505 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/06df9492-bc31-44f9-9f87-1665786f3be4-kubelet-dir\") pod \"aws-ebs-csi-driver-node-wb7vw\" (UID: \"06df9492-bc31-44f9-9f87-1665786f3be4\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-wb7vw" Jul 21 12:39:55.731355 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:39:55.730538 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/1d597db9-aea4-46bd-a743-06fcb6bc24d2-sys\") pod \"tuned-k7j9j\" (UID: \"1d597db9-aea4-46bd-a743-06fcb6bc24d2\") " pod="openshift-cluster-node-tuning-operator/tuned-k7j9j" Jul 21 12:39:55.731355 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:39:55.730549 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/490c41c0-4938-41c5-a264-1cf80533b229-os-release\") pod \"multus-pgvgl\" (UID: \"490c41c0-4938-41c5-a264-1cf80533b229\") " pod="openshift-multus/multus-pgvgl" Jul 21 12:39:55.731355 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:39:55.730528 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/490c41c0-4938-41c5-a264-1cf80533b229-multus-cni-dir\") pod \"multus-pgvgl\" (UID: \"490c41c0-4938-41c5-a264-1cf80533b229\") " pod="openshift-multus/multus-pgvgl" Jul 21 12:39:55.732267 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:39:55.730563 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/a2b48106-9844-4431-9b6f-d4e1baacb15b-etc-openvswitch\") pod \"ovnkube-node-6nf7d\" (UID: \"a2b48106-9844-4431-9b6f-d4e1baacb15b\") " pod="openshift-ovn-kubernetes/ovnkube-node-6nf7d" Jul 21 12:39:55.732267 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:39:55.730572 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/490c41c0-4938-41c5-a264-1cf80533b229-host-run-k8s-cni-cncf-io\") pod \"multus-pgvgl\" (UID: \"490c41c0-4938-41c5-a264-1cf80533b229\") " pod="openshift-multus/multus-pgvgl" Jul 21 12:39:55.732267 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:39:55.730581 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/64971bb0-35c9-4026-9680-2a7700efb382-cni-binary-copy\") pod \"multus-additional-cni-plugins-wnhr4\" (UID: \"64971bb0-35c9-4026-9680-2a7700efb382\") " pod="openshift-multus/multus-additional-cni-plugins-wnhr4" Jul 21 12:39:55.732267 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:39:55.730584 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/1d597db9-aea4-46bd-a743-06fcb6bc24d2-sys\") pod \"tuned-k7j9j\" (UID: \"1d597db9-aea4-46bd-a743-06fcb6bc24d2\") " pod="openshift-cluster-node-tuning-operator/tuned-k7j9j" Jul 21 12:39:55.732267 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:39:55.730610 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/64971bb0-35c9-4026-9680-2a7700efb382-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-wnhr4\" (UID: \"64971bb0-35c9-4026-9680-2a7700efb382\") " pod="openshift-multus/multus-additional-cni-plugins-wnhr4" Jul 21 12:39:55.732267 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:39:55.730668 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/a2b48106-9844-4431-9b6f-d4e1baacb15b-host-cni-netd\") pod \"ovnkube-node-6nf7d\" (UID: \"a2b48106-9844-4431-9b6f-d4e1baacb15b\") " pod="openshift-ovn-kubernetes/ovnkube-node-6nf7d" Jul 21 12:39:55.732267 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:39:55.730692 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/490c41c0-4938-41c5-a264-1cf80533b229-host-run-netns\") pod \"multus-pgvgl\" (UID: \"490c41c0-4938-41c5-a264-1cf80533b229\") " pod="openshift-multus/multus-pgvgl" Jul 21 12:39:55.732267 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:39:55.730733 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/1d597db9-aea4-46bd-a743-06fcb6bc24d2-etc-sysctl-d\") pod \"tuned-k7j9j\" (UID: \"1d597db9-aea4-46bd-a743-06fcb6bc24d2\") " pod="openshift-cluster-node-tuning-operator/tuned-k7j9j" Jul 21 12:39:55.732267 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:39:55.730723 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/a2b48106-9844-4431-9b6f-d4e1baacb15b-host-cni-netd\") pod \"ovnkube-node-6nf7d\" (UID: \"a2b48106-9844-4431-9b6f-d4e1baacb15b\") " pod="openshift-ovn-kubernetes/ovnkube-node-6nf7d" Jul 21 12:39:55.732267 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:39:55.730761 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/1d597db9-aea4-46bd-a743-06fcb6bc24d2-etc-systemd\") pod \"tuned-k7j9j\" (UID: \"1d597db9-aea4-46bd-a743-06fcb6bc24d2\") " pod="openshift-cluster-node-tuning-operator/tuned-k7j9j" Jul 21 12:39:55.732267 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:39:55.730765 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/490c41c0-4938-41c5-a264-1cf80533b229-host-run-netns\") pod \"multus-pgvgl\" (UID: \"490c41c0-4938-41c5-a264-1cf80533b229\") " pod="openshift-multus/multus-pgvgl" Jul 21 12:39:55.732267 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:39:55.730789 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/64971bb0-35c9-4026-9680-2a7700efb382-os-release\") pod \"multus-additional-cni-plugins-wnhr4\" (UID: \"64971bb0-35c9-4026-9680-2a7700efb382\") " pod="openshift-multus/multus-additional-cni-plugins-wnhr4" Jul 21 12:39:55.732267 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:39:55.730816 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/a2b48106-9844-4431-9b6f-d4e1baacb15b-host-cni-bin\") pod \"ovnkube-node-6nf7d\" (UID: \"a2b48106-9844-4431-9b6f-d4e1baacb15b\") " pod="openshift-ovn-kubernetes/ovnkube-node-6nf7d" Jul 21 12:39:55.732267 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:39:55.730840 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/06df9492-bc31-44f9-9f87-1665786f3be4-device-dir\") pod \"aws-ebs-csi-driver-node-wb7vw\" (UID: \"06df9492-bc31-44f9-9f87-1665786f3be4\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-wb7vw" Jul 21 12:39:55.732267 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:39:55.730866 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/4dd651ed-4103-426d-83cc-ea465d4e3d7b-host-slash\") pod \"iptables-alerter-dr85z\" (UID: \"4dd651ed-4103-426d-83cc-ea465d4e3d7b\") " pod="openshift-network-operator/iptables-alerter-dr85z" Jul 21 12:39:55.732267 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:39:55.730890 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/1d597db9-aea4-46bd-a743-06fcb6bc24d2-etc-sysctl-d\") pod \"tuned-k7j9j\" (UID: \"1d597db9-aea4-46bd-a743-06fcb6bc24d2\") " pod="openshift-cluster-node-tuning-operator/tuned-k7j9j" Jul 21 12:39:55.732267 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:39:55.730907 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/64971bb0-35c9-4026-9680-2a7700efb382-os-release\") pod \"multus-additional-cni-plugins-wnhr4\" (UID: \"64971bb0-35c9-4026-9680-2a7700efb382\") " pod="openshift-multus/multus-additional-cni-plugins-wnhr4" Jul 21 12:39:55.733106 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:39:55.730941 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/4dd651ed-4103-426d-83cc-ea465d4e3d7b-host-slash\") pod \"iptables-alerter-dr85z\" (UID: \"4dd651ed-4103-426d-83cc-ea465d4e3d7b\") " pod="openshift-network-operator/iptables-alerter-dr85z" Jul 21 12:39:55.733106 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:39:55.730944 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/06df9492-bc31-44f9-9f87-1665786f3be4-device-dir\") pod \"aws-ebs-csi-driver-node-wb7vw\" (UID: \"06df9492-bc31-44f9-9f87-1665786f3be4\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-wb7vw" Jul 21 12:39:55.733106 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:39:55.730955 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/1d597db9-aea4-46bd-a743-06fcb6bc24d2-etc-systemd\") pod \"tuned-k7j9j\" (UID: \"1d597db9-aea4-46bd-a743-06fcb6bc24d2\") " pod="openshift-cluster-node-tuning-operator/tuned-k7j9j" Jul 21 12:39:55.733106 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:39:55.730971 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-klh55\" (UniqueName: \"kubernetes.io/projected/4dd651ed-4103-426d-83cc-ea465d4e3d7b-kube-api-access-klh55\") pod \"iptables-alerter-dr85z\" (UID: \"4dd651ed-4103-426d-83cc-ea465d4e3d7b\") " pod="openshift-network-operator/iptables-alerter-dr85z" Jul 21 12:39:55.733106 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:39:55.730983 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/a2b48106-9844-4431-9b6f-d4e1baacb15b-host-cni-bin\") pod \"ovnkube-node-6nf7d\" (UID: \"a2b48106-9844-4431-9b6f-d4e1baacb15b\") " pod="openshift-ovn-kubernetes/ovnkube-node-6nf7d" Jul 21 12:39:55.733106 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:39:55.731030 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/64971bb0-35c9-4026-9680-2a7700efb382-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-wnhr4\" (UID: \"64971bb0-35c9-4026-9680-2a7700efb382\") " pod="openshift-multus/multus-additional-cni-plugins-wnhr4" Jul 21 12:39:55.733106 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:39:55.731131 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/a2b48106-9844-4431-9b6f-d4e1baacb15b-log-socket\") pod \"ovnkube-node-6nf7d\" (UID: \"a2b48106-9844-4431-9b6f-d4e1baacb15b\") " pod="openshift-ovn-kubernetes/ovnkube-node-6nf7d" Jul 21 12:39:55.733106 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:39:55.731134 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/64971bb0-35c9-4026-9680-2a7700efb382-cni-binary-copy\") pod \"multus-additional-cni-plugins-wnhr4\" (UID: \"64971bb0-35c9-4026-9680-2a7700efb382\") " pod="openshift-multus/multus-additional-cni-plugins-wnhr4" Jul 21 12:39:55.733106 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:39:55.731135 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/64971bb0-35c9-4026-9680-2a7700efb382-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-wnhr4\" (UID: \"64971bb0-35c9-4026-9680-2a7700efb382\") " pod="openshift-multus/multus-additional-cni-plugins-wnhr4" Jul 21 12:39:55.733106 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:39:55.731161 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/06df9492-bc31-44f9-9f87-1665786f3be4-socket-dir\") pod \"aws-ebs-csi-driver-node-wb7vw\" (UID: \"06df9492-bc31-44f9-9f87-1665786f3be4\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-wb7vw" Jul 21 12:39:55.733106 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:39:55.731192 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/a2b48106-9844-4431-9b6f-d4e1baacb15b-log-socket\") pod \"ovnkube-node-6nf7d\" (UID: \"a2b48106-9844-4431-9b6f-d4e1baacb15b\") " pod="openshift-ovn-kubernetes/ovnkube-node-6nf7d" Jul 21 12:39:55.733106 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:39:55.731241 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2x47f\" (UniqueName: \"kubernetes.io/projected/06df9492-bc31-44f9-9f87-1665786f3be4-kube-api-access-2x47f\") pod \"aws-ebs-csi-driver-node-wb7vw\" (UID: \"06df9492-bc31-44f9-9f87-1665786f3be4\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-wb7vw" Jul 21 12:39:55.733106 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:39:55.731276 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/490c41c0-4938-41c5-a264-1cf80533b229-multus-daemon-config\") pod \"multus-pgvgl\" (UID: \"490c41c0-4938-41c5-a264-1cf80533b229\") " pod="openshift-multus/multus-pgvgl" Jul 21 12:39:55.733106 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:39:55.731317 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/1d597db9-aea4-46bd-a743-06fcb6bc24d2-etc-kubernetes\") pod \"tuned-k7j9j\" (UID: \"1d597db9-aea4-46bd-a743-06fcb6bc24d2\") " pod="openshift-cluster-node-tuning-operator/tuned-k7j9j" Jul 21 12:39:55.733106 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:39:55.731338 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/06df9492-bc31-44f9-9f87-1665786f3be4-socket-dir\") pod \"aws-ebs-csi-driver-node-wb7vw\" (UID: \"06df9492-bc31-44f9-9f87-1665786f3be4\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-wb7vw" Jul 21 12:39:55.733106 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:39:55.731379 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/9cfd0727-8ed4-40bd-b1d5-66a531560bce-serviceca\") pod \"node-ca-km4fw\" (UID: \"9cfd0727-8ed4-40bd-b1d5-66a531560bce\") " pod="openshift-image-registry/node-ca-km4fw" Jul 21 12:39:55.733106 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:39:55.731420 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/1d597db9-aea4-46bd-a743-06fcb6bc24d2-etc-kubernetes\") pod \"tuned-k7j9j\" (UID: \"1d597db9-aea4-46bd-a743-06fcb6bc24d2\") " pod="openshift-cluster-node-tuning-operator/tuned-k7j9j" Jul 21 12:39:55.733966 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:39:55.731436 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/a2b48106-9844-4431-9b6f-d4e1baacb15b-run-openvswitch\") pod \"ovnkube-node-6nf7d\" (UID: \"a2b48106-9844-4431-9b6f-d4e1baacb15b\") " pod="openshift-ovn-kubernetes/ovnkube-node-6nf7d" Jul 21 12:39:55.733966 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:39:55.731464 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/a2b48106-9844-4431-9b6f-d4e1baacb15b-ovn-node-metrics-cert\") pod \"ovnkube-node-6nf7d\" (UID: \"a2b48106-9844-4431-9b6f-d4e1baacb15b\") " pod="openshift-ovn-kubernetes/ovnkube-node-6nf7d" Jul 21 12:39:55.733966 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:39:55.731467 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/a2b48106-9844-4431-9b6f-d4e1baacb15b-run-openvswitch\") pod \"ovnkube-node-6nf7d\" (UID: \"a2b48106-9844-4431-9b6f-d4e1baacb15b\") " pod="openshift-ovn-kubernetes/ovnkube-node-6nf7d" Jul 21 12:39:55.733966 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:39:55.731487 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/490c41c0-4938-41c5-a264-1cf80533b229-multus-socket-dir-parent\") pod \"multus-pgvgl\" (UID: \"490c41c0-4938-41c5-a264-1cf80533b229\") " pod="openshift-multus/multus-pgvgl" Jul 21 12:39:55.733966 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:39:55.731535 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/490c41c0-4938-41c5-a264-1cf80533b229-host-var-lib-kubelet\") pod \"multus-pgvgl\" (UID: \"490c41c0-4938-41c5-a264-1cf80533b229\") " pod="openshift-multus/multus-pgvgl" Jul 21 12:39:55.733966 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:39:55.731559 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/1d597db9-aea4-46bd-a743-06fcb6bc24d2-etc-tuned\") pod \"tuned-k7j9j\" (UID: \"1d597db9-aea4-46bd-a743-06fcb6bc24d2\") " pod="openshift-cluster-node-tuning-operator/tuned-k7j9j" Jul 21 12:39:55.733966 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:39:55.731564 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/64971bb0-35c9-4026-9680-2a7700efb382-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-wnhr4\" (UID: \"64971bb0-35c9-4026-9680-2a7700efb382\") " pod="openshift-multus/multus-additional-cni-plugins-wnhr4" Jul 21 12:39:55.733966 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:39:55.731584 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/490c41c0-4938-41c5-a264-1cf80533b229-host-var-lib-kubelet\") pod \"multus-pgvgl\" (UID: \"490c41c0-4938-41c5-a264-1cf80533b229\") " pod="openshift-multus/multus-pgvgl" Jul 21 12:39:55.733966 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:39:55.731584 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/490c41c0-4938-41c5-a264-1cf80533b229-cni-binary-copy\") pod \"multus-pgvgl\" (UID: \"490c41c0-4938-41c5-a264-1cf80533b229\") " pod="openshift-multus/multus-pgvgl" Jul 21 12:39:55.733966 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:39:55.731559 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/490c41c0-4938-41c5-a264-1cf80533b229-multus-socket-dir-parent\") pod \"multus-pgvgl\" (UID: \"490c41c0-4938-41c5-a264-1cf80533b229\") " pod="openshift-multus/multus-pgvgl" Jul 21 12:39:55.733966 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:39:55.731623 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/64971bb0-35c9-4026-9680-2a7700efb382-tuning-conf-dir\") pod \"multus-additional-cni-plugins-wnhr4\" (UID: \"64971bb0-35c9-4026-9680-2a7700efb382\") " pod="openshift-multus/multus-additional-cni-plugins-wnhr4" Jul 21 12:39:55.733966 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:39:55.731663 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/a2b48106-9844-4431-9b6f-d4e1baacb15b-ovnkube-script-lib\") pod \"ovnkube-node-6nf7d\" (UID: \"a2b48106-9844-4431-9b6f-d4e1baacb15b\") " pod="openshift-ovn-kubernetes/ovnkube-node-6nf7d" Jul 21 12:39:55.733966 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:39:55.731699 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/c70252ec-a8bb-4dd0-9544-2ff28e109eb9-hosts-file\") pod \"node-resolver-6crjb\" (UID: \"c70252ec-a8bb-4dd0-9544-2ff28e109eb9\") " pod="openshift-dns/node-resolver-6crjb" Jul 21 12:39:55.733966 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:39:55.731723 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/a2b48106-9844-4431-9b6f-d4e1baacb15b-host-run-ovn-kubernetes\") pod \"ovnkube-node-6nf7d\" (UID: \"a2b48106-9844-4431-9b6f-d4e1baacb15b\") " pod="openshift-ovn-kubernetes/ovnkube-node-6nf7d" Jul 21 12:39:55.733966 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:39:55.731746 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/a2b48106-9844-4431-9b6f-d4e1baacb15b-ovnkube-config\") pod \"ovnkube-node-6nf7d\" (UID: \"a2b48106-9844-4431-9b6f-d4e1baacb15b\") " pod="openshift-ovn-kubernetes/ovnkube-node-6nf7d" Jul 21 12:39:55.733966 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:39:55.731800 2576 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory" Jul 21 12:39:55.733966 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:39:55.731821 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/c70252ec-a8bb-4dd0-9544-2ff28e109eb9-hosts-file\") pod \"node-resolver-6crjb\" (UID: \"c70252ec-a8bb-4dd0-9544-2ff28e109eb9\") " pod="openshift-dns/node-resolver-6crjb" Jul 21 12:39:55.733966 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:39:55.731870 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/a2b48106-9844-4431-9b6f-d4e1baacb15b-host-run-ovn-kubernetes\") pod \"ovnkube-node-6nf7d\" (UID: \"a2b48106-9844-4431-9b6f-d4e1baacb15b\") " pod="openshift-ovn-kubernetes/ovnkube-node-6nf7d" Jul 21 12:39:55.734830 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:39:55.731882 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/64971bb0-35c9-4026-9680-2a7700efb382-tuning-conf-dir\") pod \"multus-additional-cni-plugins-wnhr4\" (UID: \"64971bb0-35c9-4026-9680-2a7700efb382\") " pod="openshift-multus/multus-additional-cni-plugins-wnhr4" Jul 21 12:39:55.734830 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:39:55.731899 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/490c41c0-4938-41c5-a264-1cf80533b229-multus-daemon-config\") pod \"multus-pgvgl\" (UID: \"490c41c0-4938-41c5-a264-1cf80533b229\") " pod="openshift-multus/multus-pgvgl" Jul 21 12:39:55.734830 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:39:55.731916 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/1d597db9-aea4-46bd-a743-06fcb6bc24d2-etc-sysconfig\") pod \"tuned-k7j9j\" (UID: \"1d597db9-aea4-46bd-a743-06fcb6bc24d2\") " pod="openshift-cluster-node-tuning-operator/tuned-k7j9j" Jul 21 12:39:55.734830 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:39:55.731827 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/1d597db9-aea4-46bd-a743-06fcb6bc24d2-etc-sysconfig\") pod \"tuned-k7j9j\" (UID: \"1d597db9-aea4-46bd-a743-06fcb6bc24d2\") " pod="openshift-cluster-node-tuning-operator/tuned-k7j9j" Jul 21 12:39:55.734830 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:39:55.731940 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/9cfd0727-8ed4-40bd-b1d5-66a531560bce-serviceca\") pod \"node-ca-km4fw\" (UID: \"9cfd0727-8ed4-40bd-b1d5-66a531560bce\") " pod="openshift-image-registry/node-ca-km4fw" Jul 21 12:39:55.734830 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:39:55.731951 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/4dd651ed-4103-426d-83cc-ea465d4e3d7b-iptables-alerter-script\") pod \"iptables-alerter-dr85z\" (UID: \"4dd651ed-4103-426d-83cc-ea465d4e3d7b\") " pod="openshift-network-operator/iptables-alerter-dr85z" Jul 21 12:39:55.734830 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:39:55.731979 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/6713665c-3b71-4751-929e-16de48263766-agent-certs\") pod \"konnectivity-agent-skc26\" (UID: \"6713665c-3b71-4751-929e-16de48263766\") " pod="kube-system/konnectivity-agent-skc26" Jul 21 12:39:55.734830 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:39:55.732109 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/490c41c0-4938-41c5-a264-1cf80533b229-cni-binary-copy\") pod \"multus-pgvgl\" (UID: \"490c41c0-4938-41c5-a264-1cf80533b229\") " pod="openshift-multus/multus-pgvgl" Jul 21 12:39:55.734830 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:39:55.732136 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/06df9492-bc31-44f9-9f87-1665786f3be4-etc-selinux\") pod \"aws-ebs-csi-driver-node-wb7vw\" (UID: \"06df9492-bc31-44f9-9f87-1665786f3be4\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-wb7vw" Jul 21 12:39:55.734830 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:39:55.732165 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/490c41c0-4938-41c5-a264-1cf80533b229-cnibin\") pod \"multus-pgvgl\" (UID: \"490c41c0-4938-41c5-a264-1cf80533b229\") " pod="openshift-multus/multus-pgvgl" Jul 21 12:39:55.734830 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:39:55.732185 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d932e6af-610e-4967-87b1-e489cead334d-metrics-certs\") pod \"network-metrics-daemon-l9rfj\" (UID: \"d932e6af-610e-4967-87b1-e489cead334d\") " pod="openshift-multus/network-metrics-daemon-l9rfj" Jul 21 12:39:55.734830 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:39:55.732203 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/1d597db9-aea4-46bd-a743-06fcb6bc24d2-etc-sysctl-conf\") pod \"tuned-k7j9j\" (UID: \"1d597db9-aea4-46bd-a743-06fcb6bc24d2\") " pod="openshift-cluster-node-tuning-operator/tuned-k7j9j" Jul 21 12:39:55.734830 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:39:55.732224 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hs9gs\" (UniqueName: \"kubernetes.io/projected/1d597db9-aea4-46bd-a743-06fcb6bc24d2-kube-api-access-hs9gs\") pod \"tuned-k7j9j\" (UID: \"1d597db9-aea4-46bd-a743-06fcb6bc24d2\") " pod="openshift-cluster-node-tuning-operator/tuned-k7j9j" Jul 21 12:39:55.734830 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:39:55.732246 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kvc8f\" (UniqueName: \"kubernetes.io/projected/ea2d55a6-d43a-4d57-b49c-b63904d8810e-kube-api-access-kvc8f\") pod \"network-check-target-6f57m\" (UID: \"ea2d55a6-d43a-4d57-b49c-b63904d8810e\") " pod="openshift-network-diagnostics/network-check-target-6f57m" Jul 21 12:39:55.734830 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:39:55.732258 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/490c41c0-4938-41c5-a264-1cf80533b229-cnibin\") pod \"multus-pgvgl\" (UID: \"490c41c0-4938-41c5-a264-1cf80533b229\") " pod="openshift-multus/multus-pgvgl" Jul 21 12:39:55.734830 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:39:55.732265 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/64971bb0-35c9-4026-9680-2a7700efb382-system-cni-dir\") pod \"multus-additional-cni-plugins-wnhr4\" (UID: \"64971bb0-35c9-4026-9680-2a7700efb382\") " pod="openshift-multus/multus-additional-cni-plugins-wnhr4" Jul 21 12:39:55.734830 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:39:55.732287 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/a2b48106-9844-4431-9b6f-d4e1baacb15b-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-6nf7d\" (UID: \"a2b48106-9844-4431-9b6f-d4e1baacb15b\") " pod="openshift-ovn-kubernetes/ovnkube-node-6nf7d" Jul 21 12:39:55.735595 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:39:55.732321 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/490c41c0-4938-41c5-a264-1cf80533b229-host-var-lib-cni-bin\") pod \"multus-pgvgl\" (UID: \"490c41c0-4938-41c5-a264-1cf80533b229\") " pod="openshift-multus/multus-pgvgl" Jul 21 12:39:55.735595 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:39:55.732341 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/490c41c0-4938-41c5-a264-1cf80533b229-multus-conf-dir\") pod \"multus-pgvgl\" (UID: \"490c41c0-4938-41c5-a264-1cf80533b229\") " pod="openshift-multus/multus-pgvgl" Jul 21 12:39:55.735595 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:39:55.732343 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/06df9492-bc31-44f9-9f87-1665786f3be4-etc-selinux\") pod \"aws-ebs-csi-driver-node-wb7vw\" (UID: \"06df9492-bc31-44f9-9f87-1665786f3be4\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-wb7vw" Jul 21 12:39:55.735595 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:39:55.732348 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/a2b48106-9844-4431-9b6f-d4e1baacb15b-ovnkube-script-lib\") pod \"ovnkube-node-6nf7d\" (UID: \"a2b48106-9844-4431-9b6f-d4e1baacb15b\") " pod="openshift-ovn-kubernetes/ovnkube-node-6nf7d" Jul 21 12:39:55.735595 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:39:55.732362 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/1d597db9-aea4-46bd-a743-06fcb6bc24d2-lib-modules\") pod \"tuned-k7j9j\" (UID: \"1d597db9-aea4-46bd-a743-06fcb6bc24d2\") " pod="openshift-cluster-node-tuning-operator/tuned-k7j9j" Jul 21 12:39:55.735595 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:39:55.732383 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/1d597db9-aea4-46bd-a743-06fcb6bc24d2-host\") pod \"tuned-k7j9j\" (UID: \"1d597db9-aea4-46bd-a743-06fcb6bc24d2\") " pod="openshift-cluster-node-tuning-operator/tuned-k7j9j" Jul 21 12:39:55.735595 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:39:55.732434 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/1d597db9-aea4-46bd-a743-06fcb6bc24d2-host\") pod \"tuned-k7j9j\" (UID: \"1d597db9-aea4-46bd-a743-06fcb6bc24d2\") " pod="openshift-cluster-node-tuning-operator/tuned-k7j9j" Jul 21 12:39:55.735595 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:39:55.732472 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/64971bb0-35c9-4026-9680-2a7700efb382-system-cni-dir\") pod \"multus-additional-cni-plugins-wnhr4\" (UID: \"64971bb0-35c9-4026-9680-2a7700efb382\") " pod="openshift-multus/multus-additional-cni-plugins-wnhr4" Jul 21 12:39:55.735595 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:39:55.732483 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/a2b48106-9844-4431-9b6f-d4e1baacb15b-ovnkube-config\") pod \"ovnkube-node-6nf7d\" (UID: \"a2b48106-9844-4431-9b6f-d4e1baacb15b\") " pod="openshift-ovn-kubernetes/ovnkube-node-6nf7d" Jul 21 12:39:55.735595 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:39:55.732492 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/4dd651ed-4103-426d-83cc-ea465d4e3d7b-iptables-alerter-script\") pod \"iptables-alerter-dr85z\" (UID: \"4dd651ed-4103-426d-83cc-ea465d4e3d7b\") " pod="openshift-network-operator/iptables-alerter-dr85z" Jul 21 12:39:55.735595 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:39:55.732504 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/1d597db9-aea4-46bd-a743-06fcb6bc24d2-etc-sysctl-conf\") pod \"tuned-k7j9j\" (UID: \"1d597db9-aea4-46bd-a743-06fcb6bc24d2\") " pod="openshift-cluster-node-tuning-operator/tuned-k7j9j" Jul 21 12:39:55.735595 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:39:55.732510 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/a2b48106-9844-4431-9b6f-d4e1baacb15b-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-6nf7d\" (UID: \"a2b48106-9844-4431-9b6f-d4e1baacb15b\") " pod="openshift-ovn-kubernetes/ovnkube-node-6nf7d" Jul 21 12:39:55.735595 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:39:55.732569 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/490c41c0-4938-41c5-a264-1cf80533b229-host-var-lib-cni-bin\") pod \"multus-pgvgl\" (UID: \"490c41c0-4938-41c5-a264-1cf80533b229\") " pod="openshift-multus/multus-pgvgl" Jul 21 12:39:55.735595 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:39:55.732582 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/490c41c0-4938-41c5-a264-1cf80533b229-multus-conf-dir\") pod \"multus-pgvgl\" (UID: \"490c41c0-4938-41c5-a264-1cf80533b229\") " pod="openshift-multus/multus-pgvgl" Jul 21 12:39:55.735595 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:39:55.732567 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/4dc9bfad-f2bb-4934-9f49-4cfd0f381ab2-kubelet-config\") pod \"global-pull-secret-syncer-g2tkf\" (UID: \"4dc9bfad-f2bb-4934-9f49-4cfd0f381ab2\") " pod="kube-system/global-pull-secret-syncer-g2tkf" Jul 21 12:39:55.735595 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:39:55.732625 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/a2b48106-9844-4431-9b6f-d4e1baacb15b-host-run-netns\") pod \"ovnkube-node-6nf7d\" (UID: \"a2b48106-9844-4431-9b6f-d4e1baacb15b\") " pod="openshift-ovn-kubernetes/ovnkube-node-6nf7d" Jul 21 12:39:55.735595 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:39:55.732630 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/4dc9bfad-f2bb-4934-9f49-4cfd0f381ab2-kubelet-config\") pod \"global-pull-secret-syncer-g2tkf\" (UID: \"4dc9bfad-f2bb-4934-9f49-4cfd0f381ab2\") " pod="kube-system/global-pull-secret-syncer-g2tkf" Jul 21 12:39:55.736096 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:39:55.732650 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/1d597db9-aea4-46bd-a743-06fcb6bc24d2-lib-modules\") pod \"tuned-k7j9j\" (UID: \"1d597db9-aea4-46bd-a743-06fcb6bc24d2\") " pod="openshift-cluster-node-tuning-operator/tuned-k7j9j" Jul 21 12:39:55.736096 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:39:55.732659 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/490c41c0-4938-41c5-a264-1cf80533b229-host-run-multus-certs\") pod \"multus-pgvgl\" (UID: \"490c41c0-4938-41c5-a264-1cf80533b229\") " pod="openshift-multus/multus-pgvgl" Jul 21 12:39:55.736096 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:39:55.732665 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/a2b48106-9844-4431-9b6f-d4e1baacb15b-host-run-netns\") pod \"ovnkube-node-6nf7d\" (UID: \"a2b48106-9844-4431-9b6f-d4e1baacb15b\") " pod="openshift-ovn-kubernetes/ovnkube-node-6nf7d" Jul 21 12:39:55.736096 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:39:55.732697 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/490c41c0-4938-41c5-a264-1cf80533b229-etc-kubernetes\") pod \"multus-pgvgl\" (UID: \"490c41c0-4938-41c5-a264-1cf80533b229\") " pod="openshift-multus/multus-pgvgl" Jul 21 12:39:55.736096 ip-10-0-130-157 kubenswrapper[2576]: E0721 12:39:55.732714 2576 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 21 12:39:55.736096 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:39:55.732720 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/1d597db9-aea4-46bd-a743-06fcb6bc24d2-tmp\") pod \"tuned-k7j9j\" (UID: \"1d597db9-aea4-46bd-a743-06fcb6bc24d2\") " pod="openshift-cluster-node-tuning-operator/tuned-k7j9j" Jul 21 12:39:55.736096 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:39:55.732779 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/490c41c0-4938-41c5-a264-1cf80533b229-etc-kubernetes\") pod \"multus-pgvgl\" (UID: \"490c41c0-4938-41c5-a264-1cf80533b229\") " pod="openshift-multus/multus-pgvgl" Jul 21 12:39:55.736096 ip-10-0-130-157 kubenswrapper[2576]: E0721 12:39:55.732798 2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d932e6af-610e-4967-87b1-e489cead334d-metrics-certs podName:d932e6af-610e-4967-87b1-e489cead334d nodeName:}" failed. No retries permitted until 2026-07-21 12:39:56.232765844 +0000 UTC m=+4.068029718 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/d932e6af-610e-4967-87b1-e489cead334d-metrics-certs") pod "network-metrics-daemon-l9rfj" (UID: "d932e6af-610e-4967-87b1-e489cead334d") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 21 12:39:55.736096 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:39:55.732815 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/4dc9bfad-f2bb-4934-9f49-4cfd0f381ab2-original-pull-secret\") pod \"global-pull-secret-syncer-g2tkf\" (UID: \"4dc9bfad-f2bb-4934-9f49-4cfd0f381ab2\") " pod="kube-system/global-pull-secret-syncer-g2tkf" Jul 21 12:39:55.736096 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:39:55.732841 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lwd2d\" (UniqueName: \"kubernetes.io/projected/c70252ec-a8bb-4dd0-9544-2ff28e109eb9-kube-api-access-lwd2d\") pod \"node-resolver-6crjb\" (UID: \"c70252ec-a8bb-4dd0-9544-2ff28e109eb9\") " pod="openshift-dns/node-resolver-6crjb" Jul 21 12:39:55.736096 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:39:55.732862 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/06df9492-bc31-44f9-9f87-1665786f3be4-registration-dir\") pod \"aws-ebs-csi-driver-node-wb7vw\" (UID: \"06df9492-bc31-44f9-9f87-1665786f3be4\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-wb7vw" Jul 21 12:39:55.736096 ip-10-0-130-157 kubenswrapper[2576]: E0721 12:39:55.732874 2576 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 21 12:39:55.736096 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:39:55.732881 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/06df9492-bc31-44f9-9f87-1665786f3be4-sys-fs\") pod \"aws-ebs-csi-driver-node-wb7vw\" (UID: \"06df9492-bc31-44f9-9f87-1665786f3be4\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-wb7vw" Jul 21 12:39:55.736096 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:39:55.732905 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/490c41c0-4938-41c5-a264-1cf80533b229-hostroot\") pod \"multus-pgvgl\" (UID: \"490c41c0-4938-41c5-a264-1cf80533b229\") " pod="openshift-multus/multus-pgvgl" Jul 21 12:39:55.736096 ip-10-0-130-157 kubenswrapper[2576]: E0721 12:39:55.732935 2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4dc9bfad-f2bb-4934-9f49-4cfd0f381ab2-original-pull-secret podName:4dc9bfad-f2bb-4934-9f49-4cfd0f381ab2 nodeName:}" failed. No retries permitted until 2026-07-21 12:39:56.232919773 +0000 UTC m=+4.068183618 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/4dc9bfad-f2bb-4934-9f49-4cfd0f381ab2-original-pull-secret") pod "global-pull-secret-syncer-g2tkf" (UID: "4dc9bfad-f2bb-4934-9f49-4cfd0f381ab2") : object "kube-system"/"original-pull-secret" not registered Jul 21 12:39:55.736096 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:39:55.732941 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/490c41c0-4938-41c5-a264-1cf80533b229-hostroot\") pod \"multus-pgvgl\" (UID: \"490c41c0-4938-41c5-a264-1cf80533b229\") " pod="openshift-multus/multus-pgvgl" Jul 21 12:39:55.736096 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:39:55.732955 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/1d597db9-aea4-46bd-a743-06fcb6bc24d2-run\") pod \"tuned-k7j9j\" (UID: \"1d597db9-aea4-46bd-a743-06fcb6bc24d2\") " pod="openshift-cluster-node-tuning-operator/tuned-k7j9j" Jul 21 12:39:55.736670 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:39:55.732982 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/4dc9bfad-f2bb-4934-9f49-4cfd0f381ab2-dbus\") pod \"global-pull-secret-syncer-g2tkf\" (UID: \"4dc9bfad-f2bb-4934-9f49-4cfd0f381ab2\") " pod="kube-system/global-pull-secret-syncer-g2tkf" Jul 21 12:39:55.736670 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:39:55.732990 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/06df9492-bc31-44f9-9f87-1665786f3be4-sys-fs\") pod \"aws-ebs-csi-driver-node-wb7vw\" (UID: \"06df9492-bc31-44f9-9f87-1665786f3be4\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-wb7vw" Jul 21 12:39:55.736670 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:39:55.733005 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/64971bb0-35c9-4026-9680-2a7700efb382-cnibin\") pod \"multus-additional-cni-plugins-wnhr4\" (UID: \"64971bb0-35c9-4026-9680-2a7700efb382\") " pod="openshift-multus/multus-additional-cni-plugins-wnhr4" Jul 21 12:39:55.736670 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:39:55.733029 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/a2b48106-9844-4431-9b6f-d4e1baacb15b-host-kubelet\") pod \"ovnkube-node-6nf7d\" (UID: \"a2b48106-9844-4431-9b6f-d4e1baacb15b\") " pod="openshift-ovn-kubernetes/ovnkube-node-6nf7d" Jul 21 12:39:55.736670 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:39:55.733034 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/06df9492-bc31-44f9-9f87-1665786f3be4-registration-dir\") pod \"aws-ebs-csi-driver-node-wb7vw\" (UID: \"06df9492-bc31-44f9-9f87-1665786f3be4\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-wb7vw" Jul 21 12:39:55.736670 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:39:55.733054 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/a2b48106-9844-4431-9b6f-d4e1baacb15b-run-systemd\") pod \"ovnkube-node-6nf7d\" (UID: \"a2b48106-9844-4431-9b6f-d4e1baacb15b\") " pod="openshift-ovn-kubernetes/ovnkube-node-6nf7d" Jul 21 12:39:55.736670 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:39:55.733083 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/1d597db9-aea4-46bd-a743-06fcb6bc24d2-run\") pod \"tuned-k7j9j\" (UID: \"1d597db9-aea4-46bd-a743-06fcb6bc24d2\") " pod="openshift-cluster-node-tuning-operator/tuned-k7j9j" Jul 21 12:39:55.736670 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:39:55.732817 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/490c41c0-4938-41c5-a264-1cf80533b229-host-run-multus-certs\") pod \"multus-pgvgl\" (UID: \"490c41c0-4938-41c5-a264-1cf80533b229\") " pod="openshift-multus/multus-pgvgl" Jul 21 12:39:55.736670 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:39:55.733094 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/64971bb0-35c9-4026-9680-2a7700efb382-cnibin\") pod \"multus-additional-cni-plugins-wnhr4\" (UID: \"64971bb0-35c9-4026-9680-2a7700efb382\") " pod="openshift-multus/multus-additional-cni-plugins-wnhr4" Jul 21 12:39:55.736670 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:39:55.733078 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/1d597db9-aea4-46bd-a743-06fcb6bc24d2-etc-modprobe-d\") pod \"tuned-k7j9j\" (UID: \"1d597db9-aea4-46bd-a743-06fcb6bc24d2\") " pod="openshift-cluster-node-tuning-operator/tuned-k7j9j" Jul 21 12:39:55.736670 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:39:55.733124 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/a2b48106-9844-4431-9b6f-d4e1baacb15b-host-kubelet\") pod \"ovnkube-node-6nf7d\" (UID: \"a2b48106-9844-4431-9b6f-d4e1baacb15b\") " pod="openshift-ovn-kubernetes/ovnkube-node-6nf7d" Jul 21 12:39:55.736670 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:39:55.733152 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/4dc9bfad-f2bb-4934-9f49-4cfd0f381ab2-dbus\") pod \"global-pull-secret-syncer-g2tkf\" (UID: \"4dc9bfad-f2bb-4934-9f49-4cfd0f381ab2\") " pod="kube-system/global-pull-secret-syncer-g2tkf" Jul 21 12:39:55.736670 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:39:55.733213 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/a2b48106-9844-4431-9b6f-d4e1baacb15b-run-systemd\") pod \"ovnkube-node-6nf7d\" (UID: \"a2b48106-9844-4431-9b6f-d4e1baacb15b\") " pod="openshift-ovn-kubernetes/ovnkube-node-6nf7d" Jul 21 12:39:55.736670 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:39:55.733233 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/1d597db9-aea4-46bd-a743-06fcb6bc24d2-etc-modprobe-d\") pod \"tuned-k7j9j\" (UID: \"1d597db9-aea4-46bd-a743-06fcb6bc24d2\") " pod="openshift-cluster-node-tuning-operator/tuned-k7j9j" Jul 21 12:39:55.736670 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:39:55.735176 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/a2b48106-9844-4431-9b6f-d4e1baacb15b-ovn-node-metrics-cert\") pod \"ovnkube-node-6nf7d\" (UID: \"a2b48106-9844-4431-9b6f-d4e1baacb15b\") " pod="openshift-ovn-kubernetes/ovnkube-node-6nf7d" Jul 21 12:39:55.736670 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:39:55.735190 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/1d597db9-aea4-46bd-a743-06fcb6bc24d2-tmp\") pod \"tuned-k7j9j\" (UID: \"1d597db9-aea4-46bd-a743-06fcb6bc24d2\") " pod="openshift-cluster-node-tuning-operator/tuned-k7j9j" Jul 21 12:39:55.736670 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:39:55.735447 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/6713665c-3b71-4751-929e-16de48263766-agent-certs\") pod \"konnectivity-agent-skc26\" (UID: \"6713665c-3b71-4751-929e-16de48263766\") " pod="kube-system/konnectivity-agent-skc26" Jul 21 12:39:55.736670 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:39:55.735657 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/1d597db9-aea4-46bd-a743-06fcb6bc24d2-etc-tuned\") pod \"tuned-k7j9j\" (UID: \"1d597db9-aea4-46bd-a743-06fcb6bc24d2\") " pod="openshift-cluster-node-tuning-operator/tuned-k7j9j" Jul 21 12:39:55.742029 ip-10-0-130-157 kubenswrapper[2576]: E0721 12:39:55.741999 2576 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 21 12:39:55.742029 ip-10-0-130-157 kubenswrapper[2576]: E0721 12:39:55.742034 2576 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 21 12:39:55.742214 ip-10-0-130-157 kubenswrapper[2576]: E0721 12:39:55.742050 2576 projected.go:196] Error preparing data for projected volume kube-api-access-kvc8f for pod openshift-network-diagnostics/network-check-target-6f57m: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 21 12:39:55.742214 ip-10-0-130-157 kubenswrapper[2576]: E0721 12:39:55.742124 2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/ea2d55a6-d43a-4d57-b49c-b63904d8810e-kube-api-access-kvc8f podName:ea2d55a6-d43a-4d57-b49c-b63904d8810e nodeName:}" failed. No retries permitted until 2026-07-21 12:39:56.242103767 +0000 UTC m=+4.077367632 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-kvc8f" (UniqueName: "kubernetes.io/projected/ea2d55a6-d43a-4d57-b49c-b63904d8810e-kube-api-access-kvc8f") pod "network-check-target-6f57m" (UID: "ea2d55a6-d43a-4d57-b49c-b63904d8810e") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 21 12:39:55.743840 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:39:55.743753 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4d7vz\" (UniqueName: \"kubernetes.io/projected/64971bb0-35c9-4026-9680-2a7700efb382-kube-api-access-4d7vz\") pod \"multus-additional-cni-plugins-wnhr4\" (UID: \"64971bb0-35c9-4026-9680-2a7700efb382\") " pod="openshift-multus/multus-additional-cni-plugins-wnhr4" Jul 21 12:39:55.743840 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:39:55.743790 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5q68t\" (UniqueName: \"kubernetes.io/projected/490c41c0-4938-41c5-a264-1cf80533b229-kube-api-access-5q68t\") pod \"multus-pgvgl\" (UID: \"490c41c0-4938-41c5-a264-1cf80533b229\") " pod="openshift-multus/multus-pgvgl" Jul 21 12:39:55.744003 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:39:55.743885 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wc78j\" (UniqueName: \"kubernetes.io/projected/9cfd0727-8ed4-40bd-b1d5-66a531560bce-kube-api-access-wc78j\") pod \"node-ca-km4fw\" (UID: \"9cfd0727-8ed4-40bd-b1d5-66a531560bce\") " pod="openshift-image-registry/node-ca-km4fw" Jul 21 12:39:55.744003 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:39:55.743975 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2x47f\" (UniqueName: \"kubernetes.io/projected/06df9492-bc31-44f9-9f87-1665786f3be4-kube-api-access-2x47f\") pod \"aws-ebs-csi-driver-node-wb7vw\" (UID: \"06df9492-bc31-44f9-9f87-1665786f3be4\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-wb7vw" Jul 21 12:39:55.744075 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:39:55.743998 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-klh55\" (UniqueName: \"kubernetes.io/projected/4dd651ed-4103-426d-83cc-ea465d4e3d7b-kube-api-access-klh55\") pod \"iptables-alerter-dr85z\" (UID: \"4dd651ed-4103-426d-83cc-ea465d4e3d7b\") " pod="openshift-network-operator/iptables-alerter-dr85z" Jul 21 12:39:55.744075 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:39:55.744021 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zkg7v\" (UniqueName: \"kubernetes.io/projected/d932e6af-610e-4967-87b1-e489cead334d-kube-api-access-zkg7v\") pod \"network-metrics-daemon-l9rfj\" (UID: \"d932e6af-610e-4967-87b1-e489cead334d\") " pod="openshift-multus/network-metrics-daemon-l9rfj" Jul 21 12:39:55.744582 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:39:55.744562 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ffzgq\" (UniqueName: \"kubernetes.io/projected/a2b48106-9844-4431-9b6f-d4e1baacb15b-kube-api-access-ffzgq\") pod \"ovnkube-node-6nf7d\" (UID: \"a2b48106-9844-4431-9b6f-d4e1baacb15b\") " pod="openshift-ovn-kubernetes/ovnkube-node-6nf7d" Jul 21 12:39:55.745137 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:39:55.745120 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hs9gs\" (UniqueName: \"kubernetes.io/projected/1d597db9-aea4-46bd-a743-06fcb6bc24d2-kube-api-access-hs9gs\") pod \"tuned-k7j9j\" (UID: \"1d597db9-aea4-46bd-a743-06fcb6bc24d2\") " pod="openshift-cluster-node-tuning-operator/tuned-k7j9j" Jul 21 12:39:55.745351 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:39:55.745334 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lwd2d\" (UniqueName: \"kubernetes.io/projected/c70252ec-a8bb-4dd0-9544-2ff28e109eb9-kube-api-access-lwd2d\") pod \"node-resolver-6crjb\" (UID: \"c70252ec-a8bb-4dd0-9544-2ff28e109eb9\") " pod="openshift-dns/node-resolver-6crjb" Jul 21 12:39:55.912221 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:39:55.912140 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-skc26" Jul 21 12:39:55.919979 ip-10-0-130-157 kubenswrapper[2576]: W0721 12:39:55.919942 2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6713665c_3b71_4751_929e_16de48263766.slice/crio-548c96075e1188986cc0f699ddfca2a880b73b68cef7e70018a31ac3c5f07dd5 WatchSource:0}: Error finding container 548c96075e1188986cc0f699ddfca2a880b73b68cef7e70018a31ac3c5f07dd5: Status 404 returned error can't find the container with id 548c96075e1188986cc0f699ddfca2a880b73b68cef7e70018a31ac3c5f07dd5 Jul 21 12:39:55.920110 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:39:55.919996 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-wb7vw" Jul 21 12:39:55.926483 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:39:55.926462 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-6crjb" Jul 21 12:39:55.927444 ip-10-0-130-157 kubenswrapper[2576]: W0721 12:39:55.927416 2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod06df9492_bc31_44f9_9f87_1665786f3be4.slice/crio-dc210945db7a93c3276170218feeac5a97f78bf68d771bf93e91066a64a244f7 WatchSource:0}: Error finding container dc210945db7a93c3276170218feeac5a97f78bf68d771bf93e91066a64a244f7: Status 404 returned error can't find the container with id dc210945db7a93c3276170218feeac5a97f78bf68d771bf93e91066a64a244f7 Jul 21 12:39:55.931343 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:39:55.931324 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-wnhr4" Jul 21 12:39:55.935500 ip-10-0-130-157 kubenswrapper[2576]: W0721 12:39:55.935474 2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc70252ec_a8bb_4dd0_9544_2ff28e109eb9.slice/crio-1e158c7a7b18a3133bef09f0682e65a5b8c64d6355e09891215e60c317143dd7 WatchSource:0}: Error finding container 1e158c7a7b18a3133bef09f0682e65a5b8c64d6355e09891215e60c317143dd7: Status 404 returned error can't find the container with id 1e158c7a7b18a3133bef09f0682e65a5b8c64d6355e09891215e60c317143dd7 Jul 21 12:39:55.936272 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:39:55.936252 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-pgvgl" Jul 21 12:39:55.940103 ip-10-0-130-157 kubenswrapper[2576]: W0721 12:39:55.940049 2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod64971bb0_35c9_4026_9680_2a7700efb382.slice/crio-a94720530444f4f2f36db2913014afce6cdcc37c94558768295ae341857df8fe WatchSource:0}: Error finding container a94720530444f4f2f36db2913014afce6cdcc37c94558768295ae341857df8fe: Status 404 returned error can't find the container with id a94720530444f4f2f36db2913014afce6cdcc37c94558768295ae341857df8fe Jul 21 12:39:55.941842 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:39:55.941644 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-k7j9j" Jul 21 12:39:55.946023 ip-10-0-130-157 kubenswrapper[2576]: W0721 12:39:55.945989 2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod490c41c0_4938_41c5_a264_1cf80533b229.slice/crio-081308fb5edc491223626bbdb61505aa9774fa3f0bc804d261888cbacdfc1236 WatchSource:0}: Error finding container 081308fb5edc491223626bbdb61505aa9774fa3f0bc804d261888cbacdfc1236: Status 404 returned error can't find the container with id 081308fb5edc491223626bbdb61505aa9774fa3f0bc804d261888cbacdfc1236 Jul 21 12:39:55.946844 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:39:55.946825 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-km4fw" Jul 21 12:39:55.951166 ip-10-0-130-157 kubenswrapper[2576]: W0721 12:39:55.951144 2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1d597db9_aea4_46bd_a743_06fcb6bc24d2.slice/crio-a5749d37c3be767552b676e1ac271a5d7fc5317c3a8f91b0c16a2e138d83d77f WatchSource:0}: Error finding container a5749d37c3be767552b676e1ac271a5d7fc5317c3a8f91b0c16a2e138d83d77f: Status 404 returned error can't find the container with id a5749d37c3be767552b676e1ac271a5d7fc5317c3a8f91b0c16a2e138d83d77f Jul 21 12:39:55.953319 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:39:55.953296 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-dr85z" Jul 21 12:39:55.957958 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:39:55.957935 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-6nf7d" Jul 21 12:39:55.958132 ip-10-0-130-157 kubenswrapper[2576]: W0721 12:39:55.958106 2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9cfd0727_8ed4_40bd_b1d5_66a531560bce.slice/crio-1c9cbbe3b57dfff1c1fd7dd47039bc1b4ac1bb589c4ab20bc637f9377b10caea WatchSource:0}: Error finding container 1c9cbbe3b57dfff1c1fd7dd47039bc1b4ac1bb589c4ab20bc637f9377b10caea: Status 404 returned error can't find the container with id 1c9cbbe3b57dfff1c1fd7dd47039bc1b4ac1bb589c4ab20bc637f9377b10caea Jul 21 12:39:55.962273 ip-10-0-130-157 kubenswrapper[2576]: W0721 12:39:55.962252 2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4dd651ed_4103_426d_83cc_ea465d4e3d7b.slice/crio-59bc20611ad70b148d0b7760deafbb82e1293784450e771a0acc990891f9bc90 WatchSource:0}: Error finding container 59bc20611ad70b148d0b7760deafbb82e1293784450e771a0acc990891f9bc90: Status 404 returned error can't find the container with id 59bc20611ad70b148d0b7760deafbb82e1293784450e771a0acc990891f9bc90 Jul 21 12:39:55.967452 ip-10-0-130-157 kubenswrapper[2576]: W0721 12:39:55.967425 2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda2b48106_9844_4431_9b6f_d4e1baacb15b.slice/crio-0fe94953705938e059445af25ad2536541d565d17a432e15f36562e9be2dc50a WatchSource:0}: Error finding container 0fe94953705938e059445af25ad2536541d565d17a432e15f36562e9be2dc50a: Status 404 returned error can't find the container with id 0fe94953705938e059445af25ad2536541d565d17a432e15f36562e9be2dc50a Jul 21 12:39:56.018055 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:39:56.018017 2576 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Jul 21 12:39:56.237305 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:39:56.237217 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d932e6af-610e-4967-87b1-e489cead334d-metrics-certs\") pod \"network-metrics-daemon-l9rfj\" (UID: \"d932e6af-610e-4967-87b1-e489cead334d\") " pod="openshift-multus/network-metrics-daemon-l9rfj" Jul 21 12:39:56.237305 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:39:56.237286 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/4dc9bfad-f2bb-4934-9f49-4cfd0f381ab2-original-pull-secret\") pod \"global-pull-secret-syncer-g2tkf\" (UID: \"4dc9bfad-f2bb-4934-9f49-4cfd0f381ab2\") " pod="kube-system/global-pull-secret-syncer-g2tkf" Jul 21 12:39:56.237536 ip-10-0-130-157 kubenswrapper[2576]: E0721 12:39:56.237371 2576 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 21 12:39:56.237536 ip-10-0-130-157 kubenswrapper[2576]: E0721 12:39:56.237429 2576 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 21 12:39:56.237536 ip-10-0-130-157 kubenswrapper[2576]: E0721 12:39:56.237443 2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d932e6af-610e-4967-87b1-e489cead334d-metrics-certs podName:d932e6af-610e-4967-87b1-e489cead334d nodeName:}" failed. No retries permitted until 2026-07-21 12:39:57.237428634 +0000 UTC m=+5.072692482 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/d932e6af-610e-4967-87b1-e489cead334d-metrics-certs") pod "network-metrics-daemon-l9rfj" (UID: "d932e6af-610e-4967-87b1-e489cead334d") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 21 12:39:56.237536 ip-10-0-130-157 kubenswrapper[2576]: E0721 12:39:56.237478 2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4dc9bfad-f2bb-4934-9f49-4cfd0f381ab2-original-pull-secret podName:4dc9bfad-f2bb-4934-9f49-4cfd0f381ab2 nodeName:}" failed. No retries permitted until 2026-07-21 12:39:57.237463514 +0000 UTC m=+5.072727363 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/4dc9bfad-f2bb-4934-9f49-4cfd0f381ab2-original-pull-secret") pod "global-pull-secret-syncer-g2tkf" (UID: "4dc9bfad-f2bb-4934-9f49-4cfd0f381ab2") : object "kube-system"/"original-pull-secret" not registered Jul 21 12:39:56.315504 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:39:56.315420 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-130-157.ec2.internal" event={"ID":"77735b552764c08668c42d1fa6bcbce3","Type":"ContainerStarted","Data":"67e473f553460710fbf88bbf52227d28d6c8bec5c668eb689e335d908bce2978"} Jul 21 12:39:56.316781 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:39:56.316750 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-157.ec2.internal" event={"ID":"e47f524b69dd5de36bff36ee415660e7","Type":"ContainerStarted","Data":"083f37808fb1cdc7a7a435396bbb2925740c7a3db9c6d9dd9afc208227cd8976"} Jul 21 12:39:56.318002 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:39:56.317977 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-km4fw" event={"ID":"9cfd0727-8ed4-40bd-b1d5-66a531560bce","Type":"ContainerStarted","Data":"1c9cbbe3b57dfff1c1fd7dd47039bc1b4ac1bb589c4ab20bc637f9377b10caea"} Jul 21 12:39:56.319282 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:39:56.319257 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-pgvgl" event={"ID":"490c41c0-4938-41c5-a264-1cf80533b229","Type":"ContainerStarted","Data":"081308fb5edc491223626bbdb61505aa9774fa3f0bc804d261888cbacdfc1236"} Jul 21 12:39:56.320917 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:39:56.320895 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-wnhr4" event={"ID":"64971bb0-35c9-4026-9680-2a7700efb382","Type":"ContainerStarted","Data":"a94720530444f4f2f36db2913014afce6cdcc37c94558768295ae341857df8fe"} Jul 21 12:39:56.322175 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:39:56.322088 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-6nf7d" event={"ID":"a2b48106-9844-4431-9b6f-d4e1baacb15b","Type":"ContainerStarted","Data":"0fe94953705938e059445af25ad2536541d565d17a432e15f36562e9be2dc50a"} Jul 21 12:39:56.323307 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:39:56.323276 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-dr85z" event={"ID":"4dd651ed-4103-426d-83cc-ea465d4e3d7b","Type":"ContainerStarted","Data":"59bc20611ad70b148d0b7760deafbb82e1293784450e771a0acc990891f9bc90"} Jul 21 12:39:56.324624 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:39:56.324600 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-k7j9j" event={"ID":"1d597db9-aea4-46bd-a743-06fcb6bc24d2","Type":"ContainerStarted","Data":"a5749d37c3be767552b676e1ac271a5d7fc5317c3a8f91b0c16a2e138d83d77f"} Jul 21 12:39:56.325741 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:39:56.325720 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-6crjb" event={"ID":"c70252ec-a8bb-4dd0-9544-2ff28e109eb9","Type":"ContainerStarted","Data":"1e158c7a7b18a3133bef09f0682e65a5b8c64d6355e09891215e60c317143dd7"} Jul 21 12:39:56.326972 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:39:56.326920 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-wb7vw" event={"ID":"06df9492-bc31-44f9-9f87-1665786f3be4","Type":"ContainerStarted","Data":"dc210945db7a93c3276170218feeac5a97f78bf68d771bf93e91066a64a244f7"} Jul 21 12:39:56.328107 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:39:56.328085 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-skc26" event={"ID":"6713665c-3b71-4751-929e-16de48263766","Type":"ContainerStarted","Data":"548c96075e1188986cc0f699ddfca2a880b73b68cef7e70018a31ac3c5f07dd5"} Jul 21 12:39:56.338534 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:39:56.338494 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kvc8f\" (UniqueName: \"kubernetes.io/projected/ea2d55a6-d43a-4d57-b49c-b63904d8810e-kube-api-access-kvc8f\") pod \"network-check-target-6f57m\" (UID: \"ea2d55a6-d43a-4d57-b49c-b63904d8810e\") " pod="openshift-network-diagnostics/network-check-target-6f57m" Jul 21 12:39:56.338725 ip-10-0-130-157 kubenswrapper[2576]: E0721 12:39:56.338698 2576 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 21 12:39:56.338725 ip-10-0-130-157 kubenswrapper[2576]: E0721 12:39:56.338725 2576 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 21 12:39:56.338892 ip-10-0-130-157 kubenswrapper[2576]: E0721 12:39:56.338737 2576 projected.go:196] Error preparing data for projected volume kube-api-access-kvc8f for pod openshift-network-diagnostics/network-check-target-6f57m: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 21 12:39:56.338892 ip-10-0-130-157 kubenswrapper[2576]: E0721 12:39:56.338796 2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/ea2d55a6-d43a-4d57-b49c-b63904d8810e-kube-api-access-kvc8f podName:ea2d55a6-d43a-4d57-b49c-b63904d8810e nodeName:}" failed. No retries permitted until 2026-07-21 12:39:57.338778949 +0000 UTC m=+5.174042795 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-kvc8f" (UniqueName: "kubernetes.io/projected/ea2d55a6-d43a-4d57-b49c-b63904d8810e-kube-api-access-kvc8f") pod "network-check-target-6f57m" (UID: "ea2d55a6-d43a-4d57-b49c-b63904d8810e") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 21 12:39:56.665944 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:39:56.665890 2576 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-20 12:34:55 +0000 UTC" deadline="2028-02-06 07:57:13.099316278 +0000 UTC" Jul 21 12:39:56.665944 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:39:56.665937 2576 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="13555h17m16.433381695s" Jul 21 12:39:57.245603 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:39:57.245498 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d932e6af-610e-4967-87b1-e489cead334d-metrics-certs\") pod \"network-metrics-daemon-l9rfj\" (UID: \"d932e6af-610e-4967-87b1-e489cead334d\") " pod="openshift-multus/network-metrics-daemon-l9rfj" Jul 21 12:39:57.246374 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:39:57.245625 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/4dc9bfad-f2bb-4934-9f49-4cfd0f381ab2-original-pull-secret\") pod \"global-pull-secret-syncer-g2tkf\" (UID: \"4dc9bfad-f2bb-4934-9f49-4cfd0f381ab2\") " pod="kube-system/global-pull-secret-syncer-g2tkf" Jul 21 12:39:57.246374 ip-10-0-130-157 kubenswrapper[2576]: E0721 12:39:57.245888 2576 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 21 12:39:57.246374 ip-10-0-130-157 kubenswrapper[2576]: E0721 12:39:57.245952 2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4dc9bfad-f2bb-4934-9f49-4cfd0f381ab2-original-pull-secret podName:4dc9bfad-f2bb-4934-9f49-4cfd0f381ab2 nodeName:}" failed. No retries permitted until 2026-07-21 12:39:59.245934369 +0000 UTC m=+7.081198215 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/4dc9bfad-f2bb-4934-9f49-4cfd0f381ab2-original-pull-secret") pod "global-pull-secret-syncer-g2tkf" (UID: "4dc9bfad-f2bb-4934-9f49-4cfd0f381ab2") : object "kube-system"/"original-pull-secret" not registered Jul 21 12:39:57.246589 ip-10-0-130-157 kubenswrapper[2576]: E0721 12:39:57.246384 2576 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 21 12:39:57.246589 ip-10-0-130-157 kubenswrapper[2576]: E0721 12:39:57.246431 2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d932e6af-610e-4967-87b1-e489cead334d-metrics-certs podName:d932e6af-610e-4967-87b1-e489cead334d nodeName:}" failed. No retries permitted until 2026-07-21 12:39:59.246415774 +0000 UTC m=+7.081679624 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/d932e6af-610e-4967-87b1-e489cead334d-metrics-certs") pod "network-metrics-daemon-l9rfj" (UID: "d932e6af-610e-4967-87b1-e489cead334d") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 21 12:39:57.306705 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:39:57.306670 2576 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160" Jul 21 12:39:57.310800 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:39:57.310765 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-g2tkf" Jul 21 12:39:57.310996 ip-10-0-130-157 kubenswrapper[2576]: E0721 12:39:57.310911 2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-g2tkf" podUID="4dc9bfad-f2bb-4934-9f49-4cfd0f381ab2" Jul 21 12:39:57.311324 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:39:57.311303 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-l9rfj" Jul 21 12:39:57.311453 ip-10-0-130-157 kubenswrapper[2576]: E0721 12:39:57.311417 2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-l9rfj" podUID="d932e6af-610e-4967-87b1-e489cead334d" Jul 21 12:39:57.311543 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:39:57.311507 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-6f57m" Jul 21 12:39:57.311605 ip-10-0-130-157 kubenswrapper[2576]: E0721 12:39:57.311592 2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-6f57m" podUID="ea2d55a6-d43a-4d57-b49c-b63904d8810e" Jul 21 12:39:57.346670 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:39:57.346630 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kvc8f\" (UniqueName: \"kubernetes.io/projected/ea2d55a6-d43a-4d57-b49c-b63904d8810e-kube-api-access-kvc8f\") pod \"network-check-target-6f57m\" (UID: \"ea2d55a6-d43a-4d57-b49c-b63904d8810e\") " pod="openshift-network-diagnostics/network-check-target-6f57m" Jul 21 12:39:57.346854 ip-10-0-130-157 kubenswrapper[2576]: E0721 12:39:57.346835 2576 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 21 12:39:57.346919 ip-10-0-130-157 kubenswrapper[2576]: E0721 12:39:57.346855 2576 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 21 12:39:57.346919 ip-10-0-130-157 kubenswrapper[2576]: E0721 12:39:57.346870 2576 projected.go:196] Error preparing data for projected volume kube-api-access-kvc8f for pod openshift-network-diagnostics/network-check-target-6f57m: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 21 12:39:57.347019 ip-10-0-130-157 kubenswrapper[2576]: E0721 12:39:57.346929 2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/ea2d55a6-d43a-4d57-b49c-b63904d8810e-kube-api-access-kvc8f podName:ea2d55a6-d43a-4d57-b49c-b63904d8810e nodeName:}" failed. No retries permitted until 2026-07-21 12:39:59.346909719 +0000 UTC m=+7.182173582 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-kvc8f" (UniqueName: "kubernetes.io/projected/ea2d55a6-d43a-4d57-b49c-b63904d8810e-kube-api-access-kvc8f") pod "network-check-target-6f57m" (UID: "ea2d55a6-d43a-4d57-b49c-b63904d8810e") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 21 12:39:57.666640 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:39:57.666595 2576 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-20 12:34:55 +0000 UTC" deadline="2028-02-10 18:00:49.7551766 +0000 UTC" Jul 21 12:39:57.666640 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:39:57.666639 2576 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="13661h20m52.088542859s" Jul 21 12:39:59.264162 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:39:59.263894 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d932e6af-610e-4967-87b1-e489cead334d-metrics-certs\") pod \"network-metrics-daemon-l9rfj\" (UID: \"d932e6af-610e-4967-87b1-e489cead334d\") " pod="openshift-multus/network-metrics-daemon-l9rfj" Jul 21 12:39:59.264162 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:39:59.263961 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/4dc9bfad-f2bb-4934-9f49-4cfd0f381ab2-original-pull-secret\") pod \"global-pull-secret-syncer-g2tkf\" (UID: \"4dc9bfad-f2bb-4934-9f49-4cfd0f381ab2\") " pod="kube-system/global-pull-secret-syncer-g2tkf" Jul 21 12:39:59.264162 ip-10-0-130-157 kubenswrapper[2576]: E0721 12:39:59.264068 2576 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 21 12:39:59.264162 ip-10-0-130-157 kubenswrapper[2576]: E0721 12:39:59.264103 2576 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 21 12:39:59.264162 ip-10-0-130-157 kubenswrapper[2576]: E0721 12:39:59.264149 2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d932e6af-610e-4967-87b1-e489cead334d-metrics-certs podName:d932e6af-610e-4967-87b1-e489cead334d nodeName:}" failed. No retries permitted until 2026-07-21 12:40:03.264127085 +0000 UTC m=+11.099390936 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/d932e6af-610e-4967-87b1-e489cead334d-metrics-certs") pod "network-metrics-daemon-l9rfj" (UID: "d932e6af-610e-4967-87b1-e489cead334d") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 21 12:39:59.264162 ip-10-0-130-157 kubenswrapper[2576]: E0721 12:39:59.264170 2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4dc9bfad-f2bb-4934-9f49-4cfd0f381ab2-original-pull-secret podName:4dc9bfad-f2bb-4934-9f49-4cfd0f381ab2 nodeName:}" failed. No retries permitted until 2026-07-21 12:40:03.264160005 +0000 UTC m=+11.099423856 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/4dc9bfad-f2bb-4934-9f49-4cfd0f381ab2-original-pull-secret") pod "global-pull-secret-syncer-g2tkf" (UID: "4dc9bfad-f2bb-4934-9f49-4cfd0f381ab2") : object "kube-system"/"original-pull-secret" not registered Jul 21 12:39:59.311387 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:39:59.310628 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-g2tkf" Jul 21 12:39:59.311387 ip-10-0-130-157 kubenswrapper[2576]: E0721 12:39:59.310765 2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-g2tkf" podUID="4dc9bfad-f2bb-4934-9f49-4cfd0f381ab2" Jul 21 12:39:59.311387 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:39:59.311133 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-l9rfj" Jul 21 12:39:59.311387 ip-10-0-130-157 kubenswrapper[2576]: E0721 12:39:59.311239 2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-l9rfj" podUID="d932e6af-610e-4967-87b1-e489cead334d" Jul 21 12:39:59.311387 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:39:59.311291 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-6f57m" Jul 21 12:39:59.311387 ip-10-0-130-157 kubenswrapper[2576]: E0721 12:39:59.311354 2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-6f57m" podUID="ea2d55a6-d43a-4d57-b49c-b63904d8810e" Jul 21 12:39:59.365381 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:39:59.364742 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kvc8f\" (UniqueName: \"kubernetes.io/projected/ea2d55a6-d43a-4d57-b49c-b63904d8810e-kube-api-access-kvc8f\") pod \"network-check-target-6f57m\" (UID: \"ea2d55a6-d43a-4d57-b49c-b63904d8810e\") " pod="openshift-network-diagnostics/network-check-target-6f57m" Jul 21 12:39:59.365381 ip-10-0-130-157 kubenswrapper[2576]: E0721 12:39:59.364928 2576 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 21 12:39:59.365381 ip-10-0-130-157 kubenswrapper[2576]: E0721 12:39:59.364948 2576 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 21 12:39:59.365381 ip-10-0-130-157 kubenswrapper[2576]: E0721 12:39:59.364961 2576 projected.go:196] Error preparing data for projected volume kube-api-access-kvc8f for pod openshift-network-diagnostics/network-check-target-6f57m: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 21 12:39:59.365381 ip-10-0-130-157 kubenswrapper[2576]: E0721 12:39:59.365016 2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/ea2d55a6-d43a-4d57-b49c-b63904d8810e-kube-api-access-kvc8f podName:ea2d55a6-d43a-4d57-b49c-b63904d8810e nodeName:}" failed. No retries permitted until 2026-07-21 12:40:03.364998544 +0000 UTC m=+11.200262394 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-kvc8f" (UniqueName: "kubernetes.io/projected/ea2d55a6-d43a-4d57-b49c-b63904d8810e-kube-api-access-kvc8f") pod "network-check-target-6f57m" (UID: "ea2d55a6-d43a-4d57-b49c-b63904d8810e") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 21 12:40:01.310786 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:40:01.310748 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-6f57m" Jul 21 12:40:01.311240 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:40:01.310806 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-g2tkf" Jul 21 12:40:01.311240 ip-10-0-130-157 kubenswrapper[2576]: E0721 12:40:01.310880 2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-6f57m" podUID="ea2d55a6-d43a-4d57-b49c-b63904d8810e" Jul 21 12:40:01.311240 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:40:01.310972 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-l9rfj" Jul 21 12:40:01.311240 ip-10-0-130-157 kubenswrapper[2576]: E0721 12:40:01.311097 2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-l9rfj" podUID="d932e6af-610e-4967-87b1-e489cead334d" Jul 21 12:40:01.311460 ip-10-0-130-157 kubenswrapper[2576]: E0721 12:40:01.311229 2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-g2tkf" podUID="4dc9bfad-f2bb-4934-9f49-4cfd0f381ab2" Jul 21 12:40:03.300214 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:40:03.300176 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d932e6af-610e-4967-87b1-e489cead334d-metrics-certs\") pod \"network-metrics-daemon-l9rfj\" (UID: \"d932e6af-610e-4967-87b1-e489cead334d\") " pod="openshift-multus/network-metrics-daemon-l9rfj" Jul 21 12:40:03.300704 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:40:03.300244 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/4dc9bfad-f2bb-4934-9f49-4cfd0f381ab2-original-pull-secret\") pod \"global-pull-secret-syncer-g2tkf\" (UID: \"4dc9bfad-f2bb-4934-9f49-4cfd0f381ab2\") " pod="kube-system/global-pull-secret-syncer-g2tkf" Jul 21 12:40:03.300704 ip-10-0-130-157 kubenswrapper[2576]: E0721 12:40:03.300408 2576 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 21 12:40:03.300704 ip-10-0-130-157 kubenswrapper[2576]: E0721 12:40:03.300468 2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4dc9bfad-f2bb-4934-9f49-4cfd0f381ab2-original-pull-secret podName:4dc9bfad-f2bb-4934-9f49-4cfd0f381ab2 nodeName:}" failed. No retries permitted until 2026-07-21 12:40:11.300450353 +0000 UTC m=+19.135714202 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/4dc9bfad-f2bb-4934-9f49-4cfd0f381ab2-original-pull-secret") pod "global-pull-secret-syncer-g2tkf" (UID: "4dc9bfad-f2bb-4934-9f49-4cfd0f381ab2") : object "kube-system"/"original-pull-secret" not registered Jul 21 12:40:03.300704 ip-10-0-130-157 kubenswrapper[2576]: E0721 12:40:03.300548 2576 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 21 12:40:03.300704 ip-10-0-130-157 kubenswrapper[2576]: E0721 12:40:03.300582 2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d932e6af-610e-4967-87b1-e489cead334d-metrics-certs podName:d932e6af-610e-4967-87b1-e489cead334d nodeName:}" failed. No retries permitted until 2026-07-21 12:40:11.300569039 +0000 UTC m=+19.135832890 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/d932e6af-610e-4967-87b1-e489cead334d-metrics-certs") pod "network-metrics-daemon-l9rfj" (UID: "d932e6af-610e-4967-87b1-e489cead334d") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 21 12:40:03.310471 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:40:03.310443 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-g2tkf" Jul 21 12:40:03.310471 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:40:03.310464 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-6f57m" Jul 21 12:40:03.310700 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:40:03.310489 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-l9rfj" Jul 21 12:40:03.310700 ip-10-0-130-157 kubenswrapper[2576]: E0721 12:40:03.310660 2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-6f57m" podUID="ea2d55a6-d43a-4d57-b49c-b63904d8810e" Jul 21 12:40:03.310802 ip-10-0-130-157 kubenswrapper[2576]: E0721 12:40:03.310728 2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-l9rfj" podUID="d932e6af-610e-4967-87b1-e489cead334d" Jul 21 12:40:03.310802 ip-10-0-130-157 kubenswrapper[2576]: E0721 12:40:03.310769 2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-g2tkf" podUID="4dc9bfad-f2bb-4934-9f49-4cfd0f381ab2" Jul 21 12:40:03.400646 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:40:03.400611 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kvc8f\" (UniqueName: \"kubernetes.io/projected/ea2d55a6-d43a-4d57-b49c-b63904d8810e-kube-api-access-kvc8f\") pod \"network-check-target-6f57m\" (UID: \"ea2d55a6-d43a-4d57-b49c-b63904d8810e\") " pod="openshift-network-diagnostics/network-check-target-6f57m" Jul 21 12:40:03.400805 ip-10-0-130-157 kubenswrapper[2576]: E0721 12:40:03.400764 2576 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 21 12:40:03.400805 ip-10-0-130-157 kubenswrapper[2576]: E0721 12:40:03.400781 2576 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 21 12:40:03.400805 ip-10-0-130-157 kubenswrapper[2576]: E0721 12:40:03.400794 2576 projected.go:196] Error preparing data for projected volume kube-api-access-kvc8f for pod openshift-network-diagnostics/network-check-target-6f57m: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 21 12:40:03.400973 ip-10-0-130-157 kubenswrapper[2576]: E0721 12:40:03.400845 2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/ea2d55a6-d43a-4d57-b49c-b63904d8810e-kube-api-access-kvc8f podName:ea2d55a6-d43a-4d57-b49c-b63904d8810e nodeName:}" failed. No retries permitted until 2026-07-21 12:40:11.400827806 +0000 UTC m=+19.236091652 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-kvc8f" (UniqueName: "kubernetes.io/projected/ea2d55a6-d43a-4d57-b49c-b63904d8810e-kube-api-access-kvc8f") pod "network-check-target-6f57m" (UID: "ea2d55a6-d43a-4d57-b49c-b63904d8810e") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 21 12:40:05.312344 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:40:05.311493 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-g2tkf" Jul 21 12:40:05.312344 ip-10-0-130-157 kubenswrapper[2576]: E0721 12:40:05.311641 2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-g2tkf" podUID="4dc9bfad-f2bb-4934-9f49-4cfd0f381ab2" Jul 21 12:40:05.312344 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:40:05.312043 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-l9rfj" Jul 21 12:40:05.312344 ip-10-0-130-157 kubenswrapper[2576]: E0721 12:40:05.312156 2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-l9rfj" podUID="d932e6af-610e-4967-87b1-e489cead334d" Jul 21 12:40:05.312344 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:40:05.312214 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-6f57m" Jul 21 12:40:05.312344 ip-10-0-130-157 kubenswrapper[2576]: E0721 12:40:05.312283 2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-6f57m" podUID="ea2d55a6-d43a-4d57-b49c-b63904d8810e" Jul 21 12:40:06.355277 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:40:06.354965 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-157.ec2.internal" event={"ID":"e47f524b69dd5de36bff36ee415660e7","Type":"ContainerStarted","Data":"707e66ef93eacc5daa19f3f3b97dca1ffb71c771a4eda7f770a2aa3c1311b330"} Jul 21 12:40:07.311265 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:40:07.311232 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-6f57m" Jul 21 12:40:07.311539 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:40:07.311278 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-g2tkf" Jul 21 12:40:07.311539 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:40:07.311278 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-l9rfj" Jul 21 12:40:07.311539 ip-10-0-130-157 kubenswrapper[2576]: E0721 12:40:07.311355 2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-6f57m" podUID="ea2d55a6-d43a-4d57-b49c-b63904d8810e" Jul 21 12:40:07.311539 ip-10-0-130-157 kubenswrapper[2576]: E0721 12:40:07.311464 2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-l9rfj" podUID="d932e6af-610e-4967-87b1-e489cead334d" Jul 21 12:40:07.311739 ip-10-0-130-157 kubenswrapper[2576]: E0721 12:40:07.311570 2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-g2tkf" podUID="4dc9bfad-f2bb-4934-9f49-4cfd0f381ab2" Jul 21 12:40:07.358727 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:40:07.358694 2576 generic.go:356] "Generic (PLEG): container finished" podID="e47f524b69dd5de36bff36ee415660e7" containerID="707e66ef93eacc5daa19f3f3b97dca1ffb71c771a4eda7f770a2aa3c1311b330" exitCode=0 Jul 21 12:40:07.359143 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:40:07.358738 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-157.ec2.internal" event={"ID":"e47f524b69dd5de36bff36ee415660e7","Type":"ContainerDied","Data":"707e66ef93eacc5daa19f3f3b97dca1ffb71c771a4eda7f770a2aa3c1311b330"} Jul 21 12:40:09.311339 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:40:09.311297 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-g2tkf" Jul 21 12:40:09.311856 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:40:09.311357 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-6f57m" Jul 21 12:40:09.311856 ip-10-0-130-157 kubenswrapper[2576]: E0721 12:40:09.311435 2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-g2tkf" podUID="4dc9bfad-f2bb-4934-9f49-4cfd0f381ab2" Jul 21 12:40:09.311856 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:40:09.311300 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-l9rfj" Jul 21 12:40:09.311856 ip-10-0-130-157 kubenswrapper[2576]: E0721 12:40:09.311573 2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-l9rfj" podUID="d932e6af-610e-4967-87b1-e489cead334d" Jul 21 12:40:09.311856 ip-10-0-130-157 kubenswrapper[2576]: E0721 12:40:09.311639 2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-6f57m" podUID="ea2d55a6-d43a-4d57-b49c-b63904d8810e" Jul 21 12:40:11.311253 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:40:11.311218 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-6f57m" Jul 21 12:40:11.311253 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:40:11.311235 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-g2tkf" Jul 21 12:40:11.311771 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:40:11.311243 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-l9rfj" Jul 21 12:40:11.311771 ip-10-0-130-157 kubenswrapper[2576]: E0721 12:40:11.311330 2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-6f57m" podUID="ea2d55a6-d43a-4d57-b49c-b63904d8810e" Jul 21 12:40:11.311771 ip-10-0-130-157 kubenswrapper[2576]: E0721 12:40:11.311420 2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-g2tkf" podUID="4dc9bfad-f2bb-4934-9f49-4cfd0f381ab2" Jul 21 12:40:11.311771 ip-10-0-130-157 kubenswrapper[2576]: E0721 12:40:11.311506 2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-l9rfj" podUID="d932e6af-610e-4967-87b1-e489cead334d" Jul 21 12:40:11.358866 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:40:11.358833 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d932e6af-610e-4967-87b1-e489cead334d-metrics-certs\") pod \"network-metrics-daemon-l9rfj\" (UID: \"d932e6af-610e-4967-87b1-e489cead334d\") " pod="openshift-multus/network-metrics-daemon-l9rfj" Jul 21 12:40:11.359025 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:40:11.358881 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/4dc9bfad-f2bb-4934-9f49-4cfd0f381ab2-original-pull-secret\") pod \"global-pull-secret-syncer-g2tkf\" (UID: \"4dc9bfad-f2bb-4934-9f49-4cfd0f381ab2\") " pod="kube-system/global-pull-secret-syncer-g2tkf" Jul 21 12:40:11.359025 ip-10-0-130-157 kubenswrapper[2576]: E0721 12:40:11.359006 2576 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 21 12:40:11.359121 ip-10-0-130-157 kubenswrapper[2576]: E0721 12:40:11.359017 2576 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 21 12:40:11.359121 ip-10-0-130-157 kubenswrapper[2576]: E0721 12:40:11.359060 2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4dc9bfad-f2bb-4934-9f49-4cfd0f381ab2-original-pull-secret podName:4dc9bfad-f2bb-4934-9f49-4cfd0f381ab2 nodeName:}" failed. No retries permitted until 2026-07-21 12:40:27.359047265 +0000 UTC m=+35.194311112 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/4dc9bfad-f2bb-4934-9f49-4cfd0f381ab2-original-pull-secret") pod "global-pull-secret-syncer-g2tkf" (UID: "4dc9bfad-f2bb-4934-9f49-4cfd0f381ab2") : object "kube-system"/"original-pull-secret" not registered Jul 21 12:40:11.361818 ip-10-0-130-157 kubenswrapper[2576]: E0721 12:40:11.359390 2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d932e6af-610e-4967-87b1-e489cead334d-metrics-certs podName:d932e6af-610e-4967-87b1-e489cead334d nodeName:}" failed. No retries permitted until 2026-07-21 12:40:27.359068748 +0000 UTC m=+35.194332592 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/d932e6af-610e-4967-87b1-e489cead334d-metrics-certs") pod "network-metrics-daemon-l9rfj" (UID: "d932e6af-610e-4967-87b1-e489cead334d") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 21 12:40:11.459988 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:40:11.459947 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kvc8f\" (UniqueName: \"kubernetes.io/projected/ea2d55a6-d43a-4d57-b49c-b63904d8810e-kube-api-access-kvc8f\") pod \"network-check-target-6f57m\" (UID: \"ea2d55a6-d43a-4d57-b49c-b63904d8810e\") " pod="openshift-network-diagnostics/network-check-target-6f57m" Jul 21 12:40:11.460151 ip-10-0-130-157 kubenswrapper[2576]: E0721 12:40:11.460122 2576 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 21 12:40:11.460151 ip-10-0-130-157 kubenswrapper[2576]: E0721 12:40:11.460146 2576 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 21 12:40:11.460236 ip-10-0-130-157 kubenswrapper[2576]: E0721 12:40:11.460161 2576 projected.go:196] Error preparing data for projected volume kube-api-access-kvc8f for pod openshift-network-diagnostics/network-check-target-6f57m: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 21 12:40:11.460346 ip-10-0-130-157 kubenswrapper[2576]: E0721 12:40:11.460237 2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/ea2d55a6-d43a-4d57-b49c-b63904d8810e-kube-api-access-kvc8f podName:ea2d55a6-d43a-4d57-b49c-b63904d8810e nodeName:}" failed. No retries permitted until 2026-07-21 12:40:27.460217727 +0000 UTC m=+35.295481570 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-kvc8f" (UniqueName: "kubernetes.io/projected/ea2d55a6-d43a-4d57-b49c-b63904d8810e-kube-api-access-kvc8f") pod "network-check-target-6f57m" (UID: "ea2d55a6-d43a-4d57-b49c-b63904d8810e") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 21 12:40:13.311043 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:40:13.311007 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-l9rfj" Jul 21 12:40:13.311465 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:40:13.311007 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-g2tkf" Jul 21 12:40:13.311465 ip-10-0-130-157 kubenswrapper[2576]: E0721 12:40:13.311144 2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-l9rfj" podUID="d932e6af-610e-4967-87b1-e489cead334d" Jul 21 12:40:13.311465 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:40:13.311013 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-6f57m" Jul 21 12:40:13.311465 ip-10-0-130-157 kubenswrapper[2576]: E0721 12:40:13.311192 2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-g2tkf" podUID="4dc9bfad-f2bb-4934-9f49-4cfd0f381ab2" Jul 21 12:40:13.311465 ip-10-0-130-157 kubenswrapper[2576]: E0721 12:40:13.311289 2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-6f57m" podUID="ea2d55a6-d43a-4d57-b49c-b63904d8810e" Jul 21 12:40:15.311311 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:40:15.310530 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-l9rfj" Jul 21 12:40:15.311311 ip-10-0-130-157 kubenswrapper[2576]: E0721 12:40:15.310654 2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-l9rfj" podUID="d932e6af-610e-4967-87b1-e489cead334d" Jul 21 12:40:15.311311 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:40:15.310729 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-6f57m" Jul 21 12:40:15.311311 ip-10-0-130-157 kubenswrapper[2576]: E0721 12:40:15.310891 2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-6f57m" podUID="ea2d55a6-d43a-4d57-b49c-b63904d8810e" Jul 21 12:40:15.311311 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:40:15.310951 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-g2tkf" Jul 21 12:40:15.311311 ip-10-0-130-157 kubenswrapper[2576]: E0721 12:40:15.311029 2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-g2tkf" podUID="4dc9bfad-f2bb-4934-9f49-4cfd0f381ab2" Jul 21 12:40:15.381912 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:40:15.381660 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-wnhr4" event={"ID":"64971bb0-35c9-4026-9680-2a7700efb382","Type":"ContainerStarted","Data":"e50c94f275e631c90c32756e16ed9eb10a2dca4d80d41a77d949ef3177c7240c"} Jul 21 12:40:15.385731 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:40:15.385555 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-k7j9j" event={"ID":"1d597db9-aea4-46bd-a743-06fcb6bc24d2","Type":"ContainerStarted","Data":"3c9c608fc617d0135124d64b999910b18c4011607202c377888c2b5021c624a5"} Jul 21 12:40:15.390447 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:40:15.390356 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-130-157.ec2.internal" event={"ID":"77735b552764c08668c42d1fa6bcbce3","Type":"ContainerStarted","Data":"5d93c8ee6db2ededf64fb990127c1fb27ab15e588836083dc2223a04f27f5b64"} Jul 21 12:40:15.393301 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:40:15.393240 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-km4fw" event={"ID":"9cfd0727-8ed4-40bd-b1d5-66a531560bce","Type":"ContainerStarted","Data":"ad50bb603ba86941071f654f55bad336bf8edc1aafde0d0c8c59f5c78a9db7f4"} Jul 21 12:40:15.424606 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:40:15.424548 2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-k7j9j" podStartSLOduration=2.176334719 podStartE2EDuration="21.424530191s" totalImagesPullingTime="19.248195472s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-21 12:39:54 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 12:40:15.424078695 +0000 UTC m=+23.259342562" watchObservedRunningTime="2026-07-21 12:40:15.424530191 +0000 UTC m=+23.259794052" Jul 21 12:40:15.438545 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:40:15.438207 2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-130-157.ec2.internal" podStartSLOduration=21.438185811 podStartE2EDuration="21.438185811s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-21 12:39:54 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 12:40:15.437616635 +0000 UTC m=+23.272880503" watchObservedRunningTime="2026-07-21 12:40:15.438185811 +0000 UTC m=+23.273449680" Jul 21 12:40:15.457991 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:40:15.457831 2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/node-ca-km4fw" podStartSLOduration=2.6300086560000002 podStartE2EDuration="21.457813419s" totalImagesPullingTime="18.827804763s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-21 12:39:54 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 12:40:15.457798628 +0000 UTC m=+23.293062495" watchObservedRunningTime="2026-07-21 12:40:15.457813419 +0000 UTC m=+23.293077279" Jul 21 12:40:16.398433 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:40:16.398121 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-6nf7d_a2b48106-9844-4431-9b6f-d4e1baacb15b/ovn-acl-logging/0.log" Jul 21 12:40:16.399060 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:40:16.398747 2576 generic.go:356] "Generic (PLEG): container finished" podID="a2b48106-9844-4431-9b6f-d4e1baacb15b" containerID="2d0f01731a3d5be1f650f4253a6b12a39138367aedcf75156779e68d1c9d0a7f" exitCode=1 Jul 21 12:40:16.399060 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:40:16.398812 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-6nf7d" event={"ID":"a2b48106-9844-4431-9b6f-d4e1baacb15b","Type":"ContainerStarted","Data":"065aeff5066fe5edce90c0268aa68106bbb4a0e1786fe790a42b3772c3c8eb26"} Jul 21 12:40:16.399060 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:40:16.398889 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-6nf7d" event={"ID":"a2b48106-9844-4431-9b6f-d4e1baacb15b","Type":"ContainerStarted","Data":"4fbcb66a639fa3566263a33aa4629ca1df5da021adf346797a7ad666b0bcefe5"} Jul 21 12:40:16.399060 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:40:16.398941 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-6nf7d" event={"ID":"a2b48106-9844-4431-9b6f-d4e1baacb15b","Type":"ContainerStarted","Data":"7861befbeeee3a4726fa566a8c00e76924325935a7f89d203635bc75b061ba40"} Jul 21 12:40:16.399060 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:40:16.398983 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-6nf7d" event={"ID":"a2b48106-9844-4431-9b6f-d4e1baacb15b","Type":"ContainerStarted","Data":"6abc415525add6ab0b65a75d6ce8962562680faf2996243140cde174042eb52d"} Jul 21 12:40:16.399060 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:40:16.399020 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-6nf7d" event={"ID":"a2b48106-9844-4431-9b6f-d4e1baacb15b","Type":"ContainerDied","Data":"2d0f01731a3d5be1f650f4253a6b12a39138367aedcf75156779e68d1c9d0a7f"} Jul 21 12:40:16.399366 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:40:16.399067 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-6nf7d" event={"ID":"a2b48106-9844-4431-9b6f-d4e1baacb15b","Type":"ContainerStarted","Data":"3dc79bc744dd440a4cfd37a594fd0c80069287db40440c8cb621bf9b9a3537a2"} Jul 21 12:40:16.401346 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:40:16.401326 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-6crjb" event={"ID":"c70252ec-a8bb-4dd0-9544-2ff28e109eb9","Type":"ContainerStarted","Data":"556c6db200337469c2aad5c2f10e8d1805dace40c12af51378bcdef354011901"} Jul 21 12:40:16.402641 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:40:16.402622 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-wb7vw" event={"ID":"06df9492-bc31-44f9-9f87-1665786f3be4","Type":"ContainerStarted","Data":"458eefafc799633a450eeb5877798ce7afbde94e11a1316fbcf38ad6e2ab9129"} Jul 21 12:40:16.403852 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:40:16.403827 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-skc26" event={"ID":"6713665c-3b71-4751-929e-16de48263766","Type":"ContainerStarted","Data":"61b8c79ef13d42a924bb69c1d48f21dbedfe3360722563d36f12fdf12cbda7c8"} Jul 21 12:40:16.405469 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:40:16.405450 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-157.ec2.internal" event={"ID":"e47f524b69dd5de36bff36ee415660e7","Type":"ContainerStarted","Data":"d5459da3a9fd858f9afa5f2100abcab5b21e0da3cb29528264a3f296d95e3679"} Jul 21 12:40:16.406672 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:40:16.406653 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-pgvgl" event={"ID":"490c41c0-4938-41c5-a264-1cf80533b229","Type":"ContainerStarted","Data":"a3287492163d3e6bbfde015abe4cf094ac67810e597a8657240db9166b455b5f"} Jul 21 12:40:16.407912 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:40:16.407894 2576 generic.go:356] "Generic (PLEG): container finished" podID="64971bb0-35c9-4026-9680-2a7700efb382" containerID="e50c94f275e631c90c32756e16ed9eb10a2dca4d80d41a77d949ef3177c7240c" exitCode=0 Jul 21 12:40:16.407998 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:40:16.407974 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-wnhr4" event={"ID":"64971bb0-35c9-4026-9680-2a7700efb382","Type":"ContainerDied","Data":"e50c94f275e631c90c32756e16ed9eb10a2dca4d80d41a77d949ef3177c7240c"} Jul 21 12:40:16.418594 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:40:16.418558 2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/node-resolver-6crjb" podStartSLOduration=3.146882446 podStartE2EDuration="22.41854612s" totalImagesPullingTime="19.271663674s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-21 12:39:54 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 12:40:16.418133395 +0000 UTC m=+24.253397261" watchObservedRunningTime="2026-07-21 12:40:16.41854612 +0000 UTC m=+24.253809985" Jul 21 12:40:16.433376 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:40:16.433346 2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-157.ec2.internal" podStartSLOduration=22.433334337 podStartE2EDuration="22.433334337s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-21 12:39:54 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 12:40:16.433007853 +0000 UTC m=+24.268271718" watchObservedRunningTime="2026-07-21 12:40:16.433334337 +0000 UTC m=+24.268598202" Jul 21 12:40:16.447312 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:40:16.447274 2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/konnectivity-agent-skc26" podStartSLOduration=3.165864846 podStartE2EDuration="22.447259974s" totalImagesPullingTime="19.281395128s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 12:39:54 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 12:40:16.446733954 +0000 UTC m=+24.281997821" watchObservedRunningTime="2026-07-21 12:40:16.447259974 +0000 UTC m=+24.282523843" Jul 21 12:40:16.488680 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:40:16.488632 2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-pgvgl" podStartSLOduration=3.233755672 podStartE2EDuration="22.488613496s" totalImagesPullingTime="19.254857824s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-21 12:39:54 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 12:40:16.488447217 +0000 UTC m=+24.323711083" watchObservedRunningTime="2026-07-21 12:40:16.488613496 +0000 UTC m=+24.323877363" Jul 21 12:40:16.642031 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:40:16.642009 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-skc26_6713665c-3b71-4751-929e-16de48263766/konnectivity-agent/0.log" limit=1024 Jul 21 12:40:16.706902 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:40:16.704920 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-skc26_6713665c-3b71-4751-929e-16de48263766/konnectivity-agent/0.log" limit=1024 Jul 21 12:40:17.287328 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:40:17.287291 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-skc26_6713665c-3b71-4751-929e-16de48263766/konnectivity-agent/0.log" limit=1024 Jul 21 12:40:17.311524 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:40:17.311491 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-6f57m" Jul 21 12:40:17.311685 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:40:17.311529 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-g2tkf" Jul 21 12:40:17.311685 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:40:17.311538 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-l9rfj" Jul 21 12:40:17.311685 ip-10-0-130-157 kubenswrapper[2576]: E0721 12:40:17.311611 2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-6f57m" podUID="ea2d55a6-d43a-4d57-b49c-b63904d8810e" Jul 21 12:40:17.311846 ip-10-0-130-157 kubenswrapper[2576]: E0721 12:40:17.311711 2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-g2tkf" podUID="4dc9bfad-f2bb-4934-9f49-4cfd0f381ab2" Jul 21 12:40:17.311846 ip-10-0-130-157 kubenswrapper[2576]: E0721 12:40:17.311789 2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-l9rfj" podUID="d932e6af-610e-4967-87b1-e489cead334d" Jul 21 12:40:17.387624 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:40:17.387598 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-skc26_6713665c-3b71-4751-929e-16de48263766/konnectivity-agent/0.log" limit=1024 Jul 21 12:40:17.412593 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:40:17.412568 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-dr85z" event={"ID":"4dd651ed-4103-426d-83cc-ea465d4e3d7b","Type":"ContainerStarted","Data":"4eace51cd6d813f0d78d238036aa385ea2c1c45de74bd001dee9eab4705aaf6e"} Jul 21 12:40:17.505245 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:40:17.505217 2576 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock" Jul 21 12:40:17.588082 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:40:17.588015 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-skc26_6713665c-3b71-4751-929e-16de48263766/konnectivity-agent/0.log" limit=1024 Jul 21 12:40:17.969481 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:40:17.969452 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-skc26_6713665c-3b71-4751-929e-16de48263766/konnectivity-agent/0.log" limit=1024 Jul 21 12:40:18.034433 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:40:18.034401 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-skc26_6713665c-3b71-4751-929e-16de48263766/konnectivity-agent/0.log" limit=1024 Jul 21 12:40:18.218473 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:40:18.218339 2576 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-07-21T12:40:17.505233836Z","UUID":"b16a9fbf-6973-4dc2-82ef-14b0dcadea84","Handler":null,"Name":"","Endpoint":""} Jul 21 12:40:18.220074 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:40:18.220007 2576 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0 Jul 21 12:40:18.220074 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:40:18.220041 2576 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock Jul 21 12:40:18.418807 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:40:18.418782 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-6nf7d_a2b48106-9844-4431-9b6f-d4e1baacb15b/ovn-acl-logging/0.log" Jul 21 12:40:18.419342 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:40:18.419314 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-6nf7d" event={"ID":"a2b48106-9844-4431-9b6f-d4e1baacb15b","Type":"ContainerStarted","Data":"0a6de32d92ad69742f4e4983aaebfd8c5694187d6a30eecd90dd077f04baebf6"} Jul 21 12:40:18.422226 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:40:18.422198 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-wb7vw" event={"ID":"06df9492-bc31-44f9-9f87-1665786f3be4","Type":"ContainerStarted","Data":"eb8912abb59b8cb63aefceb2b2dc135c1d804c49e72145e905edaea305ba0c49"} Jul 21 12:40:18.423257 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:40:18.423233 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-skc26_6713665c-3b71-4751-929e-16de48263766/konnectivity-agent/0.log" limit=1024 Jul 21 12:40:18.498829 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:40:18.498742 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-skc26_6713665c-3b71-4751-929e-16de48263766/konnectivity-agent/0.log" limit=1024 Jul 21 12:40:19.311539 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:40:19.311230 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-6f57m" Jul 21 12:40:19.311650 ip-10-0-130-157 kubenswrapper[2576]: E0721 12:40:19.311591 2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-6f57m" podUID="ea2d55a6-d43a-4d57-b49c-b63904d8810e" Jul 21 12:40:19.311650 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:40:19.311326 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-l9rfj" Jul 21 12:40:19.311754 ip-10-0-130-157 kubenswrapper[2576]: E0721 12:40:19.311738 2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-l9rfj" podUID="d932e6af-610e-4967-87b1-e489cead334d" Jul 21 12:40:19.311754 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:40:19.311315 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-g2tkf" Jul 21 12:40:19.311862 ip-10-0-130-157 kubenswrapper[2576]: E0721 12:40:19.311849 2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-g2tkf" podUID="4dc9bfad-f2bb-4934-9f49-4cfd0f381ab2" Jul 21 12:40:19.426937 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:40:19.426908 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-wb7vw" event={"ID":"06df9492-bc31-44f9-9f87-1665786f3be4","Type":"ContainerStarted","Data":"ebe73edd22257e2cc2eac4a330fe73b922cfe0925ec290f0b0320e2aa8712ee0"} Jul 21 12:40:19.446246 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:40:19.446207 2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-wb7vw" podStartSLOduration=2.642952888 podStartE2EDuration="25.446195086s" totalImagesPullingTime="22.803242198s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-21 12:39:54 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 12:40:19.445946125 +0000 UTC m=+27.281209991" watchObservedRunningTime="2026-07-21 12:40:19.446195086 +0000 UTC m=+27.281458958" Jul 21 12:40:19.446687 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:40:19.446662 2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-dr85z" podStartSLOduration=6.281568146 podStartE2EDuration="25.446654194s" totalImagesPullingTime="19.165086048s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-21 12:39:54 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 12:40:17.431420452 +0000 UTC m=+25.266684318" watchObservedRunningTime="2026-07-21 12:40:19.446654194 +0000 UTC m=+27.281918060" Jul 21 12:40:20.354993 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:40:20.354962 2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-skc26" Jul 21 12:40:20.355714 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:40:20.355681 2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-skc26" Jul 21 12:40:20.429080 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:40:20.429056 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-skc26" Jul 21 12:40:20.429501 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:40:20.429487 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-skc26" Jul 21 12:40:20.507438 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:40:20.507403 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-skc26_6713665c-3b71-4751-929e-16de48263766/konnectivity-agent/0.log" limit=1024 Jul 21 12:40:21.310860 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:40:21.310834 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-g2tkf" Jul 21 12:40:21.310939 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:40:21.310878 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-6f57m" Jul 21 12:40:21.310939 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:40:21.310846 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-l9rfj" Jul 21 12:40:21.311003 ip-10-0-130-157 kubenswrapper[2576]: E0721 12:40:21.310958 2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-g2tkf" podUID="4dc9bfad-f2bb-4934-9f49-4cfd0f381ab2" Jul 21 12:40:21.311042 ip-10-0-130-157 kubenswrapper[2576]: E0721 12:40:21.311022 2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-l9rfj" podUID="d932e6af-610e-4967-87b1-e489cead334d" Jul 21 12:40:21.311112 ip-10-0-130-157 kubenswrapper[2576]: E0721 12:40:21.311094 2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-6f57m" podUID="ea2d55a6-d43a-4d57-b49c-b63904d8810e" Jul 21 12:40:21.432531 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:40:21.432340 2576 generic.go:356] "Generic (PLEG): container finished" podID="64971bb0-35c9-4026-9680-2a7700efb382" containerID="38e4ee24a6e0814efe23dad0c731c3419379020a1221233acff4e5d569359e60" exitCode=0 Jul 21 12:40:21.432531 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:40:21.432422 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-wnhr4" event={"ID":"64971bb0-35c9-4026-9680-2a7700efb382","Type":"ContainerDied","Data":"38e4ee24a6e0814efe23dad0c731c3419379020a1221233acff4e5d569359e60"} Jul 21 12:40:21.436144 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:40:21.436126 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-6nf7d_a2b48106-9844-4431-9b6f-d4e1baacb15b/ovn-acl-logging/0.log" Jul 21 12:40:21.436553 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:40:21.436535 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-6nf7d" event={"ID":"a2b48106-9844-4431-9b6f-d4e1baacb15b","Type":"ContainerStarted","Data":"a09fd4c5089a06820e7cb353c4f3b6d695d4b613652036b435f495a00089b05d"} Jul 21 12:40:21.436943 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:40:21.436929 2576 scope.go:117] "RemoveContainer" containerID="2d0f01731a3d5be1f650f4253a6b12a39138367aedcf75156779e68d1c9d0a7f" Jul 21 12:40:22.341078 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:40:22.341042 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-skc26_6713665c-3b71-4751-929e-16de48263766/konnectivity-agent/0.log" limit=1024 Jul 21 12:40:22.442194 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:40:22.442166 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-skc26_6713665c-3b71-4751-929e-16de48263766/konnectivity-agent/0.log" limit=1024 Jul 21 12:40:22.442645 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:40:22.442623 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-6nf7d_a2b48106-9844-4431-9b6f-d4e1baacb15b/ovn-acl-logging/0.log" Jul 21 12:40:22.443087 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:40:22.443063 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-6nf7d" event={"ID":"a2b48106-9844-4431-9b6f-d4e1baacb15b","Type":"ContainerStarted","Data":"7622550cfbd7829ad3e830470cbcc3902b2c2cce45dc9204d98f32958d687c92"} Jul 21 12:40:22.443492 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:40:22.443472 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-6nf7d" Jul 21 12:40:22.443637 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:40:22.443564 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-6nf7d" Jul 21 12:40:22.443637 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:40:22.443618 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-6nf7d" Jul 21 12:40:22.457619 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:40:22.457598 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-6nf7d" Jul 21 12:40:22.459251 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:40:22.459233 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-6nf7d" Jul 21 12:40:22.474933 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:40:22.474876 2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-6nf7d" podStartSLOduration=8.842449063 podStartE2EDuration="28.474864301s" totalImagesPullingTime="19.632415238s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-21 12:39:54 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 12:40:22.474553066 +0000 UTC m=+30.309816933" watchObservedRunningTime="2026-07-21 12:40:22.474864301 +0000 UTC m=+30.310128191" Jul 21 12:40:22.952709 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:40:22.952672 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-skc26_6713665c-3b71-4751-929e-16de48263766/konnectivity-agent/0.log" limit=1024 Jul 21 12:40:23.055710 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:40:23.055683 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-skc26_6713665c-3b71-4751-929e-16de48263766/konnectivity-agent/0.log" limit=1024 Jul 21 12:40:23.134927 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:40:23.134230 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-g2tkf"] Jul 21 12:40:23.134927 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:40:23.134366 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-g2tkf" Jul 21 12:40:23.134927 ip-10-0-130-157 kubenswrapper[2576]: E0721 12:40:23.134505 2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-g2tkf" podUID="4dc9bfad-f2bb-4934-9f49-4cfd0f381ab2" Jul 21 12:40:23.138236 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:40:23.138214 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-l9rfj"] Jul 21 12:40:23.138369 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:40:23.138351 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-l9rfj" Jul 21 12:40:23.138478 ip-10-0-130-157 kubenswrapper[2576]: E0721 12:40:23.138455 2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-l9rfj" podUID="d932e6af-610e-4967-87b1-e489cead334d" Jul 21 12:40:23.139016 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:40:23.138995 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-6f57m"] Jul 21 12:40:23.139092 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:40:23.139082 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-6f57m" Jul 21 12:40:23.139163 ip-10-0-130-157 kubenswrapper[2576]: E0721 12:40:23.139148 2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-6f57m" podUID="ea2d55a6-d43a-4d57-b49c-b63904d8810e" Jul 21 12:40:23.447175 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:40:23.447141 2576 generic.go:356] "Generic (PLEG): container finished" podID="64971bb0-35c9-4026-9680-2a7700efb382" containerID="fc9054f44ef18334676547c740f95caef02a820400a19b257883413d1b251c88" exitCode=0 Jul 21 12:40:23.447576 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:40:23.447234 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-wnhr4" event={"ID":"64971bb0-35c9-4026-9680-2a7700efb382","Type":"ContainerDied","Data":"fc9054f44ef18334676547c740f95caef02a820400a19b257883413d1b251c88"} Jul 21 12:40:24.311820 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:40:24.311794 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-g2tkf" Jul 21 12:40:24.311996 ip-10-0-130-157 kubenswrapper[2576]: E0721 12:40:24.311872 2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-g2tkf" podUID="4dc9bfad-f2bb-4934-9f49-4cfd0f381ab2" Jul 21 12:40:25.311170 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:40:25.310967 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-6f57m" Jul 21 12:40:25.311574 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:40:25.311027 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-l9rfj" Jul 21 12:40:25.311574 ip-10-0-130-157 kubenswrapper[2576]: E0721 12:40:25.311223 2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-6f57m" podUID="ea2d55a6-d43a-4d57-b49c-b63904d8810e" Jul 21 12:40:25.311574 ip-10-0-130-157 kubenswrapper[2576]: E0721 12:40:25.311274 2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-l9rfj" podUID="d932e6af-610e-4967-87b1-e489cead334d" Jul 21 12:40:25.453230 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:40:25.453207 2576 generic.go:356] "Generic (PLEG): container finished" podID="64971bb0-35c9-4026-9680-2a7700efb382" containerID="3af691642077b947c6e953e87e7c63d9098bec0aa2ce082d2efc9f93ff6ec0b2" exitCode=0 Jul 21 12:40:25.453335 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:40:25.453237 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-wnhr4" event={"ID":"64971bb0-35c9-4026-9680-2a7700efb382","Type":"ContainerDied","Data":"3af691642077b947c6e953e87e7c63d9098bec0aa2ce082d2efc9f93ff6ec0b2"} Jul 21 12:40:26.311053 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:40:26.311022 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-g2tkf" Jul 21 12:40:26.311223 ip-10-0-130-157 kubenswrapper[2576]: E0721 12:40:26.311152 2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-g2tkf" podUID="4dc9bfad-f2bb-4934-9f49-4cfd0f381ab2" Jul 21 12:40:26.455768 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:40:26.455705 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-skc26_6713665c-3b71-4751-929e-16de48263766/konnectivity-agent/0.log" limit=1024 Jul 21 12:40:26.558424 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:40:26.558391 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-skc26_6713665c-3b71-4751-929e-16de48263766/konnectivity-agent/0.log" limit=1024 Jul 21 12:40:27.310587 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:40:27.310558 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-6f57m" Jul 21 12:40:27.310587 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:40:27.310577 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-l9rfj" Jul 21 12:40:27.310813 ip-10-0-130-157 kubenswrapper[2576]: E0721 12:40:27.310676 2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-6f57m" podUID="ea2d55a6-d43a-4d57-b49c-b63904d8810e" Jul 21 12:40:27.310871 ip-10-0-130-157 kubenswrapper[2576]: E0721 12:40:27.310822 2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-l9rfj" podUID="d932e6af-610e-4967-87b1-e489cead334d" Jul 21 12:40:27.375491 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:40:27.375460 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d932e6af-610e-4967-87b1-e489cead334d-metrics-certs\") pod \"network-metrics-daemon-l9rfj\" (UID: \"d932e6af-610e-4967-87b1-e489cead334d\") " pod="openshift-multus/network-metrics-daemon-l9rfj" Jul 21 12:40:27.375889 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:40:27.375507 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/4dc9bfad-f2bb-4934-9f49-4cfd0f381ab2-original-pull-secret\") pod \"global-pull-secret-syncer-g2tkf\" (UID: \"4dc9bfad-f2bb-4934-9f49-4cfd0f381ab2\") " pod="kube-system/global-pull-secret-syncer-g2tkf" Jul 21 12:40:27.375889 ip-10-0-130-157 kubenswrapper[2576]: E0721 12:40:27.375605 2576 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 21 12:40:27.375889 ip-10-0-130-157 kubenswrapper[2576]: E0721 12:40:27.375617 2576 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 21 12:40:27.375889 ip-10-0-130-157 kubenswrapper[2576]: E0721 12:40:27.375654 2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4dc9bfad-f2bb-4934-9f49-4cfd0f381ab2-original-pull-secret podName:4dc9bfad-f2bb-4934-9f49-4cfd0f381ab2 nodeName:}" failed. No retries permitted until 2026-07-21 12:40:59.37563812 +0000 UTC m=+67.210901964 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/4dc9bfad-f2bb-4934-9f49-4cfd0f381ab2-original-pull-secret") pod "global-pull-secret-syncer-g2tkf" (UID: "4dc9bfad-f2bb-4934-9f49-4cfd0f381ab2") : object "kube-system"/"original-pull-secret" not registered Jul 21 12:40:27.375889 ip-10-0-130-157 kubenswrapper[2576]: E0721 12:40:27.375678 2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d932e6af-610e-4967-87b1-e489cead334d-metrics-certs podName:d932e6af-610e-4967-87b1-e489cead334d nodeName:}" failed. No retries permitted until 2026-07-21 12:40:59.375661755 +0000 UTC m=+67.210925620 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/d932e6af-610e-4967-87b1-e489cead334d-metrics-certs") pod "network-metrics-daemon-l9rfj" (UID: "d932e6af-610e-4967-87b1-e489cead334d") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 21 12:40:27.476483 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:40:27.476445 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kvc8f\" (UniqueName: \"kubernetes.io/projected/ea2d55a6-d43a-4d57-b49c-b63904d8810e-kube-api-access-kvc8f\") pod \"network-check-target-6f57m\" (UID: \"ea2d55a6-d43a-4d57-b49c-b63904d8810e\") " pod="openshift-network-diagnostics/network-check-target-6f57m" Jul 21 12:40:27.476648 ip-10-0-130-157 kubenswrapper[2576]: E0721 12:40:27.476606 2576 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 21 12:40:27.476648 ip-10-0-130-157 kubenswrapper[2576]: E0721 12:40:27.476632 2576 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 21 12:40:27.476648 ip-10-0-130-157 kubenswrapper[2576]: E0721 12:40:27.476645 2576 projected.go:196] Error preparing data for projected volume kube-api-access-kvc8f for pod openshift-network-diagnostics/network-check-target-6f57m: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 21 12:40:27.476804 ip-10-0-130-157 kubenswrapper[2576]: E0721 12:40:27.476697 2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/ea2d55a6-d43a-4d57-b49c-b63904d8810e-kube-api-access-kvc8f podName:ea2d55a6-d43a-4d57-b49c-b63904d8810e nodeName:}" failed. No retries permitted until 2026-07-21 12:40:59.476680767 +0000 UTC m=+67.311944612 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-kvc8f" (UniqueName: "kubernetes.io/projected/ea2d55a6-d43a-4d57-b49c-b63904d8810e-kube-api-access-kvc8f") pod "network-check-target-6f57m" (UID: "ea2d55a6-d43a-4d57-b49c-b63904d8810e") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 21 12:40:28.314636 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:40:28.314612 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-g2tkf" Jul 21 12:40:28.314791 ip-10-0-130-157 kubenswrapper[2576]: E0721 12:40:28.314719 2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-g2tkf" podUID="4dc9bfad-f2bb-4934-9f49-4cfd0f381ab2" Jul 21 12:40:28.983218 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:40:28.983187 2576 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-157.ec2.internal" event="NodeReady" Jul 21 12:40:28.983833 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:40:28.983342 2576 kubelet_node_status.go:489] "Fast updating node status as it just became ready" Jul 21 12:40:29.031796 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:40:29.031710 2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-6hckk"] Jul 21 12:40:29.036587 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:40:29.036563 2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-bmvqm"] Jul 21 12:40:29.036715 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:40:29.036685 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-6hckk" Jul 21 12:40:29.038551 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:40:29.038507 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-bmvqm" Jul 21 12:40:29.038776 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:40:29.038747 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-h5b4t\"" Jul 21 12:40:29.038925 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:40:29.038901 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\"" Jul 21 12:40:29.039035 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:40:29.038943 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\"" Jul 21 12:40:29.040423 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:40:29.040218 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\"" Jul 21 12:40:29.040423 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:40:29.040248 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-7n7fg\"" Jul 21 12:40:29.040423 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:40:29.040289 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\"" Jul 21 12:40:29.040423 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:40:29.040373 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\"" Jul 21 12:40:29.042632 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:40:29.042612 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-6hckk"] Jul 21 12:40:29.045052 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:40:29.045030 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-bmvqm"] Jul 21 12:40:29.175771 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:40:29.175737 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-skc26_6713665c-3b71-4751-929e-16de48263766/konnectivity-agent/0.log" limit=1024 Jul 21 12:40:29.187770 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:40:29.187743 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/61de9ce0-c295-45c5-b657-2f2c64048340-tmp-dir\") pod \"dns-default-6hckk\" (UID: \"61de9ce0-c295-45c5-b657-2f2c64048340\") " pod="openshift-dns/dns-default-6hckk" Jul 21 12:40:29.187996 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:40:29.187793 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-w76j9\" (UniqueName: \"kubernetes.io/projected/61de9ce0-c295-45c5-b657-2f2c64048340-kube-api-access-w76j9\") pod \"dns-default-6hckk\" (UID: \"61de9ce0-c295-45c5-b657-2f2c64048340\") " pod="openshift-dns/dns-default-6hckk" Jul 21 12:40:29.187996 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:40:29.187825 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/61de9ce0-c295-45c5-b657-2f2c64048340-config-volume\") pod \"dns-default-6hckk\" (UID: \"61de9ce0-c295-45c5-b657-2f2c64048340\") " pod="openshift-dns/dns-default-6hckk" Jul 21 12:40:29.187996 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:40:29.187854 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fp6t2\" (UniqueName: \"kubernetes.io/projected/dea4cd60-192f-49cf-aeaa-d48857637421-kube-api-access-fp6t2\") pod \"ingress-canary-bmvqm\" (UID: \"dea4cd60-192f-49cf-aeaa-d48857637421\") " pod="openshift-ingress-canary/ingress-canary-bmvqm" Jul 21 12:40:29.187996 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:40:29.187951 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/61de9ce0-c295-45c5-b657-2f2c64048340-metrics-tls\") pod \"dns-default-6hckk\" (UID: \"61de9ce0-c295-45c5-b657-2f2c64048340\") " pod="openshift-dns/dns-default-6hckk" Jul 21 12:40:29.188186 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:40:29.188004 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/dea4cd60-192f-49cf-aeaa-d48857637421-cert\") pod \"ingress-canary-bmvqm\" (UID: \"dea4cd60-192f-49cf-aeaa-d48857637421\") " pod="openshift-ingress-canary/ingress-canary-bmvqm" Jul 21 12:40:29.288949 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:40:29.288865 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/61de9ce0-c295-45c5-b657-2f2c64048340-metrics-tls\") pod \"dns-default-6hckk\" (UID: \"61de9ce0-c295-45c5-b657-2f2c64048340\") " pod="openshift-dns/dns-default-6hckk" Jul 21 12:40:29.288949 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:40:29.288928 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/dea4cd60-192f-49cf-aeaa-d48857637421-cert\") pod \"ingress-canary-bmvqm\" (UID: \"dea4cd60-192f-49cf-aeaa-d48857637421\") " pod="openshift-ingress-canary/ingress-canary-bmvqm" Jul 21 12:40:29.289174 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:40:29.288964 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/61de9ce0-c295-45c5-b657-2f2c64048340-tmp-dir\") pod \"dns-default-6hckk\" (UID: \"61de9ce0-c295-45c5-b657-2f2c64048340\") " pod="openshift-dns/dns-default-6hckk" Jul 21 12:40:29.289174 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:40:29.289001 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-w76j9\" (UniqueName: \"kubernetes.io/projected/61de9ce0-c295-45c5-b657-2f2c64048340-kube-api-access-w76j9\") pod \"dns-default-6hckk\" (UID: \"61de9ce0-c295-45c5-b657-2f2c64048340\") " pod="openshift-dns/dns-default-6hckk" Jul 21 12:40:29.289174 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:40:29.289027 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/61de9ce0-c295-45c5-b657-2f2c64048340-config-volume\") pod \"dns-default-6hckk\" (UID: \"61de9ce0-c295-45c5-b657-2f2c64048340\") " pod="openshift-dns/dns-default-6hckk" Jul 21 12:40:29.289174 ip-10-0-130-157 kubenswrapper[2576]: E0721 12:40:29.289037 2576 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 21 12:40:29.289174 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:40:29.289056 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fp6t2\" (UniqueName: \"kubernetes.io/projected/dea4cd60-192f-49cf-aeaa-d48857637421-kube-api-access-fp6t2\") pod \"ingress-canary-bmvqm\" (UID: \"dea4cd60-192f-49cf-aeaa-d48857637421\") " pod="openshift-ingress-canary/ingress-canary-bmvqm" Jul 21 12:40:29.289174 ip-10-0-130-157 kubenswrapper[2576]: E0721 12:40:29.289110 2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/61de9ce0-c295-45c5-b657-2f2c64048340-metrics-tls podName:61de9ce0-c295-45c5-b657-2f2c64048340 nodeName:}" failed. No retries permitted until 2026-07-21 12:40:29.789088792 +0000 UTC m=+37.624352637 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/61de9ce0-c295-45c5-b657-2f2c64048340-metrics-tls") pod "dns-default-6hckk" (UID: "61de9ce0-c295-45c5-b657-2f2c64048340") : secret "dns-default-metrics-tls" not found Jul 21 12:40:29.289492 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:40:29.289304 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/61de9ce0-c295-45c5-b657-2f2c64048340-tmp-dir\") pod \"dns-default-6hckk\" (UID: \"61de9ce0-c295-45c5-b657-2f2c64048340\") " pod="openshift-dns/dns-default-6hckk" Jul 21 12:40:29.289762 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:40:29.289737 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/61de9ce0-c295-45c5-b657-2f2c64048340-config-volume\") pod \"dns-default-6hckk\" (UID: \"61de9ce0-c295-45c5-b657-2f2c64048340\") " pod="openshift-dns/dns-default-6hckk" Jul 21 12:40:29.293674 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:40:29.293651 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/dea4cd60-192f-49cf-aeaa-d48857637421-cert\") pod \"ingress-canary-bmvqm\" (UID: \"dea4cd60-192f-49cf-aeaa-d48857637421\") " pod="openshift-ingress-canary/ingress-canary-bmvqm" Jul 21 12:40:29.297892 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:40:29.297869 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fp6t2\" (UniqueName: \"kubernetes.io/projected/dea4cd60-192f-49cf-aeaa-d48857637421-kube-api-access-fp6t2\") pod \"ingress-canary-bmvqm\" (UID: \"dea4cd60-192f-49cf-aeaa-d48857637421\") " pod="openshift-ingress-canary/ingress-canary-bmvqm" Jul 21 12:40:29.297892 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:40:29.297888 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-w76j9\" (UniqueName: \"kubernetes.io/projected/61de9ce0-c295-45c5-b657-2f2c64048340-kube-api-access-w76j9\") pod \"dns-default-6hckk\" (UID: \"61de9ce0-c295-45c5-b657-2f2c64048340\") " pod="openshift-dns/dns-default-6hckk" Jul 21 12:40:29.310676 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:40:29.310657 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-6f57m" Jul 21 12:40:29.310930 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:40:29.310835 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-l9rfj" Jul 21 12:40:29.312940 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:40:29.312916 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" Jul 21 12:40:29.312940 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:40:29.312934 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Jul 21 12:40:29.313158 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:40:29.313138 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" Jul 21 12:40:29.313289 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:40:29.313270 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-rhfvk\"" Jul 21 12:40:29.313390 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:40:29.313373 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-tqbcg\"" Jul 21 12:40:29.356608 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:40:29.356587 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-bmvqm" Jul 21 12:40:29.794040 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:40:29.793166 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/61de9ce0-c295-45c5-b657-2f2c64048340-metrics-tls\") pod \"dns-default-6hckk\" (UID: \"61de9ce0-c295-45c5-b657-2f2c64048340\") " pod="openshift-dns/dns-default-6hckk" Jul 21 12:40:29.794040 ip-10-0-130-157 kubenswrapper[2576]: E0721 12:40:29.793619 2576 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 21 12:40:29.794040 ip-10-0-130-157 kubenswrapper[2576]: E0721 12:40:29.793705 2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/61de9ce0-c295-45c5-b657-2f2c64048340-metrics-tls podName:61de9ce0-c295-45c5-b657-2f2c64048340 nodeName:}" failed. No retries permitted until 2026-07-21 12:40:30.793683057 +0000 UTC m=+38.628946926 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/61de9ce0-c295-45c5-b657-2f2c64048340-metrics-tls") pod "dns-default-6hckk" (UID: "61de9ce0-c295-45c5-b657-2f2c64048340") : secret "dns-default-metrics-tls" not found Jul 21 12:40:29.835131 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:40:29.835111 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-bmvqm"] Jul 21 12:40:29.838550 ip-10-0-130-157 kubenswrapper[2576]: W0721 12:40:29.838500 2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddea4cd60_192f_49cf_aeaa_d48857637421.slice/crio-fb43e61b00161d71679f5fdd2eaa5ba0d4f3f244ff7aa1e7280007a35aaa35cd WatchSource:0}: Error finding container fb43e61b00161d71679f5fdd2eaa5ba0d4f3f244ff7aa1e7280007a35aaa35cd: Status 404 returned error can't find the container with id fb43e61b00161d71679f5fdd2eaa5ba0d4f3f244ff7aa1e7280007a35aaa35cd Jul 21 12:40:30.310787 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:40:30.310755 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-g2tkf" Jul 21 12:40:30.313116 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:40:30.313095 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Jul 21 12:40:30.469260 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:40:30.469208 2576 generic.go:356] "Generic (PLEG): container finished" podID="64971bb0-35c9-4026-9680-2a7700efb382" containerID="07d4a00778ef75fef3911ca761f8fb9b891a7dfb480bc35294d4f2a7375dbc89" exitCode=0 Jul 21 12:40:30.469421 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:40:30.469289 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-wnhr4" event={"ID":"64971bb0-35c9-4026-9680-2a7700efb382","Type":"ContainerDied","Data":"07d4a00778ef75fef3911ca761f8fb9b891a7dfb480bc35294d4f2a7375dbc89"} Jul 21 12:40:30.470716 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:40:30.470677 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-bmvqm" event={"ID":"dea4cd60-192f-49cf-aeaa-d48857637421","Type":"ContainerStarted","Data":"fb43e61b00161d71679f5fdd2eaa5ba0d4f3f244ff7aa1e7280007a35aaa35cd"} Jul 21 12:40:30.563406 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:40:30.563335 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-skc26_6713665c-3b71-4751-929e-16de48263766/konnectivity-agent/0.log" limit=1024 Jul 21 12:40:30.649877 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:40:30.649840 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-skc26_6713665c-3b71-4751-929e-16de48263766/konnectivity-agent/0.log" limit=1024 Jul 21 12:40:30.736928 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:40:30.736902 2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-ncggx"] Jul 21 12:40:30.739592 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:40:30.739573 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-ncggx" Jul 21 12:40:30.741868 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:40:30.741617 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\"" Jul 21 12:40:30.741868 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:40:30.741845 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\"" Jul 21 12:40:30.742039 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:40:30.741928 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-6mtb2\"" Jul 21 12:40:30.742039 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:40:30.741980 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\"" Jul 21 12:40:30.742153 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:40:30.742057 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\"" Jul 21 12:40:30.752216 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:40:30.752192 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-ncggx"] Jul 21 12:40:30.773957 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:40:30.773908 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-skc26_6713665c-3b71-4751-929e-16de48263766/konnectivity-agent/0.log" limit=1024 Jul 21 12:40:30.802215 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:40:30.802190 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/61de9ce0-c295-45c5-b657-2f2c64048340-metrics-tls\") pod \"dns-default-6hckk\" (UID: \"61de9ce0-c295-45c5-b657-2f2c64048340\") " pod="openshift-dns/dns-default-6hckk" Jul 21 12:40:30.802390 ip-10-0-130-157 kubenswrapper[2576]: E0721 12:40:30.802372 2576 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 21 12:40:30.802461 ip-10-0-130-157 kubenswrapper[2576]: E0721 12:40:30.802451 2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/61de9ce0-c295-45c5-b657-2f2c64048340-metrics-tls podName:61de9ce0-c295-45c5-b657-2f2c64048340 nodeName:}" failed. No retries permitted until 2026-07-21 12:40:32.802431969 +0000 UTC m=+40.637695815 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/61de9ce0-c295-45c5-b657-2f2c64048340-metrics-tls") pod "dns-default-6hckk" (UID: "61de9ce0-c295-45c5-b657-2f2c64048340") : secret "dns-default-metrics-tls" not found Jul 21 12:40:30.903239 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:40:30.903213 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ggvgs\" (UniqueName: \"kubernetes.io/projected/eb329e76-c1b9-4fe2-9cb4-0d85b8a10e9e-kube-api-access-ggvgs\") pod \"insights-runtime-extractor-ncggx\" (UID: \"eb329e76-c1b9-4fe2-9cb4-0d85b8a10e9e\") " pod="openshift-insights/insights-runtime-extractor-ncggx" Jul 21 12:40:30.903381 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:40:30.903250 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/eb329e76-c1b9-4fe2-9cb4-0d85b8a10e9e-crio-socket\") pod \"insights-runtime-extractor-ncggx\" (UID: \"eb329e76-c1b9-4fe2-9cb4-0d85b8a10e9e\") " pod="openshift-insights/insights-runtime-extractor-ncggx" Jul 21 12:40:30.903381 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:40:30.903274 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/eb329e76-c1b9-4fe2-9cb4-0d85b8a10e9e-data-volume\") pod \"insights-runtime-extractor-ncggx\" (UID: \"eb329e76-c1b9-4fe2-9cb4-0d85b8a10e9e\") " pod="openshift-insights/insights-runtime-extractor-ncggx" Jul 21 12:40:30.903381 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:40:30.903353 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/eb329e76-c1b9-4fe2-9cb4-0d85b8a10e9e-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-ncggx\" (UID: \"eb329e76-c1b9-4fe2-9cb4-0d85b8a10e9e\") " pod="openshift-insights/insights-runtime-extractor-ncggx" Jul 21 12:40:30.903506 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:40:30.903390 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/eb329e76-c1b9-4fe2-9cb4-0d85b8a10e9e-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-ncggx\" (UID: \"eb329e76-c1b9-4fe2-9cb4-0d85b8a10e9e\") " pod="openshift-insights/insights-runtime-extractor-ncggx" Jul 21 12:40:31.004728 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:40:31.004695 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/eb329e76-c1b9-4fe2-9cb4-0d85b8a10e9e-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-ncggx\" (UID: \"eb329e76-c1b9-4fe2-9cb4-0d85b8a10e9e\") " pod="openshift-insights/insights-runtime-extractor-ncggx" Jul 21 12:40:31.004872 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:40:31.004792 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ggvgs\" (UniqueName: \"kubernetes.io/projected/eb329e76-c1b9-4fe2-9cb4-0d85b8a10e9e-kube-api-access-ggvgs\") pod \"insights-runtime-extractor-ncggx\" (UID: \"eb329e76-c1b9-4fe2-9cb4-0d85b8a10e9e\") " pod="openshift-insights/insights-runtime-extractor-ncggx" Jul 21 12:40:31.004872 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:40:31.004828 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/eb329e76-c1b9-4fe2-9cb4-0d85b8a10e9e-crio-socket\") pod \"insights-runtime-extractor-ncggx\" (UID: \"eb329e76-c1b9-4fe2-9cb4-0d85b8a10e9e\") " pod="openshift-insights/insights-runtime-extractor-ncggx" Jul 21 12:40:31.004872 ip-10-0-130-157 kubenswrapper[2576]: E0721 12:40:31.004833 2576 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 21 12:40:31.004872 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:40:31.004853 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/eb329e76-c1b9-4fe2-9cb4-0d85b8a10e9e-data-volume\") pod \"insights-runtime-extractor-ncggx\" (UID: \"eb329e76-c1b9-4fe2-9cb4-0d85b8a10e9e\") " pod="openshift-insights/insights-runtime-extractor-ncggx" Jul 21 12:40:31.005061 ip-10-0-130-157 kubenswrapper[2576]: E0721 12:40:31.004897 2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/eb329e76-c1b9-4fe2-9cb4-0d85b8a10e9e-insights-runtime-extractor-tls podName:eb329e76-c1b9-4fe2-9cb4-0d85b8a10e9e nodeName:}" failed. No retries permitted until 2026-07-21 12:40:31.504877416 +0000 UTC m=+39.340141264 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/eb329e76-c1b9-4fe2-9cb4-0d85b8a10e9e-insights-runtime-extractor-tls") pod "insights-runtime-extractor-ncggx" (UID: "eb329e76-c1b9-4fe2-9cb4-0d85b8a10e9e") : secret "insights-runtime-extractor-tls" not found Jul 21 12:40:31.005061 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:40:31.004929 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/eb329e76-c1b9-4fe2-9cb4-0d85b8a10e9e-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-ncggx\" (UID: \"eb329e76-c1b9-4fe2-9cb4-0d85b8a10e9e\") " pod="openshift-insights/insights-runtime-extractor-ncggx" Jul 21 12:40:31.005061 ip-10-0-130-157 kubenswrapper[2576]: E0721 12:40:31.004986 2576 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 21 12:40:31.005061 ip-10-0-130-157 kubenswrapper[2576]: E0721 12:40:31.005026 2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/eb329e76-c1b9-4fe2-9cb4-0d85b8a10e9e-kube-rbac-proxy-cm podName:eb329e76-c1b9-4fe2-9cb4-0d85b8a10e9e nodeName:}" failed. No retries permitted until 2026-07-21 12:40:31.505014234 +0000 UTC m=+39.340278091 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/eb329e76-c1b9-4fe2-9cb4-0d85b8a10e9e-kube-rbac-proxy-cm") pod "insights-runtime-extractor-ncggx" (UID: "eb329e76-c1b9-4fe2-9cb4-0d85b8a10e9e") : configmap "kube-rbac-proxy" not found Jul 21 12:40:31.005260 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:40:31.005129 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/eb329e76-c1b9-4fe2-9cb4-0d85b8a10e9e-crio-socket\") pod \"insights-runtime-extractor-ncggx\" (UID: \"eb329e76-c1b9-4fe2-9cb4-0d85b8a10e9e\") " pod="openshift-insights/insights-runtime-extractor-ncggx" Jul 21 12:40:31.005316 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:40:31.005257 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/eb329e76-c1b9-4fe2-9cb4-0d85b8a10e9e-data-volume\") pod \"insights-runtime-extractor-ncggx\" (UID: \"eb329e76-c1b9-4fe2-9cb4-0d85b8a10e9e\") " pod="openshift-insights/insights-runtime-extractor-ncggx" Jul 21 12:40:31.015529 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:40:31.015494 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ggvgs\" (UniqueName: \"kubernetes.io/projected/eb329e76-c1b9-4fe2-9cb4-0d85b8a10e9e-kube-api-access-ggvgs\") pod \"insights-runtime-extractor-ncggx\" (UID: \"eb329e76-c1b9-4fe2-9cb4-0d85b8a10e9e\") " pod="openshift-insights/insights-runtime-extractor-ncggx" Jul 21 12:40:31.037589 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:40:31.037542 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-skc26_6713665c-3b71-4751-929e-16de48263766/konnectivity-agent/0.log" limit=1024 Jul 21 12:40:31.106573 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:40:31.106497 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-skc26_6713665c-3b71-4751-929e-16de48263766/konnectivity-agent/0.log" limit=1024 Jul 21 12:40:31.476681 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:40:31.476647 2576 generic.go:356] "Generic (PLEG): container finished" podID="64971bb0-35c9-4026-9680-2a7700efb382" containerID="5ce0d89f58d2a3be443e1bbb8ab69fd4ed30567d4ebcbbe7409cb7f898ee8200" exitCode=0 Jul 21 12:40:31.477143 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:40:31.476713 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-wnhr4" event={"ID":"64971bb0-35c9-4026-9680-2a7700efb382","Type":"ContainerDied","Data":"5ce0d89f58d2a3be443e1bbb8ab69fd4ed30567d4ebcbbe7409cb7f898ee8200"} Jul 21 12:40:31.508254 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:40:31.508220 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/eb329e76-c1b9-4fe2-9cb4-0d85b8a10e9e-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-ncggx\" (UID: \"eb329e76-c1b9-4fe2-9cb4-0d85b8a10e9e\") " pod="openshift-insights/insights-runtime-extractor-ncggx" Jul 21 12:40:31.508379 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:40:31.508337 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/eb329e76-c1b9-4fe2-9cb4-0d85b8a10e9e-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-ncggx\" (UID: \"eb329e76-c1b9-4fe2-9cb4-0d85b8a10e9e\") " pod="openshift-insights/insights-runtime-extractor-ncggx" Jul 21 12:40:31.508379 ip-10-0-130-157 kubenswrapper[2576]: E0721 12:40:31.508374 2576 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 21 12:40:31.508489 ip-10-0-130-157 kubenswrapper[2576]: E0721 12:40:31.508421 2576 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 21 12:40:31.508489 ip-10-0-130-157 kubenswrapper[2576]: E0721 12:40:31.508443 2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/eb329e76-c1b9-4fe2-9cb4-0d85b8a10e9e-insights-runtime-extractor-tls podName:eb329e76-c1b9-4fe2-9cb4-0d85b8a10e9e nodeName:}" failed. No retries permitted until 2026-07-21 12:40:32.508421056 +0000 UTC m=+40.343684904 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/eb329e76-c1b9-4fe2-9cb4-0d85b8a10e9e-insights-runtime-extractor-tls") pod "insights-runtime-extractor-ncggx" (UID: "eb329e76-c1b9-4fe2-9cb4-0d85b8a10e9e") : secret "insights-runtime-extractor-tls" not found Jul 21 12:40:31.508489 ip-10-0-130-157 kubenswrapper[2576]: E0721 12:40:31.508482 2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/eb329e76-c1b9-4fe2-9cb4-0d85b8a10e9e-kube-rbac-proxy-cm podName:eb329e76-c1b9-4fe2-9cb4-0d85b8a10e9e nodeName:}" failed. No retries permitted until 2026-07-21 12:40:32.508464315 +0000 UTC m=+40.343728177 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/eb329e76-c1b9-4fe2-9cb4-0d85b8a10e9e-kube-rbac-proxy-cm") pod "insights-runtime-extractor-ncggx" (UID: "eb329e76-c1b9-4fe2-9cb4-0d85b8a10e9e") : configmap "kube-rbac-proxy" not found Jul 21 12:40:32.369829 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:40:32.369673 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-skc26_6713665c-3b71-4751-929e-16de48263766/konnectivity-agent/0.log" limit=1024 Jul 21 12:40:32.470742 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:40:32.470681 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-skc26_6713665c-3b71-4751-929e-16de48263766/konnectivity-agent/0.log" limit=1024 Jul 21 12:40:32.481817 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:40:32.481797 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-wnhr4" event={"ID":"64971bb0-35c9-4026-9680-2a7700efb382","Type":"ContainerStarted","Data":"0a3fc6a1913fdec10f7d4c896ff96ab16249f74bd2dd65a8a440c55557d56ba5"} Jul 21 12:40:32.516375 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:40:32.516354 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/eb329e76-c1b9-4fe2-9cb4-0d85b8a10e9e-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-ncggx\" (UID: \"eb329e76-c1b9-4fe2-9cb4-0d85b8a10e9e\") " pod="openshift-insights/insights-runtime-extractor-ncggx" Jul 21 12:40:32.516481 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:40:32.516468 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/eb329e76-c1b9-4fe2-9cb4-0d85b8a10e9e-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-ncggx\" (UID: \"eb329e76-c1b9-4fe2-9cb4-0d85b8a10e9e\") " pod="openshift-insights/insights-runtime-extractor-ncggx" Jul 21 12:40:32.516566 ip-10-0-130-157 kubenswrapper[2576]: E0721 12:40:32.516486 2576 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 21 12:40:32.516566 ip-10-0-130-157 kubenswrapper[2576]: E0721 12:40:32.516549 2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/eb329e76-c1b9-4fe2-9cb4-0d85b8a10e9e-insights-runtime-extractor-tls podName:eb329e76-c1b9-4fe2-9cb4-0d85b8a10e9e nodeName:}" failed. No retries permitted until 2026-07-21 12:40:34.51653532 +0000 UTC m=+42.351799165 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/eb329e76-c1b9-4fe2-9cb4-0d85b8a10e9e-insights-runtime-extractor-tls") pod "insights-runtime-extractor-ncggx" (UID: "eb329e76-c1b9-4fe2-9cb4-0d85b8a10e9e") : secret "insights-runtime-extractor-tls" not found Jul 21 12:40:32.516671 ip-10-0-130-157 kubenswrapper[2576]: E0721 12:40:32.516593 2576 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 21 12:40:32.516671 ip-10-0-130-157 kubenswrapper[2576]: E0721 12:40:32.516648 2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/eb329e76-c1b9-4fe2-9cb4-0d85b8a10e9e-kube-rbac-proxy-cm podName:eb329e76-c1b9-4fe2-9cb4-0d85b8a10e9e nodeName:}" failed. No retries permitted until 2026-07-21 12:40:34.516632074 +0000 UTC m=+42.351895938 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/eb329e76-c1b9-4fe2-9cb4-0d85b8a10e9e-kube-rbac-proxy-cm") pod "insights-runtime-extractor-ncggx" (UID: "eb329e76-c1b9-4fe2-9cb4-0d85b8a10e9e") : configmap "kube-rbac-proxy" not found Jul 21 12:40:32.526340 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:40:32.526283 2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-wnhr4" podStartSLOduration=7.764011741 podStartE2EDuration="38.526263617s" totalImagesPullingTime="30.762251876s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-21 12:39:54 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 12:40:32.524860836 +0000 UTC m=+40.360124701" watchObservedRunningTime="2026-07-21 12:40:32.526263617 +0000 UTC m=+40.361527483" Jul 21 12:40:32.534360 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:40:32.534344 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-6crjb_c70252ec-a8bb-4dd0-9544-2ff28e109eb9/dns-node-resolver/0.log" Jul 21 12:40:32.671814 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:40:32.671785 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-skc26_6713665c-3b71-4751-929e-16de48263766/konnectivity-agent/0.log" limit=1024 Jul 21 12:40:32.772323 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:40:32.772252 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-skc26_6713665c-3b71-4751-929e-16de48263766/konnectivity-agent/0.log" limit=1024 Jul 21 12:40:32.819436 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:40:32.819411 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/61de9ce0-c295-45c5-b657-2f2c64048340-metrics-tls\") pod \"dns-default-6hckk\" (UID: \"61de9ce0-c295-45c5-b657-2f2c64048340\") " pod="openshift-dns/dns-default-6hckk" Jul 21 12:40:32.819587 ip-10-0-130-157 kubenswrapper[2576]: E0721 12:40:32.819571 2576 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 21 12:40:32.819644 ip-10-0-130-157 kubenswrapper[2576]: E0721 12:40:32.819634 2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/61de9ce0-c295-45c5-b657-2f2c64048340-metrics-tls podName:61de9ce0-c295-45c5-b657-2f2c64048340 nodeName:}" failed. No retries permitted until 2026-07-21 12:40:36.819618716 +0000 UTC m=+44.654882559 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/61de9ce0-c295-45c5-b657-2f2c64048340-metrics-tls") pod "dns-default-6hckk" (UID: "61de9ce0-c295-45c5-b657-2f2c64048340") : secret "dns-default-metrics-tls" not found Jul 21 12:40:32.831364 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:40:32.831333 2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-kube-storage-version-migrator/migrator-7cc9c7ffd7-gh6sz"] Jul 21 12:40:32.852213 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:40:32.852191 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator/migrator-7cc9c7ffd7-gh6sz"] Jul 21 12:40:32.852320 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:40:32.852306 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator/migrator-7cc9c7ffd7-gh6sz" Jul 21 12:40:32.855484 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:40:32.855463 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator\"/\"kube-root-ca.crt\"" Jul 21 12:40:32.855601 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:40:32.855585 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator\"/\"openshift-service-ca.crt\"" Jul 21 12:40:32.855758 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:40:32.855742 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator\"/\"kube-storage-version-migrator-sa-dockercfg-fvfm4\"" Jul 21 12:40:32.918940 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:40:32.918920 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-skc26_6713665c-3b71-4751-929e-16de48263766/konnectivity-agent/0.log" limit=1024 Jul 21 12:40:32.984638 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:40:32.984616 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-skc26_6713665c-3b71-4751-929e-16de48263766/konnectivity-agent/0.log" limit=1024 Jul 21 12:40:33.020714 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:40:33.020695 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-z582c\" (UniqueName: \"kubernetes.io/projected/2a7b9cbf-a3b9-4602-961d-73d18b1e3700-kube-api-access-z582c\") pod \"migrator-7cc9c7ffd7-gh6sz\" (UID: \"2a7b9cbf-a3b9-4602-961d-73d18b1e3700\") " pod="openshift-kube-storage-version-migrator/migrator-7cc9c7ffd7-gh6sz" Jul 21 12:40:33.121297 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:40:33.121248 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-z582c\" (UniqueName: \"kubernetes.io/projected/2a7b9cbf-a3b9-4602-961d-73d18b1e3700-kube-api-access-z582c\") pod \"migrator-7cc9c7ffd7-gh6sz\" (UID: \"2a7b9cbf-a3b9-4602-961d-73d18b1e3700\") " pod="openshift-kube-storage-version-migrator/migrator-7cc9c7ffd7-gh6sz" Jul 21 12:40:33.136630 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:40:33.136610 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-z582c\" (UniqueName: \"kubernetes.io/projected/2a7b9cbf-a3b9-4602-961d-73d18b1e3700-kube-api-access-z582c\") pod \"migrator-7cc9c7ffd7-gh6sz\" (UID: \"2a7b9cbf-a3b9-4602-961d-73d18b1e3700\") " pod="openshift-kube-storage-version-migrator/migrator-7cc9c7ffd7-gh6sz" Jul 21 12:40:33.141806 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:40:33.141783 2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-service-ca/service-ca-58598695f7-qp4sr"] Jul 21 12:40:33.156283 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:40:33.156265 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca/service-ca-58598695f7-qp4sr" Jul 21 12:40:33.156390 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:40:33.156273 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca/service-ca-58598695f7-qp4sr"] Jul 21 12:40:33.158279 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:40:33.158256 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"openshift-service-ca.crt\"" Jul 21 12:40:33.158474 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:40:33.158459 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"signing-cabundle\"" Jul 21 12:40:33.158557 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:40:33.158534 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca\"/\"signing-key\"" Jul 21 12:40:33.158927 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:40:33.158909 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca\"/\"service-ca-dockercfg-h4hgn\"" Jul 21 12:40:33.158979 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:40:33.158935 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"kube-root-ca.crt\"" Jul 21 12:40:33.161335 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:40:33.161316 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator/migrator-7cc9c7ffd7-gh6sz" Jul 21 12:40:33.176541 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:40:33.176505 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-skc26_6713665c-3b71-4751-929e-16de48263766/konnectivity-agent/0.log" limit=1024 Jul 21 12:40:33.297432 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:40:33.297406 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator/migrator-7cc9c7ffd7-gh6sz"] Jul 21 12:40:33.300956 ip-10-0-130-157 kubenswrapper[2576]: W0721 12:40:33.300931 2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2a7b9cbf_a3b9_4602_961d_73d18b1e3700.slice/crio-c426f98ef62d6c1347d5a2088462779fc0a61ceed04ffad53c4097f6e6e16e95 WatchSource:0}: Error finding container c426f98ef62d6c1347d5a2088462779fc0a61ceed04ffad53c4097f6e6e16e95: Status 404 returned error can't find the container with id c426f98ef62d6c1347d5a2088462779fc0a61ceed04ffad53c4097f6e6e16e95 Jul 21 12:40:33.314317 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:40:33.314292 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-skc26_6713665c-3b71-4751-929e-16de48263766/konnectivity-agent/0.log" limit=1024 Jul 21 12:40:33.322464 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:40:33.322443 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/5e0ce41e-4647-485b-bede-5c3b146a49df-signing-key\") pod \"service-ca-58598695f7-qp4sr\" (UID: \"5e0ce41e-4647-485b-bede-5c3b146a49df\") " pod="openshift-service-ca/service-ca-58598695f7-qp4sr" Jul 21 12:40:33.322586 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:40:33.322470 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-z7xgv\" (UniqueName: \"kubernetes.io/projected/5e0ce41e-4647-485b-bede-5c3b146a49df-kube-api-access-z7xgv\") pod \"service-ca-58598695f7-qp4sr\" (UID: \"5e0ce41e-4647-485b-bede-5c3b146a49df\") " pod="openshift-service-ca/service-ca-58598695f7-qp4sr" Jul 21 12:40:33.322651 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:40:33.322635 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/5e0ce41e-4647-485b-bede-5c3b146a49df-signing-cabundle\") pod \"service-ca-58598695f7-qp4sr\" (UID: \"5e0ce41e-4647-485b-bede-5c3b146a49df\") " pod="openshift-service-ca/service-ca-58598695f7-qp4sr" Jul 21 12:40:33.423698 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:40:33.423672 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/5e0ce41e-4647-485b-bede-5c3b146a49df-signing-key\") pod \"service-ca-58598695f7-qp4sr\" (UID: \"5e0ce41e-4647-485b-bede-5c3b146a49df\") " pod="openshift-service-ca/service-ca-58598695f7-qp4sr" Jul 21 12:40:33.423809 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:40:33.423706 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-z7xgv\" (UniqueName: \"kubernetes.io/projected/5e0ce41e-4647-485b-bede-5c3b146a49df-kube-api-access-z7xgv\") pod \"service-ca-58598695f7-qp4sr\" (UID: \"5e0ce41e-4647-485b-bede-5c3b146a49df\") " pod="openshift-service-ca/service-ca-58598695f7-qp4sr" Jul 21 12:40:33.423892 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:40:33.423843 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/5e0ce41e-4647-485b-bede-5c3b146a49df-signing-cabundle\") pod \"service-ca-58598695f7-qp4sr\" (UID: \"5e0ce41e-4647-485b-bede-5c3b146a49df\") " pod="openshift-service-ca/service-ca-58598695f7-qp4sr" Jul 21 12:40:33.424566 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:40:33.424547 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/5e0ce41e-4647-485b-bede-5c3b146a49df-signing-cabundle\") pod \"service-ca-58598695f7-qp4sr\" (UID: \"5e0ce41e-4647-485b-bede-5c3b146a49df\") " pod="openshift-service-ca/service-ca-58598695f7-qp4sr" Jul 21 12:40:33.425873 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:40:33.425855 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/5e0ce41e-4647-485b-bede-5c3b146a49df-signing-key\") pod \"service-ca-58598695f7-qp4sr\" (UID: \"5e0ce41e-4647-485b-bede-5c3b146a49df\") " pod="openshift-service-ca/service-ca-58598695f7-qp4sr" Jul 21 12:40:33.432592 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:40:33.432572 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-z7xgv\" (UniqueName: \"kubernetes.io/projected/5e0ce41e-4647-485b-bede-5c3b146a49df-kube-api-access-z7xgv\") pod \"service-ca-58598695f7-qp4sr\" (UID: \"5e0ce41e-4647-485b-bede-5c3b146a49df\") " pod="openshift-service-ca/service-ca-58598695f7-qp4sr" Jul 21 12:40:33.464654 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:40:33.464633 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca/service-ca-58598695f7-qp4sr" Jul 21 12:40:33.485756 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:40:33.485736 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-bmvqm" event={"ID":"dea4cd60-192f-49cf-aeaa-d48857637421","Type":"ContainerStarted","Data":"91c141def15841fb7bf3c171a068402449d7d6186c280e8dffc5a3f83c6274a9"} Jul 21 12:40:33.486855 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:40:33.486832 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-7cc9c7ffd7-gh6sz" event={"ID":"2a7b9cbf-a3b9-4602-961d-73d18b1e3700","Type":"ContainerStarted","Data":"c426f98ef62d6c1347d5a2088462779fc0a61ceed04ffad53c4097f6e6e16e95"} Jul 21 12:40:33.503973 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:40:33.503934 2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-bmvqm" podStartSLOduration=1.867373825 podStartE2EDuration="4.503920504s" totalImagesPullingTime="2.636546679s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 12:40:29 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 12:40:33.502882903 +0000 UTC m=+41.338146770" watchObservedRunningTime="2026-07-21 12:40:33.503920504 +0000 UTC m=+41.339184348" Jul 21 12:40:33.611988 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:40:33.611962 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca/service-ca-58598695f7-qp4sr"] Jul 21 12:40:33.614791 ip-10-0-130-157 kubenswrapper[2576]: W0721 12:40:33.614769 2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5e0ce41e_4647_485b_bede_5c3b146a49df.slice/crio-21ec92b60aa5e83a434868b22713680502c5b189c15b2b41dc6c38299c18b65b WatchSource:0}: Error finding container 21ec92b60aa5e83a434868b22713680502c5b189c15b2b41dc6c38299c18b65b: Status 404 returned error can't find the container with id 21ec92b60aa5e83a434868b22713680502c5b189c15b2b41dc6c38299c18b65b Jul 21 12:40:33.735041 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:40:33.734991 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-km4fw_9cfd0727-8ed4-40bd-b1d5-66a531560bce/node-ca/0.log" Jul 21 12:40:34.022053 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:40:34.021977 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-skc26_6713665c-3b71-4751-929e-16de48263766/konnectivity-agent/0.log" limit=1024 Jul 21 12:40:34.491027 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:40:34.490969 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca/service-ca-58598695f7-qp4sr" event={"ID":"5e0ce41e-4647-485b-bede-5c3b146a49df","Type":"ContainerStarted","Data":"21ec92b60aa5e83a434868b22713680502c5b189c15b2b41dc6c38299c18b65b"} Jul 21 12:40:34.533668 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:40:34.533634 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/eb329e76-c1b9-4fe2-9cb4-0d85b8a10e9e-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-ncggx\" (UID: \"eb329e76-c1b9-4fe2-9cb4-0d85b8a10e9e\") " pod="openshift-insights/insights-runtime-extractor-ncggx" Jul 21 12:40:34.533832 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:40:34.533757 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/eb329e76-c1b9-4fe2-9cb4-0d85b8a10e9e-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-ncggx\" (UID: \"eb329e76-c1b9-4fe2-9cb4-0d85b8a10e9e\") " pod="openshift-insights/insights-runtime-extractor-ncggx" Jul 21 12:40:34.533895 ip-10-0-130-157 kubenswrapper[2576]: E0721 12:40:34.533857 2576 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 21 12:40:34.533954 ip-10-0-130-157 kubenswrapper[2576]: E0721 12:40:34.533916 2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/eb329e76-c1b9-4fe2-9cb4-0d85b8a10e9e-kube-rbac-proxy-cm podName:eb329e76-c1b9-4fe2-9cb4-0d85b8a10e9e nodeName:}" failed. No retries permitted until 2026-07-21 12:40:38.53389765 +0000 UTC m=+46.369161510 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/eb329e76-c1b9-4fe2-9cb4-0d85b8a10e9e-kube-rbac-proxy-cm") pod "insights-runtime-extractor-ncggx" (UID: "eb329e76-c1b9-4fe2-9cb4-0d85b8a10e9e") : configmap "kube-rbac-proxy" not found Jul 21 12:40:34.534146 ip-10-0-130-157 kubenswrapper[2576]: E0721 12:40:34.534121 2576 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 21 12:40:34.534262 ip-10-0-130-157 kubenswrapper[2576]: E0721 12:40:34.534186 2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/eb329e76-c1b9-4fe2-9cb4-0d85b8a10e9e-insights-runtime-extractor-tls podName:eb329e76-c1b9-4fe2-9cb4-0d85b8a10e9e nodeName:}" failed. No retries permitted until 2026-07-21 12:40:38.534169036 +0000 UTC m=+46.369432882 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/eb329e76-c1b9-4fe2-9cb4-0d85b8a10e9e-insights-runtime-extractor-tls") pod "insights-runtime-extractor-ncggx" (UID: "eb329e76-c1b9-4fe2-9cb4-0d85b8a10e9e") : secret "insights-runtime-extractor-tls" not found Jul 21 12:40:34.535586 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:40:34.535567 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-bmvqm_dea4cd60-192f-49cf-aeaa-d48857637421/serve-healthcheck-canary/0.log" Jul 21 12:40:36.497870 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:40:36.497788 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-7cc9c7ffd7-gh6sz" event={"ID":"2a7b9cbf-a3b9-4602-961d-73d18b1e3700","Type":"ContainerStarted","Data":"3fd3e8d7c3f931f3a8091e8650aae4ded5d4010265496bfb40c1f598d27fb301"} Jul 21 12:40:36.497870 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:40:36.497843 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-7cc9c7ffd7-gh6sz" event={"ID":"2a7b9cbf-a3b9-4602-961d-73d18b1e3700","Type":"ContainerStarted","Data":"f64a3405c0ff9a85168d86780a22acfa9b05f80a8ad8c71cd8ff25abda4a21ce"} Jul 21 12:40:36.499264 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:40:36.499242 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca/service-ca-58598695f7-qp4sr" event={"ID":"5e0ce41e-4647-485b-bede-5c3b146a49df","Type":"ContainerStarted","Data":"74c02712353ead035b97e2bf49c54980c26bc0405bafedaa526946ed8631d563"} Jul 21 12:40:36.517504 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:40:36.517457 2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-kube-storage-version-migrator/migrator-7cc9c7ffd7-gh6sz" podStartSLOduration=1.9600386520000002 podStartE2EDuration="4.517439884s" totalImagesPullingTime="2.557401232s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 12:40:32 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 12:40:36.51668235 +0000 UTC m=+44.351946216" watchObservedRunningTime="2026-07-21 12:40:36.517439884 +0000 UTC m=+44.352703751" Jul 21 12:40:36.532409 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:40:36.532370 2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-service-ca/service-ca-58598695f7-qp4sr" podStartSLOduration=0.933613614 podStartE2EDuration="3.532355907s" totalImagesPullingTime="2.598742293s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 12:40:33 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 12:40:36.531642394 +0000 UTC m=+44.366906261" watchObservedRunningTime="2026-07-21 12:40:36.532355907 +0000 UTC m=+44.367619773" Jul 21 12:40:36.850239 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:40:36.850166 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/61de9ce0-c295-45c5-b657-2f2c64048340-metrics-tls\") pod \"dns-default-6hckk\" (UID: \"61de9ce0-c295-45c5-b657-2f2c64048340\") " pod="openshift-dns/dns-default-6hckk" Jul 21 12:40:36.850378 ip-10-0-130-157 kubenswrapper[2576]: E0721 12:40:36.850292 2576 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 21 12:40:36.850378 ip-10-0-130-157 kubenswrapper[2576]: E0721 12:40:36.850353 2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/61de9ce0-c295-45c5-b657-2f2c64048340-metrics-tls podName:61de9ce0-c295-45c5-b657-2f2c64048340 nodeName:}" failed. No retries permitted until 2026-07-21 12:40:44.850337083 +0000 UTC m=+52.685600949 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/61de9ce0-c295-45c5-b657-2f2c64048340-metrics-tls") pod "dns-default-6hckk" (UID: "61de9ce0-c295-45c5-b657-2f2c64048340") : secret "dns-default-metrics-tls" not found Jul 21 12:40:38.095059 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:40:38.095022 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-skc26_6713665c-3b71-4751-929e-16de48263766/konnectivity-agent/0.log" limit=1024 Jul 21 12:40:38.228760 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:40:38.228721 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-skc26_6713665c-3b71-4751-929e-16de48263766/konnectivity-agent/0.log" limit=1024 Jul 21 12:40:38.393193 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:40:38.393139 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-skc26_6713665c-3b71-4751-929e-16de48263766/konnectivity-agent/0.log" limit=1024 Jul 21 12:40:38.494658 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:40:38.494603 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-skc26_6713665c-3b71-4751-929e-16de48263766/konnectivity-agent/0.log" limit=1024 Jul 21 12:40:38.566251 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:40:38.566222 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/eb329e76-c1b9-4fe2-9cb4-0d85b8a10e9e-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-ncggx\" (UID: \"eb329e76-c1b9-4fe2-9cb4-0d85b8a10e9e\") " pod="openshift-insights/insights-runtime-extractor-ncggx" Jul 21 12:40:38.566389 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:40:38.566335 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/eb329e76-c1b9-4fe2-9cb4-0d85b8a10e9e-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-ncggx\" (UID: \"eb329e76-c1b9-4fe2-9cb4-0d85b8a10e9e\") " pod="openshift-insights/insights-runtime-extractor-ncggx" Jul 21 12:40:38.566389 ip-10-0-130-157 kubenswrapper[2576]: E0721 12:40:38.566350 2576 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 21 12:40:38.566502 ip-10-0-130-157 kubenswrapper[2576]: E0721 12:40:38.566416 2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/eb329e76-c1b9-4fe2-9cb4-0d85b8a10e9e-insights-runtime-extractor-tls podName:eb329e76-c1b9-4fe2-9cb4-0d85b8a10e9e nodeName:}" failed. No retries permitted until 2026-07-21 12:40:46.566397506 +0000 UTC m=+54.401661350 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/eb329e76-c1b9-4fe2-9cb4-0d85b8a10e9e-insights-runtime-extractor-tls") pod "insights-runtime-extractor-ncggx" (UID: "eb329e76-c1b9-4fe2-9cb4-0d85b8a10e9e") : secret "insights-runtime-extractor-tls" not found Jul 21 12:40:38.566502 ip-10-0-130-157 kubenswrapper[2576]: E0721 12:40:38.566419 2576 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 21 12:40:38.566502 ip-10-0-130-157 kubenswrapper[2576]: E0721 12:40:38.566467 2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/eb329e76-c1b9-4fe2-9cb4-0d85b8a10e9e-kube-rbac-proxy-cm podName:eb329e76-c1b9-4fe2-9cb4-0d85b8a10e9e nodeName:}" failed. No retries permitted until 2026-07-21 12:40:46.566453033 +0000 UTC m=+54.401716886 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/eb329e76-c1b9-4fe2-9cb4-0d85b8a10e9e-kube-rbac-proxy-cm") pod "insights-runtime-extractor-ncggx" (UID: "eb329e76-c1b9-4fe2-9cb4-0d85b8a10e9e") : configmap "kube-rbac-proxy" not found Jul 21 12:40:38.592755 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:40:38.592725 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-skc26_6713665c-3b71-4751-929e-16de48263766/konnectivity-agent/0.log" limit=1024 Jul 21 12:40:39.014980 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:40:39.014947 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-skc26_6713665c-3b71-4751-929e-16de48263766/konnectivity-agent/0.log" limit=1024 Jul 21 12:40:39.389568 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:40:39.389536 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-skc26_6713665c-3b71-4751-929e-16de48263766/konnectivity-agent/0.log" limit=1024 Jul 21 12:40:40.198579 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:40:40.198545 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-skc26_6713665c-3b71-4751-929e-16de48263766/konnectivity-agent/0.log" limit=1024 Jul 21 12:40:40.630482 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:40:40.630393 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-skc26_6713665c-3b71-4751-929e-16de48263766/konnectivity-agent/0.log" limit=1024 Jul 21 12:40:40.779344 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:40:40.779316 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-skc26_6713665c-3b71-4751-929e-16de48263766/konnectivity-agent/0.log" limit=1024 Jul 21 12:40:40.981750 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:40:40.981725 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-skc26_6713665c-3b71-4751-929e-16de48263766/konnectivity-agent/0.log" limit=1024 Jul 21 12:40:41.182340 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:40:41.182310 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-skc26_6713665c-3b71-4751-929e-16de48263766/konnectivity-agent/0.log" limit=1024 Jul 21 12:40:41.787752 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:40:41.787720 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-skc26_6713665c-3b71-4751-929e-16de48263766/konnectivity-agent/0.log" limit=1024 Jul 21 12:40:42.017216 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:40:42.017184 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-skc26_6713665c-3b71-4751-929e-16de48263766/konnectivity-agent/0.log" limit=1024 Jul 21 12:40:42.187489 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:40:42.187461 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-skc26_6713665c-3b71-4751-929e-16de48263766/konnectivity-agent/0.log" limit=1024 Jul 21 12:40:42.382939 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:40:42.382898 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-skc26_6713665c-3b71-4751-929e-16de48263766/konnectivity-agent/0.log" limit=1024 Jul 21 12:40:42.597473 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:40:42.597395 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-skc26_6713665c-3b71-4751-929e-16de48263766/konnectivity-agent/0.log" limit=1024 Jul 21 12:40:42.665225 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:40:42.665196 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-skc26_6713665c-3b71-4751-929e-16de48263766/konnectivity-agent/0.log" limit=1024 Jul 21 12:40:42.984387 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:40:42.984360 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-skc26_6713665c-3b71-4751-929e-16de48263766/konnectivity-agent/0.log" limit=1024 Jul 21 12:40:43.581225 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:40:43.581194 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-skc26_6713665c-3b71-4751-929e-16de48263766/konnectivity-agent/0.log" limit=1024 Jul 21 12:40:43.986390 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:40:43.986359 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-skc26_6713665c-3b71-4751-929e-16de48263766/konnectivity-agent/0.log" limit=1024 Jul 21 12:40:44.271932 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:40:44.271861 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-skc26_6713665c-3b71-4751-929e-16de48263766/konnectivity-agent/0.log" limit=1024 Jul 21 12:40:44.396418 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:40:44.396384 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-skc26_6713665c-3b71-4751-929e-16de48263766/konnectivity-agent/0.log" limit=1024 Jul 21 12:40:44.821949 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:40:44.821921 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-skc26_6713665c-3b71-4751-929e-16de48263766/konnectivity-agent/0.log" limit=1024 Jul 21 12:40:44.912065 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:40:44.912042 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/61de9ce0-c295-45c5-b657-2f2c64048340-metrics-tls\") pod \"dns-default-6hckk\" (UID: \"61de9ce0-c295-45c5-b657-2f2c64048340\") " pod="openshift-dns/dns-default-6hckk" Jul 21 12:40:44.914194 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:40:44.914172 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/61de9ce0-c295-45c5-b657-2f2c64048340-metrics-tls\") pod \"dns-default-6hckk\" (UID: \"61de9ce0-c295-45c5-b657-2f2c64048340\") " pod="openshift-dns/dns-default-6hckk" Jul 21 12:40:44.950604 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:40:44.950583 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-6hckk" Jul 21 12:40:45.010530 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:40:45.010494 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-skc26_6713665c-3b71-4751-929e-16de48263766/konnectivity-agent/0.log" limit=1024 Jul 21 12:40:45.094771 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:40:45.094711 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-6hckk"] Jul 21 12:40:45.097821 ip-10-0-130-157 kubenswrapper[2576]: W0721 12:40:45.097792 2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod61de9ce0_c295_45c5_b657_2f2c64048340.slice/crio-57c1e83158b66e8253b4b1f8099260eca12acbe6b1d51e6db31814e0f3a8a66d WatchSource:0}: Error finding container 57c1e83158b66e8253b4b1f8099260eca12acbe6b1d51e6db31814e0f3a8a66d: Status 404 returned error can't find the container with id 57c1e83158b66e8253b4b1f8099260eca12acbe6b1d51e6db31814e0f3a8a66d Jul 21 12:40:45.386208 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:40:45.386138 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-skc26_6713665c-3b71-4751-929e-16de48263766/konnectivity-agent/0.log" limit=1024 Jul 21 12:40:45.520616 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:40:45.520583 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-6hckk" event={"ID":"61de9ce0-c295-45c5-b657-2f2c64048340","Type":"ContainerStarted","Data":"57c1e83158b66e8253b4b1f8099260eca12acbe6b1d51e6db31814e0f3a8a66d"} Jul 21 12:40:45.588451 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:40:45.588423 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-skc26_6713665c-3b71-4751-929e-16de48263766/konnectivity-agent/0.log" limit=1024 Jul 21 12:40:45.780502 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:40:45.780478 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-skc26_6713665c-3b71-4751-929e-16de48263766/konnectivity-agent/0.log" limit=1024 Jul 21 12:40:45.961651 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:40:45.961624 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-skc26_6713665c-3b71-4751-929e-16de48263766/konnectivity-agent/0.log" limit=1024 Jul 21 12:40:46.018100 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:40:46.018073 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-skc26_6713665c-3b71-4751-929e-16de48263766/konnectivity-agent/0.log" limit=1024 Jul 21 12:40:46.254950 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:40:46.254920 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-skc26_6713665c-3b71-4751-929e-16de48263766/konnectivity-agent/0.log" limit=1024 Jul 21 12:40:46.625970 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:40:46.625885 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/eb329e76-c1b9-4fe2-9cb4-0d85b8a10e9e-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-ncggx\" (UID: \"eb329e76-c1b9-4fe2-9cb4-0d85b8a10e9e\") " pod="openshift-insights/insights-runtime-extractor-ncggx" Jul 21 12:40:46.625970 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:40:46.625936 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/eb329e76-c1b9-4fe2-9cb4-0d85b8a10e9e-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-ncggx\" (UID: \"eb329e76-c1b9-4fe2-9cb4-0d85b8a10e9e\") " pod="openshift-insights/insights-runtime-extractor-ncggx" Jul 21 12:40:46.626175 ip-10-0-130-157 kubenswrapper[2576]: E0721 12:40:46.626080 2576 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 21 12:40:46.626175 ip-10-0-130-157 kubenswrapper[2576]: E0721 12:40:46.626136 2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/eb329e76-c1b9-4fe2-9cb4-0d85b8a10e9e-insights-runtime-extractor-tls podName:eb329e76-c1b9-4fe2-9cb4-0d85b8a10e9e nodeName:}" failed. No retries permitted until 2026-07-21 12:41:02.626120383 +0000 UTC m=+70.461384227 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/eb329e76-c1b9-4fe2-9cb4-0d85b8a10e9e-insights-runtime-extractor-tls") pod "insights-runtime-extractor-ncggx" (UID: "eb329e76-c1b9-4fe2-9cb4-0d85b8a10e9e") : secret "insights-runtime-extractor-tls" not found Jul 21 12:40:46.626448 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:40:46.626427 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/eb329e76-c1b9-4fe2-9cb4-0d85b8a10e9e-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-ncggx\" (UID: \"eb329e76-c1b9-4fe2-9cb4-0d85b8a10e9e\") " pod="openshift-insights/insights-runtime-extractor-ncggx" Jul 21 12:40:46.788109 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:40:46.788068 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-skc26_6713665c-3b71-4751-929e-16de48263766/konnectivity-agent/0.log" limit=1024 Jul 21 12:40:47.184312 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:40:47.184282 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-skc26_6713665c-3b71-4751-929e-16de48263766/konnectivity-agent/0.log" limit=1024 Jul 21 12:40:47.589483 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:40:47.589413 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-skc26_6713665c-3b71-4751-929e-16de48263766/konnectivity-agent/0.log" limit=1024 Jul 21 12:40:47.995351 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:40:47.995320 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-skc26_6713665c-3b71-4751-929e-16de48263766/konnectivity-agent/0.log" limit=1024 Jul 21 12:40:48.382304 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:40:48.382281 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-skc26_6713665c-3b71-4751-929e-16de48263766/konnectivity-agent/0.log" limit=1024 Jul 21 12:40:48.530027 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:40:48.529952 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-6hckk" event={"ID":"61de9ce0-c295-45c5-b657-2f2c64048340","Type":"ContainerStarted","Data":"a82a6db833bb2e19b40b998da6c97828e759a5556097836e5262b8f0c8b9f7d4"} Jul 21 12:40:48.530027 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:40:48.530011 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-6hckk" event={"ID":"61de9ce0-c295-45c5-b657-2f2c64048340","Type":"ContainerStarted","Data":"606b00847ba5fe4f71172c23a56d5c4aaa677f1ef528e2decd1643f65100c56e"} Jul 21 12:40:48.530179 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:40:48.530090 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-6hckk" Jul 21 12:40:48.547852 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:40:48.547810 2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/dns-default-6hckk" podStartSLOduration=16.60022777 podStartE2EDuration="19.547798699s" totalImagesPullingTime="2.947570929s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 12:40:29 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 12:40:48.546998908 +0000 UTC m=+56.382262788" watchObservedRunningTime="2026-07-21 12:40:48.547798699 +0000 UTC m=+56.383062565" Jul 21 12:40:48.581159 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:40:48.581123 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-skc26_6713665c-3b71-4751-929e-16de48263766/konnectivity-agent/0.log" limit=1024 Jul 21 12:40:48.801417 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:40:48.801320 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-skc26_6713665c-3b71-4751-929e-16de48263766/konnectivity-agent/0.log" limit=1024 Jul 21 12:40:54.465525 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:40:54.465483 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-6nf7d" Jul 21 12:40:58.535507 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:40:58.535473 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-6hckk" Jul 21 12:40:59.393848 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:40:59.393815 2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/cluster-proxy-proxy-agent-fc5f6dffd-xhplp"] Jul 21 12:40:59.396651 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:40:59.396627 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-fc5f6dffd-xhplp" Jul 21 12:40:59.398831 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:40:59.398806 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-ca\"" Jul 21 12:40:59.399910 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:40:59.399889 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"open-cluster-management-image-pull-credentials\"" Jul 21 12:40:59.400035 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:40:59.399907 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-hub-kubeconfig\"" Jul 21 12:40:59.400035 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:40:59.399912 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-open-cluster-management.io-proxy-agent-signer-client-cert\"" Jul 21 12:40:59.400035 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:40:59.399959 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"kube-root-ca.crt\"" Jul 21 12:40:59.400389 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:40:59.400374 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-service-proxy-server-certificates\"" Jul 21 12:40:59.400483 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:40:59.400417 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"openshift-service-ca.crt\"" Jul 21 12:40:59.411451 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:40:59.411433 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca\" (UniqueName: \"kubernetes.io/secret/d0e5ef6b-17e9-4cad-ae07-2860e3556d8e-ca\") pod \"cluster-proxy-proxy-agent-fc5f6dffd-xhplp\" (UID: \"d0e5ef6b-17e9-4cad-ae07-2860e3556d8e\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-fc5f6dffd-xhplp" Jul 21 12:40:59.411553 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:40:59.411461 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/d0e5ef6b-17e9-4cad-ae07-2860e3556d8e-hub-kubeconfig\") pod \"cluster-proxy-proxy-agent-fc5f6dffd-xhplp\" (UID: \"d0e5ef6b-17e9-4cad-ae07-2860e3556d8e\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-fc5f6dffd-xhplp" Jul 21 12:40:59.411553 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:40:59.411491 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"additional-service-ca\" (UniqueName: \"kubernetes.io/configmap/d0e5ef6b-17e9-4cad-ae07-2860e3556d8e-additional-service-ca\") pod \"cluster-proxy-proxy-agent-fc5f6dffd-xhplp\" (UID: \"d0e5ef6b-17e9-4cad-ae07-2860e3556d8e\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-fc5f6dffd-xhplp" Jul 21 12:40:59.411553 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:40:59.411507 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-proxy-server-cert\" (UniqueName: \"kubernetes.io/secret/d0e5ef6b-17e9-4cad-ae07-2860e3556d8e-service-proxy-server-cert\") pod \"cluster-proxy-proxy-agent-fc5f6dffd-xhplp\" (UID: \"d0e5ef6b-17e9-4cad-ae07-2860e3556d8e\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-fc5f6dffd-xhplp" Jul 21 12:40:59.411715 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:40:59.411566 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lptcg\" (UniqueName: \"kubernetes.io/projected/d0e5ef6b-17e9-4cad-ae07-2860e3556d8e-kube-api-access-lptcg\") pod \"cluster-proxy-proxy-agent-fc5f6dffd-xhplp\" (UID: \"d0e5ef6b-17e9-4cad-ae07-2860e3556d8e\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-fc5f6dffd-xhplp" Jul 21 12:40:59.411715 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:40:59.411624 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d932e6af-610e-4967-87b1-e489cead334d-metrics-certs\") pod \"network-metrics-daemon-l9rfj\" (UID: \"d932e6af-610e-4967-87b1-e489cead334d\") " pod="openshift-multus/network-metrics-daemon-l9rfj" Jul 21 12:40:59.411715 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:40:59.411687 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/4dc9bfad-f2bb-4934-9f49-4cfd0f381ab2-original-pull-secret\") pod \"global-pull-secret-syncer-g2tkf\" (UID: \"4dc9bfad-f2bb-4934-9f49-4cfd0f381ab2\") " pod="kube-system/global-pull-secret-syncer-g2tkf" Jul 21 12:40:59.411928 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:40:59.411727 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hub\" (UniqueName: \"kubernetes.io/secret/d0e5ef6b-17e9-4cad-ae07-2860e3556d8e-hub\") pod \"cluster-proxy-proxy-agent-fc5f6dffd-xhplp\" (UID: \"d0e5ef6b-17e9-4cad-ae07-2860e3556d8e\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-fc5f6dffd-xhplp" Jul 21 12:40:59.412667 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:40:59.412647 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/cluster-proxy-proxy-agent-fc5f6dffd-xhplp"] Jul 21 12:40:59.413819 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:40:59.413804 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Jul 21 12:40:59.413984 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:40:59.413968 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Jul 21 12:40:59.424841 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:40:59.424815 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/4dc9bfad-f2bb-4934-9f49-4cfd0f381ab2-original-pull-secret\") pod \"global-pull-secret-syncer-g2tkf\" (UID: \"4dc9bfad-f2bb-4934-9f49-4cfd0f381ab2\") " pod="kube-system/global-pull-secret-syncer-g2tkf" Jul 21 12:40:59.424917 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:40:59.424858 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d932e6af-610e-4967-87b1-e489cead334d-metrics-certs\") pod \"network-metrics-daemon-l9rfj\" (UID: \"d932e6af-610e-4967-87b1-e489cead334d\") " pod="openshift-multus/network-metrics-daemon-l9rfj" Jul 21 12:40:59.512331 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:40:59.512305 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"additional-service-ca\" (UniqueName: \"kubernetes.io/configmap/d0e5ef6b-17e9-4cad-ae07-2860e3556d8e-additional-service-ca\") pod \"cluster-proxy-proxy-agent-fc5f6dffd-xhplp\" (UID: \"d0e5ef6b-17e9-4cad-ae07-2860e3556d8e\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-fc5f6dffd-xhplp" Jul 21 12:40:59.512433 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:40:59.512334 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-proxy-server-cert\" (UniqueName: \"kubernetes.io/secret/d0e5ef6b-17e9-4cad-ae07-2860e3556d8e-service-proxy-server-cert\") pod \"cluster-proxy-proxy-agent-fc5f6dffd-xhplp\" (UID: \"d0e5ef6b-17e9-4cad-ae07-2860e3556d8e\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-fc5f6dffd-xhplp" Jul 21 12:40:59.512433 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:40:59.512362 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lptcg\" (UniqueName: \"kubernetes.io/projected/d0e5ef6b-17e9-4cad-ae07-2860e3556d8e-kube-api-access-lptcg\") pod \"cluster-proxy-proxy-agent-fc5f6dffd-xhplp\" (UID: \"d0e5ef6b-17e9-4cad-ae07-2860e3556d8e\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-fc5f6dffd-xhplp" Jul 21 12:40:59.512433 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:40:59.512381 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kvc8f\" (UniqueName: \"kubernetes.io/projected/ea2d55a6-d43a-4d57-b49c-b63904d8810e-kube-api-access-kvc8f\") pod \"network-check-target-6f57m\" (UID: \"ea2d55a6-d43a-4d57-b49c-b63904d8810e\") " pod="openshift-network-diagnostics/network-check-target-6f57m" Jul 21 12:40:59.512433 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:40:59.512407 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hub\" (UniqueName: \"kubernetes.io/secret/d0e5ef6b-17e9-4cad-ae07-2860e3556d8e-hub\") pod \"cluster-proxy-proxy-agent-fc5f6dffd-xhplp\" (UID: \"d0e5ef6b-17e9-4cad-ae07-2860e3556d8e\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-fc5f6dffd-xhplp" Jul 21 12:40:59.512433 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:40:59.512426 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca\" (UniqueName: \"kubernetes.io/secret/d0e5ef6b-17e9-4cad-ae07-2860e3556d8e-ca\") pod \"cluster-proxy-proxy-agent-fc5f6dffd-xhplp\" (UID: \"d0e5ef6b-17e9-4cad-ae07-2860e3556d8e\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-fc5f6dffd-xhplp" Jul 21 12:40:59.512727 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:40:59.512449 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/d0e5ef6b-17e9-4cad-ae07-2860e3556d8e-hub-kubeconfig\") pod \"cluster-proxy-proxy-agent-fc5f6dffd-xhplp\" (UID: \"d0e5ef6b-17e9-4cad-ae07-2860e3556d8e\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-fc5f6dffd-xhplp" Jul 21 12:40:59.513074 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:40:59.513046 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"additional-service-ca\" (UniqueName: \"kubernetes.io/configmap/d0e5ef6b-17e9-4cad-ae07-2860e3556d8e-additional-service-ca\") pod \"cluster-proxy-proxy-agent-fc5f6dffd-xhplp\" (UID: \"d0e5ef6b-17e9-4cad-ae07-2860e3556d8e\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-fc5f6dffd-xhplp" Jul 21 12:40:59.514737 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:40:59.514710 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-proxy-server-cert\" (UniqueName: \"kubernetes.io/secret/d0e5ef6b-17e9-4cad-ae07-2860e3556d8e-service-proxy-server-cert\") pod \"cluster-proxy-proxy-agent-fc5f6dffd-xhplp\" (UID: \"d0e5ef6b-17e9-4cad-ae07-2860e3556d8e\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-fc5f6dffd-xhplp" Jul 21 12:40:59.514889 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:40:59.514860 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hub\" (UniqueName: \"kubernetes.io/secret/d0e5ef6b-17e9-4cad-ae07-2860e3556d8e-hub\") pod \"cluster-proxy-proxy-agent-fc5f6dffd-xhplp\" (UID: \"d0e5ef6b-17e9-4cad-ae07-2860e3556d8e\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-fc5f6dffd-xhplp" Jul 21 12:40:59.514986 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:40:59.514972 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" Jul 21 12:40:59.515211 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:40:59.515189 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ca\" (UniqueName: \"kubernetes.io/secret/d0e5ef6b-17e9-4cad-ae07-2860e3556d8e-ca\") pod \"cluster-proxy-proxy-agent-fc5f6dffd-xhplp\" (UID: \"d0e5ef6b-17e9-4cad-ae07-2860e3556d8e\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-fc5f6dffd-xhplp" Jul 21 12:40:59.515495 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:40:59.515462 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/d0e5ef6b-17e9-4cad-ae07-2860e3556d8e-hub-kubeconfig\") pod \"cluster-proxy-proxy-agent-fc5f6dffd-xhplp\" (UID: \"d0e5ef6b-17e9-4cad-ae07-2860e3556d8e\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-fc5f6dffd-xhplp" Jul 21 12:40:59.524887 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:40:59.524870 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" Jul 21 12:40:59.531467 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:40:59.531445 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lptcg\" (UniqueName: \"kubernetes.io/projected/d0e5ef6b-17e9-4cad-ae07-2860e3556d8e-kube-api-access-lptcg\") pod \"cluster-proxy-proxy-agent-fc5f6dffd-xhplp\" (UID: \"d0e5ef6b-17e9-4cad-ae07-2860e3556d8e\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-fc5f6dffd-xhplp" Jul 21 12:40:59.536282 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:40:59.536263 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kvc8f\" (UniqueName: \"kubernetes.io/projected/ea2d55a6-d43a-4d57-b49c-b63904d8810e-kube-api-access-kvc8f\") pod \"network-check-target-6f57m\" (UID: \"ea2d55a6-d43a-4d57-b49c-b63904d8810e\") " pod="openshift-network-diagnostics/network-check-target-6f57m" Jul 21 12:40:59.538737 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:40:59.538717 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-skc26_6713665c-3b71-4751-929e-16de48263766/konnectivity-agent/0.log" limit=1024 Jul 21 12:40:59.624644 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:40:59.624620 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-rhfvk\"" Jul 21 12:40:59.628087 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:40:59.628069 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-tqbcg\"" Jul 21 12:40:59.631822 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:40:59.631808 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-6f57m" Jul 21 12:40:59.636596 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:40:59.636573 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-l9rfj" Jul 21 12:40:59.641752 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:40:59.641733 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-skc26_6713665c-3b71-4751-929e-16de48263766/konnectivity-agent/0.log" limit=1024 Jul 21 12:40:59.721769 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:40:59.721668 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-g2tkf" Jul 21 12:40:59.730847 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:40:59.730824 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-fc5f6dffd-xhplp" Jul 21 12:40:59.742199 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:40:59.742089 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-skc26_6713665c-3b71-4751-929e-16de48263766/konnectivity-agent/0.log" limit=1024 Jul 21 12:40:59.764873 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:40:59.764803 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-6f57m"] Jul 21 12:40:59.767857 ip-10-0-130-157 kubenswrapper[2576]: W0721 12:40:59.767834 2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podea2d55a6_d43a_4d57_b49c_b63904d8810e.slice/crio-c56f4a63f900c0fc2701e39583b9d24b509b5b64708640f0a3cca5ef6c4a0caa WatchSource:0}: Error finding container c56f4a63f900c0fc2701e39583b9d24b509b5b64708640f0a3cca5ef6c4a0caa: Status 404 returned error can't find the container with id c56f4a63f900c0fc2701e39583b9d24b509b5b64708640f0a3cca5ef6c4a0caa Jul 21 12:40:59.784897 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:40:59.784867 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-l9rfj"] Jul 21 12:40:59.788156 ip-10-0-130-157 kubenswrapper[2576]: W0721 12:40:59.788123 2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd932e6af_610e_4967_87b1_e489cead334d.slice/crio-33ececda6115362085cd9f28925c4c7f49be92d69a29af45588f6d84d193cfda WatchSource:0}: Error finding container 33ececda6115362085cd9f28925c4c7f49be92d69a29af45588f6d84d193cfda: Status 404 returned error can't find the container with id 33ececda6115362085cd9f28925c4c7f49be92d69a29af45588f6d84d193cfda Jul 21 12:40:59.839532 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:40:59.839484 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-skc26_6713665c-3b71-4751-929e-16de48263766/konnectivity-agent/0.log" limit=1024 Jul 21 12:40:59.856185 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:40:59.856162 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-g2tkf"] Jul 21 12:40:59.858863 ip-10-0-130-157 kubenswrapper[2576]: W0721 12:40:59.858827 2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4dc9bfad_f2bb_4934_9f49_4cfd0f381ab2.slice/crio-a4ebf0e63f328bd0afceb3ea8e903f8e89f3bf165a132e0cbaca3279ea98c1fa WatchSource:0}: Error finding container a4ebf0e63f328bd0afceb3ea8e903f8e89f3bf165a132e0cbaca3279ea98c1fa: Status 404 returned error can't find the container with id a4ebf0e63f328bd0afceb3ea8e903f8e89f3bf165a132e0cbaca3279ea98c1fa Jul 21 12:40:59.880387 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:40:59.880362 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/cluster-proxy-proxy-agent-fc5f6dffd-xhplp"] Jul 21 12:40:59.882910 ip-10-0-130-157 kubenswrapper[2576]: W0721 12:40:59.882889 2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podd0e5ef6b_17e9_4cad_ae07_2860e3556d8e.slice/crio-d92741136316de86877abeddc80d8d9d4809f1717e9fbe8e22cbc2ba70c74318 WatchSource:0}: Error finding container d92741136316de86877abeddc80d8d9d4809f1717e9fbe8e22cbc2ba70c74318: Status 404 returned error can't find the container with id d92741136316de86877abeddc80d8d9d4809f1717e9fbe8e22cbc2ba70c74318 Jul 21 12:41:00.571990 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:41:00.571947 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-l9rfj" event={"ID":"d932e6af-610e-4967-87b1-e489cead334d","Type":"ContainerStarted","Data":"33ececda6115362085cd9f28925c4c7f49be92d69a29af45588f6d84d193cfda"} Jul 21 12:41:00.573374 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:41:00.573332 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-fc5f6dffd-xhplp" event={"ID":"d0e5ef6b-17e9-4cad-ae07-2860e3556d8e","Type":"ContainerStarted","Data":"d92741136316de86877abeddc80d8d9d4809f1717e9fbe8e22cbc2ba70c74318"} Jul 21 12:41:00.574945 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:41:00.574902 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-g2tkf" event={"ID":"4dc9bfad-f2bb-4934-9f49-4cfd0f381ab2","Type":"ContainerStarted","Data":"a4ebf0e63f328bd0afceb3ea8e903f8e89f3bf165a132e0cbaca3279ea98c1fa"} Jul 21 12:41:00.576247 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:41:00.576221 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-6f57m" event={"ID":"ea2d55a6-d43a-4d57-b49c-b63904d8810e","Type":"ContainerStarted","Data":"c56f4a63f900c0fc2701e39583b9d24b509b5b64708640f0a3cca5ef6c4a0caa"} Jul 21 12:41:01.582594 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:41:01.582414 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-l9rfj" event={"ID":"d932e6af-610e-4967-87b1-e489cead334d","Type":"ContainerStarted","Data":"6b4122545d83d2be24729d3d5a9a9ce7757ee1054f05d0e1acfe6883fbd4b574"} Jul 21 12:41:01.582594 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:41:01.582477 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-l9rfj" event={"ID":"d932e6af-610e-4967-87b1-e489cead334d","Type":"ContainerStarted","Data":"15aee9fcdcc8cd4249143ba2f8056a302852635fa34c0cf96daf22a51d781a9d"} Jul 21 12:41:01.605970 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:41:01.605912 2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-l9rfj" podStartSLOduration=66.448693848 podStartE2EDuration="1m7.605892037s" totalImagesPullingTime="1.157198189s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 12:39:54 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 12:41:01.6040246 +0000 UTC m=+69.439288470" watchObservedRunningTime="2026-07-21 12:41:01.605892037 +0000 UTC m=+69.441155903" Jul 21 12:41:01.762048 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:41:01.761934 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-skc26_6713665c-3b71-4751-929e-16de48263766/konnectivity-agent/0.log" limit=1024 Jul 21 12:41:01.852921 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:41:01.852783 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-skc26_6713665c-3b71-4751-929e-16de48263766/konnectivity-agent/0.log" limit=1024 Jul 21 12:41:01.961442 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:41:01.961406 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-skc26_6713665c-3b71-4751-929e-16de48263766/konnectivity-agent/0.log" limit=1024 Jul 21 12:41:02.636837 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:41:02.636496 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/eb329e76-c1b9-4fe2-9cb4-0d85b8a10e9e-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-ncggx\" (UID: \"eb329e76-c1b9-4fe2-9cb4-0d85b8a10e9e\") " pod="openshift-insights/insights-runtime-extractor-ncggx" Jul 21 12:41:02.640330 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:41:02.639409 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/eb329e76-c1b9-4fe2-9cb4-0d85b8a10e9e-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-ncggx\" (UID: \"eb329e76-c1b9-4fe2-9cb4-0d85b8a10e9e\") " pod="openshift-insights/insights-runtime-extractor-ncggx" Jul 21 12:41:02.852774 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:41:02.852561 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-6mtb2\"" Jul 21 12:41:02.860603 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:41:02.860551 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-ncggx" Jul 21 12:41:03.440997 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:41:03.440917 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-skc26_6713665c-3b71-4751-929e-16de48263766/konnectivity-agent/0.log" limit=1024 Jul 21 12:41:03.522366 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:41:03.522249 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-skc26_6713665c-3b71-4751-929e-16de48263766/konnectivity-agent/0.log" limit=1024 Jul 21 12:41:04.683310 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:41:04.683267 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-skc26_6713665c-3b71-4751-929e-16de48263766/konnectivity-agent/0.log" limit=1024 Jul 21 12:41:04.766338 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:41:04.766309 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-skc26_6713665c-3b71-4751-929e-16de48263766/konnectivity-agent/0.log" limit=1024 Jul 21 12:41:06.751105 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:41:06.750956 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-ncggx"] Jul 21 12:41:06.755109 ip-10-0-130-157 kubenswrapper[2576]: W0721 12:41:06.755084 2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podeb329e76_c1b9_4fe2_9cb4_0d85b8a10e9e.slice/crio-bcaf7c88a8971ea7864cb29458420258cda94194c9a412ee1a231aeaf1b4285e WatchSource:0}: Error finding container bcaf7c88a8971ea7864cb29458420258cda94194c9a412ee1a231aeaf1b4285e: Status 404 returned error can't find the container with id bcaf7c88a8971ea7864cb29458420258cda94194c9a412ee1a231aeaf1b4285e Jul 21 12:41:07.438150 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:41:07.438112 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-skc26_6713665c-3b71-4751-929e-16de48263766/konnectivity-agent/0.log" limit=1024 Jul 21 12:41:07.506651 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:41:07.506617 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-skc26_6713665c-3b71-4751-929e-16de48263766/konnectivity-agent/0.log" limit=1024 Jul 21 12:41:07.605854 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:41:07.605812 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-fc5f6dffd-xhplp" event={"ID":"d0e5ef6b-17e9-4cad-ae07-2860e3556d8e","Type":"ContainerStarted","Data":"cff4577b2b1ebfccd5f243825c028f563d6c1cd8c255089cecbd5a3d78dbd7e0"} Jul 21 12:41:07.606028 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:41:07.605865 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-fc5f6dffd-xhplp" event={"ID":"d0e5ef6b-17e9-4cad-ae07-2860e3556d8e","Type":"ContainerStarted","Data":"6a386d1dedf56315e4b275ca6600d06a05d7772dcc1eeb563862006564723705"} Jul 21 12:41:07.606028 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:41:07.605886 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-fc5f6dffd-xhplp" event={"ID":"d0e5ef6b-17e9-4cad-ae07-2860e3556d8e","Type":"ContainerStarted","Data":"ab23cae3ba4e6f15232079ae1da12c4cb3153f0145bed9ee800510cf41752a69"} Jul 21 12:41:07.607861 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:41:07.607834 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-g2tkf" event={"ID":"4dc9bfad-f2bb-4934-9f49-4cfd0f381ab2","Type":"ContainerStarted","Data":"258535be83075f6373f84bbb413658fb0a55862ede10d37eb1128fa85afe81e5"} Jul 21 12:41:07.609564 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:41:07.609536 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-6f57m" event={"ID":"ea2d55a6-d43a-4d57-b49c-b63904d8810e","Type":"ContainerStarted","Data":"ca40a7a0f5a3ddecfc5b3a07ec93f4fd3cdc00ccd16a37668c85eab5c157c3a5"} Jul 21 12:41:07.609690 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:41:07.609607 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-6f57m" Jul 21 12:41:07.610803 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:41:07.610772 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-ncggx" event={"ID":"eb329e76-c1b9-4fe2-9cb4-0d85b8a10e9e","Type":"ContainerStarted","Data":"bcaf7c88a8971ea7864cb29458420258cda94194c9a412ee1a231aeaf1b4285e"} Jul 21 12:41:07.628416 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:41:07.628364 2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-fc5f6dffd-xhplp" podStartSLOduration=1.8865336259999999 podStartE2EDuration="8.628350631s" totalImagesPullingTime="6.741817005s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 12:40:59 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 12:41:07.627688577 +0000 UTC m=+75.462952442" watchObservedRunningTime="2026-07-21 12:41:07.628350631 +0000 UTC m=+75.463614491" Jul 21 12:41:07.643953 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:41:07.643915 2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-g2tkf" podStartSLOduration=66.88560068 podStartE2EDuration="1m13.643898656s" totalImagesPullingTime="6.758297976s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-21 12:39:54 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 12:41:07.643146506 +0000 UTC m=+75.478410373" watchObservedRunningTime="2026-07-21 12:41:07.643898656 +0000 UTC m=+75.479162526" Jul 21 12:41:07.660648 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:41:07.660616 2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-6f57m" podStartSLOduration=66.817261417 podStartE2EDuration="1m13.660603243s" totalImagesPullingTime="6.843341826s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 12:39:54 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 12:41:07.659186679 +0000 UTC m=+75.494450547" watchObservedRunningTime="2026-07-21 12:41:07.660603243 +0000 UTC m=+75.495867109" Jul 21 12:41:07.867554 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:41:07.867454 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-skc26_6713665c-3b71-4751-929e-16de48263766/konnectivity-agent/0.log" limit=1024 Jul 21 12:41:07.947946 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:41:07.947910 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-skc26_6713665c-3b71-4751-929e-16de48263766/konnectivity-agent/0.log" limit=1024 Jul 21 12:41:08.012685 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:41:08.012653 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-skc26_6713665c-3b71-4751-929e-16de48263766/konnectivity-agent/0.log" limit=1024 Jul 21 12:41:08.543370 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:41:08.543336 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-skc26_6713665c-3b71-4751-929e-16de48263766/konnectivity-agent/0.log" limit=1024 Jul 21 12:41:08.660277 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:41:08.660238 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-skc26_6713665c-3b71-4751-929e-16de48263766/konnectivity-agent/0.log" limit=1024 Jul 21 12:41:12.631199 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:41:12.631148 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-ncggx" event={"ID":"eb329e76-c1b9-4fe2-9cb4-0d85b8a10e9e","Type":"ContainerStarted","Data":"3e5b5b77c8e7a999ab2bad07fdc4615fd89e70b5c01342b2eb348a52ac6aaf84"} Jul 21 12:41:12.643717 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:41:12.643685 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-skc26_6713665c-3b71-4751-929e-16de48263766/konnectivity-agent/0.log" limit=1024 Jul 21 12:41:13.767603 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:41:13.767566 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-skc26_6713665c-3b71-4751-929e-16de48263766/konnectivity-agent/0.log" limit=1024 Jul 21 12:41:14.652873 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:41:14.652821 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-skc26_6713665c-3b71-4751-929e-16de48263766/konnectivity-agent/0.log" limit=1024 Jul 21 12:41:15.029252 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:41:15.029183 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-skc26_6713665c-3b71-4751-929e-16de48263766/konnectivity-agent/0.log" limit=1024 Jul 21 12:41:15.218666 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:41:15.218643 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-skc26_6713665c-3b71-4751-929e-16de48263766/konnectivity-agent/0.log" limit=1024 Jul 21 12:41:15.298603 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:41:15.298539 2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-c69sx"] Jul 21 12:41:15.300799 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:41:15.300779 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-c69sx" Jul 21 12:41:15.302897 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:41:15.302876 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\"" Jul 21 12:41:15.303003 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:41:15.302881 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\"" Jul 21 12:41:15.303003 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:41:15.302881 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\"" Jul 21 12:41:15.303228 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:41:15.303212 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-nj2hc\"" Jul 21 12:41:15.303392 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:41:15.303378 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\"" Jul 21 12:41:15.303490 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:41:15.303408 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\"" Jul 21 12:41:15.303490 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:41:15.303410 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\"" Jul 21 12:41:15.325977 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:41:15.325956 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-skc26_6713665c-3b71-4751-929e-16de48263766/konnectivity-agent/0.log" limit=1024 Jul 21 12:41:15.326152 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:41:15.326133 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/a6e83d7e-1979-4094-8beb-c3f19ab6ccdf-node-exporter-accelerators-collector-config\") pod \"node-exporter-c69sx\" (UID: \"a6e83d7e-1979-4094-8beb-c3f19ab6ccdf\") " pod="openshift-monitoring/node-exporter-c69sx" Jul 21 12:41:15.326199 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:41:15.326168 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/a6e83d7e-1979-4094-8beb-c3f19ab6ccdf-sys\") pod \"node-exporter-c69sx\" (UID: \"a6e83d7e-1979-4094-8beb-c3f19ab6ccdf\") " pod="openshift-monitoring/node-exporter-c69sx" Jul 21 12:41:15.326242 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:41:15.326221 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/a6e83d7e-1979-4094-8beb-c3f19ab6ccdf-root\") pod \"node-exporter-c69sx\" (UID: \"a6e83d7e-1979-4094-8beb-c3f19ab6ccdf\") " pod="openshift-monitoring/node-exporter-c69sx" Jul 21 12:41:15.326278 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:41:15.326264 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/a6e83d7e-1979-4094-8beb-c3f19ab6ccdf-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-c69sx\" (UID: \"a6e83d7e-1979-4094-8beb-c3f19ab6ccdf\") " pod="openshift-monitoring/node-exporter-c69sx" Jul 21 12:41:15.326333 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:41:15.326317 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/a6e83d7e-1979-4094-8beb-c3f19ab6ccdf-node-exporter-wtmp\") pod \"node-exporter-c69sx\" (UID: \"a6e83d7e-1979-4094-8beb-c3f19ab6ccdf\") " pod="openshift-monitoring/node-exporter-c69sx" Jul 21 12:41:15.326369 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:41:15.326345 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/a6e83d7e-1979-4094-8beb-c3f19ab6ccdf-metrics-client-ca\") pod \"node-exporter-c69sx\" (UID: \"a6e83d7e-1979-4094-8beb-c3f19ab6ccdf\") " pod="openshift-monitoring/node-exporter-c69sx" Jul 21 12:41:15.326369 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:41:15.326362 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/a6e83d7e-1979-4094-8beb-c3f19ab6ccdf-node-exporter-tls\") pod \"node-exporter-c69sx\" (UID: \"a6e83d7e-1979-4094-8beb-c3f19ab6ccdf\") " pod="openshift-monitoring/node-exporter-c69sx" Jul 21 12:41:15.326442 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:41:15.326382 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/a6e83d7e-1979-4094-8beb-c3f19ab6ccdf-node-exporter-textfile\") pod \"node-exporter-c69sx\" (UID: \"a6e83d7e-1979-4094-8beb-c3f19ab6ccdf\") " pod="openshift-monitoring/node-exporter-c69sx" Jul 21 12:41:15.326442 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:41:15.326415 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fsz98\" (UniqueName: \"kubernetes.io/projected/a6e83d7e-1979-4094-8beb-c3f19ab6ccdf-kube-api-access-fsz98\") pod \"node-exporter-c69sx\" (UID: \"a6e83d7e-1979-4094-8beb-c3f19ab6ccdf\") " pod="openshift-monitoring/node-exporter-c69sx" Jul 21 12:41:15.410834 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:41:15.410793 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-skc26_6713665c-3b71-4751-929e-16de48263766/konnectivity-agent/0.log" limit=1024 Jul 21 12:41:15.426817 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:41:15.426795 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/a6e83d7e-1979-4094-8beb-c3f19ab6ccdf-node-exporter-textfile\") pod \"node-exporter-c69sx\" (UID: \"a6e83d7e-1979-4094-8beb-c3f19ab6ccdf\") " pod="openshift-monitoring/node-exporter-c69sx" Jul 21 12:41:15.426954 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:41:15.426827 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fsz98\" (UniqueName: \"kubernetes.io/projected/a6e83d7e-1979-4094-8beb-c3f19ab6ccdf-kube-api-access-fsz98\") pod \"node-exporter-c69sx\" (UID: \"a6e83d7e-1979-4094-8beb-c3f19ab6ccdf\") " pod="openshift-monitoring/node-exporter-c69sx" Jul 21 12:41:15.426954 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:41:15.426850 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/a6e83d7e-1979-4094-8beb-c3f19ab6ccdf-node-exporter-accelerators-collector-config\") pod \"node-exporter-c69sx\" (UID: \"a6e83d7e-1979-4094-8beb-c3f19ab6ccdf\") " pod="openshift-monitoring/node-exporter-c69sx" Jul 21 12:41:15.426954 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:41:15.426871 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/a6e83d7e-1979-4094-8beb-c3f19ab6ccdf-sys\") pod \"node-exporter-c69sx\" (UID: \"a6e83d7e-1979-4094-8beb-c3f19ab6ccdf\") " pod="openshift-monitoring/node-exporter-c69sx" Jul 21 12:41:15.426954 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:41:15.426893 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/a6e83d7e-1979-4094-8beb-c3f19ab6ccdf-root\") pod \"node-exporter-c69sx\" (UID: \"a6e83d7e-1979-4094-8beb-c3f19ab6ccdf\") " pod="openshift-monitoring/node-exporter-c69sx" Jul 21 12:41:15.426954 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:41:15.426932 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/a6e83d7e-1979-4094-8beb-c3f19ab6ccdf-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-c69sx\" (UID: \"a6e83d7e-1979-4094-8beb-c3f19ab6ccdf\") " pod="openshift-monitoring/node-exporter-c69sx" Jul 21 12:41:15.427194 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:41:15.426965 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/a6e83d7e-1979-4094-8beb-c3f19ab6ccdf-sys\") pod \"node-exporter-c69sx\" (UID: \"a6e83d7e-1979-4094-8beb-c3f19ab6ccdf\") " pod="openshift-monitoring/node-exporter-c69sx" Jul 21 12:41:15.427194 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:41:15.426973 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/a6e83d7e-1979-4094-8beb-c3f19ab6ccdf-node-exporter-wtmp\") pod \"node-exporter-c69sx\" (UID: \"a6e83d7e-1979-4094-8beb-c3f19ab6ccdf\") " pod="openshift-monitoring/node-exporter-c69sx" Jul 21 12:41:15.427194 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:41:15.427011 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/a6e83d7e-1979-4094-8beb-c3f19ab6ccdf-metrics-client-ca\") pod \"node-exporter-c69sx\" (UID: \"a6e83d7e-1979-4094-8beb-c3f19ab6ccdf\") " pod="openshift-monitoring/node-exporter-c69sx" Jul 21 12:41:15.427194 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:41:15.427035 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/a6e83d7e-1979-4094-8beb-c3f19ab6ccdf-node-exporter-tls\") pod \"node-exporter-c69sx\" (UID: \"a6e83d7e-1979-4094-8beb-c3f19ab6ccdf\") " pod="openshift-monitoring/node-exporter-c69sx" Jul 21 12:41:15.427194 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:41:15.427092 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/a6e83d7e-1979-4094-8beb-c3f19ab6ccdf-node-exporter-wtmp\") pod \"node-exporter-c69sx\" (UID: \"a6e83d7e-1979-4094-8beb-c3f19ab6ccdf\") " pod="openshift-monitoring/node-exporter-c69sx" Jul 21 12:41:15.427194 ip-10-0-130-157 kubenswrapper[2576]: E0721 12:41:15.427135 2576 secret.go:189] Couldn't get secret openshift-monitoring/node-exporter-tls: secret "node-exporter-tls" not found Jul 21 12:41:15.427194 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:41:15.427156 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/a6e83d7e-1979-4094-8beb-c3f19ab6ccdf-node-exporter-textfile\") pod \"node-exporter-c69sx\" (UID: \"a6e83d7e-1979-4094-8beb-c3f19ab6ccdf\") " pod="openshift-monitoring/node-exporter-c69sx" Jul 21 12:41:15.427194 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:41:15.427137 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/a6e83d7e-1979-4094-8beb-c3f19ab6ccdf-root\") pod \"node-exporter-c69sx\" (UID: \"a6e83d7e-1979-4094-8beb-c3f19ab6ccdf\") " pod="openshift-monitoring/node-exporter-c69sx" Jul 21 12:41:15.427194 ip-10-0-130-157 kubenswrapper[2576]: E0721 12:41:15.427194 2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a6e83d7e-1979-4094-8beb-c3f19ab6ccdf-node-exporter-tls podName:a6e83d7e-1979-4094-8beb-c3f19ab6ccdf nodeName:}" failed. No retries permitted until 2026-07-21 12:41:15.927175856 +0000 UTC m=+83.762439712 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "node-exporter-tls" (UniqueName: "kubernetes.io/secret/a6e83d7e-1979-4094-8beb-c3f19ab6ccdf-node-exporter-tls") pod "node-exporter-c69sx" (UID: "a6e83d7e-1979-4094-8beb-c3f19ab6ccdf") : secret "node-exporter-tls" not found Jul 21 12:41:15.427632 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:41:15.427442 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/a6e83d7e-1979-4094-8beb-c3f19ab6ccdf-node-exporter-accelerators-collector-config\") pod \"node-exporter-c69sx\" (UID: \"a6e83d7e-1979-4094-8beb-c3f19ab6ccdf\") " pod="openshift-monitoring/node-exporter-c69sx" Jul 21 12:41:15.427632 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:41:15.427588 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/a6e83d7e-1979-4094-8beb-c3f19ab6ccdf-metrics-client-ca\") pod \"node-exporter-c69sx\" (UID: \"a6e83d7e-1979-4094-8beb-c3f19ab6ccdf\") " pod="openshift-monitoring/node-exporter-c69sx" Jul 21 12:41:15.429355 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:41:15.429336 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/a6e83d7e-1979-4094-8beb-c3f19ab6ccdf-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-c69sx\" (UID: \"a6e83d7e-1979-4094-8beb-c3f19ab6ccdf\") " pod="openshift-monitoring/node-exporter-c69sx" Jul 21 12:41:15.438383 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:41:15.438356 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fsz98\" (UniqueName: \"kubernetes.io/projected/a6e83d7e-1979-4094-8beb-c3f19ab6ccdf-kube-api-access-fsz98\") pod \"node-exporter-c69sx\" (UID: \"a6e83d7e-1979-4094-8beb-c3f19ab6ccdf\") " pod="openshift-monitoring/node-exporter-c69sx" Jul 21 12:41:15.511031 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:41:15.511001 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-skc26_6713665c-3b71-4751-929e-16de48263766/konnectivity-agent/0.log" limit=1024 Jul 21 12:41:15.651953 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:41:15.651918 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-skc26_6713665c-3b71-4751-929e-16de48263766/konnectivity-agent/0.log" limit=1024 Jul 21 12:41:15.652784 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:41:15.652738 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-ncggx" event={"ID":"eb329e76-c1b9-4fe2-9cb4-0d85b8a10e9e","Type":"ContainerStarted","Data":"aaec20c72a7243f566ca4b9f1254692a3f933356528a3b0b45c68c32410ba7ce"} Jul 21 12:41:15.856531 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:41:15.856464 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-skc26_6713665c-3b71-4751-929e-16de48263766/konnectivity-agent/0.log" limit=1024 Jul 21 12:41:15.931041 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:41:15.930966 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/a6e83d7e-1979-4094-8beb-c3f19ab6ccdf-node-exporter-tls\") pod \"node-exporter-c69sx\" (UID: \"a6e83d7e-1979-4094-8beb-c3f19ab6ccdf\") " pod="openshift-monitoring/node-exporter-c69sx" Jul 21 12:41:15.933535 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:41:15.933472 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/a6e83d7e-1979-4094-8beb-c3f19ab6ccdf-node-exporter-tls\") pod \"node-exporter-c69sx\" (UID: \"a6e83d7e-1979-4094-8beb-c3f19ab6ccdf\") " pod="openshift-monitoring/node-exporter-c69sx" Jul 21 12:41:15.957508 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:41:15.957481 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-skc26_6713665c-3b71-4751-929e-16de48263766/konnectivity-agent/0.log" limit=1024 Jul 21 12:41:16.210456 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:41:16.210380 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-c69sx" Jul 21 12:41:16.222085 ip-10-0-130-157 kubenswrapper[2576]: W0721 12:41:16.222004 2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda6e83d7e_1979_4094_8beb_c3f19ab6ccdf.slice/crio-fced305d453fc9a46075328f35520039cad76cba17a06a51463983e632d0740e WatchSource:0}: Error finding container fced305d453fc9a46075328f35520039cad76cba17a06a51463983e632d0740e: Status 404 returned error can't find the container with id fced305d453fc9a46075328f35520039cad76cba17a06a51463983e632d0740e Jul 21 12:41:16.328663 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:41:16.328630 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-skc26_6713665c-3b71-4751-929e-16de48263766/konnectivity-agent/0.log" limit=1024 Jul 21 12:41:16.424167 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:41:16.424135 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-skc26_6713665c-3b71-4751-929e-16de48263766/konnectivity-agent/0.log" limit=1024 Jul 21 12:41:16.657861 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:41:16.657826 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-c69sx" event={"ID":"a6e83d7e-1979-4094-8beb-c3f19ab6ccdf","Type":"ContainerStarted","Data":"fced305d453fc9a46075328f35520039cad76cba17a06a51463983e632d0740e"} Jul 21 12:41:16.759100 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:41:16.759067 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-skc26_6713665c-3b71-4751-929e-16de48263766/konnectivity-agent/0.log" limit=1024 Jul 21 12:41:18.328646 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:41:18.328581 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-skc26_6713665c-3b71-4751-929e-16de48263766/konnectivity-agent/0.log" limit=1024 Jul 21 12:41:18.389931 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:41:18.389887 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-skc26_6713665c-3b71-4751-929e-16de48263766/konnectivity-agent/0.log" limit=1024 Jul 21 12:41:18.525434 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:41:18.525405 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-skc26_6713665c-3b71-4751-929e-16de48263766/konnectivity-agent/0.log" limit=1024 Jul 21 12:41:18.625252 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:41:18.625229 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-skc26_6713665c-3b71-4751-929e-16de48263766/konnectivity-agent/0.log" limit=1024 Jul 21 12:41:18.666521 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:41:18.666493 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-ncggx" event={"ID":"eb329e76-c1b9-4fe2-9cb4-0d85b8a10e9e","Type":"ContainerStarted","Data":"dd77916272943e36c3257fbbd7fe56a22db30c2380d6661f3d7e83823e9f38c9"} Jul 21 12:41:18.688720 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:41:18.688680 2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-ncggx" podStartSLOduration=36.996013206 podStartE2EDuration="48.688665514s" totalImagesPullingTime="11.692652308s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-21 12:40:30 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 12:41:18.687291253 +0000 UTC m=+86.522555119" watchObservedRunningTime="2026-07-21 12:41:18.688665514 +0000 UTC m=+86.523929379" Jul 21 12:41:19.671939 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:41:19.671906 2576 generic.go:356] "Generic (PLEG): container finished" podID="a6e83d7e-1979-4094-8beb-c3f19ab6ccdf" containerID="47c36e3f08179af4878b4c8864e598f6f30b71f51ceb0320f761864b23c89631" exitCode=0 Jul 21 12:41:19.672293 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:41:19.671996 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-c69sx" event={"ID":"a6e83d7e-1979-4094-8beb-c3f19ab6ccdf","Type":"ContainerDied","Data":"47c36e3f08179af4878b4c8864e598f6f30b71f51ceb0320f761864b23c89631"} Jul 21 12:41:19.693298 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:41:19.693275 2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/metrics-server-597974fb65-bdrwd"] Jul 21 12:41:19.697365 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:41:19.697350 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/metrics-server-597974fb65-bdrwd" Jul 21 12:41:19.699339 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:41:19.699323 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kubelet-serving-ca-bundle\"" Jul 21 12:41:19.699449 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:41:19.699355 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-dvi82aon68srg\"" Jul 21 12:41:19.699449 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:41:19.699417 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-client-certs\"" Jul 21 12:41:19.699449 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:41:19.699429 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-tls\"" Jul 21 12:41:19.699615 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:41:19.699597 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-dockercfg-298gz\"" Jul 21 12:41:19.699672 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:41:19.699663 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-server-audit-profiles\"" Jul 21 12:41:19.712115 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:41:19.712094 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/metrics-server-597974fb65-bdrwd"] Jul 21 12:41:19.762343 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:41:19.762296 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/ea945a18-e37f-4957-8c7d-bee0cb2e9421-secret-metrics-server-client-certs\") pod \"metrics-server-597974fb65-bdrwd\" (UID: \"ea945a18-e37f-4957-8c7d-bee0cb2e9421\") " pod="openshift-monitoring/metrics-server-597974fb65-bdrwd" Jul 21 12:41:19.762447 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:41:19.762357 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2sq7k\" (UniqueName: \"kubernetes.io/projected/ea945a18-e37f-4957-8c7d-bee0cb2e9421-kube-api-access-2sq7k\") pod \"metrics-server-597974fb65-bdrwd\" (UID: \"ea945a18-e37f-4957-8c7d-bee0cb2e9421\") " pod="openshift-monitoring/metrics-server-597974fb65-bdrwd" Jul 21 12:41:19.762447 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:41:19.762421 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/ea945a18-e37f-4957-8c7d-bee0cb2e9421-metrics-server-audit-profiles\") pod \"metrics-server-597974fb65-bdrwd\" (UID: \"ea945a18-e37f-4957-8c7d-bee0cb2e9421\") " pod="openshift-monitoring/metrics-server-597974fb65-bdrwd" Jul 21 12:41:19.762595 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:41:19.762556 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ea945a18-e37f-4957-8c7d-bee0cb2e9421-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-597974fb65-bdrwd\" (UID: \"ea945a18-e37f-4957-8c7d-bee0cb2e9421\") " pod="openshift-monitoring/metrics-server-597974fb65-bdrwd" Jul 21 12:41:19.762653 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:41:19.762607 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/ea945a18-e37f-4957-8c7d-bee0cb2e9421-audit-log\") pod \"metrics-server-597974fb65-bdrwd\" (UID: \"ea945a18-e37f-4957-8c7d-bee0cb2e9421\") " pod="openshift-monitoring/metrics-server-597974fb65-bdrwd" Jul 21 12:41:19.762708 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:41:19.762684 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/ea945a18-e37f-4957-8c7d-bee0cb2e9421-secret-metrics-server-tls\") pod \"metrics-server-597974fb65-bdrwd\" (UID: \"ea945a18-e37f-4957-8c7d-bee0cb2e9421\") " pod="openshift-monitoring/metrics-server-597974fb65-bdrwd" Jul 21 12:41:19.762814 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:41:19.762793 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/ea945a18-e37f-4957-8c7d-bee0cb2e9421-client-ca-bundle\") pod \"metrics-server-597974fb65-bdrwd\" (UID: \"ea945a18-e37f-4957-8c7d-bee0cb2e9421\") " pod="openshift-monitoring/metrics-server-597974fb65-bdrwd" Jul 21 12:41:19.863238 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:41:19.863212 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/ea945a18-e37f-4957-8c7d-bee0cb2e9421-metrics-server-audit-profiles\") pod \"metrics-server-597974fb65-bdrwd\" (UID: \"ea945a18-e37f-4957-8c7d-bee0cb2e9421\") " pod="openshift-monitoring/metrics-server-597974fb65-bdrwd" Jul 21 12:41:19.863357 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:41:19.863254 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ea945a18-e37f-4957-8c7d-bee0cb2e9421-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-597974fb65-bdrwd\" (UID: \"ea945a18-e37f-4957-8c7d-bee0cb2e9421\") " pod="openshift-monitoring/metrics-server-597974fb65-bdrwd" Jul 21 12:41:19.863357 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:41:19.863272 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/ea945a18-e37f-4957-8c7d-bee0cb2e9421-audit-log\") pod \"metrics-server-597974fb65-bdrwd\" (UID: \"ea945a18-e37f-4957-8c7d-bee0cb2e9421\") " pod="openshift-monitoring/metrics-server-597974fb65-bdrwd" Jul 21 12:41:19.863357 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:41:19.863291 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/ea945a18-e37f-4957-8c7d-bee0cb2e9421-secret-metrics-server-tls\") pod \"metrics-server-597974fb65-bdrwd\" (UID: \"ea945a18-e37f-4957-8c7d-bee0cb2e9421\") " pod="openshift-monitoring/metrics-server-597974fb65-bdrwd" Jul 21 12:41:19.863506 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:41:19.863407 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/ea945a18-e37f-4957-8c7d-bee0cb2e9421-client-ca-bundle\") pod \"metrics-server-597974fb65-bdrwd\" (UID: \"ea945a18-e37f-4957-8c7d-bee0cb2e9421\") " pod="openshift-monitoring/metrics-server-597974fb65-bdrwd" Jul 21 12:41:19.863506 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:41:19.863466 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/ea945a18-e37f-4957-8c7d-bee0cb2e9421-secret-metrics-server-client-certs\") pod \"metrics-server-597974fb65-bdrwd\" (UID: \"ea945a18-e37f-4957-8c7d-bee0cb2e9421\") " pod="openshift-monitoring/metrics-server-597974fb65-bdrwd" Jul 21 12:41:19.863639 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:41:19.863536 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2sq7k\" (UniqueName: \"kubernetes.io/projected/ea945a18-e37f-4957-8c7d-bee0cb2e9421-kube-api-access-2sq7k\") pod \"metrics-server-597974fb65-bdrwd\" (UID: \"ea945a18-e37f-4957-8c7d-bee0cb2e9421\") " pod="openshift-monitoring/metrics-server-597974fb65-bdrwd" Jul 21 12:41:19.863793 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:41:19.863767 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/ea945a18-e37f-4957-8c7d-bee0cb2e9421-audit-log\") pod \"metrics-server-597974fb65-bdrwd\" (UID: \"ea945a18-e37f-4957-8c7d-bee0cb2e9421\") " pod="openshift-monitoring/metrics-server-597974fb65-bdrwd" Jul 21 12:41:19.864033 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:41:19.864008 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ea945a18-e37f-4957-8c7d-bee0cb2e9421-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-597974fb65-bdrwd\" (UID: \"ea945a18-e37f-4957-8c7d-bee0cb2e9421\") " pod="openshift-monitoring/metrics-server-597974fb65-bdrwd" Jul 21 12:41:19.864249 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:41:19.864231 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/ea945a18-e37f-4957-8c7d-bee0cb2e9421-metrics-server-audit-profiles\") pod \"metrics-server-597974fb65-bdrwd\" (UID: \"ea945a18-e37f-4957-8c7d-bee0cb2e9421\") " pod="openshift-monitoring/metrics-server-597974fb65-bdrwd" Jul 21 12:41:19.865634 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:41:19.865608 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/ea945a18-e37f-4957-8c7d-bee0cb2e9421-secret-metrics-server-tls\") pod \"metrics-server-597974fb65-bdrwd\" (UID: \"ea945a18-e37f-4957-8c7d-bee0cb2e9421\") " pod="openshift-monitoring/metrics-server-597974fb65-bdrwd" Jul 21 12:41:19.865765 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:41:19.865743 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/ea945a18-e37f-4957-8c7d-bee0cb2e9421-client-ca-bundle\") pod \"metrics-server-597974fb65-bdrwd\" (UID: \"ea945a18-e37f-4957-8c7d-bee0cb2e9421\") " pod="openshift-monitoring/metrics-server-597974fb65-bdrwd" Jul 21 12:41:19.865847 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:41:19.865834 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/ea945a18-e37f-4957-8c7d-bee0cb2e9421-secret-metrics-server-client-certs\") pod \"metrics-server-597974fb65-bdrwd\" (UID: \"ea945a18-e37f-4957-8c7d-bee0cb2e9421\") " pod="openshift-monitoring/metrics-server-597974fb65-bdrwd" Jul 21 12:41:19.873412 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:41:19.873388 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2sq7k\" (UniqueName: \"kubernetes.io/projected/ea945a18-e37f-4957-8c7d-bee0cb2e9421-kube-api-access-2sq7k\") pod \"metrics-server-597974fb65-bdrwd\" (UID: \"ea945a18-e37f-4957-8c7d-bee0cb2e9421\") " pod="openshift-monitoring/metrics-server-597974fb65-bdrwd" Jul 21 12:41:20.023369 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:41:20.023313 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/metrics-server-597974fb65-bdrwd" Jul 21 12:41:20.050105 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:41:20.050072 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-skc26_6713665c-3b71-4751-929e-16de48263766/konnectivity-agent/0.log" limit=1024 Jul 21 12:41:20.148187 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:41:20.148161 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/metrics-server-597974fb65-bdrwd"] Jul 21 12:41:20.151541 ip-10-0-130-157 kubenswrapper[2576]: W0721 12:41:20.151500 2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podea945a18_e37f_4957_8c7d_bee0cb2e9421.slice/crio-9621d749ac2fa4351fc0724230536ec6ede32b47bf66a6721ce9d6f428ae3443 WatchSource:0}: Error finding container 9621d749ac2fa4351fc0724230536ec6ede32b47bf66a6721ce9d6f428ae3443: Status 404 returned error can't find the container with id 9621d749ac2fa4351fc0724230536ec6ede32b47bf66a6721ce9d6f428ae3443 Jul 21 12:41:20.283367 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:41:20.283310 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-skc26_6713665c-3b71-4751-929e-16de48263766/konnectivity-agent/0.log" limit=1024 Jul 21 12:41:20.502343 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:41:20.502307 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-skc26_6713665c-3b71-4751-929e-16de48263766/konnectivity-agent/0.log" limit=1024 Jul 21 12:41:20.581291 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:41:20.581228 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-skc26_6713665c-3b71-4751-929e-16de48263766/konnectivity-agent/0.log" limit=1024 Jul 21 12:41:20.677320 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:41:20.677288 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-c69sx" event={"ID":"a6e83d7e-1979-4094-8beb-c3f19ab6ccdf","Type":"ContainerStarted","Data":"635fb0acbf70ef60625e213ffe1194f5c6c1b8a9960320f8eecb15cdc6e6ffb3"} Jul 21 12:41:20.677701 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:41:20.677343 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-c69sx" event={"ID":"a6e83d7e-1979-4094-8beb-c3f19ab6ccdf","Type":"ContainerStarted","Data":"a003aec672cd1a0b8501593967fffb33cd39e38cd1f102d439b2cdf921953954"} Jul 21 12:41:20.678432 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:41:20.678410 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/metrics-server-597974fb65-bdrwd" event={"ID":"ea945a18-e37f-4957-8c7d-bee0cb2e9421","Type":"ContainerStarted","Data":"9621d749ac2fa4351fc0724230536ec6ede32b47bf66a6721ce9d6f428ae3443"} Jul 21 12:41:20.701453 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:41:20.701410 2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-c69sx" podStartSLOduration=2.353081587 podStartE2EDuration="5.701397711s" totalImagesPullingTime="3.285364794s" totalInitContainerRuntime="62.95133ms" isStatefulPod=true podCreationTimestamp="2026-07-21 12:41:15 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 12:41:20.699883166 +0000 UTC m=+88.535147044" watchObservedRunningTime="2026-07-21 12:41:20.701397711 +0000 UTC m=+88.536661576" Jul 21 12:41:20.764567 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:41:20.764503 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-skc26_6713665c-3b71-4751-929e-16de48263766/konnectivity-agent/0.log" limit=1024 Jul 21 12:41:21.264649 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:41:21.264616 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-skc26_6713665c-3b71-4751-929e-16de48263766/konnectivity-agent/0.log" limit=1024 Jul 21 12:41:21.365325 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:41:21.365292 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-skc26_6713665c-3b71-4751-929e-16de48263766/konnectivity-agent/0.log" limit=1024 Jul 21 12:41:21.465886 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:41:21.465855 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-skc26_6713665c-3b71-4751-929e-16de48263766/konnectivity-agent/0.log" limit=1024 Jul 21 12:41:21.679663 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:41:21.679570 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-skc26_6713665c-3b71-4751-929e-16de48263766/konnectivity-agent/0.log" limit=1024 Jul 21 12:41:21.683139 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:41:21.683101 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/metrics-server-597974fb65-bdrwd" event={"ID":"ea945a18-e37f-4957-8c7d-bee0cb2e9421","Type":"ContainerStarted","Data":"a1193cfdd9e8f935a4a59d3a3693e1d8cf991f695aa606938c09ae1b88359f8e"} Jul 21 12:41:21.701599 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:41:21.701574 2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jul 21 12:41:21.705600 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:41:21.705582 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" Jul 21 12:41:21.709688 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:41:21.709671 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls-assets-0\"" Jul 21 12:41:21.714876 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:41:21.714856 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-prometheus-http-client-file\"" Jul 21 12:41:21.714982 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:41:21.714877 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-kube-rbac-proxy-web\"" Jul 21 12:41:21.714982 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:41:21.714925 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-web-config\"" Jul 21 12:41:21.714982 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:41:21.714961 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"serving-certs-ca-bundle\"" Jul 21 12:41:21.714982 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:41:21.714977 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-1\"" Jul 21 12:41:21.715206 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:41:21.715084 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-sidecar-tls\"" Jul 21 12:41:21.715417 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:41:21.715399 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls\"" Jul 21 12:41:21.723898 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:41:21.723885 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-2\"" Jul 21 12:41:21.742009 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:41:21.741987 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s\"" Jul 21 12:41:21.742221 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:41:21.742204 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-dockercfg-w6429\"" Jul 21 12:41:21.742352 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:41:21.742336 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-client-certs\"" Jul 21 12:41:21.742756 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:41:21.742719 2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/metrics-server-597974fb65-bdrwd" podStartSLOduration=1.380445674 podStartE2EDuration="2.742704078s" totalImagesPullingTime="1.362258404s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 12:41:19 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 12:41:21.714982474 +0000 UTC m=+89.550246377" watchObservedRunningTime="2026-07-21 12:41:21.742704078 +0000 UTC m=+89.577967944" Jul 21 12:41:21.742973 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:41:21.742958 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-grpc-tls-dhjbeffbr1t5\"" Jul 21 12:41:21.743087 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:41:21.743065 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-0\"" Jul 21 12:41:21.744010 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:41:21.743988 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jul 21 12:41:21.752497 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:41:21.752473 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-rbac-proxy\"" Jul 21 12:41:21.754465 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:41:21.754447 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-trusted-ca-bundle\"" Jul 21 12:41:21.778990 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:41:21.778934 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/a2f209e9-2530-44a8-b31f-942b15f141a6-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"a2f209e9-2530-44a8-b31f-942b15f141a6\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 21 12:41:21.778990 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:41:21.778966 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/a2f209e9-2530-44a8-b31f-942b15f141a6-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"a2f209e9-2530-44a8-b31f-942b15f141a6\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 21 12:41:21.779135 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:41:21.778994 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a2f209e9-2530-44a8-b31f-942b15f141a6-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"a2f209e9-2530-44a8-b31f-942b15f141a6\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 21 12:41:21.779135 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:41:21.779072 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/a2f209e9-2530-44a8-b31f-942b15f141a6-config-out\") pod \"prometheus-k8s-0\" (UID: \"a2f209e9-2530-44a8-b31f-942b15f141a6\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 21 12:41:21.779135 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:41:21.779111 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/a2f209e9-2530-44a8-b31f-942b15f141a6-config\") pod \"prometheus-k8s-0\" (UID: \"a2f209e9-2530-44a8-b31f-942b15f141a6\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 21 12:41:21.779295 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:41:21.779279 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/a2f209e9-2530-44a8-b31f-942b15f141a6-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"a2f209e9-2530-44a8-b31f-942b15f141a6\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 21 12:41:21.779379 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:41:21.779363 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/a2f209e9-2530-44a8-b31f-942b15f141a6-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"a2f209e9-2530-44a8-b31f-942b15f141a6\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 21 12:41:21.779421 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:41:21.779398 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/a2f209e9-2530-44a8-b31f-942b15f141a6-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"a2f209e9-2530-44a8-b31f-942b15f141a6\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 21 12:41:21.779470 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:41:21.779454 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/a2f209e9-2530-44a8-b31f-942b15f141a6-web-config\") pod \"prometheus-k8s-0\" (UID: \"a2f209e9-2530-44a8-b31f-942b15f141a6\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 21 12:41:21.779510 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:41:21.779484 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a2f209e9-2530-44a8-b31f-942b15f141a6-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"a2f209e9-2530-44a8-b31f-942b15f141a6\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 21 12:41:21.779582 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:41:21.779540 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/a2f209e9-2530-44a8-b31f-942b15f141a6-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"a2f209e9-2530-44a8-b31f-942b15f141a6\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 21 12:41:21.779582 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:41:21.779564 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/a2f209e9-2530-44a8-b31f-942b15f141a6-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"a2f209e9-2530-44a8-b31f-942b15f141a6\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 21 12:41:21.779656 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:41:21.779635 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/a2f209e9-2530-44a8-b31f-942b15f141a6-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"a2f209e9-2530-44a8-b31f-942b15f141a6\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 21 12:41:21.779838 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:41:21.779818 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/a2f209e9-2530-44a8-b31f-942b15f141a6-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"a2f209e9-2530-44a8-b31f-942b15f141a6\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 21 12:41:21.779887 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:41:21.779868 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/a2f209e9-2530-44a8-b31f-942b15f141a6-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"a2f209e9-2530-44a8-b31f-942b15f141a6\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 21 12:41:21.779999 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:41:21.779982 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9v96t\" (UniqueName: \"kubernetes.io/projected/a2f209e9-2530-44a8-b31f-942b15f141a6-kube-api-access-9v96t\") pod \"prometheus-k8s-0\" (UID: \"a2f209e9-2530-44a8-b31f-942b15f141a6\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 21 12:41:21.780046 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:41:21.780019 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a2f209e9-2530-44a8-b31f-942b15f141a6-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"a2f209e9-2530-44a8-b31f-942b15f141a6\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 21 12:41:21.780082 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:41:21.780054 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/a2f209e9-2530-44a8-b31f-942b15f141a6-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"a2f209e9-2530-44a8-b31f-942b15f141a6\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 21 12:41:21.780082 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:41:21.780074 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/a2f209e9-2530-44a8-b31f-942b15f141a6-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"a2f209e9-2530-44a8-b31f-942b15f141a6\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 21 12:41:21.780151 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:41:21.780096 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/a2f209e9-2530-44a8-b31f-942b15f141a6-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"a2f209e9-2530-44a8-b31f-942b15f141a6\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 21 12:41:21.787471 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:41:21.787448 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-skc26_6713665c-3b71-4751-929e-16de48263766/konnectivity-agent/0.log" limit=1024 Jul 21 12:41:21.880793 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:41:21.880769 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/a2f209e9-2530-44a8-b31f-942b15f141a6-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"a2f209e9-2530-44a8-b31f-942b15f141a6\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 21 12:41:21.880884 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:41:21.880797 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9v96t\" (UniqueName: \"kubernetes.io/projected/a2f209e9-2530-44a8-b31f-942b15f141a6-kube-api-access-9v96t\") pod \"prometheus-k8s-0\" (UID: \"a2f209e9-2530-44a8-b31f-942b15f141a6\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 21 12:41:21.880884 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:41:21.880816 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a2f209e9-2530-44a8-b31f-942b15f141a6-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"a2f209e9-2530-44a8-b31f-942b15f141a6\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 21 12:41:21.880884 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:41:21.880834 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/a2f209e9-2530-44a8-b31f-942b15f141a6-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"a2f209e9-2530-44a8-b31f-942b15f141a6\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 21 12:41:21.880884 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:41:21.880860 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/a2f209e9-2530-44a8-b31f-942b15f141a6-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"a2f209e9-2530-44a8-b31f-942b15f141a6\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 21 12:41:21.881080 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:41:21.880886 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/a2f209e9-2530-44a8-b31f-942b15f141a6-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"a2f209e9-2530-44a8-b31f-942b15f141a6\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 21 12:41:21.881080 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:41:21.880913 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/a2f209e9-2530-44a8-b31f-942b15f141a6-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"a2f209e9-2530-44a8-b31f-942b15f141a6\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 21 12:41:21.881080 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:41:21.880935 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/a2f209e9-2530-44a8-b31f-942b15f141a6-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"a2f209e9-2530-44a8-b31f-942b15f141a6\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 21 12:41:21.881080 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:41:21.880960 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a2f209e9-2530-44a8-b31f-942b15f141a6-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"a2f209e9-2530-44a8-b31f-942b15f141a6\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 21 12:41:21.881080 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:41:21.880987 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/a2f209e9-2530-44a8-b31f-942b15f141a6-config-out\") pod \"prometheus-k8s-0\" (UID: \"a2f209e9-2530-44a8-b31f-942b15f141a6\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 21 12:41:21.881080 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:41:21.881006 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/a2f209e9-2530-44a8-b31f-942b15f141a6-config\") pod \"prometheus-k8s-0\" (UID: \"a2f209e9-2530-44a8-b31f-942b15f141a6\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 21 12:41:21.881080 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:41:21.881033 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/a2f209e9-2530-44a8-b31f-942b15f141a6-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"a2f209e9-2530-44a8-b31f-942b15f141a6\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 21 12:41:21.881411 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:41:21.881090 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/a2f209e9-2530-44a8-b31f-942b15f141a6-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"a2f209e9-2530-44a8-b31f-942b15f141a6\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 21 12:41:21.881411 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:41:21.881122 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/a2f209e9-2530-44a8-b31f-942b15f141a6-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"a2f209e9-2530-44a8-b31f-942b15f141a6\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 21 12:41:21.881411 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:41:21.881156 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/a2f209e9-2530-44a8-b31f-942b15f141a6-web-config\") pod \"prometheus-k8s-0\" (UID: \"a2f209e9-2530-44a8-b31f-942b15f141a6\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 21 12:41:21.881411 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:41:21.881184 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a2f209e9-2530-44a8-b31f-942b15f141a6-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"a2f209e9-2530-44a8-b31f-942b15f141a6\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 21 12:41:21.881411 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:41:21.881209 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/a2f209e9-2530-44a8-b31f-942b15f141a6-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"a2f209e9-2530-44a8-b31f-942b15f141a6\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 21 12:41:21.881411 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:41:21.881233 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/a2f209e9-2530-44a8-b31f-942b15f141a6-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"a2f209e9-2530-44a8-b31f-942b15f141a6\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 21 12:41:21.881411 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:41:21.881304 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/a2f209e9-2530-44a8-b31f-942b15f141a6-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"a2f209e9-2530-44a8-b31f-942b15f141a6\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 21 12:41:21.881767 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:41:21.881620 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a2f209e9-2530-44a8-b31f-942b15f141a6-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"a2f209e9-2530-44a8-b31f-942b15f141a6\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 21 12:41:21.881967 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:41:21.881944 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/a2f209e9-2530-44a8-b31f-942b15f141a6-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"a2f209e9-2530-44a8-b31f-942b15f141a6\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 21 12:41:21.882048 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:41:21.882027 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/a2f209e9-2530-44a8-b31f-942b15f141a6-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"a2f209e9-2530-44a8-b31f-942b15f141a6\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 21 12:41:21.882300 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:41:21.882266 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/a2f209e9-2530-44a8-b31f-942b15f141a6-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"a2f209e9-2530-44a8-b31f-942b15f141a6\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 21 12:41:21.883873 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:41:21.883845 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/a2f209e9-2530-44a8-b31f-942b15f141a6-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"a2f209e9-2530-44a8-b31f-942b15f141a6\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 21 12:41:21.884611 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:41:21.884315 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a2f209e9-2530-44a8-b31f-942b15f141a6-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"a2f209e9-2530-44a8-b31f-942b15f141a6\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 21 12:41:21.884611 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:41:21.884494 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/a2f209e9-2530-44a8-b31f-942b15f141a6-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"a2f209e9-2530-44a8-b31f-942b15f141a6\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 21 12:41:21.885324 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:41:21.885050 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/a2f209e9-2530-44a8-b31f-942b15f141a6-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"a2f209e9-2530-44a8-b31f-942b15f141a6\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 21 12:41:21.885956 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:41:21.885927 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a2f209e9-2530-44a8-b31f-942b15f141a6-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"a2f209e9-2530-44a8-b31f-942b15f141a6\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 21 12:41:21.887917 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:41:21.886361 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/a2f209e9-2530-44a8-b31f-942b15f141a6-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"a2f209e9-2530-44a8-b31f-942b15f141a6\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 21 12:41:21.887917 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:41:21.886925 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/a2f209e9-2530-44a8-b31f-942b15f141a6-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"a2f209e9-2530-44a8-b31f-942b15f141a6\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 21 12:41:21.887917 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:41:21.886975 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/a2f209e9-2530-44a8-b31f-942b15f141a6-config-out\") pod \"prometheus-k8s-0\" (UID: \"a2f209e9-2530-44a8-b31f-942b15f141a6\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 21 12:41:21.887917 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:41:21.887362 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/a2f209e9-2530-44a8-b31f-942b15f141a6-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"a2f209e9-2530-44a8-b31f-942b15f141a6\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 21 12:41:21.887917 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:41:21.887872 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/a2f209e9-2530-44a8-b31f-942b15f141a6-web-config\") pod \"prometheus-k8s-0\" (UID: \"a2f209e9-2530-44a8-b31f-942b15f141a6\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 21 12:41:21.891894 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:41:21.888498 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/a2f209e9-2530-44a8-b31f-942b15f141a6-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"a2f209e9-2530-44a8-b31f-942b15f141a6\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 21 12:41:21.891894 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:41:21.888656 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/a2f209e9-2530-44a8-b31f-942b15f141a6-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"a2f209e9-2530-44a8-b31f-942b15f141a6\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 21 12:41:21.891894 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:41:21.888904 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/secret/a2f209e9-2530-44a8-b31f-942b15f141a6-config\") pod \"prometheus-k8s-0\" (UID: \"a2f209e9-2530-44a8-b31f-942b15f141a6\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 21 12:41:21.891894 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:41:21.889546 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/a2f209e9-2530-44a8-b31f-942b15f141a6-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"a2f209e9-2530-44a8-b31f-942b15f141a6\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 21 12:41:21.891894 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:41:21.890267 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/a2f209e9-2530-44a8-b31f-942b15f141a6-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"a2f209e9-2530-44a8-b31f-942b15f141a6\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 21 12:41:21.893355 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:41:21.893333 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9v96t\" (UniqueName: \"kubernetes.io/projected/a2f209e9-2530-44a8-b31f-942b15f141a6-kube-api-access-9v96t\") pod \"prometheus-k8s-0\" (UID: \"a2f209e9-2530-44a8-b31f-942b15f141a6\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 21 12:41:21.893831 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:41:21.893815 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/a2f209e9-2530-44a8-b31f-942b15f141a6-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"a2f209e9-2530-44a8-b31f-942b15f141a6\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 21 12:41:21.907737 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:41:21.907710 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-skc26_6713665c-3b71-4751-929e-16de48263766/konnectivity-agent/0.log" limit=1024 Jul 21 12:41:22.014211 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:41:22.014190 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" Jul 21 12:41:22.153015 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:41:22.152989 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jul 21 12:41:22.158676 ip-10-0-130-157 kubenswrapper[2576]: W0721 12:41:22.158652 2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda2f209e9_2530_44a8_b31f_942b15f141a6.slice/crio-67b30c68410c4ce101b2b376d44cc28bc848707af276b4e03ff4466b2f4bc22d WatchSource:0}: Error finding container 67b30c68410c4ce101b2b376d44cc28bc848707af276b4e03ff4466b2f4bc22d: Status 404 returned error can't find the container with id 67b30c68410c4ce101b2b376d44cc28bc848707af276b4e03ff4466b2f4bc22d Jul 21 12:41:22.356223 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:41:22.356126 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-skc26_6713665c-3b71-4751-929e-16de48263766/konnectivity-agent/0.log" limit=1024 Jul 21 12:41:22.558699 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:41:22.558660 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-skc26_6713665c-3b71-4751-929e-16de48263766/konnectivity-agent/0.log" limit=1024 Jul 21 12:41:22.635679 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:41:22.635593 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-skc26_6713665c-3b71-4751-929e-16de48263766/konnectivity-agent/0.log" limit=1024 Jul 21 12:41:22.688388 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:41:22.688358 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"a2f209e9-2530-44a8-b31f-942b15f141a6","Type":"ContainerStarted","Data":"67b30c68410c4ce101b2b376d44cc28bc848707af276b4e03ff4466b2f4bc22d"} Jul 21 12:41:22.710369 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:41:22.710344 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-skc26_6713665c-3b71-4751-929e-16de48263766/konnectivity-agent/0.log" limit=1024 Jul 21 12:41:22.819664 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:41:22.819635 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-skc26_6713665c-3b71-4751-929e-16de48263766/konnectivity-agent/0.log" limit=1024 Jul 21 12:41:23.693088 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:41:23.693052 2576 generic.go:356] "Generic (PLEG): container finished" podID="a2f209e9-2530-44a8-b31f-942b15f141a6" containerID="0f37fbf7f8aa8483c00d738e9c25b7609ee7628075256f55f9844e002bea2bea" exitCode=0 Jul 21 12:41:23.693454 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:41:23.693154 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"a2f209e9-2530-44a8-b31f-942b15f141a6","Type":"ContainerDied","Data":"0f37fbf7f8aa8483c00d738e9c25b7609ee7628075256f55f9844e002bea2bea"} Jul 21 12:41:23.852226 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:41:23.852145 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-skc26_6713665c-3b71-4751-929e-16de48263766/konnectivity-agent/0.log" limit=1024 Jul 21 12:41:23.954771 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:41:23.954739 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-skc26_6713665c-3b71-4751-929e-16de48263766/konnectivity-agent/0.log" limit=1024 Jul 21 12:41:24.023182 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:41:24.023150 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-skc26_6713665c-3b71-4751-929e-16de48263766/konnectivity-agent/0.log" limit=1024 Jul 21 12:41:26.209371 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:41:26.209329 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-skc26_6713665c-3b71-4751-929e-16de48263766/konnectivity-agent/0.log" limit=1024 Jul 21 12:41:26.278335 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:41:26.278297 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-skc26_6713665c-3b71-4751-929e-16de48263766/konnectivity-agent/0.log" limit=1024 Jul 21 12:41:27.710964 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:41:27.710930 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"a2f209e9-2530-44a8-b31f-942b15f141a6","Type":"ContainerStarted","Data":"fd3d02813a819ea32dca9d4aa71f8e8d5e076a8d1d6f9b8647bbb81c63b87a61"} Jul 21 12:41:27.711403 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:41:27.711016 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"a2f209e9-2530-44a8-b31f-942b15f141a6","Type":"ContainerStarted","Data":"ca5a2ccb5233e6b6f3487ea8a324669b0d704e771d95c1e06f7df0435f456b17"} Jul 21 12:41:29.721812 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:41:29.721774 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"a2f209e9-2530-44a8-b31f-942b15f141a6","Type":"ContainerStarted","Data":"20258cf7c3a30e391cd7f0b855a71905a78b3e264a00af57d1c32399b62de2d8"} Jul 21 12:41:29.722269 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:41:29.721868 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"a2f209e9-2530-44a8-b31f-942b15f141a6","Type":"ContainerStarted","Data":"44051f152f926bbb4b472917bdfaa93e91a6d0742b90ac0b94a615300bbebd6c"} Jul 21 12:41:29.722269 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:41:29.721903 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"a2f209e9-2530-44a8-b31f-942b15f141a6","Type":"ContainerStarted","Data":"547f8d230bafffc154d05a87317c64a67341a8cfe84074f818634b2ea6103588"} Jul 21 12:41:29.722269 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:41:29.721927 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"a2f209e9-2530-44a8-b31f-942b15f141a6","Type":"ContainerStarted","Data":"c4d2880dab6684889918584054bc723c9b3b60958d3b3ff9b5c2504de2f1eb17"} Jul 21 12:41:29.756098 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:41:29.756041 2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-k8s-0" podStartSLOduration=2.244083283 podStartE2EDuration="8.756019719s" totalImagesPullingTime="6.489411903s" totalInitContainerRuntime="22.524533ms" isStatefulPod=false podCreationTimestamp="2026-07-21 12:41:21 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 12:41:29.754049811 +0000 UTC m=+97.589313676" watchObservedRunningTime="2026-07-21 12:41:29.756019719 +0000 UTC m=+97.591283587" Jul 21 12:41:31.012802 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:41:31.012771 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-skc26_6713665c-3b71-4751-929e-16de48263766/konnectivity-agent/0.log" limit=1024 Jul 21 12:41:31.213128 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:41:31.213100 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-skc26_6713665c-3b71-4751-929e-16de48263766/konnectivity-agent/0.log" limit=1024 Jul 21 12:41:32.014876 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:41:32.014847 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-k8s-0" Jul 21 12:41:32.990773 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:41:32.990735 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-skc26_6713665c-3b71-4751-929e-16de48263766/konnectivity-agent/0.log" limit=1024 Jul 21 12:41:33.094890 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:41:33.094859 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-skc26_6713665c-3b71-4751-929e-16de48263766/konnectivity-agent/0.log" limit=1024 Jul 21 12:41:35.980190 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:41:35.980161 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-skc26_6713665c-3b71-4751-929e-16de48263766/konnectivity-agent/0.log" limit=1024 Jul 21 12:41:36.056050 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:41:36.056024 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-skc26_6713665c-3b71-4751-929e-16de48263766/konnectivity-agent/0.log" limit=1024 Jul 21 12:41:38.617417 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:41:38.617386 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-6f57m" Jul 21 12:41:40.024334 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:41:40.024304 2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/metrics-server-597974fb65-bdrwd" Jul 21 12:41:40.024793 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:41:40.024362 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/metrics-server-597974fb65-bdrwd" Jul 21 12:41:40.416482 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:41:40.416457 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-skc26_6713665c-3b71-4751-929e-16de48263766/konnectivity-agent/0.log" limit=1024 Jul 21 12:41:40.508181 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:41:40.508152 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-skc26_6713665c-3b71-4751-929e-16de48263766/konnectivity-agent/0.log" limit=1024 Jul 21 12:41:40.994646 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:41:40.994610 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-skc26_6713665c-3b71-4751-929e-16de48263766/konnectivity-agent/0.log" limit=1024 Jul 21 12:41:41.093233 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:41:41.093203 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-skc26_6713665c-3b71-4751-929e-16de48263766/konnectivity-agent/0.log" limit=1024 Jul 21 12:41:45.668686 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:41:45.668652 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-skc26_6713665c-3b71-4751-929e-16de48263766/konnectivity-agent/0.log" limit=1024 Jul 21 12:41:45.985908 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:41:45.985829 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-skc26_6713665c-3b71-4751-929e-16de48263766/konnectivity-agent/0.log" limit=1024 Jul 21 12:41:46.066806 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:41:46.066774 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-skc26_6713665c-3b71-4751-929e-16de48263766/konnectivity-agent/0.log" limit=1024 Jul 21 12:41:46.799026 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:41:46.798992 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-skc26_6713665c-3b71-4751-929e-16de48263766/konnectivity-agent/0.log" limit=1024 Jul 21 12:41:46.872624 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:41:46.872591 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-skc26_6713665c-3b71-4751-929e-16de48263766/konnectivity-agent/0.log" limit=1024 Jul 21 12:41:46.973749 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:41:46.973720 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-skc26_6713665c-3b71-4751-929e-16de48263766/konnectivity-agent/0.log" limit=1024 Jul 21 12:41:50.714737 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:41:50.714703 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-skc26_6713665c-3b71-4751-929e-16de48263766/konnectivity-agent/0.log" limit=1024 Jul 21 12:41:50.807782 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:41:50.807751 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-skc26_6713665c-3b71-4751-929e-16de48263766/konnectivity-agent/0.log" limit=1024 Jul 21 12:42:00.029107 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:42:00.029075 2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/metrics-server-597974fb65-bdrwd" Jul 21 12:42:00.032897 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:42:00.032871 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/metrics-server-597974fb65-bdrwd" Jul 21 12:42:00.963592 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:42:00.963557 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-skc26_6713665c-3b71-4751-929e-16de48263766/konnectivity-agent/0.log" limit=1024 Jul 21 12:42:01.050716 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:42:01.050683 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-skc26_6713665c-3b71-4751-929e-16de48263766/konnectivity-agent/0.log" limit=1024 Jul 21 12:42:21.085502 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:42:21.085466 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-skc26_6713665c-3b71-4751-929e-16de48263766/konnectivity-agent/0.log" limit=1024 Jul 21 12:42:22.015034 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:42:22.014994 2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/prometheus-k8s-0" Jul 21 12:42:22.033503 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:42:22.033481 2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/prometheus-k8s-0" Jul 21 12:42:22.916160 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:42:22.916134 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-k8s-0" Jul 21 12:42:34.722216 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:42:34.722181 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-skc26_6713665c-3b71-4751-929e-16de48263766/konnectivity-agent/0.log" limit=1024 Jul 21 12:42:34.822470 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:42:34.822435 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-skc26_6713665c-3b71-4751-929e-16de48263766/konnectivity-agent/0.log" limit=1024 Jul 21 12:42:40.146908 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:42:40.146872 2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jul 21 12:42:40.147453 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:42:40.147154 2576 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="a2f209e9-2530-44a8-b31f-942b15f141a6" containerName="prometheus" containerID="cri-o://ca5a2ccb5233e6b6f3487ea8a324669b0d704e771d95c1e06f7df0435f456b17" gracePeriod=600 Jul 21 12:42:40.147453 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:42:40.147190 2576 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="a2f209e9-2530-44a8-b31f-942b15f141a6" containerName="kube-rbac-proxy" containerID="cri-o://44051f152f926bbb4b472917bdfaa93e91a6d0742b90ac0b94a615300bbebd6c" gracePeriod=600 Jul 21 12:42:40.147453 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:42:40.147208 2576 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="a2f209e9-2530-44a8-b31f-942b15f141a6" containerName="config-reloader" containerID="cri-o://fd3d02813a819ea32dca9d4aa71f8e8d5e076a8d1d6f9b8647bbb81c63b87a61" gracePeriod=600 Jul 21 12:42:40.147453 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:42:40.147224 2576 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="a2f209e9-2530-44a8-b31f-942b15f141a6" containerName="thanos-sidecar" containerID="cri-o://c4d2880dab6684889918584054bc723c9b3b60958d3b3ff9b5c2504de2f1eb17" gracePeriod=600 Jul 21 12:42:40.147453 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:42:40.147246 2576 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="a2f209e9-2530-44a8-b31f-942b15f141a6" containerName="kube-rbac-proxy-web" containerID="cri-o://547f8d230bafffc154d05a87317c64a67341a8cfe84074f818634b2ea6103588" gracePeriod=600 Jul 21 12:42:40.147453 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:42:40.147243 2576 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="a2f209e9-2530-44a8-b31f-942b15f141a6" containerName="kube-rbac-proxy-thanos" containerID="cri-o://20258cf7c3a30e391cd7f0b855a71905a78b3e264a00af57d1c32399b62de2d8" gracePeriod=600 Jul 21 12:42:40.379257 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:42:40.379233 2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" Jul 21 12:42:40.434486 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:42:40.434412 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/a2f209e9-2530-44a8-b31f-942b15f141a6-thanos-prometheus-http-client-file\") pod \"a2f209e9-2530-44a8-b31f-942b15f141a6\" (UID: \"a2f209e9-2530-44a8-b31f-942b15f141a6\") " Jul 21 12:42:40.434486 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:42:40.434452 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/a2f209e9-2530-44a8-b31f-942b15f141a6-prometheus-k8s-rulefiles-2\") pod \"a2f209e9-2530-44a8-b31f-942b15f141a6\" (UID: \"a2f209e9-2530-44a8-b31f-942b15f141a6\") " Jul 21 12:42:40.434726 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:42:40.434493 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/a2f209e9-2530-44a8-b31f-942b15f141a6-web-config\") pod \"a2f209e9-2530-44a8-b31f-942b15f141a6\" (UID: \"a2f209e9-2530-44a8-b31f-942b15f141a6\") " Jul 21 12:42:40.434726 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:42:40.434571 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/a2f209e9-2530-44a8-b31f-942b15f141a6-prometheus-k8s-rulefiles-0\") pod \"a2f209e9-2530-44a8-b31f-942b15f141a6\" (UID: \"a2f209e9-2530-44a8-b31f-942b15f141a6\") " Jul 21 12:42:40.434726 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:42:40.434600 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/a2f209e9-2530-44a8-b31f-942b15f141a6-prometheus-k8s-db\") pod \"a2f209e9-2530-44a8-b31f-942b15f141a6\" (UID: \"a2f209e9-2530-44a8-b31f-942b15f141a6\") " Jul 21 12:42:40.434726 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:42:40.434632 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/a2f209e9-2530-44a8-b31f-942b15f141a6-secret-prometheus-k8s-tls\") pod \"a2f209e9-2530-44a8-b31f-942b15f141a6\" (UID: \"a2f209e9-2530-44a8-b31f-942b15f141a6\") " Jul 21 12:42:40.434932 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:42:40.434749 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/a2f209e9-2530-44a8-b31f-942b15f141a6-tls-assets\") pod \"a2f209e9-2530-44a8-b31f-942b15f141a6\" (UID: \"a2f209e9-2530-44a8-b31f-942b15f141a6\") " Jul 21 12:42:40.434932 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:42:40.434805 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/a2f209e9-2530-44a8-b31f-942b15f141a6-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"a2f209e9-2530-44a8-b31f-942b15f141a6\" (UID: \"a2f209e9-2530-44a8-b31f-942b15f141a6\") " Jul 21 12:42:40.434932 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:42:40.434832 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/a2f209e9-2530-44a8-b31f-942b15f141a6-secret-grpc-tls\") pod \"a2f209e9-2530-44a8-b31f-942b15f141a6\" (UID: \"a2f209e9-2530-44a8-b31f-942b15f141a6\") " Jul 21 12:42:40.434932 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:42:40.434859 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a2f209e9-2530-44a8-b31f-942b15f141a6-configmap-serving-certs-ca-bundle\") pod \"a2f209e9-2530-44a8-b31f-942b15f141a6\" (UID: \"a2f209e9-2530-44a8-b31f-942b15f141a6\") " Jul 21 12:42:40.434932 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:42:40.434872 2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a2f209e9-2530-44a8-b31f-942b15f141a6-prometheus-k8s-rulefiles-2" (OuterVolumeSpecName: "prometheus-k8s-rulefiles-2") pod "a2f209e9-2530-44a8-b31f-942b15f141a6" (UID: "a2f209e9-2530-44a8-b31f-942b15f141a6"). InnerVolumeSpecName "prometheus-k8s-rulefiles-2". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 21 12:42:40.434932 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:42:40.434886 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/a2f209e9-2530-44a8-b31f-942b15f141a6-secret-kube-rbac-proxy\") pod \"a2f209e9-2530-44a8-b31f-942b15f141a6\" (UID: \"a2f209e9-2530-44a8-b31f-942b15f141a6\") " Jul 21 12:42:40.434932 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:42:40.434924 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/a2f209e9-2530-44a8-b31f-942b15f141a6-config-out\") pod \"a2f209e9-2530-44a8-b31f-942b15f141a6\" (UID: \"a2f209e9-2530-44a8-b31f-942b15f141a6\") " Jul 21 12:42:40.435265 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:42:40.434948 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/a2f209e9-2530-44a8-b31f-942b15f141a6-configmap-metrics-client-ca\") pod \"a2f209e9-2530-44a8-b31f-942b15f141a6\" (UID: \"a2f209e9-2530-44a8-b31f-942b15f141a6\") " Jul 21 12:42:40.435265 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:42:40.434983 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/a2f209e9-2530-44a8-b31f-942b15f141a6-config\") pod \"a2f209e9-2530-44a8-b31f-942b15f141a6\" (UID: \"a2f209e9-2530-44a8-b31f-942b15f141a6\") " Jul 21 12:42:40.435265 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:42:40.435008 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-9v96t\" (UniqueName: \"kubernetes.io/projected/a2f209e9-2530-44a8-b31f-942b15f141a6-kube-api-access-9v96t\") pod \"a2f209e9-2530-44a8-b31f-942b15f141a6\" (UID: \"a2f209e9-2530-44a8-b31f-942b15f141a6\") " Jul 21 12:42:40.435265 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:42:40.435047 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a2f209e9-2530-44a8-b31f-942b15f141a6-configmap-kubelet-serving-ca-bundle\") pod \"a2f209e9-2530-44a8-b31f-942b15f141a6\" (UID: \"a2f209e9-2530-44a8-b31f-942b15f141a6\") " Jul 21 12:42:40.435265 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:42:40.435090 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a2f209e9-2530-44a8-b31f-942b15f141a6-prometheus-trusted-ca-bundle\") pod \"a2f209e9-2530-44a8-b31f-942b15f141a6\" (UID: \"a2f209e9-2530-44a8-b31f-942b15f141a6\") " Jul 21 12:42:40.435265 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:42:40.435116 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/a2f209e9-2530-44a8-b31f-942b15f141a6-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"a2f209e9-2530-44a8-b31f-942b15f141a6\" (UID: \"a2f209e9-2530-44a8-b31f-942b15f141a6\") " Jul 21 12:42:40.435265 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:42:40.435146 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/a2f209e9-2530-44a8-b31f-942b15f141a6-secret-metrics-client-certs\") pod \"a2f209e9-2530-44a8-b31f-942b15f141a6\" (UID: \"a2f209e9-2530-44a8-b31f-942b15f141a6\") " Jul 21 12:42:40.435265 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:42:40.435174 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/a2f209e9-2530-44a8-b31f-942b15f141a6-prometheus-k8s-rulefiles-1\") pod \"a2f209e9-2530-44a8-b31f-942b15f141a6\" (UID: \"a2f209e9-2530-44a8-b31f-942b15f141a6\") " Jul 21 12:42:40.435661 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:42:40.435425 2576 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/a2f209e9-2530-44a8-b31f-942b15f141a6-prometheus-k8s-rulefiles-2\") on node \"ip-10-0-130-157.ec2.internal\" DevicePath \"\"" Jul 21 12:42:40.436662 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:42:40.435748 2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a2f209e9-2530-44a8-b31f-942b15f141a6-prometheus-k8s-rulefiles-1" (OuterVolumeSpecName: "prometheus-k8s-rulefiles-1") pod "a2f209e9-2530-44a8-b31f-942b15f141a6" (UID: "a2f209e9-2530-44a8-b31f-942b15f141a6"). InnerVolumeSpecName "prometheus-k8s-rulefiles-1". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 21 12:42:40.436662 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:42:40.436066 2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a2f209e9-2530-44a8-b31f-942b15f141a6-prometheus-k8s-rulefiles-0" (OuterVolumeSpecName: "prometheus-k8s-rulefiles-0") pod "a2f209e9-2530-44a8-b31f-942b15f141a6" (UID: "a2f209e9-2530-44a8-b31f-942b15f141a6"). InnerVolumeSpecName "prometheus-k8s-rulefiles-0". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 21 12:42:40.436662 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:42:40.436072 2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a2f209e9-2530-44a8-b31f-942b15f141a6-prometheus-trusted-ca-bundle" (OuterVolumeSpecName: "prometheus-trusted-ca-bundle") pod "a2f209e9-2530-44a8-b31f-942b15f141a6" (UID: "a2f209e9-2530-44a8-b31f-942b15f141a6"). InnerVolumeSpecName "prometheus-trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 21 12:42:40.436662 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:42:40.436627 2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a2f209e9-2530-44a8-b31f-942b15f141a6-configmap-kubelet-serving-ca-bundle" (OuterVolumeSpecName: "configmap-kubelet-serving-ca-bundle") pod "a2f209e9-2530-44a8-b31f-942b15f141a6" (UID: "a2f209e9-2530-44a8-b31f-942b15f141a6"). InnerVolumeSpecName "configmap-kubelet-serving-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 21 12:42:40.437232 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:42:40.437042 2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a2f209e9-2530-44a8-b31f-942b15f141a6-configmap-metrics-client-ca" (OuterVolumeSpecName: "configmap-metrics-client-ca") pod "a2f209e9-2530-44a8-b31f-942b15f141a6" (UID: "a2f209e9-2530-44a8-b31f-942b15f141a6"). InnerVolumeSpecName "configmap-metrics-client-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 21 12:42:40.437362 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:42:40.437335 2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a2f209e9-2530-44a8-b31f-942b15f141a6-prometheus-k8s-db" (OuterVolumeSpecName: "prometheus-k8s-db") pod "a2f209e9-2530-44a8-b31f-942b15f141a6" (UID: "a2f209e9-2530-44a8-b31f-942b15f141a6"). InnerVolumeSpecName "prometheus-k8s-db". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 21 12:42:40.437556 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:42:40.437532 2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a2f209e9-2530-44a8-b31f-942b15f141a6-thanos-prometheus-http-client-file" (OuterVolumeSpecName: "thanos-prometheus-http-client-file") pod "a2f209e9-2530-44a8-b31f-942b15f141a6" (UID: "a2f209e9-2530-44a8-b31f-942b15f141a6"). InnerVolumeSpecName "thanos-prometheus-http-client-file". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 21 12:42:40.437771 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:42:40.437745 2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a2f209e9-2530-44a8-b31f-942b15f141a6-secret-prometheus-k8s-kube-rbac-proxy-web" (OuterVolumeSpecName: "secret-prometheus-k8s-kube-rbac-proxy-web") pod "a2f209e9-2530-44a8-b31f-942b15f141a6" (UID: "a2f209e9-2530-44a8-b31f-942b15f141a6"). InnerVolumeSpecName "secret-prometheus-k8s-kube-rbac-proxy-web". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 21 12:42:40.437845 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:42:40.437827 2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a2f209e9-2530-44a8-b31f-942b15f141a6-configmap-serving-certs-ca-bundle" (OuterVolumeSpecName: "configmap-serving-certs-ca-bundle") pod "a2f209e9-2530-44a8-b31f-942b15f141a6" (UID: "a2f209e9-2530-44a8-b31f-942b15f141a6"). InnerVolumeSpecName "configmap-serving-certs-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 21 12:42:40.439836 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:42:40.439799 2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a2f209e9-2530-44a8-b31f-942b15f141a6-secret-metrics-client-certs" (OuterVolumeSpecName: "secret-metrics-client-certs") pod "a2f209e9-2530-44a8-b31f-942b15f141a6" (UID: "a2f209e9-2530-44a8-b31f-942b15f141a6"). InnerVolumeSpecName "secret-metrics-client-certs". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 21 12:42:40.439836 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:42:40.439823 2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a2f209e9-2530-44a8-b31f-942b15f141a6-config" (OuterVolumeSpecName: "config") pod "a2f209e9-2530-44a8-b31f-942b15f141a6" (UID: "a2f209e9-2530-44a8-b31f-942b15f141a6"). InnerVolumeSpecName "config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 21 12:42:40.440107 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:42:40.440071 2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a2f209e9-2530-44a8-b31f-942b15f141a6-kube-api-access-9v96t" (OuterVolumeSpecName: "kube-api-access-9v96t") pod "a2f209e9-2530-44a8-b31f-942b15f141a6" (UID: "a2f209e9-2530-44a8-b31f-942b15f141a6"). InnerVolumeSpecName "kube-api-access-9v96t". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 21 12:42:40.440189 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:42:40.440132 2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a2f209e9-2530-44a8-b31f-942b15f141a6-secret-prometheus-k8s-thanos-sidecar-tls" (OuterVolumeSpecName: "secret-prometheus-k8s-thanos-sidecar-tls") pod "a2f209e9-2530-44a8-b31f-942b15f141a6" (UID: "a2f209e9-2530-44a8-b31f-942b15f141a6"). InnerVolumeSpecName "secret-prometheus-k8s-thanos-sidecar-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 21 12:42:40.440189 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:42:40.440167 2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a2f209e9-2530-44a8-b31f-942b15f141a6-secret-prometheus-k8s-tls" (OuterVolumeSpecName: "secret-prometheus-k8s-tls") pod "a2f209e9-2530-44a8-b31f-942b15f141a6" (UID: "a2f209e9-2530-44a8-b31f-942b15f141a6"). InnerVolumeSpecName "secret-prometheus-k8s-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 21 12:42:40.440579 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:42:40.440533 2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a2f209e9-2530-44a8-b31f-942b15f141a6-secret-kube-rbac-proxy" (OuterVolumeSpecName: "secret-kube-rbac-proxy") pod "a2f209e9-2530-44a8-b31f-942b15f141a6" (UID: "a2f209e9-2530-44a8-b31f-942b15f141a6"). InnerVolumeSpecName "secret-kube-rbac-proxy". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 21 12:42:40.440579 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:42:40.440537 2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a2f209e9-2530-44a8-b31f-942b15f141a6-config-out" (OuterVolumeSpecName: "config-out") pod "a2f209e9-2530-44a8-b31f-942b15f141a6" (UID: "a2f209e9-2530-44a8-b31f-942b15f141a6"). InnerVolumeSpecName "config-out". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 21 12:42:40.440960 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:42:40.440930 2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a2f209e9-2530-44a8-b31f-942b15f141a6-tls-assets" (OuterVolumeSpecName: "tls-assets") pod "a2f209e9-2530-44a8-b31f-942b15f141a6" (UID: "a2f209e9-2530-44a8-b31f-942b15f141a6"). InnerVolumeSpecName "tls-assets". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 21 12:42:40.441202 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:42:40.441180 2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a2f209e9-2530-44a8-b31f-942b15f141a6-secret-grpc-tls" (OuterVolumeSpecName: "secret-grpc-tls") pod "a2f209e9-2530-44a8-b31f-942b15f141a6" (UID: "a2f209e9-2530-44a8-b31f-942b15f141a6"). InnerVolumeSpecName "secret-grpc-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 21 12:42:40.447532 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:42:40.447502 2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a2f209e9-2530-44a8-b31f-942b15f141a6-web-config" (OuterVolumeSpecName: "web-config") pod "a2f209e9-2530-44a8-b31f-942b15f141a6" (UID: "a2f209e9-2530-44a8-b31f-942b15f141a6"). InnerVolumeSpecName "web-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 21 12:42:40.536298 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:42:40.536271 2576 reconciler_common.go:299] "Volume detached for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/a2f209e9-2530-44a8-b31f-942b15f141a6-web-config\") on node \"ip-10-0-130-157.ec2.internal\" DevicePath \"\"" Jul 21 12:42:40.536298 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:42:40.536295 2576 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/a2f209e9-2530-44a8-b31f-942b15f141a6-prometheus-k8s-rulefiles-0\") on node \"ip-10-0-130-157.ec2.internal\" DevicePath \"\"" Jul 21 12:42:40.536494 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:42:40.536309 2576 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/a2f209e9-2530-44a8-b31f-942b15f141a6-prometheus-k8s-db\") on node \"ip-10-0-130-157.ec2.internal\" DevicePath \"\"" Jul 21 12:42:40.536494 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:42:40.536318 2576 reconciler_common.go:299] "Volume detached for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/a2f209e9-2530-44a8-b31f-942b15f141a6-secret-prometheus-k8s-tls\") on node \"ip-10-0-130-157.ec2.internal\" DevicePath \"\"" Jul 21 12:42:40.536494 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:42:40.536326 2576 reconciler_common.go:299] "Volume detached for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/a2f209e9-2530-44a8-b31f-942b15f141a6-tls-assets\") on node \"ip-10-0-130-157.ec2.internal\" DevicePath \"\"" Jul 21 12:42:40.536494 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:42:40.536337 2576 reconciler_common.go:299] "Volume detached for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/a2f209e9-2530-44a8-b31f-942b15f141a6-secret-prometheus-k8s-kube-rbac-proxy-web\") on node \"ip-10-0-130-157.ec2.internal\" DevicePath \"\"" Jul 21 12:42:40.536494 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:42:40.536345 2576 reconciler_common.go:299] "Volume detached for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/a2f209e9-2530-44a8-b31f-942b15f141a6-secret-grpc-tls\") on node \"ip-10-0-130-157.ec2.internal\" DevicePath \"\"" Jul 21 12:42:40.536494 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:42:40.536354 2576 reconciler_common.go:299] "Volume detached for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a2f209e9-2530-44a8-b31f-942b15f141a6-configmap-serving-certs-ca-bundle\") on node \"ip-10-0-130-157.ec2.internal\" DevicePath \"\"" Jul 21 12:42:40.536494 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:42:40.536363 2576 reconciler_common.go:299] "Volume detached for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/a2f209e9-2530-44a8-b31f-942b15f141a6-secret-kube-rbac-proxy\") on node \"ip-10-0-130-157.ec2.internal\" DevicePath \"\"" Jul 21 12:42:40.536494 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:42:40.536372 2576 reconciler_common.go:299] "Volume detached for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/a2f209e9-2530-44a8-b31f-942b15f141a6-config-out\") on node \"ip-10-0-130-157.ec2.internal\" DevicePath \"\"" Jul 21 12:42:40.536494 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:42:40.536380 2576 reconciler_common.go:299] "Volume detached for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/a2f209e9-2530-44a8-b31f-942b15f141a6-configmap-metrics-client-ca\") on node \"ip-10-0-130-157.ec2.internal\" DevicePath \"\"" Jul 21 12:42:40.536494 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:42:40.536388 2576 reconciler_common.go:299] "Volume detached for volume \"config\" (UniqueName: \"kubernetes.io/secret/a2f209e9-2530-44a8-b31f-942b15f141a6-config\") on node \"ip-10-0-130-157.ec2.internal\" DevicePath \"\"" Jul 21 12:42:40.536494 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:42:40.536396 2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-9v96t\" (UniqueName: \"kubernetes.io/projected/a2f209e9-2530-44a8-b31f-942b15f141a6-kube-api-access-9v96t\") on node \"ip-10-0-130-157.ec2.internal\" DevicePath \"\"" Jul 21 12:42:40.536494 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:42:40.536405 2576 reconciler_common.go:299] "Volume detached for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a2f209e9-2530-44a8-b31f-942b15f141a6-configmap-kubelet-serving-ca-bundle\") on node \"ip-10-0-130-157.ec2.internal\" DevicePath \"\"" Jul 21 12:42:40.536494 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:42:40.536413 2576 reconciler_common.go:299] "Volume detached for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a2f209e9-2530-44a8-b31f-942b15f141a6-prometheus-trusted-ca-bundle\") on node \"ip-10-0-130-157.ec2.internal\" DevicePath \"\"" Jul 21 12:42:40.536494 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:42:40.536421 2576 reconciler_common.go:299] "Volume detached for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/a2f209e9-2530-44a8-b31f-942b15f141a6-secret-prometheus-k8s-thanos-sidecar-tls\") on node \"ip-10-0-130-157.ec2.internal\" DevicePath \"\"" Jul 21 12:42:40.536494 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:42:40.536437 2576 reconciler_common.go:299] "Volume detached for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/a2f209e9-2530-44a8-b31f-942b15f141a6-secret-metrics-client-certs\") on node \"ip-10-0-130-157.ec2.internal\" DevicePath \"\"" Jul 21 12:42:40.536494 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:42:40.536446 2576 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/a2f209e9-2530-44a8-b31f-942b15f141a6-prometheus-k8s-rulefiles-1\") on node \"ip-10-0-130-157.ec2.internal\" DevicePath \"\"" Jul 21 12:42:40.536494 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:42:40.536454 2576 reconciler_common.go:299] "Volume detached for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/a2f209e9-2530-44a8-b31f-942b15f141a6-thanos-prometheus-http-client-file\") on node \"ip-10-0-130-157.ec2.internal\" DevicePath \"\"" Jul 21 12:42:40.967783 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:42:40.967745 2576 generic.go:356] "Generic (PLEG): container finished" podID="a2f209e9-2530-44a8-b31f-942b15f141a6" containerID="20258cf7c3a30e391cd7f0b855a71905a78b3e264a00af57d1c32399b62de2d8" exitCode=0 Jul 21 12:42:40.967783 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:42:40.967778 2576 generic.go:356] "Generic (PLEG): container finished" podID="a2f209e9-2530-44a8-b31f-942b15f141a6" containerID="44051f152f926bbb4b472917bdfaa93e91a6d0742b90ac0b94a615300bbebd6c" exitCode=0 Jul 21 12:42:40.967783 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:42:40.967788 2576 generic.go:356] "Generic (PLEG): container finished" podID="a2f209e9-2530-44a8-b31f-942b15f141a6" containerID="547f8d230bafffc154d05a87317c64a67341a8cfe84074f818634b2ea6103588" exitCode=0 Jul 21 12:42:40.968014 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:42:40.967796 2576 generic.go:356] "Generic (PLEG): container finished" podID="a2f209e9-2530-44a8-b31f-942b15f141a6" containerID="c4d2880dab6684889918584054bc723c9b3b60958d3b3ff9b5c2504de2f1eb17" exitCode=0 Jul 21 12:42:40.968014 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:42:40.967804 2576 generic.go:356] "Generic (PLEG): container finished" podID="a2f209e9-2530-44a8-b31f-942b15f141a6" containerID="fd3d02813a819ea32dca9d4aa71f8e8d5e076a8d1d6f9b8647bbb81c63b87a61" exitCode=0 Jul 21 12:42:40.968014 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:42:40.967811 2576 generic.go:356] "Generic (PLEG): container finished" podID="a2f209e9-2530-44a8-b31f-942b15f141a6" containerID="ca5a2ccb5233e6b6f3487ea8a324669b0d704e771d95c1e06f7df0435f456b17" exitCode=0 Jul 21 12:42:40.968014 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:42:40.967836 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"a2f209e9-2530-44a8-b31f-942b15f141a6","Type":"ContainerDied","Data":"20258cf7c3a30e391cd7f0b855a71905a78b3e264a00af57d1c32399b62de2d8"} Jul 21 12:42:40.968014 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:42:40.967868 2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" Jul 21 12:42:40.968014 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:42:40.967938 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"a2f209e9-2530-44a8-b31f-942b15f141a6","Type":"ContainerDied","Data":"44051f152f926bbb4b472917bdfaa93e91a6d0742b90ac0b94a615300bbebd6c"} Jul 21 12:42:40.968014 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:42:40.967966 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"a2f209e9-2530-44a8-b31f-942b15f141a6","Type":"ContainerDied","Data":"547f8d230bafffc154d05a87317c64a67341a8cfe84074f818634b2ea6103588"} Jul 21 12:42:40.968014 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:42:40.967990 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"a2f209e9-2530-44a8-b31f-942b15f141a6","Type":"ContainerDied","Data":"c4d2880dab6684889918584054bc723c9b3b60958d3b3ff9b5c2504de2f1eb17"} Jul 21 12:42:40.968014 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:42:40.968011 2576 scope.go:117] "RemoveContainer" containerID="20258cf7c3a30e391cd7f0b855a71905a78b3e264a00af57d1c32399b62de2d8" Jul 21 12:42:40.968334 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:42:40.968014 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"a2f209e9-2530-44a8-b31f-942b15f141a6","Type":"ContainerDied","Data":"fd3d02813a819ea32dca9d4aa71f8e8d5e076a8d1d6f9b8647bbb81c63b87a61"} Jul 21 12:42:40.968334 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:42:40.968116 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"a2f209e9-2530-44a8-b31f-942b15f141a6","Type":"ContainerDied","Data":"ca5a2ccb5233e6b6f3487ea8a324669b0d704e771d95c1e06f7df0435f456b17"} Jul 21 12:42:40.968334 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:42:40.968146 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"a2f209e9-2530-44a8-b31f-942b15f141a6","Type":"ContainerDied","Data":"67b30c68410c4ce101b2b376d44cc28bc848707af276b4e03ff4466b2f4bc22d"} Jul 21 12:42:40.977316 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:42:40.977297 2576 scope.go:117] "RemoveContainer" containerID="44051f152f926bbb4b472917bdfaa93e91a6d0742b90ac0b94a615300bbebd6c" Jul 21 12:42:40.984712 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:42:40.984693 2576 scope.go:117] "RemoveContainer" containerID="547f8d230bafffc154d05a87317c64a67341a8cfe84074f818634b2ea6103588" Jul 21 12:42:40.990883 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:42:40.990868 2576 scope.go:117] "RemoveContainer" containerID="c4d2880dab6684889918584054bc723c9b3b60958d3b3ff9b5c2504de2f1eb17" Jul 21 12:42:40.993792 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:42:40.993767 2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jul 21 12:42:40.997931 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:42:40.997879 2576 scope.go:117] "RemoveContainer" containerID="fd3d02813a819ea32dca9d4aa71f8e8d5e076a8d1d6f9b8647bbb81c63b87a61" Jul 21 12:42:40.999236 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:42:40.999192 2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jul 21 12:42:41.004362 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:42:41.004344 2576 scope.go:117] "RemoveContainer" containerID="ca5a2ccb5233e6b6f3487ea8a324669b0d704e771d95c1e06f7df0435f456b17" Jul 21 12:42:41.011308 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:42:41.011291 2576 scope.go:117] "RemoveContainer" containerID="0f37fbf7f8aa8483c00d738e9c25b7609ee7628075256f55f9844e002bea2bea" Jul 21 12:42:41.017389 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:42:41.017373 2576 scope.go:117] "RemoveContainer" containerID="20258cf7c3a30e391cd7f0b855a71905a78b3e264a00af57d1c32399b62de2d8" Jul 21 12:42:41.017644 ip-10-0-130-157 kubenswrapper[2576]: E0721 12:42:41.017624 2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"20258cf7c3a30e391cd7f0b855a71905a78b3e264a00af57d1c32399b62de2d8\": container with ID starting with 20258cf7c3a30e391cd7f0b855a71905a78b3e264a00af57d1c32399b62de2d8 not found: ID does not exist" containerID="20258cf7c3a30e391cd7f0b855a71905a78b3e264a00af57d1c32399b62de2d8" Jul 21 12:42:41.017696 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:42:41.017653 2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"20258cf7c3a30e391cd7f0b855a71905a78b3e264a00af57d1c32399b62de2d8"} err="failed to get container status \"20258cf7c3a30e391cd7f0b855a71905a78b3e264a00af57d1c32399b62de2d8\": rpc error: code = NotFound desc = could not find container \"20258cf7c3a30e391cd7f0b855a71905a78b3e264a00af57d1c32399b62de2d8\": container with ID starting with 20258cf7c3a30e391cd7f0b855a71905a78b3e264a00af57d1c32399b62de2d8 not found: ID does not exist" Jul 21 12:42:41.017696 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:42:41.017683 2576 scope.go:117] "RemoveContainer" containerID="44051f152f926bbb4b472917bdfaa93e91a6d0742b90ac0b94a615300bbebd6c" Jul 21 12:42:41.017925 ip-10-0-130-157 kubenswrapper[2576]: E0721 12:42:41.017910 2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"44051f152f926bbb4b472917bdfaa93e91a6d0742b90ac0b94a615300bbebd6c\": container with ID starting with 44051f152f926bbb4b472917bdfaa93e91a6d0742b90ac0b94a615300bbebd6c not found: ID does not exist" containerID="44051f152f926bbb4b472917bdfaa93e91a6d0742b90ac0b94a615300bbebd6c" Jul 21 12:42:41.017966 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:42:41.017928 2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"44051f152f926bbb4b472917bdfaa93e91a6d0742b90ac0b94a615300bbebd6c"} err="failed to get container status \"44051f152f926bbb4b472917bdfaa93e91a6d0742b90ac0b94a615300bbebd6c\": rpc error: code = NotFound desc = could not find container \"44051f152f926bbb4b472917bdfaa93e91a6d0742b90ac0b94a615300bbebd6c\": container with ID starting with 44051f152f926bbb4b472917bdfaa93e91a6d0742b90ac0b94a615300bbebd6c not found: ID does not exist" Jul 21 12:42:41.017966 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:42:41.017940 2576 scope.go:117] "RemoveContainer" containerID="547f8d230bafffc154d05a87317c64a67341a8cfe84074f818634b2ea6103588" Jul 21 12:42:41.018124 ip-10-0-130-157 kubenswrapper[2576]: E0721 12:42:41.018109 2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"547f8d230bafffc154d05a87317c64a67341a8cfe84074f818634b2ea6103588\": container with ID starting with 547f8d230bafffc154d05a87317c64a67341a8cfe84074f818634b2ea6103588 not found: ID does not exist" containerID="547f8d230bafffc154d05a87317c64a67341a8cfe84074f818634b2ea6103588" Jul 21 12:42:41.018161 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:42:41.018128 2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"547f8d230bafffc154d05a87317c64a67341a8cfe84074f818634b2ea6103588"} err="failed to get container status \"547f8d230bafffc154d05a87317c64a67341a8cfe84074f818634b2ea6103588\": rpc error: code = NotFound desc = could not find container \"547f8d230bafffc154d05a87317c64a67341a8cfe84074f818634b2ea6103588\": container with ID starting with 547f8d230bafffc154d05a87317c64a67341a8cfe84074f818634b2ea6103588 not found: ID does not exist" Jul 21 12:42:41.018161 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:42:41.018141 2576 scope.go:117] "RemoveContainer" containerID="c4d2880dab6684889918584054bc723c9b3b60958d3b3ff9b5c2504de2f1eb17" Jul 21 12:42:41.018362 ip-10-0-130-157 kubenswrapper[2576]: E0721 12:42:41.018345 2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c4d2880dab6684889918584054bc723c9b3b60958d3b3ff9b5c2504de2f1eb17\": container with ID starting with c4d2880dab6684889918584054bc723c9b3b60958d3b3ff9b5c2504de2f1eb17 not found: ID does not exist" containerID="c4d2880dab6684889918584054bc723c9b3b60958d3b3ff9b5c2504de2f1eb17" Jul 21 12:42:41.018412 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:42:41.018363 2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c4d2880dab6684889918584054bc723c9b3b60958d3b3ff9b5c2504de2f1eb17"} err="failed to get container status \"c4d2880dab6684889918584054bc723c9b3b60958d3b3ff9b5c2504de2f1eb17\": rpc error: code = NotFound desc = could not find container \"c4d2880dab6684889918584054bc723c9b3b60958d3b3ff9b5c2504de2f1eb17\": container with ID starting with c4d2880dab6684889918584054bc723c9b3b60958d3b3ff9b5c2504de2f1eb17 not found: ID does not exist" Jul 21 12:42:41.018412 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:42:41.018376 2576 scope.go:117] "RemoveContainer" containerID="fd3d02813a819ea32dca9d4aa71f8e8d5e076a8d1d6f9b8647bbb81c63b87a61" Jul 21 12:42:41.018579 ip-10-0-130-157 kubenswrapper[2576]: E0721 12:42:41.018547 2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"fd3d02813a819ea32dca9d4aa71f8e8d5e076a8d1d6f9b8647bbb81c63b87a61\": container with ID starting with fd3d02813a819ea32dca9d4aa71f8e8d5e076a8d1d6f9b8647bbb81c63b87a61 not found: ID does not exist" containerID="fd3d02813a819ea32dca9d4aa71f8e8d5e076a8d1d6f9b8647bbb81c63b87a61" Jul 21 12:42:41.018639 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:42:41.018582 2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"fd3d02813a819ea32dca9d4aa71f8e8d5e076a8d1d6f9b8647bbb81c63b87a61"} err="failed to get container status \"fd3d02813a819ea32dca9d4aa71f8e8d5e076a8d1d6f9b8647bbb81c63b87a61\": rpc error: code = NotFound desc = could not find container \"fd3d02813a819ea32dca9d4aa71f8e8d5e076a8d1d6f9b8647bbb81c63b87a61\": container with ID starting with fd3d02813a819ea32dca9d4aa71f8e8d5e076a8d1d6f9b8647bbb81c63b87a61 not found: ID does not exist" Jul 21 12:42:41.018639 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:42:41.018594 2576 scope.go:117] "RemoveContainer" containerID="ca5a2ccb5233e6b6f3487ea8a324669b0d704e771d95c1e06f7df0435f456b17" Jul 21 12:42:41.018773 ip-10-0-130-157 kubenswrapper[2576]: E0721 12:42:41.018755 2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ca5a2ccb5233e6b6f3487ea8a324669b0d704e771d95c1e06f7df0435f456b17\": container with ID starting with ca5a2ccb5233e6b6f3487ea8a324669b0d704e771d95c1e06f7df0435f456b17 not found: ID does not exist" containerID="ca5a2ccb5233e6b6f3487ea8a324669b0d704e771d95c1e06f7df0435f456b17" Jul 21 12:42:41.018831 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:42:41.018777 2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ca5a2ccb5233e6b6f3487ea8a324669b0d704e771d95c1e06f7df0435f456b17"} err="failed to get container status \"ca5a2ccb5233e6b6f3487ea8a324669b0d704e771d95c1e06f7df0435f456b17\": rpc error: code = NotFound desc = could not find container \"ca5a2ccb5233e6b6f3487ea8a324669b0d704e771d95c1e06f7df0435f456b17\": container with ID starting with ca5a2ccb5233e6b6f3487ea8a324669b0d704e771d95c1e06f7df0435f456b17 not found: ID does not exist" Jul 21 12:42:41.018831 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:42:41.018793 2576 scope.go:117] "RemoveContainer" containerID="0f37fbf7f8aa8483c00d738e9c25b7609ee7628075256f55f9844e002bea2bea" Jul 21 12:42:41.018965 ip-10-0-130-157 kubenswrapper[2576]: E0721 12:42:41.018950 2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0f37fbf7f8aa8483c00d738e9c25b7609ee7628075256f55f9844e002bea2bea\": container with ID starting with 0f37fbf7f8aa8483c00d738e9c25b7609ee7628075256f55f9844e002bea2bea not found: ID does not exist" containerID="0f37fbf7f8aa8483c00d738e9c25b7609ee7628075256f55f9844e002bea2bea" Jul 21 12:42:41.019019 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:42:41.018971 2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0f37fbf7f8aa8483c00d738e9c25b7609ee7628075256f55f9844e002bea2bea"} err="failed to get container status \"0f37fbf7f8aa8483c00d738e9c25b7609ee7628075256f55f9844e002bea2bea\": rpc error: code = NotFound desc = could not find container \"0f37fbf7f8aa8483c00d738e9c25b7609ee7628075256f55f9844e002bea2bea\": container with ID starting with 0f37fbf7f8aa8483c00d738e9c25b7609ee7628075256f55f9844e002bea2bea not found: ID does not exist" Jul 21 12:42:41.019019 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:42:41.018983 2576 scope.go:117] "RemoveContainer" containerID="20258cf7c3a30e391cd7f0b855a71905a78b3e264a00af57d1c32399b62de2d8" Jul 21 12:42:41.019200 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:42:41.019179 2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"20258cf7c3a30e391cd7f0b855a71905a78b3e264a00af57d1c32399b62de2d8"} err="failed to get container status \"20258cf7c3a30e391cd7f0b855a71905a78b3e264a00af57d1c32399b62de2d8\": rpc error: code = NotFound desc = could not find container \"20258cf7c3a30e391cd7f0b855a71905a78b3e264a00af57d1c32399b62de2d8\": container with ID starting with 20258cf7c3a30e391cd7f0b855a71905a78b3e264a00af57d1c32399b62de2d8 not found: ID does not exist" Jul 21 12:42:41.019267 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:42:41.019203 2576 scope.go:117] "RemoveContainer" containerID="44051f152f926bbb4b472917bdfaa93e91a6d0742b90ac0b94a615300bbebd6c" Jul 21 12:42:41.019406 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:42:41.019391 2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"44051f152f926bbb4b472917bdfaa93e91a6d0742b90ac0b94a615300bbebd6c"} err="failed to get container status \"44051f152f926bbb4b472917bdfaa93e91a6d0742b90ac0b94a615300bbebd6c\": rpc error: code = NotFound desc = could not find container \"44051f152f926bbb4b472917bdfaa93e91a6d0742b90ac0b94a615300bbebd6c\": container with ID starting with 44051f152f926bbb4b472917bdfaa93e91a6d0742b90ac0b94a615300bbebd6c not found: ID does not exist" Jul 21 12:42:41.019451 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:42:41.019405 2576 scope.go:117] "RemoveContainer" containerID="547f8d230bafffc154d05a87317c64a67341a8cfe84074f818634b2ea6103588" Jul 21 12:42:41.019692 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:42:41.019671 2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"547f8d230bafffc154d05a87317c64a67341a8cfe84074f818634b2ea6103588"} err="failed to get container status \"547f8d230bafffc154d05a87317c64a67341a8cfe84074f818634b2ea6103588\": rpc error: code = NotFound desc = could not find container \"547f8d230bafffc154d05a87317c64a67341a8cfe84074f818634b2ea6103588\": container with ID starting with 547f8d230bafffc154d05a87317c64a67341a8cfe84074f818634b2ea6103588 not found: ID does not exist" Jul 21 12:42:41.019775 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:42:41.019693 2576 scope.go:117] "RemoveContainer" containerID="c4d2880dab6684889918584054bc723c9b3b60958d3b3ff9b5c2504de2f1eb17" Jul 21 12:42:41.019918 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:42:41.019903 2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c4d2880dab6684889918584054bc723c9b3b60958d3b3ff9b5c2504de2f1eb17"} err="failed to get container status \"c4d2880dab6684889918584054bc723c9b3b60958d3b3ff9b5c2504de2f1eb17\": rpc error: code = NotFound desc = could not find container \"c4d2880dab6684889918584054bc723c9b3b60958d3b3ff9b5c2504de2f1eb17\": container with ID starting with c4d2880dab6684889918584054bc723c9b3b60958d3b3ff9b5c2504de2f1eb17 not found: ID does not exist" Jul 21 12:42:41.019918 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:42:41.019917 2576 scope.go:117] "RemoveContainer" containerID="fd3d02813a819ea32dca9d4aa71f8e8d5e076a8d1d6f9b8647bbb81c63b87a61" Jul 21 12:42:41.020119 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:42:41.020105 2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"fd3d02813a819ea32dca9d4aa71f8e8d5e076a8d1d6f9b8647bbb81c63b87a61"} err="failed to get container status \"fd3d02813a819ea32dca9d4aa71f8e8d5e076a8d1d6f9b8647bbb81c63b87a61\": rpc error: code = NotFound desc = could not find container \"fd3d02813a819ea32dca9d4aa71f8e8d5e076a8d1d6f9b8647bbb81c63b87a61\": container with ID starting with fd3d02813a819ea32dca9d4aa71f8e8d5e076a8d1d6f9b8647bbb81c63b87a61 not found: ID does not exist" Jul 21 12:42:41.020119 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:42:41.020118 2576 scope.go:117] "RemoveContainer" containerID="ca5a2ccb5233e6b6f3487ea8a324669b0d704e771d95c1e06f7df0435f456b17" Jul 21 12:42:41.020274 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:42:41.020261 2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ca5a2ccb5233e6b6f3487ea8a324669b0d704e771d95c1e06f7df0435f456b17"} err="failed to get container status \"ca5a2ccb5233e6b6f3487ea8a324669b0d704e771d95c1e06f7df0435f456b17\": rpc error: code = NotFound desc = could not find container \"ca5a2ccb5233e6b6f3487ea8a324669b0d704e771d95c1e06f7df0435f456b17\": container with ID starting with ca5a2ccb5233e6b6f3487ea8a324669b0d704e771d95c1e06f7df0435f456b17 not found: ID does not exist" Jul 21 12:42:41.020318 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:42:41.020274 2576 scope.go:117] "RemoveContainer" containerID="0f37fbf7f8aa8483c00d738e9c25b7609ee7628075256f55f9844e002bea2bea" Jul 21 12:42:41.020465 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:42:41.020445 2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0f37fbf7f8aa8483c00d738e9c25b7609ee7628075256f55f9844e002bea2bea"} err="failed to get container status \"0f37fbf7f8aa8483c00d738e9c25b7609ee7628075256f55f9844e002bea2bea\": rpc error: code = NotFound desc = could not find container \"0f37fbf7f8aa8483c00d738e9c25b7609ee7628075256f55f9844e002bea2bea\": container with ID starting with 0f37fbf7f8aa8483c00d738e9c25b7609ee7628075256f55f9844e002bea2bea not found: ID does not exist" Jul 21 12:42:41.020562 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:42:41.020466 2576 scope.go:117] "RemoveContainer" containerID="20258cf7c3a30e391cd7f0b855a71905a78b3e264a00af57d1c32399b62de2d8" Jul 21 12:42:41.020686 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:42:41.020668 2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"20258cf7c3a30e391cd7f0b855a71905a78b3e264a00af57d1c32399b62de2d8"} err="failed to get container status \"20258cf7c3a30e391cd7f0b855a71905a78b3e264a00af57d1c32399b62de2d8\": rpc error: code = NotFound desc = could not find container \"20258cf7c3a30e391cd7f0b855a71905a78b3e264a00af57d1c32399b62de2d8\": container with ID starting with 20258cf7c3a30e391cd7f0b855a71905a78b3e264a00af57d1c32399b62de2d8 not found: ID does not exist" Jul 21 12:42:41.020752 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:42:41.020688 2576 scope.go:117] "RemoveContainer" containerID="44051f152f926bbb4b472917bdfaa93e91a6d0742b90ac0b94a615300bbebd6c" Jul 21 12:42:41.020884 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:42:41.020869 2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"44051f152f926bbb4b472917bdfaa93e91a6d0742b90ac0b94a615300bbebd6c"} err="failed to get container status \"44051f152f926bbb4b472917bdfaa93e91a6d0742b90ac0b94a615300bbebd6c\": rpc error: code = NotFound desc = could not find container \"44051f152f926bbb4b472917bdfaa93e91a6d0742b90ac0b94a615300bbebd6c\": container with ID starting with 44051f152f926bbb4b472917bdfaa93e91a6d0742b90ac0b94a615300bbebd6c not found: ID does not exist" Jul 21 12:42:41.020930 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:42:41.020884 2576 scope.go:117] "RemoveContainer" containerID="547f8d230bafffc154d05a87317c64a67341a8cfe84074f818634b2ea6103588" Jul 21 12:42:41.021090 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:42:41.021071 2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"547f8d230bafffc154d05a87317c64a67341a8cfe84074f818634b2ea6103588"} err="failed to get container status \"547f8d230bafffc154d05a87317c64a67341a8cfe84074f818634b2ea6103588\": rpc error: code = NotFound desc = could not find container \"547f8d230bafffc154d05a87317c64a67341a8cfe84074f818634b2ea6103588\": container with ID starting with 547f8d230bafffc154d05a87317c64a67341a8cfe84074f818634b2ea6103588 not found: ID does not exist" Jul 21 12:42:41.021133 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:42:41.021091 2576 scope.go:117] "RemoveContainer" containerID="c4d2880dab6684889918584054bc723c9b3b60958d3b3ff9b5c2504de2f1eb17" Jul 21 12:42:41.021360 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:42:41.021338 2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c4d2880dab6684889918584054bc723c9b3b60958d3b3ff9b5c2504de2f1eb17"} err="failed to get container status \"c4d2880dab6684889918584054bc723c9b3b60958d3b3ff9b5c2504de2f1eb17\": rpc error: code = NotFound desc = could not find container \"c4d2880dab6684889918584054bc723c9b3b60958d3b3ff9b5c2504de2f1eb17\": container with ID starting with c4d2880dab6684889918584054bc723c9b3b60958d3b3ff9b5c2504de2f1eb17 not found: ID does not exist" Jul 21 12:42:41.021424 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:42:41.021360 2576 scope.go:117] "RemoveContainer" containerID="fd3d02813a819ea32dca9d4aa71f8e8d5e076a8d1d6f9b8647bbb81c63b87a61" Jul 21 12:42:41.021668 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:42:41.021647 2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"fd3d02813a819ea32dca9d4aa71f8e8d5e076a8d1d6f9b8647bbb81c63b87a61"} err="failed to get container status \"fd3d02813a819ea32dca9d4aa71f8e8d5e076a8d1d6f9b8647bbb81c63b87a61\": rpc error: code = NotFound desc = could not find container \"fd3d02813a819ea32dca9d4aa71f8e8d5e076a8d1d6f9b8647bbb81c63b87a61\": container with ID starting with fd3d02813a819ea32dca9d4aa71f8e8d5e076a8d1d6f9b8647bbb81c63b87a61 not found: ID does not exist" Jul 21 12:42:41.021741 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:42:41.021675 2576 scope.go:117] "RemoveContainer" containerID="ca5a2ccb5233e6b6f3487ea8a324669b0d704e771d95c1e06f7df0435f456b17" Jul 21 12:42:41.021924 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:42:41.021907 2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ca5a2ccb5233e6b6f3487ea8a324669b0d704e771d95c1e06f7df0435f456b17"} err="failed to get container status \"ca5a2ccb5233e6b6f3487ea8a324669b0d704e771d95c1e06f7df0435f456b17\": rpc error: code = NotFound desc = could not find container \"ca5a2ccb5233e6b6f3487ea8a324669b0d704e771d95c1e06f7df0435f456b17\": container with ID starting with ca5a2ccb5233e6b6f3487ea8a324669b0d704e771d95c1e06f7df0435f456b17 not found: ID does not exist" Jul 21 12:42:41.021974 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:42:41.021925 2576 scope.go:117] "RemoveContainer" containerID="0f37fbf7f8aa8483c00d738e9c25b7609ee7628075256f55f9844e002bea2bea" Jul 21 12:42:41.022147 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:42:41.022127 2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0f37fbf7f8aa8483c00d738e9c25b7609ee7628075256f55f9844e002bea2bea"} err="failed to get container status \"0f37fbf7f8aa8483c00d738e9c25b7609ee7628075256f55f9844e002bea2bea\": rpc error: code = NotFound desc = could not find container \"0f37fbf7f8aa8483c00d738e9c25b7609ee7628075256f55f9844e002bea2bea\": container with ID starting with 0f37fbf7f8aa8483c00d738e9c25b7609ee7628075256f55f9844e002bea2bea not found: ID does not exist" Jul 21 12:42:41.022195 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:42:41.022147 2576 scope.go:117] "RemoveContainer" containerID="20258cf7c3a30e391cd7f0b855a71905a78b3e264a00af57d1c32399b62de2d8" Jul 21 12:42:41.022379 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:42:41.022355 2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"20258cf7c3a30e391cd7f0b855a71905a78b3e264a00af57d1c32399b62de2d8"} err="failed to get container status \"20258cf7c3a30e391cd7f0b855a71905a78b3e264a00af57d1c32399b62de2d8\": rpc error: code = NotFound desc = could not find container \"20258cf7c3a30e391cd7f0b855a71905a78b3e264a00af57d1c32399b62de2d8\": container with ID starting with 20258cf7c3a30e391cd7f0b855a71905a78b3e264a00af57d1c32399b62de2d8 not found: ID does not exist" Jul 21 12:42:41.022457 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:42:41.022382 2576 scope.go:117] "RemoveContainer" containerID="44051f152f926bbb4b472917bdfaa93e91a6d0742b90ac0b94a615300bbebd6c" Jul 21 12:42:41.022668 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:42:41.022649 2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"44051f152f926bbb4b472917bdfaa93e91a6d0742b90ac0b94a615300bbebd6c"} err="failed to get container status \"44051f152f926bbb4b472917bdfaa93e91a6d0742b90ac0b94a615300bbebd6c\": rpc error: code = NotFound desc = could not find container \"44051f152f926bbb4b472917bdfaa93e91a6d0742b90ac0b94a615300bbebd6c\": container with ID starting with 44051f152f926bbb4b472917bdfaa93e91a6d0742b90ac0b94a615300bbebd6c not found: ID does not exist" Jul 21 12:42:41.022728 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:42:41.022668 2576 scope.go:117] "RemoveContainer" containerID="547f8d230bafffc154d05a87317c64a67341a8cfe84074f818634b2ea6103588" Jul 21 12:42:41.022894 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:42:41.022876 2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"547f8d230bafffc154d05a87317c64a67341a8cfe84074f818634b2ea6103588"} err="failed to get container status \"547f8d230bafffc154d05a87317c64a67341a8cfe84074f818634b2ea6103588\": rpc error: code = NotFound desc = could not find container \"547f8d230bafffc154d05a87317c64a67341a8cfe84074f818634b2ea6103588\": container with ID starting with 547f8d230bafffc154d05a87317c64a67341a8cfe84074f818634b2ea6103588 not found: ID does not exist" Jul 21 12:42:41.022953 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:42:41.022895 2576 scope.go:117] "RemoveContainer" containerID="c4d2880dab6684889918584054bc723c9b3b60958d3b3ff9b5c2504de2f1eb17" Jul 21 12:42:41.023108 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:42:41.023091 2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c4d2880dab6684889918584054bc723c9b3b60958d3b3ff9b5c2504de2f1eb17"} err="failed to get container status \"c4d2880dab6684889918584054bc723c9b3b60958d3b3ff9b5c2504de2f1eb17\": rpc error: code = NotFound desc = could not find container \"c4d2880dab6684889918584054bc723c9b3b60958d3b3ff9b5c2504de2f1eb17\": container with ID starting with c4d2880dab6684889918584054bc723c9b3b60958d3b3ff9b5c2504de2f1eb17 not found: ID does not exist" Jul 21 12:42:41.023179 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:42:41.023109 2576 scope.go:117] "RemoveContainer" containerID="fd3d02813a819ea32dca9d4aa71f8e8d5e076a8d1d6f9b8647bbb81c63b87a61" Jul 21 12:42:41.023325 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:42:41.023304 2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"fd3d02813a819ea32dca9d4aa71f8e8d5e076a8d1d6f9b8647bbb81c63b87a61"} err="failed to get container status \"fd3d02813a819ea32dca9d4aa71f8e8d5e076a8d1d6f9b8647bbb81c63b87a61\": rpc error: code = NotFound desc = could not find container \"fd3d02813a819ea32dca9d4aa71f8e8d5e076a8d1d6f9b8647bbb81c63b87a61\": container with ID starting with fd3d02813a819ea32dca9d4aa71f8e8d5e076a8d1d6f9b8647bbb81c63b87a61 not found: ID does not exist" Jul 21 12:42:41.023363 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:42:41.023327 2576 scope.go:117] "RemoveContainer" containerID="ca5a2ccb5233e6b6f3487ea8a324669b0d704e771d95c1e06f7df0435f456b17" Jul 21 12:42:41.023581 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:42:41.023561 2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ca5a2ccb5233e6b6f3487ea8a324669b0d704e771d95c1e06f7df0435f456b17"} err="failed to get container status \"ca5a2ccb5233e6b6f3487ea8a324669b0d704e771d95c1e06f7df0435f456b17\": rpc error: code = NotFound desc = could not find container \"ca5a2ccb5233e6b6f3487ea8a324669b0d704e771d95c1e06f7df0435f456b17\": container with ID starting with ca5a2ccb5233e6b6f3487ea8a324669b0d704e771d95c1e06f7df0435f456b17 not found: ID does not exist" Jul 21 12:42:41.023658 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:42:41.023582 2576 scope.go:117] "RemoveContainer" containerID="0f37fbf7f8aa8483c00d738e9c25b7609ee7628075256f55f9844e002bea2bea" Jul 21 12:42:41.023821 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:42:41.023805 2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0f37fbf7f8aa8483c00d738e9c25b7609ee7628075256f55f9844e002bea2bea"} err="failed to get container status \"0f37fbf7f8aa8483c00d738e9c25b7609ee7628075256f55f9844e002bea2bea\": rpc error: code = NotFound desc = could not find container \"0f37fbf7f8aa8483c00d738e9c25b7609ee7628075256f55f9844e002bea2bea\": container with ID starting with 0f37fbf7f8aa8483c00d738e9c25b7609ee7628075256f55f9844e002bea2bea not found: ID does not exist" Jul 21 12:42:41.023874 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:42:41.023820 2576 scope.go:117] "RemoveContainer" containerID="20258cf7c3a30e391cd7f0b855a71905a78b3e264a00af57d1c32399b62de2d8" Jul 21 12:42:41.024049 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:42:41.024033 2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"20258cf7c3a30e391cd7f0b855a71905a78b3e264a00af57d1c32399b62de2d8"} err="failed to get container status \"20258cf7c3a30e391cd7f0b855a71905a78b3e264a00af57d1c32399b62de2d8\": rpc error: code = NotFound desc = could not find container \"20258cf7c3a30e391cd7f0b855a71905a78b3e264a00af57d1c32399b62de2d8\": container with ID starting with 20258cf7c3a30e391cd7f0b855a71905a78b3e264a00af57d1c32399b62de2d8 not found: ID does not exist" Jul 21 12:42:41.024104 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:42:41.024049 2576 scope.go:117] "RemoveContainer" containerID="44051f152f926bbb4b472917bdfaa93e91a6d0742b90ac0b94a615300bbebd6c" Jul 21 12:42:41.024242 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:42:41.024226 2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"44051f152f926bbb4b472917bdfaa93e91a6d0742b90ac0b94a615300bbebd6c"} err="failed to get container status \"44051f152f926bbb4b472917bdfaa93e91a6d0742b90ac0b94a615300bbebd6c\": rpc error: code = NotFound desc = could not find container \"44051f152f926bbb4b472917bdfaa93e91a6d0742b90ac0b94a615300bbebd6c\": container with ID starting with 44051f152f926bbb4b472917bdfaa93e91a6d0742b90ac0b94a615300bbebd6c not found: ID does not exist" Jul 21 12:42:41.024285 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:42:41.024242 2576 scope.go:117] "RemoveContainer" containerID="547f8d230bafffc154d05a87317c64a67341a8cfe84074f818634b2ea6103588" Jul 21 12:42:41.024413 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:42:41.024390 2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"547f8d230bafffc154d05a87317c64a67341a8cfe84074f818634b2ea6103588"} err="failed to get container status \"547f8d230bafffc154d05a87317c64a67341a8cfe84074f818634b2ea6103588\": rpc error: code = NotFound desc = could not find container \"547f8d230bafffc154d05a87317c64a67341a8cfe84074f818634b2ea6103588\": container with ID starting with 547f8d230bafffc154d05a87317c64a67341a8cfe84074f818634b2ea6103588 not found: ID does not exist" Jul 21 12:42:41.024481 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:42:41.024413 2576 scope.go:117] "RemoveContainer" containerID="c4d2880dab6684889918584054bc723c9b3b60958d3b3ff9b5c2504de2f1eb17" Jul 21 12:42:41.024636 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:42:41.024615 2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c4d2880dab6684889918584054bc723c9b3b60958d3b3ff9b5c2504de2f1eb17"} err="failed to get container status \"c4d2880dab6684889918584054bc723c9b3b60958d3b3ff9b5c2504de2f1eb17\": rpc error: code = NotFound desc = could not find container \"c4d2880dab6684889918584054bc723c9b3b60958d3b3ff9b5c2504de2f1eb17\": container with ID starting with c4d2880dab6684889918584054bc723c9b3b60958d3b3ff9b5c2504de2f1eb17 not found: ID does not exist" Jul 21 12:42:41.024715 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:42:41.024638 2576 scope.go:117] "RemoveContainer" containerID="fd3d02813a819ea32dca9d4aa71f8e8d5e076a8d1d6f9b8647bbb81c63b87a61" Jul 21 12:42:41.024878 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:42:41.024861 2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"fd3d02813a819ea32dca9d4aa71f8e8d5e076a8d1d6f9b8647bbb81c63b87a61"} err="failed to get container status \"fd3d02813a819ea32dca9d4aa71f8e8d5e076a8d1d6f9b8647bbb81c63b87a61\": rpc error: code = NotFound desc = could not find container \"fd3d02813a819ea32dca9d4aa71f8e8d5e076a8d1d6f9b8647bbb81c63b87a61\": container with ID starting with fd3d02813a819ea32dca9d4aa71f8e8d5e076a8d1d6f9b8647bbb81c63b87a61 not found: ID does not exist" Jul 21 12:42:41.024924 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:42:41.024879 2576 scope.go:117] "RemoveContainer" containerID="ca5a2ccb5233e6b6f3487ea8a324669b0d704e771d95c1e06f7df0435f456b17" Jul 21 12:42:41.025069 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:42:41.025052 2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ca5a2ccb5233e6b6f3487ea8a324669b0d704e771d95c1e06f7df0435f456b17"} err="failed to get container status \"ca5a2ccb5233e6b6f3487ea8a324669b0d704e771d95c1e06f7df0435f456b17\": rpc error: code = NotFound desc = could not find container \"ca5a2ccb5233e6b6f3487ea8a324669b0d704e771d95c1e06f7df0435f456b17\": container with ID starting with ca5a2ccb5233e6b6f3487ea8a324669b0d704e771d95c1e06f7df0435f456b17 not found: ID does not exist" Jul 21 12:42:41.025108 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:42:41.025070 2576 scope.go:117] "RemoveContainer" containerID="0f37fbf7f8aa8483c00d738e9c25b7609ee7628075256f55f9844e002bea2bea" Jul 21 12:42:41.025283 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:42:41.025266 2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0f37fbf7f8aa8483c00d738e9c25b7609ee7628075256f55f9844e002bea2bea"} err="failed to get container status \"0f37fbf7f8aa8483c00d738e9c25b7609ee7628075256f55f9844e002bea2bea\": rpc error: code = NotFound desc = could not find container \"0f37fbf7f8aa8483c00d738e9c25b7609ee7628075256f55f9844e002bea2bea\": container with ID starting with 0f37fbf7f8aa8483c00d738e9c25b7609ee7628075256f55f9844e002bea2bea not found: ID does not exist" Jul 21 12:42:41.025327 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:42:41.025283 2576 scope.go:117] "RemoveContainer" containerID="20258cf7c3a30e391cd7f0b855a71905a78b3e264a00af57d1c32399b62de2d8" Jul 21 12:42:41.025488 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:42:41.025466 2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"20258cf7c3a30e391cd7f0b855a71905a78b3e264a00af57d1c32399b62de2d8"} err="failed to get container status \"20258cf7c3a30e391cd7f0b855a71905a78b3e264a00af57d1c32399b62de2d8\": rpc error: code = NotFound desc = could not find container \"20258cf7c3a30e391cd7f0b855a71905a78b3e264a00af57d1c32399b62de2d8\": container with ID starting with 20258cf7c3a30e391cd7f0b855a71905a78b3e264a00af57d1c32399b62de2d8 not found: ID does not exist" Jul 21 12:42:41.025569 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:42:41.025489 2576 scope.go:117] "RemoveContainer" containerID="44051f152f926bbb4b472917bdfaa93e91a6d0742b90ac0b94a615300bbebd6c" Jul 21 12:42:41.025766 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:42:41.025748 2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"44051f152f926bbb4b472917bdfaa93e91a6d0742b90ac0b94a615300bbebd6c"} err="failed to get container status \"44051f152f926bbb4b472917bdfaa93e91a6d0742b90ac0b94a615300bbebd6c\": rpc error: code = NotFound desc = could not find container \"44051f152f926bbb4b472917bdfaa93e91a6d0742b90ac0b94a615300bbebd6c\": container with ID starting with 44051f152f926bbb4b472917bdfaa93e91a6d0742b90ac0b94a615300bbebd6c not found: ID does not exist" Jul 21 12:42:41.025818 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:42:41.025766 2576 scope.go:117] "RemoveContainer" containerID="547f8d230bafffc154d05a87317c64a67341a8cfe84074f818634b2ea6103588" Jul 21 12:42:41.025972 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:42:41.025951 2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"547f8d230bafffc154d05a87317c64a67341a8cfe84074f818634b2ea6103588"} err="failed to get container status \"547f8d230bafffc154d05a87317c64a67341a8cfe84074f818634b2ea6103588\": rpc error: code = NotFound desc = could not find container \"547f8d230bafffc154d05a87317c64a67341a8cfe84074f818634b2ea6103588\": container with ID starting with 547f8d230bafffc154d05a87317c64a67341a8cfe84074f818634b2ea6103588 not found: ID does not exist" Jul 21 12:42:41.026013 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:42:41.025971 2576 scope.go:117] "RemoveContainer" containerID="c4d2880dab6684889918584054bc723c9b3b60958d3b3ff9b5c2504de2f1eb17" Jul 21 12:42:41.026188 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:42:41.026173 2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c4d2880dab6684889918584054bc723c9b3b60958d3b3ff9b5c2504de2f1eb17"} err="failed to get container status \"c4d2880dab6684889918584054bc723c9b3b60958d3b3ff9b5c2504de2f1eb17\": rpc error: code = NotFound desc = could not find container \"c4d2880dab6684889918584054bc723c9b3b60958d3b3ff9b5c2504de2f1eb17\": container with ID starting with c4d2880dab6684889918584054bc723c9b3b60958d3b3ff9b5c2504de2f1eb17 not found: ID does not exist" Jul 21 12:42:41.026188 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:42:41.026187 2576 scope.go:117] "RemoveContainer" containerID="fd3d02813a819ea32dca9d4aa71f8e8d5e076a8d1d6f9b8647bbb81c63b87a61" Jul 21 12:42:41.026382 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:42:41.026366 2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"fd3d02813a819ea32dca9d4aa71f8e8d5e076a8d1d6f9b8647bbb81c63b87a61"} err="failed to get container status \"fd3d02813a819ea32dca9d4aa71f8e8d5e076a8d1d6f9b8647bbb81c63b87a61\": rpc error: code = NotFound desc = could not find container \"fd3d02813a819ea32dca9d4aa71f8e8d5e076a8d1d6f9b8647bbb81c63b87a61\": container with ID starting with fd3d02813a819ea32dca9d4aa71f8e8d5e076a8d1d6f9b8647bbb81c63b87a61 not found: ID does not exist" Jul 21 12:42:41.026382 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:42:41.026381 2576 scope.go:117] "RemoveContainer" containerID="ca5a2ccb5233e6b6f3487ea8a324669b0d704e771d95c1e06f7df0435f456b17" Jul 21 12:42:41.026592 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:42:41.026574 2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ca5a2ccb5233e6b6f3487ea8a324669b0d704e771d95c1e06f7df0435f456b17"} err="failed to get container status \"ca5a2ccb5233e6b6f3487ea8a324669b0d704e771d95c1e06f7df0435f456b17\": rpc error: code = NotFound desc = could not find container \"ca5a2ccb5233e6b6f3487ea8a324669b0d704e771d95c1e06f7df0435f456b17\": container with ID starting with ca5a2ccb5233e6b6f3487ea8a324669b0d704e771d95c1e06f7df0435f456b17 not found: ID does not exist" Jul 21 12:42:41.026656 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:42:41.026592 2576 scope.go:117] "RemoveContainer" containerID="0f37fbf7f8aa8483c00d738e9c25b7609ee7628075256f55f9844e002bea2bea" Jul 21 12:42:41.026826 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:42:41.026809 2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0f37fbf7f8aa8483c00d738e9c25b7609ee7628075256f55f9844e002bea2bea"} err="failed to get container status \"0f37fbf7f8aa8483c00d738e9c25b7609ee7628075256f55f9844e002bea2bea\": rpc error: code = NotFound desc = could not find container \"0f37fbf7f8aa8483c00d738e9c25b7609ee7628075256f55f9844e002bea2bea\": container with ID starting with 0f37fbf7f8aa8483c00d738e9c25b7609ee7628075256f55f9844e002bea2bea not found: ID does not exist" Jul 21 12:42:41.030919 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:42:41.030898 2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jul 21 12:42:41.031151 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:42:41.031136 2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a2f209e9-2530-44a8-b31f-942b15f141a6" containerName="kube-rbac-proxy-web" Jul 21 12:42:41.031222 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:42:41.031154 2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="a2f209e9-2530-44a8-b31f-942b15f141a6" containerName="kube-rbac-proxy-web" Jul 21 12:42:41.031222 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:42:41.031171 2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a2f209e9-2530-44a8-b31f-942b15f141a6" containerName="init-config-reloader" Jul 21 12:42:41.031222 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:42:41.031179 2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="a2f209e9-2530-44a8-b31f-942b15f141a6" containerName="init-config-reloader" Jul 21 12:42:41.031222 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:42:41.031188 2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a2f209e9-2530-44a8-b31f-942b15f141a6" containerName="prometheus" Jul 21 12:42:41.031222 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:42:41.031197 2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="a2f209e9-2530-44a8-b31f-942b15f141a6" containerName="prometheus" Jul 21 12:42:41.031222 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:42:41.031210 2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a2f209e9-2530-44a8-b31f-942b15f141a6" containerName="kube-rbac-proxy" Jul 21 12:42:41.031222 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:42:41.031219 2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="a2f209e9-2530-44a8-b31f-942b15f141a6" containerName="kube-rbac-proxy" Jul 21 12:42:41.031573 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:42:41.031232 2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a2f209e9-2530-44a8-b31f-942b15f141a6" containerName="config-reloader" Jul 21 12:42:41.031573 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:42:41.031240 2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="a2f209e9-2530-44a8-b31f-942b15f141a6" containerName="config-reloader" Jul 21 12:42:41.031573 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:42:41.031253 2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a2f209e9-2530-44a8-b31f-942b15f141a6" containerName="kube-rbac-proxy-thanos" Jul 21 12:42:41.031573 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:42:41.031263 2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="a2f209e9-2530-44a8-b31f-942b15f141a6" containerName="kube-rbac-proxy-thanos" Jul 21 12:42:41.031573 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:42:41.031276 2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a2f209e9-2530-44a8-b31f-942b15f141a6" containerName="thanos-sidecar" Jul 21 12:42:41.031573 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:42:41.031283 2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="a2f209e9-2530-44a8-b31f-942b15f141a6" containerName="thanos-sidecar" Jul 21 12:42:41.031573 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:42:41.031348 2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="a2f209e9-2530-44a8-b31f-942b15f141a6" containerName="prometheus" Jul 21 12:42:41.031573 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:42:41.031359 2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="a2f209e9-2530-44a8-b31f-942b15f141a6" containerName="kube-rbac-proxy" Jul 21 12:42:41.031573 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:42:41.031369 2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="a2f209e9-2530-44a8-b31f-942b15f141a6" containerName="thanos-sidecar" Jul 21 12:42:41.031573 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:42:41.031377 2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="a2f209e9-2530-44a8-b31f-942b15f141a6" containerName="kube-rbac-proxy-web" Jul 21 12:42:41.031573 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:42:41.031387 2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="a2f209e9-2530-44a8-b31f-942b15f141a6" containerName="config-reloader" Jul 21 12:42:41.031573 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:42:41.031395 2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="a2f209e9-2530-44a8-b31f-942b15f141a6" containerName="kube-rbac-proxy-thanos" Jul 21 12:42:41.036063 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:42:41.036045 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" Jul 21 12:42:41.038080 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:42:41.038057 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-dockercfg-w6429\"" Jul 21 12:42:41.038080 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:42:41.038060 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls\"" Jul 21 12:42:41.038264 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:42:41.038173 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s\"" Jul 21 12:42:41.038264 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:42:41.038184 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls-assets-0\"" Jul 21 12:42:41.038264 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:42:41.038222 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-prometheus-http-client-file\"" Jul 21 12:42:41.038264 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:42:41.038224 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-grpc-tls-dhjbeffbr1t5\"" Jul 21 12:42:41.038454 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:42:41.038238 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-rbac-proxy\"" Jul 21 12:42:41.038454 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:42:41.038439 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-sidecar-tls\"" Jul 21 12:42:41.038845 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:42:41.038669 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-web-config\"" Jul 21 12:42:41.038845 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:42:41.038680 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-1\"" Jul 21 12:42:41.038845 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:42:41.038737 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-2\"" Jul 21 12:42:41.038845 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:42:41.038669 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-kube-rbac-proxy-web\"" Jul 21 12:42:41.039334 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:42:41.039314 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-client-certs\"" Jul 21 12:42:41.040378 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:42:41.040359 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"serving-certs-ca-bundle\"" Jul 21 12:42:41.042872 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:42:41.042856 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-0\"" Jul 21 12:42:41.048404 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:42:41.048388 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-trusted-ca-bundle\"" Jul 21 12:42:41.052738 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:42:41.052715 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jul 21 12:42:41.139958 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:42:41.139933 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/4911b6ac-242a-4014-933d-6821ff61e77a-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"4911b6ac-242a-4014-933d-6821ff61e77a\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 21 12:42:41.139958 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:42:41.139960 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/4911b6ac-242a-4014-933d-6821ff61e77a-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"4911b6ac-242a-4014-933d-6821ff61e77a\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 21 12:42:41.140141 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:42:41.139975 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/4911b6ac-242a-4014-933d-6821ff61e77a-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"4911b6ac-242a-4014-933d-6821ff61e77a\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 21 12:42:41.140141 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:42:41.139992 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/4911b6ac-242a-4014-933d-6821ff61e77a-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"4911b6ac-242a-4014-933d-6821ff61e77a\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 21 12:42:41.140141 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:42:41.140007 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/4911b6ac-242a-4014-933d-6821ff61e77a-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"4911b6ac-242a-4014-933d-6821ff61e77a\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 21 12:42:41.140141 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:42:41.140024 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lsxdz\" (UniqueName: \"kubernetes.io/projected/4911b6ac-242a-4014-933d-6821ff61e77a-kube-api-access-lsxdz\") pod \"prometheus-k8s-0\" (UID: \"4911b6ac-242a-4014-933d-6821ff61e77a\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 21 12:42:41.140141 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:42:41.140113 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/4911b6ac-242a-4014-933d-6821ff61e77a-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"4911b6ac-242a-4014-933d-6821ff61e77a\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 21 12:42:41.140370 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:42:41.140153 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/4911b6ac-242a-4014-933d-6821ff61e77a-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"4911b6ac-242a-4014-933d-6821ff61e77a\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 21 12:42:41.140370 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:42:41.140182 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/4911b6ac-242a-4014-933d-6821ff61e77a-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"4911b6ac-242a-4014-933d-6821ff61e77a\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 21 12:42:41.140370 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:42:41.140207 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/4911b6ac-242a-4014-933d-6821ff61e77a-config-out\") pod \"prometheus-k8s-0\" (UID: \"4911b6ac-242a-4014-933d-6821ff61e77a\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 21 12:42:41.140370 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:42:41.140249 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/4911b6ac-242a-4014-933d-6821ff61e77a-web-config\") pod \"prometheus-k8s-0\" (UID: \"4911b6ac-242a-4014-933d-6821ff61e77a\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 21 12:42:41.140370 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:42:41.140279 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/4911b6ac-242a-4014-933d-6821ff61e77a-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"4911b6ac-242a-4014-933d-6821ff61e77a\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 21 12:42:41.140370 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:42:41.140308 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/4911b6ac-242a-4014-933d-6821ff61e77a-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"4911b6ac-242a-4014-933d-6821ff61e77a\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 21 12:42:41.140370 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:42:41.140341 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/4911b6ac-242a-4014-933d-6821ff61e77a-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"4911b6ac-242a-4014-933d-6821ff61e77a\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 21 12:42:41.140684 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:42:41.140372 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/4911b6ac-242a-4014-933d-6821ff61e77a-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"4911b6ac-242a-4014-933d-6821ff61e77a\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 21 12:42:41.140684 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:42:41.140398 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/4911b6ac-242a-4014-933d-6821ff61e77a-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"4911b6ac-242a-4014-933d-6821ff61e77a\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 21 12:42:41.140684 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:42:41.140414 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/4911b6ac-242a-4014-933d-6821ff61e77a-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"4911b6ac-242a-4014-933d-6821ff61e77a\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 21 12:42:41.140684 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:42:41.140436 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/4911b6ac-242a-4014-933d-6821ff61e77a-config\") pod \"prometheus-k8s-0\" (UID: \"4911b6ac-242a-4014-933d-6821ff61e77a\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 21 12:42:41.140684 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:42:41.140488 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/4911b6ac-242a-4014-933d-6821ff61e77a-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"4911b6ac-242a-4014-933d-6821ff61e77a\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 21 12:42:41.140684 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:42:41.140571 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/4911b6ac-242a-4014-933d-6821ff61e77a-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"4911b6ac-242a-4014-933d-6821ff61e77a\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 21 12:42:41.241831 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:42:41.241760 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/4911b6ac-242a-4014-933d-6821ff61e77a-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"4911b6ac-242a-4014-933d-6821ff61e77a\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 21 12:42:41.241831 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:42:41.241803 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/4911b6ac-242a-4014-933d-6821ff61e77a-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"4911b6ac-242a-4014-933d-6821ff61e77a\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 21 12:42:41.241831 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:42:41.241826 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/4911b6ac-242a-4014-933d-6821ff61e77a-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"4911b6ac-242a-4014-933d-6821ff61e77a\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 21 12:42:41.242293 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:42:41.241843 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/4911b6ac-242a-4014-933d-6821ff61e77a-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"4911b6ac-242a-4014-933d-6821ff61e77a\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 21 12:42:41.242293 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:42:41.241860 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/4911b6ac-242a-4014-933d-6821ff61e77a-config\") pod \"prometheus-k8s-0\" (UID: \"4911b6ac-242a-4014-933d-6821ff61e77a\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 21 12:42:41.242293 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:42:41.241877 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/4911b6ac-242a-4014-933d-6821ff61e77a-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"4911b6ac-242a-4014-933d-6821ff61e77a\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 21 12:42:41.242293 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:42:41.241900 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/4911b6ac-242a-4014-933d-6821ff61e77a-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"4911b6ac-242a-4014-933d-6821ff61e77a\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 21 12:42:41.242293 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:42:41.241925 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/4911b6ac-242a-4014-933d-6821ff61e77a-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"4911b6ac-242a-4014-933d-6821ff61e77a\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 21 12:42:41.242293 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:42:41.241950 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/4911b6ac-242a-4014-933d-6821ff61e77a-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"4911b6ac-242a-4014-933d-6821ff61e77a\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 21 12:42:41.242293 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:42:41.241972 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/4911b6ac-242a-4014-933d-6821ff61e77a-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"4911b6ac-242a-4014-933d-6821ff61e77a\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 21 12:42:41.242293 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:42:41.241996 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/4911b6ac-242a-4014-933d-6821ff61e77a-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"4911b6ac-242a-4014-933d-6821ff61e77a\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 21 12:42:41.242293 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:42:41.242015 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/4911b6ac-242a-4014-933d-6821ff61e77a-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"4911b6ac-242a-4014-933d-6821ff61e77a\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 21 12:42:41.242293 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:42:41.242040 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lsxdz\" (UniqueName: \"kubernetes.io/projected/4911b6ac-242a-4014-933d-6821ff61e77a-kube-api-access-lsxdz\") pod \"prometheus-k8s-0\" (UID: \"4911b6ac-242a-4014-933d-6821ff61e77a\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 21 12:42:41.242293 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:42:41.242083 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/4911b6ac-242a-4014-933d-6821ff61e77a-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"4911b6ac-242a-4014-933d-6821ff61e77a\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 21 12:42:41.242293 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:42:41.242114 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/4911b6ac-242a-4014-933d-6821ff61e77a-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"4911b6ac-242a-4014-933d-6821ff61e77a\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 21 12:42:41.242293 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:42:41.242137 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/4911b6ac-242a-4014-933d-6821ff61e77a-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"4911b6ac-242a-4014-933d-6821ff61e77a\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 21 12:42:41.242293 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:42:41.242162 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/4911b6ac-242a-4014-933d-6821ff61e77a-config-out\") pod \"prometheus-k8s-0\" (UID: \"4911b6ac-242a-4014-933d-6821ff61e77a\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 21 12:42:41.242293 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:42:41.242206 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/4911b6ac-242a-4014-933d-6821ff61e77a-web-config\") pod \"prometheus-k8s-0\" (UID: \"4911b6ac-242a-4014-933d-6821ff61e77a\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 21 12:42:41.242293 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:42:41.242232 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/4911b6ac-242a-4014-933d-6821ff61e77a-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"4911b6ac-242a-4014-933d-6821ff61e77a\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 21 12:42:41.242293 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:42:41.242261 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/4911b6ac-242a-4014-933d-6821ff61e77a-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"4911b6ac-242a-4014-933d-6821ff61e77a\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 21 12:42:41.243143 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:42:41.242708 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/4911b6ac-242a-4014-933d-6821ff61e77a-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"4911b6ac-242a-4014-933d-6821ff61e77a\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 21 12:42:41.243143 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:42:41.242716 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/4911b6ac-242a-4014-933d-6821ff61e77a-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"4911b6ac-242a-4014-933d-6821ff61e77a\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 21 12:42:41.243677 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:42:41.243340 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/4911b6ac-242a-4014-933d-6821ff61e77a-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"4911b6ac-242a-4014-933d-6821ff61e77a\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 21 12:42:41.245102 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:42:41.245071 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/4911b6ac-242a-4014-933d-6821ff61e77a-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"4911b6ac-242a-4014-933d-6821ff61e77a\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 21 12:42:41.245203 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:42:41.245115 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/4911b6ac-242a-4014-933d-6821ff61e77a-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"4911b6ac-242a-4014-933d-6821ff61e77a\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 21 12:42:41.245203 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:42:41.245119 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/4911b6ac-242a-4014-933d-6821ff61e77a-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"4911b6ac-242a-4014-933d-6821ff61e77a\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 21 12:42:41.245693 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:42:41.245665 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/4911b6ac-242a-4014-933d-6821ff61e77a-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"4911b6ac-242a-4014-933d-6821ff61e77a\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 21 12:42:41.245693 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:42:41.245680 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/secret/4911b6ac-242a-4014-933d-6821ff61e77a-config\") pod \"prometheus-k8s-0\" (UID: \"4911b6ac-242a-4014-933d-6821ff61e77a\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 21 12:42:41.245970 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:42:41.245947 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/4911b6ac-242a-4014-933d-6821ff61e77a-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"4911b6ac-242a-4014-933d-6821ff61e77a\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 21 12:42:41.246268 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:42:41.246241 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/4911b6ac-242a-4014-933d-6821ff61e77a-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"4911b6ac-242a-4014-933d-6821ff61e77a\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 21 12:42:41.246351 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:42:41.246306 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/4911b6ac-242a-4014-933d-6821ff61e77a-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"4911b6ac-242a-4014-933d-6821ff61e77a\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 21 12:42:41.246494 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:42:41.246471 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/4911b6ac-242a-4014-933d-6821ff61e77a-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"4911b6ac-242a-4014-933d-6821ff61e77a\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 21 12:42:41.246910 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:42:41.246859 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/4911b6ac-242a-4014-933d-6821ff61e77a-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"4911b6ac-242a-4014-933d-6821ff61e77a\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 21 12:42:41.247009 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:42:41.246964 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/4911b6ac-242a-4014-933d-6821ff61e77a-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"4911b6ac-242a-4014-933d-6821ff61e77a\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 21 12:42:41.247009 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:42:41.246971 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/4911b6ac-242a-4014-933d-6821ff61e77a-config-out\") pod \"prometheus-k8s-0\" (UID: \"4911b6ac-242a-4014-933d-6821ff61e77a\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 21 12:42:41.247481 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:42:41.247453 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/4911b6ac-242a-4014-933d-6821ff61e77a-web-config\") pod \"prometheus-k8s-0\" (UID: \"4911b6ac-242a-4014-933d-6821ff61e77a\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 21 12:42:41.247825 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:42:41.247802 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/4911b6ac-242a-4014-933d-6821ff61e77a-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"4911b6ac-242a-4014-933d-6821ff61e77a\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 21 12:42:41.248141 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:42:41.248117 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/4911b6ac-242a-4014-933d-6821ff61e77a-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"4911b6ac-242a-4014-933d-6821ff61e77a\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 21 12:42:41.248360 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:42:41.248343 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/4911b6ac-242a-4014-933d-6821ff61e77a-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"4911b6ac-242a-4014-933d-6821ff61e77a\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 21 12:42:41.255981 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:42:41.255958 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lsxdz\" (UniqueName: \"kubernetes.io/projected/4911b6ac-242a-4014-933d-6821ff61e77a-kube-api-access-lsxdz\") pod \"prometheus-k8s-0\" (UID: \"4911b6ac-242a-4014-933d-6821ff61e77a\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 21 12:42:41.346601 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:42:41.346578 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" Jul 21 12:42:41.483931 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:42:41.483901 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jul 21 12:42:41.486349 ip-10-0-130-157 kubenswrapper[2576]: W0721 12:42:41.486324 2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4911b6ac_242a_4014_933d_6821ff61e77a.slice/crio-74294f99b16b20a5489fdbfeb222a37fbf3ab4f925be534bcbe072561da07508 WatchSource:0}: Error finding container 74294f99b16b20a5489fdbfeb222a37fbf3ab4f925be534bcbe072561da07508: Status 404 returned error can't find the container with id 74294f99b16b20a5489fdbfeb222a37fbf3ab4f925be534bcbe072561da07508 Jul 21 12:42:41.974320 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:42:41.974287 2576 generic.go:356] "Generic (PLEG): container finished" podID="4911b6ac-242a-4014-933d-6821ff61e77a" containerID="108cd2d5e1bca431e701279eca7543079c6406210c8aaef925bad63a2139cc8c" exitCode=0 Jul 21 12:42:41.974468 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:42:41.974340 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"4911b6ac-242a-4014-933d-6821ff61e77a","Type":"ContainerDied","Data":"108cd2d5e1bca431e701279eca7543079c6406210c8aaef925bad63a2139cc8c"} Jul 21 12:42:41.974468 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:42:41.974401 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"4911b6ac-242a-4014-933d-6821ff61e77a","Type":"ContainerStarted","Data":"74294f99b16b20a5489fdbfeb222a37fbf3ab4f925be534bcbe072561da07508"} Jul 21 12:42:42.318871 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:42:42.318432 2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a2f209e9-2530-44a8-b31f-942b15f141a6" path="/var/lib/kubelet/pods/a2f209e9-2530-44a8-b31f-942b15f141a6/volumes" Jul 21 12:42:42.981131 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:42:42.981095 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"4911b6ac-242a-4014-933d-6821ff61e77a","Type":"ContainerStarted","Data":"fdd88fe795e251115a91898695d2ece286687a1d447ec52acb2f36c44cd3bd8f"} Jul 21 12:42:42.981272 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:42:42.981159 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"4911b6ac-242a-4014-933d-6821ff61e77a","Type":"ContainerStarted","Data":"ef1f82f4f63b6d89d750fc5387c0d1e5319fa788fa27a293e8fe2ffd55169bab"} Jul 21 12:42:42.981272 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:42:42.981188 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"4911b6ac-242a-4014-933d-6821ff61e77a","Type":"ContainerStarted","Data":"e68d996e08deeb2f4cbc3ccea5934013a9ef6a9577df46538b146a1b54949693"} Jul 21 12:42:42.981272 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:42:42.981216 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"4911b6ac-242a-4014-933d-6821ff61e77a","Type":"ContainerStarted","Data":"3bbb2c3c6d1e25c5cc67e9d57d447ae1f0c33afca1e31444ef1613027abd532d"} Jul 21 12:42:42.981272 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:42:42.981241 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"4911b6ac-242a-4014-933d-6821ff61e77a","Type":"ContainerStarted","Data":"2041af44bbf81d278541929c4c9e51ea6102984d31504129046afef85653ce5e"} Jul 21 12:42:42.981272 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:42:42.981265 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"4911b6ac-242a-4014-933d-6821ff61e77a","Type":"ContainerStarted","Data":"bd6ec96706f4a8c4d0a20608d9fff64eab09eb599fdfd734766d3b02b82dfc41"} Jul 21 12:42:43.012823 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:42:43.012775 2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-k8s-0" podStartSLOduration=1.993417572 podStartE2EDuration="2.012760664s" totalImagesPullingTime="0s" totalInitContainerRuntime="19.343092ms" isStatefulPod=false podCreationTimestamp="2026-07-21 12:42:41 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 12:42:43.011546702 +0000 UTC m=+170.846810568" watchObservedRunningTime="2026-07-21 12:42:43.012760664 +0000 UTC m=+170.848024529" Jul 21 12:42:43.897910 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:42:43.897873 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-skc26_6713665c-3b71-4751-929e-16de48263766/konnectivity-agent/0.log" limit=1024 Jul 21 12:42:43.998016 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:42:43.997988 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-skc26_6713665c-3b71-4751-929e-16de48263766/konnectivity-agent/0.log" limit=1024 Jul 21 12:42:44.100397 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:42:44.100373 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-skc26_6713665c-3b71-4751-929e-16de48263766/konnectivity-agent/0.log" limit=1024 Jul 21 12:42:44.632160 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:42:44.632125 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-skc26_6713665c-3b71-4751-929e-16de48263766/konnectivity-agent/0.log" limit=1024 Jul 21 12:42:44.702982 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:42:44.702956 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-skc26_6713665c-3b71-4751-929e-16de48263766/konnectivity-agent/0.log" limit=1024 Jul 21 12:42:46.346729 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:42:46.346701 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-k8s-0" Jul 21 12:42:55.006903 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:42:55.006852 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-skc26_6713665c-3b71-4751-929e-16de48263766/konnectivity-agent/0.log" limit=1024 Jul 21 12:42:55.107052 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:42:55.107016 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-skc26_6713665c-3b71-4751-929e-16de48263766/konnectivity-agent/0.log" limit=1024 Jul 21 12:42:55.176702 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:42:55.176677 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-skc26_6713665c-3b71-4751-929e-16de48263766/konnectivity-agent/0.log" limit=1024 Jul 21 12:43:02.956310 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:43:02.956276 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-skc26_6713665c-3b71-4751-929e-16de48263766/konnectivity-agent/0.log" limit=1024 Jul 21 12:43:16.524814 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:43:16.524771 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-skc26_6713665c-3b71-4751-929e-16de48263766/konnectivity-agent/0.log" limit=1024 Jul 21 12:43:16.596712 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:43:16.596681 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-skc26_6713665c-3b71-4751-929e-16de48263766/konnectivity-agent/0.log" limit=1024 Jul 21 12:43:41.347682 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:43:41.347639 2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/prometheus-k8s-0" Jul 21 12:43:41.362503 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:43:41.362480 2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/prometheus-k8s-0" Jul 21 12:43:42.212533 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:43:42.212493 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-k8s-0" Jul 21 12:43:49.999297 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:43:49.999262 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-skc26_6713665c-3b71-4751-929e-16de48263766/konnectivity-agent/0.log" limit=1024 Jul 21 12:43:50.064047 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:43:50.064021 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-skc26_6713665c-3b71-4751-929e-16de48263766/konnectivity-agent/0.log" limit=1024 Jul 21 12:43:51.549679 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:43:51.549643 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-skc26_6713665c-3b71-4751-929e-16de48263766/konnectivity-agent/0.log" limit=1024 Jul 21 12:43:51.610659 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:43:51.610626 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-skc26_6713665c-3b71-4751-929e-16de48263766/konnectivity-agent/0.log" limit=1024 Jul 21 12:43:57.665845 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:43:57.665805 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-skc26_6713665c-3b71-4751-929e-16de48263766/konnectivity-agent/0.log" limit=1024 Jul 21 12:44:02.040490 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:44:02.040452 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-skc26_6713665c-3b71-4751-929e-16de48263766/konnectivity-agent/0.log" limit=1024 Jul 21 12:44:02.130876 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:44:02.130842 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-skc26_6713665c-3b71-4751-929e-16de48263766/konnectivity-agent/0.log" limit=1024 Jul 21 12:44:02.330575 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:44:02.330483 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-skc26_6713665c-3b71-4751-929e-16de48263766/konnectivity-agent/0.log" limit=1024 Jul 21 12:44:02.428075 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:44:02.428044 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-skc26_6713665c-3b71-4751-929e-16de48263766/konnectivity-agent/0.log" limit=1024 Jul 21 12:44:02.536620 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:44:02.536592 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-skc26_6713665c-3b71-4751-929e-16de48263766/konnectivity-agent/0.log" limit=1024 Jul 21 12:44:15.970119 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:44:15.970034 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-skc26_6713665c-3b71-4751-929e-16de48263766/konnectivity-agent/0.log" limit=1024 Jul 21 12:44:16.068465 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:44:16.068414 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-skc26_6713665c-3b71-4751-929e-16de48263766/konnectivity-agent/0.log" limit=1024 Jul 21 12:44:16.167137 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:44:16.167107 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-skc26_6713665c-3b71-4751-929e-16de48263766/konnectivity-agent/0.log" limit=1024 Jul 21 12:44:53.625953 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:44:53.625924 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-6nf7d_a2b48106-9844-4431-9b6f-d4e1baacb15b/ovn-acl-logging/0.log" Jul 21 12:44:53.631305 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:44:53.631283 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-6nf7d_a2b48106-9844-4431-9b6f-d4e1baacb15b/ovn-acl-logging/0.log" Jul 21 12:44:53.633365 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:44:53.633348 2576 kubelet.go:1648] "Image garbage collection succeeded" Jul 21 12:44:56.887284 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:44:56.887250 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-skc26_6713665c-3b71-4751-929e-16de48263766/konnectivity-agent/0.log" limit=1024 Jul 21 12:45:14.683360 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:45:14.683326 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-skc26_6713665c-3b71-4751-929e-16de48263766/konnectivity-agent/0.log" limit=1024 Jul 21 12:45:14.786485 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:45:14.786446 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-skc26_6713665c-3b71-4751-929e-16de48263766/konnectivity-agent/0.log" limit=1024 Jul 21 12:45:41.636952 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:45:41.636916 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-skc26_6713665c-3b71-4751-929e-16de48263766/konnectivity-agent/0.log" limit=1024 Jul 21 12:45:42.573745 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:45:42.573706 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-skc26_6713665c-3b71-4751-929e-16de48263766/konnectivity-agent/0.log" limit=1024 Jul 21 12:45:45.448880 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:45:45.448792 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-skc26_6713665c-3b71-4751-929e-16de48263766/konnectivity-agent/0.log" limit=1024 Jul 21 12:45:45.548888 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:45:45.548858 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-skc26_6713665c-3b71-4751-929e-16de48263766/konnectivity-agent/0.log" limit=1024 Jul 21 12:45:46.452315 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:45:46.452284 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-skc26_6713665c-3b71-4751-929e-16de48263766/konnectivity-agent/0.log" limit=1024 Jul 21 12:45:46.548686 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:45:46.548658 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-skc26_6713665c-3b71-4751-929e-16de48263766/konnectivity-agent/0.log" limit=1024 Jul 21 12:45:46.962339 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:45:46.962307 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-skc26_6713665c-3b71-4751-929e-16de48263766/konnectivity-agent/0.log" limit=1024 Jul 21 12:45:47.066606 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:45:47.066574 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-skc26_6713665c-3b71-4751-929e-16de48263766/konnectivity-agent/0.log" limit=1024 Jul 21 12:45:48.862741 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:45:48.862707 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-skc26_6713665c-3b71-4751-929e-16de48263766/konnectivity-agent/0.log" limit=1024 Jul 21 12:45:49.071923 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:45:49.071894 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-skc26_6713665c-3b71-4751-929e-16de48263766/konnectivity-agent/0.log" limit=1024 Jul 21 12:45:52.050935 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:45:52.050874 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-skc26_6713665c-3b71-4751-929e-16de48263766/konnectivity-agent/0.log" limit=1024 Jul 21 12:45:52.148998 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:45:52.148968 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-skc26_6713665c-3b71-4751-929e-16de48263766/konnectivity-agent/0.log" limit=1024 Jul 21 12:45:56.673287 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:45:56.673260 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-skc26_6713665c-3b71-4751-929e-16de48263766/konnectivity-agent/0.log" limit=1024 Jul 21 12:46:01.823623 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:46:01.823589 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-skc26_6713665c-3b71-4751-929e-16de48263766/konnectivity-agent/0.log" limit=1024 Jul 21 12:46:01.924619 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:46:01.924593 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-skc26_6713665c-3b71-4751-929e-16de48263766/konnectivity-agent/0.log" limit=1024 Jul 21 12:46:09.381024 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:46:09.380984 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-skc26_6713665c-3b71-4751-929e-16de48263766/konnectivity-agent/0.log" limit=1024 Jul 21 12:46:09.478395 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:46:09.478368 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-skc26_6713665c-3b71-4751-929e-16de48263766/konnectivity-agent/0.log" limit=1024 Jul 21 12:46:14.551507 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:46:14.551473 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-skc26_6713665c-3b71-4751-929e-16de48263766/konnectivity-agent/0.log" limit=1024 Jul 21 12:46:14.651681 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:46:14.651651 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-skc26_6713665c-3b71-4751-929e-16de48263766/konnectivity-agent/0.log" limit=1024 Jul 21 12:46:17.123770 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:46:17.123739 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-skc26_6713665c-3b71-4751-929e-16de48263766/konnectivity-agent/0.log" limit=1024 Jul 21 12:46:23.271050 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:46:23.271016 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-skc26_6713665c-3b71-4751-929e-16de48263766/konnectivity-agent/0.log" limit=1024 Jul 21 12:46:23.338928 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:46:23.338895 2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-lws-operator/lws-controller-manager-84bfb648b5-4nc5w"] Jul 21 12:46:23.341863 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:46:23.341846 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-84bfb648b5-4nc5w" Jul 21 12:46:23.344292 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:46:23.344270 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"openshift-service-ca.crt\"" Jul 21 12:46:23.344900 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:46:23.344868 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"lws-controller-manager-dockercfg-vjqqc\"" Jul 21 12:46:23.344900 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:46:23.344890 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"lws-manager-config\"" Jul 21 12:46:23.344900 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:46:23.344899 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"webhook-server-cert\"" Jul 21 12:46:23.345114 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:46:23.344918 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"metrics-server-cert\"" Jul 21 12:46:23.345114 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:46:23.344949 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"kube-root-ca.crt\"" Jul 21 12:46:23.350816 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:46:23.350783 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-skc26_6713665c-3b71-4751-929e-16de48263766/konnectivity-agent/0.log" limit=1024 Jul 21 12:46:23.352205 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:46:23.352185 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-84bfb648b5-4nc5w"] Jul 21 12:46:23.431449 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:46:23.431395 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/7050b6cd-c457-48d8-9dce-cc1dfa7ecf38-cert\") pod \"lws-controller-manager-84bfb648b5-4nc5w\" (UID: \"7050b6cd-c457-48d8-9dce-cc1dfa7ecf38\") " pod="openshift-lws-operator/lws-controller-manager-84bfb648b5-4nc5w" Jul 21 12:46:23.431669 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:46:23.431540 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tcr6l\" (UniqueName: \"kubernetes.io/projected/7050b6cd-c457-48d8-9dce-cc1dfa7ecf38-kube-api-access-tcr6l\") pod \"lws-controller-manager-84bfb648b5-4nc5w\" (UID: \"7050b6cd-c457-48d8-9dce-cc1dfa7ecf38\") " pod="openshift-lws-operator/lws-controller-manager-84bfb648b5-4nc5w" Jul 21 12:46:23.431669 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:46:23.431583 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/7050b6cd-c457-48d8-9dce-cc1dfa7ecf38-manager-config\") pod \"lws-controller-manager-84bfb648b5-4nc5w\" (UID: \"7050b6cd-c457-48d8-9dce-cc1dfa7ecf38\") " pod="openshift-lws-operator/lws-controller-manager-84bfb648b5-4nc5w" Jul 21 12:46:23.431669 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:46:23.431606 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/7050b6cd-c457-48d8-9dce-cc1dfa7ecf38-metrics-cert\") pod \"lws-controller-manager-84bfb648b5-4nc5w\" (UID: \"7050b6cd-c457-48d8-9dce-cc1dfa7ecf38\") " pod="openshift-lws-operator/lws-controller-manager-84bfb648b5-4nc5w" Jul 21 12:46:23.532483 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:46:23.532397 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/7050b6cd-c457-48d8-9dce-cc1dfa7ecf38-cert\") pod \"lws-controller-manager-84bfb648b5-4nc5w\" (UID: \"7050b6cd-c457-48d8-9dce-cc1dfa7ecf38\") " pod="openshift-lws-operator/lws-controller-manager-84bfb648b5-4nc5w" Jul 21 12:46:23.532483 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:46:23.532470 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tcr6l\" (UniqueName: \"kubernetes.io/projected/7050b6cd-c457-48d8-9dce-cc1dfa7ecf38-kube-api-access-tcr6l\") pod \"lws-controller-manager-84bfb648b5-4nc5w\" (UID: \"7050b6cd-c457-48d8-9dce-cc1dfa7ecf38\") " pod="openshift-lws-operator/lws-controller-manager-84bfb648b5-4nc5w" Jul 21 12:46:23.532727 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:46:23.532538 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/7050b6cd-c457-48d8-9dce-cc1dfa7ecf38-manager-config\") pod \"lws-controller-manager-84bfb648b5-4nc5w\" (UID: \"7050b6cd-c457-48d8-9dce-cc1dfa7ecf38\") " pod="openshift-lws-operator/lws-controller-manager-84bfb648b5-4nc5w" Jul 21 12:46:23.532727 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:46:23.532567 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/7050b6cd-c457-48d8-9dce-cc1dfa7ecf38-metrics-cert\") pod \"lws-controller-manager-84bfb648b5-4nc5w\" (UID: \"7050b6cd-c457-48d8-9dce-cc1dfa7ecf38\") " pod="openshift-lws-operator/lws-controller-manager-84bfb648b5-4nc5w" Jul 21 12:46:23.533229 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:46:23.533204 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/7050b6cd-c457-48d8-9dce-cc1dfa7ecf38-manager-config\") pod \"lws-controller-manager-84bfb648b5-4nc5w\" (UID: \"7050b6cd-c457-48d8-9dce-cc1dfa7ecf38\") " pod="openshift-lws-operator/lws-controller-manager-84bfb648b5-4nc5w" Jul 21 12:46:23.534974 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:46:23.534943 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/7050b6cd-c457-48d8-9dce-cc1dfa7ecf38-metrics-cert\") pod \"lws-controller-manager-84bfb648b5-4nc5w\" (UID: \"7050b6cd-c457-48d8-9dce-cc1dfa7ecf38\") " pod="openshift-lws-operator/lws-controller-manager-84bfb648b5-4nc5w" Jul 21 12:46:23.535069 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:46:23.534977 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/7050b6cd-c457-48d8-9dce-cc1dfa7ecf38-cert\") pod \"lws-controller-manager-84bfb648b5-4nc5w\" (UID: \"7050b6cd-c457-48d8-9dce-cc1dfa7ecf38\") " pod="openshift-lws-operator/lws-controller-manager-84bfb648b5-4nc5w" Jul 21 12:46:23.542376 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:46:23.542351 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tcr6l\" (UniqueName: \"kubernetes.io/projected/7050b6cd-c457-48d8-9dce-cc1dfa7ecf38-kube-api-access-tcr6l\") pod \"lws-controller-manager-84bfb648b5-4nc5w\" (UID: \"7050b6cd-c457-48d8-9dce-cc1dfa7ecf38\") " pod="openshift-lws-operator/lws-controller-manager-84bfb648b5-4nc5w" Jul 21 12:46:23.651538 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:46:23.651469 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-84bfb648b5-4nc5w" Jul 21 12:46:23.795231 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:46:23.795162 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-84bfb648b5-4nc5w"] Jul 21 12:46:23.798381 ip-10-0-130-157 kubenswrapper[2576]: W0721 12:46:23.798354 2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7050b6cd_c457_48d8_9dce_cc1dfa7ecf38.slice/crio-c36b1d0f7b2f4f1a7a6d83adc69f2769c17ebc9409a9f1bfa1ed00471600c445 WatchSource:0}: Error finding container c36b1d0f7b2f4f1a7a6d83adc69f2769c17ebc9409a9f1bfa1ed00471600c445: Status 404 returned error can't find the container with id c36b1d0f7b2f4f1a7a6d83adc69f2769c17ebc9409a9f1bfa1ed00471600c445 Jul 21 12:46:23.800174 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:46:23.800158 2576 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 21 12:46:24.744170 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:46:24.744099 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-84bfb648b5-4nc5w" event={"ID":"7050b6cd-c457-48d8-9dce-cc1dfa7ecf38","Type":"ContainerStarted","Data":"c36b1d0f7b2f4f1a7a6d83adc69f2769c17ebc9409a9f1bfa1ed00471600c445"} Jul 21 12:46:26.753960 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:46:26.753919 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-84bfb648b5-4nc5w" event={"ID":"7050b6cd-c457-48d8-9dce-cc1dfa7ecf38","Type":"ContainerStarted","Data":"e5f942adcdaa0032e0b581d7eb93a4a51ffd436a7b2b89750b7e2e4426289050"} Jul 21 12:46:26.754454 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:46:26.753997 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-lws-operator/lws-controller-manager-84bfb648b5-4nc5w" Jul 21 12:46:26.772567 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:46:26.772474 2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-lws-operator/lws-controller-manager-84bfb648b5-4nc5w" podStartSLOduration=1.5062950640000001 podStartE2EDuration="3.772457814s" totalImagesPullingTime="2.26616275s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 12:46:23 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 12:46:26.77159642 +0000 UTC m=+394.606860287" watchObservedRunningTime="2026-07-21 12:46:26.772457814 +0000 UTC m=+394.607721680" Jul 21 12:46:31.715452 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:46:31.715410 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-skc26_6713665c-3b71-4751-929e-16de48263766/konnectivity-agent/0.log" limit=1024 Jul 21 12:46:31.816004 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:46:31.815974 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-skc26_6713665c-3b71-4751-929e-16de48263766/konnectivity-agent/0.log" limit=1024 Jul 21 12:46:32.352858 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:46:32.352824 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-skc26_6713665c-3b71-4751-929e-16de48263766/konnectivity-agent/0.log" limit=1024 Jul 21 12:46:32.603770 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:46:32.603671 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-skc26_6713665c-3b71-4751-929e-16de48263766/konnectivity-agent/0.log" limit=1024 Jul 21 12:46:32.701110 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:46:32.701065 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-skc26_6713665c-3b71-4751-929e-16de48263766/konnectivity-agent/0.log" limit=1024 Jul 21 12:46:32.743176 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:46:32.743147 2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-579d89bddd-mk77w"] Jul 21 12:46:32.747256 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:46:32.746595 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-579d89bddd-mk77w" Jul 21 12:46:32.749148 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:46:32.749121 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-dockercfg-bh5f5\"" Jul 21 12:46:32.749374 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:46:32.749353 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-webhook-cert\"" Jul 21 12:46:32.749597 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:46:32.749574 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-service-cert\"" Jul 21 12:46:32.749813 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:46:32.749792 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\"" Jul 21 12:46:32.749894 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:46:32.749873 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\"" Jul 21 12:46:32.762550 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:46:32.762528 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-579d89bddd-mk77w"] Jul 21 12:46:32.899669 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:46:32.899642 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/3e7a2646-5d2a-498c-878f-f3c1f6834381-apiservice-cert\") pod \"opendatahub-operator-controller-manager-579d89bddd-mk77w\" (UID: \"3e7a2646-5d2a-498c-878f-f3c1f6834381\") " pod="opendatahub/opendatahub-operator-controller-manager-579d89bddd-mk77w" Jul 21 12:46:32.899810 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:46:32.899680 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/3e7a2646-5d2a-498c-878f-f3c1f6834381-webhook-cert\") pod \"opendatahub-operator-controller-manager-579d89bddd-mk77w\" (UID: \"3e7a2646-5d2a-498c-878f-f3c1f6834381\") " pod="opendatahub/opendatahub-operator-controller-manager-579d89bddd-mk77w" Jul 21 12:46:32.899810 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:46:32.899713 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/3e7a2646-5d2a-498c-878f-f3c1f6834381-manifests\") pod \"opendatahub-operator-controller-manager-579d89bddd-mk77w\" (UID: \"3e7a2646-5d2a-498c-878f-f3c1f6834381\") " pod="opendatahub/opendatahub-operator-controller-manager-579d89bddd-mk77w" Jul 21 12:46:32.899810 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:46:32.899764 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/3e7a2646-5d2a-498c-878f-f3c1f6834381-tmp\") pod \"opendatahub-operator-controller-manager-579d89bddd-mk77w\" (UID: \"3e7a2646-5d2a-498c-878f-f3c1f6834381\") " pod="opendatahub/opendatahub-operator-controller-manager-579d89bddd-mk77w" Jul 21 12:46:32.899810 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:46:32.899793 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-q4xtt\" (UniqueName: \"kubernetes.io/projected/3e7a2646-5d2a-498c-878f-f3c1f6834381-kube-api-access-q4xtt\") pod \"opendatahub-operator-controller-manager-579d89bddd-mk77w\" (UID: \"3e7a2646-5d2a-498c-878f-f3c1f6834381\") " pod="opendatahub/opendatahub-operator-controller-manager-579d89bddd-mk77w" Jul 21 12:46:32.912584 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:46:32.912558 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-skc26_6713665c-3b71-4751-929e-16de48263766/konnectivity-agent/0.log" limit=1024 Jul 21 12:46:33.001049 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:46:33.001024 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/3e7a2646-5d2a-498c-878f-f3c1f6834381-webhook-cert\") pod \"opendatahub-operator-controller-manager-579d89bddd-mk77w\" (UID: \"3e7a2646-5d2a-498c-878f-f3c1f6834381\") " pod="opendatahub/opendatahub-operator-controller-manager-579d89bddd-mk77w" Jul 21 12:46:33.001152 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:46:33.001064 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/3e7a2646-5d2a-498c-878f-f3c1f6834381-manifests\") pod \"opendatahub-operator-controller-manager-579d89bddd-mk77w\" (UID: \"3e7a2646-5d2a-498c-878f-f3c1f6834381\") " pod="opendatahub/opendatahub-operator-controller-manager-579d89bddd-mk77w" Jul 21 12:46:33.001152 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:46:33.001081 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/3e7a2646-5d2a-498c-878f-f3c1f6834381-tmp\") pod \"opendatahub-operator-controller-manager-579d89bddd-mk77w\" (UID: \"3e7a2646-5d2a-498c-878f-f3c1f6834381\") " pod="opendatahub/opendatahub-operator-controller-manager-579d89bddd-mk77w" Jul 21 12:46:33.001152 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:46:33.001100 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-q4xtt\" (UniqueName: \"kubernetes.io/projected/3e7a2646-5d2a-498c-878f-f3c1f6834381-kube-api-access-q4xtt\") pod \"opendatahub-operator-controller-manager-579d89bddd-mk77w\" (UID: \"3e7a2646-5d2a-498c-878f-f3c1f6834381\") " pod="opendatahub/opendatahub-operator-controller-manager-579d89bddd-mk77w" Jul 21 12:46:33.001152 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:46:33.001132 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/3e7a2646-5d2a-498c-878f-f3c1f6834381-apiservice-cert\") pod \"opendatahub-operator-controller-manager-579d89bddd-mk77w\" (UID: \"3e7a2646-5d2a-498c-878f-f3c1f6834381\") " pod="opendatahub/opendatahub-operator-controller-manager-579d89bddd-mk77w" Jul 21 12:46:33.001487 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:46:33.001454 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/3e7a2646-5d2a-498c-878f-f3c1f6834381-manifests\") pod \"opendatahub-operator-controller-manager-579d89bddd-mk77w\" (UID: \"3e7a2646-5d2a-498c-878f-f3c1f6834381\") " pod="opendatahub/opendatahub-operator-controller-manager-579d89bddd-mk77w" Jul 21 12:46:33.003345 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:46:33.003317 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/3e7a2646-5d2a-498c-878f-f3c1f6834381-tmp\") pod \"opendatahub-operator-controller-manager-579d89bddd-mk77w\" (UID: \"3e7a2646-5d2a-498c-878f-f3c1f6834381\") " pod="opendatahub/opendatahub-operator-controller-manager-579d89bddd-mk77w" Jul 21 12:46:33.003570 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:46:33.003554 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/3e7a2646-5d2a-498c-878f-f3c1f6834381-webhook-cert\") pod \"opendatahub-operator-controller-manager-579d89bddd-mk77w\" (UID: \"3e7a2646-5d2a-498c-878f-f3c1f6834381\") " pod="opendatahub/opendatahub-operator-controller-manager-579d89bddd-mk77w" Jul 21 12:46:33.003705 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:46:33.003684 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/3e7a2646-5d2a-498c-878f-f3c1f6834381-apiservice-cert\") pod \"opendatahub-operator-controller-manager-579d89bddd-mk77w\" (UID: \"3e7a2646-5d2a-498c-878f-f3c1f6834381\") " pod="opendatahub/opendatahub-operator-controller-manager-579d89bddd-mk77w" Jul 21 12:46:33.013309 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:46:33.013283 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-q4xtt\" (UniqueName: \"kubernetes.io/projected/3e7a2646-5d2a-498c-878f-f3c1f6834381-kube-api-access-q4xtt\") pod \"opendatahub-operator-controller-manager-579d89bddd-mk77w\" (UID: \"3e7a2646-5d2a-498c-878f-f3c1f6834381\") " pod="opendatahub/opendatahub-operator-controller-manager-579d89bddd-mk77w" Jul 21 12:46:33.014418 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:46:33.014395 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-skc26_6713665c-3b71-4751-929e-16de48263766/konnectivity-agent/0.log" limit=1024 Jul 21 12:46:33.057744 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:46:33.057723 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-579d89bddd-mk77w" Jul 21 12:46:33.108491 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:46:33.108461 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-skc26_6713665c-3b71-4751-929e-16de48263766/konnectivity-agent/0.log" limit=1024 Jul 21 12:46:33.186544 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:46:33.186405 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-579d89bddd-mk77w"] Jul 21 12:46:33.191265 ip-10-0-130-157 kubenswrapper[2576]: W0721 12:46:33.191239 2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3e7a2646_5d2a_498c_878f_f3c1f6834381.slice/crio-a1b5bbd59b20cd9b6b658499741f2df81c93de5002b28eabf80a5b3679c0927b WatchSource:0}: Error finding container a1b5bbd59b20cd9b6b658499741f2df81c93de5002b28eabf80a5b3679c0927b: Status 404 returned error can't find the container with id a1b5bbd59b20cd9b6b658499741f2df81c93de5002b28eabf80a5b3679c0927b Jul 21 12:46:33.192263 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:46:33.192240 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-skc26_6713665c-3b71-4751-929e-16de48263766/konnectivity-agent/0.log" limit=1024 Jul 21 12:46:33.274452 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:46:33.274425 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-skc26_6713665c-3b71-4751-929e-16de48263766/konnectivity-agent/0.log" limit=1024 Jul 21 12:46:33.369630 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:46:33.369601 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-skc26_6713665c-3b71-4751-929e-16de48263766/konnectivity-agent/0.log" limit=1024 Jul 21 12:46:33.780952 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:46:33.780916 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-579d89bddd-mk77w" event={"ID":"3e7a2646-5d2a-498c-878f-f3c1f6834381","Type":"ContainerStarted","Data":"a1b5bbd59b20cd9b6b658499741f2df81c93de5002b28eabf80a5b3679c0927b"} Jul 21 12:46:36.793611 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:46:36.793579 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-579d89bddd-mk77w" event={"ID":"3e7a2646-5d2a-498c-878f-f3c1f6834381","Type":"ContainerStarted","Data":"c0818498e4fd0aa6db0f0175b7679eaa5f8548c63eb7b96faa1a50d6b4eb4928"} Jul 21 12:46:37.760848 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:46:37.760815 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-lws-operator/lws-controller-manager-84bfb648b5-4nc5w" Jul 21 12:46:37.798499 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:46:37.798467 2576 generic.go:356] "Generic (PLEG): container finished" podID="3e7a2646-5d2a-498c-878f-f3c1f6834381" containerID="c0818498e4fd0aa6db0f0175b7679eaa5f8548c63eb7b96faa1a50d6b4eb4928" exitCode=0 Jul 21 12:46:37.798947 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:46:37.798528 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-579d89bddd-mk77w" event={"ID":"3e7a2646-5d2a-498c-878f-f3c1f6834381","Type":"ContainerDied","Data":"c0818498e4fd0aa6db0f0175b7679eaa5f8548c63eb7b96faa1a50d6b4eb4928"} Jul 21 12:46:38.804171 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:46:38.804118 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-579d89bddd-mk77w" event={"ID":"3e7a2646-5d2a-498c-878f-f3c1f6834381","Type":"ContainerStarted","Data":"216cd122b7d56f9ecd9874f8b66479a3637216f539fc96d7c65c1b6bac825f0e"} Jul 21 12:46:38.804791 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:46:38.804212 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/opendatahub-operator-controller-manager-579d89bddd-mk77w" Jul 21 12:46:38.825086 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:46:38.825040 2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/opendatahub-operator-controller-manager-579d89bddd-mk77w" podStartSLOduration=2.432498312 podStartE2EDuration="6.825026324s" totalImagesPullingTime="3.375509927s" totalInitContainerRuntime="1.017018085s" isStatefulPod=false podCreationTimestamp="2026-07-21 12:46:32 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 12:46:38.823256724 +0000 UTC m=+406.658520591" watchObservedRunningTime="2026-07-21 12:46:38.825026324 +0000 UTC m=+406.660290189" Jul 21 12:46:49.809946 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:46:49.809915 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/opendatahub-operator-controller-manager-579d89bddd-mk77w" Jul 21 12:46:51.648558 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:46:51.648504 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-skc26_6713665c-3b71-4751-929e-16de48263766/konnectivity-agent/0.log" limit=1024 Jul 21 12:46:51.762278 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:46:51.762249 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-skc26_6713665c-3b71-4751-929e-16de48263766/konnectivity-agent/0.log" limit=1024 Jul 21 12:46:52.234723 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:46:52.234689 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-skc26_6713665c-3b71-4751-929e-16de48263766/konnectivity-agent/0.log" limit=1024 Jul 21 12:46:52.315537 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:46:52.315498 2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/kube-auth-proxy-67cb6d6b5d-424n4"] Jul 21 12:46:52.323584 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:46:52.323556 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-67cb6d6b5d-424n4" Jul 21 12:46:52.326241 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:46:52.326209 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-dockercfg-kr5rq\"" Jul 21 12:46:52.326241 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:46:52.326222 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-creds\"" Jul 21 12:46:52.326423 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:46:52.326210 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-tls\"" Jul 21 12:46:52.326423 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:46:52.326253 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\"" Jul 21 12:46:52.326631 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:46:52.326614 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\"" Jul 21 12:46:52.331370 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:46:52.331349 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-67cb6d6b5d-424n4"] Jul 21 12:46:52.406803 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:46:52.406775 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-skc26_6713665c-3b71-4751-929e-16de48263766/konnectivity-agent/0.log" limit=1024 Jul 21 12:46:52.445469 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:46:52.445439 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/c8997198-ef13-4846-90eb-47345692f1a9-tmp\") pod \"kube-auth-proxy-67cb6d6b5d-424n4\" (UID: \"c8997198-ef13-4846-90eb-47345692f1a9\") " pod="openshift-ingress/kube-auth-proxy-67cb6d6b5d-424n4" Jul 21 12:46:52.445587 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:46:52.445474 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c8997198-ef13-4846-90eb-47345692f1a9-tls-certs\") pod \"kube-auth-proxy-67cb6d6b5d-424n4\" (UID: \"c8997198-ef13-4846-90eb-47345692f1a9\") " pod="openshift-ingress/kube-auth-proxy-67cb6d6b5d-424n4" Jul 21 12:46:52.445587 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:46:52.445525 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gkrh5\" (UniqueName: \"kubernetes.io/projected/c8997198-ef13-4846-90eb-47345692f1a9-kube-api-access-gkrh5\") pod \"kube-auth-proxy-67cb6d6b5d-424n4\" (UID: \"c8997198-ef13-4846-90eb-47345692f1a9\") " pod="openshift-ingress/kube-auth-proxy-67cb6d6b5d-424n4" Jul 21 12:46:52.505085 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:46:52.505005 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-skc26_6713665c-3b71-4751-929e-16de48263766/konnectivity-agent/0.log" limit=1024 Jul 21 12:46:52.546746 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:46:52.546719 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gkrh5\" (UniqueName: \"kubernetes.io/projected/c8997198-ef13-4846-90eb-47345692f1a9-kube-api-access-gkrh5\") pod \"kube-auth-proxy-67cb6d6b5d-424n4\" (UID: \"c8997198-ef13-4846-90eb-47345692f1a9\") " pod="openshift-ingress/kube-auth-proxy-67cb6d6b5d-424n4" Jul 21 12:46:52.546867 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:46:52.546781 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/c8997198-ef13-4846-90eb-47345692f1a9-tmp\") pod \"kube-auth-proxy-67cb6d6b5d-424n4\" (UID: \"c8997198-ef13-4846-90eb-47345692f1a9\") " pod="openshift-ingress/kube-auth-proxy-67cb6d6b5d-424n4" Jul 21 12:46:52.546867 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:46:52.546824 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c8997198-ef13-4846-90eb-47345692f1a9-tls-certs\") pod \"kube-auth-proxy-67cb6d6b5d-424n4\" (UID: \"c8997198-ef13-4846-90eb-47345692f1a9\") " pod="openshift-ingress/kube-auth-proxy-67cb6d6b5d-424n4" Jul 21 12:46:52.548944 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:46:52.548918 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/c8997198-ef13-4846-90eb-47345692f1a9-tmp\") pod \"kube-auth-proxy-67cb6d6b5d-424n4\" (UID: \"c8997198-ef13-4846-90eb-47345692f1a9\") " pod="openshift-ingress/kube-auth-proxy-67cb6d6b5d-424n4" Jul 21 12:46:52.549084 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:46:52.549065 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c8997198-ef13-4846-90eb-47345692f1a9-tls-certs\") pod \"kube-auth-proxy-67cb6d6b5d-424n4\" (UID: \"c8997198-ef13-4846-90eb-47345692f1a9\") " pod="openshift-ingress/kube-auth-proxy-67cb6d6b5d-424n4" Jul 21 12:46:52.561003 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:46:52.560982 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gkrh5\" (UniqueName: \"kubernetes.io/projected/c8997198-ef13-4846-90eb-47345692f1a9-kube-api-access-gkrh5\") pod \"kube-auth-proxy-67cb6d6b5d-424n4\" (UID: \"c8997198-ef13-4846-90eb-47345692f1a9\") " pod="openshift-ingress/kube-auth-proxy-67cb6d6b5d-424n4" Jul 21 12:46:52.613065 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:46:52.613045 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-skc26_6713665c-3b71-4751-929e-16de48263766/konnectivity-agent/0.log" limit=1024 Jul 21 12:46:52.634408 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:46:52.634390 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-67cb6d6b5d-424n4" Jul 21 12:46:52.711912 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:46:52.711885 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-skc26_6713665c-3b71-4751-929e-16de48263766/konnectivity-agent/0.log" limit=1024 Jul 21 12:46:52.775489 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:46:52.775468 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-67cb6d6b5d-424n4"] Jul 21 12:46:52.777882 ip-10-0-130-157 kubenswrapper[2576]: W0721 12:46:52.777854 2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc8997198_ef13_4846_90eb_47345692f1a9.slice/crio-2dc09dc4e24e04aad41435f7617c33bb0bc947a34bc719bcfedd153ab8e71402 WatchSource:0}: Error finding container 2dc09dc4e24e04aad41435f7617c33bb0bc947a34bc719bcfedd153ab8e71402: Status 404 returned error can't find the container with id 2dc09dc4e24e04aad41435f7617c33bb0bc947a34bc719bcfedd153ab8e71402 Jul 21 12:46:52.811482 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:46:52.811453 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-skc26_6713665c-3b71-4751-929e-16de48263766/konnectivity-agent/0.log" limit=1024 Jul 21 12:46:52.854264 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:46:52.854233 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-67cb6d6b5d-424n4" event={"ID":"c8997198-ef13-4846-90eb-47345692f1a9","Type":"ContainerStarted","Data":"2dc09dc4e24e04aad41435f7617c33bb0bc947a34bc719bcfedd153ab8e71402"} Jul 21 12:46:53.523817 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:46:53.523779 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-skc26_6713665c-3b71-4751-929e-16de48263766/konnectivity-agent/0.log" limit=1024 Jul 21 12:46:53.671029 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:46:53.670993 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-skc26_6713665c-3b71-4751-929e-16de48263766/konnectivity-agent/0.log" limit=1024 Jul 21 12:46:53.738273 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:46:53.738242 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-skc26_6713665c-3b71-4751-929e-16de48263766/konnectivity-agent/0.log" limit=1024 Jul 21 12:46:53.961602 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:46:53.961569 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-skc26_6713665c-3b71-4751-929e-16de48263766/konnectivity-agent/0.log" limit=1024 Jul 21 12:46:54.934471 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:46:54.934436 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-skc26_6713665c-3b71-4751-929e-16de48263766/konnectivity-agent/0.log" limit=1024 Jul 21 12:46:55.154792 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:46:55.154773 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-creds\"" Jul 21 12:46:55.866425 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:46:55.866394 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-67cb6d6b5d-424n4" event={"ID":"c8997198-ef13-4846-90eb-47345692f1a9","Type":"ContainerStarted","Data":"1af7fcafef4a46b4a5d05f42f9017a904aa2f1bdc1e07c41c7a855e907edfc76"} Jul 21 12:46:55.883439 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:46:55.883125 2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/kube-auth-proxy-67cb6d6b5d-424n4" podStartSLOduration=1.5099267950000002 podStartE2EDuration="3.883108035s" totalImagesPullingTime="2.37318124s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 12:46:52 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 12:46:55.88260803 +0000 UTC m=+423.717871907" watchObservedRunningTime="2026-07-21 12:46:55.883108035 +0000 UTC m=+423.718371906" Jul 21 12:46:59.177364 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:46:59.177330 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-skc26_6713665c-3b71-4751-929e-16de48263766/konnectivity-agent/0.log" limit=1024 Jul 21 12:46:59.279892 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:46:59.279862 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-skc26_6713665c-3b71-4751-929e-16de48263766/konnectivity-agent/0.log" limit=1024 Jul 21 12:47:00.064802 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:47:00.064768 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-skc26_6713665c-3b71-4751-929e-16de48263766/konnectivity-agent/0.log" limit=1024 Jul 21 12:47:00.308565 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:47:00.308504 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-skc26_6713665c-3b71-4751-929e-16de48263766/konnectivity-agent/0.log" limit=1024 Jul 21 12:47:00.393282 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:47:00.393248 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-skc26_6713665c-3b71-4751-929e-16de48263766/konnectivity-agent/0.log" limit=1024 Jul 21 12:47:02.683580 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:47:02.683543 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-skc26_6713665c-3b71-4751-929e-16de48263766/konnectivity-agent/0.log" limit=1024 Jul 21 12:47:02.777137 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:47:02.777060 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-skc26_6713665c-3b71-4751-929e-16de48263766/konnectivity-agent/0.log" limit=1024 Jul 21 12:47:02.856291 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:47:02.856238 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-skc26_6713665c-3b71-4751-929e-16de48263766/konnectivity-agent/0.log" limit=1024 Jul 21 12:47:02.978126 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:47:02.978042 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-skc26_6713665c-3b71-4751-929e-16de48263766/konnectivity-agent/0.log" limit=1024 Jul 21 12:47:03.069586 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:47:03.069549 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-skc26_6713665c-3b71-4751-929e-16de48263766/konnectivity-agent/0.log" limit=1024 Jul 21 12:47:03.176288 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:47:03.176254 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-skc26_6713665c-3b71-4751-929e-16de48263766/konnectivity-agent/0.log" limit=1024 Jul 21 12:47:03.276649 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:47:03.276544 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-skc26_6713665c-3b71-4751-929e-16de48263766/konnectivity-agent/0.log" limit=1024 Jul 21 12:47:08.961722 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:47:08.961682 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-skc26_6713665c-3b71-4751-929e-16de48263766/konnectivity-agent/0.log" limit=1024 Jul 21 12:47:26.128265 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:47:26.128168 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-skc26_6713665c-3b71-4751-929e-16de48263766/konnectivity-agent/0.log" limit=1024 Jul 21 12:47:26.228775 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:47:26.228750 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-skc26_6713665c-3b71-4751-929e-16de48263766/konnectivity-agent/0.log" limit=1024 Jul 21 12:47:53.152020 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:47:53.151970 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-skc26_6713665c-3b71-4751-929e-16de48263766/konnectivity-agent/0.log" limit=1024 Jul 21 12:47:53.253635 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:47:53.253603 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-skc26_6713665c-3b71-4751-929e-16de48263766/konnectivity-agent/0.log" limit=1024 Jul 21 12:47:53.444863 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:47:53.444779 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-skc26_6713665c-3b71-4751-929e-16de48263766/konnectivity-agent/0.log" limit=1024 Jul 21 12:48:21.439054 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:48:21.439016 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-skc26_6713665c-3b71-4751-929e-16de48263766/konnectivity-agent/0.log" limit=1024 Jul 21 12:48:21.631771 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:48:21.631739 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-skc26_6713665c-3b71-4751-929e-16de48263766/konnectivity-agent/0.log" limit=1024 Jul 21 12:48:22.014712 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:48:22.014673 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-skc26_6713665c-3b71-4751-929e-16de48263766/konnectivity-agent/0.log" limit=1024 Jul 21 12:48:25.451483 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:48:25.451446 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-skc26_6713665c-3b71-4751-929e-16de48263766/konnectivity-agent/0.log" limit=1024 Jul 21 12:48:27.403923 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:48:27.403891 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-skc26_6713665c-3b71-4751-929e-16de48263766/konnectivity-agent/0.log" limit=1024 Jul 21 12:48:29.152807 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:48:29.152769 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-skc26_6713665c-3b71-4751-929e-16de48263766/konnectivity-agent/0.log" limit=1024 Jul 21 12:48:29.398083 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:48:29.398053 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-skc26_6713665c-3b71-4751-929e-16de48263766/konnectivity-agent/0.log" limit=1024 Jul 21 12:48:30.849701 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:48:30.849673 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-skc26_6713665c-3b71-4751-929e-16de48263766/konnectivity-agent/0.log" limit=1024 Jul 21 12:48:32.411339 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:48:32.411301 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-skc26_6713665c-3b71-4751-929e-16de48263766/konnectivity-agent/0.log" limit=1024 Jul 21 12:48:32.824419 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:48:32.824309 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-skc26_6713665c-3b71-4751-929e-16de48263766/konnectivity-agent/0.log" limit=1024 Jul 21 12:48:32.924649 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:48:32.924608 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-skc26_6713665c-3b71-4751-929e-16de48263766/konnectivity-agent/0.log" limit=1024 Jul 21 12:48:33.024430 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:48:33.024405 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-skc26_6713665c-3b71-4751-929e-16de48263766/konnectivity-agent/0.log" limit=1024 Jul 21 12:48:33.123125 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:48:33.123061 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-skc26_6713665c-3b71-4751-929e-16de48263766/konnectivity-agent/0.log" limit=1024 Jul 21 12:48:33.243964 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:48:33.243934 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-skc26_6713665c-3b71-4751-929e-16de48263766/konnectivity-agent/0.log" limit=1024 Jul 21 12:48:33.358055 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:48:33.358014 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-skc26_6713665c-3b71-4751-929e-16de48263766/konnectivity-agent/0.log" limit=1024 Jul 21 12:48:35.226827 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:48:35.226794 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-skc26_6713665c-3b71-4751-929e-16de48263766/konnectivity-agent/0.log" limit=1024 Jul 21 12:48:35.426844 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:48:35.426814 2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-console-plugin-66b5774985-khhrm"] Jul 21 12:48:35.428969 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:48:35.428952 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-khhrm" Jul 21 12:48:35.431676 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:48:35.431652 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\"" Jul 21 12:48:35.431804 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:48:35.431766 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"plugin-serving-cert\"" Jul 21 12:48:35.431871 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:48:35.431799 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kuadrant-console-nginx-conf\"" Jul 21 12:48:35.432236 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:48:35.432218 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"default-dockercfg-hsjs2\"" Jul 21 12:48:35.432312 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:48:35.432233 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\"" Jul 21 12:48:35.441492 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:48:35.441472 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-console-plugin-66b5774985-khhrm"] Jul 21 12:48:35.453633 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:48:35.453604 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-skc26_6713665c-3b71-4751-929e-16de48263766/konnectivity-agent/0.log" limit=1024 Jul 21 12:48:35.485507 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:48:35.485455 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/f54afc32-9dcd-47e8-a25a-199b3a57866d-plugin-serving-cert\") pod \"kuadrant-console-plugin-66b5774985-khhrm\" (UID: \"f54afc32-9dcd-47e8-a25a-199b3a57866d\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-khhrm" Jul 21 12:48:35.485507 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:48:35.485479 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/f54afc32-9dcd-47e8-a25a-199b3a57866d-nginx-conf\") pod \"kuadrant-console-plugin-66b5774985-khhrm\" (UID: \"f54afc32-9dcd-47e8-a25a-199b3a57866d\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-khhrm" Jul 21 12:48:35.485507 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:48:35.485505 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hj5j2\" (UniqueName: \"kubernetes.io/projected/f54afc32-9dcd-47e8-a25a-199b3a57866d-kube-api-access-hj5j2\") pod \"kuadrant-console-plugin-66b5774985-khhrm\" (UID: \"f54afc32-9dcd-47e8-a25a-199b3a57866d\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-khhrm" Jul 21 12:48:35.552298 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:48:35.552268 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-skc26_6713665c-3b71-4751-929e-16de48263766/konnectivity-agent/0.log" limit=1024 Jul 21 12:48:35.586592 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:48:35.586567 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/f54afc32-9dcd-47e8-a25a-199b3a57866d-plugin-serving-cert\") pod \"kuadrant-console-plugin-66b5774985-khhrm\" (UID: \"f54afc32-9dcd-47e8-a25a-199b3a57866d\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-khhrm" Jul 21 12:48:35.586661 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:48:35.586595 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/f54afc32-9dcd-47e8-a25a-199b3a57866d-nginx-conf\") pod \"kuadrant-console-plugin-66b5774985-khhrm\" (UID: \"f54afc32-9dcd-47e8-a25a-199b3a57866d\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-khhrm" Jul 21 12:48:35.586661 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:48:35.586629 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hj5j2\" (UniqueName: \"kubernetes.io/projected/f54afc32-9dcd-47e8-a25a-199b3a57866d-kube-api-access-hj5j2\") pod \"kuadrant-console-plugin-66b5774985-khhrm\" (UID: \"f54afc32-9dcd-47e8-a25a-199b3a57866d\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-khhrm" Jul 21 12:48:35.587171 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:48:35.587139 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/f54afc32-9dcd-47e8-a25a-199b3a57866d-nginx-conf\") pod \"kuadrant-console-plugin-66b5774985-khhrm\" (UID: \"f54afc32-9dcd-47e8-a25a-199b3a57866d\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-khhrm" Jul 21 12:48:35.588942 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:48:35.588921 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/f54afc32-9dcd-47e8-a25a-199b3a57866d-plugin-serving-cert\") pod \"kuadrant-console-plugin-66b5774985-khhrm\" (UID: \"f54afc32-9dcd-47e8-a25a-199b3a57866d\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-khhrm" Jul 21 12:48:35.599764 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:48:35.599737 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hj5j2\" (UniqueName: \"kubernetes.io/projected/f54afc32-9dcd-47e8-a25a-199b3a57866d-kube-api-access-hj5j2\") pod \"kuadrant-console-plugin-66b5774985-khhrm\" (UID: \"f54afc32-9dcd-47e8-a25a-199b3a57866d\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-khhrm" Jul 21 12:48:35.737948 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:48:35.737873 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-khhrm" Jul 21 12:48:35.873890 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:48:35.873861 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-console-plugin-66b5774985-khhrm"] Jul 21 12:48:35.877230 ip-10-0-130-157 kubenswrapper[2576]: W0721 12:48:35.877199 2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podf54afc32_9dcd_47e8_a25a_199b3a57866d.slice/crio-96ec63006b5604bf60bb91aea2d798b5e3be230440520511b7703056aa0338bf WatchSource:0}: Error finding container 96ec63006b5604bf60bb91aea2d798b5e3be230440520511b7703056aa0338bf: Status 404 returned error can't find the container with id 96ec63006b5604bf60bb91aea2d798b5e3be230440520511b7703056aa0338bf Jul 21 12:48:36.231226 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:48:36.231187 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-khhrm" event={"ID":"f54afc32-9dcd-47e8-a25a-199b3a57866d","Type":"ContainerStarted","Data":"96ec63006b5604bf60bb91aea2d798b5e3be230440520511b7703056aa0338bf"} Jul 21 12:48:37.199172 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:48:37.199135 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-skc26_6713665c-3b71-4751-929e-16de48263766/konnectivity-agent/0.log" limit=1024 Jul 21 12:48:37.268741 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:48:37.268678 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-skc26_6713665c-3b71-4751-929e-16de48263766/konnectivity-agent/0.log" limit=1024 Jul 21 12:48:44.698628 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:48:44.698535 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-skc26_6713665c-3b71-4751-929e-16de48263766/konnectivity-agent/0.log" limit=1024 Jul 21 12:48:44.788691 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:48:44.788649 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-skc26_6713665c-3b71-4751-929e-16de48263766/konnectivity-agent/0.log" limit=1024 Jul 21 12:48:49.425205 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:48:49.425165 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-skc26_6713665c-3b71-4751-929e-16de48263766/konnectivity-agent/0.log" limit=1024 Jul 21 12:48:49.540077 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:48:49.540038 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-skc26_6713665c-3b71-4751-929e-16de48263766/konnectivity-agent/0.log" limit=1024 Jul 21 12:48:51.888079 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:48:51.888048 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-skc26_6713665c-3b71-4751-929e-16de48263766/konnectivity-agent/0.log" limit=1024 Jul 21 12:48:51.995996 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:48:51.995965 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-skc26_6713665c-3b71-4751-929e-16de48263766/konnectivity-agent/0.log" limit=1024 Jul 21 12:48:57.320945 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:48:57.320913 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-khhrm" event={"ID":"f54afc32-9dcd-47e8-a25a-199b3a57866d","Type":"ContainerStarted","Data":"de471cccf5ba4a90803960b57d9b74d7ca2ae1453ebf19b1a3ffb6bd6f6f04bb"} Jul 21 12:48:57.339870 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:48:57.339795 2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-khhrm" podStartSLOduration=1.146029486 podStartE2EDuration="22.339781727s" totalImagesPullingTime="21.193752241s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 12:48:35 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 12:48:57.337606375 +0000 UTC m=+545.172870241" watchObservedRunningTime="2026-07-21 12:48:57.339781727 +0000 UTC m=+545.175045617" Jul 21 12:48:57.415231 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:48:57.415198 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-skc26_6713665c-3b71-4751-929e-16de48263766/konnectivity-agent/0.log" limit=1024 Jul 21 12:48:57.519971 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:48:57.519943 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-skc26_6713665c-3b71-4751-929e-16de48263766/konnectivity-agent/0.log" limit=1024 Jul 21 12:49:04.400342 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:49:04.400301 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-skc26_6713665c-3b71-4751-929e-16de48263766/konnectivity-agent/0.log" limit=1024 Jul 21 12:49:04.499090 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:49:04.499057 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-skc26_6713665c-3b71-4751-929e-16de48263766/konnectivity-agent/0.log" limit=1024 Jul 21 12:49:09.242477 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:49:09.242428 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-skc26_6713665c-3b71-4751-929e-16de48263766/konnectivity-agent/0.log" limit=1024 Jul 21 12:49:09.358261 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:49:09.358226 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-skc26_6713665c-3b71-4751-929e-16de48263766/konnectivity-agent/0.log" limit=1024 Jul 21 12:49:11.694633 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:49:11.694595 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-skc26_6713665c-3b71-4751-929e-16de48263766/konnectivity-agent/0.log" limit=1024 Jul 21 12:49:11.805595 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:49:11.805560 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-skc26_6713665c-3b71-4751-929e-16de48263766/konnectivity-agent/0.log" limit=1024 Jul 21 12:49:15.431975 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:49:15.431935 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-skc26_6713665c-3b71-4751-929e-16de48263766/konnectivity-agent/0.log" limit=1024 Jul 21 12:49:15.537933 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:49:15.537895 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-skc26_6713665c-3b71-4751-929e-16de48263766/konnectivity-agent/0.log" limit=1024 Jul 21 12:49:17.403570 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:49:17.403525 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-skc26_6713665c-3b71-4751-929e-16de48263766/konnectivity-agent/0.log" limit=1024 Jul 21 12:49:17.469583 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:49:17.469554 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-skc26_6713665c-3b71-4751-929e-16de48263766/konnectivity-agent/0.log" limit=1024 Jul 21 12:49:20.476807 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:49:20.476770 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-skc26_6713665c-3b71-4751-929e-16de48263766/konnectivity-agent/0.log" limit=1024 Jul 21 12:49:21.210703 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:49:21.210670 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-skc26_6713665c-3b71-4751-929e-16de48263766/konnectivity-agent/0.log" limit=1024 Jul 21 12:49:21.598081 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:49:21.597989 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-skc26_6713665c-3b71-4751-929e-16de48263766/konnectivity-agent/0.log" limit=1024 Jul 21 12:49:23.181102 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:49:23.181070 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-skc26_6713665c-3b71-4751-929e-16de48263766/konnectivity-agent/0.log" limit=1024 Jul 21 12:49:23.330666 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:49:23.330634 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-skc26_6713665c-3b71-4751-929e-16de48263766/konnectivity-agent/0.log" limit=1024 Jul 21 12:49:23.475409 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:49:23.475329 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-skc26_6713665c-3b71-4751-929e-16de48263766/konnectivity-agent/0.log" limit=1024 Jul 21 12:49:23.822850 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:49:23.822778 2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-8pmg8"] Jul 21 12:49:23.835958 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:49:23.835939 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-54d6d988d5-8pmg8" Jul 21 12:49:23.840069 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:49:23.840052 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"limitador-limits-config-limitador\"" Jul 21 12:49:23.853939 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:49:23.853913 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-8pmg8"] Jul 21 12:49:23.917213 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:49:23.917184 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-8pmg8"] Jul 21 12:49:23.967308 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:49:23.967279 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2kdm5\" (UniqueName: \"kubernetes.io/projected/7225c4af-0114-4819-8db0-a306a9f04752-kube-api-access-2kdm5\") pod \"limitador-limitador-54d6d988d5-8pmg8\" (UID: \"7225c4af-0114-4819-8db0-a306a9f04752\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-8pmg8" Jul 21 12:49:23.967421 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:49:23.967335 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/7225c4af-0114-4819-8db0-a306a9f04752-config-file\") pod \"limitador-limitador-54d6d988d5-8pmg8\" (UID: \"7225c4af-0114-4819-8db0-a306a9f04752\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-8pmg8" Jul 21 12:49:24.068370 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:49:24.068344 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/7225c4af-0114-4819-8db0-a306a9f04752-config-file\") pod \"limitador-limitador-54d6d988d5-8pmg8\" (UID: \"7225c4af-0114-4819-8db0-a306a9f04752\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-8pmg8" Jul 21 12:49:24.068472 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:49:24.068388 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2kdm5\" (UniqueName: \"kubernetes.io/projected/7225c4af-0114-4819-8db0-a306a9f04752-kube-api-access-2kdm5\") pod \"limitador-limitador-54d6d988d5-8pmg8\" (UID: \"7225c4af-0114-4819-8db0-a306a9f04752\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-8pmg8" Jul 21 12:49:24.068988 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:49:24.068969 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/7225c4af-0114-4819-8db0-a306a9f04752-config-file\") pod \"limitador-limitador-54d6d988d5-8pmg8\" (UID: \"7225c4af-0114-4819-8db0-a306a9f04752\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-8pmg8" Jul 21 12:49:24.085431 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:49:24.085372 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2kdm5\" (UniqueName: \"kubernetes.io/projected/7225c4af-0114-4819-8db0-a306a9f04752-kube-api-access-2kdm5\") pod \"limitador-limitador-54d6d988d5-8pmg8\" (UID: \"7225c4af-0114-4819-8db0-a306a9f04752\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-8pmg8" Jul 21 12:49:24.145645 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:49:24.145622 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-54d6d988d5-8pmg8" Jul 21 12:49:24.282930 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:49:24.282908 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-8pmg8"] Jul 21 12:49:24.285150 ip-10-0-130-157 kubenswrapper[2576]: W0721 12:49:24.285112 2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7225c4af_0114_4819_8db0_a306a9f04752.slice/crio-cc39da80f1e3860badea1989b7c06d0b5c24fa0020b1c0fbbac81799b4e17a3c WatchSource:0}: Error finding container cc39da80f1e3860badea1989b7c06d0b5c24fa0020b1c0fbbac81799b4e17a3c: Status 404 returned error can't find the container with id cc39da80f1e3860badea1989b7c06d0b5c24fa0020b1c0fbbac81799b4e17a3c Jul 21 12:49:24.394709 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:49:24.394673 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-skc26_6713665c-3b71-4751-929e-16de48263766/konnectivity-agent/0.log" limit=1024 Jul 21 12:49:24.414568 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:49:24.414539 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-54d6d988d5-8pmg8" event={"ID":"7225c4af-0114-4819-8db0-a306a9f04752","Type":"ContainerStarted","Data":"cc39da80f1e3860badea1989b7c06d0b5c24fa0020b1c0fbbac81799b4e17a3c"} Jul 21 12:49:24.493088 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:49:24.493056 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-skc26_6713665c-3b71-4751-929e-16de48263766/konnectivity-agent/0.log" limit=1024 Jul 21 12:49:24.838841 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:49:24.838741 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-skc26_6713665c-3b71-4751-929e-16de48263766/konnectivity-agent/0.log" limit=1024 Jul 21 12:49:24.879730 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:49:24.879695 2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-75d556bd77-mz6gn"] Jul 21 12:49:24.883111 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:49:24.883090 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-75d556bd77-mz6gn" Jul 21 12:49:24.887199 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:49:24.887177 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-authorino-dockercfg-m9vxc\"" Jul 21 12:49:24.900124 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:49:24.899844 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-75d556bd77-mz6gn"] Jul 21 12:49:24.953981 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:49:24.953911 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-skc26_6713665c-3b71-4751-929e-16de48263766/konnectivity-agent/0.log" limit=1024 Jul 21 12:49:24.977722 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:49:24.977692 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-g5xjx\" (UniqueName: \"kubernetes.io/projected/f9927c89-68b9-49a5-aad3-e83b5b6763d1-kube-api-access-g5xjx\") pod \"authorino-75d556bd77-mz6gn\" (UID: \"f9927c89-68b9-49a5-aad3-e83b5b6763d1\") " pod="kuadrant-system/authorino-75d556bd77-mz6gn" Jul 21 12:49:25.078370 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:49:25.078348 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-g5xjx\" (UniqueName: \"kubernetes.io/projected/f9927c89-68b9-49a5-aad3-e83b5b6763d1-kube-api-access-g5xjx\") pod \"authorino-75d556bd77-mz6gn\" (UID: \"f9927c89-68b9-49a5-aad3-e83b5b6763d1\") " pod="kuadrant-system/authorino-75d556bd77-mz6gn" Jul 21 12:49:25.090173 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:49:25.090119 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-g5xjx\" (UniqueName: \"kubernetes.io/projected/f9927c89-68b9-49a5-aad3-e83b5b6763d1-kube-api-access-g5xjx\") pod \"authorino-75d556bd77-mz6gn\" (UID: \"f9927c89-68b9-49a5-aad3-e83b5b6763d1\") " pod="kuadrant-system/authorino-75d556bd77-mz6gn" Jul 21 12:49:25.195292 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:49:25.195264 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-75d556bd77-mz6gn" Jul 21 12:49:25.341725 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:49:25.341631 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-75d556bd77-mz6gn"] Jul 21 12:49:25.346276 ip-10-0-130-157 kubenswrapper[2576]: W0721 12:49:25.346245 2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podf9927c89_68b9_49a5_aad3_e83b5b6763d1.slice/crio-6fc27362fc80e989821d303c63a4b63f6c233ef396b41c111b2d74f7e69e1a9b WatchSource:0}: Error finding container 6fc27362fc80e989821d303c63a4b63f6c233ef396b41c111b2d74f7e69e1a9b: Status 404 returned error can't find the container with id 6fc27362fc80e989821d303c63a4b63f6c233ef396b41c111b2d74f7e69e1a9b Jul 21 12:49:25.419818 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:49:25.419774 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-75d556bd77-mz6gn" event={"ID":"f9927c89-68b9-49a5-aad3-e83b5b6763d1","Type":"ContainerStarted","Data":"6fc27362fc80e989821d303c63a4b63f6c233ef396b41c111b2d74f7e69e1a9b"} Jul 21 12:49:27.432613 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:49:27.432573 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-54d6d988d5-8pmg8" event={"ID":"7225c4af-0114-4819-8db0-a306a9f04752","Type":"ContainerStarted","Data":"a844329970912a7ec00d888dc3bc9cc52f1a7755e34289468a38b6a9c4fb4fb0"} Jul 21 12:49:27.433049 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:49:27.432704 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-limitador-54d6d988d5-8pmg8" Jul 21 12:49:27.453555 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:49:27.453497 2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-limitador-54d6d988d5-8pmg8" podStartSLOduration=1.79562493 podStartE2EDuration="4.453481919s" totalImagesPullingTime="2.657856989s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 12:49:23 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 12:49:27.451986523 +0000 UTC m=+575.287250403" watchObservedRunningTime="2026-07-21 12:49:27.453481919 +0000 UTC m=+575.288745784" Jul 21 12:49:29.442759 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:49:29.442715 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-75d556bd77-mz6gn" event={"ID":"f9927c89-68b9-49a5-aad3-e83b5b6763d1","Type":"ContainerStarted","Data":"1d0d18e86383d6412379ac9e39e96632ddcf503918caeff7b7e7f53af3e7b82b"} Jul 21 12:49:29.463887 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:49:29.463842 2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-75d556bd77-mz6gn" podStartSLOduration=1.778051944 podStartE2EDuration="5.463827413s" totalImagesPullingTime="3.685775469s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 12:49:24 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 12:49:29.460994085 +0000 UTC m=+577.296257948" watchObservedRunningTime="2026-07-21 12:49:29.463827413 +0000 UTC m=+577.299091278" Jul 21 12:49:30.657092 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:49:30.657028 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-skc26_6713665c-3b71-4751-929e-16de48263766/konnectivity-agent/0.log" limit=1024 Jul 21 12:49:30.730968 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:49:30.730931 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-skc26_6713665c-3b71-4751-929e-16de48263766/konnectivity-agent/0.log" limit=1024 Jul 21 12:49:31.885841 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:49:31.885804 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-skc26_6713665c-3b71-4751-929e-16de48263766/konnectivity-agent/0.log" limit=1024 Jul 21 12:49:31.969854 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:49:31.969826 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-skc26_6713665c-3b71-4751-929e-16de48263766/konnectivity-agent/0.log" limit=1024 Jul 21 12:49:33.762093 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:49:33.762060 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-skc26_6713665c-3b71-4751-929e-16de48263766/konnectivity-agent/0.log" limit=1024 Jul 21 12:49:33.858599 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:49:33.858567 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-skc26_6713665c-3b71-4751-929e-16de48263766/konnectivity-agent/0.log" limit=1024 Jul 21 12:49:36.286177 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:49:36.286147 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-skc26_6713665c-3b71-4751-929e-16de48263766/konnectivity-agent/0.log" limit=1024 Jul 21 12:49:36.391311 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:49:36.391281 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-skc26_6713665c-3b71-4751-929e-16de48263766/konnectivity-agent/0.log" limit=1024 Jul 21 12:49:38.438295 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:49:38.438265 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-limitador-54d6d988d5-8pmg8" Jul 21 12:49:44.254791 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:49:44.254738 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-skc26_6713665c-3b71-4751-929e-16de48263766/konnectivity-agent/0.log" limit=1024 Jul 21 12:49:44.346837 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:49:44.346809 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-skc26_6713665c-3b71-4751-929e-16de48263766/konnectivity-agent/0.log" limit=1024 Jul 21 12:49:44.730074 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:49:44.730045 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-skc26_6713665c-3b71-4751-929e-16de48263766/konnectivity-agent/0.log" limit=1024 Jul 21 12:49:53.653440 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:49:53.653400 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-6nf7d_a2b48106-9844-4431-9b6f-d4e1baacb15b/ovn-acl-logging/0.log" Jul 21 12:49:53.656646 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:49:53.656623 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-6nf7d_a2b48106-9844-4431-9b6f-d4e1baacb15b/ovn-acl-logging/0.log" Jul 21 12:49:56.370568 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:49:56.370524 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-skc26_6713665c-3b71-4751-929e-16de48263766/konnectivity-agent/0.log" limit=1024 Jul 21 12:49:56.691457 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:49:56.691430 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-skc26_6713665c-3b71-4751-929e-16de48263766/konnectivity-agent/0.log" limit=1024 Jul 21 12:49:58.150426 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:49:58.150397 2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-75d556bd77-mz6gn"] Jul 21 12:49:58.152842 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:49:58.150589 2576 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/authorino-75d556bd77-mz6gn" podUID="f9927c89-68b9-49a5-aad3-e83b5b6763d1" containerName="authorino" containerID="cri-o://1d0d18e86383d6412379ac9e39e96632ddcf503918caeff7b7e7f53af3e7b82b" gracePeriod=30 Jul 21 12:49:58.392305 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:49:58.392281 2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-75d556bd77-mz6gn" Jul 21 12:49:58.420600 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:49:58.420527 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-g5xjx\" (UniqueName: \"kubernetes.io/projected/f9927c89-68b9-49a5-aad3-e83b5b6763d1-kube-api-access-g5xjx\") pod \"f9927c89-68b9-49a5-aad3-e83b5b6763d1\" (UID: \"f9927c89-68b9-49a5-aad3-e83b5b6763d1\") " Jul 21 12:49:58.422687 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:49:58.422649 2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f9927c89-68b9-49a5-aad3-e83b5b6763d1-kube-api-access-g5xjx" (OuterVolumeSpecName: "kube-api-access-g5xjx") pod "f9927c89-68b9-49a5-aad3-e83b5b6763d1" (UID: "f9927c89-68b9-49a5-aad3-e83b5b6763d1"). InnerVolumeSpecName "kube-api-access-g5xjx". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 21 12:49:58.521352 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:49:58.521320 2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-g5xjx\" (UniqueName: \"kubernetes.io/projected/f9927c89-68b9-49a5-aad3-e83b5b6763d1-kube-api-access-g5xjx\") on node \"ip-10-0-130-157.ec2.internal\" DevicePath \"\"" Jul 21 12:49:58.558869 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:49:58.558838 2576 generic.go:356] "Generic (PLEG): container finished" podID="f9927c89-68b9-49a5-aad3-e83b5b6763d1" containerID="1d0d18e86383d6412379ac9e39e96632ddcf503918caeff7b7e7f53af3e7b82b" exitCode=0 Jul 21 12:49:58.558987 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:49:58.558876 2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-75d556bd77-mz6gn" Jul 21 12:49:58.558987 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:49:58.558901 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-75d556bd77-mz6gn" event={"ID":"f9927c89-68b9-49a5-aad3-e83b5b6763d1","Type":"ContainerDied","Data":"1d0d18e86383d6412379ac9e39e96632ddcf503918caeff7b7e7f53af3e7b82b"} Jul 21 12:49:58.558987 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:49:58.558930 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-75d556bd77-mz6gn" event={"ID":"f9927c89-68b9-49a5-aad3-e83b5b6763d1","Type":"ContainerDied","Data":"6fc27362fc80e989821d303c63a4b63f6c233ef396b41c111b2d74f7e69e1a9b"} Jul 21 12:49:58.558987 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:49:58.558947 2576 scope.go:117] "RemoveContainer" containerID="1d0d18e86383d6412379ac9e39e96632ddcf503918caeff7b7e7f53af3e7b82b" Jul 21 12:49:58.567275 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:49:58.567259 2576 scope.go:117] "RemoveContainer" containerID="1d0d18e86383d6412379ac9e39e96632ddcf503918caeff7b7e7f53af3e7b82b" Jul 21 12:49:58.567550 ip-10-0-130-157 kubenswrapper[2576]: E0721 12:49:58.567507 2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1d0d18e86383d6412379ac9e39e96632ddcf503918caeff7b7e7f53af3e7b82b\": container with ID starting with 1d0d18e86383d6412379ac9e39e96632ddcf503918caeff7b7e7f53af3e7b82b not found: ID does not exist" containerID="1d0d18e86383d6412379ac9e39e96632ddcf503918caeff7b7e7f53af3e7b82b" Jul 21 12:49:58.567605 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:49:58.567560 2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1d0d18e86383d6412379ac9e39e96632ddcf503918caeff7b7e7f53af3e7b82b"} err="failed to get container status \"1d0d18e86383d6412379ac9e39e96632ddcf503918caeff7b7e7f53af3e7b82b\": rpc error: code = NotFound desc = could not find container \"1d0d18e86383d6412379ac9e39e96632ddcf503918caeff7b7e7f53af3e7b82b\": container with ID starting with 1d0d18e86383d6412379ac9e39e96632ddcf503918caeff7b7e7f53af3e7b82b not found: ID does not exist" Jul 21 12:49:58.579628 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:49:58.579605 2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-75d556bd77-mz6gn"] Jul 21 12:49:58.588886 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:49:58.585157 2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-75d556bd77-mz6gn"] Jul 21 12:50:00.315189 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:50:00.315148 2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f9927c89-68b9-49a5-aad3-e83b5b6763d1" path="/var/lib/kubelet/pods/f9927c89-68b9-49a5-aad3-e83b5b6763d1/volumes" Jul 21 12:50:03.479741 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:50:03.479707 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-skc26_6713665c-3b71-4751-929e-16de48263766/konnectivity-agent/0.log" limit=1024 Jul 21 12:50:05.404542 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:50:05.404490 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-skc26_6713665c-3b71-4751-929e-16de48263766/konnectivity-agent/0.log" limit=1024 Jul 21 12:50:06.224917 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:50:06.224877 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-skc26_6713665c-3b71-4751-929e-16de48263766/konnectivity-agent/0.log" limit=1024 Jul 21 12:50:06.328555 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:50:06.328502 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-skc26_6713665c-3b71-4751-929e-16de48263766/konnectivity-agent/0.log" limit=1024 Jul 21 12:50:34.752128 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:50:34.752037 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-skc26_6713665c-3b71-4751-929e-16de48263766/konnectivity-agent/0.log" limit=1024 Jul 21 12:50:34.850480 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:50:34.850448 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-skc26_6713665c-3b71-4751-929e-16de48263766/konnectivity-agent/0.log" limit=1024 Jul 21 12:50:35.418204 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:50:35.418175 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-skc26_6713665c-3b71-4751-929e-16de48263766/konnectivity-agent/0.log" limit=1024 Jul 21 12:50:35.789346 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:50:35.789248 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-skc26_6713665c-3b71-4751-929e-16de48263766/konnectivity-agent/0.log" limit=1024 Jul 21 12:50:35.953737 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:50:35.953711 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-skc26_6713665c-3b71-4751-929e-16de48263766/konnectivity-agent/0.log" limit=1024 Jul 21 12:50:36.050410 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:50:36.050342 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-skc26_6713665c-3b71-4751-929e-16de48263766/konnectivity-agent/0.log" limit=1024 Jul 21 12:50:36.199837 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:50:36.199804 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-8pmg8"] Jul 21 12:50:47.414322 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:50:47.414296 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-skc26_6713665c-3b71-4751-929e-16de48263766/konnectivity-agent/0.log" limit=1024 Jul 21 12:50:47.504492 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:50:47.504458 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-skc26_6713665c-3b71-4751-929e-16de48263766/konnectivity-agent/0.log" limit=1024 Jul 21 12:50:59.647637 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:50:59.647596 2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-pvmgz"] Jul 21 12:50:59.648331 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:50:59.648294 2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f9927c89-68b9-49a5-aad3-e83b5b6763d1" containerName="authorino" Jul 21 12:50:59.648424 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:50:59.648335 2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="f9927c89-68b9-49a5-aad3-e83b5b6763d1" containerName="authorino" Jul 21 12:50:59.648546 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:50:59.648531 2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="f9927c89-68b9-49a5-aad3-e83b5b6763d1" containerName="authorino" Jul 21 12:50:59.652146 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:50:59.652120 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-pvmgz" Jul 21 12:50:59.654560 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:50:59.654538 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"kube-root-ca.crt\"" Jul 21 12:50:59.654679 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:50:59.654602 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"openshift-service-ca.crt\"" Jul 21 12:50:59.655042 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:50:59.655027 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-distinct-2-simulated-kserve-self-signed-certs\"" Jul 21 12:50:59.655129 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:50:59.655032 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"default-dockercfg-gbts6\"" Jul 21 12:50:59.660642 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:50:59.660622 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-pvmgz"] Jul 21 12:50:59.684719 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:50:59.684694 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-skc26_6713665c-3b71-4751-929e-16de48263766/konnectivity-agent/0.log" limit=1024 Jul 21 12:50:59.767287 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:50:59.767258 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9vnzs\" (UniqueName: \"kubernetes.io/projected/124c966d-681b-4916-8c7f-e2c065462afc-kube-api-access-9vnzs\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-pvmgz\" (UID: \"124c966d-681b-4916-8c7f-e2c065462afc\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-pvmgz" Jul 21 12:50:59.767411 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:50:59.767307 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/124c966d-681b-4916-8c7f-e2c065462afc-home\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-pvmgz\" (UID: \"124c966d-681b-4916-8c7f-e2c065462afc\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-pvmgz" Jul 21 12:50:59.767411 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:50:59.767367 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/124c966d-681b-4916-8c7f-e2c065462afc-dshm\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-pvmgz\" (UID: \"124c966d-681b-4916-8c7f-e2c065462afc\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-pvmgz" Jul 21 12:50:59.767411 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:50:59.767394 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/124c966d-681b-4916-8c7f-e2c065462afc-tmp-dir\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-pvmgz\" (UID: \"124c966d-681b-4916-8c7f-e2c065462afc\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-pvmgz" Jul 21 12:50:59.767586 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:50:59.767423 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/124c966d-681b-4916-8c7f-e2c065462afc-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-pvmgz\" (UID: \"124c966d-681b-4916-8c7f-e2c065462afc\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-pvmgz" Jul 21 12:50:59.767586 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:50:59.767452 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/124c966d-681b-4916-8c7f-e2c065462afc-model-cache\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-pvmgz\" (UID: \"124c966d-681b-4916-8c7f-e2c065462afc\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-pvmgz" Jul 21 12:50:59.868695 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:50:59.868672 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/124c966d-681b-4916-8c7f-e2c065462afc-home\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-pvmgz\" (UID: \"124c966d-681b-4916-8c7f-e2c065462afc\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-pvmgz" Jul 21 12:50:59.868801 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:50:59.868708 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/124c966d-681b-4916-8c7f-e2c065462afc-dshm\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-pvmgz\" (UID: \"124c966d-681b-4916-8c7f-e2c065462afc\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-pvmgz" Jul 21 12:50:59.868801 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:50:59.868726 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/124c966d-681b-4916-8c7f-e2c065462afc-tmp-dir\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-pvmgz\" (UID: \"124c966d-681b-4916-8c7f-e2c065462afc\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-pvmgz" Jul 21 12:50:59.868801 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:50:59.868746 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/124c966d-681b-4916-8c7f-e2c065462afc-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-pvmgz\" (UID: \"124c966d-681b-4916-8c7f-e2c065462afc\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-pvmgz" Jul 21 12:50:59.868801 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:50:59.868773 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/124c966d-681b-4916-8c7f-e2c065462afc-model-cache\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-pvmgz\" (UID: \"124c966d-681b-4916-8c7f-e2c065462afc\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-pvmgz" Jul 21 12:50:59.868970 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:50:59.868827 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9vnzs\" (UniqueName: \"kubernetes.io/projected/124c966d-681b-4916-8c7f-e2c065462afc-kube-api-access-9vnzs\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-pvmgz\" (UID: \"124c966d-681b-4916-8c7f-e2c065462afc\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-pvmgz" Jul 21 12:50:59.869119 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:50:59.869099 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/124c966d-681b-4916-8c7f-e2c065462afc-home\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-pvmgz\" (UID: \"124c966d-681b-4916-8c7f-e2c065462afc\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-pvmgz" Jul 21 12:50:59.869187 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:50:59.869115 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/124c966d-681b-4916-8c7f-e2c065462afc-tmp-dir\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-pvmgz\" (UID: \"124c966d-681b-4916-8c7f-e2c065462afc\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-pvmgz" Jul 21 12:50:59.869187 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:50:59.869145 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/124c966d-681b-4916-8c7f-e2c065462afc-model-cache\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-pvmgz\" (UID: \"124c966d-681b-4916-8c7f-e2c065462afc\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-pvmgz" Jul 21 12:50:59.871020 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:50:59.870999 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/124c966d-681b-4916-8c7f-e2c065462afc-dshm\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-pvmgz\" (UID: \"124c966d-681b-4916-8c7f-e2c065462afc\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-pvmgz" Jul 21 12:50:59.871263 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:50:59.871247 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/124c966d-681b-4916-8c7f-e2c065462afc-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-pvmgz\" (UID: \"124c966d-681b-4916-8c7f-e2c065462afc\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-pvmgz" Jul 21 12:50:59.878569 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:50:59.878550 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9vnzs\" (UniqueName: \"kubernetes.io/projected/124c966d-681b-4916-8c7f-e2c065462afc-kube-api-access-9vnzs\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-pvmgz\" (UID: \"124c966d-681b-4916-8c7f-e2c065462afc\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-pvmgz" Jul 21 12:50:59.964150 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:50:59.964094 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-pvmgz" Jul 21 12:51:00.098850 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:51:00.098820 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-pvmgz"] Jul 21 12:51:00.101150 ip-10-0-130-157 kubenswrapper[2576]: W0721 12:51:00.101123 2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod124c966d_681b_4916_8c7f_e2c065462afc.slice/crio-646961ece13fb3a0b05a2321fd6f2cb4aa128c12e6b09294de536ab3869d7224 WatchSource:0}: Error finding container 646961ece13fb3a0b05a2321fd6f2cb4aa128c12e6b09294de536ab3869d7224: Status 404 returned error can't find the container with id 646961ece13fb3a0b05a2321fd6f2cb4aa128c12e6b09294de536ab3869d7224 Jul 21 12:51:00.280505 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:51:00.280442 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-8pmg8"] Jul 21 12:51:00.815913 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:51:00.815869 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-pvmgz" event={"ID":"124c966d-681b-4916-8c7f-e2c065462afc","Type":"ContainerStarted","Data":"646961ece13fb3a0b05a2321fd6f2cb4aa128c12e6b09294de536ab3869d7224"} Jul 21 12:51:02.560107 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:51:02.560060 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-skc26_6713665c-3b71-4751-929e-16de48263766/konnectivity-agent/0.log" limit=1024 Jul 21 12:51:03.291015 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:51:03.290983 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-8pmg8"] Jul 21 12:51:09.183979 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:51:09.183937 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-skc26_6713665c-3b71-4751-929e-16de48263766/konnectivity-agent/0.log" limit=1024 Jul 21 12:51:11.859350 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:51:11.859310 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-pvmgz" event={"ID":"124c966d-681b-4916-8c7f-e2c065462afc","Type":"ContainerStarted","Data":"de2fb3714d8ab385993b720354426ad866f49ba21066ada343f13b14495df416"} Jul 21 12:51:11.883065 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:51:11.883015 2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-pvmgz" podStartSLOduration=1.780707118 podStartE2EDuration="12.883000581s" totalImagesPullingTime="11.102293463s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 12:50:59 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 12:51:11.882123201 +0000 UTC m=+679.717387067" watchObservedRunningTime="2026-07-21 12:51:11.883000581 +0000 UTC m=+679.718264448" Jul 21 12:51:18.884862 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:51:18.884815 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-skc26_6713665c-3b71-4751-929e-16de48263766/konnectivity-agent/0.log" limit=1024 Jul 21 12:51:18.986315 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:51:18.986280 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-skc26_6713665c-3b71-4751-929e-16de48263766/konnectivity-agent/0.log" limit=1024 Jul 21 12:51:19.454954 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:51:19.454923 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-skc26_6713665c-3b71-4751-929e-16de48263766/konnectivity-agent/0.log" limit=1024 Jul 21 12:51:19.965280 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:51:19.965251 2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-pvmgz" Jul 21 12:51:19.965680 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:51:19.965335 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-pvmgz" Jul 21 12:51:19.978169 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:51:19.978145 2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-pvmgz" Jul 21 12:51:20.493083 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:51:20.493054 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-8pmg8"] Jul 21 12:51:20.907170 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:51:20.907137 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-pvmgz" Jul 21 12:51:24.600579 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:51:24.600546 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-skc26_6713665c-3b71-4751-929e-16de48263766/konnectivity-agent/0.log" limit=1024 Jul 21 12:51:26.289205 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:51:26.289166 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-8pmg8"] Jul 21 12:51:39.181884 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:51:39.181854 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-skc26_6713665c-3b71-4751-929e-16de48263766/konnectivity-agent/0.log" limit=1024 Jul 21 12:51:41.091812 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:51:41.091774 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-8pmg8"] Jul 21 12:51:41.452051 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:51:41.452012 2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-57fc66f697-5r52s"] Jul 21 12:51:41.456854 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:51:41.456832 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-5r52s" Jul 21 12:51:41.459164 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:51:41.459147 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"premium-simulated-simulated-premium-kserve-self-signed-certs\"" Jul 21 12:51:41.466918 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:51:41.466893 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-57fc66f697-5r52s"] Jul 21 12:51:41.501088 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:51:41.501060 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-skc26_6713665c-3b71-4751-929e-16de48263766/konnectivity-agent/0.log" limit=1024 Jul 21 12:51:41.583956 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:51:41.583932 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f02a054b-0324-4744-b923-1cb3be68fcb4-home\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-5r52s\" (UID: \"f02a054b-0324-4744-b923-1cb3be68fcb4\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-5r52s" Jul 21 12:51:41.584052 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:51:41.583962 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f02a054b-0324-4744-b923-1cb3be68fcb4-dshm\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-5r52s\" (UID: \"f02a054b-0324-4744-b923-1cb3be68fcb4\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-5r52s" Jul 21 12:51:41.584052 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:51:41.583990 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f02a054b-0324-4744-b923-1cb3be68fcb4-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-5r52s\" (UID: \"f02a054b-0324-4744-b923-1cb3be68fcb4\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-5r52s" Jul 21 12:51:41.584052 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:51:41.584043 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f02a054b-0324-4744-b923-1cb3be68fcb4-tmp-dir\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-5r52s\" (UID: \"f02a054b-0324-4744-b923-1cb3be68fcb4\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-5r52s" Jul 21 12:51:41.584181 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:51:41.584123 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f02a054b-0324-4744-b923-1cb3be68fcb4-model-cache\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-5r52s\" (UID: \"f02a054b-0324-4744-b923-1cb3be68fcb4\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-5r52s" Jul 21 12:51:41.584181 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:51:41.584167 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xdq2p\" (UniqueName: \"kubernetes.io/projected/f02a054b-0324-4744-b923-1cb3be68fcb4-kube-api-access-xdq2p\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-5r52s\" (UID: \"f02a054b-0324-4744-b923-1cb3be68fcb4\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-5r52s" Jul 21 12:51:41.685489 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:51:41.685466 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f02a054b-0324-4744-b923-1cb3be68fcb4-home\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-5r52s\" (UID: \"f02a054b-0324-4744-b923-1cb3be68fcb4\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-5r52s" Jul 21 12:51:41.685585 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:51:41.685494 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f02a054b-0324-4744-b923-1cb3be68fcb4-dshm\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-5r52s\" (UID: \"f02a054b-0324-4744-b923-1cb3be68fcb4\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-5r52s" Jul 21 12:51:41.685647 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:51:41.685628 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f02a054b-0324-4744-b923-1cb3be68fcb4-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-5r52s\" (UID: \"f02a054b-0324-4744-b923-1cb3be68fcb4\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-5r52s" Jul 21 12:51:41.685700 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:51:41.685683 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f02a054b-0324-4744-b923-1cb3be68fcb4-tmp-dir\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-5r52s\" (UID: \"f02a054b-0324-4744-b923-1cb3be68fcb4\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-5r52s" Jul 21 12:51:41.685741 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:51:41.685722 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f02a054b-0324-4744-b923-1cb3be68fcb4-model-cache\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-5r52s\" (UID: \"f02a054b-0324-4744-b923-1cb3be68fcb4\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-5r52s" Jul 21 12:51:41.685785 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:51:41.685761 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xdq2p\" (UniqueName: \"kubernetes.io/projected/f02a054b-0324-4744-b923-1cb3be68fcb4-kube-api-access-xdq2p\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-5r52s\" (UID: \"f02a054b-0324-4744-b923-1cb3be68fcb4\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-5r52s" Jul 21 12:51:41.685869 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:51:41.685849 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f02a054b-0324-4744-b923-1cb3be68fcb4-home\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-5r52s\" (UID: \"f02a054b-0324-4744-b923-1cb3be68fcb4\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-5r52s" Jul 21 12:51:41.685949 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:51:41.685931 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f02a054b-0324-4744-b923-1cb3be68fcb4-tmp-dir\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-5r52s\" (UID: \"f02a054b-0324-4744-b923-1cb3be68fcb4\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-5r52s" Jul 21 12:51:41.686087 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:51:41.686068 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f02a054b-0324-4744-b923-1cb3be68fcb4-model-cache\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-5r52s\" (UID: \"f02a054b-0324-4744-b923-1cb3be68fcb4\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-5r52s" Jul 21 12:51:41.687870 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:51:41.687847 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f02a054b-0324-4744-b923-1cb3be68fcb4-dshm\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-5r52s\" (UID: \"f02a054b-0324-4744-b923-1cb3be68fcb4\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-5r52s" Jul 21 12:51:41.688067 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:51:41.688050 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f02a054b-0324-4744-b923-1cb3be68fcb4-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-5r52s\" (UID: \"f02a054b-0324-4744-b923-1cb3be68fcb4\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-5r52s" Jul 21 12:51:41.694627 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:51:41.694605 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xdq2p\" (UniqueName: \"kubernetes.io/projected/f02a054b-0324-4744-b923-1cb3be68fcb4-kube-api-access-xdq2p\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-5r52s\" (UID: \"f02a054b-0324-4744-b923-1cb3be68fcb4\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-5r52s" Jul 21 12:51:41.766419 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:51:41.766367 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-5r52s" Jul 21 12:51:41.898232 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:51:41.898133 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-57fc66f697-5r52s"] Jul 21 12:51:41.900785 ip-10-0-130-157 kubenswrapper[2576]: W0721 12:51:41.900746 2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podf02a054b_0324_4744_b923_1cb3be68fcb4.slice/crio-dbe2462e3fd5eb5177d76b1d8ac5bf478dbda13efa416af5b77f41f30d590be2 WatchSource:0}: Error finding container dbe2462e3fd5eb5177d76b1d8ac5bf478dbda13efa416af5b77f41f30d590be2: Status 404 returned error can't find the container with id dbe2462e3fd5eb5177d76b1d8ac5bf478dbda13efa416af5b77f41f30d590be2 Jul 21 12:51:41.902296 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:51:41.902279 2576 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 21 12:51:41.984352 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:51:41.984314 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-5r52s" event={"ID":"f02a054b-0324-4744-b923-1cb3be68fcb4","Type":"ContainerStarted","Data":"dbe2462e3fd5eb5177d76b1d8ac5bf478dbda13efa416af5b77f41f30d590be2"} Jul 21 12:51:49.013647 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:51:49.013611 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-5r52s" event={"ID":"f02a054b-0324-4744-b923-1cb3be68fcb4","Type":"ContainerStarted","Data":"29fa1331cff0ee8c8f03aed566ce62fc565544a645dd860fd3b11d0cfa507e0c"} Jul 21 12:51:49.033781 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:51:49.033732 2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-5r52s" podStartSLOduration=1.042889913 podStartE2EDuration="8.033717677s" totalImagesPullingTime="6.990827764s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 12:51:41 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 12:51:49.033269351 +0000 UTC m=+716.868533218" watchObservedRunningTime="2026-07-21 12:51:49.033717677 +0000 UTC m=+716.868981542" Jul 21 12:51:51.767109 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:51:51.767073 2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-5r52s" Jul 21 12:51:51.767540 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:51:51.767122 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-5r52s" Jul 21 12:51:51.779168 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:51:51.779139 2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-5r52s" Jul 21 12:51:52.036563 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:51:52.036469 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-5r52s" Jul 21 12:51:52.484803 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:51:52.484774 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-8pmg8"] Jul 21 12:52:59.430424 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:52:59.430382 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-skc26_6713665c-3b71-4751-929e-16de48263766/konnectivity-agent/0.log" limit=1024 Jul 21 12:53:00.123179 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:53:00.123137 2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-5597f86487-f7kvl"] Jul 21 12:53:00.126333 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:53:00.126315 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5597f86487-f7kvl" Jul 21 12:53:00.129977 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:53:00.129955 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-authorino-dockercfg-m9vxc\"" Jul 21 12:53:00.130088 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:53:00.130068 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-server-cert\"" Jul 21 12:53:00.136747 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:53:00.136726 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-5597f86487-f7kvl"] Jul 21 12:53:00.254448 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:53:00.254424 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7bhwd\" (UniqueName: \"kubernetes.io/projected/3eabb5f5-46c3-4820-a666-4036a4afbba0-kube-api-access-7bhwd\") pod \"authorino-5597f86487-f7kvl\" (UID: \"3eabb5f5-46c3-4820-a666-4036a4afbba0\") " pod="kuadrant-system/authorino-5597f86487-f7kvl" Jul 21 12:53:00.254565 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:53:00.254452 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/3eabb5f5-46c3-4820-a666-4036a4afbba0-tls-cert\") pod \"authorino-5597f86487-f7kvl\" (UID: \"3eabb5f5-46c3-4820-a666-4036a4afbba0\") " pod="kuadrant-system/authorino-5597f86487-f7kvl" Jul 21 12:53:00.355743 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:53:00.355719 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7bhwd\" (UniqueName: \"kubernetes.io/projected/3eabb5f5-46c3-4820-a666-4036a4afbba0-kube-api-access-7bhwd\") pod \"authorino-5597f86487-f7kvl\" (UID: \"3eabb5f5-46c3-4820-a666-4036a4afbba0\") " pod="kuadrant-system/authorino-5597f86487-f7kvl" Jul 21 12:53:00.355849 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:53:00.355746 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/3eabb5f5-46c3-4820-a666-4036a4afbba0-tls-cert\") pod \"authorino-5597f86487-f7kvl\" (UID: \"3eabb5f5-46c3-4820-a666-4036a4afbba0\") " pod="kuadrant-system/authorino-5597f86487-f7kvl" Jul 21 12:53:00.358179 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:53:00.358161 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/3eabb5f5-46c3-4820-a666-4036a4afbba0-tls-cert\") pod \"authorino-5597f86487-f7kvl\" (UID: \"3eabb5f5-46c3-4820-a666-4036a4afbba0\") " pod="kuadrant-system/authorino-5597f86487-f7kvl" Jul 21 12:53:00.364812 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:53:00.364790 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7bhwd\" (UniqueName: \"kubernetes.io/projected/3eabb5f5-46c3-4820-a666-4036a4afbba0-kube-api-access-7bhwd\") pod \"authorino-5597f86487-f7kvl\" (UID: \"3eabb5f5-46c3-4820-a666-4036a4afbba0\") " pod="kuadrant-system/authorino-5597f86487-f7kvl" Jul 21 12:53:00.435816 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:53:00.435794 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5597f86487-f7kvl" Jul 21 12:53:00.460528 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:53:00.460471 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-skc26_6713665c-3b71-4751-929e-16de48263766/konnectivity-agent/0.log" limit=1024 Jul 21 12:53:00.560229 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:53:00.560201 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-5597f86487-f7kvl"] Jul 21 12:53:00.561596 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:53:00.561572 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-skc26_6713665c-3b71-4751-929e-16de48263766/konnectivity-agent/0.log" limit=1024 Jul 21 12:53:00.563409 ip-10-0-130-157 kubenswrapper[2576]: W0721 12:53:00.563384 2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod3eabb5f5_46c3_4820_a666_4036a4afbba0.slice/crio-95bf69ac3e41772f8046a4309ff55e94ab8f71e677b8f7f3b82771865d393726 WatchSource:0}: Error finding container 95bf69ac3e41772f8046a4309ff55e94ab8f71e677b8f7f3b82771865d393726: Status 404 returned error can't find the container with id 95bf69ac3e41772f8046a4309ff55e94ab8f71e677b8f7f3b82771865d393726 Jul 21 12:53:01.316446 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:53:01.316412 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-5597f86487-f7kvl" event={"ID":"3eabb5f5-46c3-4820-a666-4036a4afbba0","Type":"ContainerStarted","Data":"90b77b38d035f8adfd06e61979a1769813a74b75c147d529f9d5a15c45010cba"} Jul 21 12:53:01.316446 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:53:01.316451 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-5597f86487-f7kvl" event={"ID":"3eabb5f5-46c3-4820-a666-4036a4afbba0","Type":"ContainerStarted","Data":"95bf69ac3e41772f8046a4309ff55e94ab8f71e677b8f7f3b82771865d393726"} Jul 21 12:53:01.332329 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:53:01.332271 2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-5597f86487-f7kvl" podStartSLOduration=0.744586192 podStartE2EDuration="1.332256422s" totalImagesPullingTime="587.67023ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 12:53:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 12:53:01.331317187 +0000 UTC m=+789.166581052" watchObservedRunningTime="2026-07-21 12:53:01.332256422 +0000 UTC m=+789.167520287" Jul 21 12:53:01.665462 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:53:01.665407 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-skc26_6713665c-3b71-4751-929e-16de48263766/konnectivity-agent/0.log" limit=1024 Jul 21 12:53:01.766554 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:53:01.766496 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-skc26_6713665c-3b71-4751-929e-16de48263766/konnectivity-agent/0.log" limit=1024 Jul 21 12:53:33.110149 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:53:33.110067 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-8pmg8"] Jul 21 12:53:42.990197 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:53:42.990163 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-8pmg8"] Jul 21 12:53:54.616458 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:53:54.616419 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-8pmg8"] Jul 21 12:54:10.301761 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:54:10.301727 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-skc26_6713665c-3b71-4751-929e-16de48263766/konnectivity-agent/0.log" limit=1024 Jul 21 12:54:10.402690 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:54:10.402656 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-skc26_6713665c-3b71-4751-929e-16de48263766/konnectivity-agent/0.log" limit=1024 Jul 21 12:54:24.807782 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:54:24.807747 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-skc26_6713665c-3b71-4751-929e-16de48263766/konnectivity-agent/0.log" limit=1024 Jul 21 12:54:53.681680 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:54:53.681650 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-6nf7d_a2b48106-9844-4431-9b6f-d4e1baacb15b/ovn-acl-logging/0.log" Jul 21 12:54:53.683463 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:54:53.683435 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-6nf7d_a2b48106-9844-4431-9b6f-d4e1baacb15b/ovn-acl-logging/0.log" Jul 21 12:55:10.004682 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:55:10.004641 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-skc26_6713665c-3b71-4751-929e-16de48263766/konnectivity-agent/0.log" limit=1024 Jul 21 12:55:11.504306 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:55:11.504273 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-skc26_6713665c-3b71-4751-929e-16de48263766/konnectivity-agent/0.log" limit=1024 Jul 21 12:55:33.703940 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:55:33.703902 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-8pmg8"] Jul 21 12:55:44.537227 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:55:44.537192 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-skc26_6713665c-3b71-4751-929e-16de48263766/konnectivity-agent/0.log" limit=1024 Jul 21 12:55:44.792236 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:55:44.792154 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-8pmg8"] Jul 21 12:55:50.260606 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:55:50.260572 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-skc26_6713665c-3b71-4751-929e-16de48263766/konnectivity-agent/0.log" limit=1024 Jul 21 12:55:50.356066 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:55:50.356041 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-skc26_6713665c-3b71-4751-929e-16de48263766/konnectivity-agent/0.log" limit=1024 Jul 21 12:55:50.497245 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:55:50.497216 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-skc26_6713665c-3b71-4751-929e-16de48263766/konnectivity-agent/0.log" limit=1024 Jul 21 12:55:57.080393 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:55:57.080362 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-skc26_6713665c-3b71-4751-929e-16de48263766/konnectivity-agent/0.log" limit=1024 Jul 21 12:55:57.187759 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:55:57.187724 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-skc26_6713665c-3b71-4751-929e-16de48263766/konnectivity-agent/0.log" limit=1024 Jul 21 12:55:57.285677 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:55:57.285650 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-skc26_6713665c-3b71-4751-929e-16de48263766/konnectivity-agent/0.log" limit=1024 Jul 21 12:55:57.478076 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:55:57.478045 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-skc26_6713665c-3b71-4751-929e-16de48263766/konnectivity-agent/0.log" limit=1024 Jul 21 12:56:04.387138 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:56:04.387104 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-skc26_6713665c-3b71-4751-929e-16de48263766/konnectivity-agent/0.log" limit=1024 Jul 21 12:56:04.512113 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:56:04.512079 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-skc26_6713665c-3b71-4751-929e-16de48263766/konnectivity-agent/0.log" limit=1024 Jul 21 12:56:12.963195 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:56:12.963149 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-skc26_6713665c-3b71-4751-929e-16de48263766/konnectivity-agent/0.log" limit=1024 Jul 21 12:56:22.236178 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:56:22.236102 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-8pmg8"] Jul 21 12:56:32.009754 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:56:32.004435 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-8pmg8"] Jul 21 12:56:42.696397 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:56:42.696360 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-8pmg8"] Jul 21 12:56:50.499349 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:56:50.499313 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-8pmg8"] Jul 21 12:57:01.692172 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:57:01.692132 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-8pmg8"] Jul 21 12:57:13.112507 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:57:13.112470 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-8pmg8"] Jul 21 12:57:24.191067 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:57:24.191037 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-8pmg8"] Jul 21 12:57:34.085714 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:57:34.085678 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-8pmg8"] Jul 21 12:57:59.690897 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:57:59.690821 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-8pmg8"] Jul 21 12:58:10.303714 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:58:10.303679 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-8pmg8"] Jul 21 12:58:35.689303 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:58:35.689267 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-8pmg8"] Jul 21 12:58:46.385968 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:58:46.385938 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-8pmg8"] Jul 21 12:59:02.084047 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:59:02.084009 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-8pmg8"] Jul 21 12:59:13.689901 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:59:13.689864 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-8pmg8"] Jul 21 12:59:23.792366 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:59:23.792329 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-8pmg8"] Jul 21 12:59:33.442473 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:59:33.442416 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-skc26_6713665c-3b71-4751-929e-16de48263766/konnectivity-agent/0.log" limit=1024 Jul 21 12:59:33.508154 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:59:33.508125 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-skc26_6713665c-3b71-4751-929e-16de48263766/konnectivity-agent/0.log" limit=1024 Jul 21 12:59:35.781359 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:59:35.781327 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-8pmg8"] Jul 21 12:59:53.715084 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:59:53.715054 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-6nf7d_a2b48106-9844-4431-9b6f-d4e1baacb15b/ovn-acl-logging/0.log" Jul 21 12:59:53.715582 ip-10-0-130-157 kubenswrapper[2576]: I0721 12:59:53.715116 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-6nf7d_a2b48106-9844-4431-9b6f-d4e1baacb15b/ovn-acl-logging/0.log" Jul 21 13:01:30.984908 ip-10-0-130-157 kubenswrapper[2576]: I0721 13:01:30.984868 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-8pmg8"] Jul 21 13:01:42.888455 ip-10-0-130-157 kubenswrapper[2576]: I0721 13:01:42.888424 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-8pmg8"] Jul 21 13:01:51.789412 ip-10-0-130-157 kubenswrapper[2576]: I0721 13:01:51.789378 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-8pmg8"] Jul 21 13:02:02.586308 ip-10-0-130-157 kubenswrapper[2576]: I0721 13:02:02.586277 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-8pmg8"] Jul 21 13:02:13.190364 ip-10-0-130-157 kubenswrapper[2576]: I0721 13:02:13.190327 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-8pmg8"] Jul 21 13:02:21.238315 ip-10-0-130-157 kubenswrapper[2576]: I0721 13:02:21.238210 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-skc26_6713665c-3b71-4751-929e-16de48263766/konnectivity-agent/0.log" limit=1024 Jul 21 13:02:23.884928 ip-10-0-130-157 kubenswrapper[2576]: I0721 13:02:23.884891 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-8pmg8"] Jul 21 13:02:34.393429 ip-10-0-130-157 kubenswrapper[2576]: I0721 13:02:34.393399 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-8pmg8"] Jul 21 13:02:40.594707 ip-10-0-130-157 kubenswrapper[2576]: I0721 13:02:40.594663 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-skc26_6713665c-3b71-4751-929e-16de48263766/konnectivity-agent/0.log" limit=1024 Jul 21 13:02:44.291464 ip-10-0-130-157 kubenswrapper[2576]: I0721 13:02:44.291427 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-8pmg8"] Jul 21 13:02:56.586968 ip-10-0-130-157 kubenswrapper[2576]: I0721 13:02:56.586930 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-8pmg8"] Jul 21 13:03:04.589356 ip-10-0-130-157 kubenswrapper[2576]: I0721 13:03:04.589318 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-8pmg8"] Jul 21 13:03:14.285224 ip-10-0-130-157 kubenswrapper[2576]: I0721 13:03:14.285187 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-8pmg8"] Jul 21 13:03:37.210699 ip-10-0-130-157 kubenswrapper[2576]: I0721 13:03:37.210658 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-8pmg8"] Jul 21 13:03:46.087347 ip-10-0-130-157 kubenswrapper[2576]: I0721 13:03:46.087258 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-8pmg8"] Jul 21 13:03:48.269632 ip-10-0-130-157 kubenswrapper[2576]: I0721 13:03:48.269598 2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-same-id-a-478d1513-kserve-85d49bc68c-8qw9n"] Jul 21 13:03:48.272721 ip-10-0-130-157 kubenswrapper[2576]: I0721 13:03:48.272704 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-a-478d1513-kserve-85d49bc68c-8qw9n" Jul 21 13:03:48.274821 ip-10-0-130-157 kubenswrapper[2576]: I0721 13:03:48.274803 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-same-id-a-478d1513-kserve-self-signed-certs\"" Jul 21 13:03:48.292830 ip-10-0-130-157 kubenswrapper[2576]: I0721 13:03:48.292808 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-same-id-a-478d1513-kserve-85d49bc68c-8qw9n"] Jul 21 13:03:48.299484 ip-10-0-130-157 kubenswrapper[2576]: I0721 13:03:48.299451 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-skc26_6713665c-3b71-4751-929e-16de48263766/konnectivity-agent/0.log" limit=1024 Jul 21 13:03:48.339930 ip-10-0-130-157 kubenswrapper[2576]: I0721 13:03:48.339909 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/fa6cc249-a955-474b-ad3c-c3d420907228-model-cache\") pod \"e2e-same-id-a-478d1513-kserve-85d49bc68c-8qw9n\" (UID: \"fa6cc249-a955-474b-ad3c-c3d420907228\") " pod="llm/e2e-same-id-a-478d1513-kserve-85d49bc68c-8qw9n" Jul 21 13:03:48.340034 ip-10-0-130-157 kubenswrapper[2576]: I0721 13:03:48.339936 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qxmvk\" (UniqueName: \"kubernetes.io/projected/fa6cc249-a955-474b-ad3c-c3d420907228-kube-api-access-qxmvk\") pod \"e2e-same-id-a-478d1513-kserve-85d49bc68c-8qw9n\" (UID: \"fa6cc249-a955-474b-ad3c-c3d420907228\") " pod="llm/e2e-same-id-a-478d1513-kserve-85d49bc68c-8qw9n" Jul 21 13:03:48.340034 ip-10-0-130-157 kubenswrapper[2576]: I0721 13:03:48.339955 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/fa6cc249-a955-474b-ad3c-c3d420907228-home\") pod \"e2e-same-id-a-478d1513-kserve-85d49bc68c-8qw9n\" (UID: \"fa6cc249-a955-474b-ad3c-c3d420907228\") " pod="llm/e2e-same-id-a-478d1513-kserve-85d49bc68c-8qw9n" Jul 21 13:03:48.340034 ip-10-0-130-157 kubenswrapper[2576]: I0721 13:03:48.339978 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/fa6cc249-a955-474b-ad3c-c3d420907228-tls-certs\") pod \"e2e-same-id-a-478d1513-kserve-85d49bc68c-8qw9n\" (UID: \"fa6cc249-a955-474b-ad3c-c3d420907228\") " pod="llm/e2e-same-id-a-478d1513-kserve-85d49bc68c-8qw9n" Jul 21 13:03:48.340169 ip-10-0-130-157 kubenswrapper[2576]: I0721 13:03:48.340063 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/fa6cc249-a955-474b-ad3c-c3d420907228-dshm\") pod \"e2e-same-id-a-478d1513-kserve-85d49bc68c-8qw9n\" (UID: \"fa6cc249-a955-474b-ad3c-c3d420907228\") " pod="llm/e2e-same-id-a-478d1513-kserve-85d49bc68c-8qw9n" Jul 21 13:03:48.340169 ip-10-0-130-157 kubenswrapper[2576]: I0721 13:03:48.340113 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/fa6cc249-a955-474b-ad3c-c3d420907228-tmp-dir\") pod \"e2e-same-id-a-478d1513-kserve-85d49bc68c-8qw9n\" (UID: \"fa6cc249-a955-474b-ad3c-c3d420907228\") " pod="llm/e2e-same-id-a-478d1513-kserve-85d49bc68c-8qw9n" Jul 21 13:03:48.440412 ip-10-0-130-157 kubenswrapper[2576]: I0721 13:03:48.440389 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/fa6cc249-a955-474b-ad3c-c3d420907228-dshm\") pod \"e2e-same-id-a-478d1513-kserve-85d49bc68c-8qw9n\" (UID: \"fa6cc249-a955-474b-ad3c-c3d420907228\") " pod="llm/e2e-same-id-a-478d1513-kserve-85d49bc68c-8qw9n" Jul 21 13:03:48.440501 ip-10-0-130-157 kubenswrapper[2576]: I0721 13:03:48.440425 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/fa6cc249-a955-474b-ad3c-c3d420907228-tmp-dir\") pod \"e2e-same-id-a-478d1513-kserve-85d49bc68c-8qw9n\" (UID: \"fa6cc249-a955-474b-ad3c-c3d420907228\") " pod="llm/e2e-same-id-a-478d1513-kserve-85d49bc68c-8qw9n" Jul 21 13:03:48.440501 ip-10-0-130-157 kubenswrapper[2576]: I0721 13:03:48.440455 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/fa6cc249-a955-474b-ad3c-c3d420907228-model-cache\") pod \"e2e-same-id-a-478d1513-kserve-85d49bc68c-8qw9n\" (UID: \"fa6cc249-a955-474b-ad3c-c3d420907228\") " pod="llm/e2e-same-id-a-478d1513-kserve-85d49bc68c-8qw9n" Jul 21 13:03:48.440501 ip-10-0-130-157 kubenswrapper[2576]: I0721 13:03:48.440471 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qxmvk\" (UniqueName: \"kubernetes.io/projected/fa6cc249-a955-474b-ad3c-c3d420907228-kube-api-access-qxmvk\") pod \"e2e-same-id-a-478d1513-kserve-85d49bc68c-8qw9n\" (UID: \"fa6cc249-a955-474b-ad3c-c3d420907228\") " pod="llm/e2e-same-id-a-478d1513-kserve-85d49bc68c-8qw9n" Jul 21 13:03:48.440501 ip-10-0-130-157 kubenswrapper[2576]: I0721 13:03:48.440487 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/fa6cc249-a955-474b-ad3c-c3d420907228-home\") pod \"e2e-same-id-a-478d1513-kserve-85d49bc68c-8qw9n\" (UID: \"fa6cc249-a955-474b-ad3c-c3d420907228\") " pod="llm/e2e-same-id-a-478d1513-kserve-85d49bc68c-8qw9n" Jul 21 13:03:48.440733 ip-10-0-130-157 kubenswrapper[2576]: I0721 13:03:48.440527 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/fa6cc249-a955-474b-ad3c-c3d420907228-tls-certs\") pod \"e2e-same-id-a-478d1513-kserve-85d49bc68c-8qw9n\" (UID: \"fa6cc249-a955-474b-ad3c-c3d420907228\") " pod="llm/e2e-same-id-a-478d1513-kserve-85d49bc68c-8qw9n" Jul 21 13:03:48.440896 ip-10-0-130-157 kubenswrapper[2576]: I0721 13:03:48.440878 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/fa6cc249-a955-474b-ad3c-c3d420907228-tmp-dir\") pod \"e2e-same-id-a-478d1513-kserve-85d49bc68c-8qw9n\" (UID: \"fa6cc249-a955-474b-ad3c-c3d420907228\") " pod="llm/e2e-same-id-a-478d1513-kserve-85d49bc68c-8qw9n" Jul 21 13:03:48.440956 ip-10-0-130-157 kubenswrapper[2576]: I0721 13:03:48.440907 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/fa6cc249-a955-474b-ad3c-c3d420907228-model-cache\") pod \"e2e-same-id-a-478d1513-kserve-85d49bc68c-8qw9n\" (UID: \"fa6cc249-a955-474b-ad3c-c3d420907228\") " pod="llm/e2e-same-id-a-478d1513-kserve-85d49bc68c-8qw9n" Jul 21 13:03:48.440956 ip-10-0-130-157 kubenswrapper[2576]: I0721 13:03:48.440933 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/fa6cc249-a955-474b-ad3c-c3d420907228-home\") pod \"e2e-same-id-a-478d1513-kserve-85d49bc68c-8qw9n\" (UID: \"fa6cc249-a955-474b-ad3c-c3d420907228\") " pod="llm/e2e-same-id-a-478d1513-kserve-85d49bc68c-8qw9n" Jul 21 13:03:48.442571 ip-10-0-130-157 kubenswrapper[2576]: I0721 13:03:48.442552 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/fa6cc249-a955-474b-ad3c-c3d420907228-dshm\") pod \"e2e-same-id-a-478d1513-kserve-85d49bc68c-8qw9n\" (UID: \"fa6cc249-a955-474b-ad3c-c3d420907228\") " pod="llm/e2e-same-id-a-478d1513-kserve-85d49bc68c-8qw9n" Jul 21 13:03:48.442865 ip-10-0-130-157 kubenswrapper[2576]: I0721 13:03:48.442850 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/fa6cc249-a955-474b-ad3c-c3d420907228-tls-certs\") pod \"e2e-same-id-a-478d1513-kserve-85d49bc68c-8qw9n\" (UID: \"fa6cc249-a955-474b-ad3c-c3d420907228\") " pod="llm/e2e-same-id-a-478d1513-kserve-85d49bc68c-8qw9n" Jul 21 13:03:48.450100 ip-10-0-130-157 kubenswrapper[2576]: I0721 13:03:48.450077 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qxmvk\" (UniqueName: \"kubernetes.io/projected/fa6cc249-a955-474b-ad3c-c3d420907228-kube-api-access-qxmvk\") pod \"e2e-same-id-a-478d1513-kserve-85d49bc68c-8qw9n\" (UID: \"fa6cc249-a955-474b-ad3c-c3d420907228\") " pod="llm/e2e-same-id-a-478d1513-kserve-85d49bc68c-8qw9n" Jul 21 13:03:48.581747 ip-10-0-130-157 kubenswrapper[2576]: I0721 13:03:48.581684 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-a-478d1513-kserve-85d49bc68c-8qw9n" Jul 21 13:03:48.706224 ip-10-0-130-157 kubenswrapper[2576]: I0721 13:03:48.706202 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-same-id-a-478d1513-kserve-85d49bc68c-8qw9n"] Jul 21 13:03:48.708919 ip-10-0-130-157 kubenswrapper[2576]: W0721 13:03:48.708894 2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podfa6cc249_a955_474b_ad3c_c3d420907228.slice/crio-6182ead3dc72fce2c2d48ceca5aaef646611090a7f2addd134493ff63a3f9077 WatchSource:0}: Error finding container 6182ead3dc72fce2c2d48ceca5aaef646611090a7f2addd134493ff63a3f9077: Status 404 returned error can't find the container with id 6182ead3dc72fce2c2d48ceca5aaef646611090a7f2addd134493ff63a3f9077 Jul 21 13:03:49.002731 ip-10-0-130-157 kubenswrapper[2576]: I0721 13:03:49.002694 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-a-478d1513-kserve-85d49bc68c-8qw9n" event={"ID":"fa6cc249-a955-474b-ad3c-c3d420907228","Type":"ContainerStarted","Data":"4efdef73bafda2ec0540a6939a1856a2c89e433d484beebbe5798c546225b24e"} Jul 21 13:03:49.002731 ip-10-0-130-157 kubenswrapper[2576]: I0721 13:03:49.002734 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-a-478d1513-kserve-85d49bc68c-8qw9n" event={"ID":"fa6cc249-a955-474b-ad3c-c3d420907228","Type":"ContainerStarted","Data":"6182ead3dc72fce2c2d48ceca5aaef646611090a7f2addd134493ff63a3f9077"} Jul 21 13:03:49.021905 ip-10-0-130-157 kubenswrapper[2576]: I0721 13:03:49.021862 2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-same-id-a-478d1513-kserve-85d49bc68c-8qw9n" podStartSLOduration=1.021847071 podStartE2EDuration="1.021847071s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 13:03:48 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 13:03:49.020698014 +0000 UTC m=+1436.855961883" watchObservedRunningTime="2026-07-21 13:03:49.021847071 +0000 UTC m=+1436.857110936" Jul 21 13:03:58.582553 ip-10-0-130-157 kubenswrapper[2576]: I0721 13:03:58.582462 2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-same-id-a-478d1513-kserve-85d49bc68c-8qw9n" Jul 21 13:03:58.582553 ip-10-0-130-157 kubenswrapper[2576]: I0721 13:03:58.582501 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-same-id-a-478d1513-kserve-85d49bc68c-8qw9n" Jul 21 13:03:58.594755 ip-10-0-130-157 kubenswrapper[2576]: I0721 13:03:58.594728 2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-same-id-a-478d1513-kserve-85d49bc68c-8qw9n" Jul 21 13:03:59.058799 ip-10-0-130-157 kubenswrapper[2576]: I0721 13:03:59.058772 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-same-id-a-478d1513-kserve-85d49bc68c-8qw9n" Jul 21 13:03:59.791260 ip-10-0-130-157 kubenswrapper[2576]: I0721 13:03:59.791222 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-8pmg8"] Jul 21 13:04:02.956146 ip-10-0-130-157 kubenswrapper[2576]: I0721 13:04:02.956113 2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-same-id-b-478d1513-kserve-84d5fb5d75-2tn5b"] Jul 21 13:04:02.959627 ip-10-0-130-157 kubenswrapper[2576]: I0721 13:04:02.959606 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-b-478d1513-kserve-84d5fb5d75-2tn5b" Jul 21 13:04:02.961428 ip-10-0-130-157 kubenswrapper[2576]: I0721 13:04:02.961402 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-same-id-b-478d1513-kserve-self-signed-certs\"" Jul 21 13:04:02.969337 ip-10-0-130-157 kubenswrapper[2576]: I0721 13:04:02.969314 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-same-id-b-478d1513-kserve-84d5fb5d75-2tn5b"] Jul 21 13:04:02.989493 ip-10-0-130-157 kubenswrapper[2576]: I0721 13:04:02.989461 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-skc26_6713665c-3b71-4751-929e-16de48263766/konnectivity-agent/0.log" limit=1024 Jul 21 13:04:03.045702 ip-10-0-130-157 kubenswrapper[2576]: I0721 13:04:03.045668 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qplmn\" (UniqueName: \"kubernetes.io/projected/4e8de9de-4d3c-4012-b131-c92d62c77e3b-kube-api-access-qplmn\") pod \"e2e-same-id-b-478d1513-kserve-84d5fb5d75-2tn5b\" (UID: \"4e8de9de-4d3c-4012-b131-c92d62c77e3b\") " pod="llm/e2e-same-id-b-478d1513-kserve-84d5fb5d75-2tn5b" Jul 21 13:04:03.045787 ip-10-0-130-157 kubenswrapper[2576]: I0721 13:04:03.045703 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4e8de9de-4d3c-4012-b131-c92d62c77e3b-tmp-dir\") pod \"e2e-same-id-b-478d1513-kserve-84d5fb5d75-2tn5b\" (UID: \"4e8de9de-4d3c-4012-b131-c92d62c77e3b\") " pod="llm/e2e-same-id-b-478d1513-kserve-84d5fb5d75-2tn5b" Jul 21 13:04:03.045787 ip-10-0-130-157 kubenswrapper[2576]: I0721 13:04:03.045726 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4e8de9de-4d3c-4012-b131-c92d62c77e3b-tls-certs\") pod \"e2e-same-id-b-478d1513-kserve-84d5fb5d75-2tn5b\" (UID: \"4e8de9de-4d3c-4012-b131-c92d62c77e3b\") " pod="llm/e2e-same-id-b-478d1513-kserve-84d5fb5d75-2tn5b" Jul 21 13:04:03.045873 ip-10-0-130-157 kubenswrapper[2576]: I0721 13:04:03.045808 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/4e8de9de-4d3c-4012-b131-c92d62c77e3b-home\") pod \"e2e-same-id-b-478d1513-kserve-84d5fb5d75-2tn5b\" (UID: \"4e8de9de-4d3c-4012-b131-c92d62c77e3b\") " pod="llm/e2e-same-id-b-478d1513-kserve-84d5fb5d75-2tn5b" Jul 21 13:04:03.045873 ip-10-0-130-157 kubenswrapper[2576]: I0721 13:04:03.045837 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/4e8de9de-4d3c-4012-b131-c92d62c77e3b-dshm\") pod \"e2e-same-id-b-478d1513-kserve-84d5fb5d75-2tn5b\" (UID: \"4e8de9de-4d3c-4012-b131-c92d62c77e3b\") " pod="llm/e2e-same-id-b-478d1513-kserve-84d5fb5d75-2tn5b" Jul 21 13:04:03.045943 ip-10-0-130-157 kubenswrapper[2576]: I0721 13:04:03.045876 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/4e8de9de-4d3c-4012-b131-c92d62c77e3b-model-cache\") pod \"e2e-same-id-b-478d1513-kserve-84d5fb5d75-2tn5b\" (UID: \"4e8de9de-4d3c-4012-b131-c92d62c77e3b\") " pod="llm/e2e-same-id-b-478d1513-kserve-84d5fb5d75-2tn5b" Jul 21 13:04:03.146738 ip-10-0-130-157 kubenswrapper[2576]: I0721 13:04:03.146711 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/4e8de9de-4d3c-4012-b131-c92d62c77e3b-home\") pod \"e2e-same-id-b-478d1513-kserve-84d5fb5d75-2tn5b\" (UID: \"4e8de9de-4d3c-4012-b131-c92d62c77e3b\") " pod="llm/e2e-same-id-b-478d1513-kserve-84d5fb5d75-2tn5b" Jul 21 13:04:03.146835 ip-10-0-130-157 kubenswrapper[2576]: I0721 13:04:03.146752 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/4e8de9de-4d3c-4012-b131-c92d62c77e3b-dshm\") pod \"e2e-same-id-b-478d1513-kserve-84d5fb5d75-2tn5b\" (UID: \"4e8de9de-4d3c-4012-b131-c92d62c77e3b\") " pod="llm/e2e-same-id-b-478d1513-kserve-84d5fb5d75-2tn5b" Jul 21 13:04:03.146835 ip-10-0-130-157 kubenswrapper[2576]: I0721 13:04:03.146777 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/4e8de9de-4d3c-4012-b131-c92d62c77e3b-model-cache\") pod \"e2e-same-id-b-478d1513-kserve-84d5fb5d75-2tn5b\" (UID: \"4e8de9de-4d3c-4012-b131-c92d62c77e3b\") " pod="llm/e2e-same-id-b-478d1513-kserve-84d5fb5d75-2tn5b" Jul 21 13:04:03.146835 ip-10-0-130-157 kubenswrapper[2576]: I0721 13:04:03.146808 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qplmn\" (UniqueName: \"kubernetes.io/projected/4e8de9de-4d3c-4012-b131-c92d62c77e3b-kube-api-access-qplmn\") pod \"e2e-same-id-b-478d1513-kserve-84d5fb5d75-2tn5b\" (UID: \"4e8de9de-4d3c-4012-b131-c92d62c77e3b\") " pod="llm/e2e-same-id-b-478d1513-kserve-84d5fb5d75-2tn5b" Jul 21 13:04:03.146835 ip-10-0-130-157 kubenswrapper[2576]: I0721 13:04:03.146827 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4e8de9de-4d3c-4012-b131-c92d62c77e3b-tmp-dir\") pod \"e2e-same-id-b-478d1513-kserve-84d5fb5d75-2tn5b\" (UID: \"4e8de9de-4d3c-4012-b131-c92d62c77e3b\") " pod="llm/e2e-same-id-b-478d1513-kserve-84d5fb5d75-2tn5b" Jul 21 13:04:03.147040 ip-10-0-130-157 kubenswrapper[2576]: I0721 13:04:03.146842 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4e8de9de-4d3c-4012-b131-c92d62c77e3b-tls-certs\") pod \"e2e-same-id-b-478d1513-kserve-84d5fb5d75-2tn5b\" (UID: \"4e8de9de-4d3c-4012-b131-c92d62c77e3b\") " pod="llm/e2e-same-id-b-478d1513-kserve-84d5fb5d75-2tn5b" Jul 21 13:04:03.147106 ip-10-0-130-157 kubenswrapper[2576]: I0721 13:04:03.147083 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/4e8de9de-4d3c-4012-b131-c92d62c77e3b-home\") pod \"e2e-same-id-b-478d1513-kserve-84d5fb5d75-2tn5b\" (UID: \"4e8de9de-4d3c-4012-b131-c92d62c77e3b\") " pod="llm/e2e-same-id-b-478d1513-kserve-84d5fb5d75-2tn5b" Jul 21 13:04:03.147250 ip-10-0-130-157 kubenswrapper[2576]: I0721 13:04:03.147226 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4e8de9de-4d3c-4012-b131-c92d62c77e3b-tmp-dir\") pod \"e2e-same-id-b-478d1513-kserve-84d5fb5d75-2tn5b\" (UID: \"4e8de9de-4d3c-4012-b131-c92d62c77e3b\") " pod="llm/e2e-same-id-b-478d1513-kserve-84d5fb5d75-2tn5b" Jul 21 13:04:03.147310 ip-10-0-130-157 kubenswrapper[2576]: I0721 13:04:03.147232 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/4e8de9de-4d3c-4012-b131-c92d62c77e3b-model-cache\") pod \"e2e-same-id-b-478d1513-kserve-84d5fb5d75-2tn5b\" (UID: \"4e8de9de-4d3c-4012-b131-c92d62c77e3b\") " pod="llm/e2e-same-id-b-478d1513-kserve-84d5fb5d75-2tn5b" Jul 21 13:04:03.149105 ip-10-0-130-157 kubenswrapper[2576]: I0721 13:04:03.149088 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/4e8de9de-4d3c-4012-b131-c92d62c77e3b-dshm\") pod \"e2e-same-id-b-478d1513-kserve-84d5fb5d75-2tn5b\" (UID: \"4e8de9de-4d3c-4012-b131-c92d62c77e3b\") " pod="llm/e2e-same-id-b-478d1513-kserve-84d5fb5d75-2tn5b" Jul 21 13:04:03.149391 ip-10-0-130-157 kubenswrapper[2576]: I0721 13:04:03.149372 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4e8de9de-4d3c-4012-b131-c92d62c77e3b-tls-certs\") pod \"e2e-same-id-b-478d1513-kserve-84d5fb5d75-2tn5b\" (UID: \"4e8de9de-4d3c-4012-b131-c92d62c77e3b\") " pod="llm/e2e-same-id-b-478d1513-kserve-84d5fb5d75-2tn5b" Jul 21 13:04:03.154891 ip-10-0-130-157 kubenswrapper[2576]: I0721 13:04:03.154872 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qplmn\" (UniqueName: \"kubernetes.io/projected/4e8de9de-4d3c-4012-b131-c92d62c77e3b-kube-api-access-qplmn\") pod \"e2e-same-id-b-478d1513-kserve-84d5fb5d75-2tn5b\" (UID: \"4e8de9de-4d3c-4012-b131-c92d62c77e3b\") " pod="llm/e2e-same-id-b-478d1513-kserve-84d5fb5d75-2tn5b" Jul 21 13:04:03.268995 ip-10-0-130-157 kubenswrapper[2576]: I0721 13:04:03.268942 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-b-478d1513-kserve-84d5fb5d75-2tn5b" Jul 21 13:04:03.399117 ip-10-0-130-157 kubenswrapper[2576]: I0721 13:04:03.399065 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-same-id-b-478d1513-kserve-84d5fb5d75-2tn5b"] Jul 21 13:04:03.401179 ip-10-0-130-157 kubenswrapper[2576]: W0721 13:04:03.401153 2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod4e8de9de_4d3c_4012_b131_c92d62c77e3b.slice/crio-3fe8f606b5ca8d3d44297c0e77979284c6abb00c2dc798e7c3885043217b3bff WatchSource:0}: Error finding container 3fe8f606b5ca8d3d44297c0e77979284c6abb00c2dc798e7c3885043217b3bff: Status 404 returned error can't find the container with id 3fe8f606b5ca8d3d44297c0e77979284c6abb00c2dc798e7c3885043217b3bff Jul 21 13:04:04.072123 ip-10-0-130-157 kubenswrapper[2576]: I0721 13:04:04.072092 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-b-478d1513-kserve-84d5fb5d75-2tn5b" event={"ID":"4e8de9de-4d3c-4012-b131-c92d62c77e3b","Type":"ContainerStarted","Data":"6826fde7e3304d79a6d996db1cffa6cda1ac401512fa8b8d30abaa2eb5eeeea7"} Jul 21 13:04:04.072507 ip-10-0-130-157 kubenswrapper[2576]: I0721 13:04:04.072133 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-b-478d1513-kserve-84d5fb5d75-2tn5b" event={"ID":"4e8de9de-4d3c-4012-b131-c92d62c77e3b","Type":"ContainerStarted","Data":"3fe8f606b5ca8d3d44297c0e77979284c6abb00c2dc798e7c3885043217b3bff"} Jul 21 13:04:04.091587 ip-10-0-130-157 kubenswrapper[2576]: I0721 13:04:04.091538 2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-same-id-b-478d1513-kserve-84d5fb5d75-2tn5b" podStartSLOduration=2.091509133 podStartE2EDuration="2.091509133s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 13:04:02 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 13:04:04.090361835 +0000 UTC m=+1451.925625700" watchObservedRunningTime="2026-07-21 13:04:04.091509133 +0000 UTC m=+1451.926772976" Jul 21 13:04:08.987732 ip-10-0-130-157 kubenswrapper[2576]: I0721 13:04:08.987699 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-8pmg8"] Jul 21 13:04:13.269719 ip-10-0-130-157 kubenswrapper[2576]: I0721 13:04:13.269687 2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-same-id-b-478d1513-kserve-84d5fb5d75-2tn5b" Jul 21 13:04:13.270113 ip-10-0-130-157 kubenswrapper[2576]: I0721 13:04:13.269733 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-same-id-b-478d1513-kserve-84d5fb5d75-2tn5b" Jul 21 13:04:13.282295 ip-10-0-130-157 kubenswrapper[2576]: I0721 13:04:13.282271 2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-same-id-b-478d1513-kserve-84d5fb5d75-2tn5b" Jul 21 13:04:14.127460 ip-10-0-130-157 kubenswrapper[2576]: I0721 13:04:14.127429 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-same-id-b-478d1513-kserve-84d5fb5d75-2tn5b" Jul 21 13:04:29.827597 ip-10-0-130-157 kubenswrapper[2576]: I0721 13:04:29.827565 2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["llm/e2e-same-id-a-478d1513-kserve-85d49bc68c-8qw9n"] Jul 21 13:04:29.858110 ip-10-0-130-157 kubenswrapper[2576]: I0721 13:04:29.858077 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-skc26_6713665c-3b71-4751-929e-16de48263766/konnectivity-agent/0.log" limit=1024 Jul 21 13:04:31.133075 ip-10-0-130-157 kubenswrapper[2576]: I0721 13:04:31.133043 2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["llm/e2e-same-id-b-478d1513-kserve-84d5fb5d75-2tn5b"] Jul 21 13:04:31.157640 ip-10-0-130-157 kubenswrapper[2576]: I0721 13:04:31.157606 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-skc26_6713665c-3b71-4751-929e-16de48263766/konnectivity-agent/0.log" limit=1024 Jul 21 13:04:34.295534 ip-10-0-130-157 kubenswrapper[2576]: I0721 13:04:34.295485 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-8pmg8"] Jul 21 13:04:44.847137 ip-10-0-130-157 kubenswrapper[2576]: I0721 13:04:44.847094 2576 kuberuntime_container.go:873] "Killing container with a grace period" pod="llm/e2e-same-id-a-478d1513-kserve-85d49bc68c-8qw9n" podUID="fa6cc249-a955-474b-ad3c-c3d420907228" containerName="main" containerID="cri-o://4efdef73bafda2ec0540a6939a1856a2c89e433d484beebbe5798c546225b24e" gracePeriod=45 Jul 21 13:04:45.085262 ip-10-0-130-157 kubenswrapper[2576]: I0721 13:04:45.085240 2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-a-478d1513-kserve-85d49bc68c-8qw9n" Jul 21 13:04:45.192103 ip-10-0-130-157 kubenswrapper[2576]: I0721 13:04:45.192076 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/fa6cc249-a955-474b-ad3c-c3d420907228-model-cache\") pod \"fa6cc249-a955-474b-ad3c-c3d420907228\" (UID: \"fa6cc249-a955-474b-ad3c-c3d420907228\") " Jul 21 13:04:45.192239 ip-10-0-130-157 kubenswrapper[2576]: I0721 13:04:45.192124 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/fa6cc249-a955-474b-ad3c-c3d420907228-home\") pod \"fa6cc249-a955-474b-ad3c-c3d420907228\" (UID: \"fa6cc249-a955-474b-ad3c-c3d420907228\") " Jul 21 13:04:45.192239 ip-10-0-130-157 kubenswrapper[2576]: I0721 13:04:45.192141 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/fa6cc249-a955-474b-ad3c-c3d420907228-tmp-dir\") pod \"fa6cc249-a955-474b-ad3c-c3d420907228\" (UID: \"fa6cc249-a955-474b-ad3c-c3d420907228\") " Jul 21 13:04:45.192239 ip-10-0-130-157 kubenswrapper[2576]: I0721 13:04:45.192170 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/fa6cc249-a955-474b-ad3c-c3d420907228-tls-certs\") pod \"fa6cc249-a955-474b-ad3c-c3d420907228\" (UID: \"fa6cc249-a955-474b-ad3c-c3d420907228\") " Jul 21 13:04:45.192239 ip-10-0-130-157 kubenswrapper[2576]: I0721 13:04:45.192197 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-qxmvk\" (UniqueName: \"kubernetes.io/projected/fa6cc249-a955-474b-ad3c-c3d420907228-kube-api-access-qxmvk\") pod \"fa6cc249-a955-474b-ad3c-c3d420907228\" (UID: \"fa6cc249-a955-474b-ad3c-c3d420907228\") " Jul 21 13:04:45.192239 ip-10-0-130-157 kubenswrapper[2576]: I0721 13:04:45.192221 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/fa6cc249-a955-474b-ad3c-c3d420907228-dshm\") pod \"fa6cc249-a955-474b-ad3c-c3d420907228\" (UID: \"fa6cc249-a955-474b-ad3c-c3d420907228\") " Jul 21 13:04:45.192496 ip-10-0-130-157 kubenswrapper[2576]: I0721 13:04:45.192388 2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fa6cc249-a955-474b-ad3c-c3d420907228-home" (OuterVolumeSpecName: "home") pod "fa6cc249-a955-474b-ad3c-c3d420907228" (UID: "fa6cc249-a955-474b-ad3c-c3d420907228"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 21 13:04:45.192496 ip-10-0-130-157 kubenswrapper[2576]: I0721 13:04:45.192405 2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fa6cc249-a955-474b-ad3c-c3d420907228-model-cache" (OuterVolumeSpecName: "model-cache") pod "fa6cc249-a955-474b-ad3c-c3d420907228" (UID: "fa6cc249-a955-474b-ad3c-c3d420907228"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 21 13:04:45.192607 ip-10-0-130-157 kubenswrapper[2576]: I0721 13:04:45.192496 2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fa6cc249-a955-474b-ad3c-c3d420907228-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "fa6cc249-a955-474b-ad3c-c3d420907228" (UID: "fa6cc249-a955-474b-ad3c-c3d420907228"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 21 13:04:45.194408 ip-10-0-130-157 kubenswrapper[2576]: I0721 13:04:45.194385 2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/fa6cc249-a955-474b-ad3c-c3d420907228-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "fa6cc249-a955-474b-ad3c-c3d420907228" (UID: "fa6cc249-a955-474b-ad3c-c3d420907228"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 21 13:04:45.194494 ip-10-0-130-157 kubenswrapper[2576]: I0721 13:04:45.194435 2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fa6cc249-a955-474b-ad3c-c3d420907228-dshm" (OuterVolumeSpecName: "dshm") pod "fa6cc249-a955-474b-ad3c-c3d420907228" (UID: "fa6cc249-a955-474b-ad3c-c3d420907228"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 21 13:04:45.194494 ip-10-0-130-157 kubenswrapper[2576]: I0721 13:04:45.194442 2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/fa6cc249-a955-474b-ad3c-c3d420907228-kube-api-access-qxmvk" (OuterVolumeSpecName: "kube-api-access-qxmvk") pod "fa6cc249-a955-474b-ad3c-c3d420907228" (UID: "fa6cc249-a955-474b-ad3c-c3d420907228"). InnerVolumeSpecName "kube-api-access-qxmvk". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 21 13:04:45.261265 ip-10-0-130-157 kubenswrapper[2576]: I0721 13:04:45.261240 2576 generic.go:356] "Generic (PLEG): container finished" podID="fa6cc249-a955-474b-ad3c-c3d420907228" containerID="4efdef73bafda2ec0540a6939a1856a2c89e433d484beebbe5798c546225b24e" exitCode=0 Jul 21 13:04:45.261383 ip-10-0-130-157 kubenswrapper[2576]: I0721 13:04:45.261281 2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-a-478d1513-kserve-85d49bc68c-8qw9n" Jul 21 13:04:45.261383 ip-10-0-130-157 kubenswrapper[2576]: I0721 13:04:45.261305 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-a-478d1513-kserve-85d49bc68c-8qw9n" event={"ID":"fa6cc249-a955-474b-ad3c-c3d420907228","Type":"ContainerDied","Data":"4efdef73bafda2ec0540a6939a1856a2c89e433d484beebbe5798c546225b24e"} Jul 21 13:04:45.261383 ip-10-0-130-157 kubenswrapper[2576]: I0721 13:04:45.261343 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-a-478d1513-kserve-85d49bc68c-8qw9n" event={"ID":"fa6cc249-a955-474b-ad3c-c3d420907228","Type":"ContainerDied","Data":"6182ead3dc72fce2c2d48ceca5aaef646611090a7f2addd134493ff63a3f9077"} Jul 21 13:04:45.261383 ip-10-0-130-157 kubenswrapper[2576]: I0721 13:04:45.261366 2576 scope.go:117] "RemoveContainer" containerID="4efdef73bafda2ec0540a6939a1856a2c89e433d484beebbe5798c546225b24e" Jul 21 13:04:45.269325 ip-10-0-130-157 kubenswrapper[2576]: I0721 13:04:45.269309 2576 scope.go:117] "RemoveContainer" containerID="4efdef73bafda2ec0540a6939a1856a2c89e433d484beebbe5798c546225b24e" Jul 21 13:04:45.269621 ip-10-0-130-157 kubenswrapper[2576]: E0721 13:04:45.269603 2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4efdef73bafda2ec0540a6939a1856a2c89e433d484beebbe5798c546225b24e\": container with ID starting with 4efdef73bafda2ec0540a6939a1856a2c89e433d484beebbe5798c546225b24e not found: ID does not exist" containerID="4efdef73bafda2ec0540a6939a1856a2c89e433d484beebbe5798c546225b24e" Jul 21 13:04:45.269691 ip-10-0-130-157 kubenswrapper[2576]: I0721 13:04:45.269628 2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4efdef73bafda2ec0540a6939a1856a2c89e433d484beebbe5798c546225b24e"} err="failed to get container status \"4efdef73bafda2ec0540a6939a1856a2c89e433d484beebbe5798c546225b24e\": rpc error: code = NotFound desc = could not find container \"4efdef73bafda2ec0540a6939a1856a2c89e433d484beebbe5798c546225b24e\": container with ID starting with 4efdef73bafda2ec0540a6939a1856a2c89e433d484beebbe5798c546225b24e not found: ID does not exist" Jul 21 13:04:45.281901 ip-10-0-130-157 kubenswrapper[2576]: I0721 13:04:45.281881 2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["llm/e2e-same-id-a-478d1513-kserve-85d49bc68c-8qw9n"] Jul 21 13:04:45.286671 ip-10-0-130-157 kubenswrapper[2576]: I0721 13:04:45.286650 2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["llm/e2e-same-id-a-478d1513-kserve-85d49bc68c-8qw9n"] Jul 21 13:04:45.293540 ip-10-0-130-157 kubenswrapper[2576]: I0721 13:04:45.293506 2576 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/fa6cc249-a955-474b-ad3c-c3d420907228-model-cache\") on node \"ip-10-0-130-157.ec2.internal\" DevicePath \"\"" Jul 21 13:04:45.293597 ip-10-0-130-157 kubenswrapper[2576]: I0721 13:04:45.293544 2576 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/fa6cc249-a955-474b-ad3c-c3d420907228-home\") on node \"ip-10-0-130-157.ec2.internal\" DevicePath \"\"" Jul 21 13:04:45.293597 ip-10-0-130-157 kubenswrapper[2576]: I0721 13:04:45.293553 2576 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/fa6cc249-a955-474b-ad3c-c3d420907228-tmp-dir\") on node \"ip-10-0-130-157.ec2.internal\" DevicePath \"\"" Jul 21 13:04:45.293597 ip-10-0-130-157 kubenswrapper[2576]: I0721 13:04:45.293560 2576 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/fa6cc249-a955-474b-ad3c-c3d420907228-tls-certs\") on node \"ip-10-0-130-157.ec2.internal\" DevicePath \"\"" Jul 21 13:04:45.293597 ip-10-0-130-157 kubenswrapper[2576]: I0721 13:04:45.293568 2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-qxmvk\" (UniqueName: \"kubernetes.io/projected/fa6cc249-a955-474b-ad3c-c3d420907228-kube-api-access-qxmvk\") on node \"ip-10-0-130-157.ec2.internal\" DevicePath \"\"" Jul 21 13:04:45.293597 ip-10-0-130-157 kubenswrapper[2576]: I0721 13:04:45.293578 2576 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/fa6cc249-a955-474b-ad3c-c3d420907228-dshm\") on node \"ip-10-0-130-157.ec2.internal\" DevicePath \"\"" Jul 21 13:04:46.153007 ip-10-0-130-157 kubenswrapper[2576]: I0721 13:04:46.152962 2576 kuberuntime_container.go:873] "Killing container with a grace period" pod="llm/e2e-same-id-b-478d1513-kserve-84d5fb5d75-2tn5b" podUID="4e8de9de-4d3c-4012-b131-c92d62c77e3b" containerName="main" containerID="cri-o://6826fde7e3304d79a6d996db1cffa6cda1ac401512fa8b8d30abaa2eb5eeeea7" gracePeriod=45 Jul 21 13:04:46.268144 ip-10-0-130-157 kubenswrapper[2576]: I0721 13:04:46.268119 2576 generic.go:356] "Generic (PLEG): container finished" podID="4e8de9de-4d3c-4012-b131-c92d62c77e3b" containerID="6826fde7e3304d79a6d996db1cffa6cda1ac401512fa8b8d30abaa2eb5eeeea7" exitCode=0 Jul 21 13:04:46.268252 ip-10-0-130-157 kubenswrapper[2576]: I0721 13:04:46.268159 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-b-478d1513-kserve-84d5fb5d75-2tn5b" event={"ID":"4e8de9de-4d3c-4012-b131-c92d62c77e3b","Type":"ContainerDied","Data":"6826fde7e3304d79a6d996db1cffa6cda1ac401512fa8b8d30abaa2eb5eeeea7"} Jul 21 13:04:46.316656 ip-10-0-130-157 kubenswrapper[2576]: I0721 13:04:46.316629 2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="fa6cc249-a955-474b-ad3c-c3d420907228" path="/var/lib/kubelet/pods/fa6cc249-a955-474b-ad3c-c3d420907228/volumes" Jul 21 13:04:46.392904 ip-10-0-130-157 kubenswrapper[2576]: I0721 13:04:46.392876 2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-b-478d1513-kserve-84d5fb5d75-2tn5b" Jul 21 13:04:46.403288 ip-10-0-130-157 kubenswrapper[2576]: I0721 13:04:46.403250 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4e8de9de-4d3c-4012-b131-c92d62c77e3b-tmp-dir\") pod \"4e8de9de-4d3c-4012-b131-c92d62c77e3b\" (UID: \"4e8de9de-4d3c-4012-b131-c92d62c77e3b\") " Jul 21 13:04:46.403379 ip-10-0-130-157 kubenswrapper[2576]: I0721 13:04:46.403287 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/4e8de9de-4d3c-4012-b131-c92d62c77e3b-model-cache\") pod \"4e8de9de-4d3c-4012-b131-c92d62c77e3b\" (UID: \"4e8de9de-4d3c-4012-b131-c92d62c77e3b\") " Jul 21 13:04:46.403379 ip-10-0-130-157 kubenswrapper[2576]: I0721 13:04:46.403310 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-qplmn\" (UniqueName: \"kubernetes.io/projected/4e8de9de-4d3c-4012-b131-c92d62c77e3b-kube-api-access-qplmn\") pod \"4e8de9de-4d3c-4012-b131-c92d62c77e3b\" (UID: \"4e8de9de-4d3c-4012-b131-c92d62c77e3b\") " Jul 21 13:04:46.403379 ip-10-0-130-157 kubenswrapper[2576]: I0721 13:04:46.403344 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4e8de9de-4d3c-4012-b131-c92d62c77e3b-tls-certs\") pod \"4e8de9de-4d3c-4012-b131-c92d62c77e3b\" (UID: \"4e8de9de-4d3c-4012-b131-c92d62c77e3b\") " Jul 21 13:04:46.403379 ip-10-0-130-157 kubenswrapper[2576]: I0721 13:04:46.403361 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/4e8de9de-4d3c-4012-b131-c92d62c77e3b-dshm\") pod \"4e8de9de-4d3c-4012-b131-c92d62c77e3b\" (UID: \"4e8de9de-4d3c-4012-b131-c92d62c77e3b\") " Jul 21 13:04:46.403603 ip-10-0-130-157 kubenswrapper[2576]: I0721 13:04:46.403422 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/4e8de9de-4d3c-4012-b131-c92d62c77e3b-home\") pod \"4e8de9de-4d3c-4012-b131-c92d62c77e3b\" (UID: \"4e8de9de-4d3c-4012-b131-c92d62c77e3b\") " Jul 21 13:04:46.403603 ip-10-0-130-157 kubenswrapper[2576]: I0721 13:04:46.403551 2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4e8de9de-4d3c-4012-b131-c92d62c77e3b-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "4e8de9de-4d3c-4012-b131-c92d62c77e3b" (UID: "4e8de9de-4d3c-4012-b131-c92d62c77e3b"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 21 13:04:46.403603 ip-10-0-130-157 kubenswrapper[2576]: I0721 13:04:46.403559 2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4e8de9de-4d3c-4012-b131-c92d62c77e3b-model-cache" (OuterVolumeSpecName: "model-cache") pod "4e8de9de-4d3c-4012-b131-c92d62c77e3b" (UID: "4e8de9de-4d3c-4012-b131-c92d62c77e3b"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 21 13:04:46.403757 ip-10-0-130-157 kubenswrapper[2576]: I0721 13:04:46.403630 2576 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4e8de9de-4d3c-4012-b131-c92d62c77e3b-tmp-dir\") on node \"ip-10-0-130-157.ec2.internal\" DevicePath \"\"" Jul 21 13:04:46.403757 ip-10-0-130-157 kubenswrapper[2576]: I0721 13:04:46.403648 2576 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/4e8de9de-4d3c-4012-b131-c92d62c77e3b-model-cache\") on node \"ip-10-0-130-157.ec2.internal\" DevicePath \"\"" Jul 21 13:04:46.403856 ip-10-0-130-157 kubenswrapper[2576]: I0721 13:04:46.403758 2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4e8de9de-4d3c-4012-b131-c92d62c77e3b-home" (OuterVolumeSpecName: "home") pod "4e8de9de-4d3c-4012-b131-c92d62c77e3b" (UID: "4e8de9de-4d3c-4012-b131-c92d62c77e3b"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 21 13:04:46.405381 ip-10-0-130-157 kubenswrapper[2576]: I0721 13:04:46.405357 2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/4e8de9de-4d3c-4012-b131-c92d62c77e3b-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "4e8de9de-4d3c-4012-b131-c92d62c77e3b" (UID: "4e8de9de-4d3c-4012-b131-c92d62c77e3b"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 21 13:04:46.405476 ip-10-0-130-157 kubenswrapper[2576]: I0721 13:04:46.405376 2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4e8de9de-4d3c-4012-b131-c92d62c77e3b-kube-api-access-qplmn" (OuterVolumeSpecName: "kube-api-access-qplmn") pod "4e8de9de-4d3c-4012-b131-c92d62c77e3b" (UID: "4e8de9de-4d3c-4012-b131-c92d62c77e3b"). InnerVolumeSpecName "kube-api-access-qplmn". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 21 13:04:46.405476 ip-10-0-130-157 kubenswrapper[2576]: I0721 13:04:46.405388 2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4e8de9de-4d3c-4012-b131-c92d62c77e3b-dshm" (OuterVolumeSpecName: "dshm") pod "4e8de9de-4d3c-4012-b131-c92d62c77e3b" (UID: "4e8de9de-4d3c-4012-b131-c92d62c77e3b"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 21 13:04:46.504880 ip-10-0-130-157 kubenswrapper[2576]: I0721 13:04:46.504857 2576 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4e8de9de-4d3c-4012-b131-c92d62c77e3b-tls-certs\") on node \"ip-10-0-130-157.ec2.internal\" DevicePath \"\"" Jul 21 13:04:46.504880 ip-10-0-130-157 kubenswrapper[2576]: I0721 13:04:46.504877 2576 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/4e8de9de-4d3c-4012-b131-c92d62c77e3b-dshm\") on node \"ip-10-0-130-157.ec2.internal\" DevicePath \"\"" Jul 21 13:04:46.504880 ip-10-0-130-157 kubenswrapper[2576]: I0721 13:04:46.504885 2576 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/4e8de9de-4d3c-4012-b131-c92d62c77e3b-home\") on node \"ip-10-0-130-157.ec2.internal\" DevicePath \"\"" Jul 21 13:04:46.505050 ip-10-0-130-157 kubenswrapper[2576]: I0721 13:04:46.504893 2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-qplmn\" (UniqueName: \"kubernetes.io/projected/4e8de9de-4d3c-4012-b131-c92d62c77e3b-kube-api-access-qplmn\") on node \"ip-10-0-130-157.ec2.internal\" DevicePath \"\"" Jul 21 13:04:46.762613 ip-10-0-130-157 kubenswrapper[2576]: I0721 13:04:46.762497 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-skc26_6713665c-3b71-4751-929e-16de48263766/konnectivity-agent/0.log" limit=1024 Jul 21 13:04:46.862366 ip-10-0-130-157 kubenswrapper[2576]: I0721 13:04:46.862339 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-skc26_6713665c-3b71-4751-929e-16de48263766/konnectivity-agent/0.log" limit=1024 Jul 21 13:04:47.273323 ip-10-0-130-157 kubenswrapper[2576]: I0721 13:04:47.273296 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-b-478d1513-kserve-84d5fb5d75-2tn5b" event={"ID":"4e8de9de-4d3c-4012-b131-c92d62c77e3b","Type":"ContainerDied","Data":"3fe8f606b5ca8d3d44297c0e77979284c6abb00c2dc798e7c3885043217b3bff"} Jul 21 13:04:47.273686 ip-10-0-130-157 kubenswrapper[2576]: I0721 13:04:47.273304 2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-b-478d1513-kserve-84d5fb5d75-2tn5b" Jul 21 13:04:47.273686 ip-10-0-130-157 kubenswrapper[2576]: I0721 13:04:47.273344 2576 scope.go:117] "RemoveContainer" containerID="6826fde7e3304d79a6d996db1cffa6cda1ac401512fa8b8d30abaa2eb5eeeea7" Jul 21 13:04:47.295303 ip-10-0-130-157 kubenswrapper[2576]: I0721 13:04:47.295281 2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["llm/e2e-same-id-b-478d1513-kserve-84d5fb5d75-2tn5b"] Jul 21 13:04:47.297442 ip-10-0-130-157 kubenswrapper[2576]: I0721 13:04:47.297422 2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["llm/e2e-same-id-b-478d1513-kserve-84d5fb5d75-2tn5b"] Jul 21 13:04:48.120558 ip-10-0-130-157 kubenswrapper[2576]: I0721 13:04:48.120508 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-8pmg8"] Jul 21 13:04:48.315036 ip-10-0-130-157 kubenswrapper[2576]: I0721 13:04:48.315005 2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="4e8de9de-4d3c-4012-b131-c92d62c77e3b" path="/var/lib/kubelet/pods/4e8de9de-4d3c-4012-b131-c92d62c77e3b/volumes" Jul 21 13:04:53.739113 ip-10-0-130-157 kubenswrapper[2576]: I0721 13:04:53.739090 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-6nf7d_a2b48106-9844-4431-9b6f-d4e1baacb15b/ovn-acl-logging/0.log" Jul 21 13:04:53.739542 ip-10-0-130-157 kubenswrapper[2576]: I0721 13:04:53.739510 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-6nf7d_a2b48106-9844-4431-9b6f-d4e1baacb15b/ovn-acl-logging/0.log" Jul 21 13:04:58.508357 ip-10-0-130-157 kubenswrapper[2576]: I0721 13:04:58.508274 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-8pmg8"] Jul 21 13:05:10.783310 ip-10-0-130-157 kubenswrapper[2576]: I0721 13:05:10.783269 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-8pmg8"] Jul 21 13:05:20.385084 ip-10-0-130-157 kubenswrapper[2576]: I0721 13:05:20.385006 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-8pmg8"] Jul 21 13:05:30.403839 ip-10-0-130-157 kubenswrapper[2576]: I0721 13:05:30.403808 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-8pmg8"] Jul 21 13:05:38.295077 ip-10-0-130-157 kubenswrapper[2576]: I0721 13:05:38.295037 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-8pmg8"] Jul 21 13:05:47.888032 ip-10-0-130-157 kubenswrapper[2576]: I0721 13:05:47.887995 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-8pmg8"] Jul 21 13:05:57.690190 ip-10-0-130-157 kubenswrapper[2576]: I0721 13:05:57.690156 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-8pmg8"] Jul 21 13:06:10.892035 ip-10-0-130-157 kubenswrapper[2576]: I0721 13:06:10.892006 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-8pmg8"] Jul 21 13:06:20.598147 ip-10-0-130-157 kubenswrapper[2576]: I0721 13:06:20.598116 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-8pmg8"] Jul 21 13:06:29.490544 ip-10-0-130-157 kubenswrapper[2576]: I0721 13:06:29.490494 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-8pmg8"] Jul 21 13:06:40.185447 ip-10-0-130-157 kubenswrapper[2576]: I0721 13:06:40.185412 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-8pmg8"] Jul 21 13:06:51.488436 ip-10-0-130-157 kubenswrapper[2576]: I0721 13:06:51.488342 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-8pmg8"] Jul 21 13:06:58.432224 ip-10-0-130-157 kubenswrapper[2576]: I0721 13:06:58.432186 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-8pmg8"] Jul 21 13:07:14.683336 ip-10-0-130-157 kubenswrapper[2576]: I0721 13:07:14.683300 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-8pmg8"] Jul 21 13:07:26.490538 ip-10-0-130-157 kubenswrapper[2576]: I0721 13:07:26.490484 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-8pmg8"] Jul 21 13:07:36.290015 ip-10-0-130-157 kubenswrapper[2576]: I0721 13:07:36.289977 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-8pmg8"] Jul 21 13:07:39.788646 ip-10-0-130-157 kubenswrapper[2576]: I0721 13:07:39.788607 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-skc26_6713665c-3b71-4751-929e-16de48263766/konnectivity-agent/0.log" limit=1024 Jul 21 13:07:48.819729 ip-10-0-130-157 kubenswrapper[2576]: I0721 13:07:48.819690 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-8pmg8"] Jul 21 13:08:00.384338 ip-10-0-130-157 kubenswrapper[2576]: I0721 13:08:00.384302 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-8pmg8"] Jul 21 13:08:00.626535 ip-10-0-130-157 kubenswrapper[2576]: I0721 13:08:00.626481 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-skc26_6713665c-3b71-4751-929e-16de48263766/konnectivity-agent/0.log" limit=1024 Jul 21 13:08:03.599674 ip-10-0-130-157 kubenswrapper[2576]: I0721 13:08:03.599641 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-skc26_6713665c-3b71-4751-929e-16de48263766/konnectivity-agent/0.log" limit=1024 Jul 21 13:08:03.698830 ip-10-0-130-157 kubenswrapper[2576]: I0721 13:08:03.698797 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-skc26_6713665c-3b71-4751-929e-16de48263766/konnectivity-agent/0.log" limit=1024 Jul 21 13:08:04.055497 ip-10-0-130-157 kubenswrapper[2576]: I0721 13:08:04.055466 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-skc26_6713665c-3b71-4751-929e-16de48263766/konnectivity-agent/0.log" limit=1024 Jul 21 13:08:10.285365 ip-10-0-130-157 kubenswrapper[2576]: I0721 13:08:10.285334 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-8pmg8"] Jul 21 13:08:10.298332 ip-10-0-130-157 kubenswrapper[2576]: I0721 13:08:10.298303 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-skc26_6713665c-3b71-4751-929e-16de48263766/konnectivity-agent/0.log" limit=1024 Jul 21 13:08:10.364353 ip-10-0-130-157 kubenswrapper[2576]: I0721 13:08:10.364321 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-skc26_6713665c-3b71-4751-929e-16de48263766/konnectivity-agent/0.log" limit=1024 Jul 21 13:08:10.542571 ip-10-0-130-157 kubenswrapper[2576]: I0721 13:08:10.542492 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-skc26_6713665c-3b71-4751-929e-16de48263766/konnectivity-agent/0.log" limit=1024 Jul 21 13:08:10.873343 ip-10-0-130-157 kubenswrapper[2576]: I0721 13:08:10.873272 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-skc26_6713665c-3b71-4751-929e-16de48263766/konnectivity-agent/0.log" limit=1024 Jul 21 13:08:11.158449 ip-10-0-130-157 kubenswrapper[2576]: I0721 13:08:11.158418 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-skc26_6713665c-3b71-4751-929e-16de48263766/konnectivity-agent/0.log" limit=1024 Jul 21 13:08:13.276752 ip-10-0-130-157 kubenswrapper[2576]: I0721 13:08:13.276716 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-skc26_6713665c-3b71-4751-929e-16de48263766/konnectivity-agent/0.log" limit=1024 Jul 21 13:08:13.382444 ip-10-0-130-157 kubenswrapper[2576]: I0721 13:08:13.382410 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-skc26_6713665c-3b71-4751-929e-16de48263766/konnectivity-agent/0.log" limit=1024 Jul 21 13:08:13.478401 ip-10-0-130-157 kubenswrapper[2576]: I0721 13:08:13.478373 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-skc26_6713665c-3b71-4751-929e-16de48263766/konnectivity-agent/0.log" limit=1024 Jul 21 13:08:13.743617 ip-10-0-130-157 kubenswrapper[2576]: I0721 13:08:13.743585 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-skc26_6713665c-3b71-4751-929e-16de48263766/konnectivity-agent/0.log" limit=1024 Jul 21 13:08:14.110648 ip-10-0-130-157 kubenswrapper[2576]: I0721 13:08:14.110550 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-skc26_6713665c-3b71-4751-929e-16de48263766/konnectivity-agent/0.log" limit=1024 Jul 21 13:08:14.208544 ip-10-0-130-157 kubenswrapper[2576]: I0721 13:08:14.208480 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-skc26_6713665c-3b71-4751-929e-16de48263766/konnectivity-agent/0.log" limit=1024 Jul 21 13:08:19.082073 ip-10-0-130-157 kubenswrapper[2576]: I0721 13:08:19.082044 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-8pmg8"] Jul 21 13:08:30.696554 ip-10-0-130-157 kubenswrapper[2576]: I0721 13:08:30.696506 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-skc26_6713665c-3b71-4751-929e-16de48263766/konnectivity-agent/0.log" limit=1024 Jul 21 13:08:49.444053 ip-10-0-130-157 kubenswrapper[2576]: I0721 13:08:49.444017 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-skc26_6713665c-3b71-4751-929e-16de48263766/konnectivity-agent/0.log" limit=1024 Jul 21 13:08:52.213017 ip-10-0-130-157 kubenswrapper[2576]: I0721 13:08:52.212986 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-8pmg8"] Jul 21 13:08:53.956869 ip-10-0-130-157 kubenswrapper[2576]: I0721 13:08:53.956837 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-skc26_6713665c-3b71-4751-929e-16de48263766/konnectivity-agent/0.log" limit=1024 Jul 21 13:08:54.108787 ip-10-0-130-157 kubenswrapper[2576]: I0721 13:08:54.108751 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-skc26_6713665c-3b71-4751-929e-16de48263766/konnectivity-agent/0.log" limit=1024 Jul 21 13:08:54.366287 ip-10-0-130-157 kubenswrapper[2576]: I0721 13:08:54.366209 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-skc26_6713665c-3b71-4751-929e-16de48263766/konnectivity-agent/0.log" limit=1024 Jul 21 13:08:54.597694 ip-10-0-130-157 kubenswrapper[2576]: I0721 13:08:54.597659 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-skc26_6713665c-3b71-4751-929e-16de48263766/konnectivity-agent/0.log" limit=1024 Jul 21 13:08:54.700846 ip-10-0-130-157 kubenswrapper[2576]: I0721 13:08:54.700820 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-skc26_6713665c-3b71-4751-929e-16de48263766/konnectivity-agent/0.log" limit=1024 Jul 21 13:08:54.813972 ip-10-0-130-157 kubenswrapper[2576]: I0721 13:08:54.813949 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-skc26_6713665c-3b71-4751-929e-16de48263766/konnectivity-agent/0.log" limit=1024 Jul 21 13:08:55.204842 ip-10-0-130-157 kubenswrapper[2576]: I0721 13:08:55.204815 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-skc26_6713665c-3b71-4751-929e-16de48263766/konnectivity-agent/0.log" limit=1024 Jul 21 13:08:55.540627 ip-10-0-130-157 kubenswrapper[2576]: I0721 13:08:55.540555 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-skc26_6713665c-3b71-4751-929e-16de48263766/konnectivity-agent/0.log" limit=1024 Jul 21 13:09:00.461667 ip-10-0-130-157 kubenswrapper[2576]: I0721 13:09:00.461634 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-skc26_6713665c-3b71-4751-929e-16de48263766/konnectivity-agent/0.log" limit=1024 Jul 21 13:09:00.600554 ip-10-0-130-157 kubenswrapper[2576]: I0721 13:09:00.600525 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-skc26_6713665c-3b71-4751-929e-16de48263766/konnectivity-agent/0.log" limit=1024 Jul 21 13:09:00.864346 ip-10-0-130-157 kubenswrapper[2576]: I0721 13:09:00.864267 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-skc26_6713665c-3b71-4751-929e-16de48263766/konnectivity-agent/0.log" limit=1024 Jul 21 13:09:01.317196 ip-10-0-130-157 kubenswrapper[2576]: I0721 13:09:01.317164 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-skc26_6713665c-3b71-4751-929e-16de48263766/konnectivity-agent/0.log" limit=1024 Jul 21 13:09:01.410612 ip-10-0-130-157 kubenswrapper[2576]: I0721 13:09:01.410586 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-skc26_6713665c-3b71-4751-929e-16de48263766/konnectivity-agent/0.log" limit=1024 Jul 21 13:09:04.889451 ip-10-0-130-157 kubenswrapper[2576]: I0721 13:09:04.889414 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-8pmg8"] Jul 21 13:09:36.211804 ip-10-0-130-157 kubenswrapper[2576]: I0721 13:09:36.211773 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-skc26_6713665c-3b71-4751-929e-16de48263766/konnectivity-agent/0.log" limit=1024 Jul 21 13:09:39.292264 ip-10-0-130-157 kubenswrapper[2576]: I0721 13:09:39.292223 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-8pmg8"] Jul 21 13:09:40.541925 ip-10-0-130-157 kubenswrapper[2576]: I0721 13:09:40.541894 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-skc26_6713665c-3b71-4751-929e-16de48263766/konnectivity-agent/0.log" limit=1024 Jul 21 13:09:40.705296 ip-10-0-130-157 kubenswrapper[2576]: I0721 13:09:40.705266 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-skc26_6713665c-3b71-4751-929e-16de48263766/konnectivity-agent/0.log" limit=1024 Jul 21 13:09:40.933694 ip-10-0-130-157 kubenswrapper[2576]: I0721 13:09:40.933657 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-skc26_6713665c-3b71-4751-929e-16de48263766/konnectivity-agent/0.log" limit=1024 Jul 21 13:09:41.164755 ip-10-0-130-157 kubenswrapper[2576]: I0721 13:09:41.164716 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-skc26_6713665c-3b71-4751-929e-16de48263766/konnectivity-agent/0.log" limit=1024 Jul 21 13:09:41.497107 ip-10-0-130-157 kubenswrapper[2576]: I0721 13:09:41.497069 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-skc26_6713665c-3b71-4751-929e-16de48263766/konnectivity-agent/0.log" limit=1024 Jul 21 13:09:41.635356 ip-10-0-130-157 kubenswrapper[2576]: I0721 13:09:41.635330 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-skc26_6713665c-3b71-4751-929e-16de48263766/konnectivity-agent/0.log" limit=1024 Jul 21 13:09:43.834794 ip-10-0-130-157 kubenswrapper[2576]: I0721 13:09:43.834758 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-skc26_6713665c-3b71-4751-929e-16de48263766/konnectivity-agent/0.log" limit=1024 Jul 21 13:09:43.928715 ip-10-0-130-157 kubenswrapper[2576]: I0721 13:09:43.928688 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-skc26_6713665c-3b71-4751-929e-16de48263766/konnectivity-agent/0.log" limit=1024 Jul 21 13:09:44.029142 ip-10-0-130-157 kubenswrapper[2576]: I0721 13:09:44.029111 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-skc26_6713665c-3b71-4751-929e-16de48263766/konnectivity-agent/0.log" limit=1024 Jul 21 13:09:44.128771 ip-10-0-130-157 kubenswrapper[2576]: I0721 13:09:44.128702 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-skc26_6713665c-3b71-4751-929e-16de48263766/konnectivity-agent/0.log" limit=1024 Jul 21 13:09:44.564418 ip-10-0-130-157 kubenswrapper[2576]: I0721 13:09:44.564386 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-skc26_6713665c-3b71-4751-929e-16de48263766/konnectivity-agent/0.log" limit=1024 Jul 21 13:09:44.665022 ip-10-0-130-157 kubenswrapper[2576]: I0721 13:09:44.664991 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-skc26_6713665c-3b71-4751-929e-16de48263766/konnectivity-agent/0.log" limit=1024 Jul 21 13:09:50.885017 ip-10-0-130-157 kubenswrapper[2576]: I0721 13:09:50.884984 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-8pmg8"] Jul 21 13:09:53.760760 ip-10-0-130-157 kubenswrapper[2576]: I0721 13:09:53.760730 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-6nf7d_a2b48106-9844-4431-9b6f-d4e1baacb15b/ovn-acl-logging/0.log" Jul 21 13:09:53.763131 ip-10-0-130-157 kubenswrapper[2576]: I0721 13:09:53.763109 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-6nf7d_a2b48106-9844-4431-9b6f-d4e1baacb15b/ovn-acl-logging/0.log" Jul 21 13:10:20.406424 ip-10-0-130-157 kubenswrapper[2576]: I0721 13:10:20.406393 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-skc26_6713665c-3b71-4751-929e-16de48263766/konnectivity-agent/0.log" limit=1024 Jul 21 13:10:23.796737 ip-10-0-130-157 kubenswrapper[2576]: I0721 13:10:23.796699 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-8pmg8"] Jul 21 13:10:24.729288 ip-10-0-130-157 kubenswrapper[2576]: I0721 13:10:24.729257 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-skc26_6713665c-3b71-4751-929e-16de48263766/konnectivity-agent/0.log" limit=1024 Jul 21 13:10:26.379178 ip-10-0-130-157 kubenswrapper[2576]: I0721 13:10:26.379149 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-skc26_6713665c-3b71-4751-929e-16de48263766/konnectivity-agent/0.log" limit=1024 Jul 21 13:10:26.479584 ip-10-0-130-157 kubenswrapper[2576]: I0721 13:10:26.479558 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-skc26_6713665c-3b71-4751-929e-16de48263766/konnectivity-agent/0.log" limit=1024 Jul 21 13:10:26.916928 ip-10-0-130-157 kubenswrapper[2576]: I0721 13:10:26.916895 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-skc26_6713665c-3b71-4751-929e-16de48263766/konnectivity-agent/0.log" limit=1024 Jul 21 13:10:27.018683 ip-10-0-130-157 kubenswrapper[2576]: I0721 13:10:27.018651 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-skc26_6713665c-3b71-4751-929e-16de48263766/konnectivity-agent/0.log" limit=1024 Jul 21 13:10:28.797290 ip-10-0-130-157 kubenswrapper[2576]: I0721 13:10:28.797258 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-skc26_6713665c-3b71-4751-929e-16de48263766/konnectivity-agent/0.log" limit=1024 Jul 21 13:10:28.896809 ip-10-0-130-157 kubenswrapper[2576]: I0721 13:10:28.896768 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-skc26_6713665c-3b71-4751-929e-16de48263766/konnectivity-agent/0.log" limit=1024 Jul 21 13:10:28.994191 ip-10-0-130-157 kubenswrapper[2576]: I0721 13:10:28.994159 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-skc26_6713665c-3b71-4751-929e-16de48263766/konnectivity-agent/0.log" limit=1024 Jul 21 13:10:29.889382 ip-10-0-130-157 kubenswrapper[2576]: I0721 13:10:29.889348 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-skc26_6713665c-3b71-4751-929e-16de48263766/konnectivity-agent/0.log" limit=1024 Jul 21 13:10:29.953126 ip-10-0-130-157 kubenswrapper[2576]: I0721 13:10:29.953097 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-skc26_6713665c-3b71-4751-929e-16de48263766/konnectivity-agent/0.log" limit=1024 Jul 21 13:10:32.827197 ip-10-0-130-157 kubenswrapper[2576]: I0721 13:10:32.827165 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-skc26_6713665c-3b71-4751-929e-16de48263766/konnectivity-agent/0.log" limit=1024 Jul 21 13:10:32.926497 ip-10-0-130-157 kubenswrapper[2576]: I0721 13:10:32.926461 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-skc26_6713665c-3b71-4751-929e-16de48263766/konnectivity-agent/0.log" limit=1024 Jul 21 13:10:35.306408 ip-10-0-130-157 kubenswrapper[2576]: I0721 13:10:35.306374 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-skc26_6713665c-3b71-4751-929e-16de48263766/konnectivity-agent/0.log" limit=1024 Jul 21 13:10:35.381653 ip-10-0-130-157 kubenswrapper[2576]: I0721 13:10:35.381626 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-8pmg8"] Jul 21 13:10:35.386059 ip-10-0-130-157 kubenswrapper[2576]: I0721 13:10:35.386036 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-skc26_6713665c-3b71-4751-929e-16de48263766/konnectivity-agent/0.log" limit=1024 Jul 21 13:10:50.538504 ip-10-0-130-157 kubenswrapper[2576]: I0721 13:10:50.538471 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-skc26_6713665c-3b71-4751-929e-16de48263766/konnectivity-agent/0.log" limit=1024 Jul 21 13:10:50.675246 ip-10-0-130-157 kubenswrapper[2576]: I0721 13:10:50.675214 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-skc26_6713665c-3b71-4751-929e-16de48263766/konnectivity-agent/0.log" limit=1024 Jul 21 13:11:04.275964 ip-10-0-130-157 kubenswrapper[2576]: I0721 13:11:04.275929 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-skc26_6713665c-3b71-4751-929e-16de48263766/konnectivity-agent/0.log" limit=1024 Jul 21 13:11:07.091911 ip-10-0-130-157 kubenswrapper[2576]: I0721 13:11:07.091877 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-8pmg8"] Jul 21 13:11:08.792943 ip-10-0-130-157 kubenswrapper[2576]: I0721 13:11:08.792904 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-skc26_6713665c-3b71-4751-929e-16de48263766/konnectivity-agent/0.log" limit=1024 Jul 21 13:11:08.948814 ip-10-0-130-157 kubenswrapper[2576]: I0721 13:11:08.948784 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-skc26_6713665c-3b71-4751-929e-16de48263766/konnectivity-agent/0.log" limit=1024 Jul 21 13:11:09.186786 ip-10-0-130-157 kubenswrapper[2576]: I0721 13:11:09.186760 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-skc26_6713665c-3b71-4751-929e-16de48263766/konnectivity-agent/0.log" limit=1024 Jul 21 13:11:09.659757 ip-10-0-130-157 kubenswrapper[2576]: I0721 13:11:09.659728 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-skc26_6713665c-3b71-4751-929e-16de48263766/konnectivity-agent/0.log" limit=1024 Jul 21 13:11:09.760225 ip-10-0-130-157 kubenswrapper[2576]: I0721 13:11:09.760193 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-skc26_6713665c-3b71-4751-929e-16de48263766/konnectivity-agent/0.log" limit=1024 Jul 21 13:11:09.859839 ip-10-0-130-157 kubenswrapper[2576]: I0721 13:11:09.859806 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-skc26_6713665c-3b71-4751-929e-16de48263766/konnectivity-agent/0.log" limit=1024 Jul 21 13:11:11.325605 ip-10-0-130-157 kubenswrapper[2576]: I0721 13:11:11.325570 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-skc26_6713665c-3b71-4751-929e-16de48263766/konnectivity-agent/0.log" limit=1024 Jul 21 13:11:18.750368 ip-10-0-130-157 kubenswrapper[2576]: I0721 13:11:18.750294 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-skc26_6713665c-3b71-4751-929e-16de48263766/konnectivity-agent/0.log" limit=1024 Jul 21 13:11:18.855481 ip-10-0-130-157 kubenswrapper[2576]: I0721 13:11:18.855445 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-skc26_6713665c-3b71-4751-929e-16de48263766/konnectivity-agent/0.log" limit=1024 Jul 21 13:11:18.952739 ip-10-0-130-157 kubenswrapper[2576]: I0721 13:11:18.952707 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-skc26_6713665c-3b71-4751-929e-16de48263766/konnectivity-agent/0.log" limit=1024 Jul 21 13:11:19.051890 ip-10-0-130-157 kubenswrapper[2576]: I0721 13:11:19.051808 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-skc26_6713665c-3b71-4751-929e-16de48263766/konnectivity-agent/0.log" limit=1024 Jul 21 13:11:24.607257 ip-10-0-130-157 kubenswrapper[2576]: I0721 13:11:24.607225 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-skc26_6713665c-3b71-4751-929e-16de48263766/konnectivity-agent/0.log" limit=1024 Jul 21 13:11:50.565424 ip-10-0-130-157 kubenswrapper[2576]: I0721 13:11:50.565379 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-skc26_6713665c-3b71-4751-929e-16de48263766/konnectivity-agent/0.log" limit=1024 Jul 21 13:11:57.638338 ip-10-0-130-157 kubenswrapper[2576]: I0721 13:11:57.638308 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-skc26_6713665c-3b71-4751-929e-16de48263766/konnectivity-agent/0.log" limit=1024 Jul 21 13:11:57.740132 ip-10-0-130-157 kubenswrapper[2576]: I0721 13:11:57.740098 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-skc26_6713665c-3b71-4751-929e-16de48263766/konnectivity-agent/0.log" limit=1024 Jul 21 13:11:57.838308 ip-10-0-130-157 kubenswrapper[2576]: I0721 13:11:57.838279 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-skc26_6713665c-3b71-4751-929e-16de48263766/konnectivity-agent/0.log" limit=1024 Jul 21 13:12:13.770176 ip-10-0-130-157 kubenswrapper[2576]: I0721 13:12:13.770141 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-skc26_6713665c-3b71-4751-929e-16de48263766/konnectivity-agent/0.log" limit=1024 Jul 21 13:12:20.308471 ip-10-0-130-157 kubenswrapper[2576]: I0721 13:12:20.308433 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-skc26_6713665c-3b71-4751-929e-16de48263766/konnectivity-agent/0.log" limit=1024 Jul 21 13:12:20.377620 ip-10-0-130-157 kubenswrapper[2576]: I0721 13:12:20.377589 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-skc26_6713665c-3b71-4751-929e-16de48263766/konnectivity-agent/0.log" limit=1024 Jul 21 13:12:20.478719 ip-10-0-130-157 kubenswrapper[2576]: I0721 13:12:20.478684 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-skc26_6713665c-3b71-4751-929e-16de48263766/konnectivity-agent/0.log" limit=1024 Jul 21 13:12:20.575435 ip-10-0-130-157 kubenswrapper[2576]: I0721 13:12:20.575367 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-skc26_6713665c-3b71-4751-929e-16de48263766/konnectivity-agent/0.log" limit=1024 Jul 21 13:12:31.060884 ip-10-0-130-157 kubenswrapper[2576]: I0721 13:12:31.060842 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-skc26_6713665c-3b71-4751-929e-16de48263766/konnectivity-agent/0.log" limit=1024 Jul 21 13:12:31.866269 ip-10-0-130-157 kubenswrapper[2576]: I0721 13:12:31.866235 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-skc26_6713665c-3b71-4751-929e-16de48263766/konnectivity-agent/0.log" limit=1024 Jul 21 13:12:39.206927 ip-10-0-130-157 kubenswrapper[2576]: I0721 13:12:39.206875 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-skc26_6713665c-3b71-4751-929e-16de48263766/konnectivity-agent/0.log" limit=1024 Jul 21 13:12:39.312469 ip-10-0-130-157 kubenswrapper[2576]: I0721 13:12:39.312436 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-skc26_6713665c-3b71-4751-929e-16de48263766/konnectivity-agent/0.log" limit=1024 Jul 21 13:12:39.409315 ip-10-0-130-157 kubenswrapper[2576]: I0721 13:12:39.409279 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-skc26_6713665c-3b71-4751-929e-16de48263766/konnectivity-agent/0.log" limit=1024 Jul 21 13:12:39.509489 ip-10-0-130-157 kubenswrapper[2576]: I0721 13:12:39.509386 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-skc26_6713665c-3b71-4751-929e-16de48263766/konnectivity-agent/0.log" limit=1024 Jul 21 13:12:40.277357 ip-10-0-130-157 kubenswrapper[2576]: I0721 13:12:40.277325 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-skc26_6713665c-3b71-4751-929e-16de48263766/konnectivity-agent/0.log" limit=1024 Jul 21 13:12:40.372317 ip-10-0-130-157 kubenswrapper[2576]: I0721 13:12:40.372276 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-skc26_6713665c-3b71-4751-929e-16de48263766/konnectivity-agent/0.log" limit=1024 Jul 21 13:12:40.470854 ip-10-0-130-157 kubenswrapper[2576]: I0721 13:12:40.470824 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-skc26_6713665c-3b71-4751-929e-16de48263766/konnectivity-agent/0.log" limit=1024 Jul 21 13:12:42.090381 ip-10-0-130-157 kubenswrapper[2576]: I0721 13:12:42.090341 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-skc26_6713665c-3b71-4751-929e-16de48263766/konnectivity-agent/0.log" limit=1024 Jul 21 13:12:42.497574 ip-10-0-130-157 kubenswrapper[2576]: I0721 13:12:42.497542 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-8pmg8"] Jul 21 13:12:48.702842 ip-10-0-130-157 kubenswrapper[2576]: I0721 13:12:48.702766 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-skc26_6713665c-3b71-4751-929e-16de48263766/konnectivity-agent/0.log" limit=1024 Jul 21 13:12:48.804142 ip-10-0-130-157 kubenswrapper[2576]: I0721 13:12:48.804112 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-skc26_6713665c-3b71-4751-929e-16de48263766/konnectivity-agent/0.log" limit=1024 Jul 21 13:12:49.387364 ip-10-0-130-157 kubenswrapper[2576]: I0721 13:12:49.387336 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-skc26_6713665c-3b71-4751-929e-16de48263766/konnectivity-agent/0.log" limit=1024 Jul 21 13:12:55.184046 ip-10-0-130-157 kubenswrapper[2576]: I0721 13:12:55.184007 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-8pmg8"] Jul 21 13:12:56.271542 ip-10-0-130-157 kubenswrapper[2576]: I0721 13:12:56.271496 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-skc26_6713665c-3b71-4751-929e-16de48263766/konnectivity-agent/0.log" limit=1024 Jul 21 13:12:56.370908 ip-10-0-130-157 kubenswrapper[2576]: I0721 13:12:56.370878 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-skc26_6713665c-3b71-4751-929e-16de48263766/konnectivity-agent/0.log" limit=1024 Jul 21 13:12:56.530144 ip-10-0-130-157 kubenswrapper[2576]: I0721 13:12:56.530055 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-skc26_6713665c-3b71-4751-929e-16de48263766/konnectivity-agent/0.log" limit=1024 Jul 21 13:12:56.871757 ip-10-0-130-157 kubenswrapper[2576]: I0721 13:12:56.871669 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-skc26_6713665c-3b71-4751-929e-16de48263766/konnectivity-agent/0.log" limit=1024 Jul 21 13:12:56.979483 ip-10-0-130-157 kubenswrapper[2576]: I0721 13:12:56.979449 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-skc26_6713665c-3b71-4751-929e-16de48263766/konnectivity-agent/0.log" limit=1024 Jul 21 13:13:00.239435 ip-10-0-130-157 kubenswrapper[2576]: I0721 13:13:00.239394 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-skc26_6713665c-3b71-4751-929e-16de48263766/konnectivity-agent/0.log" limit=1024 Jul 21 13:13:00.339235 ip-10-0-130-157 kubenswrapper[2576]: I0721 13:13:00.339210 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-skc26_6713665c-3b71-4751-929e-16de48263766/konnectivity-agent/0.log" limit=1024 Jul 21 13:13:00.439344 ip-10-0-130-157 kubenswrapper[2576]: I0721 13:13:00.439316 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-skc26_6713665c-3b71-4751-929e-16de48263766/konnectivity-agent/0.log" limit=1024 Jul 21 13:13:00.643723 ip-10-0-130-157 kubenswrapper[2576]: I0721 13:13:00.643696 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-skc26_6713665c-3b71-4751-929e-16de48263766/konnectivity-agent/0.log" limit=1024 Jul 21 13:13:01.065467 ip-10-0-130-157 kubenswrapper[2576]: I0721 13:13:01.065377 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-skc26_6713665c-3b71-4751-929e-16de48263766/konnectivity-agent/0.log" limit=1024 Jul 21 13:13:01.167233 ip-10-0-130-157 kubenswrapper[2576]: I0721 13:13:01.167199 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-skc26_6713665c-3b71-4751-929e-16de48263766/konnectivity-agent/0.log" limit=1024 Jul 21 13:13:04.734311 ip-10-0-130-157 kubenswrapper[2576]: I0721 13:13:04.734270 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-skc26_6713665c-3b71-4751-929e-16de48263766/konnectivity-agent/0.log" limit=1024 Jul 21 13:13:05.417413 ip-10-0-130-157 kubenswrapper[2576]: I0721 13:13:05.417382 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-skc26_6713665c-3b71-4751-929e-16de48263766/konnectivity-agent/0.log" limit=1024 Jul 21 13:13:05.517175 ip-10-0-130-157 kubenswrapper[2576]: I0721 13:13:05.517147 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-skc26_6713665c-3b71-4751-929e-16de48263766/konnectivity-agent/0.log" limit=1024 Jul 21 13:13:05.883904 ip-10-0-130-157 kubenswrapper[2576]: I0721 13:13:05.883819 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-8pmg8"] Jul 21 13:13:06.146735 ip-10-0-130-157 kubenswrapper[2576]: I0721 13:13:06.146701 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-skc26_6713665c-3b71-4751-929e-16de48263766/konnectivity-agent/0.log" limit=1024 Jul 21 13:13:12.280237 ip-10-0-130-157 kubenswrapper[2576]: I0721 13:13:12.280201 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-skc26_6713665c-3b71-4751-929e-16de48263766/konnectivity-agent/0.log" limit=1024 Jul 21 13:13:12.356532 ip-10-0-130-157 kubenswrapper[2576]: I0721 13:13:12.356487 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-skc26_6713665c-3b71-4751-929e-16de48263766/konnectivity-agent/0.log" limit=1024 Jul 21 13:13:23.533690 ip-10-0-130-157 kubenswrapper[2576]: I0721 13:13:23.533655 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-skc26_6713665c-3b71-4751-929e-16de48263766/konnectivity-agent/0.log" limit=1024 Jul 21 13:13:26.687577 ip-10-0-130-157 kubenswrapper[2576]: I0721 13:13:26.687539 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-8pmg8"] Jul 21 13:13:27.751679 ip-10-0-130-157 kubenswrapper[2576]: I0721 13:13:27.751646 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-skc26_6713665c-3b71-4751-929e-16de48263766/konnectivity-agent/0.log" limit=1024 Jul 21 13:13:28.159700 ip-10-0-130-157 kubenswrapper[2576]: I0721 13:13:28.159669 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-skc26_6713665c-3b71-4751-929e-16de48263766/konnectivity-agent/0.log" limit=1024 Jul 21 13:13:34.755441 ip-10-0-130-157 kubenswrapper[2576]: I0721 13:13:34.755406 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-skc26_6713665c-3b71-4751-929e-16de48263766/konnectivity-agent/0.log" limit=1024 Jul 21 13:13:34.829046 ip-10-0-130-157 kubenswrapper[2576]: I0721 13:13:34.829016 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-skc26_6713665c-3b71-4751-929e-16de48263766/konnectivity-agent/0.log" limit=1024 Jul 21 13:13:35.057120 ip-10-0-130-157 kubenswrapper[2576]: I0721 13:13:35.057039 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-skc26_6713665c-3b71-4751-929e-16de48263766/konnectivity-agent/0.log" limit=1024 Jul 21 13:13:35.278576 ip-10-0-130-157 kubenswrapper[2576]: I0721 13:13:35.278540 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-skc26_6713665c-3b71-4751-929e-16de48263766/konnectivity-agent/0.log" limit=1024 Jul 21 13:13:35.397296 ip-10-0-130-157 kubenswrapper[2576]: I0721 13:13:35.397268 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-skc26_6713665c-3b71-4751-929e-16de48263766/konnectivity-agent/0.log" limit=1024 Jul 21 13:13:35.501208 ip-10-0-130-157 kubenswrapper[2576]: I0721 13:13:35.501180 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-skc26_6713665c-3b71-4751-929e-16de48263766/konnectivity-agent/0.log" limit=1024 Jul 21 13:13:35.611169 ip-10-0-130-157 kubenswrapper[2576]: I0721 13:13:35.611142 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-skc26_6713665c-3b71-4751-929e-16de48263766/konnectivity-agent/0.log" limit=1024 Jul 21 13:13:41.709329 ip-10-0-130-157 kubenswrapper[2576]: I0721 13:13:41.709291 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-skc26_6713665c-3b71-4751-929e-16de48263766/konnectivity-agent/0.log" limit=1024 Jul 21 13:13:41.817891 ip-10-0-130-157 kubenswrapper[2576]: I0721 13:13:41.817856 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-skc26_6713665c-3b71-4751-929e-16de48263766/konnectivity-agent/0.log" limit=1024 Jul 21 13:13:41.913731 ip-10-0-130-157 kubenswrapper[2576]: I0721 13:13:41.913701 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-skc26_6713665c-3b71-4751-929e-16de48263766/konnectivity-agent/0.log" limit=1024 Jul 21 13:13:42.078339 ip-10-0-130-157 kubenswrapper[2576]: I0721 13:13:42.078265 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-skc26_6713665c-3b71-4751-929e-16de48263766/konnectivity-agent/0.log" limit=1024 Jul 21 13:13:48.577364 ip-10-0-130-157 kubenswrapper[2576]: I0721 13:13:48.577329 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-skc26_6713665c-3b71-4751-929e-16de48263766/konnectivity-agent/0.log" limit=1024 Jul 21 13:13:48.675767 ip-10-0-130-157 kubenswrapper[2576]: I0721 13:13:48.675731 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-skc26_6713665c-3b71-4751-929e-16de48263766/konnectivity-agent/0.log" limit=1024 Jul 21 13:13:55.417434 ip-10-0-130-157 kubenswrapper[2576]: I0721 13:13:55.417396 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-skc26_6713665c-3b71-4751-929e-16de48263766/konnectivity-agent/0.log" limit=1024 Jul 21 13:13:55.749746 ip-10-0-130-157 kubenswrapper[2576]: I0721 13:13:55.749668 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-skc26_6713665c-3b71-4751-929e-16de48263766/konnectivity-agent/0.log" limit=1024 Jul 21 13:13:55.884166 ip-10-0-130-157 kubenswrapper[2576]: I0721 13:13:55.884137 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-skc26_6713665c-3b71-4751-929e-16de48263766/konnectivity-agent/0.log" limit=1024 Jul 21 13:13:58.528975 ip-10-0-130-157 kubenswrapper[2576]: I0721 13:13:58.528940 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-skc26_6713665c-3b71-4751-929e-16de48263766/konnectivity-agent/0.log" limit=1024 Jul 21 13:13:58.629233 ip-10-0-130-157 kubenswrapper[2576]: I0721 13:13:58.629201 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-skc26_6713665c-3b71-4751-929e-16de48263766/konnectivity-agent/0.log" limit=1024 Jul 21 13:13:58.729967 ip-10-0-130-157 kubenswrapper[2576]: I0721 13:13:58.729936 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-skc26_6713665c-3b71-4751-929e-16de48263766/konnectivity-agent/0.log" limit=1024 Jul 21 13:13:58.964451 ip-10-0-130-157 kubenswrapper[2576]: I0721 13:13:58.964423 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-skc26_6713665c-3b71-4751-929e-16de48263766/konnectivity-agent/0.log" limit=1024 Jul 21 13:13:59.365733 ip-10-0-130-157 kubenswrapper[2576]: I0721 13:13:59.365649 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-skc26_6713665c-3b71-4751-929e-16de48263766/konnectivity-agent/0.log" limit=1024 Jul 21 13:13:59.466535 ip-10-0-130-157 kubenswrapper[2576]: I0721 13:13:59.466486 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-skc26_6713665c-3b71-4751-929e-16de48263766/konnectivity-agent/0.log" limit=1024 Jul 21 13:13:59.859341 ip-10-0-130-157 kubenswrapper[2576]: I0721 13:13:59.859314 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-skc26_6713665c-3b71-4751-929e-16de48263766/konnectivity-agent/0.log" limit=1024 Jul 21 13:13:59.959839 ip-10-0-130-157 kubenswrapper[2576]: I0721 13:13:59.959814 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-skc26_6713665c-3b71-4751-929e-16de48263766/konnectivity-agent/0.log" limit=1024 Jul 21 13:14:03.016729 ip-10-0-130-157 kubenswrapper[2576]: I0721 13:14:03.016681 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-skc26_6713665c-3b71-4751-929e-16de48263766/konnectivity-agent/0.log" limit=1024 Jul 21 13:14:03.626414 ip-10-0-130-157 kubenswrapper[2576]: I0721 13:14:03.626382 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-8pmg8"] Jul 21 13:14:20.158742 ip-10-0-130-157 kubenswrapper[2576]: I0721 13:14:20.158656 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-skc26_6713665c-3b71-4751-929e-16de48263766/konnectivity-agent/0.log" limit=1024 Jul 21 13:14:23.593369 ip-10-0-130-157 kubenswrapper[2576]: I0721 13:14:23.593335 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-8pmg8"] Jul 21 13:14:38.110176 ip-10-0-130-157 kubenswrapper[2576]: I0721 13:14:38.110136 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-8pmg8"] Jul 21 13:14:53.783842 ip-10-0-130-157 kubenswrapper[2576]: I0721 13:14:53.783808 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-6nf7d_a2b48106-9844-4431-9b6f-d4e1baacb15b/ovn-acl-logging/0.log" Jul 21 13:14:53.786735 ip-10-0-130-157 kubenswrapper[2576]: I0721 13:14:53.786712 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-6nf7d_a2b48106-9844-4431-9b6f-d4e1baacb15b/ovn-acl-logging/0.log" Jul 21 13:14:56.031190 ip-10-0-130-157 kubenswrapper[2576]: I0721 13:14:56.031158 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-8pmg8"] Jul 21 13:14:57.341830 ip-10-0-130-157 kubenswrapper[2576]: I0721 13:14:57.341772 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-skc26_6713665c-3b71-4751-929e-16de48263766/konnectivity-agent/0.log" limit=1024 Jul 21 13:15:00.147971 ip-10-0-130-157 kubenswrapper[2576]: I0721 13:15:00.147941 2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29743995-gp29b"] Jul 21 13:15:00.148346 ip-10-0-130-157 kubenswrapper[2576]: I0721 13:15:00.148223 2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="fa6cc249-a955-474b-ad3c-c3d420907228" containerName="main" Jul 21 13:15:00.148346 ip-10-0-130-157 kubenswrapper[2576]: I0721 13:15:00.148234 2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="fa6cc249-a955-474b-ad3c-c3d420907228" containerName="main" Jul 21 13:15:00.148346 ip-10-0-130-157 kubenswrapper[2576]: I0721 13:15:00.148249 2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4e8de9de-4d3c-4012-b131-c92d62c77e3b" containerName="main" Jul 21 13:15:00.148346 ip-10-0-130-157 kubenswrapper[2576]: I0721 13:15:00.148255 2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="4e8de9de-4d3c-4012-b131-c92d62c77e3b" containerName="main" Jul 21 13:15:00.148346 ip-10-0-130-157 kubenswrapper[2576]: I0721 13:15:00.148308 2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="fa6cc249-a955-474b-ad3c-c3d420907228" containerName="main" Jul 21 13:15:00.148346 ip-10-0-130-157 kubenswrapper[2576]: I0721 13:15:00.148316 2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="4e8de9de-4d3c-4012-b131-c92d62c77e3b" containerName="main" Jul 21 13:15:00.151003 ip-10-0-130-157 kubenswrapper[2576]: I0721 13:15:00.150974 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29743995-gp29b" Jul 21 13:15:00.154421 ip-10-0-130-157 kubenswrapper[2576]: I0721 13:15:00.154403 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-5nsrr\"" Jul 21 13:15:00.160388 ip-10-0-130-157 kubenswrapper[2576]: I0721 13:15:00.160372 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29743995-gp29b" Jul 21 13:15:00.163462 ip-10-0-130-157 kubenswrapper[2576]: I0721 13:15:00.163237 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29743995-gp29b"] Jul 21 13:15:00.288541 ip-10-0-130-157 kubenswrapper[2576]: I0721 13:15:00.288496 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29743995-gp29b"] Jul 21 13:15:00.291372 ip-10-0-130-157 kubenswrapper[2576]: W0721 13:15:00.291343 2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7556cf0b_54d0_4ed8_bb42_a2a1055d688a.slice/crio-f4b7c174982d790c1172c1a2898a78a370dc7bd149bb83e1079a757a8f9c120d WatchSource:0}: Error finding container f4b7c174982d790c1172c1a2898a78a370dc7bd149bb83e1079a757a8f9c120d: Status 404 returned error can't find the container with id f4b7c174982d790c1172c1a2898a78a370dc7bd149bb83e1079a757a8f9c120d Jul 21 13:15:00.293282 ip-10-0-130-157 kubenswrapper[2576]: I0721 13:15:00.293266 2576 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 21 13:15:00.830944 ip-10-0-130-157 kubenswrapper[2576]: I0721 13:15:00.830902 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29743995-gp29b" event={"ID":"7556cf0b-54d0-4ed8-bb42-a2a1055d688a","Type":"ContainerStarted","Data":"f4b7c174982d790c1172c1a2898a78a370dc7bd149bb83e1079a757a8f9c120d"} Jul 21 13:15:01.737726 ip-10-0-130-157 kubenswrapper[2576]: I0721 13:15:01.737686 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-skc26_6713665c-3b71-4751-929e-16de48263766/konnectivity-agent/0.log" limit=1024 Jul 21 13:15:04.047834 ip-10-0-130-157 kubenswrapper[2576]: I0721 13:15:04.047792 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-skc26_6713665c-3b71-4751-929e-16de48263766/konnectivity-agent/0.log" limit=1024 Jul 21 13:15:04.390000 ip-10-0-130-157 kubenswrapper[2576]: I0721 13:15:04.389970 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-8pmg8"] Jul 21 13:15:09.085554 ip-10-0-130-157 kubenswrapper[2576]: I0721 13:15:09.085487 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-skc26_6713665c-3b71-4751-929e-16de48263766/konnectivity-agent/0.log" limit=1024 Jul 21 13:15:12.891785 ip-10-0-130-157 kubenswrapper[2576]: I0721 13:15:12.891751 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29743995-gp29b" event={"ID":"7556cf0b-54d0-4ed8-bb42-a2a1055d688a","Type":"ContainerStarted","Data":"3c2a57aa253deec078f1818664073e4782b11f1c2c40e2d63b533b1004dec9fb"} Jul 21 13:15:12.909109 ip-10-0-130-157 kubenswrapper[2576]: I0721 13:15:12.909064 2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29743995-gp29b" podStartSLOduration=1.153196796 podStartE2EDuration="12.909049977s" totalImagesPullingTime="11.755853181s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 13:15:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 13:15:12.90800436 +0000 UTC m=+2120.743268251" watchObservedRunningTime="2026-07-21 13:15:12.909049977 +0000 UTC m=+2120.744313843" Jul 21 13:15:13.897653 ip-10-0-130-157 kubenswrapper[2576]: I0721 13:15:13.897621 2576 generic.go:356] "Generic (PLEG): container finished" podID="7556cf0b-54d0-4ed8-bb42-a2a1055d688a" containerID="3c2a57aa253deec078f1818664073e4782b11f1c2c40e2d63b533b1004dec9fb" exitCode=0 Jul 21 13:15:13.898025 ip-10-0-130-157 kubenswrapper[2576]: I0721 13:15:13.897663 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29743995-gp29b" event={"ID":"7556cf0b-54d0-4ed8-bb42-a2a1055d688a","Type":"ContainerDied","Data":"3c2a57aa253deec078f1818664073e4782b11f1c2c40e2d63b533b1004dec9fb"} Jul 21 13:15:15.024316 ip-10-0-130-157 kubenswrapper[2576]: I0721 13:15:15.024283 2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29743995-gp29b" Jul 21 13:15:15.906714 ip-10-0-130-157 kubenswrapper[2576]: I0721 13:15:15.906679 2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29743995-gp29b" Jul 21 13:15:15.906714 ip-10-0-130-157 kubenswrapper[2576]: I0721 13:15:15.906701 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29743995-gp29b" event={"ID":"7556cf0b-54d0-4ed8-bb42-a2a1055d688a","Type":"ContainerDied","Data":"f4b7c174982d790c1172c1a2898a78a370dc7bd149bb83e1079a757a8f9c120d"} Jul 21 13:15:15.906924 ip-10-0-130-157 kubenswrapper[2576]: I0721 13:15:15.906751 2576 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="f4b7c174982d790c1172c1a2898a78a370dc7bd149bb83e1079a757a8f9c120d" Jul 21 13:15:29.594919 ip-10-0-130-157 kubenswrapper[2576]: I0721 13:15:29.594883 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-skc26_6713665c-3b71-4751-929e-16de48263766/konnectivity-agent/0.log" limit=1024 Jul 21 13:15:35.534791 ip-10-0-130-157 kubenswrapper[2576]: I0721 13:15:35.534753 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-skc26_6713665c-3b71-4751-929e-16de48263766/konnectivity-agent/0.log" limit=1024 Jul 21 13:15:35.613949 ip-10-0-130-157 kubenswrapper[2576]: I0721 13:15:35.613921 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-skc26_6713665c-3b71-4751-929e-16de48263766/konnectivity-agent/0.log" limit=1024 Jul 21 13:15:45.713278 ip-10-0-130-157 kubenswrapper[2576]: I0721 13:15:45.713202 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-skc26_6713665c-3b71-4751-929e-16de48263766/konnectivity-agent/0.log" limit=1024 Jul 21 13:15:47.198663 ip-10-0-130-157 kubenswrapper[2576]: I0721 13:15:47.198616 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-skc26_6713665c-3b71-4751-929e-16de48263766/konnectivity-agent/0.log" limit=1024 Jul 21 13:15:53.450161 ip-10-0-130-157 kubenswrapper[2576]: I0721 13:15:53.450127 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-skc26_6713665c-3b71-4751-929e-16de48263766/konnectivity-agent/0.log" limit=1024 Jul 21 13:15:53.653742 ip-10-0-130-157 kubenswrapper[2576]: I0721 13:15:53.653716 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-skc26_6713665c-3b71-4751-929e-16de48263766/konnectivity-agent/0.log" limit=1024 Jul 21 13:15:53.757307 ip-10-0-130-157 kubenswrapper[2576]: I0721 13:15:53.757239 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-skc26_6713665c-3b71-4751-929e-16de48263766/konnectivity-agent/0.log" limit=1024 Jul 21 13:15:53.994058 ip-10-0-130-157 kubenswrapper[2576]: I0721 13:15:53.994030 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-skc26_6713665c-3b71-4751-929e-16de48263766/konnectivity-agent/0.log" limit=1024 Jul 21 13:15:56.577280 ip-10-0-130-157 kubenswrapper[2576]: I0721 13:15:56.577245 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-skc26_6713665c-3b71-4751-929e-16de48263766/konnectivity-agent/0.log" limit=1024 Jul 21 13:15:56.685095 ip-10-0-130-157 kubenswrapper[2576]: I0721 13:15:56.685071 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-skc26_6713665c-3b71-4751-929e-16de48263766/konnectivity-agent/0.log" limit=1024 Jul 21 13:15:57.755416 ip-10-0-130-157 kubenswrapper[2576]: I0721 13:15:57.755375 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-skc26_6713665c-3b71-4751-929e-16de48263766/konnectivity-agent/0.log" limit=1024 Jul 21 13:15:57.799078 ip-10-0-130-157 kubenswrapper[2576]: I0721 13:15:57.799047 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-8pmg8"] Jul 21 13:16:04.267203 ip-10-0-130-157 kubenswrapper[2576]: I0721 13:16:04.267169 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-skc26_6713665c-3b71-4751-929e-16de48263766/konnectivity-agent/0.log" limit=1024 Jul 21 13:16:04.367995 ip-10-0-130-157 kubenswrapper[2576]: I0721 13:16:04.367968 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-skc26_6713665c-3b71-4751-929e-16de48263766/konnectivity-agent/0.log" limit=1024 Jul 21 13:16:04.952356 ip-10-0-130-157 kubenswrapper[2576]: I0721 13:16:04.952324 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-skc26_6713665c-3b71-4751-929e-16de48263766/konnectivity-agent/0.log" limit=1024 Jul 21 13:16:08.294409 ip-10-0-130-157 kubenswrapper[2576]: I0721 13:16:08.294372 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-8pmg8"] Jul 21 13:16:12.179204 ip-10-0-130-157 kubenswrapper[2576]: I0721 13:16:12.179167 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-skc26_6713665c-3b71-4751-929e-16de48263766/konnectivity-agent/0.log" limit=1024 Jul 21 13:16:12.259807 ip-10-0-130-157 kubenswrapper[2576]: I0721 13:16:12.259774 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-skc26_6713665c-3b71-4751-929e-16de48263766/konnectivity-agent/0.log" limit=1024 Jul 21 13:16:12.345685 ip-10-0-130-157 kubenswrapper[2576]: I0721 13:16:12.345644 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-skc26_6713665c-3b71-4751-929e-16de48263766/konnectivity-agent/0.log" limit=1024 Jul 21 13:16:22.914055 ip-10-0-130-157 kubenswrapper[2576]: I0721 13:16:22.914023 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-skc26_6713665c-3b71-4751-929e-16de48263766/konnectivity-agent/0.log" limit=1024 Jul 21 13:16:23.255284 ip-10-0-130-157 kubenswrapper[2576]: I0721 13:16:23.255209 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-skc26_6713665c-3b71-4751-929e-16de48263766/konnectivity-agent/0.log" limit=1024 Jul 21 13:16:44.282773 ip-10-0-130-157 kubenswrapper[2576]: I0721 13:16:44.282733 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-skc26_6713665c-3b71-4751-929e-16de48263766/konnectivity-agent/0.log" limit=1024 Jul 21 13:16:51.390742 ip-10-0-130-157 kubenswrapper[2576]: I0721 13:16:51.390710 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-skc26_6713665c-3b71-4751-929e-16de48263766/konnectivity-agent/0.log" limit=1024 Jul 21 13:16:51.496645 ip-10-0-130-157 kubenswrapper[2576]: I0721 13:16:51.496601 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-skc26_6713665c-3b71-4751-929e-16de48263766/konnectivity-agent/0.log" limit=1024 Jul 21 13:16:51.749843 ip-10-0-130-157 kubenswrapper[2576]: I0721 13:16:51.749747 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-skc26_6713665c-3b71-4751-929e-16de48263766/konnectivity-agent/0.log" limit=1024 Jul 21 13:16:52.085175 ip-10-0-130-157 kubenswrapper[2576]: I0721 13:16:52.085097 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-skc26_6713665c-3b71-4751-929e-16de48263766/konnectivity-agent/0.log" limit=1024 Jul 21 13:16:52.165445 ip-10-0-130-157 kubenswrapper[2576]: I0721 13:16:52.165415 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-skc26_6713665c-3b71-4751-929e-16de48263766/konnectivity-agent/0.log" limit=1024 Jul 21 13:16:55.639255 ip-10-0-130-157 kubenswrapper[2576]: I0721 13:16:55.639222 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-skc26_6713665c-3b71-4751-929e-16de48263766/konnectivity-agent/0.log" limit=1024 Jul 21 13:16:55.730311 ip-10-0-130-157 kubenswrapper[2576]: I0721 13:16:55.730283 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-skc26_6713665c-3b71-4751-929e-16de48263766/konnectivity-agent/0.log" limit=1024 Jul 21 13:16:55.920727 ip-10-0-130-157 kubenswrapper[2576]: I0721 13:16:55.920656 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-skc26_6713665c-3b71-4751-929e-16de48263766/konnectivity-agent/0.log" limit=1024 Jul 21 13:16:56.221986 ip-10-0-130-157 kubenswrapper[2576]: I0721 13:16:56.221914 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-skc26_6713665c-3b71-4751-929e-16de48263766/konnectivity-agent/0.log" limit=1024 Jul 21 13:16:56.577012 ip-10-0-130-157 kubenswrapper[2576]: I0721 13:16:56.576935 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-skc26_6713665c-3b71-4751-929e-16de48263766/konnectivity-agent/0.log" limit=1024 Jul 21 13:16:56.677438 ip-10-0-130-157 kubenswrapper[2576]: I0721 13:16:56.677402 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-skc26_6713665c-3b71-4751-929e-16de48263766/konnectivity-agent/0.log" limit=1024 Jul 21 13:16:59.612128 ip-10-0-130-157 kubenswrapper[2576]: I0721 13:16:59.612092 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-skc26_6713665c-3b71-4751-929e-16de48263766/konnectivity-agent/0.log" limit=1024 Jul 21 13:17:00.180338 ip-10-0-130-157 kubenswrapper[2576]: I0721 13:17:00.180304 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-skc26_6713665c-3b71-4751-929e-16de48263766/konnectivity-agent/0.log" limit=1024 Jul 21 13:17:00.204049 ip-10-0-130-157 kubenswrapper[2576]: I0721 13:17:00.204020 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-8pmg8"] Jul 21 13:17:01.806327 ip-10-0-130-157 kubenswrapper[2576]: I0721 13:17:01.806286 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-skc26_6713665c-3b71-4751-929e-16de48263766/konnectivity-agent/0.log" limit=1024 Jul 21 13:17:01.903437 ip-10-0-130-157 kubenswrapper[2576]: I0721 13:17:01.903403 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-skc26_6713665c-3b71-4751-929e-16de48263766/konnectivity-agent/0.log" limit=1024 Jul 21 13:17:03.632728 ip-10-0-130-157 kubenswrapper[2576]: I0721 13:17:03.632696 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-skc26_6713665c-3b71-4751-929e-16de48263766/konnectivity-agent/0.log" limit=1024 Jul 21 13:17:03.732788 ip-10-0-130-157 kubenswrapper[2576]: I0721 13:17:03.732758 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-skc26_6713665c-3b71-4751-929e-16de48263766/konnectivity-agent/0.log" limit=1024 Jul 21 13:17:03.968869 ip-10-0-130-157 kubenswrapper[2576]: I0721 13:17:03.968839 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-skc26_6713665c-3b71-4751-929e-16de48263766/konnectivity-agent/0.log" limit=1024 Jul 21 13:17:04.431548 ip-10-0-130-157 kubenswrapper[2576]: I0721 13:17:04.431496 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-skc26_6713665c-3b71-4751-929e-16de48263766/konnectivity-agent/0.log" limit=1024 Jul 21 13:17:04.639009 ip-10-0-130-157 kubenswrapper[2576]: I0721 13:17:04.638979 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-skc26_6713665c-3b71-4751-929e-16de48263766/konnectivity-agent/0.log" limit=1024 Jul 21 13:17:04.736098 ip-10-0-130-157 kubenswrapper[2576]: I0721 13:17:04.736022 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-skc26_6713665c-3b71-4751-929e-16de48263766/konnectivity-agent/0.log" limit=1024 Jul 21 13:17:08.165803 ip-10-0-130-157 kubenswrapper[2576]: I0721 13:17:08.165759 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-skc26_6713665c-3b71-4751-929e-16de48263766/konnectivity-agent/0.log" limit=1024 Jul 21 13:17:10.594783 ip-10-0-130-157 kubenswrapper[2576]: I0721 13:17:10.594751 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-8pmg8"] Jul 21 13:17:15.694351 ip-10-0-130-157 kubenswrapper[2576]: I0721 13:17:15.694278 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-skc26_6713665c-3b71-4751-929e-16de48263766/konnectivity-agent/0.log" limit=1024 Jul 21 13:17:18.558551 ip-10-0-130-157 kubenswrapper[2576]: I0721 13:17:18.558505 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-skc26_6713665c-3b71-4751-929e-16de48263766/konnectivity-agent/0.log" limit=1024 Jul 21 13:17:18.667080 ip-10-0-130-157 kubenswrapper[2576]: I0721 13:17:18.667053 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-skc26_6713665c-3b71-4751-929e-16de48263766/konnectivity-agent/0.log" limit=1024 Jul 21 13:17:27.031921 ip-10-0-130-157 kubenswrapper[2576]: I0721 13:17:27.031891 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-skc26_6713665c-3b71-4751-929e-16de48263766/konnectivity-agent/0.log" limit=1024 Jul 21 13:17:27.127853 ip-10-0-130-157 kubenswrapper[2576]: I0721 13:17:27.127823 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-skc26_6713665c-3b71-4751-929e-16de48263766/konnectivity-agent/0.log" limit=1024 Jul 21 13:17:28.391907 ip-10-0-130-157 kubenswrapper[2576]: I0721 13:17:28.391866 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-8pmg8"] Jul 21 13:17:39.058009 ip-10-0-130-157 kubenswrapper[2576]: I0721 13:17:39.057977 2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-inf-b-823a18/test-model-823a18-kserve-7f799f7887-z7w9s"] Jul 21 13:17:39.058466 ip-10-0-130-157 kubenswrapper[2576]: I0721 13:17:39.058409 2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7556cf0b-54d0-4ed8-bb42-a2a1055d688a" containerName="cleanup" Jul 21 13:17:39.058466 ip-10-0-130-157 kubenswrapper[2576]: I0721 13:17:39.058426 2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="7556cf0b-54d0-4ed8-bb42-a2a1055d688a" containerName="cleanup" Jul 21 13:17:39.058619 ip-10-0-130-157 kubenswrapper[2576]: I0721 13:17:39.058556 2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="7556cf0b-54d0-4ed8-bb42-a2a1055d688a" containerName="cleanup" Jul 21 13:17:39.060503 ip-10-0-130-157 kubenswrapper[2576]: I0721 13:17:39.060482 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-823a18/test-model-823a18-kserve-7f799f7887-z7w9s" Jul 21 13:17:39.063259 ip-10-0-130-157 kubenswrapper[2576]: I0721 13:17:39.063227 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-b-823a18\"/\"test-model-823a18-kserve-self-signed-certs\"" Jul 21 13:17:39.063259 ip-10-0-130-157 kubenswrapper[2576]: I0721 13:17:39.063252 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-b-823a18\"/\"openshift-service-ca.crt\"" Jul 21 13:17:39.063418 ip-10-0-130-157 kubenswrapper[2576]: I0721 13:17:39.063254 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-b-823a18\"/\"default-dockercfg-dvq2r\"" Jul 21 13:17:39.063418 ip-10-0-130-157 kubenswrapper[2576]: I0721 13:17:39.063235 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-b-823a18\"/\"kube-root-ca.crt\"" Jul 21 13:17:39.072647 ip-10-0-130-157 kubenswrapper[2576]: I0721 13:17:39.072623 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-b-823a18/test-model-823a18-kserve-7f799f7887-z7w9s"] Jul 21 13:17:39.096670 ip-10-0-130-157 kubenswrapper[2576]: I0721 13:17:39.096627 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-skc26_6713665c-3b71-4751-929e-16de48263766/konnectivity-agent/0.log" limit=1024 Jul 21 13:17:39.210290 ip-10-0-130-157 kubenswrapper[2576]: I0721 13:17:39.210256 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d2095623-f543-47b5-b2c1-6282534832b8-dshm\") pod \"test-model-823a18-kserve-7f799f7887-z7w9s\" (UID: \"d2095623-f543-47b5-b2c1-6282534832b8\") " pod="ai-tenant-e2e-inf-b-823a18/test-model-823a18-kserve-7f799f7887-z7w9s" Jul 21 13:17:39.210396 ip-10-0-130-157 kubenswrapper[2576]: I0721 13:17:39.210307 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rkb5l\" (UniqueName: \"kubernetes.io/projected/d2095623-f543-47b5-b2c1-6282534832b8-kube-api-access-rkb5l\") pod \"test-model-823a18-kserve-7f799f7887-z7w9s\" (UID: \"d2095623-f543-47b5-b2c1-6282534832b8\") " pod="ai-tenant-e2e-inf-b-823a18/test-model-823a18-kserve-7f799f7887-z7w9s" Jul 21 13:17:39.210396 ip-10-0-130-157 kubenswrapper[2576]: I0721 13:17:39.210358 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d2095623-f543-47b5-b2c1-6282534832b8-model-cache\") pod \"test-model-823a18-kserve-7f799f7887-z7w9s\" (UID: \"d2095623-f543-47b5-b2c1-6282534832b8\") " pod="ai-tenant-e2e-inf-b-823a18/test-model-823a18-kserve-7f799f7887-z7w9s" Jul 21 13:17:39.210396 ip-10-0-130-157 kubenswrapper[2576]: I0721 13:17:39.210379 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d2095623-f543-47b5-b2c1-6282534832b8-tmp-dir\") pod \"test-model-823a18-kserve-7f799f7887-z7w9s\" (UID: \"d2095623-f543-47b5-b2c1-6282534832b8\") " pod="ai-tenant-e2e-inf-b-823a18/test-model-823a18-kserve-7f799f7887-z7w9s" Jul 21 13:17:39.210542 ip-10-0-130-157 kubenswrapper[2576]: I0721 13:17:39.210401 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d2095623-f543-47b5-b2c1-6282534832b8-home\") pod \"test-model-823a18-kserve-7f799f7887-z7w9s\" (UID: \"d2095623-f543-47b5-b2c1-6282534832b8\") " pod="ai-tenant-e2e-inf-b-823a18/test-model-823a18-kserve-7f799f7887-z7w9s" Jul 21 13:17:39.210542 ip-10-0-130-157 kubenswrapper[2576]: I0721 13:17:39.210429 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d2095623-f543-47b5-b2c1-6282534832b8-tls-certs\") pod \"test-model-823a18-kserve-7f799f7887-z7w9s\" (UID: \"d2095623-f543-47b5-b2c1-6282534832b8\") " pod="ai-tenant-e2e-inf-b-823a18/test-model-823a18-kserve-7f799f7887-z7w9s" Jul 21 13:17:39.311630 ip-10-0-130-157 kubenswrapper[2576]: I0721 13:17:39.311567 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d2095623-f543-47b5-b2c1-6282534832b8-dshm\") pod \"test-model-823a18-kserve-7f799f7887-z7w9s\" (UID: \"d2095623-f543-47b5-b2c1-6282534832b8\") " pod="ai-tenant-e2e-inf-b-823a18/test-model-823a18-kserve-7f799f7887-z7w9s" Jul 21 13:17:39.311738 ip-10-0-130-157 kubenswrapper[2576]: I0721 13:17:39.311625 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rkb5l\" (UniqueName: \"kubernetes.io/projected/d2095623-f543-47b5-b2c1-6282534832b8-kube-api-access-rkb5l\") pod \"test-model-823a18-kserve-7f799f7887-z7w9s\" (UID: \"d2095623-f543-47b5-b2c1-6282534832b8\") " pod="ai-tenant-e2e-inf-b-823a18/test-model-823a18-kserve-7f799f7887-z7w9s" Jul 21 13:17:39.311738 ip-10-0-130-157 kubenswrapper[2576]: I0721 13:17:39.311677 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d2095623-f543-47b5-b2c1-6282534832b8-model-cache\") pod \"test-model-823a18-kserve-7f799f7887-z7w9s\" (UID: \"d2095623-f543-47b5-b2c1-6282534832b8\") " pod="ai-tenant-e2e-inf-b-823a18/test-model-823a18-kserve-7f799f7887-z7w9s" Jul 21 13:17:39.311738 ip-10-0-130-157 kubenswrapper[2576]: I0721 13:17:39.311712 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d2095623-f543-47b5-b2c1-6282534832b8-tmp-dir\") pod \"test-model-823a18-kserve-7f799f7887-z7w9s\" (UID: \"d2095623-f543-47b5-b2c1-6282534832b8\") " pod="ai-tenant-e2e-inf-b-823a18/test-model-823a18-kserve-7f799f7887-z7w9s" Jul 21 13:17:39.311905 ip-10-0-130-157 kubenswrapper[2576]: I0721 13:17:39.311735 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d2095623-f543-47b5-b2c1-6282534832b8-home\") pod \"test-model-823a18-kserve-7f799f7887-z7w9s\" (UID: \"d2095623-f543-47b5-b2c1-6282534832b8\") " pod="ai-tenant-e2e-inf-b-823a18/test-model-823a18-kserve-7f799f7887-z7w9s" Jul 21 13:17:39.311905 ip-10-0-130-157 kubenswrapper[2576]: I0721 13:17:39.311769 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d2095623-f543-47b5-b2c1-6282534832b8-tls-certs\") pod \"test-model-823a18-kserve-7f799f7887-z7w9s\" (UID: \"d2095623-f543-47b5-b2c1-6282534832b8\") " pod="ai-tenant-e2e-inf-b-823a18/test-model-823a18-kserve-7f799f7887-z7w9s" Jul 21 13:17:39.312052 ip-10-0-130-157 kubenswrapper[2576]: I0721 13:17:39.312030 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d2095623-f543-47b5-b2c1-6282534832b8-model-cache\") pod \"test-model-823a18-kserve-7f799f7887-z7w9s\" (UID: \"d2095623-f543-47b5-b2c1-6282534832b8\") " pod="ai-tenant-e2e-inf-b-823a18/test-model-823a18-kserve-7f799f7887-z7w9s" Jul 21 13:17:39.312119 ip-10-0-130-157 kubenswrapper[2576]: I0721 13:17:39.312089 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d2095623-f543-47b5-b2c1-6282534832b8-tmp-dir\") pod \"test-model-823a18-kserve-7f799f7887-z7w9s\" (UID: \"d2095623-f543-47b5-b2c1-6282534832b8\") " pod="ai-tenant-e2e-inf-b-823a18/test-model-823a18-kserve-7f799f7887-z7w9s" Jul 21 13:17:39.312119 ip-10-0-130-157 kubenswrapper[2576]: I0721 13:17:39.312106 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d2095623-f543-47b5-b2c1-6282534832b8-home\") pod \"test-model-823a18-kserve-7f799f7887-z7w9s\" (UID: \"d2095623-f543-47b5-b2c1-6282534832b8\") " pod="ai-tenant-e2e-inf-b-823a18/test-model-823a18-kserve-7f799f7887-z7w9s" Jul 21 13:17:39.313810 ip-10-0-130-157 kubenswrapper[2576]: I0721 13:17:39.313786 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d2095623-f543-47b5-b2c1-6282534832b8-dshm\") pod \"test-model-823a18-kserve-7f799f7887-z7w9s\" (UID: \"d2095623-f543-47b5-b2c1-6282534832b8\") " pod="ai-tenant-e2e-inf-b-823a18/test-model-823a18-kserve-7f799f7887-z7w9s" Jul 21 13:17:39.314168 ip-10-0-130-157 kubenswrapper[2576]: I0721 13:17:39.314149 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d2095623-f543-47b5-b2c1-6282534832b8-tls-certs\") pod \"test-model-823a18-kserve-7f799f7887-z7w9s\" (UID: \"d2095623-f543-47b5-b2c1-6282534832b8\") " pod="ai-tenant-e2e-inf-b-823a18/test-model-823a18-kserve-7f799f7887-z7w9s" Jul 21 13:17:39.320359 ip-10-0-130-157 kubenswrapper[2576]: I0721 13:17:39.320335 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rkb5l\" (UniqueName: \"kubernetes.io/projected/d2095623-f543-47b5-b2c1-6282534832b8-kube-api-access-rkb5l\") pod \"test-model-823a18-kserve-7f799f7887-z7w9s\" (UID: \"d2095623-f543-47b5-b2c1-6282534832b8\") " pod="ai-tenant-e2e-inf-b-823a18/test-model-823a18-kserve-7f799f7887-z7w9s" Jul 21 13:17:39.369472 ip-10-0-130-157 kubenswrapper[2576]: I0721 13:17:39.369449 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-823a18/test-model-823a18-kserve-7f799f7887-z7w9s" Jul 21 13:17:39.493544 ip-10-0-130-157 kubenswrapper[2576]: I0721 13:17:39.493496 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-b-823a18/test-model-823a18-kserve-7f799f7887-z7w9s"] Jul 21 13:17:39.497380 ip-10-0-130-157 kubenswrapper[2576]: W0721 13:17:39.497349 2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podd2095623_f543_47b5_b2c1_6282534832b8.slice/crio-76abefdde0a85be7966251036e3d63cb628e6450a8c6d73aded65ea6e8ca2700 WatchSource:0}: Error finding container 76abefdde0a85be7966251036e3d63cb628e6450a8c6d73aded65ea6e8ca2700: Status 404 returned error can't find the container with id 76abefdde0a85be7966251036e3d63cb628e6450a8c6d73aded65ea6e8ca2700 Jul 21 13:17:39.518185 ip-10-0-130-157 kubenswrapper[2576]: I0721 13:17:39.518156 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-823a18/test-model-823a18-kserve-7f799f7887-z7w9s" event={"ID":"d2095623-f543-47b5-b2c1-6282534832b8","Type":"ContainerStarted","Data":"76abefdde0a85be7966251036e3d63cb628e6450a8c6d73aded65ea6e8ca2700"} Jul 21 13:17:40.523086 ip-10-0-130-157 kubenswrapper[2576]: I0721 13:17:40.523053 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-823a18/test-model-823a18-kserve-7f799f7887-z7w9s" event={"ID":"d2095623-f543-47b5-b2c1-6282534832b8","Type":"ContainerStarted","Data":"b2343fae27dfffa9ff75087bbf42bdcef49c2c22e6af53dc2776a394f3a21604"} Jul 21 13:17:40.542719 ip-10-0-130-157 kubenswrapper[2576]: I0721 13:17:40.542679 2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-inf-b-823a18/test-model-823a18-kserve-7f799f7887-z7w9s" podStartSLOduration=1.542664712 podStartE2EDuration="1.542664712s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 13:17:39 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 13:17:40.54177634 +0000 UTC m=+2268.377040204" watchObservedRunningTime="2026-07-21 13:17:40.542664712 +0000 UTC m=+2268.377928590" Jul 21 13:17:45.686530 ip-10-0-130-157 kubenswrapper[2576]: I0721 13:17:45.686478 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-8pmg8"] Jul 21 13:17:49.370531 ip-10-0-130-157 kubenswrapper[2576]: I0721 13:17:49.370490 2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-inf-b-823a18/test-model-823a18-kserve-7f799f7887-z7w9s" Jul 21 13:17:49.370912 ip-10-0-130-157 kubenswrapper[2576]: I0721 13:17:49.370565 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-inf-b-823a18/test-model-823a18-kserve-7f799f7887-z7w9s" Jul 21 13:17:49.382645 ip-10-0-130-157 kubenswrapper[2576]: I0721 13:17:49.382622 2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-inf-b-823a18/test-model-823a18-kserve-7f799f7887-z7w9s" Jul 21 13:17:49.569422 ip-10-0-130-157 kubenswrapper[2576]: I0721 13:17:49.569395 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-inf-b-823a18/test-model-823a18-kserve-7f799f7887-z7w9s" Jul 21 13:18:17.881076 ip-10-0-130-157 kubenswrapper[2576]: I0721 13:18:17.881041 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-skc26_6713665c-3b71-4751-929e-16de48263766/konnectivity-agent/0.log" limit=1024 Jul 21 13:18:18.782369 ip-10-0-130-157 kubenswrapper[2576]: I0721 13:18:18.782337 2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-b-823a18/test-model-823a18-kserve-7f799f7887-z7w9s"] Jul 21 13:18:18.934536 ip-10-0-130-157 kubenswrapper[2576]: I0721 13:18:18.934488 2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-360ca4-6psx2"] Jul 21 13:18:18.938028 ip-10-0-130-157 kubenswrapper[2576]: I0721 13:18:18.938002 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-360ca4-6psx2" Jul 21 13:18:18.940208 ip-10-0-130-157 kubenswrapper[2576]: I0721 13:18:18.940187 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\"" Jul 21 13:18:18.940335 ip-10-0-130-157 kubenswrapper[2576]: I0721 13:18:18.940189 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-5nsrr\"" Jul 21 13:18:18.943477 ip-10-0-130-157 kubenswrapper[2576]: I0721 13:18:18.943451 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-360ca4-6psx2"] Jul 21 13:18:18.992777 ip-10-0-130-157 kubenswrapper[2576]: I0721 13:18:18.992750 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/5cba722c-4219-463d-94dd-bf920a0828ea-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-a-360ca4-6psx2\" (UID: \"5cba722c-4219-463d-94dd-bf920a0828ea\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-360ca4-6psx2" Jul 21 13:18:19.093669 ip-10-0-130-157 kubenswrapper[2576]: I0721 13:18:19.093614 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/5cba722c-4219-463d-94dd-bf920a0828ea-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-a-360ca4-6psx2\" (UID: \"5cba722c-4219-463d-94dd-bf920a0828ea\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-360ca4-6psx2" Jul 21 13:18:19.094167 ip-10-0-130-157 kubenswrapper[2576]: I0721 13:18:19.094148 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/5cba722c-4219-463d-94dd-bf920a0828ea-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-a-360ca4-6psx2\" (UID: \"5cba722c-4219-463d-94dd-bf920a0828ea\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-360ca4-6psx2" Jul 21 13:18:19.249690 ip-10-0-130-157 kubenswrapper[2576]: I0721 13:18:19.249668 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-360ca4-6psx2" Jul 21 13:18:19.372908 ip-10-0-130-157 kubenswrapper[2576]: I0721 13:18:19.372790 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-360ca4-6psx2"] Jul 21 13:18:19.375572 ip-10-0-130-157 kubenswrapper[2576]: W0721 13:18:19.375546 2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5cba722c_4219_463d_94dd_bf920a0828ea.slice/crio-f7b316009c92250e4371014c339b21cc9f18ea03cbe3f5333795b9fa49140db0 WatchSource:0}: Error finding container f7b316009c92250e4371014c339b21cc9f18ea03cbe3f5333795b9fa49140db0: Status 404 returned error can't find the container with id f7b316009c92250e4371014c339b21cc9f18ea03cbe3f5333795b9fa49140db0 Jul 21 13:18:19.689482 ip-10-0-130-157 kubenswrapper[2576]: I0721 13:18:19.689455 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-360ca4-6psx2" event={"ID":"5cba722c-4219-463d-94dd-bf920a0828ea","Type":"ContainerStarted","Data":"36c94ac152f1858ebcce4b4618c6c8c5abb3bb923624f50d701154b66cae519f"} Jul 21 13:18:19.689621 ip-10-0-130-157 kubenswrapper[2576]: I0721 13:18:19.689510 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-360ca4-6psx2" event={"ID":"5cba722c-4219-463d-94dd-bf920a0828ea","Type":"ContainerStarted","Data":"f7b316009c92250e4371014c339b21cc9f18ea03cbe3f5333795b9fa49140db0"} Jul 21 13:18:20.552181 ip-10-0-130-157 kubenswrapper[2576]: I0721 13:18:20.552147 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-skc26_6713665c-3b71-4751-929e-16de48263766/konnectivity-agent/0.log" limit=1024 Jul 21 13:18:20.695132 ip-10-0-130-157 kubenswrapper[2576]: I0721 13:18:20.695103 2576 generic.go:356] "Generic (PLEG): container finished" podID="5cba722c-4219-463d-94dd-bf920a0828ea" containerID="36c94ac152f1858ebcce4b4618c6c8c5abb3bb923624f50d701154b66cae519f" exitCode=0 Jul 21 13:18:20.695416 ip-10-0-130-157 kubenswrapper[2576]: I0721 13:18:20.695159 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-360ca4-6psx2" event={"ID":"5cba722c-4219-463d-94dd-bf920a0828ea","Type":"ContainerDied","Data":"36c94ac152f1858ebcce4b4618c6c8c5abb3bb923624f50d701154b66cae519f"} Jul 21 13:18:21.827972 ip-10-0-130-157 kubenswrapper[2576]: I0721 13:18:21.827950 2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-360ca4-6psx2" Jul 21 13:18:21.911014 ip-10-0-130-157 kubenswrapper[2576]: I0721 13:18:21.910991 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/5cba722c-4219-463d-94dd-bf920a0828ea-maas-api-service-ca\") pod \"5cba722c-4219-463d-94dd-bf920a0828ea\" (UID: \"5cba722c-4219-463d-94dd-bf920a0828ea\") " Jul 21 13:18:21.911313 ip-10-0-130-157 kubenswrapper[2576]: I0721 13:18:21.911292 2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5cba722c-4219-463d-94dd-bf920a0828ea-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "5cba722c-4219-463d-94dd-bf920a0828ea" (UID: "5cba722c-4219-463d-94dd-bf920a0828ea"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 21 13:18:22.012167 ip-10-0-130-157 kubenswrapper[2576]: I0721 13:18:22.012102 2576 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/5cba722c-4219-463d-94dd-bf920a0828ea-maas-api-service-ca\") on node \"ip-10-0-130-157.ec2.internal\" DevicePath \"\"" Jul 21 13:18:22.585880 ip-10-0-130-157 kubenswrapper[2576]: I0721 13:18:22.585805 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-8pmg8"] Jul 21 13:18:22.705587 ip-10-0-130-157 kubenswrapper[2576]: I0721 13:18:22.705550 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-360ca4-6psx2" event={"ID":"5cba722c-4219-463d-94dd-bf920a0828ea","Type":"ContainerDied","Data":"f7b316009c92250e4371014c339b21cc9f18ea03cbe3f5333795b9fa49140db0"} Jul 21 13:18:22.705729 ip-10-0-130-157 kubenswrapper[2576]: I0721 13:18:22.705608 2576 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="f7b316009c92250e4371014c339b21cc9f18ea03cbe3f5333795b9fa49140db0" Jul 21 13:18:22.705729 ip-10-0-130-157 kubenswrapper[2576]: I0721 13:18:22.705565 2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-360ca4-6psx2" Jul 21 13:18:30.625277 ip-10-0-130-157 kubenswrapper[2576]: I0721 13:18:30.625248 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-skc26_6713665c-3b71-4751-929e-16de48263766/konnectivity-agent/0.log" limit=1024 Jul 21 13:18:33.802004 ip-10-0-130-157 kubenswrapper[2576]: I0721 13:18:33.801968 2576 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-inf-b-823a18/test-model-823a18-kserve-7f799f7887-z7w9s" podUID="d2095623-f543-47b5-b2c1-6282534832b8" containerName="main" containerID="cri-o://b2343fae27dfffa9ff75087bbf42bdcef49c2c22e6af53dc2776a394f3a21604" gracePeriod=45 Jul 21 13:18:34.044115 ip-10-0-130-157 kubenswrapper[2576]: I0721 13:18:34.044092 2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-823a18/test-model-823a18-kserve-7f799f7887-z7w9s" Jul 21 13:18:34.060067 ip-10-0-130-157 kubenswrapper[2576]: I0721 13:18:34.060018 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-skc26_6713665c-3b71-4751-929e-16de48263766/konnectivity-agent/0.log" limit=1024 Jul 21 13:18:34.084482 ip-10-0-130-157 kubenswrapper[2576]: I0721 13:18:34.084458 2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-360ca4-6psx2"] Jul 21 13:18:34.089330 ip-10-0-130-157 kubenswrapper[2576]: I0721 13:18:34.089311 2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-360ca4-6psx2"] Jul 21 13:18:34.104359 ip-10-0-130-157 kubenswrapper[2576]: I0721 13:18:34.104337 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d2095623-f543-47b5-b2c1-6282534832b8-model-cache\") pod \"d2095623-f543-47b5-b2c1-6282534832b8\" (UID: \"d2095623-f543-47b5-b2c1-6282534832b8\") " Jul 21 13:18:34.104436 ip-10-0-130-157 kubenswrapper[2576]: I0721 13:18:34.104384 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d2095623-f543-47b5-b2c1-6282534832b8-tls-certs\") pod \"d2095623-f543-47b5-b2c1-6282534832b8\" (UID: \"d2095623-f543-47b5-b2c1-6282534832b8\") " Jul 21 13:18:34.104436 ip-10-0-130-157 kubenswrapper[2576]: I0721 13:18:34.104405 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d2095623-f543-47b5-b2c1-6282534832b8-home\") pod \"d2095623-f543-47b5-b2c1-6282534832b8\" (UID: \"d2095623-f543-47b5-b2c1-6282534832b8\") " Jul 21 13:18:34.104583 ip-10-0-130-157 kubenswrapper[2576]: I0721 13:18:34.104442 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-rkb5l\" (UniqueName: \"kubernetes.io/projected/d2095623-f543-47b5-b2c1-6282534832b8-kube-api-access-rkb5l\") pod \"d2095623-f543-47b5-b2c1-6282534832b8\" (UID: \"d2095623-f543-47b5-b2c1-6282534832b8\") " Jul 21 13:18:34.104583 ip-10-0-130-157 kubenswrapper[2576]: I0721 13:18:34.104471 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d2095623-f543-47b5-b2c1-6282534832b8-dshm\") pod \"d2095623-f543-47b5-b2c1-6282534832b8\" (UID: \"d2095623-f543-47b5-b2c1-6282534832b8\") " Jul 21 13:18:34.104583 ip-10-0-130-157 kubenswrapper[2576]: I0721 13:18:34.104539 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d2095623-f543-47b5-b2c1-6282534832b8-tmp-dir\") pod \"d2095623-f543-47b5-b2c1-6282534832b8\" (UID: \"d2095623-f543-47b5-b2c1-6282534832b8\") " Jul 21 13:18:34.104750 ip-10-0-130-157 kubenswrapper[2576]: I0721 13:18:34.104666 2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d2095623-f543-47b5-b2c1-6282534832b8-model-cache" (OuterVolumeSpecName: "model-cache") pod "d2095623-f543-47b5-b2c1-6282534832b8" (UID: "d2095623-f543-47b5-b2c1-6282534832b8"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 21 13:18:34.104750 ip-10-0-130-157 kubenswrapper[2576]: I0721 13:18:34.104679 2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d2095623-f543-47b5-b2c1-6282534832b8-home" (OuterVolumeSpecName: "home") pod "d2095623-f543-47b5-b2c1-6282534832b8" (UID: "d2095623-f543-47b5-b2c1-6282534832b8"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 21 13:18:34.104870 ip-10-0-130-157 kubenswrapper[2576]: I0721 13:18:34.104851 2576 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d2095623-f543-47b5-b2c1-6282534832b8-home\") on node \"ip-10-0-130-157.ec2.internal\" DevicePath \"\"" Jul 21 13:18:34.104919 ip-10-0-130-157 kubenswrapper[2576]: I0721 13:18:34.104876 2576 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d2095623-f543-47b5-b2c1-6282534832b8-model-cache\") on node \"ip-10-0-130-157.ec2.internal\" DevicePath \"\"" Jul 21 13:18:34.104919 ip-10-0-130-157 kubenswrapper[2576]: I0721 13:18:34.104852 2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d2095623-f543-47b5-b2c1-6282534832b8-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "d2095623-f543-47b5-b2c1-6282534832b8" (UID: "d2095623-f543-47b5-b2c1-6282534832b8"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 21 13:18:34.106698 ip-10-0-130-157 kubenswrapper[2576]: I0721 13:18:34.106676 2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d2095623-f543-47b5-b2c1-6282534832b8-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "d2095623-f543-47b5-b2c1-6282534832b8" (UID: "d2095623-f543-47b5-b2c1-6282534832b8"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 21 13:18:34.106698 ip-10-0-130-157 kubenswrapper[2576]: I0721 13:18:34.106682 2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d2095623-f543-47b5-b2c1-6282534832b8-dshm" (OuterVolumeSpecName: "dshm") pod "d2095623-f543-47b5-b2c1-6282534832b8" (UID: "d2095623-f543-47b5-b2c1-6282534832b8"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 21 13:18:34.106698 ip-10-0-130-157 kubenswrapper[2576]: I0721 13:18:34.106691 2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d2095623-f543-47b5-b2c1-6282534832b8-kube-api-access-rkb5l" (OuterVolumeSpecName: "kube-api-access-rkb5l") pod "d2095623-f543-47b5-b2c1-6282534832b8" (UID: "d2095623-f543-47b5-b2c1-6282534832b8"). InnerVolumeSpecName "kube-api-access-rkb5l". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 21 13:18:34.205781 ip-10-0-130-157 kubenswrapper[2576]: I0721 13:18:34.205758 2576 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d2095623-f543-47b5-b2c1-6282534832b8-tls-certs\") on node \"ip-10-0-130-157.ec2.internal\" DevicePath \"\"" Jul 21 13:18:34.205781 ip-10-0-130-157 kubenswrapper[2576]: I0721 13:18:34.205778 2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-rkb5l\" (UniqueName: \"kubernetes.io/projected/d2095623-f543-47b5-b2c1-6282534832b8-kube-api-access-rkb5l\") on node \"ip-10-0-130-157.ec2.internal\" DevicePath \"\"" Jul 21 13:18:34.205899 ip-10-0-130-157 kubenswrapper[2576]: I0721 13:18:34.205788 2576 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d2095623-f543-47b5-b2c1-6282534832b8-dshm\") on node \"ip-10-0-130-157.ec2.internal\" DevicePath \"\"" Jul 21 13:18:34.205899 ip-10-0-130-157 kubenswrapper[2576]: I0721 13:18:34.205815 2576 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d2095623-f543-47b5-b2c1-6282534832b8-tmp-dir\") on node \"ip-10-0-130-157.ec2.internal\" DevicePath \"\"" Jul 21 13:18:34.315606 ip-10-0-130-157 kubenswrapper[2576]: I0721 13:18:34.315556 2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="5cba722c-4219-463d-94dd-bf920a0828ea" path="/var/lib/kubelet/pods/5cba722c-4219-463d-94dd-bf920a0828ea/volumes" Jul 21 13:18:34.482885 ip-10-0-130-157 kubenswrapper[2576]: I0721 13:18:34.482856 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-skc26_6713665c-3b71-4751-929e-16de48263766/konnectivity-agent/0.log" limit=1024 Jul 21 13:18:34.757009 ip-10-0-130-157 kubenswrapper[2576]: I0721 13:18:34.756973 2576 generic.go:356] "Generic (PLEG): container finished" podID="d2095623-f543-47b5-b2c1-6282534832b8" containerID="b2343fae27dfffa9ff75087bbf42bdcef49c2c22e6af53dc2776a394f3a21604" exitCode=0 Jul 21 13:18:34.757175 ip-10-0-130-157 kubenswrapper[2576]: I0721 13:18:34.757030 2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-823a18/test-model-823a18-kserve-7f799f7887-z7w9s" Jul 21 13:18:34.757175 ip-10-0-130-157 kubenswrapper[2576]: I0721 13:18:34.757032 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-823a18/test-model-823a18-kserve-7f799f7887-z7w9s" event={"ID":"d2095623-f543-47b5-b2c1-6282534832b8","Type":"ContainerDied","Data":"b2343fae27dfffa9ff75087bbf42bdcef49c2c22e6af53dc2776a394f3a21604"} Jul 21 13:18:34.757175 ip-10-0-130-157 kubenswrapper[2576]: I0721 13:18:34.757145 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-823a18/test-model-823a18-kserve-7f799f7887-z7w9s" event={"ID":"d2095623-f543-47b5-b2c1-6282534832b8","Type":"ContainerDied","Data":"76abefdde0a85be7966251036e3d63cb628e6450a8c6d73aded65ea6e8ca2700"} Jul 21 13:18:34.757175 ip-10-0-130-157 kubenswrapper[2576]: I0721 13:18:34.757169 2576 scope.go:117] "RemoveContainer" containerID="b2343fae27dfffa9ff75087bbf42bdcef49c2c22e6af53dc2776a394f3a21604" Jul 21 13:18:34.766993 ip-10-0-130-157 kubenswrapper[2576]: I0721 13:18:34.766954 2576 scope.go:117] "RemoveContainer" containerID="b2343fae27dfffa9ff75087bbf42bdcef49c2c22e6af53dc2776a394f3a21604" Jul 21 13:18:34.767264 ip-10-0-130-157 kubenswrapper[2576]: E0721 13:18:34.767245 2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b2343fae27dfffa9ff75087bbf42bdcef49c2c22e6af53dc2776a394f3a21604\": container with ID starting with b2343fae27dfffa9ff75087bbf42bdcef49c2c22e6af53dc2776a394f3a21604 not found: ID does not exist" containerID="b2343fae27dfffa9ff75087bbf42bdcef49c2c22e6af53dc2776a394f3a21604" Jul 21 13:18:34.767329 ip-10-0-130-157 kubenswrapper[2576]: I0721 13:18:34.767271 2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b2343fae27dfffa9ff75087bbf42bdcef49c2c22e6af53dc2776a394f3a21604"} err="failed to get container status \"b2343fae27dfffa9ff75087bbf42bdcef49c2c22e6af53dc2776a394f3a21604\": rpc error: code = NotFound desc = could not find container \"b2343fae27dfffa9ff75087bbf42bdcef49c2c22e6af53dc2776a394f3a21604\": container with ID starting with b2343fae27dfffa9ff75087bbf42bdcef49c2c22e6af53dc2776a394f3a21604 not found: ID does not exist" Jul 21 13:18:34.773967 ip-10-0-130-157 kubenswrapper[2576]: I0721 13:18:34.773944 2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-b-823a18/test-model-823a18-kserve-7f799f7887-z7w9s"] Jul 21 13:18:34.780413 ip-10-0-130-157 kubenswrapper[2576]: I0721 13:18:34.780392 2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-inf-b-823a18/test-model-823a18-kserve-7f799f7887-z7w9s"] Jul 21 13:18:36.315907 ip-10-0-130-157 kubenswrapper[2576]: I0721 13:18:36.315874 2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d2095623-f543-47b5-b2c1-6282534832b8" path="/var/lib/kubelet/pods/d2095623-f543-47b5-b2c1-6282534832b8/volumes" Jul 21 13:18:38.894754 ip-10-0-130-157 kubenswrapper[2576]: I0721 13:18:38.894719 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-8pmg8"] Jul 21 13:18:41.147053 ip-10-0-130-157 kubenswrapper[2576]: I0721 13:18:41.147016 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-skc26_6713665c-3b71-4751-929e-16de48263766/konnectivity-agent/0.log" limit=1024 Jul 21 13:18:41.261228 ip-10-0-130-157 kubenswrapper[2576]: I0721 13:18:41.261186 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-skc26_6713665c-3b71-4751-929e-16de48263766/konnectivity-agent/0.log" limit=1024 Jul 21 13:18:41.464583 ip-10-0-130-157 kubenswrapper[2576]: I0721 13:18:41.464487 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-skc26_6713665c-3b71-4751-929e-16de48263766/konnectivity-agent/0.log" limit=1024 Jul 21 13:18:41.696975 ip-10-0-130-157 kubenswrapper[2576]: I0721 13:18:41.696931 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-skc26_6713665c-3b71-4751-929e-16de48263766/konnectivity-agent/0.log" limit=1024 Jul 21 13:18:41.796266 ip-10-0-130-157 kubenswrapper[2576]: I0721 13:18:41.796150 2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-823a18-9rk5m"] Jul 21 13:18:41.796462 ip-10-0-130-157 kubenswrapper[2576]: I0721 13:18:41.796449 2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5cba722c-4219-463d-94dd-bf920a0828ea" containerName="revoke-keys" Jul 21 13:18:41.796538 ip-10-0-130-157 kubenswrapper[2576]: I0721 13:18:41.796463 2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="5cba722c-4219-463d-94dd-bf920a0828ea" containerName="revoke-keys" Jul 21 13:18:41.796538 ip-10-0-130-157 kubenswrapper[2576]: I0721 13:18:41.796487 2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d2095623-f543-47b5-b2c1-6282534832b8" containerName="main" Jul 21 13:18:41.796538 ip-10-0-130-157 kubenswrapper[2576]: I0721 13:18:41.796493 2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="d2095623-f543-47b5-b2c1-6282534832b8" containerName="main" Jul 21 13:18:41.796649 ip-10-0-130-157 kubenswrapper[2576]: I0721 13:18:41.796572 2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="5cba722c-4219-463d-94dd-bf920a0828ea" containerName="revoke-keys" Jul 21 13:18:41.796649 ip-10-0-130-157 kubenswrapper[2576]: I0721 13:18:41.796582 2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="d2095623-f543-47b5-b2c1-6282534832b8" containerName="main" Jul 21 13:18:41.800828 ip-10-0-130-157 kubenswrapper[2576]: I0721 13:18:41.800809 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-823a18-9rk5m" Jul 21 13:18:41.803237 ip-10-0-130-157 kubenswrapper[2576]: I0721 13:18:41.803218 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\"" Jul 21 13:18:41.803331 ip-10-0-130-157 kubenswrapper[2576]: I0721 13:18:41.803218 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-5nsrr\"" Jul 21 13:18:41.808692 ip-10-0-130-157 kubenswrapper[2576]: I0721 13:18:41.808421 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-823a18-9rk5m"] Jul 21 13:18:41.866164 ip-10-0-130-157 kubenswrapper[2576]: I0721 13:18:41.866141 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/42a14bc4-06fc-4a51-9f8f-8c72ce8104d6-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-b-823a18-9rk5m\" (UID: \"42a14bc4-06fc-4a51-9f8f-8c72ce8104d6\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-823a18-9rk5m" Jul 21 13:18:41.967461 ip-10-0-130-157 kubenswrapper[2576]: I0721 13:18:41.967433 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/42a14bc4-06fc-4a51-9f8f-8c72ce8104d6-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-b-823a18-9rk5m\" (UID: \"42a14bc4-06fc-4a51-9f8f-8c72ce8104d6\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-823a18-9rk5m" Jul 21 13:18:41.968034 ip-10-0-130-157 kubenswrapper[2576]: I0721 13:18:41.968016 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/42a14bc4-06fc-4a51-9f8f-8c72ce8104d6-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-b-823a18-9rk5m\" (UID: \"42a14bc4-06fc-4a51-9f8f-8c72ce8104d6\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-823a18-9rk5m" Jul 21 13:18:42.112030 ip-10-0-130-157 kubenswrapper[2576]: I0721 13:18:42.111971 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-823a18-9rk5m" Jul 21 13:18:42.246792 ip-10-0-130-157 kubenswrapper[2576]: I0721 13:18:42.246747 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-823a18-9rk5m"] Jul 21 13:18:42.250944 ip-10-0-130-157 kubenswrapper[2576]: W0721 13:18:42.250918 2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod42a14bc4_06fc_4a51_9f8f_8c72ce8104d6.slice/crio-fda0994c5690ee902448310658a99a714c152c81bd3888d44120173d0492637e WatchSource:0}: Error finding container fda0994c5690ee902448310658a99a714c152c81bd3888d44120173d0492637e: Status 404 returned error can't find the container with id fda0994c5690ee902448310658a99a714c152c81bd3888d44120173d0492637e Jul 21 13:18:42.688578 ip-10-0-130-157 kubenswrapper[2576]: I0721 13:18:42.688541 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-skc26_6713665c-3b71-4751-929e-16de48263766/konnectivity-agent/0.log" limit=1024 Jul 21 13:18:42.794242 ip-10-0-130-157 kubenswrapper[2576]: I0721 13:18:42.794158 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-skc26_6713665c-3b71-4751-929e-16de48263766/konnectivity-agent/0.log" limit=1024 Jul 21 13:18:42.795204 ip-10-0-130-157 kubenswrapper[2576]: I0721 13:18:42.795178 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-823a18-9rk5m" event={"ID":"42a14bc4-06fc-4a51-9f8f-8c72ce8104d6","Type":"ContainerStarted","Data":"a628c296182018d88f49e9e40665ce177561e609817e28b06e5f1402700c561b"} Jul 21 13:18:42.795338 ip-10-0-130-157 kubenswrapper[2576]: I0721 13:18:42.795230 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-823a18-9rk5m" event={"ID":"42a14bc4-06fc-4a51-9f8f-8c72ce8104d6","Type":"ContainerStarted","Data":"fda0994c5690ee902448310658a99a714c152c81bd3888d44120173d0492637e"} Jul 21 13:18:42.816836 ip-10-0-130-157 kubenswrapper[2576]: I0721 13:18:42.816788 2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-823a18-9rk5m" podStartSLOduration=1.816771187 podStartE2EDuration="1.816771187s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 13:18:41 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 13:18:42.816057346 +0000 UTC m=+2330.651321213" watchObservedRunningTime="2026-07-21 13:18:42.816771187 +0000 UTC m=+2330.652035054" Jul 21 13:18:43.801079 ip-10-0-130-157 kubenswrapper[2576]: I0721 13:18:43.801044 2576 generic.go:356] "Generic (PLEG): container finished" podID="42a14bc4-06fc-4a51-9f8f-8c72ce8104d6" containerID="a628c296182018d88f49e9e40665ce177561e609817e28b06e5f1402700c561b" exitCode=0 Jul 21 13:18:43.801469 ip-10-0-130-157 kubenswrapper[2576]: I0721 13:18:43.801128 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-823a18-9rk5m" event={"ID":"42a14bc4-06fc-4a51-9f8f-8c72ce8104d6","Type":"ContainerDied","Data":"a628c296182018d88f49e9e40665ce177561e609817e28b06e5f1402700c561b"} Jul 21 13:18:44.927199 ip-10-0-130-157 kubenswrapper[2576]: I0721 13:18:44.927172 2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-823a18-9rk5m" Jul 21 13:18:44.990698 ip-10-0-130-157 kubenswrapper[2576]: I0721 13:18:44.990661 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/42a14bc4-06fc-4a51-9f8f-8c72ce8104d6-maas-api-service-ca\") pod \"42a14bc4-06fc-4a51-9f8f-8c72ce8104d6\" (UID: \"42a14bc4-06fc-4a51-9f8f-8c72ce8104d6\") " Jul 21 13:18:44.991037 ip-10-0-130-157 kubenswrapper[2576]: I0721 13:18:44.991018 2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/42a14bc4-06fc-4a51-9f8f-8c72ce8104d6-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "42a14bc4-06fc-4a51-9f8f-8c72ce8104d6" (UID: "42a14bc4-06fc-4a51-9f8f-8c72ce8104d6"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 21 13:18:45.091642 ip-10-0-130-157 kubenswrapper[2576]: I0721 13:18:45.091564 2576 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/42a14bc4-06fc-4a51-9f8f-8c72ce8104d6-maas-api-service-ca\") on node \"ip-10-0-130-157.ec2.internal\" DevicePath \"\"" Jul 21 13:18:45.810709 ip-10-0-130-157 kubenswrapper[2576]: I0721 13:18:45.810680 2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-823a18-9rk5m" Jul 21 13:18:45.810709 ip-10-0-130-157 kubenswrapper[2576]: I0721 13:18:45.810696 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-823a18-9rk5m" event={"ID":"42a14bc4-06fc-4a51-9f8f-8c72ce8104d6","Type":"ContainerDied","Data":"fda0994c5690ee902448310658a99a714c152c81bd3888d44120173d0492637e"} Jul 21 13:18:45.810930 ip-10-0-130-157 kubenswrapper[2576]: I0721 13:18:45.810738 2576 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="fda0994c5690ee902448310658a99a714c152c81bd3888d44120173d0492637e" Jul 21 13:18:49.212556 ip-10-0-130-157 kubenswrapper[2576]: I0721 13:18:49.212511 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-skc26_6713665c-3b71-4751-929e-16de48263766/konnectivity-agent/0.log" limit=1024 Jul 21 13:18:49.303174 ip-10-0-130-157 kubenswrapper[2576]: I0721 13:18:49.303144 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-skc26_6713665c-3b71-4751-929e-16de48263766/konnectivity-agent/0.log" limit=1024 Jul 21 13:18:52.390533 ip-10-0-130-157 kubenswrapper[2576]: I0721 13:18:52.390482 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-skc26_6713665c-3b71-4751-929e-16de48263766/konnectivity-agent/0.log" limit=1024 Jul 21 13:18:55.485694 ip-10-0-130-157 kubenswrapper[2576]: I0721 13:18:55.485658 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-8pmg8"] Jul 21 13:18:56.901005 ip-10-0-130-157 kubenswrapper[2576]: I0721 13:18:56.900973 2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-823a18-9rk5m"] Jul 21 13:18:56.902016 ip-10-0-130-157 kubenswrapper[2576]: I0721 13:18:56.901992 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-skc26_6713665c-3b71-4751-929e-16de48263766/konnectivity-agent/0.log" limit=1024 Jul 21 13:18:56.904479 ip-10-0-130-157 kubenswrapper[2576]: I0721 13:18:56.904457 2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-823a18-9rk5m"] Jul 21 13:18:57.290576 ip-10-0-130-157 kubenswrapper[2576]: I0721 13:18:57.290489 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-skc26_6713665c-3b71-4751-929e-16de48263766/konnectivity-agent/0.log" limit=1024 Jul 21 13:18:58.315762 ip-10-0-130-157 kubenswrapper[2576]: I0721 13:18:58.315726 2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="42a14bc4-06fc-4a51-9f8f-8c72ce8104d6" path="/var/lib/kubelet/pods/42a14bc4-06fc-4a51-9f8f-8c72ce8104d6/volumes" Jul 21 13:19:03.940472 ip-10-0-130-157 kubenswrapper[2576]: I0721 13:19:03.940438 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-skc26_6713665c-3b71-4751-929e-16de48263766/konnectivity-agent/0.log" limit=1024 Jul 21 13:19:04.139497 ip-10-0-130-157 kubenswrapper[2576]: I0721 13:19:04.139465 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-skc26_6713665c-3b71-4751-929e-16de48263766/konnectivity-agent/0.log" limit=1024 Jul 21 13:19:04.239629 ip-10-0-130-157 kubenswrapper[2576]: I0721 13:19:04.239555 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-skc26_6713665c-3b71-4751-929e-16de48263766/konnectivity-agent/0.log" limit=1024 Jul 21 13:19:04.495299 ip-10-0-130-157 kubenswrapper[2576]: I0721 13:19:04.495225 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-skc26_6713665c-3b71-4751-929e-16de48263766/konnectivity-agent/0.log" limit=1024 Jul 21 13:19:04.853158 ip-10-0-130-157 kubenswrapper[2576]: I0721 13:19:04.853084 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-skc26_6713665c-3b71-4751-929e-16de48263766/konnectivity-agent/0.log" limit=1024 Jul 21 13:19:05.261284 ip-10-0-130-157 kubenswrapper[2576]: I0721 13:19:05.261255 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-skc26_6713665c-3b71-4751-929e-16de48263766/konnectivity-agent/0.log" limit=1024 Jul 21 13:19:05.692628 ip-10-0-130-157 kubenswrapper[2576]: I0721 13:19:05.692572 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-skc26_6713665c-3b71-4751-929e-16de48263766/konnectivity-agent/0.log" limit=1024 Jul 21 13:19:10.010913 ip-10-0-130-157 kubenswrapper[2576]: I0721 13:19:10.010877 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-skc26_6713665c-3b71-4751-929e-16de48263766/konnectivity-agent/0.log" limit=1024 Jul 21 13:19:10.102727 ip-10-0-130-157 kubenswrapper[2576]: I0721 13:19:10.102699 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-skc26_6713665c-3b71-4751-929e-16de48263766/konnectivity-agent/0.log" limit=1024 Jul 21 13:19:13.182768 ip-10-0-130-157 kubenswrapper[2576]: I0721 13:19:13.182732 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-skc26_6713665c-3b71-4751-929e-16de48263766/konnectivity-agent/0.log" limit=1024 Jul 21 13:19:13.304198 ip-10-0-130-157 kubenswrapper[2576]: I0721 13:19:13.304167 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-skc26_6713665c-3b71-4751-929e-16de48263766/konnectivity-agent/0.log" limit=1024 Jul 21 13:19:13.708074 ip-10-0-130-157 kubenswrapper[2576]: I0721 13:19:13.708043 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-skc26_6713665c-3b71-4751-929e-16de48263766/konnectivity-agent/0.log" limit=1024 Jul 21 13:19:14.123869 ip-10-0-130-157 kubenswrapper[2576]: I0721 13:19:14.123795 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-skc26_6713665c-3b71-4751-929e-16de48263766/konnectivity-agent/0.log" limit=1024 Jul 21 13:19:14.461247 ip-10-0-130-157 kubenswrapper[2576]: I0721 13:19:14.461220 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-skc26_6713665c-3b71-4751-929e-16de48263766/konnectivity-agent/0.log" limit=1024 Jul 21 13:19:14.486220 ip-10-0-130-157 kubenswrapper[2576]: I0721 13:19:14.486198 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-8pmg8"] Jul 21 13:19:17.428802 ip-10-0-130-157 kubenswrapper[2576]: I0721 13:19:17.428771 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-skc26_6713665c-3b71-4751-929e-16de48263766/konnectivity-agent/0.log" limit=1024 Jul 21 13:19:17.542739 ip-10-0-130-157 kubenswrapper[2576]: I0721 13:19:17.542694 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-skc26_6713665c-3b71-4751-929e-16de48263766/konnectivity-agent/0.log" limit=1024 Jul 21 13:19:17.641873 ip-10-0-130-157 kubenswrapper[2576]: I0721 13:19:17.641832 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-skc26_6713665c-3b71-4751-929e-16de48263766/konnectivity-agent/0.log" limit=1024 Jul 21 13:19:19.035823 ip-10-0-130-157 kubenswrapper[2576]: I0721 13:19:19.035787 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-skc26_6713665c-3b71-4751-929e-16de48263766/konnectivity-agent/0.log" limit=1024 Jul 21 13:19:19.138394 ip-10-0-130-157 kubenswrapper[2576]: I0721 13:19:19.138365 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-skc26_6713665c-3b71-4751-929e-16de48263766/konnectivity-agent/0.log" limit=1024 Jul 21 13:19:21.928685 ip-10-0-130-157 kubenswrapper[2576]: I0721 13:19:21.928651 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-skc26_6713665c-3b71-4751-929e-16de48263766/konnectivity-agent/0.log" limit=1024 Jul 21 13:19:22.728201 ip-10-0-130-157 kubenswrapper[2576]: I0721 13:19:22.728165 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-8pmg8"] Jul 21 13:19:25.783742 ip-10-0-130-157 kubenswrapper[2576]: I0721 13:19:25.783706 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-skc26_6713665c-3b71-4751-929e-16de48263766/konnectivity-agent/0.log" limit=1024 Jul 21 13:19:31.066123 ip-10-0-130-157 kubenswrapper[2576]: I0721 13:19:31.066087 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-skc26_6713665c-3b71-4751-929e-16de48263766/konnectivity-agent/0.log" limit=1024 Jul 21 13:19:31.166024 ip-10-0-130-157 kubenswrapper[2576]: I0721 13:19:31.165995 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-skc26_6713665c-3b71-4751-929e-16de48263766/konnectivity-agent/0.log" limit=1024 Jul 21 13:19:39.369461 ip-10-0-130-157 kubenswrapper[2576]: I0721 13:19:39.369425 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-skc26_6713665c-3b71-4751-929e-16de48263766/konnectivity-agent/0.log" limit=1024 Jul 21 13:19:39.469897 ip-10-0-130-157 kubenswrapper[2576]: I0721 13:19:39.469864 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-skc26_6713665c-3b71-4751-929e-16de48263766/konnectivity-agent/0.log" limit=1024 Jul 21 13:19:45.386058 ip-10-0-130-157 kubenswrapper[2576]: I0721 13:19:45.386020 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-skc26_6713665c-3b71-4751-929e-16de48263766/konnectivity-agent/0.log" limit=1024 Jul 21 13:19:48.489560 ip-10-0-130-157 kubenswrapper[2576]: I0721 13:19:48.489525 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-8pmg8"] Jul 21 13:19:50.330668 ip-10-0-130-157 kubenswrapper[2576]: I0721 13:19:50.330630 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-skc26_6713665c-3b71-4751-929e-16de48263766/konnectivity-agent/0.log" limit=1024 Jul 21 13:19:50.773842 ip-10-0-130-157 kubenswrapper[2576]: I0721 13:19:50.773800 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-skc26_6713665c-3b71-4751-929e-16de48263766/konnectivity-agent/0.log" limit=1024 Jul 21 13:19:53.808589 ip-10-0-130-157 kubenswrapper[2576]: I0721 13:19:53.808556 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-6nf7d_a2b48106-9844-4431-9b6f-d4e1baacb15b/ovn-acl-logging/0.log" Jul 21 13:19:53.812000 ip-10-0-130-157 kubenswrapper[2576]: I0721 13:19:53.811975 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-6nf7d_a2b48106-9844-4431-9b6f-d4e1baacb15b/ovn-acl-logging/0.log" Jul 21 13:19:57.727917 ip-10-0-130-157 kubenswrapper[2576]: I0721 13:19:57.727883 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-skc26_6713665c-3b71-4751-929e-16de48263766/konnectivity-agent/0.log" limit=1024 Jul 21 13:19:57.805499 ip-10-0-130-157 kubenswrapper[2576]: I0721 13:19:57.805474 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-skc26_6713665c-3b71-4751-929e-16de48263766/konnectivity-agent/0.log" limit=1024 Jul 21 13:19:58.013342 ip-10-0-130-157 kubenswrapper[2576]: I0721 13:19:58.013273 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-skc26_6713665c-3b71-4751-929e-16de48263766/konnectivity-agent/0.log" limit=1024 Jul 21 13:19:58.225492 ip-10-0-130-157 kubenswrapper[2576]: I0721 13:19:58.225458 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-skc26_6713665c-3b71-4751-929e-16de48263766/konnectivity-agent/0.log" limit=1024 Jul 21 13:20:14.066247 ip-10-0-130-157 kubenswrapper[2576]: I0721 13:20:14.066212 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-skc26_6713665c-3b71-4751-929e-16de48263766/konnectivity-agent/0.log" limit=1024 Jul 21 13:20:18.484327 ip-10-0-130-157 kubenswrapper[2576]: I0721 13:20:18.484295 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-skc26_6713665c-3b71-4751-929e-16de48263766/konnectivity-agent/0.log" limit=1024 Jul 21 13:20:18.919586 ip-10-0-130-157 kubenswrapper[2576]: I0721 13:20:18.919556 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-skc26_6713665c-3b71-4751-929e-16de48263766/konnectivity-agent/0.log" limit=1024 Jul 21 13:20:19.305107 ip-10-0-130-157 kubenswrapper[2576]: I0721 13:20:19.305027 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-8pmg8"] Jul 21 13:20:25.765490 ip-10-0-130-157 kubenswrapper[2576]: I0721 13:20:25.765449 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-skc26_6713665c-3b71-4751-929e-16de48263766/konnectivity-agent/0.log" limit=1024 Jul 21 13:20:26.291909 ip-10-0-130-157 kubenswrapper[2576]: I0721 13:20:26.291878 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-skc26_6713665c-3b71-4751-929e-16de48263766/konnectivity-agent/0.log" limit=1024 Jul 21 13:20:26.400765 ip-10-0-130-157 kubenswrapper[2576]: I0721 13:20:26.400735 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-skc26_6713665c-3b71-4751-929e-16de48263766/konnectivity-agent/0.log" limit=1024 Jul 21 13:20:26.666850 ip-10-0-130-157 kubenswrapper[2576]: I0721 13:20:26.666819 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-skc26_6713665c-3b71-4751-929e-16de48263766/konnectivity-agent/0.log" limit=1024 Jul 21 13:20:27.046971 ip-10-0-130-157 kubenswrapper[2576]: I0721 13:20:27.046902 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-skc26_6713665c-3b71-4751-929e-16de48263766/konnectivity-agent/0.log" limit=1024 Jul 21 13:20:27.335282 ip-10-0-130-157 kubenswrapper[2576]: I0721 13:20:27.335204 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-skc26_6713665c-3b71-4751-929e-16de48263766/konnectivity-agent/0.log" limit=1024 Jul 21 13:20:30.124263 ip-10-0-130-157 kubenswrapper[2576]: I0721 13:20:30.124229 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-skc26_6713665c-3b71-4751-929e-16de48263766/konnectivity-agent/0.log" limit=1024 Jul 21 13:20:30.226740 ip-10-0-130-157 kubenswrapper[2576]: I0721 13:20:30.226709 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-skc26_6713665c-3b71-4751-929e-16de48263766/konnectivity-agent/0.log" limit=1024 Jul 21 13:20:30.334553 ip-10-0-130-157 kubenswrapper[2576]: I0721 13:20:30.334503 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-skc26_6713665c-3b71-4751-929e-16de48263766/konnectivity-agent/0.log" limit=1024 Jul 21 13:20:30.609386 ip-10-0-130-157 kubenswrapper[2576]: I0721 13:20:30.609361 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-skc26_6713665c-3b71-4751-929e-16de48263766/konnectivity-agent/0.log" limit=1024 Jul 21 13:20:31.134441 ip-10-0-130-157 kubenswrapper[2576]: I0721 13:20:31.134408 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-skc26_6713665c-3b71-4751-929e-16de48263766/konnectivity-agent/0.log" limit=1024 Jul 21 13:20:31.235909 ip-10-0-130-157 kubenswrapper[2576]: I0721 13:20:31.235866 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-skc26_6713665c-3b71-4751-929e-16de48263766/konnectivity-agent/0.log" limit=1024 Jul 21 13:20:34.621975 ip-10-0-130-157 kubenswrapper[2576]: I0721 13:20:34.621943 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-skc26_6713665c-3b71-4751-929e-16de48263766/konnectivity-agent/0.log" limit=1024 Jul 21 13:20:34.915325 ip-10-0-130-157 kubenswrapper[2576]: I0721 13:20:34.915294 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-skc26_6713665c-3b71-4751-929e-16de48263766/konnectivity-agent/0.log" limit=1024 Jul 21 13:20:35.480589 ip-10-0-130-157 kubenswrapper[2576]: I0721 13:20:35.480557 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-8pmg8"] Jul 21 13:20:35.585333 ip-10-0-130-157 kubenswrapper[2576]: I0721 13:20:35.585309 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-skc26_6713665c-3b71-4751-929e-16de48263766/konnectivity-agent/0.log" limit=1024 Jul 21 13:20:35.677843 ip-10-0-130-157 kubenswrapper[2576]: I0721 13:20:35.677820 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-skc26_6713665c-3b71-4751-929e-16de48263766/konnectivity-agent/0.log" limit=1024 Jul 21 13:20:37.900345 ip-10-0-130-157 kubenswrapper[2576]: I0721 13:20:37.900302 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-skc26_6713665c-3b71-4751-929e-16de48263766/konnectivity-agent/0.log" limit=1024 Jul 21 13:20:38.000047 ip-10-0-130-157 kubenswrapper[2576]: I0721 13:20:38.000018 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-skc26_6713665c-3b71-4751-929e-16de48263766/konnectivity-agent/0.log" limit=1024 Jul 21 13:20:38.100476 ip-10-0-130-157 kubenswrapper[2576]: I0721 13:20:38.100449 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-skc26_6713665c-3b71-4751-929e-16de48263766/konnectivity-agent/0.log" limit=1024 Jul 21 13:20:38.201913 ip-10-0-130-157 kubenswrapper[2576]: I0721 13:20:38.201850 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-skc26_6713665c-3b71-4751-929e-16de48263766/konnectivity-agent/0.log" limit=1024 Jul 21 13:20:38.714346 ip-10-0-130-157 kubenswrapper[2576]: I0721 13:20:38.714317 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-skc26_6713665c-3b71-4751-929e-16de48263766/konnectivity-agent/0.log" limit=1024 Jul 21 13:20:38.816206 ip-10-0-130-157 kubenswrapper[2576]: I0721 13:20:38.816180 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-skc26_6713665c-3b71-4751-929e-16de48263766/konnectivity-agent/0.log" limit=1024 Jul 21 13:20:42.454895 ip-10-0-130-157 kubenswrapper[2576]: I0721 13:20:42.454858 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-skc26_6713665c-3b71-4751-929e-16de48263766/konnectivity-agent/0.log" limit=1024 Jul 21 13:20:49.802725 ip-10-0-130-157 kubenswrapper[2576]: I0721 13:20:49.802681 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-8pmg8"] Jul 21 13:20:52.111156 ip-10-0-130-157 kubenswrapper[2576]: I0721 13:20:52.111123 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-skc26_6713665c-3b71-4751-929e-16de48263766/konnectivity-agent/0.log" limit=1024 Jul 21 13:20:52.208151 ip-10-0-130-157 kubenswrapper[2576]: I0721 13:20:52.208122 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-skc26_6713665c-3b71-4751-929e-16de48263766/konnectivity-agent/0.log" limit=1024 Jul 21 13:20:59.712203 ip-10-0-130-157 kubenswrapper[2576]: I0721 13:20:59.712167 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-skc26_6713665c-3b71-4751-929e-16de48263766/konnectivity-agent/0.log" limit=1024 Jul 21 13:20:59.815940 ip-10-0-130-157 kubenswrapper[2576]: I0721 13:20:59.815914 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-skc26_6713665c-3b71-4751-929e-16de48263766/konnectivity-agent/0.log" limit=1024 Jul 21 13:21:00.518146 ip-10-0-130-157 kubenswrapper[2576]: I0721 13:21:00.518097 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-skc26_6713665c-3b71-4751-929e-16de48263766/konnectivity-agent/0.log" limit=1024 Jul 21 13:21:03.086093 ip-10-0-130-157 kubenswrapper[2576]: I0721 13:21:03.086058 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-8pmg8"] Jul 21 13:21:28.566650 ip-10-0-130-157 kubenswrapper[2576]: I0721 13:21:28.566617 2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-b-cfed01/auth-test-model-cfed01-kserve-76fcd9c9dc-hj85s"] Jul 21 13:21:28.567097 ip-10-0-130-157 kubenswrapper[2576]: I0721 13:21:28.566903 2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="42a14bc4-06fc-4a51-9f8f-8c72ce8104d6" containerName="revoke-keys" Jul 21 13:21:28.567097 ip-10-0-130-157 kubenswrapper[2576]: I0721 13:21:28.566914 2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="42a14bc4-06fc-4a51-9f8f-8c72ce8104d6" containerName="revoke-keys" Jul 21 13:21:28.567097 ip-10-0-130-157 kubenswrapper[2576]: I0721 13:21:28.566984 2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="42a14bc4-06fc-4a51-9f8f-8c72ce8104d6" containerName="revoke-keys" Jul 21 13:21:28.569698 ip-10-0-130-157 kubenswrapper[2576]: I0721 13:21:28.569680 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-cfed01/auth-test-model-cfed01-kserve-76fcd9c9dc-hj85s" Jul 21 13:21:28.576180 ip-10-0-130-157 kubenswrapper[2576]: I0721 13:21:28.576157 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-cfed01\"/\"default-dockercfg-4qzr4\"" Jul 21 13:21:28.576375 ip-10-0-130-157 kubenswrapper[2576]: I0721 13:21:28.576175 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-cfed01\"/\"openshift-service-ca.crt\"" Jul 21 13:21:28.576375 ip-10-0-130-157 kubenswrapper[2576]: I0721 13:21:28.576180 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-cfed01\"/\"kube-root-ca.crt\"" Jul 21 13:21:28.576375 ip-10-0-130-157 kubenswrapper[2576]: I0721 13:21:28.576260 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-cfed01\"/\"auth-test-model-cfed01-kserve-self-signed-certs\"" Jul 21 13:21:28.584804 ip-10-0-130-157 kubenswrapper[2576]: I0721 13:21:28.584784 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-cfed01/auth-test-model-cfed01-kserve-76fcd9c9dc-hj85s"] Jul 21 13:21:28.610859 ip-10-0-130-157 kubenswrapper[2576]: I0721 13:21:28.610834 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/22d0d372-952e-41b4-a3fc-23b3f3abe858-model-cache\") pod \"auth-test-model-cfed01-kserve-76fcd9c9dc-hj85s\" (UID: \"22d0d372-952e-41b4-a3fc-23b3f3abe858\") " pod="ai-tenant-e2e-shared-b-cfed01/auth-test-model-cfed01-kserve-76fcd9c9dc-hj85s" Jul 21 13:21:28.610948 ip-10-0-130-157 kubenswrapper[2576]: I0721 13:21:28.610863 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qg47c\" (UniqueName: \"kubernetes.io/projected/22d0d372-952e-41b4-a3fc-23b3f3abe858-kube-api-access-qg47c\") pod \"auth-test-model-cfed01-kserve-76fcd9c9dc-hj85s\" (UID: \"22d0d372-952e-41b4-a3fc-23b3f3abe858\") " pod="ai-tenant-e2e-shared-b-cfed01/auth-test-model-cfed01-kserve-76fcd9c9dc-hj85s" Jul 21 13:21:28.610990 ip-10-0-130-157 kubenswrapper[2576]: I0721 13:21:28.610962 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/22d0d372-952e-41b4-a3fc-23b3f3abe858-home\") pod \"auth-test-model-cfed01-kserve-76fcd9c9dc-hj85s\" (UID: \"22d0d372-952e-41b4-a3fc-23b3f3abe858\") " pod="ai-tenant-e2e-shared-b-cfed01/auth-test-model-cfed01-kserve-76fcd9c9dc-hj85s" Jul 21 13:21:28.611026 ip-10-0-130-157 kubenswrapper[2576]: I0721 13:21:28.610989 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/22d0d372-952e-41b4-a3fc-23b3f3abe858-tls-certs\") pod \"auth-test-model-cfed01-kserve-76fcd9c9dc-hj85s\" (UID: \"22d0d372-952e-41b4-a3fc-23b3f3abe858\") " pod="ai-tenant-e2e-shared-b-cfed01/auth-test-model-cfed01-kserve-76fcd9c9dc-hj85s" Jul 21 13:21:28.611026 ip-10-0-130-157 kubenswrapper[2576]: I0721 13:21:28.611009 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/22d0d372-952e-41b4-a3fc-23b3f3abe858-dshm\") pod \"auth-test-model-cfed01-kserve-76fcd9c9dc-hj85s\" (UID: \"22d0d372-952e-41b4-a3fc-23b3f3abe858\") " pod="ai-tenant-e2e-shared-b-cfed01/auth-test-model-cfed01-kserve-76fcd9c9dc-hj85s" Jul 21 13:21:28.611094 ip-10-0-130-157 kubenswrapper[2576]: I0721 13:21:28.611036 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/22d0d372-952e-41b4-a3fc-23b3f3abe858-tmp-dir\") pod \"auth-test-model-cfed01-kserve-76fcd9c9dc-hj85s\" (UID: \"22d0d372-952e-41b4-a3fc-23b3f3abe858\") " pod="ai-tenant-e2e-shared-b-cfed01/auth-test-model-cfed01-kserve-76fcd9c9dc-hj85s" Jul 21 13:21:28.711875 ip-10-0-130-157 kubenswrapper[2576]: I0721 13:21:28.711844 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/22d0d372-952e-41b4-a3fc-23b3f3abe858-model-cache\") pod \"auth-test-model-cfed01-kserve-76fcd9c9dc-hj85s\" (UID: \"22d0d372-952e-41b4-a3fc-23b3f3abe858\") " pod="ai-tenant-e2e-shared-b-cfed01/auth-test-model-cfed01-kserve-76fcd9c9dc-hj85s" Jul 21 13:21:28.711875 ip-10-0-130-157 kubenswrapper[2576]: I0721 13:21:28.711877 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qg47c\" (UniqueName: \"kubernetes.io/projected/22d0d372-952e-41b4-a3fc-23b3f3abe858-kube-api-access-qg47c\") pod \"auth-test-model-cfed01-kserve-76fcd9c9dc-hj85s\" (UID: \"22d0d372-952e-41b4-a3fc-23b3f3abe858\") " pod="ai-tenant-e2e-shared-b-cfed01/auth-test-model-cfed01-kserve-76fcd9c9dc-hj85s" Jul 21 13:21:28.712044 ip-10-0-130-157 kubenswrapper[2576]: I0721 13:21:28.711929 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/22d0d372-952e-41b4-a3fc-23b3f3abe858-home\") pod \"auth-test-model-cfed01-kserve-76fcd9c9dc-hj85s\" (UID: \"22d0d372-952e-41b4-a3fc-23b3f3abe858\") " pod="ai-tenant-e2e-shared-b-cfed01/auth-test-model-cfed01-kserve-76fcd9c9dc-hj85s" Jul 21 13:21:28.712044 ip-10-0-130-157 kubenswrapper[2576]: I0721 13:21:28.711956 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/22d0d372-952e-41b4-a3fc-23b3f3abe858-tls-certs\") pod \"auth-test-model-cfed01-kserve-76fcd9c9dc-hj85s\" (UID: \"22d0d372-952e-41b4-a3fc-23b3f3abe858\") " pod="ai-tenant-e2e-shared-b-cfed01/auth-test-model-cfed01-kserve-76fcd9c9dc-hj85s" Jul 21 13:21:28.712044 ip-10-0-130-157 kubenswrapper[2576]: I0721 13:21:28.711999 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/22d0d372-952e-41b4-a3fc-23b3f3abe858-dshm\") pod \"auth-test-model-cfed01-kserve-76fcd9c9dc-hj85s\" (UID: \"22d0d372-952e-41b4-a3fc-23b3f3abe858\") " pod="ai-tenant-e2e-shared-b-cfed01/auth-test-model-cfed01-kserve-76fcd9c9dc-hj85s" Jul 21 13:21:28.712044 ip-10-0-130-157 kubenswrapper[2576]: I0721 13:21:28.712034 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/22d0d372-952e-41b4-a3fc-23b3f3abe858-tmp-dir\") pod \"auth-test-model-cfed01-kserve-76fcd9c9dc-hj85s\" (UID: \"22d0d372-952e-41b4-a3fc-23b3f3abe858\") " pod="ai-tenant-e2e-shared-b-cfed01/auth-test-model-cfed01-kserve-76fcd9c9dc-hj85s" Jul 21 13:21:28.712303 ip-10-0-130-157 kubenswrapper[2576]: I0721 13:21:28.712280 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/22d0d372-952e-41b4-a3fc-23b3f3abe858-home\") pod \"auth-test-model-cfed01-kserve-76fcd9c9dc-hj85s\" (UID: \"22d0d372-952e-41b4-a3fc-23b3f3abe858\") " pod="ai-tenant-e2e-shared-b-cfed01/auth-test-model-cfed01-kserve-76fcd9c9dc-hj85s" Jul 21 13:21:28.712373 ip-10-0-130-157 kubenswrapper[2576]: I0721 13:21:28.712281 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/22d0d372-952e-41b4-a3fc-23b3f3abe858-model-cache\") pod \"auth-test-model-cfed01-kserve-76fcd9c9dc-hj85s\" (UID: \"22d0d372-952e-41b4-a3fc-23b3f3abe858\") " pod="ai-tenant-e2e-shared-b-cfed01/auth-test-model-cfed01-kserve-76fcd9c9dc-hj85s" Jul 21 13:21:28.712419 ip-10-0-130-157 kubenswrapper[2576]: I0721 13:21:28.712374 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/22d0d372-952e-41b4-a3fc-23b3f3abe858-tmp-dir\") pod \"auth-test-model-cfed01-kserve-76fcd9c9dc-hj85s\" (UID: \"22d0d372-952e-41b4-a3fc-23b3f3abe858\") " pod="ai-tenant-e2e-shared-b-cfed01/auth-test-model-cfed01-kserve-76fcd9c9dc-hj85s" Jul 21 13:21:28.714290 ip-10-0-130-157 kubenswrapper[2576]: I0721 13:21:28.714267 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/22d0d372-952e-41b4-a3fc-23b3f3abe858-dshm\") pod \"auth-test-model-cfed01-kserve-76fcd9c9dc-hj85s\" (UID: \"22d0d372-952e-41b4-a3fc-23b3f3abe858\") " pod="ai-tenant-e2e-shared-b-cfed01/auth-test-model-cfed01-kserve-76fcd9c9dc-hj85s" Jul 21 13:21:28.714469 ip-10-0-130-157 kubenswrapper[2576]: I0721 13:21:28.714452 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/22d0d372-952e-41b4-a3fc-23b3f3abe858-tls-certs\") pod \"auth-test-model-cfed01-kserve-76fcd9c9dc-hj85s\" (UID: \"22d0d372-952e-41b4-a3fc-23b3f3abe858\") " pod="ai-tenant-e2e-shared-b-cfed01/auth-test-model-cfed01-kserve-76fcd9c9dc-hj85s" Jul 21 13:21:28.720897 ip-10-0-130-157 kubenswrapper[2576]: I0721 13:21:28.720875 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qg47c\" (UniqueName: \"kubernetes.io/projected/22d0d372-952e-41b4-a3fc-23b3f3abe858-kube-api-access-qg47c\") pod \"auth-test-model-cfed01-kserve-76fcd9c9dc-hj85s\" (UID: \"22d0d372-952e-41b4-a3fc-23b3f3abe858\") " pod="ai-tenant-e2e-shared-b-cfed01/auth-test-model-cfed01-kserve-76fcd9c9dc-hj85s" Jul 21 13:21:28.878789 ip-10-0-130-157 kubenswrapper[2576]: I0721 13:21:28.878696 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-cfed01/auth-test-model-cfed01-kserve-76fcd9c9dc-hj85s" Jul 21 13:21:29.003187 ip-10-0-130-157 kubenswrapper[2576]: I0721 13:21:29.003047 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-cfed01/auth-test-model-cfed01-kserve-76fcd9c9dc-hj85s"] Jul 21 13:21:29.005886 ip-10-0-130-157 kubenswrapper[2576]: W0721 13:21:29.005860 2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod22d0d372_952e_41b4_a3fc_23b3f3abe858.slice/crio-b8949960b0a0097eb9e43e7cc723bd4ff87a3f9dbc6ce11f37365e66c7bf3003 WatchSource:0}: Error finding container b8949960b0a0097eb9e43e7cc723bd4ff87a3f9dbc6ce11f37365e66c7bf3003: Status 404 returned error can't find the container with id b8949960b0a0097eb9e43e7cc723bd4ff87a3f9dbc6ce11f37365e66c7bf3003 Jul 21 13:21:29.522983 ip-10-0-130-157 kubenswrapper[2576]: I0721 13:21:29.522945 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-cfed01/auth-test-model-cfed01-kserve-76fcd9c9dc-hj85s" event={"ID":"22d0d372-952e-41b4-a3fc-23b3f3abe858","Type":"ContainerStarted","Data":"9966b9a6ea7a9312f2b2fcb28ef4cc32da24789542ac4f69116dc114af41945f"} Jul 21 13:21:29.522983 ip-10-0-130-157 kubenswrapper[2576]: I0721 13:21:29.522985 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-cfed01/auth-test-model-cfed01-kserve-76fcd9c9dc-hj85s" event={"ID":"22d0d372-952e-41b4-a3fc-23b3f3abe858","Type":"ContainerStarted","Data":"b8949960b0a0097eb9e43e7cc723bd4ff87a3f9dbc6ce11f37365e66c7bf3003"} Jul 21 13:21:29.542089 ip-10-0-130-157 kubenswrapper[2576]: I0721 13:21:29.542018 2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-b-cfed01/auth-test-model-cfed01-kserve-76fcd9c9dc-hj85s" podStartSLOduration=1.5420006590000002 podStartE2EDuration="1.542000659s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 13:21:28 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 13:21:29.541460737 +0000 UTC m=+2497.376724605" watchObservedRunningTime="2026-07-21 13:21:29.542000659 +0000 UTC m=+2497.377264526" Jul 21 13:21:29.557110 ip-10-0-130-157 kubenswrapper[2576]: I0721 13:21:29.557087 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-skc26_6713665c-3b71-4751-929e-16de48263766/konnectivity-agent/0.log" limit=1024 Jul 21 13:21:32.495177 ip-10-0-130-157 kubenswrapper[2576]: I0721 13:21:32.495144 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-8pmg8"] Jul 21 13:21:38.878833 ip-10-0-130-157 kubenswrapper[2576]: I0721 13:21:38.878795 2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-b-cfed01/auth-test-model-cfed01-kserve-76fcd9c9dc-hj85s" Jul 21 13:21:38.878833 ip-10-0-130-157 kubenswrapper[2576]: I0721 13:21:38.878842 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-b-cfed01/auth-test-model-cfed01-kserve-76fcd9c9dc-hj85s" Jul 21 13:21:38.891166 ip-10-0-130-157 kubenswrapper[2576]: I0721 13:21:38.891140 2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-b-cfed01/auth-test-model-cfed01-kserve-76fcd9c9dc-hj85s" Jul 21 13:21:39.576558 ip-10-0-130-157 kubenswrapper[2576]: I0721 13:21:39.576532 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-b-cfed01/auth-test-model-cfed01-kserve-76fcd9c9dc-hj85s" Jul 21 13:21:40.052726 ip-10-0-130-157 kubenswrapper[2576]: I0721 13:21:40.052694 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-skc26_6713665c-3b71-4751-929e-16de48263766/konnectivity-agent/0.log" limit=1024 Jul 21 13:21:49.989574 ip-10-0-130-157 kubenswrapper[2576]: I0721 13:21:49.989479 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-8pmg8"] Jul 21 13:22:06.037885 ip-10-0-130-157 kubenswrapper[2576]: I0721 13:22:06.037843 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-skc26_6713665c-3b71-4751-929e-16de48263766/konnectivity-agent/0.log" limit=1024 Jul 21 13:22:06.140717 ip-10-0-130-157 kubenswrapper[2576]: I0721 13:22:06.140688 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-skc26_6713665c-3b71-4751-929e-16de48263766/konnectivity-agent/0.log" limit=1024 Jul 21 13:22:08.390367 ip-10-0-130-157 kubenswrapper[2576]: I0721 13:22:08.390329 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-8pmg8"] Jul 21 13:22:22.584880 ip-10-0-130-157 kubenswrapper[2576]: I0721 13:22:22.584845 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-8pmg8"] Jul 21 13:22:37.588098 ip-10-0-130-157 kubenswrapper[2576]: I0721 13:22:37.588064 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-8pmg8"] Jul 21 13:22:49.380909 ip-10-0-130-157 kubenswrapper[2576]: I0721 13:22:49.380875 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-8pmg8"] Jul 21 13:23:04.392715 ip-10-0-130-157 kubenswrapper[2576]: I0721 13:23:04.392677 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-8pmg8"] Jul 21 13:23:23.297470 ip-10-0-130-157 kubenswrapper[2576]: I0721 13:23:23.297386 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-8pmg8"] Jul 21 13:23:28.156106 ip-10-0-130-157 kubenswrapper[2576]: I0721 13:23:28.156075 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-skc26_6713665c-3b71-4751-929e-16de48263766/konnectivity-agent/0.log" limit=1024 Jul 21 13:23:29.058079 ip-10-0-130-157 kubenswrapper[2576]: I0721 13:23:29.058047 2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-cfed01/auth-test-model-cfed01-kserve-76fcd9c9dc-hj85s"] Jul 21 13:23:29.407380 ip-10-0-130-157 kubenswrapper[2576]: I0721 13:23:29.407348 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-skc26_6713665c-3b71-4751-929e-16de48263766/konnectivity-agent/0.log" limit=1024 Jul 21 13:23:41.496539 ip-10-0-130-157 kubenswrapper[2576]: I0721 13:23:41.496489 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-8pmg8"] Jul 21 13:23:44.075673 ip-10-0-130-157 kubenswrapper[2576]: I0721 13:23:44.075645 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-skc26_6713665c-3b71-4751-929e-16de48263766/konnectivity-agent/0.log" limit=1024 Jul 21 13:23:44.077634 ip-10-0-130-157 kubenswrapper[2576]: I0721 13:23:44.077605 2576 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-b-cfed01/auth-test-model-cfed01-kserve-76fcd9c9dc-hj85s" podUID="22d0d372-952e-41b4-a3fc-23b3f3abe858" containerName="main" containerID="cri-o://9966b9a6ea7a9312f2b2fcb28ef4cc32da24789542ac4f69116dc114af41945f" gracePeriod=45 Jul 21 13:23:44.323637 ip-10-0-130-157 kubenswrapper[2576]: I0721 13:23:44.323612 2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-cfed01/auth-test-model-cfed01-kserve-76fcd9c9dc-hj85s" Jul 21 13:23:44.368552 ip-10-0-130-157 kubenswrapper[2576]: I0721 13:23:44.368449 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/22d0d372-952e-41b4-a3fc-23b3f3abe858-model-cache\") pod \"22d0d372-952e-41b4-a3fc-23b3f3abe858\" (UID: \"22d0d372-952e-41b4-a3fc-23b3f3abe858\") " Jul 21 13:23:44.368552 ip-10-0-130-157 kubenswrapper[2576]: I0721 13:23:44.368487 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/22d0d372-952e-41b4-a3fc-23b3f3abe858-tmp-dir\") pod \"22d0d372-952e-41b4-a3fc-23b3f3abe858\" (UID: \"22d0d372-952e-41b4-a3fc-23b3f3abe858\") " Jul 21 13:23:44.368552 ip-10-0-130-157 kubenswrapper[2576]: I0721 13:23:44.368525 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/22d0d372-952e-41b4-a3fc-23b3f3abe858-dshm\") pod \"22d0d372-952e-41b4-a3fc-23b3f3abe858\" (UID: \"22d0d372-952e-41b4-a3fc-23b3f3abe858\") " Jul 21 13:23:44.368811 ip-10-0-130-157 kubenswrapper[2576]: I0721 13:23:44.368605 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/22d0d372-952e-41b4-a3fc-23b3f3abe858-tls-certs\") pod \"22d0d372-952e-41b4-a3fc-23b3f3abe858\" (UID: \"22d0d372-952e-41b4-a3fc-23b3f3abe858\") " Jul 21 13:23:44.368811 ip-10-0-130-157 kubenswrapper[2576]: I0721 13:23:44.368665 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-qg47c\" (UniqueName: \"kubernetes.io/projected/22d0d372-952e-41b4-a3fc-23b3f3abe858-kube-api-access-qg47c\") pod \"22d0d372-952e-41b4-a3fc-23b3f3abe858\" (UID: \"22d0d372-952e-41b4-a3fc-23b3f3abe858\") " Jul 21 13:23:44.368811 ip-10-0-130-157 kubenswrapper[2576]: I0721 13:23:44.368686 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/22d0d372-952e-41b4-a3fc-23b3f3abe858-home\") pod \"22d0d372-952e-41b4-a3fc-23b3f3abe858\" (UID: \"22d0d372-952e-41b4-a3fc-23b3f3abe858\") " Jul 21 13:23:44.368811 ip-10-0-130-157 kubenswrapper[2576]: I0721 13:23:44.368758 2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/22d0d372-952e-41b4-a3fc-23b3f3abe858-model-cache" (OuterVolumeSpecName: "model-cache") pod "22d0d372-952e-41b4-a3fc-23b3f3abe858" (UID: "22d0d372-952e-41b4-a3fc-23b3f3abe858"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 21 13:23:44.368811 ip-10-0-130-157 kubenswrapper[2576]: I0721 13:23:44.368764 2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/22d0d372-952e-41b4-a3fc-23b3f3abe858-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "22d0d372-952e-41b4-a3fc-23b3f3abe858" (UID: "22d0d372-952e-41b4-a3fc-23b3f3abe858"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 21 13:23:44.369045 ip-10-0-130-157 kubenswrapper[2576]: I0721 13:23:44.368948 2576 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/22d0d372-952e-41b4-a3fc-23b3f3abe858-model-cache\") on node \"ip-10-0-130-157.ec2.internal\" DevicePath \"\"" Jul 21 13:23:44.369045 ip-10-0-130-157 kubenswrapper[2576]: I0721 13:23:44.368966 2576 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/22d0d372-952e-41b4-a3fc-23b3f3abe858-tmp-dir\") on node \"ip-10-0-130-157.ec2.internal\" DevicePath \"\"" Jul 21 13:23:44.369045 ip-10-0-130-157 kubenswrapper[2576]: I0721 13:23:44.368997 2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/22d0d372-952e-41b4-a3fc-23b3f3abe858-home" (OuterVolumeSpecName: "home") pod "22d0d372-952e-41b4-a3fc-23b3f3abe858" (UID: "22d0d372-952e-41b4-a3fc-23b3f3abe858"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 21 13:23:44.370873 ip-10-0-130-157 kubenswrapper[2576]: I0721 13:23:44.370850 2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/22d0d372-952e-41b4-a3fc-23b3f3abe858-dshm" (OuterVolumeSpecName: "dshm") pod "22d0d372-952e-41b4-a3fc-23b3f3abe858" (UID: "22d0d372-952e-41b4-a3fc-23b3f3abe858"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 21 13:23:44.370983 ip-10-0-130-157 kubenswrapper[2576]: I0721 13:23:44.370906 2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/22d0d372-952e-41b4-a3fc-23b3f3abe858-kube-api-access-qg47c" (OuterVolumeSpecName: "kube-api-access-qg47c") pod "22d0d372-952e-41b4-a3fc-23b3f3abe858" (UID: "22d0d372-952e-41b4-a3fc-23b3f3abe858"). InnerVolumeSpecName "kube-api-access-qg47c". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 21 13:23:44.370983 ip-10-0-130-157 kubenswrapper[2576]: I0721 13:23:44.370918 2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/22d0d372-952e-41b4-a3fc-23b3f3abe858-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "22d0d372-952e-41b4-a3fc-23b3f3abe858" (UID: "22d0d372-952e-41b4-a3fc-23b3f3abe858"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 21 13:23:44.469904 ip-10-0-130-157 kubenswrapper[2576]: I0721 13:23:44.469873 2576 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/22d0d372-952e-41b4-a3fc-23b3f3abe858-tls-certs\") on node \"ip-10-0-130-157.ec2.internal\" DevicePath \"\"" Jul 21 13:23:44.469904 ip-10-0-130-157 kubenswrapper[2576]: I0721 13:23:44.469904 2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-qg47c\" (UniqueName: \"kubernetes.io/projected/22d0d372-952e-41b4-a3fc-23b3f3abe858-kube-api-access-qg47c\") on node \"ip-10-0-130-157.ec2.internal\" DevicePath \"\"" Jul 21 13:23:44.470090 ip-10-0-130-157 kubenswrapper[2576]: I0721 13:23:44.469919 2576 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/22d0d372-952e-41b4-a3fc-23b3f3abe858-home\") on node \"ip-10-0-130-157.ec2.internal\" DevicePath \"\"" Jul 21 13:23:44.470090 ip-10-0-130-157 kubenswrapper[2576]: I0721 13:23:44.469932 2576 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/22d0d372-952e-41b4-a3fc-23b3f3abe858-dshm\") on node \"ip-10-0-130-157.ec2.internal\" DevicePath \"\"" Jul 21 13:23:45.122611 ip-10-0-130-157 kubenswrapper[2576]: I0721 13:23:45.122575 2576 generic.go:356] "Generic (PLEG): container finished" podID="22d0d372-952e-41b4-a3fc-23b3f3abe858" containerID="9966b9a6ea7a9312f2b2fcb28ef4cc32da24789542ac4f69116dc114af41945f" exitCode=0 Jul 21 13:23:45.123020 ip-10-0-130-157 kubenswrapper[2576]: I0721 13:23:45.122627 2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-cfed01/auth-test-model-cfed01-kserve-76fcd9c9dc-hj85s" Jul 21 13:23:45.123020 ip-10-0-130-157 kubenswrapper[2576]: I0721 13:23:45.122654 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-cfed01/auth-test-model-cfed01-kserve-76fcd9c9dc-hj85s" event={"ID":"22d0d372-952e-41b4-a3fc-23b3f3abe858","Type":"ContainerDied","Data":"9966b9a6ea7a9312f2b2fcb28ef4cc32da24789542ac4f69116dc114af41945f"} Jul 21 13:23:45.123020 ip-10-0-130-157 kubenswrapper[2576]: I0721 13:23:45.122696 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-cfed01/auth-test-model-cfed01-kserve-76fcd9c9dc-hj85s" event={"ID":"22d0d372-952e-41b4-a3fc-23b3f3abe858","Type":"ContainerDied","Data":"b8949960b0a0097eb9e43e7cc723bd4ff87a3f9dbc6ce11f37365e66c7bf3003"} Jul 21 13:23:45.123020 ip-10-0-130-157 kubenswrapper[2576]: I0721 13:23:45.122715 2576 scope.go:117] "RemoveContainer" containerID="9966b9a6ea7a9312f2b2fcb28ef4cc32da24789542ac4f69116dc114af41945f" Jul 21 13:23:45.131122 ip-10-0-130-157 kubenswrapper[2576]: I0721 13:23:45.131107 2576 scope.go:117] "RemoveContainer" containerID="9966b9a6ea7a9312f2b2fcb28ef4cc32da24789542ac4f69116dc114af41945f" Jul 21 13:23:45.131383 ip-10-0-130-157 kubenswrapper[2576]: E0721 13:23:45.131366 2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9966b9a6ea7a9312f2b2fcb28ef4cc32da24789542ac4f69116dc114af41945f\": container with ID starting with 9966b9a6ea7a9312f2b2fcb28ef4cc32da24789542ac4f69116dc114af41945f not found: ID does not exist" containerID="9966b9a6ea7a9312f2b2fcb28ef4cc32da24789542ac4f69116dc114af41945f" Jul 21 13:23:45.131435 ip-10-0-130-157 kubenswrapper[2576]: I0721 13:23:45.131391 2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9966b9a6ea7a9312f2b2fcb28ef4cc32da24789542ac4f69116dc114af41945f"} err="failed to get container status \"9966b9a6ea7a9312f2b2fcb28ef4cc32da24789542ac4f69116dc114af41945f\": rpc error: code = NotFound desc = could not find container \"9966b9a6ea7a9312f2b2fcb28ef4cc32da24789542ac4f69116dc114af41945f\": container with ID starting with 9966b9a6ea7a9312f2b2fcb28ef4cc32da24789542ac4f69116dc114af41945f not found: ID does not exist" Jul 21 13:23:45.143264 ip-10-0-130-157 kubenswrapper[2576]: I0721 13:23:45.143245 2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-cfed01/auth-test-model-cfed01-kserve-76fcd9c9dc-hj85s"] Jul 21 13:23:45.150570 ip-10-0-130-157 kubenswrapper[2576]: I0721 13:23:45.150509 2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-b-cfed01/auth-test-model-cfed01-kserve-76fcd9c9dc-hj85s"] Jul 21 13:23:46.316389 ip-10-0-130-157 kubenswrapper[2576]: I0721 13:23:46.316359 2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="22d0d372-952e-41b4-a3fc-23b3f3abe858" path="/var/lib/kubelet/pods/22d0d372-952e-41b4-a3fc-23b3f3abe858/volumes" Jul 21 13:23:53.587347 ip-10-0-130-157 kubenswrapper[2576]: I0721 13:23:53.587313 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-8pmg8"] Jul 21 13:24:19.142878 ip-10-0-130-157 kubenswrapper[2576]: I0721 13:24:19.142849 2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-b-cfed01/sub-test-model-cfed01-kserve-659bd84577-lccf8"] Jul 21 13:24:19.143322 ip-10-0-130-157 kubenswrapper[2576]: I0721 13:24:19.143131 2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="22d0d372-952e-41b4-a3fc-23b3f3abe858" containerName="main" Jul 21 13:24:19.143322 ip-10-0-130-157 kubenswrapper[2576]: I0721 13:24:19.143143 2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="22d0d372-952e-41b4-a3fc-23b3f3abe858" containerName="main" Jul 21 13:24:19.143322 ip-10-0-130-157 kubenswrapper[2576]: I0721 13:24:19.143200 2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="22d0d372-952e-41b4-a3fc-23b3f3abe858" containerName="main" Jul 21 13:24:19.145969 ip-10-0-130-157 kubenswrapper[2576]: I0721 13:24:19.145952 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-cfed01/sub-test-model-cfed01-kserve-659bd84577-lccf8" Jul 21 13:24:19.148068 ip-10-0-130-157 kubenswrapper[2576]: I0721 13:24:19.148048 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-cfed01\"/\"openshift-service-ca.crt\"" Jul 21 13:24:19.148199 ip-10-0-130-157 kubenswrapper[2576]: I0721 13:24:19.148119 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-cfed01\"/\"kube-root-ca.crt\"" Jul 21 13:24:19.148770 ip-10-0-130-157 kubenswrapper[2576]: I0721 13:24:19.148754 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-cfed01\"/\"default-dockercfg-4qzr4\"" Jul 21 13:24:19.148883 ip-10-0-130-157 kubenswrapper[2576]: I0721 13:24:19.148860 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-cfed01\"/\"sub-test-model-cfed01-kserve-self-signed-certs\"" Jul 21 13:24:19.156613 ip-10-0-130-157 kubenswrapper[2576]: I0721 13:24:19.156594 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-cfed01/sub-test-model-cfed01-kserve-659bd84577-lccf8"] Jul 21 13:24:19.167478 ip-10-0-130-157 kubenswrapper[2576]: I0721 13:24:19.167453 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-skc26_6713665c-3b71-4751-929e-16de48263766/konnectivity-agent/0.log" limit=1024 Jul 21 13:24:19.225174 ip-10-0-130-157 kubenswrapper[2576]: I0721 13:24:19.225151 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/65942337-d38d-4be6-a5d1-427c2cdac308-dshm\") pod \"sub-test-model-cfed01-kserve-659bd84577-lccf8\" (UID: \"65942337-d38d-4be6-a5d1-427c2cdac308\") " pod="ai-tenant-e2e-shared-b-cfed01/sub-test-model-cfed01-kserve-659bd84577-lccf8" Jul 21 13:24:19.225267 ip-10-0-130-157 kubenswrapper[2576]: I0721 13:24:19.225181 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/65942337-d38d-4be6-a5d1-427c2cdac308-home\") pod \"sub-test-model-cfed01-kserve-659bd84577-lccf8\" (UID: \"65942337-d38d-4be6-a5d1-427c2cdac308\") " pod="ai-tenant-e2e-shared-b-cfed01/sub-test-model-cfed01-kserve-659bd84577-lccf8" Jul 21 13:24:19.225267 ip-10-0-130-157 kubenswrapper[2576]: I0721 13:24:19.225200 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/65942337-d38d-4be6-a5d1-427c2cdac308-model-cache\") pod \"sub-test-model-cfed01-kserve-659bd84577-lccf8\" (UID: \"65942337-d38d-4be6-a5d1-427c2cdac308\") " pod="ai-tenant-e2e-shared-b-cfed01/sub-test-model-cfed01-kserve-659bd84577-lccf8" Jul 21 13:24:19.225267 ip-10-0-130-157 kubenswrapper[2576]: I0721 13:24:19.225256 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5kgb2\" (UniqueName: \"kubernetes.io/projected/65942337-d38d-4be6-a5d1-427c2cdac308-kube-api-access-5kgb2\") pod \"sub-test-model-cfed01-kserve-659bd84577-lccf8\" (UID: \"65942337-d38d-4be6-a5d1-427c2cdac308\") " pod="ai-tenant-e2e-shared-b-cfed01/sub-test-model-cfed01-kserve-659bd84577-lccf8" Jul 21 13:24:19.225370 ip-10-0-130-157 kubenswrapper[2576]: I0721 13:24:19.225328 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/65942337-d38d-4be6-a5d1-427c2cdac308-tls-certs\") pod \"sub-test-model-cfed01-kserve-659bd84577-lccf8\" (UID: \"65942337-d38d-4be6-a5d1-427c2cdac308\") " pod="ai-tenant-e2e-shared-b-cfed01/sub-test-model-cfed01-kserve-659bd84577-lccf8" Jul 21 13:24:19.225370 ip-10-0-130-157 kubenswrapper[2576]: I0721 13:24:19.225353 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/65942337-d38d-4be6-a5d1-427c2cdac308-tmp-dir\") pod \"sub-test-model-cfed01-kserve-659bd84577-lccf8\" (UID: \"65942337-d38d-4be6-a5d1-427c2cdac308\") " pod="ai-tenant-e2e-shared-b-cfed01/sub-test-model-cfed01-kserve-659bd84577-lccf8" Jul 21 13:24:19.325909 ip-10-0-130-157 kubenswrapper[2576]: I0721 13:24:19.325889 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/65942337-d38d-4be6-a5d1-427c2cdac308-tls-certs\") pod \"sub-test-model-cfed01-kserve-659bd84577-lccf8\" (UID: \"65942337-d38d-4be6-a5d1-427c2cdac308\") " pod="ai-tenant-e2e-shared-b-cfed01/sub-test-model-cfed01-kserve-659bd84577-lccf8" Jul 21 13:24:19.326005 ip-10-0-130-157 kubenswrapper[2576]: I0721 13:24:19.325920 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/65942337-d38d-4be6-a5d1-427c2cdac308-tmp-dir\") pod \"sub-test-model-cfed01-kserve-659bd84577-lccf8\" (UID: \"65942337-d38d-4be6-a5d1-427c2cdac308\") " pod="ai-tenant-e2e-shared-b-cfed01/sub-test-model-cfed01-kserve-659bd84577-lccf8" Jul 21 13:24:19.326065 ip-10-0-130-157 kubenswrapper[2576]: I0721 13:24:19.326048 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/65942337-d38d-4be6-a5d1-427c2cdac308-dshm\") pod \"sub-test-model-cfed01-kserve-659bd84577-lccf8\" (UID: \"65942337-d38d-4be6-a5d1-427c2cdac308\") " pod="ai-tenant-e2e-shared-b-cfed01/sub-test-model-cfed01-kserve-659bd84577-lccf8" Jul 21 13:24:19.326103 ip-10-0-130-157 kubenswrapper[2576]: I0721 13:24:19.326085 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/65942337-d38d-4be6-a5d1-427c2cdac308-home\") pod \"sub-test-model-cfed01-kserve-659bd84577-lccf8\" (UID: \"65942337-d38d-4be6-a5d1-427c2cdac308\") " pod="ai-tenant-e2e-shared-b-cfed01/sub-test-model-cfed01-kserve-659bd84577-lccf8" Jul 21 13:24:19.326150 ip-10-0-130-157 kubenswrapper[2576]: I0721 13:24:19.326112 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/65942337-d38d-4be6-a5d1-427c2cdac308-model-cache\") pod \"sub-test-model-cfed01-kserve-659bd84577-lccf8\" (UID: \"65942337-d38d-4be6-a5d1-427c2cdac308\") " pod="ai-tenant-e2e-shared-b-cfed01/sub-test-model-cfed01-kserve-659bd84577-lccf8" Jul 21 13:24:19.326188 ip-10-0-130-157 kubenswrapper[2576]: I0721 13:24:19.326148 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5kgb2\" (UniqueName: \"kubernetes.io/projected/65942337-d38d-4be6-a5d1-427c2cdac308-kube-api-access-5kgb2\") pod \"sub-test-model-cfed01-kserve-659bd84577-lccf8\" (UID: \"65942337-d38d-4be6-a5d1-427c2cdac308\") " pod="ai-tenant-e2e-shared-b-cfed01/sub-test-model-cfed01-kserve-659bd84577-lccf8" Jul 21 13:24:19.326329 ip-10-0-130-157 kubenswrapper[2576]: I0721 13:24:19.326305 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/65942337-d38d-4be6-a5d1-427c2cdac308-tmp-dir\") pod \"sub-test-model-cfed01-kserve-659bd84577-lccf8\" (UID: \"65942337-d38d-4be6-a5d1-427c2cdac308\") " pod="ai-tenant-e2e-shared-b-cfed01/sub-test-model-cfed01-kserve-659bd84577-lccf8" Jul 21 13:24:19.326402 ip-10-0-130-157 kubenswrapper[2576]: I0721 13:24:19.326380 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/65942337-d38d-4be6-a5d1-427c2cdac308-home\") pod \"sub-test-model-cfed01-kserve-659bd84577-lccf8\" (UID: \"65942337-d38d-4be6-a5d1-427c2cdac308\") " pod="ai-tenant-e2e-shared-b-cfed01/sub-test-model-cfed01-kserve-659bd84577-lccf8" Jul 21 13:24:19.326539 ip-10-0-130-157 kubenswrapper[2576]: I0721 13:24:19.326497 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/65942337-d38d-4be6-a5d1-427c2cdac308-model-cache\") pod \"sub-test-model-cfed01-kserve-659bd84577-lccf8\" (UID: \"65942337-d38d-4be6-a5d1-427c2cdac308\") " pod="ai-tenant-e2e-shared-b-cfed01/sub-test-model-cfed01-kserve-659bd84577-lccf8" Jul 21 13:24:19.328216 ip-10-0-130-157 kubenswrapper[2576]: I0721 13:24:19.328193 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/65942337-d38d-4be6-a5d1-427c2cdac308-dshm\") pod \"sub-test-model-cfed01-kserve-659bd84577-lccf8\" (UID: \"65942337-d38d-4be6-a5d1-427c2cdac308\") " pod="ai-tenant-e2e-shared-b-cfed01/sub-test-model-cfed01-kserve-659bd84577-lccf8" Jul 21 13:24:19.328332 ip-10-0-130-157 kubenswrapper[2576]: I0721 13:24:19.328314 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/65942337-d38d-4be6-a5d1-427c2cdac308-tls-certs\") pod \"sub-test-model-cfed01-kserve-659bd84577-lccf8\" (UID: \"65942337-d38d-4be6-a5d1-427c2cdac308\") " pod="ai-tenant-e2e-shared-b-cfed01/sub-test-model-cfed01-kserve-659bd84577-lccf8" Jul 21 13:24:19.336182 ip-10-0-130-157 kubenswrapper[2576]: I0721 13:24:19.336163 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5kgb2\" (UniqueName: \"kubernetes.io/projected/65942337-d38d-4be6-a5d1-427c2cdac308-kube-api-access-5kgb2\") pod \"sub-test-model-cfed01-kserve-659bd84577-lccf8\" (UID: \"65942337-d38d-4be6-a5d1-427c2cdac308\") " pod="ai-tenant-e2e-shared-b-cfed01/sub-test-model-cfed01-kserve-659bd84577-lccf8" Jul 21 13:24:19.455430 ip-10-0-130-157 kubenswrapper[2576]: I0721 13:24:19.455376 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-cfed01/sub-test-model-cfed01-kserve-659bd84577-lccf8" Jul 21 13:24:19.587637 ip-10-0-130-157 kubenswrapper[2576]: I0721 13:24:19.587580 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-cfed01/sub-test-model-cfed01-kserve-659bd84577-lccf8"] Jul 21 13:24:19.592009 ip-10-0-130-157 kubenswrapper[2576]: W0721 13:24:19.591981 2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod65942337_d38d_4be6_a5d1_427c2cdac308.slice/crio-3701a5e07ae0e8c9047336efed935764a8f319f6722ec7cf93c8932433d550ec WatchSource:0}: Error finding container 3701a5e07ae0e8c9047336efed935764a8f319f6722ec7cf93c8932433d550ec: Status 404 returned error can't find the container with id 3701a5e07ae0e8c9047336efed935764a8f319f6722ec7cf93c8932433d550ec Jul 21 13:24:20.283659 ip-10-0-130-157 kubenswrapper[2576]: I0721 13:24:20.283616 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-cfed01/sub-test-model-cfed01-kserve-659bd84577-lccf8" event={"ID":"65942337-d38d-4be6-a5d1-427c2cdac308","Type":"ContainerStarted","Data":"2e76f91144a32a8af0661733611f751d63d6f90f374f67fb79f9d27565f8752a"} Jul 21 13:24:20.284119 ip-10-0-130-157 kubenswrapper[2576]: I0721 13:24:20.283666 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-cfed01/sub-test-model-cfed01-kserve-659bd84577-lccf8" event={"ID":"65942337-d38d-4be6-a5d1-427c2cdac308","Type":"ContainerStarted","Data":"3701a5e07ae0e8c9047336efed935764a8f319f6722ec7cf93c8932433d550ec"} Jul 21 13:24:20.304335 ip-10-0-130-157 kubenswrapper[2576]: I0721 13:24:20.304287 2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-b-cfed01/sub-test-model-cfed01-kserve-659bd84577-lccf8" podStartSLOduration=1.304259366 podStartE2EDuration="1.304259366s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 13:24:19 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 13:24:20.303569331 +0000 UTC m=+2668.138833200" watchObservedRunningTime="2026-07-21 13:24:20.304259366 +0000 UTC m=+2668.139523210" Jul 21 13:24:23.090189 ip-10-0-130-157 kubenswrapper[2576]: I0721 13:24:23.090153 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-8pmg8"] Jul 21 13:24:29.455873 ip-10-0-130-157 kubenswrapper[2576]: I0721 13:24:29.455839 2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-b-cfed01/sub-test-model-cfed01-kserve-659bd84577-lccf8" Jul 21 13:24:29.456357 ip-10-0-130-157 kubenswrapper[2576]: I0721 13:24:29.455887 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-b-cfed01/sub-test-model-cfed01-kserve-659bd84577-lccf8" Jul 21 13:24:29.468163 ip-10-0-130-157 kubenswrapper[2576]: I0721 13:24:29.468136 2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-b-cfed01/sub-test-model-cfed01-kserve-659bd84577-lccf8" Jul 21 13:24:30.340083 ip-10-0-130-157 kubenswrapper[2576]: I0721 13:24:30.340051 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-b-cfed01/sub-test-model-cfed01-kserve-659bd84577-lccf8" Jul 21 13:24:37.200321 ip-10-0-130-157 kubenswrapper[2576]: I0721 13:24:37.200271 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-8pmg8"] Jul 21 13:24:51.607226 ip-10-0-130-157 kubenswrapper[2576]: I0721 13:24:51.607147 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-skc26_6713665c-3b71-4751-929e-16de48263766/konnectivity-agent/0.log" limit=1024 Jul 21 13:24:52.441604 ip-10-0-130-157 kubenswrapper[2576]: I0721 13:24:52.441576 2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-cfed01/sub-test-model-cfed01-kserve-659bd84577-lccf8"] Jul 21 13:24:52.463150 ip-10-0-130-157 kubenswrapper[2576]: I0721 13:24:52.463124 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-skc26_6713665c-3b71-4751-929e-16de48263766/konnectivity-agent/0.log" limit=1024 Jul 21 13:24:53.839320 ip-10-0-130-157 kubenswrapper[2576]: I0721 13:24:53.839283 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-6nf7d_a2b48106-9844-4431-9b6f-d4e1baacb15b/ovn-acl-logging/0.log" Jul 21 13:24:53.842796 ip-10-0-130-157 kubenswrapper[2576]: I0721 13:24:53.842775 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-6nf7d_a2b48106-9844-4431-9b6f-d4e1baacb15b/ovn-acl-logging/0.log" Jul 21 13:24:53.901469 ip-10-0-130-157 kubenswrapper[2576]: I0721 13:24:53.901446 2576 scope.go:117] "RemoveContainer" containerID="a628c296182018d88f49e9e40665ce177561e609817e28b06e5f1402700c561b" Jul 21 13:24:53.908959 ip-10-0-130-157 kubenswrapper[2576]: I0721 13:24:53.908941 2576 scope.go:117] "RemoveContainer" containerID="36c94ac152f1858ebcce4b4618c6c8c5abb3bb923624f50d701154b66cae519f" Jul 21 13:24:57.387608 ip-10-0-130-157 kubenswrapper[2576]: I0721 13:24:57.387577 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-8pmg8"] Jul 21 13:25:05.665484 ip-10-0-130-157 kubenswrapper[2576]: I0721 13:25:05.665448 2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-a-b3a013/rate-test-model-b3a013-kserve-85d8bc796c-7zb6p"] Jul 21 13:25:05.668936 ip-10-0-130-157 kubenswrapper[2576]: I0721 13:25:05.668914 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-b3a013/rate-test-model-b3a013-kserve-85d8bc796c-7zb6p" Jul 21 13:25:05.670902 ip-10-0-130-157 kubenswrapper[2576]: I0721 13:25:05.670876 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-b3a013\"/\"kube-root-ca.crt\"" Jul 21 13:25:05.671398 ip-10-0-130-157 kubenswrapper[2576]: I0721 13:25:05.671379 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-b3a013\"/\"rate-test-model-b3a013-kserve-self-signed-certs\"" Jul 21 13:25:05.671544 ip-10-0-130-157 kubenswrapper[2576]: I0721 13:25:05.671407 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-b3a013\"/\"default-dockercfg-bt7lb\"" Jul 21 13:25:05.671544 ip-10-0-130-157 kubenswrapper[2576]: I0721 13:25:05.671410 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-b3a013\"/\"openshift-service-ca.crt\"" Jul 21 13:25:05.677951 ip-10-0-130-157 kubenswrapper[2576]: I0721 13:25:05.677929 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-b3a013/rate-test-model-b3a013-kserve-85d8bc796c-7zb6p"] Jul 21 13:25:05.712900 ip-10-0-130-157 kubenswrapper[2576]: I0721 13:25:05.712875 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-skc26_6713665c-3b71-4751-929e-16de48263766/konnectivity-agent/0.log" limit=1024 Jul 21 13:25:05.781112 ip-10-0-130-157 kubenswrapper[2576]: I0721 13:25:05.781085 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-n6lp7\" (UniqueName: \"kubernetes.io/projected/b5f85806-c45d-4317-831f-614dde87a6e9-kube-api-access-n6lp7\") pod \"rate-test-model-b3a013-kserve-85d8bc796c-7zb6p\" (UID: \"b5f85806-c45d-4317-831f-614dde87a6e9\") " pod="ai-tenant-e2e-shared-a-b3a013/rate-test-model-b3a013-kserve-85d8bc796c-7zb6p" Jul 21 13:25:05.781228 ip-10-0-130-157 kubenswrapper[2576]: I0721 13:25:05.781119 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b5f85806-c45d-4317-831f-614dde87a6e9-tmp-dir\") pod \"rate-test-model-b3a013-kserve-85d8bc796c-7zb6p\" (UID: \"b5f85806-c45d-4317-831f-614dde87a6e9\") " pod="ai-tenant-e2e-shared-a-b3a013/rate-test-model-b3a013-kserve-85d8bc796c-7zb6p" Jul 21 13:25:05.781228 ip-10-0-130-157 kubenswrapper[2576]: I0721 13:25:05.781143 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b5f85806-c45d-4317-831f-614dde87a6e9-dshm\") pod \"rate-test-model-b3a013-kserve-85d8bc796c-7zb6p\" (UID: \"b5f85806-c45d-4317-831f-614dde87a6e9\") " pod="ai-tenant-e2e-shared-a-b3a013/rate-test-model-b3a013-kserve-85d8bc796c-7zb6p" Jul 21 13:25:05.781319 ip-10-0-130-157 kubenswrapper[2576]: I0721 13:25:05.781234 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b5f85806-c45d-4317-831f-614dde87a6e9-tls-certs\") pod \"rate-test-model-b3a013-kserve-85d8bc796c-7zb6p\" (UID: \"b5f85806-c45d-4317-831f-614dde87a6e9\") " pod="ai-tenant-e2e-shared-a-b3a013/rate-test-model-b3a013-kserve-85d8bc796c-7zb6p" Jul 21 13:25:05.781319 ip-10-0-130-157 kubenswrapper[2576]: I0721 13:25:05.781257 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b5f85806-c45d-4317-831f-614dde87a6e9-home\") pod \"rate-test-model-b3a013-kserve-85d8bc796c-7zb6p\" (UID: \"b5f85806-c45d-4317-831f-614dde87a6e9\") " pod="ai-tenant-e2e-shared-a-b3a013/rate-test-model-b3a013-kserve-85d8bc796c-7zb6p" Jul 21 13:25:05.781319 ip-10-0-130-157 kubenswrapper[2576]: I0721 13:25:05.781273 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b5f85806-c45d-4317-831f-614dde87a6e9-model-cache\") pod \"rate-test-model-b3a013-kserve-85d8bc796c-7zb6p\" (UID: \"b5f85806-c45d-4317-831f-614dde87a6e9\") " pod="ai-tenant-e2e-shared-a-b3a013/rate-test-model-b3a013-kserve-85d8bc796c-7zb6p" Jul 21 13:25:05.882435 ip-10-0-130-157 kubenswrapper[2576]: I0721 13:25:05.882408 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-n6lp7\" (UniqueName: \"kubernetes.io/projected/b5f85806-c45d-4317-831f-614dde87a6e9-kube-api-access-n6lp7\") pod \"rate-test-model-b3a013-kserve-85d8bc796c-7zb6p\" (UID: \"b5f85806-c45d-4317-831f-614dde87a6e9\") " pod="ai-tenant-e2e-shared-a-b3a013/rate-test-model-b3a013-kserve-85d8bc796c-7zb6p" Jul 21 13:25:05.882601 ip-10-0-130-157 kubenswrapper[2576]: I0721 13:25:05.882439 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b5f85806-c45d-4317-831f-614dde87a6e9-tmp-dir\") pod \"rate-test-model-b3a013-kserve-85d8bc796c-7zb6p\" (UID: \"b5f85806-c45d-4317-831f-614dde87a6e9\") " pod="ai-tenant-e2e-shared-a-b3a013/rate-test-model-b3a013-kserve-85d8bc796c-7zb6p" Jul 21 13:25:05.882601 ip-10-0-130-157 kubenswrapper[2576]: I0721 13:25:05.882458 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b5f85806-c45d-4317-831f-614dde87a6e9-dshm\") pod \"rate-test-model-b3a013-kserve-85d8bc796c-7zb6p\" (UID: \"b5f85806-c45d-4317-831f-614dde87a6e9\") " pod="ai-tenant-e2e-shared-a-b3a013/rate-test-model-b3a013-kserve-85d8bc796c-7zb6p" Jul 21 13:25:05.882601 ip-10-0-130-157 kubenswrapper[2576]: I0721 13:25:05.882487 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b5f85806-c45d-4317-831f-614dde87a6e9-tls-certs\") pod \"rate-test-model-b3a013-kserve-85d8bc796c-7zb6p\" (UID: \"b5f85806-c45d-4317-831f-614dde87a6e9\") " pod="ai-tenant-e2e-shared-a-b3a013/rate-test-model-b3a013-kserve-85d8bc796c-7zb6p" Jul 21 13:25:05.882601 ip-10-0-130-157 kubenswrapper[2576]: I0721 13:25:05.882531 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b5f85806-c45d-4317-831f-614dde87a6e9-home\") pod \"rate-test-model-b3a013-kserve-85d8bc796c-7zb6p\" (UID: \"b5f85806-c45d-4317-831f-614dde87a6e9\") " pod="ai-tenant-e2e-shared-a-b3a013/rate-test-model-b3a013-kserve-85d8bc796c-7zb6p" Jul 21 13:25:05.882601 ip-10-0-130-157 kubenswrapper[2576]: I0721 13:25:05.882557 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b5f85806-c45d-4317-831f-614dde87a6e9-model-cache\") pod \"rate-test-model-b3a013-kserve-85d8bc796c-7zb6p\" (UID: \"b5f85806-c45d-4317-831f-614dde87a6e9\") " pod="ai-tenant-e2e-shared-a-b3a013/rate-test-model-b3a013-kserve-85d8bc796c-7zb6p" Jul 21 13:25:05.882883 ip-10-0-130-157 kubenswrapper[2576]: I0721 13:25:05.882863 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b5f85806-c45d-4317-831f-614dde87a6e9-tmp-dir\") pod \"rate-test-model-b3a013-kserve-85d8bc796c-7zb6p\" (UID: \"b5f85806-c45d-4317-831f-614dde87a6e9\") " pod="ai-tenant-e2e-shared-a-b3a013/rate-test-model-b3a013-kserve-85d8bc796c-7zb6p" Jul 21 13:25:05.882944 ip-10-0-130-157 kubenswrapper[2576]: I0721 13:25:05.882927 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b5f85806-c45d-4317-831f-614dde87a6e9-home\") pod \"rate-test-model-b3a013-kserve-85d8bc796c-7zb6p\" (UID: \"b5f85806-c45d-4317-831f-614dde87a6e9\") " pod="ai-tenant-e2e-shared-a-b3a013/rate-test-model-b3a013-kserve-85d8bc796c-7zb6p" Jul 21 13:25:05.882995 ip-10-0-130-157 kubenswrapper[2576]: I0721 13:25:05.882984 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b5f85806-c45d-4317-831f-614dde87a6e9-model-cache\") pod \"rate-test-model-b3a013-kserve-85d8bc796c-7zb6p\" (UID: \"b5f85806-c45d-4317-831f-614dde87a6e9\") " pod="ai-tenant-e2e-shared-a-b3a013/rate-test-model-b3a013-kserve-85d8bc796c-7zb6p" Jul 21 13:25:05.884737 ip-10-0-130-157 kubenswrapper[2576]: I0721 13:25:05.884716 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b5f85806-c45d-4317-831f-614dde87a6e9-dshm\") pod \"rate-test-model-b3a013-kserve-85d8bc796c-7zb6p\" (UID: \"b5f85806-c45d-4317-831f-614dde87a6e9\") " pod="ai-tenant-e2e-shared-a-b3a013/rate-test-model-b3a013-kserve-85d8bc796c-7zb6p" Jul 21 13:25:05.885010 ip-10-0-130-157 kubenswrapper[2576]: I0721 13:25:05.884994 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b5f85806-c45d-4317-831f-614dde87a6e9-tls-certs\") pod \"rate-test-model-b3a013-kserve-85d8bc796c-7zb6p\" (UID: \"b5f85806-c45d-4317-831f-614dde87a6e9\") " pod="ai-tenant-e2e-shared-a-b3a013/rate-test-model-b3a013-kserve-85d8bc796c-7zb6p" Jul 21 13:25:05.891994 ip-10-0-130-157 kubenswrapper[2576]: I0721 13:25:05.891973 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-n6lp7\" (UniqueName: \"kubernetes.io/projected/b5f85806-c45d-4317-831f-614dde87a6e9-kube-api-access-n6lp7\") pod \"rate-test-model-b3a013-kserve-85d8bc796c-7zb6p\" (UID: \"b5f85806-c45d-4317-831f-614dde87a6e9\") " pod="ai-tenant-e2e-shared-a-b3a013/rate-test-model-b3a013-kserve-85d8bc796c-7zb6p" Jul 21 13:25:05.979312 ip-10-0-130-157 kubenswrapper[2576]: I0721 13:25:05.979234 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-b3a013/rate-test-model-b3a013-kserve-85d8bc796c-7zb6p" Jul 21 13:25:06.108763 ip-10-0-130-157 kubenswrapper[2576]: I0721 13:25:06.108680 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-b3a013/rate-test-model-b3a013-kserve-85d8bc796c-7zb6p"] Jul 21 13:25:06.111306 ip-10-0-130-157 kubenswrapper[2576]: W0721 13:25:06.111275 2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podb5f85806_c45d_4317_831f_614dde87a6e9.slice/crio-a889a40ae0f35db849e146a014ddba8045341d6e0bfc8ab8541864d06c6ccb4a WatchSource:0}: Error finding container a889a40ae0f35db849e146a014ddba8045341d6e0bfc8ab8541864d06c6ccb4a: Status 404 returned error can't find the container with id a889a40ae0f35db849e146a014ddba8045341d6e0bfc8ab8541864d06c6ccb4a Jul 21 13:25:06.488786 ip-10-0-130-157 kubenswrapper[2576]: I0721 13:25:06.488749 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-b3a013/rate-test-model-b3a013-kserve-85d8bc796c-7zb6p" event={"ID":"b5f85806-c45d-4317-831f-614dde87a6e9","Type":"ContainerStarted","Data":"0ec6e6da59b49b9b9e97c120ed30affcc45f0f714467877ba004b028ab1e5af9"} Jul 21 13:25:06.488960 ip-10-0-130-157 kubenswrapper[2576]: I0721 13:25:06.488797 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-b3a013/rate-test-model-b3a013-kserve-85d8bc796c-7zb6p" event={"ID":"b5f85806-c45d-4317-831f-614dde87a6e9","Type":"ContainerStarted","Data":"a889a40ae0f35db849e146a014ddba8045341d6e0bfc8ab8541864d06c6ccb4a"} Jul 21 13:25:06.513651 ip-10-0-130-157 kubenswrapper[2576]: I0721 13:25:06.513604 2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-a-b3a013/rate-test-model-b3a013-kserve-85d8bc796c-7zb6p" podStartSLOduration=1.5135899560000001 podStartE2EDuration="1.513589956s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 13:25:05 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 13:25:06.511774025 +0000 UTC m=+2714.347037890" watchObservedRunningTime="2026-07-21 13:25:06.513589956 +0000 UTC m=+2714.348853821" Jul 21 13:25:07.464207 ip-10-0-130-157 kubenswrapper[2576]: I0721 13:25:07.464165 2576 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-b-cfed01/sub-test-model-cfed01-kserve-659bd84577-lccf8" podUID="65942337-d38d-4be6-a5d1-427c2cdac308" containerName="main" containerID="cri-o://2e76f91144a32a8af0661733611f751d63d6f90f374f67fb79f9d27565f8752a" gracePeriod=45 Jul 21 13:25:07.718704 ip-10-0-130-157 kubenswrapper[2576]: I0721 13:25:07.718652 2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-cfed01/sub-test-model-cfed01-kserve-659bd84577-lccf8" Jul 21 13:25:07.799106 ip-10-0-130-157 kubenswrapper[2576]: I0721 13:25:07.799075 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/65942337-d38d-4be6-a5d1-427c2cdac308-tls-certs\") pod \"65942337-d38d-4be6-a5d1-427c2cdac308\" (UID: \"65942337-d38d-4be6-a5d1-427c2cdac308\") " Jul 21 13:25:07.799250 ip-10-0-130-157 kubenswrapper[2576]: I0721 13:25:07.799112 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/65942337-d38d-4be6-a5d1-427c2cdac308-dshm\") pod \"65942337-d38d-4be6-a5d1-427c2cdac308\" (UID: \"65942337-d38d-4be6-a5d1-427c2cdac308\") " Jul 21 13:25:07.799250 ip-10-0-130-157 kubenswrapper[2576]: I0721 13:25:07.799170 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/65942337-d38d-4be6-a5d1-427c2cdac308-model-cache\") pod \"65942337-d38d-4be6-a5d1-427c2cdac308\" (UID: \"65942337-d38d-4be6-a5d1-427c2cdac308\") " Jul 21 13:25:07.799250 ip-10-0-130-157 kubenswrapper[2576]: I0721 13:25:07.799222 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/65942337-d38d-4be6-a5d1-427c2cdac308-tmp-dir\") pod \"65942337-d38d-4be6-a5d1-427c2cdac308\" (UID: \"65942337-d38d-4be6-a5d1-427c2cdac308\") " Jul 21 13:25:07.799417 ip-10-0-130-157 kubenswrapper[2576]: I0721 13:25:07.799252 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/65942337-d38d-4be6-a5d1-427c2cdac308-home\") pod \"65942337-d38d-4be6-a5d1-427c2cdac308\" (UID: \"65942337-d38d-4be6-a5d1-427c2cdac308\") " Jul 21 13:25:07.799417 ip-10-0-130-157 kubenswrapper[2576]: I0721 13:25:07.799303 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-5kgb2\" (UniqueName: \"kubernetes.io/projected/65942337-d38d-4be6-a5d1-427c2cdac308-kube-api-access-5kgb2\") pod \"65942337-d38d-4be6-a5d1-427c2cdac308\" (UID: \"65942337-d38d-4be6-a5d1-427c2cdac308\") " Jul 21 13:25:07.799509 ip-10-0-130-157 kubenswrapper[2576]: I0721 13:25:07.799414 2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/65942337-d38d-4be6-a5d1-427c2cdac308-model-cache" (OuterVolumeSpecName: "model-cache") pod "65942337-d38d-4be6-a5d1-427c2cdac308" (UID: "65942337-d38d-4be6-a5d1-427c2cdac308"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 21 13:25:07.799578 ip-10-0-130-157 kubenswrapper[2576]: I0721 13:25:07.799555 2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/65942337-d38d-4be6-a5d1-427c2cdac308-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "65942337-d38d-4be6-a5d1-427c2cdac308" (UID: "65942337-d38d-4be6-a5d1-427c2cdac308"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 21 13:25:07.799628 ip-10-0-130-157 kubenswrapper[2576]: I0721 13:25:07.799565 2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/65942337-d38d-4be6-a5d1-427c2cdac308-home" (OuterVolumeSpecName: "home") pod "65942337-d38d-4be6-a5d1-427c2cdac308" (UID: "65942337-d38d-4be6-a5d1-427c2cdac308"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 21 13:25:07.799676 ip-10-0-130-157 kubenswrapper[2576]: I0721 13:25:07.799622 2576 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/65942337-d38d-4be6-a5d1-427c2cdac308-model-cache\") on node \"ip-10-0-130-157.ec2.internal\" DevicePath \"\"" Jul 21 13:25:07.801224 ip-10-0-130-157 kubenswrapper[2576]: I0721 13:25:07.801194 2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/65942337-d38d-4be6-a5d1-427c2cdac308-dshm" (OuterVolumeSpecName: "dshm") pod "65942337-d38d-4be6-a5d1-427c2cdac308" (UID: "65942337-d38d-4be6-a5d1-427c2cdac308"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 21 13:25:07.801339 ip-10-0-130-157 kubenswrapper[2576]: I0721 13:25:07.801280 2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/65942337-d38d-4be6-a5d1-427c2cdac308-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "65942337-d38d-4be6-a5d1-427c2cdac308" (UID: "65942337-d38d-4be6-a5d1-427c2cdac308"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 21 13:25:07.801339 ip-10-0-130-157 kubenswrapper[2576]: I0721 13:25:07.801284 2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/65942337-d38d-4be6-a5d1-427c2cdac308-kube-api-access-5kgb2" (OuterVolumeSpecName: "kube-api-access-5kgb2") pod "65942337-d38d-4be6-a5d1-427c2cdac308" (UID: "65942337-d38d-4be6-a5d1-427c2cdac308"). InnerVolumeSpecName "kube-api-access-5kgb2". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 21 13:25:07.900832 ip-10-0-130-157 kubenswrapper[2576]: I0721 13:25:07.900804 2576 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/65942337-d38d-4be6-a5d1-427c2cdac308-tmp-dir\") on node \"ip-10-0-130-157.ec2.internal\" DevicePath \"\"" Jul 21 13:25:07.900832 ip-10-0-130-157 kubenswrapper[2576]: I0721 13:25:07.900827 2576 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/65942337-d38d-4be6-a5d1-427c2cdac308-home\") on node \"ip-10-0-130-157.ec2.internal\" DevicePath \"\"" Jul 21 13:25:07.900980 ip-10-0-130-157 kubenswrapper[2576]: I0721 13:25:07.900838 2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-5kgb2\" (UniqueName: \"kubernetes.io/projected/65942337-d38d-4be6-a5d1-427c2cdac308-kube-api-access-5kgb2\") on node \"ip-10-0-130-157.ec2.internal\" DevicePath \"\"" Jul 21 13:25:07.900980 ip-10-0-130-157 kubenswrapper[2576]: I0721 13:25:07.900848 2576 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/65942337-d38d-4be6-a5d1-427c2cdac308-tls-certs\") on node \"ip-10-0-130-157.ec2.internal\" DevicePath \"\"" Jul 21 13:25:07.900980 ip-10-0-130-157 kubenswrapper[2576]: I0721 13:25:07.900856 2576 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/65942337-d38d-4be6-a5d1-427c2cdac308-dshm\") on node \"ip-10-0-130-157.ec2.internal\" DevicePath \"\"" Jul 21 13:25:08.498583 ip-10-0-130-157 kubenswrapper[2576]: I0721 13:25:08.498551 2576 generic.go:356] "Generic (PLEG): container finished" podID="65942337-d38d-4be6-a5d1-427c2cdac308" containerID="2e76f91144a32a8af0661733611f751d63d6f90f374f67fb79f9d27565f8752a" exitCode=0 Jul 21 13:25:08.498961 ip-10-0-130-157 kubenswrapper[2576]: I0721 13:25:08.498593 2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-cfed01/sub-test-model-cfed01-kserve-659bd84577-lccf8" Jul 21 13:25:08.498961 ip-10-0-130-157 kubenswrapper[2576]: I0721 13:25:08.498643 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-cfed01/sub-test-model-cfed01-kserve-659bd84577-lccf8" event={"ID":"65942337-d38d-4be6-a5d1-427c2cdac308","Type":"ContainerDied","Data":"2e76f91144a32a8af0661733611f751d63d6f90f374f67fb79f9d27565f8752a"} Jul 21 13:25:08.498961 ip-10-0-130-157 kubenswrapper[2576]: I0721 13:25:08.498686 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-cfed01/sub-test-model-cfed01-kserve-659bd84577-lccf8" event={"ID":"65942337-d38d-4be6-a5d1-427c2cdac308","Type":"ContainerDied","Data":"3701a5e07ae0e8c9047336efed935764a8f319f6722ec7cf93c8932433d550ec"} Jul 21 13:25:08.498961 ip-10-0-130-157 kubenswrapper[2576]: I0721 13:25:08.498706 2576 scope.go:117] "RemoveContainer" containerID="2e76f91144a32a8af0661733611f751d63d6f90f374f67fb79f9d27565f8752a" Jul 21 13:25:08.506844 ip-10-0-130-157 kubenswrapper[2576]: I0721 13:25:08.506828 2576 scope.go:117] "RemoveContainer" containerID="2e76f91144a32a8af0661733611f751d63d6f90f374f67fb79f9d27565f8752a" Jul 21 13:25:08.507110 ip-10-0-130-157 kubenswrapper[2576]: E0721 13:25:08.507092 2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2e76f91144a32a8af0661733611f751d63d6f90f374f67fb79f9d27565f8752a\": container with ID starting with 2e76f91144a32a8af0661733611f751d63d6f90f374f67fb79f9d27565f8752a not found: ID does not exist" containerID="2e76f91144a32a8af0661733611f751d63d6f90f374f67fb79f9d27565f8752a" Jul 21 13:25:08.507181 ip-10-0-130-157 kubenswrapper[2576]: I0721 13:25:08.507120 2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2e76f91144a32a8af0661733611f751d63d6f90f374f67fb79f9d27565f8752a"} err="failed to get container status \"2e76f91144a32a8af0661733611f751d63d6f90f374f67fb79f9d27565f8752a\": rpc error: code = NotFound desc = could not find container \"2e76f91144a32a8af0661733611f751d63d6f90f374f67fb79f9d27565f8752a\": container with ID starting with 2e76f91144a32a8af0661733611f751d63d6f90f374f67fb79f9d27565f8752a not found: ID does not exist" Jul 21 13:25:08.517271 ip-10-0-130-157 kubenswrapper[2576]: I0721 13:25:08.517248 2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-cfed01/sub-test-model-cfed01-kserve-659bd84577-lccf8"] Jul 21 13:25:08.522435 ip-10-0-130-157 kubenswrapper[2576]: I0721 13:25:08.522414 2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-b-cfed01/sub-test-model-cfed01-kserve-659bd84577-lccf8"] Jul 21 13:25:10.316333 ip-10-0-130-157 kubenswrapper[2576]: I0721 13:25:10.316301 2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="65942337-d38d-4be6-a5d1-427c2cdac308" path="/var/lib/kubelet/pods/65942337-d38d-4be6-a5d1-427c2cdac308/volumes" Jul 21 13:25:11.189395 ip-10-0-130-157 kubenswrapper[2576]: I0721 13:25:11.189364 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-8pmg8"] Jul 21 13:25:15.979469 ip-10-0-130-157 kubenswrapper[2576]: I0721 13:25:15.979432 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-a-b3a013/rate-test-model-b3a013-kserve-85d8bc796c-7zb6p" Jul 21 13:25:15.979469 ip-10-0-130-157 kubenswrapper[2576]: I0721 13:25:15.979479 2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-a-b3a013/rate-test-model-b3a013-kserve-85d8bc796c-7zb6p" Jul 21 13:25:15.991822 ip-10-0-130-157 kubenswrapper[2576]: I0721 13:25:15.991799 2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-a-b3a013/rate-test-model-b3a013-kserve-85d8bc796c-7zb6p" Jul 21 13:25:16.544618 ip-10-0-130-157 kubenswrapper[2576]: I0721 13:25:16.544588 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-a-b3a013/rate-test-model-b3a013-kserve-85d8bc796c-7zb6p" Jul 21 13:25:17.828789 ip-10-0-130-157 kubenswrapper[2576]: I0721 13:25:17.828755 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-skc26_6713665c-3b71-4751-929e-16de48263766/konnectivity-agent/0.log" limit=1024 Jul 21 13:25:17.947452 ip-10-0-130-157 kubenswrapper[2576]: I0721 13:25:17.947421 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-skc26_6713665c-3b71-4751-929e-16de48263766/konnectivity-agent/0.log" limit=1024 Jul 21 13:25:26.173771 ip-10-0-130-157 kubenswrapper[2576]: I0721 13:25:26.173736 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-skc26_6713665c-3b71-4751-929e-16de48263766/konnectivity-agent/0.log" limit=1024 Jul 21 13:25:26.274228 ip-10-0-130-157 kubenswrapper[2576]: I0721 13:25:26.274202 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-skc26_6713665c-3b71-4751-929e-16de48263766/konnectivity-agent/0.log" limit=1024 Jul 21 13:25:35.783272 ip-10-0-130-157 kubenswrapper[2576]: I0721 13:25:35.783235 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-skc26_6713665c-3b71-4751-929e-16de48263766/konnectivity-agent/0.log" limit=1024 Jul 21 13:25:35.885077 ip-10-0-130-157 kubenswrapper[2576]: I0721 13:25:35.885051 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-skc26_6713665c-3b71-4751-929e-16de48263766/konnectivity-agent/0.log" limit=1024 Jul 21 13:25:36.704729 ip-10-0-130-157 kubenswrapper[2576]: I0721 13:25:36.704702 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-skc26_6713665c-3b71-4751-929e-16de48263766/konnectivity-agent/0.log" limit=1024 Jul 21 13:25:40.186282 ip-10-0-130-157 kubenswrapper[2576]: I0721 13:25:40.186239 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-8pmg8"] Jul 21 13:25:53.234146 ip-10-0-130-157 kubenswrapper[2576]: I0721 13:25:53.234115 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-skc26_6713665c-3b71-4751-929e-16de48263766/konnectivity-agent/0.log" limit=1024 Jul 21 13:25:53.331874 ip-10-0-130-157 kubenswrapper[2576]: I0721 13:25:53.331844 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-skc26_6713665c-3b71-4751-929e-16de48263766/konnectivity-agent/0.log" limit=1024 Jul 21 13:25:54.896731 ip-10-0-130-157 kubenswrapper[2576]: I0721 13:25:54.896693 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-8pmg8"] Jul 21 13:26:00.721055 ip-10-0-130-157 kubenswrapper[2576]: I0721 13:26:00.721019 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-skc26_6713665c-3b71-4751-929e-16de48263766/konnectivity-agent/0.log" limit=1024 Jul 21 13:26:00.820771 ip-10-0-130-157 kubenswrapper[2576]: I0721 13:26:00.820740 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-skc26_6713665c-3b71-4751-929e-16de48263766/konnectivity-agent/0.log" limit=1024 Jul 21 13:26:09.578255 ip-10-0-130-157 kubenswrapper[2576]: I0721 13:26:09.578218 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-8pmg8"] Jul 21 13:26:16.773373 ip-10-0-130-157 kubenswrapper[2576]: I0721 13:26:16.773294 2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-b3a013/rate-test-model-b3a013-kserve-85d8bc796c-7zb6p"] Jul 21 13:26:16.797620 ip-10-0-130-157 kubenswrapper[2576]: I0721 13:26:16.797593 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-skc26_6713665c-3b71-4751-929e-16de48263766/konnectivity-agent/0.log" limit=1024 Jul 21 13:26:17.776209 ip-10-0-130-157 kubenswrapper[2576]: I0721 13:26:17.776181 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-skc26_6713665c-3b71-4751-929e-16de48263766/konnectivity-agent/0.log" limit=1024 Jul 21 13:26:18.763984 ip-10-0-130-157 kubenswrapper[2576]: I0721 13:26:18.763949 2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7292-140084598090160"] Jul 21 13:26:18.764276 ip-10-0-130-157 kubenswrapper[2576]: I0721 13:26:18.764263 2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="65942337-d38d-4be6-a5d1-427c2cdac308" containerName="main" Jul 21 13:26:18.764276 ip-10-0-130-157 kubenswrapper[2576]: I0721 13:26:18.764277 2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="65942337-d38d-4be6-a5d1-427c2cdac308" containerName="main" Jul 21 13:26:18.764367 ip-10-0-130-157 kubenswrapper[2576]: I0721 13:26:18.764329 2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="65942337-d38d-4be6-a5d1-427c2cdac308" containerName="main" Jul 21 13:26:18.767348 ip-10-0-130-157 kubenswrapper[2576]: I0721 13:26:18.767329 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7292-140084598090160" Jul 21 13:26:18.769312 ip-10-0-130-157 kubenswrapper[2576]: I0721 13:26:18.769289 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"default-dockercfg-bmbjm\"" Jul 21 13:26:18.774668 ip-10-0-130-157 kubenswrapper[2576]: I0721 13:26:18.774643 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7292-140084598090160"] Jul 21 13:26:18.816328 ip-10-0-130-157 kubenswrapper[2576]: I0721 13:26:18.816306 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mbrpl\" (UniqueName: \"kubernetes.io/projected/58614b6c-1f26-4496-8bb1-9ac9c46423db-kube-api-access-mbrpl\") pod \"test-curl-7292-140084598090160\" (UID: \"58614b6c-1f26-4496-8bb1-9ac9c46423db\") " pod="opendatahub/test-curl-7292-140084598090160" Jul 21 13:26:18.917647 ip-10-0-130-157 kubenswrapper[2576]: I0721 13:26:18.917627 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mbrpl\" (UniqueName: \"kubernetes.io/projected/58614b6c-1f26-4496-8bb1-9ac9c46423db-kube-api-access-mbrpl\") pod \"test-curl-7292-140084598090160\" (UID: \"58614b6c-1f26-4496-8bb1-9ac9c46423db\") " pod="opendatahub/test-curl-7292-140084598090160" Jul 21 13:26:18.926563 ip-10-0-130-157 kubenswrapper[2576]: I0721 13:26:18.926541 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mbrpl\" (UniqueName: \"kubernetes.io/projected/58614b6c-1f26-4496-8bb1-9ac9c46423db-kube-api-access-mbrpl\") pod \"test-curl-7292-140084598090160\" (UID: \"58614b6c-1f26-4496-8bb1-9ac9c46423db\") " pod="opendatahub/test-curl-7292-140084598090160" Jul 21 13:26:19.077705 ip-10-0-130-157 kubenswrapper[2576]: I0721 13:26:19.077648 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7292-140084598090160" Jul 21 13:26:19.202329 ip-10-0-130-157 kubenswrapper[2576]: I0721 13:26:19.202224 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7292-140084598090160"] Jul 21 13:26:19.204944 ip-10-0-130-157 kubenswrapper[2576]: W0721 13:26:19.204918 2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod58614b6c_1f26_4496_8bb1_9ac9c46423db.slice/crio-aa5447226143f9ee1a9d2700910254f97d76b53547235f54bc690309e669a211 WatchSource:0}: Error finding container aa5447226143f9ee1a9d2700910254f97d76b53547235f54bc690309e669a211: Status 404 returned error can't find the container with id aa5447226143f9ee1a9d2700910254f97d76b53547235f54bc690309e669a211 Jul 21 13:26:19.206627 ip-10-0-130-157 kubenswrapper[2576]: I0721 13:26:19.206608 2576 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 21 13:26:19.823152 ip-10-0-130-157 kubenswrapper[2576]: I0721 13:26:19.823118 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7292-140084598090160" event={"ID":"58614b6c-1f26-4496-8bb1-9ac9c46423db","Type":"ContainerStarted","Data":"aa5447226143f9ee1a9d2700910254f97d76b53547235f54bc690309e669a211"} Jul 21 13:26:20.829377 ip-10-0-130-157 kubenswrapper[2576]: I0721 13:26:20.829333 2576 generic.go:356] "Generic (PLEG): container finished" podID="58614b6c-1f26-4496-8bb1-9ac9c46423db" containerID="9d331f9ab97a7ebb92fccdad593422012ad1943a22b230fa9eecb038909236ae" exitCode=0 Jul 21 13:26:20.829806 ip-10-0-130-157 kubenswrapper[2576]: I0721 13:26:20.829426 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7292-140084598090160" event={"ID":"58614b6c-1f26-4496-8bb1-9ac9c46423db","Type":"ContainerDied","Data":"9d331f9ab97a7ebb92fccdad593422012ad1943a22b230fa9eecb038909236ae"} Jul 21 13:26:21.958868 ip-10-0-130-157 kubenswrapper[2576]: I0721 13:26:21.958848 2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7292-140084598090160" Jul 21 13:26:21.982430 ip-10-0-130-157 kubenswrapper[2576]: I0721 13:26:21.982397 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_test-curl-7292-140084598090160_58614b6c-1f26-4496-8bb1-9ac9c46423db/test-curl-7292-140084598090160/0.log" Jul 21 13:26:22.010323 ip-10-0-130-157 kubenswrapper[2576]: I0721 13:26:22.010299 2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7292-140084598090160"] Jul 21 13:26:22.014918 ip-10-0-130-157 kubenswrapper[2576]: I0721 13:26:22.014896 2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7292-140084598090160"] Jul 21 13:26:22.041085 ip-10-0-130-157 kubenswrapper[2576]: I0721 13:26:22.041058 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-mbrpl\" (UniqueName: \"kubernetes.io/projected/58614b6c-1f26-4496-8bb1-9ac9c46423db-kube-api-access-mbrpl\") pod \"58614b6c-1f26-4496-8bb1-9ac9c46423db\" (UID: \"58614b6c-1f26-4496-8bb1-9ac9c46423db\") " Jul 21 13:26:22.043097 ip-10-0-130-157 kubenswrapper[2576]: I0721 13:26:22.043064 2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/58614b6c-1f26-4496-8bb1-9ac9c46423db-kube-api-access-mbrpl" (OuterVolumeSpecName: "kube-api-access-mbrpl") pod "58614b6c-1f26-4496-8bb1-9ac9c46423db" (UID: "58614b6c-1f26-4496-8bb1-9ac9c46423db"). InnerVolumeSpecName "kube-api-access-mbrpl". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 21 13:26:22.141925 ip-10-0-130-157 kubenswrapper[2576]: I0721 13:26:22.141900 2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-mbrpl\" (UniqueName: \"kubernetes.io/projected/58614b6c-1f26-4496-8bb1-9ac9c46423db-kube-api-access-mbrpl\") on node \"ip-10-0-130-157.ec2.internal\" DevicePath \"\"" Jul 21 13:26:22.316590 ip-10-0-130-157 kubenswrapper[2576]: I0721 13:26:22.316557 2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="58614b6c-1f26-4496-8bb1-9ac9c46423db" path="/var/lib/kubelet/pods/58614b6c-1f26-4496-8bb1-9ac9c46423db/volumes" Jul 21 13:26:22.839421 ip-10-0-130-157 kubenswrapper[2576]: I0721 13:26:22.839390 2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7292-140084598090160" Jul 21 13:26:22.839421 ip-10-0-130-157 kubenswrapper[2576]: I0721 13:26:22.839417 2576 scope.go:117] "RemoveContainer" containerID="9d331f9ab97a7ebb92fccdad593422012ad1943a22b230fa9eecb038909236ae" Jul 21 13:26:24.482443 ip-10-0-130-157 kubenswrapper[2576]: I0721 13:26:24.482414 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-8pmg8"] Jul 21 13:26:31.792740 ip-10-0-130-157 kubenswrapper[2576]: I0721 13:26:31.792704 2576 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-a-b3a013/rate-test-model-b3a013-kserve-85d8bc796c-7zb6p" podUID="b5f85806-c45d-4317-831f-614dde87a6e9" containerName="main" containerID="cri-o://0ec6e6da59b49b9b9e97c120ed30affcc45f0f714467877ba004b028ab1e5af9" gracePeriod=45 Jul 21 13:26:32.030163 ip-10-0-130-157 kubenswrapper[2576]: I0721 13:26:32.030139 2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-b3a013/rate-test-model-b3a013-kserve-85d8bc796c-7zb6p" Jul 21 13:26:32.120128 ip-10-0-130-157 kubenswrapper[2576]: I0721 13:26:32.120048 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b5f85806-c45d-4317-831f-614dde87a6e9-model-cache\") pod \"b5f85806-c45d-4317-831f-614dde87a6e9\" (UID: \"b5f85806-c45d-4317-831f-614dde87a6e9\") " Jul 21 13:26:32.120128 ip-10-0-130-157 kubenswrapper[2576]: I0721 13:26:32.120120 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b5f85806-c45d-4317-831f-614dde87a6e9-tls-certs\") pod \"b5f85806-c45d-4317-831f-614dde87a6e9\" (UID: \"b5f85806-c45d-4317-831f-614dde87a6e9\") " Jul 21 13:26:32.120322 ip-10-0-130-157 kubenswrapper[2576]: I0721 13:26:32.120156 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b5f85806-c45d-4317-831f-614dde87a6e9-dshm\") pod \"b5f85806-c45d-4317-831f-614dde87a6e9\" (UID: \"b5f85806-c45d-4317-831f-614dde87a6e9\") " Jul 21 13:26:32.120322 ip-10-0-130-157 kubenswrapper[2576]: I0721 13:26:32.120189 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b5f85806-c45d-4317-831f-614dde87a6e9-tmp-dir\") pod \"b5f85806-c45d-4317-831f-614dde87a6e9\" (UID: \"b5f85806-c45d-4317-831f-614dde87a6e9\") " Jul 21 13:26:32.120322 ip-10-0-130-157 kubenswrapper[2576]: I0721 13:26:32.120227 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b5f85806-c45d-4317-831f-614dde87a6e9-home\") pod \"b5f85806-c45d-4317-831f-614dde87a6e9\" (UID: \"b5f85806-c45d-4317-831f-614dde87a6e9\") " Jul 21 13:26:32.120322 ip-10-0-130-157 kubenswrapper[2576]: I0721 13:26:32.120265 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-n6lp7\" (UniqueName: \"kubernetes.io/projected/b5f85806-c45d-4317-831f-614dde87a6e9-kube-api-access-n6lp7\") pod \"b5f85806-c45d-4317-831f-614dde87a6e9\" (UID: \"b5f85806-c45d-4317-831f-614dde87a6e9\") " Jul 21 13:26:32.120322 ip-10-0-130-157 kubenswrapper[2576]: I0721 13:26:32.120303 2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b5f85806-c45d-4317-831f-614dde87a6e9-model-cache" (OuterVolumeSpecName: "model-cache") pod "b5f85806-c45d-4317-831f-614dde87a6e9" (UID: "b5f85806-c45d-4317-831f-614dde87a6e9"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 21 13:26:32.120643 ip-10-0-130-157 kubenswrapper[2576]: I0721 13:26:32.120433 2576 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b5f85806-c45d-4317-831f-614dde87a6e9-model-cache\") on node \"ip-10-0-130-157.ec2.internal\" DevicePath \"\"" Jul 21 13:26:32.120733 ip-10-0-130-157 kubenswrapper[2576]: I0721 13:26:32.120709 2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b5f85806-c45d-4317-831f-614dde87a6e9-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "b5f85806-c45d-4317-831f-614dde87a6e9" (UID: "b5f85806-c45d-4317-831f-614dde87a6e9"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 21 13:26:32.120837 ip-10-0-130-157 kubenswrapper[2576]: I0721 13:26:32.120793 2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b5f85806-c45d-4317-831f-614dde87a6e9-home" (OuterVolumeSpecName: "home") pod "b5f85806-c45d-4317-831f-614dde87a6e9" (UID: "b5f85806-c45d-4317-831f-614dde87a6e9"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 21 13:26:32.122536 ip-10-0-130-157 kubenswrapper[2576]: I0721 13:26:32.122494 2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b5f85806-c45d-4317-831f-614dde87a6e9-dshm" (OuterVolumeSpecName: "dshm") pod "b5f85806-c45d-4317-831f-614dde87a6e9" (UID: "b5f85806-c45d-4317-831f-614dde87a6e9"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 21 13:26:32.122640 ip-10-0-130-157 kubenswrapper[2576]: I0721 13:26:32.122580 2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b5f85806-c45d-4317-831f-614dde87a6e9-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "b5f85806-c45d-4317-831f-614dde87a6e9" (UID: "b5f85806-c45d-4317-831f-614dde87a6e9"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 21 13:26:32.122640 ip-10-0-130-157 kubenswrapper[2576]: I0721 13:26:32.122615 2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b5f85806-c45d-4317-831f-614dde87a6e9-kube-api-access-n6lp7" (OuterVolumeSpecName: "kube-api-access-n6lp7") pod "b5f85806-c45d-4317-831f-614dde87a6e9" (UID: "b5f85806-c45d-4317-831f-614dde87a6e9"). InnerVolumeSpecName "kube-api-access-n6lp7". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 21 13:26:32.221182 ip-10-0-130-157 kubenswrapper[2576]: I0721 13:26:32.221160 2576 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b5f85806-c45d-4317-831f-614dde87a6e9-home\") on node \"ip-10-0-130-157.ec2.internal\" DevicePath \"\"" Jul 21 13:26:32.221182 ip-10-0-130-157 kubenswrapper[2576]: I0721 13:26:32.221181 2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-n6lp7\" (UniqueName: \"kubernetes.io/projected/b5f85806-c45d-4317-831f-614dde87a6e9-kube-api-access-n6lp7\") on node \"ip-10-0-130-157.ec2.internal\" DevicePath \"\"" Jul 21 13:26:32.221310 ip-10-0-130-157 kubenswrapper[2576]: I0721 13:26:32.221192 2576 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b5f85806-c45d-4317-831f-614dde87a6e9-tls-certs\") on node \"ip-10-0-130-157.ec2.internal\" DevicePath \"\"" Jul 21 13:26:32.221310 ip-10-0-130-157 kubenswrapper[2576]: I0721 13:26:32.221200 2576 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b5f85806-c45d-4317-831f-614dde87a6e9-dshm\") on node \"ip-10-0-130-157.ec2.internal\" DevicePath \"\"" Jul 21 13:26:32.221310 ip-10-0-130-157 kubenswrapper[2576]: I0721 13:26:32.221209 2576 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b5f85806-c45d-4317-831f-614dde87a6e9-tmp-dir\") on node \"ip-10-0-130-157.ec2.internal\" DevicePath \"\"" Jul 21 13:26:32.885805 ip-10-0-130-157 kubenswrapper[2576]: I0721 13:26:32.885769 2576 generic.go:356] "Generic (PLEG): container finished" podID="b5f85806-c45d-4317-831f-614dde87a6e9" containerID="0ec6e6da59b49b9b9e97c120ed30affcc45f0f714467877ba004b028ab1e5af9" exitCode=0 Jul 21 13:26:32.886248 ip-10-0-130-157 kubenswrapper[2576]: I0721 13:26:32.885833 2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-b3a013/rate-test-model-b3a013-kserve-85d8bc796c-7zb6p" Jul 21 13:26:32.886248 ip-10-0-130-157 kubenswrapper[2576]: I0721 13:26:32.885856 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-b3a013/rate-test-model-b3a013-kserve-85d8bc796c-7zb6p" event={"ID":"b5f85806-c45d-4317-831f-614dde87a6e9","Type":"ContainerDied","Data":"0ec6e6da59b49b9b9e97c120ed30affcc45f0f714467877ba004b028ab1e5af9"} Jul 21 13:26:32.886248 ip-10-0-130-157 kubenswrapper[2576]: I0721 13:26:32.885898 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-b3a013/rate-test-model-b3a013-kserve-85d8bc796c-7zb6p" event={"ID":"b5f85806-c45d-4317-831f-614dde87a6e9","Type":"ContainerDied","Data":"a889a40ae0f35db849e146a014ddba8045341d6e0bfc8ab8541864d06c6ccb4a"} Jul 21 13:26:32.886248 ip-10-0-130-157 kubenswrapper[2576]: I0721 13:26:32.885921 2576 scope.go:117] "RemoveContainer" containerID="0ec6e6da59b49b9b9e97c120ed30affcc45f0f714467877ba004b028ab1e5af9" Jul 21 13:26:32.894203 ip-10-0-130-157 kubenswrapper[2576]: I0721 13:26:32.894184 2576 scope.go:117] "RemoveContainer" containerID="0ec6e6da59b49b9b9e97c120ed30affcc45f0f714467877ba004b028ab1e5af9" Jul 21 13:26:32.894431 ip-10-0-130-157 kubenswrapper[2576]: E0721 13:26:32.894413 2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0ec6e6da59b49b9b9e97c120ed30affcc45f0f714467877ba004b028ab1e5af9\": container with ID starting with 0ec6e6da59b49b9b9e97c120ed30affcc45f0f714467877ba004b028ab1e5af9 not found: ID does not exist" containerID="0ec6e6da59b49b9b9e97c120ed30affcc45f0f714467877ba004b028ab1e5af9" Jul 21 13:26:32.894480 ip-10-0-130-157 kubenswrapper[2576]: I0721 13:26:32.894440 2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0ec6e6da59b49b9b9e97c120ed30affcc45f0f714467877ba004b028ab1e5af9"} err="failed to get container status \"0ec6e6da59b49b9b9e97c120ed30affcc45f0f714467877ba004b028ab1e5af9\": rpc error: code = NotFound desc = could not find container \"0ec6e6da59b49b9b9e97c120ed30affcc45f0f714467877ba004b028ab1e5af9\": container with ID starting with 0ec6e6da59b49b9b9e97c120ed30affcc45f0f714467877ba004b028ab1e5af9 not found: ID does not exist" Jul 21 13:26:32.905998 ip-10-0-130-157 kubenswrapper[2576]: I0721 13:26:32.905974 2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-b3a013/rate-test-model-b3a013-kserve-85d8bc796c-7zb6p"] Jul 21 13:26:32.908791 ip-10-0-130-157 kubenswrapper[2576]: I0721 13:26:32.908772 2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-a-b3a013/rate-test-model-b3a013-kserve-85d8bc796c-7zb6p"] Jul 21 13:26:34.315817 ip-10-0-130-157 kubenswrapper[2576]: I0721 13:26:34.315787 2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b5f85806-c45d-4317-831f-614dde87a6e9" path="/var/lib/kubelet/pods/b5f85806-c45d-4317-831f-614dde87a6e9/volumes" Jul 21 13:26:38.292259 ip-10-0-130-157 kubenswrapper[2576]: I0721 13:26:38.292227 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-skc26_6713665c-3b71-4751-929e-16de48263766/konnectivity-agent/0.log" limit=1024 Jul 21 13:26:38.389176 ip-10-0-130-157 kubenswrapper[2576]: I0721 13:26:38.389143 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-skc26_6713665c-3b71-4751-929e-16de48263766/konnectivity-agent/0.log" limit=1024 Jul 21 13:26:48.279440 ip-10-0-130-157 kubenswrapper[2576]: I0721 13:26:48.279408 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-skc26_6713665c-3b71-4751-929e-16de48263766/konnectivity-agent/0.log" limit=1024 Jul 21 13:26:51.504826 ip-10-0-130-157 kubenswrapper[2576]: I0721 13:26:51.504792 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-8pmg8"] Jul 21 13:26:52.672948 ip-10-0-130-157 kubenswrapper[2576]: I0721 13:26:52.672912 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-skc26_6713665c-3b71-4751-929e-16de48263766/konnectivity-agent/0.log" limit=1024 Jul 21 13:26:53.103782 ip-10-0-130-157 kubenswrapper[2576]: I0721 13:26:53.103705 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-skc26_6713665c-3b71-4751-929e-16de48263766/konnectivity-agent/0.log" limit=1024 Jul 21 13:27:00.186489 ip-10-0-130-157 kubenswrapper[2576]: I0721 13:27:00.186455 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-skc26_6713665c-3b71-4751-929e-16de48263766/konnectivity-agent/0.log" limit=1024 Jul 21 13:27:00.285090 ip-10-0-130-157 kubenswrapper[2576]: I0721 13:27:00.285059 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-skc26_6713665c-3b71-4751-929e-16de48263766/konnectivity-agent/0.log" limit=1024 Jul 21 13:27:00.478529 ip-10-0-130-157 kubenswrapper[2576]: I0721 13:27:00.478438 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-skc26_6713665c-3b71-4751-929e-16de48263766/konnectivity-agent/0.log" limit=1024 Jul 21 13:27:00.703372 ip-10-0-130-157 kubenswrapper[2576]: I0721 13:27:00.703347 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-skc26_6713665c-3b71-4751-929e-16de48263766/konnectivity-agent/0.log" limit=1024 Jul 21 13:27:00.810427 ip-10-0-130-157 kubenswrapper[2576]: I0721 13:27:00.810361 2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-cfed01-mbhcs"] Jul 21 13:27:00.810673 ip-10-0-130-157 kubenswrapper[2576]: I0721 13:27:00.810660 2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b5f85806-c45d-4317-831f-614dde87a6e9" containerName="main" Jul 21 13:27:00.810721 ip-10-0-130-157 kubenswrapper[2576]: I0721 13:27:00.810674 2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="b5f85806-c45d-4317-831f-614dde87a6e9" containerName="main" Jul 21 13:27:00.810721 ip-10-0-130-157 kubenswrapper[2576]: I0721 13:27:00.810691 2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="58614b6c-1f26-4496-8bb1-9ac9c46423db" containerName="test-curl-7292-140084598090160" Jul 21 13:27:00.810721 ip-10-0-130-157 kubenswrapper[2576]: I0721 13:27:00.810699 2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="58614b6c-1f26-4496-8bb1-9ac9c46423db" containerName="test-curl-7292-140084598090160" Jul 21 13:27:00.810818 ip-10-0-130-157 kubenswrapper[2576]: I0721 13:27:00.810744 2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="58614b6c-1f26-4496-8bb1-9ac9c46423db" containerName="test-curl-7292-140084598090160" Jul 21 13:27:00.810818 ip-10-0-130-157 kubenswrapper[2576]: I0721 13:27:00.810754 2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="b5f85806-c45d-4317-831f-614dde87a6e9" containerName="main" Jul 21 13:27:00.815989 ip-10-0-130-157 kubenswrapper[2576]: I0721 13:27:00.815970 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-cfed01-mbhcs" Jul 21 13:27:00.820186 ip-10-0-130-157 kubenswrapper[2576]: I0721 13:27:00.820169 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\"" Jul 21 13:27:00.820301 ip-10-0-130-157 kubenswrapper[2576]: I0721 13:27:00.820241 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-5nsrr\"" Jul 21 13:27:00.823875 ip-10-0-130-157 kubenswrapper[2576]: I0721 13:27:00.823852 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-cfed01-mbhcs"] Jul 21 13:27:00.931987 ip-10-0-130-157 kubenswrapper[2576]: I0721 13:27:00.931951 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/7491f36c-fc60-43cc-93ad-358050f79f19-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-b-cfed01-mbhcs\" (UID: \"7491f36c-fc60-43cc-93ad-358050f79f19\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-cfed01-mbhcs" Jul 21 13:27:01.032909 ip-10-0-130-157 kubenswrapper[2576]: I0721 13:27:01.032886 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/7491f36c-fc60-43cc-93ad-358050f79f19-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-b-cfed01-mbhcs\" (UID: \"7491f36c-fc60-43cc-93ad-358050f79f19\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-cfed01-mbhcs" Jul 21 13:27:01.033462 ip-10-0-130-157 kubenswrapper[2576]: I0721 13:27:01.033445 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/7491f36c-fc60-43cc-93ad-358050f79f19-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-b-cfed01-mbhcs\" (UID: \"7491f36c-fc60-43cc-93ad-358050f79f19\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-cfed01-mbhcs" Jul 21 13:27:01.127009 ip-10-0-130-157 kubenswrapper[2576]: I0721 13:27:01.126951 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-cfed01-mbhcs" Jul 21 13:27:01.257479 ip-10-0-130-157 kubenswrapper[2576]: I0721 13:27:01.257447 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-cfed01-mbhcs"] Jul 21 13:27:01.260742 ip-10-0-130-157 kubenswrapper[2576]: W0721 13:27:01.260711 2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7491f36c_fc60_43cc_93ad_358050f79f19.slice/crio-fc07b92e1e3ecec8c409f009d7cb290d306fca378b9fd9b205084954de400094 WatchSource:0}: Error finding container fc07b92e1e3ecec8c409f009d7cb290d306fca378b9fd9b205084954de400094: Status 404 returned error can't find the container with id fc07b92e1e3ecec8c409f009d7cb290d306fca378b9fd9b205084954de400094 Jul 21 13:27:01.509339 ip-10-0-130-157 kubenswrapper[2576]: I0721 13:27:01.509312 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-skc26_6713665c-3b71-4751-929e-16de48263766/konnectivity-agent/0.log" limit=1024 Jul 21 13:27:01.606095 ip-10-0-130-157 kubenswrapper[2576]: I0721 13:27:01.606071 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-skc26_6713665c-3b71-4751-929e-16de48263766/konnectivity-agent/0.log" limit=1024 Jul 21 13:27:02.012069 ip-10-0-130-157 kubenswrapper[2576]: I0721 13:27:02.012042 2576 generic.go:356] "Generic (PLEG): container finished" podID="7491f36c-fc60-43cc-93ad-358050f79f19" containerID="c4e5106adacbcb5d209bf2a9ca9eb3e8667bc1ec66740265c28b6be7d6c3a265" exitCode=0 Jul 21 13:27:02.012169 ip-10-0-130-157 kubenswrapper[2576]: I0721 13:27:02.012104 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-cfed01-mbhcs" event={"ID":"7491f36c-fc60-43cc-93ad-358050f79f19","Type":"ContainerDied","Data":"c4e5106adacbcb5d209bf2a9ca9eb3e8667bc1ec66740265c28b6be7d6c3a265"} Jul 21 13:27:02.012169 ip-10-0-130-157 kubenswrapper[2576]: I0721 13:27:02.012141 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-cfed01-mbhcs" event={"ID":"7491f36c-fc60-43cc-93ad-358050f79f19","Type":"ContainerStarted","Data":"fc07b92e1e3ecec8c409f009d7cb290d306fca378b9fd9b205084954de400094"} Jul 21 13:27:03.145582 ip-10-0-130-157 kubenswrapper[2576]: I0721 13:27:03.145557 2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-cfed01-mbhcs" Jul 21 13:27:03.251592 ip-10-0-130-157 kubenswrapper[2576]: I0721 13:27:03.251560 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/7491f36c-fc60-43cc-93ad-358050f79f19-maas-api-service-ca\") pod \"7491f36c-fc60-43cc-93ad-358050f79f19\" (UID: \"7491f36c-fc60-43cc-93ad-358050f79f19\") " Jul 21 13:27:03.251901 ip-10-0-130-157 kubenswrapper[2576]: I0721 13:27:03.251878 2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7491f36c-fc60-43cc-93ad-358050f79f19-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "7491f36c-fc60-43cc-93ad-358050f79f19" (UID: "7491f36c-fc60-43cc-93ad-358050f79f19"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 21 13:27:03.352776 ip-10-0-130-157 kubenswrapper[2576]: I0721 13:27:03.352699 2576 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/7491f36c-fc60-43cc-93ad-358050f79f19-maas-api-service-ca\") on node \"ip-10-0-130-157.ec2.internal\" DevicePath \"\"" Jul 21 13:27:04.021834 ip-10-0-130-157 kubenswrapper[2576]: I0721 13:27:04.021801 2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-cfed01-mbhcs" Jul 21 13:27:04.021834 ip-10-0-130-157 kubenswrapper[2576]: I0721 13:27:04.021811 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-cfed01-mbhcs" event={"ID":"7491f36c-fc60-43cc-93ad-358050f79f19","Type":"ContainerDied","Data":"fc07b92e1e3ecec8c409f009d7cb290d306fca378b9fd9b205084954de400094"} Jul 21 13:27:04.022047 ip-10-0-130-157 kubenswrapper[2576]: I0721 13:27:04.021862 2576 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="fc07b92e1e3ecec8c409f009d7cb290d306fca378b9fd9b205084954de400094" Jul 21 13:27:11.419553 ip-10-0-130-157 kubenswrapper[2576]: I0721 13:27:11.419505 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-skc26_6713665c-3b71-4751-929e-16de48263766/konnectivity-agent/0.log" limit=1024 Jul 21 13:27:14.403408 ip-10-0-130-157 kubenswrapper[2576]: I0721 13:27:14.403371 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-8pmg8"] Jul 21 13:27:15.917295 ip-10-0-130-157 kubenswrapper[2576]: I0721 13:27:15.917261 2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-cfed01-mbhcs"] Jul 21 13:27:15.917820 ip-10-0-130-157 kubenswrapper[2576]: I0721 13:27:15.917796 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-skc26_6713665c-3b71-4751-929e-16de48263766/konnectivity-agent/0.log" limit=1024 Jul 21 13:27:15.919878 ip-10-0-130-157 kubenswrapper[2576]: I0721 13:27:15.919852 2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-cfed01-mbhcs"] Jul 21 13:27:16.312104 ip-10-0-130-157 kubenswrapper[2576]: I0721 13:27:16.312030 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-skc26_6713665c-3b71-4751-929e-16de48263766/konnectivity-agent/0.log" limit=1024 Jul 21 13:27:16.315968 ip-10-0-130-157 kubenswrapper[2576]: I0721 13:27:16.315943 2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="7491f36c-fc60-43cc-93ad-358050f79f19" path="/var/lib/kubelet/pods/7491f36c-fc60-43cc-93ad-358050f79f19/volumes" Jul 21 13:27:22.813174 ip-10-0-130-157 kubenswrapper[2576]: I0721 13:27:22.813138 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-skc26_6713665c-3b71-4751-929e-16de48263766/konnectivity-agent/0.log" limit=1024 Jul 21 13:27:22.917842 ip-10-0-130-157 kubenswrapper[2576]: I0721 13:27:22.917810 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-skc26_6713665c-3b71-4751-929e-16de48263766/konnectivity-agent/0.log" limit=1024 Jul 21 13:27:23.161000 ip-10-0-130-157 kubenswrapper[2576]: I0721 13:27:23.160970 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-skc26_6713665c-3b71-4751-929e-16de48263766/konnectivity-agent/0.log" limit=1024 Jul 21 13:27:23.366027 ip-10-0-130-157 kubenswrapper[2576]: I0721 13:27:23.365996 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-skc26_6713665c-3b71-4751-929e-16de48263766/konnectivity-agent/0.log" limit=1024 Jul 21 13:27:23.815907 ip-10-0-130-157 kubenswrapper[2576]: I0721 13:27:23.815877 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-5597f86487-f7kvl_3eabb5f5-46c3-4820-a666-4036a4afbba0/authorino/0.log" Jul 21 13:27:30.215705 ip-10-0-130-157 kubenswrapper[2576]: I0721 13:27:30.215677 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-579d89bddd-mk77w_3e7a2646-5d2a-498c-878f-f3c1f6834381/copy-manifests/0.log" Jul 21 13:27:30.225744 ip-10-0-130-157 kubenswrapper[2576]: I0721 13:27:30.225715 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-579d89bddd-mk77w_3e7a2646-5d2a-498c-878f-f3c1f6834381/manager/0.log" Jul 21 13:27:31.113865 ip-10-0-130-157 kubenswrapper[2576]: I0721 13:27:31.113835 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-key-cleanup-29743995-gp29b_7556cf0b-54d0-4ed8-bb42-a2a1055d688a/cleanup/0.log" Jul 21 13:27:31.150099 ip-10-0-130-157 kubenswrapper[2576]: I0721 13:27:31.150072 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-skc26_6713665c-3b71-4751-929e-16de48263766/konnectivity-agent/0.log" limit=1024 Jul 21 13:27:32.638822 ip-10-0-130-157 kubenswrapper[2576]: I0721 13:27:32.638796 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-5597f86487-f7kvl_3eabb5f5-46c3-4820-a666-4036a4afbba0/authorino/0.log" Jul 21 13:27:33.026895 ip-10-0-130-157 kubenswrapper[2576]: I0721 13:27:33.026823 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-console-plugin-66b5774985-khhrm_f54afc32-9dcd-47e8-a25a-199b3a57866d/kuadrant-console-plugin/0.log" Jul 21 13:27:33.425881 ip-10-0-130-157 kubenswrapper[2576]: I0721 13:27:33.425856 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-limitador-54d6d988d5-8pmg8_7225c4af-0114-4819-8db0-a306a9f04752/limitador/0.log" Jul 21 13:27:34.172329 ip-10-0-130-157 kubenswrapper[2576]: I0721 13:27:34.172299 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-67cb6d6b5d-424n4_c8997198-ef13-4846-90eb-47345692f1a9/kube-auth-proxy/0.log" Jul 21 13:27:35.176282 ip-10-0-130-157 kubenswrapper[2576]: I0721 13:27:35.176254 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-distinct-2-simulated-kserve-67d8d5d6d-pvmgz_124c966d-681b-4916-8c7f-e2c065462afc/main/0.log" Jul 21 13:27:35.793057 ip-10-0-130-157 kubenswrapper[2576]: I0721 13:27:35.793021 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_premium-simulated-simulated-premium-kserve-57fc66f697-5r52s_f02a054b-0324-4744-b923-1cb3be68fcb4/main/0.log" Jul 21 13:27:42.119291 ip-10-0-130-157 kubenswrapper[2576]: I0721 13:27:42.119253 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-skc26_6713665c-3b71-4751-929e-16de48263766/konnectivity-agent/0.log" limit=1024 Jul 21 13:27:42.218568 ip-10-0-130-157 kubenswrapper[2576]: I0721 13:27:42.218538 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-skc26_6713665c-3b71-4751-929e-16de48263766/konnectivity-agent/0.log" limit=1024 Jul 21 13:27:44.987184 ip-10-0-130-157 kubenswrapper[2576]: I0721 13:27:44.987104 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-g2tkf_4dc9bfad-f2bb-4934-9f49-4cfd0f381ab2/global-pull-secret-syncer/0.log" Jul 21 13:27:45.228126 ip-10-0-130-157 kubenswrapper[2576]: I0721 13:27:45.228099 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-skc26_6713665c-3b71-4751-929e-16de48263766/konnectivity-agent/0.log" Jul 21 13:27:45.387164 ip-10-0-130-157 kubenswrapper[2576]: I0721 13:27:45.387142 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-130-157.ec2.internal_77735b552764c08668c42d1fa6bcbce3/haproxy/0.log" Jul 21 13:27:50.501049 ip-10-0-130-157 kubenswrapper[2576]: I0721 13:27:50.501016 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-5597f86487-f7kvl_3eabb5f5-46c3-4820-a666-4036a4afbba0/authorino/0.log" Jul 21 13:27:50.625334 ip-10-0-130-157 kubenswrapper[2576]: I0721 13:27:50.625270 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-console-plugin-66b5774985-khhrm_f54afc32-9dcd-47e8-a25a-199b3a57866d/kuadrant-console-plugin/0.log" Jul 21 13:27:51.065031 ip-10-0-130-157 kubenswrapper[2576]: I0721 13:27:51.065001 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-limitador-54d6d988d5-8pmg8_7225c4af-0114-4819-8db0-a306a9f04752/limitador/0.log" Jul 21 13:27:53.274164 ip-10-0-130-157 kubenswrapper[2576]: I0721 13:27:53.274085 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_metrics-server-597974fb65-bdrwd_ea945a18-e37f-4957-8c7d-bee0cb2e9421/metrics-server/0.log" Jul 21 13:27:53.363422 ip-10-0-130-157 kubenswrapper[2576]: I0721 13:27:53.363396 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-c69sx_a6e83d7e-1979-4094-8beb-c3f19ab6ccdf/node-exporter/0.log" Jul 21 13:27:53.393397 ip-10-0-130-157 kubenswrapper[2576]: I0721 13:27:53.393373 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-c69sx_a6e83d7e-1979-4094-8beb-c3f19ab6ccdf/kube-rbac-proxy/0.log" Jul 21 13:27:53.428609 ip-10-0-130-157 kubenswrapper[2576]: I0721 13:27:53.428593 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-c69sx_a6e83d7e-1979-4094-8beb-c3f19ab6ccdf/init-textfile/0.log" Jul 21 13:27:53.769237 ip-10-0-130-157 kubenswrapper[2576]: I0721 13:27:53.769213 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_4911b6ac-242a-4014-933d-6821ff61e77a/prometheus/0.log" Jul 21 13:27:53.800001 ip-10-0-130-157 kubenswrapper[2576]: I0721 13:27:53.799981 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_4911b6ac-242a-4014-933d-6821ff61e77a/config-reloader/0.log" Jul 21 13:27:53.831373 ip-10-0-130-157 kubenswrapper[2576]: I0721 13:27:53.831357 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_4911b6ac-242a-4014-933d-6821ff61e77a/thanos-sidecar/0.log" Jul 21 13:27:53.862212 ip-10-0-130-157 kubenswrapper[2576]: I0721 13:27:53.862197 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_4911b6ac-242a-4014-933d-6821ff61e77a/kube-rbac-proxy-web/0.log" Jul 21 13:27:53.893599 ip-10-0-130-157 kubenswrapper[2576]: I0721 13:27:53.893584 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_4911b6ac-242a-4014-933d-6821ff61e77a/kube-rbac-proxy/0.log" Jul 21 13:27:53.942129 ip-10-0-130-157 kubenswrapper[2576]: I0721 13:27:53.942107 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_4911b6ac-242a-4014-933d-6821ff61e77a/kube-rbac-proxy-thanos/0.log" Jul 21 13:27:53.988868 ip-10-0-130-157 kubenswrapper[2576]: I0721 13:27:53.988848 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_4911b6ac-242a-4014-933d-6821ff61e77a/init-config-reloader/0.log" Jul 21 13:27:54.757020 ip-10-0-130-157 kubenswrapper[2576]: I0721 13:27:54.756991 2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-s9mdb/perf-node-gather-daemonset-7shbm"] Jul 21 13:27:54.757379 ip-10-0-130-157 kubenswrapper[2576]: I0721 13:27:54.757273 2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7491f36c-fc60-43cc-93ad-358050f79f19" containerName="revoke-keys" Jul 21 13:27:54.757379 ip-10-0-130-157 kubenswrapper[2576]: I0721 13:27:54.757284 2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="7491f36c-fc60-43cc-93ad-358050f79f19" containerName="revoke-keys" Jul 21 13:27:54.757379 ip-10-0-130-157 kubenswrapper[2576]: I0721 13:27:54.757339 2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="7491f36c-fc60-43cc-93ad-358050f79f19" containerName="revoke-keys" Jul 21 13:27:54.760592 ip-10-0-130-157 kubenswrapper[2576]: I0721 13:27:54.760570 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-s9mdb/perf-node-gather-daemonset-7shbm" Jul 21 13:27:54.763714 ip-10-0-130-157 kubenswrapper[2576]: I0721 13:27:54.763693 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-s9mdb\"/\"kube-root-ca.crt\"" Jul 21 13:27:54.763814 ip-10-0-130-157 kubenswrapper[2576]: I0721 13:27:54.763738 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-s9mdb\"/\"openshift-service-ca.crt\"" Jul 21 13:27:54.763814 ip-10-0-130-157 kubenswrapper[2576]: I0721 13:27:54.763775 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-s9mdb\"/\"default-dockercfg-kpxh2\"" Jul 21 13:27:54.773334 ip-10-0-130-157 kubenswrapper[2576]: I0721 13:27:54.773312 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-s9mdb/perf-node-gather-daemonset-7shbm"] Jul 21 13:27:54.830407 ip-10-0-130-157 kubenswrapper[2576]: I0721 13:27:54.830380 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/9a7235f4-65e9-425a-90d3-f65f16d37848-proc\") pod \"perf-node-gather-daemonset-7shbm\" (UID: \"9a7235f4-65e9-425a-90d3-f65f16d37848\") " pod="openshift-must-gather-s9mdb/perf-node-gather-daemonset-7shbm" Jul 21 13:27:54.830537 ip-10-0-130-157 kubenswrapper[2576]: I0721 13:27:54.830412 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/9a7235f4-65e9-425a-90d3-f65f16d37848-lib-modules\") pod \"perf-node-gather-daemonset-7shbm\" (UID: \"9a7235f4-65e9-425a-90d3-f65f16d37848\") " pod="openshift-must-gather-s9mdb/perf-node-gather-daemonset-7shbm" Jul 21 13:27:54.830537 ip-10-0-130-157 kubenswrapper[2576]: I0721 13:27:54.830430 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/9a7235f4-65e9-425a-90d3-f65f16d37848-podres\") pod \"perf-node-gather-daemonset-7shbm\" (UID: \"9a7235f4-65e9-425a-90d3-f65f16d37848\") " pod="openshift-must-gather-s9mdb/perf-node-gather-daemonset-7shbm" Jul 21 13:27:54.830615 ip-10-0-130-157 kubenswrapper[2576]: I0721 13:27:54.830544 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8m7z2\" (UniqueName: \"kubernetes.io/projected/9a7235f4-65e9-425a-90d3-f65f16d37848-kube-api-access-8m7z2\") pod \"perf-node-gather-daemonset-7shbm\" (UID: \"9a7235f4-65e9-425a-90d3-f65f16d37848\") " pod="openshift-must-gather-s9mdb/perf-node-gather-daemonset-7shbm" Jul 21 13:27:54.830654 ip-10-0-130-157 kubenswrapper[2576]: I0721 13:27:54.830627 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/9a7235f4-65e9-425a-90d3-f65f16d37848-sys\") pod \"perf-node-gather-daemonset-7shbm\" (UID: \"9a7235f4-65e9-425a-90d3-f65f16d37848\") " pod="openshift-must-gather-s9mdb/perf-node-gather-daemonset-7shbm" Jul 21 13:27:54.931610 ip-10-0-130-157 kubenswrapper[2576]: I0721 13:27:54.931586 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/9a7235f4-65e9-425a-90d3-f65f16d37848-sys\") pod \"perf-node-gather-daemonset-7shbm\" (UID: \"9a7235f4-65e9-425a-90d3-f65f16d37848\") " pod="openshift-must-gather-s9mdb/perf-node-gather-daemonset-7shbm" Jul 21 13:27:54.931716 ip-10-0-130-157 kubenswrapper[2576]: I0721 13:27:54.931616 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/9a7235f4-65e9-425a-90d3-f65f16d37848-proc\") pod \"perf-node-gather-daemonset-7shbm\" (UID: \"9a7235f4-65e9-425a-90d3-f65f16d37848\") " pod="openshift-must-gather-s9mdb/perf-node-gather-daemonset-7shbm" Jul 21 13:27:54.931716 ip-10-0-130-157 kubenswrapper[2576]: I0721 13:27:54.931635 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/9a7235f4-65e9-425a-90d3-f65f16d37848-lib-modules\") pod \"perf-node-gather-daemonset-7shbm\" (UID: \"9a7235f4-65e9-425a-90d3-f65f16d37848\") " pod="openshift-must-gather-s9mdb/perf-node-gather-daemonset-7shbm" Jul 21 13:27:54.931716 ip-10-0-130-157 kubenswrapper[2576]: I0721 13:27:54.931672 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/9a7235f4-65e9-425a-90d3-f65f16d37848-podres\") pod \"perf-node-gather-daemonset-7shbm\" (UID: \"9a7235f4-65e9-425a-90d3-f65f16d37848\") " pod="openshift-must-gather-s9mdb/perf-node-gather-daemonset-7shbm" Jul 21 13:27:54.931716 ip-10-0-130-157 kubenswrapper[2576]: I0721 13:27:54.931695 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/9a7235f4-65e9-425a-90d3-f65f16d37848-sys\") pod \"perf-node-gather-daemonset-7shbm\" (UID: \"9a7235f4-65e9-425a-90d3-f65f16d37848\") " pod="openshift-must-gather-s9mdb/perf-node-gather-daemonset-7shbm" Jul 21 13:27:54.931716 ip-10-0-130-157 kubenswrapper[2576]: I0721 13:27:54.931709 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/9a7235f4-65e9-425a-90d3-f65f16d37848-proc\") pod \"perf-node-gather-daemonset-7shbm\" (UID: \"9a7235f4-65e9-425a-90d3-f65f16d37848\") " pod="openshift-must-gather-s9mdb/perf-node-gather-daemonset-7shbm" Jul 21 13:27:54.931901 ip-10-0-130-157 kubenswrapper[2576]: I0721 13:27:54.931720 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8m7z2\" (UniqueName: \"kubernetes.io/projected/9a7235f4-65e9-425a-90d3-f65f16d37848-kube-api-access-8m7z2\") pod \"perf-node-gather-daemonset-7shbm\" (UID: \"9a7235f4-65e9-425a-90d3-f65f16d37848\") " pod="openshift-must-gather-s9mdb/perf-node-gather-daemonset-7shbm" Jul 21 13:27:54.931901 ip-10-0-130-157 kubenswrapper[2576]: I0721 13:27:54.931796 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/9a7235f4-65e9-425a-90d3-f65f16d37848-lib-modules\") pod \"perf-node-gather-daemonset-7shbm\" (UID: \"9a7235f4-65e9-425a-90d3-f65f16d37848\") " pod="openshift-must-gather-s9mdb/perf-node-gather-daemonset-7shbm" Jul 21 13:27:54.931901 ip-10-0-130-157 kubenswrapper[2576]: I0721 13:27:54.931810 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/9a7235f4-65e9-425a-90d3-f65f16d37848-podres\") pod \"perf-node-gather-daemonset-7shbm\" (UID: \"9a7235f4-65e9-425a-90d3-f65f16d37848\") " pod="openshift-must-gather-s9mdb/perf-node-gather-daemonset-7shbm" Jul 21 13:27:54.941531 ip-10-0-130-157 kubenswrapper[2576]: I0721 13:27:54.941493 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8m7z2\" (UniqueName: \"kubernetes.io/projected/9a7235f4-65e9-425a-90d3-f65f16d37848-kube-api-access-8m7z2\") pod \"perf-node-gather-daemonset-7shbm\" (UID: \"9a7235f4-65e9-425a-90d3-f65f16d37848\") " pod="openshift-must-gather-s9mdb/perf-node-gather-daemonset-7shbm" Jul 21 13:27:55.070175 ip-10-0-130-157 kubenswrapper[2576]: I0721 13:27:55.070114 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-s9mdb/perf-node-gather-daemonset-7shbm" Jul 21 13:27:55.200008 ip-10-0-130-157 kubenswrapper[2576]: I0721 13:27:55.199851 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-s9mdb/perf-node-gather-daemonset-7shbm"] Jul 21 13:27:55.202708 ip-10-0-130-157 kubenswrapper[2576]: W0721 13:27:55.202680 2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-pod9a7235f4_65e9_425a_90d3_f65f16d37848.slice/crio-2335cf91613f46d7ff3aefb88c917333943b02dd792ebcd1202e36e055b6cd6c WatchSource:0}: Error finding container 2335cf91613f46d7ff3aefb88c917333943b02dd792ebcd1202e36e055b6cd6c: Status 404 returned error can't find the container with id 2335cf91613f46d7ff3aefb88c917333943b02dd792ebcd1202e36e055b6cd6c Jul 21 13:27:55.240045 ip-10-0-130-157 kubenswrapper[2576]: I0721 13:27:55.240023 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-s9mdb/perf-node-gather-daemonset-7shbm" event={"ID":"9a7235f4-65e9-425a-90d3-f65f16d37848","Type":"ContainerStarted","Data":"2335cf91613f46d7ff3aefb88c917333943b02dd792ebcd1202e36e055b6cd6c"} Jul 21 13:27:56.246289 ip-10-0-130-157 kubenswrapper[2576]: I0721 13:27:56.246255 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-s9mdb/perf-node-gather-daemonset-7shbm" event={"ID":"9a7235f4-65e9-425a-90d3-f65f16d37848","Type":"ContainerStarted","Data":"d9b3e7bda2d7e31a2035e51a2ff9dbe6d8eb13813f9735a31fbf0653db6ad3f4"} Jul 21 13:27:56.246713 ip-10-0-130-157 kubenswrapper[2576]: I0721 13:27:56.246372 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-s9mdb/perf-node-gather-daemonset-7shbm" Jul 21 13:27:56.265866 ip-10-0-130-157 kubenswrapper[2576]: I0721 13:27:56.265820 2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-s9mdb/perf-node-gather-daemonset-7shbm" podStartSLOduration=2.265803772 podStartE2EDuration="2.265803772s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-21 13:27:54 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 13:27:56.264316484 +0000 UTC m=+2884.099580350" watchObservedRunningTime="2026-07-21 13:27:56.265803772 +0000 UTC m=+2884.101067639" Jul 21 13:27:58.959169 ip-10-0-130-157 kubenswrapper[2576]: I0721 13:27:58.959140 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-6hckk_61de9ce0-c295-45c5-b657-2f2c64048340/dns/0.log" Jul 21 13:27:59.009694 ip-10-0-130-157 kubenswrapper[2576]: I0721 13:27:59.009653 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-6hckk_61de9ce0-c295-45c5-b657-2f2c64048340/kube-rbac-proxy/0.log" Jul 21 13:27:59.247812 ip-10-0-130-157 kubenswrapper[2576]: I0721 13:27:59.247742 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-6crjb_c70252ec-a8bb-4dd0-9544-2ff28e109eb9/dns-node-resolver/0.log" Jul 21 13:27:59.348674 ip-10-0-130-157 kubenswrapper[2576]: I0721 13:27:59.348643 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-skc26_6713665c-3b71-4751-929e-16de48263766/konnectivity-agent/0.log" limit=1024 Jul 21 13:27:59.424663 ip-10-0-130-157 kubenswrapper[2576]: I0721 13:27:59.424638 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-skc26_6713665c-3b71-4751-929e-16de48263766/konnectivity-agent/0.log" limit=1024 Jul 21 13:28:00.093116 ip-10-0-130-157 kubenswrapper[2576]: I0721 13:28:00.093090 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-km4fw_9cfd0727-8ed4-40bd-b1d5-66a531560bce/node-ca/0.log" Jul 21 13:28:01.290964 ip-10-0-130-157 kubenswrapper[2576]: I0721 13:28:01.290936 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-67cb6d6b5d-424n4_c8997198-ef13-4846-90eb-47345692f1a9/kube-auth-proxy/0.log" Jul 21 13:28:02.259046 ip-10-0-130-157 kubenswrapper[2576]: I0721 13:28:02.259023 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-s9mdb/perf-node-gather-daemonset-7shbm" Jul 21 13:28:02.318809 ip-10-0-130-157 kubenswrapper[2576]: I0721 13:28:02.318783 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-bmvqm_dea4cd60-192f-49cf-aeaa-d48857637421/serve-healthcheck-canary/0.log" Jul 21 13:28:02.983391 ip-10-0-130-157 kubenswrapper[2576]: I0721 13:28:02.983367 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-ncggx_eb329e76-c1b9-4fe2-9cb4-0d85b8a10e9e/kube-rbac-proxy/0.log" Jul 21 13:28:03.010082 ip-10-0-130-157 kubenswrapper[2576]: I0721 13:28:03.010058 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-ncggx_eb329e76-c1b9-4fe2-9cb4-0d85b8a10e9e/exporter/0.log" Jul 21 13:28:03.042116 ip-10-0-130-157 kubenswrapper[2576]: I0721 13:28:03.042084 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-ncggx_eb329e76-c1b9-4fe2-9cb4-0d85b8a10e9e/extractor/0.log" Jul 21 13:28:05.499491 ip-10-0-130-157 kubenswrapper[2576]: I0721 13:28:05.499458 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-67cb6d6b5d-424n4_c8997198-ef13-4846-90eb-47345692f1a9/kube-auth-proxy/0.log" Jul 21 13:28:06.170410 ip-10-0-130-157 kubenswrapper[2576]: I0721 13:28:06.170325 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-579d89bddd-mk77w_3e7a2646-5d2a-498c-878f-f3c1f6834381/manager/0.log" Jul 21 13:28:06.202447 ip-10-0-130-157 kubenswrapper[2576]: I0721 13:28:06.202427 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-579d89bddd-mk77w_3e7a2646-5d2a-498c-878f-f3c1f6834381/copy-manifests/0.log" Jul 21 13:28:07.100529 ip-10-0-130-157 kubenswrapper[2576]: I0721 13:28:07.100486 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-67cb6d6b5d-424n4_c8997198-ef13-4846-90eb-47345692f1a9/kube-auth-proxy/0.log" Jul 21 13:28:07.573587 ip-10-0-130-157 kubenswrapper[2576]: I0721 13:28:07.573559 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-lws-operator_lws-controller-manager-84bfb648b5-4nc5w_7050b6cd-c457-48d8-9dce-cc1dfa7ecf38/manager/0.log"