Jul 29 08:19:25.952468 ip-10-0-131-138 systemd[1]: kubelet.service: Failed to load environment files: No such file or directory Jul 29 08:19:25.952479 ip-10-0-131-138 systemd[1]: kubelet.service: Failed to run 'start-pre' task: No such file or directory Jul 29 08:19:25.952489 ip-10-0-131-138 systemd[1]: kubelet.service: Failed with result 'resources'. Jul 29 08:19:25.952842 ip-10-0-131-138 systemd[1]: Failed to start Kubernetes Kubelet. Jul 29 08:19:35.639462 ip-10-0-131-138 systemd[1]: kubelet.service: Failed to schedule restart job: Unit crio.service not found. Jul 29 08:19:35.639479 ip-10-0-131-138 systemd[1]: kubelet.service: Failed with result 'resources'. -- Boot 9da740822c184ee4bc51bbb2eff87894 -- Jul 29 08:21:25.856665 ip-10-0-131-138 systemd[1]: Starting Kubernetes Kubelet... Jul 29 08:21:26.290824 ip-10-0-131-138 kubenswrapper[2563]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 29 08:21:26.290824 ip-10-0-131-138 kubenswrapper[2563]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version. Jul 29 08:21:26.290824 ip-10-0-131-138 kubenswrapper[2563]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 29 08:21:26.290824 ip-10-0-131-138 kubenswrapper[2563]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI. Jul 29 08:21:26.290824 ip-10-0-131-138 kubenswrapper[2563]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 29 08:21:26.293231 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:21:26.293139 2563 server.go:213] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime" Jul 29 08:21:26.298755 ip-10-0-131-138 kubenswrapper[2563]: W0729 08:21:26.298725 2563 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 29 08:21:26.298755 ip-10-0-131-138 kubenswrapper[2563]: W0729 08:21:26.298746 2563 feature_gate.go:328] unrecognized feature gate: Example Jul 29 08:21:26.298755 ip-10-0-131-138 kubenswrapper[2563]: W0729 08:21:26.298750 2563 feature_gate.go:328] unrecognized feature gate: Example2 Jul 29 08:21:26.298755 ip-10-0-131-138 kubenswrapper[2563]: W0729 08:21:26.298755 2563 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 29 08:21:26.298755 ip-10-0-131-138 kubenswrapper[2563]: W0729 08:21:26.298759 2563 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 29 08:21:26.298755 ip-10-0-131-138 kubenswrapper[2563]: W0729 08:21:26.298761 2563 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 29 08:21:26.298993 ip-10-0-131-138 kubenswrapper[2563]: W0729 08:21:26.298765 2563 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 29 08:21:26.298993 ip-10-0-131-138 kubenswrapper[2563]: W0729 08:21:26.298769 2563 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 29 08:21:26.298993 ip-10-0-131-138 kubenswrapper[2563]: W0729 08:21:26.298772 2563 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 29 08:21:26.298993 ip-10-0-131-138 kubenswrapper[2563]: W0729 08:21:26.298775 2563 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 29 08:21:26.298993 ip-10-0-131-138 kubenswrapper[2563]: W0729 08:21:26.298777 2563 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 29 08:21:26.298993 ip-10-0-131-138 kubenswrapper[2563]: W0729 08:21:26.298781 2563 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 29 08:21:26.298993 ip-10-0-131-138 kubenswrapper[2563]: W0729 08:21:26.298785 2563 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 29 08:21:26.298993 ip-10-0-131-138 kubenswrapper[2563]: W0729 08:21:26.298788 2563 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 29 08:21:26.298993 ip-10-0-131-138 kubenswrapper[2563]: W0729 08:21:26.298790 2563 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 29 08:21:26.298993 ip-10-0-131-138 kubenswrapper[2563]: W0729 08:21:26.298793 2563 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 29 08:21:26.298993 ip-10-0-131-138 kubenswrapper[2563]: W0729 08:21:26.298796 2563 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 29 08:21:26.298993 ip-10-0-131-138 kubenswrapper[2563]: W0729 08:21:26.298799 2563 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 29 08:21:26.298993 ip-10-0-131-138 kubenswrapper[2563]: W0729 08:21:26.298801 2563 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 29 08:21:26.298993 ip-10-0-131-138 kubenswrapper[2563]: W0729 08:21:26.298804 2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 29 08:21:26.298993 ip-10-0-131-138 kubenswrapper[2563]: W0729 08:21:26.298807 2563 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 29 08:21:26.298993 ip-10-0-131-138 kubenswrapper[2563]: W0729 08:21:26.298809 2563 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 29 08:21:26.298993 ip-10-0-131-138 kubenswrapper[2563]: W0729 08:21:26.298812 2563 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 29 08:21:26.298993 ip-10-0-131-138 kubenswrapper[2563]: W0729 08:21:26.298814 2563 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 29 08:21:26.298993 ip-10-0-131-138 kubenswrapper[2563]: W0729 08:21:26.298816 2563 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 29 08:21:26.298993 ip-10-0-131-138 kubenswrapper[2563]: W0729 08:21:26.298819 2563 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 29 08:21:26.299486 ip-10-0-131-138 kubenswrapper[2563]: W0729 08:21:26.298822 2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 29 08:21:26.299486 ip-10-0-131-138 kubenswrapper[2563]: W0729 08:21:26.298825 2563 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 29 08:21:26.299486 ip-10-0-131-138 kubenswrapper[2563]: W0729 08:21:26.298828 2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 29 08:21:26.299486 ip-10-0-131-138 kubenswrapper[2563]: W0729 08:21:26.298830 2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 29 08:21:26.299486 ip-10-0-131-138 kubenswrapper[2563]: W0729 08:21:26.298832 2563 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 29 08:21:26.299486 ip-10-0-131-138 kubenswrapper[2563]: W0729 08:21:26.298835 2563 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 29 08:21:26.299486 ip-10-0-131-138 kubenswrapper[2563]: W0729 08:21:26.298844 2563 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 29 08:21:26.299486 ip-10-0-131-138 kubenswrapper[2563]: W0729 08:21:26.298847 2563 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 29 08:21:26.299486 ip-10-0-131-138 kubenswrapper[2563]: W0729 08:21:26.298849 2563 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 29 08:21:26.299486 ip-10-0-131-138 kubenswrapper[2563]: W0729 08:21:26.298852 2563 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 29 08:21:26.299486 ip-10-0-131-138 kubenswrapper[2563]: W0729 08:21:26.298855 2563 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 29 08:21:26.299486 ip-10-0-131-138 kubenswrapper[2563]: W0729 08:21:26.298857 2563 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 29 08:21:26.299486 ip-10-0-131-138 kubenswrapper[2563]: W0729 08:21:26.298860 2563 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 29 08:21:26.299486 ip-10-0-131-138 kubenswrapper[2563]: W0729 08:21:26.298863 2563 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 29 08:21:26.299486 ip-10-0-131-138 kubenswrapper[2563]: W0729 08:21:26.298865 2563 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 29 08:21:26.299486 ip-10-0-131-138 kubenswrapper[2563]: W0729 08:21:26.298868 2563 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 29 08:21:26.299486 ip-10-0-131-138 kubenswrapper[2563]: W0729 08:21:26.298870 2563 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 29 08:21:26.299486 ip-10-0-131-138 kubenswrapper[2563]: W0729 08:21:26.298873 2563 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 29 08:21:26.299486 ip-10-0-131-138 kubenswrapper[2563]: W0729 08:21:26.298876 2563 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 29 08:21:26.299486 ip-10-0-131-138 kubenswrapper[2563]: W0729 08:21:26.298878 2563 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 29 08:21:26.299970 ip-10-0-131-138 kubenswrapper[2563]: W0729 08:21:26.298880 2563 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 29 08:21:26.299970 ip-10-0-131-138 kubenswrapper[2563]: W0729 08:21:26.298883 2563 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 29 08:21:26.299970 ip-10-0-131-138 kubenswrapper[2563]: W0729 08:21:26.298885 2563 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 29 08:21:26.299970 ip-10-0-131-138 kubenswrapper[2563]: W0729 08:21:26.298888 2563 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 29 08:21:26.299970 ip-10-0-131-138 kubenswrapper[2563]: W0729 08:21:26.298890 2563 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 29 08:21:26.299970 ip-10-0-131-138 kubenswrapper[2563]: W0729 08:21:26.298893 2563 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 29 08:21:26.299970 ip-10-0-131-138 kubenswrapper[2563]: W0729 08:21:26.298895 2563 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 29 08:21:26.299970 ip-10-0-131-138 kubenswrapper[2563]: W0729 08:21:26.298898 2563 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 29 08:21:26.299970 ip-10-0-131-138 kubenswrapper[2563]: W0729 08:21:26.298900 2563 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 29 08:21:26.299970 ip-10-0-131-138 kubenswrapper[2563]: W0729 08:21:26.298903 2563 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 29 08:21:26.299970 ip-10-0-131-138 kubenswrapper[2563]: W0729 08:21:26.298905 2563 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 29 08:21:26.299970 ip-10-0-131-138 kubenswrapper[2563]: W0729 08:21:26.298908 2563 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 29 08:21:26.299970 ip-10-0-131-138 kubenswrapper[2563]: W0729 08:21:26.298910 2563 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 29 08:21:26.299970 ip-10-0-131-138 kubenswrapper[2563]: W0729 08:21:26.298913 2563 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 29 08:21:26.299970 ip-10-0-131-138 kubenswrapper[2563]: W0729 08:21:26.298915 2563 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 29 08:21:26.299970 ip-10-0-131-138 kubenswrapper[2563]: W0729 08:21:26.298918 2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 29 08:21:26.299970 ip-10-0-131-138 kubenswrapper[2563]: W0729 08:21:26.298920 2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 29 08:21:26.299970 ip-10-0-131-138 kubenswrapper[2563]: W0729 08:21:26.298922 2563 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 29 08:21:26.299970 ip-10-0-131-138 kubenswrapper[2563]: W0729 08:21:26.298925 2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 29 08:21:26.299970 ip-10-0-131-138 kubenswrapper[2563]: W0729 08:21:26.298927 2563 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 29 08:21:26.300506 ip-10-0-131-138 kubenswrapper[2563]: W0729 08:21:26.298930 2563 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 29 08:21:26.300506 ip-10-0-131-138 kubenswrapper[2563]: W0729 08:21:26.298932 2563 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 29 08:21:26.300506 ip-10-0-131-138 kubenswrapper[2563]: W0729 08:21:26.298935 2563 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 29 08:21:26.300506 ip-10-0-131-138 kubenswrapper[2563]: W0729 08:21:26.298937 2563 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 29 08:21:26.300506 ip-10-0-131-138 kubenswrapper[2563]: W0729 08:21:26.298940 2563 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 29 08:21:26.300506 ip-10-0-131-138 kubenswrapper[2563]: W0729 08:21:26.298944 2563 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 29 08:21:26.300506 ip-10-0-131-138 kubenswrapper[2563]: W0729 08:21:26.298947 2563 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 29 08:21:26.300506 ip-10-0-131-138 kubenswrapper[2563]: W0729 08:21:26.298949 2563 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 29 08:21:26.300506 ip-10-0-131-138 kubenswrapper[2563]: W0729 08:21:26.298951 2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 29 08:21:26.300506 ip-10-0-131-138 kubenswrapper[2563]: W0729 08:21:26.298956 2563 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 29 08:21:26.300506 ip-10-0-131-138 kubenswrapper[2563]: W0729 08:21:26.298960 2563 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 29 08:21:26.300506 ip-10-0-131-138 kubenswrapper[2563]: W0729 08:21:26.298962 2563 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 29 08:21:26.300506 ip-10-0-131-138 kubenswrapper[2563]: W0729 08:21:26.298965 2563 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 29 08:21:26.300506 ip-10-0-131-138 kubenswrapper[2563]: W0729 08:21:26.298967 2563 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 29 08:21:26.300506 ip-10-0-131-138 kubenswrapper[2563]: W0729 08:21:26.298970 2563 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 29 08:21:26.300506 ip-10-0-131-138 kubenswrapper[2563]: W0729 08:21:26.298972 2563 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 29 08:21:26.300506 ip-10-0-131-138 kubenswrapper[2563]: W0729 08:21:26.298974 2563 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 29 08:21:26.300506 ip-10-0-131-138 kubenswrapper[2563]: W0729 08:21:26.298977 2563 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 29 08:21:26.300506 ip-10-0-131-138 kubenswrapper[2563]: W0729 08:21:26.298980 2563 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 29 08:21:26.300506 ip-10-0-131-138 kubenswrapper[2563]: W0729 08:21:26.298983 2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 29 08:21:26.300998 ip-10-0-131-138 kubenswrapper[2563]: W0729 08:21:26.298985 2563 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 29 08:21:26.300998 ip-10-0-131-138 kubenswrapper[2563]: W0729 08:21:26.298988 2563 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 29 08:21:26.300998 ip-10-0-131-138 kubenswrapper[2563]: W0729 08:21:26.298991 2563 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 29 08:21:26.300998 ip-10-0-131-138 kubenswrapper[2563]: W0729 08:21:26.298993 2563 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 29 08:21:26.300998 ip-10-0-131-138 kubenswrapper[2563]: W0729 08:21:26.298996 2563 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 29 08:21:26.300998 ip-10-0-131-138 kubenswrapper[2563]: W0729 08:21:26.298998 2563 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 29 08:21:26.300998 ip-10-0-131-138 kubenswrapper[2563]: W0729 08:21:26.299001 2563 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 29 08:21:26.300998 ip-10-0-131-138 kubenswrapper[2563]: W0729 08:21:26.299004 2563 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 29 08:21:26.300998 ip-10-0-131-138 kubenswrapper[2563]: W0729 08:21:26.299007 2563 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 29 08:21:26.300998 ip-10-0-131-138 kubenswrapper[2563]: W0729 08:21:26.299009 2563 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 29 08:21:26.300998 ip-10-0-131-138 kubenswrapper[2563]: W0729 08:21:26.299012 2563 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 29 08:21:26.300998 ip-10-0-131-138 kubenswrapper[2563]: W0729 08:21:26.299015 2563 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 29 08:21:26.300998 ip-10-0-131-138 kubenswrapper[2563]: W0729 08:21:26.299017 2563 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 29 08:21:26.300998 ip-10-0-131-138 kubenswrapper[2563]: W0729 08:21:26.299764 2563 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 29 08:21:26.300998 ip-10-0-131-138 kubenswrapper[2563]: W0729 08:21:26.299771 2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 29 08:21:26.300998 ip-10-0-131-138 kubenswrapper[2563]: W0729 08:21:26.299775 2563 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 29 08:21:26.300998 ip-10-0-131-138 kubenswrapper[2563]: W0729 08:21:26.299779 2563 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 29 08:21:26.300998 ip-10-0-131-138 kubenswrapper[2563]: W0729 08:21:26.299782 2563 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 29 08:21:26.300998 ip-10-0-131-138 kubenswrapper[2563]: W0729 08:21:26.299784 2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 29 08:21:26.301485 ip-10-0-131-138 kubenswrapper[2563]: W0729 08:21:26.299787 2563 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 29 08:21:26.301485 ip-10-0-131-138 kubenswrapper[2563]: W0729 08:21:26.299789 2563 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 29 08:21:26.301485 ip-10-0-131-138 kubenswrapper[2563]: W0729 08:21:26.299792 2563 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 29 08:21:26.301485 ip-10-0-131-138 kubenswrapper[2563]: W0729 08:21:26.299794 2563 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 29 08:21:26.301485 ip-10-0-131-138 kubenswrapper[2563]: W0729 08:21:26.299799 2563 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 29 08:21:26.301485 ip-10-0-131-138 kubenswrapper[2563]: W0729 08:21:26.299803 2563 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 29 08:21:26.301485 ip-10-0-131-138 kubenswrapper[2563]: W0729 08:21:26.299806 2563 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 29 08:21:26.301485 ip-10-0-131-138 kubenswrapper[2563]: W0729 08:21:26.299809 2563 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 29 08:21:26.301485 ip-10-0-131-138 kubenswrapper[2563]: W0729 08:21:26.299812 2563 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 29 08:21:26.301485 ip-10-0-131-138 kubenswrapper[2563]: W0729 08:21:26.299815 2563 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 29 08:21:26.301485 ip-10-0-131-138 kubenswrapper[2563]: W0729 08:21:26.299817 2563 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 29 08:21:26.301485 ip-10-0-131-138 kubenswrapper[2563]: W0729 08:21:26.299821 2563 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 29 08:21:26.301485 ip-10-0-131-138 kubenswrapper[2563]: W0729 08:21:26.299824 2563 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 29 08:21:26.301485 ip-10-0-131-138 kubenswrapper[2563]: W0729 08:21:26.299827 2563 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 29 08:21:26.301485 ip-10-0-131-138 kubenswrapper[2563]: W0729 08:21:26.299829 2563 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 29 08:21:26.301485 ip-10-0-131-138 kubenswrapper[2563]: W0729 08:21:26.299832 2563 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 29 08:21:26.301485 ip-10-0-131-138 kubenswrapper[2563]: W0729 08:21:26.299835 2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 29 08:21:26.301485 ip-10-0-131-138 kubenswrapper[2563]: W0729 08:21:26.299838 2563 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 29 08:21:26.301485 ip-10-0-131-138 kubenswrapper[2563]: W0729 08:21:26.299842 2563 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 29 08:21:26.301958 ip-10-0-131-138 kubenswrapper[2563]: W0729 08:21:26.299845 2563 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 29 08:21:26.301958 ip-10-0-131-138 kubenswrapper[2563]: W0729 08:21:26.299847 2563 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 29 08:21:26.301958 ip-10-0-131-138 kubenswrapper[2563]: W0729 08:21:26.299850 2563 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 29 08:21:26.301958 ip-10-0-131-138 kubenswrapper[2563]: W0729 08:21:26.299852 2563 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 29 08:21:26.301958 ip-10-0-131-138 kubenswrapper[2563]: W0729 08:21:26.299855 2563 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 29 08:21:26.301958 ip-10-0-131-138 kubenswrapper[2563]: W0729 08:21:26.299857 2563 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 29 08:21:26.301958 ip-10-0-131-138 kubenswrapper[2563]: W0729 08:21:26.299860 2563 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 29 08:21:26.301958 ip-10-0-131-138 kubenswrapper[2563]: W0729 08:21:26.299863 2563 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 29 08:21:26.301958 ip-10-0-131-138 kubenswrapper[2563]: W0729 08:21:26.299866 2563 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 29 08:21:26.301958 ip-10-0-131-138 kubenswrapper[2563]: W0729 08:21:26.299869 2563 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 29 08:21:26.301958 ip-10-0-131-138 kubenswrapper[2563]: W0729 08:21:26.299872 2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 29 08:21:26.301958 ip-10-0-131-138 kubenswrapper[2563]: W0729 08:21:26.299874 2563 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 29 08:21:26.301958 ip-10-0-131-138 kubenswrapper[2563]: W0729 08:21:26.299877 2563 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 29 08:21:26.301958 ip-10-0-131-138 kubenswrapper[2563]: W0729 08:21:26.299879 2563 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 29 08:21:26.301958 ip-10-0-131-138 kubenswrapper[2563]: W0729 08:21:26.299882 2563 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 29 08:21:26.301958 ip-10-0-131-138 kubenswrapper[2563]: W0729 08:21:26.299885 2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 29 08:21:26.301958 ip-10-0-131-138 kubenswrapper[2563]: W0729 08:21:26.299887 2563 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 29 08:21:26.301958 ip-10-0-131-138 kubenswrapper[2563]: W0729 08:21:26.299889 2563 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 29 08:21:26.301958 ip-10-0-131-138 kubenswrapper[2563]: W0729 08:21:26.299892 2563 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 29 08:21:26.301958 ip-10-0-131-138 kubenswrapper[2563]: W0729 08:21:26.299894 2563 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 29 08:21:26.302453 ip-10-0-131-138 kubenswrapper[2563]: W0729 08:21:26.299896 2563 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 29 08:21:26.302453 ip-10-0-131-138 kubenswrapper[2563]: W0729 08:21:26.299900 2563 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 29 08:21:26.302453 ip-10-0-131-138 kubenswrapper[2563]: W0729 08:21:26.299903 2563 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 29 08:21:26.302453 ip-10-0-131-138 kubenswrapper[2563]: W0729 08:21:26.299906 2563 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 29 08:21:26.302453 ip-10-0-131-138 kubenswrapper[2563]: W0729 08:21:26.299908 2563 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 29 08:21:26.302453 ip-10-0-131-138 kubenswrapper[2563]: W0729 08:21:26.299912 2563 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 29 08:21:26.302453 ip-10-0-131-138 kubenswrapper[2563]: W0729 08:21:26.299915 2563 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 29 08:21:26.302453 ip-10-0-131-138 kubenswrapper[2563]: W0729 08:21:26.299917 2563 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 29 08:21:26.302453 ip-10-0-131-138 kubenswrapper[2563]: W0729 08:21:26.299920 2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 29 08:21:26.302453 ip-10-0-131-138 kubenswrapper[2563]: W0729 08:21:26.299922 2563 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 29 08:21:26.302453 ip-10-0-131-138 kubenswrapper[2563]: W0729 08:21:26.299924 2563 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 29 08:21:26.302453 ip-10-0-131-138 kubenswrapper[2563]: W0729 08:21:26.299927 2563 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 29 08:21:26.302453 ip-10-0-131-138 kubenswrapper[2563]: W0729 08:21:26.299929 2563 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 29 08:21:26.302453 ip-10-0-131-138 kubenswrapper[2563]: W0729 08:21:26.299931 2563 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 29 08:21:26.302453 ip-10-0-131-138 kubenswrapper[2563]: W0729 08:21:26.299935 2563 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 29 08:21:26.302453 ip-10-0-131-138 kubenswrapper[2563]: W0729 08:21:26.299939 2563 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 29 08:21:26.302453 ip-10-0-131-138 kubenswrapper[2563]: W0729 08:21:26.299942 2563 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 29 08:21:26.302453 ip-10-0-131-138 kubenswrapper[2563]: W0729 08:21:26.299944 2563 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 29 08:21:26.302453 ip-10-0-131-138 kubenswrapper[2563]: W0729 08:21:26.299947 2563 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 29 08:21:26.302924 ip-10-0-131-138 kubenswrapper[2563]: W0729 08:21:26.299949 2563 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 29 08:21:26.302924 ip-10-0-131-138 kubenswrapper[2563]: W0729 08:21:26.299953 2563 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 29 08:21:26.302924 ip-10-0-131-138 kubenswrapper[2563]: W0729 08:21:26.299956 2563 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 29 08:21:26.302924 ip-10-0-131-138 kubenswrapper[2563]: W0729 08:21:26.299958 2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 29 08:21:26.302924 ip-10-0-131-138 kubenswrapper[2563]: W0729 08:21:26.299961 2563 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 29 08:21:26.302924 ip-10-0-131-138 kubenswrapper[2563]: W0729 08:21:26.299963 2563 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 29 08:21:26.302924 ip-10-0-131-138 kubenswrapper[2563]: W0729 08:21:26.299965 2563 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 29 08:21:26.302924 ip-10-0-131-138 kubenswrapper[2563]: W0729 08:21:26.299968 2563 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 29 08:21:26.302924 ip-10-0-131-138 kubenswrapper[2563]: W0729 08:21:26.299970 2563 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 29 08:21:26.302924 ip-10-0-131-138 kubenswrapper[2563]: W0729 08:21:26.299973 2563 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 29 08:21:26.302924 ip-10-0-131-138 kubenswrapper[2563]: W0729 08:21:26.299975 2563 feature_gate.go:328] unrecognized feature gate: Example Jul 29 08:21:26.302924 ip-10-0-131-138 kubenswrapper[2563]: W0729 08:21:26.299978 2563 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 29 08:21:26.302924 ip-10-0-131-138 kubenswrapper[2563]: W0729 08:21:26.299981 2563 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 29 08:21:26.302924 ip-10-0-131-138 kubenswrapper[2563]: W0729 08:21:26.299983 2563 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 29 08:21:26.302924 ip-10-0-131-138 kubenswrapper[2563]: W0729 08:21:26.299986 2563 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 29 08:21:26.302924 ip-10-0-131-138 kubenswrapper[2563]: W0729 08:21:26.299988 2563 feature_gate.go:328] unrecognized feature gate: Example2 Jul 29 08:21:26.302924 ip-10-0-131-138 kubenswrapper[2563]: W0729 08:21:26.299990 2563 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 29 08:21:26.302924 ip-10-0-131-138 kubenswrapper[2563]: W0729 08:21:26.299993 2563 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 29 08:21:26.302924 ip-10-0-131-138 kubenswrapper[2563]: W0729 08:21:26.299995 2563 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 29 08:21:26.302924 ip-10-0-131-138 kubenswrapper[2563]: W0729 08:21:26.299998 2563 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 29 08:21:26.303438 ip-10-0-131-138 kubenswrapper[2563]: W0729 08:21:26.300001 2563 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 29 08:21:26.303438 ip-10-0-131-138 kubenswrapper[2563]: W0729 08:21:26.300003 2563 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 29 08:21:26.303438 ip-10-0-131-138 kubenswrapper[2563]: W0729 08:21:26.300006 2563 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 29 08:21:26.303438 ip-10-0-131-138 kubenswrapper[2563]: W0729 08:21:26.300008 2563 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 29 08:21:26.303438 ip-10-0-131-138 kubenswrapper[2563]: W0729 08:21:26.300011 2563 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 29 08:21:26.303438 ip-10-0-131-138 kubenswrapper[2563]: W0729 08:21:26.300014 2563 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 29 08:21:26.303438 ip-10-0-131-138 kubenswrapper[2563]: W0729 08:21:26.300016 2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 29 08:21:26.303438 ip-10-0-131-138 kubenswrapper[2563]: W0729 08:21:26.300019 2563 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 29 08:21:26.303438 ip-10-0-131-138 kubenswrapper[2563]: W0729 08:21:26.300021 2563 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 29 08:21:26.303438 ip-10-0-131-138 kubenswrapper[2563]: W0729 08:21:26.300024 2563 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 29 08:21:26.303438 ip-10-0-131-138 kubenswrapper[2563]: W0729 08:21:26.300026 2563 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 29 08:21:26.303438 ip-10-0-131-138 kubenswrapper[2563]: W0729 08:21:26.300029 2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 29 08:21:26.303438 ip-10-0-131-138 kubenswrapper[2563]: W0729 08:21:26.300031 2563 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 29 08:21:26.303438 ip-10-0-131-138 kubenswrapper[2563]: W0729 08:21:26.300034 2563 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 29 08:21:26.303438 ip-10-0-131-138 kubenswrapper[2563]: W0729 08:21:26.300036 2563 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 29 08:21:26.303438 ip-10-0-131-138 kubenswrapper[2563]: W0729 08:21:26.300419 2563 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 29 08:21:26.303438 ip-10-0-131-138 kubenswrapper[2563]: W0729 08:21:26.300424 2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 29 08:21:26.303438 ip-10-0-131-138 kubenswrapper[2563]: W0729 08:21:26.300427 2563 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 29 08:21:26.303438 ip-10-0-131-138 kubenswrapper[2563]: W0729 08:21:26.300431 2563 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 29 08:21:26.303438 ip-10-0-131-138 kubenswrapper[2563]: W0729 08:21:26.300434 2563 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 29 08:21:26.303923 ip-10-0-131-138 kubenswrapper[2563]: W0729 08:21:26.300436 2563 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 29 08:21:26.303923 ip-10-0-131-138 kubenswrapper[2563]: W0729 08:21:26.300439 2563 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 29 08:21:26.303923 ip-10-0-131-138 kubenswrapper[2563]: W0729 08:21:26.300441 2563 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 29 08:21:26.303923 ip-10-0-131-138 kubenswrapper[2563]: W0729 08:21:26.300444 2563 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 29 08:21:26.303923 ip-10-0-131-138 kubenswrapper[2563]: W0729 08:21:26.300447 2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 29 08:21:26.303923 ip-10-0-131-138 kubenswrapper[2563]: W0729 08:21:26.300449 2563 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 29 08:21:26.303923 ip-10-0-131-138 kubenswrapper[2563]: W0729 08:21:26.300452 2563 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 29 08:21:26.303923 ip-10-0-131-138 kubenswrapper[2563]: W0729 08:21:26.300454 2563 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 29 08:21:26.303923 ip-10-0-131-138 kubenswrapper[2563]: W0729 08:21:26.300457 2563 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 29 08:21:26.303923 ip-10-0-131-138 kubenswrapper[2563]: W0729 08:21:26.300459 2563 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 29 08:21:26.303923 ip-10-0-131-138 kubenswrapper[2563]: W0729 08:21:26.300462 2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 29 08:21:26.303923 ip-10-0-131-138 kubenswrapper[2563]: W0729 08:21:26.300465 2563 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 29 08:21:26.303923 ip-10-0-131-138 kubenswrapper[2563]: W0729 08:21:26.300467 2563 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 29 08:21:26.303923 ip-10-0-131-138 kubenswrapper[2563]: W0729 08:21:26.300470 2563 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 29 08:21:26.303923 ip-10-0-131-138 kubenswrapper[2563]: W0729 08:21:26.300472 2563 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 29 08:21:26.303923 ip-10-0-131-138 kubenswrapper[2563]: W0729 08:21:26.300475 2563 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 29 08:21:26.303923 ip-10-0-131-138 kubenswrapper[2563]: W0729 08:21:26.300477 2563 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 29 08:21:26.303923 ip-10-0-131-138 kubenswrapper[2563]: W0729 08:21:26.300481 2563 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 29 08:21:26.303923 ip-10-0-131-138 kubenswrapper[2563]: W0729 08:21:26.300483 2563 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 29 08:21:26.303923 ip-10-0-131-138 kubenswrapper[2563]: W0729 08:21:26.300486 2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 29 08:21:26.304408 ip-10-0-131-138 kubenswrapper[2563]: W0729 08:21:26.300488 2563 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 29 08:21:26.304408 ip-10-0-131-138 kubenswrapper[2563]: W0729 08:21:26.300491 2563 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 29 08:21:26.304408 ip-10-0-131-138 kubenswrapper[2563]: W0729 08:21:26.300493 2563 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 29 08:21:26.304408 ip-10-0-131-138 kubenswrapper[2563]: W0729 08:21:26.300495 2563 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 29 08:21:26.304408 ip-10-0-131-138 kubenswrapper[2563]: W0729 08:21:26.300498 2563 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 29 08:21:26.304408 ip-10-0-131-138 kubenswrapper[2563]: W0729 08:21:26.300500 2563 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 29 08:21:26.304408 ip-10-0-131-138 kubenswrapper[2563]: W0729 08:21:26.300503 2563 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 29 08:21:26.304408 ip-10-0-131-138 kubenswrapper[2563]: W0729 08:21:26.300505 2563 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 29 08:21:26.304408 ip-10-0-131-138 kubenswrapper[2563]: W0729 08:21:26.300508 2563 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 29 08:21:26.304408 ip-10-0-131-138 kubenswrapper[2563]: W0729 08:21:26.300511 2563 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 29 08:21:26.304408 ip-10-0-131-138 kubenswrapper[2563]: W0729 08:21:26.300515 2563 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 29 08:21:26.304408 ip-10-0-131-138 kubenswrapper[2563]: W0729 08:21:26.300518 2563 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 29 08:21:26.304408 ip-10-0-131-138 kubenswrapper[2563]: W0729 08:21:26.300520 2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 29 08:21:26.304408 ip-10-0-131-138 kubenswrapper[2563]: W0729 08:21:26.300523 2563 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 29 08:21:26.304408 ip-10-0-131-138 kubenswrapper[2563]: W0729 08:21:26.300532 2563 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 29 08:21:26.304408 ip-10-0-131-138 kubenswrapper[2563]: W0729 08:21:26.300535 2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 29 08:21:26.304408 ip-10-0-131-138 kubenswrapper[2563]: W0729 08:21:26.300537 2563 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 29 08:21:26.304408 ip-10-0-131-138 kubenswrapper[2563]: W0729 08:21:26.300540 2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 29 08:21:26.304408 ip-10-0-131-138 kubenswrapper[2563]: W0729 08:21:26.300542 2563 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 29 08:21:26.304911 ip-10-0-131-138 kubenswrapper[2563]: W0729 08:21:26.300545 2563 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 29 08:21:26.304911 ip-10-0-131-138 kubenswrapper[2563]: W0729 08:21:26.300548 2563 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 29 08:21:26.304911 ip-10-0-131-138 kubenswrapper[2563]: W0729 08:21:26.300550 2563 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 29 08:21:26.304911 ip-10-0-131-138 kubenswrapper[2563]: W0729 08:21:26.300553 2563 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 29 08:21:26.304911 ip-10-0-131-138 kubenswrapper[2563]: W0729 08:21:26.300555 2563 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 29 08:21:26.304911 ip-10-0-131-138 kubenswrapper[2563]: W0729 08:21:26.300562 2563 feature_gate.go:328] unrecognized feature gate: Example Jul 29 08:21:26.304911 ip-10-0-131-138 kubenswrapper[2563]: W0729 08:21:26.300565 2563 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 29 08:21:26.304911 ip-10-0-131-138 kubenswrapper[2563]: W0729 08:21:26.300568 2563 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 29 08:21:26.304911 ip-10-0-131-138 kubenswrapper[2563]: W0729 08:21:26.300570 2563 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 29 08:21:26.304911 ip-10-0-131-138 kubenswrapper[2563]: W0729 08:21:26.300573 2563 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 29 08:21:26.304911 ip-10-0-131-138 kubenswrapper[2563]: W0729 08:21:26.300576 2563 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 29 08:21:26.304911 ip-10-0-131-138 kubenswrapper[2563]: W0729 08:21:26.300578 2563 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 29 08:21:26.304911 ip-10-0-131-138 kubenswrapper[2563]: W0729 08:21:26.300581 2563 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 29 08:21:26.304911 ip-10-0-131-138 kubenswrapper[2563]: W0729 08:21:26.300583 2563 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 29 08:21:26.304911 ip-10-0-131-138 kubenswrapper[2563]: W0729 08:21:26.300586 2563 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 29 08:21:26.304911 ip-10-0-131-138 kubenswrapper[2563]: W0729 08:21:26.300588 2563 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 29 08:21:26.304911 ip-10-0-131-138 kubenswrapper[2563]: W0729 08:21:26.300591 2563 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 29 08:21:26.304911 ip-10-0-131-138 kubenswrapper[2563]: W0729 08:21:26.300593 2563 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 29 08:21:26.304911 ip-10-0-131-138 kubenswrapper[2563]: W0729 08:21:26.300597 2563 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 29 08:21:26.305433 ip-10-0-131-138 kubenswrapper[2563]: W0729 08:21:26.300601 2563 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 29 08:21:26.305433 ip-10-0-131-138 kubenswrapper[2563]: W0729 08:21:26.300603 2563 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 29 08:21:26.305433 ip-10-0-131-138 kubenswrapper[2563]: W0729 08:21:26.300606 2563 feature_gate.go:328] unrecognized feature gate: Example2 Jul 29 08:21:26.305433 ip-10-0-131-138 kubenswrapper[2563]: W0729 08:21:26.300609 2563 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 29 08:21:26.305433 ip-10-0-131-138 kubenswrapper[2563]: W0729 08:21:26.300612 2563 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 29 08:21:26.305433 ip-10-0-131-138 kubenswrapper[2563]: W0729 08:21:26.300614 2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 29 08:21:26.305433 ip-10-0-131-138 kubenswrapper[2563]: W0729 08:21:26.300616 2563 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 29 08:21:26.305433 ip-10-0-131-138 kubenswrapper[2563]: W0729 08:21:26.300619 2563 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 29 08:21:26.305433 ip-10-0-131-138 kubenswrapper[2563]: W0729 08:21:26.300621 2563 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 29 08:21:26.305433 ip-10-0-131-138 kubenswrapper[2563]: W0729 08:21:26.300624 2563 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 29 08:21:26.305433 ip-10-0-131-138 kubenswrapper[2563]: W0729 08:21:26.300627 2563 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 29 08:21:26.305433 ip-10-0-131-138 kubenswrapper[2563]: W0729 08:21:26.300629 2563 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 29 08:21:26.305433 ip-10-0-131-138 kubenswrapper[2563]: W0729 08:21:26.300632 2563 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 29 08:21:26.305433 ip-10-0-131-138 kubenswrapper[2563]: W0729 08:21:26.300634 2563 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 29 08:21:26.305433 ip-10-0-131-138 kubenswrapper[2563]: W0729 08:21:26.300636 2563 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 29 08:21:26.305433 ip-10-0-131-138 kubenswrapper[2563]: W0729 08:21:26.300639 2563 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 29 08:21:26.305433 ip-10-0-131-138 kubenswrapper[2563]: W0729 08:21:26.300641 2563 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 29 08:21:26.305433 ip-10-0-131-138 kubenswrapper[2563]: W0729 08:21:26.300643 2563 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 29 08:21:26.305433 ip-10-0-131-138 kubenswrapper[2563]: W0729 08:21:26.300646 2563 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 29 08:21:26.305433 ip-10-0-131-138 kubenswrapper[2563]: W0729 08:21:26.300649 2563 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 29 08:21:26.305907 ip-10-0-131-138 kubenswrapper[2563]: W0729 08:21:26.300652 2563 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 29 08:21:26.305907 ip-10-0-131-138 kubenswrapper[2563]: W0729 08:21:26.300656 2563 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 29 08:21:26.305907 ip-10-0-131-138 kubenswrapper[2563]: W0729 08:21:26.300658 2563 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 29 08:21:26.305907 ip-10-0-131-138 kubenswrapper[2563]: W0729 08:21:26.300661 2563 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 29 08:21:26.305907 ip-10-0-131-138 kubenswrapper[2563]: W0729 08:21:26.300663 2563 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 29 08:21:26.305907 ip-10-0-131-138 kubenswrapper[2563]: W0729 08:21:26.300666 2563 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 29 08:21:26.305907 ip-10-0-131-138 kubenswrapper[2563]: W0729 08:21:26.300669 2563 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 29 08:21:26.305907 ip-10-0-131-138 kubenswrapper[2563]: W0729 08:21:26.300672 2563 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 29 08:21:26.305907 ip-10-0-131-138 kubenswrapper[2563]: W0729 08:21:26.300675 2563 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 29 08:21:26.305907 ip-10-0-131-138 kubenswrapper[2563]: W0729 08:21:26.300678 2563 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 29 08:21:26.305907 ip-10-0-131-138 kubenswrapper[2563]: W0729 08:21:26.300681 2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 29 08:21:26.305907 ip-10-0-131-138 kubenswrapper[2563]: W0729 08:21:26.300683 2563 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 29 08:21:26.305907 ip-10-0-131-138 kubenswrapper[2563]: W0729 08:21:26.300686 2563 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 29 08:21:26.305907 ip-10-0-131-138 kubenswrapper[2563]: W0729 08:21:26.300688 2563 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 29 08:21:26.305907 ip-10-0-131-138 kubenswrapper[2563]: W0729 08:21:26.300691 2563 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 29 08:21:26.305907 ip-10-0-131-138 kubenswrapper[2563]: W0729 08:21:26.300693 2563 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 29 08:21:26.305907 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:21:26.300759 2563 flags.go:64] FLAG: --address="0.0.0.0" Jul 29 08:21:26.305907 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:21:26.300765 2563 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]" Jul 29 08:21:26.305907 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:21:26.300771 2563 flags.go:64] FLAG: --anonymous-auth="true" Jul 29 08:21:26.305907 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:21:26.300776 2563 flags.go:64] FLAG: --application-metrics-count-limit="100" Jul 29 08:21:26.305907 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:21:26.300780 2563 flags.go:64] FLAG: --authentication-token-webhook="false" Jul 29 08:21:26.306426 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:21:26.300783 2563 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s" Jul 29 08:21:26.306426 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:21:26.300787 2563 flags.go:64] FLAG: --authorization-mode="AlwaysAllow" Jul 29 08:21:26.306426 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:21:26.300791 2563 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s" Jul 29 08:21:26.306426 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:21:26.300793 2563 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s" Jul 29 08:21:26.306426 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:21:26.300796 2563 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id" Jul 29 08:21:26.306426 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:21:26.300800 2563 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig" Jul 29 08:21:26.306426 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:21:26.300803 2563 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki" Jul 29 08:21:26.306426 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:21:26.300806 2563 flags.go:64] FLAG: --cgroup-driver="cgroupfs" Jul 29 08:21:26.306426 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:21:26.300809 2563 flags.go:64] FLAG: --cgroup-root="" Jul 29 08:21:26.306426 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:21:26.300811 2563 flags.go:64] FLAG: --cgroups-per-qos="true" Jul 29 08:21:26.306426 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:21:26.300814 2563 flags.go:64] FLAG: --client-ca-file="" Jul 29 08:21:26.306426 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:21:26.300818 2563 flags.go:64] FLAG: --cloud-provider="external" Jul 29 08:21:26.306426 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:21:26.300821 2563 flags.go:64] FLAG: --cluster-dns="[]" Jul 29 08:21:26.306426 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:21:26.300825 2563 flags.go:64] FLAG: --cluster-domain="" Jul 29 08:21:26.306426 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:21:26.300828 2563 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf" Jul 29 08:21:26.306426 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:21:26.300831 2563 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d" Jul 29 08:21:26.306426 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:21:26.300834 2563 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json" Jul 29 08:21:26.306426 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:21:26.300837 2563 flags.go:64] FLAG: --container-log-max-files="5" Jul 29 08:21:26.306426 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:21:26.300845 2563 flags.go:64] FLAG: --container-log-max-size="10Mi" Jul 29 08:21:26.306426 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:21:26.300848 2563 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock" Jul 29 08:21:26.306426 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:21:26.300850 2563 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock" Jul 29 08:21:26.306426 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:21:26.300853 2563 flags.go:64] FLAG: --containerd-namespace="k8s.io" Jul 29 08:21:26.306426 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:21:26.300856 2563 flags.go:64] FLAG: --contention-profiling="false" Jul 29 08:21:26.306426 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:21:26.300859 2563 flags.go:64] FLAG: --cpu-cfs-quota="true" Jul 29 08:21:26.306998 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:21:26.300862 2563 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms" Jul 29 08:21:26.306998 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:21:26.300865 2563 flags.go:64] FLAG: --cpu-manager-policy="none" Jul 29 08:21:26.306998 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:21:26.300868 2563 flags.go:64] FLAG: --cpu-manager-policy-options="" Jul 29 08:21:26.306998 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:21:26.300872 2563 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s" Jul 29 08:21:26.306998 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:21:26.300875 2563 flags.go:64] FLAG: --enable-controller-attach-detach="true" Jul 29 08:21:26.306998 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:21:26.300878 2563 flags.go:64] FLAG: --enable-debugging-handlers="true" Jul 29 08:21:26.306998 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:21:26.300880 2563 flags.go:64] FLAG: --enable-load-reader="false" Jul 29 08:21:26.306998 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:21:26.300883 2563 flags.go:64] FLAG: --enable-server="true" Jul 29 08:21:26.306998 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:21:26.300886 2563 flags.go:64] FLAG: --enforce-node-allocatable="[pods]" Jul 29 08:21:26.306998 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:21:26.300891 2563 flags.go:64] FLAG: --event-burst="100" Jul 29 08:21:26.306998 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:21:26.300896 2563 flags.go:64] FLAG: --event-qps="50" Jul 29 08:21:26.306998 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:21:26.300899 2563 flags.go:64] FLAG: --event-storage-age-limit="default=0" Jul 29 08:21:26.306998 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:21:26.300901 2563 flags.go:64] FLAG: --event-storage-event-limit="default=0" Jul 29 08:21:26.306998 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:21:26.300904 2563 flags.go:64] FLAG: --eviction-hard="" Jul 29 08:21:26.306998 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:21:26.300908 2563 flags.go:64] FLAG: --eviction-max-pod-grace-period="0" Jul 29 08:21:26.306998 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:21:26.300911 2563 flags.go:64] FLAG: --eviction-minimum-reclaim="" Jul 29 08:21:26.306998 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:21:26.300913 2563 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s" Jul 29 08:21:26.306998 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:21:26.300916 2563 flags.go:64] FLAG: --eviction-soft="" Jul 29 08:21:26.306998 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:21:26.300919 2563 flags.go:64] FLAG: --eviction-soft-grace-period="" Jul 29 08:21:26.306998 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:21:26.300922 2563 flags.go:64] FLAG: --exit-on-lock-contention="false" Jul 29 08:21:26.306998 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:21:26.300925 2563 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false" Jul 29 08:21:26.306998 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:21:26.300928 2563 flags.go:64] FLAG: --experimental-mounter-path="" Jul 29 08:21:26.306998 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:21:26.300933 2563 flags.go:64] FLAG: --fail-cgroupv1="false" Jul 29 08:21:26.306998 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:21:26.300935 2563 flags.go:64] FLAG: --fail-swap-on="true" Jul 29 08:21:26.306998 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:21:26.300938 2563 flags.go:64] FLAG: --feature-gates="" Jul 29 08:21:26.307670 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:21:26.300942 2563 flags.go:64] FLAG: --file-check-frequency="20s" Jul 29 08:21:26.307670 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:21:26.300945 2563 flags.go:64] FLAG: --global-housekeeping-interval="1m0s" Jul 29 08:21:26.307670 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:21:26.300961 2563 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge" Jul 29 08:21:26.307670 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:21:26.300965 2563 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1" Jul 29 08:21:26.307670 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:21:26.300968 2563 flags.go:64] FLAG: --healthz-port="10248" Jul 29 08:21:26.307670 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:21:26.300971 2563 flags.go:64] FLAG: --help="false" Jul 29 08:21:26.307670 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:21:26.300974 2563 flags.go:64] FLAG: --hostname-override="ip-10-0-131-138.ec2.internal" Jul 29 08:21:26.307670 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:21:26.300977 2563 flags.go:64] FLAG: --housekeeping-interval="10s" Jul 29 08:21:26.307670 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:21:26.300979 2563 flags.go:64] FLAG: --http-check-frequency="20s" Jul 29 08:21:26.307670 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:21:26.300982 2563 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins" Jul 29 08:21:26.307670 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:21:26.300985 2563 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml" Jul 29 08:21:26.307670 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:21:26.300988 2563 flags.go:64] FLAG: --image-gc-high-threshold="85" Jul 29 08:21:26.307670 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:21:26.300991 2563 flags.go:64] FLAG: --image-gc-low-threshold="80" Jul 29 08:21:26.307670 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:21:26.300994 2563 flags.go:64] FLAG: --image-service-endpoint="" Jul 29 08:21:26.307670 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:21:26.300996 2563 flags.go:64] FLAG: --kernel-memcg-notification="false" Jul 29 08:21:26.307670 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:21:26.300999 2563 flags.go:64] FLAG: --kube-api-burst="100" Jul 29 08:21:26.307670 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:21:26.301002 2563 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf" Jul 29 08:21:26.307670 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:21:26.301008 2563 flags.go:64] FLAG: --kube-api-qps="50" Jul 29 08:21:26.307670 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:21:26.301011 2563 flags.go:64] FLAG: --kube-reserved="" Jul 29 08:21:26.307670 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:21:26.301014 2563 flags.go:64] FLAG: --kube-reserved-cgroup="" Jul 29 08:21:26.307670 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:21:26.301016 2563 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig" Jul 29 08:21:26.307670 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:21:26.301019 2563 flags.go:64] FLAG: --kubelet-cgroups="" Jul 29 08:21:26.307670 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:21:26.301022 2563 flags.go:64] FLAG: --local-storage-capacity-isolation="true" Jul 29 08:21:26.307670 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:21:26.301025 2563 flags.go:64] FLAG: --lock-file="" Jul 29 08:21:26.308256 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:21:26.301027 2563 flags.go:64] FLAG: --log-cadvisor-usage="false" Jul 29 08:21:26.308256 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:21:26.301030 2563 flags.go:64] FLAG: --log-flush-frequency="5s" Jul 29 08:21:26.308256 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:21:26.301033 2563 flags.go:64] FLAG: --log-json-info-buffer-size="0" Jul 29 08:21:26.308256 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:21:26.301038 2563 flags.go:64] FLAG: --log-json-split-stream="false" Jul 29 08:21:26.308256 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:21:26.301041 2563 flags.go:64] FLAG: --log-text-info-buffer-size="0" Jul 29 08:21:26.308256 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:21:26.301046 2563 flags.go:64] FLAG: --log-text-split-stream="false" Jul 29 08:21:26.308256 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:21:26.301049 2563 flags.go:64] FLAG: --logging-format="text" Jul 29 08:21:26.308256 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:21:26.301051 2563 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id" Jul 29 08:21:26.308256 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:21:26.301054 2563 flags.go:64] FLAG: --make-iptables-util-chains="true" Jul 29 08:21:26.308256 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:21:26.301057 2563 flags.go:64] FLAG: --manifest-url="" Jul 29 08:21:26.308256 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:21:26.301068 2563 flags.go:64] FLAG: --manifest-url-header="" Jul 29 08:21:26.308256 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:21:26.301073 2563 flags.go:64] FLAG: --max-housekeeping-interval="15s" Jul 29 08:21:26.308256 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:21:26.301076 2563 flags.go:64] FLAG: --max-open-files="1000000" Jul 29 08:21:26.308256 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:21:26.301079 2563 flags.go:64] FLAG: --max-pods="110" Jul 29 08:21:26.308256 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:21:26.301083 2563 flags.go:64] FLAG: --maximum-dead-containers="-1" Jul 29 08:21:26.308256 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:21:26.301086 2563 flags.go:64] FLAG: --maximum-dead-containers-per-container="1" Jul 29 08:21:26.308256 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:21:26.301088 2563 flags.go:64] FLAG: --memory-manager-policy="None" Jul 29 08:21:26.308256 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:21:26.301091 2563 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s" Jul 29 08:21:26.308256 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:21:26.301094 2563 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s" Jul 29 08:21:26.308256 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:21:26.301097 2563 flags.go:64] FLAG: --node-ip="0.0.0.0" Jul 29 08:21:26.308256 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:21:26.301099 2563 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel" Jul 29 08:21:26.308256 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:21:26.301106 2563 flags.go:64] FLAG: --node-status-max-images="50" Jul 29 08:21:26.308256 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:21:26.301112 2563 flags.go:64] FLAG: --node-status-update-frequency="10s" Jul 29 08:21:26.308256 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:21:26.301115 2563 flags.go:64] FLAG: --oom-score-adj="-999" Jul 29 08:21:26.308256 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:21:26.301118 2563 flags.go:64] FLAG: --pod-cidr="" Jul 29 08:21:26.308906 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:21:26.301122 2563 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:7505336a406394b8c1c77d14d890a9f628aa4cfff172312179cb9af8c28c30bb" Jul 29 08:21:26.308906 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:21:26.301127 2563 flags.go:64] FLAG: --pod-manifest-path="" Jul 29 08:21:26.308906 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:21:26.301130 2563 flags.go:64] FLAG: --pod-max-pids="-1" Jul 29 08:21:26.308906 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:21:26.301133 2563 flags.go:64] FLAG: --pods-per-core="0" Jul 29 08:21:26.308906 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:21:26.301136 2563 flags.go:64] FLAG: --port="10250" Jul 29 08:21:26.308906 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:21:26.301139 2563 flags.go:64] FLAG: --protect-kernel-defaults="false" Jul 29 08:21:26.308906 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:21:26.301142 2563 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-0480972e843fe5845" Jul 29 08:21:26.308906 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:21:26.301145 2563 flags.go:64] FLAG: --qos-reserved="" Jul 29 08:21:26.308906 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:21:26.301148 2563 flags.go:64] FLAG: --read-only-port="10255" Jul 29 08:21:26.308906 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:21:26.301151 2563 flags.go:64] FLAG: --register-node="true" Jul 29 08:21:26.308906 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:21:26.301153 2563 flags.go:64] FLAG: --register-with-taints="" Jul 29 08:21:26.308906 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:21:26.301157 2563 flags.go:64] FLAG: --registry-burst="10" Jul 29 08:21:26.308906 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:21:26.301161 2563 flags.go:64] FLAG: --registry-qps="5" Jul 29 08:21:26.308906 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:21:26.301164 2563 flags.go:64] FLAG: --reserved-cpus="" Jul 29 08:21:26.308906 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:21:26.301167 2563 flags.go:64] FLAG: --reserved-memory="" Jul 29 08:21:26.308906 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:21:26.301170 2563 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf" Jul 29 08:21:26.308906 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:21:26.301173 2563 flags.go:64] FLAG: --root-dir="/var/lib/kubelet" Jul 29 08:21:26.308906 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:21:26.301176 2563 flags.go:64] FLAG: --rotate-certificates="false" Jul 29 08:21:26.308906 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:21:26.301179 2563 flags.go:64] FLAG: --rotate-server-certificates="false" Jul 29 08:21:26.308906 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:21:26.301181 2563 flags.go:64] FLAG: --runonce="false" Jul 29 08:21:26.308906 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:21:26.301184 2563 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service" Jul 29 08:21:26.308906 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:21:26.301187 2563 flags.go:64] FLAG: --runtime-request-timeout="2m0s" Jul 29 08:21:26.308906 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:21:26.301190 2563 flags.go:64] FLAG: --seccomp-default="false" Jul 29 08:21:26.308906 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:21:26.301193 2563 flags.go:64] FLAG: --serialize-image-pulls="true" Jul 29 08:21:26.308906 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:21:26.301196 2563 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s" Jul 29 08:21:26.309554 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:21:26.301199 2563 flags.go:64] FLAG: --storage-driver-db="cadvisor" Jul 29 08:21:26.309554 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:21:26.301202 2563 flags.go:64] FLAG: --storage-driver-host="localhost:8086" Jul 29 08:21:26.309554 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:21:26.301205 2563 flags.go:64] FLAG: --storage-driver-password="root" Jul 29 08:21:26.309554 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:21:26.301207 2563 flags.go:64] FLAG: --storage-driver-secure="false" Jul 29 08:21:26.309554 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:21:26.301210 2563 flags.go:64] FLAG: --storage-driver-table="stats" Jul 29 08:21:26.309554 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:21:26.301214 2563 flags.go:64] FLAG: --storage-driver-user="root" Jul 29 08:21:26.309554 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:21:26.301216 2563 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s" Jul 29 08:21:26.309554 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:21:26.301219 2563 flags.go:64] FLAG: --sync-frequency="1m0s" Jul 29 08:21:26.309554 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:21:26.301223 2563 flags.go:64] FLAG: --system-cgroups="" Jul 29 08:21:26.309554 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:21:26.301226 2563 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi" Jul 29 08:21:26.309554 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:21:26.301231 2563 flags.go:64] FLAG: --system-reserved-cgroup="" Jul 29 08:21:26.309554 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:21:26.301233 2563 flags.go:64] FLAG: --tls-cert-file="" Jul 29 08:21:26.309554 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:21:26.301236 2563 flags.go:64] FLAG: --tls-cipher-suites="[]" Jul 29 08:21:26.309554 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:21:26.301239 2563 flags.go:64] FLAG: --tls-min-version="" Jul 29 08:21:26.309554 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:21:26.301242 2563 flags.go:64] FLAG: --tls-private-key-file="" Jul 29 08:21:26.309554 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:21:26.301244 2563 flags.go:64] FLAG: --topology-manager-policy="none" Jul 29 08:21:26.309554 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:21:26.301247 2563 flags.go:64] FLAG: --topology-manager-policy-options="" Jul 29 08:21:26.309554 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:21:26.301250 2563 flags.go:64] FLAG: --topology-manager-scope="container" Jul 29 08:21:26.309554 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:21:26.301252 2563 flags.go:64] FLAG: --v="2" Jul 29 08:21:26.309554 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:21:26.301257 2563 flags.go:64] FLAG: --version="false" Jul 29 08:21:26.309554 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:21:26.301261 2563 flags.go:64] FLAG: --vmodule="" Jul 29 08:21:26.309554 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:21:26.301265 2563 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec" Jul 29 08:21:26.309554 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:21:26.301268 2563 flags.go:64] FLAG: --volume-stats-agg-period="1m0s" Jul 29 08:21:26.309554 ip-10-0-131-138 kubenswrapper[2563]: W0729 08:21:26.301399 2563 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 29 08:21:26.309554 ip-10-0-131-138 kubenswrapper[2563]: W0729 08:21:26.301403 2563 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 29 08:21:26.310160 ip-10-0-131-138 kubenswrapper[2563]: W0729 08:21:26.301407 2563 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 29 08:21:26.310160 ip-10-0-131-138 kubenswrapper[2563]: W0729 08:21:26.301410 2563 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 29 08:21:26.310160 ip-10-0-131-138 kubenswrapper[2563]: W0729 08:21:26.301413 2563 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 29 08:21:26.310160 ip-10-0-131-138 kubenswrapper[2563]: W0729 08:21:26.301415 2563 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 29 08:21:26.310160 ip-10-0-131-138 kubenswrapper[2563]: W0729 08:21:26.301418 2563 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 29 08:21:26.310160 ip-10-0-131-138 kubenswrapper[2563]: W0729 08:21:26.301420 2563 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 29 08:21:26.310160 ip-10-0-131-138 kubenswrapper[2563]: W0729 08:21:26.301423 2563 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 29 08:21:26.310160 ip-10-0-131-138 kubenswrapper[2563]: W0729 08:21:26.301425 2563 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 29 08:21:26.310160 ip-10-0-131-138 kubenswrapper[2563]: W0729 08:21:26.301428 2563 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 29 08:21:26.310160 ip-10-0-131-138 kubenswrapper[2563]: W0729 08:21:26.301430 2563 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 29 08:21:26.310160 ip-10-0-131-138 kubenswrapper[2563]: W0729 08:21:26.301433 2563 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 29 08:21:26.310160 ip-10-0-131-138 kubenswrapper[2563]: W0729 08:21:26.301435 2563 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 29 08:21:26.310160 ip-10-0-131-138 kubenswrapper[2563]: W0729 08:21:26.301438 2563 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 29 08:21:26.310160 ip-10-0-131-138 kubenswrapper[2563]: W0729 08:21:26.301441 2563 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 29 08:21:26.310160 ip-10-0-131-138 kubenswrapper[2563]: W0729 08:21:26.301445 2563 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 29 08:21:26.310160 ip-10-0-131-138 kubenswrapper[2563]: W0729 08:21:26.301449 2563 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 29 08:21:26.310160 ip-10-0-131-138 kubenswrapper[2563]: W0729 08:21:26.301452 2563 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 29 08:21:26.310160 ip-10-0-131-138 kubenswrapper[2563]: W0729 08:21:26.301455 2563 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 29 08:21:26.310160 ip-10-0-131-138 kubenswrapper[2563]: W0729 08:21:26.301458 2563 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 29 08:21:26.310160 ip-10-0-131-138 kubenswrapper[2563]: W0729 08:21:26.301461 2563 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 29 08:21:26.310679 ip-10-0-131-138 kubenswrapper[2563]: W0729 08:21:26.301463 2563 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 29 08:21:26.310679 ip-10-0-131-138 kubenswrapper[2563]: W0729 08:21:26.301466 2563 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 29 08:21:26.310679 ip-10-0-131-138 kubenswrapper[2563]: W0729 08:21:26.301469 2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 29 08:21:26.310679 ip-10-0-131-138 kubenswrapper[2563]: W0729 08:21:26.301471 2563 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 29 08:21:26.310679 ip-10-0-131-138 kubenswrapper[2563]: W0729 08:21:26.301474 2563 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 29 08:21:26.310679 ip-10-0-131-138 kubenswrapper[2563]: W0729 08:21:26.301476 2563 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 29 08:21:26.310679 ip-10-0-131-138 kubenswrapper[2563]: W0729 08:21:26.301480 2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 29 08:21:26.310679 ip-10-0-131-138 kubenswrapper[2563]: W0729 08:21:26.301484 2563 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 29 08:21:26.310679 ip-10-0-131-138 kubenswrapper[2563]: W0729 08:21:26.301488 2563 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 29 08:21:26.310679 ip-10-0-131-138 kubenswrapper[2563]: W0729 08:21:26.301490 2563 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 29 08:21:26.310679 ip-10-0-131-138 kubenswrapper[2563]: W0729 08:21:26.301493 2563 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 29 08:21:26.310679 ip-10-0-131-138 kubenswrapper[2563]: W0729 08:21:26.301496 2563 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 29 08:21:26.310679 ip-10-0-131-138 kubenswrapper[2563]: W0729 08:21:26.301498 2563 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 29 08:21:26.310679 ip-10-0-131-138 kubenswrapper[2563]: W0729 08:21:26.301501 2563 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 29 08:21:26.310679 ip-10-0-131-138 kubenswrapper[2563]: W0729 08:21:26.301503 2563 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 29 08:21:26.310679 ip-10-0-131-138 kubenswrapper[2563]: W0729 08:21:26.301505 2563 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 29 08:21:26.310679 ip-10-0-131-138 kubenswrapper[2563]: W0729 08:21:26.301508 2563 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 29 08:21:26.310679 ip-10-0-131-138 kubenswrapper[2563]: W0729 08:21:26.301510 2563 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 29 08:21:26.310679 ip-10-0-131-138 kubenswrapper[2563]: W0729 08:21:26.301513 2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 29 08:21:26.311140 ip-10-0-131-138 kubenswrapper[2563]: W0729 08:21:26.301515 2563 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 29 08:21:26.311140 ip-10-0-131-138 kubenswrapper[2563]: W0729 08:21:26.301517 2563 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 29 08:21:26.311140 ip-10-0-131-138 kubenswrapper[2563]: W0729 08:21:26.301520 2563 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 29 08:21:26.311140 ip-10-0-131-138 kubenswrapper[2563]: W0729 08:21:26.301522 2563 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 29 08:21:26.311140 ip-10-0-131-138 kubenswrapper[2563]: W0729 08:21:26.301526 2563 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 29 08:21:26.311140 ip-10-0-131-138 kubenswrapper[2563]: W0729 08:21:26.301529 2563 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 29 08:21:26.311140 ip-10-0-131-138 kubenswrapper[2563]: W0729 08:21:26.301532 2563 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 29 08:21:26.311140 ip-10-0-131-138 kubenswrapper[2563]: W0729 08:21:26.301535 2563 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 29 08:21:26.311140 ip-10-0-131-138 kubenswrapper[2563]: W0729 08:21:26.301539 2563 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 29 08:21:26.311140 ip-10-0-131-138 kubenswrapper[2563]: W0729 08:21:26.301541 2563 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 29 08:21:26.311140 ip-10-0-131-138 kubenswrapper[2563]: W0729 08:21:26.301544 2563 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 29 08:21:26.311140 ip-10-0-131-138 kubenswrapper[2563]: W0729 08:21:26.301546 2563 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 29 08:21:26.311140 ip-10-0-131-138 kubenswrapper[2563]: W0729 08:21:26.301549 2563 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 29 08:21:26.311140 ip-10-0-131-138 kubenswrapper[2563]: W0729 08:21:26.301551 2563 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 29 08:21:26.311140 ip-10-0-131-138 kubenswrapper[2563]: W0729 08:21:26.301554 2563 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 29 08:21:26.311140 ip-10-0-131-138 kubenswrapper[2563]: W0729 08:21:26.301556 2563 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 29 08:21:26.311140 ip-10-0-131-138 kubenswrapper[2563]: W0729 08:21:26.301559 2563 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 29 08:21:26.311140 ip-10-0-131-138 kubenswrapper[2563]: W0729 08:21:26.301561 2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 29 08:21:26.311140 ip-10-0-131-138 kubenswrapper[2563]: W0729 08:21:26.301564 2563 feature_gate.go:328] unrecognized feature gate: Example Jul 29 08:21:26.311622 ip-10-0-131-138 kubenswrapper[2563]: W0729 08:21:26.301567 2563 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 29 08:21:26.311622 ip-10-0-131-138 kubenswrapper[2563]: W0729 08:21:26.301570 2563 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 29 08:21:26.311622 ip-10-0-131-138 kubenswrapper[2563]: W0729 08:21:26.301572 2563 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 29 08:21:26.311622 ip-10-0-131-138 kubenswrapper[2563]: W0729 08:21:26.301575 2563 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 29 08:21:26.311622 ip-10-0-131-138 kubenswrapper[2563]: W0729 08:21:26.301578 2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 29 08:21:26.311622 ip-10-0-131-138 kubenswrapper[2563]: W0729 08:21:26.301580 2563 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 29 08:21:26.311622 ip-10-0-131-138 kubenswrapper[2563]: W0729 08:21:26.301582 2563 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 29 08:21:26.311622 ip-10-0-131-138 kubenswrapper[2563]: W0729 08:21:26.301585 2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 29 08:21:26.311622 ip-10-0-131-138 kubenswrapper[2563]: W0729 08:21:26.301587 2563 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 29 08:21:26.311622 ip-10-0-131-138 kubenswrapper[2563]: W0729 08:21:26.301590 2563 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 29 08:21:26.311622 ip-10-0-131-138 kubenswrapper[2563]: W0729 08:21:26.301592 2563 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 29 08:21:26.311622 ip-10-0-131-138 kubenswrapper[2563]: W0729 08:21:26.301594 2563 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 29 08:21:26.311622 ip-10-0-131-138 kubenswrapper[2563]: W0729 08:21:26.301597 2563 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 29 08:21:26.311622 ip-10-0-131-138 kubenswrapper[2563]: W0729 08:21:26.301599 2563 feature_gate.go:328] unrecognized feature gate: Example2 Jul 29 08:21:26.311622 ip-10-0-131-138 kubenswrapper[2563]: W0729 08:21:26.301601 2563 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 29 08:21:26.311622 ip-10-0-131-138 kubenswrapper[2563]: W0729 08:21:26.301604 2563 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 29 08:21:26.311622 ip-10-0-131-138 kubenswrapper[2563]: W0729 08:21:26.301606 2563 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 29 08:21:26.311622 ip-10-0-131-138 kubenswrapper[2563]: W0729 08:21:26.301609 2563 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 29 08:21:26.311622 ip-10-0-131-138 kubenswrapper[2563]: W0729 08:21:26.301611 2563 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 29 08:21:26.311622 ip-10-0-131-138 kubenswrapper[2563]: W0729 08:21:26.301613 2563 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 29 08:21:26.312112 ip-10-0-131-138 kubenswrapper[2563]: W0729 08:21:26.301616 2563 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 29 08:21:26.312112 ip-10-0-131-138 kubenswrapper[2563]: W0729 08:21:26.301620 2563 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 29 08:21:26.312112 ip-10-0-131-138 kubenswrapper[2563]: W0729 08:21:26.301623 2563 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 29 08:21:26.312112 ip-10-0-131-138 kubenswrapper[2563]: W0729 08:21:26.301625 2563 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 29 08:21:26.312112 ip-10-0-131-138 kubenswrapper[2563]: W0729 08:21:26.301628 2563 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 29 08:21:26.312112 ip-10-0-131-138 kubenswrapper[2563]: W0729 08:21:26.301630 2563 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 29 08:21:26.312112 ip-10-0-131-138 kubenswrapper[2563]: W0729 08:21:26.301632 2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 29 08:21:26.312112 ip-10-0-131-138 kubenswrapper[2563]: W0729 08:21:26.301635 2563 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 29 08:21:26.312112 ip-10-0-131-138 kubenswrapper[2563]: W0729 08:21:26.301638 2563 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 29 08:21:26.312112 ip-10-0-131-138 kubenswrapper[2563]: W0729 08:21:26.301640 2563 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 29 08:21:26.312112 ip-10-0-131-138 kubenswrapper[2563]: W0729 08:21:26.301642 2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 29 08:21:26.312112 ip-10-0-131-138 kubenswrapper[2563]: W0729 08:21:26.301645 2563 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 29 08:21:26.312112 ip-10-0-131-138 kubenswrapper[2563]: W0729 08:21:26.301650 2563 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 29 08:21:26.312112 ip-10-0-131-138 kubenswrapper[2563]: W0729 08:21:26.301653 2563 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 29 08:21:26.312112 ip-10-0-131-138 kubenswrapper[2563]: W0729 08:21:26.301655 2563 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 29 08:21:26.312112 ip-10-0-131-138 kubenswrapper[2563]: W0729 08:21:26.301657 2563 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 29 08:21:26.312112 ip-10-0-131-138 kubenswrapper[2563]: W0729 08:21:26.301660 2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 29 08:21:26.312112 ip-10-0-131-138 kubenswrapper[2563]: W0729 08:21:26.301663 2563 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 29 08:21:26.312112 ip-10-0-131-138 kubenswrapper[2563]: W0729 08:21:26.301665 2563 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 29 08:21:26.312603 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:21:26.302291 2563 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 29 08:21:26.312603 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:21:26.308983 2563 server.go:529] "Kubelet version" kubeletVersion="v1.34.9" Jul 29 08:21:26.312603 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:21:26.309009 2563 server.go:531] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK="" Jul 29 08:21:26.312603 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:21:26.309028 2563 watchdog_linux.go:95] "Systemd watchdog is not enabled" Jul 29 08:21:26.312603 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:21:26.309034 2563 watchdog_linux.go:137] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started." Jul 29 08:21:26.312603 ip-10-0-131-138 kubenswrapper[2563]: W0729 08:21:26.309096 2563 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 29 08:21:26.312603 ip-10-0-131-138 kubenswrapper[2563]: W0729 08:21:26.309102 2563 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 29 08:21:26.312603 ip-10-0-131-138 kubenswrapper[2563]: W0729 08:21:26.309105 2563 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 29 08:21:26.312603 ip-10-0-131-138 kubenswrapper[2563]: W0729 08:21:26.309108 2563 feature_gate.go:328] unrecognized feature gate: Example2 Jul 29 08:21:26.312603 ip-10-0-131-138 kubenswrapper[2563]: W0729 08:21:26.309111 2563 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 29 08:21:26.312603 ip-10-0-131-138 kubenswrapper[2563]: W0729 08:21:26.309115 2563 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 29 08:21:26.312603 ip-10-0-131-138 kubenswrapper[2563]: W0729 08:21:26.309119 2563 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 29 08:21:26.312603 ip-10-0-131-138 kubenswrapper[2563]: W0729 08:21:26.309123 2563 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 29 08:21:26.312603 ip-10-0-131-138 kubenswrapper[2563]: W0729 08:21:26.309126 2563 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 29 08:21:26.312960 ip-10-0-131-138 kubenswrapper[2563]: W0729 08:21:26.309129 2563 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 29 08:21:26.312960 ip-10-0-131-138 kubenswrapper[2563]: W0729 08:21:26.309131 2563 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 29 08:21:26.312960 ip-10-0-131-138 kubenswrapper[2563]: W0729 08:21:26.309134 2563 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 29 08:21:26.312960 ip-10-0-131-138 kubenswrapper[2563]: W0729 08:21:26.309137 2563 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 29 08:21:26.312960 ip-10-0-131-138 kubenswrapper[2563]: W0729 08:21:26.309139 2563 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 29 08:21:26.312960 ip-10-0-131-138 kubenswrapper[2563]: W0729 08:21:26.309142 2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 29 08:21:26.312960 ip-10-0-131-138 kubenswrapper[2563]: W0729 08:21:26.309145 2563 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 29 08:21:26.312960 ip-10-0-131-138 kubenswrapper[2563]: W0729 08:21:26.309148 2563 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 29 08:21:26.312960 ip-10-0-131-138 kubenswrapper[2563]: W0729 08:21:26.309150 2563 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 29 08:21:26.312960 ip-10-0-131-138 kubenswrapper[2563]: W0729 08:21:26.309153 2563 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 29 08:21:26.312960 ip-10-0-131-138 kubenswrapper[2563]: W0729 08:21:26.309155 2563 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 29 08:21:26.312960 ip-10-0-131-138 kubenswrapper[2563]: W0729 08:21:26.309158 2563 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 29 08:21:26.312960 ip-10-0-131-138 kubenswrapper[2563]: W0729 08:21:26.309160 2563 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 29 08:21:26.312960 ip-10-0-131-138 kubenswrapper[2563]: W0729 08:21:26.309163 2563 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 29 08:21:26.312960 ip-10-0-131-138 kubenswrapper[2563]: W0729 08:21:26.309167 2563 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 29 08:21:26.312960 ip-10-0-131-138 kubenswrapper[2563]: W0729 08:21:26.309171 2563 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 29 08:21:26.312960 ip-10-0-131-138 kubenswrapper[2563]: W0729 08:21:26.309174 2563 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 29 08:21:26.312960 ip-10-0-131-138 kubenswrapper[2563]: W0729 08:21:26.309177 2563 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 29 08:21:26.312960 ip-10-0-131-138 kubenswrapper[2563]: W0729 08:21:26.309180 2563 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 29 08:21:26.313561 ip-10-0-131-138 kubenswrapper[2563]: W0729 08:21:26.309183 2563 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 29 08:21:26.313561 ip-10-0-131-138 kubenswrapper[2563]: W0729 08:21:26.309187 2563 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 29 08:21:26.313561 ip-10-0-131-138 kubenswrapper[2563]: W0729 08:21:26.309189 2563 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 29 08:21:26.313561 ip-10-0-131-138 kubenswrapper[2563]: W0729 08:21:26.309192 2563 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 29 08:21:26.313561 ip-10-0-131-138 kubenswrapper[2563]: W0729 08:21:26.309195 2563 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 29 08:21:26.313561 ip-10-0-131-138 kubenswrapper[2563]: W0729 08:21:26.309197 2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 29 08:21:26.313561 ip-10-0-131-138 kubenswrapper[2563]: W0729 08:21:26.309199 2563 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 29 08:21:26.313561 ip-10-0-131-138 kubenswrapper[2563]: W0729 08:21:26.309202 2563 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 29 08:21:26.313561 ip-10-0-131-138 kubenswrapper[2563]: W0729 08:21:26.309204 2563 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 29 08:21:26.313561 ip-10-0-131-138 kubenswrapper[2563]: W0729 08:21:26.309207 2563 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 29 08:21:26.313561 ip-10-0-131-138 kubenswrapper[2563]: W0729 08:21:26.309209 2563 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 29 08:21:26.313561 ip-10-0-131-138 kubenswrapper[2563]: W0729 08:21:26.309211 2563 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 29 08:21:26.313561 ip-10-0-131-138 kubenswrapper[2563]: W0729 08:21:26.309214 2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 29 08:21:26.313561 ip-10-0-131-138 kubenswrapper[2563]: W0729 08:21:26.309216 2563 feature_gate.go:328] unrecognized feature gate: Example Jul 29 08:21:26.313561 ip-10-0-131-138 kubenswrapper[2563]: W0729 08:21:26.309219 2563 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 29 08:21:26.313561 ip-10-0-131-138 kubenswrapper[2563]: W0729 08:21:26.309221 2563 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 29 08:21:26.313561 ip-10-0-131-138 kubenswrapper[2563]: W0729 08:21:26.309223 2563 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 29 08:21:26.313561 ip-10-0-131-138 kubenswrapper[2563]: W0729 08:21:26.309226 2563 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 29 08:21:26.313561 ip-10-0-131-138 kubenswrapper[2563]: W0729 08:21:26.309228 2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 29 08:21:26.313561 ip-10-0-131-138 kubenswrapper[2563]: W0729 08:21:26.309231 2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 29 08:21:26.314045 ip-10-0-131-138 kubenswrapper[2563]: W0729 08:21:26.309233 2563 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 29 08:21:26.314045 ip-10-0-131-138 kubenswrapper[2563]: W0729 08:21:26.309236 2563 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 29 08:21:26.314045 ip-10-0-131-138 kubenswrapper[2563]: W0729 08:21:26.309238 2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 29 08:21:26.314045 ip-10-0-131-138 kubenswrapper[2563]: W0729 08:21:26.309240 2563 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 29 08:21:26.314045 ip-10-0-131-138 kubenswrapper[2563]: W0729 08:21:26.309243 2563 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 29 08:21:26.314045 ip-10-0-131-138 kubenswrapper[2563]: W0729 08:21:26.309245 2563 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 29 08:21:26.314045 ip-10-0-131-138 kubenswrapper[2563]: W0729 08:21:26.309248 2563 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 29 08:21:26.314045 ip-10-0-131-138 kubenswrapper[2563]: W0729 08:21:26.309250 2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 29 08:21:26.314045 ip-10-0-131-138 kubenswrapper[2563]: W0729 08:21:26.309253 2563 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 29 08:21:26.314045 ip-10-0-131-138 kubenswrapper[2563]: W0729 08:21:26.309255 2563 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 29 08:21:26.314045 ip-10-0-131-138 kubenswrapper[2563]: W0729 08:21:26.309258 2563 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 29 08:21:26.314045 ip-10-0-131-138 kubenswrapper[2563]: W0729 08:21:26.309260 2563 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 29 08:21:26.314045 ip-10-0-131-138 kubenswrapper[2563]: W0729 08:21:26.309262 2563 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 29 08:21:26.314045 ip-10-0-131-138 kubenswrapper[2563]: W0729 08:21:26.309266 2563 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 29 08:21:26.314045 ip-10-0-131-138 kubenswrapper[2563]: W0729 08:21:26.309269 2563 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 29 08:21:26.314045 ip-10-0-131-138 kubenswrapper[2563]: W0729 08:21:26.309271 2563 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 29 08:21:26.314045 ip-10-0-131-138 kubenswrapper[2563]: W0729 08:21:26.309274 2563 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 29 08:21:26.314045 ip-10-0-131-138 kubenswrapper[2563]: W0729 08:21:26.309276 2563 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 29 08:21:26.314045 ip-10-0-131-138 kubenswrapper[2563]: W0729 08:21:26.309278 2563 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 29 08:21:26.314045 ip-10-0-131-138 kubenswrapper[2563]: W0729 08:21:26.309281 2563 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 29 08:21:26.314560 ip-10-0-131-138 kubenswrapper[2563]: W0729 08:21:26.309283 2563 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 29 08:21:26.314560 ip-10-0-131-138 kubenswrapper[2563]: W0729 08:21:26.309286 2563 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 29 08:21:26.314560 ip-10-0-131-138 kubenswrapper[2563]: W0729 08:21:26.309288 2563 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 29 08:21:26.314560 ip-10-0-131-138 kubenswrapper[2563]: W0729 08:21:26.309290 2563 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 29 08:21:26.314560 ip-10-0-131-138 kubenswrapper[2563]: W0729 08:21:26.309293 2563 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 29 08:21:26.314560 ip-10-0-131-138 kubenswrapper[2563]: W0729 08:21:26.309295 2563 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 29 08:21:26.314560 ip-10-0-131-138 kubenswrapper[2563]: W0729 08:21:26.309298 2563 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 29 08:21:26.314560 ip-10-0-131-138 kubenswrapper[2563]: W0729 08:21:26.309301 2563 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 29 08:21:26.314560 ip-10-0-131-138 kubenswrapper[2563]: W0729 08:21:26.309303 2563 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 29 08:21:26.314560 ip-10-0-131-138 kubenswrapper[2563]: W0729 08:21:26.309306 2563 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 29 08:21:26.314560 ip-10-0-131-138 kubenswrapper[2563]: W0729 08:21:26.309308 2563 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 29 08:21:26.314560 ip-10-0-131-138 kubenswrapper[2563]: W0729 08:21:26.309311 2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 29 08:21:26.314560 ip-10-0-131-138 kubenswrapper[2563]: W0729 08:21:26.309313 2563 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 29 08:21:26.314560 ip-10-0-131-138 kubenswrapper[2563]: W0729 08:21:26.309316 2563 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 29 08:21:26.314560 ip-10-0-131-138 kubenswrapper[2563]: W0729 08:21:26.309318 2563 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 29 08:21:26.314560 ip-10-0-131-138 kubenswrapper[2563]: W0729 08:21:26.309320 2563 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 29 08:21:26.314560 ip-10-0-131-138 kubenswrapper[2563]: W0729 08:21:26.309323 2563 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 29 08:21:26.314560 ip-10-0-131-138 kubenswrapper[2563]: W0729 08:21:26.309325 2563 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 29 08:21:26.314560 ip-10-0-131-138 kubenswrapper[2563]: W0729 08:21:26.309328 2563 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 29 08:21:26.314560 ip-10-0-131-138 kubenswrapper[2563]: W0729 08:21:26.309330 2563 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 29 08:21:26.315037 ip-10-0-131-138 kubenswrapper[2563]: W0729 08:21:26.309353 2563 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 29 08:21:26.315037 ip-10-0-131-138 kubenswrapper[2563]: W0729 08:21:26.309357 2563 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 29 08:21:26.315037 ip-10-0-131-138 kubenswrapper[2563]: W0729 08:21:26.309361 2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 29 08:21:26.315037 ip-10-0-131-138 kubenswrapper[2563]: W0729 08:21:26.309365 2563 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 29 08:21:26.315037 ip-10-0-131-138 kubenswrapper[2563]: W0729 08:21:26.309369 2563 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 29 08:21:26.315037 ip-10-0-131-138 kubenswrapper[2563]: W0729 08:21:26.309374 2563 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 29 08:21:26.315037 ip-10-0-131-138 kubenswrapper[2563]: W0729 08:21:26.309376 2563 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 29 08:21:26.315037 ip-10-0-131-138 kubenswrapper[2563]: W0729 08:21:26.309380 2563 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 29 08:21:26.315037 ip-10-0-131-138 kubenswrapper[2563]: W0729 08:21:26.309382 2563 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 29 08:21:26.315037 ip-10-0-131-138 kubenswrapper[2563]: W0729 08:21:26.309384 2563 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 29 08:21:26.315037 ip-10-0-131-138 kubenswrapper[2563]: W0729 08:21:26.309387 2563 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 29 08:21:26.315037 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:21:26.309392 2563 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 29 08:21:26.315037 ip-10-0-131-138 kubenswrapper[2563]: W0729 08:21:26.309505 2563 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 29 08:21:26.315037 ip-10-0-131-138 kubenswrapper[2563]: W0729 08:21:26.309510 2563 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 29 08:21:26.315388 ip-10-0-131-138 kubenswrapper[2563]: W0729 08:21:26.309513 2563 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 29 08:21:26.315388 ip-10-0-131-138 kubenswrapper[2563]: W0729 08:21:26.309515 2563 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 29 08:21:26.315388 ip-10-0-131-138 kubenswrapper[2563]: W0729 08:21:26.309518 2563 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 29 08:21:26.315388 ip-10-0-131-138 kubenswrapper[2563]: W0729 08:21:26.309520 2563 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 29 08:21:26.315388 ip-10-0-131-138 kubenswrapper[2563]: W0729 08:21:26.309523 2563 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 29 08:21:26.315388 ip-10-0-131-138 kubenswrapper[2563]: W0729 08:21:26.309525 2563 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 29 08:21:26.315388 ip-10-0-131-138 kubenswrapper[2563]: W0729 08:21:26.309527 2563 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 29 08:21:26.315388 ip-10-0-131-138 kubenswrapper[2563]: W0729 08:21:26.309530 2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 29 08:21:26.315388 ip-10-0-131-138 kubenswrapper[2563]: W0729 08:21:26.309533 2563 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 29 08:21:26.315388 ip-10-0-131-138 kubenswrapper[2563]: W0729 08:21:26.309535 2563 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 29 08:21:26.315388 ip-10-0-131-138 kubenswrapper[2563]: W0729 08:21:26.309538 2563 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 29 08:21:26.315388 ip-10-0-131-138 kubenswrapper[2563]: W0729 08:21:26.309540 2563 feature_gate.go:328] unrecognized feature gate: Example2 Jul 29 08:21:26.315388 ip-10-0-131-138 kubenswrapper[2563]: W0729 08:21:26.309542 2563 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 29 08:21:26.315388 ip-10-0-131-138 kubenswrapper[2563]: W0729 08:21:26.309545 2563 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 29 08:21:26.315388 ip-10-0-131-138 kubenswrapper[2563]: W0729 08:21:26.309547 2563 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 29 08:21:26.315388 ip-10-0-131-138 kubenswrapper[2563]: W0729 08:21:26.309550 2563 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 29 08:21:26.315388 ip-10-0-131-138 kubenswrapper[2563]: W0729 08:21:26.309552 2563 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 29 08:21:26.315388 ip-10-0-131-138 kubenswrapper[2563]: W0729 08:21:26.309555 2563 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 29 08:21:26.315388 ip-10-0-131-138 kubenswrapper[2563]: W0729 08:21:26.309557 2563 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 29 08:21:26.315388 ip-10-0-131-138 kubenswrapper[2563]: W0729 08:21:26.309560 2563 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 29 08:21:26.315865 ip-10-0-131-138 kubenswrapper[2563]: W0729 08:21:26.309562 2563 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 29 08:21:26.315865 ip-10-0-131-138 kubenswrapper[2563]: W0729 08:21:26.309565 2563 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 29 08:21:26.315865 ip-10-0-131-138 kubenswrapper[2563]: W0729 08:21:26.309568 2563 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 29 08:21:26.315865 ip-10-0-131-138 kubenswrapper[2563]: W0729 08:21:26.309570 2563 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 29 08:21:26.315865 ip-10-0-131-138 kubenswrapper[2563]: W0729 08:21:26.309573 2563 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 29 08:21:26.315865 ip-10-0-131-138 kubenswrapper[2563]: W0729 08:21:26.309575 2563 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 29 08:21:26.315865 ip-10-0-131-138 kubenswrapper[2563]: W0729 08:21:26.309579 2563 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 29 08:21:26.315865 ip-10-0-131-138 kubenswrapper[2563]: W0729 08:21:26.309581 2563 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 29 08:21:26.315865 ip-10-0-131-138 kubenswrapper[2563]: W0729 08:21:26.309583 2563 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 29 08:21:26.315865 ip-10-0-131-138 kubenswrapper[2563]: W0729 08:21:26.309586 2563 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 29 08:21:26.315865 ip-10-0-131-138 kubenswrapper[2563]: W0729 08:21:26.309588 2563 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 29 08:21:26.315865 ip-10-0-131-138 kubenswrapper[2563]: W0729 08:21:26.309591 2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 29 08:21:26.315865 ip-10-0-131-138 kubenswrapper[2563]: W0729 08:21:26.309593 2563 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 29 08:21:26.315865 ip-10-0-131-138 kubenswrapper[2563]: W0729 08:21:26.309596 2563 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 29 08:21:26.315865 ip-10-0-131-138 kubenswrapper[2563]: W0729 08:21:26.309598 2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 29 08:21:26.315865 ip-10-0-131-138 kubenswrapper[2563]: W0729 08:21:26.309600 2563 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 29 08:21:26.315865 ip-10-0-131-138 kubenswrapper[2563]: W0729 08:21:26.309603 2563 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 29 08:21:26.315865 ip-10-0-131-138 kubenswrapper[2563]: W0729 08:21:26.309605 2563 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 29 08:21:26.315865 ip-10-0-131-138 kubenswrapper[2563]: W0729 08:21:26.309607 2563 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 29 08:21:26.315865 ip-10-0-131-138 kubenswrapper[2563]: W0729 08:21:26.309610 2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 29 08:21:26.316510 ip-10-0-131-138 kubenswrapper[2563]: W0729 08:21:26.309612 2563 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 29 08:21:26.316510 ip-10-0-131-138 kubenswrapper[2563]: W0729 08:21:26.309615 2563 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 29 08:21:26.316510 ip-10-0-131-138 kubenswrapper[2563]: W0729 08:21:26.309619 2563 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 29 08:21:26.316510 ip-10-0-131-138 kubenswrapper[2563]: W0729 08:21:26.309623 2563 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 29 08:21:26.316510 ip-10-0-131-138 kubenswrapper[2563]: W0729 08:21:26.309627 2563 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 29 08:21:26.316510 ip-10-0-131-138 kubenswrapper[2563]: W0729 08:21:26.309630 2563 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 29 08:21:26.316510 ip-10-0-131-138 kubenswrapper[2563]: W0729 08:21:26.309632 2563 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 29 08:21:26.316510 ip-10-0-131-138 kubenswrapper[2563]: W0729 08:21:26.309635 2563 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 29 08:21:26.316510 ip-10-0-131-138 kubenswrapper[2563]: W0729 08:21:26.309637 2563 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 29 08:21:26.316510 ip-10-0-131-138 kubenswrapper[2563]: W0729 08:21:26.309640 2563 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 29 08:21:26.316510 ip-10-0-131-138 kubenswrapper[2563]: W0729 08:21:26.309642 2563 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 29 08:21:26.316510 ip-10-0-131-138 kubenswrapper[2563]: W0729 08:21:26.309645 2563 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 29 08:21:26.316510 ip-10-0-131-138 kubenswrapper[2563]: W0729 08:21:26.309647 2563 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 29 08:21:26.316510 ip-10-0-131-138 kubenswrapper[2563]: W0729 08:21:26.309650 2563 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 29 08:21:26.316510 ip-10-0-131-138 kubenswrapper[2563]: W0729 08:21:26.309652 2563 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 29 08:21:26.316510 ip-10-0-131-138 kubenswrapper[2563]: W0729 08:21:26.309655 2563 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 29 08:21:26.316510 ip-10-0-131-138 kubenswrapper[2563]: W0729 08:21:26.309658 2563 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 29 08:21:26.316510 ip-10-0-131-138 kubenswrapper[2563]: W0729 08:21:26.309660 2563 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 29 08:21:26.316510 ip-10-0-131-138 kubenswrapper[2563]: W0729 08:21:26.309663 2563 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 29 08:21:26.316962 ip-10-0-131-138 kubenswrapper[2563]: W0729 08:21:26.309665 2563 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 29 08:21:26.316962 ip-10-0-131-138 kubenswrapper[2563]: W0729 08:21:26.309668 2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 29 08:21:26.316962 ip-10-0-131-138 kubenswrapper[2563]: W0729 08:21:26.309670 2563 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 29 08:21:26.316962 ip-10-0-131-138 kubenswrapper[2563]: W0729 08:21:26.309673 2563 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 29 08:21:26.316962 ip-10-0-131-138 kubenswrapper[2563]: W0729 08:21:26.309675 2563 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 29 08:21:26.316962 ip-10-0-131-138 kubenswrapper[2563]: W0729 08:21:26.309678 2563 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 29 08:21:26.316962 ip-10-0-131-138 kubenswrapper[2563]: W0729 08:21:26.309680 2563 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 29 08:21:26.316962 ip-10-0-131-138 kubenswrapper[2563]: W0729 08:21:26.309683 2563 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 29 08:21:26.316962 ip-10-0-131-138 kubenswrapper[2563]: W0729 08:21:26.309685 2563 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 29 08:21:26.316962 ip-10-0-131-138 kubenswrapper[2563]: W0729 08:21:26.309687 2563 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 29 08:21:26.316962 ip-10-0-131-138 kubenswrapper[2563]: W0729 08:21:26.309690 2563 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 29 08:21:26.316962 ip-10-0-131-138 kubenswrapper[2563]: W0729 08:21:26.309692 2563 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 29 08:21:26.316962 ip-10-0-131-138 kubenswrapper[2563]: W0729 08:21:26.309694 2563 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 29 08:21:26.316962 ip-10-0-131-138 kubenswrapper[2563]: W0729 08:21:26.309697 2563 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 29 08:21:26.316962 ip-10-0-131-138 kubenswrapper[2563]: W0729 08:21:26.309700 2563 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 29 08:21:26.316962 ip-10-0-131-138 kubenswrapper[2563]: W0729 08:21:26.309702 2563 feature_gate.go:328] unrecognized feature gate: Example Jul 29 08:21:26.316962 ip-10-0-131-138 kubenswrapper[2563]: W0729 08:21:26.309704 2563 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 29 08:21:26.316962 ip-10-0-131-138 kubenswrapper[2563]: W0729 08:21:26.309707 2563 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 29 08:21:26.316962 ip-10-0-131-138 kubenswrapper[2563]: W0729 08:21:26.309709 2563 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 29 08:21:26.316962 ip-10-0-131-138 kubenswrapper[2563]: W0729 08:21:26.309711 2563 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 29 08:21:26.317464 ip-10-0-131-138 kubenswrapper[2563]: W0729 08:21:26.309714 2563 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 29 08:21:26.317464 ip-10-0-131-138 kubenswrapper[2563]: W0729 08:21:26.309716 2563 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 29 08:21:26.317464 ip-10-0-131-138 kubenswrapper[2563]: W0729 08:21:26.309719 2563 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 29 08:21:26.317464 ip-10-0-131-138 kubenswrapper[2563]: W0729 08:21:26.309721 2563 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 29 08:21:26.317464 ip-10-0-131-138 kubenswrapper[2563]: W0729 08:21:26.309723 2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 29 08:21:26.317464 ip-10-0-131-138 kubenswrapper[2563]: W0729 08:21:26.309725 2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 29 08:21:26.317464 ip-10-0-131-138 kubenswrapper[2563]: W0729 08:21:26.309728 2563 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 29 08:21:26.317464 ip-10-0-131-138 kubenswrapper[2563]: W0729 08:21:26.309730 2563 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 29 08:21:26.317464 ip-10-0-131-138 kubenswrapper[2563]: W0729 08:21:26.309733 2563 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 29 08:21:26.317464 ip-10-0-131-138 kubenswrapper[2563]: W0729 08:21:26.309737 2563 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 29 08:21:26.317464 ip-10-0-131-138 kubenswrapper[2563]: W0729 08:21:26.309740 2563 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 29 08:21:26.317464 ip-10-0-131-138 kubenswrapper[2563]: W0729 08:21:26.309743 2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 29 08:21:26.317464 ip-10-0-131-138 kubenswrapper[2563]: W0729 08:21:26.309745 2563 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 29 08:21:26.317464 ip-10-0-131-138 kubenswrapper[2563]: W0729 08:21:26.309748 2563 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 29 08:21:26.317464 ip-10-0-131-138 kubenswrapper[2563]: W0729 08:21:26.309751 2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 29 08:21:26.317464 ip-10-0-131-138 kubenswrapper[2563]: W0729 08:21:26.309754 2563 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 29 08:21:26.317464 ip-10-0-131-138 kubenswrapper[2563]: W0729 08:21:26.309756 2563 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 29 08:21:26.317464 ip-10-0-131-138 kubenswrapper[2563]: W0729 08:21:26.309759 2563 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 29 08:21:26.317928 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:21:26.309764 2563 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 29 08:21:26.317928 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:21:26.309860 2563 server.go:962] "Client rotation is on, will bootstrap in background" Jul 29 08:21:26.317928 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:21:26.311860 2563 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir" Jul 29 08:21:26.317928 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:21:26.313404 2563 server.go:1019] "Starting client certificate rotation" Jul 29 08:21:26.317928 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:21:26.313508 2563 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet" Jul 29 08:21:26.317928 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:21:26.313534 2563 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet" Jul 29 08:21:26.318238 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:21:26.318225 2563 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Jul 29 08:21:26.321328 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:21:26.321303 2563 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Jul 29 08:21:26.334935 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:21:26.334906 2563 log.go:25] "Validated CRI v1 runtime API" Jul 29 08:21:26.337525 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:21:26.337494 2563 log.go:25] "Validated CRI v1 image API" Jul 29 08:21:26.338681 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:21:26.338667 2563 server.go:1429] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd" Jul 29 08:21:26.342205 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:21:26.342183 2563 fs.go:135] Filesystem UUIDs: map[582524e7-a52b-48ab-af96-00560255b888:/dev/nvme0n1p4 7B77-95E7:/dev/nvme0n1p2 df53b3d5-7b4b-4bff-ade1-116d2c99ce3a:/dev/nvme0n1p3] Jul 29 08:21:26.342280 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:21:26.342204 2563 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}] Jul 29 08:21:26.347660 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:21:26.347557 2563 manager.go:216] Machine: {Timestamp:2026-07-29 08:21:26.34607021 +0000 UTC m=+0.365277889 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3099902 MemoryCapacity:32812175360 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec20d63af09c76d60ed5b4a8aae0be1a SystemUUID:ec20d63a-f09c-76d6-0ed5-b4a8aae0be1a BootID:9da74082-2c18-4ee4-bc51-bbb2eff87894 Filesystems:[{Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16406085632 Type:vfs Inodes:4005392 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6562435072 Type:vfs Inodes:819200 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6111232 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16406089728 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:7a:cd:9e:f7:13 Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:7a:cd:9e:f7:13 Speed:0 Mtu:9001} {Name:ovs-system MacAddress:9a:b3:ca:91:60:60 Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:32812175360 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:34603008 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None} Jul 29 08:21:26.347660 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:21:26.347653 2563 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available. Jul 29 08:21:26.347773 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:21:26.347730 2563 manager.go:232] Version: {KernelVersion:5.14.0-570.129.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260717-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:} Jul 29 08:21:26.349707 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:21:26.349688 2563 reflector.go:439] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Jul 29 08:21:26.351297 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:21:26.351272 2563 container_manager_linux.go:273] "Container manager verified user specified cgroup-root exists" cgroupRoot=[] Jul 29 08:21:26.351458 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:21:26.351301 2563 container_manager_linux.go:278] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-131-138.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2} Jul 29 08:21:26.352172 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:21:26.352160 2563 topology_manager.go:138] "Creating topology manager with none policy" Jul 29 08:21:26.352224 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:21:26.352174 2563 container_manager_linux.go:309] "Creating device plugin manager" Jul 29 08:21:26.352224 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:21:26.352182 2563 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock" Jul 29 08:21:26.352725 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:21:26.352715 2563 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock" Jul 29 08:21:26.353631 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:21:26.353621 2563 container_manager_linux.go:318] "Creating Dynamic Resource Allocation (DRA) manager" Jul 29 08:21:26.355505 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:21:26.355494 2563 state_mem.go:36] "Initialized new in-memory state store" Jul 29 08:21:26.355625 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:21:26.355616 2563 server.go:1240] "Using root directory" path="/var/lib/kubelet" Jul 29 08:21:26.355655 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:21:26.355650 2563 kubelet.go:486] "Attempting to sync node with API server" Jul 29 08:21:26.355681 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:21:26.355668 2563 kubelet.go:387] "Adding static pod path" path="/etc/kubernetes/manifests" Jul 29 08:21:26.355681 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:21:26.355678 2563 file.go:69] "Watching path" path="/etc/kubernetes/manifests" Jul 29 08:21:26.355736 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:21:26.355688 2563 kubelet.go:398] "Adding apiserver pod source" Jul 29 08:21:26.355736 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:21:26.355696 2563 apiserver.go:42] "Waiting for node sync before watching apiserver pods" Jul 29 08:21:26.356169 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:21:26.356155 2563 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Jul 29 08:21:26.356210 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:21:26.356182 2563 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Jul 29 08:21:26.358464 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:21:26.358439 2563 kuberuntime_manager.go:301] "Container runtime initialized" containerRuntime="cri-o" version="1.34.10-2.rhaos4.21.git83ee25b.el9" apiVersion="v1" Jul 29 08:21:26.360591 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:21:26.360564 2563 kubelet.go:960] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled" Jul 29 08:21:26.360683 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:21:26.360607 2563 kubelet.go:984] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled" Jul 29 08:21:26.362462 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:21:26.362446 2563 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume" Jul 29 08:21:26.362537 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:21:26.362485 2563 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir" Jul 29 08:21:26.362537 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:21:26.362496 2563 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo" Jul 29 08:21:26.362537 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:21:26.362505 2563 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path" Jul 29 08:21:26.362537 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:21:26.362518 2563 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs" Jul 29 08:21:26.362537 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:21:26.362527 2563 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret" Jul 29 08:21:26.362722 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:21:26.362539 2563 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi" Jul 29 08:21:26.362722 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:21:26.362548 2563 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api" Jul 29 08:21:26.362722 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:21:26.362560 2563 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc" Jul 29 08:21:26.362722 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:21:26.362568 2563 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap" Jul 29 08:21:26.362722 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:21:26.362589 2563 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected" Jul 29 08:21:26.362722 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:21:26.362599 2563 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume" Jul 29 08:21:26.364196 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:21:26.364179 2563 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi" Jul 29 08:21:26.364317 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:21:26.364210 2563 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image" Jul 29 08:21:26.367672 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:21:26.367651 2563 server.go:1268] "Started kubelet" Jul 29 08:21:26.367770 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:21:26.367729 2563 server.go:180] "Starting to listen" address="0.0.0.0" port=10250 Jul 29 08:21:26.368283 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:21:26.367903 2563 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10 Jul 29 08:21:26.368283 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:21:26.367963 2563 server_v1.go:49] "podresources" method="list" useActivePods=true Jul 29 08:21:26.368525 ip-10-0-131-138 systemd[1]: Started Kubernetes Kubelet. Jul 29 08:21:26.368710 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:21:26.368559 2563 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-131-138.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope Jul 29 08:21:26.368799 ip-10-0-131-138 kubenswrapper[2563]: E0729 08:21:26.368737 2563 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service" Jul 29 08:21:26.369124 ip-10-0-131-138 kubenswrapper[2563]: E0729 08:21:26.369058 2563 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-131-138.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node" Jul 29 08:21:26.369984 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:21:26.369966 2563 server.go:249] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock" Jul 29 08:21:26.370573 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:21:26.370556 2563 server.go:310] "Adding debug handlers to kubelet server" Jul 29 08:21:26.373398 ip-10-0-131-138 kubenswrapper[2563]: E0729 08:21:26.372428 2563 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-131-138.ec2.internal.18c6b4f996ba24a6 default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-131-138.ec2.internal,UID:ip-10-0-131-138.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-131-138.ec2.internal,},FirstTimestamp:2026-07-29 08:21:26.367618214 +0000 UTC m=+0.386825895,LastTimestamp:2026-07-29 08:21:26.367618214 +0000 UTC m=+0.386825895,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-131-138.ec2.internal,}" Jul 29 08:21:26.375151 ip-10-0-131-138 kubenswrapper[2563]: E0729 08:21:26.375134 2563 kubelet.go:1638] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache" Jul 29 08:21:26.375928 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:21:26.375911 2563 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-s84pj" Jul 29 08:21:26.377576 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:21:26.377560 2563 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving" Jul 29 08:21:26.377635 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:21:26.377576 2563 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer" Jul 29 08:21:26.377840 ip-10-0-131-138 kubenswrapper[2563]: E0729 08:21:26.377815 2563 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-131-138.ec2.internal\" not found" Jul 29 08:21:26.378376 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:21:26.378359 2563 desired_state_of_world_populator.go:146] "Desired state populator starts to run" Jul 29 08:21:26.378376 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:21:26.378363 2563 volume_manager.go:311] "The desired_state_of_world populator starts" Jul 29 08:21:26.378496 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:21:26.378387 2563 volume_manager.go:313] "Starting Kubelet Volume Manager" Jul 29 08:21:26.378496 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:21:26.378478 2563 reconstruct.go:97] "Volume reconstruction finished" Jul 29 08:21:26.378496 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:21:26.378488 2563 reconciler.go:29] "Reconciler: start to sync state" Jul 29 08:21:26.378603 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:21:26.378542 2563 factory.go:55] Registering systemd factory Jul 29 08:21:26.378603 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:21:26.378565 2563 factory.go:223] Registration of the systemd container factory successfully Jul 29 08:21:26.378748 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:21:26.378738 2563 factory.go:153] Registering CRI-O factory Jul 29 08:21:26.378748 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:21:26.378749 2563 factory.go:223] Registration of the crio container factory successfully Jul 29 08:21:26.378815 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:21:26.378785 2563 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory Jul 29 08:21:26.378815 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:21:26.378805 2563 factory.go:103] Registering Raw factory Jul 29 08:21:26.378880 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:21:26.378820 2563 manager.go:1196] Started watching for new ooms in manager Jul 29 08:21:26.379278 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:21:26.379266 2563 manager.go:349] Starting recovery of all containers Jul 29 08:21:26.382183 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:21:26.382155 2563 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-s84pj" Jul 29 08:21:26.386851 ip-10-0-131-138 kubenswrapper[2563]: E0729 08:21:26.386798 2563 reflector.go:205] "Failed to watch" err="failed to list *v1.CSIDriver: csidrivers.storage.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"csidrivers\" in API group \"storage.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver" Jul 29 08:21:26.387053 ip-10-0-131-138 kubenswrapper[2563]: E0729 08:21:26.387013 2563 controller.go:145] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-131-138.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="200ms" Jul 29 08:21:26.388969 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:21:26.388949 2563 manager.go:354] Recovery completed Jul 29 08:21:26.394529 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:21:26.394430 2563 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 29 08:21:26.396207 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:21:26.396194 2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-138.ec2.internal" event="NodeHasSufficientMemory" Jul 29 08:21:26.396272 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:21:26.396226 2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-138.ec2.internal" event="NodeHasNoDiskPressure" Jul 29 08:21:26.396272 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:21:26.396237 2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-138.ec2.internal" event="NodeHasSufficientPID" Jul 29 08:21:26.396704 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:21:26.396691 2563 cpu_manager.go:222] "Starting CPU manager" policy="none" Jul 29 08:21:26.396704 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:21:26.396702 2563 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s" Jul 29 08:21:26.396810 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:21:26.396716 2563 state_mem.go:36] "Initialized new in-memory state store" Jul 29 08:21:26.396810 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:21:26.396752 2563 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 29 08:21:26.398177 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:21:26.398161 2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-138.ec2.internal" event="NodeHasSufficientMemory" Jul 29 08:21:26.398242 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:21:26.398188 2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-138.ec2.internal" event="NodeHasNoDiskPressure" Jul 29 08:21:26.398242 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:21:26.398198 2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-138.ec2.internal" event="NodeHasSufficientPID" Jul 29 08:21:26.398395 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:21:26.398383 2563 policy_none.go:49] "None policy: Start" Jul 29 08:21:26.398431 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:21:26.398399 2563 memory_manager.go:187] "Starting memorymanager" policy="None" Jul 29 08:21:26.398431 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:21:26.398409 2563 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint" Jul 29 08:21:26.399424 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:21:26.399415 2563 policy_none.go:47] "Start" Jul 29 08:21:26.435888 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:21:26.435872 2563 manager.go:337] "Starting Device Plugin manager" Jul 29 08:21:26.435972 ip-10-0-131-138 kubenswrapper[2563]: E0729 08:21:26.435902 2563 manager.go:513] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint" Jul 29 08:21:26.435972 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:21:26.435912 2563 server.go:87] "Starting device plugin registration server" Jul 29 08:21:26.436172 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:21:26.436149 2563 eviction_manager.go:189] "Eviction manager: starting control loop" Jul 29 08:21:26.436262 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:21:26.436163 2563 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s" Jul 29 08:21:26.436262 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:21:26.436250 2563 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry" Jul 29 08:21:26.436360 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:21:26.436319 2563 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts" Jul 29 08:21:26.436360 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:21:26.436327 2563 plugin_manager.go:118] "Starting Kubelet Plugin Manager" Jul 29 08:21:26.436877 ip-10-0-131-138 kubenswrapper[2563]: E0729 08:21:26.436853 2563 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\"" Jul 29 08:21:26.436954 ip-10-0-131-138 kubenswrapper[2563]: E0729 08:21:26.436913 2563 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-131-138.ec2.internal\" not found" Jul 29 08:21:26.517626 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:21:26.517587 2563 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4" Jul 29 08:21:26.518940 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:21:26.518792 2563 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6" Jul 29 08:21:26.518940 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:21:26.518823 2563 status_manager.go:244] "Starting to sync pod status with apiserver" Jul 29 08:21:26.518940 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:21:26.518844 2563 kubelet.go:2448] "Starting kubelet main sync loop" Jul 29 08:21:26.518940 ip-10-0-131-138 kubenswrapper[2563]: E0729 08:21:26.518874 2563 kubelet.go:2472] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful" Jul 29 08:21:26.521883 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:21:26.521867 2563 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Jul 29 08:21:26.537222 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:21:26.537199 2563 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 29 08:21:26.537989 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:21:26.537969 2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-138.ec2.internal" event="NodeHasSufficientMemory" Jul 29 08:21:26.538077 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:21:26.538003 2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-138.ec2.internal" event="NodeHasNoDiskPressure" Jul 29 08:21:26.538077 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:21:26.538018 2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-138.ec2.internal" event="NodeHasSufficientPID" Jul 29 08:21:26.538077 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:21:26.538047 2563 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-131-138.ec2.internal" Jul 29 08:21:26.547176 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:21:26.547127 2563 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-131-138.ec2.internal" Jul 29 08:21:26.547176 ip-10-0-131-138 kubenswrapper[2563]: E0729 08:21:26.547148 2563 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-131-138.ec2.internal\": node \"ip-10-0-131-138.ec2.internal\" not found" Jul 29 08:21:26.591648 ip-10-0-131-138 kubenswrapper[2563]: E0729 08:21:26.591619 2563 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-131-138.ec2.internal\" not found" Jul 29 08:21:26.619380 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:21:26.619360 2563 kubelet.go:2535] "SyncLoop ADD" source="file" pods=["kube-system/kube-apiserver-proxy-ip-10-0-131-138.ec2.internal","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-138.ec2.internal"] Jul 29 08:21:26.619456 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:21:26.619413 2563 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 29 08:21:26.620411 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:21:26.620394 2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-138.ec2.internal" event="NodeHasSufficientMemory" Jul 29 08:21:26.620507 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:21:26.620423 2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-138.ec2.internal" event="NodeHasNoDiskPressure" Jul 29 08:21:26.620507 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:21:26.620432 2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-138.ec2.internal" event="NodeHasSufficientPID" Jul 29 08:21:26.621817 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:21:26.621804 2563 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 29 08:21:26.621917 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:21:26.621904 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-131-138.ec2.internal" Jul 29 08:21:26.621957 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:21:26.621929 2563 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 29 08:21:26.622547 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:21:26.622532 2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-138.ec2.internal" event="NodeHasSufficientMemory" Jul 29 08:21:26.622640 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:21:26.622564 2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-138.ec2.internal" event="NodeHasNoDiskPressure" Jul 29 08:21:26.622640 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:21:26.622579 2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-138.ec2.internal" event="NodeHasSufficientPID" Jul 29 08:21:26.622640 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:21:26.622585 2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-138.ec2.internal" event="NodeHasSufficientMemory" Jul 29 08:21:26.622640 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:21:26.622608 2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-138.ec2.internal" event="NodeHasNoDiskPressure" Jul 29 08:21:26.622640 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:21:26.622622 2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-138.ec2.internal" event="NodeHasSufficientPID" Jul 29 08:21:26.623801 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:21:26.623787 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-138.ec2.internal" Jul 29 08:21:26.623877 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:21:26.623817 2563 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 29 08:21:26.624433 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:21:26.624417 2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-138.ec2.internal" event="NodeHasSufficientMemory" Jul 29 08:21:26.624506 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:21:26.624447 2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-138.ec2.internal" event="NodeHasNoDiskPressure" Jul 29 08:21:26.624506 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:21:26.624459 2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-138.ec2.internal" event="NodeHasSufficientPID" Jul 29 08:21:26.649025 ip-10-0-131-138 kubenswrapper[2563]: E0729 08:21:26.649001 2563 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-131-138.ec2.internal\" not found" node="ip-10-0-131-138.ec2.internal" Jul 29 08:21:26.653538 ip-10-0-131-138 kubenswrapper[2563]: E0729 08:21:26.653524 2563 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-131-138.ec2.internal\" not found" node="ip-10-0-131-138.ec2.internal" Jul 29 08:21:26.679695 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:21:26.679672 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/99103139fa564168399467f94d133369-config\") pod \"kube-apiserver-proxy-ip-10-0-131-138.ec2.internal\" (UID: \"99103139fa564168399467f94d133369\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-131-138.ec2.internal" Jul 29 08:21:26.679772 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:21:26.679699 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/9f8ab28df5c602f62a77b9c2aed7b570-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-131-138.ec2.internal\" (UID: \"9f8ab28df5c602f62a77b9c2aed7b570\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-138.ec2.internal" Jul 29 08:21:26.679772 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:21:26.679719 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/9f8ab28df5c602f62a77b9c2aed7b570-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-131-138.ec2.internal\" (UID: \"9f8ab28df5c602f62a77b9c2aed7b570\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-138.ec2.internal" Jul 29 08:21:26.692048 ip-10-0-131-138 kubenswrapper[2563]: E0729 08:21:26.692028 2563 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-131-138.ec2.internal\" not found" Jul 29 08:21:26.779849 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:21:26.779817 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/99103139fa564168399467f94d133369-config\") pod \"kube-apiserver-proxy-ip-10-0-131-138.ec2.internal\" (UID: \"99103139fa564168399467f94d133369\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-131-138.ec2.internal" Jul 29 08:21:26.779849 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:21:26.779850 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/9f8ab28df5c602f62a77b9c2aed7b570-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-131-138.ec2.internal\" (UID: \"9f8ab28df5c602f62a77b9c2aed7b570\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-138.ec2.internal" Jul 29 08:21:26.780043 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:21:26.779870 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/9f8ab28df5c602f62a77b9c2aed7b570-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-131-138.ec2.internal\" (UID: \"9f8ab28df5c602f62a77b9c2aed7b570\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-138.ec2.internal" Jul 29 08:21:26.780043 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:21:26.779915 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/9f8ab28df5c602f62a77b9c2aed7b570-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-131-138.ec2.internal\" (UID: \"9f8ab28df5c602f62a77b9c2aed7b570\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-138.ec2.internal" Jul 29 08:21:26.780043 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:21:26.779927 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/99103139fa564168399467f94d133369-config\") pod \"kube-apiserver-proxy-ip-10-0-131-138.ec2.internal\" (UID: \"99103139fa564168399467f94d133369\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-131-138.ec2.internal" Jul 29 08:21:26.780043 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:21:26.779971 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/9f8ab28df5c602f62a77b9c2aed7b570-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-131-138.ec2.internal\" (UID: \"9f8ab28df5c602f62a77b9c2aed7b570\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-138.ec2.internal" Jul 29 08:21:26.792936 ip-10-0-131-138 kubenswrapper[2563]: E0729 08:21:26.792908 2563 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-131-138.ec2.internal\" not found" Jul 29 08:21:26.893477 ip-10-0-131-138 kubenswrapper[2563]: E0729 08:21:26.893410 2563 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-131-138.ec2.internal\" not found" Jul 29 08:21:26.951653 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:21:26.951616 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-131-138.ec2.internal" Jul 29 08:21:26.957377 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:21:26.957359 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-138.ec2.internal" Jul 29 08:21:28.432008 ip-10-0-131-138 kubenswrapper[2563]: E0729 08:21:28.431981 2563 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-131-138.ec2.internal\" not found" Jul 29 08:21:28.532634 ip-10-0-131-138 kubenswrapper[2563]: E0729 08:21:28.532583 2563 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-131-138.ec2.internal\" not found" Jul 29 08:21:28.633149 ip-10-0-131-138 kubenswrapper[2563]: E0729 08:21:28.633069 2563 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-131-138.ec2.internal\" not found" Jul 29 08:21:28.648890 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:21:28.648861 2563 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160" Jul 29 08:21:28.733662 ip-10-0-131-138 kubenswrapper[2563]: E0729 08:21:28.733619 2563 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-131-138.ec2.internal\" not found" Jul 29 08:21:28.750813 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:21:28.750782 2563 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials" Jul 29 08:21:28.750977 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:21:28.750955 2563 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received" Jul 29 08:21:28.751040 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:21:28.750986 2563 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received" Jul 29 08:21:28.824717 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:21:28.824678 2563 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-07-28 08:16:27 +0000 UTC" deadline="2028-03-12 21:21:43.169200522 +0000 UTC" Jul 29 08:21:28.824717 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:21:28.824709 2563 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="14221h0m14.34449561s" Jul 29 08:21:28.833961 ip-10-0-131-138 kubenswrapper[2563]: E0729 08:21:28.833935 2563 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-131-138.ec2.internal\" not found" Jul 29 08:21:28.934753 ip-10-0-131-138 kubenswrapper[2563]: E0729 08:21:28.934677 2563 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-131-138.ec2.internal\" not found" Jul 29 08:21:28.960923 ip-10-0-131-138 kubenswrapper[2563]: W0729 08:21:28.960880 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9f8ab28df5c602f62a77b9c2aed7b570.slice/crio-6d9b70b2bafedfc223766a379c31cad15edf6121cdc08f1418e9d90bfb3f5c2f WatchSource:0}: Error finding container 6d9b70b2bafedfc223766a379c31cad15edf6121cdc08f1418e9d90bfb3f5c2f: Status 404 returned error can't find the container with id 6d9b70b2bafedfc223766a379c31cad15edf6121cdc08f1418e9d90bfb3f5c2f Jul 29 08:21:28.962502 ip-10-0-131-138 kubenswrapper[2563]: W0729 08:21:28.962479 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod99103139fa564168399467f94d133369.slice/crio-5886551be8113ceaf8ded0289ac287837f19a5a20e3a3374da6d0ced569fc249 WatchSource:0}: Error finding container 5886551be8113ceaf8ded0289ac287837f19a5a20e3a3374da6d0ced569fc249: Status 404 returned error can't find the container with id 5886551be8113ceaf8ded0289ac287837f19a5a20e3a3374da6d0ced569fc249 Jul 29 08:21:28.965125 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:21:28.965110 2563 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 29 08:21:29.010845 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:21:29.010817 2563 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160" Jul 29 08:21:29.016587 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:21:29.016569 2563 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-138.ec2.internal" Jul 29 08:21:29.033924 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:21:29.033902 2563 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Jul 29 08:21:29.035066 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:21:29.035047 2563 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-131-138.ec2.internal" Jul 29 08:21:29.046570 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:21:29.046549 2563 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Jul 29 08:21:29.088560 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:21:29.088537 2563 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160" Jul 29 08:21:29.794356 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:21:29.794324 2563 apiserver.go:52] "Watching apiserver" Jul 29 08:21:29.803703 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:21:29.803672 2563 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66" Jul 29 08:21:29.804100 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:21:29.804064 2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kube-system/kube-apiserver-proxy-ip-10-0-131-138.ec2.internal","openshift-cluster-node-tuning-operator/tuned-95gmk","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-138.ec2.internal","openshift-multus/network-metrics-daemon-qk9kb","openshift-network-operator/iptables-alerter-z2c22","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-52bkd","openshift-dns/node-resolver-bgb7d","openshift-image-registry/node-ca-9d4f5","openshift-multus/multus-additional-cni-plugins-qs8zl","openshift-multus/multus-v5xrb","openshift-network-diagnostics/network-check-target-rdclp","openshift-ovn-kubernetes/ovnkube-node-vqdfh","kube-system/global-pull-secret-syncer-rn9kc","kube-system/konnectivity-agent-8rp9g"] Jul 29 08:21:29.805635 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:21:29.805613 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-52bkd" Jul 29 08:21:29.806848 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:21:29.806826 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-bgb7d" Jul 29 08:21:29.808308 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:21:29.808283 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-qk9kb" Jul 29 08:21:29.808432 ip-10-0-131-138 kubenswrapper[2563]: E0729 08:21:29.808347 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-qk9kb" podUID="35e49848-94f6-43d9-b7c4-6c817dd5dde7" Jul 29 08:21:29.808643 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:21:29.808509 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\"" Jul 29 08:21:29.808643 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:21:29.808609 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-5h44f\"" Jul 29 08:21:29.808789 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:21:29.808661 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\"" Jul 29 08:21:29.808987 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:21:29.808971 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\"" Jul 29 08:21:29.809243 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:21:29.809227 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-qjq2g\"" Jul 29 08:21:29.809626 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:21:29.809478 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\"" Jul 29 08:21:29.809626 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:21:29.809616 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-z2c22" Jul 29 08:21:29.810083 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:21:29.810064 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\"" Jul 29 08:21:29.810857 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:21:29.810841 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-95gmk" Jul 29 08:21:29.812233 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:21:29.812144 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\"" Jul 29 08:21:29.812233 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:21:29.812171 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\"" Jul 29 08:21:29.812579 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:21:29.812408 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-9pn9p\"" Jul 29 08:21:29.812727 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:21:29.812711 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\"" Jul 29 08:21:29.812857 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:21:29.812842 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-9d4f5" Jul 29 08:21:29.813736 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:21:29.813524 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-mjwqc\"" Jul 29 08:21:29.813736 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:21:29.813534 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\"" Jul 29 08:21:29.813880 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:21:29.813745 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\"" Jul 29 08:21:29.814333 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:21:29.814314 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-qs8zl" Jul 29 08:21:29.815358 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:21:29.815333 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-v5xrb" Jul 29 08:21:29.817100 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:21:29.815813 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\"" Jul 29 08:21:29.817100 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:21:29.816110 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\"" Jul 29 08:21:29.817100 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:21:29.816221 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-qvmpd\"" Jul 29 08:21:29.817100 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:21:29.816378 2563 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving" Jul 29 08:21:29.817100 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:21:29.816722 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-rdclp" Jul 29 08:21:29.817100 ip-10-0-131-138 kubenswrapper[2563]: E0729 08:21:29.816783 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-rdclp" podUID="ce06e4b8-c4ec-4edf-a9c4-d7d86a03d844" Jul 29 08:21:29.817447 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:21:29.817218 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-2fhbh\"" Jul 29 08:21:29.817447 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:21:29.817347 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\"" Jul 29 08:21:29.817447 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:21:29.817434 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\"" Jul 29 08:21:29.817854 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:21:29.817638 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\"" Jul 29 08:21:29.817854 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:21:29.817689 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\"" Jul 29 08:21:29.817854 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:21:29.817843 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\"" Jul 29 08:21:29.818601 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:21:29.818582 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-vqdfh" Jul 29 08:21:29.818770 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:21:29.818754 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\"" Jul 29 08:21:29.818996 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:21:29.818972 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-q5vtr\"" Jul 29 08:21:29.819177 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:21:29.819162 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\"" Jul 29 08:21:29.820242 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:21:29.820200 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-rn9kc" Jul 29 08:21:29.820365 ip-10-0-131-138 kubenswrapper[2563]: E0729 08:21:29.820291 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-rn9kc" podUID="b14e5aed-a2b3-41bf-9389-701d69342953" Jul 29 08:21:29.821450 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:21:29.821432 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\"" Jul 29 08:21:29.821782 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:21:29.821750 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\"" Jul 29 08:21:29.823236 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:21:29.822563 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-8rp9g" Jul 29 08:21:29.823236 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:21:29.822925 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-6dk2v\"" Jul 29 08:21:29.823236 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:21:29.822941 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\"" Jul 29 08:21:29.823236 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:21:29.823122 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\"" Jul 29 08:21:29.823236 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:21:29.823164 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\"" Jul 29 08:21:29.823527 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:21:29.823442 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\"" Jul 29 08:21:29.832819 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:21:29.832779 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\"" Jul 29 08:21:29.833242 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:21:29.833226 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\"" Jul 29 08:21:29.833414 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:21:29.833398 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-slwh4\"" Jul 29 08:21:29.833848 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:21:29.833591 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/93ba91b4-4c35-43be-9094-08b0b3a473b3-etc-selinux\") pod \"aws-ebs-csi-driver-node-52bkd\" (UID: \"93ba91b4-4c35-43be-9094-08b0b3a473b3\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-52bkd" Jul 29 08:21:29.833848 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:21:29.833654 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/30ff3fec-359e-4b75-82eb-9f1a194766f7-etc-modprobe-d\") pod \"tuned-95gmk\" (UID: \"30ff3fec-359e-4b75-82eb-9f1a194766f7\") " pod="openshift-cluster-node-tuning-operator/tuned-95gmk" Jul 29 08:21:29.833848 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:21:29.833696 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/bb5d06c1-cbbb-4e40-9e0a-43aa22af698f-cnibin\") pod \"multus-additional-cni-plugins-qs8zl\" (UID: \"bb5d06c1-cbbb-4e40-9e0a-43aa22af698f\") " pod="openshift-multus/multus-additional-cni-plugins-qs8zl" Jul 29 08:21:29.833848 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:21:29.833743 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/bb5d06c1-cbbb-4e40-9e0a-43aa22af698f-tuning-conf-dir\") pod \"multus-additional-cni-plugins-qs8zl\" (UID: \"bb5d06c1-cbbb-4e40-9e0a-43aa22af698f\") " pod="openshift-multus/multus-additional-cni-plugins-qs8zl" Jul 29 08:21:29.833848 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:21:29.833795 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/2b20d931-6631-4e41-aff5-ed22c5fdafa7-host-var-lib-cni-multus\") pod \"multus-v5xrb\" (UID: \"2b20d931-6631-4e41-aff5-ed22c5fdafa7\") " pod="openshift-multus/multus-v5xrb" Jul 29 08:21:29.833848 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:21:29.833835 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/35e49848-94f6-43d9-b7c4-6c817dd5dde7-metrics-certs\") pod \"network-metrics-daemon-qk9kb\" (UID: \"35e49848-94f6-43d9-b7c4-6c817dd5dde7\") " pod="openshift-multus/network-metrics-daemon-qk9kb" Jul 29 08:21:29.834175 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:21:29.833882 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/93ba91b4-4c35-43be-9094-08b0b3a473b3-sys-fs\") pod \"aws-ebs-csi-driver-node-52bkd\" (UID: \"93ba91b4-4c35-43be-9094-08b0b3a473b3\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-52bkd" Jul 29 08:21:29.834175 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:21:29.833921 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/30ff3fec-359e-4b75-82eb-9f1a194766f7-etc-tuned\") pod \"tuned-95gmk\" (UID: \"30ff3fec-359e-4b75-82eb-9f1a194766f7\") " pod="openshift-cluster-node-tuning-operator/tuned-95gmk" Jul 29 08:21:29.834175 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:21:29.833984 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pvwwt\" (UniqueName: \"kubernetes.io/projected/5bce82ca-dd94-425f-83d2-cd44232d8cdb-kube-api-access-pvwwt\") pod \"node-ca-9d4f5\" (UID: \"5bce82ca-dd94-425f-83d2-cd44232d8cdb\") " pod="openshift-image-registry/node-ca-9d4f5" Jul 29 08:21:29.834175 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:21:29.834014 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/2b20d931-6631-4e41-aff5-ed22c5fdafa7-host-var-lib-kubelet\") pod \"multus-v5xrb\" (UID: \"2b20d931-6631-4e41-aff5-ed22c5fdafa7\") " pod="openshift-multus/multus-v5xrb" Jul 29 08:21:29.834175 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:21:29.834049 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/30ff3fec-359e-4b75-82eb-9f1a194766f7-etc-kubernetes\") pod \"tuned-95gmk\" (UID: \"30ff3fec-359e-4b75-82eb-9f1a194766f7\") " pod="openshift-cluster-node-tuning-operator/tuned-95gmk" Jul 29 08:21:29.834175 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:21:29.834077 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/2b20d931-6631-4e41-aff5-ed22c5fdafa7-hostroot\") pod \"multus-v5xrb\" (UID: \"2b20d931-6631-4e41-aff5-ed22c5fdafa7\") " pod="openshift-multus/multus-v5xrb" Jul 29 08:21:29.834422 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:21:29.834243 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/2b20d931-6631-4e41-aff5-ed22c5fdafa7-etc-kubernetes\") pod \"multus-v5xrb\" (UID: \"2b20d931-6631-4e41-aff5-ed22c5fdafa7\") " pod="openshift-multus/multus-v5xrb" Jul 29 08:21:29.834422 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:21:29.834299 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/5bce82ca-dd94-425f-83d2-cd44232d8cdb-host\") pod \"node-ca-9d4f5\" (UID: \"5bce82ca-dd94-425f-83d2-cd44232d8cdb\") " pod="openshift-image-registry/node-ca-9d4f5" Jul 29 08:21:29.834422 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:21:29.834353 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/bb5d06c1-cbbb-4e40-9e0a-43aa22af698f-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-qs8zl\" (UID: \"bb5d06c1-cbbb-4e40-9e0a-43aa22af698f\") " pod="openshift-multus/multus-additional-cni-plugins-qs8zl" Jul 29 08:21:29.834422 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:21:29.834383 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/2b20d931-6631-4e41-aff5-ed22c5fdafa7-host-run-netns\") pod \"multus-v5xrb\" (UID: \"2b20d931-6631-4e41-aff5-ed22c5fdafa7\") " pod="openshift-multus/multus-v5xrb" Jul 29 08:21:29.834422 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:21:29.834404 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/30ff3fec-359e-4b75-82eb-9f1a194766f7-tmp\") pod \"tuned-95gmk\" (UID: \"30ff3fec-359e-4b75-82eb-9f1a194766f7\") " pod="openshift-cluster-node-tuning-operator/tuned-95gmk" Jul 29 08:21:29.834422 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:21:29.834421 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/2b20d931-6631-4e41-aff5-ed22c5fdafa7-host-var-lib-cni-bin\") pod \"multus-v5xrb\" (UID: \"2b20d931-6631-4e41-aff5-ed22c5fdafa7\") " pod="openshift-multus/multus-v5xrb" Jul 29 08:21:29.834585 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:21:29.834442 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/93ba91b4-4c35-43be-9094-08b0b3a473b3-socket-dir\") pod \"aws-ebs-csi-driver-node-52bkd\" (UID: \"93ba91b4-4c35-43be-9094-08b0b3a473b3\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-52bkd" Jul 29 08:21:29.834585 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:21:29.834466 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/30ff3fec-359e-4b75-82eb-9f1a194766f7-etc-sysconfig\") pod \"tuned-95gmk\" (UID: \"30ff3fec-359e-4b75-82eb-9f1a194766f7\") " pod="openshift-cluster-node-tuning-operator/tuned-95gmk" Jul 29 08:21:29.834585 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:21:29.834532 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/30ff3fec-359e-4b75-82eb-9f1a194766f7-run\") pod \"tuned-95gmk\" (UID: \"30ff3fec-359e-4b75-82eb-9f1a194766f7\") " pod="openshift-cluster-node-tuning-operator/tuned-95gmk" Jul 29 08:21:29.834585 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:21:29.834552 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/30ff3fec-359e-4b75-82eb-9f1a194766f7-lib-modules\") pod \"tuned-95gmk\" (UID: \"30ff3fec-359e-4b75-82eb-9f1a194766f7\") " pod="openshift-cluster-node-tuning-operator/tuned-95gmk" Jul 29 08:21:29.834585 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:21:29.834569 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/93ba91b4-4c35-43be-9094-08b0b3a473b3-kubelet-dir\") pod \"aws-ebs-csi-driver-node-52bkd\" (UID: \"93ba91b4-4c35-43be-9094-08b0b3a473b3\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-52bkd" Jul 29 08:21:29.834756 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:21:29.834585 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/93ba91b4-4c35-43be-9094-08b0b3a473b3-registration-dir\") pod \"aws-ebs-csi-driver-node-52bkd\" (UID: \"93ba91b4-4c35-43be-9094-08b0b3a473b3\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-52bkd" Jul 29 08:21:29.834756 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:21:29.834602 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/30ff3fec-359e-4b75-82eb-9f1a194766f7-sys\") pod \"tuned-95gmk\" (UID: \"30ff3fec-359e-4b75-82eb-9f1a194766f7\") " pod="openshift-cluster-node-tuning-operator/tuned-95gmk" Jul 29 08:21:29.834756 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:21:29.834619 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/30ff3fec-359e-4b75-82eb-9f1a194766f7-var-lib-kubelet\") pod \"tuned-95gmk\" (UID: \"30ff3fec-359e-4b75-82eb-9f1a194766f7\") " pod="openshift-cluster-node-tuning-operator/tuned-95gmk" Jul 29 08:21:29.834756 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:21:29.834724 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/2b20d931-6631-4e41-aff5-ed22c5fdafa7-os-release\") pod \"multus-v5xrb\" (UID: \"2b20d931-6631-4e41-aff5-ed22c5fdafa7\") " pod="openshift-multus/multus-v5xrb" Jul 29 08:21:29.834928 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:21:29.834772 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/30ff3fec-359e-4b75-82eb-9f1a194766f7-etc-systemd\") pod \"tuned-95gmk\" (UID: \"30ff3fec-359e-4b75-82eb-9f1a194766f7\") " pod="openshift-cluster-node-tuning-operator/tuned-95gmk" Jul 29 08:21:29.834928 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:21:29.834859 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-55kqw\" (UniqueName: \"kubernetes.io/projected/2b20d931-6631-4e41-aff5-ed22c5fdafa7-kube-api-access-55kqw\") pod \"multus-v5xrb\" (UID: \"2b20d931-6631-4e41-aff5-ed22c5fdafa7\") " pod="openshift-multus/multus-v5xrb" Jul 29 08:21:29.834928 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:21:29.834887 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/db249d48-ad47-4cc0-8a37-37078055ed38-iptables-alerter-script\") pod \"iptables-alerter-z2c22\" (UID: \"db249d48-ad47-4cc0-8a37-37078055ed38\") " pod="openshift-network-operator/iptables-alerter-z2c22" Jul 29 08:21:29.835034 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:21:29.834935 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/db249d48-ad47-4cc0-8a37-37078055ed38-host-slash\") pod \"iptables-alerter-z2c22\" (UID: \"db249d48-ad47-4cc0-8a37-37078055ed38\") " pod="openshift-network-operator/iptables-alerter-z2c22" Jul 29 08:21:29.835034 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:21:29.834973 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-b4tfp\" (UniqueName: \"kubernetes.io/projected/bc118958-c5f8-44b8-80a5-bfc2d80fad1d-kube-api-access-b4tfp\") pod \"node-resolver-bgb7d\" (UID: \"bc118958-c5f8-44b8-80a5-bfc2d80fad1d\") " pod="openshift-dns/node-resolver-bgb7d" Jul 29 08:21:29.835105 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:21:29.835030 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/93ba91b4-4c35-43be-9094-08b0b3a473b3-device-dir\") pod \"aws-ebs-csi-driver-node-52bkd\" (UID: \"93ba91b4-4c35-43be-9094-08b0b3a473b3\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-52bkd" Jul 29 08:21:29.835105 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:21:29.835066 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tdnsd\" (UniqueName: \"kubernetes.io/projected/93ba91b4-4c35-43be-9094-08b0b3a473b3-kube-api-access-tdnsd\") pod \"aws-ebs-csi-driver-node-52bkd\" (UID: \"93ba91b4-4c35-43be-9094-08b0b3a473b3\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-52bkd" Jul 29 08:21:29.835105 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:21:29.835098 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ptjkx\" (UniqueName: \"kubernetes.io/projected/30ff3fec-359e-4b75-82eb-9f1a194766f7-kube-api-access-ptjkx\") pod \"tuned-95gmk\" (UID: \"30ff3fec-359e-4b75-82eb-9f1a194766f7\") " pod="openshift-cluster-node-tuning-operator/tuned-95gmk" Jul 29 08:21:29.835203 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:21:29.835126 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vsl94\" (UniqueName: \"kubernetes.io/projected/bb5d06c1-cbbb-4e40-9e0a-43aa22af698f-kube-api-access-vsl94\") pod \"multus-additional-cni-plugins-qs8zl\" (UID: \"bb5d06c1-cbbb-4e40-9e0a-43aa22af698f\") " pod="openshift-multus/multus-additional-cni-plugins-qs8zl" Jul 29 08:21:29.835203 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:21:29.835162 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/2b20d931-6631-4e41-aff5-ed22c5fdafa7-system-cni-dir\") pod \"multus-v5xrb\" (UID: \"2b20d931-6631-4e41-aff5-ed22c5fdafa7\") " pod="openshift-multus/multus-v5xrb" Jul 29 08:21:29.835293 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:21:29.835211 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/2b20d931-6631-4e41-aff5-ed22c5fdafa7-multus-cni-dir\") pod \"multus-v5xrb\" (UID: \"2b20d931-6631-4e41-aff5-ed22c5fdafa7\") " pod="openshift-multus/multus-v5xrb" Jul 29 08:21:29.835293 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:21:29.835240 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/2b20d931-6631-4e41-aff5-ed22c5fdafa7-cnibin\") pod \"multus-v5xrb\" (UID: \"2b20d931-6631-4e41-aff5-ed22c5fdafa7\") " pod="openshift-multus/multus-v5xrb" Jul 29 08:21:29.835390 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:21:29.835296 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/bc118958-c5f8-44b8-80a5-bfc2d80fad1d-hosts-file\") pod \"node-resolver-bgb7d\" (UID: \"bc118958-c5f8-44b8-80a5-bfc2d80fad1d\") " pod="openshift-dns/node-resolver-bgb7d" Jul 29 08:21:29.835390 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:21:29.835321 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/30ff3fec-359e-4b75-82eb-9f1a194766f7-etc-sysctl-d\") pod \"tuned-95gmk\" (UID: \"30ff3fec-359e-4b75-82eb-9f1a194766f7\") " pod="openshift-cluster-node-tuning-operator/tuned-95gmk" Jul 29 08:21:29.835390 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:21:29.835365 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/bb5d06c1-cbbb-4e40-9e0a-43aa22af698f-system-cni-dir\") pod \"multus-additional-cni-plugins-qs8zl\" (UID: \"bb5d06c1-cbbb-4e40-9e0a-43aa22af698f\") " pod="openshift-multus/multus-additional-cni-plugins-qs8zl" Jul 29 08:21:29.835521 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:21:29.835399 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/2b20d931-6631-4e41-aff5-ed22c5fdafa7-multus-socket-dir-parent\") pod \"multus-v5xrb\" (UID: \"2b20d931-6631-4e41-aff5-ed22c5fdafa7\") " pod="openshift-multus/multus-v5xrb" Jul 29 08:21:29.835521 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:21:29.835431 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/2b20d931-6631-4e41-aff5-ed22c5fdafa7-host-run-multus-certs\") pod \"multus-v5xrb\" (UID: \"2b20d931-6631-4e41-aff5-ed22c5fdafa7\") " pod="openshift-multus/multus-v5xrb" Jul 29 08:21:29.835521 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:21:29.835461 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/bc118958-c5f8-44b8-80a5-bfc2d80fad1d-tmp-dir\") pod \"node-resolver-bgb7d\" (UID: \"bc118958-c5f8-44b8-80a5-bfc2d80fad1d\") " pod="openshift-dns/node-resolver-bgb7d" Jul 29 08:21:29.835631 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:21:29.835542 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/30ff3fec-359e-4b75-82eb-9f1a194766f7-etc-sysctl-conf\") pod \"tuned-95gmk\" (UID: \"30ff3fec-359e-4b75-82eb-9f1a194766f7\") " pod="openshift-cluster-node-tuning-operator/tuned-95gmk" Jul 29 08:21:29.835631 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:21:29.835582 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/bb5d06c1-cbbb-4e40-9e0a-43aa22af698f-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-qs8zl\" (UID: \"bb5d06c1-cbbb-4e40-9e0a-43aa22af698f\") " pod="openshift-multus/multus-additional-cni-plugins-qs8zl" Jul 29 08:21:29.835692 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:21:29.835628 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/2b20d931-6631-4e41-aff5-ed22c5fdafa7-cni-binary-copy\") pod \"multus-v5xrb\" (UID: \"2b20d931-6631-4e41-aff5-ed22c5fdafa7\") " pod="openshift-multus/multus-v5xrb" Jul 29 08:21:29.837294 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:21:29.835727 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/2b20d931-6631-4e41-aff5-ed22c5fdafa7-host-run-k8s-cni-cncf-io\") pod \"multus-v5xrb\" (UID: \"2b20d931-6631-4e41-aff5-ed22c5fdafa7\") " pod="openshift-multus/multus-v5xrb" Jul 29 08:21:29.837294 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:21:29.835820 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/bb5d06c1-cbbb-4e40-9e0a-43aa22af698f-os-release\") pod \"multus-additional-cni-plugins-qs8zl\" (UID: \"bb5d06c1-cbbb-4e40-9e0a-43aa22af698f\") " pod="openshift-multus/multus-additional-cni-plugins-qs8zl" Jul 29 08:21:29.837294 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:21:29.835918 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/bb5d06c1-cbbb-4e40-9e0a-43aa22af698f-cni-binary-copy\") pod \"multus-additional-cni-plugins-qs8zl\" (UID: \"bb5d06c1-cbbb-4e40-9e0a-43aa22af698f\") " pod="openshift-multus/multus-additional-cni-plugins-qs8zl" Jul 29 08:21:29.837294 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:21:29.835944 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nxn6h\" (UniqueName: \"kubernetes.io/projected/ce06e4b8-c4ec-4edf-a9c4-d7d86a03d844-kube-api-access-nxn6h\") pod \"network-check-target-rdclp\" (UID: \"ce06e4b8-c4ec-4edf-a9c4-d7d86a03d844\") " pod="openshift-network-diagnostics/network-check-target-rdclp" Jul 29 08:21:29.837294 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:21:29.835988 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8dzmg\" (UniqueName: \"kubernetes.io/projected/35e49848-94f6-43d9-b7c4-6c817dd5dde7-kube-api-access-8dzmg\") pod \"network-metrics-daemon-qk9kb\" (UID: \"35e49848-94f6-43d9-b7c4-6c817dd5dde7\") " pod="openshift-multus/network-metrics-daemon-qk9kb" Jul 29 08:21:29.837294 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:21:29.836045 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/2b20d931-6631-4e41-aff5-ed22c5fdafa7-multus-conf-dir\") pod \"multus-v5xrb\" (UID: \"2b20d931-6631-4e41-aff5-ed22c5fdafa7\") " pod="openshift-multus/multus-v5xrb" Jul 29 08:21:29.837294 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:21:29.836081 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/2b20d931-6631-4e41-aff5-ed22c5fdafa7-multus-daemon-config\") pod \"multus-v5xrb\" (UID: \"2b20d931-6631-4e41-aff5-ed22c5fdafa7\") " pod="openshift-multus/multus-v5xrb" Jul 29 08:21:29.837294 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:21:29.836140 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/30ff3fec-359e-4b75-82eb-9f1a194766f7-host\") pod \"tuned-95gmk\" (UID: \"30ff3fec-359e-4b75-82eb-9f1a194766f7\") " pod="openshift-cluster-node-tuning-operator/tuned-95gmk" Jul 29 08:21:29.837294 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:21:29.836190 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/5bce82ca-dd94-425f-83d2-cd44232d8cdb-serviceca\") pod \"node-ca-9d4f5\" (UID: \"5bce82ca-dd94-425f-83d2-cd44232d8cdb\") " pod="openshift-image-registry/node-ca-9d4f5" Jul 29 08:21:29.837294 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:21:29.836398 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2fnff\" (UniqueName: \"kubernetes.io/projected/db249d48-ad47-4cc0-8a37-37078055ed38-kube-api-access-2fnff\") pod \"iptables-alerter-z2c22\" (UID: \"db249d48-ad47-4cc0-8a37-37078055ed38\") " pod="openshift-network-operator/iptables-alerter-z2c22" Jul 29 08:21:29.838945 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:21:29.838926 2563 reflector.go:439] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Jul 29 08:21:29.856170 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:21:29.856145 2563 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-jsx9p" Jul 29 08:21:29.866689 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:21:29.866666 2563 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-jsx9p" Jul 29 08:21:29.917510 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:21:29.917480 2563 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world" Jul 29 08:21:29.937019 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:21:29.936985 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/49600d58-879f-477d-aa43-11177f411977-etc-openvswitch\") pod \"ovnkube-node-vqdfh\" (UID: \"49600d58-879f-477d-aa43-11177f411977\") " pod="openshift-ovn-kubernetes/ovnkube-node-vqdfh" Jul 29 08:21:29.937206 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:21:29.937028 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/49600d58-879f-477d-aa43-11177f411977-ovn-node-metrics-cert\") pod \"ovnkube-node-vqdfh\" (UID: \"49600d58-879f-477d-aa43-11177f411977\") " pod="openshift-ovn-kubernetes/ovnkube-node-vqdfh" Jul 29 08:21:29.937206 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:21:29.937053 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/cfd1d8df-e6b7-4a67-81f2-5d83da541e20-agent-certs\") pod \"konnectivity-agent-8rp9g\" (UID: \"cfd1d8df-e6b7-4a67-81f2-5d83da541e20\") " pod="kube-system/konnectivity-agent-8rp9g" Jul 29 08:21:29.937206 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:21:29.937083 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/93ba91b4-4c35-43be-9094-08b0b3a473b3-device-dir\") pod \"aws-ebs-csi-driver-node-52bkd\" (UID: \"93ba91b4-4c35-43be-9094-08b0b3a473b3\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-52bkd" Jul 29 08:21:29.937206 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:21:29.937105 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tdnsd\" (UniqueName: \"kubernetes.io/projected/93ba91b4-4c35-43be-9094-08b0b3a473b3-kube-api-access-tdnsd\") pod \"aws-ebs-csi-driver-node-52bkd\" (UID: \"93ba91b4-4c35-43be-9094-08b0b3a473b3\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-52bkd" Jul 29 08:21:29.937206 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:21:29.937131 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ptjkx\" (UniqueName: \"kubernetes.io/projected/30ff3fec-359e-4b75-82eb-9f1a194766f7-kube-api-access-ptjkx\") pod \"tuned-95gmk\" (UID: \"30ff3fec-359e-4b75-82eb-9f1a194766f7\") " pod="openshift-cluster-node-tuning-operator/tuned-95gmk" Jul 29 08:21:29.937206 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:21:29.937155 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vsl94\" (UniqueName: \"kubernetes.io/projected/bb5d06c1-cbbb-4e40-9e0a-43aa22af698f-kube-api-access-vsl94\") pod \"multus-additional-cni-plugins-qs8zl\" (UID: \"bb5d06c1-cbbb-4e40-9e0a-43aa22af698f\") " pod="openshift-multus/multus-additional-cni-plugins-qs8zl" Jul 29 08:21:29.937206 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:21:29.937176 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/2b20d931-6631-4e41-aff5-ed22c5fdafa7-system-cni-dir\") pod \"multus-v5xrb\" (UID: \"2b20d931-6631-4e41-aff5-ed22c5fdafa7\") " pod="openshift-multus/multus-v5xrb" Jul 29 08:21:29.937206 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:21:29.937197 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/2b20d931-6631-4e41-aff5-ed22c5fdafa7-multus-cni-dir\") pod \"multus-v5xrb\" (UID: \"2b20d931-6631-4e41-aff5-ed22c5fdafa7\") " pod="openshift-multus/multus-v5xrb" Jul 29 08:21:29.937664 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:21:29.937218 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/2b20d931-6631-4e41-aff5-ed22c5fdafa7-cnibin\") pod \"multus-v5xrb\" (UID: \"2b20d931-6631-4e41-aff5-ed22c5fdafa7\") " pod="openshift-multus/multus-v5xrb" Jul 29 08:21:29.937664 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:21:29.937242 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/bc118958-c5f8-44b8-80a5-bfc2d80fad1d-hosts-file\") pod \"node-resolver-bgb7d\" (UID: \"bc118958-c5f8-44b8-80a5-bfc2d80fad1d\") " pod="openshift-dns/node-resolver-bgb7d" Jul 29 08:21:29.937664 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:21:29.937283 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/30ff3fec-359e-4b75-82eb-9f1a194766f7-etc-sysctl-d\") pod \"tuned-95gmk\" (UID: \"30ff3fec-359e-4b75-82eb-9f1a194766f7\") " pod="openshift-cluster-node-tuning-operator/tuned-95gmk" Jul 29 08:21:29.937664 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:21:29.937308 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/bb5d06c1-cbbb-4e40-9e0a-43aa22af698f-system-cni-dir\") pod \"multus-additional-cni-plugins-qs8zl\" (UID: \"bb5d06c1-cbbb-4e40-9e0a-43aa22af698f\") " pod="openshift-multus/multus-additional-cni-plugins-qs8zl" Jul 29 08:21:29.937664 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:21:29.937334 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/2b20d931-6631-4e41-aff5-ed22c5fdafa7-multus-socket-dir-parent\") pod \"multus-v5xrb\" (UID: \"2b20d931-6631-4e41-aff5-ed22c5fdafa7\") " pod="openshift-multus/multus-v5xrb" Jul 29 08:21:29.937664 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:21:29.937370 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/2b20d931-6631-4e41-aff5-ed22c5fdafa7-host-run-multus-certs\") pod \"multus-v5xrb\" (UID: \"2b20d931-6631-4e41-aff5-ed22c5fdafa7\") " pod="openshift-multus/multus-v5xrb" Jul 29 08:21:29.937664 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:21:29.937395 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/bc118958-c5f8-44b8-80a5-bfc2d80fad1d-tmp-dir\") pod \"node-resolver-bgb7d\" (UID: \"bc118958-c5f8-44b8-80a5-bfc2d80fad1d\") " pod="openshift-dns/node-resolver-bgb7d" Jul 29 08:21:29.937664 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:21:29.937422 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/49600d58-879f-477d-aa43-11177f411977-run-systemd\") pod \"ovnkube-node-vqdfh\" (UID: \"49600d58-879f-477d-aa43-11177f411977\") " pod="openshift-ovn-kubernetes/ovnkube-node-vqdfh" Jul 29 08:21:29.937664 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:21:29.937478 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/49600d58-879f-477d-aa43-11177f411977-log-socket\") pod \"ovnkube-node-vqdfh\" (UID: \"49600d58-879f-477d-aa43-11177f411977\") " pod="openshift-ovn-kubernetes/ovnkube-node-vqdfh" Jul 29 08:21:29.937664 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:21:29.937506 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/49600d58-879f-477d-aa43-11177f411977-ovnkube-script-lib\") pod \"ovnkube-node-vqdfh\" (UID: \"49600d58-879f-477d-aa43-11177f411977\") " pod="openshift-ovn-kubernetes/ovnkube-node-vqdfh" Jul 29 08:21:29.937664 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:21:29.937531 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/30ff3fec-359e-4b75-82eb-9f1a194766f7-etc-sysctl-conf\") pod \"tuned-95gmk\" (UID: \"30ff3fec-359e-4b75-82eb-9f1a194766f7\") " pod="openshift-cluster-node-tuning-operator/tuned-95gmk" Jul 29 08:21:29.937664 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:21:29.937561 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/bb5d06c1-cbbb-4e40-9e0a-43aa22af698f-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-qs8zl\" (UID: \"bb5d06c1-cbbb-4e40-9e0a-43aa22af698f\") " pod="openshift-multus/multus-additional-cni-plugins-qs8zl" Jul 29 08:21:29.937664 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:21:29.937586 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/2b20d931-6631-4e41-aff5-ed22c5fdafa7-cni-binary-copy\") pod \"multus-v5xrb\" (UID: \"2b20d931-6631-4e41-aff5-ed22c5fdafa7\") " pod="openshift-multus/multus-v5xrb" Jul 29 08:21:29.937664 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:21:29.937616 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/2b20d931-6631-4e41-aff5-ed22c5fdafa7-host-run-k8s-cni-cncf-io\") pod \"multus-v5xrb\" (UID: \"2b20d931-6631-4e41-aff5-ed22c5fdafa7\") " pod="openshift-multus/multus-v5xrb" Jul 29 08:21:29.937664 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:21:29.937640 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/b14e5aed-a2b3-41bf-9389-701d69342953-dbus\") pod \"global-pull-secret-syncer-rn9kc\" (UID: \"b14e5aed-a2b3-41bf-9389-701d69342953\") " pod="kube-system/global-pull-secret-syncer-rn9kc" Jul 29 08:21:29.937664 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:21:29.937666 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/bb5d06c1-cbbb-4e40-9e0a-43aa22af698f-os-release\") pod \"multus-additional-cni-plugins-qs8zl\" (UID: \"bb5d06c1-cbbb-4e40-9e0a-43aa22af698f\") " pod="openshift-multus/multus-additional-cni-plugins-qs8zl" Jul 29 08:21:29.938455 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:21:29.937692 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/bb5d06c1-cbbb-4e40-9e0a-43aa22af698f-cni-binary-copy\") pod \"multus-additional-cni-plugins-qs8zl\" (UID: \"bb5d06c1-cbbb-4e40-9e0a-43aa22af698f\") " pod="openshift-multus/multus-additional-cni-plugins-qs8zl" Jul 29 08:21:29.938455 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:21:29.937719 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nxn6h\" (UniqueName: \"kubernetes.io/projected/ce06e4b8-c4ec-4edf-a9c4-d7d86a03d844-kube-api-access-nxn6h\") pod \"network-check-target-rdclp\" (UID: \"ce06e4b8-c4ec-4edf-a9c4-d7d86a03d844\") " pod="openshift-network-diagnostics/network-check-target-rdclp" Jul 29 08:21:29.938455 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:21:29.937744 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8dzmg\" (UniqueName: \"kubernetes.io/projected/35e49848-94f6-43d9-b7c4-6c817dd5dde7-kube-api-access-8dzmg\") pod \"network-metrics-daemon-qk9kb\" (UID: \"35e49848-94f6-43d9-b7c4-6c817dd5dde7\") " pod="openshift-multus/network-metrics-daemon-qk9kb" Jul 29 08:21:29.938455 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:21:29.937772 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/49600d58-879f-477d-aa43-11177f411977-host-cni-bin\") pod \"ovnkube-node-vqdfh\" (UID: \"49600d58-879f-477d-aa43-11177f411977\") " pod="openshift-ovn-kubernetes/ovnkube-node-vqdfh" Jul 29 08:21:29.938455 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:21:29.937797 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/2b20d931-6631-4e41-aff5-ed22c5fdafa7-multus-conf-dir\") pod \"multus-v5xrb\" (UID: \"2b20d931-6631-4e41-aff5-ed22c5fdafa7\") " pod="openshift-multus/multus-v5xrb" Jul 29 08:21:29.938455 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:21:29.937821 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/2b20d931-6631-4e41-aff5-ed22c5fdafa7-multus-daemon-config\") pod \"multus-v5xrb\" (UID: \"2b20d931-6631-4e41-aff5-ed22c5fdafa7\") " pod="openshift-multus/multus-v5xrb" Jul 29 08:21:29.938455 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:21:29.937847 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/49600d58-879f-477d-aa43-11177f411977-systemd-units\") pod \"ovnkube-node-vqdfh\" (UID: \"49600d58-879f-477d-aa43-11177f411977\") " pod="openshift-ovn-kubernetes/ovnkube-node-vqdfh" Jul 29 08:21:29.938455 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:21:29.937871 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/b14e5aed-a2b3-41bf-9389-701d69342953-original-pull-secret\") pod \"global-pull-secret-syncer-rn9kc\" (UID: \"b14e5aed-a2b3-41bf-9389-701d69342953\") " pod="kube-system/global-pull-secret-syncer-rn9kc" Jul 29 08:21:29.938455 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:21:29.937895 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/30ff3fec-359e-4b75-82eb-9f1a194766f7-host\") pod \"tuned-95gmk\" (UID: \"30ff3fec-359e-4b75-82eb-9f1a194766f7\") " pod="openshift-cluster-node-tuning-operator/tuned-95gmk" Jul 29 08:21:29.938455 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:21:29.937924 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/5bce82ca-dd94-425f-83d2-cd44232d8cdb-serviceca\") pod \"node-ca-9d4f5\" (UID: \"5bce82ca-dd94-425f-83d2-cd44232d8cdb\") " pod="openshift-image-registry/node-ca-9d4f5" Jul 29 08:21:29.938455 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:21:29.937956 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2fnff\" (UniqueName: \"kubernetes.io/projected/db249d48-ad47-4cc0-8a37-37078055ed38-kube-api-access-2fnff\") pod \"iptables-alerter-z2c22\" (UID: \"db249d48-ad47-4cc0-8a37-37078055ed38\") " pod="openshift-network-operator/iptables-alerter-z2c22" Jul 29 08:21:29.938455 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:21:29.937985 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/93ba91b4-4c35-43be-9094-08b0b3a473b3-etc-selinux\") pod \"aws-ebs-csi-driver-node-52bkd\" (UID: \"93ba91b4-4c35-43be-9094-08b0b3a473b3\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-52bkd" Jul 29 08:21:29.938455 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:21:29.938011 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/30ff3fec-359e-4b75-82eb-9f1a194766f7-etc-modprobe-d\") pod \"tuned-95gmk\" (UID: \"30ff3fec-359e-4b75-82eb-9f1a194766f7\") " pod="openshift-cluster-node-tuning-operator/tuned-95gmk" Jul 29 08:21:29.938455 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:21:29.938042 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/bb5d06c1-cbbb-4e40-9e0a-43aa22af698f-cnibin\") pod \"multus-additional-cni-plugins-qs8zl\" (UID: \"bb5d06c1-cbbb-4e40-9e0a-43aa22af698f\") " pod="openshift-multus/multus-additional-cni-plugins-qs8zl" Jul 29 08:21:29.938455 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:21:29.938068 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/bb5d06c1-cbbb-4e40-9e0a-43aa22af698f-tuning-conf-dir\") pod \"multus-additional-cni-plugins-qs8zl\" (UID: \"bb5d06c1-cbbb-4e40-9e0a-43aa22af698f\") " pod="openshift-multus/multus-additional-cni-plugins-qs8zl" Jul 29 08:21:29.938455 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:21:29.938097 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/2b20d931-6631-4e41-aff5-ed22c5fdafa7-host-var-lib-cni-multus\") pod \"multus-v5xrb\" (UID: \"2b20d931-6631-4e41-aff5-ed22c5fdafa7\") " pod="openshift-multus/multus-v5xrb" Jul 29 08:21:29.938455 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:21:29.938131 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/35e49848-94f6-43d9-b7c4-6c817dd5dde7-metrics-certs\") pod \"network-metrics-daemon-qk9kb\" (UID: \"35e49848-94f6-43d9-b7c4-6c817dd5dde7\") " pod="openshift-multus/network-metrics-daemon-qk9kb" Jul 29 08:21:29.939234 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:21:29.938158 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/49600d58-879f-477d-aa43-11177f411977-host-kubelet\") pod \"ovnkube-node-vqdfh\" (UID: \"49600d58-879f-477d-aa43-11177f411977\") " pod="openshift-ovn-kubernetes/ovnkube-node-vqdfh" Jul 29 08:21:29.939234 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:21:29.938186 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/93ba91b4-4c35-43be-9094-08b0b3a473b3-sys-fs\") pod \"aws-ebs-csi-driver-node-52bkd\" (UID: \"93ba91b4-4c35-43be-9094-08b0b3a473b3\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-52bkd" Jul 29 08:21:29.939234 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:21:29.938211 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/30ff3fec-359e-4b75-82eb-9f1a194766f7-etc-tuned\") pod \"tuned-95gmk\" (UID: \"30ff3fec-359e-4b75-82eb-9f1a194766f7\") " pod="openshift-cluster-node-tuning-operator/tuned-95gmk" Jul 29 08:21:29.939234 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:21:29.938244 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pvwwt\" (UniqueName: \"kubernetes.io/projected/5bce82ca-dd94-425f-83d2-cd44232d8cdb-kube-api-access-pvwwt\") pod \"node-ca-9d4f5\" (UID: \"5bce82ca-dd94-425f-83d2-cd44232d8cdb\") " pod="openshift-image-registry/node-ca-9d4f5" Jul 29 08:21:29.939234 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:21:29.938292 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/2b20d931-6631-4e41-aff5-ed22c5fdafa7-host-var-lib-kubelet\") pod \"multus-v5xrb\" (UID: \"2b20d931-6631-4e41-aff5-ed22c5fdafa7\") " pod="openshift-multus/multus-v5xrb" Jul 29 08:21:29.939234 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:21:29.938352 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/2b20d931-6631-4e41-aff5-ed22c5fdafa7-host-var-lib-kubelet\") pod \"multus-v5xrb\" (UID: \"2b20d931-6631-4e41-aff5-ed22c5fdafa7\") " pod="openshift-multus/multus-v5xrb" Jul 29 08:21:29.939234 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:21:29.938357 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/2b20d931-6631-4e41-aff5-ed22c5fdafa7-host-run-k8s-cni-cncf-io\") pod \"multus-v5xrb\" (UID: \"2b20d931-6631-4e41-aff5-ed22c5fdafa7\") " pod="openshift-multus/multus-v5xrb" Jul 29 08:21:29.939234 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:21:29.938384 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/bc118958-c5f8-44b8-80a5-bfc2d80fad1d-hosts-file\") pod \"node-resolver-bgb7d\" (UID: \"bc118958-c5f8-44b8-80a5-bfc2d80fad1d\") " pod="openshift-dns/node-resolver-bgb7d" Jul 29 08:21:29.939234 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:21:29.938451 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/bb5d06c1-cbbb-4e40-9e0a-43aa22af698f-os-release\") pod \"multus-additional-cni-plugins-qs8zl\" (UID: \"bb5d06c1-cbbb-4e40-9e0a-43aa22af698f\") " pod="openshift-multus/multus-additional-cni-plugins-qs8zl" Jul 29 08:21:29.939234 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:21:29.938469 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/93ba91b4-4c35-43be-9094-08b0b3a473b3-device-dir\") pod \"aws-ebs-csi-driver-node-52bkd\" (UID: \"93ba91b4-4c35-43be-9094-08b0b3a473b3\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-52bkd" Jul 29 08:21:29.939234 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:21:29.938549 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/30ff3fec-359e-4b75-82eb-9f1a194766f7-host\") pod \"tuned-95gmk\" (UID: \"30ff3fec-359e-4b75-82eb-9f1a194766f7\") " pod="openshift-cluster-node-tuning-operator/tuned-95gmk" Jul 29 08:21:29.939234 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:21:29.938810 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/30ff3fec-359e-4b75-82eb-9f1a194766f7-etc-sysctl-d\") pod \"tuned-95gmk\" (UID: \"30ff3fec-359e-4b75-82eb-9f1a194766f7\") " pod="openshift-cluster-node-tuning-operator/tuned-95gmk" Jul 29 08:21:29.939234 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:21:29.938863 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/bb5d06c1-cbbb-4e40-9e0a-43aa22af698f-system-cni-dir\") pod \"multus-additional-cni-plugins-qs8zl\" (UID: \"bb5d06c1-cbbb-4e40-9e0a-43aa22af698f\") " pod="openshift-multus/multus-additional-cni-plugins-qs8zl" Jul 29 08:21:29.939234 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:21:29.938917 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/2b20d931-6631-4e41-aff5-ed22c5fdafa7-multus-socket-dir-parent\") pod \"multus-v5xrb\" (UID: \"2b20d931-6631-4e41-aff5-ed22c5fdafa7\") " pod="openshift-multus/multus-v5xrb" Jul 29 08:21:29.939234 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:21:29.938954 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/2b20d931-6631-4e41-aff5-ed22c5fdafa7-host-run-multus-certs\") pod \"multus-v5xrb\" (UID: \"2b20d931-6631-4e41-aff5-ed22c5fdafa7\") " pod="openshift-multus/multus-v5xrb" Jul 29 08:21:29.939234 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:21:29.939005 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/5bce82ca-dd94-425f-83d2-cd44232d8cdb-serviceca\") pod \"node-ca-9d4f5\" (UID: \"5bce82ca-dd94-425f-83d2-cd44232d8cdb\") " pod="openshift-image-registry/node-ca-9d4f5" Jul 29 08:21:29.939234 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:21:29.939094 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/bb5d06c1-cbbb-4e40-9e0a-43aa22af698f-cni-binary-copy\") pod \"multus-additional-cni-plugins-qs8zl\" (UID: \"bb5d06c1-cbbb-4e40-9e0a-43aa22af698f\") " pod="openshift-multus/multus-additional-cni-plugins-qs8zl" Jul 29 08:21:29.939234 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:21:29.939149 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/2b20d931-6631-4e41-aff5-ed22c5fdafa7-multus-conf-dir\") pod \"multus-v5xrb\" (UID: \"2b20d931-6631-4e41-aff5-ed22c5fdafa7\") " pod="openshift-multus/multus-v5xrb" Jul 29 08:21:29.940068 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:21:29.939237 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/bc118958-c5f8-44b8-80a5-bfc2d80fad1d-tmp-dir\") pod \"node-resolver-bgb7d\" (UID: \"bc118958-c5f8-44b8-80a5-bfc2d80fad1d\") " pod="openshift-dns/node-resolver-bgb7d" Jul 29 08:21:29.940068 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:21:29.939391 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/93ba91b4-4c35-43be-9094-08b0b3a473b3-etc-selinux\") pod \"aws-ebs-csi-driver-node-52bkd\" (UID: \"93ba91b4-4c35-43be-9094-08b0b3a473b3\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-52bkd" Jul 29 08:21:29.940068 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:21:29.939453 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/30ff3fec-359e-4b75-82eb-9f1a194766f7-etc-sysctl-conf\") pod \"tuned-95gmk\" (UID: \"30ff3fec-359e-4b75-82eb-9f1a194766f7\") " pod="openshift-cluster-node-tuning-operator/tuned-95gmk" Jul 29 08:21:29.940068 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:21:29.939489 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/30ff3fec-359e-4b75-82eb-9f1a194766f7-etc-modprobe-d\") pod \"tuned-95gmk\" (UID: \"30ff3fec-359e-4b75-82eb-9f1a194766f7\") " pod="openshift-cluster-node-tuning-operator/tuned-95gmk" Jul 29 08:21:29.940068 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:21:29.939536 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/bb5d06c1-cbbb-4e40-9e0a-43aa22af698f-cnibin\") pod \"multus-additional-cni-plugins-qs8zl\" (UID: \"bb5d06c1-cbbb-4e40-9e0a-43aa22af698f\") " pod="openshift-multus/multus-additional-cni-plugins-qs8zl" Jul 29 08:21:29.940068 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:21:29.939551 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/2b20d931-6631-4e41-aff5-ed22c5fdafa7-multus-daemon-config\") pod \"multus-v5xrb\" (UID: \"2b20d931-6631-4e41-aff5-ed22c5fdafa7\") " pod="openshift-multus/multus-v5xrb" Jul 29 08:21:29.940068 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:21:29.939675 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/bb5d06c1-cbbb-4e40-9e0a-43aa22af698f-tuning-conf-dir\") pod \"multus-additional-cni-plugins-qs8zl\" (UID: \"bb5d06c1-cbbb-4e40-9e0a-43aa22af698f\") " pod="openshift-multus/multus-additional-cni-plugins-qs8zl" Jul 29 08:21:29.940068 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:21:29.939724 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/2b20d931-6631-4e41-aff5-ed22c5fdafa7-host-var-lib-cni-multus\") pod \"multus-v5xrb\" (UID: \"2b20d931-6631-4e41-aff5-ed22c5fdafa7\") " pod="openshift-multus/multus-v5xrb" Jul 29 08:21:29.940068 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:21:29.939798 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/93ba91b4-4c35-43be-9094-08b0b3a473b3-sys-fs\") pod \"aws-ebs-csi-driver-node-52bkd\" (UID: \"93ba91b4-4c35-43be-9094-08b0b3a473b3\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-52bkd" Jul 29 08:21:29.940068 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:21:29.939848 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/bb5d06c1-cbbb-4e40-9e0a-43aa22af698f-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-qs8zl\" (UID: \"bb5d06c1-cbbb-4e40-9e0a-43aa22af698f\") " pod="openshift-multus/multus-additional-cni-plugins-qs8zl" Jul 29 08:21:29.940068 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:21:29.940016 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/2b20d931-6631-4e41-aff5-ed22c5fdafa7-system-cni-dir\") pod \"multus-v5xrb\" (UID: \"2b20d931-6631-4e41-aff5-ed22c5fdafa7\") " pod="openshift-multus/multus-v5xrb" Jul 29 08:21:29.940570 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:21:29.940069 2563 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory" Jul 29 08:21:29.940570 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:21:29.940121 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/2b20d931-6631-4e41-aff5-ed22c5fdafa7-multus-cni-dir\") pod \"multus-v5xrb\" (UID: \"2b20d931-6631-4e41-aff5-ed22c5fdafa7\") " pod="openshift-multus/multus-v5xrb" Jul 29 08:21:29.940570 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:21:29.940146 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/2b20d931-6631-4e41-aff5-ed22c5fdafa7-cni-binary-copy\") pod \"multus-v5xrb\" (UID: \"2b20d931-6631-4e41-aff5-ed22c5fdafa7\") " pod="openshift-multus/multus-v5xrb" Jul 29 08:21:29.940570 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:21:29.940161 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/2b20d931-6631-4e41-aff5-ed22c5fdafa7-cnibin\") pod \"multus-v5xrb\" (UID: \"2b20d931-6631-4e41-aff5-ed22c5fdafa7\") " pod="openshift-multus/multus-v5xrb" Jul 29 08:21:29.940570 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:21:29.940189 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/49600d58-879f-477d-aa43-11177f411977-node-log\") pod \"ovnkube-node-vqdfh\" (UID: \"49600d58-879f-477d-aa43-11177f411977\") " pod="openshift-ovn-kubernetes/ovnkube-node-vqdfh" Jul 29 08:21:29.940570 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:21:29.940219 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/49600d58-879f-477d-aa43-11177f411977-host-run-ovn-kubernetes\") pod \"ovnkube-node-vqdfh\" (UID: \"49600d58-879f-477d-aa43-11177f411977\") " pod="openshift-ovn-kubernetes/ovnkube-node-vqdfh" Jul 29 08:21:29.940570 ip-10-0-131-138 kubenswrapper[2563]: E0729 08:21:29.940225 2563 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 29 08:21:29.940570 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:21:29.940248 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/cfd1d8df-e6b7-4a67-81f2-5d83da541e20-konnectivity-ca\") pod \"konnectivity-agent-8rp9g\" (UID: \"cfd1d8df-e6b7-4a67-81f2-5d83da541e20\") " pod="kube-system/konnectivity-agent-8rp9g" Jul 29 08:21:29.940570 ip-10-0-131-138 kubenswrapper[2563]: E0729 08:21:29.940312 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/35e49848-94f6-43d9-b7c4-6c817dd5dde7-metrics-certs podName:35e49848-94f6-43d9-b7c4-6c817dd5dde7 nodeName:}" failed. No retries permitted until 2026-07-29 08:21:30.440281875 +0000 UTC m=+3.021570647 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/35e49848-94f6-43d9-b7c4-6c817dd5dde7-metrics-certs") pod "network-metrics-daemon-qk9kb" (UID: "35e49848-94f6-43d9-b7c4-6c817dd5dde7") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 29 08:21:29.940570 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:21:29.940330 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/30ff3fec-359e-4b75-82eb-9f1a194766f7-etc-kubernetes\") pod \"tuned-95gmk\" (UID: \"30ff3fec-359e-4b75-82eb-9f1a194766f7\") " pod="openshift-cluster-node-tuning-operator/tuned-95gmk" Jul 29 08:21:29.940570 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:21:29.940348 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/2b20d931-6631-4e41-aff5-ed22c5fdafa7-hostroot\") pod \"multus-v5xrb\" (UID: \"2b20d931-6631-4e41-aff5-ed22c5fdafa7\") " pod="openshift-multus/multus-v5xrb" Jul 29 08:21:29.940570 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:21:29.940369 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/2b20d931-6631-4e41-aff5-ed22c5fdafa7-etc-kubernetes\") pod \"multus-v5xrb\" (UID: \"2b20d931-6631-4e41-aff5-ed22c5fdafa7\") " pod="openshift-multus/multus-v5xrb" Jul 29 08:21:29.940570 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:21:29.940419 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/49600d58-879f-477d-aa43-11177f411977-ovnkube-config\") pod \"ovnkube-node-vqdfh\" (UID: \"49600d58-879f-477d-aa43-11177f411977\") " pod="openshift-ovn-kubernetes/ovnkube-node-vqdfh" Jul 29 08:21:29.940570 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:21:29.940445 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/b14e5aed-a2b3-41bf-9389-701d69342953-kubelet-config\") pod \"global-pull-secret-syncer-rn9kc\" (UID: \"b14e5aed-a2b3-41bf-9389-701d69342953\") " pod="kube-system/global-pull-secret-syncer-rn9kc" Jul 29 08:21:29.940570 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:21:29.940442 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/30ff3fec-359e-4b75-82eb-9f1a194766f7-etc-kubernetes\") pod \"tuned-95gmk\" (UID: \"30ff3fec-359e-4b75-82eb-9f1a194766f7\") " pod="openshift-cluster-node-tuning-operator/tuned-95gmk" Jul 29 08:21:29.940570 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:21:29.940461 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/2b20d931-6631-4e41-aff5-ed22c5fdafa7-hostroot\") pod \"multus-v5xrb\" (UID: \"2b20d931-6631-4e41-aff5-ed22c5fdafa7\") " pod="openshift-multus/multus-v5xrb" Jul 29 08:21:29.940570 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:21:29.940469 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/5bce82ca-dd94-425f-83d2-cd44232d8cdb-host\") pod \"node-ca-9d4f5\" (UID: \"5bce82ca-dd94-425f-83d2-cd44232d8cdb\") " pod="openshift-image-registry/node-ca-9d4f5" Jul 29 08:21:29.940570 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:21:29.940512 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/5bce82ca-dd94-425f-83d2-cd44232d8cdb-host\") pod \"node-ca-9d4f5\" (UID: \"5bce82ca-dd94-425f-83d2-cd44232d8cdb\") " pod="openshift-image-registry/node-ca-9d4f5" Jul 29 08:21:29.941366 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:21:29.940522 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/bb5d06c1-cbbb-4e40-9e0a-43aa22af698f-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-qs8zl\" (UID: \"bb5d06c1-cbbb-4e40-9e0a-43aa22af698f\") " pod="openshift-multus/multus-additional-cni-plugins-qs8zl" Jul 29 08:21:29.941366 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:21:29.940546 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/2b20d931-6631-4e41-aff5-ed22c5fdafa7-etc-kubernetes\") pod \"multus-v5xrb\" (UID: \"2b20d931-6631-4e41-aff5-ed22c5fdafa7\") " pod="openshift-multus/multus-v5xrb" Jul 29 08:21:29.941366 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:21:29.940561 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/2b20d931-6631-4e41-aff5-ed22c5fdafa7-host-run-netns\") pod \"multus-v5xrb\" (UID: \"2b20d931-6631-4e41-aff5-ed22c5fdafa7\") " pod="openshift-multus/multus-v5xrb" Jul 29 08:21:29.941366 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:21:29.940646 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/49600d58-879f-477d-aa43-11177f411977-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-vqdfh\" (UID: \"49600d58-879f-477d-aa43-11177f411977\") " pod="openshift-ovn-kubernetes/ovnkube-node-vqdfh" Jul 29 08:21:29.941366 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:21:29.940676 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/2b20d931-6631-4e41-aff5-ed22c5fdafa7-host-run-netns\") pod \"multus-v5xrb\" (UID: \"2b20d931-6631-4e41-aff5-ed22c5fdafa7\") " pod="openshift-multus/multus-v5xrb" Jul 29 08:21:29.941366 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:21:29.940716 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/30ff3fec-359e-4b75-82eb-9f1a194766f7-tmp\") pod \"tuned-95gmk\" (UID: \"30ff3fec-359e-4b75-82eb-9f1a194766f7\") " pod="openshift-cluster-node-tuning-operator/tuned-95gmk" Jul 29 08:21:29.941366 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:21:29.940747 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/2b20d931-6631-4e41-aff5-ed22c5fdafa7-host-var-lib-cni-bin\") pod \"multus-v5xrb\" (UID: \"2b20d931-6631-4e41-aff5-ed22c5fdafa7\") " pod="openshift-multus/multus-v5xrb" Jul 29 08:21:29.941366 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:21:29.940775 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/49600d58-879f-477d-aa43-11177f411977-run-ovn\") pod \"ovnkube-node-vqdfh\" (UID: \"49600d58-879f-477d-aa43-11177f411977\") " pod="openshift-ovn-kubernetes/ovnkube-node-vqdfh" Jul 29 08:21:29.941366 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:21:29.940799 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/49600d58-879f-477d-aa43-11177f411977-host-cni-netd\") pod \"ovnkube-node-vqdfh\" (UID: \"49600d58-879f-477d-aa43-11177f411977\") " pod="openshift-ovn-kubernetes/ovnkube-node-vqdfh" Jul 29 08:21:29.941366 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:21:29.940811 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/2b20d931-6631-4e41-aff5-ed22c5fdafa7-host-var-lib-cni-bin\") pod \"multus-v5xrb\" (UID: \"2b20d931-6631-4e41-aff5-ed22c5fdafa7\") " pod="openshift-multus/multus-v5xrb" Jul 29 08:21:29.941366 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:21:29.940824 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/93ba91b4-4c35-43be-9094-08b0b3a473b3-socket-dir\") pod \"aws-ebs-csi-driver-node-52bkd\" (UID: \"93ba91b4-4c35-43be-9094-08b0b3a473b3\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-52bkd" Jul 29 08:21:29.941366 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:21:29.940848 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/49600d58-879f-477d-aa43-11177f411977-host-run-netns\") pod \"ovnkube-node-vqdfh\" (UID: \"49600d58-879f-477d-aa43-11177f411977\") " pod="openshift-ovn-kubernetes/ovnkube-node-vqdfh" Jul 29 08:21:29.941366 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:21:29.940872 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/30ff3fec-359e-4b75-82eb-9f1a194766f7-etc-sysconfig\") pod \"tuned-95gmk\" (UID: \"30ff3fec-359e-4b75-82eb-9f1a194766f7\") " pod="openshift-cluster-node-tuning-operator/tuned-95gmk" Jul 29 08:21:29.941366 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:21:29.940895 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/30ff3fec-359e-4b75-82eb-9f1a194766f7-run\") pod \"tuned-95gmk\" (UID: \"30ff3fec-359e-4b75-82eb-9f1a194766f7\") " pod="openshift-cluster-node-tuning-operator/tuned-95gmk" Jul 29 08:21:29.941366 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:21:29.940919 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/30ff3fec-359e-4b75-82eb-9f1a194766f7-lib-modules\") pod \"tuned-95gmk\" (UID: \"30ff3fec-359e-4b75-82eb-9f1a194766f7\") " pod="openshift-cluster-node-tuning-operator/tuned-95gmk" Jul 29 08:21:29.941366 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:21:29.940937 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/93ba91b4-4c35-43be-9094-08b0b3a473b3-socket-dir\") pod \"aws-ebs-csi-driver-node-52bkd\" (UID: \"93ba91b4-4c35-43be-9094-08b0b3a473b3\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-52bkd" Jul 29 08:21:29.941366 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:21:29.940943 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/49600d58-879f-477d-aa43-11177f411977-host-slash\") pod \"ovnkube-node-vqdfh\" (UID: \"49600d58-879f-477d-aa43-11177f411977\") " pod="openshift-ovn-kubernetes/ovnkube-node-vqdfh" Jul 29 08:21:29.943160 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:21:29.940976 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/49600d58-879f-477d-aa43-11177f411977-var-lib-openvswitch\") pod \"ovnkube-node-vqdfh\" (UID: \"49600d58-879f-477d-aa43-11177f411977\") " pod="openshift-ovn-kubernetes/ovnkube-node-vqdfh" Jul 29 08:21:29.943160 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:21:29.941000 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/49600d58-879f-477d-aa43-11177f411977-env-overrides\") pod \"ovnkube-node-vqdfh\" (UID: \"49600d58-879f-477d-aa43-11177f411977\") " pod="openshift-ovn-kubernetes/ovnkube-node-vqdfh" Jul 29 08:21:29.943160 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:21:29.941020 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/30ff3fec-359e-4b75-82eb-9f1a194766f7-etc-sysconfig\") pod \"tuned-95gmk\" (UID: \"30ff3fec-359e-4b75-82eb-9f1a194766f7\") " pod="openshift-cluster-node-tuning-operator/tuned-95gmk" Jul 29 08:21:29.943160 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:21:29.941030 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/93ba91b4-4c35-43be-9094-08b0b3a473b3-kubelet-dir\") pod \"aws-ebs-csi-driver-node-52bkd\" (UID: \"93ba91b4-4c35-43be-9094-08b0b3a473b3\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-52bkd" Jul 29 08:21:29.943160 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:21:29.941044 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/bb5d06c1-cbbb-4e40-9e0a-43aa22af698f-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-qs8zl\" (UID: \"bb5d06c1-cbbb-4e40-9e0a-43aa22af698f\") " pod="openshift-multus/multus-additional-cni-plugins-qs8zl" Jul 29 08:21:29.943160 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:21:29.941057 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/93ba91b4-4c35-43be-9094-08b0b3a473b3-registration-dir\") pod \"aws-ebs-csi-driver-node-52bkd\" (UID: \"93ba91b4-4c35-43be-9094-08b0b3a473b3\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-52bkd" Jul 29 08:21:29.943160 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:21:29.941082 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/30ff3fec-359e-4b75-82eb-9f1a194766f7-sys\") pod \"tuned-95gmk\" (UID: \"30ff3fec-359e-4b75-82eb-9f1a194766f7\") " pod="openshift-cluster-node-tuning-operator/tuned-95gmk" Jul 29 08:21:29.943160 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:21:29.941105 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/30ff3fec-359e-4b75-82eb-9f1a194766f7-run\") pod \"tuned-95gmk\" (UID: \"30ff3fec-359e-4b75-82eb-9f1a194766f7\") " pod="openshift-cluster-node-tuning-operator/tuned-95gmk" Jul 29 08:21:29.943160 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:21:29.941138 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/30ff3fec-359e-4b75-82eb-9f1a194766f7-lib-modules\") pod \"tuned-95gmk\" (UID: \"30ff3fec-359e-4b75-82eb-9f1a194766f7\") " pod="openshift-cluster-node-tuning-operator/tuned-95gmk" Jul 29 08:21:29.943160 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:21:29.941147 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/93ba91b4-4c35-43be-9094-08b0b3a473b3-registration-dir\") pod \"aws-ebs-csi-driver-node-52bkd\" (UID: \"93ba91b4-4c35-43be-9094-08b0b3a473b3\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-52bkd" Jul 29 08:21:29.943160 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:21:29.941106 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/30ff3fec-359e-4b75-82eb-9f1a194766f7-var-lib-kubelet\") pod \"tuned-95gmk\" (UID: \"30ff3fec-359e-4b75-82eb-9f1a194766f7\") " pod="openshift-cluster-node-tuning-operator/tuned-95gmk" Jul 29 08:21:29.943160 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:21:29.941179 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/2b20d931-6631-4e41-aff5-ed22c5fdafa7-os-release\") pod \"multus-v5xrb\" (UID: \"2b20d931-6631-4e41-aff5-ed22c5fdafa7\") " pod="openshift-multus/multus-v5xrb" Jul 29 08:21:29.943160 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:21:29.941185 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/93ba91b4-4c35-43be-9094-08b0b3a473b3-kubelet-dir\") pod \"aws-ebs-csi-driver-node-52bkd\" (UID: \"93ba91b4-4c35-43be-9094-08b0b3a473b3\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-52bkd" Jul 29 08:21:29.943160 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:21:29.941191 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/30ff3fec-359e-4b75-82eb-9f1a194766f7-var-lib-kubelet\") pod \"tuned-95gmk\" (UID: \"30ff3fec-359e-4b75-82eb-9f1a194766f7\") " pod="openshift-cluster-node-tuning-operator/tuned-95gmk" Jul 29 08:21:29.943160 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:21:29.941150 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/30ff3fec-359e-4b75-82eb-9f1a194766f7-sys\") pod \"tuned-95gmk\" (UID: \"30ff3fec-359e-4b75-82eb-9f1a194766f7\") " pod="openshift-cluster-node-tuning-operator/tuned-95gmk" Jul 29 08:21:29.943160 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:21:29.941204 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/30ff3fec-359e-4b75-82eb-9f1a194766f7-etc-systemd\") pod \"tuned-95gmk\" (UID: \"30ff3fec-359e-4b75-82eb-9f1a194766f7\") " pod="openshift-cluster-node-tuning-operator/tuned-95gmk" Jul 29 08:21:29.943160 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:21:29.941230 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-55kqw\" (UniqueName: \"kubernetes.io/projected/2b20d931-6631-4e41-aff5-ed22c5fdafa7-kube-api-access-55kqw\") pod \"multus-v5xrb\" (UID: \"2b20d931-6631-4e41-aff5-ed22c5fdafa7\") " pod="openshift-multus/multus-v5xrb" Jul 29 08:21:29.943967 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:21:29.941246 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/2b20d931-6631-4e41-aff5-ed22c5fdafa7-os-release\") pod \"multus-v5xrb\" (UID: \"2b20d931-6631-4e41-aff5-ed22c5fdafa7\") " pod="openshift-multus/multus-v5xrb" Jul 29 08:21:29.943967 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:21:29.941276 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/db249d48-ad47-4cc0-8a37-37078055ed38-iptables-alerter-script\") pod \"iptables-alerter-z2c22\" (UID: \"db249d48-ad47-4cc0-8a37-37078055ed38\") " pod="openshift-network-operator/iptables-alerter-z2c22" Jul 29 08:21:29.943967 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:21:29.941308 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/30ff3fec-359e-4b75-82eb-9f1a194766f7-etc-systemd\") pod \"tuned-95gmk\" (UID: \"30ff3fec-359e-4b75-82eb-9f1a194766f7\") " pod="openshift-cluster-node-tuning-operator/tuned-95gmk" Jul 29 08:21:29.943967 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:21:29.941413 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/db249d48-ad47-4cc0-8a37-37078055ed38-host-slash\") pod \"iptables-alerter-z2c22\" (UID: \"db249d48-ad47-4cc0-8a37-37078055ed38\") " pod="openshift-network-operator/iptables-alerter-z2c22" Jul 29 08:21:29.943967 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:21:29.941442 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/49600d58-879f-477d-aa43-11177f411977-run-openvswitch\") pod \"ovnkube-node-vqdfh\" (UID: \"49600d58-879f-477d-aa43-11177f411977\") " pod="openshift-ovn-kubernetes/ovnkube-node-vqdfh" Jul 29 08:21:29.943967 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:21:29.941482 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/db249d48-ad47-4cc0-8a37-37078055ed38-host-slash\") pod \"iptables-alerter-z2c22\" (UID: \"db249d48-ad47-4cc0-8a37-37078055ed38\") " pod="openshift-network-operator/iptables-alerter-z2c22" Jul 29 08:21:29.943967 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:21:29.941516 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5bdcj\" (UniqueName: \"kubernetes.io/projected/49600d58-879f-477d-aa43-11177f411977-kube-api-access-5bdcj\") pod \"ovnkube-node-vqdfh\" (UID: \"49600d58-879f-477d-aa43-11177f411977\") " pod="openshift-ovn-kubernetes/ovnkube-node-vqdfh" Jul 29 08:21:29.943967 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:21:29.941548 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-b4tfp\" (UniqueName: \"kubernetes.io/projected/bc118958-c5f8-44b8-80a5-bfc2d80fad1d-kube-api-access-b4tfp\") pod \"node-resolver-bgb7d\" (UID: \"bc118958-c5f8-44b8-80a5-bfc2d80fad1d\") " pod="openshift-dns/node-resolver-bgb7d" Jul 29 08:21:29.943967 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:21:29.941748 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/db249d48-ad47-4cc0-8a37-37078055ed38-iptables-alerter-script\") pod \"iptables-alerter-z2c22\" (UID: \"db249d48-ad47-4cc0-8a37-37078055ed38\") " pod="openshift-network-operator/iptables-alerter-z2c22" Jul 29 08:21:29.944753 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:21:29.944734 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/30ff3fec-359e-4b75-82eb-9f1a194766f7-etc-tuned\") pod \"tuned-95gmk\" (UID: \"30ff3fec-359e-4b75-82eb-9f1a194766f7\") " pod="openshift-cluster-node-tuning-operator/tuned-95gmk" Jul 29 08:21:29.946624 ip-10-0-131-138 kubenswrapper[2563]: E0729 08:21:29.945503 2563 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 29 08:21:29.946624 ip-10-0-131-138 kubenswrapper[2563]: E0729 08:21:29.945532 2563 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 29 08:21:29.946624 ip-10-0-131-138 kubenswrapper[2563]: E0729 08:21:29.945545 2563 projected.go:196] Error preparing data for projected volume kube-api-access-nxn6h for pod openshift-network-diagnostics/network-check-target-rdclp: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 29 08:21:29.946624 ip-10-0-131-138 kubenswrapper[2563]: E0729 08:21:29.945602 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/ce06e4b8-c4ec-4edf-a9c4-d7d86a03d844-kube-api-access-nxn6h podName:ce06e4b8-c4ec-4edf-a9c4-d7d86a03d844 nodeName:}" failed. No retries permitted until 2026-07-29 08:21:30.445585244 +0000 UTC m=+3.026874019 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-nxn6h" (UniqueName: "kubernetes.io/projected/ce06e4b8-c4ec-4edf-a9c4-d7d86a03d844-kube-api-access-nxn6h") pod "network-check-target-rdclp" (UID: "ce06e4b8-c4ec-4edf-a9c4-d7d86a03d844") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 29 08:21:29.947098 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:21:29.946894 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/30ff3fec-359e-4b75-82eb-9f1a194766f7-tmp\") pod \"tuned-95gmk\" (UID: \"30ff3fec-359e-4b75-82eb-9f1a194766f7\") " pod="openshift-cluster-node-tuning-operator/tuned-95gmk" Jul 29 08:21:29.950472 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:21:29.950405 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2fnff\" (UniqueName: \"kubernetes.io/projected/db249d48-ad47-4cc0-8a37-37078055ed38-kube-api-access-2fnff\") pod \"iptables-alerter-z2c22\" (UID: \"db249d48-ad47-4cc0-8a37-37078055ed38\") " pod="openshift-network-operator/iptables-alerter-z2c22" Jul 29 08:21:29.951469 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:21:29.951405 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-b4tfp\" (UniqueName: \"kubernetes.io/projected/bc118958-c5f8-44b8-80a5-bfc2d80fad1d-kube-api-access-b4tfp\") pod \"node-resolver-bgb7d\" (UID: \"bc118958-c5f8-44b8-80a5-bfc2d80fad1d\") " pod="openshift-dns/node-resolver-bgb7d" Jul 29 08:21:29.952839 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:21:29.952815 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-55kqw\" (UniqueName: \"kubernetes.io/projected/2b20d931-6631-4e41-aff5-ed22c5fdafa7-kube-api-access-55kqw\") pod \"multus-v5xrb\" (UID: \"2b20d931-6631-4e41-aff5-ed22c5fdafa7\") " pod="openshift-multus/multus-v5xrb" Jul 29 08:21:29.953637 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:21:29.953609 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-pvwwt\" (UniqueName: \"kubernetes.io/projected/5bce82ca-dd94-425f-83d2-cd44232d8cdb-kube-api-access-pvwwt\") pod \"node-ca-9d4f5\" (UID: \"5bce82ca-dd94-425f-83d2-cd44232d8cdb\") " pod="openshift-image-registry/node-ca-9d4f5" Jul 29 08:21:29.954439 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:21:29.954400 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tdnsd\" (UniqueName: \"kubernetes.io/projected/93ba91b4-4c35-43be-9094-08b0b3a473b3-kube-api-access-tdnsd\") pod \"aws-ebs-csi-driver-node-52bkd\" (UID: \"93ba91b4-4c35-43be-9094-08b0b3a473b3\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-52bkd" Jul 29 08:21:29.955106 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:21:29.954810 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ptjkx\" (UniqueName: \"kubernetes.io/projected/30ff3fec-359e-4b75-82eb-9f1a194766f7-kube-api-access-ptjkx\") pod \"tuned-95gmk\" (UID: \"30ff3fec-359e-4b75-82eb-9f1a194766f7\") " pod="openshift-cluster-node-tuning-operator/tuned-95gmk" Jul 29 08:21:29.955106 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:21:29.955014 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8dzmg\" (UniqueName: \"kubernetes.io/projected/35e49848-94f6-43d9-b7c4-6c817dd5dde7-kube-api-access-8dzmg\") pod \"network-metrics-daemon-qk9kb\" (UID: \"35e49848-94f6-43d9-b7c4-6c817dd5dde7\") " pod="openshift-multus/network-metrics-daemon-qk9kb" Jul 29 08:21:29.955106 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:21:29.955027 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vsl94\" (UniqueName: \"kubernetes.io/projected/bb5d06c1-cbbb-4e40-9e0a-43aa22af698f-kube-api-access-vsl94\") pod \"multus-additional-cni-plugins-qs8zl\" (UID: \"bb5d06c1-cbbb-4e40-9e0a-43aa22af698f\") " pod="openshift-multus/multus-additional-cni-plugins-qs8zl" Jul 29 08:21:29.963335 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:21:29.963281 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-131-138.ec2.internal" event={"ID":"99103139fa564168399467f94d133369","Type":"ContainerStarted","Data":"5886551be8113ceaf8ded0289ac287837f19a5a20e3a3374da6d0ced569fc249"} Jul 29 08:21:29.964759 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:21:29.964733 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-138.ec2.internal" event={"ID":"9f8ab28df5c602f62a77b9c2aed7b570","Type":"ContainerStarted","Data":"6d9b70b2bafedfc223766a379c31cad15edf6121cdc08f1418e9d90bfb3f5c2f"} Jul 29 08:21:30.041892 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:21:30.041853 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/49600d58-879f-477d-aa43-11177f411977-run-systemd\") pod \"ovnkube-node-vqdfh\" (UID: \"49600d58-879f-477d-aa43-11177f411977\") " pod="openshift-ovn-kubernetes/ovnkube-node-vqdfh" Jul 29 08:21:30.041892 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:21:30.041900 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/49600d58-879f-477d-aa43-11177f411977-log-socket\") pod \"ovnkube-node-vqdfh\" (UID: \"49600d58-879f-477d-aa43-11177f411977\") " pod="openshift-ovn-kubernetes/ovnkube-node-vqdfh" Jul 29 08:21:30.042131 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:21:30.041921 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/49600d58-879f-477d-aa43-11177f411977-ovnkube-script-lib\") pod \"ovnkube-node-vqdfh\" (UID: \"49600d58-879f-477d-aa43-11177f411977\") " pod="openshift-ovn-kubernetes/ovnkube-node-vqdfh" Jul 29 08:21:30.042131 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:21:30.041948 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/b14e5aed-a2b3-41bf-9389-701d69342953-dbus\") pod \"global-pull-secret-syncer-rn9kc\" (UID: \"b14e5aed-a2b3-41bf-9389-701d69342953\") " pod="kube-system/global-pull-secret-syncer-rn9kc" Jul 29 08:21:30.042131 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:21:30.041986 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/49600d58-879f-477d-aa43-11177f411977-host-cni-bin\") pod \"ovnkube-node-vqdfh\" (UID: \"49600d58-879f-477d-aa43-11177f411977\") " pod="openshift-ovn-kubernetes/ovnkube-node-vqdfh" Jul 29 08:21:30.042131 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:21:30.042012 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/49600d58-879f-477d-aa43-11177f411977-systemd-units\") pod \"ovnkube-node-vqdfh\" (UID: \"49600d58-879f-477d-aa43-11177f411977\") " pod="openshift-ovn-kubernetes/ovnkube-node-vqdfh" Jul 29 08:21:30.042131 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:21:30.042034 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/b14e5aed-a2b3-41bf-9389-701d69342953-original-pull-secret\") pod \"global-pull-secret-syncer-rn9kc\" (UID: \"b14e5aed-a2b3-41bf-9389-701d69342953\") " pod="kube-system/global-pull-secret-syncer-rn9kc" Jul 29 08:21:30.042131 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:21:30.042074 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/49600d58-879f-477d-aa43-11177f411977-host-kubelet\") pod \"ovnkube-node-vqdfh\" (UID: \"49600d58-879f-477d-aa43-11177f411977\") " pod="openshift-ovn-kubernetes/ovnkube-node-vqdfh" Jul 29 08:21:30.042131 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:21:30.042101 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/49600d58-879f-477d-aa43-11177f411977-node-log\") pod \"ovnkube-node-vqdfh\" (UID: \"49600d58-879f-477d-aa43-11177f411977\") " pod="openshift-ovn-kubernetes/ovnkube-node-vqdfh" Jul 29 08:21:30.042131 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:21:30.042123 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/49600d58-879f-477d-aa43-11177f411977-host-run-ovn-kubernetes\") pod \"ovnkube-node-vqdfh\" (UID: \"49600d58-879f-477d-aa43-11177f411977\") " pod="openshift-ovn-kubernetes/ovnkube-node-vqdfh" Jul 29 08:21:30.042453 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:21:30.042140 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/cfd1d8df-e6b7-4a67-81f2-5d83da541e20-konnectivity-ca\") pod \"konnectivity-agent-8rp9g\" (UID: \"cfd1d8df-e6b7-4a67-81f2-5d83da541e20\") " pod="kube-system/konnectivity-agent-8rp9g" Jul 29 08:21:30.042453 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:21:30.042164 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/49600d58-879f-477d-aa43-11177f411977-ovnkube-config\") pod \"ovnkube-node-vqdfh\" (UID: \"49600d58-879f-477d-aa43-11177f411977\") " pod="openshift-ovn-kubernetes/ovnkube-node-vqdfh" Jul 29 08:21:30.042453 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:21:30.042185 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/b14e5aed-a2b3-41bf-9389-701d69342953-kubelet-config\") pod \"global-pull-secret-syncer-rn9kc\" (UID: \"b14e5aed-a2b3-41bf-9389-701d69342953\") " pod="kube-system/global-pull-secret-syncer-rn9kc" Jul 29 08:21:30.042453 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:21:30.042204 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/49600d58-879f-477d-aa43-11177f411977-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-vqdfh\" (UID: \"49600d58-879f-477d-aa43-11177f411977\") " pod="openshift-ovn-kubernetes/ovnkube-node-vqdfh" Jul 29 08:21:30.042453 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:21:30.042222 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/49600d58-879f-477d-aa43-11177f411977-run-ovn\") pod \"ovnkube-node-vqdfh\" (UID: \"49600d58-879f-477d-aa43-11177f411977\") " pod="openshift-ovn-kubernetes/ovnkube-node-vqdfh" Jul 29 08:21:30.042453 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:21:30.042241 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/49600d58-879f-477d-aa43-11177f411977-host-cni-netd\") pod \"ovnkube-node-vqdfh\" (UID: \"49600d58-879f-477d-aa43-11177f411977\") " pod="openshift-ovn-kubernetes/ovnkube-node-vqdfh" Jul 29 08:21:30.042453 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:21:30.042282 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/49600d58-879f-477d-aa43-11177f411977-host-run-netns\") pod \"ovnkube-node-vqdfh\" (UID: \"49600d58-879f-477d-aa43-11177f411977\") " pod="openshift-ovn-kubernetes/ovnkube-node-vqdfh" Jul 29 08:21:30.042453 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:21:30.042310 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/49600d58-879f-477d-aa43-11177f411977-host-slash\") pod \"ovnkube-node-vqdfh\" (UID: \"49600d58-879f-477d-aa43-11177f411977\") " pod="openshift-ovn-kubernetes/ovnkube-node-vqdfh" Jul 29 08:21:30.042453 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:21:30.042328 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/49600d58-879f-477d-aa43-11177f411977-var-lib-openvswitch\") pod \"ovnkube-node-vqdfh\" (UID: \"49600d58-879f-477d-aa43-11177f411977\") " pod="openshift-ovn-kubernetes/ovnkube-node-vqdfh" Jul 29 08:21:30.042453 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:21:30.042345 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/49600d58-879f-477d-aa43-11177f411977-env-overrides\") pod \"ovnkube-node-vqdfh\" (UID: \"49600d58-879f-477d-aa43-11177f411977\") " pod="openshift-ovn-kubernetes/ovnkube-node-vqdfh" Jul 29 08:21:30.042453 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:21:30.042370 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/49600d58-879f-477d-aa43-11177f411977-run-openvswitch\") pod \"ovnkube-node-vqdfh\" (UID: \"49600d58-879f-477d-aa43-11177f411977\") " pod="openshift-ovn-kubernetes/ovnkube-node-vqdfh" Jul 29 08:21:30.042453 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:21:30.042391 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5bdcj\" (UniqueName: \"kubernetes.io/projected/49600d58-879f-477d-aa43-11177f411977-kube-api-access-5bdcj\") pod \"ovnkube-node-vqdfh\" (UID: \"49600d58-879f-477d-aa43-11177f411977\") " pod="openshift-ovn-kubernetes/ovnkube-node-vqdfh" Jul 29 08:21:30.042453 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:21:30.042415 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/49600d58-879f-477d-aa43-11177f411977-etc-openvswitch\") pod \"ovnkube-node-vqdfh\" (UID: \"49600d58-879f-477d-aa43-11177f411977\") " pod="openshift-ovn-kubernetes/ovnkube-node-vqdfh" Jul 29 08:21:30.042453 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:21:30.042439 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/49600d58-879f-477d-aa43-11177f411977-ovn-node-metrics-cert\") pod \"ovnkube-node-vqdfh\" (UID: \"49600d58-879f-477d-aa43-11177f411977\") " pod="openshift-ovn-kubernetes/ovnkube-node-vqdfh" Jul 29 08:21:30.042453 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:21:30.042459 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/cfd1d8df-e6b7-4a67-81f2-5d83da541e20-agent-certs\") pod \"konnectivity-agent-8rp9g\" (UID: \"cfd1d8df-e6b7-4a67-81f2-5d83da541e20\") " pod="kube-system/konnectivity-agent-8rp9g" Jul 29 08:21:30.043890 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:21:30.043171 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/49600d58-879f-477d-aa43-11177f411977-host-slash\") pod \"ovnkube-node-vqdfh\" (UID: \"49600d58-879f-477d-aa43-11177f411977\") " pod="openshift-ovn-kubernetes/ovnkube-node-vqdfh" Jul 29 08:21:30.043890 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:21:30.043282 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/b14e5aed-a2b3-41bf-9389-701d69342953-kubelet-config\") pod \"global-pull-secret-syncer-rn9kc\" (UID: \"b14e5aed-a2b3-41bf-9389-701d69342953\") " pod="kube-system/global-pull-secret-syncer-rn9kc" Jul 29 08:21:30.043890 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:21:30.043331 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/49600d58-879f-477d-aa43-11177f411977-host-cni-netd\") pod \"ovnkube-node-vqdfh\" (UID: \"49600d58-879f-477d-aa43-11177f411977\") " pod="openshift-ovn-kubernetes/ovnkube-node-vqdfh" Jul 29 08:21:30.043890 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:21:30.043361 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/49600d58-879f-477d-aa43-11177f411977-systemd-units\") pod \"ovnkube-node-vqdfh\" (UID: \"49600d58-879f-477d-aa43-11177f411977\") " pod="openshift-ovn-kubernetes/ovnkube-node-vqdfh" Jul 29 08:21:30.043890 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:21:30.043367 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/49600d58-879f-477d-aa43-11177f411977-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-vqdfh\" (UID: \"49600d58-879f-477d-aa43-11177f411977\") " pod="openshift-ovn-kubernetes/ovnkube-node-vqdfh" Jul 29 08:21:30.043890 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:21:30.043378 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/49600d58-879f-477d-aa43-11177f411977-run-openvswitch\") pod \"ovnkube-node-vqdfh\" (UID: \"49600d58-879f-477d-aa43-11177f411977\") " pod="openshift-ovn-kubernetes/ovnkube-node-vqdfh" Jul 29 08:21:30.043890 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:21:30.043378 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/49600d58-879f-477d-aa43-11177f411977-host-run-netns\") pod \"ovnkube-node-vqdfh\" (UID: \"49600d58-879f-477d-aa43-11177f411977\") " pod="openshift-ovn-kubernetes/ovnkube-node-vqdfh" Jul 29 08:21:30.043890 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:21:30.043372 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/49600d58-879f-477d-aa43-11177f411977-ovnkube-config\") pod \"ovnkube-node-vqdfh\" (UID: \"49600d58-879f-477d-aa43-11177f411977\") " pod="openshift-ovn-kubernetes/ovnkube-node-vqdfh" Jul 29 08:21:30.043890 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:21:30.043418 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/49600d58-879f-477d-aa43-11177f411977-run-ovn\") pod \"ovnkube-node-vqdfh\" (UID: \"49600d58-879f-477d-aa43-11177f411977\") " pod="openshift-ovn-kubernetes/ovnkube-node-vqdfh" Jul 29 08:21:30.043890 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:21:30.043424 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/49600d58-879f-477d-aa43-11177f411977-var-lib-openvswitch\") pod \"ovnkube-node-vqdfh\" (UID: \"49600d58-879f-477d-aa43-11177f411977\") " pod="openshift-ovn-kubernetes/ovnkube-node-vqdfh" Jul 29 08:21:30.043890 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:21:30.043430 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/49600d58-879f-477d-aa43-11177f411977-run-systemd\") pod \"ovnkube-node-vqdfh\" (UID: \"49600d58-879f-477d-aa43-11177f411977\") " pod="openshift-ovn-kubernetes/ovnkube-node-vqdfh" Jul 29 08:21:30.043890 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:21:30.043465 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/49600d58-879f-477d-aa43-11177f411977-node-log\") pod \"ovnkube-node-vqdfh\" (UID: \"49600d58-879f-477d-aa43-11177f411977\") " pod="openshift-ovn-kubernetes/ovnkube-node-vqdfh" Jul 29 08:21:30.043890 ip-10-0-131-138 kubenswrapper[2563]: E0729 08:21:30.043472 2563 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 29 08:21:30.043890 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:21:30.043467 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/49600d58-879f-477d-aa43-11177f411977-etc-openvswitch\") pod \"ovnkube-node-vqdfh\" (UID: \"49600d58-879f-477d-aa43-11177f411977\") " pod="openshift-ovn-kubernetes/ovnkube-node-vqdfh" Jul 29 08:21:30.043890 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:21:30.043496 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/49600d58-879f-477d-aa43-11177f411977-host-cni-bin\") pod \"ovnkube-node-vqdfh\" (UID: \"49600d58-879f-477d-aa43-11177f411977\") " pod="openshift-ovn-kubernetes/ovnkube-node-vqdfh" Jul 29 08:21:30.043890 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:21:30.043522 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/49600d58-879f-477d-aa43-11177f411977-host-kubelet\") pod \"ovnkube-node-vqdfh\" (UID: \"49600d58-879f-477d-aa43-11177f411977\") " pod="openshift-ovn-kubernetes/ovnkube-node-vqdfh" Jul 29 08:21:30.043890 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:21:30.043543 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/49600d58-879f-477d-aa43-11177f411977-host-run-ovn-kubernetes\") pod \"ovnkube-node-vqdfh\" (UID: \"49600d58-879f-477d-aa43-11177f411977\") " pod="openshift-ovn-kubernetes/ovnkube-node-vqdfh" Jul 29 08:21:30.044837 ip-10-0-131-138 kubenswrapper[2563]: E0729 08:21:30.043558 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b14e5aed-a2b3-41bf-9389-701d69342953-original-pull-secret podName:b14e5aed-a2b3-41bf-9389-701d69342953 nodeName:}" failed. No retries permitted until 2026-07-29 08:21:30.543522523 +0000 UTC m=+3.124811281 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/b14e5aed-a2b3-41bf-9389-701d69342953-original-pull-secret") pod "global-pull-secret-syncer-rn9kc" (UID: "b14e5aed-a2b3-41bf-9389-701d69342953") : object "kube-system"/"original-pull-secret" not registered Jul 29 08:21:30.044837 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:21:30.043801 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/49600d58-879f-477d-aa43-11177f411977-env-overrides\") pod \"ovnkube-node-vqdfh\" (UID: \"49600d58-879f-477d-aa43-11177f411977\") " pod="openshift-ovn-kubernetes/ovnkube-node-vqdfh" Jul 29 08:21:30.044837 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:21:30.043833 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/b14e5aed-a2b3-41bf-9389-701d69342953-dbus\") pod \"global-pull-secret-syncer-rn9kc\" (UID: \"b14e5aed-a2b3-41bf-9389-701d69342953\") " pod="kube-system/global-pull-secret-syncer-rn9kc" Jul 29 08:21:30.044837 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:21:30.043852 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/49600d58-879f-477d-aa43-11177f411977-log-socket\") pod \"ovnkube-node-vqdfh\" (UID: \"49600d58-879f-477d-aa43-11177f411977\") " pod="openshift-ovn-kubernetes/ovnkube-node-vqdfh" Jul 29 08:21:30.044837 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:21:30.044091 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/cfd1d8df-e6b7-4a67-81f2-5d83da541e20-konnectivity-ca\") pod \"konnectivity-agent-8rp9g\" (UID: \"cfd1d8df-e6b7-4a67-81f2-5d83da541e20\") " pod="kube-system/konnectivity-agent-8rp9g" Jul 29 08:21:30.044837 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:21:30.044113 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/49600d58-879f-477d-aa43-11177f411977-ovnkube-script-lib\") pod \"ovnkube-node-vqdfh\" (UID: \"49600d58-879f-477d-aa43-11177f411977\") " pod="openshift-ovn-kubernetes/ovnkube-node-vqdfh" Jul 29 08:21:30.045927 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:21:30.045901 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/cfd1d8df-e6b7-4a67-81f2-5d83da541e20-agent-certs\") pod \"konnectivity-agent-8rp9g\" (UID: \"cfd1d8df-e6b7-4a67-81f2-5d83da541e20\") " pod="kube-system/konnectivity-agent-8rp9g" Jul 29 08:21:30.046146 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:21:30.046126 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/49600d58-879f-477d-aa43-11177f411977-ovn-node-metrics-cert\") pod \"ovnkube-node-vqdfh\" (UID: \"49600d58-879f-477d-aa43-11177f411977\") " pod="openshift-ovn-kubernetes/ovnkube-node-vqdfh" Jul 29 08:21:30.052767 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:21:30.052724 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5bdcj\" (UniqueName: \"kubernetes.io/projected/49600d58-879f-477d-aa43-11177f411977-kube-api-access-5bdcj\") pod \"ovnkube-node-vqdfh\" (UID: \"49600d58-879f-477d-aa43-11177f411977\") " pod="openshift-ovn-kubernetes/ovnkube-node-vqdfh" Jul 29 08:21:30.092039 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:21:30.092007 2563 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Jul 29 08:21:30.118342 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:21:30.118058 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-52bkd" Jul 29 08:21:30.138900 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:21:30.138872 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-bgb7d" Jul 29 08:21:30.149945 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:21:30.149921 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-z2c22" Jul 29 08:21:30.164687 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:21:30.164665 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-95gmk" Jul 29 08:21:30.172437 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:21:30.172409 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-9d4f5" Jul 29 08:21:30.182272 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:21:30.182229 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-qs8zl" Jul 29 08:21:30.190971 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:21:30.190950 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-v5xrb" Jul 29 08:21:30.197822 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:21:30.197798 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-vqdfh" Jul 29 08:21:30.204568 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:21:30.204547 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-8rp9g" Jul 29 08:21:30.444449 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:21:30.444360 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/35e49848-94f6-43d9-b7c4-6c817dd5dde7-metrics-certs\") pod \"network-metrics-daemon-qk9kb\" (UID: \"35e49848-94f6-43d9-b7c4-6c817dd5dde7\") " pod="openshift-multus/network-metrics-daemon-qk9kb" Jul 29 08:21:30.444622 ip-10-0-131-138 kubenswrapper[2563]: E0729 08:21:30.444534 2563 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 29 08:21:30.444622 ip-10-0-131-138 kubenswrapper[2563]: E0729 08:21:30.444615 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/35e49848-94f6-43d9-b7c4-6c817dd5dde7-metrics-certs podName:35e49848-94f6-43d9-b7c4-6c817dd5dde7 nodeName:}" failed. No retries permitted until 2026-07-29 08:21:31.444589668 +0000 UTC m=+4.025878443 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/35e49848-94f6-43d9-b7c4-6c817dd5dde7-metrics-certs") pod "network-metrics-daemon-qk9kb" (UID: "35e49848-94f6-43d9-b7c4-6c817dd5dde7") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 29 08:21:30.544899 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:21:30.544861 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nxn6h\" (UniqueName: \"kubernetes.io/projected/ce06e4b8-c4ec-4edf-a9c4-d7d86a03d844-kube-api-access-nxn6h\") pod \"network-check-target-rdclp\" (UID: \"ce06e4b8-c4ec-4edf-a9c4-d7d86a03d844\") " pod="openshift-network-diagnostics/network-check-target-rdclp" Jul 29 08:21:30.545098 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:21:30.544916 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/b14e5aed-a2b3-41bf-9389-701d69342953-original-pull-secret\") pod \"global-pull-secret-syncer-rn9kc\" (UID: \"b14e5aed-a2b3-41bf-9389-701d69342953\") " pod="kube-system/global-pull-secret-syncer-rn9kc" Jul 29 08:21:30.545098 ip-10-0-131-138 kubenswrapper[2563]: E0729 08:21:30.545019 2563 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 29 08:21:30.545098 ip-10-0-131-138 kubenswrapper[2563]: E0729 08:21:30.545045 2563 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 29 08:21:30.545098 ip-10-0-131-138 kubenswrapper[2563]: E0729 08:21:30.545056 2563 projected.go:196] Error preparing data for projected volume kube-api-access-nxn6h for pod openshift-network-diagnostics/network-check-target-rdclp: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 29 08:21:30.545098 ip-10-0-131-138 kubenswrapper[2563]: E0729 08:21:30.545058 2563 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 29 08:21:30.545380 ip-10-0-131-138 kubenswrapper[2563]: E0729 08:21:30.545115 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/ce06e4b8-c4ec-4edf-a9c4-d7d86a03d844-kube-api-access-nxn6h podName:ce06e4b8-c4ec-4edf-a9c4-d7d86a03d844 nodeName:}" failed. No retries permitted until 2026-07-29 08:21:31.545095792 +0000 UTC m=+4.126384548 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-nxn6h" (UniqueName: "kubernetes.io/projected/ce06e4b8-c4ec-4edf-a9c4-d7d86a03d844-kube-api-access-nxn6h") pod "network-check-target-rdclp" (UID: "ce06e4b8-c4ec-4edf-a9c4-d7d86a03d844") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 29 08:21:30.545380 ip-10-0-131-138 kubenswrapper[2563]: E0729 08:21:30.545137 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b14e5aed-a2b3-41bf-9389-701d69342953-original-pull-secret podName:b14e5aed-a2b3-41bf-9389-701d69342953 nodeName:}" failed. No retries permitted until 2026-07-29 08:21:31.545126194 +0000 UTC m=+4.126414950 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/b14e5aed-a2b3-41bf-9389-701d69342953-original-pull-secret") pod "global-pull-secret-syncer-rn9kc" (UID: "b14e5aed-a2b3-41bf-9389-701d69342953") : object "kube-system"/"original-pull-secret" not registered Jul 29 08:21:30.869734 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:21:30.869699 2563 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-28 08:16:29 +0000 UTC" deadline="2028-02-01 19:02:57.815569603 +0000 UTC" Jul 29 08:21:30.869734 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:21:30.869732 2563 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="13258h41m26.945843641s" Jul 29 08:21:30.957562 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:21:30.957528 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-qk9kb" Jul 29 08:21:30.957706 ip-10-0-131-138 kubenswrapper[2563]: E0729 08:21:30.957662 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-qk9kb" podUID="35e49848-94f6-43d9-b7c4-6c817dd5dde7" Jul 29 08:21:30.957744 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:21:30.957720 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-rdclp" Jul 29 08:21:30.957873 ip-10-0-131-138 kubenswrapper[2563]: E0729 08:21:30.957854 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-rdclp" podUID="ce06e4b8-c4ec-4edf-a9c4-d7d86a03d844" Jul 29 08:21:30.991949 ip-10-0-131-138 kubenswrapper[2563]: W0729 08:21:30.991855 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod30ff3fec_359e_4b75_82eb_9f1a194766f7.slice/crio-3ede07caab13dba8153f38242503fe8ca906d81cb39595b605c4c6817b68d5b8 WatchSource:0}: Error finding container 3ede07caab13dba8153f38242503fe8ca906d81cb39595b605c4c6817b68d5b8: Status 404 returned error can't find the container with id 3ede07caab13dba8153f38242503fe8ca906d81cb39595b605c4c6817b68d5b8 Jul 29 08:21:30.995653 ip-10-0-131-138 kubenswrapper[2563]: W0729 08:21:30.995629 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddb249d48_ad47_4cc0_8a37_37078055ed38.slice/crio-fdede6719d3f37de03af799b6c2578e4a9ffc8f750549ec57b60ff11f4e1e631 WatchSource:0}: Error finding container fdede6719d3f37de03af799b6c2578e4a9ffc8f750549ec57b60ff11f4e1e631: Status 404 returned error can't find the container with id fdede6719d3f37de03af799b6c2578e4a9ffc8f750549ec57b60ff11f4e1e631 Jul 29 08:21:30.997279 ip-10-0-131-138 kubenswrapper[2563]: W0729 08:21:30.997219 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2b20d931_6631_4e41_aff5_ed22c5fdafa7.slice/crio-2d46a6a3f1dfc5864ceafef4fbe7d1f76dbbfd5a18be1b2b76580e79d7731d5a WatchSource:0}: Error finding container 2d46a6a3f1dfc5864ceafef4fbe7d1f76dbbfd5a18be1b2b76580e79d7731d5a: Status 404 returned error can't find the container with id 2d46a6a3f1dfc5864ceafef4fbe7d1f76dbbfd5a18be1b2b76580e79d7731d5a Jul 29 08:21:31.000607 ip-10-0-131-138 kubenswrapper[2563]: W0729 08:21:31.000588 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbc118958_c5f8_44b8_80a5_bfc2d80fad1d.slice/crio-91eeb3b446d21668908bc0c58f7aaba29170c9af69cc3eab8b285ba5638b6ad5 WatchSource:0}: Error finding container 91eeb3b446d21668908bc0c58f7aaba29170c9af69cc3eab8b285ba5638b6ad5: Status 404 returned error can't find the container with id 91eeb3b446d21668908bc0c58f7aaba29170c9af69cc3eab8b285ba5638b6ad5 Jul 29 08:21:31.001424 ip-10-0-131-138 kubenswrapper[2563]: W0729 08:21:31.001404 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbb5d06c1_cbbb_4e40_9e0a_43aa22af698f.slice/crio-b49a4c9a0f3d7441f0f389dff1bc37a9e12feb25b7c1e583ce35c260ca2fa318 WatchSource:0}: Error finding container b49a4c9a0f3d7441f0f389dff1bc37a9e12feb25b7c1e583ce35c260ca2fa318: Status 404 returned error can't find the container with id b49a4c9a0f3d7441f0f389dff1bc37a9e12feb25b7c1e583ce35c260ca2fa318 Jul 29 08:21:31.004649 ip-10-0-131-138 kubenswrapper[2563]: W0729 08:21:31.003960 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod93ba91b4_4c35_43be_9094_08b0b3a473b3.slice/crio-dc3a30fa698636d6508bac7f5dd1caaa69c4983d8dcde2206e9f4fd2770b3ca0 WatchSource:0}: Error finding container dc3a30fa698636d6508bac7f5dd1caaa69c4983d8dcde2206e9f4fd2770b3ca0: Status 404 returned error can't find the container with id dc3a30fa698636d6508bac7f5dd1caaa69c4983d8dcde2206e9f4fd2770b3ca0 Jul 29 08:21:31.452045 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:21:31.451775 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/35e49848-94f6-43d9-b7c4-6c817dd5dde7-metrics-certs\") pod \"network-metrics-daemon-qk9kb\" (UID: \"35e49848-94f6-43d9-b7c4-6c817dd5dde7\") " pod="openshift-multus/network-metrics-daemon-qk9kb" Jul 29 08:21:31.452045 ip-10-0-131-138 kubenswrapper[2563]: E0729 08:21:31.451951 2563 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 29 08:21:31.452299 ip-10-0-131-138 kubenswrapper[2563]: E0729 08:21:31.452107 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/35e49848-94f6-43d9-b7c4-6c817dd5dde7-metrics-certs podName:35e49848-94f6-43d9-b7c4-6c817dd5dde7 nodeName:}" failed. No retries permitted until 2026-07-29 08:21:33.452086246 +0000 UTC m=+6.033375015 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/35e49848-94f6-43d9-b7c4-6c817dd5dde7-metrics-certs") pod "network-metrics-daemon-qk9kb" (UID: "35e49848-94f6-43d9-b7c4-6c817dd5dde7") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 29 08:21:31.553135 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:21:31.553088 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nxn6h\" (UniqueName: \"kubernetes.io/projected/ce06e4b8-c4ec-4edf-a9c4-d7d86a03d844-kube-api-access-nxn6h\") pod \"network-check-target-rdclp\" (UID: \"ce06e4b8-c4ec-4edf-a9c4-d7d86a03d844\") " pod="openshift-network-diagnostics/network-check-target-rdclp" Jul 29 08:21:31.553347 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:21:31.553149 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/b14e5aed-a2b3-41bf-9389-701d69342953-original-pull-secret\") pod \"global-pull-secret-syncer-rn9kc\" (UID: \"b14e5aed-a2b3-41bf-9389-701d69342953\") " pod="kube-system/global-pull-secret-syncer-rn9kc" Jul 29 08:21:31.553347 ip-10-0-131-138 kubenswrapper[2563]: E0729 08:21:31.553292 2563 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 29 08:21:31.553524 ip-10-0-131-138 kubenswrapper[2563]: E0729 08:21:31.553353 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b14e5aed-a2b3-41bf-9389-701d69342953-original-pull-secret podName:b14e5aed-a2b3-41bf-9389-701d69342953 nodeName:}" failed. No retries permitted until 2026-07-29 08:21:33.553335088 +0000 UTC m=+6.134623859 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/b14e5aed-a2b3-41bf-9389-701d69342953-original-pull-secret") pod "global-pull-secret-syncer-rn9kc" (UID: "b14e5aed-a2b3-41bf-9389-701d69342953") : object "kube-system"/"original-pull-secret" not registered Jul 29 08:21:31.553791 ip-10-0-131-138 kubenswrapper[2563]: E0729 08:21:31.553769 2563 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 29 08:21:31.553875 ip-10-0-131-138 kubenswrapper[2563]: E0729 08:21:31.553799 2563 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 29 08:21:31.553875 ip-10-0-131-138 kubenswrapper[2563]: E0729 08:21:31.553813 2563 projected.go:196] Error preparing data for projected volume kube-api-access-nxn6h for pod openshift-network-diagnostics/network-check-target-rdclp: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 29 08:21:31.553875 ip-10-0-131-138 kubenswrapper[2563]: E0729 08:21:31.553856 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/ce06e4b8-c4ec-4edf-a9c4-d7d86a03d844-kube-api-access-nxn6h podName:ce06e4b8-c4ec-4edf-a9c4-d7d86a03d844 nodeName:}" failed. No retries permitted until 2026-07-29 08:21:33.553842745 +0000 UTC m=+6.135131507 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-nxn6h" (UniqueName: "kubernetes.io/projected/ce06e4b8-c4ec-4edf-a9c4-d7d86a03d844-kube-api-access-nxn6h") pod "network-check-target-rdclp" (UID: "ce06e4b8-c4ec-4edf-a9c4-d7d86a03d844") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 29 08:21:31.601536 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:21:31.601214 2563 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160" Jul 29 08:21:31.870150 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:21:31.870043 2563 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-28 08:16:29 +0000 UTC" deadline="2028-04-13 19:15:39.792135084 +0000 UTC" Jul 29 08:21:31.870150 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:21:31.870102 2563 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="14986h54m7.922037397s" Jul 29 08:21:31.960238 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:21:31.960203 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-rn9kc" Jul 29 08:21:31.960425 ip-10-0-131-138 kubenswrapper[2563]: E0729 08:21:31.960354 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-rn9kc" podUID="b14e5aed-a2b3-41bf-9389-701d69342953" Jul 29 08:21:31.976606 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:21:31.976571 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-z2c22" event={"ID":"db249d48-ad47-4cc0-8a37-37078055ed38","Type":"ContainerStarted","Data":"fdede6719d3f37de03af799b6c2578e4a9ffc8f750549ec57b60ff11f4e1e631"} Jul 29 08:21:31.984139 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:21:31.984104 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-131-138.ec2.internal" event={"ID":"99103139fa564168399467f94d133369","Type":"ContainerStarted","Data":"ea7b634fb580d3f42933e6f591277eb8ea5cef40944908b3d3a411628102a9d6"} Jul 29 08:21:31.993662 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:21:31.993627 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-52bkd" event={"ID":"93ba91b4-4c35-43be-9094-08b0b3a473b3","Type":"ContainerStarted","Data":"dc3a30fa698636d6508bac7f5dd1caaa69c4983d8dcde2206e9f4fd2770b3ca0"} Jul 29 08:21:32.000873 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:21:32.000844 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-bgb7d" event={"ID":"bc118958-c5f8-44b8-80a5-bfc2d80fad1d","Type":"ContainerStarted","Data":"91eeb3b446d21668908bc0c58f7aaba29170c9af69cc3eab8b285ba5638b6ad5"} Jul 29 08:21:32.004152 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:21:32.004125 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-v5xrb" event={"ID":"2b20d931-6631-4e41-aff5-ed22c5fdafa7","Type":"ContainerStarted","Data":"2d46a6a3f1dfc5864ceafef4fbe7d1f76dbbfd5a18be1b2b76580e79d7731d5a"} Jul 29 08:21:32.005410 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:21:32.004929 2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-131-138.ec2.internal" podStartSLOduration=3.004912731 podStartE2EDuration="3.004912731s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-29 08:21:29 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 08:21:32.004214263 +0000 UTC m=+4.585503042" watchObservedRunningTime="2026-07-29 08:21:32.004912731 +0000 UTC m=+4.586201510" Jul 29 08:21:32.016698 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:21:32.016641 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-9d4f5" event={"ID":"5bce82ca-dd94-425f-83d2-cd44232d8cdb","Type":"ContainerStarted","Data":"1f340e7e2b4ea956b6cc7867b620ae786a535f792c78b41850f0d0e4e69c116a"} Jul 29 08:21:32.022563 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:21:32.022482 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-8rp9g" event={"ID":"cfd1d8df-e6b7-4a67-81f2-5d83da541e20","Type":"ContainerStarted","Data":"25d60a3e1208a218516512506b3261cd90eadae6595e6a010303fc9b42d70e27"} Jul 29 08:21:32.035556 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:21:32.035529 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-95gmk" event={"ID":"30ff3fec-359e-4b75-82eb-9f1a194766f7","Type":"ContainerStarted","Data":"3ede07caab13dba8153f38242503fe8ca906d81cb39595b605c4c6817b68d5b8"} Jul 29 08:21:32.047004 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:21:32.046799 2563 generic.go:356] "Generic (PLEG): container finished" podID="9f8ab28df5c602f62a77b9c2aed7b570" containerID="c27b011553660c91cace03d67ccf185f3dbedca461610a1cabf9589970d4b6b4" exitCode=0 Jul 29 08:21:32.047004 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:21:32.046875 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-138.ec2.internal" event={"ID":"9f8ab28df5c602f62a77b9c2aed7b570","Type":"ContainerDied","Data":"c27b011553660c91cace03d67ccf185f3dbedca461610a1cabf9589970d4b6b4"} Jul 29 08:21:32.066828 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:21:32.066802 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-vqdfh" event={"ID":"49600d58-879f-477d-aa43-11177f411977","Type":"ContainerStarted","Data":"3831116ee96213241f40d1e394cf7aeff2ff4b92a5051f9928f64f6ab36e46fe"} Jul 29 08:21:32.071331 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:21:32.071302 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-qs8zl" event={"ID":"bb5d06c1-cbbb-4e40-9e0a-43aa22af698f","Type":"ContainerStarted","Data":"b49a4c9a0f3d7441f0f389dff1bc37a9e12feb25b7c1e583ce35c260ca2fa318"} Jul 29 08:21:32.958658 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:21:32.957997 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-qk9kb" Jul 29 08:21:32.958658 ip-10-0-131-138 kubenswrapper[2563]: E0729 08:21:32.958140 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-qk9kb" podUID="35e49848-94f6-43d9-b7c4-6c817dd5dde7" Jul 29 08:21:32.958658 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:21:32.958220 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-rdclp" Jul 29 08:21:32.958658 ip-10-0-131-138 kubenswrapper[2563]: E0729 08:21:32.958303 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-rdclp" podUID="ce06e4b8-c4ec-4edf-a9c4-d7d86a03d844" Jul 29 08:21:33.092356 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:21:33.092316 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-138.ec2.internal" event={"ID":"9f8ab28df5c602f62a77b9c2aed7b570","Type":"ContainerStarted","Data":"89b85309f4eebc1ef349af8c524a1f7ff277efa6e5114136247d9f9af75aa05b"} Jul 29 08:21:33.469163 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:21:33.469128 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/35e49848-94f6-43d9-b7c4-6c817dd5dde7-metrics-certs\") pod \"network-metrics-daemon-qk9kb\" (UID: \"35e49848-94f6-43d9-b7c4-6c817dd5dde7\") " pod="openshift-multus/network-metrics-daemon-qk9kb" Jul 29 08:21:33.469373 ip-10-0-131-138 kubenswrapper[2563]: E0729 08:21:33.469308 2563 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 29 08:21:33.469373 ip-10-0-131-138 kubenswrapper[2563]: E0729 08:21:33.469370 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/35e49848-94f6-43d9-b7c4-6c817dd5dde7-metrics-certs podName:35e49848-94f6-43d9-b7c4-6c817dd5dde7 nodeName:}" failed. No retries permitted until 2026-07-29 08:21:37.469352101 +0000 UTC m=+10.050640874 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/35e49848-94f6-43d9-b7c4-6c817dd5dde7-metrics-certs") pod "network-metrics-daemon-qk9kb" (UID: "35e49848-94f6-43d9-b7c4-6c817dd5dde7") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 29 08:21:33.569762 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:21:33.569717 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nxn6h\" (UniqueName: \"kubernetes.io/projected/ce06e4b8-c4ec-4edf-a9c4-d7d86a03d844-kube-api-access-nxn6h\") pod \"network-check-target-rdclp\" (UID: \"ce06e4b8-c4ec-4edf-a9c4-d7d86a03d844\") " pod="openshift-network-diagnostics/network-check-target-rdclp" Jul 29 08:21:33.569949 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:21:33.569817 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/b14e5aed-a2b3-41bf-9389-701d69342953-original-pull-secret\") pod \"global-pull-secret-syncer-rn9kc\" (UID: \"b14e5aed-a2b3-41bf-9389-701d69342953\") " pod="kube-system/global-pull-secret-syncer-rn9kc" Jul 29 08:21:33.570012 ip-10-0-131-138 kubenswrapper[2563]: E0729 08:21:33.569984 2563 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 29 08:21:33.570066 ip-10-0-131-138 kubenswrapper[2563]: E0729 08:21:33.570047 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b14e5aed-a2b3-41bf-9389-701d69342953-original-pull-secret podName:b14e5aed-a2b3-41bf-9389-701d69342953 nodeName:}" failed. No retries permitted until 2026-07-29 08:21:37.570028446 +0000 UTC m=+10.151317204 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/b14e5aed-a2b3-41bf-9389-701d69342953-original-pull-secret") pod "global-pull-secret-syncer-rn9kc" (UID: "b14e5aed-a2b3-41bf-9389-701d69342953") : object "kube-system"/"original-pull-secret" not registered Jul 29 08:21:33.570158 ip-10-0-131-138 kubenswrapper[2563]: E0729 08:21:33.570146 2563 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 29 08:21:33.570214 ip-10-0-131-138 kubenswrapper[2563]: E0729 08:21:33.570168 2563 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 29 08:21:33.570214 ip-10-0-131-138 kubenswrapper[2563]: E0729 08:21:33.570181 2563 projected.go:196] Error preparing data for projected volume kube-api-access-nxn6h for pod openshift-network-diagnostics/network-check-target-rdclp: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 29 08:21:33.570350 ip-10-0-131-138 kubenswrapper[2563]: E0729 08:21:33.570220 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/ce06e4b8-c4ec-4edf-a9c4-d7d86a03d844-kube-api-access-nxn6h podName:ce06e4b8-c4ec-4edf-a9c4-d7d86a03d844 nodeName:}" failed. No retries permitted until 2026-07-29 08:21:37.570208638 +0000 UTC m=+10.151497397 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-nxn6h" (UniqueName: "kubernetes.io/projected/ce06e4b8-c4ec-4edf-a9c4-d7d86a03d844-kube-api-access-nxn6h") pod "network-check-target-rdclp" (UID: "ce06e4b8-c4ec-4edf-a9c4-d7d86a03d844") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 29 08:21:33.960162 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:21:33.959577 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-rn9kc" Jul 29 08:21:33.960162 ip-10-0-131-138 kubenswrapper[2563]: E0729 08:21:33.959709 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-rn9kc" podUID="b14e5aed-a2b3-41bf-9389-701d69342953" Jul 29 08:21:34.957586 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:21:34.957531 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-qk9kb" Jul 29 08:21:34.957806 ip-10-0-131-138 kubenswrapper[2563]: E0729 08:21:34.957676 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-qk9kb" podUID="35e49848-94f6-43d9-b7c4-6c817dd5dde7" Jul 29 08:21:34.958051 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:21:34.958025 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-rdclp" Jul 29 08:21:34.958177 ip-10-0-131-138 kubenswrapper[2563]: E0729 08:21:34.958109 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-rdclp" podUID="ce06e4b8-c4ec-4edf-a9c4-d7d86a03d844" Jul 29 08:21:35.957813 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:21:35.957730 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-rn9kc" Jul 29 08:21:35.958270 ip-10-0-131-138 kubenswrapper[2563]: E0729 08:21:35.957873 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-rn9kc" podUID="b14e5aed-a2b3-41bf-9389-701d69342953" Jul 29 08:21:36.957463 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:21:36.957429 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-qk9kb" Jul 29 08:21:36.957647 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:21:36.957438 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-rdclp" Jul 29 08:21:36.957647 ip-10-0-131-138 kubenswrapper[2563]: E0729 08:21:36.957622 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-qk9kb" podUID="35e49848-94f6-43d9-b7c4-6c817dd5dde7" Jul 29 08:21:36.957758 ip-10-0-131-138 kubenswrapper[2563]: E0729 08:21:36.957698 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-rdclp" podUID="ce06e4b8-c4ec-4edf-a9c4-d7d86a03d844" Jul 29 08:21:37.504459 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:21:37.504424 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/35e49848-94f6-43d9-b7c4-6c817dd5dde7-metrics-certs\") pod \"network-metrics-daemon-qk9kb\" (UID: \"35e49848-94f6-43d9-b7c4-6c817dd5dde7\") " pod="openshift-multus/network-metrics-daemon-qk9kb" Jul 29 08:21:37.504932 ip-10-0-131-138 kubenswrapper[2563]: E0729 08:21:37.504573 2563 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 29 08:21:37.504932 ip-10-0-131-138 kubenswrapper[2563]: E0729 08:21:37.504630 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/35e49848-94f6-43d9-b7c4-6c817dd5dde7-metrics-certs podName:35e49848-94f6-43d9-b7c4-6c817dd5dde7 nodeName:}" failed. No retries permitted until 2026-07-29 08:21:45.504615923 +0000 UTC m=+18.085904682 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/35e49848-94f6-43d9-b7c4-6c817dd5dde7-metrics-certs") pod "network-metrics-daemon-qk9kb" (UID: "35e49848-94f6-43d9-b7c4-6c817dd5dde7") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 29 08:21:37.605390 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:21:37.605348 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nxn6h\" (UniqueName: \"kubernetes.io/projected/ce06e4b8-c4ec-4edf-a9c4-d7d86a03d844-kube-api-access-nxn6h\") pod \"network-check-target-rdclp\" (UID: \"ce06e4b8-c4ec-4edf-a9c4-d7d86a03d844\") " pod="openshift-network-diagnostics/network-check-target-rdclp" Jul 29 08:21:37.605554 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:21:37.605404 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/b14e5aed-a2b3-41bf-9389-701d69342953-original-pull-secret\") pod \"global-pull-secret-syncer-rn9kc\" (UID: \"b14e5aed-a2b3-41bf-9389-701d69342953\") " pod="kube-system/global-pull-secret-syncer-rn9kc" Jul 29 08:21:37.605617 ip-10-0-131-138 kubenswrapper[2563]: E0729 08:21:37.605549 2563 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 29 08:21:37.605617 ip-10-0-131-138 kubenswrapper[2563]: E0729 08:21:37.605572 2563 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 29 08:21:37.605617 ip-10-0-131-138 kubenswrapper[2563]: E0729 08:21:37.605576 2563 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 29 08:21:37.605617 ip-10-0-131-138 kubenswrapper[2563]: E0729 08:21:37.605597 2563 projected.go:196] Error preparing data for projected volume kube-api-access-nxn6h for pod openshift-network-diagnostics/network-check-target-rdclp: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 29 08:21:37.605782 ip-10-0-131-138 kubenswrapper[2563]: E0729 08:21:37.605643 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b14e5aed-a2b3-41bf-9389-701d69342953-original-pull-secret podName:b14e5aed-a2b3-41bf-9389-701d69342953 nodeName:}" failed. No retries permitted until 2026-07-29 08:21:45.60562445 +0000 UTC m=+18.186913222 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/b14e5aed-a2b3-41bf-9389-701d69342953-original-pull-secret") pod "global-pull-secret-syncer-rn9kc" (UID: "b14e5aed-a2b3-41bf-9389-701d69342953") : object "kube-system"/"original-pull-secret" not registered Jul 29 08:21:37.605782 ip-10-0-131-138 kubenswrapper[2563]: E0729 08:21:37.605662 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/ce06e4b8-c4ec-4edf-a9c4-d7d86a03d844-kube-api-access-nxn6h podName:ce06e4b8-c4ec-4edf-a9c4-d7d86a03d844 nodeName:}" failed. No retries permitted until 2026-07-29 08:21:45.60565242 +0000 UTC m=+18.186941179 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-nxn6h" (UniqueName: "kubernetes.io/projected/ce06e4b8-c4ec-4edf-a9c4-d7d86a03d844-kube-api-access-nxn6h") pod "network-check-target-rdclp" (UID: "ce06e4b8-c4ec-4edf-a9c4-d7d86a03d844") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 29 08:21:37.959155 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:21:37.957519 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-rn9kc" Jul 29 08:21:37.959155 ip-10-0-131-138 kubenswrapper[2563]: E0729 08:21:37.957669 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-rn9kc" podUID="b14e5aed-a2b3-41bf-9389-701d69342953" Jul 29 08:21:38.957473 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:21:38.957430 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-qk9kb" Jul 29 08:21:38.957945 ip-10-0-131-138 kubenswrapper[2563]: E0729 08:21:38.957551 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-qk9kb" podUID="35e49848-94f6-43d9-b7c4-6c817dd5dde7" Jul 29 08:21:38.957945 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:21:38.957616 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-rdclp" Jul 29 08:21:38.957945 ip-10-0-131-138 kubenswrapper[2563]: E0729 08:21:38.957669 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-rdclp" podUID="ce06e4b8-c4ec-4edf-a9c4-d7d86a03d844" Jul 29 08:21:39.959227 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:21:39.959195 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-rn9kc" Jul 29 08:21:39.959757 ip-10-0-131-138 kubenswrapper[2563]: E0729 08:21:39.959342 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-rn9kc" podUID="b14e5aed-a2b3-41bf-9389-701d69342953" Jul 29 08:21:40.957661 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:21:40.957626 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-qk9kb" Jul 29 08:21:40.957836 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:21:40.957626 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-rdclp" Jul 29 08:21:40.957836 ip-10-0-131-138 kubenswrapper[2563]: E0729 08:21:40.957776 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-qk9kb" podUID="35e49848-94f6-43d9-b7c4-6c817dd5dde7" Jul 29 08:21:40.957836 ip-10-0-131-138 kubenswrapper[2563]: E0729 08:21:40.957828 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-rdclp" podUID="ce06e4b8-c4ec-4edf-a9c4-d7d86a03d844" Jul 29 08:21:41.957430 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:21:41.957332 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-rn9kc" Jul 29 08:21:41.957875 ip-10-0-131-138 kubenswrapper[2563]: E0729 08:21:41.957465 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-rn9kc" podUID="b14e5aed-a2b3-41bf-9389-701d69342953" Jul 29 08:21:42.957391 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:21:42.957349 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-rdclp" Jul 29 08:21:42.957592 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:21:42.957473 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-qk9kb" Jul 29 08:21:42.957592 ip-10-0-131-138 kubenswrapper[2563]: E0729 08:21:42.957503 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-rdclp" podUID="ce06e4b8-c4ec-4edf-a9c4-d7d86a03d844" Jul 29 08:21:42.957942 ip-10-0-131-138 kubenswrapper[2563]: E0729 08:21:42.957597 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-qk9kb" podUID="35e49848-94f6-43d9-b7c4-6c817dd5dde7" Jul 29 08:21:43.957749 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:21:43.957708 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-rn9kc" Jul 29 08:21:43.958276 ip-10-0-131-138 kubenswrapper[2563]: E0729 08:21:43.957862 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-rn9kc" podUID="b14e5aed-a2b3-41bf-9389-701d69342953" Jul 29 08:21:44.957549 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:21:44.957509 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-qk9kb" Jul 29 08:21:44.957836 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:21:44.957558 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-rdclp" Jul 29 08:21:44.957836 ip-10-0-131-138 kubenswrapper[2563]: E0729 08:21:44.957650 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-qk9kb" podUID="35e49848-94f6-43d9-b7c4-6c817dd5dde7" Jul 29 08:21:44.957836 ip-10-0-131-138 kubenswrapper[2563]: E0729 08:21:44.957764 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-rdclp" podUID="ce06e4b8-c4ec-4edf-a9c4-d7d86a03d844" Jul 29 08:21:45.568435 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:21:45.568403 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/35e49848-94f6-43d9-b7c4-6c817dd5dde7-metrics-certs\") pod \"network-metrics-daemon-qk9kb\" (UID: \"35e49848-94f6-43d9-b7c4-6c817dd5dde7\") " pod="openshift-multus/network-metrics-daemon-qk9kb" Jul 29 08:21:45.568598 ip-10-0-131-138 kubenswrapper[2563]: E0729 08:21:45.568521 2563 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 29 08:21:45.568598 ip-10-0-131-138 kubenswrapper[2563]: E0729 08:21:45.568578 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/35e49848-94f6-43d9-b7c4-6c817dd5dde7-metrics-certs podName:35e49848-94f6-43d9-b7c4-6c817dd5dde7 nodeName:}" failed. No retries permitted until 2026-07-29 08:22:01.568565436 +0000 UTC m=+34.149854191 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/35e49848-94f6-43d9-b7c4-6c817dd5dde7-metrics-certs") pod "network-metrics-daemon-qk9kb" (UID: "35e49848-94f6-43d9-b7c4-6c817dd5dde7") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 29 08:21:45.669817 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:21:45.669738 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nxn6h\" (UniqueName: \"kubernetes.io/projected/ce06e4b8-c4ec-4edf-a9c4-d7d86a03d844-kube-api-access-nxn6h\") pod \"network-check-target-rdclp\" (UID: \"ce06e4b8-c4ec-4edf-a9c4-d7d86a03d844\") " pod="openshift-network-diagnostics/network-check-target-rdclp" Jul 29 08:21:45.669817 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:21:45.669781 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/b14e5aed-a2b3-41bf-9389-701d69342953-original-pull-secret\") pod \"global-pull-secret-syncer-rn9kc\" (UID: \"b14e5aed-a2b3-41bf-9389-701d69342953\") " pod="kube-system/global-pull-secret-syncer-rn9kc" Jul 29 08:21:45.670051 ip-10-0-131-138 kubenswrapper[2563]: E0729 08:21:45.669903 2563 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 29 08:21:45.670051 ip-10-0-131-138 kubenswrapper[2563]: E0729 08:21:45.669914 2563 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 29 08:21:45.670051 ip-10-0-131-138 kubenswrapper[2563]: E0729 08:21:45.669931 2563 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 29 08:21:45.670051 ip-10-0-131-138 kubenswrapper[2563]: E0729 08:21:45.669944 2563 projected.go:196] Error preparing data for projected volume kube-api-access-nxn6h for pod openshift-network-diagnostics/network-check-target-rdclp: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 29 08:21:45.670051 ip-10-0-131-138 kubenswrapper[2563]: E0729 08:21:45.669971 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b14e5aed-a2b3-41bf-9389-701d69342953-original-pull-secret podName:b14e5aed-a2b3-41bf-9389-701d69342953 nodeName:}" failed. No retries permitted until 2026-07-29 08:22:01.669952927 +0000 UTC m=+34.251241705 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/b14e5aed-a2b3-41bf-9389-701d69342953-original-pull-secret") pod "global-pull-secret-syncer-rn9kc" (UID: "b14e5aed-a2b3-41bf-9389-701d69342953") : object "kube-system"/"original-pull-secret" not registered Jul 29 08:21:45.670051 ip-10-0-131-138 kubenswrapper[2563]: E0729 08:21:45.669990 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/ce06e4b8-c4ec-4edf-a9c4-d7d86a03d844-kube-api-access-nxn6h podName:ce06e4b8-c4ec-4edf-a9c4-d7d86a03d844 nodeName:}" failed. No retries permitted until 2026-07-29 08:22:01.669980068 +0000 UTC m=+34.251268832 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-nxn6h" (UniqueName: "kubernetes.io/projected/ce06e4b8-c4ec-4edf-a9c4-d7d86a03d844-kube-api-access-nxn6h") pod "network-check-target-rdclp" (UID: "ce06e4b8-c4ec-4edf-a9c4-d7d86a03d844") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 29 08:21:45.957437 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:21:45.957348 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-rn9kc" Jul 29 08:21:45.957603 ip-10-0-131-138 kubenswrapper[2563]: E0729 08:21:45.957489 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-rn9kc" podUID="b14e5aed-a2b3-41bf-9389-701d69342953" Jul 29 08:21:46.957469 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:21:46.957440 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-rdclp" Jul 29 08:21:46.957979 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:21:46.957483 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-qk9kb" Jul 29 08:21:46.957979 ip-10-0-131-138 kubenswrapper[2563]: E0729 08:21:46.957561 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-rdclp" podUID="ce06e4b8-c4ec-4edf-a9c4-d7d86a03d844" Jul 29 08:21:46.957979 ip-10-0-131-138 kubenswrapper[2563]: E0729 08:21:46.957666 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-qk9kb" podUID="35e49848-94f6-43d9-b7c4-6c817dd5dde7" Jul 29 08:21:47.958073 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:21:47.958032 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-rn9kc" Jul 29 08:21:47.958596 ip-10-0-131-138 kubenswrapper[2563]: E0729 08:21:47.958136 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-rn9kc" podUID="b14e5aed-a2b3-41bf-9389-701d69342953" Jul 29 08:21:48.956945 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:21:48.956913 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-rdclp" Jul 29 08:21:48.957106 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:21:48.956916 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-qk9kb" Jul 29 08:21:48.957106 ip-10-0-131-138 kubenswrapper[2563]: E0729 08:21:48.957016 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-rdclp" podUID="ce06e4b8-c4ec-4edf-a9c4-d7d86a03d844" Jul 29 08:21:48.957213 ip-10-0-131-138 kubenswrapper[2563]: E0729 08:21:48.957111 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-qk9kb" podUID="35e49848-94f6-43d9-b7c4-6c817dd5dde7" Jul 29 08:21:49.958526 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:21:49.958064 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-rn9kc" Jul 29 08:21:49.959179 ip-10-0-131-138 kubenswrapper[2563]: E0729 08:21:49.958650 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-rn9kc" podUID="b14e5aed-a2b3-41bf-9389-701d69342953" Jul 29 08:21:50.130584 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:21:50.130554 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-52bkd" event={"ID":"93ba91b4-4c35-43be-9094-08b0b3a473b3","Type":"ContainerStarted","Data":"2ee7b0e38d3201ac089ec4d1b3d7c312d378697bd0e7f8b4da3480eba9b47ddf"} Jul 29 08:21:50.131934 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:21:50.131907 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-bgb7d" event={"ID":"bc118958-c5f8-44b8-80a5-bfc2d80fad1d","Type":"ContainerStarted","Data":"0291eaf7420d43721a8889b49860d615b4676c944aab2d192aa8e5126f8182b3"} Jul 29 08:21:50.133305 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:21:50.133281 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-v5xrb" event={"ID":"2b20d931-6631-4e41-aff5-ed22c5fdafa7","Type":"ContainerStarted","Data":"91b43321ff7e0ddbd405e9779642a8e973903eef8c035aa051471d66ce618c19"} Jul 29 08:21:50.134707 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:21:50.134687 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-9d4f5" event={"ID":"5bce82ca-dd94-425f-83d2-cd44232d8cdb","Type":"ContainerStarted","Data":"713fefdbb80c087075e73845a41a773205e1fb2a41a60212ebb66b6e7583818c"} Jul 29 08:21:50.136404 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:21:50.136380 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-8rp9g" event={"ID":"cfd1d8df-e6b7-4a67-81f2-5d83da541e20","Type":"ContainerStarted","Data":"745f5453b00a0445b7b414f7e0d2e2b84496db2f14a3b79d2286572210f13b11"} Jul 29 08:21:50.137997 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:21:50.137956 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-95gmk" event={"ID":"30ff3fec-359e-4b75-82eb-9f1a194766f7","Type":"ContainerStarted","Data":"469c859b1adcb5e22bc6aae277a284d12431aeb295fec8cf98c490fdccfa9a5d"} Jul 29 08:21:50.140749 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:21:50.140728 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-vqdfh" event={"ID":"49600d58-879f-477d-aa43-11177f411977","Type":"ContainerStarted","Data":"fbdc646cb5c26284ee679ad396244edde0e637dc2fd2e438cf52191970c6ab36"} Jul 29 08:21:50.140816 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:21:50.140779 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-vqdfh" event={"ID":"49600d58-879f-477d-aa43-11177f411977","Type":"ContainerStarted","Data":"72fbbe63b0c4468a039ea77b484fc8f9fd2827f1ffebb8a8760d034137ba0252"} Jul 29 08:21:50.140816 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:21:50.140807 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-vqdfh" event={"ID":"49600d58-879f-477d-aa43-11177f411977","Type":"ContainerStarted","Data":"8efa149560d1d66521727212af620795f0f025bfd5a8bf12b0593ff6cc0af4ab"} Jul 29 08:21:50.140915 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:21:50.140835 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-vqdfh" event={"ID":"49600d58-879f-477d-aa43-11177f411977","Type":"ContainerStarted","Data":"4bd497df5c23332d853a2465fb2352a29c65594e3c85c71ded528fec3666ab8c"} Jul 29 08:21:50.140915 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:21:50.140861 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-vqdfh" event={"ID":"49600d58-879f-477d-aa43-11177f411977","Type":"ContainerStarted","Data":"c286ddba2a4db3a74c9cd89805f8f97d00338946fab093f6e981849a4685a239"} Jul 29 08:21:50.142034 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:21:50.142012 2563 generic.go:356] "Generic (PLEG): container finished" podID="bb5d06c1-cbbb-4e40-9e0a-43aa22af698f" containerID="e7b6f5e3e7b9cba6c8966ef2846cef109ae7600cbb984b4a2427d64a331f33f6" exitCode=0 Jul 29 08:21:50.142112 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:21:50.142045 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-qs8zl" event={"ID":"bb5d06c1-cbbb-4e40-9e0a-43aa22af698f","Type":"ContainerDied","Data":"e7b6f5e3e7b9cba6c8966ef2846cef109ae7600cbb984b4a2427d64a331f33f6"} Jul 29 08:21:50.150776 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:21:50.150720 2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-138.ec2.internal" podStartSLOduration=21.150707698 podStartE2EDuration="21.150707698s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-29 08:21:29 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 08:21:33.110400758 +0000 UTC m=+5.691689542" watchObservedRunningTime="2026-07-29 08:21:50.150707698 +0000 UTC m=+22.731996476" Jul 29 08:21:50.173430 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:21:50.173396 2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/node-resolver-bgb7d" podStartSLOduration=4.608936834 podStartE2EDuration="23.17338693s" totalImagesPullingTime="18.564450096s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-29 08:21:27 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 08:21:50.150351354 +0000 UTC m=+22.731640132" watchObservedRunningTime="2026-07-29 08:21:50.17338693 +0000 UTC m=+22.754675707" Jul 29 08:21:50.173818 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:21:50.173795 2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-95gmk" podStartSLOduration=3.577263732 podStartE2EDuration="22.173789979s" totalImagesPullingTime="18.596526247s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-29 08:21:28 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 08:21:50.17308878 +0000 UTC m=+22.754377555" watchObservedRunningTime="2026-07-29 08:21:50.173789979 +0000 UTC m=+22.755078754" Jul 29 08:21:50.217329 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:21:50.217288 2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/node-ca-9d4f5" podStartSLOduration=3.650680099 podStartE2EDuration="22.21724951s" totalImagesPullingTime="18.566569411s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-29 08:21:28 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 08:21:50.189840575 +0000 UTC m=+22.771129354" watchObservedRunningTime="2026-07-29 08:21:50.21724951 +0000 UTC m=+22.798538288" Jul 29 08:21:50.242596 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:21:50.242546 2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/konnectivity-agent-8rp9g" podStartSLOduration=7.822286696 podStartE2EDuration="22.242531514s" totalImagesPullingTime="14.420244818s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 08:21:28 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 08:21:50.242065697 +0000 UTC m=+22.823354476" watchObservedRunningTime="2026-07-29 08:21:50.242531514 +0000 UTC m=+22.823820292" Jul 29 08:21:50.261537 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:21:50.261489 2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-v5xrb" podStartSLOduration=3.640107131 podStartE2EDuration="22.261472171s" totalImagesPullingTime="18.62136504s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-29 08:21:28 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 08:21:50.261049141 +0000 UTC m=+22.842337919" watchObservedRunningTime="2026-07-29 08:21:50.261472171 +0000 UTC m=+22.842760949" Jul 29 08:21:50.292517 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:21:50.292486 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8rp9g_cfd1d8df-e6b7-4a67-81f2-5d83da541e20/konnectivity-agent/0.log" limit=1024 Jul 29 08:21:50.362533 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:21:50.362462 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8rp9g_cfd1d8df-e6b7-4a67-81f2-5d83da541e20/konnectivity-agent/0.log" limit=1024 Jul 29 08:21:50.462854 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:21:50.462822 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8rp9g_cfd1d8df-e6b7-4a67-81f2-5d83da541e20/konnectivity-agent/0.log" limit=1024 Jul 29 08:21:50.564881 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:21:50.564848 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8rp9g_cfd1d8df-e6b7-4a67-81f2-5d83da541e20/konnectivity-agent/0.log" limit=1024 Jul 29 08:21:50.934966 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:21:50.934923 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8rp9g_cfd1d8df-e6b7-4a67-81f2-5d83da541e20/konnectivity-agent/0.log" limit=1024 Jul 29 08:21:50.957996 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:21:50.957969 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-qk9kb" Jul 29 08:21:50.957996 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:21:50.958004 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-rdclp" Jul 29 08:21:50.958177 ip-10-0-131-138 kubenswrapper[2563]: E0729 08:21:50.958096 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-qk9kb" podUID="35e49848-94f6-43d9-b7c4-6c817dd5dde7" Jul 29 08:21:50.958224 ip-10-0-131-138 kubenswrapper[2563]: E0729 08:21:50.958208 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-rdclp" podUID="ce06e4b8-c4ec-4edf-a9c4-d7d86a03d844" Jul 29 08:21:51.035029 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:21:51.034933 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8rp9g_cfd1d8df-e6b7-4a67-81f2-5d83da541e20/konnectivity-agent/0.log" limit=1024 Jul 29 08:21:51.146811 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:21:51.146780 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-z2c22" event={"ID":"db249d48-ad47-4cc0-8a37-37078055ed38","Type":"ContainerStarted","Data":"f785a802c6e9d8691df1745448f4a3c3c5a132e3e0fd0aac0d3861763aefe0a9"} Jul 29 08:21:51.150144 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:21:51.150044 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-vqdfh" event={"ID":"49600d58-879f-477d-aa43-11177f411977","Type":"ContainerStarted","Data":"160920cf28e96e5d7627c6accbd9bf0a701dd179371821de08bdc2860d13dad0"} Jul 29 08:21:51.208315 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:21:51.208288 2563 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock" Jul 29 08:21:51.293543 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:21:51.293455 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8rp9g_cfd1d8df-e6b7-4a67-81f2-5d83da541e20/konnectivity-agent/0.log" limit=1024 Jul 29 08:21:51.394425 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:21:51.394397 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8rp9g_cfd1d8df-e6b7-4a67-81f2-5d83da541e20/konnectivity-agent/0.log" limit=1024 Jul 29 08:21:51.888369 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:21:51.888248 2563 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-07-29T08:21:51.208306615Z","UUID":"27ff9ccd-d999-4c9c-aa01-1a6abfe3f8f1","Handler":null,"Name":"","Endpoint":""} Jul 29 08:21:51.890043 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:21:51.890001 2563 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0 Jul 29 08:21:51.890043 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:21:51.890046 2563 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock Jul 29 08:21:51.957633 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:21:51.957602 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-rn9kc" Jul 29 08:21:51.957817 ip-10-0-131-138 kubenswrapper[2563]: E0729 08:21:51.957733 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-rn9kc" podUID="b14e5aed-a2b3-41bf-9389-701d69342953" Jul 29 08:21:52.154480 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:21:52.154392 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-52bkd" event={"ID":"93ba91b4-4c35-43be-9094-08b0b3a473b3","Type":"ContainerStarted","Data":"5945e85fb8bfea8d26ce80468a17f7df2a54d704ca57c9118a8dda759e0adbf3"} Jul 29 08:21:52.173937 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:21:52.173911 2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-8rp9g" Jul 29 08:21:52.174541 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:21:52.174519 2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-8rp9g" Jul 29 08:21:52.198330 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:21:52.198276 2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-z2c22" podStartSLOduration=5.6355017 podStartE2EDuration="24.198239279s" totalImagesPullingTime="18.562737579s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-29 08:21:28 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 08:21:51.166598648 +0000 UTC m=+23.747887427" watchObservedRunningTime="2026-07-29 08:21:52.198239279 +0000 UTC m=+24.779528056" Jul 29 08:21:52.957975 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:21:52.957937 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-qk9kb" Jul 29 08:21:52.958285 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:21:52.957953 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-rdclp" Jul 29 08:21:52.958285 ip-10-0-131-138 kubenswrapper[2563]: E0729 08:21:52.958070 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-qk9kb" podUID="35e49848-94f6-43d9-b7c4-6c817dd5dde7" Jul 29 08:21:52.958285 ip-10-0-131-138 kubenswrapper[2563]: E0729 08:21:52.958096 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-rdclp" podUID="ce06e4b8-c4ec-4edf-a9c4-d7d86a03d844" Jul 29 08:21:53.159621 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:21:53.159567 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-52bkd" event={"ID":"93ba91b4-4c35-43be-9094-08b0b3a473b3","Type":"ContainerStarted","Data":"6aa196cce1d82e1200d0339fe9fb90cc067984cbd3851b0bfc5cfd4f01aabdf4"} Jul 29 08:21:53.167491 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:21:53.167464 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-vqdfh" event={"ID":"49600d58-879f-477d-aa43-11177f411977","Type":"ContainerStarted","Data":"adb0ff99415ca4cc8405e37a6a2c490c6a505be6ec86ac81263e33d00fe85f2e"} Jul 29 08:21:53.167782 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:21:53.167754 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-8rp9g" Jul 29 08:21:53.168213 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:21:53.168196 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-8rp9g" Jul 29 08:21:53.186988 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:21:53.186935 2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-52bkd" podStartSLOduration=5.264727104 podStartE2EDuration="26.186917627s" totalImagesPullingTime="20.922190523s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-29 08:21:27 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 08:21:53.186614624 +0000 UTC m=+25.767903392" watchObservedRunningTime="2026-07-29 08:21:53.186917627 +0000 UTC m=+25.768206407" Jul 29 08:21:53.258279 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:21:53.258185 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8rp9g_cfd1d8df-e6b7-4a67-81f2-5d83da541e20/konnectivity-agent/0.log" limit=1024 Jul 29 08:21:53.957479 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:21:53.957445 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-rn9kc" Jul 29 08:21:53.957670 ip-10-0-131-138 kubenswrapper[2563]: E0729 08:21:53.957569 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-rn9kc" podUID="b14e5aed-a2b3-41bf-9389-701d69342953" Jul 29 08:21:54.957148 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:21:54.956944 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-qk9kb" Jul 29 08:21:54.957754 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:21:54.956972 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-rdclp" Jul 29 08:21:54.957754 ip-10-0-131-138 kubenswrapper[2563]: E0729 08:21:54.957233 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-qk9kb" podUID="35e49848-94f6-43d9-b7c4-6c817dd5dde7" Jul 29 08:21:54.957754 ip-10-0-131-138 kubenswrapper[2563]: E0729 08:21:54.957343 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-rdclp" podUID="ce06e4b8-c4ec-4edf-a9c4-d7d86a03d844" Jul 29 08:21:55.175436 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:21:55.175358 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-vqdfh" event={"ID":"49600d58-879f-477d-aa43-11177f411977","Type":"ContainerStarted","Data":"1eac3e915f9ad7e9e5000cbd88866d75baffcfa52be8c2023a4b597714838940"} Jul 29 08:21:55.175662 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:21:55.175645 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-vqdfh" Jul 29 08:21:55.175751 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:21:55.175709 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-vqdfh" Jul 29 08:21:55.178124 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:21:55.178097 2563 generic.go:356] "Generic (PLEG): container finished" podID="bb5d06c1-cbbb-4e40-9e0a-43aa22af698f" containerID="d40fdde0b9f950de937d945d0ebd64b684d8a1541e45613a8a70067c1c03f5e5" exitCode=0 Jul 29 08:21:55.178226 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:21:55.178160 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-qs8zl" event={"ID":"bb5d06c1-cbbb-4e40-9e0a-43aa22af698f","Type":"ContainerDied","Data":"d40fdde0b9f950de937d945d0ebd64b684d8a1541e45613a8a70067c1c03f5e5"} Jul 29 08:21:55.192082 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:21:55.192057 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-vqdfh" Jul 29 08:21:55.245088 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:21:55.245049 2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-vqdfh" podStartSLOduration=8.60858856 podStartE2EDuration="27.245033968s" totalImagesPullingTime="18.636445408s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-29 08:21:28 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 08:21:55.216430231 +0000 UTC m=+27.797719009" watchObservedRunningTime="2026-07-29 08:21:55.245033968 +0000 UTC m=+27.826322724" Jul 29 08:21:55.957364 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:21:55.957091 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-rn9kc" Jul 29 08:21:55.957823 ip-10-0-131-138 kubenswrapper[2563]: E0729 08:21:55.957480 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-rn9kc" podUID="b14e5aed-a2b3-41bf-9389-701d69342953" Jul 29 08:21:56.181316 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:21:56.181286 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-vqdfh" Jul 29 08:21:56.196684 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:21:56.196655 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-vqdfh" Jul 29 08:21:56.731287 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:21:56.731239 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-rn9kc"] Jul 29 08:21:56.731521 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:21:56.731353 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-rn9kc" Jul 29 08:21:56.731521 ip-10-0-131-138 kubenswrapper[2563]: E0729 08:21:56.731443 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-rn9kc" podUID="b14e5aed-a2b3-41bf-9389-701d69342953" Jul 29 08:21:56.736379 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:21:56.736343 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-rdclp"] Jul 29 08:21:56.736505 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:21:56.736463 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-rdclp" Jul 29 08:21:56.736576 ip-10-0-131-138 kubenswrapper[2563]: E0729 08:21:56.736531 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-rdclp" podUID="ce06e4b8-c4ec-4edf-a9c4-d7d86a03d844" Jul 29 08:21:56.736958 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:21:56.736937 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-qk9kb"] Jul 29 08:21:56.737067 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:21:56.737040 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-qk9kb" Jul 29 08:21:56.737126 ip-10-0-131-138 kubenswrapper[2563]: E0729 08:21:56.737115 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-qk9kb" podUID="35e49848-94f6-43d9-b7c4-6c817dd5dde7" Jul 29 08:21:57.185278 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:21:57.185223 2563 generic.go:356] "Generic (PLEG): container finished" podID="bb5d06c1-cbbb-4e40-9e0a-43aa22af698f" containerID="fe4b2dd9defd5d97e166057f8435228427778dc0ddbde206135a61078674ee81" exitCode=0 Jul 29 08:21:57.185679 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:21:57.185322 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-qs8zl" event={"ID":"bb5d06c1-cbbb-4e40-9e0a-43aa22af698f","Type":"ContainerDied","Data":"fe4b2dd9defd5d97e166057f8435228427778dc0ddbde206135a61078674ee81"} Jul 29 08:21:57.958784 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:21:57.958753 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-rdclp" Jul 29 08:21:57.958939 ip-10-0-131-138 kubenswrapper[2563]: E0729 08:21:57.958842 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-rdclp" podUID="ce06e4b8-c4ec-4edf-a9c4-d7d86a03d844" Jul 29 08:21:58.957267 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:21:58.956989 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-rn9kc" Jul 29 08:21:58.957601 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:21:58.957008 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-qk9kb" Jul 29 08:21:58.957601 ip-10-0-131-138 kubenswrapper[2563]: E0729 08:21:58.957401 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-rn9kc" podUID="b14e5aed-a2b3-41bf-9389-701d69342953" Jul 29 08:21:58.957601 ip-10-0-131-138 kubenswrapper[2563]: E0729 08:21:58.957493 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-qk9kb" podUID="35e49848-94f6-43d9-b7c4-6c817dd5dde7" Jul 29 08:21:59.193093 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:21:59.193018 2563 generic.go:356] "Generic (PLEG): container finished" podID="bb5d06c1-cbbb-4e40-9e0a-43aa22af698f" containerID="c737d1210a9ce997da0a3aa84e183b72a0d8bf87d75506ee7a2b1bbc57bfe04c" exitCode=0 Jul 29 08:21:59.193093 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:21:59.193065 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-qs8zl" event={"ID":"bb5d06c1-cbbb-4e40-9e0a-43aa22af698f","Type":"ContainerDied","Data":"c737d1210a9ce997da0a3aa84e183b72a0d8bf87d75506ee7a2b1bbc57bfe04c"} Jul 29 08:21:59.957541 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:21:59.957505 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-rdclp" Jul 29 08:21:59.957991 ip-10-0-131-138 kubenswrapper[2563]: E0729 08:21:59.957639 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-rdclp" podUID="ce06e4b8-c4ec-4edf-a9c4-d7d86a03d844" Jul 29 08:22:00.957601 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:22:00.957562 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-qk9kb" Jul 29 08:22:00.958023 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:22:00.957563 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-rn9kc" Jul 29 08:22:00.958023 ip-10-0-131-138 kubenswrapper[2563]: E0729 08:22:00.957712 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-qk9kb" podUID="35e49848-94f6-43d9-b7c4-6c817dd5dde7" Jul 29 08:22:00.958023 ip-10-0-131-138 kubenswrapper[2563]: E0729 08:22:00.957746 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-rn9kc" podUID="b14e5aed-a2b3-41bf-9389-701d69342953" Jul 29 08:22:01.593204 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:22:01.593159 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/35e49848-94f6-43d9-b7c4-6c817dd5dde7-metrics-certs\") pod \"network-metrics-daemon-qk9kb\" (UID: \"35e49848-94f6-43d9-b7c4-6c817dd5dde7\") " pod="openshift-multus/network-metrics-daemon-qk9kb" Jul 29 08:22:01.593421 ip-10-0-131-138 kubenswrapper[2563]: E0729 08:22:01.593338 2563 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 29 08:22:01.593421 ip-10-0-131-138 kubenswrapper[2563]: E0729 08:22:01.593404 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/35e49848-94f6-43d9-b7c4-6c817dd5dde7-metrics-certs podName:35e49848-94f6-43d9-b7c4-6c817dd5dde7 nodeName:}" failed. No retries permitted until 2026-07-29 08:22:33.593385576 +0000 UTC m=+66.174674353 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/35e49848-94f6-43d9-b7c4-6c817dd5dde7-metrics-certs") pod "network-metrics-daemon-qk9kb" (UID: "35e49848-94f6-43d9-b7c4-6c817dd5dde7") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 29 08:22:01.694318 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:22:01.694286 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nxn6h\" (UniqueName: \"kubernetes.io/projected/ce06e4b8-c4ec-4edf-a9c4-d7d86a03d844-kube-api-access-nxn6h\") pod \"network-check-target-rdclp\" (UID: \"ce06e4b8-c4ec-4edf-a9c4-d7d86a03d844\") " pod="openshift-network-diagnostics/network-check-target-rdclp" Jul 29 08:22:01.694318 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:22:01.694327 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/b14e5aed-a2b3-41bf-9389-701d69342953-original-pull-secret\") pod \"global-pull-secret-syncer-rn9kc\" (UID: \"b14e5aed-a2b3-41bf-9389-701d69342953\") " pod="kube-system/global-pull-secret-syncer-rn9kc" Jul 29 08:22:01.694548 ip-10-0-131-138 kubenswrapper[2563]: E0729 08:22:01.694460 2563 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 29 08:22:01.694548 ip-10-0-131-138 kubenswrapper[2563]: E0729 08:22:01.694479 2563 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 29 08:22:01.694548 ip-10-0-131-138 kubenswrapper[2563]: E0729 08:22:01.694495 2563 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 29 08:22:01.694548 ip-10-0-131-138 kubenswrapper[2563]: E0729 08:22:01.694506 2563 projected.go:196] Error preparing data for projected volume kube-api-access-nxn6h for pod openshift-network-diagnostics/network-check-target-rdclp: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 29 08:22:01.694548 ip-10-0-131-138 kubenswrapper[2563]: E0729 08:22:01.694528 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b14e5aed-a2b3-41bf-9389-701d69342953-original-pull-secret podName:b14e5aed-a2b3-41bf-9389-701d69342953 nodeName:}" failed. No retries permitted until 2026-07-29 08:22:33.694508813 +0000 UTC m=+66.275797586 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/b14e5aed-a2b3-41bf-9389-701d69342953-original-pull-secret") pod "global-pull-secret-syncer-rn9kc" (UID: "b14e5aed-a2b3-41bf-9389-701d69342953") : object "kube-system"/"original-pull-secret" not registered Jul 29 08:22:01.694548 ip-10-0-131-138 kubenswrapper[2563]: E0729 08:22:01.694545 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/ce06e4b8-c4ec-4edf-a9c4-d7d86a03d844-kube-api-access-nxn6h podName:ce06e4b8-c4ec-4edf-a9c4-d7d86a03d844 nodeName:}" failed. No retries permitted until 2026-07-29 08:22:33.69453535 +0000 UTC m=+66.275824130 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-nxn6h" (UniqueName: "kubernetes.io/projected/ce06e4b8-c4ec-4edf-a9c4-d7d86a03d844-kube-api-access-nxn6h") pod "network-check-target-rdclp" (UID: "ce06e4b8-c4ec-4edf-a9c4-d7d86a03d844") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 29 08:22:01.957133 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:22:01.957099 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-rdclp" Jul 29 08:22:01.957334 ip-10-0-131-138 kubenswrapper[2563]: E0729 08:22:01.957245 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-rdclp" podUID="ce06e4b8-c4ec-4edf-a9c4-d7d86a03d844" Jul 29 08:22:02.268821 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:22:02.268731 2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-138.ec2.internal" event="NodeReady" Jul 29 08:22:02.269368 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:22:02.268902 2563 kubelet_node_status.go:489] "Fast updating node status as it just became ready" Jul 29 08:22:02.317214 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:22:02.317178 2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-nhklh"] Jul 29 08:22:02.319851 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:22:02.319821 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-nhklh" Jul 29 08:22:02.320158 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:22:02.320122 2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-pwqh7"] Jul 29 08:22:02.321957 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:22:02.321936 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-pwqh7" Jul 29 08:22:02.323145 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:22:02.322971 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-wsvft\"" Jul 29 08:22:02.323613 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:22:02.323473 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\"" Jul 29 08:22:02.323613 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:22:02.323512 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\"" Jul 29 08:22:02.324703 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:22:02.324685 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\"" Jul 29 08:22:02.325224 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:22:02.325006 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\"" Jul 29 08:22:02.325224 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:22:02.325023 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-xzvt9\"" Jul 29 08:22:02.325224 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:22:02.325056 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\"" Jul 29 08:22:02.333838 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:22:02.333816 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-nhklh"] Jul 29 08:22:02.333939 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:22:02.333873 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-pwqh7"] Jul 29 08:22:02.399117 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:22:02.399084 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e521e734-a8d9-4748-ba28-c5f2beb27610-tmp-dir\") pod \"dns-default-nhklh\" (UID: \"e521e734-a8d9-4748-ba28-c5f2beb27610\") " pod="openshift-dns/dns-default-nhklh" Jul 29 08:22:02.399117 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:22:02.399122 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4qkg4\" (UniqueName: \"kubernetes.io/projected/e521e734-a8d9-4748-ba28-c5f2beb27610-kube-api-access-4qkg4\") pod \"dns-default-nhklh\" (UID: \"e521e734-a8d9-4748-ba28-c5f2beb27610\") " pod="openshift-dns/dns-default-nhklh" Jul 29 08:22:02.399377 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:22:02.399202 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/b052d187-d65d-4894-aaa2-5dbb4b06600e-cert\") pod \"ingress-canary-pwqh7\" (UID: \"b052d187-d65d-4894-aaa2-5dbb4b06600e\") " pod="openshift-ingress-canary/ingress-canary-pwqh7" Jul 29 08:22:02.399377 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:22:02.399217 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dg9wt\" (UniqueName: \"kubernetes.io/projected/b052d187-d65d-4894-aaa2-5dbb4b06600e-kube-api-access-dg9wt\") pod \"ingress-canary-pwqh7\" (UID: \"b052d187-d65d-4894-aaa2-5dbb4b06600e\") " pod="openshift-ingress-canary/ingress-canary-pwqh7" Jul 29 08:22:02.399377 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:22:02.399275 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/e521e734-a8d9-4748-ba28-c5f2beb27610-config-volume\") pod \"dns-default-nhklh\" (UID: \"e521e734-a8d9-4748-ba28-c5f2beb27610\") " pod="openshift-dns/dns-default-nhklh" Jul 29 08:22:02.399377 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:22:02.399328 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/e521e734-a8d9-4748-ba28-c5f2beb27610-metrics-tls\") pod \"dns-default-nhklh\" (UID: \"e521e734-a8d9-4748-ba28-c5f2beb27610\") " pod="openshift-dns/dns-default-nhklh" Jul 29 08:22:02.466493 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:22:02.466461 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8rp9g_cfd1d8df-e6b7-4a67-81f2-5d83da541e20/konnectivity-agent/0.log" limit=1024 Jul 29 08:22:02.500328 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:22:02.500286 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e521e734-a8d9-4748-ba28-c5f2beb27610-tmp-dir\") pod \"dns-default-nhklh\" (UID: \"e521e734-a8d9-4748-ba28-c5f2beb27610\") " pod="openshift-dns/dns-default-nhklh" Jul 29 08:22:02.500491 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:22:02.500330 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4qkg4\" (UniqueName: \"kubernetes.io/projected/e521e734-a8d9-4748-ba28-c5f2beb27610-kube-api-access-4qkg4\") pod \"dns-default-nhklh\" (UID: \"e521e734-a8d9-4748-ba28-c5f2beb27610\") " pod="openshift-dns/dns-default-nhklh" Jul 29 08:22:02.500491 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:22:02.500424 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/b052d187-d65d-4894-aaa2-5dbb4b06600e-cert\") pod \"ingress-canary-pwqh7\" (UID: \"b052d187-d65d-4894-aaa2-5dbb4b06600e\") " pod="openshift-ingress-canary/ingress-canary-pwqh7" Jul 29 08:22:02.500491 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:22:02.500444 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dg9wt\" (UniqueName: \"kubernetes.io/projected/b052d187-d65d-4894-aaa2-5dbb4b06600e-kube-api-access-dg9wt\") pod \"ingress-canary-pwqh7\" (UID: \"b052d187-d65d-4894-aaa2-5dbb4b06600e\") " pod="openshift-ingress-canary/ingress-canary-pwqh7" Jul 29 08:22:02.500491 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:22:02.500464 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/e521e734-a8d9-4748-ba28-c5f2beb27610-config-volume\") pod \"dns-default-nhklh\" (UID: \"e521e734-a8d9-4748-ba28-c5f2beb27610\") " pod="openshift-dns/dns-default-nhklh" Jul 29 08:22:02.500711 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:22:02.500507 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/e521e734-a8d9-4748-ba28-c5f2beb27610-metrics-tls\") pod \"dns-default-nhklh\" (UID: \"e521e734-a8d9-4748-ba28-c5f2beb27610\") " pod="openshift-dns/dns-default-nhklh" Jul 29 08:22:02.500711 ip-10-0-131-138 kubenswrapper[2563]: E0729 08:22:02.500649 2563 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 29 08:22:02.500711 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:22:02.500668 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e521e734-a8d9-4748-ba28-c5f2beb27610-tmp-dir\") pod \"dns-default-nhklh\" (UID: \"e521e734-a8d9-4748-ba28-c5f2beb27610\") " pod="openshift-dns/dns-default-nhklh" Jul 29 08:22:02.500882 ip-10-0-131-138 kubenswrapper[2563]: E0729 08:22:02.500775 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e521e734-a8d9-4748-ba28-c5f2beb27610-metrics-tls podName:e521e734-a8d9-4748-ba28-c5f2beb27610 nodeName:}" failed. No retries permitted until 2026-07-29 08:22:03.000751471 +0000 UTC m=+35.582040228 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/e521e734-a8d9-4748-ba28-c5f2beb27610-metrics-tls") pod "dns-default-nhklh" (UID: "e521e734-a8d9-4748-ba28-c5f2beb27610") : secret "dns-default-metrics-tls" not found Jul 29 08:22:02.501218 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:22:02.501179 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/e521e734-a8d9-4748-ba28-c5f2beb27610-config-volume\") pod \"dns-default-nhklh\" (UID: \"e521e734-a8d9-4748-ba28-c5f2beb27610\") " pod="openshift-dns/dns-default-nhklh" Jul 29 08:22:02.505374 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:22:02.505351 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/b052d187-d65d-4894-aaa2-5dbb4b06600e-cert\") pod \"ingress-canary-pwqh7\" (UID: \"b052d187-d65d-4894-aaa2-5dbb4b06600e\") " pod="openshift-ingress-canary/ingress-canary-pwqh7" Jul 29 08:22:02.510447 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:22:02.510426 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4qkg4\" (UniqueName: \"kubernetes.io/projected/e521e734-a8d9-4748-ba28-c5f2beb27610-kube-api-access-4qkg4\") pod \"dns-default-nhklh\" (UID: \"e521e734-a8d9-4748-ba28-c5f2beb27610\") " pod="openshift-dns/dns-default-nhklh" Jul 29 08:22:02.510558 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:22:02.510507 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dg9wt\" (UniqueName: \"kubernetes.io/projected/b052d187-d65d-4894-aaa2-5dbb4b06600e-kube-api-access-dg9wt\") pod \"ingress-canary-pwqh7\" (UID: \"b052d187-d65d-4894-aaa2-5dbb4b06600e\") " pod="openshift-ingress-canary/ingress-canary-pwqh7" Jul 29 08:22:02.567560 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:22:02.567480 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8rp9g_cfd1d8df-e6b7-4a67-81f2-5d83da541e20/konnectivity-agent/0.log" limit=1024 Jul 29 08:22:02.639588 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:22:02.639553 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-pwqh7" Jul 29 08:22:02.956983 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:22:02.956934 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-rn9kc" Jul 29 08:22:02.957160 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:22:02.956957 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-qk9kb" Jul 29 08:22:02.961820 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:22:02.961795 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Jul 29 08:22:02.961820 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:22:02.961809 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Jul 29 08:22:02.962000 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:22:02.961808 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-7c8hq\"" Jul 29 08:22:03.005273 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:22:03.005232 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/e521e734-a8d9-4748-ba28-c5f2beb27610-metrics-tls\") pod \"dns-default-nhklh\" (UID: \"e521e734-a8d9-4748-ba28-c5f2beb27610\") " pod="openshift-dns/dns-default-nhklh" Jul 29 08:22:03.005431 ip-10-0-131-138 kubenswrapper[2563]: E0729 08:22:03.005398 2563 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 29 08:22:03.005509 ip-10-0-131-138 kubenswrapper[2563]: E0729 08:22:03.005464 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e521e734-a8d9-4748-ba28-c5f2beb27610-metrics-tls podName:e521e734-a8d9-4748-ba28-c5f2beb27610 nodeName:}" failed. No retries permitted until 2026-07-29 08:22:04.005447437 +0000 UTC m=+36.586736196 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/e521e734-a8d9-4748-ba28-c5f2beb27610-metrics-tls") pod "dns-default-nhklh" (UID: "e521e734-a8d9-4748-ba28-c5f2beb27610") : secret "dns-default-metrics-tls" not found Jul 29 08:22:03.250624 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:22:03.250466 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-pwqh7"] Jul 29 08:22:03.254866 ip-10-0-131-138 kubenswrapper[2563]: W0729 08:22:03.254840 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb052d187_d65d_4894_aaa2_5dbb4b06600e.slice/crio-9937fb26ede0b2d9191e1d528b4ab22abb1741bba1522a55a22990c1133a3807 WatchSource:0}: Error finding container 9937fb26ede0b2d9191e1d528b4ab22abb1741bba1522a55a22990c1133a3807: Status 404 returned error can't find the container with id 9937fb26ede0b2d9191e1d528b4ab22abb1741bba1522a55a22990c1133a3807 Jul 29 08:22:03.957576 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:22:03.957543 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-rdclp" Jul 29 08:22:03.960838 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:22:03.960801 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" Jul 29 08:22:03.960970 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:22:03.960864 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-wscl8\"" Jul 29 08:22:03.961898 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:22:03.961883 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" Jul 29 08:22:04.013009 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:22:04.012975 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/e521e734-a8d9-4748-ba28-c5f2beb27610-metrics-tls\") pod \"dns-default-nhklh\" (UID: \"e521e734-a8d9-4748-ba28-c5f2beb27610\") " pod="openshift-dns/dns-default-nhklh" Jul 29 08:22:04.013186 ip-10-0-131-138 kubenswrapper[2563]: E0729 08:22:04.013138 2563 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 29 08:22:04.013245 ip-10-0-131-138 kubenswrapper[2563]: E0729 08:22:04.013210 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e521e734-a8d9-4748-ba28-c5f2beb27610-metrics-tls podName:e521e734-a8d9-4748-ba28-c5f2beb27610 nodeName:}" failed. No retries permitted until 2026-07-29 08:22:06.01318886 +0000 UTC m=+38.594477623 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/e521e734-a8d9-4748-ba28-c5f2beb27610-metrics-tls") pod "dns-default-nhklh" (UID: "e521e734-a8d9-4748-ba28-c5f2beb27610") : secret "dns-default-metrics-tls" not found Jul 29 08:22:04.207419 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:22:04.207378 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-pwqh7" event={"ID":"b052d187-d65d-4894-aaa2-5dbb4b06600e","Type":"ContainerStarted","Data":"9937fb26ede0b2d9191e1d528b4ab22abb1741bba1522a55a22990c1133a3807"} Jul 29 08:22:04.210727 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:22:04.210657 2563 generic.go:356] "Generic (PLEG): container finished" podID="bb5d06c1-cbbb-4e40-9e0a-43aa22af698f" containerID="92d3de1b110a2b23af12ffb2220bc1909de95141d466c561f28bd5550eb62af4" exitCode=0 Jul 29 08:22:04.210727 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:22:04.210701 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-qs8zl" event={"ID":"bb5d06c1-cbbb-4e40-9e0a-43aa22af698f","Type":"ContainerDied","Data":"92d3de1b110a2b23af12ffb2220bc1909de95141d466c561f28bd5550eb62af4"} Jul 29 08:22:05.217210 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:22:05.217172 2563 generic.go:356] "Generic (PLEG): container finished" podID="bb5d06c1-cbbb-4e40-9e0a-43aa22af698f" containerID="6b08876eabe5c94c9eea65d62dc24d8605f02e8f2fc24f42508d82f5577b81c0" exitCode=0 Jul 29 08:22:05.217723 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:22:05.217270 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-qs8zl" event={"ID":"bb5d06c1-cbbb-4e40-9e0a-43aa22af698f","Type":"ContainerDied","Data":"6b08876eabe5c94c9eea65d62dc24d8605f02e8f2fc24f42508d82f5577b81c0"} Jul 29 08:22:06.032357 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:22:06.032083 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/e521e734-a8d9-4748-ba28-c5f2beb27610-metrics-tls\") pod \"dns-default-nhklh\" (UID: \"e521e734-a8d9-4748-ba28-c5f2beb27610\") " pod="openshift-dns/dns-default-nhklh" Jul 29 08:22:06.034287 ip-10-0-131-138 kubenswrapper[2563]: E0729 08:22:06.032532 2563 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 29 08:22:06.034287 ip-10-0-131-138 kubenswrapper[2563]: E0729 08:22:06.032622 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e521e734-a8d9-4748-ba28-c5f2beb27610-metrics-tls podName:e521e734-a8d9-4748-ba28-c5f2beb27610 nodeName:}" failed. No retries permitted until 2026-07-29 08:22:10.032600867 +0000 UTC m=+42.613889637 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/e521e734-a8d9-4748-ba28-c5f2beb27610-metrics-tls") pod "dns-default-nhklh" (UID: "e521e734-a8d9-4748-ba28-c5f2beb27610") : secret "dns-default-metrics-tls" not found Jul 29 08:22:06.219593 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:22:06.219477 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8rp9g_cfd1d8df-e6b7-4a67-81f2-5d83da541e20/konnectivity-agent/0.log" limit=1024 Jul 29 08:22:06.220830 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:22:06.220802 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-pwqh7" event={"ID":"b052d187-d65d-4894-aaa2-5dbb4b06600e","Type":"ContainerStarted","Data":"960b505ff4759500a4d3027570492d51193b270d7255cf31f9296dc32f1a9155"} Jul 29 08:22:06.224231 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:22:06.224193 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-qs8zl" event={"ID":"bb5d06c1-cbbb-4e40-9e0a-43aa22af698f","Type":"ContainerStarted","Data":"e5a23ae560b88f62cb0642143fa68576967684dc1fd2d213a32c48eef91e4fff"} Jul 29 08:22:06.240031 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:22:06.239987 2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-pwqh7" podStartSLOduration=1.6706375439999999 podStartE2EDuration="4.239974557s" totalImagesPullingTime="2.569337013s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 08:22:02 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 08:22:06.239879397 +0000 UTC m=+38.821168175" watchObservedRunningTime="2026-07-29 08:22:06.239974557 +0000 UTC m=+38.821263334" Jul 29 08:22:06.269105 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:22:06.269046 2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-qs8zl" podStartSLOduration=9.09670506 podStartE2EDuration="38.269025528s" totalImagesPullingTime="29.172320468s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-29 08:21:28 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 08:22:06.266849955 +0000 UTC m=+38.848138744" watchObservedRunningTime="2026-07-29 08:22:06.269025528 +0000 UTC m=+38.850314303" Jul 29 08:22:06.319635 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:22:06.319591 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8rp9g_cfd1d8df-e6b7-4a67-81f2-5d83da541e20/konnectivity-agent/0.log" limit=1024 Jul 29 08:22:06.427120 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:22:06.427087 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8rp9g_cfd1d8df-e6b7-4a67-81f2-5d83da541e20/konnectivity-agent/0.log" limit=1024 Jul 29 08:22:06.498214 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:22:06.498137 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8rp9g_cfd1d8df-e6b7-4a67-81f2-5d83da541e20/konnectivity-agent/0.log" limit=1024 Jul 29 08:22:07.760596 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:22:07.760558 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8rp9g_cfd1d8df-e6b7-4a67-81f2-5d83da541e20/konnectivity-agent/0.log" limit=1024 Jul 29 08:22:07.763797 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:22:07.763775 2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-548b489cb8-ch694"] Jul 29 08:22:07.772022 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:22:07.772002 2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/router-default-7c7545584f-2d6gk"] Jul 29 08:22:07.772152 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:22:07.772137 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-ch694" Jul 29 08:22:07.775479 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:22:07.775460 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-tls\"" Jul 29 08:22:07.775847 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:22:07.775829 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\"" Jul 29 08:22:07.775934 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:22:07.775868 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-dockercfg-8shpd\"" Jul 29 08:22:07.778106 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:22:07.778087 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/router-default-7c7545584f-2d6gk" Jul 29 08:22:07.778907 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:22:07.778869 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\"" Jul 29 08:22:07.779087 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:22:07.779067 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemetry-config\"" Jul 29 08:22:07.781239 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:22:07.781222 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-dockercfg-v82zv\"" Jul 29 08:22:07.781341 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:22:07.781248 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-stats-default\"" Jul 29 08:22:07.781401 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:22:07.781223 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\"" Jul 29 08:22:07.781546 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:22:07.781524 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-metrics-certs-default\"" Jul 29 08:22:07.781877 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:22:07.781859 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"default-ingress-cert\"" Jul 29 08:22:07.781968 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:22:07.781955 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"service-ca-bundle\"" Jul 29 08:22:07.782928 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:22:07.782911 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\"" Jul 29 08:22:07.783318 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:22:07.783301 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-548b489cb8-ch694"] Jul 29 08:22:07.788493 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:22:07.788468 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/router-default-7c7545584f-2d6gk"] Jul 29 08:22:07.843335 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:22:07.843304 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8rp9g_cfd1d8df-e6b7-4a67-81f2-5d83da541e20/konnectivity-agent/0.log" limit=1024 Jul 29 08:22:07.926992 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:22:07.926961 2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-6748489656-q2lqd"] Jul 29 08:22:07.946125 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:22:07.946087 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/bf0cdb1f-7233-4c76-8159-b233968ce8e5-service-ca-bundle\") pod \"router-default-7c7545584f-2d6gk\" (UID: \"bf0cdb1f-7233-4c76-8159-b233968ce8e5\") " pod="openshift-ingress/router-default-7c7545584f-2d6gk" Jul 29 08:22:07.946125 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:22:07.946120 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7br7c\" (UniqueName: \"kubernetes.io/projected/bf0cdb1f-7233-4c76-8159-b233968ce8e5-kube-api-access-7br7c\") pod \"router-default-7c7545584f-2d6gk\" (UID: \"bf0cdb1f-7233-4c76-8159-b233968ce8e5\") " pod="openshift-ingress/router-default-7c7545584f-2d6gk" Jul 29 08:22:07.946430 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:22:07.946140 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/ed443996-4f98-44b5-9476-37a0c3c23099-telemetry-config\") pod \"cluster-monitoring-operator-548b489cb8-ch694\" (UID: \"ed443996-4f98-44b5-9476-37a0c3c23099\") " pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-ch694" Jul 29 08:22:07.946430 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:22:07.946208 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/bf0cdb1f-7233-4c76-8159-b233968ce8e5-default-certificate\") pod \"router-default-7c7545584f-2d6gk\" (UID: \"bf0cdb1f-7233-4c76-8159-b233968ce8e5\") " pod="openshift-ingress/router-default-7c7545584f-2d6gk" Jul 29 08:22:07.946430 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:22:07.946243 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-56m8k\" (UniqueName: \"kubernetes.io/projected/ed443996-4f98-44b5-9476-37a0c3c23099-kube-api-access-56m8k\") pod \"cluster-monitoring-operator-548b489cb8-ch694\" (UID: \"ed443996-4f98-44b5-9476-37a0c3c23099\") " pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-ch694" Jul 29 08:22:07.946430 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:22:07.946356 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/bf0cdb1f-7233-4c76-8159-b233968ce8e5-stats-auth\") pod \"router-default-7c7545584f-2d6gk\" (UID: \"bf0cdb1f-7233-4c76-8159-b233968ce8e5\") " pod="openshift-ingress/router-default-7c7545584f-2d6gk" Jul 29 08:22:07.946430 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:22:07.946382 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/ed443996-4f98-44b5-9476-37a0c3c23099-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-548b489cb8-ch694\" (UID: \"ed443996-4f98-44b5-9476-37a0c3c23099\") " pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-ch694" Jul 29 08:22:07.946430 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:22:07.946404 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/bf0cdb1f-7233-4c76-8159-b233968ce8e5-metrics-certs\") pod \"router-default-7c7545584f-2d6gk\" (UID: \"bf0cdb1f-7233-4c76-8159-b233968ce8e5\") " pod="openshift-ingress/router-default-7c7545584f-2d6gk" Jul 29 08:22:07.951937 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:22:07.951914 2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-549c6d54cf-pc5qx"] Jul 29 08:22:07.952058 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:22:07.952029 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-samples-operator/cluster-samples-operator-6748489656-q2lqd" Jul 29 08:22:07.953833 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:22:07.953702 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8rp9g_cfd1d8df-e6b7-4a67-81f2-5d83da541e20/konnectivity-agent/0.log" limit=1024 Jul 29 08:22:07.954676 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:22:07.954655 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-samples-operator\"/\"kube-root-ca.crt\"" Jul 29 08:22:07.954825 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:22:07.954808 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-samples-operator\"/\"samples-operator-tls\"" Jul 29 08:22:07.954990 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:22:07.954969 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-samples-operator\"/\"openshift-service-ca.crt\"" Jul 29 08:22:07.955132 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:22:07.955114 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-samples-operator\"/\"cluster-samples-operator-dockercfg-2zxzh\"" Jul 29 08:22:07.976944 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:22:07.976917 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-storage-operator/volume-data-source-validator-549c6d54cf-pc5qx" Jul 29 08:22:07.980146 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:22:07.980128 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-6748489656-q2lqd"] Jul 29 08:22:07.980233 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:22:07.980168 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-549c6d54cf-pc5qx"] Jul 29 08:22:07.980818 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:22:07.980794 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-storage-operator\"/\"openshift-service-ca.crt\"" Jul 29 08:22:07.980818 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:22:07.980803 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-storage-operator\"/\"kube-root-ca.crt\"" Jul 29 08:22:07.980959 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:22:07.980827 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-storage-operator\"/\"volume-data-source-validator-dockercfg-g7v49\"" Jul 29 08:22:07.994178 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:22:07.994156 2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-operator-5885fdb759-84zrp"] Jul 29 08:22:08.012130 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:22:08.012075 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-operator-5885fdb759-84zrp"] Jul 29 08:22:08.012228 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:22:08.012159 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-operator-5885fdb759-84zrp" Jul 29 08:22:08.015603 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:22:08.015583 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"openshift-insights-serving-cert\"" Jul 29 08:22:08.015769 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:22:08.015747 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\"" Jul 29 08:22:08.015859 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:22:08.015844 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\"" Jul 29 08:22:08.016532 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:22:08.016515 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"operator-dockercfg-zkc27\"" Jul 29 08:22:08.016974 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:22:08.016959 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"service-ca-bundle\"" Jul 29 08:22:08.021412 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:22:08.021396 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"trusted-ca-bundle\"" Jul 29 08:22:08.046783 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:22:08.046754 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/bf0cdb1f-7233-4c76-8159-b233968ce8e5-service-ca-bundle\") pod \"router-default-7c7545584f-2d6gk\" (UID: \"bf0cdb1f-7233-4c76-8159-b233968ce8e5\") " pod="openshift-ingress/router-default-7c7545584f-2d6gk" Jul 29 08:22:08.046964 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:22:08.046792 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7br7c\" (UniqueName: \"kubernetes.io/projected/bf0cdb1f-7233-4c76-8159-b233968ce8e5-kube-api-access-7br7c\") pod \"router-default-7c7545584f-2d6gk\" (UID: \"bf0cdb1f-7233-4c76-8159-b233968ce8e5\") " pod="openshift-ingress/router-default-7c7545584f-2d6gk" Jul 29 08:22:08.046964 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:22:08.046817 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/ed443996-4f98-44b5-9476-37a0c3c23099-telemetry-config\") pod \"cluster-monitoring-operator-548b489cb8-ch694\" (UID: \"ed443996-4f98-44b5-9476-37a0c3c23099\") " pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-ch694" Jul 29 08:22:08.046964 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:22:08.046851 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/bf0cdb1f-7233-4c76-8159-b233968ce8e5-default-certificate\") pod \"router-default-7c7545584f-2d6gk\" (UID: \"bf0cdb1f-7233-4c76-8159-b233968ce8e5\") " pod="openshift-ingress/router-default-7c7545584f-2d6gk" Jul 29 08:22:08.046964 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:22:08.046875 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-56m8k\" (UniqueName: \"kubernetes.io/projected/ed443996-4f98-44b5-9476-37a0c3c23099-kube-api-access-56m8k\") pod \"cluster-monitoring-operator-548b489cb8-ch694\" (UID: \"ed443996-4f98-44b5-9476-37a0c3c23099\") " pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-ch694" Jul 29 08:22:08.046964 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:22:08.046903 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4rj2h\" (UniqueName: \"kubernetes.io/projected/d2793bc6-e6c2-4799-845e-ee32f0a85693-kube-api-access-4rj2h\") pod \"cluster-samples-operator-6748489656-q2lqd\" (UID: \"d2793bc6-e6c2-4799-845e-ee32f0a85693\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6748489656-q2lqd" Jul 29 08:22:08.046964 ip-10-0-131-138 kubenswrapper[2563]: E0729 08:22:08.046945 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/bf0cdb1f-7233-4c76-8159-b233968ce8e5-service-ca-bundle podName:bf0cdb1f-7233-4c76-8159-b233968ce8e5 nodeName:}" failed. No retries permitted until 2026-07-29 08:22:08.54692385 +0000 UTC m=+41.128212627 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/bf0cdb1f-7233-4c76-8159-b233968ce8e5-service-ca-bundle") pod "router-default-7c7545584f-2d6gk" (UID: "bf0cdb1f-7233-4c76-8159-b233968ce8e5") : configmap references non-existent config key: service-ca.crt Jul 29 08:22:08.047280 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:22:08.047019 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/bf0cdb1f-7233-4c76-8159-b233968ce8e5-stats-auth\") pod \"router-default-7c7545584f-2d6gk\" (UID: \"bf0cdb1f-7233-4c76-8159-b233968ce8e5\") " pod="openshift-ingress/router-default-7c7545584f-2d6gk" Jul 29 08:22:08.047280 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:22:08.047047 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/ed443996-4f98-44b5-9476-37a0c3c23099-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-548b489cb8-ch694\" (UID: \"ed443996-4f98-44b5-9476-37a0c3c23099\") " pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-ch694" Jul 29 08:22:08.047280 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:22:08.047114 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/d2793bc6-e6c2-4799-845e-ee32f0a85693-samples-operator-tls\") pod \"cluster-samples-operator-6748489656-q2lqd\" (UID: \"d2793bc6-e6c2-4799-845e-ee32f0a85693\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6748489656-q2lqd" Jul 29 08:22:08.047280 ip-10-0-131-138 kubenswrapper[2563]: E0729 08:22:08.047123 2563 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Jul 29 08:22:08.047280 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:22:08.047172 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/bf0cdb1f-7233-4c76-8159-b233968ce8e5-metrics-certs\") pod \"router-default-7c7545584f-2d6gk\" (UID: \"bf0cdb1f-7233-4c76-8159-b233968ce8e5\") " pod="openshift-ingress/router-default-7c7545584f-2d6gk" Jul 29 08:22:08.047280 ip-10-0-131-138 kubenswrapper[2563]: E0729 08:22:08.047209 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ed443996-4f98-44b5-9476-37a0c3c23099-cluster-monitoring-operator-tls podName:ed443996-4f98-44b5-9476-37a0c3c23099 nodeName:}" failed. No retries permitted until 2026-07-29 08:22:08.547189506 +0000 UTC m=+41.128478288 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/ed443996-4f98-44b5-9476-37a0c3c23099-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-548b489cb8-ch694" (UID: "ed443996-4f98-44b5-9476-37a0c3c23099") : secret "cluster-monitoring-operator-tls" not found Jul 29 08:22:08.047280 ip-10-0-131-138 kubenswrapper[2563]: E0729 08:22:08.047238 2563 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Jul 29 08:22:08.047514 ip-10-0-131-138 kubenswrapper[2563]: E0729 08:22:08.047315 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/bf0cdb1f-7233-4c76-8159-b233968ce8e5-metrics-certs podName:bf0cdb1f-7233-4c76-8159-b233968ce8e5 nodeName:}" failed. No retries permitted until 2026-07-29 08:22:08.547300714 +0000 UTC m=+41.128589470 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/bf0cdb1f-7233-4c76-8159-b233968ce8e5-metrics-certs") pod "router-default-7c7545584f-2d6gk" (UID: "bf0cdb1f-7233-4c76-8159-b233968ce8e5") : secret "router-metrics-certs-default" not found Jul 29 08:22:08.047669 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:22:08.047643 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/ed443996-4f98-44b5-9476-37a0c3c23099-telemetry-config\") pod \"cluster-monitoring-operator-548b489cb8-ch694\" (UID: \"ed443996-4f98-44b5-9476-37a0c3c23099\") " pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-ch694" Jul 29 08:22:08.051100 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:22:08.051080 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/bf0cdb1f-7233-4c76-8159-b233968ce8e5-default-certificate\") pod \"router-default-7c7545584f-2d6gk\" (UID: \"bf0cdb1f-7233-4c76-8159-b233968ce8e5\") " pod="openshift-ingress/router-default-7c7545584f-2d6gk" Jul 29 08:22:08.051181 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:22:08.051084 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/bf0cdb1f-7233-4c76-8159-b233968ce8e5-stats-auth\") pod \"router-default-7c7545584f-2d6gk\" (UID: \"bf0cdb1f-7233-4c76-8159-b233968ce8e5\") " pod="openshift-ingress/router-default-7c7545584f-2d6gk" Jul 29 08:22:08.052223 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:22:08.052199 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8rp9g_cfd1d8df-e6b7-4a67-81f2-5d83da541e20/konnectivity-agent/0.log" limit=1024 Jul 29 08:22:08.058629 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:22:08.058609 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-56m8k\" (UniqueName: \"kubernetes.io/projected/ed443996-4f98-44b5-9476-37a0c3c23099-kube-api-access-56m8k\") pod \"cluster-monitoring-operator-548b489cb8-ch694\" (UID: \"ed443996-4f98-44b5-9476-37a0c3c23099\") " pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-ch694" Jul 29 08:22:08.059787 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:22:08.059764 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7br7c\" (UniqueName: \"kubernetes.io/projected/bf0cdb1f-7233-4c76-8159-b233968ce8e5-kube-api-access-7br7c\") pod \"router-default-7c7545584f-2d6gk\" (UID: \"bf0cdb1f-7233-4c76-8159-b233968ce8e5\") " pod="openshift-ingress/router-default-7c7545584f-2d6gk" Jul 29 08:22:08.124677 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:22:08.124627 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8rp9g_cfd1d8df-e6b7-4a67-81f2-5d83da541e20/konnectivity-agent/0.log" limit=1024 Jul 29 08:22:08.148361 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:22:08.148327 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cbxzc\" (UniqueName: \"kubernetes.io/projected/791f1046-8a1f-41c4-b7a6-70862705aad6-kube-api-access-cbxzc\") pod \"insights-operator-5885fdb759-84zrp\" (UID: \"791f1046-8a1f-41c4-b7a6-70862705aad6\") " pod="openshift-insights/insights-operator-5885fdb759-84zrp" Jul 29 08:22:08.148504 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:22:08.148373 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/d2793bc6-e6c2-4799-845e-ee32f0a85693-samples-operator-tls\") pod \"cluster-samples-operator-6748489656-q2lqd\" (UID: \"d2793bc6-e6c2-4799-845e-ee32f0a85693\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6748489656-q2lqd" Jul 29 08:22:08.148504 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:22:08.148428 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/791f1046-8a1f-41c4-b7a6-70862705aad6-tmp\") pod \"insights-operator-5885fdb759-84zrp\" (UID: \"791f1046-8a1f-41c4-b7a6-70862705aad6\") " pod="openshift-insights/insights-operator-5885fdb759-84zrp" Jul 29 08:22:08.148504 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:22:08.148456 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/791f1046-8a1f-41c4-b7a6-70862705aad6-snapshots\") pod \"insights-operator-5885fdb759-84zrp\" (UID: \"791f1046-8a1f-41c4-b7a6-70862705aad6\") " pod="openshift-insights/insights-operator-5885fdb759-84zrp" Jul 29 08:22:08.148504 ip-10-0-131-138 kubenswrapper[2563]: E0729 08:22:08.148486 2563 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Jul 29 08:22:08.148705 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:22:08.148534 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/791f1046-8a1f-41c4-b7a6-70862705aad6-service-ca-bundle\") pod \"insights-operator-5885fdb759-84zrp\" (UID: \"791f1046-8a1f-41c4-b7a6-70862705aad6\") " pod="openshift-insights/insights-operator-5885fdb759-84zrp" Jul 29 08:22:08.148705 ip-10-0-131-138 kubenswrapper[2563]: E0729 08:22:08.148542 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d2793bc6-e6c2-4799-845e-ee32f0a85693-samples-operator-tls podName:d2793bc6-e6c2-4799-845e-ee32f0a85693 nodeName:}" failed. No retries permitted until 2026-07-29 08:22:08.648523632 +0000 UTC m=+41.229812389 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/d2793bc6-e6c2-4799-845e-ee32f0a85693-samples-operator-tls") pod "cluster-samples-operator-6748489656-q2lqd" (UID: "d2793bc6-e6c2-4799-845e-ee32f0a85693") : secret "samples-operator-tls" not found Jul 29 08:22:08.148705 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:22:08.148565 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/791f1046-8a1f-41c4-b7a6-70862705aad6-serving-cert\") pod \"insights-operator-5885fdb759-84zrp\" (UID: \"791f1046-8a1f-41c4-b7a6-70862705aad6\") " pod="openshift-insights/insights-operator-5885fdb759-84zrp" Jul 29 08:22:08.148705 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:22:08.148595 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-99hpt\" (UniqueName: \"kubernetes.io/projected/c61e3f1b-de05-4466-97b5-21d2e47af9df-kube-api-access-99hpt\") pod \"volume-data-source-validator-549c6d54cf-pc5qx\" (UID: \"c61e3f1b-de05-4466-97b5-21d2e47af9df\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-549c6d54cf-pc5qx" Jul 29 08:22:08.148705 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:22:08.148616 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/791f1046-8a1f-41c4-b7a6-70862705aad6-trusted-ca-bundle\") pod \"insights-operator-5885fdb759-84zrp\" (UID: \"791f1046-8a1f-41c4-b7a6-70862705aad6\") " pod="openshift-insights/insights-operator-5885fdb759-84zrp" Jul 29 08:22:08.148885 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:22:08.148705 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4rj2h\" (UniqueName: \"kubernetes.io/projected/d2793bc6-e6c2-4799-845e-ee32f0a85693-kube-api-access-4rj2h\") pod \"cluster-samples-operator-6748489656-q2lqd\" (UID: \"d2793bc6-e6c2-4799-845e-ee32f0a85693\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6748489656-q2lqd" Jul 29 08:22:08.158239 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:22:08.158210 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4rj2h\" (UniqueName: \"kubernetes.io/projected/d2793bc6-e6c2-4799-845e-ee32f0a85693-kube-api-access-4rj2h\") pod \"cluster-samples-operator-6748489656-q2lqd\" (UID: \"d2793bc6-e6c2-4799-845e-ee32f0a85693\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6748489656-q2lqd" Jul 29 08:22:08.250024 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:22:08.249988 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cbxzc\" (UniqueName: \"kubernetes.io/projected/791f1046-8a1f-41c4-b7a6-70862705aad6-kube-api-access-cbxzc\") pod \"insights-operator-5885fdb759-84zrp\" (UID: \"791f1046-8a1f-41c4-b7a6-70862705aad6\") " pod="openshift-insights/insights-operator-5885fdb759-84zrp" Jul 29 08:22:08.250194 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:22:08.250036 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/791f1046-8a1f-41c4-b7a6-70862705aad6-tmp\") pod \"insights-operator-5885fdb759-84zrp\" (UID: \"791f1046-8a1f-41c4-b7a6-70862705aad6\") " pod="openshift-insights/insights-operator-5885fdb759-84zrp" Jul 29 08:22:08.250194 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:22:08.250058 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/791f1046-8a1f-41c4-b7a6-70862705aad6-snapshots\") pod \"insights-operator-5885fdb759-84zrp\" (UID: \"791f1046-8a1f-41c4-b7a6-70862705aad6\") " pod="openshift-insights/insights-operator-5885fdb759-84zrp" Jul 29 08:22:08.250194 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:22:08.250102 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/791f1046-8a1f-41c4-b7a6-70862705aad6-service-ca-bundle\") pod \"insights-operator-5885fdb759-84zrp\" (UID: \"791f1046-8a1f-41c4-b7a6-70862705aad6\") " pod="openshift-insights/insights-operator-5885fdb759-84zrp" Jul 29 08:22:08.250194 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:22:08.250124 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/791f1046-8a1f-41c4-b7a6-70862705aad6-serving-cert\") pod \"insights-operator-5885fdb759-84zrp\" (UID: \"791f1046-8a1f-41c4-b7a6-70862705aad6\") " pod="openshift-insights/insights-operator-5885fdb759-84zrp" Jul 29 08:22:08.250194 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:22:08.250155 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-99hpt\" (UniqueName: \"kubernetes.io/projected/c61e3f1b-de05-4466-97b5-21d2e47af9df-kube-api-access-99hpt\") pod \"volume-data-source-validator-549c6d54cf-pc5qx\" (UID: \"c61e3f1b-de05-4466-97b5-21d2e47af9df\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-549c6d54cf-pc5qx" Jul 29 08:22:08.250194 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:22:08.250182 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/791f1046-8a1f-41c4-b7a6-70862705aad6-trusted-ca-bundle\") pod \"insights-operator-5885fdb759-84zrp\" (UID: \"791f1046-8a1f-41c4-b7a6-70862705aad6\") " pod="openshift-insights/insights-operator-5885fdb759-84zrp" Jul 29 08:22:08.250532 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:22:08.250507 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/791f1046-8a1f-41c4-b7a6-70862705aad6-tmp\") pod \"insights-operator-5885fdb759-84zrp\" (UID: \"791f1046-8a1f-41c4-b7a6-70862705aad6\") " pod="openshift-insights/insights-operator-5885fdb759-84zrp" Jul 29 08:22:08.250719 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:22:08.250698 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/791f1046-8a1f-41c4-b7a6-70862705aad6-snapshots\") pod \"insights-operator-5885fdb759-84zrp\" (UID: \"791f1046-8a1f-41c4-b7a6-70862705aad6\") " pod="openshift-insights/insights-operator-5885fdb759-84zrp" Jul 29 08:22:08.250780 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:22:08.250701 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/791f1046-8a1f-41c4-b7a6-70862705aad6-service-ca-bundle\") pod \"insights-operator-5885fdb759-84zrp\" (UID: \"791f1046-8a1f-41c4-b7a6-70862705aad6\") " pod="openshift-insights/insights-operator-5885fdb759-84zrp" Jul 29 08:22:08.251020 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:22:08.251000 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/791f1046-8a1f-41c4-b7a6-70862705aad6-trusted-ca-bundle\") pod \"insights-operator-5885fdb759-84zrp\" (UID: \"791f1046-8a1f-41c4-b7a6-70862705aad6\") " pod="openshift-insights/insights-operator-5885fdb759-84zrp" Jul 29 08:22:08.252730 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:22:08.252713 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/791f1046-8a1f-41c4-b7a6-70862705aad6-serving-cert\") pod \"insights-operator-5885fdb759-84zrp\" (UID: \"791f1046-8a1f-41c4-b7a6-70862705aad6\") " pod="openshift-insights/insights-operator-5885fdb759-84zrp" Jul 29 08:22:08.259679 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:22:08.259660 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-cbxzc\" (UniqueName: \"kubernetes.io/projected/791f1046-8a1f-41c4-b7a6-70862705aad6-kube-api-access-cbxzc\") pod \"insights-operator-5885fdb759-84zrp\" (UID: \"791f1046-8a1f-41c4-b7a6-70862705aad6\") " pod="openshift-insights/insights-operator-5885fdb759-84zrp" Jul 29 08:22:08.259766 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:22:08.259736 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-99hpt\" (UniqueName: \"kubernetes.io/projected/c61e3f1b-de05-4466-97b5-21d2e47af9df-kube-api-access-99hpt\") pod \"volume-data-source-validator-549c6d54cf-pc5qx\" (UID: \"c61e3f1b-de05-4466-97b5-21d2e47af9df\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-549c6d54cf-pc5qx" Jul 29 08:22:08.285847 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:22:08.285784 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-storage-operator/volume-data-source-validator-549c6d54cf-pc5qx" Jul 29 08:22:08.321179 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:22:08.321162 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-operator-5885fdb759-84zrp" Jul 29 08:22:08.475714 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:22:08.475677 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-549c6d54cf-pc5qx"] Jul 29 08:22:08.478228 ip-10-0-131-138 kubenswrapper[2563]: W0729 08:22:08.478201 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc61e3f1b_de05_4466_97b5_21d2e47af9df.slice/crio-079a408c2b157d432a862f8ee7518746a7d4ed12f05e8355fe96b4cda974ec14 WatchSource:0}: Error finding container 079a408c2b157d432a862f8ee7518746a7d4ed12f05e8355fe96b4cda974ec14: Status 404 returned error can't find the container with id 079a408c2b157d432a862f8ee7518746a7d4ed12f05e8355fe96b4cda974ec14 Jul 29 08:22:08.490136 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:22:08.490116 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-operator-5885fdb759-84zrp"] Jul 29 08:22:08.492854 ip-10-0-131-138 kubenswrapper[2563]: W0729 08:22:08.492831 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod791f1046_8a1f_41c4_b7a6_70862705aad6.slice/crio-d763f9518f3fd0b759d09820274683bba4c3273696f126a3b0d6575dcf829f4b WatchSource:0}: Error finding container d763f9518f3fd0b759d09820274683bba4c3273696f126a3b0d6575dcf829f4b: Status 404 returned error can't find the container with id d763f9518f3fd0b759d09820274683bba4c3273696f126a3b0d6575dcf829f4b Jul 29 08:22:08.553073 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:22:08.552984 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/bf0cdb1f-7233-4c76-8159-b233968ce8e5-service-ca-bundle\") pod \"router-default-7c7545584f-2d6gk\" (UID: \"bf0cdb1f-7233-4c76-8159-b233968ce8e5\") " pod="openshift-ingress/router-default-7c7545584f-2d6gk" Jul 29 08:22:08.553073 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:22:08.553046 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/ed443996-4f98-44b5-9476-37a0c3c23099-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-548b489cb8-ch694\" (UID: \"ed443996-4f98-44b5-9476-37a0c3c23099\") " pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-ch694" Jul 29 08:22:08.553073 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:22:08.553076 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/bf0cdb1f-7233-4c76-8159-b233968ce8e5-metrics-certs\") pod \"router-default-7c7545584f-2d6gk\" (UID: \"bf0cdb1f-7233-4c76-8159-b233968ce8e5\") " pod="openshift-ingress/router-default-7c7545584f-2d6gk" Jul 29 08:22:08.553358 ip-10-0-131-138 kubenswrapper[2563]: E0729 08:22:08.553157 2563 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Jul 29 08:22:08.553358 ip-10-0-131-138 kubenswrapper[2563]: E0729 08:22:08.553174 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/bf0cdb1f-7233-4c76-8159-b233968ce8e5-service-ca-bundle podName:bf0cdb1f-7233-4c76-8159-b233968ce8e5 nodeName:}" failed. No retries permitted until 2026-07-29 08:22:09.553150376 +0000 UTC m=+42.134439149 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/bf0cdb1f-7233-4c76-8159-b233968ce8e5-service-ca-bundle") pod "router-default-7c7545584f-2d6gk" (UID: "bf0cdb1f-7233-4c76-8159-b233968ce8e5") : configmap references non-existent config key: service-ca.crt Jul 29 08:22:08.553358 ip-10-0-131-138 kubenswrapper[2563]: E0729 08:22:08.553184 2563 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Jul 29 08:22:08.553358 ip-10-0-131-138 kubenswrapper[2563]: E0729 08:22:08.553201 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/bf0cdb1f-7233-4c76-8159-b233968ce8e5-metrics-certs podName:bf0cdb1f-7233-4c76-8159-b233968ce8e5 nodeName:}" failed. No retries permitted until 2026-07-29 08:22:09.553190203 +0000 UTC m=+42.134478959 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/bf0cdb1f-7233-4c76-8159-b233968ce8e5-metrics-certs") pod "router-default-7c7545584f-2d6gk" (UID: "bf0cdb1f-7233-4c76-8159-b233968ce8e5") : secret "router-metrics-certs-default" not found Jul 29 08:22:08.553358 ip-10-0-131-138 kubenswrapper[2563]: E0729 08:22:08.553227 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ed443996-4f98-44b5-9476-37a0c3c23099-cluster-monitoring-operator-tls podName:ed443996-4f98-44b5-9476-37a0c3c23099 nodeName:}" failed. No retries permitted until 2026-07-29 08:22:09.55321514 +0000 UTC m=+42.134503900 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/ed443996-4f98-44b5-9476-37a0c3c23099-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-548b489cb8-ch694" (UID: "ed443996-4f98-44b5-9476-37a0c3c23099") : secret "cluster-monitoring-operator-tls" not found Jul 29 08:22:08.653897 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:22:08.653859 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/d2793bc6-e6c2-4799-845e-ee32f0a85693-samples-operator-tls\") pod \"cluster-samples-operator-6748489656-q2lqd\" (UID: \"d2793bc6-e6c2-4799-845e-ee32f0a85693\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6748489656-q2lqd" Jul 29 08:22:08.654071 ip-10-0-131-138 kubenswrapper[2563]: E0729 08:22:08.654013 2563 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Jul 29 08:22:08.654110 ip-10-0-131-138 kubenswrapper[2563]: E0729 08:22:08.654086 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d2793bc6-e6c2-4799-845e-ee32f0a85693-samples-operator-tls podName:d2793bc6-e6c2-4799-845e-ee32f0a85693 nodeName:}" failed. No retries permitted until 2026-07-29 08:22:09.654070174 +0000 UTC m=+42.235358931 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/d2793bc6-e6c2-4799-845e-ee32f0a85693-samples-operator-tls") pod "cluster-samples-operator-6748489656-q2lqd" (UID: "d2793bc6-e6c2-4799-845e-ee32f0a85693") : secret "samples-operator-tls" not found Jul 29 08:22:09.232758 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:22:09.232719 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-5885fdb759-84zrp" event={"ID":"791f1046-8a1f-41c4-b7a6-70862705aad6","Type":"ContainerStarted","Data":"d763f9518f3fd0b759d09820274683bba4c3273696f126a3b0d6575dcf829f4b"} Jul 29 08:22:09.233734 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:22:09.233711 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-storage-operator/volume-data-source-validator-549c6d54cf-pc5qx" event={"ID":"c61e3f1b-de05-4466-97b5-21d2e47af9df","Type":"ContainerStarted","Data":"079a408c2b157d432a862f8ee7518746a7d4ed12f05e8355fe96b4cda974ec14"} Jul 29 08:22:09.562139 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:22:09.562046 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/ed443996-4f98-44b5-9476-37a0c3c23099-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-548b489cb8-ch694\" (UID: \"ed443996-4f98-44b5-9476-37a0c3c23099\") " pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-ch694" Jul 29 08:22:09.562139 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:22:09.562097 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/bf0cdb1f-7233-4c76-8159-b233968ce8e5-metrics-certs\") pod \"router-default-7c7545584f-2d6gk\" (UID: \"bf0cdb1f-7233-4c76-8159-b233968ce8e5\") " pod="openshift-ingress/router-default-7c7545584f-2d6gk" Jul 29 08:22:09.562386 ip-10-0-131-138 kubenswrapper[2563]: E0729 08:22:09.562188 2563 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Jul 29 08:22:09.562386 ip-10-0-131-138 kubenswrapper[2563]: E0729 08:22:09.562204 2563 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Jul 29 08:22:09.562386 ip-10-0-131-138 kubenswrapper[2563]: E0729 08:22:09.562270 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/bf0cdb1f-7233-4c76-8159-b233968ce8e5-metrics-certs podName:bf0cdb1f-7233-4c76-8159-b233968ce8e5 nodeName:}" failed. No retries permitted until 2026-07-29 08:22:11.562235139 +0000 UTC m=+44.143523896 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/bf0cdb1f-7233-4c76-8159-b233968ce8e5-metrics-certs") pod "router-default-7c7545584f-2d6gk" (UID: "bf0cdb1f-7233-4c76-8159-b233968ce8e5") : secret "router-metrics-certs-default" not found Jul 29 08:22:09.562386 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:22:09.562304 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/bf0cdb1f-7233-4c76-8159-b233968ce8e5-service-ca-bundle\") pod \"router-default-7c7545584f-2d6gk\" (UID: \"bf0cdb1f-7233-4c76-8159-b233968ce8e5\") " pod="openshift-ingress/router-default-7c7545584f-2d6gk" Jul 29 08:22:09.562386 ip-10-0-131-138 kubenswrapper[2563]: E0729 08:22:09.562341 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ed443996-4f98-44b5-9476-37a0c3c23099-cluster-monitoring-operator-tls podName:ed443996-4f98-44b5-9476-37a0c3c23099 nodeName:}" failed. No retries permitted until 2026-07-29 08:22:11.562306143 +0000 UTC m=+44.143594903 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/ed443996-4f98-44b5-9476-37a0c3c23099-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-548b489cb8-ch694" (UID: "ed443996-4f98-44b5-9476-37a0c3c23099") : secret "cluster-monitoring-operator-tls" not found Jul 29 08:22:09.562386 ip-10-0-131-138 kubenswrapper[2563]: E0729 08:22:09.562366 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/bf0cdb1f-7233-4c76-8159-b233968ce8e5-service-ca-bundle podName:bf0cdb1f-7233-4c76-8159-b233968ce8e5 nodeName:}" failed. No retries permitted until 2026-07-29 08:22:11.562358806 +0000 UTC m=+44.143647562 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/bf0cdb1f-7233-4c76-8159-b233968ce8e5-service-ca-bundle") pod "router-default-7c7545584f-2d6gk" (UID: "bf0cdb1f-7233-4c76-8159-b233968ce8e5") : configmap references non-existent config key: service-ca.crt Jul 29 08:22:09.663274 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:22:09.663224 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/d2793bc6-e6c2-4799-845e-ee32f0a85693-samples-operator-tls\") pod \"cluster-samples-operator-6748489656-q2lqd\" (UID: \"d2793bc6-e6c2-4799-845e-ee32f0a85693\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6748489656-q2lqd" Jul 29 08:22:09.663413 ip-10-0-131-138 kubenswrapper[2563]: E0729 08:22:09.663385 2563 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Jul 29 08:22:09.663472 ip-10-0-131-138 kubenswrapper[2563]: E0729 08:22:09.663458 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d2793bc6-e6c2-4799-845e-ee32f0a85693-samples-operator-tls podName:d2793bc6-e6c2-4799-845e-ee32f0a85693 nodeName:}" failed. No retries permitted until 2026-07-29 08:22:11.66344021 +0000 UTC m=+44.244728980 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/d2793bc6-e6c2-4799-845e-ee32f0a85693-samples-operator-tls") pod "cluster-samples-operator-6748489656-q2lqd" (UID: "d2793bc6-e6c2-4799-845e-ee32f0a85693") : secret "samples-operator-tls" not found Jul 29 08:22:10.066722 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:22:10.066686 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/e521e734-a8d9-4748-ba28-c5f2beb27610-metrics-tls\") pod \"dns-default-nhklh\" (UID: \"e521e734-a8d9-4748-ba28-c5f2beb27610\") " pod="openshift-dns/dns-default-nhklh" Jul 29 08:22:10.066929 ip-10-0-131-138 kubenswrapper[2563]: E0729 08:22:10.066872 2563 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 29 08:22:10.066997 ip-10-0-131-138 kubenswrapper[2563]: E0729 08:22:10.066939 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e521e734-a8d9-4748-ba28-c5f2beb27610-metrics-tls podName:e521e734-a8d9-4748-ba28-c5f2beb27610 nodeName:}" failed. No retries permitted until 2026-07-29 08:22:18.066922027 +0000 UTC m=+50.648210783 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/e521e734-a8d9-4748-ba28-c5f2beb27610-metrics-tls") pod "dns-default-nhklh" (UID: "e521e734-a8d9-4748-ba28-c5f2beb27610") : secret "dns-default-metrics-tls" not found Jul 29 08:22:11.240053 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:22:11.240016 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-storage-operator/volume-data-source-validator-549c6d54cf-pc5qx" event={"ID":"c61e3f1b-de05-4466-97b5-21d2e47af9df","Type":"ContainerStarted","Data":"f3ae71e798b25e1c0ec229744cbbbff8518f423ecf72377aeee7b467c9f426cb"} Jul 29 08:22:11.259154 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:22:11.259106 2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-storage-operator/volume-data-source-validator-549c6d54cf-pc5qx" podStartSLOduration=2.363650676 podStartE2EDuration="4.259091414s" totalImagesPullingTime="1.895440738s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 08:22:07 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 08:22:11.259021633 +0000 UTC m=+43.840310411" watchObservedRunningTime="2026-07-29 08:22:11.259091414 +0000 UTC m=+43.840380252" Jul 29 08:22:11.338008 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:22:11.337977 2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-network-diagnostics/network-check-source-7d88cb55d8-dr9xk"] Jul 29 08:22:11.350977 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:22:11.350942 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-source-7d88cb55d8-dr9xk" Jul 29 08:22:11.351601 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:22:11.351573 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-source-7d88cb55d8-dr9xk"] Jul 29 08:22:11.353750 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:22:11.353730 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"network-diagnostics-dockercfg-rlnbb\"" Jul 29 08:22:11.375803 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:22:11.375781 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8rp9g_cfd1d8df-e6b7-4a67-81f2-5d83da541e20/konnectivity-agent/0.log" limit=1024 Jul 29 08:22:11.435161 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:22:11.435125 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8rp9g_cfd1d8df-e6b7-4a67-81f2-5d83da541e20/konnectivity-agent/0.log" limit=1024 Jul 29 08:22:11.479015 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:22:11.478979 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zdpg9\" (UniqueName: \"kubernetes.io/projected/3eef9ce5-17eb-4fbe-82d3-dd9b1cb17e93-kube-api-access-zdpg9\") pod \"network-check-source-7d88cb55d8-dr9xk\" (UID: \"3eef9ce5-17eb-4fbe-82d3-dd9b1cb17e93\") " pod="openshift-network-diagnostics/network-check-source-7d88cb55d8-dr9xk" Jul 29 08:22:11.579696 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:22:11.579669 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zdpg9\" (UniqueName: \"kubernetes.io/projected/3eef9ce5-17eb-4fbe-82d3-dd9b1cb17e93-kube-api-access-zdpg9\") pod \"network-check-source-7d88cb55d8-dr9xk\" (UID: \"3eef9ce5-17eb-4fbe-82d3-dd9b1cb17e93\") " pod="openshift-network-diagnostics/network-check-source-7d88cb55d8-dr9xk" Jul 29 08:22:11.579785 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:22:11.579740 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/bf0cdb1f-7233-4c76-8159-b233968ce8e5-service-ca-bundle\") pod \"router-default-7c7545584f-2d6gk\" (UID: \"bf0cdb1f-7233-4c76-8159-b233968ce8e5\") " pod="openshift-ingress/router-default-7c7545584f-2d6gk" Jul 29 08:22:11.579847 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:22:11.579815 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/ed443996-4f98-44b5-9476-37a0c3c23099-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-548b489cb8-ch694\" (UID: \"ed443996-4f98-44b5-9476-37a0c3c23099\") " pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-ch694" Jul 29 08:22:11.579900 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:22:11.579859 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/bf0cdb1f-7233-4c76-8159-b233968ce8e5-metrics-certs\") pod \"router-default-7c7545584f-2d6gk\" (UID: \"bf0cdb1f-7233-4c76-8159-b233968ce8e5\") " pod="openshift-ingress/router-default-7c7545584f-2d6gk" Jul 29 08:22:11.579948 ip-10-0-131-138 kubenswrapper[2563]: E0729 08:22:11.579924 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/bf0cdb1f-7233-4c76-8159-b233968ce8e5-service-ca-bundle podName:bf0cdb1f-7233-4c76-8159-b233968ce8e5 nodeName:}" failed. No retries permitted until 2026-07-29 08:22:15.579899781 +0000 UTC m=+48.161188545 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/bf0cdb1f-7233-4c76-8159-b233968ce8e5-service-ca-bundle") pod "router-default-7c7545584f-2d6gk" (UID: "bf0cdb1f-7233-4c76-8159-b233968ce8e5") : configmap references non-existent config key: service-ca.crt Jul 29 08:22:11.580001 ip-10-0-131-138 kubenswrapper[2563]: E0729 08:22:11.579952 2563 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Jul 29 08:22:11.580001 ip-10-0-131-138 kubenswrapper[2563]: E0729 08:22:11.579959 2563 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Jul 29 08:22:11.580001 ip-10-0-131-138 kubenswrapper[2563]: E0729 08:22:11.579998 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/bf0cdb1f-7233-4c76-8159-b233968ce8e5-metrics-certs podName:bf0cdb1f-7233-4c76-8159-b233968ce8e5 nodeName:}" failed. No retries permitted until 2026-07-29 08:22:15.579983171 +0000 UTC m=+48.161271929 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/bf0cdb1f-7233-4c76-8159-b233968ce8e5-metrics-certs") pod "router-default-7c7545584f-2d6gk" (UID: "bf0cdb1f-7233-4c76-8159-b233968ce8e5") : secret "router-metrics-certs-default" not found Jul 29 08:22:11.580141 ip-10-0-131-138 kubenswrapper[2563]: E0729 08:22:11.580012 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ed443996-4f98-44b5-9476-37a0c3c23099-cluster-monitoring-operator-tls podName:ed443996-4f98-44b5-9476-37a0c3c23099 nodeName:}" failed. No retries permitted until 2026-07-29 08:22:15.580005954 +0000 UTC m=+48.161294710 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/ed443996-4f98-44b5-9476-37a0c3c23099-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-548b489cb8-ch694" (UID: "ed443996-4f98-44b5-9476-37a0c3c23099") : secret "cluster-monitoring-operator-tls" not found Jul 29 08:22:11.588735 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:22:11.588709 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zdpg9\" (UniqueName: \"kubernetes.io/projected/3eef9ce5-17eb-4fbe-82d3-dd9b1cb17e93-kube-api-access-zdpg9\") pod \"network-check-source-7d88cb55d8-dr9xk\" (UID: \"3eef9ce5-17eb-4fbe-82d3-dd9b1cb17e93\") " pod="openshift-network-diagnostics/network-check-source-7d88cb55d8-dr9xk" Jul 29 08:22:11.660400 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:22:11.660376 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-source-7d88cb55d8-dr9xk" Jul 29 08:22:11.680883 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:22:11.680855 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/d2793bc6-e6c2-4799-845e-ee32f0a85693-samples-operator-tls\") pod \"cluster-samples-operator-6748489656-q2lqd\" (UID: \"d2793bc6-e6c2-4799-845e-ee32f0a85693\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6748489656-q2lqd" Jul 29 08:22:11.680999 ip-10-0-131-138 kubenswrapper[2563]: E0729 08:22:11.680980 2563 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Jul 29 08:22:11.681049 ip-10-0-131-138 kubenswrapper[2563]: E0729 08:22:11.681037 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d2793bc6-e6c2-4799-845e-ee32f0a85693-samples-operator-tls podName:d2793bc6-e6c2-4799-845e-ee32f0a85693 nodeName:}" failed. No retries permitted until 2026-07-29 08:22:15.68102054 +0000 UTC m=+48.262309296 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/d2793bc6-e6c2-4799-845e-ee32f0a85693-samples-operator-tls") pod "cluster-samples-operator-6748489656-q2lqd" (UID: "d2793bc6-e6c2-4799-845e-ee32f0a85693") : secret "samples-operator-tls" not found Jul 29 08:22:11.805315 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:22:11.805285 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-source-7d88cb55d8-dr9xk"] Jul 29 08:22:11.808606 ip-10-0-131-138 kubenswrapper[2563]: W0729 08:22:11.808568 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3eef9ce5_17eb_4fbe_82d3_dd9b1cb17e93.slice/crio-c8dfc0454290ba024656c3a0452aea94a453274490744698e79777f6ce050048 WatchSource:0}: Error finding container c8dfc0454290ba024656c3a0452aea94a453274490744698e79777f6ce050048: Status 404 returned error can't find the container with id c8dfc0454290ba024656c3a0452aea94a453274490744698e79777f6ce050048 Jul 29 08:22:11.998146 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:22:11.998076 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8rp9g_cfd1d8df-e6b7-4a67-81f2-5d83da541e20/konnectivity-agent/0.log" limit=1024 Jul 29 08:22:12.099557 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:22:12.099528 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8rp9g_cfd1d8df-e6b7-4a67-81f2-5d83da541e20/konnectivity-agent/0.log" limit=1024 Jul 29 08:22:12.199749 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:22:12.199720 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8rp9g_cfd1d8df-e6b7-4a67-81f2-5d83da541e20/konnectivity-agent/0.log" limit=1024 Jul 29 08:22:12.244883 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:22:12.244838 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-5885fdb759-84zrp" event={"ID":"791f1046-8a1f-41c4-b7a6-70862705aad6","Type":"ContainerStarted","Data":"b67755562dc6fc88e5252d84491e15c8d5e3a7db12930c51046b3316f8c81c71"} Jul 29 08:22:12.246124 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:22:12.246084 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-source-7d88cb55d8-dr9xk" event={"ID":"3eef9ce5-17eb-4fbe-82d3-dd9b1cb17e93","Type":"ContainerStarted","Data":"c8dfc0454290ba024656c3a0452aea94a453274490744698e79777f6ce050048"} Jul 29 08:22:12.301772 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:22:12.301606 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8rp9g_cfd1d8df-e6b7-4a67-81f2-5d83da541e20/konnectivity-agent/0.log" limit=1024 Jul 29 08:22:12.364422 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:22:12.364384 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8rp9g_cfd1d8df-e6b7-4a67-81f2-5d83da541e20/konnectivity-agent/0.log" limit=1024 Jul 29 08:22:12.531957 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:22:12.531916 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8rp9g_cfd1d8df-e6b7-4a67-81f2-5d83da541e20/konnectivity-agent/0.log" limit=1024 Jul 29 08:22:12.597508 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:22:12.597428 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8rp9g_cfd1d8df-e6b7-4a67-81f2-5d83da541e20/konnectivity-agent/0.log" limit=1024 Jul 29 08:22:12.735117 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:22:12.735077 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8rp9g_cfd1d8df-e6b7-4a67-81f2-5d83da541e20/konnectivity-agent/0.log" limit=1024 Jul 29 08:22:12.819998 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:22:12.819936 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8rp9g_cfd1d8df-e6b7-4a67-81f2-5d83da541e20/konnectivity-agent/0.log" limit=1024 Jul 29 08:22:12.997506 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:22:12.997449 2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-operator-5885fdb759-84zrp" podStartSLOduration=2.927247754 podStartE2EDuration="5.997425225s" totalImagesPullingTime="3.070177471s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 08:22:07 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 08:22:12.289696854 +0000 UTC m=+44.870985634" watchObservedRunningTime="2026-07-29 08:22:12.997425225 +0000 UTC m=+45.578714011" Jul 29 08:22:12.998151 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:22:12.998123 2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-29q8s"] Jul 29 08:22:13.020713 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:22:13.020683 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-29q8s"] Jul 29 08:22:13.020869 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:22:13.020801 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-29q8s" Jul 29 08:22:13.023715 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:22:13.023693 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\"" Jul 29 08:22:13.023873 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:22:13.023853 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-x95nv\"" Jul 29 08:22:13.023971 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:22:13.023957 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\"" Jul 29 08:22:13.043705 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:22:13.043661 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8rp9g_cfd1d8df-e6b7-4a67-81f2-5d83da541e20/konnectivity-agent/0.log" limit=1024 Jul 29 08:22:13.106707 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:22:13.106675 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8rp9g_cfd1d8df-e6b7-4a67-81f2-5d83da541e20/konnectivity-agent/0.log" limit=1024 Jul 29 08:22:13.197372 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:22:13.197327 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/bb16ee9e-483f-465c-8deb-4a36b0db9743-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-29q8s\" (UID: \"bb16ee9e-483f-465c-8deb-4a36b0db9743\") " pod="openshift-insights/insights-runtime-extractor-29q8s" Jul 29 08:22:13.197372 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:22:13.197371 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/bb16ee9e-483f-465c-8deb-4a36b0db9743-crio-socket\") pod \"insights-runtime-extractor-29q8s\" (UID: \"bb16ee9e-483f-465c-8deb-4a36b0db9743\") " pod="openshift-insights/insights-runtime-extractor-29q8s" Jul 29 08:22:13.197614 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:22:13.197496 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/bb16ee9e-483f-465c-8deb-4a36b0db9743-data-volume\") pod \"insights-runtime-extractor-29q8s\" (UID: \"bb16ee9e-483f-465c-8deb-4a36b0db9743\") " pod="openshift-insights/insights-runtime-extractor-29q8s" Jul 29 08:22:13.197688 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:22:13.197665 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/bb16ee9e-483f-465c-8deb-4a36b0db9743-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-29q8s\" (UID: \"bb16ee9e-483f-465c-8deb-4a36b0db9743\") " pod="openshift-insights/insights-runtime-extractor-29q8s" Jul 29 08:22:13.197739 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:22:13.197704 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-k7q7p\" (UniqueName: \"kubernetes.io/projected/bb16ee9e-483f-465c-8deb-4a36b0db9743-kube-api-access-k7q7p\") pod \"insights-runtime-extractor-29q8s\" (UID: \"bb16ee9e-483f-465c-8deb-4a36b0db9743\") " pod="openshift-insights/insights-runtime-extractor-29q8s" Jul 29 08:22:13.298807 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:22:13.298716 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/bb16ee9e-483f-465c-8deb-4a36b0db9743-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-29q8s\" (UID: \"bb16ee9e-483f-465c-8deb-4a36b0db9743\") " pod="openshift-insights/insights-runtime-extractor-29q8s" Jul 29 08:22:13.298807 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:22:13.298766 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/bb16ee9e-483f-465c-8deb-4a36b0db9743-crio-socket\") pod \"insights-runtime-extractor-29q8s\" (UID: \"bb16ee9e-483f-465c-8deb-4a36b0db9743\") " pod="openshift-insights/insights-runtime-extractor-29q8s" Jul 29 08:22:13.299362 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:22:13.298825 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/bb16ee9e-483f-465c-8deb-4a36b0db9743-data-volume\") pod \"insights-runtime-extractor-29q8s\" (UID: \"bb16ee9e-483f-465c-8deb-4a36b0db9743\") " pod="openshift-insights/insights-runtime-extractor-29q8s" Jul 29 08:22:13.299362 ip-10-0-131-138 kubenswrapper[2563]: E0729 08:22:13.298852 2563 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 29 08:22:13.299362 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:22:13.298903 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/bb16ee9e-483f-465c-8deb-4a36b0db9743-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-29q8s\" (UID: \"bb16ee9e-483f-465c-8deb-4a36b0db9743\") " pod="openshift-insights/insights-runtime-extractor-29q8s" Jul 29 08:22:13.299362 ip-10-0-131-138 kubenswrapper[2563]: E0729 08:22:13.298927 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/bb16ee9e-483f-465c-8deb-4a36b0db9743-kube-rbac-proxy-cm podName:bb16ee9e-483f-465c-8deb-4a36b0db9743 nodeName:}" failed. No retries permitted until 2026-07-29 08:22:13.798903374 +0000 UTC m=+46.380192137 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/bb16ee9e-483f-465c-8deb-4a36b0db9743-kube-rbac-proxy-cm") pod "insights-runtime-extractor-29q8s" (UID: "bb16ee9e-483f-465c-8deb-4a36b0db9743") : configmap "kube-rbac-proxy" not found Jul 29 08:22:13.299362 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:22:13.298980 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-k7q7p\" (UniqueName: \"kubernetes.io/projected/bb16ee9e-483f-465c-8deb-4a36b0db9743-kube-api-access-k7q7p\") pod \"insights-runtime-extractor-29q8s\" (UID: \"bb16ee9e-483f-465c-8deb-4a36b0db9743\") " pod="openshift-insights/insights-runtime-extractor-29q8s" Jul 29 08:22:13.299362 ip-10-0-131-138 kubenswrapper[2563]: E0729 08:22:13.299003 2563 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 29 08:22:13.299362 ip-10-0-131-138 kubenswrapper[2563]: E0729 08:22:13.299055 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/bb16ee9e-483f-465c-8deb-4a36b0db9743-insights-runtime-extractor-tls podName:bb16ee9e-483f-465c-8deb-4a36b0db9743 nodeName:}" failed. No retries permitted until 2026-07-29 08:22:13.799039763 +0000 UTC m=+46.380328521 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/bb16ee9e-483f-465c-8deb-4a36b0db9743-insights-runtime-extractor-tls") pod "insights-runtime-extractor-29q8s" (UID: "bb16ee9e-483f-465c-8deb-4a36b0db9743") : secret "insights-runtime-extractor-tls" not found Jul 29 08:22:13.299362 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:22:13.299232 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/bb16ee9e-483f-465c-8deb-4a36b0db9743-crio-socket\") pod \"insights-runtime-extractor-29q8s\" (UID: \"bb16ee9e-483f-465c-8deb-4a36b0db9743\") " pod="openshift-insights/insights-runtime-extractor-29q8s" Jul 29 08:22:13.299760 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:22:13.299580 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/bb16ee9e-483f-465c-8deb-4a36b0db9743-data-volume\") pod \"insights-runtime-extractor-29q8s\" (UID: \"bb16ee9e-483f-465c-8deb-4a36b0db9743\") " pod="openshift-insights/insights-runtime-extractor-29q8s" Jul 29 08:22:13.309441 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:22:13.309314 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-k7q7p\" (UniqueName: \"kubernetes.io/projected/bb16ee9e-483f-465c-8deb-4a36b0db9743-kube-api-access-k7q7p\") pod \"insights-runtime-extractor-29q8s\" (UID: \"bb16ee9e-483f-465c-8deb-4a36b0db9743\") " pod="openshift-insights/insights-runtime-extractor-29q8s" Jul 29 08:22:13.804137 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:22:13.804099 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/bb16ee9e-483f-465c-8deb-4a36b0db9743-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-29q8s\" (UID: \"bb16ee9e-483f-465c-8deb-4a36b0db9743\") " pod="openshift-insights/insights-runtime-extractor-29q8s" Jul 29 08:22:13.804355 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:22:13.804212 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/bb16ee9e-483f-465c-8deb-4a36b0db9743-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-29q8s\" (UID: \"bb16ee9e-483f-465c-8deb-4a36b0db9743\") " pod="openshift-insights/insights-runtime-extractor-29q8s" Jul 29 08:22:13.804355 ip-10-0-131-138 kubenswrapper[2563]: E0729 08:22:13.804225 2563 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 29 08:22:13.804355 ip-10-0-131-138 kubenswrapper[2563]: E0729 08:22:13.804307 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/bb16ee9e-483f-465c-8deb-4a36b0db9743-kube-rbac-proxy-cm podName:bb16ee9e-483f-465c-8deb-4a36b0db9743 nodeName:}" failed. No retries permitted until 2026-07-29 08:22:14.80428757 +0000 UTC m=+47.385576328 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/bb16ee9e-483f-465c-8deb-4a36b0db9743-kube-rbac-proxy-cm") pod "insights-runtime-extractor-29q8s" (UID: "bb16ee9e-483f-465c-8deb-4a36b0db9743") : configmap "kube-rbac-proxy" not found Jul 29 08:22:13.804537 ip-10-0-131-138 kubenswrapper[2563]: E0729 08:22:13.804372 2563 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 29 08:22:13.804537 ip-10-0-131-138 kubenswrapper[2563]: E0729 08:22:13.804430 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/bb16ee9e-483f-465c-8deb-4a36b0db9743-insights-runtime-extractor-tls podName:bb16ee9e-483f-465c-8deb-4a36b0db9743 nodeName:}" failed. No retries permitted until 2026-07-29 08:22:14.804414936 +0000 UTC m=+47.385703696 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/bb16ee9e-483f-465c-8deb-4a36b0db9743-insights-runtime-extractor-tls") pod "insights-runtime-extractor-29q8s" (UID: "bb16ee9e-483f-465c-8deb-4a36b0db9743") : secret "insights-runtime-extractor-tls" not found Jul 29 08:22:14.382529 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:22:14.382496 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8rp9g_cfd1d8df-e6b7-4a67-81f2-5d83da541e20/konnectivity-agent/0.log" limit=1024 Jul 29 08:22:14.450692 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:22:14.450653 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8rp9g_cfd1d8df-e6b7-4a67-81f2-5d83da541e20/konnectivity-agent/0.log" limit=1024 Jul 29 08:22:14.812169 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:22:14.812142 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/bb16ee9e-483f-465c-8deb-4a36b0db9743-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-29q8s\" (UID: \"bb16ee9e-483f-465c-8deb-4a36b0db9743\") " pod="openshift-insights/insights-runtime-extractor-29q8s" Jul 29 08:22:14.812350 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:22:14.812205 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/bb16ee9e-483f-465c-8deb-4a36b0db9743-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-29q8s\" (UID: \"bb16ee9e-483f-465c-8deb-4a36b0db9743\") " pod="openshift-insights/insights-runtime-extractor-29q8s" Jul 29 08:22:14.812350 ip-10-0-131-138 kubenswrapper[2563]: E0729 08:22:14.812303 2563 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 29 08:22:14.812350 ip-10-0-131-138 kubenswrapper[2563]: E0729 08:22:14.812318 2563 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 29 08:22:14.812452 ip-10-0-131-138 kubenswrapper[2563]: E0729 08:22:14.812355 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/bb16ee9e-483f-465c-8deb-4a36b0db9743-kube-rbac-proxy-cm podName:bb16ee9e-483f-465c-8deb-4a36b0db9743 nodeName:}" failed. No retries permitted until 2026-07-29 08:22:16.812341347 +0000 UTC m=+49.393630102 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/bb16ee9e-483f-465c-8deb-4a36b0db9743-kube-rbac-proxy-cm") pod "insights-runtime-extractor-29q8s" (UID: "bb16ee9e-483f-465c-8deb-4a36b0db9743") : configmap "kube-rbac-proxy" not found Jul 29 08:22:14.812452 ip-10-0-131-138 kubenswrapper[2563]: E0729 08:22:14.812369 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/bb16ee9e-483f-465c-8deb-4a36b0db9743-insights-runtime-extractor-tls podName:bb16ee9e-483f-465c-8deb-4a36b0db9743 nodeName:}" failed. No retries permitted until 2026-07-29 08:22:16.812362667 +0000 UTC m=+49.393651422 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/bb16ee9e-483f-465c-8deb-4a36b0db9743-insights-runtime-extractor-tls") pod "insights-runtime-extractor-29q8s" (UID: "bb16ee9e-483f-465c-8deb-4a36b0db9743") : secret "insights-runtime-extractor-tls" not found Jul 29 08:22:14.821603 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:22:14.821574 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8rp9g_cfd1d8df-e6b7-4a67-81f2-5d83da541e20/konnectivity-agent/0.log" limit=1024 Jul 29 08:22:14.923836 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:22:14.923797 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8rp9g_cfd1d8df-e6b7-4a67-81f2-5d83da541e20/konnectivity-agent/0.log" limit=1024 Jul 29 08:22:15.255325 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:22:15.255291 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-source-7d88cb55d8-dr9xk" event={"ID":"3eef9ce5-17eb-4fbe-82d3-dd9b1cb17e93","Type":"ContainerStarted","Data":"63255f287b3b5c51fad1b5e8b45fb6870390f57ef00205c8b7740ea6ab31f3c3"} Jul 29 08:22:15.358385 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:22:15.358224 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8rp9g_cfd1d8df-e6b7-4a67-81f2-5d83da541e20/konnectivity-agent/0.log" limit=1024 Jul 29 08:22:15.591220 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:22:15.591193 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-bgb7d_bc118958-c5f8-44b8-80a5-bfc2d80fad1d/dns-node-resolver/0.log" Jul 29 08:22:15.619703 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:22:15.619631 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/bf0cdb1f-7233-4c76-8159-b233968ce8e5-service-ca-bundle\") pod \"router-default-7c7545584f-2d6gk\" (UID: \"bf0cdb1f-7233-4c76-8159-b233968ce8e5\") " pod="openshift-ingress/router-default-7c7545584f-2d6gk" Jul 29 08:22:15.619846 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:22:15.619744 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/ed443996-4f98-44b5-9476-37a0c3c23099-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-548b489cb8-ch694\" (UID: \"ed443996-4f98-44b5-9476-37a0c3c23099\") " pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-ch694" Jul 29 08:22:15.619846 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:22:15.619766 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/bf0cdb1f-7233-4c76-8159-b233968ce8e5-metrics-certs\") pod \"router-default-7c7545584f-2d6gk\" (UID: \"bf0cdb1f-7233-4c76-8159-b233968ce8e5\") " pod="openshift-ingress/router-default-7c7545584f-2d6gk" Jul 29 08:22:15.619846 ip-10-0-131-138 kubenswrapper[2563]: E0729 08:22:15.619800 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/bf0cdb1f-7233-4c76-8159-b233968ce8e5-service-ca-bundle podName:bf0cdb1f-7233-4c76-8159-b233968ce8e5 nodeName:}" failed. No retries permitted until 2026-07-29 08:22:23.619782685 +0000 UTC m=+56.201071440 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/bf0cdb1f-7233-4c76-8159-b233968ce8e5-service-ca-bundle") pod "router-default-7c7545584f-2d6gk" (UID: "bf0cdb1f-7233-4c76-8159-b233968ce8e5") : configmap references non-existent config key: service-ca.crt Jul 29 08:22:15.620015 ip-10-0-131-138 kubenswrapper[2563]: E0729 08:22:15.619851 2563 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Jul 29 08:22:15.620015 ip-10-0-131-138 kubenswrapper[2563]: E0729 08:22:15.619906 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/bf0cdb1f-7233-4c76-8159-b233968ce8e5-metrics-certs podName:bf0cdb1f-7233-4c76-8159-b233968ce8e5 nodeName:}" failed. No retries permitted until 2026-07-29 08:22:23.619890443 +0000 UTC m=+56.201179201 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/bf0cdb1f-7233-4c76-8159-b233968ce8e5-metrics-certs") pod "router-default-7c7545584f-2d6gk" (UID: "bf0cdb1f-7233-4c76-8159-b233968ce8e5") : secret "router-metrics-certs-default" not found Jul 29 08:22:15.620015 ip-10-0-131-138 kubenswrapper[2563]: E0729 08:22:15.619913 2563 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Jul 29 08:22:15.620015 ip-10-0-131-138 kubenswrapper[2563]: E0729 08:22:15.619974 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ed443996-4f98-44b5-9476-37a0c3c23099-cluster-monitoring-operator-tls podName:ed443996-4f98-44b5-9476-37a0c3c23099 nodeName:}" failed. No retries permitted until 2026-07-29 08:22:23.619958409 +0000 UTC m=+56.201247178 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/ed443996-4f98-44b5-9476-37a0c3c23099-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-548b489cb8-ch694" (UID: "ed443996-4f98-44b5-9476-37a0c3c23099") : secret "cluster-monitoring-operator-tls" not found Jul 29 08:22:15.720741 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:22:15.720704 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/d2793bc6-e6c2-4799-845e-ee32f0a85693-samples-operator-tls\") pod \"cluster-samples-operator-6748489656-q2lqd\" (UID: \"d2793bc6-e6c2-4799-845e-ee32f0a85693\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6748489656-q2lqd" Jul 29 08:22:15.720878 ip-10-0-131-138 kubenswrapper[2563]: E0729 08:22:15.720852 2563 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Jul 29 08:22:15.720933 ip-10-0-131-138 kubenswrapper[2563]: E0729 08:22:15.720923 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d2793bc6-e6c2-4799-845e-ee32f0a85693-samples-operator-tls podName:d2793bc6-e6c2-4799-845e-ee32f0a85693 nodeName:}" failed. No retries permitted until 2026-07-29 08:22:23.720908064 +0000 UTC m=+56.302196819 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/d2793bc6-e6c2-4799-845e-ee32f0a85693-samples-operator-tls") pod "cluster-samples-operator-6748489656-q2lqd" (UID: "d2793bc6-e6c2-4799-845e-ee32f0a85693") : secret "samples-operator-tls" not found Jul 29 08:22:16.591359 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:22:16.591327 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-9d4f5_5bce82ca-dd94-425f-83d2-cd44232d8cdb/node-ca/0.log" Jul 29 08:22:16.831531 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:22:16.831485 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/bb16ee9e-483f-465c-8deb-4a36b0db9743-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-29q8s\" (UID: \"bb16ee9e-483f-465c-8deb-4a36b0db9743\") " pod="openshift-insights/insights-runtime-extractor-29q8s" Jul 29 08:22:16.831716 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:22:16.831584 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/bb16ee9e-483f-465c-8deb-4a36b0db9743-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-29q8s\" (UID: \"bb16ee9e-483f-465c-8deb-4a36b0db9743\") " pod="openshift-insights/insights-runtime-extractor-29q8s" Jul 29 08:22:16.831716 ip-10-0-131-138 kubenswrapper[2563]: E0729 08:22:16.831638 2563 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 29 08:22:16.831716 ip-10-0-131-138 kubenswrapper[2563]: E0729 08:22:16.831706 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/bb16ee9e-483f-465c-8deb-4a36b0db9743-insights-runtime-extractor-tls podName:bb16ee9e-483f-465c-8deb-4a36b0db9743 nodeName:}" failed. No retries permitted until 2026-07-29 08:22:20.831690137 +0000 UTC m=+53.412978893 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/bb16ee9e-483f-465c-8deb-4a36b0db9743-insights-runtime-extractor-tls") pod "insights-runtime-extractor-29q8s" (UID: "bb16ee9e-483f-465c-8deb-4a36b0db9743") : secret "insights-runtime-extractor-tls" not found Jul 29 08:22:16.831846 ip-10-0-131-138 kubenswrapper[2563]: E0729 08:22:16.831714 2563 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 29 08:22:16.831846 ip-10-0-131-138 kubenswrapper[2563]: E0729 08:22:16.831764 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/bb16ee9e-483f-465c-8deb-4a36b0db9743-kube-rbac-proxy-cm podName:bb16ee9e-483f-465c-8deb-4a36b0db9743 nodeName:}" failed. No retries permitted until 2026-07-29 08:22:20.831750593 +0000 UTC m=+53.413039349 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/bb16ee9e-483f-465c-8deb-4a36b0db9743-kube-rbac-proxy-cm") pod "insights-runtime-extractor-29q8s" (UID: "bb16ee9e-483f-465c-8deb-4a36b0db9743") : configmap "kube-rbac-proxy" not found Jul 29 08:22:17.791948 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:22:17.791910 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-pwqh7_b052d187-d65d-4894-aaa2-5dbb4b06600e/serve-healthcheck-canary/0.log" Jul 29 08:22:18.098741 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:22:18.098656 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8rp9g_cfd1d8df-e6b7-4a67-81f2-5d83da541e20/konnectivity-agent/0.log" limit=1024 Jul 29 08:22:18.140768 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:22:18.140745 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/e521e734-a8d9-4748-ba28-c5f2beb27610-metrics-tls\") pod \"dns-default-nhklh\" (UID: \"e521e734-a8d9-4748-ba28-c5f2beb27610\") " pod="openshift-dns/dns-default-nhklh" Jul 29 08:22:18.140856 ip-10-0-131-138 kubenswrapper[2563]: E0729 08:22:18.140844 2563 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 29 08:22:18.140903 ip-10-0-131-138 kubenswrapper[2563]: E0729 08:22:18.140894 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e521e734-a8d9-4748-ba28-c5f2beb27610-metrics-tls podName:e521e734-a8d9-4748-ba28-c5f2beb27610 nodeName:}" failed. No retries permitted until 2026-07-29 08:22:34.140881428 +0000 UTC m=+66.722170184 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/e521e734-a8d9-4748-ba28-c5f2beb27610-metrics-tls") pod "dns-default-nhklh" (UID: "e521e734-a8d9-4748-ba28-c5f2beb27610") : secret "dns-default-metrics-tls" not found Jul 29 08:22:18.839936 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:22:18.839902 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8rp9g_cfd1d8df-e6b7-4a67-81f2-5d83da541e20/konnectivity-agent/0.log" limit=1024 Jul 29 08:22:18.965777 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:22:18.965735 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8rp9g_cfd1d8df-e6b7-4a67-81f2-5d83da541e20/konnectivity-agent/0.log" limit=1024 Jul 29 08:22:19.078673 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:22:19.078636 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8rp9g_cfd1d8df-e6b7-4a67-81f2-5d83da541e20/konnectivity-agent/0.log" limit=1024 Jul 29 08:22:19.178888 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:22:19.178785 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8rp9g_cfd1d8df-e6b7-4a67-81f2-5d83da541e20/konnectivity-agent/0.log" limit=1024 Jul 29 08:22:19.351356 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:22:19.351323 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8rp9g_cfd1d8df-e6b7-4a67-81f2-5d83da541e20/konnectivity-agent/0.log" limit=1024 Jul 29 08:22:19.497811 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:22:19.497778 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8rp9g_cfd1d8df-e6b7-4a67-81f2-5d83da541e20/konnectivity-agent/0.log" limit=1024 Jul 29 08:22:19.595175 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:22:19.595123 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8rp9g_cfd1d8df-e6b7-4a67-81f2-5d83da541e20/konnectivity-agent/0.log" limit=1024 Jul 29 08:22:19.911648 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:22:19.911559 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8rp9g_cfd1d8df-e6b7-4a67-81f2-5d83da541e20/konnectivity-agent/0.log" limit=1024 Jul 29 08:22:20.864580 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:22:20.864535 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/bb16ee9e-483f-465c-8deb-4a36b0db9743-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-29q8s\" (UID: \"bb16ee9e-483f-465c-8deb-4a36b0db9743\") " pod="openshift-insights/insights-runtime-extractor-29q8s" Jul 29 08:22:20.864769 ip-10-0-131-138 kubenswrapper[2563]: E0729 08:22:20.864653 2563 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 29 08:22:20.864769 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:22:20.864676 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/bb16ee9e-483f-465c-8deb-4a36b0db9743-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-29q8s\" (UID: \"bb16ee9e-483f-465c-8deb-4a36b0db9743\") " pod="openshift-insights/insights-runtime-extractor-29q8s" Jul 29 08:22:20.864769 ip-10-0-131-138 kubenswrapper[2563]: E0729 08:22:20.864714 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/bb16ee9e-483f-465c-8deb-4a36b0db9743-kube-rbac-proxy-cm podName:bb16ee9e-483f-465c-8deb-4a36b0db9743 nodeName:}" failed. No retries permitted until 2026-07-29 08:22:28.864698889 +0000 UTC m=+61.445987646 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/bb16ee9e-483f-465c-8deb-4a36b0db9743-kube-rbac-proxy-cm") pod "insights-runtime-extractor-29q8s" (UID: "bb16ee9e-483f-465c-8deb-4a36b0db9743") : configmap "kube-rbac-proxy" not found Jul 29 08:22:20.864925 ip-10-0-131-138 kubenswrapper[2563]: E0729 08:22:20.864780 2563 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 29 08:22:20.864925 ip-10-0-131-138 kubenswrapper[2563]: E0729 08:22:20.864838 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/bb16ee9e-483f-465c-8deb-4a36b0db9743-insights-runtime-extractor-tls podName:bb16ee9e-483f-465c-8deb-4a36b0db9743 nodeName:}" failed. No retries permitted until 2026-07-29 08:22:28.864821883 +0000 UTC m=+61.446110658 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/bb16ee9e-483f-465c-8deb-4a36b0db9743-insights-runtime-extractor-tls") pod "insights-runtime-extractor-29q8s" (UID: "bb16ee9e-483f-465c-8deb-4a36b0db9743") : secret "insights-runtime-extractor-tls" not found Jul 29 08:22:20.905804 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:22:20.905741 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8rp9g_cfd1d8df-e6b7-4a67-81f2-5d83da541e20/konnectivity-agent/0.log" limit=1024 Jul 29 08:22:21.332744 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:22:21.332707 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8rp9g_cfd1d8df-e6b7-4a67-81f2-5d83da541e20/konnectivity-agent/0.log" limit=1024 Jul 29 08:22:21.403832 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:22:21.403782 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8rp9g_cfd1d8df-e6b7-4a67-81f2-5d83da541e20/konnectivity-agent/0.log" limit=1024 Jul 29 08:22:21.508739 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:22:21.508705 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8rp9g_cfd1d8df-e6b7-4a67-81f2-5d83da541e20/konnectivity-agent/0.log" limit=1024 Jul 29 08:22:21.713144 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:22:21.713112 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8rp9g_cfd1d8df-e6b7-4a67-81f2-5d83da541e20/konnectivity-agent/0.log" limit=1024 Jul 29 08:22:22.105528 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:22:22.105447 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8rp9g_cfd1d8df-e6b7-4a67-81f2-5d83da541e20/konnectivity-agent/0.log" limit=1024 Jul 29 08:22:22.441566 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:22:22.441487 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8rp9g_cfd1d8df-e6b7-4a67-81f2-5d83da541e20/konnectivity-agent/0.log" limit=1024 Jul 29 08:22:22.542336 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:22:22.542294 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8rp9g_cfd1d8df-e6b7-4a67-81f2-5d83da541e20/konnectivity-agent/0.log" limit=1024 Jul 29 08:22:22.711429 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:22:22.711403 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8rp9g_cfd1d8df-e6b7-4a67-81f2-5d83da541e20/konnectivity-agent/0.log" limit=1024 Jul 29 08:22:23.213600 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:22:23.213573 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8rp9g_cfd1d8df-e6b7-4a67-81f2-5d83da541e20/konnectivity-agent/0.log" limit=1024 Jul 29 08:22:23.273887 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:22:23.273857 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8rp9g_cfd1d8df-e6b7-4a67-81f2-5d83da541e20/konnectivity-agent/0.log" limit=1024 Jul 29 08:22:23.506914 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:22:23.506825 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8rp9g_cfd1d8df-e6b7-4a67-81f2-5d83da541e20/konnectivity-agent/0.log" limit=1024 Jul 29 08:22:23.688048 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:22:23.688014 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/bf0cdb1f-7233-4c76-8159-b233968ce8e5-service-ca-bundle\") pod \"router-default-7c7545584f-2d6gk\" (UID: \"bf0cdb1f-7233-4c76-8159-b233968ce8e5\") " pod="openshift-ingress/router-default-7c7545584f-2d6gk" Jul 29 08:22:23.688194 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:22:23.688161 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/ed443996-4f98-44b5-9476-37a0c3c23099-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-548b489cb8-ch694\" (UID: \"ed443996-4f98-44b5-9476-37a0c3c23099\") " pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-ch694" Jul 29 08:22:23.688194 ip-10-0-131-138 kubenswrapper[2563]: E0729 08:22:23.688185 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/bf0cdb1f-7233-4c76-8159-b233968ce8e5-service-ca-bundle podName:bf0cdb1f-7233-4c76-8159-b233968ce8e5 nodeName:}" failed. No retries permitted until 2026-07-29 08:22:39.688164528 +0000 UTC m=+72.269453302 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/bf0cdb1f-7233-4c76-8159-b233968ce8e5-service-ca-bundle") pod "router-default-7c7545584f-2d6gk" (UID: "bf0cdb1f-7233-4c76-8159-b233968ce8e5") : configmap references non-existent config key: service-ca.crt Jul 29 08:22:23.688311 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:22:23.688221 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/bf0cdb1f-7233-4c76-8159-b233968ce8e5-metrics-certs\") pod \"router-default-7c7545584f-2d6gk\" (UID: \"bf0cdb1f-7233-4c76-8159-b233968ce8e5\") " pod="openshift-ingress/router-default-7c7545584f-2d6gk" Jul 29 08:22:23.688311 ip-10-0-131-138 kubenswrapper[2563]: E0729 08:22:23.688245 2563 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Jul 29 08:22:23.688377 ip-10-0-131-138 kubenswrapper[2563]: E0729 08:22:23.688337 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ed443996-4f98-44b5-9476-37a0c3c23099-cluster-monitoring-operator-tls podName:ed443996-4f98-44b5-9476-37a0c3c23099 nodeName:}" failed. No retries permitted until 2026-07-29 08:22:39.688322546 +0000 UTC m=+72.269611303 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/ed443996-4f98-44b5-9476-37a0c3c23099-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-548b489cb8-ch694" (UID: "ed443996-4f98-44b5-9476-37a0c3c23099") : secret "cluster-monitoring-operator-tls" not found Jul 29 08:22:23.690753 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:22:23.690717 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/bf0cdb1f-7233-4c76-8159-b233968ce8e5-metrics-certs\") pod \"router-default-7c7545584f-2d6gk\" (UID: \"bf0cdb1f-7233-4c76-8159-b233968ce8e5\") " pod="openshift-ingress/router-default-7c7545584f-2d6gk" Jul 29 08:22:23.707103 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:22:23.707078 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8rp9g_cfd1d8df-e6b7-4a67-81f2-5d83da541e20/konnectivity-agent/0.log" limit=1024 Jul 29 08:22:23.788785 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:22:23.788705 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/d2793bc6-e6c2-4799-845e-ee32f0a85693-samples-operator-tls\") pod \"cluster-samples-operator-6748489656-q2lqd\" (UID: \"d2793bc6-e6c2-4799-845e-ee32f0a85693\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6748489656-q2lqd" Jul 29 08:22:23.791142 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:22:23.791122 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/d2793bc6-e6c2-4799-845e-ee32f0a85693-samples-operator-tls\") pod \"cluster-samples-operator-6748489656-q2lqd\" (UID: \"d2793bc6-e6c2-4799-845e-ee32f0a85693\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6748489656-q2lqd" Jul 29 08:22:23.862376 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:22:23.862326 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-samples-operator/cluster-samples-operator-6748489656-q2lqd" Jul 29 08:22:24.007193 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:22:24.007142 2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-source-7d88cb55d8-dr9xk" podStartSLOduration=9.780838861 podStartE2EDuration="13.007122965s" totalImagesPullingTime="3.226284104s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 08:22:11 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 08:22:15.290150147 +0000 UTC m=+47.871438924" watchObservedRunningTime="2026-07-29 08:22:24.007122965 +0000 UTC m=+56.588411744" Jul 29 08:22:24.007552 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:22:24.007532 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-6748489656-q2lqd"] Jul 29 08:22:24.161013 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:22:24.160977 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8rp9g_cfd1d8df-e6b7-4a67-81f2-5d83da541e20/konnectivity-agent/0.log" limit=1024 Jul 29 08:22:24.285465 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:22:24.285431 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-6748489656-q2lqd" event={"ID":"d2793bc6-e6c2-4799-845e-ee32f0a85693","Type":"ContainerStarted","Data":"54e9d3564acfcdf603df306954d4babceab64efbdf33c40ee75b89d8f61af2e3"} Jul 29 08:22:24.712555 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:22:24.712513 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8rp9g_cfd1d8df-e6b7-4a67-81f2-5d83da541e20/konnectivity-agent/0.log" limit=1024 Jul 29 08:22:24.908033 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:22:24.907999 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8rp9g_cfd1d8df-e6b7-4a67-81f2-5d83da541e20/konnectivity-agent/0.log" limit=1024 Jul 29 08:22:25.106379 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:22:25.106279 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8rp9g_cfd1d8df-e6b7-4a67-81f2-5d83da541e20/konnectivity-agent/0.log" limit=1024 Jul 29 08:22:25.204763 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:22:25.204651 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8rp9g_cfd1d8df-e6b7-4a67-81f2-5d83da541e20/konnectivity-agent/0.log" limit=1024 Jul 29 08:22:25.339503 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:22:25.339448 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8rp9g_cfd1d8df-e6b7-4a67-81f2-5d83da541e20/konnectivity-agent/0.log" limit=1024 Jul 29 08:22:26.293018 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:22:26.292984 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-6748489656-q2lqd" event={"ID":"d2793bc6-e6c2-4799-845e-ee32f0a85693","Type":"ContainerStarted","Data":"c26945365bf6bafa8ae6e3e7afc88dcf2824904d3901433b821089b8e48df687"} Jul 29 08:22:26.293403 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:22:26.293065 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-6748489656-q2lqd" event={"ID":"d2793bc6-e6c2-4799-845e-ee32f0a85693","Type":"ContainerStarted","Data":"ef56e7d0340a28ae8e3c62421fba1cf4ed4f9b3b236cec0ae084463ca64d671f"} Jul 29 08:22:26.307377 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:22:26.307339 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8rp9g_cfd1d8df-e6b7-4a67-81f2-5d83da541e20/konnectivity-agent/0.log" limit=1024 Jul 29 08:22:26.310994 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:22:26.310936 2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-samples-operator/cluster-samples-operator-6748489656-q2lqd" podStartSLOduration=17.749691834 podStartE2EDuration="19.310919322s" totalImagesPullingTime="1.561227488s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 08:22:07 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 08:22:26.310176379 +0000 UTC m=+58.891465157" watchObservedRunningTime="2026-07-29 08:22:26.310919322 +0000 UTC m=+58.892208101" Jul 29 08:22:26.509006 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:22:26.508963 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8rp9g_cfd1d8df-e6b7-4a67-81f2-5d83da541e20/konnectivity-agent/0.log" limit=1024 Jul 29 08:22:26.855020 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:22:26.854979 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8rp9g_cfd1d8df-e6b7-4a67-81f2-5d83da541e20/konnectivity-agent/0.log" limit=1024 Jul 29 08:22:26.955455 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:22:26.955426 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8rp9g_cfd1d8df-e6b7-4a67-81f2-5d83da541e20/konnectivity-agent/0.log" limit=1024 Jul 29 08:22:27.110113 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:22:27.110022 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8rp9g_cfd1d8df-e6b7-4a67-81f2-5d83da541e20/konnectivity-agent/0.log" limit=1024 Jul 29 08:22:28.113760 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:22:28.113720 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8rp9g_cfd1d8df-e6b7-4a67-81f2-5d83da541e20/konnectivity-agent/0.log" limit=1024 Jul 29 08:22:28.206169 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:22:28.206144 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-vqdfh" Jul 29 08:22:28.512623 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:22:28.512585 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8rp9g_cfd1d8df-e6b7-4a67-81f2-5d83da541e20/konnectivity-agent/0.log" limit=1024 Jul 29 08:22:28.707775 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:22:28.707734 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8rp9g_cfd1d8df-e6b7-4a67-81f2-5d83da541e20/konnectivity-agent/0.log" limit=1024 Jul 29 08:22:28.913115 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:22:28.913031 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8rp9g_cfd1d8df-e6b7-4a67-81f2-5d83da541e20/konnectivity-agent/0.log" limit=1024 Jul 29 08:22:28.932044 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:22:28.932015 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/bb16ee9e-483f-465c-8deb-4a36b0db9743-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-29q8s\" (UID: \"bb16ee9e-483f-465c-8deb-4a36b0db9743\") " pod="openshift-insights/insights-runtime-extractor-29q8s" Jul 29 08:22:28.932181 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:22:28.932102 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/bb16ee9e-483f-465c-8deb-4a36b0db9743-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-29q8s\" (UID: \"bb16ee9e-483f-465c-8deb-4a36b0db9743\") " pod="openshift-insights/insights-runtime-extractor-29q8s" Jul 29 08:22:28.932181 ip-10-0-131-138 kubenswrapper[2563]: E0729 08:22:28.932162 2563 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 29 08:22:28.932288 ip-10-0-131-138 kubenswrapper[2563]: E0729 08:22:28.932238 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/bb16ee9e-483f-465c-8deb-4a36b0db9743-insights-runtime-extractor-tls podName:bb16ee9e-483f-465c-8deb-4a36b0db9743 nodeName:}" failed. No retries permitted until 2026-07-29 08:22:44.932222733 +0000 UTC m=+77.513511490 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/bb16ee9e-483f-465c-8deb-4a36b0db9743-insights-runtime-extractor-tls") pod "insights-runtime-extractor-29q8s" (UID: "bb16ee9e-483f-465c-8deb-4a36b0db9743") : secret "insights-runtime-extractor-tls" not found Jul 29 08:22:28.932623 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:22:28.932604 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/bb16ee9e-483f-465c-8deb-4a36b0db9743-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-29q8s\" (UID: \"bb16ee9e-483f-465c-8deb-4a36b0db9743\") " pod="openshift-insights/insights-runtime-extractor-29q8s" Jul 29 08:22:29.115521 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:22:29.115480 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8rp9g_cfd1d8df-e6b7-4a67-81f2-5d83da541e20/konnectivity-agent/0.log" limit=1024 Jul 29 08:22:29.509879 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:22:29.509844 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8rp9g_cfd1d8df-e6b7-4a67-81f2-5d83da541e20/konnectivity-agent/0.log" limit=1024 Jul 29 08:22:33.669872 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:22:33.669825 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/35e49848-94f6-43d9-b7c4-6c817dd5dde7-metrics-certs\") pod \"network-metrics-daemon-qk9kb\" (UID: \"35e49848-94f6-43d9-b7c4-6c817dd5dde7\") " pod="openshift-multus/network-metrics-daemon-qk9kb" Jul 29 08:22:33.672732 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:22:33.672713 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Jul 29 08:22:33.683301 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:22:33.683250 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/35e49848-94f6-43d9-b7c4-6c817dd5dde7-metrics-certs\") pod \"network-metrics-daemon-qk9kb\" (UID: \"35e49848-94f6-43d9-b7c4-6c817dd5dde7\") " pod="openshift-multus/network-metrics-daemon-qk9kb" Jul 29 08:22:33.771367 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:22:33.771319 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nxn6h\" (UniqueName: \"kubernetes.io/projected/ce06e4b8-c4ec-4edf-a9c4-d7d86a03d844-kube-api-access-nxn6h\") pod \"network-check-target-rdclp\" (UID: \"ce06e4b8-c4ec-4edf-a9c4-d7d86a03d844\") " pod="openshift-network-diagnostics/network-check-target-rdclp" Jul 29 08:22:33.771367 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:22:33.771362 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/b14e5aed-a2b3-41bf-9389-701d69342953-original-pull-secret\") pod \"global-pull-secret-syncer-rn9kc\" (UID: \"b14e5aed-a2b3-41bf-9389-701d69342953\") " pod="kube-system/global-pull-secret-syncer-rn9kc" Jul 29 08:22:33.772229 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:22:33.772203 2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-74f6c649f4-m8w74"] Jul 29 08:22:33.774068 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:22:33.774035 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nxn6h\" (UniqueName: \"kubernetes.io/projected/ce06e4b8-c4ec-4edf-a9c4-d7d86a03d844-kube-api-access-nxn6h\") pod \"network-check-target-rdclp\" (UID: \"ce06e4b8-c4ec-4edf-a9c4-d7d86a03d844\") " pod="openshift-network-diagnostics/network-check-target-rdclp" Jul 29 08:22:33.774205 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:22:33.774189 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Jul 29 08:22:33.775133 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:22:33.775118 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-74f6c649f4-m8w74" Jul 29 08:22:33.777706 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:22:33.777687 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"open-cluster-management-image-pull-credentials\"" Jul 29 08:22:33.778829 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:22:33.778795 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"managed-serviceaccount-hub-kubeconfig\"" Jul 29 08:22:33.778946 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:22:33.778845 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"kube-root-ca.crt\"" Jul 29 08:22:33.778946 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:22:33.778865 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"openshift-service-ca.crt\"" Jul 29 08:22:33.778946 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:22:33.778845 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"managed-serviceaccount-dockercfg-gnkjj\"" Jul 29 08:22:33.780507 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:22:33.780488 2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-56cbc87d66-2t2lj"] Jul 29 08:22:33.782472 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:22:33.782454 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-56cbc87d66-2t2lj" Jul 29 08:22:33.783978 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:22:33.783958 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/b14e5aed-a2b3-41bf-9389-701d69342953-original-pull-secret\") pod \"global-pull-secret-syncer-rn9kc\" (UID: \"b14e5aed-a2b3-41bf-9389-701d69342953\") " pod="kube-system/global-pull-secret-syncer-rn9kc" Jul 29 08:22:33.785055 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:22:33.785034 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"work-manager-hub-kubeconfig\"" Jul 29 08:22:33.793088 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:22:33.793067 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-74f6c649f4-m8w74"] Jul 29 08:22:33.811720 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:22:33.811696 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-56cbc87d66-2t2lj"] Jul 29 08:22:33.829905 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:22:33.829878 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8rp9g_cfd1d8df-e6b7-4a67-81f2-5d83da541e20/konnectivity-agent/0.log" limit=1024 Jul 29 08:22:33.871658 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:22:33.871623 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-rn9kc" Jul 29 08:22:33.871862 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:22:33.871833 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/160e73ad-6c93-4ba9-b01a-7aea4e3fddfa-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-74f6c649f4-m8w74\" (UID: \"160e73ad-6c93-4ba9-b01a-7aea4e3fddfa\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-74f6c649f4-m8w74" Jul 29 08:22:33.871935 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:22:33.871862 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/3c999580-b677-4bbf-b2e0-f425901a074e-tmp\") pod \"klusterlet-addon-workmgr-56cbc87d66-2t2lj\" (UID: \"3c999580-b677-4bbf-b2e0-f425901a074e\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-56cbc87d66-2t2lj" Jul 29 08:22:33.871935 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:22:33.871904 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ghz4z\" (UniqueName: \"kubernetes.io/projected/160e73ad-6c93-4ba9-b01a-7aea4e3fddfa-kube-api-access-ghz4z\") pod \"managed-serviceaccount-addon-agent-74f6c649f4-m8w74\" (UID: \"160e73ad-6c93-4ba9-b01a-7aea4e3fddfa\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-74f6c649f4-m8w74" Jul 29 08:22:33.872018 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:22:33.871956 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/3c999580-b677-4bbf-b2e0-f425901a074e-klusterlet-config\") pod \"klusterlet-addon-workmgr-56cbc87d66-2t2lj\" (UID: \"3c999580-b677-4bbf-b2e0-f425901a074e\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-56cbc87d66-2t2lj" Jul 29 08:22:33.872018 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:22:33.871981 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qtc6f\" (UniqueName: \"kubernetes.io/projected/3c999580-b677-4bbf-b2e0-f425901a074e-kube-api-access-qtc6f\") pod \"klusterlet-addon-workmgr-56cbc87d66-2t2lj\" (UID: \"3c999580-b677-4bbf-b2e0-f425901a074e\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-56cbc87d66-2t2lj" Jul 29 08:22:33.880335 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:22:33.880314 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-7c8hq\"" Jul 29 08:22:33.887060 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:22:33.887038 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-qk9kb" Jul 29 08:22:33.908890 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:22:33.908844 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8rp9g_cfd1d8df-e6b7-4a67-81f2-5d83da541e20/konnectivity-agent/0.log" limit=1024 Jul 29 08:22:33.971809 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:22:33.971754 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-wscl8\"" Jul 29 08:22:33.973059 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:22:33.973035 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/160e73ad-6c93-4ba9-b01a-7aea4e3fddfa-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-74f6c649f4-m8w74\" (UID: \"160e73ad-6c93-4ba9-b01a-7aea4e3fddfa\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-74f6c649f4-m8w74" Jul 29 08:22:33.973194 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:22:33.973069 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/3c999580-b677-4bbf-b2e0-f425901a074e-tmp\") pod \"klusterlet-addon-workmgr-56cbc87d66-2t2lj\" (UID: \"3c999580-b677-4bbf-b2e0-f425901a074e\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-56cbc87d66-2t2lj" Jul 29 08:22:33.973194 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:22:33.973113 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ghz4z\" (UniqueName: \"kubernetes.io/projected/160e73ad-6c93-4ba9-b01a-7aea4e3fddfa-kube-api-access-ghz4z\") pod \"managed-serviceaccount-addon-agent-74f6c649f4-m8w74\" (UID: \"160e73ad-6c93-4ba9-b01a-7aea4e3fddfa\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-74f6c649f4-m8w74" Jul 29 08:22:33.973194 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:22:33.973141 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/3c999580-b677-4bbf-b2e0-f425901a074e-klusterlet-config\") pod \"klusterlet-addon-workmgr-56cbc87d66-2t2lj\" (UID: \"3c999580-b677-4bbf-b2e0-f425901a074e\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-56cbc87d66-2t2lj" Jul 29 08:22:33.973194 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:22:33.973167 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qtc6f\" (UniqueName: \"kubernetes.io/projected/3c999580-b677-4bbf-b2e0-f425901a074e-kube-api-access-qtc6f\") pod \"klusterlet-addon-workmgr-56cbc87d66-2t2lj\" (UID: \"3c999580-b677-4bbf-b2e0-f425901a074e\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-56cbc87d66-2t2lj" Jul 29 08:22:33.973851 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:22:33.973814 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/3c999580-b677-4bbf-b2e0-f425901a074e-tmp\") pod \"klusterlet-addon-workmgr-56cbc87d66-2t2lj\" (UID: \"3c999580-b677-4bbf-b2e0-f425901a074e\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-56cbc87d66-2t2lj" Jul 29 08:22:33.976444 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:22:33.976402 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/160e73ad-6c93-4ba9-b01a-7aea4e3fddfa-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-74f6c649f4-m8w74\" (UID: \"160e73ad-6c93-4ba9-b01a-7aea4e3fddfa\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-74f6c649f4-m8w74" Jul 29 08:22:33.976725 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:22:33.976696 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/3c999580-b677-4bbf-b2e0-f425901a074e-klusterlet-config\") pod \"klusterlet-addon-workmgr-56cbc87d66-2t2lj\" (UID: \"3c999580-b677-4bbf-b2e0-f425901a074e\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-56cbc87d66-2t2lj" Jul 29 08:22:33.979679 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:22:33.979393 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-rdclp" Jul 29 08:22:33.985157 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:22:33.985134 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qtc6f\" (UniqueName: \"kubernetes.io/projected/3c999580-b677-4bbf-b2e0-f425901a074e-kube-api-access-qtc6f\") pod \"klusterlet-addon-workmgr-56cbc87d66-2t2lj\" (UID: \"3c999580-b677-4bbf-b2e0-f425901a074e\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-56cbc87d66-2t2lj" Jul 29 08:22:33.986054 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:22:33.986034 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ghz4z\" (UniqueName: \"kubernetes.io/projected/160e73ad-6c93-4ba9-b01a-7aea4e3fddfa-kube-api-access-ghz4z\") pod \"managed-serviceaccount-addon-agent-74f6c649f4-m8w74\" (UID: \"160e73ad-6c93-4ba9-b01a-7aea4e3fddfa\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-74f6c649f4-m8w74" Jul 29 08:22:34.017160 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:22:34.017125 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8rp9g_cfd1d8df-e6b7-4a67-81f2-5d83da541e20/konnectivity-agent/0.log" limit=1024 Jul 29 08:22:34.023451 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:22:34.023425 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-rn9kc"] Jul 29 08:22:34.027435 ip-10-0-131-138 kubenswrapper[2563]: W0729 08:22:34.027410 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb14e5aed_a2b3_41bf_9389_701d69342953.slice/crio-2f57e0dbcd97199d25eaa81ebaacc2b00ffe0c6d1b0d7576b0c570f998eb6204 WatchSource:0}: Error finding container 2f57e0dbcd97199d25eaa81ebaacc2b00ffe0c6d1b0d7576b0c570f998eb6204: Status 404 returned error can't find the container with id 2f57e0dbcd97199d25eaa81ebaacc2b00ffe0c6d1b0d7576b0c570f998eb6204 Jul 29 08:22:34.042162 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:22:34.042131 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-qk9kb"] Jul 29 08:22:34.042719 ip-10-0-131-138 kubenswrapper[2563]: W0729 08:22:34.042691 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod35e49848_94f6_43d9_b7c4_6c817dd5dde7.slice/crio-1e9d39cdd3bc3679475059a0a02863ab64dce8b601b80213612ebe21e3bada0f WatchSource:0}: Error finding container 1e9d39cdd3bc3679475059a0a02863ab64dce8b601b80213612ebe21e3bada0f: Status 404 returned error can't find the container with id 1e9d39cdd3bc3679475059a0a02863ab64dce8b601b80213612ebe21e3bada0f Jul 29 08:22:34.112306 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:22:34.112272 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-74f6c649f4-m8w74" Jul 29 08:22:34.117060 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:22:34.117035 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-56cbc87d66-2t2lj" Jul 29 08:22:34.137580 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:22:34.137555 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-rdclp"] Jul 29 08:22:34.141226 ip-10-0-131-138 kubenswrapper[2563]: W0729 08:22:34.141199 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podce06e4b8_c4ec_4edf_a9c4_d7d86a03d844.slice/crio-eed68f36ffbf75a996b86724a1b39aa47e92c2f8e8fddc05932ef0524512350a WatchSource:0}: Error finding container eed68f36ffbf75a996b86724a1b39aa47e92c2f8e8fddc05932ef0524512350a: Status 404 returned error can't find the container with id eed68f36ffbf75a996b86724a1b39aa47e92c2f8e8fddc05932ef0524512350a Jul 29 08:22:34.174876 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:22:34.174386 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/e521e734-a8d9-4748-ba28-c5f2beb27610-metrics-tls\") pod \"dns-default-nhklh\" (UID: \"e521e734-a8d9-4748-ba28-c5f2beb27610\") " pod="openshift-dns/dns-default-nhklh" Jul 29 08:22:34.177280 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:22:34.177207 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/e521e734-a8d9-4748-ba28-c5f2beb27610-metrics-tls\") pod \"dns-default-nhklh\" (UID: \"e521e734-a8d9-4748-ba28-c5f2beb27610\") " pod="openshift-dns/dns-default-nhklh" Jul 29 08:22:34.272190 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:22:34.272151 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-74f6c649f4-m8w74"] Jul 29 08:22:34.275686 ip-10-0-131-138 kubenswrapper[2563]: W0729 08:22:34.275655 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod160e73ad_6c93_4ba9_b01a_7aea4e3fddfa.slice/crio-1255b0613a32e825a0534dcfcdd4395b824666bcf0dbf3c487c85e5c48696d8d WatchSource:0}: Error finding container 1255b0613a32e825a0534dcfcdd4395b824666bcf0dbf3c487c85e5c48696d8d: Status 404 returned error can't find the container with id 1255b0613a32e825a0534dcfcdd4395b824666bcf0dbf3c487c85e5c48696d8d Jul 29 08:22:34.293080 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:22:34.293051 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-56cbc87d66-2t2lj"] Jul 29 08:22:34.295966 ip-10-0-131-138 kubenswrapper[2563]: W0729 08:22:34.295942 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3c999580_b677_4bbf_b2e0_f425901a074e.slice/crio-9cd3dbf076366db89c9d1f7b930e05345c98b75a63403461de6ee4a61e230fa8 WatchSource:0}: Error finding container 9cd3dbf076366db89c9d1f7b930e05345c98b75a63403461de6ee4a61e230fa8: Status 404 returned error can't find the container with id 9cd3dbf076366db89c9d1f7b930e05345c98b75a63403461de6ee4a61e230fa8 Jul 29 08:22:34.314306 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:22:34.314277 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8rp9g_cfd1d8df-e6b7-4a67-81f2-5d83da541e20/konnectivity-agent/0.log" limit=1024 Jul 29 08:22:34.323549 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:22:34.323520 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-74f6c649f4-m8w74" event={"ID":"160e73ad-6c93-4ba9-b01a-7aea4e3fddfa","Type":"ContainerStarted","Data":"1255b0613a32e825a0534dcfcdd4395b824666bcf0dbf3c487c85e5c48696d8d"} Jul 29 08:22:34.325148 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:22:34.325124 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-rdclp" event={"ID":"ce06e4b8-c4ec-4edf-a9c4-d7d86a03d844","Type":"ContainerStarted","Data":"908f7020fc814e7afdb86ec7b855b36185ca9f04cb05d36eff2c3c49529ade03"} Jul 29 08:22:34.325269 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:22:34.325163 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-rdclp" event={"ID":"ce06e4b8-c4ec-4edf-a9c4-d7d86a03d844","Type":"ContainerStarted","Data":"eed68f36ffbf75a996b86724a1b39aa47e92c2f8e8fddc05932ef0524512350a"} Jul 29 08:22:34.325269 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:22:34.325217 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-rdclp" Jul 29 08:22:34.326210 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:22:34.326193 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-56cbc87d66-2t2lj" event={"ID":"3c999580-b677-4bbf-b2e0-f425901a074e","Type":"ContainerStarted","Data":"9cd3dbf076366db89c9d1f7b930e05345c98b75a63403461de6ee4a61e230fa8"} Jul 29 08:22:34.327296 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:22:34.327278 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-qk9kb" event={"ID":"35e49848-94f6-43d9-b7c4-6c817dd5dde7","Type":"ContainerStarted","Data":"1e9d39cdd3bc3679475059a0a02863ab64dce8b601b80213612ebe21e3bada0f"} Jul 29 08:22:34.328237 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:22:34.328219 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-rn9kc" event={"ID":"b14e5aed-a2b3-41bf-9389-701d69342953","Type":"ContainerStarted","Data":"2f57e0dbcd97199d25eaa81ebaacc2b00ffe0c6d1b0d7576b0c570f998eb6204"} Jul 29 08:22:34.343861 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:22:34.343788 2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-rdclp" podStartSLOduration=66.343772537 podStartE2EDuration="1m6.343772537s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 08:21:28 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 08:22:34.343302544 +0000 UTC m=+66.924591323" watchObservedRunningTime="2026-07-29 08:22:34.343772537 +0000 UTC m=+66.925061315" Jul 29 08:22:34.376523 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:22:34.376490 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8rp9g_cfd1d8df-e6b7-4a67-81f2-5d83da541e20/konnectivity-agent/0.log" limit=1024 Jul 29 08:22:34.435475 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:22:34.435394 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-wsvft\"" Jul 29 08:22:34.442890 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:22:34.442865 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-nhklh" Jul 29 08:22:34.568105 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:22:34.568072 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8rp9g_cfd1d8df-e6b7-4a67-81f2-5d83da541e20/konnectivity-agent/0.log" limit=1024 Jul 29 08:22:34.613699 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:22:34.613669 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-nhklh"] Jul 29 08:22:34.617092 ip-10-0-131-138 kubenswrapper[2563]: W0729 08:22:34.617061 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode521e734_a8d9_4748_ba28_c5f2beb27610.slice/crio-370995a615a9025e89d938933ed554e4041f6a7cd0921372b559f78e1cfe1327 WatchSource:0}: Error finding container 370995a615a9025e89d938933ed554e4041f6a7cd0921372b559f78e1cfe1327: Status 404 returned error can't find the container with id 370995a615a9025e89d938933ed554e4041f6a7cd0921372b559f78e1cfe1327 Jul 29 08:22:34.993804 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:22:34.993740 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8rp9g_cfd1d8df-e6b7-4a67-81f2-5d83da541e20/konnectivity-agent/0.log" limit=1024 Jul 29 08:22:35.077580 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:22:35.077544 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8rp9g_cfd1d8df-e6b7-4a67-81f2-5d83da541e20/konnectivity-agent/0.log" limit=1024 Jul 29 08:22:35.170539 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:22:35.170503 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8rp9g_cfd1d8df-e6b7-4a67-81f2-5d83da541e20/konnectivity-agent/0.log" limit=1024 Jul 29 08:22:35.253742 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:22:35.253659 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8rp9g_cfd1d8df-e6b7-4a67-81f2-5d83da541e20/konnectivity-agent/0.log" limit=1024 Jul 29 08:22:35.336808 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:22:35.336767 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-nhklh" event={"ID":"e521e734-a8d9-4748-ba28-c5f2beb27610","Type":"ContainerStarted","Data":"370995a615a9025e89d938933ed554e4041f6a7cd0921372b559f78e1cfe1327"} Jul 29 08:22:35.380096 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:22:35.380062 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8rp9g_cfd1d8df-e6b7-4a67-81f2-5d83da541e20/konnectivity-agent/0.log" limit=1024 Jul 29 08:22:35.480092 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:22:35.480064 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8rp9g_cfd1d8df-e6b7-4a67-81f2-5d83da541e20/konnectivity-agent/0.log" limit=1024 Jul 29 08:22:35.930124 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:22:35.930001 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8rp9g_cfd1d8df-e6b7-4a67-81f2-5d83da541e20/konnectivity-agent/0.log" limit=1024 Jul 29 08:22:36.036185 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:22:36.036057 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8rp9g_cfd1d8df-e6b7-4a67-81f2-5d83da541e20/konnectivity-agent/0.log" limit=1024 Jul 29 08:22:36.175433 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:22:36.175344 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8rp9g_cfd1d8df-e6b7-4a67-81f2-5d83da541e20/konnectivity-agent/0.log" limit=1024 Jul 29 08:22:36.277171 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:22:36.277134 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8rp9g_cfd1d8df-e6b7-4a67-81f2-5d83da541e20/konnectivity-agent/0.log" limit=1024 Jul 29 08:22:36.346364 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:22:36.346327 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-qk9kb" event={"ID":"35e49848-94f6-43d9-b7c4-6c817dd5dde7","Type":"ContainerStarted","Data":"642c5ae4a10395e3525a8eba75de427bdefc23b210152a2fcb1727b64145322c"} Jul 29 08:22:36.346545 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:22:36.346398 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-qk9kb" event={"ID":"35e49848-94f6-43d9-b7c4-6c817dd5dde7","Type":"ContainerStarted","Data":"a994be1d4a95a6dfea60ebd14580976fd18b499ed1c64d1c5dd4a8693c0df703"} Jul 29 08:22:36.369915 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:22:36.369881 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8rp9g_cfd1d8df-e6b7-4a67-81f2-5d83da541e20/konnectivity-agent/0.log" limit=1024 Jul 29 08:22:36.432738 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:22:36.432651 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8rp9g_cfd1d8df-e6b7-4a67-81f2-5d83da541e20/konnectivity-agent/0.log" limit=1024 Jul 29 08:22:36.534907 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:22:36.534833 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8rp9g_cfd1d8df-e6b7-4a67-81f2-5d83da541e20/konnectivity-agent/0.log" limit=1024 Jul 29 08:22:36.761120 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:22:36.761057 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8rp9g_cfd1d8df-e6b7-4a67-81f2-5d83da541e20/konnectivity-agent/0.log" limit=1024 Jul 29 08:22:36.857413 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:22:36.857248 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8rp9g_cfd1d8df-e6b7-4a67-81f2-5d83da541e20/konnectivity-agent/0.log" limit=1024 Jul 29 08:22:36.929445 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:22:36.929397 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8rp9g_cfd1d8df-e6b7-4a67-81f2-5d83da541e20/konnectivity-agent/0.log" limit=1024 Jul 29 08:22:39.729655 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:22:39.729610 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/ed443996-4f98-44b5-9476-37a0c3c23099-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-548b489cb8-ch694\" (UID: \"ed443996-4f98-44b5-9476-37a0c3c23099\") " pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-ch694" Jul 29 08:22:39.730126 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:22:39.729682 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/bf0cdb1f-7233-4c76-8159-b233968ce8e5-service-ca-bundle\") pod \"router-default-7c7545584f-2d6gk\" (UID: \"bf0cdb1f-7233-4c76-8159-b233968ce8e5\") " pod="openshift-ingress/router-default-7c7545584f-2d6gk" Jul 29 08:22:39.730421 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:22:39.730392 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/bf0cdb1f-7233-4c76-8159-b233968ce8e5-service-ca-bundle\") pod \"router-default-7c7545584f-2d6gk\" (UID: \"bf0cdb1f-7233-4c76-8159-b233968ce8e5\") " pod="openshift-ingress/router-default-7c7545584f-2d6gk" Jul 29 08:22:39.732439 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:22:39.732417 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/ed443996-4f98-44b5-9476-37a0c3c23099-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-548b489cb8-ch694\" (UID: \"ed443996-4f98-44b5-9476-37a0c3c23099\") " pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-ch694" Jul 29 08:22:39.885171 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:22:39.885140 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-dockercfg-8shpd\"" Jul 29 08:22:39.891434 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:22:39.891396 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-dockercfg-v82zv\"" Jul 29 08:22:39.892352 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:22:39.892326 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-ch694" Jul 29 08:22:39.899131 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:22:39.899102 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/router-default-7c7545584f-2d6gk" Jul 29 08:22:41.331237 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:22:41.330531 2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-qk9kb" podStartSLOduration=71.973134531 podStartE2EDuration="1m13.330506352s" totalImagesPullingTime="1.357371821s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 08:21:28 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 08:22:36.365435227 +0000 UTC m=+68.946724007" watchObservedRunningTime="2026-07-29 08:22:41.330506352 +0000 UTC m=+73.911795130" Jul 29 08:22:41.331237 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:22:41.331168 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/router-default-7c7545584f-2d6gk"] Jul 29 08:22:41.360524 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:22:41.360327 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-548b489cb8-ch694"] Jul 29 08:22:41.369837 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:22:41.369807 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-74f6c649f4-m8w74" event={"ID":"160e73ad-6c93-4ba9-b01a-7aea4e3fddfa","Type":"ContainerStarted","Data":"d8bde6a4c3c277f8934f1498f44da6c735891f4c5c51c6997028fe8a292d3059"} Jul 29 08:22:41.374998 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:22:41.374924 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-nhklh" event={"ID":"e521e734-a8d9-4748-ba28-c5f2beb27610","Type":"ContainerStarted","Data":"7b109fab13ae9fd2db467621da8cde68951af4adcadc8ae20167b98fb4cf39fc"} Jul 29 08:22:41.377154 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:22:41.377105 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/router-default-7c7545584f-2d6gk" event={"ID":"bf0cdb1f-7233-4c76-8159-b233968ce8e5","Type":"ContainerStarted","Data":"98d09c0e11b76a9ebed2b47fc28409f8cb813bba5dee7b0de8f9c5b758eda36f"} Jul 29 08:22:41.378824 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:22:41.378768 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-56cbc87d66-2t2lj" event={"ID":"3c999580-b677-4bbf-b2e0-f425901a074e","Type":"ContainerStarted","Data":"c5daa85392516ef1ad41ef926e1d16b416f3c6fe25632854a8a1a4423a7dba3b"} Jul 29 08:22:41.379286 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:22:41.379203 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-56cbc87d66-2t2lj" Jul 29 08:22:41.380896 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:22:41.380756 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-56cbc87d66-2t2lj" Jul 29 08:22:41.380896 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:22:41.380886 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-rn9kc" event={"ID":"b14e5aed-a2b3-41bf-9389-701d69342953","Type":"ContainerStarted","Data":"d03954e3a433f1d4b1d58b278804c5b47f8128b9cae912c871a608defae96d9e"} Jul 29 08:22:41.389267 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:22:41.389196 2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-74f6c649f4-m8w74" podStartSLOduration=1.51613966 podStartE2EDuration="8.389175738s" totalImagesPullingTime="6.873036078s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 08:22:33 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 08:22:41.388057588 +0000 UTC m=+73.969346367" watchObservedRunningTime="2026-07-29 08:22:41.389175738 +0000 UTC m=+73.970464517" Jul 29 08:22:41.407782 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:22:41.407736 2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-56cbc87d66-2t2lj" podStartSLOduration=1.539534809 podStartE2EDuration="8.40770982s" totalImagesPullingTime="6.868175011s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 08:22:33 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 08:22:41.407091371 +0000 UTC m=+73.988380143" watchObservedRunningTime="2026-07-29 08:22:41.40770982 +0000 UTC m=+73.988998598" Jul 29 08:22:41.424698 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:22:41.424641 2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-rn9kc" podStartSLOduration=66.300132399 podStartE2EDuration="1m13.424622578s" totalImagesPullingTime="7.124490179s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-29 08:21:28 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 08:22:41.422998792 +0000 UTC m=+74.004287571" watchObservedRunningTime="2026-07-29 08:22:41.424622578 +0000 UTC m=+74.005911356" Jul 29 08:22:42.388422 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:22:42.388385 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-nhklh" event={"ID":"e521e734-a8d9-4748-ba28-c5f2beb27610","Type":"ContainerStarted","Data":"4f64c1c09ddc82ab959ddf8ba56d9ba15079af0d262d8a31da1fe83eef9a76fa"} Jul 29 08:22:42.388871 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:22:42.388489 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-nhklh" Jul 29 08:22:42.389892 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:22:42.389865 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/router-default-7c7545584f-2d6gk" event={"ID":"bf0cdb1f-7233-4c76-8159-b233968ce8e5","Type":"ContainerStarted","Data":"181ae8bc884a4d61f35bdbaf88c50241ca19c4b0f0df26a9c237577975270070"} Jul 29 08:22:42.391121 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:22:42.391088 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-ch694" event={"ID":"ed443996-4f98-44b5-9476-37a0c3c23099","Type":"ContainerStarted","Data":"714d45c2036e274c7cc8afd307ba33f939bed5c29a49a8335503e5930d22909e"} Jul 29 08:22:42.417261 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:22:42.417218 2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/dns-default-nhklh" podStartSLOduration=33.889349388 podStartE2EDuration="40.417203667s" totalImagesPullingTime="6.527854279s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 08:22:02 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 08:22:42.416203616 +0000 UTC m=+74.997492395" watchObservedRunningTime="2026-07-29 08:22:42.417203667 +0000 UTC m=+74.998492444" Jul 29 08:22:42.460169 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:22:42.460120 2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/router-default-7c7545584f-2d6gk" podStartSLOduration=35.460101421 podStartE2EDuration="35.460101421s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 08:22:07 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 08:22:42.460061061 +0000 UTC m=+75.041349839" watchObservedRunningTime="2026-07-29 08:22:42.460101421 +0000 UTC m=+75.041390196" Jul 29 08:22:42.899586 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:22:42.899553 2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/router-default-7c7545584f-2d6gk" Jul 29 08:22:42.902544 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:22:42.902508 2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/router-default-7c7545584f-2d6gk" Jul 29 08:22:43.396365 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:22:43.396331 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-ch694" event={"ID":"ed443996-4f98-44b5-9476-37a0c3c23099","Type":"ContainerStarted","Data":"cf06d7a36922eee2ba9fcee48fd5a97b3e4f428c829868af4d2cea282a4d9fdf"} Jul 29 08:22:43.396820 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:22:43.396576 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/router-default-7c7545584f-2d6gk" Jul 29 08:22:43.397168 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:22:43.397142 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8rp9g_cfd1d8df-e6b7-4a67-81f2-5d83da541e20/konnectivity-agent/0.log" limit=1024 Jul 29 08:22:43.397765 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:22:43.397746 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/router-default-7c7545584f-2d6gk" Jul 29 08:22:43.415970 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:22:43.415921 2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-ch694" podStartSLOduration=34.738234654 podStartE2EDuration="36.415902761s" totalImagesPullingTime="1.677668107s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 08:22:07 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 08:22:43.415845547 +0000 UTC m=+75.997134327" watchObservedRunningTime="2026-07-29 08:22:43.415902761 +0000 UTC m=+75.997191543" Jul 29 08:22:43.665050 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:22:43.664968 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8rp9g_cfd1d8df-e6b7-4a67-81f2-5d83da541e20/konnectivity-agent/0.log" limit=1024 Jul 29 08:22:43.992600 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:22:43.992560 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8rp9g_cfd1d8df-e6b7-4a67-81f2-5d83da541e20/konnectivity-agent/0.log" limit=1024 Jul 29 08:22:44.214335 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:22:44.214301 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8rp9g_cfd1d8df-e6b7-4a67-81f2-5d83da541e20/konnectivity-agent/0.log" limit=1024 Jul 29 08:22:44.277608 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:22:44.277532 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8rp9g_cfd1d8df-e6b7-4a67-81f2-5d83da541e20/konnectivity-agent/0.log" limit=1024 Jul 29 08:22:44.975673 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:22:44.975642 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/bb16ee9e-483f-465c-8deb-4a36b0db9743-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-29q8s\" (UID: \"bb16ee9e-483f-465c-8deb-4a36b0db9743\") " pod="openshift-insights/insights-runtime-extractor-29q8s" Jul 29 08:22:44.978038 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:22:44.978018 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/bb16ee9e-483f-465c-8deb-4a36b0db9743-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-29q8s\" (UID: \"bb16ee9e-483f-465c-8deb-4a36b0db9743\") " pod="openshift-insights/insights-runtime-extractor-29q8s" Jul 29 08:22:45.135534 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:22:45.135503 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-x95nv\"" Jul 29 08:22:45.143092 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:22:45.143064 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-29q8s" Jul 29 08:22:45.272792 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:22:45.272611 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-29q8s"] Jul 29 08:22:45.275208 ip-10-0-131-138 kubenswrapper[2563]: W0729 08:22:45.275188 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbb16ee9e_483f_465c_8deb_4a36b0db9743.slice/crio-b61536b1d0667addd3578651f128e039b7f1cfeaecf563b51e25bec067d75c4c WatchSource:0}: Error finding container b61536b1d0667addd3578651f128e039b7f1cfeaecf563b51e25bec067d75c4c: Status 404 returned error can't find the container with id b61536b1d0667addd3578651f128e039b7f1cfeaecf563b51e25bec067d75c4c Jul 29 08:22:45.403846 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:22:45.403813 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-29q8s" event={"ID":"bb16ee9e-483f-465c-8deb-4a36b0db9743","Type":"ContainerStarted","Data":"b61536b1d0667addd3578651f128e039b7f1cfeaecf563b51e25bec067d75c4c"} Jul 29 08:22:45.820320 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:22:45.820283 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8rp9g_cfd1d8df-e6b7-4a67-81f2-5d83da541e20/konnectivity-agent/0.log" limit=1024 Jul 29 08:22:45.901146 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:22:45.901114 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8rp9g_cfd1d8df-e6b7-4a67-81f2-5d83da541e20/konnectivity-agent/0.log" limit=1024 Jul 29 08:22:50.427060 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:22:50.427028 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-29q8s" event={"ID":"bb16ee9e-483f-465c-8deb-4a36b0db9743","Type":"ContainerStarted","Data":"2a883af20ae37800bf1e713283f3d62d04a9ee684fee1575cb900f3d79cdadef"} Jul 29 08:22:50.533388 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:22:50.533353 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8rp9g_cfd1d8df-e6b7-4a67-81f2-5d83da541e20/konnectivity-agent/0.log" limit=1024 Jul 29 08:22:50.592753 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:22:50.592675 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8rp9g_cfd1d8df-e6b7-4a67-81f2-5d83da541e20/konnectivity-agent/0.log" limit=1024 Jul 29 08:22:52.399088 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:22:52.399040 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-nhklh" Jul 29 08:22:53.022653 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:22:53.022608 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8rp9g_cfd1d8df-e6b7-4a67-81f2-5d83da541e20/konnectivity-agent/0.log" limit=1024 Jul 29 08:22:53.122995 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:22:53.122959 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8rp9g_cfd1d8df-e6b7-4a67-81f2-5d83da541e20/konnectivity-agent/0.log" limit=1024 Jul 29 08:22:54.448332 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:22:54.448291 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-29q8s" event={"ID":"bb16ee9e-483f-465c-8deb-4a36b0db9743","Type":"ContainerStarted","Data":"87160db83ae779172083b715e76cd02a5a56e15c64206c23e985bce1b3ca1c5b"} Jul 29 08:22:55.716281 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:22:55.716233 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8rp9g_cfd1d8df-e6b7-4a67-81f2-5d83da541e20/konnectivity-agent/0.log" limit=1024 Jul 29 08:22:55.826115 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:22:55.826085 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8rp9g_cfd1d8df-e6b7-4a67-81f2-5d83da541e20/konnectivity-agent/0.log" limit=1024 Jul 29 08:22:55.999693 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:22:55.999596 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8rp9g_cfd1d8df-e6b7-4a67-81f2-5d83da541e20/konnectivity-agent/0.log" limit=1024 Jul 29 08:22:57.029729 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:22:57.029695 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8rp9g_cfd1d8df-e6b7-4a67-81f2-5d83da541e20/konnectivity-agent/0.log" limit=1024 Jul 29 08:22:57.093598 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:22:57.093538 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8rp9g_cfd1d8df-e6b7-4a67-81f2-5d83da541e20/konnectivity-agent/0.log" limit=1024 Jul 29 08:22:57.391861 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:22:57.391784 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8rp9g_cfd1d8df-e6b7-4a67-81f2-5d83da541e20/konnectivity-agent/0.log" limit=1024 Jul 29 08:22:57.548574 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:22:57.548544 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8rp9g_cfd1d8df-e6b7-4a67-81f2-5d83da541e20/konnectivity-agent/0.log" limit=1024 Jul 29 08:22:58.464305 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:22:58.464271 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-29q8s" event={"ID":"bb16ee9e-483f-465c-8deb-4a36b0db9743","Type":"ContainerStarted","Data":"200bb263bbc927c013ea19de4a37687fb3b62076740ae2bd6458e6c36ac7cee7"} Jul 29 08:22:58.485959 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:22:58.485913 2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-29q8s" podStartSLOduration=34.41657835 podStartE2EDuration="46.485897973s" totalImagesPullingTime="12.069319623s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-29 08:22:12 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 08:22:58.484152861 +0000 UTC m=+91.065441667" watchObservedRunningTime="2026-07-29 08:22:58.485897973 +0000 UTC m=+91.067186751" Jul 29 08:23:01.654588 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:23:01.654556 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8rp9g_cfd1d8df-e6b7-4a67-81f2-5d83da541e20/konnectivity-agent/0.log" limit=1024 Jul 29 08:23:01.754819 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:23:01.754778 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8rp9g_cfd1d8df-e6b7-4a67-81f2-5d83da541e20/konnectivity-agent/0.log" limit=1024 Jul 29 08:23:03.330739 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:23:03.330699 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8rp9g_cfd1d8df-e6b7-4a67-81f2-5d83da541e20/konnectivity-agent/0.log" limit=1024 Jul 29 08:23:03.391725 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:23:03.391693 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8rp9g_cfd1d8df-e6b7-4a67-81f2-5d83da541e20/konnectivity-agent/0.log" limit=1024 Jul 29 08:23:05.203690 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:23:05.203645 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8rp9g_cfd1d8df-e6b7-4a67-81f2-5d83da541e20/konnectivity-agent/0.log" limit=1024 Jul 29 08:23:05.269223 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:23:05.269178 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8rp9g_cfd1d8df-e6b7-4a67-81f2-5d83da541e20/konnectivity-agent/0.log" limit=1024 Jul 29 08:23:05.339548 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:23:05.339521 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-rdclp" Jul 29 08:23:05.476248 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:23:05.476171 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8rp9g_cfd1d8df-e6b7-4a67-81f2-5d83da541e20/konnectivity-agent/0.log" limit=1024 Jul 29 08:23:05.545053 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:23:05.545018 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8rp9g_cfd1d8df-e6b7-4a67-81f2-5d83da541e20/konnectivity-agent/0.log" limit=1024 Jul 29 08:23:06.373438 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:23:06.373405 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8rp9g_cfd1d8df-e6b7-4a67-81f2-5d83da541e20/konnectivity-agent/0.log" limit=1024 Jul 29 08:23:06.475914 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:23:06.475879 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8rp9g_cfd1d8df-e6b7-4a67-81f2-5d83da541e20/konnectivity-agent/0.log" limit=1024 Jul 29 08:23:07.112379 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:23:07.112346 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8rp9g_cfd1d8df-e6b7-4a67-81f2-5d83da541e20/konnectivity-agent/0.log" limit=1024 Jul 29 08:23:07.185578 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:23:07.185550 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8rp9g_cfd1d8df-e6b7-4a67-81f2-5d83da541e20/konnectivity-agent/0.log" limit=1024 Jul 29 08:23:08.932919 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:23:08.932887 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8rp9g_cfd1d8df-e6b7-4a67-81f2-5d83da541e20/konnectivity-agent/0.log" limit=1024 Jul 29 08:23:09.046142 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:23:09.046109 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8rp9g_cfd1d8df-e6b7-4a67-81f2-5d83da541e20/konnectivity-agent/0.log" limit=1024 Jul 29 08:23:09.146945 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:23:09.146913 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8rp9g_cfd1d8df-e6b7-4a67-81f2-5d83da541e20/konnectivity-agent/0.log" limit=1024 Jul 29 08:23:09.260649 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:23:09.260610 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8rp9g_cfd1d8df-e6b7-4a67-81f2-5d83da541e20/konnectivity-agent/0.log" limit=1024 Jul 29 08:23:09.437930 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:23:09.437893 2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/kube-state-metrics-56d9f944b4-9cctr"] Jul 29 08:23:09.442109 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:23:09.442092 2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-s44d8"] Jul 29 08:23:09.442239 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:23:09.442223 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/kube-state-metrics-56d9f944b4-9cctr" Jul 29 08:23:09.447140 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:23:09.447109 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\"" Jul 29 08:23:09.448572 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:23:09.448554 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-s44d8" Jul 29 08:23:09.460938 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:23:09.460899 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\"" Jul 29 08:23:09.461061 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:23:09.461027 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\"" Jul 29 08:23:09.461219 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:23:09.461202 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\"" Jul 29 08:23:09.461329 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:23:09.461300 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/39642d9c-85e5-42e1-8b0d-90db3adaf3ef-kube-state-metrics-tls\") pod \"kube-state-metrics-56d9f944b4-9cctr\" (UID: \"39642d9c-85e5-42e1-8b0d-90db3adaf3ef\") " pod="openshift-monitoring/kube-state-metrics-56d9f944b4-9cctr" Jul 29 08:23:09.461389 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:23:09.461335 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/39642d9c-85e5-42e1-8b0d-90db3adaf3ef-volume-directive-shadow\") pod \"kube-state-metrics-56d9f944b4-9cctr\" (UID: \"39642d9c-85e5-42e1-8b0d-90db3adaf3ef\") " pod="openshift-monitoring/kube-state-metrics-56d9f944b4-9cctr" Jul 29 08:23:09.461389 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:23:09.461354 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/66322420-ef49-4c4c-99c5-1a19d67d4244-node-exporter-tls\") pod \"node-exporter-s44d8\" (UID: \"66322420-ef49-4c4c-99c5-1a19d67d4244\") " pod="openshift-monitoring/node-exporter-s44d8" Jul 29 08:23:09.461389 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:23:09.461373 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/66322420-ef49-4c4c-99c5-1a19d67d4244-node-exporter-wtmp\") pod \"node-exporter-s44d8\" (UID: \"66322420-ef49-4c4c-99c5-1a19d67d4244\") " pod="openshift-monitoring/node-exporter-s44d8" Jul 29 08:23:09.461488 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:23:09.461430 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/39642d9c-85e5-42e1-8b0d-90db3adaf3ef-metrics-client-ca\") pod \"kube-state-metrics-56d9f944b4-9cctr\" (UID: \"39642d9c-85e5-42e1-8b0d-90db3adaf3ef\") " pod="openshift-monitoring/kube-state-metrics-56d9f944b4-9cctr" Jul 29 08:23:09.461488 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:23:09.461443 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-86ckw\" (UniqueName: \"kubernetes.io/projected/39642d9c-85e5-42e1-8b0d-90db3adaf3ef-kube-api-access-86ckw\") pod \"kube-state-metrics-56d9f944b4-9cctr\" (UID: \"39642d9c-85e5-42e1-8b0d-90db3adaf3ef\") " pod="openshift-monitoring/kube-state-metrics-56d9f944b4-9cctr" Jul 29 08:23:09.461488 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:23:09.461458 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/66322420-ef49-4c4c-99c5-1a19d67d4244-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-s44d8\" (UID: \"66322420-ef49-4c4c-99c5-1a19d67d4244\") " pod="openshift-monitoring/node-exporter-s44d8" Jul 29 08:23:09.461488 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:23:09.461472 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/66322420-ef49-4c4c-99c5-1a19d67d4244-node-exporter-accelerators-collector-config\") pod \"node-exporter-s44d8\" (UID: \"66322420-ef49-4c4c-99c5-1a19d67d4244\") " pod="openshift-monitoring/node-exporter-s44d8" Jul 29 08:23:09.461627 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:23:09.461488 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-q5x85\" (UniqueName: \"kubernetes.io/projected/66322420-ef49-4c4c-99c5-1a19d67d4244-kube-api-access-q5x85\") pod \"node-exporter-s44d8\" (UID: \"66322420-ef49-4c4c-99c5-1a19d67d4244\") " pod="openshift-monitoring/node-exporter-s44d8" Jul 29 08:23:09.461627 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:23:09.461507 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/66322420-ef49-4c4c-99c5-1a19d67d4244-sys\") pod \"node-exporter-s44d8\" (UID: \"66322420-ef49-4c4c-99c5-1a19d67d4244\") " pod="openshift-monitoring/node-exporter-s44d8" Jul 29 08:23:09.461627 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:23:09.461545 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/66322420-ef49-4c4c-99c5-1a19d67d4244-node-exporter-textfile\") pod \"node-exporter-s44d8\" (UID: \"66322420-ef49-4c4c-99c5-1a19d67d4244\") " pod="openshift-monitoring/node-exporter-s44d8" Jul 29 08:23:09.461627 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:23:09.461571 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/66322420-ef49-4c4c-99c5-1a19d67d4244-metrics-client-ca\") pod \"node-exporter-s44d8\" (UID: \"66322420-ef49-4c4c-99c5-1a19d67d4244\") " pod="openshift-monitoring/node-exporter-s44d8" Jul 29 08:23:09.461627 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:23:09.461593 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/39642d9c-85e5-42e1-8b0d-90db3adaf3ef-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-56d9f944b4-9cctr\" (UID: \"39642d9c-85e5-42e1-8b0d-90db3adaf3ef\") " pod="openshift-monitoring/kube-state-metrics-56d9f944b4-9cctr" Jul 29 08:23:09.461627 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:23:09.461610 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/66322420-ef49-4c4c-99c5-1a19d67d4244-root\") pod \"node-exporter-s44d8\" (UID: \"66322420-ef49-4c4c-99c5-1a19d67d4244\") " pod="openshift-monitoring/node-exporter-s44d8" Jul 29 08:23:09.461820 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:23:09.461640 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/39642d9c-85e5-42e1-8b0d-90db3adaf3ef-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-56d9f944b4-9cctr\" (UID: \"39642d9c-85e5-42e1-8b0d-90db3adaf3ef\") " pod="openshift-monitoring/kube-state-metrics-56d9f944b4-9cctr" Jul 29 08:23:09.465144 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:23:09.465126 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-tls\"" Jul 29 08:23:09.465202 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:23:09.465149 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-kube-rbac-proxy-config\"" Jul 29 08:23:09.465413 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:23:09.465399 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-mf9b9\"" Jul 29 08:23:09.465909 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:23:09.465895 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-custom-resource-state-configmap\"" Jul 29 08:23:09.471395 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:23:09.471377 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/kube-state-metrics-56d9f944b4-9cctr"] Jul 29 08:23:09.485613 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:23:09.485586 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-dockercfg-dbfxp\"" Jul 29 08:23:09.562053 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:23:09.561969 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/66322420-ef49-4c4c-99c5-1a19d67d4244-node-exporter-tls\") pod \"node-exporter-s44d8\" (UID: \"66322420-ef49-4c4c-99c5-1a19d67d4244\") " pod="openshift-monitoring/node-exporter-s44d8" Jul 29 08:23:09.562053 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:23:09.562005 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/66322420-ef49-4c4c-99c5-1a19d67d4244-node-exporter-wtmp\") pod \"node-exporter-s44d8\" (UID: \"66322420-ef49-4c4c-99c5-1a19d67d4244\") " pod="openshift-monitoring/node-exporter-s44d8" Jul 29 08:23:09.562053 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:23:09.562031 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/39642d9c-85e5-42e1-8b0d-90db3adaf3ef-metrics-client-ca\") pod \"kube-state-metrics-56d9f944b4-9cctr\" (UID: \"39642d9c-85e5-42e1-8b0d-90db3adaf3ef\") " pod="openshift-monitoring/kube-state-metrics-56d9f944b4-9cctr" Jul 29 08:23:09.562338 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:23:09.562056 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-86ckw\" (UniqueName: \"kubernetes.io/projected/39642d9c-85e5-42e1-8b0d-90db3adaf3ef-kube-api-access-86ckw\") pod \"kube-state-metrics-56d9f944b4-9cctr\" (UID: \"39642d9c-85e5-42e1-8b0d-90db3adaf3ef\") " pod="openshift-monitoring/kube-state-metrics-56d9f944b4-9cctr" Jul 29 08:23:09.562338 ip-10-0-131-138 kubenswrapper[2563]: E0729 08:23:09.562133 2563 secret.go:189] Couldn't get secret openshift-monitoring/node-exporter-tls: secret "node-exporter-tls" not found Jul 29 08:23:09.562338 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:23:09.562175 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/66322420-ef49-4c4c-99c5-1a19d67d4244-node-exporter-wtmp\") pod \"node-exporter-s44d8\" (UID: \"66322420-ef49-4c4c-99c5-1a19d67d4244\") " pod="openshift-monitoring/node-exporter-s44d8" Jul 29 08:23:09.562338 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:23:09.562176 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/66322420-ef49-4c4c-99c5-1a19d67d4244-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-s44d8\" (UID: \"66322420-ef49-4c4c-99c5-1a19d67d4244\") " pod="openshift-monitoring/node-exporter-s44d8" Jul 29 08:23:09.562338 ip-10-0-131-138 kubenswrapper[2563]: E0729 08:23:09.562206 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/66322420-ef49-4c4c-99c5-1a19d67d4244-node-exporter-tls podName:66322420-ef49-4c4c-99c5-1a19d67d4244 nodeName:}" failed. No retries permitted until 2026-07-29 08:23:10.062183403 +0000 UTC m=+102.643472162 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "node-exporter-tls" (UniqueName: "kubernetes.io/secret/66322420-ef49-4c4c-99c5-1a19d67d4244-node-exporter-tls") pod "node-exporter-s44d8" (UID: "66322420-ef49-4c4c-99c5-1a19d67d4244") : secret "node-exporter-tls" not found Jul 29 08:23:09.562338 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:23:09.562297 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/66322420-ef49-4c4c-99c5-1a19d67d4244-node-exporter-accelerators-collector-config\") pod \"node-exporter-s44d8\" (UID: \"66322420-ef49-4c4c-99c5-1a19d67d4244\") " pod="openshift-monitoring/node-exporter-s44d8" Jul 29 08:23:09.562635 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:23:09.562340 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-q5x85\" (UniqueName: \"kubernetes.io/projected/66322420-ef49-4c4c-99c5-1a19d67d4244-kube-api-access-q5x85\") pod \"node-exporter-s44d8\" (UID: \"66322420-ef49-4c4c-99c5-1a19d67d4244\") " pod="openshift-monitoring/node-exporter-s44d8" Jul 29 08:23:09.562635 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:23:09.562373 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/66322420-ef49-4c4c-99c5-1a19d67d4244-sys\") pod \"node-exporter-s44d8\" (UID: \"66322420-ef49-4c4c-99c5-1a19d67d4244\") " pod="openshift-monitoring/node-exporter-s44d8" Jul 29 08:23:09.562635 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:23:09.562411 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/66322420-ef49-4c4c-99c5-1a19d67d4244-node-exporter-textfile\") pod \"node-exporter-s44d8\" (UID: \"66322420-ef49-4c4c-99c5-1a19d67d4244\") " pod="openshift-monitoring/node-exporter-s44d8" Jul 29 08:23:09.562635 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:23:09.562438 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/66322420-ef49-4c4c-99c5-1a19d67d4244-metrics-client-ca\") pod \"node-exporter-s44d8\" (UID: \"66322420-ef49-4c4c-99c5-1a19d67d4244\") " pod="openshift-monitoring/node-exporter-s44d8" Jul 29 08:23:09.562635 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:23:09.562453 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/66322420-ef49-4c4c-99c5-1a19d67d4244-sys\") pod \"node-exporter-s44d8\" (UID: \"66322420-ef49-4c4c-99c5-1a19d67d4244\") " pod="openshift-monitoring/node-exporter-s44d8" Jul 29 08:23:09.562635 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:23:09.562470 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/39642d9c-85e5-42e1-8b0d-90db3adaf3ef-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-56d9f944b4-9cctr\" (UID: \"39642d9c-85e5-42e1-8b0d-90db3adaf3ef\") " pod="openshift-monitoring/kube-state-metrics-56d9f944b4-9cctr" Jul 29 08:23:09.562635 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:23:09.562496 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/66322420-ef49-4c4c-99c5-1a19d67d4244-root\") pod \"node-exporter-s44d8\" (UID: \"66322420-ef49-4c4c-99c5-1a19d67d4244\") " pod="openshift-monitoring/node-exporter-s44d8" Jul 29 08:23:09.562635 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:23:09.562556 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/39642d9c-85e5-42e1-8b0d-90db3adaf3ef-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-56d9f944b4-9cctr\" (UID: \"39642d9c-85e5-42e1-8b0d-90db3adaf3ef\") " pod="openshift-monitoring/kube-state-metrics-56d9f944b4-9cctr" Jul 29 08:23:09.562635 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:23:09.562587 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/39642d9c-85e5-42e1-8b0d-90db3adaf3ef-kube-state-metrics-tls\") pod \"kube-state-metrics-56d9f944b4-9cctr\" (UID: \"39642d9c-85e5-42e1-8b0d-90db3adaf3ef\") " pod="openshift-monitoring/kube-state-metrics-56d9f944b4-9cctr" Jul 29 08:23:09.563047 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:23:09.562641 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/39642d9c-85e5-42e1-8b0d-90db3adaf3ef-volume-directive-shadow\") pod \"kube-state-metrics-56d9f944b4-9cctr\" (UID: \"39642d9c-85e5-42e1-8b0d-90db3adaf3ef\") " pod="openshift-monitoring/kube-state-metrics-56d9f944b4-9cctr" Jul 29 08:23:09.563047 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:23:09.562678 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/66322420-ef49-4c4c-99c5-1a19d67d4244-root\") pod \"node-exporter-s44d8\" (UID: \"66322420-ef49-4c4c-99c5-1a19d67d4244\") " pod="openshift-monitoring/node-exporter-s44d8" Jul 29 08:23:09.563047 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:23:09.562749 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/66322420-ef49-4c4c-99c5-1a19d67d4244-node-exporter-textfile\") pod \"node-exporter-s44d8\" (UID: \"66322420-ef49-4c4c-99c5-1a19d67d4244\") " pod="openshift-monitoring/node-exporter-s44d8" Jul 29 08:23:09.563047 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:23:09.562885 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/39642d9c-85e5-42e1-8b0d-90db3adaf3ef-metrics-client-ca\") pod \"kube-state-metrics-56d9f944b4-9cctr\" (UID: \"39642d9c-85e5-42e1-8b0d-90db3adaf3ef\") " pod="openshift-monitoring/kube-state-metrics-56d9f944b4-9cctr" Jul 29 08:23:09.563047 ip-10-0-131-138 kubenswrapper[2563]: E0729 08:23:09.562966 2563 secret.go:189] Couldn't get secret openshift-monitoring/kube-state-metrics-tls: secret "kube-state-metrics-tls" not found Jul 29 08:23:09.563047 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:23:09.562989 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/66322420-ef49-4c4c-99c5-1a19d67d4244-metrics-client-ca\") pod \"node-exporter-s44d8\" (UID: \"66322420-ef49-4c4c-99c5-1a19d67d4244\") " pod="openshift-monitoring/node-exporter-s44d8" Jul 29 08:23:09.563047 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:23:09.563013 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/39642d9c-85e5-42e1-8b0d-90db3adaf3ef-volume-directive-shadow\") pod \"kube-state-metrics-56d9f944b4-9cctr\" (UID: \"39642d9c-85e5-42e1-8b0d-90db3adaf3ef\") " pod="openshift-monitoring/kube-state-metrics-56d9f944b4-9cctr" Jul 29 08:23:09.563047 ip-10-0-131-138 kubenswrapper[2563]: E0729 08:23:09.563025 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/39642d9c-85e5-42e1-8b0d-90db3adaf3ef-kube-state-metrics-tls podName:39642d9c-85e5-42e1-8b0d-90db3adaf3ef nodeName:}" failed. No retries permitted until 2026-07-29 08:23:10.063010199 +0000 UTC m=+102.644298955 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-state-metrics-tls" (UniqueName: "kubernetes.io/secret/39642d9c-85e5-42e1-8b0d-90db3adaf3ef-kube-state-metrics-tls") pod "kube-state-metrics-56d9f944b4-9cctr" (UID: "39642d9c-85e5-42e1-8b0d-90db3adaf3ef") : secret "kube-state-metrics-tls" not found Jul 29 08:23:09.563331 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:23:09.563130 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/66322420-ef49-4c4c-99c5-1a19d67d4244-node-exporter-accelerators-collector-config\") pod \"node-exporter-s44d8\" (UID: \"66322420-ef49-4c4c-99c5-1a19d67d4244\") " pod="openshift-monitoring/node-exporter-s44d8" Jul 29 08:23:09.563372 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:23:09.563335 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/39642d9c-85e5-42e1-8b0d-90db3adaf3ef-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-56d9f944b4-9cctr\" (UID: \"39642d9c-85e5-42e1-8b0d-90db3adaf3ef\") " pod="openshift-monitoring/kube-state-metrics-56d9f944b4-9cctr" Jul 29 08:23:09.564746 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:23:09.564716 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/66322420-ef49-4c4c-99c5-1a19d67d4244-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-s44d8\" (UID: \"66322420-ef49-4c4c-99c5-1a19d67d4244\") " pod="openshift-monitoring/node-exporter-s44d8" Jul 29 08:23:09.565041 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:23:09.565024 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/39642d9c-85e5-42e1-8b0d-90db3adaf3ef-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-56d9f944b4-9cctr\" (UID: \"39642d9c-85e5-42e1-8b0d-90db3adaf3ef\") " pod="openshift-monitoring/kube-state-metrics-56d9f944b4-9cctr" Jul 29 08:23:09.639247 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:23:09.639213 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-86ckw\" (UniqueName: \"kubernetes.io/projected/39642d9c-85e5-42e1-8b0d-90db3adaf3ef-kube-api-access-86ckw\") pod \"kube-state-metrics-56d9f944b4-9cctr\" (UID: \"39642d9c-85e5-42e1-8b0d-90db3adaf3ef\") " pod="openshift-monitoring/kube-state-metrics-56d9f944b4-9cctr" Jul 29 08:23:09.640250 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:23:09.640229 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-q5x85\" (UniqueName: \"kubernetes.io/projected/66322420-ef49-4c4c-99c5-1a19d67d4244-kube-api-access-q5x85\") pod \"node-exporter-s44d8\" (UID: \"66322420-ef49-4c4c-99c5-1a19d67d4244\") " pod="openshift-monitoring/node-exporter-s44d8" Jul 29 08:23:09.843328 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:23:09.843232 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8rp9g_cfd1d8df-e6b7-4a67-81f2-5d83da541e20/konnectivity-agent/0.log" limit=1024 Jul 29 08:23:09.976024 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:23:09.975988 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8rp9g_cfd1d8df-e6b7-4a67-81f2-5d83da541e20/konnectivity-agent/0.log" limit=1024 Jul 29 08:23:10.066028 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:23:10.065991 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/66322420-ef49-4c4c-99c5-1a19d67d4244-node-exporter-tls\") pod \"node-exporter-s44d8\" (UID: \"66322420-ef49-4c4c-99c5-1a19d67d4244\") " pod="openshift-monitoring/node-exporter-s44d8" Jul 29 08:23:10.066199 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:23:10.066079 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/39642d9c-85e5-42e1-8b0d-90db3adaf3ef-kube-state-metrics-tls\") pod \"kube-state-metrics-56d9f944b4-9cctr\" (UID: \"39642d9c-85e5-42e1-8b0d-90db3adaf3ef\") " pod="openshift-monitoring/kube-state-metrics-56d9f944b4-9cctr" Jul 29 08:23:10.068523 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:23:10.068505 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/66322420-ef49-4c4c-99c5-1a19d67d4244-node-exporter-tls\") pod \"node-exporter-s44d8\" (UID: \"66322420-ef49-4c4c-99c5-1a19d67d4244\") " pod="openshift-monitoring/node-exporter-s44d8" Jul 29 08:23:10.068651 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:23:10.068631 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/39642d9c-85e5-42e1-8b0d-90db3adaf3ef-kube-state-metrics-tls\") pod \"kube-state-metrics-56d9f944b4-9cctr\" (UID: \"39642d9c-85e5-42e1-8b0d-90db3adaf3ef\") " pod="openshift-monitoring/kube-state-metrics-56d9f944b4-9cctr" Jul 29 08:23:10.074339 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:23:10.074311 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8rp9g_cfd1d8df-e6b7-4a67-81f2-5d83da541e20/konnectivity-agent/0.log" limit=1024 Jul 29 08:23:10.173668 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:23:10.173582 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8rp9g_cfd1d8df-e6b7-4a67-81f2-5d83da541e20/konnectivity-agent/0.log" limit=1024 Jul 29 08:23:10.278773 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:23:10.278733 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8rp9g_cfd1d8df-e6b7-4a67-81f2-5d83da541e20/konnectivity-agent/0.log" limit=1024 Jul 29 08:23:10.355395 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:23:10.355365 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/kube-state-metrics-56d9f944b4-9cctr" Jul 29 08:23:10.360471 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:23:10.360450 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-s44d8" Jul 29 08:23:10.395041 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:23:10.395009 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8rp9g_cfd1d8df-e6b7-4a67-81f2-5d83da541e20/konnectivity-agent/0.log" limit=1024 Jul 29 08:23:10.502693 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:23:10.502659 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-s44d8" event={"ID":"66322420-ef49-4c4c-99c5-1a19d67d4244","Type":"ContainerStarted","Data":"bb82405b704a2eb90cc6f97a723e40196dbdfec0a611784aae7d046bdadefa49"} Jul 29 08:23:10.537614 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:23:10.537399 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/kube-state-metrics-56d9f944b4-9cctr"] Jul 29 08:23:10.539689 ip-10-0-131-138 kubenswrapper[2563]: W0729 08:23:10.539659 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod39642d9c_85e5_42e1_8b0d_90db3adaf3ef.slice/crio-08985f53c6e1a189e36505476b638d4fd6eff5491e5dd74d4c85c44e3689a676 WatchSource:0}: Error finding container 08985f53c6e1a189e36505476b638d4fd6eff5491e5dd74d4c85c44e3689a676: Status 404 returned error can't find the container with id 08985f53c6e1a189e36505476b638d4fd6eff5491e5dd74d4c85c44e3689a676 Jul 29 08:23:10.669425 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:23:10.669390 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8rp9g_cfd1d8df-e6b7-4a67-81f2-5d83da541e20/konnectivity-agent/0.log" limit=1024 Jul 29 08:23:10.996409 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:23:10.996373 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8rp9g_cfd1d8df-e6b7-4a67-81f2-5d83da541e20/konnectivity-agent/0.log" limit=1024 Jul 29 08:23:11.088759 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:23:11.088721 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8rp9g_cfd1d8df-e6b7-4a67-81f2-5d83da541e20/konnectivity-agent/0.log" limit=1024 Jul 29 08:23:11.507463 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:23:11.507426 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-56d9f944b4-9cctr" event={"ID":"39642d9c-85e5-42e1-8b0d-90db3adaf3ef","Type":"ContainerStarted","Data":"08985f53c6e1a189e36505476b638d4fd6eff5491e5dd74d4c85c44e3689a676"} Jul 29 08:23:12.467911 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:23:12.467879 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8rp9g_cfd1d8df-e6b7-4a67-81f2-5d83da541e20/konnectivity-agent/0.log" limit=1024 Jul 29 08:23:12.516741 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:23:12.516670 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-s44d8" event={"ID":"66322420-ef49-4c4c-99c5-1a19d67d4244","Type":"ContainerStarted","Data":"c3a37c3b6ec99b82edc96024da002dc9b08f5fae61aa410e370915007f5d935d"} Jul 29 08:23:12.548126 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:23:12.548087 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8rp9g_cfd1d8df-e6b7-4a67-81f2-5d83da541e20/konnectivity-agent/0.log" limit=1024 Jul 29 08:23:12.625833 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:23:12.625802 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8rp9g_cfd1d8df-e6b7-4a67-81f2-5d83da541e20/konnectivity-agent/0.log" limit=1024 Jul 29 08:23:12.734015 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:23:12.733973 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8rp9g_cfd1d8df-e6b7-4a67-81f2-5d83da541e20/konnectivity-agent/0.log" limit=1024 Jul 29 08:23:13.521760 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:23:13.521727 2563 generic.go:356] "Generic (PLEG): container finished" podID="66322420-ef49-4c4c-99c5-1a19d67d4244" containerID="c3a37c3b6ec99b82edc96024da002dc9b08f5fae61aa410e370915007f5d935d" exitCode=0 Jul 29 08:23:13.522223 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:23:13.521804 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-s44d8" event={"ID":"66322420-ef49-4c4c-99c5-1a19d67d4244","Type":"ContainerDied","Data":"c3a37c3b6ec99b82edc96024da002dc9b08f5fae61aa410e370915007f5d935d"} Jul 29 08:23:13.524107 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:23:13.524081 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-56d9f944b4-9cctr" event={"ID":"39642d9c-85e5-42e1-8b0d-90db3adaf3ef","Type":"ContainerStarted","Data":"7eeaeeeacd8e3e8c639894f4f562a0c362e6905eefc41710ace27c48cd743b52"} Jul 29 08:23:13.524236 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:23:13.524130 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-56d9f944b4-9cctr" event={"ID":"39642d9c-85e5-42e1-8b0d-90db3adaf3ef","Type":"ContainerStarted","Data":"8d5fc0965481f149a6bd1ed0968f7dca34084f43b1251278fa285edbd6b13490"} Jul 29 08:23:13.524236 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:23:13.524158 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-56d9f944b4-9cctr" event={"ID":"39642d9c-85e5-42e1-8b0d-90db3adaf3ef","Type":"ContainerStarted","Data":"183eb0200343cd2c86351cbcc8b6766ac962c4361971c86316a677abf43fd01f"} Jul 29 08:23:13.547525 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:23:13.547494 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8rp9g_cfd1d8df-e6b7-4a67-81f2-5d83da541e20/konnectivity-agent/0.log" limit=1024 Jul 29 08:23:13.721697 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:23:13.721651 2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/kube-state-metrics-56d9f944b4-9cctr" podStartSLOduration=2.703806598 podStartE2EDuration="4.721634514s" totalImagesPullingTime="2.017827916s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 08:23:09 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 08:23:13.721330575 +0000 UTC m=+106.302619352" watchObservedRunningTime="2026-07-29 08:23:13.721634514 +0000 UTC m=+106.302923292" Jul 29 08:23:13.805112 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:23:13.805027 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8rp9g_cfd1d8df-e6b7-4a67-81f2-5d83da541e20/konnectivity-agent/0.log" limit=1024 Jul 29 08:23:14.034972 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:23:14.034907 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8rp9g_cfd1d8df-e6b7-4a67-81f2-5d83da541e20/konnectivity-agent/0.log" limit=1024 Jul 29 08:23:14.296216 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:23:14.296177 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8rp9g_cfd1d8df-e6b7-4a67-81f2-5d83da541e20/konnectivity-agent/0.log" limit=1024 Jul 29 08:23:14.410997 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:23:14.410958 2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/metrics-server-66c8d585d-h6fnr"] Jul 29 08:23:14.437308 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:23:14.437281 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/metrics-server-66c8d585d-h6fnr" Jul 29 08:23:14.442220 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:23:14.442194 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-client-certs\"" Jul 29 08:23:14.443531 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:23:14.443508 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-aeepsto6c348c\"" Jul 29 08:23:14.443531 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:23:14.443526 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-dockercfg-7ccvq\"" Jul 29 08:23:14.443701 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:23:14.443515 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-tls\"" Jul 29 08:23:14.444053 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:23:14.444036 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-server-audit-profiles\"" Jul 29 08:23:14.447721 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:23:14.446152 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kubelet-serving-ca-bundle\"" Jul 29 08:23:14.447721 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:23:14.447439 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/metrics-server-66c8d585d-h6fnr"] Jul 29 08:23:14.496622 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:23:14.496592 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wsjqc\" (UniqueName: \"kubernetes.io/projected/584f572a-103c-4b50-9c84-ba4e693f120d-kube-api-access-wsjqc\") pod \"metrics-server-66c8d585d-h6fnr\" (UID: \"584f572a-103c-4b50-9c84-ba4e693f120d\") " pod="openshift-monitoring/metrics-server-66c8d585d-h6fnr" Jul 29 08:23:14.496622 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:23:14.496630 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/584f572a-103c-4b50-9c84-ba4e693f120d-client-ca-bundle\") pod \"metrics-server-66c8d585d-h6fnr\" (UID: \"584f572a-103c-4b50-9c84-ba4e693f120d\") " pod="openshift-monitoring/metrics-server-66c8d585d-h6fnr" Jul 29 08:23:14.496827 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:23:14.496650 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/584f572a-103c-4b50-9c84-ba4e693f120d-audit-log\") pod \"metrics-server-66c8d585d-h6fnr\" (UID: \"584f572a-103c-4b50-9c84-ba4e693f120d\") " pod="openshift-monitoring/metrics-server-66c8d585d-h6fnr" Jul 29 08:23:14.496827 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:23:14.496678 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/584f572a-103c-4b50-9c84-ba4e693f120d-secret-metrics-server-tls\") pod \"metrics-server-66c8d585d-h6fnr\" (UID: \"584f572a-103c-4b50-9c84-ba4e693f120d\") " pod="openshift-monitoring/metrics-server-66c8d585d-h6fnr" Jul 29 08:23:14.496827 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:23:14.496719 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/584f572a-103c-4b50-9c84-ba4e693f120d-secret-metrics-server-client-certs\") pod \"metrics-server-66c8d585d-h6fnr\" (UID: \"584f572a-103c-4b50-9c84-ba4e693f120d\") " pod="openshift-monitoring/metrics-server-66c8d585d-h6fnr" Jul 29 08:23:14.496827 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:23:14.496782 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/584f572a-103c-4b50-9c84-ba4e693f120d-metrics-server-audit-profiles\") pod \"metrics-server-66c8d585d-h6fnr\" (UID: \"584f572a-103c-4b50-9c84-ba4e693f120d\") " pod="openshift-monitoring/metrics-server-66c8d585d-h6fnr" Jul 29 08:23:14.496827 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:23:14.496806 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/584f572a-103c-4b50-9c84-ba4e693f120d-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-66c8d585d-h6fnr\" (UID: \"584f572a-103c-4b50-9c84-ba4e693f120d\") " pod="openshift-monitoring/metrics-server-66c8d585d-h6fnr" Jul 29 08:23:14.497052 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:23:14.496979 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8rp9g_cfd1d8df-e6b7-4a67-81f2-5d83da541e20/konnectivity-agent/0.log" limit=1024 Jul 29 08:23:14.530545 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:23:14.530507 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-s44d8" event={"ID":"66322420-ef49-4c4c-99c5-1a19d67d4244","Type":"ContainerStarted","Data":"4fcf0da9c21e0d1d03d8dd55105442579e51ab5abc46de3ef8d6bbc5c546f096"} Jul 29 08:23:14.530888 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:23:14.530565 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-s44d8" event={"ID":"66322420-ef49-4c4c-99c5-1a19d67d4244","Type":"ContainerStarted","Data":"7bcd1d7785fe6e95e6a174e60805cd826d12e733867f35bbdb13aa9a77a2d0d3"} Jul 29 08:23:14.577070 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:23:14.576966 2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-s44d8" podStartSLOduration=3.591161525 podStartE2EDuration="5.576947019s" totalImagesPullingTime="1.887815956s" totalInitContainerRuntime="97.969538ms" isStatefulPod=true podCreationTimestamp="2026-07-29 08:23:09 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 08:23:14.575599368 +0000 UTC m=+107.156888147" watchObservedRunningTime="2026-07-29 08:23:14.576947019 +0000 UTC m=+107.158235803" Jul 29 08:23:14.597829 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:23:14.597795 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/584f572a-103c-4b50-9c84-ba4e693f120d-secret-metrics-server-client-certs\") pod \"metrics-server-66c8d585d-h6fnr\" (UID: \"584f572a-103c-4b50-9c84-ba4e693f120d\") " pod="openshift-monitoring/metrics-server-66c8d585d-h6fnr" Jul 29 08:23:14.598247 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:23:14.598059 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/584f572a-103c-4b50-9c84-ba4e693f120d-metrics-server-audit-profiles\") pod \"metrics-server-66c8d585d-h6fnr\" (UID: \"584f572a-103c-4b50-9c84-ba4e693f120d\") " pod="openshift-monitoring/metrics-server-66c8d585d-h6fnr" Jul 29 08:23:14.598247 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:23:14.598169 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/584f572a-103c-4b50-9c84-ba4e693f120d-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-66c8d585d-h6fnr\" (UID: \"584f572a-103c-4b50-9c84-ba4e693f120d\") " pod="openshift-monitoring/metrics-server-66c8d585d-h6fnr" Jul 29 08:23:14.598247 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:23:14.598221 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wsjqc\" (UniqueName: \"kubernetes.io/projected/584f572a-103c-4b50-9c84-ba4e693f120d-kube-api-access-wsjqc\") pod \"metrics-server-66c8d585d-h6fnr\" (UID: \"584f572a-103c-4b50-9c84-ba4e693f120d\") " pod="openshift-monitoring/metrics-server-66c8d585d-h6fnr" Jul 29 08:23:14.598247 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:23:14.598276 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/584f572a-103c-4b50-9c84-ba4e693f120d-client-ca-bundle\") pod \"metrics-server-66c8d585d-h6fnr\" (UID: \"584f572a-103c-4b50-9c84-ba4e693f120d\") " pod="openshift-monitoring/metrics-server-66c8d585d-h6fnr" Jul 29 08:23:14.598617 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:23:14.598329 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/584f572a-103c-4b50-9c84-ba4e693f120d-audit-log\") pod \"metrics-server-66c8d585d-h6fnr\" (UID: \"584f572a-103c-4b50-9c84-ba4e693f120d\") " pod="openshift-monitoring/metrics-server-66c8d585d-h6fnr" Jul 29 08:23:14.598617 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:23:14.598357 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/584f572a-103c-4b50-9c84-ba4e693f120d-secret-metrics-server-tls\") pod \"metrics-server-66c8d585d-h6fnr\" (UID: \"584f572a-103c-4b50-9c84-ba4e693f120d\") " pod="openshift-monitoring/metrics-server-66c8d585d-h6fnr" Jul 29 08:23:14.598727 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:23:14.598658 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/584f572a-103c-4b50-9c84-ba4e693f120d-audit-log\") pod \"metrics-server-66c8d585d-h6fnr\" (UID: \"584f572a-103c-4b50-9c84-ba4e693f120d\") " pod="openshift-monitoring/metrics-server-66c8d585d-h6fnr" Jul 29 08:23:14.598906 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:23:14.598850 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/584f572a-103c-4b50-9c84-ba4e693f120d-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-66c8d585d-h6fnr\" (UID: \"584f572a-103c-4b50-9c84-ba4e693f120d\") " pod="openshift-monitoring/metrics-server-66c8d585d-h6fnr" Jul 29 08:23:14.599361 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:23:14.599333 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/584f572a-103c-4b50-9c84-ba4e693f120d-metrics-server-audit-profiles\") pod \"metrics-server-66c8d585d-h6fnr\" (UID: \"584f572a-103c-4b50-9c84-ba4e693f120d\") " pod="openshift-monitoring/metrics-server-66c8d585d-h6fnr" Jul 29 08:23:14.600697 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:23:14.600673 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/584f572a-103c-4b50-9c84-ba4e693f120d-secret-metrics-server-client-certs\") pod \"metrics-server-66c8d585d-h6fnr\" (UID: \"584f572a-103c-4b50-9c84-ba4e693f120d\") " pod="openshift-monitoring/metrics-server-66c8d585d-h6fnr" Jul 29 08:23:14.600791 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:23:14.600720 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/584f572a-103c-4b50-9c84-ba4e693f120d-client-ca-bundle\") pod \"metrics-server-66c8d585d-h6fnr\" (UID: \"584f572a-103c-4b50-9c84-ba4e693f120d\") " pod="openshift-monitoring/metrics-server-66c8d585d-h6fnr" Jul 29 08:23:14.600990 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:23:14.600971 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/584f572a-103c-4b50-9c84-ba4e693f120d-secret-metrics-server-tls\") pod \"metrics-server-66c8d585d-h6fnr\" (UID: \"584f572a-103c-4b50-9c84-ba4e693f120d\") " pod="openshift-monitoring/metrics-server-66c8d585d-h6fnr" Jul 29 08:23:14.612783 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:23:14.612761 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8rp9g_cfd1d8df-e6b7-4a67-81f2-5d83da541e20/konnectivity-agent/0.log" limit=1024 Jul 29 08:23:14.616299 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:23:14.616283 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wsjqc\" (UniqueName: \"kubernetes.io/projected/584f572a-103c-4b50-9c84-ba4e693f120d-kube-api-access-wsjqc\") pod \"metrics-server-66c8d585d-h6fnr\" (UID: \"584f572a-103c-4b50-9c84-ba4e693f120d\") " pod="openshift-monitoring/metrics-server-66c8d585d-h6fnr" Jul 29 08:23:14.752063 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:23:14.752029 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/metrics-server-66c8d585d-h6fnr" Jul 29 08:23:14.808767 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:23:14.808725 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8rp9g_cfd1d8df-e6b7-4a67-81f2-5d83da541e20/konnectivity-agent/0.log" limit=1024 Jul 29 08:23:14.946938 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:23:14.946910 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/metrics-server-66c8d585d-h6fnr"] Jul 29 08:23:14.949811 ip-10-0-131-138 kubenswrapper[2563]: W0729 08:23:14.949788 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod584f572a_103c_4b50_9c84_ba4e693f120d.slice/crio-68642451490d26821d9960cbe276931a9132775e7b900d7cd95d568657f7e708 WatchSource:0}: Error finding container 68642451490d26821d9960cbe276931a9132775e7b900d7cd95d568657f7e708: Status 404 returned error can't find the container with id 68642451490d26821d9960cbe276931a9132775e7b900d7cd95d568657f7e708 Jul 29 08:23:14.955614 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:23:14.955580 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8rp9g_cfd1d8df-e6b7-4a67-81f2-5d83da541e20/konnectivity-agent/0.log" limit=1024 Jul 29 08:23:15.426132 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:23:15.426096 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8rp9g_cfd1d8df-e6b7-4a67-81f2-5d83da541e20/konnectivity-agent/0.log" limit=1024 Jul 29 08:23:15.536467 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:23:15.536428 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/metrics-server-66c8d585d-h6fnr" event={"ID":"584f572a-103c-4b50-9c84-ba4e693f120d","Type":"ContainerStarted","Data":"68642451490d26821d9960cbe276931a9132775e7b900d7cd95d568657f7e708"} Jul 29 08:23:15.552801 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:23:15.552760 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8rp9g_cfd1d8df-e6b7-4a67-81f2-5d83da541e20/konnectivity-agent/0.log" limit=1024 Jul 29 08:23:16.041490 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:23:16.041454 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8rp9g_cfd1d8df-e6b7-4a67-81f2-5d83da541e20/konnectivity-agent/0.log" limit=1024 Jul 29 08:23:17.548755 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:23:17.548721 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/metrics-server-66c8d585d-h6fnr" event={"ID":"584f572a-103c-4b50-9c84-ba4e693f120d","Type":"ContainerStarted","Data":"a7c573d39757d44421baed7cb14a369ba6b1b1cbd23aa3355821e9258531cbd9"} Jul 29 08:23:17.660800 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:23:17.660762 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8rp9g_cfd1d8df-e6b7-4a67-81f2-5d83da541e20/konnectivity-agent/0.log" limit=1024 Jul 29 08:23:17.807701 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:23:17.807611 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8rp9g_cfd1d8df-e6b7-4a67-81f2-5d83da541e20/konnectivity-agent/0.log" limit=1024 Jul 29 08:23:19.241970 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:23:19.241932 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8rp9g_cfd1d8df-e6b7-4a67-81f2-5d83da541e20/konnectivity-agent/0.log" limit=1024 Jul 29 08:23:19.365607 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:23:19.365570 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8rp9g_cfd1d8df-e6b7-4a67-81f2-5d83da541e20/konnectivity-agent/0.log" limit=1024 Jul 29 08:23:21.012065 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:23:21.012033 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8rp9g_cfd1d8df-e6b7-4a67-81f2-5d83da541e20/konnectivity-agent/0.log" limit=1024 Jul 29 08:23:21.110088 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:23:21.110057 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8rp9g_cfd1d8df-e6b7-4a67-81f2-5d83da541e20/konnectivity-agent/0.log" limit=1024 Jul 29 08:23:21.285217 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:23:21.285125 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8rp9g_cfd1d8df-e6b7-4a67-81f2-5d83da541e20/konnectivity-agent/0.log" limit=1024 Jul 29 08:23:21.404082 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:23:21.404052 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8rp9g_cfd1d8df-e6b7-4a67-81f2-5d83da541e20/konnectivity-agent/0.log" limit=1024 Jul 29 08:23:21.533020 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:23:21.532980 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8rp9g_cfd1d8df-e6b7-4a67-81f2-5d83da541e20/konnectivity-agent/0.log" limit=1024 Jul 29 08:23:21.623679 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:23:21.623591 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8rp9g_cfd1d8df-e6b7-4a67-81f2-5d83da541e20/konnectivity-agent/0.log" limit=1024 Jul 29 08:23:25.409945 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:23:25.409911 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8rp9g_cfd1d8df-e6b7-4a67-81f2-5d83da541e20/konnectivity-agent/0.log" limit=1024 Jul 29 08:23:25.517193 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:23:25.517155 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8rp9g_cfd1d8df-e6b7-4a67-81f2-5d83da541e20/konnectivity-agent/0.log" limit=1024 Jul 29 08:23:28.699750 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:23:28.699702 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8rp9g_cfd1d8df-e6b7-4a67-81f2-5d83da541e20/konnectivity-agent/0.log" limit=1024 Jul 29 08:23:28.870482 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:23:28.870429 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8rp9g_cfd1d8df-e6b7-4a67-81f2-5d83da541e20/konnectivity-agent/0.log" limit=1024 Jul 29 08:23:28.959380 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:23:28.959349 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8rp9g_cfd1d8df-e6b7-4a67-81f2-5d83da541e20/konnectivity-agent/0.log" limit=1024 Jul 29 08:23:32.603688 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:23:32.603654 2563 generic.go:356] "Generic (PLEG): container finished" podID="791f1046-8a1f-41c4-b7a6-70862705aad6" containerID="b67755562dc6fc88e5252d84491e15c8d5e3a7db12930c51046b3316f8c81c71" exitCode=0 Jul 29 08:23:32.604063 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:23:32.603731 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-5885fdb759-84zrp" event={"ID":"791f1046-8a1f-41c4-b7a6-70862705aad6","Type":"ContainerDied","Data":"b67755562dc6fc88e5252d84491e15c8d5e3a7db12930c51046b3316f8c81c71"} Jul 29 08:23:32.604129 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:23:32.604113 2563 scope.go:117] "RemoveContainer" containerID="b67755562dc6fc88e5252d84491e15c8d5e3a7db12930c51046b3316f8c81c71" Jul 29 08:23:32.637028 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:23:32.636977 2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/metrics-server-66c8d585d-h6fnr" podStartSLOduration=16.830143852 podStartE2EDuration="18.636960344s" totalImagesPullingTime="1.806816492s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 08:23:14 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 08:23:17.595446987 +0000 UTC m=+110.176735765" watchObservedRunningTime="2026-07-29 08:23:32.636960344 +0000 UTC m=+125.218249121" Jul 29 08:23:33.608863 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:23:33.608827 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-5885fdb759-84zrp" event={"ID":"791f1046-8a1f-41c4-b7a6-70862705aad6","Type":"ContainerStarted","Data":"607fc85c4b412cfd449291c2cce5552b0b71bea745046101456bc599fbccf07a"} Jul 29 08:23:34.752656 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:23:34.752622 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/metrics-server-66c8d585d-h6fnr" Jul 29 08:23:34.753131 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:23:34.752686 2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/metrics-server-66c8d585d-h6fnr" Jul 29 08:23:35.981198 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:23:35.981154 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8rp9g_cfd1d8df-e6b7-4a67-81f2-5d83da541e20/konnectivity-agent/0.log" limit=1024 Jul 29 08:23:36.071091 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:23:36.071060 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8rp9g_cfd1d8df-e6b7-4a67-81f2-5d83da541e20/konnectivity-agent/0.log" limit=1024 Jul 29 08:23:54.757451 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:23:54.757420 2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/metrics-server-66c8d585d-h6fnr" Jul 29 08:23:54.761420 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:23:54.761398 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/metrics-server-66c8d585d-h6fnr" Jul 29 08:24:28.877908 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:24:28.877824 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8rp9g_cfd1d8df-e6b7-4a67-81f2-5d83da541e20/konnectivity-agent/0.log" limit=1024 Jul 29 08:24:29.211645 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:24:29.211610 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8rp9g_cfd1d8df-e6b7-4a67-81f2-5d83da541e20/konnectivity-agent/0.log" limit=1024 Jul 29 08:24:39.175373 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:24:39.175334 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8rp9g_cfd1d8df-e6b7-4a67-81f2-5d83da541e20/konnectivity-agent/0.log" limit=1024 Jul 29 08:24:49.706806 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:24:49.706766 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8rp9g_cfd1d8df-e6b7-4a67-81f2-5d83da541e20/konnectivity-agent/0.log" limit=1024 Jul 29 08:24:49.808400 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:24:49.808361 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8rp9g_cfd1d8df-e6b7-4a67-81f2-5d83da541e20/konnectivity-agent/0.log" limit=1024 Jul 29 08:24:49.879880 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:24:49.879839 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8rp9g_cfd1d8df-e6b7-4a67-81f2-5d83da541e20/konnectivity-agent/0.log" limit=1024 Jul 29 08:25:03.503712 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:25:03.503673 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8rp9g_cfd1d8df-e6b7-4a67-81f2-5d83da541e20/konnectivity-agent/0.log" limit=1024 Jul 29 08:25:03.559178 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:25:03.559143 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8rp9g_cfd1d8df-e6b7-4a67-81f2-5d83da541e20/konnectivity-agent/0.log" limit=1024 Jul 29 08:25:44.179790 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:25:44.179743 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8rp9g_cfd1d8df-e6b7-4a67-81f2-5d83da541e20/konnectivity-agent/0.log" limit=1024 Jul 29 08:25:44.262296 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:25:44.262234 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8rp9g_cfd1d8df-e6b7-4a67-81f2-5d83da541e20/konnectivity-agent/0.log" limit=1024 Jul 29 08:25:44.380206 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:25:44.380158 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8rp9g_cfd1d8df-e6b7-4a67-81f2-5d83da541e20/konnectivity-agent/0.log" limit=1024 Jul 29 08:25:49.714132 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:25:49.714059 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8rp9g_cfd1d8df-e6b7-4a67-81f2-5d83da541e20/konnectivity-agent/0.log" limit=1024 Jul 29 08:25:54.967320 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:25:54.967283 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8rp9g_cfd1d8df-e6b7-4a67-81f2-5d83da541e20/konnectivity-agent/0.log" limit=1024 Jul 29 08:25:55.079383 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:25:55.079347 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8rp9g_cfd1d8df-e6b7-4a67-81f2-5d83da541e20/konnectivity-agent/0.log" limit=1024 Jul 29 08:25:55.138892 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:25:55.138864 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8rp9g_cfd1d8df-e6b7-4a67-81f2-5d83da541e20/konnectivity-agent/0.log" limit=1024 Jul 29 08:25:56.072190 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:25:56.072138 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8rp9g_cfd1d8df-e6b7-4a67-81f2-5d83da541e20/konnectivity-agent/0.log" limit=1024 Jul 29 08:25:56.174170 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:25:56.174125 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8rp9g_cfd1d8df-e6b7-4a67-81f2-5d83da541e20/konnectivity-agent/0.log" limit=1024 Jul 29 08:26:05.936181 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:26:05.936143 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8rp9g_cfd1d8df-e6b7-4a67-81f2-5d83da541e20/konnectivity-agent/0.log" limit=1024 Jul 29 08:26:06.036503 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:26:06.036469 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8rp9g_cfd1d8df-e6b7-4a67-81f2-5d83da541e20/konnectivity-agent/0.log" limit=1024 Jul 29 08:26:06.111044 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:26:06.111014 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8rp9g_cfd1d8df-e6b7-4a67-81f2-5d83da541e20/konnectivity-agent/0.log" limit=1024 Jul 29 08:26:24.006132 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:26:24.006096 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8rp9g_cfd1d8df-e6b7-4a67-81f2-5d83da541e20/konnectivity-agent/0.log" limit=1024 Jul 29 08:26:27.834008 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:26:27.833980 2563 kubelet.go:1648] "Image garbage collection succeeded" Jul 29 08:27:51.649666 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:27:51.649631 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8rp9g_cfd1d8df-e6b7-4a67-81f2-5d83da541e20/konnectivity-agent/0.log" limit=1024 Jul 29 08:28:02.331541 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:28:02.331505 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8rp9g_cfd1d8df-e6b7-4a67-81f2-5d83da541e20/konnectivity-agent/0.log" limit=1024 Jul 29 08:28:02.432156 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:28:02.432122 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8rp9g_cfd1d8df-e6b7-4a67-81f2-5d83da541e20/konnectivity-agent/0.log" limit=1024 Jul 29 08:28:03.113531 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:28:03.113497 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8rp9g_cfd1d8df-e6b7-4a67-81f2-5d83da541e20/konnectivity-agent/0.log" limit=1024 Jul 29 08:28:39.842469 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:28:39.842430 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8rp9g_cfd1d8df-e6b7-4a67-81f2-5d83da541e20/konnectivity-agent/0.log" limit=1024 Jul 29 08:28:39.942778 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:28:39.942741 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8rp9g_cfd1d8df-e6b7-4a67-81f2-5d83da541e20/konnectivity-agent/0.log" limit=1024 Jul 29 08:28:40.180678 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:28:40.180597 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8rp9g_cfd1d8df-e6b7-4a67-81f2-5d83da541e20/konnectivity-agent/0.log" limit=1024 Jul 29 08:28:40.282143 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:28:40.282107 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8rp9g_cfd1d8df-e6b7-4a67-81f2-5d83da541e20/konnectivity-agent/0.log" limit=1024 Jul 29 08:28:40.318078 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:28:40.318047 2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["test-ns-feast/feast-simple-feast-setup-57d7459654-znh42"] Jul 29 08:28:40.323826 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:28:40.323797 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/feast-simple-feast-setup-57d7459654-znh42" Jul 29 08:28:40.327459 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:28:40.327107 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-feast\"/\"feast-simple-feast-setup-ui-tls\"" Jul 29 08:28:40.327459 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:28:40.327192 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"test-ns-feast\"/\"kube-root-ca.crt\"" Jul 29 08:28:40.328327 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:28:40.328303 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-feast\"/\"feast-simple-feast-setup-dockercfg-wlmx8\"" Jul 29 08:28:40.328422 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:28:40.328390 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-feast\"/\"feast-simple-feast-setup-registry-tls\"" Jul 29 08:28:40.328596 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:28:40.328582 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-feast\"/\"feast-simple-feast-setup-online-tls\"" Jul 29 08:28:40.328692 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:28:40.328667 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"test-ns-feast\"/\"feast-simple-feast-setup-intra-comm\"" Jul 29 08:28:40.328809 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:28:40.328795 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-feast\"/\"feast-simple-feast-setup-offline-tls\"" Jul 29 08:28:40.328848 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:28:40.328836 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"test-ns-feast\"/\"openshift-service-ca.crt\"" Jul 29 08:28:40.337925 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:28:40.337902 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["test-ns-feast/feast-simple-feast-setup-57d7459654-znh42"] Jul 29 08:28:40.374599 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:28:40.374568 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8rp9g_cfd1d8df-e6b7-4a67-81f2-5d83da541e20/konnectivity-agent/0.log" limit=1024 Jul 29 08:28:40.430510 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:28:40.430472 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8rp9g_cfd1d8df-e6b7-4a67-81f2-5d83da541e20/konnectivity-agent/0.log" limit=1024 Jul 29 08:28:40.446564 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:28:40.446483 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/23f9dea4-c464-4cca-8568-c9934ca89211-offline-tls\") pod \"feast-simple-feast-setup-57d7459654-znh42\" (UID: \"23f9dea4-c464-4cca-8568-c9934ca89211\") " pod="test-ns-feast/feast-simple-feast-setup-57d7459654-znh42" Jul 29 08:28:40.446564 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:28:40.446515 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/23f9dea4-c464-4cca-8568-c9934ca89211-feast-data\") pod \"feast-simple-feast-setup-57d7459654-znh42\" (UID: \"23f9dea4-c464-4cca-8568-c9934ca89211\") " pod="test-ns-feast/feast-simple-feast-setup-57d7459654-znh42" Jul 29 08:28:40.446564 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:28:40.446537 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-td45w\" (UniqueName: \"kubernetes.io/projected/23f9dea4-c464-4cca-8568-c9934ca89211-kube-api-access-td45w\") pod \"feast-simple-feast-setup-57d7459654-znh42\" (UID: \"23f9dea4-c464-4cca-8568-c9934ca89211\") " pod="test-ns-feast/feast-simple-feast-setup-57d7459654-znh42" Jul 29 08:28:40.446778 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:28:40.446604 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/23f9dea4-c464-4cca-8568-c9934ca89211-registry-tls\") pod \"feast-simple-feast-setup-57d7459654-znh42\" (UID: \"23f9dea4-c464-4cca-8568-c9934ca89211\") " pod="test-ns-feast/feast-simple-feast-setup-57d7459654-znh42" Jul 29 08:28:40.446778 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:28:40.446646 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/23f9dea4-c464-4cca-8568-c9934ca89211-ui-tls\") pod \"feast-simple-feast-setup-57d7459654-znh42\" (UID: \"23f9dea4-c464-4cca-8568-c9934ca89211\") " pod="test-ns-feast/feast-simple-feast-setup-57d7459654-znh42" Jul 29 08:28:40.446778 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:28:40.446673 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/23f9dea4-c464-4cca-8568-c9934ca89211-online-tls\") pod \"feast-simple-feast-setup-57d7459654-znh42\" (UID: \"23f9dea4-c464-4cca-8568-c9934ca89211\") " pod="test-ns-feast/feast-simple-feast-setup-57d7459654-znh42" Jul 29 08:28:40.530740 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:28:40.530705 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8rp9g_cfd1d8df-e6b7-4a67-81f2-5d83da541e20/konnectivity-agent/0.log" limit=1024 Jul 29 08:28:40.547797 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:28:40.547769 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/23f9dea4-c464-4cca-8568-c9934ca89211-registry-tls\") pod \"feast-simple-feast-setup-57d7459654-znh42\" (UID: \"23f9dea4-c464-4cca-8568-c9934ca89211\") " pod="test-ns-feast/feast-simple-feast-setup-57d7459654-znh42" Jul 29 08:28:40.547887 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:28:40.547807 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/23f9dea4-c464-4cca-8568-c9934ca89211-ui-tls\") pod \"feast-simple-feast-setup-57d7459654-znh42\" (UID: \"23f9dea4-c464-4cca-8568-c9934ca89211\") " pod="test-ns-feast/feast-simple-feast-setup-57d7459654-znh42" Jul 29 08:28:40.547887 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:28:40.547836 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/23f9dea4-c464-4cca-8568-c9934ca89211-online-tls\") pod \"feast-simple-feast-setup-57d7459654-znh42\" (UID: \"23f9dea4-c464-4cca-8568-c9934ca89211\") " pod="test-ns-feast/feast-simple-feast-setup-57d7459654-znh42" Jul 29 08:28:40.547960 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:28:40.547891 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/23f9dea4-c464-4cca-8568-c9934ca89211-offline-tls\") pod \"feast-simple-feast-setup-57d7459654-znh42\" (UID: \"23f9dea4-c464-4cca-8568-c9934ca89211\") " pod="test-ns-feast/feast-simple-feast-setup-57d7459654-znh42" Jul 29 08:28:40.547960 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:28:40.547913 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/23f9dea4-c464-4cca-8568-c9934ca89211-feast-data\") pod \"feast-simple-feast-setup-57d7459654-znh42\" (UID: \"23f9dea4-c464-4cca-8568-c9934ca89211\") " pod="test-ns-feast/feast-simple-feast-setup-57d7459654-znh42" Jul 29 08:28:40.547960 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:28:40.547929 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-td45w\" (UniqueName: \"kubernetes.io/projected/23f9dea4-c464-4cca-8568-c9934ca89211-kube-api-access-td45w\") pod \"feast-simple-feast-setup-57d7459654-znh42\" (UID: \"23f9dea4-c464-4cca-8568-c9934ca89211\") " pod="test-ns-feast/feast-simple-feast-setup-57d7459654-znh42" Jul 29 08:28:40.548392 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:28:40.548362 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/23f9dea4-c464-4cca-8568-c9934ca89211-feast-data\") pod \"feast-simple-feast-setup-57d7459654-znh42\" (UID: \"23f9dea4-c464-4cca-8568-c9934ca89211\") " pod="test-ns-feast/feast-simple-feast-setup-57d7459654-znh42" Jul 29 08:28:40.550555 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:28:40.550524 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/23f9dea4-c464-4cca-8568-c9934ca89211-offline-tls\") pod \"feast-simple-feast-setup-57d7459654-znh42\" (UID: \"23f9dea4-c464-4cca-8568-c9934ca89211\") " pod="test-ns-feast/feast-simple-feast-setup-57d7459654-znh42" Jul 29 08:28:40.550668 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:28:40.550554 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/23f9dea4-c464-4cca-8568-c9934ca89211-ui-tls\") pod \"feast-simple-feast-setup-57d7459654-znh42\" (UID: \"23f9dea4-c464-4cca-8568-c9934ca89211\") " pod="test-ns-feast/feast-simple-feast-setup-57d7459654-znh42" Jul 29 08:28:40.550668 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:28:40.550558 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/23f9dea4-c464-4cca-8568-c9934ca89211-online-tls\") pod \"feast-simple-feast-setup-57d7459654-znh42\" (UID: \"23f9dea4-c464-4cca-8568-c9934ca89211\") " pod="test-ns-feast/feast-simple-feast-setup-57d7459654-znh42" Jul 29 08:28:40.550668 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:28:40.550651 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/23f9dea4-c464-4cca-8568-c9934ca89211-registry-tls\") pod \"feast-simple-feast-setup-57d7459654-znh42\" (UID: \"23f9dea4-c464-4cca-8568-c9934ca89211\") " pod="test-ns-feast/feast-simple-feast-setup-57d7459654-znh42" Jul 29 08:28:40.556713 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:28:40.556689 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-td45w\" (UniqueName: \"kubernetes.io/projected/23f9dea4-c464-4cca-8568-c9934ca89211-kube-api-access-td45w\") pod \"feast-simple-feast-setup-57d7459654-znh42\" (UID: \"23f9dea4-c464-4cca-8568-c9934ca89211\") " pod="test-ns-feast/feast-simple-feast-setup-57d7459654-znh42" Jul 29 08:28:40.630575 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:28:40.630534 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8rp9g_cfd1d8df-e6b7-4a67-81f2-5d83da541e20/konnectivity-agent/0.log" limit=1024 Jul 29 08:28:40.634474 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:28:40.634451 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/feast-simple-feast-setup-57d7459654-znh42" Jul 29 08:28:40.783371 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:28:40.783334 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["test-ns-feast/feast-simple-feast-setup-57d7459654-znh42"] Jul 29 08:28:40.787510 ip-10-0-131-138 kubenswrapper[2563]: W0729 08:28:40.787487 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod23f9dea4_c464_4cca_8568_c9934ca89211.slice/crio-4239e10642466a1cef6edaa732d560d8992a089b02fe49daaa06ca8f0b7a7041 WatchSource:0}: Error finding container 4239e10642466a1cef6edaa732d560d8992a089b02fe49daaa06ca8f0b7a7041: Status 404 returned error can't find the container with id 4239e10642466a1cef6edaa732d560d8992a089b02fe49daaa06ca8f0b7a7041 Jul 29 08:28:40.789394 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:28:40.789376 2563 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 29 08:28:41.697963 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:28:41.697923 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-57d7459654-znh42" event={"ID":"23f9dea4-c464-4cca-8568-c9934ca89211","Type":"ContainerStarted","Data":"4239e10642466a1cef6edaa732d560d8992a089b02fe49daaa06ca8f0b7a7041"} Jul 29 08:28:51.420070 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:28:51.419977 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8rp9g_cfd1d8df-e6b7-4a67-81f2-5d83da541e20/konnectivity-agent/0.log" limit=1024 Jul 29 08:29:08.818925 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:29:08.818887 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-57d7459654-znh42" event={"ID":"23f9dea4-c464-4cca-8568-c9934ca89211","Type":"ContainerStarted","Data":"e487d6ece51ea1839b8f3e7927f0ba70c6f83fed53ae5c79d10f505e1edb07a5"} Jul 29 08:29:10.470434 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:29:10.470405 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8rp9g_cfd1d8df-e6b7-4a67-81f2-5d83da541e20/konnectivity-agent/0.log" limit=1024 Jul 29 08:29:10.566894 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:29:10.566858 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8rp9g_cfd1d8df-e6b7-4a67-81f2-5d83da541e20/konnectivity-agent/0.log" limit=1024 Jul 29 08:29:12.835010 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:29:12.834934 2563 generic.go:356] "Generic (PLEG): container finished" podID="23f9dea4-c464-4cca-8568-c9934ca89211" containerID="e487d6ece51ea1839b8f3e7927f0ba70c6f83fed53ae5c79d10f505e1edb07a5" exitCode=0 Jul 29 08:29:12.835398 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:29:12.835006 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-57d7459654-znh42" event={"ID":"23f9dea4-c464-4cca-8568-c9934ca89211","Type":"ContainerDied","Data":"e487d6ece51ea1839b8f3e7927f0ba70c6f83fed53ae5c79d10f505e1edb07a5"} Jul 29 08:29:13.840946 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:29:13.840916 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-57d7459654-znh42" event={"ID":"23f9dea4-c464-4cca-8568-c9934ca89211","Type":"ContainerStarted","Data":"a657ad1567cc2753c77a42503ff91b69c93b5272437767f381e0003e9e34097c"} Jul 29 08:29:18.861910 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:29:18.861801 2563 generic.go:356] "Generic (PLEG): container finished" podID="23f9dea4-c464-4cca-8568-c9934ca89211" containerID="a657ad1567cc2753c77a42503ff91b69c93b5272437767f381e0003e9e34097c" exitCode=0 Jul 29 08:29:18.861910 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:29:18.861884 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-57d7459654-znh42" event={"ID":"23f9dea4-c464-4cca-8568-c9934ca89211","Type":"ContainerDied","Data":"a657ad1567cc2753c77a42503ff91b69c93b5272437767f381e0003e9e34097c"} Jul 29 08:29:19.871601 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:29:19.871552 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-57d7459654-znh42" event={"ID":"23f9dea4-c464-4cca-8568-c9934ca89211","Type":"ContainerStarted","Data":"cf710d226934ad5427b130a0accf8c9eaa1af371a11c74bd50e06e8fd409091a"} Jul 29 08:29:19.872075 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:29:19.871618 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-57d7459654-znh42" event={"ID":"23f9dea4-c464-4cca-8568-c9934ca89211","Type":"ContainerStarted","Data":"01d85ec9d66eb753e86bb82dcadd63ba603b1a875541a3e7d05194895fe1e311"} Jul 29 08:29:19.872075 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:29:19.871639 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-57d7459654-znh42" event={"ID":"23f9dea4-c464-4cca-8568-c9934ca89211","Type":"ContainerStarted","Data":"9688ba3fd4a54b7581da999201cff27f4df6274de7b6eecd0429a1808e587532"} Jul 29 08:29:19.872075 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:29:19.871656 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-57d7459654-znh42" event={"ID":"23f9dea4-c464-4cca-8568-c9934ca89211","Type":"ContainerStarted","Data":"e4146548f670e4da5b1bd66a73997e6a89177c25c63d458e92898fe9ab34b832"} Jul 29 08:29:19.906567 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:29:19.906500 2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="test-ns-feast/feast-simple-feast-setup-57d7459654-znh42" podStartSLOduration=12.873682409 podStartE2EDuration="39.90647684s" totalImagesPullingTime="27.032794431s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 08:28:40 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 08:29:19.905024404 +0000 UTC m=+472.486313196" watchObservedRunningTime="2026-07-29 08:29:19.90647684 +0000 UTC m=+472.487765620" Jul 29 08:29:20.634680 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:29:20.634635 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-57d7459654-znh42" Jul 29 08:29:20.634877 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:29:20.634710 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-57d7459654-znh42" Jul 29 08:29:20.634877 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:29:20.634739 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-57d7459654-znh42" Jul 29 08:29:20.634877 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:29:20.634758 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-57d7459654-znh42" Jul 29 08:29:22.635641 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:29:22.635595 2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-feast/feast-simple-feast-setup-57d7459654-znh42" Jul 29 08:29:22.636137 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:29:22.635659 2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-feast/feast-simple-feast-setup-57d7459654-znh42" Jul 29 08:29:22.636137 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:29:22.635839 2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-feast/feast-simple-feast-setup-57d7459654-znh42" Jul 29 08:29:22.636137 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:29:22.635868 2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-feast/feast-simple-feast-setup-57d7459654-znh42" Jul 29 08:29:22.638031 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:29:22.637974 2563 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-57d7459654-znh42" podUID="23f9dea4-c464-4cca-8568-c9934ca89211" containerName="ui" probeResult="failure" output="dial tcp 10.134.0.19:8443: connect: connection refused" Jul 29 08:29:22.638031 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:29:22.638013 2563 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-57d7459654-znh42" podUID="23f9dea4-c464-4cca-8568-c9934ca89211" containerName="registry" probeResult="failure" output="dial tcp 10.134.0.19:6571: connect: connection refused" Jul 29 08:29:22.638273 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:29:22.638024 2563 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-57d7459654-znh42" podUID="23f9dea4-c464-4cca-8568-c9934ca89211" containerName="online" probeResult="failure" output="Get \"https://10.134.0.19:6567/health\": dial tcp 10.134.0.19:6567: connect: connection refused" Jul 29 08:29:22.638273 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:29:22.638090 2563 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-57d7459654-znh42" podUID="23f9dea4-c464-4cca-8568-c9934ca89211" containerName="offline" probeResult="failure" output="dial tcp 10.134.0.19:8816: connect: connection refused" Jul 29 08:29:25.636627 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:29:25.636600 2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-feast/feast-simple-feast-setup-57d7459654-znh42" Jul 29 08:29:25.637016 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:29:25.636752 2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-feast/feast-simple-feast-setup-57d7459654-znh42" Jul 29 08:29:25.637180 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:29:25.637160 2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-feast/feast-simple-feast-setup-57d7459654-znh42" Jul 29 08:29:25.637428 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:29:25.637396 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-feast/feast-simple-feast-setup-57d7459654-znh42" Jul 29 08:29:25.637637 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:29:25.637621 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-feast/feast-simple-feast-setup-57d7459654-znh42" Jul 29 08:29:25.641688 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:29:25.641671 2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-feast/feast-simple-feast-setup-57d7459654-znh42" Jul 29 08:29:25.901400 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:29:25.901316 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-feast/feast-simple-feast-setup-57d7459654-znh42" Jul 29 08:29:25.903944 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:29:25.903924 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-feast/feast-simple-feast-setup-57d7459654-znh42" Jul 29 08:29:26.313604 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:29:26.313573 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8rp9g_cfd1d8df-e6b7-4a67-81f2-5d83da541e20/konnectivity-agent/0.log" limit=1024 Jul 29 08:29:26.922494 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:29:26.922454 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8rp9g_cfd1d8df-e6b7-4a67-81f2-5d83da541e20/konnectivity-agent/0.log" limit=1024 Jul 29 08:29:39.875544 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:29:39.875508 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8rp9g_cfd1d8df-e6b7-4a67-81f2-5d83da541e20/konnectivity-agent/0.log" limit=1024 Jul 29 08:29:40.017710 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:29:40.017672 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8rp9g_cfd1d8df-e6b7-4a67-81f2-5d83da541e20/konnectivity-agent/0.log" limit=1024 Jul 29 08:29:40.080991 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:29:40.080959 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8rp9g_cfd1d8df-e6b7-4a67-81f2-5d83da541e20/konnectivity-agent/0.log" limit=1024 Jul 29 08:29:40.619446 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:29:40.619410 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8rp9g_cfd1d8df-e6b7-4a67-81f2-5d83da541e20/konnectivity-agent/0.log" limit=1024 Jul 29 08:29:40.713724 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:29:40.713682 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8rp9g_cfd1d8df-e6b7-4a67-81f2-5d83da541e20/konnectivity-agent/0.log" limit=1024 Jul 29 08:29:47.933627 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:29:47.933596 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8rp9g_cfd1d8df-e6b7-4a67-81f2-5d83da541e20/konnectivity-agent/0.log" limit=1024 Jul 29 08:29:50.926137 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:29:50.926070 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8rp9g_cfd1d8df-e6b7-4a67-81f2-5d83da541e20/konnectivity-agent/0.log" limit=1024 Jul 29 08:29:50.989798 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:29:50.989756 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8rp9g_cfd1d8df-e6b7-4a67-81f2-5d83da541e20/konnectivity-agent/0.log" limit=1024 Jul 29 08:29:51.057726 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:29:51.057692 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8rp9g_cfd1d8df-e6b7-4a67-81f2-5d83da541e20/konnectivity-agent/0.log" limit=1024 Jul 29 08:30:27.705188 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:30:27.705097 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8rp9g_cfd1d8df-e6b7-4a67-81f2-5d83da541e20/konnectivity-agent/0.log" limit=1024 Jul 29 08:30:27.731090 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:30:27.731044 2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["test-ns-feast/feast-simple-feast-setup-57d7459654-znh42"] Jul 29 08:30:27.731401 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:30:27.731371 2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="test-ns-feast/feast-simple-feast-setup-57d7459654-znh42" podUID="23f9dea4-c464-4cca-8568-c9934ca89211" containerName="registry" containerID="cri-o://e4146548f670e4da5b1bd66a73997e6a89177c25c63d458e92898fe9ab34b832" gracePeriod=30 Jul 29 08:30:27.731513 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:30:27.731454 2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="test-ns-feast/feast-simple-feast-setup-57d7459654-znh42" podUID="23f9dea4-c464-4cca-8568-c9934ca89211" containerName="ui" containerID="cri-o://cf710d226934ad5427b130a0accf8c9eaa1af371a11c74bd50e06e8fd409091a" gracePeriod=30 Jul 29 08:30:27.731575 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:30:27.731452 2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="test-ns-feast/feast-simple-feast-setup-57d7459654-znh42" podUID="23f9dea4-c464-4cca-8568-c9934ca89211" containerName="online" containerID="cri-o://9688ba3fd4a54b7581da999201cff27f4df6274de7b6eecd0429a1808e587532" gracePeriod=30 Jul 29 08:30:27.731662 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:30:27.731452 2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="test-ns-feast/feast-simple-feast-setup-57d7459654-znh42" podUID="23f9dea4-c464-4cca-8568-c9934ca89211" containerName="offline" containerID="cri-o://01d85ec9d66eb753e86bb82dcadd63ba603b1a875541a3e7d05194895fe1e311" gracePeriod=30 Jul 29 08:30:27.772744 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:30:27.772699 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8rp9g_cfd1d8df-e6b7-4a67-81f2-5d83da541e20/konnectivity-agent/0.log" limit=1024 Jul 29 08:30:27.897858 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:30:27.897814 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8rp9g_cfd1d8df-e6b7-4a67-81f2-5d83da541e20/konnectivity-agent/0.log" limit=1024 Jul 29 08:30:27.993720 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:30:27.993630 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8rp9g_cfd1d8df-e6b7-4a67-81f2-5d83da541e20/konnectivity-agent/0.log" limit=1024 Jul 29 08:30:28.038822 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:30:28.038783 2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["test-ns-feast/feast-simple-feast-setup-57d7459654-769sh"] Jul 29 08:30:28.043076 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:30:28.043049 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/feast-simple-feast-setup-57d7459654-769sh" Jul 29 08:30:28.046685 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:30:28.046653 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-feast\"/\"feast-simple-feast-setup-dockercfg-vltbn\"" Jul 29 08:30:28.056391 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:30:28.056365 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["test-ns-feast/feast-simple-feast-setup-57d7459654-769sh"] Jul 29 08:30:28.060092 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:30:28.060053 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8rp9g_cfd1d8df-e6b7-4a67-81f2-5d83da541e20/konnectivity-agent/0.log" limit=1024 Jul 29 08:30:28.122277 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:30:28.117137 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/562b5f5f-6c61-4618-b11e-41518922e7a6-ui-tls\") pod \"feast-simple-feast-setup-57d7459654-769sh\" (UID: \"562b5f5f-6c61-4618-b11e-41518922e7a6\") " pod="test-ns-feast/feast-simple-feast-setup-57d7459654-769sh" Jul 29 08:30:28.122277 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:30:28.117280 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/562b5f5f-6c61-4618-b11e-41518922e7a6-feast-data\") pod \"feast-simple-feast-setup-57d7459654-769sh\" (UID: \"562b5f5f-6c61-4618-b11e-41518922e7a6\") " pod="test-ns-feast/feast-simple-feast-setup-57d7459654-769sh" Jul 29 08:30:28.122277 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:30:28.117333 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/562b5f5f-6c61-4618-b11e-41518922e7a6-offline-tls\") pod \"feast-simple-feast-setup-57d7459654-769sh\" (UID: \"562b5f5f-6c61-4618-b11e-41518922e7a6\") " pod="test-ns-feast/feast-simple-feast-setup-57d7459654-769sh" Jul 29 08:30:28.122277 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:30:28.117370 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/562b5f5f-6c61-4618-b11e-41518922e7a6-online-tls\") pod \"feast-simple-feast-setup-57d7459654-769sh\" (UID: \"562b5f5f-6c61-4618-b11e-41518922e7a6\") " pod="test-ns-feast/feast-simple-feast-setup-57d7459654-769sh" Jul 29 08:30:28.122277 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:30:28.117436 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kkx4w\" (UniqueName: \"kubernetes.io/projected/562b5f5f-6c61-4618-b11e-41518922e7a6-kube-api-access-kkx4w\") pod \"feast-simple-feast-setup-57d7459654-769sh\" (UID: \"562b5f5f-6c61-4618-b11e-41518922e7a6\") " pod="test-ns-feast/feast-simple-feast-setup-57d7459654-769sh" Jul 29 08:30:28.122277 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:30:28.117605 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/562b5f5f-6c61-4618-b11e-41518922e7a6-registry-tls\") pod \"feast-simple-feast-setup-57d7459654-769sh\" (UID: \"562b5f5f-6c61-4618-b11e-41518922e7a6\") " pod="test-ns-feast/feast-simple-feast-setup-57d7459654-769sh" Jul 29 08:30:28.209288 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:30:28.209221 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8rp9g_cfd1d8df-e6b7-4a67-81f2-5d83da541e20/konnectivity-agent/0.log" limit=1024 Jul 29 08:30:28.219001 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:30:28.218957 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/562b5f5f-6c61-4618-b11e-41518922e7a6-feast-data\") pod \"feast-simple-feast-setup-57d7459654-769sh\" (UID: \"562b5f5f-6c61-4618-b11e-41518922e7a6\") " pod="test-ns-feast/feast-simple-feast-setup-57d7459654-769sh" Jul 29 08:30:28.219001 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:30:28.219003 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/562b5f5f-6c61-4618-b11e-41518922e7a6-offline-tls\") pod \"feast-simple-feast-setup-57d7459654-769sh\" (UID: \"562b5f5f-6c61-4618-b11e-41518922e7a6\") " pod="test-ns-feast/feast-simple-feast-setup-57d7459654-769sh" Jul 29 08:30:28.219249 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:30:28.219028 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/562b5f5f-6c61-4618-b11e-41518922e7a6-online-tls\") pod \"feast-simple-feast-setup-57d7459654-769sh\" (UID: \"562b5f5f-6c61-4618-b11e-41518922e7a6\") " pod="test-ns-feast/feast-simple-feast-setup-57d7459654-769sh" Jul 29 08:30:28.219249 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:30:28.219069 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kkx4w\" (UniqueName: \"kubernetes.io/projected/562b5f5f-6c61-4618-b11e-41518922e7a6-kube-api-access-kkx4w\") pod \"feast-simple-feast-setup-57d7459654-769sh\" (UID: \"562b5f5f-6c61-4618-b11e-41518922e7a6\") " pod="test-ns-feast/feast-simple-feast-setup-57d7459654-769sh" Jul 29 08:30:28.219249 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:30:28.219129 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/562b5f5f-6c61-4618-b11e-41518922e7a6-registry-tls\") pod \"feast-simple-feast-setup-57d7459654-769sh\" (UID: \"562b5f5f-6c61-4618-b11e-41518922e7a6\") " pod="test-ns-feast/feast-simple-feast-setup-57d7459654-769sh" Jul 29 08:30:28.219249 ip-10-0-131-138 kubenswrapper[2563]: E0729 08:30:28.219149 2563 secret.go:189] Couldn't get secret test-ns-feast/feast-simple-feast-setup-offline-tls: secret "feast-simple-feast-setup-offline-tls" not found Jul 29 08:30:28.219249 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:30:28.219180 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/562b5f5f-6c61-4618-b11e-41518922e7a6-ui-tls\") pod \"feast-simple-feast-setup-57d7459654-769sh\" (UID: \"562b5f5f-6c61-4618-b11e-41518922e7a6\") " pod="test-ns-feast/feast-simple-feast-setup-57d7459654-769sh" Jul 29 08:30:28.219249 ip-10-0-131-138 kubenswrapper[2563]: E0729 08:30:28.219235 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/562b5f5f-6c61-4618-b11e-41518922e7a6-offline-tls podName:562b5f5f-6c61-4618-b11e-41518922e7a6 nodeName:}" failed. No retries permitted until 2026-07-29 08:30:28.719213114 +0000 UTC m=+541.300501885 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "offline-tls" (UniqueName: "kubernetes.io/secret/562b5f5f-6c61-4618-b11e-41518922e7a6-offline-tls") pod "feast-simple-feast-setup-57d7459654-769sh" (UID: "562b5f5f-6c61-4618-b11e-41518922e7a6") : secret "feast-simple-feast-setup-offline-tls" not found Jul 29 08:30:28.219556 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:30:28.219376 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/562b5f5f-6c61-4618-b11e-41518922e7a6-feast-data\") pod \"feast-simple-feast-setup-57d7459654-769sh\" (UID: \"562b5f5f-6c61-4618-b11e-41518922e7a6\") " pod="test-ns-feast/feast-simple-feast-setup-57d7459654-769sh" Jul 29 08:30:28.221911 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:30:28.221880 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/562b5f5f-6c61-4618-b11e-41518922e7a6-online-tls\") pod \"feast-simple-feast-setup-57d7459654-769sh\" (UID: \"562b5f5f-6c61-4618-b11e-41518922e7a6\") " pod="test-ns-feast/feast-simple-feast-setup-57d7459654-769sh" Jul 29 08:30:28.221911 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:30:28.221896 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/562b5f5f-6c61-4618-b11e-41518922e7a6-registry-tls\") pod \"feast-simple-feast-setup-57d7459654-769sh\" (UID: \"562b5f5f-6c61-4618-b11e-41518922e7a6\") " pod="test-ns-feast/feast-simple-feast-setup-57d7459654-769sh" Jul 29 08:30:28.222107 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:30:28.222020 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/562b5f5f-6c61-4618-b11e-41518922e7a6-ui-tls\") pod \"feast-simple-feast-setup-57d7459654-769sh\" (UID: \"562b5f5f-6c61-4618-b11e-41518922e7a6\") " pod="test-ns-feast/feast-simple-feast-setup-57d7459654-769sh" Jul 29 08:30:28.231202 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:30:28.231172 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kkx4w\" (UniqueName: \"kubernetes.io/projected/562b5f5f-6c61-4618-b11e-41518922e7a6-kube-api-access-kkx4w\") pod \"feast-simple-feast-setup-57d7459654-769sh\" (UID: \"562b5f5f-6c61-4618-b11e-41518922e7a6\") " pod="test-ns-feast/feast-simple-feast-setup-57d7459654-769sh" Jul 29 08:30:28.309311 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:30:28.309249 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8rp9g_cfd1d8df-e6b7-4a67-81f2-5d83da541e20/konnectivity-agent/0.log" limit=1024 Jul 29 08:30:28.486794 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:30:28.486754 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8rp9g_cfd1d8df-e6b7-4a67-81f2-5d83da541e20/konnectivity-agent/0.log" limit=1024 Jul 29 08:30:28.724372 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:30:28.724338 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/562b5f5f-6c61-4618-b11e-41518922e7a6-offline-tls\") pod \"feast-simple-feast-setup-57d7459654-769sh\" (UID: \"562b5f5f-6c61-4618-b11e-41518922e7a6\") " pod="test-ns-feast/feast-simple-feast-setup-57d7459654-769sh" Jul 29 08:30:28.726843 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:30:28.726822 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/562b5f5f-6c61-4618-b11e-41518922e7a6-offline-tls\") pod \"feast-simple-feast-setup-57d7459654-769sh\" (UID: \"562b5f5f-6c61-4618-b11e-41518922e7a6\") " pod="test-ns-feast/feast-simple-feast-setup-57d7459654-769sh" Jul 29 08:30:28.955585 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:30:28.955550 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/feast-simple-feast-setup-57d7459654-769sh" Jul 29 08:30:29.104684 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:30:29.104597 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["test-ns-feast/feast-simple-feast-setup-57d7459654-769sh"] Jul 29 08:30:29.107339 ip-10-0-131-138 kubenswrapper[2563]: W0729 08:30:29.107313 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod562b5f5f_6c61_4618_b11e_41518922e7a6.slice/crio-46db27c2018b716e1b70f109e48d14fd9dfccf47cfe80a7567430aca9f6da44b WatchSource:0}: Error finding container 46db27c2018b716e1b70f109e48d14fd9dfccf47cfe80a7567430aca9f6da44b: Status 404 returned error can't find the container with id 46db27c2018b716e1b70f109e48d14fd9dfccf47cfe80a7567430aca9f6da44b Jul 29 08:30:29.132022 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:30:29.131989 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-57d7459654-769sh" event={"ID":"562b5f5f-6c61-4618-b11e-41518922e7a6","Type":"ContainerStarted","Data":"46db27c2018b716e1b70f109e48d14fd9dfccf47cfe80a7567430aca9f6da44b"} Jul 29 08:30:29.134627 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:30:29.134604 2563 generic.go:356] "Generic (PLEG): container finished" podID="23f9dea4-c464-4cca-8568-c9934ca89211" containerID="cf710d226934ad5427b130a0accf8c9eaa1af371a11c74bd50e06e8fd409091a" exitCode=0 Jul 29 08:30:29.134627 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:30:29.134624 2563 generic.go:356] "Generic (PLEG): container finished" podID="23f9dea4-c464-4cca-8568-c9934ca89211" containerID="9688ba3fd4a54b7581da999201cff27f4df6274de7b6eecd0429a1808e587532" exitCode=0 Jul 29 08:30:29.134758 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:30:29.134654 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-57d7459654-znh42" event={"ID":"23f9dea4-c464-4cca-8568-c9934ca89211","Type":"ContainerDied","Data":"cf710d226934ad5427b130a0accf8c9eaa1af371a11c74bd50e06e8fd409091a"} Jul 29 08:30:29.134758 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:30:29.134681 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-57d7459654-znh42" event={"ID":"23f9dea4-c464-4cca-8568-c9934ca89211","Type":"ContainerDied","Data":"9688ba3fd4a54b7581da999201cff27f4df6274de7b6eecd0429a1808e587532"} Jul 29 08:30:30.140299 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:30:30.140242 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-57d7459654-769sh" event={"ID":"562b5f5f-6c61-4618-b11e-41518922e7a6","Type":"ContainerStarted","Data":"0d3a1df021e0ed10c199857cdacadaad9cb46533321cc945ef88989f80364b01"} Jul 29 08:30:34.156481 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:30:34.156442 2563 generic.go:356] "Generic (PLEG): container finished" podID="562b5f5f-6c61-4618-b11e-41518922e7a6" containerID="0d3a1df021e0ed10c199857cdacadaad9cb46533321cc945ef88989f80364b01" exitCode=0 Jul 29 08:30:34.156869 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:30:34.156517 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-57d7459654-769sh" event={"ID":"562b5f5f-6c61-4618-b11e-41518922e7a6","Type":"ContainerDied","Data":"0d3a1df021e0ed10c199857cdacadaad9cb46533321cc945ef88989f80364b01"} Jul 29 08:30:35.162861 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:30:35.162824 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-57d7459654-769sh" event={"ID":"562b5f5f-6c61-4618-b11e-41518922e7a6","Type":"ContainerStarted","Data":"5eeb177ee807a924207351fcfc9934115876f7f210102e091191b7a93c4b0117"} Jul 29 08:30:35.638357 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:30:35.638314 2563 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-57d7459654-znh42" podUID="23f9dea4-c464-4cca-8568-c9934ca89211" containerName="ui" probeResult="failure" output="dial tcp 10.134.0.19:8443: connect: connection refused" Jul 29 08:30:35.901966 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:30:35.901875 2563 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-57d7459654-znh42" podUID="23f9dea4-c464-4cca-8568-c9934ca89211" containerName="online" probeResult="failure" output="Get \"https://10.134.0.19:6567/health\": dial tcp 10.134.0.19:6567: connect: connection refused" Jul 29 08:30:40.186165 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:30:40.186127 2563 generic.go:356] "Generic (PLEG): container finished" podID="562b5f5f-6c61-4618-b11e-41518922e7a6" containerID="5eeb177ee807a924207351fcfc9934115876f7f210102e091191b7a93c4b0117" exitCode=0 Jul 29 08:30:40.186582 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:30:40.186193 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-57d7459654-769sh" event={"ID":"562b5f5f-6c61-4618-b11e-41518922e7a6","Type":"ContainerDied","Data":"5eeb177ee807a924207351fcfc9934115876f7f210102e091191b7a93c4b0117"} Jul 29 08:30:41.196678 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:30:41.196636 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-57d7459654-769sh" event={"ID":"562b5f5f-6c61-4618-b11e-41518922e7a6","Type":"ContainerStarted","Data":"0a60ff42153cd7243a08db6d5b6e7fb1191cfd7ebf7b0d8d51f0a668bf60bd01"} Jul 29 08:30:41.197148 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:30:41.196701 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-57d7459654-769sh" event={"ID":"562b5f5f-6c61-4618-b11e-41518922e7a6","Type":"ContainerStarted","Data":"c491f99363c3d4556759693b49458785e5e44635b52b0a40c0cff91ca24d0f44"} Jul 29 08:30:41.197148 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:30:41.196721 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-57d7459654-769sh" event={"ID":"562b5f5f-6c61-4618-b11e-41518922e7a6","Type":"ContainerStarted","Data":"b541318a41ca2d9895c30890b4850d0514fd543f57db33c885e5b6d82ba17687"} Jul 29 08:30:41.197148 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:30:41.196739 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-57d7459654-769sh" event={"ID":"562b5f5f-6c61-4618-b11e-41518922e7a6","Type":"ContainerStarted","Data":"799f32f92a307c4ebc64fd16ef84a1a5ba6f6b34cc71af2fa9184496756dde21"} Jul 29 08:30:41.227128 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:30:41.227050 2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="test-ns-feast/feast-simple-feast-setup-57d7459654-769sh" podStartSLOduration=13.227024042 podStartE2EDuration="13.227024042s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 08:30:28 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 08:30:41.225316004 +0000 UTC m=+553.806604788" watchObservedRunningTime="2026-07-29 08:30:41.227024042 +0000 UTC m=+553.808312821" Jul 29 08:30:43.956476 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:30:43.956436 2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-feast/feast-simple-feast-setup-57d7459654-769sh" Jul 29 08:30:43.957205 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:30:43.957172 2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-feast/feast-simple-feast-setup-57d7459654-769sh" Jul 29 08:30:43.957378 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:30:43.957213 2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-feast/feast-simple-feast-setup-57d7459654-769sh" Jul 29 08:30:43.957378 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:30:43.957232 2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-feast/feast-simple-feast-setup-57d7459654-769sh" Jul 29 08:30:43.958755 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:30:43.958724 2563 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-57d7459654-769sh" podUID="562b5f5f-6c61-4618-b11e-41518922e7a6" containerName="offline" probeResult="failure" output="dial tcp 10.134.0.20:8816: connect: connection refused" Jul 29 08:30:43.958971 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:30:43.958757 2563 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-57d7459654-769sh" podUID="562b5f5f-6c61-4618-b11e-41518922e7a6" containerName="online" probeResult="failure" output="Get \"https://10.134.0.20:6567/health\": dial tcp 10.134.0.20:6567: connect: connection refused" Jul 29 08:30:43.959060 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:30:43.958800 2563 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-57d7459654-769sh" podUID="562b5f5f-6c61-4618-b11e-41518922e7a6" containerName="registry" probeResult="failure" output="dial tcp 10.134.0.20:6571: connect: connection refused" Jul 29 08:30:43.959060 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:30:43.958807 2563 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-57d7459654-769sh" podUID="562b5f5f-6c61-4618-b11e-41518922e7a6" containerName="ui" probeResult="failure" output="dial tcp 10.134.0.20:8443: connect: connection refused" Jul 29 08:30:45.638162 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:30:45.638113 2563 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-57d7459654-znh42" podUID="23f9dea4-c464-4cca-8568-c9934ca89211" containerName="ui" probeResult="failure" output="dial tcp 10.134.0.19:8443: connect: connection refused" Jul 29 08:30:45.901696 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:30:45.901577 2563 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-57d7459654-znh42" podUID="23f9dea4-c464-4cca-8568-c9934ca89211" containerName="online" probeResult="failure" output="Get \"https://10.134.0.19:6567/health\": dial tcp 10.134.0.19:6567: connect: connection refused" Jul 29 08:30:46.957444 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:30:46.957411 2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-feast/feast-simple-feast-setup-57d7459654-769sh" Jul 29 08:30:46.957816 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:30:46.957528 2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-feast/feast-simple-feast-setup-57d7459654-769sh" Jul 29 08:30:46.957816 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:30:46.957751 2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-feast/feast-simple-feast-setup-57d7459654-769sh" Jul 29 08:30:46.958111 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:30:46.958082 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-57d7459654-769sh" Jul 29 08:30:46.958218 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:30:46.958125 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-57d7459654-769sh" Jul 29 08:30:46.958218 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:30:46.958143 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-57d7459654-769sh" Jul 29 08:30:46.958409 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:30:46.958383 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-feast/feast-simple-feast-setup-57d7459654-769sh" Jul 29 08:30:46.958545 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:30:46.958461 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-feast/feast-simple-feast-setup-57d7459654-769sh" Jul 29 08:30:46.958668 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:30:46.958653 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-feast/feast-simple-feast-setup-57d7459654-769sh" Jul 29 08:30:46.962692 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:30:46.962664 2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-feast/feast-simple-feast-setup-57d7459654-769sh" Jul 29 08:30:47.232281 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:30:47.232179 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-57d7459654-769sh" Jul 29 08:30:47.235462 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:30:47.235441 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-feast/feast-simple-feast-setup-57d7459654-769sh" Jul 29 08:30:47.529395 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:30:47.529307 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8rp9g_cfd1d8df-e6b7-4a67-81f2-5d83da541e20/konnectivity-agent/0.log" limit=1024 Jul 29 08:30:55.637881 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:30:55.637831 2563 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-57d7459654-znh42" podUID="23f9dea4-c464-4cca-8568-c9934ca89211" containerName="ui" probeResult="failure" output="dial tcp 10.134.0.19:8443: connect: connection refused" Jul 29 08:30:55.638352 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:30:55.637975 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-57d7459654-znh42" Jul 29 08:30:55.901037 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:30:55.900928 2563 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-57d7459654-znh42" podUID="23f9dea4-c464-4cca-8568-c9934ca89211" containerName="online" probeResult="failure" output="Get \"https://10.134.0.19:6567/health\": dial tcp 10.134.0.19:6567: connect: connection refused" Jul 29 08:30:55.901216 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:30:55.901081 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-57d7459654-znh42" Jul 29 08:30:58.278662 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:30:58.278626 2563 generic.go:356] "Generic (PLEG): container finished" podID="23f9dea4-c464-4cca-8568-c9934ca89211" containerID="01d85ec9d66eb753e86bb82dcadd63ba603b1a875541a3e7d05194895fe1e311" exitCode=137 Jul 29 08:30:58.278662 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:30:58.278656 2563 generic.go:356] "Generic (PLEG): container finished" podID="23f9dea4-c464-4cca-8568-c9934ca89211" containerID="e4146548f670e4da5b1bd66a73997e6a89177c25c63d458e92898fe9ab34b832" exitCode=137 Jul 29 08:30:58.279055 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:30:58.278675 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-57d7459654-znh42" event={"ID":"23f9dea4-c464-4cca-8568-c9934ca89211","Type":"ContainerDied","Data":"01d85ec9d66eb753e86bb82dcadd63ba603b1a875541a3e7d05194895fe1e311"} Jul 29 08:30:58.279055 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:30:58.278722 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-57d7459654-znh42" event={"ID":"23f9dea4-c464-4cca-8568-c9934ca89211","Type":"ContainerDied","Data":"e4146548f670e4da5b1bd66a73997e6a89177c25c63d458e92898fe9ab34b832"} Jul 29 08:30:58.381042 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:30:58.381005 2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/feast-simple-feast-setup-57d7459654-znh42" Jul 29 08:30:58.485212 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:30:58.485177 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/23f9dea4-c464-4cca-8568-c9934ca89211-offline-tls\") pod \"23f9dea4-c464-4cca-8568-c9934ca89211\" (UID: \"23f9dea4-c464-4cca-8568-c9934ca89211\") " Jul 29 08:30:58.485419 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:30:58.485229 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/23f9dea4-c464-4cca-8568-c9934ca89211-registry-tls\") pod \"23f9dea4-c464-4cca-8568-c9934ca89211\" (UID: \"23f9dea4-c464-4cca-8568-c9934ca89211\") " Jul 29 08:30:58.485419 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:30:58.485310 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/23f9dea4-c464-4cca-8568-c9934ca89211-ui-tls\") pod \"23f9dea4-c464-4cca-8568-c9934ca89211\" (UID: \"23f9dea4-c464-4cca-8568-c9934ca89211\") " Jul 29 08:30:58.485419 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:30:58.485343 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/23f9dea4-c464-4cca-8568-c9934ca89211-online-tls\") pod \"23f9dea4-c464-4cca-8568-c9934ca89211\" (UID: \"23f9dea4-c464-4cca-8568-c9934ca89211\") " Jul 29 08:30:58.485419 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:30:58.485388 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-td45w\" (UniqueName: \"kubernetes.io/projected/23f9dea4-c464-4cca-8568-c9934ca89211-kube-api-access-td45w\") pod \"23f9dea4-c464-4cca-8568-c9934ca89211\" (UID: \"23f9dea4-c464-4cca-8568-c9934ca89211\") " Jul 29 08:30:58.485609 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:30:58.485423 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/23f9dea4-c464-4cca-8568-c9934ca89211-feast-data\") pod \"23f9dea4-c464-4cca-8568-c9934ca89211\" (UID: \"23f9dea4-c464-4cca-8568-c9934ca89211\") " Jul 29 08:30:58.486125 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:30:58.486043 2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/23f9dea4-c464-4cca-8568-c9934ca89211-feast-data" (OuterVolumeSpecName: "feast-data") pod "23f9dea4-c464-4cca-8568-c9934ca89211" (UID: "23f9dea4-c464-4cca-8568-c9934ca89211"). InnerVolumeSpecName "feast-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 29 08:30:58.487812 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:30:58.487782 2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/23f9dea4-c464-4cca-8568-c9934ca89211-ui-tls" (OuterVolumeSpecName: "ui-tls") pod "23f9dea4-c464-4cca-8568-c9934ca89211" (UID: "23f9dea4-c464-4cca-8568-c9934ca89211"). InnerVolumeSpecName "ui-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 29 08:30:58.488192 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:30:58.488161 2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/23f9dea4-c464-4cca-8568-c9934ca89211-offline-tls" (OuterVolumeSpecName: "offline-tls") pod "23f9dea4-c464-4cca-8568-c9934ca89211" (UID: "23f9dea4-c464-4cca-8568-c9934ca89211"). InnerVolumeSpecName "offline-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 29 08:30:58.488338 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:30:58.488184 2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/23f9dea4-c464-4cca-8568-c9934ca89211-kube-api-access-td45w" (OuterVolumeSpecName: "kube-api-access-td45w") pod "23f9dea4-c464-4cca-8568-c9934ca89211" (UID: "23f9dea4-c464-4cca-8568-c9934ca89211"). InnerVolumeSpecName "kube-api-access-td45w". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 29 08:30:58.488338 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:30:58.488233 2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/23f9dea4-c464-4cca-8568-c9934ca89211-registry-tls" (OuterVolumeSpecName: "registry-tls") pod "23f9dea4-c464-4cca-8568-c9934ca89211" (UID: "23f9dea4-c464-4cca-8568-c9934ca89211"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 29 08:30:58.488338 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:30:58.488249 2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/23f9dea4-c464-4cca-8568-c9934ca89211-online-tls" (OuterVolumeSpecName: "online-tls") pod "23f9dea4-c464-4cca-8568-c9934ca89211" (UID: "23f9dea4-c464-4cca-8568-c9934ca89211"). InnerVolumeSpecName "online-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 29 08:30:58.586098 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:30:58.586008 2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-td45w\" (UniqueName: \"kubernetes.io/projected/23f9dea4-c464-4cca-8568-c9934ca89211-kube-api-access-td45w\") on node \"ip-10-0-131-138.ec2.internal\" DevicePath \"\"" Jul 29 08:30:58.586098 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:30:58.586043 2563 reconciler_common.go:299] "Volume detached for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/23f9dea4-c464-4cca-8568-c9934ca89211-feast-data\") on node \"ip-10-0-131-138.ec2.internal\" DevicePath \"\"" Jul 29 08:30:58.586098 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:30:58.586056 2563 reconciler_common.go:299] "Volume detached for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/23f9dea4-c464-4cca-8568-c9934ca89211-offline-tls\") on node \"ip-10-0-131-138.ec2.internal\" DevicePath \"\"" Jul 29 08:30:58.586098 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:30:58.586064 2563 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/23f9dea4-c464-4cca-8568-c9934ca89211-registry-tls\") on node \"ip-10-0-131-138.ec2.internal\" DevicePath \"\"" Jul 29 08:30:58.586098 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:30:58.586072 2563 reconciler_common.go:299] "Volume detached for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/23f9dea4-c464-4cca-8568-c9934ca89211-ui-tls\") on node \"ip-10-0-131-138.ec2.internal\" DevicePath \"\"" Jul 29 08:30:58.586098 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:30:58.586102 2563 reconciler_common.go:299] "Volume detached for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/23f9dea4-c464-4cca-8568-c9934ca89211-online-tls\") on node \"ip-10-0-131-138.ec2.internal\" DevicePath \"\"" Jul 29 08:30:59.285605 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:30:59.285571 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-57d7459654-znh42" event={"ID":"23f9dea4-c464-4cca-8568-c9934ca89211","Type":"ContainerDied","Data":"4239e10642466a1cef6edaa732d560d8992a089b02fe49daaa06ca8f0b7a7041"} Jul 29 08:30:59.286084 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:30:59.285629 2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/feast-simple-feast-setup-57d7459654-znh42" Jul 29 08:30:59.286084 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:30:59.285647 2563 scope.go:117] "RemoveContainer" containerID="cf710d226934ad5427b130a0accf8c9eaa1af371a11c74bd50e06e8fd409091a" Jul 29 08:30:59.295105 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:30:59.295083 2563 scope.go:117] "RemoveContainer" containerID="01d85ec9d66eb753e86bb82dcadd63ba603b1a875541a3e7d05194895fe1e311" Jul 29 08:30:59.303492 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:30:59.303471 2563 scope.go:117] "RemoveContainer" containerID="9688ba3fd4a54b7581da999201cff27f4df6274de7b6eecd0429a1808e587532" Jul 29 08:30:59.310511 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:30:59.310479 2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["test-ns-feast/feast-simple-feast-setup-57d7459654-znh42"] Jul 29 08:30:59.311998 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:30:59.311980 2563 scope.go:117] "RemoveContainer" containerID="e4146548f670e4da5b1bd66a73997e6a89177c25c63d458e92898fe9ab34b832" Jul 29 08:30:59.317207 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:30:59.317184 2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["test-ns-feast/feast-simple-feast-setup-57d7459654-znh42"] Jul 29 08:30:59.320006 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:30:59.319985 2563 scope.go:117] "RemoveContainer" containerID="a657ad1567cc2753c77a42503ff91b69c93b5272437767f381e0003e9e34097c" Jul 29 08:30:59.327454 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:30:59.327434 2563 scope.go:117] "RemoveContainer" containerID="e487d6ece51ea1839b8f3e7927f0ba70c6f83fed53ae5c79d10f505e1edb07a5" Jul 29 08:30:59.962192 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:30:59.962158 2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="23f9dea4-c464-4cca-8568-c9934ca89211" path="/var/lib/kubelet/pods/23f9dea4-c464-4cca-8568-c9934ca89211/volumes" Jul 29 08:31:49.431413 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:31:49.431330 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8rp9g_cfd1d8df-e6b7-4a67-81f2-5d83da541e20/konnectivity-agent/0.log" limit=1024 Jul 29 08:31:49.518811 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:31:49.518777 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8rp9g_cfd1d8df-e6b7-4a67-81f2-5d83da541e20/konnectivity-agent/0.log" limit=1024 Jul 29 08:31:49.617944 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:31:49.617903 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8rp9g_cfd1d8df-e6b7-4a67-81f2-5d83da541e20/konnectivity-agent/0.log" limit=1024 Jul 29 08:31:49.720729 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:31:49.720659 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8rp9g_cfd1d8df-e6b7-4a67-81f2-5d83da541e20/konnectivity-agent/0.log" limit=1024 Jul 29 08:31:49.733185 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:31:49.733153 2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["test-ns-remote-registry/feast-simple-feast-remote-setup-6944688bc8-fhd6x"] Jul 29 08:31:49.733480 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:31:49.733468 2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="23f9dea4-c464-4cca-8568-c9934ca89211" containerName="feast-apply" Jul 29 08:31:49.733522 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:31:49.733482 2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="23f9dea4-c464-4cca-8568-c9934ca89211" containerName="feast-apply" Jul 29 08:31:49.733522 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:31:49.733494 2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="23f9dea4-c464-4cca-8568-c9934ca89211" containerName="online" Jul 29 08:31:49.733522 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:31:49.733500 2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="23f9dea4-c464-4cca-8568-c9934ca89211" containerName="online" Jul 29 08:31:49.733522 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:31:49.733508 2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="23f9dea4-c464-4cca-8568-c9934ca89211" containerName="offline" Jul 29 08:31:49.733522 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:31:49.733514 2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="23f9dea4-c464-4cca-8568-c9934ca89211" containerName="offline" Jul 29 08:31:49.733693 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:31:49.733524 2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="23f9dea4-c464-4cca-8568-c9934ca89211" containerName="ui" Jul 29 08:31:49.733693 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:31:49.733530 2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="23f9dea4-c464-4cca-8568-c9934ca89211" containerName="ui" Jul 29 08:31:49.733693 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:31:49.733538 2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="23f9dea4-c464-4cca-8568-c9934ca89211" containerName="feast-init" Jul 29 08:31:49.733693 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:31:49.733544 2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="23f9dea4-c464-4cca-8568-c9934ca89211" containerName="feast-init" Jul 29 08:31:49.733693 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:31:49.733552 2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="23f9dea4-c464-4cca-8568-c9934ca89211" containerName="registry" Jul 29 08:31:49.733693 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:31:49.733557 2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="23f9dea4-c464-4cca-8568-c9934ca89211" containerName="registry" Jul 29 08:31:49.733693 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:31:49.733603 2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="23f9dea4-c464-4cca-8568-c9934ca89211" containerName="registry" Jul 29 08:31:49.733693 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:31:49.733609 2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="23f9dea4-c464-4cca-8568-c9934ca89211" containerName="online" Jul 29 08:31:49.733693 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:31:49.733616 2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="23f9dea4-c464-4cca-8568-c9934ca89211" containerName="offline" Jul 29 08:31:49.733693 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:31:49.733621 2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="23f9dea4-c464-4cca-8568-c9934ca89211" containerName="ui" Jul 29 08:31:49.737882 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:31:49.737863 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-6944688bc8-fhd6x" Jul 29 08:31:49.741169 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:31:49.741146 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"test-ns-remote-registry\"/\"feast-simple-feast-remote-setup-client-ca\"" Jul 29 08:31:49.741306 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:31:49.741284 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"test-ns-remote-registry\"/\"kube-root-ca.crt\"" Jul 29 08:31:49.741531 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:31:49.741516 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-remote-registry\"/\"feast-simple-feast-remote-setup-offline-tls\"" Jul 29 08:31:49.741629 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:31:49.741601 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-remote-registry\"/\"feast-simple-feast-remote-setup-online-tls\"" Jul 29 08:31:49.741689 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:31:49.741648 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-remote-registry\"/\"feast-simple-feast-remote-setup-dockercfg-ldwll\"" Jul 29 08:31:49.741767 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:31:49.741752 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"test-ns-remote-registry\"/\"feast-simple-feast-remote-setup-intra-comm\"" Jul 29 08:31:49.741854 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:31:49.741838 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"test-ns-remote-registry\"/\"openshift-service-ca.crt\"" Jul 29 08:31:49.742369 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:31:49.742356 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-remote-registry\"/\"feast-simple-feast-remote-setup-ui-tls\"" Jul 29 08:31:49.755424 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:31:49.755401 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["test-ns-remote-registry/feast-simple-feast-remote-setup-6944688bc8-fhd6x"] Jul 29 08:31:49.785046 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:31:49.785016 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/e283ad45-0159-43dd-a7ec-2be6de4297d5-feast-data\") pod \"feast-simple-feast-remote-setup-6944688bc8-fhd6x\" (UID: \"e283ad45-0159-43dd-a7ec-2be6de4297d5\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-6944688bc8-fhd6x" Jul 29 08:31:49.785046 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:31:49.785048 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/e283ad45-0159-43dd-a7ec-2be6de4297d5-offline-tls\") pod \"feast-simple-feast-remote-setup-6944688bc8-fhd6x\" (UID: \"e283ad45-0159-43dd-a7ec-2be6de4297d5\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-6944688bc8-fhd6x" Jul 29 08:31:49.785241 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:31:49.785068 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/e283ad45-0159-43dd-a7ec-2be6de4297d5-ui-tls\") pod \"feast-simple-feast-remote-setup-6944688bc8-fhd6x\" (UID: \"e283ad45-0159-43dd-a7ec-2be6de4297d5\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-6944688bc8-fhd6x" Jul 29 08:31:49.785241 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:31:49.785109 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/configmap/e283ad45-0159-43dd-a7ec-2be6de4297d5-registry-tls\") pod \"feast-simple-feast-remote-setup-6944688bc8-fhd6x\" (UID: \"e283ad45-0159-43dd-a7ec-2be6de4297d5\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-6944688bc8-fhd6x" Jul 29 08:31:49.785241 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:31:49.785136 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/e283ad45-0159-43dd-a7ec-2be6de4297d5-online-tls\") pod \"feast-simple-feast-remote-setup-6944688bc8-fhd6x\" (UID: \"e283ad45-0159-43dd-a7ec-2be6de4297d5\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-6944688bc8-fhd6x" Jul 29 08:31:49.785241 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:31:49.785159 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gdzc2\" (UniqueName: \"kubernetes.io/projected/e283ad45-0159-43dd-a7ec-2be6de4297d5-kube-api-access-gdzc2\") pod \"feast-simple-feast-remote-setup-6944688bc8-fhd6x\" (UID: \"e283ad45-0159-43dd-a7ec-2be6de4297d5\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-6944688bc8-fhd6x" Jul 29 08:31:49.808552 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:31:49.808523 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8rp9g_cfd1d8df-e6b7-4a67-81f2-5d83da541e20/konnectivity-agent/0.log" limit=1024 Jul 29 08:31:49.885609 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:31:49.885574 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/e283ad45-0159-43dd-a7ec-2be6de4297d5-feast-data\") pod \"feast-simple-feast-remote-setup-6944688bc8-fhd6x\" (UID: \"e283ad45-0159-43dd-a7ec-2be6de4297d5\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-6944688bc8-fhd6x" Jul 29 08:31:49.885609 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:31:49.885608 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/e283ad45-0159-43dd-a7ec-2be6de4297d5-offline-tls\") pod \"feast-simple-feast-remote-setup-6944688bc8-fhd6x\" (UID: \"e283ad45-0159-43dd-a7ec-2be6de4297d5\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-6944688bc8-fhd6x" Jul 29 08:31:49.885828 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:31:49.885628 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/e283ad45-0159-43dd-a7ec-2be6de4297d5-ui-tls\") pod \"feast-simple-feast-remote-setup-6944688bc8-fhd6x\" (UID: \"e283ad45-0159-43dd-a7ec-2be6de4297d5\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-6944688bc8-fhd6x" Jul 29 08:31:49.885828 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:31:49.885652 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/configmap/e283ad45-0159-43dd-a7ec-2be6de4297d5-registry-tls\") pod \"feast-simple-feast-remote-setup-6944688bc8-fhd6x\" (UID: \"e283ad45-0159-43dd-a7ec-2be6de4297d5\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-6944688bc8-fhd6x" Jul 29 08:31:49.885828 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:31:49.885675 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/e283ad45-0159-43dd-a7ec-2be6de4297d5-online-tls\") pod \"feast-simple-feast-remote-setup-6944688bc8-fhd6x\" (UID: \"e283ad45-0159-43dd-a7ec-2be6de4297d5\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-6944688bc8-fhd6x" Jul 29 08:31:49.885828 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:31:49.885708 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gdzc2\" (UniqueName: \"kubernetes.io/projected/e283ad45-0159-43dd-a7ec-2be6de4297d5-kube-api-access-gdzc2\") pod \"feast-simple-feast-remote-setup-6944688bc8-fhd6x\" (UID: \"e283ad45-0159-43dd-a7ec-2be6de4297d5\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-6944688bc8-fhd6x" Jul 29 08:31:49.885828 ip-10-0-131-138 kubenswrapper[2563]: E0729 08:31:49.885745 2563 secret.go:189] Couldn't get secret test-ns-remote-registry/feast-simple-feast-remote-setup-offline-tls: secret "feast-simple-feast-remote-setup-offline-tls" not found Jul 29 08:31:49.886070 ip-10-0-131-138 kubenswrapper[2563]: E0729 08:31:49.885838 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e283ad45-0159-43dd-a7ec-2be6de4297d5-offline-tls podName:e283ad45-0159-43dd-a7ec-2be6de4297d5 nodeName:}" failed. No retries permitted until 2026-07-29 08:31:50.385817606 +0000 UTC m=+622.967106361 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "offline-tls" (UniqueName: "kubernetes.io/secret/e283ad45-0159-43dd-a7ec-2be6de4297d5-offline-tls") pod "feast-simple-feast-remote-setup-6944688bc8-fhd6x" (UID: "e283ad45-0159-43dd-a7ec-2be6de4297d5") : secret "feast-simple-feast-remote-setup-offline-tls" not found Jul 29 08:31:49.886070 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:31:49.885978 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/e283ad45-0159-43dd-a7ec-2be6de4297d5-feast-data\") pod \"feast-simple-feast-remote-setup-6944688bc8-fhd6x\" (UID: \"e283ad45-0159-43dd-a7ec-2be6de4297d5\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-6944688bc8-fhd6x" Jul 29 08:31:49.886336 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:31:49.886312 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/configmap/e283ad45-0159-43dd-a7ec-2be6de4297d5-registry-tls\") pod \"feast-simple-feast-remote-setup-6944688bc8-fhd6x\" (UID: \"e283ad45-0159-43dd-a7ec-2be6de4297d5\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-6944688bc8-fhd6x" Jul 29 08:31:49.888343 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:31:49.888326 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/e283ad45-0159-43dd-a7ec-2be6de4297d5-ui-tls\") pod \"feast-simple-feast-remote-setup-6944688bc8-fhd6x\" (UID: \"e283ad45-0159-43dd-a7ec-2be6de4297d5\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-6944688bc8-fhd6x" Jul 29 08:31:49.888343 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:31:49.888333 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/e283ad45-0159-43dd-a7ec-2be6de4297d5-online-tls\") pod \"feast-simple-feast-remote-setup-6944688bc8-fhd6x\" (UID: \"e283ad45-0159-43dd-a7ec-2be6de4297d5\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-6944688bc8-fhd6x" Jul 29 08:31:49.898094 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:31:49.898067 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gdzc2\" (UniqueName: \"kubernetes.io/projected/e283ad45-0159-43dd-a7ec-2be6de4297d5-kube-api-access-gdzc2\") pod \"feast-simple-feast-remote-setup-6944688bc8-fhd6x\" (UID: \"e283ad45-0159-43dd-a7ec-2be6de4297d5\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-6944688bc8-fhd6x" Jul 29 08:31:49.921148 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:31:49.921118 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8rp9g_cfd1d8df-e6b7-4a67-81f2-5d83da541e20/konnectivity-agent/0.log" limit=1024 Jul 29 08:31:50.020591 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:31:50.020499 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8rp9g_cfd1d8df-e6b7-4a67-81f2-5d83da541e20/konnectivity-agent/0.log" limit=1024 Jul 29 08:31:50.390879 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:31:50.390787 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/e283ad45-0159-43dd-a7ec-2be6de4297d5-offline-tls\") pod \"feast-simple-feast-remote-setup-6944688bc8-fhd6x\" (UID: \"e283ad45-0159-43dd-a7ec-2be6de4297d5\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-6944688bc8-fhd6x" Jul 29 08:31:50.393500 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:31:50.393469 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/e283ad45-0159-43dd-a7ec-2be6de4297d5-offline-tls\") pod \"feast-simple-feast-remote-setup-6944688bc8-fhd6x\" (UID: \"e283ad45-0159-43dd-a7ec-2be6de4297d5\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-6944688bc8-fhd6x" Jul 29 08:31:50.647201 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:31:50.647098 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-6944688bc8-fhd6x" Jul 29 08:31:50.788570 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:31:50.788540 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["test-ns-remote-registry/feast-simple-feast-remote-setup-6944688bc8-fhd6x"] Jul 29 08:31:50.791560 ip-10-0-131-138 kubenswrapper[2563]: W0729 08:31:50.791534 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pode283ad45_0159_43dd_a7ec_2be6de4297d5.slice/crio-27033e4f0a14af6caedd2df32bc1a1111a8f88fa35ea19e9ba7a21266d6d7a31 WatchSource:0}: Error finding container 27033e4f0a14af6caedd2df32bc1a1111a8f88fa35ea19e9ba7a21266d6d7a31: Status 404 returned error can't find the container with id 27033e4f0a14af6caedd2df32bc1a1111a8f88fa35ea19e9ba7a21266d6d7a31 Jul 29 08:31:51.489048 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:31:51.489010 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-6944688bc8-fhd6x" event={"ID":"e283ad45-0159-43dd-a7ec-2be6de4297d5","Type":"ContainerStarted","Data":"b82fd41b6379ca361e70a8acc8d781a9c7e7c7cea73f621004d5dbfd1bd45634"} Jul 29 08:31:51.489218 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:31:51.489062 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-6944688bc8-fhd6x" event={"ID":"e283ad45-0159-43dd-a7ec-2be6de4297d5","Type":"ContainerStarted","Data":"27033e4f0a14af6caedd2df32bc1a1111a8f88fa35ea19e9ba7a21266d6d7a31"} Jul 29 08:31:56.510955 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:31:56.510924 2563 generic.go:356] "Generic (PLEG): container finished" podID="e283ad45-0159-43dd-a7ec-2be6de4297d5" containerID="b82fd41b6379ca361e70a8acc8d781a9c7e7c7cea73f621004d5dbfd1bd45634" exitCode=0 Jul 29 08:31:56.511421 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:31:56.510978 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-6944688bc8-fhd6x" event={"ID":"e283ad45-0159-43dd-a7ec-2be6de4297d5","Type":"ContainerDied","Data":"b82fd41b6379ca361e70a8acc8d781a9c7e7c7cea73f621004d5dbfd1bd45634"} Jul 29 08:31:57.517235 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:31:57.517200 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-6944688bc8-fhd6x" event={"ID":"e283ad45-0159-43dd-a7ec-2be6de4297d5","Type":"ContainerStarted","Data":"f8707946bc3d75bacdea36c91ad4bcce99670af4a1767f6301de09d3fd969c9d"} Jul 29 08:32:03.543049 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:32:03.543010 2563 generic.go:356] "Generic (PLEG): container finished" podID="e283ad45-0159-43dd-a7ec-2be6de4297d5" containerID="f8707946bc3d75bacdea36c91ad4bcce99670af4a1767f6301de09d3fd969c9d" exitCode=0 Jul 29 08:32:03.543478 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:32:03.543089 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-6944688bc8-fhd6x" event={"ID":"e283ad45-0159-43dd-a7ec-2be6de4297d5","Type":"ContainerDied","Data":"f8707946bc3d75bacdea36c91ad4bcce99670af4a1767f6301de09d3fd969c9d"} Jul 29 08:32:04.553223 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:32:04.553183 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-6944688bc8-fhd6x" event={"ID":"e283ad45-0159-43dd-a7ec-2be6de4297d5","Type":"ContainerStarted","Data":"576279d0750abefc3fc27ea71fa9bf1d109e2168fdc2cd5e1d8431d059e3041b"} Jul 29 08:32:04.553763 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:32:04.553244 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-6944688bc8-fhd6x" event={"ID":"e283ad45-0159-43dd-a7ec-2be6de4297d5","Type":"ContainerStarted","Data":"01b0b27383ddb1bbb2b0d30371a7c042228f3b403e46fa57e8bcad567f4452d8"} Jul 29 08:32:04.553763 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:32:04.553292 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-6944688bc8-fhd6x" event={"ID":"e283ad45-0159-43dd-a7ec-2be6de4297d5","Type":"ContainerStarted","Data":"9825ac430725d413c408b45a211cf06bf7491f5d2511be1113104c950f49db91"} Jul 29 08:32:04.582911 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:32:04.582856 2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-6944688bc8-fhd6x" podStartSLOduration=15.582833886 podStartE2EDuration="15.582833886s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 08:31:49 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 08:32:04.580043761 +0000 UTC m=+637.161332541" watchObservedRunningTime="2026-07-29 08:32:04.582833886 +0000 UTC m=+637.164122663" Jul 29 08:32:05.648277 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:32:05.648215 2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-6944688bc8-fhd6x" Jul 29 08:32:05.648739 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:32:05.648301 2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-6944688bc8-fhd6x" Jul 29 08:32:05.648739 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:32:05.648321 2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-6944688bc8-fhd6x" Jul 29 08:32:05.649812 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:32:05.649732 2563 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-6944688bc8-fhd6x" podUID="e283ad45-0159-43dd-a7ec-2be6de4297d5" containerName="ui" probeResult="failure" output="dial tcp 10.134.0.21:8443: connect: connection refused" Jul 29 08:32:05.649812 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:32:05.649765 2563 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-6944688bc8-fhd6x" podUID="e283ad45-0159-43dd-a7ec-2be6de4297d5" containerName="offline" probeResult="failure" output="dial tcp 10.134.0.21:8816: connect: connection refused" Jul 29 08:32:05.650023 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:32:05.649772 2563 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-6944688bc8-fhd6x" podUID="e283ad45-0159-43dd-a7ec-2be6de4297d5" containerName="online" probeResult="failure" output="Get \"https://10.134.0.21:6567/health\": dial tcp 10.134.0.21:6567: connect: connection refused" Jul 29 08:32:08.648429 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:32:08.648382 2563 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-6944688bc8-fhd6x" podUID="e283ad45-0159-43dd-a7ec-2be6de4297d5" containerName="ui" probeResult="failure" output="dial tcp 10.134.0.21:8443: connect: connection refused" Jul 29 08:32:08.648854 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:32:08.648378 2563 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-6944688bc8-fhd6x" podUID="e283ad45-0159-43dd-a7ec-2be6de4297d5" containerName="offline" probeResult="failure" output="dial tcp 10.134.0.21:8816: connect: connection refused" Jul 29 08:32:08.648854 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:32:08.648469 2563 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-6944688bc8-fhd6x" podUID="e283ad45-0159-43dd-a7ec-2be6de4297d5" containerName="online" probeResult="failure" output="Get \"https://10.134.0.21:6567/health\": dial tcp 10.134.0.21:6567: connect: connection refused" Jul 29 08:32:10.648279 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:32:10.648214 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-6944688bc8-fhd6x" Jul 29 08:32:10.648279 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:32:10.648284 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-6944688bc8-fhd6x" Jul 29 08:32:10.650746 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:32:10.648297 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-6944688bc8-fhd6x" Jul 29 08:32:11.648566 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:32:11.648535 2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-6944688bc8-fhd6x" Jul 29 08:32:11.649040 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:32:11.648640 2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-6944688bc8-fhd6x" Jul 29 08:32:11.649380 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:32:11.649359 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-6944688bc8-fhd6x" Jul 29 08:32:11.649481 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:32:11.649417 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-6944688bc8-fhd6x" Jul 29 08:32:11.655304 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:32:11.655279 2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-6944688bc8-fhd6x" Jul 29 08:32:12.593699 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:32:12.593664 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-6944688bc8-fhd6x" Jul 29 08:32:12.944608 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:32:12.944524 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8rp9g_cfd1d8df-e6b7-4a67-81f2-5d83da541e20/konnectivity-agent/0.log" limit=1024 Jul 29 08:33:08.380731 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:33:08.380696 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8rp9g_cfd1d8df-e6b7-4a67-81f2-5d83da541e20/konnectivity-agent/0.log" limit=1024 Jul 29 08:33:15.008500 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:33:15.008464 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8rp9g_cfd1d8df-e6b7-4a67-81f2-5d83da541e20/konnectivity-agent/0.log" limit=1024 Jul 29 08:33:15.024102 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:33:15.024058 2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["test-ns-remote-registry/feast-simple-feast-remote-setup-6944688bc8-fhd6x"] Jul 29 08:33:15.024368 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:33:15.024335 2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-6944688bc8-fhd6x" podUID="e283ad45-0159-43dd-a7ec-2be6de4297d5" containerName="online" containerID="cri-o://9825ac430725d413c408b45a211cf06bf7491f5d2511be1113104c950f49db91" gracePeriod=30 Jul 29 08:33:15.024497 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:33:15.024396 2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-6944688bc8-fhd6x" podUID="e283ad45-0159-43dd-a7ec-2be6de4297d5" containerName="offline" containerID="cri-o://01b0b27383ddb1bbb2b0d30371a7c042228f3b403e46fa57e8bcad567f4452d8" gracePeriod=30 Jul 29 08:33:15.024559 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:33:15.024526 2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-6944688bc8-fhd6x" podUID="e283ad45-0159-43dd-a7ec-2be6de4297d5" containerName="ui" containerID="cri-o://576279d0750abefc3fc27ea71fa9bf1d109e2168fdc2cd5e1d8431d059e3041b" gracePeriod=30 Jul 29 08:33:15.112981 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:33:15.112946 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8rp9g_cfd1d8df-e6b7-4a67-81f2-5d83da541e20/konnectivity-agent/0.log" limit=1024 Jul 29 08:33:15.137929 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:33:15.137897 2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["test-ns-feast/feast-simple-feast-setup-57d7459654-769sh"] Jul 29 08:33:15.138221 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:33:15.138190 2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="test-ns-feast/feast-simple-feast-setup-57d7459654-769sh" podUID="562b5f5f-6c61-4618-b11e-41518922e7a6" containerName="registry" containerID="cri-o://799f32f92a307c4ebc64fd16ef84a1a5ba6f6b34cc71af2fa9184496756dde21" gracePeriod=30 Jul 29 08:33:15.138494 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:33:15.138246 2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="test-ns-feast/feast-simple-feast-setup-57d7459654-769sh" podUID="562b5f5f-6c61-4618-b11e-41518922e7a6" containerName="ui" containerID="cri-o://0a60ff42153cd7243a08db6d5b6e7fb1191cfd7ebf7b0d8d51f0a668bf60bd01" gracePeriod=30 Jul 29 08:33:15.138595 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:33:15.138246 2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="test-ns-feast/feast-simple-feast-setup-57d7459654-769sh" podUID="562b5f5f-6c61-4618-b11e-41518922e7a6" containerName="online" containerID="cri-o://b541318a41ca2d9895c30890b4850d0514fd543f57db33c885e5b6d82ba17687" gracePeriod=30 Jul 29 08:33:15.138666 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:33:15.138303 2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="test-ns-feast/feast-simple-feast-setup-57d7459654-769sh" podUID="562b5f5f-6c61-4618-b11e-41518922e7a6" containerName="offline" containerID="cri-o://c491f99363c3d4556759693b49458785e5e44635b52b0a40c0cff91ca24d0f44" gracePeriod=30 Jul 29 08:33:15.209237 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:33:15.209197 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8rp9g_cfd1d8df-e6b7-4a67-81f2-5d83da541e20/konnectivity-agent/0.log" limit=1024 Jul 29 08:33:15.267446 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:33:15.267359 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8rp9g_cfd1d8df-e6b7-4a67-81f2-5d83da541e20/konnectivity-agent/0.log" limit=1024 Jul 29 08:33:15.827662 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:33:15.827617 2563 generic.go:356] "Generic (PLEG): container finished" podID="e283ad45-0159-43dd-a7ec-2be6de4297d5" containerID="576279d0750abefc3fc27ea71fa9bf1d109e2168fdc2cd5e1d8431d059e3041b" exitCode=0 Jul 29 08:33:15.827796 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:33:15.827688 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-6944688bc8-fhd6x" event={"ID":"e283ad45-0159-43dd-a7ec-2be6de4297d5","Type":"ContainerDied","Data":"576279d0750abefc3fc27ea71fa9bf1d109e2168fdc2cd5e1d8431d059e3041b"} Jul 29 08:33:15.831024 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:33:15.830996 2563 generic.go:356] "Generic (PLEG): container finished" podID="562b5f5f-6c61-4618-b11e-41518922e7a6" containerID="0a60ff42153cd7243a08db6d5b6e7fb1191cfd7ebf7b0d8d51f0a668bf60bd01" exitCode=0 Jul 29 08:33:15.831129 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:33:15.831056 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-57d7459654-769sh" event={"ID":"562b5f5f-6c61-4618-b11e-41518922e7a6","Type":"ContainerDied","Data":"0a60ff42153cd7243a08db6d5b6e7fb1191cfd7ebf7b0d8d51f0a668bf60bd01"} Jul 29 08:33:16.837449 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:33:16.837418 2563 generic.go:356] "Generic (PLEG): container finished" podID="e283ad45-0159-43dd-a7ec-2be6de4297d5" containerID="9825ac430725d413c408b45a211cf06bf7491f5d2511be1113104c950f49db91" exitCode=0 Jul 29 08:33:16.837850 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:33:16.837487 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-6944688bc8-fhd6x" event={"ID":"e283ad45-0159-43dd-a7ec-2be6de4297d5","Type":"ContainerDied","Data":"9825ac430725d413c408b45a211cf06bf7491f5d2511be1113104c950f49db91"} Jul 29 08:33:16.840111 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:33:16.840088 2563 generic.go:356] "Generic (PLEG): container finished" podID="562b5f5f-6c61-4618-b11e-41518922e7a6" containerID="b541318a41ca2d9895c30890b4850d0514fd543f57db33c885e5b6d82ba17687" exitCode=0 Jul 29 08:33:16.840226 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:33:16.840134 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-57d7459654-769sh" event={"ID":"562b5f5f-6c61-4618-b11e-41518922e7a6","Type":"ContainerDied","Data":"b541318a41ca2d9895c30890b4850d0514fd543f57db33c885e5b6d82ba17687"} Jul 29 08:33:16.958623 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:33:16.958571 2563 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-57d7459654-769sh" podUID="562b5f5f-6c61-4618-b11e-41518922e7a6" containerName="ui" probeResult="failure" output="dial tcp 10.134.0.20:8443: connect: connection refused" Jul 29 08:33:17.233108 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:33:17.233072 2563 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-57d7459654-769sh" podUID="562b5f5f-6c61-4618-b11e-41518922e7a6" containerName="online" probeResult="failure" output="Get \"https://10.134.0.20:6567/health\": dial tcp 10.134.0.20:6567: connect: connection refused" Jul 29 08:33:20.302700 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:33:20.302609 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8rp9g_cfd1d8df-e6b7-4a67-81f2-5d83da541e20/konnectivity-agent/0.log" limit=1024 Jul 29 08:33:21.649367 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:33:21.649323 2563 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-6944688bc8-fhd6x" podUID="e283ad45-0159-43dd-a7ec-2be6de4297d5" containerName="ui" probeResult="failure" output="dial tcp 10.134.0.21:8443: connect: connection refused" Jul 29 08:33:22.590175 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:33:22.590126 2563 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-6944688bc8-fhd6x" podUID="e283ad45-0159-43dd-a7ec-2be6de4297d5" containerName="online" probeResult="failure" output="Get \"https://10.134.0.21:6567/health\": dial tcp 10.134.0.21:6567: connect: connection refused" Jul 29 08:33:26.958489 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:33:26.958441 2563 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-57d7459654-769sh" podUID="562b5f5f-6c61-4618-b11e-41518922e7a6" containerName="ui" probeResult="failure" output="dial tcp 10.134.0.20:8443: connect: connection refused" Jul 29 08:33:27.233325 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:33:27.233222 2563 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-57d7459654-769sh" podUID="562b5f5f-6c61-4618-b11e-41518922e7a6" containerName="online" probeResult="failure" output="Get \"https://10.134.0.20:6567/health\": dial tcp 10.134.0.20:6567: connect: connection refused" Jul 29 08:33:31.649363 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:33:31.649318 2563 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-6944688bc8-fhd6x" podUID="e283ad45-0159-43dd-a7ec-2be6de4297d5" containerName="ui" probeResult="failure" output="dial tcp 10.134.0.21:8443: connect: connection refused" Jul 29 08:33:32.589467 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:33:32.589418 2563 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-6944688bc8-fhd6x" podUID="e283ad45-0159-43dd-a7ec-2be6de4297d5" containerName="online" probeResult="failure" output="Get \"https://10.134.0.21:6567/health\": dial tcp 10.134.0.21:6567: connect: connection refused" Jul 29 08:33:36.958960 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:33:36.958910 2563 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-57d7459654-769sh" podUID="562b5f5f-6c61-4618-b11e-41518922e7a6" containerName="ui" probeResult="failure" output="dial tcp 10.134.0.20:8443: connect: connection refused" Jul 29 08:33:36.959472 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:33:36.959077 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-57d7459654-769sh" Jul 29 08:33:37.232438 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:33:37.232349 2563 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-57d7459654-769sh" podUID="562b5f5f-6c61-4618-b11e-41518922e7a6" containerName="online" probeResult="failure" output="Get \"https://10.134.0.20:6567/health\": dial tcp 10.134.0.20:6567: connect: connection refused" Jul 29 08:33:37.232596 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:33:37.232465 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-57d7459654-769sh" Jul 29 08:33:41.650069 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:33:41.650021 2563 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-6944688bc8-fhd6x" podUID="e283ad45-0159-43dd-a7ec-2be6de4297d5" containerName="ui" probeResult="failure" output="dial tcp 10.134.0.21:8443: connect: connection refused" Jul 29 08:33:41.650532 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:33:41.650159 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-6944688bc8-fhd6x" Jul 29 08:33:42.590070 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:33:42.590024 2563 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-6944688bc8-fhd6x" podUID="e283ad45-0159-43dd-a7ec-2be6de4297d5" containerName="online" probeResult="failure" output="Get \"https://10.134.0.21:6567/health\": dial tcp 10.134.0.21:6567: connect: connection refused" Jul 29 08:33:42.590278 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:33:42.590156 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-6944688bc8-fhd6x" Jul 29 08:33:45.681823 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:33:45.681798 2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-6944688bc8-fhd6x" Jul 29 08:33:45.773318 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:33:45.773289 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/configmap/e283ad45-0159-43dd-a7ec-2be6de4297d5-registry-tls\") pod \"e283ad45-0159-43dd-a7ec-2be6de4297d5\" (UID: \"e283ad45-0159-43dd-a7ec-2be6de4297d5\") " Jul 29 08:33:45.773494 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:33:45.773339 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/e283ad45-0159-43dd-a7ec-2be6de4297d5-feast-data\") pod \"e283ad45-0159-43dd-a7ec-2be6de4297d5\" (UID: \"e283ad45-0159-43dd-a7ec-2be6de4297d5\") " Jul 29 08:33:45.773494 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:33:45.773359 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/e283ad45-0159-43dd-a7ec-2be6de4297d5-ui-tls\") pod \"e283ad45-0159-43dd-a7ec-2be6de4297d5\" (UID: \"e283ad45-0159-43dd-a7ec-2be6de4297d5\") " Jul 29 08:33:45.773494 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:33:45.773404 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-gdzc2\" (UniqueName: \"kubernetes.io/projected/e283ad45-0159-43dd-a7ec-2be6de4297d5-kube-api-access-gdzc2\") pod \"e283ad45-0159-43dd-a7ec-2be6de4297d5\" (UID: \"e283ad45-0159-43dd-a7ec-2be6de4297d5\") " Jul 29 08:33:45.773494 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:33:45.773466 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/e283ad45-0159-43dd-a7ec-2be6de4297d5-offline-tls\") pod \"e283ad45-0159-43dd-a7ec-2be6de4297d5\" (UID: \"e283ad45-0159-43dd-a7ec-2be6de4297d5\") " Jul 29 08:33:45.773700 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:33:45.773513 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/e283ad45-0159-43dd-a7ec-2be6de4297d5-online-tls\") pod \"e283ad45-0159-43dd-a7ec-2be6de4297d5\" (UID: \"e283ad45-0159-43dd-a7ec-2be6de4297d5\") " Jul 29 08:33:45.773756 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:33:45.773704 2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e283ad45-0159-43dd-a7ec-2be6de4297d5-registry-tls" (OuterVolumeSpecName: "registry-tls") pod "e283ad45-0159-43dd-a7ec-2be6de4297d5" (UID: "e283ad45-0159-43dd-a7ec-2be6de4297d5"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 29 08:33:45.773934 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:33:45.773845 2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e283ad45-0159-43dd-a7ec-2be6de4297d5-feast-data" (OuterVolumeSpecName: "feast-data") pod "e283ad45-0159-43dd-a7ec-2be6de4297d5" (UID: "e283ad45-0159-43dd-a7ec-2be6de4297d5"). InnerVolumeSpecName "feast-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 29 08:33:45.775933 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:33:45.775904 2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e283ad45-0159-43dd-a7ec-2be6de4297d5-offline-tls" (OuterVolumeSpecName: "offline-tls") pod "e283ad45-0159-43dd-a7ec-2be6de4297d5" (UID: "e283ad45-0159-43dd-a7ec-2be6de4297d5"). InnerVolumeSpecName "offline-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 29 08:33:45.776019 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:33:45.775942 2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e283ad45-0159-43dd-a7ec-2be6de4297d5-kube-api-access-gdzc2" (OuterVolumeSpecName: "kube-api-access-gdzc2") pod "e283ad45-0159-43dd-a7ec-2be6de4297d5" (UID: "e283ad45-0159-43dd-a7ec-2be6de4297d5"). InnerVolumeSpecName "kube-api-access-gdzc2". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 29 08:33:45.776059 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:33:45.776042 2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e283ad45-0159-43dd-a7ec-2be6de4297d5-ui-tls" (OuterVolumeSpecName: "ui-tls") pod "e283ad45-0159-43dd-a7ec-2be6de4297d5" (UID: "e283ad45-0159-43dd-a7ec-2be6de4297d5"). InnerVolumeSpecName "ui-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 29 08:33:45.776149 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:33:45.776119 2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e283ad45-0159-43dd-a7ec-2be6de4297d5-online-tls" (OuterVolumeSpecName: "online-tls") pod "e283ad45-0159-43dd-a7ec-2be6de4297d5" (UID: "e283ad45-0159-43dd-a7ec-2be6de4297d5"). InnerVolumeSpecName "online-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 29 08:33:45.793031 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:33:45.792978 2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/feast-simple-feast-setup-57d7459654-769sh" Jul 29 08:33:45.874843 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:33:45.874817 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/562b5f5f-6c61-4618-b11e-41518922e7a6-offline-tls\") pod \"562b5f5f-6c61-4618-b11e-41518922e7a6\" (UID: \"562b5f5f-6c61-4618-b11e-41518922e7a6\") " Jul 29 08:33:45.874979 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:33:45.874876 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/562b5f5f-6c61-4618-b11e-41518922e7a6-online-tls\") pod \"562b5f5f-6c61-4618-b11e-41518922e7a6\" (UID: \"562b5f5f-6c61-4618-b11e-41518922e7a6\") " Jul 29 08:33:45.874979 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:33:45.874903 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/562b5f5f-6c61-4618-b11e-41518922e7a6-feast-data\") pod \"562b5f5f-6c61-4618-b11e-41518922e7a6\" (UID: \"562b5f5f-6c61-4618-b11e-41518922e7a6\") " Jul 29 08:33:45.874979 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:33:45.874937 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/562b5f5f-6c61-4618-b11e-41518922e7a6-ui-tls\") pod \"562b5f5f-6c61-4618-b11e-41518922e7a6\" (UID: \"562b5f5f-6c61-4618-b11e-41518922e7a6\") " Jul 29 08:33:45.874979 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:33:45.874968 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/562b5f5f-6c61-4618-b11e-41518922e7a6-registry-tls\") pod \"562b5f5f-6c61-4618-b11e-41518922e7a6\" (UID: \"562b5f5f-6c61-4618-b11e-41518922e7a6\") " Jul 29 08:33:45.875186 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:33:45.875002 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-kkx4w\" (UniqueName: \"kubernetes.io/projected/562b5f5f-6c61-4618-b11e-41518922e7a6-kube-api-access-kkx4w\") pod \"562b5f5f-6c61-4618-b11e-41518922e7a6\" (UID: \"562b5f5f-6c61-4618-b11e-41518922e7a6\") " Jul 29 08:33:45.875246 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:33:45.875231 2563 reconciler_common.go:299] "Volume detached for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/e283ad45-0159-43dd-a7ec-2be6de4297d5-offline-tls\") on node \"ip-10-0-131-138.ec2.internal\" DevicePath \"\"" Jul 29 08:33:45.875335 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:33:45.875275 2563 reconciler_common.go:299] "Volume detached for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/e283ad45-0159-43dd-a7ec-2be6de4297d5-online-tls\") on node \"ip-10-0-131-138.ec2.internal\" DevicePath \"\"" Jul 29 08:33:45.875335 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:33:45.875290 2563 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/configmap/e283ad45-0159-43dd-a7ec-2be6de4297d5-registry-tls\") on node \"ip-10-0-131-138.ec2.internal\" DevicePath \"\"" Jul 29 08:33:45.875335 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:33:45.875302 2563 reconciler_common.go:299] "Volume detached for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/e283ad45-0159-43dd-a7ec-2be6de4297d5-feast-data\") on node \"ip-10-0-131-138.ec2.internal\" DevicePath \"\"" Jul 29 08:33:45.875335 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:33:45.875314 2563 reconciler_common.go:299] "Volume detached for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/e283ad45-0159-43dd-a7ec-2be6de4297d5-ui-tls\") on node \"ip-10-0-131-138.ec2.internal\" DevicePath \"\"" Jul 29 08:33:45.875335 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:33:45.875328 2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-gdzc2\" (UniqueName: \"kubernetes.io/projected/e283ad45-0159-43dd-a7ec-2be6de4297d5-kube-api-access-gdzc2\") on node \"ip-10-0-131-138.ec2.internal\" DevicePath \"\"" Jul 29 08:33:45.875574 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:33:45.875484 2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/562b5f5f-6c61-4618-b11e-41518922e7a6-feast-data" (OuterVolumeSpecName: "feast-data") pod "562b5f5f-6c61-4618-b11e-41518922e7a6" (UID: "562b5f5f-6c61-4618-b11e-41518922e7a6"). InnerVolumeSpecName "feast-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 29 08:33:45.876981 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:33:45.876959 2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/562b5f5f-6c61-4618-b11e-41518922e7a6-online-tls" (OuterVolumeSpecName: "online-tls") pod "562b5f5f-6c61-4618-b11e-41518922e7a6" (UID: "562b5f5f-6c61-4618-b11e-41518922e7a6"). InnerVolumeSpecName "online-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 29 08:33:45.877118 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:33:45.877103 2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/562b5f5f-6c61-4618-b11e-41518922e7a6-ui-tls" (OuterVolumeSpecName: "ui-tls") pod "562b5f5f-6c61-4618-b11e-41518922e7a6" (UID: "562b5f5f-6c61-4618-b11e-41518922e7a6"). InnerVolumeSpecName "ui-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 29 08:33:45.877243 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:33:45.877222 2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/562b5f5f-6c61-4618-b11e-41518922e7a6-kube-api-access-kkx4w" (OuterVolumeSpecName: "kube-api-access-kkx4w") pod "562b5f5f-6c61-4618-b11e-41518922e7a6" (UID: "562b5f5f-6c61-4618-b11e-41518922e7a6"). InnerVolumeSpecName "kube-api-access-kkx4w". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 29 08:33:45.877526 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:33:45.877498 2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/562b5f5f-6c61-4618-b11e-41518922e7a6-registry-tls" (OuterVolumeSpecName: "registry-tls") pod "562b5f5f-6c61-4618-b11e-41518922e7a6" (UID: "562b5f5f-6c61-4618-b11e-41518922e7a6"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 29 08:33:45.877526 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:33:45.877508 2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/562b5f5f-6c61-4618-b11e-41518922e7a6-offline-tls" (OuterVolumeSpecName: "offline-tls") pod "562b5f5f-6c61-4618-b11e-41518922e7a6" (UID: "562b5f5f-6c61-4618-b11e-41518922e7a6"). InnerVolumeSpecName "offline-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 29 08:33:45.950495 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:33:45.950463 2563 generic.go:356] "Generic (PLEG): container finished" podID="e283ad45-0159-43dd-a7ec-2be6de4297d5" containerID="01b0b27383ddb1bbb2b0d30371a7c042228f3b403e46fa57e8bcad567f4452d8" exitCode=137 Jul 29 08:33:45.950663 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:33:45.950548 2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-6944688bc8-fhd6x" Jul 29 08:33:45.950663 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:33:45.950555 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-6944688bc8-fhd6x" event={"ID":"e283ad45-0159-43dd-a7ec-2be6de4297d5","Type":"ContainerDied","Data":"01b0b27383ddb1bbb2b0d30371a7c042228f3b403e46fa57e8bcad567f4452d8"} Jul 29 08:33:45.950663 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:33:45.950603 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-6944688bc8-fhd6x" event={"ID":"e283ad45-0159-43dd-a7ec-2be6de4297d5","Type":"ContainerDied","Data":"27033e4f0a14af6caedd2df32bc1a1111a8f88fa35ea19e9ba7a21266d6d7a31"} Jul 29 08:33:45.950663 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:33:45.950630 2563 scope.go:117] "RemoveContainer" containerID="576279d0750abefc3fc27ea71fa9bf1d109e2168fdc2cd5e1d8431d059e3041b" Jul 29 08:33:45.953726 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:33:45.953704 2563 generic.go:356] "Generic (PLEG): container finished" podID="562b5f5f-6c61-4618-b11e-41518922e7a6" containerID="c491f99363c3d4556759693b49458785e5e44635b52b0a40c0cff91ca24d0f44" exitCode=137 Jul 29 08:33:45.953726 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:33:45.953728 2563 generic.go:356] "Generic (PLEG): container finished" podID="562b5f5f-6c61-4618-b11e-41518922e7a6" containerID="799f32f92a307c4ebc64fd16ef84a1a5ba6f6b34cc71af2fa9184496756dde21" exitCode=137 Jul 29 08:33:45.953874 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:33:45.953735 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-57d7459654-769sh" event={"ID":"562b5f5f-6c61-4618-b11e-41518922e7a6","Type":"ContainerDied","Data":"c491f99363c3d4556759693b49458785e5e44635b52b0a40c0cff91ca24d0f44"} Jul 29 08:33:45.953874 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:33:45.953775 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-57d7459654-769sh" event={"ID":"562b5f5f-6c61-4618-b11e-41518922e7a6","Type":"ContainerDied","Data":"799f32f92a307c4ebc64fd16ef84a1a5ba6f6b34cc71af2fa9184496756dde21"} Jul 29 08:33:45.953874 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:33:45.953793 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-57d7459654-769sh" event={"ID":"562b5f5f-6c61-4618-b11e-41518922e7a6","Type":"ContainerDied","Data":"46db27c2018b716e1b70f109e48d14fd9dfccf47cfe80a7567430aca9f6da44b"} Jul 29 08:33:45.953874 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:33:45.953805 2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/feast-simple-feast-setup-57d7459654-769sh" Jul 29 08:33:45.962117 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:33:45.961741 2563 scope.go:117] "RemoveContainer" containerID="01b0b27383ddb1bbb2b0d30371a7c042228f3b403e46fa57e8bcad567f4452d8" Jul 29 08:33:45.971837 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:33:45.971820 2563 scope.go:117] "RemoveContainer" containerID="9825ac430725d413c408b45a211cf06bf7491f5d2511be1113104c950f49db91" Jul 29 08:33:45.975888 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:33:45.975870 2563 reconciler_common.go:299] "Volume detached for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/562b5f5f-6c61-4618-b11e-41518922e7a6-offline-tls\") on node \"ip-10-0-131-138.ec2.internal\" DevicePath \"\"" Jul 29 08:33:45.975980 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:33:45.975911 2563 reconciler_common.go:299] "Volume detached for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/562b5f5f-6c61-4618-b11e-41518922e7a6-online-tls\") on node \"ip-10-0-131-138.ec2.internal\" DevicePath \"\"" Jul 29 08:33:45.975980 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:33:45.975924 2563 reconciler_common.go:299] "Volume detached for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/562b5f5f-6c61-4618-b11e-41518922e7a6-feast-data\") on node \"ip-10-0-131-138.ec2.internal\" DevicePath \"\"" Jul 29 08:33:45.975980 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:33:45.975936 2563 reconciler_common.go:299] "Volume detached for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/562b5f5f-6c61-4618-b11e-41518922e7a6-ui-tls\") on node \"ip-10-0-131-138.ec2.internal\" DevicePath \"\"" Jul 29 08:33:45.975980 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:33:45.975948 2563 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/562b5f5f-6c61-4618-b11e-41518922e7a6-registry-tls\") on node \"ip-10-0-131-138.ec2.internal\" DevicePath \"\"" Jul 29 08:33:45.975980 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:33:45.975960 2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-kkx4w\" (UniqueName: \"kubernetes.io/projected/562b5f5f-6c61-4618-b11e-41518922e7a6-kube-api-access-kkx4w\") on node \"ip-10-0-131-138.ec2.internal\" DevicePath \"\"" Jul 29 08:33:45.979089 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:33:45.979072 2563 scope.go:117] "RemoveContainer" containerID="f8707946bc3d75bacdea36c91ad4bcce99670af4a1767f6301de09d3fd969c9d" Jul 29 08:33:45.984593 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:33:45.984574 2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["test-ns-remote-registry/feast-simple-feast-remote-setup-6944688bc8-fhd6x"] Jul 29 08:33:45.986519 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:33:45.986477 2563 scope.go:117] "RemoveContainer" containerID="b82fd41b6379ca361e70a8acc8d781a9c7e7c7cea73f621004d5dbfd1bd45634" Jul 29 08:33:45.988347 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:33:45.988326 2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["test-ns-remote-registry/feast-simple-feast-remote-setup-6944688bc8-fhd6x"] Jul 29 08:33:45.993966 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:33:45.993952 2563 scope.go:117] "RemoveContainer" containerID="576279d0750abefc3fc27ea71fa9bf1d109e2168fdc2cd5e1d8431d059e3041b" Jul 29 08:33:45.994213 ip-10-0-131-138 kubenswrapper[2563]: E0729 08:33:45.994192 2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"576279d0750abefc3fc27ea71fa9bf1d109e2168fdc2cd5e1d8431d059e3041b\": container with ID starting with 576279d0750abefc3fc27ea71fa9bf1d109e2168fdc2cd5e1d8431d059e3041b not found: ID does not exist" containerID="576279d0750abefc3fc27ea71fa9bf1d109e2168fdc2cd5e1d8431d059e3041b" Jul 29 08:33:45.994293 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:33:45.994225 2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"576279d0750abefc3fc27ea71fa9bf1d109e2168fdc2cd5e1d8431d059e3041b"} err="failed to get container status \"576279d0750abefc3fc27ea71fa9bf1d109e2168fdc2cd5e1d8431d059e3041b\": rpc error: code = NotFound desc = could not find container \"576279d0750abefc3fc27ea71fa9bf1d109e2168fdc2cd5e1d8431d059e3041b\": container with ID starting with 576279d0750abefc3fc27ea71fa9bf1d109e2168fdc2cd5e1d8431d059e3041b not found: ID does not exist" Jul 29 08:33:45.994293 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:33:45.994269 2563 scope.go:117] "RemoveContainer" containerID="01b0b27383ddb1bbb2b0d30371a7c042228f3b403e46fa57e8bcad567f4452d8" Jul 29 08:33:45.994492 ip-10-0-131-138 kubenswrapper[2563]: E0729 08:33:45.994477 2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"01b0b27383ddb1bbb2b0d30371a7c042228f3b403e46fa57e8bcad567f4452d8\": container with ID starting with 01b0b27383ddb1bbb2b0d30371a7c042228f3b403e46fa57e8bcad567f4452d8 not found: ID does not exist" containerID="01b0b27383ddb1bbb2b0d30371a7c042228f3b403e46fa57e8bcad567f4452d8" Jul 29 08:33:45.994530 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:33:45.994497 2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"01b0b27383ddb1bbb2b0d30371a7c042228f3b403e46fa57e8bcad567f4452d8"} err="failed to get container status \"01b0b27383ddb1bbb2b0d30371a7c042228f3b403e46fa57e8bcad567f4452d8\": rpc error: code = NotFound desc = could not find container \"01b0b27383ddb1bbb2b0d30371a7c042228f3b403e46fa57e8bcad567f4452d8\": container with ID starting with 01b0b27383ddb1bbb2b0d30371a7c042228f3b403e46fa57e8bcad567f4452d8 not found: ID does not exist" Jul 29 08:33:45.994530 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:33:45.994511 2563 scope.go:117] "RemoveContainer" containerID="9825ac430725d413c408b45a211cf06bf7491f5d2511be1113104c950f49db91" Jul 29 08:33:45.994736 ip-10-0-131-138 kubenswrapper[2563]: E0729 08:33:45.994721 2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9825ac430725d413c408b45a211cf06bf7491f5d2511be1113104c950f49db91\": container with ID starting with 9825ac430725d413c408b45a211cf06bf7491f5d2511be1113104c950f49db91 not found: ID does not exist" containerID="9825ac430725d413c408b45a211cf06bf7491f5d2511be1113104c950f49db91" Jul 29 08:33:45.994774 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:33:45.994750 2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9825ac430725d413c408b45a211cf06bf7491f5d2511be1113104c950f49db91"} err="failed to get container status \"9825ac430725d413c408b45a211cf06bf7491f5d2511be1113104c950f49db91\": rpc error: code = NotFound desc = could not find container \"9825ac430725d413c408b45a211cf06bf7491f5d2511be1113104c950f49db91\": container with ID starting with 9825ac430725d413c408b45a211cf06bf7491f5d2511be1113104c950f49db91 not found: ID does not exist" Jul 29 08:33:45.994774 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:33:45.994764 2563 scope.go:117] "RemoveContainer" containerID="f8707946bc3d75bacdea36c91ad4bcce99670af4a1767f6301de09d3fd969c9d" Jul 29 08:33:45.994967 ip-10-0-131-138 kubenswrapper[2563]: E0729 08:33:45.994953 2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f8707946bc3d75bacdea36c91ad4bcce99670af4a1767f6301de09d3fd969c9d\": container with ID starting with f8707946bc3d75bacdea36c91ad4bcce99670af4a1767f6301de09d3fd969c9d not found: ID does not exist" containerID="f8707946bc3d75bacdea36c91ad4bcce99670af4a1767f6301de09d3fd969c9d" Jul 29 08:33:45.994999 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:33:45.994972 2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f8707946bc3d75bacdea36c91ad4bcce99670af4a1767f6301de09d3fd969c9d"} err="failed to get container status \"f8707946bc3d75bacdea36c91ad4bcce99670af4a1767f6301de09d3fd969c9d\": rpc error: code = NotFound desc = could not find container \"f8707946bc3d75bacdea36c91ad4bcce99670af4a1767f6301de09d3fd969c9d\": container with ID starting with f8707946bc3d75bacdea36c91ad4bcce99670af4a1767f6301de09d3fd969c9d not found: ID does not exist" Jul 29 08:33:45.994999 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:33:45.994985 2563 scope.go:117] "RemoveContainer" containerID="b82fd41b6379ca361e70a8acc8d781a9c7e7c7cea73f621004d5dbfd1bd45634" Jul 29 08:33:45.995202 ip-10-0-131-138 kubenswrapper[2563]: E0729 08:33:45.995187 2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b82fd41b6379ca361e70a8acc8d781a9c7e7c7cea73f621004d5dbfd1bd45634\": container with ID starting with b82fd41b6379ca361e70a8acc8d781a9c7e7c7cea73f621004d5dbfd1bd45634 not found: ID does not exist" containerID="b82fd41b6379ca361e70a8acc8d781a9c7e7c7cea73f621004d5dbfd1bd45634" Jul 29 08:33:45.995237 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:33:45.995205 2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b82fd41b6379ca361e70a8acc8d781a9c7e7c7cea73f621004d5dbfd1bd45634"} err="failed to get container status \"b82fd41b6379ca361e70a8acc8d781a9c7e7c7cea73f621004d5dbfd1bd45634\": rpc error: code = NotFound desc = could not find container \"b82fd41b6379ca361e70a8acc8d781a9c7e7c7cea73f621004d5dbfd1bd45634\": container with ID starting with b82fd41b6379ca361e70a8acc8d781a9c7e7c7cea73f621004d5dbfd1bd45634 not found: ID does not exist" Jul 29 08:33:45.995237 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:33:45.995215 2563 scope.go:117] "RemoveContainer" containerID="0a60ff42153cd7243a08db6d5b6e7fb1191cfd7ebf7b0d8d51f0a668bf60bd01" Jul 29 08:33:46.002536 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:33:46.002462 2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["test-ns-feast/feast-simple-feast-setup-57d7459654-769sh"] Jul 29 08:33:46.002589 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:33:46.002549 2563 scope.go:117] "RemoveContainer" containerID="c491f99363c3d4556759693b49458785e5e44635b52b0a40c0cff91ca24d0f44" Jul 29 08:33:46.008432 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:33:46.008412 2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["test-ns-feast/feast-simple-feast-setup-57d7459654-769sh"] Jul 29 08:33:46.010054 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:33:46.010040 2563 scope.go:117] "RemoveContainer" containerID="b541318a41ca2d9895c30890b4850d0514fd543f57db33c885e5b6d82ba17687" Jul 29 08:33:46.016649 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:33:46.016632 2563 scope.go:117] "RemoveContainer" containerID="799f32f92a307c4ebc64fd16ef84a1a5ba6f6b34cc71af2fa9184496756dde21" Jul 29 08:33:46.023395 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:33:46.023378 2563 scope.go:117] "RemoveContainer" containerID="5eeb177ee807a924207351fcfc9934115876f7f210102e091191b7a93c4b0117" Jul 29 08:33:46.029972 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:33:46.029806 2563 scope.go:117] "RemoveContainer" containerID="0d3a1df021e0ed10c199857cdacadaad9cb46533321cc945ef88989f80364b01" Jul 29 08:33:46.044013 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:33:46.043999 2563 scope.go:117] "RemoveContainer" containerID="0a60ff42153cd7243a08db6d5b6e7fb1191cfd7ebf7b0d8d51f0a668bf60bd01" Jul 29 08:33:46.044243 ip-10-0-131-138 kubenswrapper[2563]: E0729 08:33:46.044227 2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0a60ff42153cd7243a08db6d5b6e7fb1191cfd7ebf7b0d8d51f0a668bf60bd01\": container with ID starting with 0a60ff42153cd7243a08db6d5b6e7fb1191cfd7ebf7b0d8d51f0a668bf60bd01 not found: ID does not exist" containerID="0a60ff42153cd7243a08db6d5b6e7fb1191cfd7ebf7b0d8d51f0a668bf60bd01" Jul 29 08:33:46.044303 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:33:46.044249 2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0a60ff42153cd7243a08db6d5b6e7fb1191cfd7ebf7b0d8d51f0a668bf60bd01"} err="failed to get container status \"0a60ff42153cd7243a08db6d5b6e7fb1191cfd7ebf7b0d8d51f0a668bf60bd01\": rpc error: code = NotFound desc = could not find container \"0a60ff42153cd7243a08db6d5b6e7fb1191cfd7ebf7b0d8d51f0a668bf60bd01\": container with ID starting with 0a60ff42153cd7243a08db6d5b6e7fb1191cfd7ebf7b0d8d51f0a668bf60bd01 not found: ID does not exist" Jul 29 08:33:46.044303 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:33:46.044283 2563 scope.go:117] "RemoveContainer" containerID="c491f99363c3d4556759693b49458785e5e44635b52b0a40c0cff91ca24d0f44" Jul 29 08:33:46.044521 ip-10-0-131-138 kubenswrapper[2563]: E0729 08:33:46.044503 2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c491f99363c3d4556759693b49458785e5e44635b52b0a40c0cff91ca24d0f44\": container with ID starting with c491f99363c3d4556759693b49458785e5e44635b52b0a40c0cff91ca24d0f44 not found: ID does not exist" containerID="c491f99363c3d4556759693b49458785e5e44635b52b0a40c0cff91ca24d0f44" Jul 29 08:33:46.044560 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:33:46.044525 2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c491f99363c3d4556759693b49458785e5e44635b52b0a40c0cff91ca24d0f44"} err="failed to get container status \"c491f99363c3d4556759693b49458785e5e44635b52b0a40c0cff91ca24d0f44\": rpc error: code = NotFound desc = could not find container \"c491f99363c3d4556759693b49458785e5e44635b52b0a40c0cff91ca24d0f44\": container with ID starting with c491f99363c3d4556759693b49458785e5e44635b52b0a40c0cff91ca24d0f44 not found: ID does not exist" Jul 29 08:33:46.044560 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:33:46.044540 2563 scope.go:117] "RemoveContainer" containerID="b541318a41ca2d9895c30890b4850d0514fd543f57db33c885e5b6d82ba17687" Jul 29 08:33:46.044727 ip-10-0-131-138 kubenswrapper[2563]: E0729 08:33:46.044710 2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b541318a41ca2d9895c30890b4850d0514fd543f57db33c885e5b6d82ba17687\": container with ID starting with b541318a41ca2d9895c30890b4850d0514fd543f57db33c885e5b6d82ba17687 not found: ID does not exist" containerID="b541318a41ca2d9895c30890b4850d0514fd543f57db33c885e5b6d82ba17687" Jul 29 08:33:46.044795 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:33:46.044732 2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b541318a41ca2d9895c30890b4850d0514fd543f57db33c885e5b6d82ba17687"} err="failed to get container status \"b541318a41ca2d9895c30890b4850d0514fd543f57db33c885e5b6d82ba17687\": rpc error: code = NotFound desc = could not find container \"b541318a41ca2d9895c30890b4850d0514fd543f57db33c885e5b6d82ba17687\": container with ID starting with b541318a41ca2d9895c30890b4850d0514fd543f57db33c885e5b6d82ba17687 not found: ID does not exist" Jul 29 08:33:46.044795 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:33:46.044746 2563 scope.go:117] "RemoveContainer" containerID="799f32f92a307c4ebc64fd16ef84a1a5ba6f6b34cc71af2fa9184496756dde21" Jul 29 08:33:46.044922 ip-10-0-131-138 kubenswrapper[2563]: E0729 08:33:46.044908 2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"799f32f92a307c4ebc64fd16ef84a1a5ba6f6b34cc71af2fa9184496756dde21\": container with ID starting with 799f32f92a307c4ebc64fd16ef84a1a5ba6f6b34cc71af2fa9184496756dde21 not found: ID does not exist" containerID="799f32f92a307c4ebc64fd16ef84a1a5ba6f6b34cc71af2fa9184496756dde21" Jul 29 08:33:46.044958 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:33:46.044926 2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"799f32f92a307c4ebc64fd16ef84a1a5ba6f6b34cc71af2fa9184496756dde21"} err="failed to get container status \"799f32f92a307c4ebc64fd16ef84a1a5ba6f6b34cc71af2fa9184496756dde21\": rpc error: code = NotFound desc = could not find container \"799f32f92a307c4ebc64fd16ef84a1a5ba6f6b34cc71af2fa9184496756dde21\": container with ID starting with 799f32f92a307c4ebc64fd16ef84a1a5ba6f6b34cc71af2fa9184496756dde21 not found: ID does not exist" Jul 29 08:33:46.044958 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:33:46.044939 2563 scope.go:117] "RemoveContainer" containerID="5eeb177ee807a924207351fcfc9934115876f7f210102e091191b7a93c4b0117" Jul 29 08:33:46.045164 ip-10-0-131-138 kubenswrapper[2563]: E0729 08:33:46.045147 2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5eeb177ee807a924207351fcfc9934115876f7f210102e091191b7a93c4b0117\": container with ID starting with 5eeb177ee807a924207351fcfc9934115876f7f210102e091191b7a93c4b0117 not found: ID does not exist" containerID="5eeb177ee807a924207351fcfc9934115876f7f210102e091191b7a93c4b0117" Jul 29 08:33:46.045214 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:33:46.045167 2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5eeb177ee807a924207351fcfc9934115876f7f210102e091191b7a93c4b0117"} err="failed to get container status \"5eeb177ee807a924207351fcfc9934115876f7f210102e091191b7a93c4b0117\": rpc error: code = NotFound desc = could not find container \"5eeb177ee807a924207351fcfc9934115876f7f210102e091191b7a93c4b0117\": container with ID starting with 5eeb177ee807a924207351fcfc9934115876f7f210102e091191b7a93c4b0117 not found: ID does not exist" Jul 29 08:33:46.045214 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:33:46.045179 2563 scope.go:117] "RemoveContainer" containerID="0d3a1df021e0ed10c199857cdacadaad9cb46533321cc945ef88989f80364b01" Jul 29 08:33:46.045452 ip-10-0-131-138 kubenswrapper[2563]: E0729 08:33:46.045436 2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0d3a1df021e0ed10c199857cdacadaad9cb46533321cc945ef88989f80364b01\": container with ID starting with 0d3a1df021e0ed10c199857cdacadaad9cb46533321cc945ef88989f80364b01 not found: ID does not exist" containerID="0d3a1df021e0ed10c199857cdacadaad9cb46533321cc945ef88989f80364b01" Jul 29 08:33:46.045499 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:33:46.045457 2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0d3a1df021e0ed10c199857cdacadaad9cb46533321cc945ef88989f80364b01"} err="failed to get container status \"0d3a1df021e0ed10c199857cdacadaad9cb46533321cc945ef88989f80364b01\": rpc error: code = NotFound desc = could not find container \"0d3a1df021e0ed10c199857cdacadaad9cb46533321cc945ef88989f80364b01\": container with ID starting with 0d3a1df021e0ed10c199857cdacadaad9cb46533321cc945ef88989f80364b01 not found: ID does not exist" Jul 29 08:33:46.045499 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:33:46.045470 2563 scope.go:117] "RemoveContainer" containerID="0a60ff42153cd7243a08db6d5b6e7fb1191cfd7ebf7b0d8d51f0a668bf60bd01" Jul 29 08:33:46.045697 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:33:46.045680 2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0a60ff42153cd7243a08db6d5b6e7fb1191cfd7ebf7b0d8d51f0a668bf60bd01"} err="failed to get container status \"0a60ff42153cd7243a08db6d5b6e7fb1191cfd7ebf7b0d8d51f0a668bf60bd01\": rpc error: code = NotFound desc = could not find container \"0a60ff42153cd7243a08db6d5b6e7fb1191cfd7ebf7b0d8d51f0a668bf60bd01\": container with ID starting with 0a60ff42153cd7243a08db6d5b6e7fb1191cfd7ebf7b0d8d51f0a668bf60bd01 not found: ID does not exist" Jul 29 08:33:46.045730 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:33:46.045699 2563 scope.go:117] "RemoveContainer" containerID="c491f99363c3d4556759693b49458785e5e44635b52b0a40c0cff91ca24d0f44" Jul 29 08:33:46.045901 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:33:46.045881 2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c491f99363c3d4556759693b49458785e5e44635b52b0a40c0cff91ca24d0f44"} err="failed to get container status \"c491f99363c3d4556759693b49458785e5e44635b52b0a40c0cff91ca24d0f44\": rpc error: code = NotFound desc = could not find container \"c491f99363c3d4556759693b49458785e5e44635b52b0a40c0cff91ca24d0f44\": container with ID starting with c491f99363c3d4556759693b49458785e5e44635b52b0a40c0cff91ca24d0f44 not found: ID does not exist" Jul 29 08:33:46.045970 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:33:46.045902 2563 scope.go:117] "RemoveContainer" containerID="b541318a41ca2d9895c30890b4850d0514fd543f57db33c885e5b6d82ba17687" Jul 29 08:33:46.046096 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:33:46.046081 2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b541318a41ca2d9895c30890b4850d0514fd543f57db33c885e5b6d82ba17687"} err="failed to get container status \"b541318a41ca2d9895c30890b4850d0514fd543f57db33c885e5b6d82ba17687\": rpc error: code = NotFound desc = could not find container \"b541318a41ca2d9895c30890b4850d0514fd543f57db33c885e5b6d82ba17687\": container with ID starting with b541318a41ca2d9895c30890b4850d0514fd543f57db33c885e5b6d82ba17687 not found: ID does not exist" Jul 29 08:33:46.046143 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:33:46.046096 2563 scope.go:117] "RemoveContainer" containerID="799f32f92a307c4ebc64fd16ef84a1a5ba6f6b34cc71af2fa9184496756dde21" Jul 29 08:33:46.046303 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:33:46.046286 2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"799f32f92a307c4ebc64fd16ef84a1a5ba6f6b34cc71af2fa9184496756dde21"} err="failed to get container status \"799f32f92a307c4ebc64fd16ef84a1a5ba6f6b34cc71af2fa9184496756dde21\": rpc error: code = NotFound desc = could not find container \"799f32f92a307c4ebc64fd16ef84a1a5ba6f6b34cc71af2fa9184496756dde21\": container with ID starting with 799f32f92a307c4ebc64fd16ef84a1a5ba6f6b34cc71af2fa9184496756dde21 not found: ID does not exist" Jul 29 08:33:46.046376 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:33:46.046305 2563 scope.go:117] "RemoveContainer" containerID="5eeb177ee807a924207351fcfc9934115876f7f210102e091191b7a93c4b0117" Jul 29 08:33:46.046505 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:33:46.046490 2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5eeb177ee807a924207351fcfc9934115876f7f210102e091191b7a93c4b0117"} err="failed to get container status \"5eeb177ee807a924207351fcfc9934115876f7f210102e091191b7a93c4b0117\": rpc error: code = NotFound desc = could not find container \"5eeb177ee807a924207351fcfc9934115876f7f210102e091191b7a93c4b0117\": container with ID starting with 5eeb177ee807a924207351fcfc9934115876f7f210102e091191b7a93c4b0117 not found: ID does not exist" Jul 29 08:33:46.046571 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:33:46.046505 2563 scope.go:117] "RemoveContainer" containerID="0d3a1df021e0ed10c199857cdacadaad9cb46533321cc945ef88989f80364b01" Jul 29 08:33:46.046700 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:33:46.046683 2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0d3a1df021e0ed10c199857cdacadaad9cb46533321cc945ef88989f80364b01"} err="failed to get container status \"0d3a1df021e0ed10c199857cdacadaad9cb46533321cc945ef88989f80364b01\": rpc error: code = NotFound desc = could not find container \"0d3a1df021e0ed10c199857cdacadaad9cb46533321cc945ef88989f80364b01\": container with ID starting with 0d3a1df021e0ed10c199857cdacadaad9cb46533321cc945ef88989f80364b01 not found: ID does not exist" Jul 29 08:33:47.962942 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:33:47.962902 2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="562b5f5f-6c61-4618-b11e-41518922e7a6" path="/var/lib/kubelet/pods/562b5f5f-6c61-4618-b11e-41518922e7a6/volumes" Jul 29 08:33:47.963736 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:33:47.963712 2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e283ad45-0159-43dd-a7ec-2be6de4297d5" path="/var/lib/kubelet/pods/e283ad45-0159-43dd-a7ec-2be6de4297d5/volumes" Jul 29 08:33:58.150635 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:33:58.150591 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8rp9g_cfd1d8df-e6b7-4a67-81f2-5d83da541e20/konnectivity-agent/0.log" limit=1024 Jul 29 08:33:58.385539 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:33:58.385506 2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["test-ns-feast/postgres-864846d4f5-mgknv"] Jul 29 08:33:58.385774 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:33:58.385762 2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="562b5f5f-6c61-4618-b11e-41518922e7a6" containerName="ui" Jul 29 08:33:58.385832 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:33:58.385776 2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="562b5f5f-6c61-4618-b11e-41518922e7a6" containerName="ui" Jul 29 08:33:58.385832 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:33:58.385785 2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e283ad45-0159-43dd-a7ec-2be6de4297d5" containerName="feast-init" Jul 29 08:33:58.385832 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:33:58.385790 2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="e283ad45-0159-43dd-a7ec-2be6de4297d5" containerName="feast-init" Jul 29 08:33:58.385832 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:33:58.385798 2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e283ad45-0159-43dd-a7ec-2be6de4297d5" containerName="online" Jul 29 08:33:58.385832 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:33:58.385804 2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="e283ad45-0159-43dd-a7ec-2be6de4297d5" containerName="online" Jul 29 08:33:58.385832 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:33:58.385810 2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="562b5f5f-6c61-4618-b11e-41518922e7a6" containerName="feast-init" Jul 29 08:33:58.385832 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:33:58.385814 2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="562b5f5f-6c61-4618-b11e-41518922e7a6" containerName="feast-init" Jul 29 08:33:58.385832 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:33:58.385824 2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="562b5f5f-6c61-4618-b11e-41518922e7a6" containerName="offline" Jul 29 08:33:58.385832 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:33:58.385831 2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="562b5f5f-6c61-4618-b11e-41518922e7a6" containerName="offline" Jul 29 08:33:58.386119 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:33:58.385841 2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="562b5f5f-6c61-4618-b11e-41518922e7a6" containerName="feast-apply" Jul 29 08:33:58.386119 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:33:58.385845 2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="562b5f5f-6c61-4618-b11e-41518922e7a6" containerName="feast-apply" Jul 29 08:33:58.386119 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:33:58.385851 2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e283ad45-0159-43dd-a7ec-2be6de4297d5" containerName="ui" Jul 29 08:33:58.386119 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:33:58.385856 2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="e283ad45-0159-43dd-a7ec-2be6de4297d5" containerName="ui" Jul 29 08:33:58.386119 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:33:58.385864 2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="562b5f5f-6c61-4618-b11e-41518922e7a6" containerName="registry" Jul 29 08:33:58.386119 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:33:58.385868 2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="562b5f5f-6c61-4618-b11e-41518922e7a6" containerName="registry" Jul 29 08:33:58.386119 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:33:58.385874 2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="562b5f5f-6c61-4618-b11e-41518922e7a6" containerName="online" Jul 29 08:33:58.386119 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:33:58.385880 2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="562b5f5f-6c61-4618-b11e-41518922e7a6" containerName="online" Jul 29 08:33:58.386119 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:33:58.385884 2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e283ad45-0159-43dd-a7ec-2be6de4297d5" containerName="offline" Jul 29 08:33:58.386119 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:33:58.385889 2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="e283ad45-0159-43dd-a7ec-2be6de4297d5" containerName="offline" Jul 29 08:33:58.386119 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:33:58.385896 2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e283ad45-0159-43dd-a7ec-2be6de4297d5" containerName="feast-apply" Jul 29 08:33:58.386119 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:33:58.385902 2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="e283ad45-0159-43dd-a7ec-2be6de4297d5" containerName="feast-apply" Jul 29 08:33:58.386119 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:33:58.385944 2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="e283ad45-0159-43dd-a7ec-2be6de4297d5" containerName="offline" Jul 29 08:33:58.386119 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:33:58.385952 2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="e283ad45-0159-43dd-a7ec-2be6de4297d5" containerName="ui" Jul 29 08:33:58.386119 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:33:58.385957 2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="562b5f5f-6c61-4618-b11e-41518922e7a6" containerName="online" Jul 29 08:33:58.386119 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:33:58.385962 2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="e283ad45-0159-43dd-a7ec-2be6de4297d5" containerName="online" Jul 29 08:33:58.386119 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:33:58.385967 2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="562b5f5f-6c61-4618-b11e-41518922e7a6" containerName="ui" Jul 29 08:33:58.386119 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:33:58.385974 2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="562b5f5f-6c61-4618-b11e-41518922e7a6" containerName="registry" Jul 29 08:33:58.386119 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:33:58.385978 2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="562b5f5f-6c61-4618-b11e-41518922e7a6" containerName="offline" Jul 29 08:33:58.388850 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:33:58.388833 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/postgres-864846d4f5-mgknv" Jul 29 08:33:58.391840 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:33:58.391812 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"test-ns-feast\"/\"kube-root-ca.crt\"" Jul 29 08:33:58.391987 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:33:58.391880 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-feast\"/\"postgres-secret\"" Jul 29 08:33:58.396684 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:33:58.393281 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"test-ns-feast\"/\"openshift-service-ca.crt\"" Jul 29 08:33:58.396684 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:33:58.393555 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-feast\"/\"default-dockercfg-tl7r7\"" Jul 29 08:33:58.400309 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:33:58.400284 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["test-ns-feast/postgres-864846d4f5-mgknv"] Jul 29 08:33:58.418923 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:33:58.418883 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8rp9g_cfd1d8df-e6b7-4a67-81f2-5d83da541e20/konnectivity-agent/0.log" limit=1024 Jul 29 08:33:58.476128 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:33:58.476085 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"postgresdata\" (UniqueName: \"kubernetes.io/empty-dir/85fab982-522a-4046-8a2d-a2e4fc39365b-postgresdata\") pod \"postgres-864846d4f5-mgknv\" (UID: \"85fab982-522a-4046-8a2d-a2e4fc39365b\") " pod="test-ns-feast/postgres-864846d4f5-mgknv" Jul 29 08:33:58.476344 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:33:58.476144 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cslxt\" (UniqueName: \"kubernetes.io/projected/85fab982-522a-4046-8a2d-a2e4fc39365b-kube-api-access-cslxt\") pod \"postgres-864846d4f5-mgknv\" (UID: \"85fab982-522a-4046-8a2d-a2e4fc39365b\") " pod="test-ns-feast/postgres-864846d4f5-mgknv" Jul 29 08:33:58.479015 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:33:58.478981 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8rp9g_cfd1d8df-e6b7-4a67-81f2-5d83da541e20/konnectivity-agent/0.log" limit=1024 Jul 29 08:33:58.577229 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:33:58.577185 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"postgresdata\" (UniqueName: \"kubernetes.io/empty-dir/85fab982-522a-4046-8a2d-a2e4fc39365b-postgresdata\") pod \"postgres-864846d4f5-mgknv\" (UID: \"85fab982-522a-4046-8a2d-a2e4fc39365b\") " pod="test-ns-feast/postgres-864846d4f5-mgknv" Jul 29 08:33:58.577419 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:33:58.577246 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cslxt\" (UniqueName: \"kubernetes.io/projected/85fab982-522a-4046-8a2d-a2e4fc39365b-kube-api-access-cslxt\") pod \"postgres-864846d4f5-mgknv\" (UID: \"85fab982-522a-4046-8a2d-a2e4fc39365b\") " pod="test-ns-feast/postgres-864846d4f5-mgknv" Jul 29 08:33:58.577621 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:33:58.577601 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"postgresdata\" (UniqueName: \"kubernetes.io/empty-dir/85fab982-522a-4046-8a2d-a2e4fc39365b-postgresdata\") pod \"postgres-864846d4f5-mgknv\" (UID: \"85fab982-522a-4046-8a2d-a2e4fc39365b\") " pod="test-ns-feast/postgres-864846d4f5-mgknv" Jul 29 08:33:58.588517 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:33:58.588493 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-cslxt\" (UniqueName: \"kubernetes.io/projected/85fab982-522a-4046-8a2d-a2e4fc39365b-kube-api-access-cslxt\") pod \"postgres-864846d4f5-mgknv\" (UID: \"85fab982-522a-4046-8a2d-a2e4fc39365b\") " pod="test-ns-feast/postgres-864846d4f5-mgknv" Jul 29 08:33:58.704302 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:33:58.704273 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/postgres-864846d4f5-mgknv" Jul 29 08:33:58.850535 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:33:58.850513 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["test-ns-feast/postgres-864846d4f5-mgknv"] Jul 29 08:33:58.852418 ip-10-0-131-138 kubenswrapper[2563]: W0729 08:33:58.852388 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod85fab982_522a_4046_8a2d_a2e4fc39365b.slice/crio-e017b254fc5063b9069c3e3514f45725810621cb79652a5cf9963f3d1ef509c8 WatchSource:0}: Error finding container e017b254fc5063b9069c3e3514f45725810621cb79652a5cf9963f3d1ef509c8: Status 404 returned error can't find the container with id e017b254fc5063b9069c3e3514f45725810621cb79652a5cf9963f3d1ef509c8 Jul 29 08:33:58.854301 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:33:58.854286 2563 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 29 08:33:59.003352 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:33:59.003235 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/postgres-864846d4f5-mgknv" event={"ID":"85fab982-522a-4046-8a2d-a2e4fc39365b","Type":"ContainerStarted","Data":"e017b254fc5063b9069c3e3514f45725810621cb79652a5cf9963f3d1ef509c8"} Jul 29 08:34:05.028904 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:34:05.028867 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/postgres-864846d4f5-mgknv" event={"ID":"85fab982-522a-4046-8a2d-a2e4fc39365b","Type":"ContainerStarted","Data":"d2303db63169e2ccc093007bddc9a8740d9c99d065cc362e6871e145a36ce4bf"} Jul 29 08:34:05.051204 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:34:05.051141 2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="test-ns-feast/postgres-864846d4f5-mgknv" podStartSLOduration=1.794887017 podStartE2EDuration="7.051120883s" totalImagesPullingTime="5.256233866s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 08:33:58 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 08:34:05.050346231 +0000 UTC m=+757.631635008" watchObservedRunningTime="2026-07-29 08:34:05.051120883 +0000 UTC m=+757.632409661" Jul 29 08:34:08.905635 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:34:08.905602 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8rp9g_cfd1d8df-e6b7-4a67-81f2-5d83da541e20/konnectivity-agent/0.log" limit=1024 Jul 29 08:34:09.015319 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:34:09.015277 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8rp9g_cfd1d8df-e6b7-4a67-81f2-5d83da541e20/konnectivity-agent/0.log" limit=1024 Jul 29 08:34:09.029027 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:34:09.028988 2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["test-ns-feast/feast-credit-scoring-846b688979-bfbk9"] Jul 29 08:34:09.032469 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:34:09.032451 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/feast-credit-scoring-846b688979-bfbk9" Jul 29 08:34:09.035784 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:34:09.035757 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-feast\"/\"feast-credit-scoring-registry-tls\"" Jul 29 08:34:09.035917 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:34:09.035900 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"test-ns-feast\"/\"feast-credit-scoring-intra-comm\"" Jul 29 08:34:09.036120 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:34:09.036089 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-feast\"/\"feast-credit-scoring-dockercfg-c9j8h\"" Jul 29 08:34:09.036374 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:34:09.036359 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-feast\"/\"feast-credit-scoring-online-tls\"" Jul 29 08:34:09.036459 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:34:09.036414 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-feast\"/\"feast-credit-scoring-offline-tls\"" Jul 29 08:34:09.054177 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:34:09.054147 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["test-ns-feast/feast-credit-scoring-846b688979-bfbk9"] Jul 29 08:34:09.057896 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:34:09.057870 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/184062b6-40dd-42e4-9596-e8fd50d0486a-registry-tls\") pod \"feast-credit-scoring-846b688979-bfbk9\" (UID: \"184062b6-40dd-42e4-9596-e8fd50d0486a\") " pod="test-ns-feast/feast-credit-scoring-846b688979-bfbk9" Jul 29 08:34:09.058038 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:34:09.057906 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/184062b6-40dd-42e4-9596-e8fd50d0486a-feast-data\") pod \"feast-credit-scoring-846b688979-bfbk9\" (UID: \"184062b6-40dd-42e4-9596-e8fd50d0486a\") " pod="test-ns-feast/feast-credit-scoring-846b688979-bfbk9" Jul 29 08:34:09.058038 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:34:09.057923 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/184062b6-40dd-42e4-9596-e8fd50d0486a-online-tls\") pod \"feast-credit-scoring-846b688979-bfbk9\" (UID: \"184062b6-40dd-42e4-9596-e8fd50d0486a\") " pod="test-ns-feast/feast-credit-scoring-846b688979-bfbk9" Jul 29 08:34:09.058156 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:34:09.058031 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/184062b6-40dd-42e4-9596-e8fd50d0486a-offline-tls\") pod \"feast-credit-scoring-846b688979-bfbk9\" (UID: \"184062b6-40dd-42e4-9596-e8fd50d0486a\") " pod="test-ns-feast/feast-credit-scoring-846b688979-bfbk9" Jul 29 08:34:09.058156 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:34:09.058099 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2lvml\" (UniqueName: \"kubernetes.io/projected/184062b6-40dd-42e4-9596-e8fd50d0486a-kube-api-access-2lvml\") pod \"feast-credit-scoring-846b688979-bfbk9\" (UID: \"184062b6-40dd-42e4-9596-e8fd50d0486a\") " pod="test-ns-feast/feast-credit-scoring-846b688979-bfbk9" Jul 29 08:34:09.086060 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:34:09.086023 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8rp9g_cfd1d8df-e6b7-4a67-81f2-5d83da541e20/konnectivity-agent/0.log" limit=1024 Jul 29 08:34:09.158534 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:34:09.158448 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/184062b6-40dd-42e4-9596-e8fd50d0486a-offline-tls\") pod \"feast-credit-scoring-846b688979-bfbk9\" (UID: \"184062b6-40dd-42e4-9596-e8fd50d0486a\") " pod="test-ns-feast/feast-credit-scoring-846b688979-bfbk9" Jul 29 08:34:09.158534 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:34:09.158505 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2lvml\" (UniqueName: \"kubernetes.io/projected/184062b6-40dd-42e4-9596-e8fd50d0486a-kube-api-access-2lvml\") pod \"feast-credit-scoring-846b688979-bfbk9\" (UID: \"184062b6-40dd-42e4-9596-e8fd50d0486a\") " pod="test-ns-feast/feast-credit-scoring-846b688979-bfbk9" Jul 29 08:34:09.158786 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:34:09.158546 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/184062b6-40dd-42e4-9596-e8fd50d0486a-registry-tls\") pod \"feast-credit-scoring-846b688979-bfbk9\" (UID: \"184062b6-40dd-42e4-9596-e8fd50d0486a\") " pod="test-ns-feast/feast-credit-scoring-846b688979-bfbk9" Jul 29 08:34:09.158786 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:34:09.158576 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/184062b6-40dd-42e4-9596-e8fd50d0486a-feast-data\") pod \"feast-credit-scoring-846b688979-bfbk9\" (UID: \"184062b6-40dd-42e4-9596-e8fd50d0486a\") " pod="test-ns-feast/feast-credit-scoring-846b688979-bfbk9" Jul 29 08:34:09.158786 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:34:09.158602 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/184062b6-40dd-42e4-9596-e8fd50d0486a-online-tls\") pod \"feast-credit-scoring-846b688979-bfbk9\" (UID: \"184062b6-40dd-42e4-9596-e8fd50d0486a\") " pod="test-ns-feast/feast-credit-scoring-846b688979-bfbk9" Jul 29 08:34:09.158786 ip-10-0-131-138 kubenswrapper[2563]: E0729 08:34:09.158668 2563 secret.go:189] Couldn't get secret test-ns-feast/feast-credit-scoring-registry-tls: secret "feast-credit-scoring-registry-tls" not found Jul 29 08:34:09.158786 ip-10-0-131-138 kubenswrapper[2563]: E0729 08:34:09.158722 2563 secret.go:189] Couldn't get secret test-ns-feast/feast-credit-scoring-online-tls: secret "feast-credit-scoring-online-tls" not found Jul 29 08:34:09.158786 ip-10-0-131-138 kubenswrapper[2563]: E0729 08:34:09.158742 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/184062b6-40dd-42e4-9596-e8fd50d0486a-registry-tls podName:184062b6-40dd-42e4-9596-e8fd50d0486a nodeName:}" failed. No retries permitted until 2026-07-29 08:34:09.658720169 +0000 UTC m=+762.240008938 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/secret/184062b6-40dd-42e4-9596-e8fd50d0486a-registry-tls") pod "feast-credit-scoring-846b688979-bfbk9" (UID: "184062b6-40dd-42e4-9596-e8fd50d0486a") : secret "feast-credit-scoring-registry-tls" not found Jul 29 08:34:09.158786 ip-10-0-131-138 kubenswrapper[2563]: E0729 08:34:09.158771 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/184062b6-40dd-42e4-9596-e8fd50d0486a-online-tls podName:184062b6-40dd-42e4-9596-e8fd50d0486a nodeName:}" failed. No retries permitted until 2026-07-29 08:34:09.658754968 +0000 UTC m=+762.240043741 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "online-tls" (UniqueName: "kubernetes.io/secret/184062b6-40dd-42e4-9596-e8fd50d0486a-online-tls") pod "feast-credit-scoring-846b688979-bfbk9" (UID: "184062b6-40dd-42e4-9596-e8fd50d0486a") : secret "feast-credit-scoring-online-tls" not found Jul 29 08:34:09.159151 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:34:09.158982 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/184062b6-40dd-42e4-9596-e8fd50d0486a-feast-data\") pod \"feast-credit-scoring-846b688979-bfbk9\" (UID: \"184062b6-40dd-42e4-9596-e8fd50d0486a\") " pod="test-ns-feast/feast-credit-scoring-846b688979-bfbk9" Jul 29 08:34:09.161169 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:34:09.161152 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/184062b6-40dd-42e4-9596-e8fd50d0486a-offline-tls\") pod \"feast-credit-scoring-846b688979-bfbk9\" (UID: \"184062b6-40dd-42e4-9596-e8fd50d0486a\") " pod="test-ns-feast/feast-credit-scoring-846b688979-bfbk9" Jul 29 08:34:09.171961 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:34:09.171939 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2lvml\" (UniqueName: \"kubernetes.io/projected/184062b6-40dd-42e4-9596-e8fd50d0486a-kube-api-access-2lvml\") pod \"feast-credit-scoring-846b688979-bfbk9\" (UID: \"184062b6-40dd-42e4-9596-e8fd50d0486a\") " pod="test-ns-feast/feast-credit-scoring-846b688979-bfbk9" Jul 29 08:34:09.237893 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:34:09.237859 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8rp9g_cfd1d8df-e6b7-4a67-81f2-5d83da541e20/konnectivity-agent/0.log" limit=1024 Jul 29 08:34:09.338580 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:34:09.338530 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8rp9g_cfd1d8df-e6b7-4a67-81f2-5d83da541e20/konnectivity-agent/0.log" limit=1024 Jul 29 08:34:09.662292 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:34:09.662232 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/184062b6-40dd-42e4-9596-e8fd50d0486a-registry-tls\") pod \"feast-credit-scoring-846b688979-bfbk9\" (UID: \"184062b6-40dd-42e4-9596-e8fd50d0486a\") " pod="test-ns-feast/feast-credit-scoring-846b688979-bfbk9" Jul 29 08:34:09.662292 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:34:09.662297 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/184062b6-40dd-42e4-9596-e8fd50d0486a-online-tls\") pod \"feast-credit-scoring-846b688979-bfbk9\" (UID: \"184062b6-40dd-42e4-9596-e8fd50d0486a\") " pod="test-ns-feast/feast-credit-scoring-846b688979-bfbk9" Jul 29 08:34:09.664794 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:34:09.664768 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/184062b6-40dd-42e4-9596-e8fd50d0486a-online-tls\") pod \"feast-credit-scoring-846b688979-bfbk9\" (UID: \"184062b6-40dd-42e4-9596-e8fd50d0486a\") " pod="test-ns-feast/feast-credit-scoring-846b688979-bfbk9" Jul 29 08:34:09.664966 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:34:09.664944 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/184062b6-40dd-42e4-9596-e8fd50d0486a-registry-tls\") pod \"feast-credit-scoring-846b688979-bfbk9\" (UID: \"184062b6-40dd-42e4-9596-e8fd50d0486a\") " pod="test-ns-feast/feast-credit-scoring-846b688979-bfbk9" Jul 29 08:34:09.941365 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:34:09.941249 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/feast-credit-scoring-846b688979-bfbk9" Jul 29 08:34:10.113268 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:34:10.113230 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["test-ns-feast/feast-credit-scoring-846b688979-bfbk9"] Jul 29 08:34:10.115740 ip-10-0-131-138 kubenswrapper[2563]: W0729 08:34:10.115707 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod184062b6_40dd_42e4_9596_e8fd50d0486a.slice/crio-af891eeb54b1b5cfc9d34069cfa03f987a652e6bd08b32abcabb59e7ebe000bc WatchSource:0}: Error finding container af891eeb54b1b5cfc9d34069cfa03f987a652e6bd08b32abcabb59e7ebe000bc: Status 404 returned error can't find the container with id af891eeb54b1b5cfc9d34069cfa03f987a652e6bd08b32abcabb59e7ebe000bc Jul 29 08:34:11.068979 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:34:11.068943 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-credit-scoring-846b688979-bfbk9" event={"ID":"184062b6-40dd-42e4-9596-e8fd50d0486a","Type":"ContainerStarted","Data":"c97b5d064527ea7c2edb0e9a76b7e89b7f349197b25a8fce01e0ffc18b33dfff"} Jul 29 08:34:11.069479 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:34:11.068999 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-credit-scoring-846b688979-bfbk9" event={"ID":"184062b6-40dd-42e4-9596-e8fd50d0486a","Type":"ContainerStarted","Data":"af891eeb54b1b5cfc9d34069cfa03f987a652e6bd08b32abcabb59e7ebe000bc"} Jul 29 08:34:12.073527 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:34:12.073492 2563 generic.go:356] "Generic (PLEG): container finished" podID="184062b6-40dd-42e4-9596-e8fd50d0486a" containerID="c97b5d064527ea7c2edb0e9a76b7e89b7f349197b25a8fce01e0ffc18b33dfff" exitCode=0 Jul 29 08:34:12.074024 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:34:12.073539 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-credit-scoring-846b688979-bfbk9" event={"ID":"184062b6-40dd-42e4-9596-e8fd50d0486a","Type":"ContainerDied","Data":"c97b5d064527ea7c2edb0e9a76b7e89b7f349197b25a8fce01e0ffc18b33dfff"} Jul 29 08:34:13.078714 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:34:13.078677 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-credit-scoring-846b688979-bfbk9" event={"ID":"184062b6-40dd-42e4-9596-e8fd50d0486a","Type":"ContainerStarted","Data":"3039a44aa60c4dbcd2e2a9cd2c365626ef20d9a1edcc74d66642b0eadfc48bb3"} Jul 29 08:34:19.103763 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:34:19.103676 2563 generic.go:356] "Generic (PLEG): container finished" podID="184062b6-40dd-42e4-9596-e8fd50d0486a" containerID="3039a44aa60c4dbcd2e2a9cd2c365626ef20d9a1edcc74d66642b0eadfc48bb3" exitCode=0 Jul 29 08:34:19.103763 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:34:19.103728 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-credit-scoring-846b688979-bfbk9" event={"ID":"184062b6-40dd-42e4-9596-e8fd50d0486a","Type":"ContainerDied","Data":"3039a44aa60c4dbcd2e2a9cd2c365626ef20d9a1edcc74d66642b0eadfc48bb3"} Jul 29 08:34:20.111564 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:34:20.111518 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-credit-scoring-846b688979-bfbk9" event={"ID":"184062b6-40dd-42e4-9596-e8fd50d0486a","Type":"ContainerStarted","Data":"7bcabefaa82efbc90ee4c0350b2fa37dafcb7b1f6f1fd8337c68809aa2e64f81"} Jul 29 08:34:20.112037 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:34:20.111593 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-credit-scoring-846b688979-bfbk9" event={"ID":"184062b6-40dd-42e4-9596-e8fd50d0486a","Type":"ContainerStarted","Data":"63313f530f4ed1731598afc51f735cbf58f567faf5e72e6d02dbec188116b299"} Jul 29 08:34:20.112037 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:34:20.111612 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-credit-scoring-846b688979-bfbk9" event={"ID":"184062b6-40dd-42e4-9596-e8fd50d0486a","Type":"ContainerStarted","Data":"a6b3828055a01eb2a27fc16341f987f3c860f40c34848060613f9e8aa6b7839c"} Jul 29 08:34:20.138484 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:34:20.138414 2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="test-ns-feast/feast-credit-scoring-846b688979-bfbk9" podStartSLOduration=12.138391761 podStartE2EDuration="12.138391761s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 08:34:08 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 08:34:20.136795543 +0000 UTC m=+772.718084324" watchObservedRunningTime="2026-07-29 08:34:20.138391761 +0000 UTC m=+772.719680541" Jul 29 08:34:21.941497 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:34:21.941442 2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-feast/feast-credit-scoring-846b688979-bfbk9" Jul 29 08:34:21.942002 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:34:21.941526 2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-feast/feast-credit-scoring-846b688979-bfbk9" Jul 29 08:34:21.942002 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:34:21.941551 2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-feast/feast-credit-scoring-846b688979-bfbk9" Jul 29 08:34:21.943118 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:34:21.943084 2563 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-credit-scoring-846b688979-bfbk9" podUID="184062b6-40dd-42e4-9596-e8fd50d0486a" containerName="offline" probeResult="failure" output="dial tcp 10.134.0.23:8816: connect: connection refused" Jul 29 08:34:21.943285 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:34:21.943122 2563 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-credit-scoring-846b688979-bfbk9" podUID="184062b6-40dd-42e4-9596-e8fd50d0486a" containerName="online" probeResult="failure" output="Get \"https://10.134.0.23:6567/health\": dial tcp 10.134.0.23:6567: connect: connection refused" Jul 29 08:34:21.943285 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:34:21.943127 2563 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-credit-scoring-846b688979-bfbk9" podUID="184062b6-40dd-42e4-9596-e8fd50d0486a" containerName="registry" probeResult="failure" output="dial tcp 10.134.0.23:6571: connect: connection refused" Jul 29 08:34:24.942566 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:34:24.942528 2563 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-credit-scoring-846b688979-bfbk9" podUID="184062b6-40dd-42e4-9596-e8fd50d0486a" containerName="registry" probeResult="failure" output="dial tcp 10.134.0.23:6571: connect: connection refused" Jul 29 08:34:24.943082 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:34:24.942625 2563 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-credit-scoring-846b688979-bfbk9" podUID="184062b6-40dd-42e4-9596-e8fd50d0486a" containerName="online" probeResult="failure" output="Get \"https://10.134.0.23:6567/health\": dial tcp 10.134.0.23:6567: connect: connection refused" Jul 29 08:34:24.943560 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:34:24.943537 2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-feast/feast-credit-scoring-846b688979-bfbk9" Jul 29 08:34:24.944050 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:34:24.944026 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-credit-scoring-846b688979-bfbk9" Jul 29 08:34:24.944451 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:34:24.944430 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-feast/feast-credit-scoring-846b688979-bfbk9" Jul 29 08:34:27.941865 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:34:27.941824 2563 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-credit-scoring-846b688979-bfbk9" podUID="184062b6-40dd-42e4-9596-e8fd50d0486a" containerName="registry" probeResult="failure" output="dial tcp 10.134.0.23:6571: connect: connection refused" Jul 29 08:34:27.947454 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:34:27.947428 2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-feast/feast-credit-scoring-846b688979-bfbk9" Jul 29 08:34:27.947942 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:34:27.947919 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-credit-scoring-846b688979-bfbk9" Jul 29 08:34:27.950984 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:34:27.950962 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-feast/feast-credit-scoring-846b688979-bfbk9" Jul 29 08:34:29.941935 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:34:29.941887 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-credit-scoring-846b688979-bfbk9" Jul 29 08:34:30.942208 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:34:30.942173 2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-feast/feast-credit-scoring-846b688979-bfbk9" Jul 29 08:34:31.162203 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:34:31.162172 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-feast/feast-credit-scoring-846b688979-bfbk9" Jul 29 08:34:31.485472 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:34:31.485443 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8rp9g_cfd1d8df-e6b7-4a67-81f2-5d83da541e20/konnectivity-agent/0.log" limit=1024 Jul 29 08:35:30.027470 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:35:30.027434 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8rp9g_cfd1d8df-e6b7-4a67-81f2-5d83da541e20/konnectivity-agent/0.log" limit=1024 Jul 29 08:35:34.985686 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:35:34.985652 2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["test-ns-feast/feast-credit-scoring-846b688979-bfbk9"] Jul 29 08:35:34.986109 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:35:34.985910 2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="test-ns-feast/feast-credit-scoring-846b688979-bfbk9" podUID="184062b6-40dd-42e4-9596-e8fd50d0486a" containerName="registry" containerID="cri-o://a6b3828055a01eb2a27fc16341f987f3c860f40c34848060613f9e8aa6b7839c" gracePeriod=30 Jul 29 08:35:34.986109 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:35:34.985977 2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="test-ns-feast/feast-credit-scoring-846b688979-bfbk9" podUID="184062b6-40dd-42e4-9596-e8fd50d0486a" containerName="online" containerID="cri-o://63313f530f4ed1731598afc51f735cbf58f567faf5e72e6d02dbec188116b299" gracePeriod=30 Jul 29 08:35:34.986109 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:35:34.985977 2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="test-ns-feast/feast-credit-scoring-846b688979-bfbk9" podUID="184062b6-40dd-42e4-9596-e8fd50d0486a" containerName="offline" containerID="cri-o://7bcabefaa82efbc90ee4c0350b2fa37dafcb7b1f6f1fd8337c68809aa2e64f81" gracePeriod=30 Jul 29 08:35:35.001915 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:35:35.001889 2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["test-ns-feast/postgres-864846d4f5-mgknv"] Jul 29 08:35:35.002097 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:35:35.002065 2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="test-ns-feast/postgres-864846d4f5-mgknv" podUID="85fab982-522a-4046-8a2d-a2e4fc39365b" containerName="postgres" containerID="cri-o://d2303db63169e2ccc093007bddc9a8740d9c99d065cc362e6871e145a36ce4bf" gracePeriod=30 Jul 29 08:35:35.079714 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:35:35.079681 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8rp9g_cfd1d8df-e6b7-4a67-81f2-5d83da541e20/konnectivity-agent/0.log" limit=1024 Jul 29 08:35:35.249903 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:35:35.249872 2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/postgres-864846d4f5-mgknv" Jul 29 08:35:35.373644 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:35:35.373616 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-cslxt\" (UniqueName: \"kubernetes.io/projected/85fab982-522a-4046-8a2d-a2e4fc39365b-kube-api-access-cslxt\") pod \"85fab982-522a-4046-8a2d-a2e4fc39365b\" (UID: \"85fab982-522a-4046-8a2d-a2e4fc39365b\") " Jul 29 08:35:35.373826 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:35:35.373681 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"postgresdata\" (UniqueName: \"kubernetes.io/empty-dir/85fab982-522a-4046-8a2d-a2e4fc39365b-postgresdata\") pod \"85fab982-522a-4046-8a2d-a2e4fc39365b\" (UID: \"85fab982-522a-4046-8a2d-a2e4fc39365b\") " Jul 29 08:35:35.375844 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:35:35.375816 2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/85fab982-522a-4046-8a2d-a2e4fc39365b-kube-api-access-cslxt" (OuterVolumeSpecName: "kube-api-access-cslxt") pod "85fab982-522a-4046-8a2d-a2e4fc39365b" (UID: "85fab982-522a-4046-8a2d-a2e4fc39365b"). InnerVolumeSpecName "kube-api-access-cslxt". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 29 08:35:35.408021 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:35:35.407984 2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/85fab982-522a-4046-8a2d-a2e4fc39365b-postgresdata" (OuterVolumeSpecName: "postgresdata") pod "85fab982-522a-4046-8a2d-a2e4fc39365b" (UID: "85fab982-522a-4046-8a2d-a2e4fc39365b"). InnerVolumeSpecName "postgresdata". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 29 08:35:35.412032 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:35:35.412001 2563 generic.go:356] "Generic (PLEG): container finished" podID="85fab982-522a-4046-8a2d-a2e4fc39365b" containerID="d2303db63169e2ccc093007bddc9a8740d9c99d065cc362e6871e145a36ce4bf" exitCode=0 Jul 29 08:35:35.412181 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:35:35.412043 2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/postgres-864846d4f5-mgknv" Jul 29 08:35:35.412181 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:35:35.412043 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/postgres-864846d4f5-mgknv" event={"ID":"85fab982-522a-4046-8a2d-a2e4fc39365b","Type":"ContainerDied","Data":"d2303db63169e2ccc093007bddc9a8740d9c99d065cc362e6871e145a36ce4bf"} Jul 29 08:35:35.412181 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:35:35.412083 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/postgres-864846d4f5-mgknv" event={"ID":"85fab982-522a-4046-8a2d-a2e4fc39365b","Type":"ContainerDied","Data":"e017b254fc5063b9069c3e3514f45725810621cb79652a5cf9963f3d1ef509c8"} Jul 29 08:35:35.412181 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:35:35.412104 2563 scope.go:117] "RemoveContainer" containerID="d2303db63169e2ccc093007bddc9a8740d9c99d065cc362e6871e145a36ce4bf" Jul 29 08:35:35.420677 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:35:35.420658 2563 scope.go:117] "RemoveContainer" containerID="d2303db63169e2ccc093007bddc9a8740d9c99d065cc362e6871e145a36ce4bf" Jul 29 08:35:35.420945 ip-10-0-131-138 kubenswrapper[2563]: E0729 08:35:35.420924 2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d2303db63169e2ccc093007bddc9a8740d9c99d065cc362e6871e145a36ce4bf\": container with ID starting with d2303db63169e2ccc093007bddc9a8740d9c99d065cc362e6871e145a36ce4bf not found: ID does not exist" containerID="d2303db63169e2ccc093007bddc9a8740d9c99d065cc362e6871e145a36ce4bf" Jul 29 08:35:35.421008 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:35:35.420954 2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d2303db63169e2ccc093007bddc9a8740d9c99d065cc362e6871e145a36ce4bf"} err="failed to get container status \"d2303db63169e2ccc093007bddc9a8740d9c99d065cc362e6871e145a36ce4bf\": rpc error: code = NotFound desc = could not find container \"d2303db63169e2ccc093007bddc9a8740d9c99d065cc362e6871e145a36ce4bf\": container with ID starting with d2303db63169e2ccc093007bddc9a8740d9c99d065cc362e6871e145a36ce4bf not found: ID does not exist" Jul 29 08:35:35.434527 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:35:35.434501 2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["test-ns-feast/postgres-864846d4f5-mgknv"] Jul 29 08:35:35.438284 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:35:35.438248 2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["test-ns-feast/postgres-864846d4f5-mgknv"] Jul 29 08:35:35.474699 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:35:35.474664 2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-cslxt\" (UniqueName: \"kubernetes.io/projected/85fab982-522a-4046-8a2d-a2e4fc39365b-kube-api-access-cslxt\") on node \"ip-10-0-131-138.ec2.internal\" DevicePath \"\"" Jul 29 08:35:35.474699 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:35:35.474695 2563 reconciler_common.go:299] "Volume detached for volume \"postgresdata\" (UniqueName: \"kubernetes.io/empty-dir/85fab982-522a-4046-8a2d-a2e4fc39365b-postgresdata\") on node \"ip-10-0-131-138.ec2.internal\" DevicePath \"\"" Jul 29 08:35:35.961499 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:35:35.961473 2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="85fab982-522a-4046-8a2d-a2e4fc39365b" path="/var/lib/kubelet/pods/85fab982-522a-4046-8a2d-a2e4fc39365b/volumes" Jul 29 08:35:36.417732 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:35:36.417649 2563 generic.go:356] "Generic (PLEG): container finished" podID="184062b6-40dd-42e4-9596-e8fd50d0486a" containerID="63313f530f4ed1731598afc51f735cbf58f567faf5e72e6d02dbec188116b299" exitCode=0 Jul 29 08:35:36.417732 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:35:36.417719 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-credit-scoring-846b688979-bfbk9" event={"ID":"184062b6-40dd-42e4-9596-e8fd50d0486a","Type":"ContainerDied","Data":"63313f530f4ed1731598afc51f735cbf58f567faf5e72e6d02dbec188116b299"} Jul 29 08:35:37.948821 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:35:37.948779 2563 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-credit-scoring-846b688979-bfbk9" podUID="184062b6-40dd-42e4-9596-e8fd50d0486a" containerName="online" probeResult="failure" output="Get \"https://10.134.0.23:6567/health\": dial tcp 10.134.0.23:6567: connect: connection refused" Jul 29 08:35:47.948158 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:35:47.948115 2563 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-credit-scoring-846b688979-bfbk9" podUID="184062b6-40dd-42e4-9596-e8fd50d0486a" containerName="online" probeResult="failure" output="Get \"https://10.134.0.23:6567/health\": dial tcp 10.134.0.23:6567: connect: connection refused" Jul 29 08:35:57.948744 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:35:57.948689 2563 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-credit-scoring-846b688979-bfbk9" podUID="184062b6-40dd-42e4-9596-e8fd50d0486a" containerName="online" probeResult="failure" output="Get \"https://10.134.0.23:6567/health\": dial tcp 10.134.0.23:6567: connect: connection refused" Jul 29 08:35:57.949128 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:35:57.948841 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-credit-scoring-846b688979-bfbk9" Jul 29 08:36:05.525631 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:36:05.525602 2563 generic.go:356] "Generic (PLEG): container finished" podID="184062b6-40dd-42e4-9596-e8fd50d0486a" containerID="7bcabefaa82efbc90ee4c0350b2fa37dafcb7b1f6f1fd8337c68809aa2e64f81" exitCode=137 Jul 29 08:36:05.525631 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:36:05.525626 2563 generic.go:356] "Generic (PLEG): container finished" podID="184062b6-40dd-42e4-9596-e8fd50d0486a" containerID="a6b3828055a01eb2a27fc16341f987f3c860f40c34848060613f9e8aa6b7839c" exitCode=137 Jul 29 08:36:05.526005 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:36:05.525652 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-credit-scoring-846b688979-bfbk9" event={"ID":"184062b6-40dd-42e4-9596-e8fd50d0486a","Type":"ContainerDied","Data":"7bcabefaa82efbc90ee4c0350b2fa37dafcb7b1f6f1fd8337c68809aa2e64f81"} Jul 29 08:36:05.526005 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:36:05.525713 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-credit-scoring-846b688979-bfbk9" event={"ID":"184062b6-40dd-42e4-9596-e8fd50d0486a","Type":"ContainerDied","Data":"a6b3828055a01eb2a27fc16341f987f3c860f40c34848060613f9e8aa6b7839c"} Jul 29 08:36:05.651770 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:36:05.651747 2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/feast-credit-scoring-846b688979-bfbk9" Jul 29 08:36:05.817666 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:36:05.817592 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/184062b6-40dd-42e4-9596-e8fd50d0486a-offline-tls\") pod \"184062b6-40dd-42e4-9596-e8fd50d0486a\" (UID: \"184062b6-40dd-42e4-9596-e8fd50d0486a\") " Jul 29 08:36:05.817666 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:36:05.817655 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/184062b6-40dd-42e4-9596-e8fd50d0486a-online-tls\") pod \"184062b6-40dd-42e4-9596-e8fd50d0486a\" (UID: \"184062b6-40dd-42e4-9596-e8fd50d0486a\") " Jul 29 08:36:05.817894 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:36:05.817692 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-2lvml\" (UniqueName: \"kubernetes.io/projected/184062b6-40dd-42e4-9596-e8fd50d0486a-kube-api-access-2lvml\") pod \"184062b6-40dd-42e4-9596-e8fd50d0486a\" (UID: \"184062b6-40dd-42e4-9596-e8fd50d0486a\") " Jul 29 08:36:05.817894 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:36:05.817752 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/184062b6-40dd-42e4-9596-e8fd50d0486a-registry-tls\") pod \"184062b6-40dd-42e4-9596-e8fd50d0486a\" (UID: \"184062b6-40dd-42e4-9596-e8fd50d0486a\") " Jul 29 08:36:05.817894 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:36:05.817773 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/184062b6-40dd-42e4-9596-e8fd50d0486a-feast-data\") pod \"184062b6-40dd-42e4-9596-e8fd50d0486a\" (UID: \"184062b6-40dd-42e4-9596-e8fd50d0486a\") " Jul 29 08:36:05.820008 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:36:05.819979 2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/184062b6-40dd-42e4-9596-e8fd50d0486a-online-tls" (OuterVolumeSpecName: "online-tls") pod "184062b6-40dd-42e4-9596-e8fd50d0486a" (UID: "184062b6-40dd-42e4-9596-e8fd50d0486a"). InnerVolumeSpecName "online-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 29 08:36:05.820008 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:36:05.819996 2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/184062b6-40dd-42e4-9596-e8fd50d0486a-kube-api-access-2lvml" (OuterVolumeSpecName: "kube-api-access-2lvml") pod "184062b6-40dd-42e4-9596-e8fd50d0486a" (UID: "184062b6-40dd-42e4-9596-e8fd50d0486a"). InnerVolumeSpecName "kube-api-access-2lvml". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 29 08:36:05.820154 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:36:05.820038 2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/184062b6-40dd-42e4-9596-e8fd50d0486a-offline-tls" (OuterVolumeSpecName: "offline-tls") pod "184062b6-40dd-42e4-9596-e8fd50d0486a" (UID: "184062b6-40dd-42e4-9596-e8fd50d0486a"). InnerVolumeSpecName "offline-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 29 08:36:05.820209 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:36:05.820183 2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/184062b6-40dd-42e4-9596-e8fd50d0486a-registry-tls" (OuterVolumeSpecName: "registry-tls") pod "184062b6-40dd-42e4-9596-e8fd50d0486a" (UID: "184062b6-40dd-42e4-9596-e8fd50d0486a"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 29 08:36:05.826547 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:36:05.826521 2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/184062b6-40dd-42e4-9596-e8fd50d0486a-feast-data" (OuterVolumeSpecName: "feast-data") pod "184062b6-40dd-42e4-9596-e8fd50d0486a" (UID: "184062b6-40dd-42e4-9596-e8fd50d0486a"). InnerVolumeSpecName "feast-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 29 08:36:05.918464 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:36:05.918436 2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-2lvml\" (UniqueName: \"kubernetes.io/projected/184062b6-40dd-42e4-9596-e8fd50d0486a-kube-api-access-2lvml\") on node \"ip-10-0-131-138.ec2.internal\" DevicePath \"\"" Jul 29 08:36:05.918464 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:36:05.918461 2563 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/184062b6-40dd-42e4-9596-e8fd50d0486a-registry-tls\") on node \"ip-10-0-131-138.ec2.internal\" DevicePath \"\"" Jul 29 08:36:05.918464 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:36:05.918470 2563 reconciler_common.go:299] "Volume detached for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/184062b6-40dd-42e4-9596-e8fd50d0486a-feast-data\") on node \"ip-10-0-131-138.ec2.internal\" DevicePath \"\"" Jul 29 08:36:05.918655 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:36:05.918478 2563 reconciler_common.go:299] "Volume detached for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/184062b6-40dd-42e4-9596-e8fd50d0486a-offline-tls\") on node \"ip-10-0-131-138.ec2.internal\" DevicePath \"\"" Jul 29 08:36:05.918655 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:36:05.918506 2563 reconciler_common.go:299] "Volume detached for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/184062b6-40dd-42e4-9596-e8fd50d0486a-online-tls\") on node \"ip-10-0-131-138.ec2.internal\" DevicePath \"\"" Jul 29 08:36:06.531824 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:36:06.531787 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-credit-scoring-846b688979-bfbk9" event={"ID":"184062b6-40dd-42e4-9596-e8fd50d0486a","Type":"ContainerDied","Data":"af891eeb54b1b5cfc9d34069cfa03f987a652e6bd08b32abcabb59e7ebe000bc"} Jul 29 08:36:06.532346 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:36:06.531835 2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/feast-credit-scoring-846b688979-bfbk9" Jul 29 08:36:06.532346 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:36:06.531850 2563 scope.go:117] "RemoveContainer" containerID="7bcabefaa82efbc90ee4c0350b2fa37dafcb7b1f6f1fd8337c68809aa2e64f81" Jul 29 08:36:06.540400 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:36:06.540385 2563 scope.go:117] "RemoveContainer" containerID="63313f530f4ed1731598afc51f735cbf58f567faf5e72e6d02dbec188116b299" Jul 29 08:36:06.547966 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:36:06.547947 2563 scope.go:117] "RemoveContainer" containerID="a6b3828055a01eb2a27fc16341f987f3c860f40c34848060613f9e8aa6b7839c" Jul 29 08:36:06.553562 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:36:06.553541 2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["test-ns-feast/feast-credit-scoring-846b688979-bfbk9"] Jul 29 08:36:06.555816 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:36:06.555801 2563 scope.go:117] "RemoveContainer" containerID="3039a44aa60c4dbcd2e2a9cd2c365626ef20d9a1edcc74d66642b0eadfc48bb3" Jul 29 08:36:06.558471 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:36:06.558451 2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["test-ns-feast/feast-credit-scoring-846b688979-bfbk9"] Jul 29 08:36:06.563178 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:36:06.563164 2563 scope.go:117] "RemoveContainer" containerID="c97b5d064527ea7c2edb0e9a76b7e89b7f349197b25a8fce01e0ffc18b33dfff" Jul 29 08:36:07.962311 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:36:07.962277 2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="184062b6-40dd-42e4-9596-e8fd50d0486a" path="/var/lib/kubelet/pods/184062b6-40dd-42e4-9596-e8fd50d0486a/volumes" Jul 29 08:36:13.325591 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:36:13.325548 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8rp9g_cfd1d8df-e6b7-4a67-81f2-5d83da541e20/konnectivity-agent/0.log" limit=1024 Jul 29 08:36:13.425964 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:36:13.425917 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8rp9g_cfd1d8df-e6b7-4a67-81f2-5d83da541e20/konnectivity-agent/0.log" limit=1024 Jul 29 08:36:13.627454 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:36:13.627371 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8rp9g_cfd1d8df-e6b7-4a67-81f2-5d83da541e20/konnectivity-agent/0.log" limit=1024 Jul 29 08:36:13.685207 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:36:13.685168 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8rp9g_cfd1d8df-e6b7-4a67-81f2-5d83da541e20/konnectivity-agent/0.log" limit=1024 Jul 29 08:36:13.802423 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:36:13.802389 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8rp9g_cfd1d8df-e6b7-4a67-81f2-5d83da541e20/konnectivity-agent/0.log" limit=1024 Jul 29 08:38:08.615641 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:38:08.615553 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8rp9g_cfd1d8df-e6b7-4a67-81f2-5d83da541e20/konnectivity-agent/0.log" limit=1024 Jul 29 08:38:51.779533 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:38:51.779504 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8rp9g_cfd1d8df-e6b7-4a67-81f2-5d83da541e20/konnectivity-agent/0.log" limit=1024 Jul 29 08:39:27.625602 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:39:27.625504 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8rp9g_cfd1d8df-e6b7-4a67-81f2-5d83da541e20/konnectivity-agent/0.log" limit=1024 Jul 29 08:39:27.795215 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:39:27.795183 2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["test-ns-feast-rest/redis-69bbd798b6-zvdfn"] Jul 29 08:39:27.795495 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:39:27.795469 2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="184062b6-40dd-42e4-9596-e8fd50d0486a" containerName="offline" Jul 29 08:39:27.795495 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:39:27.795493 2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="184062b6-40dd-42e4-9596-e8fd50d0486a" containerName="offline" Jul 29 08:39:27.795678 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:39:27.795512 2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="184062b6-40dd-42e4-9596-e8fd50d0486a" containerName="registry" Jul 29 08:39:27.795678 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:39:27.795523 2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="184062b6-40dd-42e4-9596-e8fd50d0486a" containerName="registry" Jul 29 08:39:27.795678 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:39:27.795538 2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="184062b6-40dd-42e4-9596-e8fd50d0486a" containerName="online" Jul 29 08:39:27.795678 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:39:27.795547 2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="184062b6-40dd-42e4-9596-e8fd50d0486a" containerName="online" Jul 29 08:39:27.795678 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:39:27.795558 2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="184062b6-40dd-42e4-9596-e8fd50d0486a" containerName="feast-apply" Jul 29 08:39:27.795678 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:39:27.795566 2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="184062b6-40dd-42e4-9596-e8fd50d0486a" containerName="feast-apply" Jul 29 08:39:27.795678 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:39:27.795579 2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="184062b6-40dd-42e4-9596-e8fd50d0486a" containerName="feast-init" Jul 29 08:39:27.795678 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:39:27.795587 2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="184062b6-40dd-42e4-9596-e8fd50d0486a" containerName="feast-init" Jul 29 08:39:27.795678 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:39:27.795598 2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="85fab982-522a-4046-8a2d-a2e4fc39365b" containerName="postgres" Jul 29 08:39:27.795678 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:39:27.795608 2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="85fab982-522a-4046-8a2d-a2e4fc39365b" containerName="postgres" Jul 29 08:39:27.795678 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:39:27.795677 2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="85fab982-522a-4046-8a2d-a2e4fc39365b" containerName="postgres" Jul 29 08:39:27.796221 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:39:27.795687 2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="184062b6-40dd-42e4-9596-e8fd50d0486a" containerName="registry" Jul 29 08:39:27.796221 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:39:27.795695 2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="184062b6-40dd-42e4-9596-e8fd50d0486a" containerName="online" Jul 29 08:39:27.796221 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:39:27.795704 2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="184062b6-40dd-42e4-9596-e8fd50d0486a" containerName="offline" Jul 29 08:39:27.798742 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:39:27.798725 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="test-ns-feast-rest/redis-69bbd798b6-zvdfn" Jul 29 08:39:27.801898 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:39:27.801877 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-feast-rest\"/\"default-dockercfg-zsr8f\"" Jul 29 08:39:27.801996 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:39:27.801911 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"test-ns-feast-rest\"/\"openshift-service-ca.crt\"" Jul 29 08:39:27.801996 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:39:27.801951 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"test-ns-feast-rest\"/\"kube-root-ca.crt\"" Jul 29 08:39:27.818318 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:39:27.818293 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["test-ns-feast-rest/redis-69bbd798b6-zvdfn"] Jul 29 08:39:27.899591 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:39:27.899526 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bb9nk\" (UniqueName: \"kubernetes.io/projected/e4d4d2b5-4009-44e5-95e5-7d850e3de60e-kube-api-access-bb9nk\") pod \"redis-69bbd798b6-zvdfn\" (UID: \"e4d4d2b5-4009-44e5-95e5-7d850e3de60e\") " pod="test-ns-feast-rest/redis-69bbd798b6-zvdfn" Jul 29 08:39:27.999988 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:39:27.999950 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bb9nk\" (UniqueName: \"kubernetes.io/projected/e4d4d2b5-4009-44e5-95e5-7d850e3de60e-kube-api-access-bb9nk\") pod \"redis-69bbd798b6-zvdfn\" (UID: \"e4d4d2b5-4009-44e5-95e5-7d850e3de60e\") " pod="test-ns-feast-rest/redis-69bbd798b6-zvdfn" Jul 29 08:39:28.009743 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:39:28.009720 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"test-ns-feast-rest\"/\"kube-root-ca.crt\"" Jul 29 08:39:28.020574 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:39:28.020551 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"test-ns-feast-rest\"/\"openshift-service-ca.crt\"" Jul 29 08:39:28.030585 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:39:28.030566 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bb9nk\" (UniqueName: \"kubernetes.io/projected/e4d4d2b5-4009-44e5-95e5-7d850e3de60e-kube-api-access-bb9nk\") pod \"redis-69bbd798b6-zvdfn\" (UID: \"e4d4d2b5-4009-44e5-95e5-7d850e3de60e\") " pod="test-ns-feast-rest/redis-69bbd798b6-zvdfn" Jul 29 08:39:28.111228 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:39:28.111201 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-feast-rest\"/\"default-dockercfg-zsr8f\"" Jul 29 08:39:28.118326 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:39:28.118299 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="test-ns-feast-rest/redis-69bbd798b6-zvdfn" Jul 29 08:39:28.250698 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:39:28.250667 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["test-ns-feast-rest/redis-69bbd798b6-zvdfn"] Jul 29 08:39:28.253460 ip-10-0-131-138 kubenswrapper[2563]: W0729 08:39:28.253422 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pode4d4d2b5_4009_44e5_95e5_7d850e3de60e.slice/crio-42cd3238bf859a0be64c67919420b58665e7d348e453932090bedf212ccb2085 WatchSource:0}: Error finding container 42cd3238bf859a0be64c67919420b58665e7d348e453932090bedf212ccb2085: Status 404 returned error can't find the container with id 42cd3238bf859a0be64c67919420b58665e7d348e453932090bedf212ccb2085 Jul 29 08:39:28.255316 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:39:28.255296 2563 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 29 08:39:28.894865 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:39:28.894833 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8rp9g_cfd1d8df-e6b7-4a67-81f2-5d83da541e20/konnectivity-agent/0.log" limit=1024 Jul 29 08:39:28.995679 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:39:28.995646 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8rp9g_cfd1d8df-e6b7-4a67-81f2-5d83da541e20/konnectivity-agent/0.log" limit=1024 Jul 29 08:39:29.243496 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:39:29.243455 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast-rest/redis-69bbd798b6-zvdfn" event={"ID":"e4d4d2b5-4009-44e5-95e5-7d850e3de60e","Type":"ContainerStarted","Data":"42cd3238bf859a0be64c67919420b58665e7d348e453932090bedf212ccb2085"} Jul 29 08:39:30.249066 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:39:30.249034 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast-rest/redis-69bbd798b6-zvdfn" event={"ID":"e4d4d2b5-4009-44e5-95e5-7d850e3de60e","Type":"ContainerStarted","Data":"eecd9f4474595daa29bb187c8a4f76865ec2c97a5aa94ce74a087ef1cd001226"} Jul 29 08:39:30.265884 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:39:30.265842 2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="test-ns-feast-rest/redis-69bbd798b6-zvdfn" podStartSLOduration=1.851567409 podStartE2EDuration="3.265827776s" totalImagesPullingTime="1.414260367s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 08:39:27 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 08:39:30.265534132 +0000 UTC m=+1082.846822910" watchObservedRunningTime="2026-07-29 08:39:30.265827776 +0000 UTC m=+1082.847116554" Jul 29 08:39:33.195393 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:39:33.195351 2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["test-ns-feast-rest/postgres-864846d4f5-mt8rr"] Jul 29 08:39:33.199102 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:39:33.199079 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="test-ns-feast-rest/postgres-864846d4f5-mt8rr" Jul 29 08:39:33.202034 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:39:33.202013 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-feast-rest\"/\"postgres-secret\"" Jul 29 08:39:33.205572 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:39:33.205550 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["test-ns-feast-rest/postgres-864846d4f5-mt8rr"] Jul 29 08:39:33.227383 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:39:33.227360 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8rp9g_cfd1d8df-e6b7-4a67-81f2-5d83da541e20/konnectivity-agent/0.log" limit=1024 Jul 29 08:39:33.340139 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:39:33.340107 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-j2xfn\" (UniqueName: \"kubernetes.io/projected/4902e580-8aed-491a-87e9-4102b87962c7-kube-api-access-j2xfn\") pod \"postgres-864846d4f5-mt8rr\" (UID: \"4902e580-8aed-491a-87e9-4102b87962c7\") " pod="test-ns-feast-rest/postgres-864846d4f5-mt8rr" Jul 29 08:39:33.340304 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:39:33.340184 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"postgresdata\" (UniqueName: \"kubernetes.io/empty-dir/4902e580-8aed-491a-87e9-4102b87962c7-postgresdata\") pod \"postgres-864846d4f5-mt8rr\" (UID: \"4902e580-8aed-491a-87e9-4102b87962c7\") " pod="test-ns-feast-rest/postgres-864846d4f5-mt8rr" Jul 29 08:39:33.441088 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:39:33.441047 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-j2xfn\" (UniqueName: \"kubernetes.io/projected/4902e580-8aed-491a-87e9-4102b87962c7-kube-api-access-j2xfn\") pod \"postgres-864846d4f5-mt8rr\" (UID: \"4902e580-8aed-491a-87e9-4102b87962c7\") " pod="test-ns-feast-rest/postgres-864846d4f5-mt8rr" Jul 29 08:39:33.441309 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:39:33.441126 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"postgresdata\" (UniqueName: \"kubernetes.io/empty-dir/4902e580-8aed-491a-87e9-4102b87962c7-postgresdata\") pod \"postgres-864846d4f5-mt8rr\" (UID: \"4902e580-8aed-491a-87e9-4102b87962c7\") " pod="test-ns-feast-rest/postgres-864846d4f5-mt8rr" Jul 29 08:39:33.441501 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:39:33.441486 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"postgresdata\" (UniqueName: \"kubernetes.io/empty-dir/4902e580-8aed-491a-87e9-4102b87962c7-postgresdata\") pod \"postgres-864846d4f5-mt8rr\" (UID: \"4902e580-8aed-491a-87e9-4102b87962c7\") " pod="test-ns-feast-rest/postgres-864846d4f5-mt8rr" Jul 29 08:39:33.450747 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:39:33.450692 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-j2xfn\" (UniqueName: \"kubernetes.io/projected/4902e580-8aed-491a-87e9-4102b87962c7-kube-api-access-j2xfn\") pod \"postgres-864846d4f5-mt8rr\" (UID: \"4902e580-8aed-491a-87e9-4102b87962c7\") " pod="test-ns-feast-rest/postgres-864846d4f5-mt8rr" Jul 29 08:39:33.509485 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:39:33.509462 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="test-ns-feast-rest/postgres-864846d4f5-mt8rr" Jul 29 08:39:33.635199 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:39:33.635169 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["test-ns-feast-rest/postgres-864846d4f5-mt8rr"] Jul 29 08:39:33.637969 ip-10-0-131-138 kubenswrapper[2563]: W0729 08:39:33.637926 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod4902e580_8aed_491a_87e9_4102b87962c7.slice/crio-167c1b12e4b671b62a0ae9b50fca89572ad713bd4a37ce68f365db34399a3fc5 WatchSource:0}: Error finding container 167c1b12e4b671b62a0ae9b50fca89572ad713bd4a37ce68f365db34399a3fc5: Status 404 returned error can't find the container with id 167c1b12e4b671b62a0ae9b50fca89572ad713bd4a37ce68f365db34399a3fc5 Jul 29 08:39:34.266864 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:39:34.266824 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast-rest/postgres-864846d4f5-mt8rr" event={"ID":"4902e580-8aed-491a-87e9-4102b87962c7","Type":"ContainerStarted","Data":"242c76a462978822711dc92388a2eff54a5e97fd134e6e36745741230a06ee00"} Jul 29 08:39:34.266864 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:39:34.266868 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast-rest/postgres-864846d4f5-mt8rr" event={"ID":"4902e580-8aed-491a-87e9-4102b87962c7","Type":"ContainerStarted","Data":"167c1b12e4b671b62a0ae9b50fca89572ad713bd4a37ce68f365db34399a3fc5"} Jul 29 08:39:34.285566 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:39:34.285511 2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="test-ns-feast-rest/postgres-864846d4f5-mt8rr" podStartSLOduration=1.285495067 podStartE2EDuration="1.285495067s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 08:39:33 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 08:39:34.284116611 +0000 UTC m=+1086.865405393" watchObservedRunningTime="2026-07-29 08:39:34.285495067 +0000 UTC m=+1086.866783844" Jul 29 08:39:38.664815 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:39:38.664782 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8rp9g_cfd1d8df-e6b7-4a67-81f2-5d83da541e20/konnectivity-agent/0.log" limit=1024 Jul 29 08:39:38.763024 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:39:38.762995 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8rp9g_cfd1d8df-e6b7-4a67-81f2-5d83da541e20/konnectivity-agent/0.log" limit=1024 Jul 29 08:39:38.853120 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:39:38.853081 2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["test-ns-feast-rest/feast-credit-scoring-5fd7cf9d69-mkmpb"] Jul 29 08:39:38.859217 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:39:38.859195 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="test-ns-feast-rest/feast-credit-scoring-5fd7cf9d69-mkmpb" Jul 29 08:39:38.862211 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:39:38.862187 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"test-ns-feast-rest\"/\"feast-credit-scoring-intra-comm\"" Jul 29 08:39:38.862362 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:39:38.862294 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-feast-rest\"/\"feast-credit-scoring-dockercfg-wrckx\"" Jul 29 08:39:38.862362 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:39:38.862313 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-feast-rest\"/\"feast-credit-scoring-online-tls\"" Jul 29 08:39:38.862362 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:39:38.862330 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-feast-rest\"/\"feast-credit-scoring-registry-tls\"" Jul 29 08:39:38.870952 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:39:38.870933 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["test-ns-feast-rest/feast-credit-scoring-5fd7cf9d69-mkmpb"] Jul 29 08:39:38.985593 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:39:38.985557 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-w9pcz\" (UniqueName: \"kubernetes.io/projected/33172513-89e2-4a6e-b352-cb6c516b95dd-kube-api-access-w9pcz\") pod \"feast-credit-scoring-5fd7cf9d69-mkmpb\" (UID: \"33172513-89e2-4a6e-b352-cb6c516b95dd\") " pod="test-ns-feast-rest/feast-credit-scoring-5fd7cf9d69-mkmpb" Jul 29 08:39:38.985768 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:39:38.985605 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/33172513-89e2-4a6e-b352-cb6c516b95dd-online-tls\") pod \"feast-credit-scoring-5fd7cf9d69-mkmpb\" (UID: \"33172513-89e2-4a6e-b352-cb6c516b95dd\") " pod="test-ns-feast-rest/feast-credit-scoring-5fd7cf9d69-mkmpb" Jul 29 08:39:38.985768 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:39:38.985633 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/33172513-89e2-4a6e-b352-cb6c516b95dd-feast-data\") pod \"feast-credit-scoring-5fd7cf9d69-mkmpb\" (UID: \"33172513-89e2-4a6e-b352-cb6c516b95dd\") " pod="test-ns-feast-rest/feast-credit-scoring-5fd7cf9d69-mkmpb" Jul 29 08:39:38.985768 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:39:38.985657 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/33172513-89e2-4a6e-b352-cb6c516b95dd-registry-tls\") pod \"feast-credit-scoring-5fd7cf9d69-mkmpb\" (UID: \"33172513-89e2-4a6e-b352-cb6c516b95dd\") " pod="test-ns-feast-rest/feast-credit-scoring-5fd7cf9d69-mkmpb" Jul 29 08:39:39.086436 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:39:39.086404 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/33172513-89e2-4a6e-b352-cb6c516b95dd-registry-tls\") pod \"feast-credit-scoring-5fd7cf9d69-mkmpb\" (UID: \"33172513-89e2-4a6e-b352-cb6c516b95dd\") " pod="test-ns-feast-rest/feast-credit-scoring-5fd7cf9d69-mkmpb" Jul 29 08:39:39.086595 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:39:39.086464 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-w9pcz\" (UniqueName: \"kubernetes.io/projected/33172513-89e2-4a6e-b352-cb6c516b95dd-kube-api-access-w9pcz\") pod \"feast-credit-scoring-5fd7cf9d69-mkmpb\" (UID: \"33172513-89e2-4a6e-b352-cb6c516b95dd\") " pod="test-ns-feast-rest/feast-credit-scoring-5fd7cf9d69-mkmpb" Jul 29 08:39:39.086595 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:39:39.086490 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/33172513-89e2-4a6e-b352-cb6c516b95dd-online-tls\") pod \"feast-credit-scoring-5fd7cf9d69-mkmpb\" (UID: \"33172513-89e2-4a6e-b352-cb6c516b95dd\") " pod="test-ns-feast-rest/feast-credit-scoring-5fd7cf9d69-mkmpb" Jul 29 08:39:39.086595 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:39:39.086512 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/33172513-89e2-4a6e-b352-cb6c516b95dd-feast-data\") pod \"feast-credit-scoring-5fd7cf9d69-mkmpb\" (UID: \"33172513-89e2-4a6e-b352-cb6c516b95dd\") " pod="test-ns-feast-rest/feast-credit-scoring-5fd7cf9d69-mkmpb" Jul 29 08:39:39.086924 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:39:39.086903 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/33172513-89e2-4a6e-b352-cb6c516b95dd-feast-data\") pod \"feast-credit-scoring-5fd7cf9d69-mkmpb\" (UID: \"33172513-89e2-4a6e-b352-cb6c516b95dd\") " pod="test-ns-feast-rest/feast-credit-scoring-5fd7cf9d69-mkmpb" Jul 29 08:39:39.089179 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:39:39.089139 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/33172513-89e2-4a6e-b352-cb6c516b95dd-registry-tls\") pod \"feast-credit-scoring-5fd7cf9d69-mkmpb\" (UID: \"33172513-89e2-4a6e-b352-cb6c516b95dd\") " pod="test-ns-feast-rest/feast-credit-scoring-5fd7cf9d69-mkmpb" Jul 29 08:39:39.089179 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:39:39.089171 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/33172513-89e2-4a6e-b352-cb6c516b95dd-online-tls\") pod \"feast-credit-scoring-5fd7cf9d69-mkmpb\" (UID: \"33172513-89e2-4a6e-b352-cb6c516b95dd\") " pod="test-ns-feast-rest/feast-credit-scoring-5fd7cf9d69-mkmpb" Jul 29 08:39:39.096456 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:39:39.096425 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-w9pcz\" (UniqueName: \"kubernetes.io/projected/33172513-89e2-4a6e-b352-cb6c516b95dd-kube-api-access-w9pcz\") pod \"feast-credit-scoring-5fd7cf9d69-mkmpb\" (UID: \"33172513-89e2-4a6e-b352-cb6c516b95dd\") " pod="test-ns-feast-rest/feast-credit-scoring-5fd7cf9d69-mkmpb" Jul 29 08:39:39.169307 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:39:39.169281 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="test-ns-feast-rest/feast-credit-scoring-5fd7cf9d69-mkmpb" Jul 29 08:39:39.301626 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:39:39.301595 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["test-ns-feast-rest/feast-credit-scoring-5fd7cf9d69-mkmpb"] Jul 29 08:39:39.303979 ip-10-0-131-138 kubenswrapper[2563]: W0729 08:39:39.303954 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod33172513_89e2_4a6e_b352_cb6c516b95dd.slice/crio-30bd4672f3528e2728c78da1f8fd049b06694a38ebafb533180fd57a90d744ce WatchSource:0}: Error finding container 30bd4672f3528e2728c78da1f8fd049b06694a38ebafb533180fd57a90d744ce: Status 404 returned error can't find the container with id 30bd4672f3528e2728c78da1f8fd049b06694a38ebafb533180fd57a90d744ce Jul 29 08:39:39.805242 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:39:39.805204 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8rp9g_cfd1d8df-e6b7-4a67-81f2-5d83da541e20/konnectivity-agent/0.log" limit=1024 Jul 29 08:39:39.864543 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:39:39.864509 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8rp9g_cfd1d8df-e6b7-4a67-81f2-5d83da541e20/konnectivity-agent/0.log" limit=1024 Jul 29 08:39:40.290226 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:39:40.290190 2563 generic.go:356] "Generic (PLEG): container finished" podID="33172513-89e2-4a6e-b352-cb6c516b95dd" containerID="626b2f737df24bd6941bca10994ad572e5bde4efa42737f3dcc33ac9efafa57f" exitCode=0 Jul 29 08:39:40.290417 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:39:40.290245 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast-rest/feast-credit-scoring-5fd7cf9d69-mkmpb" event={"ID":"33172513-89e2-4a6e-b352-cb6c516b95dd","Type":"ContainerDied","Data":"626b2f737df24bd6941bca10994ad572e5bde4efa42737f3dcc33ac9efafa57f"} Jul 29 08:39:40.290417 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:39:40.290299 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast-rest/feast-credit-scoring-5fd7cf9d69-mkmpb" event={"ID":"33172513-89e2-4a6e-b352-cb6c516b95dd","Type":"ContainerStarted","Data":"30bd4672f3528e2728c78da1f8fd049b06694a38ebafb533180fd57a90d744ce"} Jul 29 08:39:41.296636 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:39:41.296590 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast-rest/feast-credit-scoring-5fd7cf9d69-mkmpb" event={"ID":"33172513-89e2-4a6e-b352-cb6c516b95dd","Type":"ContainerStarted","Data":"6aab0cc7a329af26bdaefdb899ca2600297e2a6d14542439af0c0e1a8e892435"} Jul 29 08:39:47.319905 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:39:47.319873 2563 generic.go:356] "Generic (PLEG): container finished" podID="33172513-89e2-4a6e-b352-cb6c516b95dd" containerID="6aab0cc7a329af26bdaefdb899ca2600297e2a6d14542439af0c0e1a8e892435" exitCode=0 Jul 29 08:39:47.320356 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:39:47.319923 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast-rest/feast-credit-scoring-5fd7cf9d69-mkmpb" event={"ID":"33172513-89e2-4a6e-b352-cb6c516b95dd","Type":"ContainerDied","Data":"6aab0cc7a329af26bdaefdb899ca2600297e2a6d14542439af0c0e1a8e892435"} Jul 29 08:39:48.327207 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:39:48.327171 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast-rest/feast-credit-scoring-5fd7cf9d69-mkmpb" event={"ID":"33172513-89e2-4a6e-b352-cb6c516b95dd","Type":"ContainerStarted","Data":"1482e4ea424dfa72b51e1a41c5cd84f3a167db0c648d2fd7bf9cdf1e19dd40e6"} Jul 29 08:39:48.327607 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:39:48.327231 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast-rest/feast-credit-scoring-5fd7cf9d69-mkmpb" event={"ID":"33172513-89e2-4a6e-b352-cb6c516b95dd","Type":"ContainerStarted","Data":"27ffaa166d65048a9fbd6031c7317bcc874f6b50d3a86338a9cf333cfccc7d2c"} Jul 29 08:39:48.350836 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:39:48.350766 2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="test-ns-feast-rest/feast-credit-scoring-5fd7cf9d69-mkmpb" podStartSLOduration=10.350744747 podStartE2EDuration="10.350744747s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 08:39:38 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 08:39:48.348647815 +0000 UTC m=+1100.929936605" watchObservedRunningTime="2026-07-29 08:39:48.350744747 +0000 UTC m=+1100.932033526" Jul 29 08:39:49.169962 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:39:49.169928 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast-rest/feast-credit-scoring-5fd7cf9d69-mkmpb" Jul 29 08:39:49.170128 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:39:49.169987 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast-rest/feast-credit-scoring-5fd7cf9d69-mkmpb" Jul 29 08:39:51.169837 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:39:51.169800 2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-feast-rest/feast-credit-scoring-5fd7cf9d69-mkmpb" Jul 29 08:39:51.170235 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:39:51.169892 2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-feast-rest/feast-credit-scoring-5fd7cf9d69-mkmpb" Jul 29 08:39:51.171681 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:39:51.171647 2563 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast-rest/feast-credit-scoring-5fd7cf9d69-mkmpb" podUID="33172513-89e2-4a6e-b352-cb6c516b95dd" containerName="registry" probeResult="failure" output="dial tcp 10.134.0.26:6571: connect: connection refused" Jul 29 08:39:51.171681 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:39:51.171664 2563 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast-rest/feast-credit-scoring-5fd7cf9d69-mkmpb" podUID="33172513-89e2-4a6e-b352-cb6c516b95dd" containerName="online" probeResult="failure" output="Get \"https://10.134.0.26:6567/health\": dial tcp 10.134.0.26:6567: connect: connection refused" Jul 29 08:39:54.170164 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:39:54.170125 2563 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast-rest/feast-credit-scoring-5fd7cf9d69-mkmpb" podUID="33172513-89e2-4a6e-b352-cb6c516b95dd" containerName="registry" probeResult="failure" output="dial tcp 10.134.0.26:6571: connect: connection refused" Jul 29 08:39:54.177891 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:39:54.177870 2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-feast-rest/feast-credit-scoring-5fd7cf9d69-mkmpb" Jul 29 08:39:54.182033 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:39:54.182012 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-feast-rest/feast-credit-scoring-5fd7cf9d69-mkmpb" Jul 29 08:39:57.169646 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:39:57.169591 2563 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast-rest/feast-credit-scoring-5fd7cf9d69-mkmpb" podUID="33172513-89e2-4a6e-b352-cb6c516b95dd" containerName="registry" probeResult="failure" output="dial tcp 10.134.0.26:6571: connect: connection refused" Jul 29 08:40:00.171094 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:40:00.171050 2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-feast-rest/feast-credit-scoring-5fd7cf9d69-mkmpb" Jul 29 08:40:00.171999 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:40:00.171976 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-feast-rest/feast-credit-scoring-5fd7cf9d69-mkmpb" Jul 29 08:40:01.158998 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:40:01.158961 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8rp9g_cfd1d8df-e6b7-4a67-81f2-5d83da541e20/konnectivity-agent/0.log" limit=1024 Jul 29 08:40:04.447117 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:40:04.447084 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8rp9g_cfd1d8df-e6b7-4a67-81f2-5d83da541e20/konnectivity-agent/0.log" limit=1024 Jul 29 08:40:04.527281 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:40:04.527228 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8rp9g_cfd1d8df-e6b7-4a67-81f2-5d83da541e20/konnectivity-agent/0.log" limit=1024 Jul 29 08:40:04.531101 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:40:04.531077 2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["test-ns-feast-rest/feast-driver-ranking-6cfb9bff58-hk598"] Jul 29 08:40:04.534655 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:40:04.534640 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="test-ns-feast-rest/feast-driver-ranking-6cfb9bff58-hk598" Jul 29 08:40:04.537301 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:40:04.537281 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-feast-rest\"/\"feast-driver-ranking-online-tls\"" Jul 29 08:40:04.537410 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:40:04.537360 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-feast-rest\"/\"feast-driver-ranking-registry-tls\"" Jul 29 08:40:04.537410 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:40:04.537386 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"test-ns-feast-rest\"/\"feast-driver-ranking-intra-comm\"" Jul 29 08:40:04.537543 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:40:04.537411 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-feast-rest\"/\"feast-driver-ranking-dockercfg-xgjwn\"" Jul 29 08:40:04.545137 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:40:04.545115 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["test-ns-feast-rest/feast-driver-ranking-6cfb9bff58-hk598"] Jul 29 08:40:04.583403 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:40:04.583376 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/5d928686-8806-43d0-8f74-148b81fa2b03-feast-data\") pod \"feast-driver-ranking-6cfb9bff58-hk598\" (UID: \"5d928686-8806-43d0-8f74-148b81fa2b03\") " pod="test-ns-feast-rest/feast-driver-ranking-6cfb9bff58-hk598" Jul 29 08:40:04.583508 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:40:04.583416 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/5d928686-8806-43d0-8f74-148b81fa2b03-online-tls\") pod \"feast-driver-ranking-6cfb9bff58-hk598\" (UID: \"5d928686-8806-43d0-8f74-148b81fa2b03\") " pod="test-ns-feast-rest/feast-driver-ranking-6cfb9bff58-hk598" Jul 29 08:40:04.583508 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:40:04.583463 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4fdst\" (UniqueName: \"kubernetes.io/projected/5d928686-8806-43d0-8f74-148b81fa2b03-kube-api-access-4fdst\") pod \"feast-driver-ranking-6cfb9bff58-hk598\" (UID: \"5d928686-8806-43d0-8f74-148b81fa2b03\") " pod="test-ns-feast-rest/feast-driver-ranking-6cfb9bff58-hk598" Jul 29 08:40:04.583508 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:40:04.583480 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/5d928686-8806-43d0-8f74-148b81fa2b03-registry-tls\") pod \"feast-driver-ranking-6cfb9bff58-hk598\" (UID: \"5d928686-8806-43d0-8f74-148b81fa2b03\") " pod="test-ns-feast-rest/feast-driver-ranking-6cfb9bff58-hk598" Jul 29 08:40:04.590938 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:40:04.590910 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8rp9g_cfd1d8df-e6b7-4a67-81f2-5d83da541e20/konnectivity-agent/0.log" limit=1024 Jul 29 08:40:04.684791 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:40:04.684759 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/5d928686-8806-43d0-8f74-148b81fa2b03-online-tls\") pod \"feast-driver-ranking-6cfb9bff58-hk598\" (UID: \"5d928686-8806-43d0-8f74-148b81fa2b03\") " pod="test-ns-feast-rest/feast-driver-ranking-6cfb9bff58-hk598" Jul 29 08:40:04.684963 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:40:04.684806 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4fdst\" (UniqueName: \"kubernetes.io/projected/5d928686-8806-43d0-8f74-148b81fa2b03-kube-api-access-4fdst\") pod \"feast-driver-ranking-6cfb9bff58-hk598\" (UID: \"5d928686-8806-43d0-8f74-148b81fa2b03\") " pod="test-ns-feast-rest/feast-driver-ranking-6cfb9bff58-hk598" Jul 29 08:40:04.684963 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:40:04.684824 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/5d928686-8806-43d0-8f74-148b81fa2b03-registry-tls\") pod \"feast-driver-ranking-6cfb9bff58-hk598\" (UID: \"5d928686-8806-43d0-8f74-148b81fa2b03\") " pod="test-ns-feast-rest/feast-driver-ranking-6cfb9bff58-hk598" Jul 29 08:40:04.684963 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:40:04.684858 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/5d928686-8806-43d0-8f74-148b81fa2b03-feast-data\") pod \"feast-driver-ranking-6cfb9bff58-hk598\" (UID: \"5d928686-8806-43d0-8f74-148b81fa2b03\") " pod="test-ns-feast-rest/feast-driver-ranking-6cfb9bff58-hk598" Jul 29 08:40:04.684963 ip-10-0-131-138 kubenswrapper[2563]: E0729 08:40:04.684917 2563 secret.go:189] Couldn't get secret test-ns-feast-rest/feast-driver-ranking-online-tls: secret "feast-driver-ranking-online-tls" not found Jul 29 08:40:04.685114 ip-10-0-131-138 kubenswrapper[2563]: E0729 08:40:04.684997 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5d928686-8806-43d0-8f74-148b81fa2b03-online-tls podName:5d928686-8806-43d0-8f74-148b81fa2b03 nodeName:}" failed. No retries permitted until 2026-07-29 08:40:05.18497455 +0000 UTC m=+1117.766263333 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "online-tls" (UniqueName: "kubernetes.io/secret/5d928686-8806-43d0-8f74-148b81fa2b03-online-tls") pod "feast-driver-ranking-6cfb9bff58-hk598" (UID: "5d928686-8806-43d0-8f74-148b81fa2b03") : secret "feast-driver-ranking-online-tls" not found Jul 29 08:40:04.685185 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:40:04.685169 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/5d928686-8806-43d0-8f74-148b81fa2b03-feast-data\") pod \"feast-driver-ranking-6cfb9bff58-hk598\" (UID: \"5d928686-8806-43d0-8f74-148b81fa2b03\") " pod="test-ns-feast-rest/feast-driver-ranking-6cfb9bff58-hk598" Jul 29 08:40:04.687431 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:40:04.687409 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/5d928686-8806-43d0-8f74-148b81fa2b03-registry-tls\") pod \"feast-driver-ranking-6cfb9bff58-hk598\" (UID: \"5d928686-8806-43d0-8f74-148b81fa2b03\") " pod="test-ns-feast-rest/feast-driver-ranking-6cfb9bff58-hk598" Jul 29 08:40:04.694435 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:40:04.694413 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4fdst\" (UniqueName: \"kubernetes.io/projected/5d928686-8806-43d0-8f74-148b81fa2b03-kube-api-access-4fdst\") pod \"feast-driver-ranking-6cfb9bff58-hk598\" (UID: \"5d928686-8806-43d0-8f74-148b81fa2b03\") " pod="test-ns-feast-rest/feast-driver-ranking-6cfb9bff58-hk598" Jul 29 08:40:05.188270 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:40:05.188219 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/5d928686-8806-43d0-8f74-148b81fa2b03-online-tls\") pod \"feast-driver-ranking-6cfb9bff58-hk598\" (UID: \"5d928686-8806-43d0-8f74-148b81fa2b03\") " pod="test-ns-feast-rest/feast-driver-ranking-6cfb9bff58-hk598" Jul 29 08:40:05.188445 ip-10-0-131-138 kubenswrapper[2563]: E0729 08:40:05.188358 2563 secret.go:189] Couldn't get secret test-ns-feast-rest/feast-driver-ranking-online-tls: secret "feast-driver-ranking-online-tls" not found Jul 29 08:40:05.188445 ip-10-0-131-138 kubenswrapper[2563]: E0729 08:40:05.188424 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5d928686-8806-43d0-8f74-148b81fa2b03-online-tls podName:5d928686-8806-43d0-8f74-148b81fa2b03 nodeName:}" failed. No retries permitted until 2026-07-29 08:40:06.188408546 +0000 UTC m=+1118.769697304 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "online-tls" (UniqueName: "kubernetes.io/secret/5d928686-8806-43d0-8f74-148b81fa2b03-online-tls") pod "feast-driver-ranking-6cfb9bff58-hk598" (UID: "5d928686-8806-43d0-8f74-148b81fa2b03") : secret "feast-driver-ranking-online-tls" not found Jul 29 08:40:05.632428 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:40:05.632399 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8rp9g_cfd1d8df-e6b7-4a67-81f2-5d83da541e20/konnectivity-agent/0.log" limit=1024 Jul 29 08:40:06.195393 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:40:06.195358 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/5d928686-8806-43d0-8f74-148b81fa2b03-online-tls\") pod \"feast-driver-ranking-6cfb9bff58-hk598\" (UID: \"5d928686-8806-43d0-8f74-148b81fa2b03\") " pod="test-ns-feast-rest/feast-driver-ranking-6cfb9bff58-hk598" Jul 29 08:40:06.197735 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:40:06.197701 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/5d928686-8806-43d0-8f74-148b81fa2b03-online-tls\") pod \"feast-driver-ranking-6cfb9bff58-hk598\" (UID: \"5d928686-8806-43d0-8f74-148b81fa2b03\") " pod="test-ns-feast-rest/feast-driver-ranking-6cfb9bff58-hk598" Jul 29 08:40:06.344695 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:40:06.344666 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="test-ns-feast-rest/feast-driver-ranking-6cfb9bff58-hk598" Jul 29 08:40:06.479558 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:40:06.479531 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["test-ns-feast-rest/feast-driver-ranking-6cfb9bff58-hk598"] Jul 29 08:40:06.480356 ip-10-0-131-138 kubenswrapper[2563]: W0729 08:40:06.480321 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5d928686_8806_43d0_8f74_148b81fa2b03.slice/crio-de4311312dfaa6316f0bd3634b76076d0bba687c2950eab16ee90b19eb9578a0 WatchSource:0}: Error finding container de4311312dfaa6316f0bd3634b76076d0bba687c2950eab16ee90b19eb9578a0: Status 404 returned error can't find the container with id de4311312dfaa6316f0bd3634b76076d0bba687c2950eab16ee90b19eb9578a0 Jul 29 08:40:07.408196 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:40:07.408158 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast-rest/feast-driver-ranking-6cfb9bff58-hk598" event={"ID":"5d928686-8806-43d0-8f74-148b81fa2b03","Type":"ContainerStarted","Data":"a1a821050d843601c424fc7b7e121f4280eb82b4550cc3f7932711662e1673d8"} Jul 29 08:40:07.408670 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:40:07.408225 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast-rest/feast-driver-ranking-6cfb9bff58-hk598" event={"ID":"5d928686-8806-43d0-8f74-148b81fa2b03","Type":"ContainerStarted","Data":"de4311312dfaa6316f0bd3634b76076d0bba687c2950eab16ee90b19eb9578a0"} Jul 29 08:40:11.424605 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:40:11.424525 2563 generic.go:356] "Generic (PLEG): container finished" podID="5d928686-8806-43d0-8f74-148b81fa2b03" containerID="a1a821050d843601c424fc7b7e121f4280eb82b4550cc3f7932711662e1673d8" exitCode=0 Jul 29 08:40:11.424959 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:40:11.424595 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast-rest/feast-driver-ranking-6cfb9bff58-hk598" event={"ID":"5d928686-8806-43d0-8f74-148b81fa2b03","Type":"ContainerDied","Data":"a1a821050d843601c424fc7b7e121f4280eb82b4550cc3f7932711662e1673d8"} Jul 29 08:40:12.433992 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:40:12.433958 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast-rest/feast-driver-ranking-6cfb9bff58-hk598" event={"ID":"5d928686-8806-43d0-8f74-148b81fa2b03","Type":"ContainerStarted","Data":"6cc111e130c5855ccd98a8071e33e3f9c72892b0292e68a1ff1577ec9df9ebcf"} Jul 29 08:40:18.457322 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:40:18.457291 2563 generic.go:356] "Generic (PLEG): container finished" podID="5d928686-8806-43d0-8f74-148b81fa2b03" containerID="6cc111e130c5855ccd98a8071e33e3f9c72892b0292e68a1ff1577ec9df9ebcf" exitCode=0 Jul 29 08:40:18.457702 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:40:18.457370 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast-rest/feast-driver-ranking-6cfb9bff58-hk598" event={"ID":"5d928686-8806-43d0-8f74-148b81fa2b03","Type":"ContainerDied","Data":"6cc111e130c5855ccd98a8071e33e3f9c72892b0292e68a1ff1577ec9df9ebcf"} Jul 29 08:40:19.464392 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:40:19.464362 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast-rest/feast-driver-ranking-6cfb9bff58-hk598" event={"ID":"5d928686-8806-43d0-8f74-148b81fa2b03","Type":"ContainerStarted","Data":"39d60b1346d1e327c2648f5dd582b4391b00b7eacb08e199caa7a6ab4593165f"} Jul 29 08:40:19.464873 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:40:19.464422 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast-rest/feast-driver-ranking-6cfb9bff58-hk598" event={"ID":"5d928686-8806-43d0-8f74-148b81fa2b03","Type":"ContainerStarted","Data":"f9c50fd646e266133284cbc329657e3473bafb6fa8e40d4183ed7a5da5548ff5"} Jul 29 08:40:19.490102 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:40:19.490038 2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="test-ns-feast-rest/feast-driver-ranking-6cfb9bff58-hk598" podStartSLOduration=15.490016105 podStartE2EDuration="15.490016105s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 08:40:04 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 08:40:19.486681259 +0000 UTC m=+1132.067970042" watchObservedRunningTime="2026-07-29 08:40:19.490016105 +0000 UTC m=+1132.071304884" Jul 29 08:40:21.345824 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:40:21.345786 2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-feast-rest/feast-driver-ranking-6cfb9bff58-hk598" Jul 29 08:40:21.346317 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:40:21.345861 2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-feast-rest/feast-driver-ranking-6cfb9bff58-hk598" Jul 29 08:40:21.347197 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:40:21.347164 2563 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast-rest/feast-driver-ranking-6cfb9bff58-hk598" podUID="5d928686-8806-43d0-8f74-148b81fa2b03" containerName="registry" probeResult="failure" output="dial tcp 10.134.0.27:6571: connect: connection refused" Jul 29 08:40:21.347197 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:40:21.347185 2563 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast-rest/feast-driver-ranking-6cfb9bff58-hk598" podUID="5d928686-8806-43d0-8f74-148b81fa2b03" containerName="online" probeResult="failure" output="Get \"https://10.134.0.27:6567/health\": dial tcp 10.134.0.27:6567: connect: connection refused" Jul 29 08:40:24.345294 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:40:24.345228 2563 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast-rest/feast-driver-ranking-6cfb9bff58-hk598" podUID="5d928686-8806-43d0-8f74-148b81fa2b03" containerName="registry" probeResult="failure" output="dial tcp 10.134.0.27:6571: connect: connection refused" Jul 29 08:40:24.706589 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:40:24.706557 2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-feast-rest/feast-driver-ranking-6cfb9bff58-hk598" Jul 29 08:40:24.707029 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:40:24.707004 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast-rest/feast-driver-ranking-6cfb9bff58-hk598" Jul 29 08:40:24.710018 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:40:24.709979 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-feast-rest/feast-driver-ranking-6cfb9bff58-hk598" Jul 29 08:40:26.345311 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:40:26.345273 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast-rest/feast-driver-ranking-6cfb9bff58-hk598" Jul 29 08:40:27.345552 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:40:27.345513 2563 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast-rest/feast-driver-ranking-6cfb9bff58-hk598" podUID="5d928686-8806-43d0-8f74-148b81fa2b03" containerName="registry" probeResult="failure" output="dial tcp 10.134.0.27:6571: connect: connection refused" Jul 29 08:40:30.345628 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:40:30.345596 2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-feast-rest/feast-driver-ranking-6cfb9bff58-hk598" Jul 29 08:40:30.346386 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:40:30.346366 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-feast-rest/feast-driver-ranking-6cfb9bff58-hk598" Jul 29 08:41:24.808797 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:41:24.808759 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8rp9g_cfd1d8df-e6b7-4a67-81f2-5d83da541e20/konnectivity-agent/0.log" limit=1024 Jul 29 08:41:26.691645 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:41:26.691613 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8rp9g_cfd1d8df-e6b7-4a67-81f2-5d83da541e20/konnectivity-agent/0.log" limit=1024 Jul 29 08:41:45.885942 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:41:45.885909 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8rp9g_cfd1d8df-e6b7-4a67-81f2-5d83da541e20/konnectivity-agent/0.log" limit=1024 Jul 29 08:41:48.092971 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:41:48.092935 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8rp9g_cfd1d8df-e6b7-4a67-81f2-5d83da541e20/konnectivity-agent/0.log" limit=1024 Jul 29 08:41:48.481695 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:41:48.481662 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8rp9g_cfd1d8df-e6b7-4a67-81f2-5d83da541e20/konnectivity-agent/0.log" limit=1024 Jul 29 08:41:48.691439 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:41:48.691404 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8rp9g_cfd1d8df-e6b7-4a67-81f2-5d83da541e20/konnectivity-agent/0.log" limit=1024 Jul 29 08:41:48.729875 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:41:48.729846 2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["test-ns-feast-rest/feast-credit-scoring-5fd7cf9d69-mkmpb"] Jul 29 08:41:48.730153 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:41:48.730123 2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="test-ns-feast-rest/feast-credit-scoring-5fd7cf9d69-mkmpb" podUID="33172513-89e2-4a6e-b352-cb6c516b95dd" containerName="registry" containerID="cri-o://27ffaa166d65048a9fbd6031c7317bcc874f6b50d3a86338a9cf333cfccc7d2c" gracePeriod=30 Jul 29 08:41:48.730268 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:41:48.730212 2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="test-ns-feast-rest/feast-credit-scoring-5fd7cf9d69-mkmpb" podUID="33172513-89e2-4a6e-b352-cb6c516b95dd" containerName="online" containerID="cri-o://1482e4ea424dfa72b51e1a41c5cd84f3a167db0c648d2fd7bf9cdf1e19dd40e6" gracePeriod=30 Jul 29 08:41:48.735291 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:41:48.735208 2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["test-ns-feast-rest/feast-driver-ranking-6cfb9bff58-hk598"] Jul 29 08:41:48.735448 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:41:48.735421 2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="test-ns-feast-rest/feast-driver-ranking-6cfb9bff58-hk598" podUID="5d928686-8806-43d0-8f74-148b81fa2b03" containerName="registry" containerID="cri-o://f9c50fd646e266133284cbc329657e3473bafb6fa8e40d4183ed7a5da5548ff5" gracePeriod=30 Jul 29 08:41:48.735561 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:41:48.735456 2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="test-ns-feast-rest/feast-driver-ranking-6cfb9bff58-hk598" podUID="5d928686-8806-43d0-8f74-148b81fa2b03" containerName="online" containerID="cri-o://39d60b1346d1e327c2648f5dd582b4391b00b7eacb08e199caa7a6ab4593165f" gracePeriod=30 Jul 29 08:41:48.753698 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:41:48.753660 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8rp9g_cfd1d8df-e6b7-4a67-81f2-5d83da541e20/konnectivity-agent/0.log" limit=1024 Jul 29 08:41:49.834728 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:41:49.834642 2563 generic.go:356] "Generic (PLEG): container finished" podID="5d928686-8806-43d0-8f74-148b81fa2b03" containerID="39d60b1346d1e327c2648f5dd582b4391b00b7eacb08e199caa7a6ab4593165f" exitCode=0 Jul 29 08:41:49.834728 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:41:49.834714 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast-rest/feast-driver-ranking-6cfb9bff58-hk598" event={"ID":"5d928686-8806-43d0-8f74-148b81fa2b03","Type":"ContainerDied","Data":"39d60b1346d1e327c2648f5dd582b4391b00b7eacb08e199caa7a6ab4593165f"} Jul 29 08:41:49.836918 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:41:49.836895 2563 generic.go:356] "Generic (PLEG): container finished" podID="33172513-89e2-4a6e-b352-cb6c516b95dd" containerID="1482e4ea424dfa72b51e1a41c5cd84f3a167db0c648d2fd7bf9cdf1e19dd40e6" exitCode=0 Jul 29 08:41:49.837035 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:41:49.836951 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast-rest/feast-credit-scoring-5fd7cf9d69-mkmpb" event={"ID":"33172513-89e2-4a6e-b352-cb6c516b95dd","Type":"ContainerDied","Data":"1482e4ea424dfa72b51e1a41c5cd84f3a167db0c648d2fd7bf9cdf1e19dd40e6"} Jul 29 08:41:50.872942 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:41:50.872909 2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["test-ns-feast-rest/postgres-864846d4f5-mt8rr"] Jul 29 08:41:50.873410 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:41:50.873116 2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="test-ns-feast-rest/postgres-864846d4f5-mt8rr" podUID="4902e580-8aed-491a-87e9-4102b87962c7" containerName="postgres" containerID="cri-o://242c76a462978822711dc92388a2eff54a5e97fd134e6e36745741230a06ee00" gracePeriod=30 Jul 29 08:41:50.877928 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:41:50.877900 2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["test-ns-feast-rest/redis-69bbd798b6-zvdfn"] Jul 29 08:41:50.878322 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:41:50.878291 2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="test-ns-feast-rest/redis-69bbd798b6-zvdfn" podUID="e4d4d2b5-4009-44e5-95e5-7d850e3de60e" containerName="redis" containerID="cri-o://eecd9f4474595daa29bb187c8a4f76865ec2c97a5aa94ce74a087ef1cd001226" gracePeriod=30 Jul 29 08:41:50.945898 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:41:50.945875 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8rp9g_cfd1d8df-e6b7-4a67-81f2-5d83da541e20/konnectivity-agent/0.log" limit=1024 Jul 29 08:41:51.122461 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:41:51.122439 2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="test-ns-feast-rest/postgres-864846d4f5-mt8rr" Jul 29 08:41:51.144723 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:41:51.144700 2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="test-ns-feast-rest/redis-69bbd798b6-zvdfn" Jul 29 08:41:51.148471 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:41:51.148448 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"postgresdata\" (UniqueName: \"kubernetes.io/empty-dir/4902e580-8aed-491a-87e9-4102b87962c7-postgresdata\") pod \"4902e580-8aed-491a-87e9-4102b87962c7\" (UID: \"4902e580-8aed-491a-87e9-4102b87962c7\") " Jul 29 08:41:51.148604 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:41:51.148498 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-j2xfn\" (UniqueName: \"kubernetes.io/projected/4902e580-8aed-491a-87e9-4102b87962c7-kube-api-access-j2xfn\") pod \"4902e580-8aed-491a-87e9-4102b87962c7\" (UID: \"4902e580-8aed-491a-87e9-4102b87962c7\") " Jul 29 08:41:51.150786 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:41:51.150746 2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4902e580-8aed-491a-87e9-4102b87962c7-kube-api-access-j2xfn" (OuterVolumeSpecName: "kube-api-access-j2xfn") pod "4902e580-8aed-491a-87e9-4102b87962c7" (UID: "4902e580-8aed-491a-87e9-4102b87962c7"). InnerVolumeSpecName "kube-api-access-j2xfn". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 29 08:41:51.185620 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:41:51.185584 2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4902e580-8aed-491a-87e9-4102b87962c7-postgresdata" (OuterVolumeSpecName: "postgresdata") pod "4902e580-8aed-491a-87e9-4102b87962c7" (UID: "4902e580-8aed-491a-87e9-4102b87962c7"). InnerVolumeSpecName "postgresdata". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 29 08:41:51.249185 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:41:51.249158 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-bb9nk\" (UniqueName: \"kubernetes.io/projected/e4d4d2b5-4009-44e5-95e5-7d850e3de60e-kube-api-access-bb9nk\") pod \"e4d4d2b5-4009-44e5-95e5-7d850e3de60e\" (UID: \"e4d4d2b5-4009-44e5-95e5-7d850e3de60e\") " Jul 29 08:41:51.249333 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:41:51.249285 2563 reconciler_common.go:299] "Volume detached for volume \"postgresdata\" (UniqueName: \"kubernetes.io/empty-dir/4902e580-8aed-491a-87e9-4102b87962c7-postgresdata\") on node \"ip-10-0-131-138.ec2.internal\" DevicePath \"\"" Jul 29 08:41:51.249333 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:41:51.249297 2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-j2xfn\" (UniqueName: \"kubernetes.io/projected/4902e580-8aed-491a-87e9-4102b87962c7-kube-api-access-j2xfn\") on node \"ip-10-0-131-138.ec2.internal\" DevicePath \"\"" Jul 29 08:41:51.251273 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:41:51.251224 2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e4d4d2b5-4009-44e5-95e5-7d850e3de60e-kube-api-access-bb9nk" (OuterVolumeSpecName: "kube-api-access-bb9nk") pod "e4d4d2b5-4009-44e5-95e5-7d850e3de60e" (UID: "e4d4d2b5-4009-44e5-95e5-7d850e3de60e"). InnerVolumeSpecName "kube-api-access-bb9nk". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 29 08:41:51.350194 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:41:51.350169 2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-bb9nk\" (UniqueName: \"kubernetes.io/projected/e4d4d2b5-4009-44e5-95e5-7d850e3de60e-kube-api-access-bb9nk\") on node \"ip-10-0-131-138.ec2.internal\" DevicePath \"\"" Jul 29 08:41:51.845505 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:41:51.845472 2563 generic.go:356] "Generic (PLEG): container finished" podID="e4d4d2b5-4009-44e5-95e5-7d850e3de60e" containerID="eecd9f4474595daa29bb187c8a4f76865ec2c97a5aa94ce74a087ef1cd001226" exitCode=0 Jul 29 08:41:51.845712 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:41:51.845537 2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="test-ns-feast-rest/redis-69bbd798b6-zvdfn" Jul 29 08:41:51.845712 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:41:51.845541 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast-rest/redis-69bbd798b6-zvdfn" event={"ID":"e4d4d2b5-4009-44e5-95e5-7d850e3de60e","Type":"ContainerDied","Data":"eecd9f4474595daa29bb187c8a4f76865ec2c97a5aa94ce74a087ef1cd001226"} Jul 29 08:41:51.845712 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:41:51.845668 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast-rest/redis-69bbd798b6-zvdfn" event={"ID":"e4d4d2b5-4009-44e5-95e5-7d850e3de60e","Type":"ContainerDied","Data":"42cd3238bf859a0be64c67919420b58665e7d348e453932090bedf212ccb2085"} Jul 29 08:41:51.845712 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:41:51.845701 2563 scope.go:117] "RemoveContainer" containerID="eecd9f4474595daa29bb187c8a4f76865ec2c97a5aa94ce74a087ef1cd001226" Jul 29 08:41:51.846882 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:41:51.846859 2563 generic.go:356] "Generic (PLEG): container finished" podID="4902e580-8aed-491a-87e9-4102b87962c7" containerID="242c76a462978822711dc92388a2eff54a5e97fd134e6e36745741230a06ee00" exitCode=0 Jul 29 08:41:51.846963 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:41:51.846910 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast-rest/postgres-864846d4f5-mt8rr" event={"ID":"4902e580-8aed-491a-87e9-4102b87962c7","Type":"ContainerDied","Data":"242c76a462978822711dc92388a2eff54a5e97fd134e6e36745741230a06ee00"} Jul 29 08:41:51.846963 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:41:51.846937 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast-rest/postgres-864846d4f5-mt8rr" event={"ID":"4902e580-8aed-491a-87e9-4102b87962c7","Type":"ContainerDied","Data":"167c1b12e4b671b62a0ae9b50fca89572ad713bd4a37ce68f365db34399a3fc5"} Jul 29 08:41:51.846963 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:41:51.846913 2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="test-ns-feast-rest/postgres-864846d4f5-mt8rr" Jul 29 08:41:51.854838 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:41:51.854823 2563 scope.go:117] "RemoveContainer" containerID="eecd9f4474595daa29bb187c8a4f76865ec2c97a5aa94ce74a087ef1cd001226" Jul 29 08:41:51.855099 ip-10-0-131-138 kubenswrapper[2563]: E0729 08:41:51.855080 2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"eecd9f4474595daa29bb187c8a4f76865ec2c97a5aa94ce74a087ef1cd001226\": container with ID starting with eecd9f4474595daa29bb187c8a4f76865ec2c97a5aa94ce74a087ef1cd001226 not found: ID does not exist" containerID="eecd9f4474595daa29bb187c8a4f76865ec2c97a5aa94ce74a087ef1cd001226" Jul 29 08:41:51.855191 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:41:51.855109 2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"eecd9f4474595daa29bb187c8a4f76865ec2c97a5aa94ce74a087ef1cd001226"} err="failed to get container status \"eecd9f4474595daa29bb187c8a4f76865ec2c97a5aa94ce74a087ef1cd001226\": rpc error: code = NotFound desc = could not find container \"eecd9f4474595daa29bb187c8a4f76865ec2c97a5aa94ce74a087ef1cd001226\": container with ID starting with eecd9f4474595daa29bb187c8a4f76865ec2c97a5aa94ce74a087ef1cd001226 not found: ID does not exist" Jul 29 08:41:51.855191 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:41:51.855133 2563 scope.go:117] "RemoveContainer" containerID="242c76a462978822711dc92388a2eff54a5e97fd134e6e36745741230a06ee00" Jul 29 08:41:51.862653 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:41:51.862637 2563 scope.go:117] "RemoveContainer" containerID="242c76a462978822711dc92388a2eff54a5e97fd134e6e36745741230a06ee00" Jul 29 08:41:51.862908 ip-10-0-131-138 kubenswrapper[2563]: E0729 08:41:51.862888 2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"242c76a462978822711dc92388a2eff54a5e97fd134e6e36745741230a06ee00\": container with ID starting with 242c76a462978822711dc92388a2eff54a5e97fd134e6e36745741230a06ee00 not found: ID does not exist" containerID="242c76a462978822711dc92388a2eff54a5e97fd134e6e36745741230a06ee00" Jul 29 08:41:51.862965 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:41:51.862915 2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"242c76a462978822711dc92388a2eff54a5e97fd134e6e36745741230a06ee00"} err="failed to get container status \"242c76a462978822711dc92388a2eff54a5e97fd134e6e36745741230a06ee00\": rpc error: code = NotFound desc = could not find container \"242c76a462978822711dc92388a2eff54a5e97fd134e6e36745741230a06ee00\": container with ID starting with 242c76a462978822711dc92388a2eff54a5e97fd134e6e36745741230a06ee00 not found: ID does not exist" Jul 29 08:41:51.870269 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:41:51.870233 2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["test-ns-feast-rest/postgres-864846d4f5-mt8rr"] Jul 29 08:41:51.873673 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:41:51.873655 2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["test-ns-feast-rest/postgres-864846d4f5-mt8rr"] Jul 29 08:41:51.884799 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:41:51.884778 2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["test-ns-feast-rest/redis-69bbd798b6-zvdfn"] Jul 29 08:41:51.888662 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:41:51.888644 2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["test-ns-feast-rest/redis-69bbd798b6-zvdfn"] Jul 29 08:41:51.961848 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:41:51.961814 2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="4902e580-8aed-491a-87e9-4102b87962c7" path="/var/lib/kubelet/pods/4902e580-8aed-491a-87e9-4102b87962c7/volumes" Jul 29 08:41:51.962156 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:41:51.962141 2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e4d4d2b5-4009-44e5-95e5-7d850e3de60e" path="/var/lib/kubelet/pods/e4d4d2b5-4009-44e5-95e5-7d850e3de60e/volumes" Jul 29 08:41:54.179350 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:41:54.179310 2563 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast-rest/feast-credit-scoring-5fd7cf9d69-mkmpb" podUID="33172513-89e2-4a6e-b352-cb6c516b95dd" containerName="online" probeResult="failure" output="Get \"https://10.134.0.26:6567/health\": dial tcp 10.134.0.26:6567: connect: connection refused" Jul 29 08:41:54.707597 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:41:54.707551 2563 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast-rest/feast-driver-ranking-6cfb9bff58-hk598" podUID="5d928686-8806-43d0-8f74-148b81fa2b03" containerName="online" probeResult="failure" output="Get \"https://10.134.0.27:6567/health\": dial tcp 10.134.0.27:6567: connect: connection refused" Jul 29 08:41:54.958913 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:41:54.958824 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8rp9g_cfd1d8df-e6b7-4a67-81f2-5d83da541e20/konnectivity-agent/0.log" limit=1024 Jul 29 08:41:55.022856 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:41:55.022825 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8rp9g_cfd1d8df-e6b7-4a67-81f2-5d83da541e20/konnectivity-agent/0.log" limit=1024 Jul 29 08:41:59.890552 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:41:59.890519 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8rp9g_cfd1d8df-e6b7-4a67-81f2-5d83da541e20/konnectivity-agent/0.log" limit=1024 Jul 29 08:41:59.978595 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:41:59.978542 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8rp9g_cfd1d8df-e6b7-4a67-81f2-5d83da541e20/konnectivity-agent/0.log" limit=1024 Jul 29 08:42:00.466635 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:42:00.466601 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8rp9g_cfd1d8df-e6b7-4a67-81f2-5d83da541e20/konnectivity-agent/0.log" limit=1024 Jul 29 08:42:04.178594 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:42:04.178548 2563 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast-rest/feast-credit-scoring-5fd7cf9d69-mkmpb" podUID="33172513-89e2-4a6e-b352-cb6c516b95dd" containerName="online" probeResult="failure" output="Get \"https://10.134.0.26:6567/health\": dial tcp 10.134.0.26:6567: connect: connection refused" Jul 29 08:42:04.707273 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:42:04.707220 2563 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast-rest/feast-driver-ranking-6cfb9bff58-hk598" podUID="5d928686-8806-43d0-8f74-148b81fa2b03" containerName="online" probeResult="failure" output="Get \"https://10.134.0.27:6567/health\": dial tcp 10.134.0.27:6567: connect: connection refused" Jul 29 08:42:14.178612 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:42:14.178562 2563 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast-rest/feast-credit-scoring-5fd7cf9d69-mkmpb" podUID="33172513-89e2-4a6e-b352-cb6c516b95dd" containerName="online" probeResult="failure" output="Get \"https://10.134.0.26:6567/health\": dial tcp 10.134.0.26:6567: connect: connection refused" Jul 29 08:42:14.179036 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:42:14.178684 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast-rest/feast-credit-scoring-5fd7cf9d69-mkmpb" Jul 29 08:42:14.707702 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:42:14.707657 2563 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast-rest/feast-driver-ranking-6cfb9bff58-hk598" podUID="5d928686-8806-43d0-8f74-148b81fa2b03" containerName="online" probeResult="failure" output="Get \"https://10.134.0.27:6567/health\": dial tcp 10.134.0.27:6567: connect: connection refused" Jul 29 08:42:14.707902 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:42:14.707800 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast-rest/feast-driver-ranking-6cfb9bff58-hk598" Jul 29 08:42:16.685719 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:42:16.685681 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8rp9g_cfd1d8df-e6b7-4a67-81f2-5d83da541e20/konnectivity-agent/0.log" limit=1024 Jul 29 08:42:16.787355 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:42:16.787322 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8rp9g_cfd1d8df-e6b7-4a67-81f2-5d83da541e20/konnectivity-agent/0.log" limit=1024 Jul 29 08:42:17.031276 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:42:17.031222 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8rp9g_cfd1d8df-e6b7-4a67-81f2-5d83da541e20/konnectivity-agent/0.log" limit=1024 Jul 29 08:42:17.130376 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:42:17.130346 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8rp9g_cfd1d8df-e6b7-4a67-81f2-5d83da541e20/konnectivity-agent/0.log" limit=1024 Jul 29 08:42:17.375774 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:42:17.375689 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8rp9g_cfd1d8df-e6b7-4a67-81f2-5d83da541e20/konnectivity-agent/0.log" limit=1024 Jul 29 08:42:17.476087 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:42:17.476052 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8rp9g_cfd1d8df-e6b7-4a67-81f2-5d83da541e20/konnectivity-agent/0.log" limit=1024 Jul 29 08:42:18.100370 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:42:18.100276 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8rp9g_cfd1d8df-e6b7-4a67-81f2-5d83da541e20/konnectivity-agent/0.log" limit=1024 Jul 29 08:42:18.199849 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:42:18.199817 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8rp9g_cfd1d8df-e6b7-4a67-81f2-5d83da541e20/konnectivity-agent/0.log" limit=1024 Jul 29 08:42:18.949369 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:42:18.949343 2563 generic.go:356] "Generic (PLEG): container finished" podID="5d928686-8806-43d0-8f74-148b81fa2b03" containerID="f9c50fd646e266133284cbc329657e3473bafb6fa8e40d4183ed7a5da5548ff5" exitCode=137 Jul 29 08:42:18.949479 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:42:18.949418 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast-rest/feast-driver-ranking-6cfb9bff58-hk598" event={"ID":"5d928686-8806-43d0-8f74-148b81fa2b03","Type":"ContainerDied","Data":"f9c50fd646e266133284cbc329657e3473bafb6fa8e40d4183ed7a5da5548ff5"} Jul 29 08:42:18.951650 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:42:18.951628 2563 generic.go:356] "Generic (PLEG): container finished" podID="33172513-89e2-4a6e-b352-cb6c516b95dd" containerID="27ffaa166d65048a9fbd6031c7317bcc874f6b50d3a86338a9cf333cfccc7d2c" exitCode=137 Jul 29 08:42:18.951750 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:42:18.951684 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast-rest/feast-credit-scoring-5fd7cf9d69-mkmpb" event={"ID":"33172513-89e2-4a6e-b352-cb6c516b95dd","Type":"ContainerDied","Data":"27ffaa166d65048a9fbd6031c7317bcc874f6b50d3a86338a9cf333cfccc7d2c"} Jul 29 08:42:19.402010 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:42:19.401991 2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="test-ns-feast-rest/feast-credit-scoring-5fd7cf9d69-mkmpb" Jul 29 08:42:19.404615 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:42:19.404600 2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="test-ns-feast-rest/feast-driver-ranking-6cfb9bff58-hk598" Jul 29 08:42:19.462509 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:42:19.462483 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/33172513-89e2-4a6e-b352-cb6c516b95dd-registry-tls\") pod \"33172513-89e2-4a6e-b352-cb6c516b95dd\" (UID: \"33172513-89e2-4a6e-b352-cb6c516b95dd\") " Jul 29 08:42:19.462681 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:42:19.462518 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/33172513-89e2-4a6e-b352-cb6c516b95dd-feast-data\") pod \"33172513-89e2-4a6e-b352-cb6c516b95dd\" (UID: \"33172513-89e2-4a6e-b352-cb6c516b95dd\") " Jul 29 08:42:19.462681 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:42:19.462540 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/5d928686-8806-43d0-8f74-148b81fa2b03-online-tls\") pod \"5d928686-8806-43d0-8f74-148b81fa2b03\" (UID: \"5d928686-8806-43d0-8f74-148b81fa2b03\") " Jul 29 08:42:19.462681 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:42:19.462569 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-w9pcz\" (UniqueName: \"kubernetes.io/projected/33172513-89e2-4a6e-b352-cb6c516b95dd-kube-api-access-w9pcz\") pod \"33172513-89e2-4a6e-b352-cb6c516b95dd\" (UID: \"33172513-89e2-4a6e-b352-cb6c516b95dd\") " Jul 29 08:42:19.462681 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:42:19.462600 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/5d928686-8806-43d0-8f74-148b81fa2b03-feast-data\") pod \"5d928686-8806-43d0-8f74-148b81fa2b03\" (UID: \"5d928686-8806-43d0-8f74-148b81fa2b03\") " Jul 29 08:42:19.462681 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:42:19.462631 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/33172513-89e2-4a6e-b352-cb6c516b95dd-online-tls\") pod \"33172513-89e2-4a6e-b352-cb6c516b95dd\" (UID: \"33172513-89e2-4a6e-b352-cb6c516b95dd\") " Jul 29 08:42:19.462681 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:42:19.462680 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/5d928686-8806-43d0-8f74-148b81fa2b03-registry-tls\") pod \"5d928686-8806-43d0-8f74-148b81fa2b03\" (UID: \"5d928686-8806-43d0-8f74-148b81fa2b03\") " Jul 29 08:42:19.462996 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:42:19.462749 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-4fdst\" (UniqueName: \"kubernetes.io/projected/5d928686-8806-43d0-8f74-148b81fa2b03-kube-api-access-4fdst\") pod \"5d928686-8806-43d0-8f74-148b81fa2b03\" (UID: \"5d928686-8806-43d0-8f74-148b81fa2b03\") " Jul 29 08:42:19.463196 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:42:19.463140 2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5d928686-8806-43d0-8f74-148b81fa2b03-feast-data" (OuterVolumeSpecName: "feast-data") pod "5d928686-8806-43d0-8f74-148b81fa2b03" (UID: "5d928686-8806-43d0-8f74-148b81fa2b03"). InnerVolumeSpecName "feast-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 29 08:42:19.465565 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:42:19.465526 2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5d928686-8806-43d0-8f74-148b81fa2b03-kube-api-access-4fdst" (OuterVolumeSpecName: "kube-api-access-4fdst") pod "5d928686-8806-43d0-8f74-148b81fa2b03" (UID: "5d928686-8806-43d0-8f74-148b81fa2b03"). InnerVolumeSpecName "kube-api-access-4fdst". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 29 08:42:19.465760 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:42:19.465736 2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/33172513-89e2-4a6e-b352-cb6c516b95dd-online-tls" (OuterVolumeSpecName: "online-tls") pod "33172513-89e2-4a6e-b352-cb6c516b95dd" (UID: "33172513-89e2-4a6e-b352-cb6c516b95dd"). InnerVolumeSpecName "online-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 29 08:42:19.465866 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:42:19.465785 2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/33172513-89e2-4a6e-b352-cb6c516b95dd-registry-tls" (OuterVolumeSpecName: "registry-tls") pod "33172513-89e2-4a6e-b352-cb6c516b95dd" (UID: "33172513-89e2-4a6e-b352-cb6c516b95dd"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 29 08:42:19.465983 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:42:19.465955 2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/33172513-89e2-4a6e-b352-cb6c516b95dd-kube-api-access-w9pcz" (OuterVolumeSpecName: "kube-api-access-w9pcz") pod "33172513-89e2-4a6e-b352-cb6c516b95dd" (UID: "33172513-89e2-4a6e-b352-cb6c516b95dd"). InnerVolumeSpecName "kube-api-access-w9pcz". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 29 08:42:19.466377 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:42:19.466346 2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5d928686-8806-43d0-8f74-148b81fa2b03-online-tls" (OuterVolumeSpecName: "online-tls") pod "5d928686-8806-43d0-8f74-148b81fa2b03" (UID: "5d928686-8806-43d0-8f74-148b81fa2b03"). InnerVolumeSpecName "online-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 29 08:42:19.466487 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:42:19.466408 2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5d928686-8806-43d0-8f74-148b81fa2b03-registry-tls" (OuterVolumeSpecName: "registry-tls") pod "5d928686-8806-43d0-8f74-148b81fa2b03" (UID: "5d928686-8806-43d0-8f74-148b81fa2b03"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 29 08:42:19.473304 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:42:19.473276 2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/33172513-89e2-4a6e-b352-cb6c516b95dd-feast-data" (OuterVolumeSpecName: "feast-data") pod "33172513-89e2-4a6e-b352-cb6c516b95dd" (UID: "33172513-89e2-4a6e-b352-cb6c516b95dd"). InnerVolumeSpecName "feast-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 29 08:42:19.563668 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:42:19.563600 2563 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/5d928686-8806-43d0-8f74-148b81fa2b03-registry-tls\") on node \"ip-10-0-131-138.ec2.internal\" DevicePath \"\"" Jul 29 08:42:19.563668 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:42:19.563624 2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-4fdst\" (UniqueName: \"kubernetes.io/projected/5d928686-8806-43d0-8f74-148b81fa2b03-kube-api-access-4fdst\") on node \"ip-10-0-131-138.ec2.internal\" DevicePath \"\"" Jul 29 08:42:19.563668 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:42:19.563635 2563 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/33172513-89e2-4a6e-b352-cb6c516b95dd-registry-tls\") on node \"ip-10-0-131-138.ec2.internal\" DevicePath \"\"" Jul 29 08:42:19.563668 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:42:19.563643 2563 reconciler_common.go:299] "Volume detached for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/33172513-89e2-4a6e-b352-cb6c516b95dd-feast-data\") on node \"ip-10-0-131-138.ec2.internal\" DevicePath \"\"" Jul 29 08:42:19.563668 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:42:19.563650 2563 reconciler_common.go:299] "Volume detached for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/5d928686-8806-43d0-8f74-148b81fa2b03-online-tls\") on node \"ip-10-0-131-138.ec2.internal\" DevicePath \"\"" Jul 29 08:42:19.563668 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:42:19.563658 2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-w9pcz\" (UniqueName: \"kubernetes.io/projected/33172513-89e2-4a6e-b352-cb6c516b95dd-kube-api-access-w9pcz\") on node \"ip-10-0-131-138.ec2.internal\" DevicePath \"\"" Jul 29 08:42:19.563668 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:42:19.563665 2563 reconciler_common.go:299] "Volume detached for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/5d928686-8806-43d0-8f74-148b81fa2b03-feast-data\") on node \"ip-10-0-131-138.ec2.internal\" DevicePath \"\"" Jul 29 08:42:19.563668 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:42:19.563673 2563 reconciler_common.go:299] "Volume detached for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/33172513-89e2-4a6e-b352-cb6c516b95dd-online-tls\") on node \"ip-10-0-131-138.ec2.internal\" DevicePath \"\"" Jul 29 08:42:19.957888 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:42:19.957862 2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="test-ns-feast-rest/feast-driver-ranking-6cfb9bff58-hk598" Jul 29 08:42:19.960308 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:42:19.960289 2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="test-ns-feast-rest/feast-credit-scoring-5fd7cf9d69-mkmpb" Jul 29 08:42:19.962994 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:42:19.962965 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast-rest/feast-driver-ranking-6cfb9bff58-hk598" event={"ID":"5d928686-8806-43d0-8f74-148b81fa2b03","Type":"ContainerDied","Data":"de4311312dfaa6316f0bd3634b76076d0bba687c2950eab16ee90b19eb9578a0"} Jul 29 08:42:19.963102 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:42:19.963021 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast-rest/feast-credit-scoring-5fd7cf9d69-mkmpb" event={"ID":"33172513-89e2-4a6e-b352-cb6c516b95dd","Type":"ContainerDied","Data":"30bd4672f3528e2728c78da1f8fd049b06694a38ebafb533180fd57a90d744ce"} Jul 29 08:42:19.963102 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:42:19.963056 2563 scope.go:117] "RemoveContainer" containerID="39d60b1346d1e327c2648f5dd582b4391b00b7eacb08e199caa7a6ab4593165f" Jul 29 08:42:19.973637 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:42:19.973623 2563 scope.go:117] "RemoveContainer" containerID="f9c50fd646e266133284cbc329657e3473bafb6fa8e40d4183ed7a5da5548ff5" Jul 29 08:42:19.981060 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:42:19.981043 2563 scope.go:117] "RemoveContainer" containerID="6cc111e130c5855ccd98a8071e33e3f9c72892b0292e68a1ff1577ec9df9ebcf" Jul 29 08:42:19.987377 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:42:19.987356 2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["test-ns-feast-rest/feast-driver-ranking-6cfb9bff58-hk598"] Jul 29 08:42:19.988139 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:42:19.988123 2563 scope.go:117] "RemoveContainer" containerID="a1a821050d843601c424fc7b7e121f4280eb82b4550cc3f7932711662e1673d8" Jul 29 08:42:19.994080 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:42:19.994058 2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["test-ns-feast-rest/feast-driver-ranking-6cfb9bff58-hk598"] Jul 29 08:42:19.995389 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:42:19.995374 2563 scope.go:117] "RemoveContainer" containerID="1482e4ea424dfa72b51e1a41c5cd84f3a167db0c648d2fd7bf9cdf1e19dd40e6" Jul 29 08:42:20.002362 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:42:20.002349 2563 scope.go:117] "RemoveContainer" containerID="27ffaa166d65048a9fbd6031c7317bcc874f6b50d3a86338a9cf333cfccc7d2c" Jul 29 08:42:20.009743 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:42:20.009726 2563 scope.go:117] "RemoveContainer" containerID="6aab0cc7a329af26bdaefdb899ca2600297e2a6d14542439af0c0e1a8e892435" Jul 29 08:42:20.010638 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:42:20.010574 2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["test-ns-feast-rest/feast-credit-scoring-5fd7cf9d69-mkmpb"] Jul 29 08:42:20.012894 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:42:20.012875 2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["test-ns-feast-rest/feast-credit-scoring-5fd7cf9d69-mkmpb"] Jul 29 08:42:20.016510 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:42:20.016494 2563 scope.go:117] "RemoveContainer" containerID="626b2f737df24bd6941bca10994ad572e5bde4efa42737f3dcc33ac9efafa57f" Jul 29 08:42:21.961643 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:42:21.961607 2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="33172513-89e2-4a6e-b352-cb6c516b95dd" path="/var/lib/kubelet/pods/33172513-89e2-4a6e-b352-cb6c516b95dd/volumes" Jul 29 08:42:21.962088 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:42:21.962075 2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="5d928686-8806-43d0-8f74-148b81fa2b03" path="/var/lib/kubelet/pods/5d928686-8806-43d0-8f74-148b81fa2b03/volumes" Jul 29 08:42:28.433771 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:42:28.433689 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8rp9g_cfd1d8df-e6b7-4a67-81f2-5d83da541e20/konnectivity-agent/0.log" limit=1024 Jul 29 08:42:28.501632 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:42:28.501588 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8rp9g_cfd1d8df-e6b7-4a67-81f2-5d83da541e20/konnectivity-agent/0.log" limit=1024 Jul 29 08:42:29.192714 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:42:29.192683 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8rp9g_cfd1d8df-e6b7-4a67-81f2-5d83da541e20/konnectivity-agent/0.log" limit=1024 Jul 29 08:43:52.566526 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:43:52.566441 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8rp9g_cfd1d8df-e6b7-4a67-81f2-5d83da541e20/konnectivity-agent/0.log" limit=1024 Jul 29 08:43:52.665131 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:43:52.665093 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8rp9g_cfd1d8df-e6b7-4a67-81f2-5d83da541e20/konnectivity-agent/0.log" limit=1024 Jul 29 08:43:52.767777 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:43:52.767747 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8rp9g_cfd1d8df-e6b7-4a67-81f2-5d83da541e20/konnectivity-agent/0.log" limit=1024 Jul 29 08:43:52.873622 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:43:52.873530 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8rp9g_cfd1d8df-e6b7-4a67-81f2-5d83da541e20/konnectivity-agent/0.log" limit=1024 Jul 29 08:43:53.061883 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:43:53.061852 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8rp9g_cfd1d8df-e6b7-4a67-81f2-5d83da541e20/konnectivity-agent/0.log" limit=1024 Jul 29 08:43:53.306489 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:43:53.306452 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8rp9g_cfd1d8df-e6b7-4a67-81f2-5d83da541e20/konnectivity-agent/0.log" limit=1024 Jul 29 08:43:55.178222 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:43:55.178192 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8rp9g_cfd1d8df-e6b7-4a67-81f2-5d83da541e20/konnectivity-agent/0.log" limit=1024 Jul 29 08:45:16.486199 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:45:16.486168 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8rp9g_cfd1d8df-e6b7-4a67-81f2-5d83da541e20/konnectivity-agent/0.log" limit=1024 Jul 29 08:45:16.584674 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:45:16.584632 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8rp9g_cfd1d8df-e6b7-4a67-81f2-5d83da541e20/konnectivity-agent/0.log" limit=1024 Jul 29 08:45:16.687535 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:45:16.687498 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8rp9g_cfd1d8df-e6b7-4a67-81f2-5d83da541e20/konnectivity-agent/0.log" limit=1024 Jul 29 08:45:16.787102 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:45:16.787014 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8rp9g_cfd1d8df-e6b7-4a67-81f2-5d83da541e20/konnectivity-agent/0.log" limit=1024 Jul 29 08:45:16.995607 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:45:16.995571 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8rp9g_cfd1d8df-e6b7-4a67-81f2-5d83da541e20/konnectivity-agent/0.log" limit=1024 Jul 29 08:45:17.098230 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:45:17.098157 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8rp9g_cfd1d8df-e6b7-4a67-81f2-5d83da541e20/konnectivity-agent/0.log" limit=1024 Jul 29 08:45:17.371071 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:45:17.370989 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8rp9g_cfd1d8df-e6b7-4a67-81f2-5d83da541e20/konnectivity-agent/0.log" limit=1024 Jul 29 08:46:39.356280 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:46:39.356217 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8rp9g_cfd1d8df-e6b7-4a67-81f2-5d83da541e20/konnectivity-agent/0.log" limit=1024 Jul 29 08:46:39.413887 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:46:39.413862 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8rp9g_cfd1d8df-e6b7-4a67-81f2-5d83da541e20/konnectivity-agent/0.log" limit=1024 Jul 29 08:46:43.462020 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:46:43.461983 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8rp9g_cfd1d8df-e6b7-4a67-81f2-5d83da541e20/konnectivity-agent/0.log" limit=1024 Jul 29 08:47:21.249402 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:47:21.249367 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8rp9g_cfd1d8df-e6b7-4a67-81f2-5d83da541e20/konnectivity-agent/0.log" limit=1024 Jul 29 08:47:21.375053 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:47:21.375020 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8rp9g_cfd1d8df-e6b7-4a67-81f2-5d83da541e20/konnectivity-agent/0.log" limit=1024 Jul 29 08:47:24.663189 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:47:24.663151 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8rp9g_cfd1d8df-e6b7-4a67-81f2-5d83da541e20/konnectivity-agent/0.log" limit=1024 Jul 29 08:47:27.242088 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:47:27.242041 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8rp9g_cfd1d8df-e6b7-4a67-81f2-5d83da541e20/konnectivity-agent/0.log" limit=1024 Jul 29 08:47:27.297455 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:47:27.297410 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8rp9g_cfd1d8df-e6b7-4a67-81f2-5d83da541e20/konnectivity-agent/0.log" limit=1024 Jul 29 08:47:48.672277 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:47:48.672227 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8rp9g_cfd1d8df-e6b7-4a67-81f2-5d83da541e20/konnectivity-agent/0.log" limit=1024 Jul 29 08:47:48.729162 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:47:48.729133 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8rp9g_cfd1d8df-e6b7-4a67-81f2-5d83da541e20/konnectivity-agent/0.log" limit=1024 Jul 29 08:47:49.486705 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:47:49.486675 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8rp9g_cfd1d8df-e6b7-4a67-81f2-5d83da541e20/konnectivity-agent/0.log" limit=1024 Jul 29 08:47:49.585666 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:47:49.585610 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8rp9g_cfd1d8df-e6b7-4a67-81f2-5d83da541e20/konnectivity-agent/0.log" limit=1024 Jul 29 08:47:49.687985 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:47:49.687928 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8rp9g_cfd1d8df-e6b7-4a67-81f2-5d83da541e20/konnectivity-agent/0.log" limit=1024 Jul 29 08:47:49.829235 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:47:49.829144 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8rp9g_cfd1d8df-e6b7-4a67-81f2-5d83da541e20/konnectivity-agent/0.log" limit=1024 Jul 29 08:47:49.930421 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:47:49.930378 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8rp9g_cfd1d8df-e6b7-4a67-81f2-5d83da541e20/konnectivity-agent/0.log" limit=1024 Jul 29 08:48:40.998397 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:48:40.998310 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8rp9g_cfd1d8df-e6b7-4a67-81f2-5d83da541e20/konnectivity-agent/0.log" limit=1024 Jul 29 08:49:10.039668 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:49:10.039634 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8rp9g_cfd1d8df-e6b7-4a67-81f2-5d83da541e20/konnectivity-agent/0.log" limit=1024 Jul 29 08:49:11.248398 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:49:11.248364 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8rp9g_cfd1d8df-e6b7-4a67-81f2-5d83da541e20/konnectivity-agent/0.log" limit=1024 Jul 29 08:50:31.830882 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:50:31.830846 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8rp9g_cfd1d8df-e6b7-4a67-81f2-5d83da541e20/konnectivity-agent/0.log" limit=1024 Jul 29 08:50:32.273136 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:50:32.273099 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8rp9g_cfd1d8df-e6b7-4a67-81f2-5d83da541e20/konnectivity-agent/0.log" limit=1024 Jul 29 08:50:32.353090 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:50:32.353032 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8rp9g_cfd1d8df-e6b7-4a67-81f2-5d83da541e20/konnectivity-agent/0.log" limit=1024 Jul 29 08:51:02.522171 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:51:02.522140 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8rp9g_cfd1d8df-e6b7-4a67-81f2-5d83da541e20/konnectivity-agent/0.log" limit=1024 Jul 29 08:51:02.603864 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:51:02.603833 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8rp9g_cfd1d8df-e6b7-4a67-81f2-5d83da541e20/konnectivity-agent/0.log" limit=1024 Jul 29 08:51:52.886535 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:51:52.886503 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8rp9g_cfd1d8df-e6b7-4a67-81f2-5d83da541e20/konnectivity-agent/0.log" limit=1024 Jul 29 08:51:52.960213 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:51:52.960179 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8rp9g_cfd1d8df-e6b7-4a67-81f2-5d83da541e20/konnectivity-agent/0.log" limit=1024 Jul 29 08:51:53.032033 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:51:53.032004 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8rp9g_cfd1d8df-e6b7-4a67-81f2-5d83da541e20/konnectivity-agent/0.log" limit=1024 Jul 29 08:51:53.097973 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:51:53.097945 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8rp9g_cfd1d8df-e6b7-4a67-81f2-5d83da541e20/konnectivity-agent/0.log" limit=1024 Jul 29 08:51:58.179416 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:51:58.179374 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8rp9g_cfd1d8df-e6b7-4a67-81f2-5d83da541e20/konnectivity-agent/0.log" limit=1024 Jul 29 08:52:23.296246 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:52:23.296209 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8rp9g_cfd1d8df-e6b7-4a67-81f2-5d83da541e20/konnectivity-agent/0.log" limit=1024 Jul 29 08:52:23.396493 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:52:23.396463 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8rp9g_cfd1d8df-e6b7-4a67-81f2-5d83da541e20/konnectivity-agent/0.log" limit=1024 Jul 29 08:52:36.325417 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:52:36.325382 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8rp9g_cfd1d8df-e6b7-4a67-81f2-5d83da541e20/konnectivity-agent/0.log" limit=1024 Jul 29 08:52:36.463052 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:52:36.463007 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8rp9g_cfd1d8df-e6b7-4a67-81f2-5d83da541e20/konnectivity-agent/0.log" limit=1024 Jul 29 08:52:42.127463 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:52:42.127427 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8rp9g_cfd1d8df-e6b7-4a67-81f2-5d83da541e20/konnectivity-agent/0.log" limit=1024 Jul 29 08:52:42.186312 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:52:42.186279 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8rp9g_cfd1d8df-e6b7-4a67-81f2-5d83da541e20/konnectivity-agent/0.log" limit=1024 Jul 29 08:52:42.706892 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:52:42.706856 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8rp9g_cfd1d8df-e6b7-4a67-81f2-5d83da541e20/konnectivity-agent/0.log" limit=1024 Jul 29 08:52:42.761473 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:52:42.761438 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8rp9g_cfd1d8df-e6b7-4a67-81f2-5d83da541e20/konnectivity-agent/0.log" limit=1024 Jul 29 08:52:59.817520 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:52:59.817441 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8rp9g_cfd1d8df-e6b7-4a67-81f2-5d83da541e20/konnectivity-agent/0.log" limit=1024 Jul 29 08:52:59.942324 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:52:59.942295 2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-qrb25/must-gather-tsvpr"] Jul 29 08:52:59.942568 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:52:59.942555 2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5d928686-8806-43d0-8f74-148b81fa2b03" containerName="registry" Jul 29 08:52:59.942611 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:52:59.942570 2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="5d928686-8806-43d0-8f74-148b81fa2b03" containerName="registry" Jul 29 08:52:59.942611 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:52:59.942587 2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4902e580-8aed-491a-87e9-4102b87962c7" containerName="postgres" Jul 29 08:52:59.942611 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:52:59.942592 2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="4902e580-8aed-491a-87e9-4102b87962c7" containerName="postgres" Jul 29 08:52:59.942611 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:52:59.942600 2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="33172513-89e2-4a6e-b352-cb6c516b95dd" containerName="feast-init" Jul 29 08:52:59.942611 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:52:59.942605 2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="33172513-89e2-4a6e-b352-cb6c516b95dd" containerName="feast-init" Jul 29 08:52:59.942770 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:52:59.942612 2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="33172513-89e2-4a6e-b352-cb6c516b95dd" containerName="online" Jul 29 08:52:59.942770 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:52:59.942620 2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="33172513-89e2-4a6e-b352-cb6c516b95dd" containerName="online" Jul 29 08:52:59.942770 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:52:59.942628 2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="33172513-89e2-4a6e-b352-cb6c516b95dd" containerName="feast-apply" Jul 29 08:52:59.942770 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:52:59.942633 2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="33172513-89e2-4a6e-b352-cb6c516b95dd" containerName="feast-apply" Jul 29 08:52:59.942770 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:52:59.942639 2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="33172513-89e2-4a6e-b352-cb6c516b95dd" containerName="registry" Jul 29 08:52:59.942770 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:52:59.942643 2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="33172513-89e2-4a6e-b352-cb6c516b95dd" containerName="registry" Jul 29 08:52:59.942770 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:52:59.942649 2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5d928686-8806-43d0-8f74-148b81fa2b03" containerName="feast-init" Jul 29 08:52:59.942770 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:52:59.942654 2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="5d928686-8806-43d0-8f74-148b81fa2b03" containerName="feast-init" Jul 29 08:52:59.942770 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:52:59.942660 2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e4d4d2b5-4009-44e5-95e5-7d850e3de60e" containerName="redis" Jul 29 08:52:59.942770 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:52:59.942665 2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="e4d4d2b5-4009-44e5-95e5-7d850e3de60e" containerName="redis" Jul 29 08:52:59.942770 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:52:59.942670 2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5d928686-8806-43d0-8f74-148b81fa2b03" containerName="feast-apply" Jul 29 08:52:59.942770 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:52:59.942675 2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="5d928686-8806-43d0-8f74-148b81fa2b03" containerName="feast-apply" Jul 29 08:52:59.942770 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:52:59.942680 2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5d928686-8806-43d0-8f74-148b81fa2b03" containerName="online" Jul 29 08:52:59.942770 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:52:59.942685 2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="5d928686-8806-43d0-8f74-148b81fa2b03" containerName="online" Jul 29 08:52:59.942770 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:52:59.942727 2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="5d928686-8806-43d0-8f74-148b81fa2b03" containerName="online" Jul 29 08:52:59.942770 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:52:59.942734 2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="5d928686-8806-43d0-8f74-148b81fa2b03" containerName="registry" Jul 29 08:52:59.942770 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:52:59.942740 2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="33172513-89e2-4a6e-b352-cb6c516b95dd" containerName="online" Jul 29 08:52:59.942770 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:52:59.942747 2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="e4d4d2b5-4009-44e5-95e5-7d850e3de60e" containerName="redis" Jul 29 08:52:59.942770 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:52:59.942752 2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="4902e580-8aed-491a-87e9-4102b87962c7" containerName="postgres" Jul 29 08:52:59.942770 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:52:59.942757 2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="33172513-89e2-4a6e-b352-cb6c516b95dd" containerName="registry" Jul 29 08:52:59.945575 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:52:59.945557 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-qrb25/must-gather-tsvpr" Jul 29 08:52:59.949428 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:52:59.949406 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-qrb25\"/\"openshift-service-ca.crt\"" Jul 29 08:52:59.949552 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:52:59.949414 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-qrb25\"/\"default-dockercfg-p4qm6\"" Jul 29 08:52:59.951467 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:52:59.951412 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-qrb25\"/\"kube-root-ca.crt\"" Jul 29 08:52:59.954073 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:52:59.954055 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-qrb25/must-gather-tsvpr"] Jul 29 08:52:59.960643 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:52:59.960619 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8rp9g_cfd1d8df-e6b7-4a67-81f2-5d83da541e20/konnectivity-agent/0.log" limit=1024 Jul 29 08:53:00.020304 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:53:00.020271 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8rp9g_cfd1d8df-e6b7-4a67-81f2-5d83da541e20/konnectivity-agent/0.log" limit=1024 Jul 29 08:53:00.034446 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:53:00.034419 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6gnjq\" (UniqueName: \"kubernetes.io/projected/e4508ba3-3b53-4260-af41-1c226f9f71a7-kube-api-access-6gnjq\") pod \"must-gather-tsvpr\" (UID: \"e4508ba3-3b53-4260-af41-1c226f9f71a7\") " pod="openshift-must-gather-qrb25/must-gather-tsvpr" Jul 29 08:53:00.034562 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:53:00.034459 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/e4508ba3-3b53-4260-af41-1c226f9f71a7-must-gather-output\") pod \"must-gather-tsvpr\" (UID: \"e4508ba3-3b53-4260-af41-1c226f9f71a7\") " pod="openshift-must-gather-qrb25/must-gather-tsvpr" Jul 29 08:53:00.135869 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:53:00.135805 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6gnjq\" (UniqueName: \"kubernetes.io/projected/e4508ba3-3b53-4260-af41-1c226f9f71a7-kube-api-access-6gnjq\") pod \"must-gather-tsvpr\" (UID: \"e4508ba3-3b53-4260-af41-1c226f9f71a7\") " pod="openshift-must-gather-qrb25/must-gather-tsvpr" Jul 29 08:53:00.135869 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:53:00.135857 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/e4508ba3-3b53-4260-af41-1c226f9f71a7-must-gather-output\") pod \"must-gather-tsvpr\" (UID: \"e4508ba3-3b53-4260-af41-1c226f9f71a7\") " pod="openshift-must-gather-qrb25/must-gather-tsvpr" Jul 29 08:53:00.136112 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:53:00.136098 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/e4508ba3-3b53-4260-af41-1c226f9f71a7-must-gather-output\") pod \"must-gather-tsvpr\" (UID: \"e4508ba3-3b53-4260-af41-1c226f9f71a7\") " pod="openshift-must-gather-qrb25/must-gather-tsvpr" Jul 29 08:53:00.144771 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:53:00.144745 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6gnjq\" (UniqueName: \"kubernetes.io/projected/e4508ba3-3b53-4260-af41-1c226f9f71a7-kube-api-access-6gnjq\") pod \"must-gather-tsvpr\" (UID: \"e4508ba3-3b53-4260-af41-1c226f9f71a7\") " pod="openshift-must-gather-qrb25/must-gather-tsvpr" Jul 29 08:53:00.255203 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:53:00.255158 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-qrb25/must-gather-tsvpr" Jul 29 08:53:00.384744 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:53:00.384713 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-qrb25/must-gather-tsvpr"] Jul 29 08:53:00.388208 ip-10-0-131-138 kubenswrapper[2563]: W0729 08:53:00.388130 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pode4508ba3_3b53_4260_af41_1c226f9f71a7.slice/crio-9f39ad3c0bbf5e6d268ad74b54a961215284e355b60a0f6e08c3273f793457e9 WatchSource:0}: Error finding container 9f39ad3c0bbf5e6d268ad74b54a961215284e355b60a0f6e08c3273f793457e9: Status 404 returned error can't find the container with id 9f39ad3c0bbf5e6d268ad74b54a961215284e355b60a0f6e08c3273f793457e9 Jul 29 08:53:00.390375 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:53:00.390358 2563 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 29 08:53:01.208855 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:53:01.208822 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-qrb25/must-gather-tsvpr" event={"ID":"e4508ba3-3b53-4260-af41-1c226f9f71a7","Type":"ContainerStarted","Data":"9f39ad3c0bbf5e6d268ad74b54a961215284e355b60a0f6e08c3273f793457e9"} Jul 29 08:53:02.216054 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:53:02.215851 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-qrb25/must-gather-tsvpr" event={"ID":"e4508ba3-3b53-4260-af41-1c226f9f71a7","Type":"ContainerStarted","Data":"7c49ecf17c63c515fc5ac4c46ec2be35b4fbe194089f35dce8cf2fa0d9376ab6"} Jul 29 08:53:02.216054 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:53:02.215908 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-qrb25/must-gather-tsvpr" event={"ID":"e4508ba3-3b53-4260-af41-1c226f9f71a7","Type":"ContainerStarted","Data":"7bc0baa0451866e7b7b48f9d77ad0e75b1150d5e72084d152baf2bb91f6e13f4"} Jul 29 08:53:02.692600 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:53:02.692519 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-rn9kc_b14e5aed-a2b3-41bf-9389-701d69342953/global-pull-secret-syncer/0.log" Jul 29 08:53:02.743712 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:53:02.743683 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-8rp9g_cfd1d8df-e6b7-4a67-81f2-5d83da541e20/konnectivity-agent/0.log" Jul 29 08:53:02.834077 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:53:02.834041 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-131-138.ec2.internal_99103139fa564168399467f94d133369/haproxy/0.log" Jul 29 08:53:05.647332 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:53:05.647291 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8rp9g_cfd1d8df-e6b7-4a67-81f2-5d83da541e20/konnectivity-agent/0.log" limit=1024 Jul 29 08:53:05.704079 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:53:05.704020 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8rp9g_cfd1d8df-e6b7-4a67-81f2-5d83da541e20/konnectivity-agent/0.log" limit=1024 Jul 29 08:53:06.962223 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:53:06.962186 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_cluster-monitoring-operator-548b489cb8-ch694_ed443996-4f98-44b5-9476-37a0c3c23099/cluster-monitoring-operator/0.log" Jul 29 08:53:07.059896 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:53:07.059868 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-56d9f944b4-9cctr_39642d9c-85e5-42e1-8b0d-90db3adaf3ef/kube-state-metrics/0.log" Jul 29 08:53:07.102117 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:53:07.102078 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-56d9f944b4-9cctr_39642d9c-85e5-42e1-8b0d-90db3adaf3ef/kube-rbac-proxy-main/0.log" Jul 29 08:53:07.138185 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:53:07.138160 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-56d9f944b4-9cctr_39642d9c-85e5-42e1-8b0d-90db3adaf3ef/kube-rbac-proxy-self/0.log" Jul 29 08:53:07.196618 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:53:07.196594 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_metrics-server-66c8d585d-h6fnr_584f572a-103c-4b50-9c84-ba4e693f120d/metrics-server/0.log" Jul 29 08:53:07.383478 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:53:07.383396 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-s44d8_66322420-ef49-4c4c-99c5-1a19d67d4244/node-exporter/0.log" Jul 29 08:53:07.422326 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:53:07.422293 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-s44d8_66322420-ef49-4c4c-99c5-1a19d67d4244/kube-rbac-proxy/0.log" Jul 29 08:53:07.468040 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:53:07.468014 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-s44d8_66322420-ef49-4c4c-99c5-1a19d67d4244/init-textfile/0.log" Jul 29 08:53:09.798434 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:53:09.798372 2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-qrb25/must-gather-tsvpr" podStartSLOduration=9.902726079 podStartE2EDuration="10.798353789s" totalImagesPullingTime="895.62771ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 08:52:59 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 08:53:02.233626123 +0000 UTC m=+1894.814914901" watchObservedRunningTime="2026-07-29 08:53:09.798353789 +0000 UTC m=+1902.379642567" Jul 29 08:53:09.800931 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:53:09.800903 2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-qrb25/perf-node-gather-daemonset-2mjgh"] Jul 29 08:53:09.806349 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:53:09.806321 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-qrb25/perf-node-gather-daemonset-2mjgh" Jul 29 08:53:09.811914 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:53:09.811881 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-qrb25/perf-node-gather-daemonset-2mjgh"] Jul 29 08:53:09.925836 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:53:09.925796 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cjsmh\" (UniqueName: \"kubernetes.io/projected/bc48a53e-53f9-4de2-af7f-96fe84e534f6-kube-api-access-cjsmh\") pod \"perf-node-gather-daemonset-2mjgh\" (UID: \"bc48a53e-53f9-4de2-af7f-96fe84e534f6\") " pod="openshift-must-gather-qrb25/perf-node-gather-daemonset-2mjgh" Jul 29 08:53:09.926110 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:53:09.926087 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/bc48a53e-53f9-4de2-af7f-96fe84e534f6-proc\") pod \"perf-node-gather-daemonset-2mjgh\" (UID: \"bc48a53e-53f9-4de2-af7f-96fe84e534f6\") " pod="openshift-must-gather-qrb25/perf-node-gather-daemonset-2mjgh" Jul 29 08:53:09.926308 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:53:09.926291 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/bc48a53e-53f9-4de2-af7f-96fe84e534f6-podres\") pod \"perf-node-gather-daemonset-2mjgh\" (UID: \"bc48a53e-53f9-4de2-af7f-96fe84e534f6\") " pod="openshift-must-gather-qrb25/perf-node-gather-daemonset-2mjgh" Jul 29 08:53:09.926433 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:53:09.926417 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/bc48a53e-53f9-4de2-af7f-96fe84e534f6-lib-modules\") pod \"perf-node-gather-daemonset-2mjgh\" (UID: \"bc48a53e-53f9-4de2-af7f-96fe84e534f6\") " pod="openshift-must-gather-qrb25/perf-node-gather-daemonset-2mjgh" Jul 29 08:53:09.926565 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:53:09.926539 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/bc48a53e-53f9-4de2-af7f-96fe84e534f6-sys\") pod \"perf-node-gather-daemonset-2mjgh\" (UID: \"bc48a53e-53f9-4de2-af7f-96fe84e534f6\") " pod="openshift-must-gather-qrb25/perf-node-gather-daemonset-2mjgh" Jul 29 08:53:10.027823 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:53:10.027771 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/bc48a53e-53f9-4de2-af7f-96fe84e534f6-podres\") pod \"perf-node-gather-daemonset-2mjgh\" (UID: \"bc48a53e-53f9-4de2-af7f-96fe84e534f6\") " pod="openshift-must-gather-qrb25/perf-node-gather-daemonset-2mjgh" Jul 29 08:53:10.028022 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:53:10.027840 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/bc48a53e-53f9-4de2-af7f-96fe84e534f6-lib-modules\") pod \"perf-node-gather-daemonset-2mjgh\" (UID: \"bc48a53e-53f9-4de2-af7f-96fe84e534f6\") " pod="openshift-must-gather-qrb25/perf-node-gather-daemonset-2mjgh" Jul 29 08:53:10.028022 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:53:10.027879 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/bc48a53e-53f9-4de2-af7f-96fe84e534f6-sys\") pod \"perf-node-gather-daemonset-2mjgh\" (UID: \"bc48a53e-53f9-4de2-af7f-96fe84e534f6\") " pod="openshift-must-gather-qrb25/perf-node-gather-daemonset-2mjgh" Jul 29 08:53:10.028022 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:53:10.027922 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cjsmh\" (UniqueName: \"kubernetes.io/projected/bc48a53e-53f9-4de2-af7f-96fe84e534f6-kube-api-access-cjsmh\") pod \"perf-node-gather-daemonset-2mjgh\" (UID: \"bc48a53e-53f9-4de2-af7f-96fe84e534f6\") " pod="openshift-must-gather-qrb25/perf-node-gather-daemonset-2mjgh" Jul 29 08:53:10.028022 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:53:10.027961 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/bc48a53e-53f9-4de2-af7f-96fe84e534f6-proc\") pod \"perf-node-gather-daemonset-2mjgh\" (UID: \"bc48a53e-53f9-4de2-af7f-96fe84e534f6\") " pod="openshift-must-gather-qrb25/perf-node-gather-daemonset-2mjgh" Jul 29 08:53:10.028022 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:53:10.027979 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/bc48a53e-53f9-4de2-af7f-96fe84e534f6-podres\") pod \"perf-node-gather-daemonset-2mjgh\" (UID: \"bc48a53e-53f9-4de2-af7f-96fe84e534f6\") " pod="openshift-must-gather-qrb25/perf-node-gather-daemonset-2mjgh" Jul 29 08:53:10.028285 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:53:10.028015 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/bc48a53e-53f9-4de2-af7f-96fe84e534f6-sys\") pod \"perf-node-gather-daemonset-2mjgh\" (UID: \"bc48a53e-53f9-4de2-af7f-96fe84e534f6\") " pod="openshift-must-gather-qrb25/perf-node-gather-daemonset-2mjgh" Jul 29 08:53:10.028285 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:53:10.028043 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/bc48a53e-53f9-4de2-af7f-96fe84e534f6-lib-modules\") pod \"perf-node-gather-daemonset-2mjgh\" (UID: \"bc48a53e-53f9-4de2-af7f-96fe84e534f6\") " pod="openshift-must-gather-qrb25/perf-node-gather-daemonset-2mjgh" Jul 29 08:53:10.028285 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:53:10.028099 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/bc48a53e-53f9-4de2-af7f-96fe84e534f6-proc\") pod \"perf-node-gather-daemonset-2mjgh\" (UID: \"bc48a53e-53f9-4de2-af7f-96fe84e534f6\") " pod="openshift-must-gather-qrb25/perf-node-gather-daemonset-2mjgh" Jul 29 08:53:10.038436 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:53:10.038400 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-cjsmh\" (UniqueName: \"kubernetes.io/projected/bc48a53e-53f9-4de2-af7f-96fe84e534f6-kube-api-access-cjsmh\") pod \"perf-node-gather-daemonset-2mjgh\" (UID: \"bc48a53e-53f9-4de2-af7f-96fe84e534f6\") " pod="openshift-must-gather-qrb25/perf-node-gather-daemonset-2mjgh" Jul 29 08:53:10.119724 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:53:10.119628 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-qrb25/perf-node-gather-daemonset-2mjgh" Jul 29 08:53:10.290616 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:53:10.290577 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-qrb25/perf-node-gather-daemonset-2mjgh"] Jul 29 08:53:10.295433 ip-10-0-131-138 kubenswrapper[2563]: W0729 08:53:10.295402 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-podbc48a53e_53f9_4de2_af7f_96fe84e534f6.slice/crio-7f29f2e8ec474f98ceab8d458dd8ef8bcda5474ae1361b0b9e6f25e1ec82a550 WatchSource:0}: Error finding container 7f29f2e8ec474f98ceab8d458dd8ef8bcda5474ae1361b0b9e6f25e1ec82a550: Status 404 returned error can't find the container with id 7f29f2e8ec474f98ceab8d458dd8ef8bcda5474ae1361b0b9e6f25e1ec82a550 Jul 29 08:53:10.764100 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:53:10.764067 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-cluster-storage-operator_volume-data-source-validator-549c6d54cf-pc5qx_c61e3f1b-de05-4466-97b5-21d2e47af9df/volume-data-source-validator/0.log" Jul 29 08:53:11.263564 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:53:11.263521 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-qrb25/perf-node-gather-daemonset-2mjgh" event={"ID":"bc48a53e-53f9-4de2-af7f-96fe84e534f6","Type":"ContainerStarted","Data":"c8c7358ab7bb9c87e363d743413204fe8b9be9c916d0e79b71e6ce56ba840968"} Jul 29 08:53:11.264051 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:53:11.263591 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-qrb25/perf-node-gather-daemonset-2mjgh" event={"ID":"bc48a53e-53f9-4de2-af7f-96fe84e534f6","Type":"ContainerStarted","Data":"7f29f2e8ec474f98ceab8d458dd8ef8bcda5474ae1361b0b9e6f25e1ec82a550"} Jul 29 08:53:11.264408 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:53:11.264380 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-qrb25/perf-node-gather-daemonset-2mjgh" Jul 29 08:53:11.282205 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:53:11.282154 2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-qrb25/perf-node-gather-daemonset-2mjgh" podStartSLOduration=2.282137376 podStartE2EDuration="2.282137376s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-29 08:53:09 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 08:53:11.280434218 +0000 UTC m=+1903.861722996" watchObservedRunningTime="2026-07-29 08:53:11.282137376 +0000 UTC m=+1903.863426201" Jul 29 08:53:11.567567 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:53:11.567498 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-nhklh_e521e734-a8d9-4748-ba28-c5f2beb27610/dns/0.log" Jul 29 08:53:11.585969 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:53:11.585943 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-nhklh_e521e734-a8d9-4748-ba28-c5f2beb27610/kube-rbac-proxy/0.log" Jul 29 08:53:11.606776 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:53:11.606749 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-bgb7d_bc118958-c5f8-44b8-80a5-bfc2d80fad1d/dns-node-resolver/0.log" Jul 29 08:53:12.087799 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:53:12.087769 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-9d4f5_5bce82ca-dd94-425f-83d2-cd44232d8cdb/node-ca/0.log" Jul 29 08:53:12.870840 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:53:12.870812 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-7c7545584f-2d6gk_bf0cdb1f-7233-4c76-8159-b233968ce8e5/router/0.log" Jul 29 08:53:13.249003 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:53:13.248968 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-pwqh7_b052d187-d65d-4894-aaa2-5dbb4b06600e/serve-healthcheck-canary/0.log" Jul 29 08:53:13.574495 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:53:13.574419 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-operator-5885fdb759-84zrp_791f1046-8a1f-41c4-b7a6-70862705aad6/insights-operator/1.log" Jul 29 08:53:13.574495 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:53:13.574420 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-operator-5885fdb759-84zrp_791f1046-8a1f-41c4-b7a6-70862705aad6/insights-operator/0.log" Jul 29 08:53:13.595913 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:53:13.595888 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-29q8s_bb16ee9e-483f-465c-8deb-4a36b0db9743/kube-rbac-proxy/0.log" Jul 29 08:53:13.615732 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:53:13.615698 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-29q8s_bb16ee9e-483f-465c-8deb-4a36b0db9743/exporter/0.log" Jul 29 08:53:13.635335 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:53:13.635304 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-29q8s_bb16ee9e-483f-465c-8deb-4a36b0db9743/extractor/0.log" Jul 29 08:53:18.282417 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:53:18.282389 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-qrb25/perf-node-gather-daemonset-2mjgh" Jul 29 08:53:19.144312 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:53:19.144286 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-qs8zl_bb5d06c1-cbbb-4e40-9e0a-43aa22af698f/kube-multus-additional-cni-plugins/0.log" Jul 29 08:53:19.167661 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:53:19.167630 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-qs8zl_bb5d06c1-cbbb-4e40-9e0a-43aa22af698f/egress-router-binary-copy/0.log" Jul 29 08:53:19.187390 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:53:19.187365 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-qs8zl_bb5d06c1-cbbb-4e40-9e0a-43aa22af698f/cni-plugins/0.log" Jul 29 08:53:19.211206 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:53:19.211166 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-qs8zl_bb5d06c1-cbbb-4e40-9e0a-43aa22af698f/bond-cni-plugin/0.log" Jul 29 08:53:19.231651 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:53:19.231618 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-qs8zl_bb5d06c1-cbbb-4e40-9e0a-43aa22af698f/routeoverride-cni/0.log" Jul 29 08:53:19.256129 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:53:19.256105 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-qs8zl_bb5d06c1-cbbb-4e40-9e0a-43aa22af698f/whereabouts-cni-bincopy/0.log" Jul 29 08:53:19.277202 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:53:19.277177 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-qs8zl_bb5d06c1-cbbb-4e40-9e0a-43aa22af698f/whereabouts-cni/0.log" Jul 29 08:53:19.523997 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:53:19.523965 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-v5xrb_2b20d931-6631-4e41-aff5-ed22c5fdafa7/kube-multus/0.log" Jul 29 08:53:19.593644 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:53:19.593605 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-qk9kb_35e49848-94f6-43d9-b7c4-6c817dd5dde7/network-metrics-daemon/0.log" Jul 29 08:53:19.611472 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:53:19.611437 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-qk9kb_35e49848-94f6-43d9-b7c4-6c817dd5dde7/kube-rbac-proxy/0.log" Jul 29 08:53:20.875113 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:53:20.875088 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-vqdfh_49600d58-879f-477d-aa43-11177f411977/ovn-controller/0.log" Jul 29 08:53:20.892895 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:53:20.892873 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-vqdfh_49600d58-879f-477d-aa43-11177f411977/ovn-acl-logging/0.log" Jul 29 08:53:20.916093 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:53:20.916069 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-vqdfh_49600d58-879f-477d-aa43-11177f411977/kube-rbac-proxy-node/0.log" Jul 29 08:53:20.942718 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:53:20.942692 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-vqdfh_49600d58-879f-477d-aa43-11177f411977/kube-rbac-proxy-ovn-metrics/0.log" Jul 29 08:53:20.963317 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:53:20.963288 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-vqdfh_49600d58-879f-477d-aa43-11177f411977/northd/0.log" Jul 29 08:53:20.984432 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:53:20.984388 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-vqdfh_49600d58-879f-477d-aa43-11177f411977/nbdb/0.log" Jul 29 08:53:21.002740 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:53:21.002709 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-vqdfh_49600d58-879f-477d-aa43-11177f411977/sbdb/0.log" Jul 29 08:53:21.048146 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:53:21.048115 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-vqdfh_49600d58-879f-477d-aa43-11177f411977/ovnkube-controller/0.log" Jul 29 08:53:22.081393 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:53:22.081275 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-network-diagnostics_network-check-source-7d88cb55d8-dr9xk_3eef9ce5-17eb-4fbe-82d3-dd9b1cb17e93/check-endpoints/0.log" Jul 29 08:53:22.156942 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:53:22.156911 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-network-diagnostics_network-check-target-rdclp_ce06e4b8-c4ec-4edf-a9c4-d7d86a03d844/network-check-target-container/0.log" Jul 29 08:53:23.069664 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:53:23.069633 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-network-operator_iptables-alerter-z2c22_db249d48-ad47-4cc0-8a37-37078055ed38/iptables-alerter/0.log" Jul 29 08:53:23.680866 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:53:23.680836 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-cluster-node-tuning-operator_tuned-95gmk_30ff3fec-359e-4b75-82eb-9f1a194766f7/tuned/0.log" Jul 29 08:53:25.371418 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:53:25.371386 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-cluster-samples-operator_cluster-samples-operator-6748489656-q2lqd_d2793bc6-e6c2-4799-845e-ee32f0a85693/cluster-samples-operator/0.log" Jul 29 08:53:25.389128 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:53:25.389104 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-cluster-samples-operator_cluster-samples-operator-6748489656-q2lqd_d2793bc6-e6c2-4799-845e-ee32f0a85693/cluster-samples-operator-watch/0.log" Jul 29 08:53:27.027306 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:53:27.027274 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-cluster-csi-drivers_aws-ebs-csi-driver-node-52bkd_93ba91b4-4c35-43be-9094-08b0b3a473b3/csi-driver/0.log" Jul 29 08:53:27.051836 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:53:27.051805 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-cluster-csi-drivers_aws-ebs-csi-driver-node-52bkd_93ba91b4-4c35-43be-9094-08b0b3a473b3/csi-node-driver-registrar/0.log" Jul 29 08:53:27.074798 ip-10-0-131-138 kubenswrapper[2563]: I0729 08:53:27.074771 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-cluster-csi-drivers_aws-ebs-csi-driver-node-52bkd_93ba91b4-4c35-43be-9094-08b0b3a473b3/csi-liveness-probe/0.log"