Jul 06 02:39:14.830966 ip-10-0-139-206 systemd[1]: Starting Kubernetes Kubelet... Jul 06 02:39:15.345249 ip-10-0-139-206 kubenswrapper[2557]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 06 02:39:15.345249 ip-10-0-139-206 kubenswrapper[2557]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version. Jul 06 02:39:15.345249 ip-10-0-139-206 kubenswrapper[2557]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 06 02:39:15.345249 ip-10-0-139-206 kubenswrapper[2557]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI. Jul 06 02:39:15.345249 ip-10-0-139-206 kubenswrapper[2557]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 06 02:39:15.345249 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:39:15.259769 2557 server.go:213] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime" Jul 06 02:39:15.345249 ip-10-0-139-206 kubenswrapper[2557]: W0706 02:39:15.263789 2557 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 06 02:39:15.345249 ip-10-0-139-206 kubenswrapper[2557]: W0706 02:39:15.263806 2557 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 06 02:39:15.345249 ip-10-0-139-206 kubenswrapper[2557]: W0706 02:39:15.263809 2557 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 06 02:39:15.345249 ip-10-0-139-206 kubenswrapper[2557]: W0706 02:39:15.263812 2557 feature_gate.go:328] unrecognized feature gate: Example Jul 06 02:39:15.345249 ip-10-0-139-206 kubenswrapper[2557]: W0706 02:39:15.263815 2557 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 06 02:39:15.345249 ip-10-0-139-206 kubenswrapper[2557]: W0706 02:39:15.263817 2557 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 06 02:39:15.345249 ip-10-0-139-206 kubenswrapper[2557]: W0706 02:39:15.263820 2557 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 06 02:39:15.345249 ip-10-0-139-206 kubenswrapper[2557]: W0706 02:39:15.263823 2557 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 06 02:39:15.347189 ip-10-0-139-206 kubenswrapper[2557]: W0706 02:39:15.263826 2557 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 06 02:39:15.347189 ip-10-0-139-206 kubenswrapper[2557]: W0706 02:39:15.263828 2557 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 06 02:39:15.347189 ip-10-0-139-206 kubenswrapper[2557]: W0706 02:39:15.263831 2557 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 06 02:39:15.347189 ip-10-0-139-206 kubenswrapper[2557]: W0706 02:39:15.263839 2557 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 06 02:39:15.347189 ip-10-0-139-206 kubenswrapper[2557]: W0706 02:39:15.263842 2557 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 06 02:39:15.347189 ip-10-0-139-206 kubenswrapper[2557]: W0706 02:39:15.263845 2557 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 06 02:39:15.347189 ip-10-0-139-206 kubenswrapper[2557]: W0706 02:39:15.263848 2557 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 06 02:39:15.347189 ip-10-0-139-206 kubenswrapper[2557]: W0706 02:39:15.263851 2557 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 06 02:39:15.347189 ip-10-0-139-206 kubenswrapper[2557]: W0706 02:39:15.263853 2557 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 06 02:39:15.347189 ip-10-0-139-206 kubenswrapper[2557]: W0706 02:39:15.263856 2557 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 06 02:39:15.347189 ip-10-0-139-206 kubenswrapper[2557]: W0706 02:39:15.263860 2557 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 06 02:39:15.347189 ip-10-0-139-206 kubenswrapper[2557]: W0706 02:39:15.263862 2557 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 06 02:39:15.347189 ip-10-0-139-206 kubenswrapper[2557]: W0706 02:39:15.263865 2557 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 06 02:39:15.347189 ip-10-0-139-206 kubenswrapper[2557]: W0706 02:39:15.263868 2557 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 06 02:39:15.347189 ip-10-0-139-206 kubenswrapper[2557]: W0706 02:39:15.263870 2557 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 06 02:39:15.347189 ip-10-0-139-206 kubenswrapper[2557]: W0706 02:39:15.263873 2557 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 06 02:39:15.347189 ip-10-0-139-206 kubenswrapper[2557]: W0706 02:39:15.263876 2557 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 06 02:39:15.347189 ip-10-0-139-206 kubenswrapper[2557]: W0706 02:39:15.263878 2557 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 06 02:39:15.347189 ip-10-0-139-206 kubenswrapper[2557]: W0706 02:39:15.263881 2557 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 06 02:39:15.347189 ip-10-0-139-206 kubenswrapper[2557]: W0706 02:39:15.263885 2557 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 06 02:39:15.353188 ip-10-0-139-206 kubenswrapper[2557]: W0706 02:39:15.263888 2557 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 06 02:39:15.353188 ip-10-0-139-206 kubenswrapper[2557]: W0706 02:39:15.263890 2557 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 06 02:39:15.353188 ip-10-0-139-206 kubenswrapper[2557]: W0706 02:39:15.263893 2557 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 06 02:39:15.353188 ip-10-0-139-206 kubenswrapper[2557]: W0706 02:39:15.263895 2557 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 06 02:39:15.353188 ip-10-0-139-206 kubenswrapper[2557]: W0706 02:39:15.263898 2557 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 06 02:39:15.353188 ip-10-0-139-206 kubenswrapper[2557]: W0706 02:39:15.263901 2557 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 06 02:39:15.353188 ip-10-0-139-206 kubenswrapper[2557]: W0706 02:39:15.263903 2557 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 06 02:39:15.353188 ip-10-0-139-206 kubenswrapper[2557]: W0706 02:39:15.263905 2557 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 06 02:39:15.353188 ip-10-0-139-206 kubenswrapper[2557]: W0706 02:39:15.263908 2557 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 06 02:39:15.353188 ip-10-0-139-206 kubenswrapper[2557]: W0706 02:39:15.263910 2557 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 06 02:39:15.353188 ip-10-0-139-206 kubenswrapper[2557]: W0706 02:39:15.263913 2557 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 06 02:39:15.353188 ip-10-0-139-206 kubenswrapper[2557]: W0706 02:39:15.263915 2557 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 06 02:39:15.353188 ip-10-0-139-206 kubenswrapper[2557]: W0706 02:39:15.263917 2557 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 06 02:39:15.353188 ip-10-0-139-206 kubenswrapper[2557]: W0706 02:39:15.263920 2557 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 06 02:39:15.353188 ip-10-0-139-206 kubenswrapper[2557]: W0706 02:39:15.263923 2557 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 06 02:39:15.353188 ip-10-0-139-206 kubenswrapper[2557]: W0706 02:39:15.263926 2557 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 06 02:39:15.353188 ip-10-0-139-206 kubenswrapper[2557]: W0706 02:39:15.263929 2557 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 06 02:39:15.353188 ip-10-0-139-206 kubenswrapper[2557]: W0706 02:39:15.263931 2557 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 06 02:39:15.353188 ip-10-0-139-206 kubenswrapper[2557]: W0706 02:39:15.263934 2557 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 06 02:39:15.355772 ip-10-0-139-206 kubenswrapper[2557]: W0706 02:39:15.263937 2557 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 06 02:39:15.355772 ip-10-0-139-206 kubenswrapper[2557]: W0706 02:39:15.263940 2557 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 06 02:39:15.355772 ip-10-0-139-206 kubenswrapper[2557]: W0706 02:39:15.263943 2557 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 06 02:39:15.355772 ip-10-0-139-206 kubenswrapper[2557]: W0706 02:39:15.263946 2557 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 06 02:39:15.355772 ip-10-0-139-206 kubenswrapper[2557]: W0706 02:39:15.263950 2557 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 06 02:39:15.355772 ip-10-0-139-206 kubenswrapper[2557]: W0706 02:39:15.263954 2557 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 06 02:39:15.355772 ip-10-0-139-206 kubenswrapper[2557]: W0706 02:39:15.263957 2557 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 06 02:39:15.355772 ip-10-0-139-206 kubenswrapper[2557]: W0706 02:39:15.263959 2557 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 06 02:39:15.355772 ip-10-0-139-206 kubenswrapper[2557]: W0706 02:39:15.263962 2557 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 06 02:39:15.355772 ip-10-0-139-206 kubenswrapper[2557]: W0706 02:39:15.263965 2557 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 06 02:39:15.355772 ip-10-0-139-206 kubenswrapper[2557]: W0706 02:39:15.263968 2557 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 06 02:39:15.355772 ip-10-0-139-206 kubenswrapper[2557]: W0706 02:39:15.263972 2557 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 06 02:39:15.355772 ip-10-0-139-206 kubenswrapper[2557]: W0706 02:39:15.263981 2557 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 06 02:39:15.355772 ip-10-0-139-206 kubenswrapper[2557]: W0706 02:39:15.263985 2557 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 06 02:39:15.355772 ip-10-0-139-206 kubenswrapper[2557]: W0706 02:39:15.263989 2557 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 06 02:39:15.355772 ip-10-0-139-206 kubenswrapper[2557]: W0706 02:39:15.263993 2557 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 06 02:39:15.355772 ip-10-0-139-206 kubenswrapper[2557]: W0706 02:39:15.263996 2557 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 06 02:39:15.355772 ip-10-0-139-206 kubenswrapper[2557]: W0706 02:39:15.263999 2557 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 06 02:39:15.355772 ip-10-0-139-206 kubenswrapper[2557]: W0706 02:39:15.264001 2557 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 06 02:39:15.360196 ip-10-0-139-206 kubenswrapper[2557]: W0706 02:39:15.264004 2557 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 06 02:39:15.360196 ip-10-0-139-206 kubenswrapper[2557]: W0706 02:39:15.264009 2557 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 06 02:39:15.360196 ip-10-0-139-206 kubenswrapper[2557]: W0706 02:39:15.264013 2557 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 06 02:39:15.360196 ip-10-0-139-206 kubenswrapper[2557]: W0706 02:39:15.264017 2557 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 06 02:39:15.360196 ip-10-0-139-206 kubenswrapper[2557]: W0706 02:39:15.264020 2557 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 06 02:39:15.360196 ip-10-0-139-206 kubenswrapper[2557]: W0706 02:39:15.264023 2557 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 06 02:39:15.360196 ip-10-0-139-206 kubenswrapper[2557]: W0706 02:39:15.264026 2557 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 06 02:39:15.360196 ip-10-0-139-206 kubenswrapper[2557]: W0706 02:39:15.264028 2557 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 06 02:39:15.360196 ip-10-0-139-206 kubenswrapper[2557]: W0706 02:39:15.264032 2557 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 06 02:39:15.360196 ip-10-0-139-206 kubenswrapper[2557]: W0706 02:39:15.264035 2557 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 06 02:39:15.360196 ip-10-0-139-206 kubenswrapper[2557]: W0706 02:39:15.264096 2557 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 06 02:39:15.360196 ip-10-0-139-206 kubenswrapper[2557]: W0706 02:39:15.264160 2557 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 06 02:39:15.360196 ip-10-0-139-206 kubenswrapper[2557]: W0706 02:39:15.264166 2557 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 06 02:39:15.360196 ip-10-0-139-206 kubenswrapper[2557]: W0706 02:39:15.264171 2557 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 06 02:39:15.360196 ip-10-0-139-206 kubenswrapper[2557]: W0706 02:39:15.264177 2557 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 06 02:39:15.360196 ip-10-0-139-206 kubenswrapper[2557]: W0706 02:39:15.264182 2557 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 06 02:39:15.360196 ip-10-0-139-206 kubenswrapper[2557]: W0706 02:39:15.264187 2557 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 06 02:39:15.360196 ip-10-0-139-206 kubenswrapper[2557]: W0706 02:39:15.264191 2557 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 06 02:39:15.360196 ip-10-0-139-206 kubenswrapper[2557]: W0706 02:39:15.264193 2557 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 06 02:39:15.360196 ip-10-0-139-206 kubenswrapper[2557]: W0706 02:39:15.264196 2557 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 06 02:39:15.361535 ip-10-0-139-206 kubenswrapper[2557]: W0706 02:39:15.264199 2557 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 06 02:39:15.361535 ip-10-0-139-206 kubenswrapper[2557]: W0706 02:39:15.264204 2557 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 06 02:39:15.361535 ip-10-0-139-206 kubenswrapper[2557]: W0706 02:39:15.264207 2557 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 06 02:39:15.361535 ip-10-0-139-206 kubenswrapper[2557]: W0706 02:39:15.264211 2557 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 06 02:39:15.361535 ip-10-0-139-206 kubenswrapper[2557]: W0706 02:39:15.264221 2557 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 06 02:39:15.361535 ip-10-0-139-206 kubenswrapper[2557]: W0706 02:39:15.264226 2557 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 06 02:39:15.361535 ip-10-0-139-206 kubenswrapper[2557]: W0706 02:39:15.264230 2557 feature_gate.go:328] unrecognized feature gate: Example2 Jul 06 02:39:15.361535 ip-10-0-139-206 kubenswrapper[2557]: W0706 02:39:15.264234 2557 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 06 02:39:15.361535 ip-10-0-139-206 kubenswrapper[2557]: W0706 02:39:15.265307 2557 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 06 02:39:15.361535 ip-10-0-139-206 kubenswrapper[2557]: W0706 02:39:15.265327 2557 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 06 02:39:15.361535 ip-10-0-139-206 kubenswrapper[2557]: W0706 02:39:15.265333 2557 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 06 02:39:15.361535 ip-10-0-139-206 kubenswrapper[2557]: W0706 02:39:15.265338 2557 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 06 02:39:15.361535 ip-10-0-139-206 kubenswrapper[2557]: W0706 02:39:15.265342 2557 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 06 02:39:15.361535 ip-10-0-139-206 kubenswrapper[2557]: W0706 02:39:15.266730 2557 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 06 02:39:15.361535 ip-10-0-139-206 kubenswrapper[2557]: W0706 02:39:15.266742 2557 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 06 02:39:15.361535 ip-10-0-139-206 kubenswrapper[2557]: W0706 02:39:15.266747 2557 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 06 02:39:15.361535 ip-10-0-139-206 kubenswrapper[2557]: W0706 02:39:15.266750 2557 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 06 02:39:15.361535 ip-10-0-139-206 kubenswrapper[2557]: W0706 02:39:15.266753 2557 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 06 02:39:15.361535 ip-10-0-139-206 kubenswrapper[2557]: W0706 02:39:15.266756 2557 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 06 02:39:15.364210 ip-10-0-139-206 kubenswrapper[2557]: W0706 02:39:15.266758 2557 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 06 02:39:15.364210 ip-10-0-139-206 kubenswrapper[2557]: W0706 02:39:15.266761 2557 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 06 02:39:15.364210 ip-10-0-139-206 kubenswrapper[2557]: W0706 02:39:15.266764 2557 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 06 02:39:15.364210 ip-10-0-139-206 kubenswrapper[2557]: W0706 02:39:15.266767 2557 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 06 02:39:15.364210 ip-10-0-139-206 kubenswrapper[2557]: W0706 02:39:15.266770 2557 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 06 02:39:15.364210 ip-10-0-139-206 kubenswrapper[2557]: W0706 02:39:15.266773 2557 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 06 02:39:15.364210 ip-10-0-139-206 kubenswrapper[2557]: W0706 02:39:15.266776 2557 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 06 02:39:15.364210 ip-10-0-139-206 kubenswrapper[2557]: W0706 02:39:15.266778 2557 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 06 02:39:15.364210 ip-10-0-139-206 kubenswrapper[2557]: W0706 02:39:15.266781 2557 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 06 02:39:15.364210 ip-10-0-139-206 kubenswrapper[2557]: W0706 02:39:15.266789 2557 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 06 02:39:15.364210 ip-10-0-139-206 kubenswrapper[2557]: W0706 02:39:15.266791 2557 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 06 02:39:15.364210 ip-10-0-139-206 kubenswrapper[2557]: W0706 02:39:15.266794 2557 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 06 02:39:15.364210 ip-10-0-139-206 kubenswrapper[2557]: W0706 02:39:15.266797 2557 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 06 02:39:15.364210 ip-10-0-139-206 kubenswrapper[2557]: W0706 02:39:15.266799 2557 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 06 02:39:15.364210 ip-10-0-139-206 kubenswrapper[2557]: W0706 02:39:15.266802 2557 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 06 02:39:15.364210 ip-10-0-139-206 kubenswrapper[2557]: W0706 02:39:15.266805 2557 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 06 02:39:15.364210 ip-10-0-139-206 kubenswrapper[2557]: W0706 02:39:15.266807 2557 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 06 02:39:15.364210 ip-10-0-139-206 kubenswrapper[2557]: W0706 02:39:15.266810 2557 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 06 02:39:15.364210 ip-10-0-139-206 kubenswrapper[2557]: W0706 02:39:15.266813 2557 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 06 02:39:15.364210 ip-10-0-139-206 kubenswrapper[2557]: W0706 02:39:15.266815 2557 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 06 02:39:15.365057 ip-10-0-139-206 kubenswrapper[2557]: W0706 02:39:15.266818 2557 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 06 02:39:15.365057 ip-10-0-139-206 kubenswrapper[2557]: W0706 02:39:15.266821 2557 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 06 02:39:15.365057 ip-10-0-139-206 kubenswrapper[2557]: W0706 02:39:15.266824 2557 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 06 02:39:15.365057 ip-10-0-139-206 kubenswrapper[2557]: W0706 02:39:15.266827 2557 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 06 02:39:15.365057 ip-10-0-139-206 kubenswrapper[2557]: W0706 02:39:15.266830 2557 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 06 02:39:15.365057 ip-10-0-139-206 kubenswrapper[2557]: W0706 02:39:15.266833 2557 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 06 02:39:15.365057 ip-10-0-139-206 kubenswrapper[2557]: W0706 02:39:15.266835 2557 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 06 02:39:15.365057 ip-10-0-139-206 kubenswrapper[2557]: W0706 02:39:15.266839 2557 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 06 02:39:15.365057 ip-10-0-139-206 kubenswrapper[2557]: W0706 02:39:15.266842 2557 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 06 02:39:15.365057 ip-10-0-139-206 kubenswrapper[2557]: W0706 02:39:15.266844 2557 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 06 02:39:15.365057 ip-10-0-139-206 kubenswrapper[2557]: W0706 02:39:15.266847 2557 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 06 02:39:15.365057 ip-10-0-139-206 kubenswrapper[2557]: W0706 02:39:15.266850 2557 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 06 02:39:15.365057 ip-10-0-139-206 kubenswrapper[2557]: W0706 02:39:15.266852 2557 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 06 02:39:15.365057 ip-10-0-139-206 kubenswrapper[2557]: W0706 02:39:15.266855 2557 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 06 02:39:15.365057 ip-10-0-139-206 kubenswrapper[2557]: W0706 02:39:15.266858 2557 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 06 02:39:15.365057 ip-10-0-139-206 kubenswrapper[2557]: W0706 02:39:15.266861 2557 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 06 02:39:15.365057 ip-10-0-139-206 kubenswrapper[2557]: W0706 02:39:15.266863 2557 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 06 02:39:15.365057 ip-10-0-139-206 kubenswrapper[2557]: W0706 02:39:15.266866 2557 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 06 02:39:15.365057 ip-10-0-139-206 kubenswrapper[2557]: W0706 02:39:15.266870 2557 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 06 02:39:15.369417 ip-10-0-139-206 kubenswrapper[2557]: W0706 02:39:15.266875 2557 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 06 02:39:15.369417 ip-10-0-139-206 kubenswrapper[2557]: W0706 02:39:15.266878 2557 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 06 02:39:15.369417 ip-10-0-139-206 kubenswrapper[2557]: W0706 02:39:15.266882 2557 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 06 02:39:15.369417 ip-10-0-139-206 kubenswrapper[2557]: W0706 02:39:15.266885 2557 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 06 02:39:15.369417 ip-10-0-139-206 kubenswrapper[2557]: W0706 02:39:15.266888 2557 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 06 02:39:15.369417 ip-10-0-139-206 kubenswrapper[2557]: W0706 02:39:15.266892 2557 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 06 02:39:15.369417 ip-10-0-139-206 kubenswrapper[2557]: W0706 02:39:15.266895 2557 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 06 02:39:15.369417 ip-10-0-139-206 kubenswrapper[2557]: W0706 02:39:15.266898 2557 feature_gate.go:328] unrecognized feature gate: Example Jul 06 02:39:15.369417 ip-10-0-139-206 kubenswrapper[2557]: W0706 02:39:15.266900 2557 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 06 02:39:15.369417 ip-10-0-139-206 kubenswrapper[2557]: W0706 02:39:15.266903 2557 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 06 02:39:15.369417 ip-10-0-139-206 kubenswrapper[2557]: W0706 02:39:15.266911 2557 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 06 02:39:15.369417 ip-10-0-139-206 kubenswrapper[2557]: W0706 02:39:15.266915 2557 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 06 02:39:15.369417 ip-10-0-139-206 kubenswrapper[2557]: W0706 02:39:15.266918 2557 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 06 02:39:15.369417 ip-10-0-139-206 kubenswrapper[2557]: W0706 02:39:15.266921 2557 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 06 02:39:15.369417 ip-10-0-139-206 kubenswrapper[2557]: W0706 02:39:15.266923 2557 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 06 02:39:15.369417 ip-10-0-139-206 kubenswrapper[2557]: W0706 02:39:15.266926 2557 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 06 02:39:15.369417 ip-10-0-139-206 kubenswrapper[2557]: W0706 02:39:15.266929 2557 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 06 02:39:15.369417 ip-10-0-139-206 kubenswrapper[2557]: W0706 02:39:15.266931 2557 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 06 02:39:15.369417 ip-10-0-139-206 kubenswrapper[2557]: W0706 02:39:15.266934 2557 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 06 02:39:15.370333 ip-10-0-139-206 kubenswrapper[2557]: W0706 02:39:15.266936 2557 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 06 02:39:15.370333 ip-10-0-139-206 kubenswrapper[2557]: W0706 02:39:15.266939 2557 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 06 02:39:15.370333 ip-10-0-139-206 kubenswrapper[2557]: W0706 02:39:15.266942 2557 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 06 02:39:15.370333 ip-10-0-139-206 kubenswrapper[2557]: W0706 02:39:15.266945 2557 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 06 02:39:15.370333 ip-10-0-139-206 kubenswrapper[2557]: W0706 02:39:15.266947 2557 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 06 02:39:15.370333 ip-10-0-139-206 kubenswrapper[2557]: W0706 02:39:15.266950 2557 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 06 02:39:15.370333 ip-10-0-139-206 kubenswrapper[2557]: W0706 02:39:15.266952 2557 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 06 02:39:15.370333 ip-10-0-139-206 kubenswrapper[2557]: W0706 02:39:15.266955 2557 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 06 02:39:15.370333 ip-10-0-139-206 kubenswrapper[2557]: W0706 02:39:15.266957 2557 feature_gate.go:328] unrecognized feature gate: Example2 Jul 06 02:39:15.370333 ip-10-0-139-206 kubenswrapper[2557]: W0706 02:39:15.266960 2557 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 06 02:39:15.370333 ip-10-0-139-206 kubenswrapper[2557]: W0706 02:39:15.266963 2557 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 06 02:39:15.370333 ip-10-0-139-206 kubenswrapper[2557]: W0706 02:39:15.266965 2557 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 06 02:39:15.370333 ip-10-0-139-206 kubenswrapper[2557]: W0706 02:39:15.266968 2557 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 06 02:39:15.370333 ip-10-0-139-206 kubenswrapper[2557]: W0706 02:39:15.266970 2557 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 06 02:39:15.370333 ip-10-0-139-206 kubenswrapper[2557]: W0706 02:39:15.266973 2557 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 06 02:39:15.370333 ip-10-0-139-206 kubenswrapper[2557]: W0706 02:39:15.266975 2557 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 06 02:39:15.370333 ip-10-0-139-206 kubenswrapper[2557]: W0706 02:39:15.266984 2557 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 06 02:39:15.370333 ip-10-0-139-206 kubenswrapper[2557]: W0706 02:39:15.266987 2557 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 06 02:39:15.370333 ip-10-0-139-206 kubenswrapper[2557]: W0706 02:39:15.266990 2557 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 06 02:39:15.370333 ip-10-0-139-206 kubenswrapper[2557]: W0706 02:39:15.266994 2557 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 06 02:39:15.373350 ip-10-0-139-206 kubenswrapper[2557]: W0706 02:39:15.266996 2557 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 06 02:39:15.373350 ip-10-0-139-206 kubenswrapper[2557]: W0706 02:39:15.266999 2557 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 06 02:39:15.373350 ip-10-0-139-206 kubenswrapper[2557]: W0706 02:39:15.267001 2557 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 06 02:39:15.373350 ip-10-0-139-206 kubenswrapper[2557]: W0706 02:39:15.267004 2557 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 06 02:39:15.373350 ip-10-0-139-206 kubenswrapper[2557]: W0706 02:39:15.267007 2557 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 06 02:39:15.373350 ip-10-0-139-206 kubenswrapper[2557]: W0706 02:39:15.267009 2557 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 06 02:39:15.373350 ip-10-0-139-206 kubenswrapper[2557]: W0706 02:39:15.267012 2557 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 06 02:39:15.373350 ip-10-0-139-206 kubenswrapper[2557]: W0706 02:39:15.267014 2557 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 06 02:39:15.373350 ip-10-0-139-206 kubenswrapper[2557]: W0706 02:39:15.267018 2557 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 06 02:39:15.373350 ip-10-0-139-206 kubenswrapper[2557]: W0706 02:39:15.267022 2557 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 06 02:39:15.373350 ip-10-0-139-206 kubenswrapper[2557]: W0706 02:39:15.267025 2557 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 06 02:39:15.373350 ip-10-0-139-206 kubenswrapper[2557]: W0706 02:39:15.267027 2557 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 06 02:39:15.373350 ip-10-0-139-206 kubenswrapper[2557]: W0706 02:39:15.267030 2557 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 06 02:39:15.373350 ip-10-0-139-206 kubenswrapper[2557]: W0706 02:39:15.267032 2557 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 06 02:39:15.373350 ip-10-0-139-206 kubenswrapper[2557]: W0706 02:39:15.267036 2557 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 06 02:39:15.373350 ip-10-0-139-206 kubenswrapper[2557]: W0706 02:39:15.267448 2557 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 06 02:39:15.373350 ip-10-0-139-206 kubenswrapper[2557]: W0706 02:39:15.267453 2557 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 06 02:39:15.373350 ip-10-0-139-206 kubenswrapper[2557]: W0706 02:39:15.267455 2557 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 06 02:39:15.373350 ip-10-0-139-206 kubenswrapper[2557]: W0706 02:39:15.267458 2557 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 06 02:39:15.373350 ip-10-0-139-206 kubenswrapper[2557]: W0706 02:39:15.267461 2557 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 06 02:39:15.374217 ip-10-0-139-206 kubenswrapper[2557]: W0706 02:39:15.267464 2557 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 06 02:39:15.374217 ip-10-0-139-206 kubenswrapper[2557]: W0706 02:39:15.267467 2557 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 06 02:39:15.374217 ip-10-0-139-206 kubenswrapper[2557]: W0706 02:39:15.267469 2557 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 06 02:39:15.374217 ip-10-0-139-206 kubenswrapper[2557]: W0706 02:39:15.267472 2557 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 06 02:39:15.374217 ip-10-0-139-206 kubenswrapper[2557]: W0706 02:39:15.267475 2557 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 06 02:39:15.374217 ip-10-0-139-206 kubenswrapper[2557]: W0706 02:39:15.267477 2557 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 06 02:39:15.374217 ip-10-0-139-206 kubenswrapper[2557]: W0706 02:39:15.267480 2557 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 06 02:39:15.374217 ip-10-0-139-206 kubenswrapper[2557]: W0706 02:39:15.267482 2557 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 06 02:39:15.374217 ip-10-0-139-206 kubenswrapper[2557]: W0706 02:39:15.267487 2557 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 06 02:39:15.374217 ip-10-0-139-206 kubenswrapper[2557]: W0706 02:39:15.267490 2557 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 06 02:39:15.374217 ip-10-0-139-206 kubenswrapper[2557]: W0706 02:39:15.267494 2557 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 06 02:39:15.374217 ip-10-0-139-206 kubenswrapper[2557]: W0706 02:39:15.267498 2557 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 06 02:39:15.374217 ip-10-0-139-206 kubenswrapper[2557]: W0706 02:39:15.267501 2557 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 06 02:39:15.374217 ip-10-0-139-206 kubenswrapper[2557]: W0706 02:39:15.267503 2557 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 06 02:39:15.374217 ip-10-0-139-206 kubenswrapper[2557]: W0706 02:39:15.267506 2557 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 06 02:39:15.374217 ip-10-0-139-206 kubenswrapper[2557]: W0706 02:39:15.267509 2557 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 06 02:39:15.374217 ip-10-0-139-206 kubenswrapper[2557]: W0706 02:39:15.267511 2557 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 06 02:39:15.374217 ip-10-0-139-206 kubenswrapper[2557]: W0706 02:39:15.267514 2557 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 06 02:39:15.374217 ip-10-0-139-206 kubenswrapper[2557]: W0706 02:39:15.267516 2557 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 06 02:39:15.378686 ip-10-0-139-206 kubenswrapper[2557]: W0706 02:39:15.267519 2557 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 06 02:39:15.378686 ip-10-0-139-206 kubenswrapper[2557]: W0706 02:39:15.267522 2557 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 06 02:39:15.378686 ip-10-0-139-206 kubenswrapper[2557]: W0706 02:39:15.267525 2557 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 06 02:39:15.378686 ip-10-0-139-206 kubenswrapper[2557]: W0706 02:39:15.267528 2557 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 06 02:39:15.378686 ip-10-0-139-206 kubenswrapper[2557]: W0706 02:39:15.267530 2557 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 06 02:39:15.378686 ip-10-0-139-206 kubenswrapper[2557]: W0706 02:39:15.267533 2557 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 06 02:39:15.378686 ip-10-0-139-206 kubenswrapper[2557]: W0706 02:39:15.267535 2557 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 06 02:39:15.378686 ip-10-0-139-206 kubenswrapper[2557]: W0706 02:39:15.267538 2557 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 06 02:39:15.378686 ip-10-0-139-206 kubenswrapper[2557]: W0706 02:39:15.267540 2557 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 06 02:39:15.378686 ip-10-0-139-206 kubenswrapper[2557]: W0706 02:39:15.267543 2557 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 06 02:39:15.378686 ip-10-0-139-206 kubenswrapper[2557]: W0706 02:39:15.267545 2557 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 06 02:39:15.378686 ip-10-0-139-206 kubenswrapper[2557]: W0706 02:39:15.267548 2557 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 06 02:39:15.378686 ip-10-0-139-206 kubenswrapper[2557]: W0706 02:39:15.267550 2557 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 06 02:39:15.378686 ip-10-0-139-206 kubenswrapper[2557]: W0706 02:39:15.267553 2557 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 06 02:39:15.378686 ip-10-0-139-206 kubenswrapper[2557]: W0706 02:39:15.267555 2557 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 06 02:39:15.378686 ip-10-0-139-206 kubenswrapper[2557]: W0706 02:39:15.267558 2557 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 06 02:39:15.378686 ip-10-0-139-206 kubenswrapper[2557]: W0706 02:39:15.267560 2557 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 06 02:39:15.378686 ip-10-0-139-206 kubenswrapper[2557]: W0706 02:39:15.267562 2557 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 06 02:39:15.378686 ip-10-0-139-206 kubenswrapper[2557]: W0706 02:39:15.267566 2557 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 06 02:39:15.378686 ip-10-0-139-206 kubenswrapper[2557]: W0706 02:39:15.267569 2557 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 06 02:39:15.379580 ip-10-0-139-206 kubenswrapper[2557]: W0706 02:39:15.267572 2557 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 06 02:39:15.379580 ip-10-0-139-206 kubenswrapper[2557]: W0706 02:39:15.267576 2557 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 06 02:39:15.379580 ip-10-0-139-206 kubenswrapper[2557]: W0706 02:39:15.267578 2557 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 06 02:39:15.379580 ip-10-0-139-206 kubenswrapper[2557]: W0706 02:39:15.267581 2557 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 06 02:39:15.379580 ip-10-0-139-206 kubenswrapper[2557]: W0706 02:39:15.267584 2557 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 06 02:39:15.379580 ip-10-0-139-206 kubenswrapper[2557]: W0706 02:39:15.267587 2557 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 06 02:39:15.379580 ip-10-0-139-206 kubenswrapper[2557]: W0706 02:39:15.267589 2557 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 06 02:39:15.379580 ip-10-0-139-206 kubenswrapper[2557]: W0706 02:39:15.267592 2557 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 06 02:39:15.379580 ip-10-0-139-206 kubenswrapper[2557]: W0706 02:39:15.267594 2557 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 06 02:39:15.379580 ip-10-0-139-206 kubenswrapper[2557]: W0706 02:39:15.267597 2557 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 06 02:39:15.379580 ip-10-0-139-206 kubenswrapper[2557]: W0706 02:39:15.267599 2557 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 06 02:39:15.379580 ip-10-0-139-206 kubenswrapper[2557]: W0706 02:39:15.267602 2557 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 06 02:39:15.379580 ip-10-0-139-206 kubenswrapper[2557]: W0706 02:39:15.267605 2557 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 06 02:39:15.379580 ip-10-0-139-206 kubenswrapper[2557]: W0706 02:39:15.267607 2557 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 06 02:39:15.379580 ip-10-0-139-206 kubenswrapper[2557]: W0706 02:39:15.267610 2557 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 06 02:39:15.379580 ip-10-0-139-206 kubenswrapper[2557]: W0706 02:39:15.267615 2557 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 06 02:39:15.379580 ip-10-0-139-206 kubenswrapper[2557]: W0706 02:39:15.267618 2557 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 06 02:39:15.379580 ip-10-0-139-206 kubenswrapper[2557]: W0706 02:39:15.267621 2557 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 06 02:39:15.379580 ip-10-0-139-206 kubenswrapper[2557]: W0706 02:39:15.267623 2557 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 06 02:39:15.384413 ip-10-0-139-206 kubenswrapper[2557]: W0706 02:39:15.267626 2557 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 06 02:39:15.384413 ip-10-0-139-206 kubenswrapper[2557]: W0706 02:39:15.267644 2557 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 06 02:39:15.384413 ip-10-0-139-206 kubenswrapper[2557]: W0706 02:39:15.267647 2557 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 06 02:39:15.384413 ip-10-0-139-206 kubenswrapper[2557]: W0706 02:39:15.267650 2557 feature_gate.go:328] unrecognized feature gate: Example2 Jul 06 02:39:15.384413 ip-10-0-139-206 kubenswrapper[2557]: W0706 02:39:15.267653 2557 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 06 02:39:15.384413 ip-10-0-139-206 kubenswrapper[2557]: W0706 02:39:15.267655 2557 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 06 02:39:15.384413 ip-10-0-139-206 kubenswrapper[2557]: W0706 02:39:15.267657 2557 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 06 02:39:15.384413 ip-10-0-139-206 kubenswrapper[2557]: W0706 02:39:15.267660 2557 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 06 02:39:15.384413 ip-10-0-139-206 kubenswrapper[2557]: W0706 02:39:15.267663 2557 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 06 02:39:15.384413 ip-10-0-139-206 kubenswrapper[2557]: W0706 02:39:15.267666 2557 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 06 02:39:15.384413 ip-10-0-139-206 kubenswrapper[2557]: W0706 02:39:15.267669 2557 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 06 02:39:15.384413 ip-10-0-139-206 kubenswrapper[2557]: W0706 02:39:15.267671 2557 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 06 02:39:15.384413 ip-10-0-139-206 kubenswrapper[2557]: W0706 02:39:15.267674 2557 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 06 02:39:15.384413 ip-10-0-139-206 kubenswrapper[2557]: W0706 02:39:15.267677 2557 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 06 02:39:15.384413 ip-10-0-139-206 kubenswrapper[2557]: W0706 02:39:15.267681 2557 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 06 02:39:15.384413 ip-10-0-139-206 kubenswrapper[2557]: W0706 02:39:15.267684 2557 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 06 02:39:15.384413 ip-10-0-139-206 kubenswrapper[2557]: W0706 02:39:15.267686 2557 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 06 02:39:15.384413 ip-10-0-139-206 kubenswrapper[2557]: W0706 02:39:15.267689 2557 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 06 02:39:15.384413 ip-10-0-139-206 kubenswrapper[2557]: W0706 02:39:15.267692 2557 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 06 02:39:15.384413 ip-10-0-139-206 kubenswrapper[2557]: W0706 02:39:15.267694 2557 feature_gate.go:328] unrecognized feature gate: Example Jul 06 02:39:15.385326 ip-10-0-139-206 kubenswrapper[2557]: W0706 02:39:15.267699 2557 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 06 02:39:15.385326 ip-10-0-139-206 kubenswrapper[2557]: W0706 02:39:15.267701 2557 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 06 02:39:15.385326 ip-10-0-139-206 kubenswrapper[2557]: W0706 02:39:15.267704 2557 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 06 02:39:15.385326 ip-10-0-139-206 kubenswrapper[2557]: W0706 02:39:15.267708 2557 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 06 02:39:15.385326 ip-10-0-139-206 kubenswrapper[2557]: W0706 02:39:15.267710 2557 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 06 02:39:15.385326 ip-10-0-139-206 kubenswrapper[2557]: W0706 02:39:15.267713 2557 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 06 02:39:15.385326 ip-10-0-139-206 kubenswrapper[2557]: W0706 02:39:15.267716 2557 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 06 02:39:15.385326 ip-10-0-139-206 kubenswrapper[2557]: W0706 02:39:15.267718 2557 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 06 02:39:15.385326 ip-10-0-139-206 kubenswrapper[2557]: W0706 02:39:15.267721 2557 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 06 02:39:15.385326 ip-10-0-139-206 kubenswrapper[2557]: W0706 02:39:15.267724 2557 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 06 02:39:15.385326 ip-10-0-139-206 kubenswrapper[2557]: W0706 02:39:15.267726 2557 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 06 02:39:15.385326 ip-10-0-139-206 kubenswrapper[2557]: W0706 02:39:15.267729 2557 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 06 02:39:15.385326 ip-10-0-139-206 kubenswrapper[2557]: W0706 02:39:15.267731 2557 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 06 02:39:15.385326 ip-10-0-139-206 kubenswrapper[2557]: W0706 02:39:15.267735 2557 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 06 02:39:15.385326 ip-10-0-139-206 kubenswrapper[2557]: W0706 02:39:15.267739 2557 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 06 02:39:15.385326 ip-10-0-139-206 kubenswrapper[2557]: W0706 02:39:15.267742 2557 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 06 02:39:15.385326 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:39:15.267818 2557 flags.go:64] FLAG: --address="0.0.0.0" Jul 06 02:39:15.385326 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:39:15.267826 2557 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]" Jul 06 02:39:15.385326 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:39:15.267833 2557 flags.go:64] FLAG: --anonymous-auth="true" Jul 06 02:39:15.385326 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:39:15.267838 2557 flags.go:64] FLAG: --application-metrics-count-limit="100" Jul 06 02:39:15.385326 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:39:15.267842 2557 flags.go:64] FLAG: --authentication-token-webhook="false" Jul 06 02:39:15.388785 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:39:15.267846 2557 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s" Jul 06 02:39:15.388785 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:39:15.267850 2557 flags.go:64] FLAG: --authorization-mode="AlwaysAllow" Jul 06 02:39:15.388785 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:39:15.267854 2557 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s" Jul 06 02:39:15.388785 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:39:15.267858 2557 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s" Jul 06 02:39:15.388785 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:39:15.267861 2557 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id" Jul 06 02:39:15.388785 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:39:15.267866 2557 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig" Jul 06 02:39:15.388785 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:39:15.267869 2557 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki" Jul 06 02:39:15.388785 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:39:15.267872 2557 flags.go:64] FLAG: --cgroup-driver="cgroupfs" Jul 06 02:39:15.388785 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:39:15.267875 2557 flags.go:64] FLAG: --cgroup-root="" Jul 06 02:39:15.388785 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:39:15.267878 2557 flags.go:64] FLAG: --cgroups-per-qos="true" Jul 06 02:39:15.388785 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:39:15.267881 2557 flags.go:64] FLAG: --client-ca-file="" Jul 06 02:39:15.388785 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:39:15.267885 2557 flags.go:64] FLAG: --cloud-provider="external" Jul 06 02:39:15.388785 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:39:15.267888 2557 flags.go:64] FLAG: --cluster-dns="[]" Jul 06 02:39:15.388785 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:39:15.267892 2557 flags.go:64] FLAG: --cluster-domain="" Jul 06 02:39:15.388785 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:39:15.267895 2557 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf" Jul 06 02:39:15.388785 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:39:15.267898 2557 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d" Jul 06 02:39:15.388785 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:39:15.267901 2557 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json" Jul 06 02:39:15.388785 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:39:15.267904 2557 flags.go:64] FLAG: --container-log-max-files="5" Jul 06 02:39:15.388785 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:39:15.267908 2557 flags.go:64] FLAG: --container-log-max-size="10Mi" Jul 06 02:39:15.388785 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:39:15.267911 2557 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock" Jul 06 02:39:15.388785 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:39:15.267914 2557 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock" Jul 06 02:39:15.388785 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:39:15.267919 2557 flags.go:64] FLAG: --containerd-namespace="k8s.io" Jul 06 02:39:15.388785 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:39:15.267922 2557 flags.go:64] FLAG: --contention-profiling="false" Jul 06 02:39:15.388785 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:39:15.267925 2557 flags.go:64] FLAG: --cpu-cfs-quota="true" Jul 06 02:39:15.389587 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:39:15.267928 2557 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms" Jul 06 02:39:15.389587 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:39:15.267931 2557 flags.go:64] FLAG: --cpu-manager-policy="none" Jul 06 02:39:15.389587 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:39:15.267934 2557 flags.go:64] FLAG: --cpu-manager-policy-options="" Jul 06 02:39:15.389587 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:39:15.267942 2557 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s" Jul 06 02:39:15.389587 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:39:15.267945 2557 flags.go:64] FLAG: --enable-controller-attach-detach="true" Jul 06 02:39:15.389587 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:39:15.267949 2557 flags.go:64] FLAG: --enable-debugging-handlers="true" Jul 06 02:39:15.389587 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:39:15.267952 2557 flags.go:64] FLAG: --enable-load-reader="false" Jul 06 02:39:15.389587 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:39:15.267955 2557 flags.go:64] FLAG: --enable-server="true" Jul 06 02:39:15.389587 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:39:15.267958 2557 flags.go:64] FLAG: --enforce-node-allocatable="[pods]" Jul 06 02:39:15.389587 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:39:15.267963 2557 flags.go:64] FLAG: --event-burst="100" Jul 06 02:39:15.389587 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:39:15.267966 2557 flags.go:64] FLAG: --event-qps="50" Jul 06 02:39:15.389587 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:39:15.267969 2557 flags.go:64] FLAG: --event-storage-age-limit="default=0" Jul 06 02:39:15.389587 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:39:15.267973 2557 flags.go:64] FLAG: --event-storage-event-limit="default=0" Jul 06 02:39:15.389587 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:39:15.267987 2557 flags.go:64] FLAG: --eviction-hard="" Jul 06 02:39:15.389587 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:39:15.267992 2557 flags.go:64] FLAG: --eviction-max-pod-grace-period="0" Jul 06 02:39:15.389587 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:39:15.267995 2557 flags.go:64] FLAG: --eviction-minimum-reclaim="" Jul 06 02:39:15.389587 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:39:15.267998 2557 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s" Jul 06 02:39:15.389587 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:39:15.268002 2557 flags.go:64] FLAG: --eviction-soft="" Jul 06 02:39:15.389587 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:39:15.268004 2557 flags.go:64] FLAG: --eviction-soft-grace-period="" Jul 06 02:39:15.389587 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:39:15.268009 2557 flags.go:64] FLAG: --exit-on-lock-contention="false" Jul 06 02:39:15.389587 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:39:15.268012 2557 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false" Jul 06 02:39:15.389587 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:39:15.268015 2557 flags.go:64] FLAG: --experimental-mounter-path="" Jul 06 02:39:15.389587 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:39:15.268018 2557 flags.go:64] FLAG: --fail-cgroupv1="false" Jul 06 02:39:15.389587 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:39:15.268021 2557 flags.go:64] FLAG: --fail-swap-on="true" Jul 06 02:39:15.389587 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:39:15.268024 2557 flags.go:64] FLAG: --feature-gates="" Jul 06 02:39:15.393058 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:39:15.268027 2557 flags.go:64] FLAG: --file-check-frequency="20s" Jul 06 02:39:15.393058 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:39:15.268030 2557 flags.go:64] FLAG: --global-housekeeping-interval="1m0s" Jul 06 02:39:15.393058 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:39:15.268034 2557 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge" Jul 06 02:39:15.393058 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:39:15.268037 2557 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1" Jul 06 02:39:15.393058 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:39:15.268040 2557 flags.go:64] FLAG: --healthz-port="10248" Jul 06 02:39:15.393058 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:39:15.268048 2557 flags.go:64] FLAG: --help="false" Jul 06 02:39:15.393058 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:39:15.268051 2557 flags.go:64] FLAG: --hostname-override="ip-10-0-139-206.ec2.internal" Jul 06 02:39:15.393058 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:39:15.268054 2557 flags.go:64] FLAG: --housekeeping-interval="10s" Jul 06 02:39:15.393058 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:39:15.268057 2557 flags.go:64] FLAG: --http-check-frequency="20s" Jul 06 02:39:15.393058 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:39:15.268060 2557 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins" Jul 06 02:39:15.393058 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:39:15.268064 2557 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml" Jul 06 02:39:15.393058 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:39:15.268067 2557 flags.go:64] FLAG: --image-gc-high-threshold="85" Jul 06 02:39:15.393058 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:39:15.268070 2557 flags.go:64] FLAG: --image-gc-low-threshold="80" Jul 06 02:39:15.393058 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:39:15.268073 2557 flags.go:64] FLAG: --image-service-endpoint="" Jul 06 02:39:15.393058 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:39:15.268075 2557 flags.go:64] FLAG: --kernel-memcg-notification="false" Jul 06 02:39:15.393058 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:39:15.268078 2557 flags.go:64] FLAG: --kube-api-burst="100" Jul 06 02:39:15.393058 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:39:15.268081 2557 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf" Jul 06 02:39:15.393058 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:39:15.268084 2557 flags.go:64] FLAG: --kube-api-qps="50" Jul 06 02:39:15.393058 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:39:15.268087 2557 flags.go:64] FLAG: --kube-reserved="" Jul 06 02:39:15.393058 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:39:15.268090 2557 flags.go:64] FLAG: --kube-reserved-cgroup="" Jul 06 02:39:15.393058 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:39:15.268094 2557 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig" Jul 06 02:39:15.393058 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:39:15.268097 2557 flags.go:64] FLAG: --kubelet-cgroups="" Jul 06 02:39:15.393058 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:39:15.268100 2557 flags.go:64] FLAG: --local-storage-capacity-isolation="true" Jul 06 02:39:15.393058 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:39:15.268103 2557 flags.go:64] FLAG: --lock-file="" Jul 06 02:39:15.393870 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:39:15.268106 2557 flags.go:64] FLAG: --log-cadvisor-usage="false" Jul 06 02:39:15.393870 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:39:15.268108 2557 flags.go:64] FLAG: --log-flush-frequency="5s" Jul 06 02:39:15.393870 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:39:15.268113 2557 flags.go:64] FLAG: --log-json-info-buffer-size="0" Jul 06 02:39:15.393870 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:39:15.268118 2557 flags.go:64] FLAG: --log-json-split-stream="false" Jul 06 02:39:15.393870 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:39:15.268121 2557 flags.go:64] FLAG: --log-text-info-buffer-size="0" Jul 06 02:39:15.393870 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:39:15.268124 2557 flags.go:64] FLAG: --log-text-split-stream="false" Jul 06 02:39:15.393870 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:39:15.268127 2557 flags.go:64] FLAG: --logging-format="text" Jul 06 02:39:15.393870 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:39:15.268129 2557 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id" Jul 06 02:39:15.393870 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:39:15.268133 2557 flags.go:64] FLAG: --make-iptables-util-chains="true" Jul 06 02:39:15.393870 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:39:15.268136 2557 flags.go:64] FLAG: --manifest-url="" Jul 06 02:39:15.393870 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:39:15.268138 2557 flags.go:64] FLAG: --manifest-url-header="" Jul 06 02:39:15.393870 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:39:15.268143 2557 flags.go:64] FLAG: --max-housekeeping-interval="15s" Jul 06 02:39:15.393870 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:39:15.268146 2557 flags.go:64] FLAG: --max-open-files="1000000" Jul 06 02:39:15.393870 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:39:15.268151 2557 flags.go:64] FLAG: --max-pods="110" Jul 06 02:39:15.393870 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:39:15.268155 2557 flags.go:64] FLAG: --maximum-dead-containers="-1" Jul 06 02:39:15.393870 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:39:15.268158 2557 flags.go:64] FLAG: --maximum-dead-containers-per-container="1" Jul 06 02:39:15.393870 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:39:15.268161 2557 flags.go:64] FLAG: --memory-manager-policy="None" Jul 06 02:39:15.393870 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:39:15.268164 2557 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s" Jul 06 02:39:15.393870 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:39:15.268167 2557 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s" Jul 06 02:39:15.393870 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:39:15.268169 2557 flags.go:64] FLAG: --node-ip="0.0.0.0" Jul 06 02:39:15.393870 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:39:15.268173 2557 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel" Jul 06 02:39:15.393870 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:39:15.268180 2557 flags.go:64] FLAG: --node-status-max-images="50" Jul 06 02:39:15.393870 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:39:15.268183 2557 flags.go:64] FLAG: --node-status-update-frequency="10s" Jul 06 02:39:15.393870 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:39:15.268186 2557 flags.go:64] FLAG: --oom-score-adj="-999" Jul 06 02:39:15.393870 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:39:15.268189 2557 flags.go:64] FLAG: --pod-cidr="" Jul 06 02:39:15.501798 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:39:15.268192 2557 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:ce93f6b4197b9c62a26a3131f8057ffe9c9379c5079a4f62e5919b75151394e5" Jul 06 02:39:15.501798 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:39:15.268197 2557 flags.go:64] FLAG: --pod-manifest-path="" Jul 06 02:39:15.501798 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:39:15.268200 2557 flags.go:64] FLAG: --pod-max-pids="-1" Jul 06 02:39:15.501798 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:39:15.268203 2557 flags.go:64] FLAG: --pods-per-core="0" Jul 06 02:39:15.501798 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:39:15.268207 2557 flags.go:64] FLAG: --port="10250" Jul 06 02:39:15.501798 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:39:15.268210 2557 flags.go:64] FLAG: --protect-kernel-defaults="false" Jul 06 02:39:15.501798 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:39:15.268214 2557 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-0b3fc373f4d5b814c" Jul 06 02:39:15.501798 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:39:15.268217 2557 flags.go:64] FLAG: --qos-reserved="" Jul 06 02:39:15.501798 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:39:15.268220 2557 flags.go:64] FLAG: --read-only-port="10255" Jul 06 02:39:15.501798 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:39:15.268224 2557 flags.go:64] FLAG: --register-node="true" Jul 06 02:39:15.501798 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:39:15.268227 2557 flags.go:64] FLAG: --register-with-taints="" Jul 06 02:39:15.501798 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:39:15.268230 2557 flags.go:64] FLAG: --registry-burst="10" Jul 06 02:39:15.501798 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:39:15.268233 2557 flags.go:64] FLAG: --registry-qps="5" Jul 06 02:39:15.501798 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:39:15.268236 2557 flags.go:64] FLAG: --reserved-cpus="" Jul 06 02:39:15.501798 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:39:15.268239 2557 flags.go:64] FLAG: --reserved-memory="" Jul 06 02:39:15.501798 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:39:15.268243 2557 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf" Jul 06 02:39:15.501798 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:39:15.268246 2557 flags.go:64] FLAG: --root-dir="/var/lib/kubelet" Jul 06 02:39:15.501798 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:39:15.268249 2557 flags.go:64] FLAG: --rotate-certificates="false" Jul 06 02:39:15.501798 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:39:15.268252 2557 flags.go:64] FLAG: --rotate-server-certificates="false" Jul 06 02:39:15.501798 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:39:15.268255 2557 flags.go:64] FLAG: --runonce="false" Jul 06 02:39:15.501798 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:39:15.268258 2557 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service" Jul 06 02:39:15.501798 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:39:15.268261 2557 flags.go:64] FLAG: --runtime-request-timeout="2m0s" Jul 06 02:39:15.501798 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:39:15.268264 2557 flags.go:64] FLAG: --seccomp-default="false" Jul 06 02:39:15.501798 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:39:15.268267 2557 flags.go:64] FLAG: --serialize-image-pulls="true" Jul 06 02:39:15.501798 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:39:15.268270 2557 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s" Jul 06 02:39:15.503130 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:39:15.268273 2557 flags.go:64] FLAG: --storage-driver-db="cadvisor" Jul 06 02:39:15.503130 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:39:15.268276 2557 flags.go:64] FLAG: --storage-driver-host="localhost:8086" Jul 06 02:39:15.503130 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:39:15.268279 2557 flags.go:64] FLAG: --storage-driver-password="root" Jul 06 02:39:15.503130 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:39:15.268282 2557 flags.go:64] FLAG: --storage-driver-secure="false" Jul 06 02:39:15.503130 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:39:15.268285 2557 flags.go:64] FLAG: --storage-driver-table="stats" Jul 06 02:39:15.503130 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:39:15.268287 2557 flags.go:64] FLAG: --storage-driver-user="root" Jul 06 02:39:15.503130 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:39:15.268290 2557 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s" Jul 06 02:39:15.503130 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:39:15.268293 2557 flags.go:64] FLAG: --sync-frequency="1m0s" Jul 06 02:39:15.503130 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:39:15.268296 2557 flags.go:64] FLAG: --system-cgroups="" Jul 06 02:39:15.503130 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:39:15.268299 2557 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi" Jul 06 02:39:15.503130 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:39:15.268304 2557 flags.go:64] FLAG: --system-reserved-cgroup="" Jul 06 02:39:15.503130 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:39:15.268308 2557 flags.go:64] FLAG: --tls-cert-file="" Jul 06 02:39:15.503130 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:39:15.268311 2557 flags.go:64] FLAG: --tls-cipher-suites="[]" Jul 06 02:39:15.503130 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:39:15.268315 2557 flags.go:64] FLAG: --tls-min-version="" Jul 06 02:39:15.503130 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:39:15.268318 2557 flags.go:64] FLAG: --tls-private-key-file="" Jul 06 02:39:15.503130 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:39:15.268321 2557 flags.go:64] FLAG: --topology-manager-policy="none" Jul 06 02:39:15.503130 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:39:15.268325 2557 flags.go:64] FLAG: --topology-manager-policy-options="" Jul 06 02:39:15.503130 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:39:15.268328 2557 flags.go:64] FLAG: --topology-manager-scope="container" Jul 06 02:39:15.503130 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:39:15.268332 2557 flags.go:64] FLAG: --v="2" Jul 06 02:39:15.503130 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:39:15.268336 2557 flags.go:64] FLAG: --version="false" Jul 06 02:39:15.503130 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:39:15.268340 2557 flags.go:64] FLAG: --vmodule="" Jul 06 02:39:15.503130 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:39:15.268343 2557 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec" Jul 06 02:39:15.503130 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:39:15.268347 2557 flags.go:64] FLAG: --volume-stats-agg-period="1m0s" Jul 06 02:39:15.503130 ip-10-0-139-206 kubenswrapper[2557]: W0706 02:39:15.268492 2557 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 06 02:39:15.503130 ip-10-0-139-206 kubenswrapper[2557]: W0706 02:39:15.268497 2557 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 06 02:39:15.507503 ip-10-0-139-206 kubenswrapper[2557]: W0706 02:39:15.268500 2557 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 06 02:39:15.507503 ip-10-0-139-206 kubenswrapper[2557]: W0706 02:39:15.268504 2557 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 06 02:39:15.507503 ip-10-0-139-206 kubenswrapper[2557]: W0706 02:39:15.268508 2557 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 06 02:39:15.507503 ip-10-0-139-206 kubenswrapper[2557]: W0706 02:39:15.268511 2557 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 06 02:39:15.507503 ip-10-0-139-206 kubenswrapper[2557]: W0706 02:39:15.268514 2557 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 06 02:39:15.507503 ip-10-0-139-206 kubenswrapper[2557]: W0706 02:39:15.268520 2557 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 06 02:39:15.507503 ip-10-0-139-206 kubenswrapper[2557]: W0706 02:39:15.268523 2557 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 06 02:39:15.507503 ip-10-0-139-206 kubenswrapper[2557]: W0706 02:39:15.268526 2557 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 06 02:39:15.507503 ip-10-0-139-206 kubenswrapper[2557]: W0706 02:39:15.268530 2557 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 06 02:39:15.507503 ip-10-0-139-206 kubenswrapper[2557]: W0706 02:39:15.268533 2557 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 06 02:39:15.507503 ip-10-0-139-206 kubenswrapper[2557]: W0706 02:39:15.268537 2557 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 06 02:39:15.507503 ip-10-0-139-206 kubenswrapper[2557]: W0706 02:39:15.268539 2557 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 06 02:39:15.507503 ip-10-0-139-206 kubenswrapper[2557]: W0706 02:39:15.268542 2557 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 06 02:39:15.507503 ip-10-0-139-206 kubenswrapper[2557]: W0706 02:39:15.268545 2557 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 06 02:39:15.507503 ip-10-0-139-206 kubenswrapper[2557]: W0706 02:39:15.268547 2557 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 06 02:39:15.507503 ip-10-0-139-206 kubenswrapper[2557]: W0706 02:39:15.268550 2557 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 06 02:39:15.507503 ip-10-0-139-206 kubenswrapper[2557]: W0706 02:39:15.268552 2557 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 06 02:39:15.507503 ip-10-0-139-206 kubenswrapper[2557]: W0706 02:39:15.268555 2557 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 06 02:39:15.507503 ip-10-0-139-206 kubenswrapper[2557]: W0706 02:39:15.268559 2557 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 06 02:39:15.508322 ip-10-0-139-206 kubenswrapper[2557]: W0706 02:39:15.268561 2557 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 06 02:39:15.508322 ip-10-0-139-206 kubenswrapper[2557]: W0706 02:39:15.268564 2557 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 06 02:39:15.508322 ip-10-0-139-206 kubenswrapper[2557]: W0706 02:39:15.268567 2557 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 06 02:39:15.508322 ip-10-0-139-206 kubenswrapper[2557]: W0706 02:39:15.268569 2557 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 06 02:39:15.508322 ip-10-0-139-206 kubenswrapper[2557]: W0706 02:39:15.268573 2557 feature_gate.go:328] unrecognized feature gate: Example2 Jul 06 02:39:15.508322 ip-10-0-139-206 kubenswrapper[2557]: W0706 02:39:15.268576 2557 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 06 02:39:15.508322 ip-10-0-139-206 kubenswrapper[2557]: W0706 02:39:15.268578 2557 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 06 02:39:15.508322 ip-10-0-139-206 kubenswrapper[2557]: W0706 02:39:15.268581 2557 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 06 02:39:15.508322 ip-10-0-139-206 kubenswrapper[2557]: W0706 02:39:15.268583 2557 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 06 02:39:15.508322 ip-10-0-139-206 kubenswrapper[2557]: W0706 02:39:15.268586 2557 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 06 02:39:15.508322 ip-10-0-139-206 kubenswrapper[2557]: W0706 02:39:15.268590 2557 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 06 02:39:15.508322 ip-10-0-139-206 kubenswrapper[2557]: W0706 02:39:15.268593 2557 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 06 02:39:15.508322 ip-10-0-139-206 kubenswrapper[2557]: W0706 02:39:15.268596 2557 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 06 02:39:15.508322 ip-10-0-139-206 kubenswrapper[2557]: W0706 02:39:15.268599 2557 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 06 02:39:15.508322 ip-10-0-139-206 kubenswrapper[2557]: W0706 02:39:15.268602 2557 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 06 02:39:15.508322 ip-10-0-139-206 kubenswrapper[2557]: W0706 02:39:15.268605 2557 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 06 02:39:15.508322 ip-10-0-139-206 kubenswrapper[2557]: W0706 02:39:15.268608 2557 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 06 02:39:15.508322 ip-10-0-139-206 kubenswrapper[2557]: W0706 02:39:15.268611 2557 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 06 02:39:15.508322 ip-10-0-139-206 kubenswrapper[2557]: W0706 02:39:15.268614 2557 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 06 02:39:15.508322 ip-10-0-139-206 kubenswrapper[2557]: W0706 02:39:15.268620 2557 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 06 02:39:15.512200 ip-10-0-139-206 kubenswrapper[2557]: W0706 02:39:15.268623 2557 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 06 02:39:15.512200 ip-10-0-139-206 kubenswrapper[2557]: W0706 02:39:15.268626 2557 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 06 02:39:15.512200 ip-10-0-139-206 kubenswrapper[2557]: W0706 02:39:15.268641 2557 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 06 02:39:15.512200 ip-10-0-139-206 kubenswrapper[2557]: W0706 02:39:15.268644 2557 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 06 02:39:15.512200 ip-10-0-139-206 kubenswrapper[2557]: W0706 02:39:15.268647 2557 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 06 02:39:15.512200 ip-10-0-139-206 kubenswrapper[2557]: W0706 02:39:15.268649 2557 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 06 02:39:15.512200 ip-10-0-139-206 kubenswrapper[2557]: W0706 02:39:15.268652 2557 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 06 02:39:15.512200 ip-10-0-139-206 kubenswrapper[2557]: W0706 02:39:15.268654 2557 feature_gate.go:328] unrecognized feature gate: Example Jul 06 02:39:15.512200 ip-10-0-139-206 kubenswrapper[2557]: W0706 02:39:15.268656 2557 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 06 02:39:15.512200 ip-10-0-139-206 kubenswrapper[2557]: W0706 02:39:15.268660 2557 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 06 02:39:15.512200 ip-10-0-139-206 kubenswrapper[2557]: W0706 02:39:15.268662 2557 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 06 02:39:15.512200 ip-10-0-139-206 kubenswrapper[2557]: W0706 02:39:15.268666 2557 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 06 02:39:15.512200 ip-10-0-139-206 kubenswrapper[2557]: W0706 02:39:15.268668 2557 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 06 02:39:15.512200 ip-10-0-139-206 kubenswrapper[2557]: W0706 02:39:15.268671 2557 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 06 02:39:15.512200 ip-10-0-139-206 kubenswrapper[2557]: W0706 02:39:15.268673 2557 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 06 02:39:15.512200 ip-10-0-139-206 kubenswrapper[2557]: W0706 02:39:15.268676 2557 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 06 02:39:15.512200 ip-10-0-139-206 kubenswrapper[2557]: W0706 02:39:15.268680 2557 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 06 02:39:15.512200 ip-10-0-139-206 kubenswrapper[2557]: W0706 02:39:15.268682 2557 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 06 02:39:15.512200 ip-10-0-139-206 kubenswrapper[2557]: W0706 02:39:15.268685 2557 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 06 02:39:15.512200 ip-10-0-139-206 kubenswrapper[2557]: W0706 02:39:15.268687 2557 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 06 02:39:15.513070 ip-10-0-139-206 kubenswrapper[2557]: W0706 02:39:15.268690 2557 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 06 02:39:15.513070 ip-10-0-139-206 kubenswrapper[2557]: W0706 02:39:15.268693 2557 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 06 02:39:15.513070 ip-10-0-139-206 kubenswrapper[2557]: W0706 02:39:15.268696 2557 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 06 02:39:15.513070 ip-10-0-139-206 kubenswrapper[2557]: W0706 02:39:15.268698 2557 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 06 02:39:15.513070 ip-10-0-139-206 kubenswrapper[2557]: W0706 02:39:15.268701 2557 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 06 02:39:15.513070 ip-10-0-139-206 kubenswrapper[2557]: W0706 02:39:15.268703 2557 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 06 02:39:15.513070 ip-10-0-139-206 kubenswrapper[2557]: W0706 02:39:15.268706 2557 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 06 02:39:15.513070 ip-10-0-139-206 kubenswrapper[2557]: W0706 02:39:15.268709 2557 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 06 02:39:15.513070 ip-10-0-139-206 kubenswrapper[2557]: W0706 02:39:15.268711 2557 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 06 02:39:15.513070 ip-10-0-139-206 kubenswrapper[2557]: W0706 02:39:15.268714 2557 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 06 02:39:15.513070 ip-10-0-139-206 kubenswrapper[2557]: W0706 02:39:15.268716 2557 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 06 02:39:15.513070 ip-10-0-139-206 kubenswrapper[2557]: W0706 02:39:15.268719 2557 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 06 02:39:15.513070 ip-10-0-139-206 kubenswrapper[2557]: W0706 02:39:15.268722 2557 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 06 02:39:15.513070 ip-10-0-139-206 kubenswrapper[2557]: W0706 02:39:15.268725 2557 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 06 02:39:15.513070 ip-10-0-139-206 kubenswrapper[2557]: W0706 02:39:15.268727 2557 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 06 02:39:15.513070 ip-10-0-139-206 kubenswrapper[2557]: W0706 02:39:15.268730 2557 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 06 02:39:15.513070 ip-10-0-139-206 kubenswrapper[2557]: W0706 02:39:15.268732 2557 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 06 02:39:15.513070 ip-10-0-139-206 kubenswrapper[2557]: W0706 02:39:15.268735 2557 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 06 02:39:15.513070 ip-10-0-139-206 kubenswrapper[2557]: W0706 02:39:15.268737 2557 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 06 02:39:15.513070 ip-10-0-139-206 kubenswrapper[2557]: W0706 02:39:15.268739 2557 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 06 02:39:15.516111 ip-10-0-139-206 kubenswrapper[2557]: W0706 02:39:15.268742 2557 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 06 02:39:15.516111 ip-10-0-139-206 kubenswrapper[2557]: W0706 02:39:15.268744 2557 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 06 02:39:15.516111 ip-10-0-139-206 kubenswrapper[2557]: W0706 02:39:15.268746 2557 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 06 02:39:15.516111 ip-10-0-139-206 kubenswrapper[2557]: W0706 02:39:15.268752 2557 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 06 02:39:15.516111 ip-10-0-139-206 kubenswrapper[2557]: W0706 02:39:15.268755 2557 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 06 02:39:15.516111 ip-10-0-139-206 kubenswrapper[2557]: W0706 02:39:15.268758 2557 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 06 02:39:15.516111 ip-10-0-139-206 kubenswrapper[2557]: W0706 02:39:15.268760 2557 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 06 02:39:15.516111 ip-10-0-139-206 kubenswrapper[2557]: W0706 02:39:15.268763 2557 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 06 02:39:15.516111 ip-10-0-139-206 kubenswrapper[2557]: W0706 02:39:15.268767 2557 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 06 02:39:15.516111 ip-10-0-139-206 kubenswrapper[2557]: W0706 02:39:15.268770 2557 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 06 02:39:15.516111 ip-10-0-139-206 kubenswrapper[2557]: W0706 02:39:15.268772 2557 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 06 02:39:15.516111 ip-10-0-139-206 kubenswrapper[2557]: W0706 02:39:15.268775 2557 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 06 02:39:15.516111 ip-10-0-139-206 kubenswrapper[2557]: W0706 02:39:15.268777 2557 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 06 02:39:15.516111 ip-10-0-139-206 kubenswrapper[2557]: W0706 02:39:15.268780 2557 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 06 02:39:15.516111 ip-10-0-139-206 kubenswrapper[2557]: W0706 02:39:15.268782 2557 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 06 02:39:15.516111 ip-10-0-139-206 kubenswrapper[2557]: W0706 02:39:15.268785 2557 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 06 02:39:15.516111 ip-10-0-139-206 kubenswrapper[2557]: W0706 02:39:15.268787 2557 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 06 02:39:15.516111 ip-10-0-139-206 kubenswrapper[2557]: W0706 02:39:15.268790 2557 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 06 02:39:15.516969 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:39:15.268795 2557 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 06 02:39:15.516969 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:39:15.275970 2557 server.go:529] "Kubelet version" kubeletVersion="v1.34.8" Jul 06 02:39:15.516969 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:39:15.275998 2557 server.go:531] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK="" Jul 06 02:39:15.516969 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:39:15.276019 2557 watchdog_linux.go:95] "Systemd watchdog is not enabled" Jul 06 02:39:15.516969 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:39:15.276025 2557 watchdog_linux.go:137] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started." Jul 06 02:39:15.516969 ip-10-0-139-206 kubenswrapper[2557]: W0706 02:39:15.276093 2557 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 06 02:39:15.516969 ip-10-0-139-206 kubenswrapper[2557]: W0706 02:39:15.276099 2557 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 06 02:39:15.516969 ip-10-0-139-206 kubenswrapper[2557]: W0706 02:39:15.276102 2557 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 06 02:39:15.516969 ip-10-0-139-206 kubenswrapper[2557]: W0706 02:39:15.276105 2557 feature_gate.go:328] unrecognized feature gate: Example Jul 06 02:39:15.516969 ip-10-0-139-206 kubenswrapper[2557]: W0706 02:39:15.276108 2557 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 06 02:39:15.516969 ip-10-0-139-206 kubenswrapper[2557]: W0706 02:39:15.276111 2557 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 06 02:39:15.516969 ip-10-0-139-206 kubenswrapper[2557]: W0706 02:39:15.276115 2557 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 06 02:39:15.516969 ip-10-0-139-206 kubenswrapper[2557]: W0706 02:39:15.276118 2557 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 06 02:39:15.516969 ip-10-0-139-206 kubenswrapper[2557]: W0706 02:39:15.276121 2557 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 06 02:39:15.516969 ip-10-0-139-206 kubenswrapper[2557]: W0706 02:39:15.276124 2557 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 06 02:39:15.520809 ip-10-0-139-206 kubenswrapper[2557]: W0706 02:39:15.276126 2557 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 06 02:39:15.520809 ip-10-0-139-206 kubenswrapper[2557]: W0706 02:39:15.276129 2557 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 06 02:39:15.520809 ip-10-0-139-206 kubenswrapper[2557]: W0706 02:39:15.276132 2557 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 06 02:39:15.520809 ip-10-0-139-206 kubenswrapper[2557]: W0706 02:39:15.276135 2557 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 06 02:39:15.520809 ip-10-0-139-206 kubenswrapper[2557]: W0706 02:39:15.276137 2557 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 06 02:39:15.520809 ip-10-0-139-206 kubenswrapper[2557]: W0706 02:39:15.276140 2557 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 06 02:39:15.520809 ip-10-0-139-206 kubenswrapper[2557]: W0706 02:39:15.276142 2557 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 06 02:39:15.520809 ip-10-0-139-206 kubenswrapper[2557]: W0706 02:39:15.276145 2557 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 06 02:39:15.520809 ip-10-0-139-206 kubenswrapper[2557]: W0706 02:39:15.276148 2557 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 06 02:39:15.520809 ip-10-0-139-206 kubenswrapper[2557]: W0706 02:39:15.276151 2557 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 06 02:39:15.520809 ip-10-0-139-206 kubenswrapper[2557]: W0706 02:39:15.276154 2557 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 06 02:39:15.520809 ip-10-0-139-206 kubenswrapper[2557]: W0706 02:39:15.276156 2557 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 06 02:39:15.520809 ip-10-0-139-206 kubenswrapper[2557]: W0706 02:39:15.276159 2557 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 06 02:39:15.520809 ip-10-0-139-206 kubenswrapper[2557]: W0706 02:39:15.276161 2557 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 06 02:39:15.520809 ip-10-0-139-206 kubenswrapper[2557]: W0706 02:39:15.276164 2557 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 06 02:39:15.520809 ip-10-0-139-206 kubenswrapper[2557]: W0706 02:39:15.276167 2557 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 06 02:39:15.520809 ip-10-0-139-206 kubenswrapper[2557]: W0706 02:39:15.276169 2557 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 06 02:39:15.520809 ip-10-0-139-206 kubenswrapper[2557]: W0706 02:39:15.276172 2557 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 06 02:39:15.520809 ip-10-0-139-206 kubenswrapper[2557]: W0706 02:39:15.276175 2557 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 06 02:39:15.520809 ip-10-0-139-206 kubenswrapper[2557]: W0706 02:39:15.276179 2557 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 06 02:39:15.521669 ip-10-0-139-206 kubenswrapper[2557]: W0706 02:39:15.276182 2557 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 06 02:39:15.521669 ip-10-0-139-206 kubenswrapper[2557]: W0706 02:39:15.276185 2557 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 06 02:39:15.521669 ip-10-0-139-206 kubenswrapper[2557]: W0706 02:39:15.276187 2557 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 06 02:39:15.521669 ip-10-0-139-206 kubenswrapper[2557]: W0706 02:39:15.276190 2557 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 06 02:39:15.521669 ip-10-0-139-206 kubenswrapper[2557]: W0706 02:39:15.276193 2557 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 06 02:39:15.521669 ip-10-0-139-206 kubenswrapper[2557]: W0706 02:39:15.276195 2557 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 06 02:39:15.521669 ip-10-0-139-206 kubenswrapper[2557]: W0706 02:39:15.276198 2557 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 06 02:39:15.521669 ip-10-0-139-206 kubenswrapper[2557]: W0706 02:39:15.276200 2557 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 06 02:39:15.521669 ip-10-0-139-206 kubenswrapper[2557]: W0706 02:39:15.276203 2557 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 06 02:39:15.521669 ip-10-0-139-206 kubenswrapper[2557]: W0706 02:39:15.276205 2557 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 06 02:39:15.521669 ip-10-0-139-206 kubenswrapper[2557]: W0706 02:39:15.276208 2557 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 06 02:39:15.521669 ip-10-0-139-206 kubenswrapper[2557]: W0706 02:39:15.276210 2557 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 06 02:39:15.521669 ip-10-0-139-206 kubenswrapper[2557]: W0706 02:39:15.276213 2557 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 06 02:39:15.521669 ip-10-0-139-206 kubenswrapper[2557]: W0706 02:39:15.276215 2557 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 06 02:39:15.521669 ip-10-0-139-206 kubenswrapper[2557]: W0706 02:39:15.276218 2557 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 06 02:39:15.521669 ip-10-0-139-206 kubenswrapper[2557]: W0706 02:39:15.276221 2557 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 06 02:39:15.521669 ip-10-0-139-206 kubenswrapper[2557]: W0706 02:39:15.276223 2557 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 06 02:39:15.521669 ip-10-0-139-206 kubenswrapper[2557]: W0706 02:39:15.276230 2557 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 06 02:39:15.521669 ip-10-0-139-206 kubenswrapper[2557]: W0706 02:39:15.276233 2557 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 06 02:39:15.521669 ip-10-0-139-206 kubenswrapper[2557]: W0706 02:39:15.276236 2557 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 06 02:39:15.526840 ip-10-0-139-206 kubenswrapper[2557]: W0706 02:39:15.276238 2557 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 06 02:39:15.526840 ip-10-0-139-206 kubenswrapper[2557]: W0706 02:39:15.276241 2557 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 06 02:39:15.526840 ip-10-0-139-206 kubenswrapper[2557]: W0706 02:39:15.276246 2557 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 06 02:39:15.526840 ip-10-0-139-206 kubenswrapper[2557]: W0706 02:39:15.276250 2557 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 06 02:39:15.526840 ip-10-0-139-206 kubenswrapper[2557]: W0706 02:39:15.276253 2557 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 06 02:39:15.526840 ip-10-0-139-206 kubenswrapper[2557]: W0706 02:39:15.276255 2557 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 06 02:39:15.526840 ip-10-0-139-206 kubenswrapper[2557]: W0706 02:39:15.276258 2557 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 06 02:39:15.526840 ip-10-0-139-206 kubenswrapper[2557]: W0706 02:39:15.276261 2557 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 06 02:39:15.526840 ip-10-0-139-206 kubenswrapper[2557]: W0706 02:39:15.276264 2557 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 06 02:39:15.526840 ip-10-0-139-206 kubenswrapper[2557]: W0706 02:39:15.276267 2557 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 06 02:39:15.526840 ip-10-0-139-206 kubenswrapper[2557]: W0706 02:39:15.276270 2557 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 06 02:39:15.526840 ip-10-0-139-206 kubenswrapper[2557]: W0706 02:39:15.276273 2557 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 06 02:39:15.526840 ip-10-0-139-206 kubenswrapper[2557]: W0706 02:39:15.276276 2557 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 06 02:39:15.526840 ip-10-0-139-206 kubenswrapper[2557]: W0706 02:39:15.276279 2557 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 06 02:39:15.526840 ip-10-0-139-206 kubenswrapper[2557]: W0706 02:39:15.276282 2557 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 06 02:39:15.526840 ip-10-0-139-206 kubenswrapper[2557]: W0706 02:39:15.276285 2557 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 06 02:39:15.526840 ip-10-0-139-206 kubenswrapper[2557]: W0706 02:39:15.276287 2557 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 06 02:39:15.526840 ip-10-0-139-206 kubenswrapper[2557]: W0706 02:39:15.276290 2557 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 06 02:39:15.526840 ip-10-0-139-206 kubenswrapper[2557]: W0706 02:39:15.276292 2557 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 06 02:39:15.527665 ip-10-0-139-206 kubenswrapper[2557]: W0706 02:39:15.276295 2557 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 06 02:39:15.527665 ip-10-0-139-206 kubenswrapper[2557]: W0706 02:39:15.276298 2557 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 06 02:39:15.527665 ip-10-0-139-206 kubenswrapper[2557]: W0706 02:39:15.276300 2557 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 06 02:39:15.527665 ip-10-0-139-206 kubenswrapper[2557]: W0706 02:39:15.276303 2557 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 06 02:39:15.527665 ip-10-0-139-206 kubenswrapper[2557]: W0706 02:39:15.276306 2557 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 06 02:39:15.527665 ip-10-0-139-206 kubenswrapper[2557]: W0706 02:39:15.276308 2557 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 06 02:39:15.527665 ip-10-0-139-206 kubenswrapper[2557]: W0706 02:39:15.276311 2557 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 06 02:39:15.527665 ip-10-0-139-206 kubenswrapper[2557]: W0706 02:39:15.276315 2557 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 06 02:39:15.527665 ip-10-0-139-206 kubenswrapper[2557]: W0706 02:39:15.276318 2557 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 06 02:39:15.527665 ip-10-0-139-206 kubenswrapper[2557]: W0706 02:39:15.276321 2557 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 06 02:39:15.527665 ip-10-0-139-206 kubenswrapper[2557]: W0706 02:39:15.276327 2557 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 06 02:39:15.527665 ip-10-0-139-206 kubenswrapper[2557]: W0706 02:39:15.276331 2557 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 06 02:39:15.527665 ip-10-0-139-206 kubenswrapper[2557]: W0706 02:39:15.276335 2557 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 06 02:39:15.527665 ip-10-0-139-206 kubenswrapper[2557]: W0706 02:39:15.276338 2557 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 06 02:39:15.527665 ip-10-0-139-206 kubenswrapper[2557]: W0706 02:39:15.276341 2557 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 06 02:39:15.527665 ip-10-0-139-206 kubenswrapper[2557]: W0706 02:39:15.276344 2557 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 06 02:39:15.527665 ip-10-0-139-206 kubenswrapper[2557]: W0706 02:39:15.276346 2557 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 06 02:39:15.527665 ip-10-0-139-206 kubenswrapper[2557]: W0706 02:39:15.276349 2557 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 06 02:39:15.527665 ip-10-0-139-206 kubenswrapper[2557]: W0706 02:39:15.276352 2557 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 06 02:39:15.528502 ip-10-0-139-206 kubenswrapper[2557]: W0706 02:39:15.276354 2557 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 06 02:39:15.528502 ip-10-0-139-206 kubenswrapper[2557]: W0706 02:39:15.276356 2557 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 06 02:39:15.528502 ip-10-0-139-206 kubenswrapper[2557]: W0706 02:39:15.276359 2557 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 06 02:39:15.528502 ip-10-0-139-206 kubenswrapper[2557]: W0706 02:39:15.276362 2557 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 06 02:39:15.528502 ip-10-0-139-206 kubenswrapper[2557]: W0706 02:39:15.276364 2557 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 06 02:39:15.528502 ip-10-0-139-206 kubenswrapper[2557]: W0706 02:39:15.276367 2557 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 06 02:39:15.528502 ip-10-0-139-206 kubenswrapper[2557]: W0706 02:39:15.276369 2557 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 06 02:39:15.528502 ip-10-0-139-206 kubenswrapper[2557]: W0706 02:39:15.276372 2557 feature_gate.go:328] unrecognized feature gate: Example2 Jul 06 02:39:15.528502 ip-10-0-139-206 kubenswrapper[2557]: W0706 02:39:15.276375 2557 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 06 02:39:15.528502 ip-10-0-139-206 kubenswrapper[2557]: W0706 02:39:15.276377 2557 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 06 02:39:15.528502 ip-10-0-139-206 kubenswrapper[2557]: W0706 02:39:15.276380 2557 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 06 02:39:15.528502 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:39:15.276385 2557 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 06 02:39:15.528502 ip-10-0-139-206 kubenswrapper[2557]: W0706 02:39:15.276524 2557 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 06 02:39:15.528502 ip-10-0-139-206 kubenswrapper[2557]: W0706 02:39:15.276529 2557 feature_gate.go:328] unrecognized feature gate: Example2 Jul 06 02:39:15.528502 ip-10-0-139-206 kubenswrapper[2557]: W0706 02:39:15.276533 2557 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 06 02:39:15.533224 ip-10-0-139-206 kubenswrapper[2557]: W0706 02:39:15.276537 2557 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 06 02:39:15.533224 ip-10-0-139-206 kubenswrapper[2557]: W0706 02:39:15.276540 2557 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 06 02:39:15.533224 ip-10-0-139-206 kubenswrapper[2557]: W0706 02:39:15.276543 2557 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 06 02:39:15.533224 ip-10-0-139-206 kubenswrapper[2557]: W0706 02:39:15.276546 2557 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 06 02:39:15.533224 ip-10-0-139-206 kubenswrapper[2557]: W0706 02:39:15.276548 2557 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 06 02:39:15.533224 ip-10-0-139-206 kubenswrapper[2557]: W0706 02:39:15.276551 2557 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 06 02:39:15.533224 ip-10-0-139-206 kubenswrapper[2557]: W0706 02:39:15.276553 2557 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 06 02:39:15.533224 ip-10-0-139-206 kubenswrapper[2557]: W0706 02:39:15.276556 2557 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 06 02:39:15.533224 ip-10-0-139-206 kubenswrapper[2557]: W0706 02:39:15.276558 2557 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 06 02:39:15.533224 ip-10-0-139-206 kubenswrapper[2557]: W0706 02:39:15.276562 2557 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 06 02:39:15.533224 ip-10-0-139-206 kubenswrapper[2557]: W0706 02:39:15.276565 2557 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 06 02:39:15.533224 ip-10-0-139-206 kubenswrapper[2557]: W0706 02:39:15.276568 2557 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 06 02:39:15.533224 ip-10-0-139-206 kubenswrapper[2557]: W0706 02:39:15.276570 2557 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 06 02:39:15.533224 ip-10-0-139-206 kubenswrapper[2557]: W0706 02:39:15.276573 2557 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 06 02:39:15.533224 ip-10-0-139-206 kubenswrapper[2557]: W0706 02:39:15.276575 2557 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 06 02:39:15.533224 ip-10-0-139-206 kubenswrapper[2557]: W0706 02:39:15.276578 2557 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 06 02:39:15.533224 ip-10-0-139-206 kubenswrapper[2557]: W0706 02:39:15.276581 2557 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 06 02:39:15.533224 ip-10-0-139-206 kubenswrapper[2557]: W0706 02:39:15.276583 2557 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 06 02:39:15.533224 ip-10-0-139-206 kubenswrapper[2557]: W0706 02:39:15.276586 2557 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 06 02:39:15.533224 ip-10-0-139-206 kubenswrapper[2557]: W0706 02:39:15.276589 2557 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 06 02:39:15.534071 ip-10-0-139-206 kubenswrapper[2557]: W0706 02:39:15.276591 2557 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 06 02:39:15.534071 ip-10-0-139-206 kubenswrapper[2557]: W0706 02:39:15.276594 2557 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 06 02:39:15.534071 ip-10-0-139-206 kubenswrapper[2557]: W0706 02:39:15.276596 2557 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 06 02:39:15.534071 ip-10-0-139-206 kubenswrapper[2557]: W0706 02:39:15.276599 2557 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 06 02:39:15.534071 ip-10-0-139-206 kubenswrapper[2557]: W0706 02:39:15.276601 2557 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 06 02:39:15.534071 ip-10-0-139-206 kubenswrapper[2557]: W0706 02:39:15.276604 2557 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 06 02:39:15.534071 ip-10-0-139-206 kubenswrapper[2557]: W0706 02:39:15.276607 2557 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 06 02:39:15.534071 ip-10-0-139-206 kubenswrapper[2557]: W0706 02:39:15.276610 2557 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 06 02:39:15.534071 ip-10-0-139-206 kubenswrapper[2557]: W0706 02:39:15.276612 2557 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 06 02:39:15.534071 ip-10-0-139-206 kubenswrapper[2557]: W0706 02:39:15.276615 2557 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 06 02:39:15.534071 ip-10-0-139-206 kubenswrapper[2557]: W0706 02:39:15.276617 2557 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 06 02:39:15.534071 ip-10-0-139-206 kubenswrapper[2557]: W0706 02:39:15.276619 2557 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 06 02:39:15.534071 ip-10-0-139-206 kubenswrapper[2557]: W0706 02:39:15.276622 2557 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 06 02:39:15.534071 ip-10-0-139-206 kubenswrapper[2557]: W0706 02:39:15.276624 2557 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 06 02:39:15.534071 ip-10-0-139-206 kubenswrapper[2557]: W0706 02:39:15.276627 2557 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 06 02:39:15.534071 ip-10-0-139-206 kubenswrapper[2557]: W0706 02:39:15.276648 2557 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 06 02:39:15.534071 ip-10-0-139-206 kubenswrapper[2557]: W0706 02:39:15.276652 2557 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 06 02:39:15.534071 ip-10-0-139-206 kubenswrapper[2557]: W0706 02:39:15.276655 2557 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 06 02:39:15.534071 ip-10-0-139-206 kubenswrapper[2557]: W0706 02:39:15.276657 2557 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 06 02:39:15.534071 ip-10-0-139-206 kubenswrapper[2557]: W0706 02:39:15.276660 2557 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 06 02:39:15.534964 ip-10-0-139-206 kubenswrapper[2557]: W0706 02:39:15.276663 2557 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 06 02:39:15.534964 ip-10-0-139-206 kubenswrapper[2557]: W0706 02:39:15.276667 2557 feature_gate.go:328] unrecognized feature gate: Example Jul 06 02:39:15.534964 ip-10-0-139-206 kubenswrapper[2557]: W0706 02:39:15.276672 2557 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 06 02:39:15.534964 ip-10-0-139-206 kubenswrapper[2557]: W0706 02:39:15.276674 2557 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 06 02:39:15.534964 ip-10-0-139-206 kubenswrapper[2557]: W0706 02:39:15.276677 2557 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 06 02:39:15.534964 ip-10-0-139-206 kubenswrapper[2557]: W0706 02:39:15.276680 2557 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 06 02:39:15.534964 ip-10-0-139-206 kubenswrapper[2557]: W0706 02:39:15.276682 2557 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 06 02:39:15.534964 ip-10-0-139-206 kubenswrapper[2557]: W0706 02:39:15.276685 2557 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 06 02:39:15.534964 ip-10-0-139-206 kubenswrapper[2557]: W0706 02:39:15.276688 2557 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 06 02:39:15.534964 ip-10-0-139-206 kubenswrapper[2557]: W0706 02:39:15.276691 2557 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 06 02:39:15.534964 ip-10-0-139-206 kubenswrapper[2557]: W0706 02:39:15.276693 2557 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 06 02:39:15.534964 ip-10-0-139-206 kubenswrapper[2557]: W0706 02:39:15.276696 2557 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 06 02:39:15.534964 ip-10-0-139-206 kubenswrapper[2557]: W0706 02:39:15.276698 2557 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 06 02:39:15.534964 ip-10-0-139-206 kubenswrapper[2557]: W0706 02:39:15.276701 2557 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 06 02:39:15.534964 ip-10-0-139-206 kubenswrapper[2557]: W0706 02:39:15.276703 2557 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 06 02:39:15.534964 ip-10-0-139-206 kubenswrapper[2557]: W0706 02:39:15.276706 2557 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 06 02:39:15.534964 ip-10-0-139-206 kubenswrapper[2557]: W0706 02:39:15.276710 2557 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 06 02:39:15.534964 ip-10-0-139-206 kubenswrapper[2557]: W0706 02:39:15.276713 2557 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 06 02:39:15.534964 ip-10-0-139-206 kubenswrapper[2557]: W0706 02:39:15.276721 2557 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 06 02:39:15.534964 ip-10-0-139-206 kubenswrapper[2557]: W0706 02:39:15.276724 2557 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 06 02:39:15.642966 ip-10-0-139-206 kubenswrapper[2557]: W0706 02:39:15.276727 2557 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 06 02:39:15.642966 ip-10-0-139-206 kubenswrapper[2557]: W0706 02:39:15.276730 2557 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 06 02:39:15.642966 ip-10-0-139-206 kubenswrapper[2557]: W0706 02:39:15.276733 2557 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 06 02:39:15.642966 ip-10-0-139-206 kubenswrapper[2557]: W0706 02:39:15.276735 2557 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 06 02:39:15.642966 ip-10-0-139-206 kubenswrapper[2557]: W0706 02:39:15.276738 2557 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 06 02:39:15.642966 ip-10-0-139-206 kubenswrapper[2557]: W0706 02:39:15.276741 2557 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 06 02:39:15.642966 ip-10-0-139-206 kubenswrapper[2557]: W0706 02:39:15.276743 2557 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 06 02:39:15.642966 ip-10-0-139-206 kubenswrapper[2557]: W0706 02:39:15.276746 2557 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 06 02:39:15.642966 ip-10-0-139-206 kubenswrapper[2557]: W0706 02:39:15.276748 2557 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 06 02:39:15.642966 ip-10-0-139-206 kubenswrapper[2557]: W0706 02:39:15.276751 2557 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 06 02:39:15.642966 ip-10-0-139-206 kubenswrapper[2557]: W0706 02:39:15.276753 2557 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 06 02:39:15.642966 ip-10-0-139-206 kubenswrapper[2557]: W0706 02:39:15.276756 2557 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 06 02:39:15.642966 ip-10-0-139-206 kubenswrapper[2557]: W0706 02:39:15.276759 2557 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 06 02:39:15.642966 ip-10-0-139-206 kubenswrapper[2557]: W0706 02:39:15.276763 2557 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 06 02:39:15.642966 ip-10-0-139-206 kubenswrapper[2557]: W0706 02:39:15.276767 2557 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 06 02:39:15.642966 ip-10-0-139-206 kubenswrapper[2557]: W0706 02:39:15.276770 2557 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 06 02:39:15.642966 ip-10-0-139-206 kubenswrapper[2557]: W0706 02:39:15.276772 2557 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 06 02:39:15.642966 ip-10-0-139-206 kubenswrapper[2557]: W0706 02:39:15.276775 2557 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 06 02:39:15.642966 ip-10-0-139-206 kubenswrapper[2557]: W0706 02:39:15.276777 2557 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 06 02:39:15.643829 ip-10-0-139-206 kubenswrapper[2557]: W0706 02:39:15.276780 2557 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 06 02:39:15.643829 ip-10-0-139-206 kubenswrapper[2557]: W0706 02:39:15.276783 2557 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 06 02:39:15.643829 ip-10-0-139-206 kubenswrapper[2557]: W0706 02:39:15.276785 2557 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 06 02:39:15.643829 ip-10-0-139-206 kubenswrapper[2557]: W0706 02:39:15.276788 2557 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 06 02:39:15.643829 ip-10-0-139-206 kubenswrapper[2557]: W0706 02:39:15.276791 2557 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 06 02:39:15.643829 ip-10-0-139-206 kubenswrapper[2557]: W0706 02:39:15.276793 2557 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 06 02:39:15.643829 ip-10-0-139-206 kubenswrapper[2557]: W0706 02:39:15.276796 2557 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 06 02:39:15.643829 ip-10-0-139-206 kubenswrapper[2557]: W0706 02:39:15.276798 2557 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 06 02:39:15.643829 ip-10-0-139-206 kubenswrapper[2557]: W0706 02:39:15.276801 2557 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 06 02:39:15.643829 ip-10-0-139-206 kubenswrapper[2557]: W0706 02:39:15.276803 2557 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 06 02:39:15.643829 ip-10-0-139-206 kubenswrapper[2557]: W0706 02:39:15.276806 2557 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 06 02:39:15.643829 ip-10-0-139-206 kubenswrapper[2557]: W0706 02:39:15.276809 2557 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 06 02:39:15.643829 ip-10-0-139-206 kubenswrapper[2557]: W0706 02:39:15.276812 2557 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 06 02:39:15.643829 ip-10-0-139-206 kubenswrapper[2557]: W0706 02:39:15.276814 2557 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 06 02:39:15.643829 ip-10-0-139-206 kubenswrapper[2557]: W0706 02:39:15.276817 2557 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 06 02:39:15.643829 ip-10-0-139-206 kubenswrapper[2557]: W0706 02:39:15.276819 2557 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 06 02:39:15.643829 ip-10-0-139-206 kubenswrapper[2557]: W0706 02:39:15.276822 2557 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 06 02:39:15.647317 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:39:15.276826 2557 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 06 02:39:15.647317 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:39:15.276971 2557 server.go:962] "Client rotation is on, will bootstrap in background" Jul 06 02:39:15.647317 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:39:15.331253 2557 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir" Jul 06 02:39:15.647317 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:39:15.332921 2557 server.go:1019] "Starting client certificate rotation" Jul 06 02:39:15.647317 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:39:15.333032 2557 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet" Jul 06 02:39:15.647317 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:39:15.333109 2557 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet" Jul 06 02:39:15.647317 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:39:15.336073 2557 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Jul 06 02:39:15.647317 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:39:15.338931 2557 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Jul 06 02:39:17.032396 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:39:17.032374 2557 log.go:25] "Validated CRI v1 runtime API" Jul 06 02:39:17.034839 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:39:17.034821 2557 log.go:25] "Validated CRI v1 image API" Jul 06 02:39:17.035839 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:39:17.035818 2557 server.go:1429] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd" Jul 06 02:39:17.041675 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:39:17.041639 2557 fs.go:135] Filesystem UUIDs: map[7B77-95E7:/dev/nvme0n1p2 c4fd35d5-947d-49a3-87b4-716c5d170d77:/dev/nvme0n1p3 f1ecafb7-1cc4-425e-b7a4-7e895d412026:/dev/nvme0n1p4] Jul 06 02:39:17.041796 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:39:17.041676 2557 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}] Jul 06 02:39:17.107079 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:39:17.106932 2557 manager.go:216] Machine: {Timestamp:2026-07-06 02:39:17.105616387 +0000 UTC m=+0.911902435 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3099300 MemoryCapacity:33164500992 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec2c5f948a494cbf3a556f68267647e4 SystemUUID:ec2c5f94-8a49-4cbf-3a55-6f68267647e4 BootID:a876fb16-23d0-4739-9e23-f3a93697e0f9 Filesystems:[{Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6107136 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16582250496 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true} {Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16582250496 Type:vfs Inodes:4048401 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6632902656 Type:vfs Inodes:819200 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:d2:7d:22:f0:61 Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:d2:7d:22:f0:61 Speed:0 Mtu:9001} {Name:ovs-system MacAddress:c6:d0:cb:22:a3:ce Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:33164500992 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:37486592 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None} Jul 06 02:39:17.107079 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:39:17.107058 2557 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available. Jul 06 02:39:17.107290 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:39:17.107155 2557 manager.go:232] Version: {KernelVersion:5.14.0-570.125.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260630-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:} Jul 06 02:39:17.108319 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:39:17.108281 2557 container_manager_linux.go:273] "Container manager verified user specified cgroup-root exists" cgroupRoot=[] Jul 06 02:39:17.108548 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:39:17.108322 2557 container_manager_linux.go:278] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-139-206.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2} Jul 06 02:39:17.109113 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:39:17.109101 2557 topology_manager.go:138] "Creating topology manager with none policy" Jul 06 02:39:17.109164 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:39:17.109119 2557 container_manager_linux.go:309] "Creating device plugin manager" Jul 06 02:39:17.109164 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:39:17.109130 2557 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock" Jul 06 02:39:17.109783 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:39:17.109769 2557 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock" Jul 06 02:39:17.113319 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:39:17.113302 2557 container_manager_linux.go:318] "Creating Dynamic Resource Allocation (DRA) manager" Jul 06 02:39:17.117336 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:39:17.117320 2557 state_mem.go:36] "Initialized new in-memory state store" Jul 06 02:39:17.117505 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:39:17.117494 2557 server.go:1240] "Using root directory" path="/var/lib/kubelet" Jul 06 02:39:17.118161 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:39:17.118151 2557 kubelet.go:486] "Attempting to sync node with API server" Jul 06 02:39:17.118229 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:39:17.118169 2557 kubelet.go:387] "Adding static pod path" path="/etc/kubernetes/manifests" Jul 06 02:39:17.118229 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:39:17.118189 2557 file.go:69] "Watching path" path="/etc/kubernetes/manifests" Jul 06 02:39:17.118229 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:39:17.118204 2557 kubelet.go:398] "Adding apiserver pod source" Jul 06 02:39:17.118229 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:39:17.118216 2557 apiserver.go:42] "Waiting for node sync before watching apiserver pods" Jul 06 02:39:17.118646 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:39:17.118631 2557 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Jul 06 02:39:17.118710 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:39:17.118655 2557 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Jul 06 02:39:17.121067 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:39:17.121034 2557 kuberuntime_manager.go:301] "Container runtime initialized" containerRuntime="cri-o" version="1.34.8-3.rhaos4.21.git91b4b0b.el9" apiVersion="v1" Jul 06 02:39:17.123318 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:39:17.123304 2557 kubelet.go:960] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled" Jul 06 02:39:17.123375 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:39:17.123336 2557 kubelet.go:984] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled" Jul 06 02:39:17.124136 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:39:17.124122 2557 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume" Jul 06 02:39:17.124193 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:39:17.124139 2557 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir" Jul 06 02:39:17.124193 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:39:17.124145 2557 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo" Jul 06 02:39:17.124193 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:39:17.124151 2557 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path" Jul 06 02:39:17.124193 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:39:17.124164 2557 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs" Jul 06 02:39:17.124193 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:39:17.124172 2557 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret" Jul 06 02:39:17.124193 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:39:17.124184 2557 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi" Jul 06 02:39:17.124193 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:39:17.124192 2557 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api" Jul 06 02:39:17.124440 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:39:17.124202 2557 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc" Jul 06 02:39:17.124440 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:39:17.124210 2557 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap" Jul 06 02:39:17.124440 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:39:17.124222 2557 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected" Jul 06 02:39:17.124440 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:39:17.124229 2557 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume" Jul 06 02:39:17.124982 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:39:17.124970 2557 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi" Jul 06 02:39:17.124982 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:39:17.124982 2557 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image" Jul 06 02:39:17.125493 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:39:17.125475 2557 reflector.go:439] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Jul 06 02:39:17.128253 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:39:17.128238 2557 server.go:1268] "Started kubelet" Jul 06 02:39:17.128437 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:39:17.128406 2557 server.go:180] "Starting to listen" address="0.0.0.0" port=10250 Jul 06 02:39:17.128521 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:39:17.128452 2557 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10 Jul 06 02:39:17.128521 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:39:17.128519 2557 server_v1.go:49] "podresources" method="list" useActivePods=true Jul 06 02:39:17.129091 ip-10-0-139-206 systemd[1]: Started Kubernetes Kubelet. Jul 06 02:39:17.130496 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:39:17.130478 2557 server.go:249] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock" Jul 06 02:39:17.131398 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:39:17.131367 2557 server.go:310] "Adding debug handlers to kubelet server" Jul 06 02:39:17.132908 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:39:17.132887 2557 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-139-206.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope Jul 06 02:39:17.132989 ip-10-0-139-206 kubenswrapper[2557]: E0706 02:39:17.132935 2557 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-139-206.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node" Jul 06 02:39:17.133078 ip-10-0-139-206 kubenswrapper[2557]: E0706 02:39:17.133055 2557 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service" Jul 06 02:39:17.133926 ip-10-0-139-206 kubenswrapper[2557]: E0706 02:39:17.133907 2557 kubelet.go:1638] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache" Jul 06 02:39:17.136445 ip-10-0-139-206 kubenswrapper[2557]: E0706 02:39:17.135324 2557 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-139-206.ec2.internal.18bf92f4b29b6574 default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-139-206.ec2.internal,UID:ip-10-0-139-206.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-139-206.ec2.internal,},FirstTimestamp:2026-07-06 02:39:17.12820978 +0000 UTC m=+0.934495831,LastTimestamp:2026-07-06 02:39:17.12820978 +0000 UTC m=+0.934495831,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-139-206.ec2.internal,}" Jul 06 02:39:17.138950 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:39:17.138905 2557 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer" Jul 06 02:39:17.138950 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:39:17.138929 2557 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving" Jul 06 02:39:17.139075 ip-10-0-139-206 kubenswrapper[2557]: E0706 02:39:17.138979 2557 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-139-206.ec2.internal\" not found" Jul 06 02:39:17.139581 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:39:17.139564 2557 desired_state_of_world_populator.go:146] "Desired state populator starts to run" Jul 06 02:39:17.139581 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:39:17.139565 2557 volume_manager.go:311] "The desired_state_of_world populator starts" Jul 06 02:39:17.139735 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:39:17.139596 2557 volume_manager.go:313] "Starting Kubelet Volume Manager" Jul 06 02:39:17.140278 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:39:17.140259 2557 reconstruct.go:97] "Volume reconstruction finished" Jul 06 02:39:17.140338 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:39:17.140279 2557 reconciler.go:29] "Reconciler: start to sync state" Jul 06 02:39:17.140660 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:39:17.140639 2557 factory.go:55] Registering systemd factory Jul 06 02:39:17.140733 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:39:17.140672 2557 factory.go:223] Registration of the systemd container factory successfully Jul 06 02:39:17.141118 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:39:17.141102 2557 factory.go:153] Registering CRI-O factory Jul 06 02:39:17.142708 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:39:17.142686 2557 factory.go:223] Registration of the crio container factory successfully Jul 06 02:39:17.142784 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:39:17.142767 2557 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory Jul 06 02:39:17.142855 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:39:17.142792 2557 factory.go:103] Registering Raw factory Jul 06 02:39:17.142855 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:39:17.142811 2557 manager.go:1196] Started watching for new ooms in manager Jul 06 02:39:17.143803 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:39:17.143766 2557 manager.go:349] Starting recovery of all containers Jul 06 02:39:17.145187 ip-10-0-139-206 kubenswrapper[2557]: E0706 02:39:17.145148 2557 reflector.go:205] "Failed to watch" err="failed to list *v1.CSIDriver: csidrivers.storage.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"csidrivers\" in API group \"storage.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver" Jul 06 02:39:17.145187 ip-10-0-139-206 kubenswrapper[2557]: E0706 02:39:17.145177 2557 controller.go:145] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-139-206.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="200ms" Jul 06 02:39:17.149118 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:39:17.149091 2557 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-4vdsh" Jul 06 02:39:17.154482 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:39:17.154460 2557 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-4vdsh" Jul 06 02:39:17.154715 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:39:17.154700 2557 manager.go:354] Recovery completed Jul 06 02:39:17.158934 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:39:17.158921 2557 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 06 02:39:17.161064 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:39:17.161049 2557 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-206.ec2.internal" event="NodeHasSufficientMemory" Jul 06 02:39:17.161125 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:39:17.161080 2557 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-206.ec2.internal" event="NodeHasNoDiskPressure" Jul 06 02:39:17.161125 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:39:17.161090 2557 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-206.ec2.internal" event="NodeHasSufficientPID" Jul 06 02:39:17.161637 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:39:17.161621 2557 cpu_manager.go:222] "Starting CPU manager" policy="none" Jul 06 02:39:17.161637 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:39:17.161635 2557 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s" Jul 06 02:39:17.161768 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:39:17.161650 2557 state_mem.go:36] "Initialized new in-memory state store" Jul 06 02:39:17.161768 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:39:17.161651 2557 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 06 02:39:17.162443 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:39:17.162427 2557 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-206.ec2.internal" event="NodeHasSufficientMemory" Jul 06 02:39:17.162511 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:39:17.162456 2557 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-206.ec2.internal" event="NodeHasNoDiskPressure" Jul 06 02:39:17.162511 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:39:17.162466 2557 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-206.ec2.internal" event="NodeHasSufficientPID" Jul 06 02:39:17.162925 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:39:17.162912 2557 policy_none.go:49] "None policy: Start" Jul 06 02:39:17.162958 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:39:17.162930 2557 memory_manager.go:187] "Starting memorymanager" policy="None" Jul 06 02:39:17.162958 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:39:17.162940 2557 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint" Jul 06 02:39:17.163665 ip-10-0-139-206 kubenswrapper[2557]: E0706 02:39:17.163593 2557 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-139-206.ec2.internal.18bf92f4b490b766 default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-139-206.ec2.internal,UID:ip-10-0-139-206.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientMemory,Message:Node ip-10-0-139-206.ec2.internal status is now: NodeHasSufficientMemory,Source:EventSource{Component:kubelet,Host:ip-10-0-139-206.ec2.internal,},FirstTimestamp:2026-07-06 02:39:17.161064294 +0000 UTC m=+0.967350343,LastTimestamp:2026-07-06 02:39:17.161064294 +0000 UTC m=+0.967350343,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-139-206.ec2.internal,}" Jul 06 02:39:17.164053 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:39:17.164042 2557 policy_none.go:47] "Start" Jul 06 02:39:17.205282 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:39:17.205055 2557 manager.go:337] "Starting Device Plugin manager" Jul 06 02:39:17.216777 ip-10-0-139-206 kubenswrapper[2557]: E0706 02:39:17.205288 2557 manager.go:513] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint" Jul 06 02:39:17.216777 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:39:17.205304 2557 server.go:87] "Starting device plugin registration server" Jul 06 02:39:17.216777 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:39:17.205589 2557 eviction_manager.go:189] "Eviction manager: starting control loop" Jul 06 02:39:17.216777 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:39:17.205600 2557 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s" Jul 06 02:39:17.216777 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:39:17.205709 2557 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry" Jul 06 02:39:17.216777 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:39:17.205798 2557 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts" Jul 06 02:39:17.216777 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:39:17.205807 2557 plugin_manager.go:118] "Starting Kubelet Plugin Manager" Jul 06 02:39:17.216777 ip-10-0-139-206 kubenswrapper[2557]: E0706 02:39:17.206366 2557 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\"" Jul 06 02:39:17.216777 ip-10-0-139-206 kubenswrapper[2557]: E0706 02:39:17.206442 2557 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-139-206.ec2.internal\" not found" Jul 06 02:39:17.294244 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:39:17.294208 2557 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4" Jul 06 02:39:17.295491 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:39:17.295472 2557 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6" Jul 06 02:39:17.295543 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:39:17.295522 2557 status_manager.go:244] "Starting to sync pod status with apiserver" Jul 06 02:39:17.295585 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:39:17.295550 2557 kubelet.go:2448] "Starting kubelet main sync loop" Jul 06 02:39:17.295630 ip-10-0-139-206 kubenswrapper[2557]: E0706 02:39:17.295594 2557 kubelet.go:2472] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful" Jul 06 02:39:17.297716 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:39:17.297686 2557 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Jul 06 02:39:17.306475 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:39:17.306455 2557 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 06 02:39:17.308429 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:39:17.308404 2557 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-206.ec2.internal" event="NodeHasSufficientMemory" Jul 06 02:39:17.308544 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:39:17.308436 2557 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-206.ec2.internal" event="NodeHasNoDiskPressure" Jul 06 02:39:17.308544 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:39:17.308452 2557 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-206.ec2.internal" event="NodeHasSufficientPID" Jul 06 02:39:17.308544 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:39:17.308480 2557 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-139-206.ec2.internal" Jul 06 02:39:17.318342 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:39:17.318318 2557 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-139-206.ec2.internal" Jul 06 02:39:17.318464 ip-10-0-139-206 kubenswrapper[2557]: E0706 02:39:17.318363 2557 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-139-206.ec2.internal\": node \"ip-10-0-139-206.ec2.internal\" not found" Jul 06 02:39:17.332297 ip-10-0-139-206 kubenswrapper[2557]: E0706 02:39:17.332270 2557 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-139-206.ec2.internal\" not found" Jul 06 02:39:17.395980 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:39:17.395890 2557 kubelet.go:2535] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-206.ec2.internal","kube-system/kube-apiserver-proxy-ip-10-0-139-206.ec2.internal"] Jul 06 02:39:17.396110 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:39:17.395995 2557 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 06 02:39:17.396995 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:39:17.396971 2557 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-206.ec2.internal" event="NodeHasSufficientMemory" Jul 06 02:39:17.397110 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:39:17.397010 2557 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-206.ec2.internal" event="NodeHasNoDiskPressure" Jul 06 02:39:17.397110 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:39:17.397023 2557 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-206.ec2.internal" event="NodeHasSufficientPID" Jul 06 02:39:17.398306 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:39:17.398291 2557 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 06 02:39:17.398434 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:39:17.398418 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-206.ec2.internal" Jul 06 02:39:17.398482 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:39:17.398450 2557 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 06 02:39:17.399117 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:39:17.399099 2557 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-206.ec2.internal" event="NodeHasSufficientMemory" Jul 06 02:39:17.399200 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:39:17.399130 2557 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-206.ec2.internal" event="NodeHasNoDiskPressure" Jul 06 02:39:17.399200 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:39:17.399143 2557 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-206.ec2.internal" event="NodeHasSufficientPID" Jul 06 02:39:17.400468 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:39:17.400452 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-139-206.ec2.internal" Jul 06 02:39:17.400555 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:39:17.400484 2557 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 06 02:39:17.400844 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:39:17.400828 2557 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-206.ec2.internal" event="NodeHasSufficientMemory" Jul 06 02:39:17.400915 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:39:17.400857 2557 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-206.ec2.internal" event="NodeHasNoDiskPressure" Jul 06 02:39:17.400915 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:39:17.400868 2557 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-206.ec2.internal" event="NodeHasSufficientPID" Jul 06 02:39:17.401198 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:39:17.401184 2557 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-206.ec2.internal" event="NodeHasSufficientMemory" Jul 06 02:39:17.401262 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:39:17.401207 2557 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-206.ec2.internal" event="NodeHasNoDiskPressure" Jul 06 02:39:17.401262 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:39:17.401216 2557 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-206.ec2.internal" event="NodeHasSufficientPID" Jul 06 02:39:17.432718 ip-10-0-139-206 kubenswrapper[2557]: E0706 02:39:17.432689 2557 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-139-206.ec2.internal\" not found" Jul 06 02:39:17.437229 ip-10-0-139-206 kubenswrapper[2557]: E0706 02:39:17.437212 2557 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-139-206.ec2.internal\" not found" node="ip-10-0-139-206.ec2.internal" Jul 06 02:39:17.441863 ip-10-0-139-206 kubenswrapper[2557]: E0706 02:39:17.441845 2557 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-139-206.ec2.internal\" not found" node="ip-10-0-139-206.ec2.internal" Jul 06 02:39:17.442888 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:39:17.442872 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/2e5d7d54b6363f2418e74312edd961df-config\") pod \"kube-apiserver-proxy-ip-10-0-139-206.ec2.internal\" (UID: \"2e5d7d54b6363f2418e74312edd961df\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-139-206.ec2.internal" Jul 06 02:39:17.442937 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:39:17.442902 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/d4d0c49eb31a9149fde1244abdabe200-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-139-206.ec2.internal\" (UID: \"d4d0c49eb31a9149fde1244abdabe200\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-206.ec2.internal" Jul 06 02:39:17.442937 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:39:17.442923 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/d4d0c49eb31a9149fde1244abdabe200-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-139-206.ec2.internal\" (UID: \"d4d0c49eb31a9149fde1244abdabe200\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-206.ec2.internal" Jul 06 02:39:17.533731 ip-10-0-139-206 kubenswrapper[2557]: E0706 02:39:17.533677 2557 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-139-206.ec2.internal\" not found" Jul 06 02:39:17.544037 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:39:17.544012 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/d4d0c49eb31a9149fde1244abdabe200-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-139-206.ec2.internal\" (UID: \"d4d0c49eb31a9149fde1244abdabe200\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-206.ec2.internal" Jul 06 02:39:17.544184 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:39:17.544044 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/2e5d7d54b6363f2418e74312edd961df-config\") pod \"kube-apiserver-proxy-ip-10-0-139-206.ec2.internal\" (UID: \"2e5d7d54b6363f2418e74312edd961df\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-139-206.ec2.internal" Jul 06 02:39:17.544184 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:39:17.544063 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/d4d0c49eb31a9149fde1244abdabe200-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-139-206.ec2.internal\" (UID: \"d4d0c49eb31a9149fde1244abdabe200\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-206.ec2.internal" Jul 06 02:39:17.544184 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:39:17.544107 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/d4d0c49eb31a9149fde1244abdabe200-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-139-206.ec2.internal\" (UID: \"d4d0c49eb31a9149fde1244abdabe200\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-206.ec2.internal" Jul 06 02:39:17.544184 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:39:17.544133 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/d4d0c49eb31a9149fde1244abdabe200-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-139-206.ec2.internal\" (UID: \"d4d0c49eb31a9149fde1244abdabe200\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-206.ec2.internal" Jul 06 02:39:17.544184 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:39:17.544146 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/2e5d7d54b6363f2418e74312edd961df-config\") pod \"kube-apiserver-proxy-ip-10-0-139-206.ec2.internal\" (UID: \"2e5d7d54b6363f2418e74312edd961df\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-139-206.ec2.internal" Jul 06 02:39:17.578185 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:39:17.578155 2557 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials" Jul 06 02:39:17.578405 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:39:17.578385 2557 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received" Jul 06 02:39:17.634687 ip-10-0-139-206 kubenswrapper[2557]: E0706 02:39:17.634638 2557 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-139-206.ec2.internal\" not found" Jul 06 02:39:17.735208 ip-10-0-139-206 kubenswrapper[2557]: E0706 02:39:17.735124 2557 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-139-206.ec2.internal\" not found" Jul 06 02:39:17.741468 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:39:17.741446 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-206.ec2.internal" Jul 06 02:39:17.746594 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:39:17.746578 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-139-206.ec2.internal" Jul 06 02:39:17.835929 ip-10-0-139-206 kubenswrapper[2557]: E0706 02:39:17.835888 2557 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-139-206.ec2.internal\" not found" Jul 06 02:39:17.936421 ip-10-0-139-206 kubenswrapper[2557]: E0706 02:39:17.936388 2557 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-139-206.ec2.internal\" not found" Jul 06 02:39:18.037084 ip-10-0-139-206 kubenswrapper[2557]: E0706 02:39:18.036991 2557 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-139-206.ec2.internal\" not found" Jul 06 02:39:18.137548 ip-10-0-139-206 kubenswrapper[2557]: E0706 02:39:18.137521 2557 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-139-206.ec2.internal\" not found" Jul 06 02:39:18.156086 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:39:18.156052 2557 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-07-05 02:34:17 +0000 UTC" deadline="2028-03-21 17:55:06.772282612 +0000 UTC" Jul 06 02:39:18.156086 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:39:18.156080 2557 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="14991h15m48.616205906s" Jul 06 02:39:18.238446 ip-10-0-139-206 kubenswrapper[2557]: E0706 02:39:18.238416 2557 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-139-206.ec2.internal\" not found" Jul 06 02:39:18.249246 ip-10-0-139-206 kubenswrapper[2557]: W0706 02:39:18.249215 2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2e5d7d54b6363f2418e74312edd961df.slice/crio-fdce852115230f8ac85c8a17ffb757f19ddb9aa6a232b75a089c8693fa9b7bc3 WatchSource:0}: Error finding container fdce852115230f8ac85c8a17ffb757f19ddb9aa6a232b75a089c8693fa9b7bc3: Status 404 returned error can't find the container with id fdce852115230f8ac85c8a17ffb757f19ddb9aa6a232b75a089c8693fa9b7bc3 Jul 06 02:39:18.249550 ip-10-0-139-206 kubenswrapper[2557]: W0706 02:39:18.249530 2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd4d0c49eb31a9149fde1244abdabe200.slice/crio-7123f569cad51921c41d230daf33ed1ad660cd027f43a4bc298e8c041b0d5c97 WatchSource:0}: Error finding container 7123f569cad51921c41d230daf33ed1ad660cd027f43a4bc298e8c041b0d5c97: Status 404 returned error can't find the container with id 7123f569cad51921c41d230daf33ed1ad660cd027f43a4bc298e8c041b0d5c97 Jul 06 02:39:18.256838 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:39:18.256821 2557 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 06 02:39:18.299558 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:39:18.299454 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-206.ec2.internal" event={"ID":"d4d0c49eb31a9149fde1244abdabe200","Type":"ContainerStarted","Data":"7123f569cad51921c41d230daf33ed1ad660cd027f43a4bc298e8c041b0d5c97"} Jul 06 02:39:18.300400 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:39:18.300377 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-139-206.ec2.internal" event={"ID":"2e5d7d54b6363f2418e74312edd961df","Type":"ContainerStarted","Data":"fdce852115230f8ac85c8a17ffb757f19ddb9aa6a232b75a089c8693fa9b7bc3"} Jul 06 02:39:18.338529 ip-10-0-139-206 kubenswrapper[2557]: E0706 02:39:18.338488 2557 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-139-206.ec2.internal\" not found" Jul 06 02:39:18.439008 ip-10-0-139-206 kubenswrapper[2557]: E0706 02:39:18.438965 2557 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-139-206.ec2.internal\" not found" Jul 06 02:39:18.487681 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:39:18.487646 2557 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160" Jul 06 02:39:18.539521 ip-10-0-139-206 kubenswrapper[2557]: E0706 02:39:18.539485 2557 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-139-206.ec2.internal\" not found" Jul 06 02:39:18.600245 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:39:18.600157 2557 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Jul 06 02:39:18.606358 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:39:18.606323 2557 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160" Jul 06 02:39:18.639418 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:39:18.639375 2557 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-206.ec2.internal" Jul 06 02:39:18.650510 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:39:18.650479 2557 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Jul 06 02:39:18.652493 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:39:18.652463 2557 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-139-206.ec2.internal" Jul 06 02:39:18.660733 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:39:18.660709 2557 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Jul 06 02:39:18.665136 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:39:18.665113 2557 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160" Jul 06 02:39:19.120049 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:39:19.119927 2557 apiserver.go:52] "Watching apiserver" Jul 06 02:39:19.127503 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:39:19.127450 2557 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66" Jul 06 02:39:19.127992 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:39:19.127961 2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kube-system/kube-apiserver-proxy-ip-10-0-139-206.ec2.internal","openshift-cluster-node-tuning-operator/tuned-fxqsb","openshift-dns/node-resolver-bhp46","openshift-image-registry/node-ca-lwmh8","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-206.ec2.internal","openshift-multus/multus-additional-cni-plugins-7nlb2","openshift-multus/multus-v4pnk","openshift-network-operator/iptables-alerter-hgnzv","kube-system/global-pull-secret-syncer-xjlbh","kube-system/konnectivity-agent-twqqj","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-84mq4","openshift-multus/network-metrics-daemon-756n2","openshift-network-diagnostics/network-check-target-l2btb","openshift-ovn-kubernetes/ovnkube-node-cw2f7"] Jul 06 02:39:19.130024 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:39:19.129650 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-v4pnk" Jul 06 02:39:19.130768 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:39:19.130745 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-84mq4" Jul 06 02:39:19.132134 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:39:19.132099 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-bhp46" Jul 06 02:39:19.132372 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:39:19.132338 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\"" Jul 06 02:39:19.132633 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:39:19.132613 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-x8rxk\"" Jul 06 02:39:19.132633 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:39:19.132624 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\"" Jul 06 02:39:19.132783 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:39:19.132671 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\"" Jul 06 02:39:19.132926 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:39:19.132908 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\"" Jul 06 02:39:19.133308 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:39:19.133283 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\"" Jul 06 02:39:19.133441 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:39:19.133407 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-lwmh8" Jul 06 02:39:19.133964 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:39:19.133946 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\"" Jul 06 02:39:19.134063 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:39:19.133992 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-4s8pz\"" Jul 06 02:39:19.134117 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:39:19.134077 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\"" Jul 06 02:39:19.134297 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:39:19.134279 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\"" Jul 06 02:39:19.134384 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:39:19.134281 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-ht6cl\"" Jul 06 02:39:19.134921 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:39:19.134725 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\"" Jul 06 02:39:19.134921 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:39:19.134819 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-cw2f7" Jul 06 02:39:19.136439 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:39:19.136155 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-7nlb2" Jul 06 02:39:19.136540 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:39:19.136508 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-2dwl6\"" Jul 06 02:39:19.137052 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:39:19.136739 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\"" Jul 06 02:39:19.137052 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:39:19.136772 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\"" Jul 06 02:39:19.137052 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:39:19.136795 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\"" Jul 06 02:39:19.138436 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:39:19.137923 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\"" Jul 06 02:39:19.138436 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:39:19.138022 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\"" Jul 06 02:39:19.138436 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:39:19.138133 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\"" Jul 06 02:39:19.138436 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:39:19.138209 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\"" Jul 06 02:39:19.138436 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:39:19.137927 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\"" Jul 06 02:39:19.138436 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:39:19.138434 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\"" Jul 06 02:39:19.138755 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:39:19.138564 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-xq2dl\"" Jul 06 02:39:19.138755 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:39:19.138612 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-xjlbh" Jul 06 02:39:19.138755 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:39:19.138623 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\"" Jul 06 02:39:19.139145 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:39:19.139004 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\"" Jul 06 02:39:19.139145 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:39:19.139012 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-2xstb\"" Jul 06 02:39:19.139260 ip-10-0-139-206 kubenswrapper[2557]: E0706 02:39:19.139156 2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-xjlbh" podUID="8f9fee4a-de24-49dc-b667-3dbc852d7ae3" Jul 06 02:39:19.140375 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:39:19.140340 2557 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving" Jul 06 02:39:19.140462 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:39:19.140392 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-l2btb" Jul 06 02:39:19.140514 ip-10-0-139-206 kubenswrapper[2557]: E0706 02:39:19.140485 2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-l2btb" podUID="c3ee201f-4897-4a18-bf37-f6c327d25e1e" Jul 06 02:39:19.144463 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:39:19.144445 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-fxqsb" Jul 06 02:39:19.146429 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:39:19.146395 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-twqqj" Jul 06 02:39:19.146917 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:39:19.146870 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-2b5fx\"" Jul 06 02:39:19.146917 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:39:19.146870 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\"" Jul 06 02:39:19.147389 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:39:19.147219 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\"" Jul 06 02:39:19.149237 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:39:19.148798 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\"" Jul 06 02:39:19.149237 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:39:19.148810 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\"" Jul 06 02:39:19.149237 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:39:19.148893 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-k78gp\"" Jul 06 02:39:19.149237 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:39:19.149078 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-hgnzv" Jul 06 02:39:19.150453 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:39:19.150227 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-756n2" Jul 06 02:39:19.150453 ip-10-0-139-206 kubenswrapper[2557]: E0706 02:39:19.150288 2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-756n2" podUID="26f69a47-e949-4e18-9d9c-6648a959e281" Jul 06 02:39:19.151174 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:39:19.151154 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\"" Jul 06 02:39:19.151316 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:39:19.151228 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\"" Jul 06 02:39:19.151607 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:39:19.151592 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\"" Jul 06 02:39:19.151775 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:39:19.151717 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-g77bq\"" Jul 06 02:39:19.153081 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:39:19.152895 2557 reflector.go:439] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Jul 06 02:39:19.153806 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:39:19.153782 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-t4wsh\" (UniqueName: \"kubernetes.io/projected/c3ee201f-4897-4a18-bf37-f6c327d25e1e-kube-api-access-t4wsh\") pod \"network-check-target-l2btb\" (UID: \"c3ee201f-4897-4a18-bf37-f6c327d25e1e\") " pod="openshift-network-diagnostics/network-check-target-l2btb" Jul 06 02:39:19.153925 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:39:19.153821 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/b6d9cfba-1e8c-48ce-a3ed-20464e26c44e-sys\") pod \"tuned-fxqsb\" (UID: \"b6d9cfba-1e8c-48ce-a3ed-20464e26c44e\") " pod="openshift-cluster-node-tuning-operator/tuned-fxqsb" Jul 06 02:39:19.153925 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:39:19.153846 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/b6d9cfba-1e8c-48ce-a3ed-20464e26c44e-tmp\") pod \"tuned-fxqsb\" (UID: \"b6d9cfba-1e8c-48ce-a3ed-20464e26c44e\") " pod="openshift-cluster-node-tuning-operator/tuned-fxqsb" Jul 06 02:39:19.153925 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:39:19.153870 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/af7822ef-5b14-477a-9fdd-5ef1fac67e23-host-var-lib-cni-multus\") pod \"multus-v4pnk\" (UID: \"af7822ef-5b14-477a-9fdd-5ef1fac67e23\") " pod="openshift-multus/multus-v4pnk" Jul 06 02:39:19.153925 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:39:19.153894 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/af7822ef-5b14-477a-9fdd-5ef1fac67e23-host-run-multus-certs\") pod \"multus-v4pnk\" (UID: \"af7822ef-5b14-477a-9fdd-5ef1fac67e23\") " pod="openshift-multus/multus-v4pnk" Jul 06 02:39:19.153925 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:39:19.153918 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-c64jj\" (UniqueName: \"kubernetes.io/projected/5b80db07-d76f-4470-a3d5-3176fdf63aa2-kube-api-access-c64jj\") pod \"node-ca-lwmh8\" (UID: \"5b80db07-d76f-4470-a3d5-3176fdf63aa2\") " pod="openshift-image-registry/node-ca-lwmh8" Jul 06 02:39:19.154247 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:39:19.153942 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/8f9fee4a-de24-49dc-b667-3dbc852d7ae3-original-pull-secret\") pod \"global-pull-secret-syncer-xjlbh\" (UID: \"8f9fee4a-de24-49dc-b667-3dbc852d7ae3\") " pod="kube-system/global-pull-secret-syncer-xjlbh" Jul 06 02:39:19.154247 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:39:19.153972 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/af7822ef-5b14-477a-9fdd-5ef1fac67e23-multus-socket-dir-parent\") pod \"multus-v4pnk\" (UID: \"af7822ef-5b14-477a-9fdd-5ef1fac67e23\") " pod="openshift-multus/multus-v4pnk" Jul 06 02:39:19.154247 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:39:19.154005 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/af7822ef-5b14-477a-9fdd-5ef1fac67e23-host-var-lib-kubelet\") pod \"multus-v4pnk\" (UID: \"af7822ef-5b14-477a-9fdd-5ef1fac67e23\") " pod="openshift-multus/multus-v4pnk" Jul 06 02:39:19.154247 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:39:19.154042 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/3c23051e-1536-45e2-8a89-dc077a09d982-socket-dir\") pod \"aws-ebs-csi-driver-node-84mq4\" (UID: \"3c23051e-1536-45e2-8a89-dc077a09d982\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-84mq4" Jul 06 02:39:19.154247 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:39:19.154083 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kpq6h\" (UniqueName: \"kubernetes.io/projected/efcf5e11-1e5d-4e2a-9b38-303a8f0787c4-kube-api-access-kpq6h\") pod \"ovnkube-node-cw2f7\" (UID: \"efcf5e11-1e5d-4e2a-9b38-303a8f0787c4\") " pod="openshift-ovn-kubernetes/ovnkube-node-cw2f7" Jul 06 02:39:19.154247 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:39:19.154120 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wm5x6\" (UniqueName: \"kubernetes.io/projected/29184621-7845-4a4a-9b81-04b439571ca3-kube-api-access-wm5x6\") pod \"multus-additional-cni-plugins-7nlb2\" (UID: \"29184621-7845-4a4a-9b81-04b439571ca3\") " pod="openshift-multus/multus-additional-cni-plugins-7nlb2" Jul 06 02:39:19.154247 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:39:19.154149 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/b6d9cfba-1e8c-48ce-a3ed-20464e26c44e-lib-modules\") pod \"tuned-fxqsb\" (UID: \"b6d9cfba-1e8c-48ce-a3ed-20464e26c44e\") " pod="openshift-cluster-node-tuning-operator/tuned-fxqsb" Jul 06 02:39:19.154247 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:39:19.154171 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/efcf5e11-1e5d-4e2a-9b38-303a8f0787c4-host-kubelet\") pod \"ovnkube-node-cw2f7\" (UID: \"efcf5e11-1e5d-4e2a-9b38-303a8f0787c4\") " pod="openshift-ovn-kubernetes/ovnkube-node-cw2f7" Jul 06 02:39:19.154247 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:39:19.154193 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/b6d9cfba-1e8c-48ce-a3ed-20464e26c44e-var-lib-kubelet\") pod \"tuned-fxqsb\" (UID: \"b6d9cfba-1e8c-48ce-a3ed-20464e26c44e\") " pod="openshift-cluster-node-tuning-operator/tuned-fxqsb" Jul 06 02:39:19.154247 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:39:19.154214 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/b6d9cfba-1e8c-48ce-a3ed-20464e26c44e-etc-tuned\") pod \"tuned-fxqsb\" (UID: \"b6d9cfba-1e8c-48ce-a3ed-20464e26c44e\") " pod="openshift-cluster-node-tuning-operator/tuned-fxqsb" Jul 06 02:39:19.154247 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:39:19.154235 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/5b80db07-d76f-4470-a3d5-3176fdf63aa2-host\") pod \"node-ca-lwmh8\" (UID: \"5b80db07-d76f-4470-a3d5-3176fdf63aa2\") " pod="openshift-image-registry/node-ca-lwmh8" Jul 06 02:39:19.154914 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:39:19.154257 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/efcf5e11-1e5d-4e2a-9b38-303a8f0787c4-host-slash\") pod \"ovnkube-node-cw2f7\" (UID: \"efcf5e11-1e5d-4e2a-9b38-303a8f0787c4\") " pod="openshift-ovn-kubernetes/ovnkube-node-cw2f7" Jul 06 02:39:19.154914 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:39:19.154283 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/efcf5e11-1e5d-4e2a-9b38-303a8f0787c4-ovnkube-config\") pod \"ovnkube-node-cw2f7\" (UID: \"efcf5e11-1e5d-4e2a-9b38-303a8f0787c4\") " pod="openshift-ovn-kubernetes/ovnkube-node-cw2f7" Jul 06 02:39:19.154914 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:39:19.154304 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/dd0a7167-560c-4195-857e-b69abeede707-hosts-file\") pod \"node-resolver-bhp46\" (UID: \"dd0a7167-560c-4195-857e-b69abeede707\") " pod="openshift-dns/node-resolver-bhp46" Jul 06 02:39:19.154914 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:39:19.154324 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/29184621-7845-4a4a-9b81-04b439571ca3-system-cni-dir\") pod \"multus-additional-cni-plugins-7nlb2\" (UID: \"29184621-7845-4a4a-9b81-04b439571ca3\") " pod="openshift-multus/multus-additional-cni-plugins-7nlb2" Jul 06 02:39:19.154914 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:39:19.154455 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/29184621-7845-4a4a-9b81-04b439571ca3-cni-binary-copy\") pod \"multus-additional-cni-plugins-7nlb2\" (UID: \"29184621-7845-4a4a-9b81-04b439571ca3\") " pod="openshift-multus/multus-additional-cni-plugins-7nlb2" Jul 06 02:39:19.154914 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:39:19.154496 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/b6d9cfba-1e8c-48ce-a3ed-20464e26c44e-run\") pod \"tuned-fxqsb\" (UID: \"b6d9cfba-1e8c-48ce-a3ed-20464e26c44e\") " pod="openshift-cluster-node-tuning-operator/tuned-fxqsb" Jul 06 02:39:19.154914 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:39:19.154538 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/af7822ef-5b14-477a-9fdd-5ef1fac67e23-host-run-netns\") pod \"multus-v4pnk\" (UID: \"af7822ef-5b14-477a-9fdd-5ef1fac67e23\") " pod="openshift-multus/multus-v4pnk" Jul 06 02:39:19.154914 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:39:19.154581 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-grjn2\" (UniqueName: \"kubernetes.io/projected/3c23051e-1536-45e2-8a89-dc077a09d982-kube-api-access-grjn2\") pod \"aws-ebs-csi-driver-node-84mq4\" (UID: \"3c23051e-1536-45e2-8a89-dc077a09d982\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-84mq4" Jul 06 02:39:19.154914 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:39:19.154623 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/efcf5e11-1e5d-4e2a-9b38-303a8f0787c4-etc-openvswitch\") pod \"ovnkube-node-cw2f7\" (UID: \"efcf5e11-1e5d-4e2a-9b38-303a8f0787c4\") " pod="openshift-ovn-kubernetes/ovnkube-node-cw2f7" Jul 06 02:39:19.154914 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:39:19.154661 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/efcf5e11-1e5d-4e2a-9b38-303a8f0787c4-run-ovn\") pod \"ovnkube-node-cw2f7\" (UID: \"efcf5e11-1e5d-4e2a-9b38-303a8f0787c4\") " pod="openshift-ovn-kubernetes/ovnkube-node-cw2f7" Jul 06 02:39:19.154914 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:39:19.154687 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/29184621-7845-4a4a-9b81-04b439571ca3-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-7nlb2\" (UID: \"29184621-7845-4a4a-9b81-04b439571ca3\") " pod="openshift-multus/multus-additional-cni-plugins-7nlb2" Jul 06 02:39:19.154914 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:39:19.154713 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/b6d9cfba-1e8c-48ce-a3ed-20464e26c44e-etc-modprobe-d\") pod \"tuned-fxqsb\" (UID: \"b6d9cfba-1e8c-48ce-a3ed-20464e26c44e\") " pod="openshift-cluster-node-tuning-operator/tuned-fxqsb" Jul 06 02:39:19.154914 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:39:19.154736 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/b6d9cfba-1e8c-48ce-a3ed-20464e26c44e-etc-sysctl-d\") pod \"tuned-fxqsb\" (UID: \"b6d9cfba-1e8c-48ce-a3ed-20464e26c44e\") " pod="openshift-cluster-node-tuning-operator/tuned-fxqsb" Jul 06 02:39:19.154914 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:39:19.154759 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/af7822ef-5b14-477a-9fdd-5ef1fac67e23-host-run-k8s-cni-cncf-io\") pod \"multus-v4pnk\" (UID: \"af7822ef-5b14-477a-9fdd-5ef1fac67e23\") " pod="openshift-multus/multus-v4pnk" Jul 06 02:39:19.154914 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:39:19.154781 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/af7822ef-5b14-477a-9fdd-5ef1fac67e23-multus-daemon-config\") pod \"multus-v4pnk\" (UID: \"af7822ef-5b14-477a-9fdd-5ef1fac67e23\") " pod="openshift-multus/multus-v4pnk" Jul 06 02:39:19.154914 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:39:19.154802 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/efcf5e11-1e5d-4e2a-9b38-303a8f0787c4-ovn-node-metrics-cert\") pod \"ovnkube-node-cw2f7\" (UID: \"efcf5e11-1e5d-4e2a-9b38-303a8f0787c4\") " pod="openshift-ovn-kubernetes/ovnkube-node-cw2f7" Jul 06 02:39:19.155683 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:39:19.154837 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/3c23051e-1536-45e2-8a89-dc077a09d982-registration-dir\") pod \"aws-ebs-csi-driver-node-84mq4\" (UID: \"3c23051e-1536-45e2-8a89-dc077a09d982\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-84mq4" Jul 06 02:39:19.155683 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:39:19.154864 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/efcf5e11-1e5d-4e2a-9b38-303a8f0787c4-systemd-units\") pod \"ovnkube-node-cw2f7\" (UID: \"efcf5e11-1e5d-4e2a-9b38-303a8f0787c4\") " pod="openshift-ovn-kubernetes/ovnkube-node-cw2f7" Jul 06 02:39:19.155683 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:39:19.154895 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/efcf5e11-1e5d-4e2a-9b38-303a8f0787c4-host-cni-bin\") pod \"ovnkube-node-cw2f7\" (UID: \"efcf5e11-1e5d-4e2a-9b38-303a8f0787c4\") " pod="openshift-ovn-kubernetes/ovnkube-node-cw2f7" Jul 06 02:39:19.155683 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:39:19.154919 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/8f9fee4a-de24-49dc-b667-3dbc852d7ae3-kubelet-config\") pod \"global-pull-secret-syncer-xjlbh\" (UID: \"8f9fee4a-de24-49dc-b667-3dbc852d7ae3\") " pod="kube-system/global-pull-secret-syncer-xjlbh" Jul 06 02:39:19.155683 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:39:19.154941 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/b6d9cfba-1e8c-48ce-a3ed-20464e26c44e-etc-systemd\") pod \"tuned-fxqsb\" (UID: \"b6d9cfba-1e8c-48ce-a3ed-20464e26c44e\") " pod="openshift-cluster-node-tuning-operator/tuned-fxqsb" Jul 06 02:39:19.155683 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:39:19.154961 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/af7822ef-5b14-477a-9fdd-5ef1fac67e23-cnibin\") pod \"multus-v4pnk\" (UID: \"af7822ef-5b14-477a-9fdd-5ef1fac67e23\") " pod="openshift-multus/multus-v4pnk" Jul 06 02:39:19.155683 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:39:19.154981 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/af7822ef-5b14-477a-9fdd-5ef1fac67e23-host-var-lib-cni-bin\") pod \"multus-v4pnk\" (UID: \"af7822ef-5b14-477a-9fdd-5ef1fac67e23\") " pod="openshift-multus/multus-v4pnk" Jul 06 02:39:19.155683 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:39:19.155007 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/af7822ef-5b14-477a-9fdd-5ef1fac67e23-hostroot\") pod \"multus-v4pnk\" (UID: \"af7822ef-5b14-477a-9fdd-5ef1fac67e23\") " pod="openshift-multus/multus-v4pnk" Jul 06 02:39:19.155683 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:39:19.155027 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/efcf5e11-1e5d-4e2a-9b38-303a8f0787c4-var-lib-openvswitch\") pod \"ovnkube-node-cw2f7\" (UID: \"efcf5e11-1e5d-4e2a-9b38-303a8f0787c4\") " pod="openshift-ovn-kubernetes/ovnkube-node-cw2f7" Jul 06 02:39:19.155683 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:39:19.155075 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/efcf5e11-1e5d-4e2a-9b38-303a8f0787c4-log-socket\") pod \"ovnkube-node-cw2f7\" (UID: \"efcf5e11-1e5d-4e2a-9b38-303a8f0787c4\") " pod="openshift-ovn-kubernetes/ovnkube-node-cw2f7" Jul 06 02:39:19.155683 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:39:19.155097 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/efcf5e11-1e5d-4e2a-9b38-303a8f0787c4-env-overrides\") pod \"ovnkube-node-cw2f7\" (UID: \"efcf5e11-1e5d-4e2a-9b38-303a8f0787c4\") " pod="openshift-ovn-kubernetes/ovnkube-node-cw2f7" Jul 06 02:39:19.155683 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:39:19.155118 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/efcf5e11-1e5d-4e2a-9b38-303a8f0787c4-ovnkube-script-lib\") pod \"ovnkube-node-cw2f7\" (UID: \"efcf5e11-1e5d-4e2a-9b38-303a8f0787c4\") " pod="openshift-ovn-kubernetes/ovnkube-node-cw2f7" Jul 06 02:39:19.155683 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:39:19.155139 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/efcf5e11-1e5d-4e2a-9b38-303a8f0787c4-node-log\") pod \"ovnkube-node-cw2f7\" (UID: \"efcf5e11-1e5d-4e2a-9b38-303a8f0787c4\") " pod="openshift-ovn-kubernetes/ovnkube-node-cw2f7" Jul 06 02:39:19.155683 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:39:19.155161 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/efcf5e11-1e5d-4e2a-9b38-303a8f0787c4-host-run-ovn-kubernetes\") pod \"ovnkube-node-cw2f7\" (UID: \"efcf5e11-1e5d-4e2a-9b38-303a8f0787c4\") " pod="openshift-ovn-kubernetes/ovnkube-node-cw2f7" Jul 06 02:39:19.155683 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:39:19.155196 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/29184621-7845-4a4a-9b81-04b439571ca3-os-release\") pod \"multus-additional-cni-plugins-7nlb2\" (UID: \"29184621-7845-4a4a-9b81-04b439571ca3\") " pod="openshift-multus/multus-additional-cni-plugins-7nlb2" Jul 06 02:39:19.155683 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:39:19.155221 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/b6d9cfba-1e8c-48ce-a3ed-20464e26c44e-etc-kubernetes\") pod \"tuned-fxqsb\" (UID: \"b6d9cfba-1e8c-48ce-a3ed-20464e26c44e\") " pod="openshift-cluster-node-tuning-operator/tuned-fxqsb" Jul 06 02:39:19.156331 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:39:19.155242 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/af7822ef-5b14-477a-9fdd-5ef1fac67e23-system-cni-dir\") pod \"multus-v4pnk\" (UID: \"af7822ef-5b14-477a-9fdd-5ef1fac67e23\") " pod="openshift-multus/multus-v4pnk" Jul 06 02:39:19.156331 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:39:19.155265 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hx2xl\" (UniqueName: \"kubernetes.io/projected/af7822ef-5b14-477a-9fdd-5ef1fac67e23-kube-api-access-hx2xl\") pod \"multus-v4pnk\" (UID: \"af7822ef-5b14-477a-9fdd-5ef1fac67e23\") " pod="openshift-multus/multus-v4pnk" Jul 06 02:39:19.156331 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:39:19.155302 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/5b80db07-d76f-4470-a3d5-3176fdf63aa2-serviceca\") pod \"node-ca-lwmh8\" (UID: \"5b80db07-d76f-4470-a3d5-3176fdf63aa2\") " pod="openshift-image-registry/node-ca-lwmh8" Jul 06 02:39:19.156331 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:39:19.155324 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/efcf5e11-1e5d-4e2a-9b38-303a8f0787c4-run-openvswitch\") pod \"ovnkube-node-cw2f7\" (UID: \"efcf5e11-1e5d-4e2a-9b38-303a8f0787c4\") " pod="openshift-ovn-kubernetes/ovnkube-node-cw2f7" Jul 06 02:39:19.156331 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:39:19.155363 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/efcf5e11-1e5d-4e2a-9b38-303a8f0787c4-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-cw2f7\" (UID: \"efcf5e11-1e5d-4e2a-9b38-303a8f0787c4\") " pod="openshift-ovn-kubernetes/ovnkube-node-cw2f7" Jul 06 02:39:19.156331 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:39:19.155401 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/dd0a7167-560c-4195-857e-b69abeede707-tmp-dir\") pod \"node-resolver-bhp46\" (UID: \"dd0a7167-560c-4195-857e-b69abeede707\") " pod="openshift-dns/node-resolver-bhp46" Jul 06 02:39:19.156331 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:39:19.155453 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/29184621-7845-4a4a-9b81-04b439571ca3-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-7nlb2\" (UID: \"29184621-7845-4a4a-9b81-04b439571ca3\") " pod="openshift-multus/multus-additional-cni-plugins-7nlb2" Jul 06 02:39:19.156331 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:39:19.155483 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/af7822ef-5b14-477a-9fdd-5ef1fac67e23-cni-binary-copy\") pod \"multus-v4pnk\" (UID: \"af7822ef-5b14-477a-9fdd-5ef1fac67e23\") " pod="openshift-multus/multus-v4pnk" Jul 06 02:39:19.156331 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:39:19.155508 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/3c23051e-1536-45e2-8a89-dc077a09d982-kubelet-dir\") pod \"aws-ebs-csi-driver-node-84mq4\" (UID: \"3c23051e-1536-45e2-8a89-dc077a09d982\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-84mq4" Jul 06 02:39:19.156331 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:39:19.155536 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/efcf5e11-1e5d-4e2a-9b38-303a8f0787c4-host-cni-netd\") pod \"ovnkube-node-cw2f7\" (UID: \"efcf5e11-1e5d-4e2a-9b38-303a8f0787c4\") " pod="openshift-ovn-kubernetes/ovnkube-node-cw2f7" Jul 06 02:39:19.156331 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:39:19.155558 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/b6d9cfba-1e8c-48ce-a3ed-20464e26c44e-etc-sysctl-conf\") pod \"tuned-fxqsb\" (UID: \"b6d9cfba-1e8c-48ce-a3ed-20464e26c44e\") " pod="openshift-cluster-node-tuning-operator/tuned-fxqsb" Jul 06 02:39:19.156331 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:39:19.155581 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/b6d9cfba-1e8c-48ce-a3ed-20464e26c44e-host\") pod \"tuned-fxqsb\" (UID: \"b6d9cfba-1e8c-48ce-a3ed-20464e26c44e\") " pod="openshift-cluster-node-tuning-operator/tuned-fxqsb" Jul 06 02:39:19.156331 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:39:19.155609 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/3c23051e-1536-45e2-8a89-dc077a09d982-etc-selinux\") pod \"aws-ebs-csi-driver-node-84mq4\" (UID: \"3c23051e-1536-45e2-8a89-dc077a09d982\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-84mq4" Jul 06 02:39:19.156331 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:39:19.155630 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/8f9fee4a-de24-49dc-b667-3dbc852d7ae3-dbus\") pod \"global-pull-secret-syncer-xjlbh\" (UID: \"8f9fee4a-de24-49dc-b667-3dbc852d7ae3\") " pod="kube-system/global-pull-secret-syncer-xjlbh" Jul 06 02:39:19.156331 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:39:19.155652 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-r55mb\" (UniqueName: \"kubernetes.io/projected/dd0a7167-560c-4195-857e-b69abeede707-kube-api-access-r55mb\") pod \"node-resolver-bhp46\" (UID: \"dd0a7167-560c-4195-857e-b69abeede707\") " pod="openshift-dns/node-resolver-bhp46" Jul 06 02:39:19.156331 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:39:19.155695 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/b6d9cfba-1e8c-48ce-a3ed-20464e26c44e-etc-sysconfig\") pod \"tuned-fxqsb\" (UID: \"b6d9cfba-1e8c-48ce-a3ed-20464e26c44e\") " pod="openshift-cluster-node-tuning-operator/tuned-fxqsb" Jul 06 02:39:19.157198 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:39:19.155718 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ncwnw\" (UniqueName: \"kubernetes.io/projected/b6d9cfba-1e8c-48ce-a3ed-20464e26c44e-kube-api-access-ncwnw\") pod \"tuned-fxqsb\" (UID: \"b6d9cfba-1e8c-48ce-a3ed-20464e26c44e\") " pod="openshift-cluster-node-tuning-operator/tuned-fxqsb" Jul 06 02:39:19.157198 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:39:19.155739 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/af7822ef-5b14-477a-9fdd-5ef1fac67e23-multus-cni-dir\") pod \"multus-v4pnk\" (UID: \"af7822ef-5b14-477a-9fdd-5ef1fac67e23\") " pod="openshift-multus/multus-v4pnk" Jul 06 02:39:19.157198 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:39:19.155769 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/3c23051e-1536-45e2-8a89-dc077a09d982-sys-fs\") pod \"aws-ebs-csi-driver-node-84mq4\" (UID: \"3c23051e-1536-45e2-8a89-dc077a09d982\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-84mq4" Jul 06 02:39:19.157198 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:39:19.155793 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/29184621-7845-4a4a-9b81-04b439571ca3-cnibin\") pod \"multus-additional-cni-plugins-7nlb2\" (UID: \"29184621-7845-4a4a-9b81-04b439571ca3\") " pod="openshift-multus/multus-additional-cni-plugins-7nlb2" Jul 06 02:39:19.157198 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:39:19.155815 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/af7822ef-5b14-477a-9fdd-5ef1fac67e23-os-release\") pod \"multus-v4pnk\" (UID: \"af7822ef-5b14-477a-9fdd-5ef1fac67e23\") " pod="openshift-multus/multus-v4pnk" Jul 06 02:39:19.157198 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:39:19.155836 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/af7822ef-5b14-477a-9fdd-5ef1fac67e23-multus-conf-dir\") pod \"multus-v4pnk\" (UID: \"af7822ef-5b14-477a-9fdd-5ef1fac67e23\") " pod="openshift-multus/multus-v4pnk" Jul 06 02:39:19.157198 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:39:19.155857 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/af7822ef-5b14-477a-9fdd-5ef1fac67e23-etc-kubernetes\") pod \"multus-v4pnk\" (UID: \"af7822ef-5b14-477a-9fdd-5ef1fac67e23\") " pod="openshift-multus/multus-v4pnk" Jul 06 02:39:19.157198 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:39:19.155878 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/3c23051e-1536-45e2-8a89-dc077a09d982-device-dir\") pod \"aws-ebs-csi-driver-node-84mq4\" (UID: \"3c23051e-1536-45e2-8a89-dc077a09d982\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-84mq4" Jul 06 02:39:19.157198 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:39:19.155915 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/efcf5e11-1e5d-4e2a-9b38-303a8f0787c4-host-run-netns\") pod \"ovnkube-node-cw2f7\" (UID: \"efcf5e11-1e5d-4e2a-9b38-303a8f0787c4\") " pod="openshift-ovn-kubernetes/ovnkube-node-cw2f7" Jul 06 02:39:19.157198 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:39:19.155959 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/efcf5e11-1e5d-4e2a-9b38-303a8f0787c4-run-systemd\") pod \"ovnkube-node-cw2f7\" (UID: \"efcf5e11-1e5d-4e2a-9b38-303a8f0787c4\") " pod="openshift-ovn-kubernetes/ovnkube-node-cw2f7" Jul 06 02:39:19.157198 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:39:19.155996 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/29184621-7845-4a4a-9b81-04b439571ca3-tuning-conf-dir\") pod \"multus-additional-cni-plugins-7nlb2\" (UID: \"29184621-7845-4a4a-9b81-04b439571ca3\") " pod="openshift-multus/multus-additional-cni-plugins-7nlb2" Jul 06 02:39:19.173951 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:39:19.173925 2557 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-5wdz8" Jul 06 02:39:19.182467 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:39:19.182439 2557 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-5wdz8" Jul 06 02:39:19.241228 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:39:19.241191 2557 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world" Jul 06 02:39:19.256666 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:39:19.256628 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/efcf5e11-1e5d-4e2a-9b38-303a8f0787c4-var-lib-openvswitch\") pod \"ovnkube-node-cw2f7\" (UID: \"efcf5e11-1e5d-4e2a-9b38-303a8f0787c4\") " pod="openshift-ovn-kubernetes/ovnkube-node-cw2f7" Jul 06 02:39:19.256854 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:39:19.256674 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/efcf5e11-1e5d-4e2a-9b38-303a8f0787c4-log-socket\") pod \"ovnkube-node-cw2f7\" (UID: \"efcf5e11-1e5d-4e2a-9b38-303a8f0787c4\") " pod="openshift-ovn-kubernetes/ovnkube-node-cw2f7" Jul 06 02:39:19.256854 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:39:19.256702 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/efcf5e11-1e5d-4e2a-9b38-303a8f0787c4-env-overrides\") pod \"ovnkube-node-cw2f7\" (UID: \"efcf5e11-1e5d-4e2a-9b38-303a8f0787c4\") " pod="openshift-ovn-kubernetes/ovnkube-node-cw2f7" Jul 06 02:39:19.256854 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:39:19.256726 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/efcf5e11-1e5d-4e2a-9b38-303a8f0787c4-ovnkube-script-lib\") pod \"ovnkube-node-cw2f7\" (UID: \"efcf5e11-1e5d-4e2a-9b38-303a8f0787c4\") " pod="openshift-ovn-kubernetes/ovnkube-node-cw2f7" Jul 06 02:39:19.256854 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:39:19.256739 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/efcf5e11-1e5d-4e2a-9b38-303a8f0787c4-var-lib-openvswitch\") pod \"ovnkube-node-cw2f7\" (UID: \"efcf5e11-1e5d-4e2a-9b38-303a8f0787c4\") " pod="openshift-ovn-kubernetes/ovnkube-node-cw2f7" Jul 06 02:39:19.256854 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:39:19.256749 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/efcf5e11-1e5d-4e2a-9b38-303a8f0787c4-node-log\") pod \"ovnkube-node-cw2f7\" (UID: \"efcf5e11-1e5d-4e2a-9b38-303a8f0787c4\") " pod="openshift-ovn-kubernetes/ovnkube-node-cw2f7" Jul 06 02:39:19.256854 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:39:19.256799 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/efcf5e11-1e5d-4e2a-9b38-303a8f0787c4-host-run-ovn-kubernetes\") pod \"ovnkube-node-cw2f7\" (UID: \"efcf5e11-1e5d-4e2a-9b38-303a8f0787c4\") " pod="openshift-ovn-kubernetes/ovnkube-node-cw2f7" Jul 06 02:39:19.256854 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:39:19.256829 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/29184621-7845-4a4a-9b81-04b439571ca3-os-release\") pod \"multus-additional-cni-plugins-7nlb2\" (UID: \"29184621-7845-4a4a-9b81-04b439571ca3\") " pod="openshift-multus/multus-additional-cni-plugins-7nlb2" Jul 06 02:39:19.256854 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:39:19.256827 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/efcf5e11-1e5d-4e2a-9b38-303a8f0787c4-log-socket\") pod \"ovnkube-node-cw2f7\" (UID: \"efcf5e11-1e5d-4e2a-9b38-303a8f0787c4\") " pod="openshift-ovn-kubernetes/ovnkube-node-cw2f7" Jul 06 02:39:19.256854 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:39:19.256851 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/b6d9cfba-1e8c-48ce-a3ed-20464e26c44e-etc-kubernetes\") pod \"tuned-fxqsb\" (UID: \"b6d9cfba-1e8c-48ce-a3ed-20464e26c44e\") " pod="openshift-cluster-node-tuning-operator/tuned-fxqsb" Jul 06 02:39:19.257300 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:39:19.256874 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/af7822ef-5b14-477a-9fdd-5ef1fac67e23-system-cni-dir\") pod \"multus-v4pnk\" (UID: \"af7822ef-5b14-477a-9fdd-5ef1fac67e23\") " pod="openshift-multus/multus-v4pnk" Jul 06 02:39:19.257300 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:39:19.256889 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/efcf5e11-1e5d-4e2a-9b38-303a8f0787c4-host-run-ovn-kubernetes\") pod \"ovnkube-node-cw2f7\" (UID: \"efcf5e11-1e5d-4e2a-9b38-303a8f0787c4\") " pod="openshift-ovn-kubernetes/ovnkube-node-cw2f7" Jul 06 02:39:19.257300 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:39:19.256896 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hx2xl\" (UniqueName: \"kubernetes.io/projected/af7822ef-5b14-477a-9fdd-5ef1fac67e23-kube-api-access-hx2xl\") pod \"multus-v4pnk\" (UID: \"af7822ef-5b14-477a-9fdd-5ef1fac67e23\") " pod="openshift-multus/multus-v4pnk" Jul 06 02:39:19.257300 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:39:19.256892 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/efcf5e11-1e5d-4e2a-9b38-303a8f0787c4-node-log\") pod \"ovnkube-node-cw2f7\" (UID: \"efcf5e11-1e5d-4e2a-9b38-303a8f0787c4\") " pod="openshift-ovn-kubernetes/ovnkube-node-cw2f7" Jul 06 02:39:19.257300 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:39:19.256919 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/5b80db07-d76f-4470-a3d5-3176fdf63aa2-serviceca\") pod \"node-ca-lwmh8\" (UID: \"5b80db07-d76f-4470-a3d5-3176fdf63aa2\") " pod="openshift-image-registry/node-ca-lwmh8" Jul 06 02:39:19.257300 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:39:19.256942 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/efcf5e11-1e5d-4e2a-9b38-303a8f0787c4-run-openvswitch\") pod \"ovnkube-node-cw2f7\" (UID: \"efcf5e11-1e5d-4e2a-9b38-303a8f0787c4\") " pod="openshift-ovn-kubernetes/ovnkube-node-cw2f7" Jul 06 02:39:19.257300 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:39:19.256959 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/af7822ef-5b14-477a-9fdd-5ef1fac67e23-system-cni-dir\") pod \"multus-v4pnk\" (UID: \"af7822ef-5b14-477a-9fdd-5ef1fac67e23\") " pod="openshift-multus/multus-v4pnk" Jul 06 02:39:19.257300 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:39:19.256968 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/efcf5e11-1e5d-4e2a-9b38-303a8f0787c4-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-cw2f7\" (UID: \"efcf5e11-1e5d-4e2a-9b38-303a8f0787c4\") " pod="openshift-ovn-kubernetes/ovnkube-node-cw2f7" Jul 06 02:39:19.257300 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:39:19.256994 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/dd0a7167-560c-4195-857e-b69abeede707-tmp-dir\") pod \"node-resolver-bhp46\" (UID: \"dd0a7167-560c-4195-857e-b69abeede707\") " pod="openshift-dns/node-resolver-bhp46" Jul 06 02:39:19.257300 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:39:19.256943 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/b6d9cfba-1e8c-48ce-a3ed-20464e26c44e-etc-kubernetes\") pod \"tuned-fxqsb\" (UID: \"b6d9cfba-1e8c-48ce-a3ed-20464e26c44e\") " pod="openshift-cluster-node-tuning-operator/tuned-fxqsb" Jul 06 02:39:19.257300 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:39:19.257020 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/29184621-7845-4a4a-9b81-04b439571ca3-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-7nlb2\" (UID: \"29184621-7845-4a4a-9b81-04b439571ca3\") " pod="openshift-multus/multus-additional-cni-plugins-7nlb2" Jul 06 02:39:19.257300 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:39:19.257039 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/efcf5e11-1e5d-4e2a-9b38-303a8f0787c4-run-openvswitch\") pod \"ovnkube-node-cw2f7\" (UID: \"efcf5e11-1e5d-4e2a-9b38-303a8f0787c4\") " pod="openshift-ovn-kubernetes/ovnkube-node-cw2f7" Jul 06 02:39:19.257300 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:39:19.257044 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/af7822ef-5b14-477a-9fdd-5ef1fac67e23-cni-binary-copy\") pod \"multus-v4pnk\" (UID: \"af7822ef-5b14-477a-9fdd-5ef1fac67e23\") " pod="openshift-multus/multus-v4pnk" Jul 06 02:39:19.257300 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:39:19.257069 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/3c23051e-1536-45e2-8a89-dc077a09d982-kubelet-dir\") pod \"aws-ebs-csi-driver-node-84mq4\" (UID: \"3c23051e-1536-45e2-8a89-dc077a09d982\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-84mq4" Jul 06 02:39:19.257300 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:39:19.257084 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/efcf5e11-1e5d-4e2a-9b38-303a8f0787c4-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-cw2f7\" (UID: \"efcf5e11-1e5d-4e2a-9b38-303a8f0787c4\") " pod="openshift-ovn-kubernetes/ovnkube-node-cw2f7" Jul 06 02:39:19.257300 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:39:19.257092 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/efcf5e11-1e5d-4e2a-9b38-303a8f0787c4-host-cni-netd\") pod \"ovnkube-node-cw2f7\" (UID: \"efcf5e11-1e5d-4e2a-9b38-303a8f0787c4\") " pod="openshift-ovn-kubernetes/ovnkube-node-cw2f7" Jul 06 02:39:19.257300 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:39:19.257130 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-l5g9n\" (UniqueName: \"kubernetes.io/projected/1b70e00f-2790-4bb2-8fc6-4ce6fb7e5ab1-kube-api-access-l5g9n\") pod \"iptables-alerter-hgnzv\" (UID: \"1b70e00f-2790-4bb2-8fc6-4ce6fb7e5ab1\") " pod="openshift-network-operator/iptables-alerter-hgnzv" Jul 06 02:39:19.258122 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:39:19.257177 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/b6d9cfba-1e8c-48ce-a3ed-20464e26c44e-etc-sysctl-conf\") pod \"tuned-fxqsb\" (UID: \"b6d9cfba-1e8c-48ce-a3ed-20464e26c44e\") " pod="openshift-cluster-node-tuning-operator/tuned-fxqsb" Jul 06 02:39:19.258122 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:39:19.257203 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/b6d9cfba-1e8c-48ce-a3ed-20464e26c44e-host\") pod \"tuned-fxqsb\" (UID: \"b6d9cfba-1e8c-48ce-a3ed-20464e26c44e\") " pod="openshift-cluster-node-tuning-operator/tuned-fxqsb" Jul 06 02:39:19.258122 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:39:19.257226 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/3c23051e-1536-45e2-8a89-dc077a09d982-etc-selinux\") pod \"aws-ebs-csi-driver-node-84mq4\" (UID: \"3c23051e-1536-45e2-8a89-dc077a09d982\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-84mq4" Jul 06 02:39:19.258122 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:39:19.257250 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/8f9fee4a-de24-49dc-b667-3dbc852d7ae3-dbus\") pod \"global-pull-secret-syncer-xjlbh\" (UID: \"8f9fee4a-de24-49dc-b667-3dbc852d7ae3\") " pod="kube-system/global-pull-secret-syncer-xjlbh" Jul 06 02:39:19.258122 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:39:19.257273 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-r55mb\" (UniqueName: \"kubernetes.io/projected/dd0a7167-560c-4195-857e-b69abeede707-kube-api-access-r55mb\") pod \"node-resolver-bhp46\" (UID: \"dd0a7167-560c-4195-857e-b69abeede707\") " pod="openshift-dns/node-resolver-bhp46" Jul 06 02:39:19.258122 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:39:19.257296 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/b6d9cfba-1e8c-48ce-a3ed-20464e26c44e-etc-sysconfig\") pod \"tuned-fxqsb\" (UID: \"b6d9cfba-1e8c-48ce-a3ed-20464e26c44e\") " pod="openshift-cluster-node-tuning-operator/tuned-fxqsb" Jul 06 02:39:19.258122 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:39:19.257321 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ncwnw\" (UniqueName: \"kubernetes.io/projected/b6d9cfba-1e8c-48ce-a3ed-20464e26c44e-kube-api-access-ncwnw\") pod \"tuned-fxqsb\" (UID: \"b6d9cfba-1e8c-48ce-a3ed-20464e26c44e\") " pod="openshift-cluster-node-tuning-operator/tuned-fxqsb" Jul 06 02:39:19.258122 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:39:19.256995 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/29184621-7845-4a4a-9b81-04b439571ca3-os-release\") pod \"multus-additional-cni-plugins-7nlb2\" (UID: \"29184621-7845-4a4a-9b81-04b439571ca3\") " pod="openshift-multus/multus-additional-cni-plugins-7nlb2" Jul 06 02:39:19.258122 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:39:19.257397 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/00981958-29f3-4942-8ba6-0be9f03348a4-konnectivity-ca\") pod \"konnectivity-agent-twqqj\" (UID: \"00981958-29f3-4942-8ba6-0be9f03348a4\") " pod="kube-system/konnectivity-agent-twqqj" Jul 06 02:39:19.258122 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:39:19.257426 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/af7822ef-5b14-477a-9fdd-5ef1fac67e23-multus-cni-dir\") pod \"multus-v4pnk\" (UID: \"af7822ef-5b14-477a-9fdd-5ef1fac67e23\") " pod="openshift-multus/multus-v4pnk" Jul 06 02:39:19.258122 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:39:19.257430 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/5b80db07-d76f-4470-a3d5-3176fdf63aa2-serviceca\") pod \"node-ca-lwmh8\" (UID: \"5b80db07-d76f-4470-a3d5-3176fdf63aa2\") " pod="openshift-image-registry/node-ca-lwmh8" Jul 06 02:39:19.258122 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:39:19.257459 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/3c23051e-1536-45e2-8a89-dc077a09d982-sys-fs\") pod \"aws-ebs-csi-driver-node-84mq4\" (UID: \"3c23051e-1536-45e2-8a89-dc077a09d982\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-84mq4" Jul 06 02:39:19.258122 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:39:19.257500 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/efcf5e11-1e5d-4e2a-9b38-303a8f0787c4-ovnkube-script-lib\") pod \"ovnkube-node-cw2f7\" (UID: \"efcf5e11-1e5d-4e2a-9b38-303a8f0787c4\") " pod="openshift-ovn-kubernetes/ovnkube-node-cw2f7" Jul 06 02:39:19.258122 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:39:19.257489 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/29184621-7845-4a4a-9b81-04b439571ca3-cnibin\") pod \"multus-additional-cni-plugins-7nlb2\" (UID: \"29184621-7845-4a4a-9b81-04b439571ca3\") " pod="openshift-multus/multus-additional-cni-plugins-7nlb2" Jul 06 02:39:19.258122 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:39:19.257533 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/af7822ef-5b14-477a-9fdd-5ef1fac67e23-os-release\") pod \"multus-v4pnk\" (UID: \"af7822ef-5b14-477a-9fdd-5ef1fac67e23\") " pod="openshift-multus/multus-v4pnk" Jul 06 02:39:19.258122 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:39:19.257541 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/b6d9cfba-1e8c-48ce-a3ed-20464e26c44e-etc-sysctl-conf\") pod \"tuned-fxqsb\" (UID: \"b6d9cfba-1e8c-48ce-a3ed-20464e26c44e\") " pod="openshift-cluster-node-tuning-operator/tuned-fxqsb" Jul 06 02:39:19.258122 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:39:19.257557 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/af7822ef-5b14-477a-9fdd-5ef1fac67e23-multus-conf-dir\") pod \"multus-v4pnk\" (UID: \"af7822ef-5b14-477a-9fdd-5ef1fac67e23\") " pod="openshift-multus/multus-v4pnk" Jul 06 02:39:19.258944 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:39:19.257582 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/af7822ef-5b14-477a-9fdd-5ef1fac67e23-etc-kubernetes\") pod \"multus-v4pnk\" (UID: \"af7822ef-5b14-477a-9fdd-5ef1fac67e23\") " pod="openshift-multus/multus-v4pnk" Jul 06 02:39:19.258944 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:39:19.257600 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/b6d9cfba-1e8c-48ce-a3ed-20464e26c44e-host\") pod \"tuned-fxqsb\" (UID: \"b6d9cfba-1e8c-48ce-a3ed-20464e26c44e\") " pod="openshift-cluster-node-tuning-operator/tuned-fxqsb" Jul 06 02:39:19.258944 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:39:19.257612 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/3c23051e-1536-45e2-8a89-dc077a09d982-device-dir\") pod \"aws-ebs-csi-driver-node-84mq4\" (UID: \"3c23051e-1536-45e2-8a89-dc077a09d982\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-84mq4" Jul 06 02:39:19.258944 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:39:19.257637 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/efcf5e11-1e5d-4e2a-9b38-303a8f0787c4-host-run-netns\") pod \"ovnkube-node-cw2f7\" (UID: \"efcf5e11-1e5d-4e2a-9b38-303a8f0787c4\") " pod="openshift-ovn-kubernetes/ovnkube-node-cw2f7" Jul 06 02:39:19.258944 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:39:19.257662 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/efcf5e11-1e5d-4e2a-9b38-303a8f0787c4-run-systemd\") pod \"ovnkube-node-cw2f7\" (UID: \"efcf5e11-1e5d-4e2a-9b38-303a8f0787c4\") " pod="openshift-ovn-kubernetes/ovnkube-node-cw2f7" Jul 06 02:39:19.258944 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:39:19.257686 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/29184621-7845-4a4a-9b81-04b439571ca3-tuning-conf-dir\") pod \"multus-additional-cni-plugins-7nlb2\" (UID: \"29184621-7845-4a4a-9b81-04b439571ca3\") " pod="openshift-multus/multus-additional-cni-plugins-7nlb2" Jul 06 02:39:19.258944 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:39:19.257708 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/dd0a7167-560c-4195-857e-b69abeede707-tmp-dir\") pod \"node-resolver-bhp46\" (UID: \"dd0a7167-560c-4195-857e-b69abeede707\") " pod="openshift-dns/node-resolver-bhp46" Jul 06 02:39:19.258944 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:39:19.257711 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/3c23051e-1536-45e2-8a89-dc077a09d982-etc-selinux\") pod \"aws-ebs-csi-driver-node-84mq4\" (UID: \"3c23051e-1536-45e2-8a89-dc077a09d982\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-84mq4" Jul 06 02:39:19.258944 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:39:19.257719 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-t4wsh\" (UniqueName: \"kubernetes.io/projected/c3ee201f-4897-4a18-bf37-f6c327d25e1e-kube-api-access-t4wsh\") pod \"network-check-target-l2btb\" (UID: \"c3ee201f-4897-4a18-bf37-f6c327d25e1e\") " pod="openshift-network-diagnostics/network-check-target-l2btb" Jul 06 02:39:19.258944 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:39:19.257744 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/b6d9cfba-1e8c-48ce-a3ed-20464e26c44e-sys\") pod \"tuned-fxqsb\" (UID: \"b6d9cfba-1e8c-48ce-a3ed-20464e26c44e\") " pod="openshift-cluster-node-tuning-operator/tuned-fxqsb" Jul 06 02:39:19.258944 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:39:19.257788 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/b6d9cfba-1e8c-48ce-a3ed-20464e26c44e-tmp\") pod \"tuned-fxqsb\" (UID: \"b6d9cfba-1e8c-48ce-a3ed-20464e26c44e\") " pod="openshift-cluster-node-tuning-operator/tuned-fxqsb" Jul 06 02:39:19.258944 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:39:19.257791 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/29184621-7845-4a4a-9b81-04b439571ca3-cnibin\") pod \"multus-additional-cni-plugins-7nlb2\" (UID: \"29184621-7845-4a4a-9b81-04b439571ca3\") " pod="openshift-multus/multus-additional-cni-plugins-7nlb2" Jul 06 02:39:19.258944 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:39:19.257812 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/af7822ef-5b14-477a-9fdd-5ef1fac67e23-host-var-lib-cni-multus\") pod \"multus-v4pnk\" (UID: \"af7822ef-5b14-477a-9fdd-5ef1fac67e23\") " pod="openshift-multus/multus-v4pnk" Jul 06 02:39:19.258944 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:39:19.257836 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/af7822ef-5b14-477a-9fdd-5ef1fac67e23-host-run-multus-certs\") pod \"multus-v4pnk\" (UID: \"af7822ef-5b14-477a-9fdd-5ef1fac67e23\") " pod="openshift-multus/multus-v4pnk" Jul 06 02:39:19.258944 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:39:19.257856 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/af7822ef-5b14-477a-9fdd-5ef1fac67e23-os-release\") pod \"multus-v4pnk\" (UID: \"af7822ef-5b14-477a-9fdd-5ef1fac67e23\") " pod="openshift-multus/multus-v4pnk" Jul 06 02:39:19.258944 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:39:19.257860 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-c64jj\" (UniqueName: \"kubernetes.io/projected/5b80db07-d76f-4470-a3d5-3176fdf63aa2-kube-api-access-c64jj\") pod \"node-ca-lwmh8\" (UID: \"5b80db07-d76f-4470-a3d5-3176fdf63aa2\") " pod="openshift-image-registry/node-ca-lwmh8" Jul 06 02:39:19.258944 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:39:19.257895 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/8f9fee4a-de24-49dc-b667-3dbc852d7ae3-original-pull-secret\") pod \"global-pull-secret-syncer-xjlbh\" (UID: \"8f9fee4a-de24-49dc-b667-3dbc852d7ae3\") " pod="kube-system/global-pull-secret-syncer-xjlbh" Jul 06 02:39:19.259771 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:39:19.257913 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/29184621-7845-4a4a-9b81-04b439571ca3-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-7nlb2\" (UID: \"29184621-7845-4a4a-9b81-04b439571ca3\") " pod="openshift-multus/multus-additional-cni-plugins-7nlb2" Jul 06 02:39:19.259771 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:39:19.257924 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/af7822ef-5b14-477a-9fdd-5ef1fac67e23-multus-socket-dir-parent\") pod \"multus-v4pnk\" (UID: \"af7822ef-5b14-477a-9fdd-5ef1fac67e23\") " pod="openshift-multus/multus-v4pnk" Jul 06 02:39:19.259771 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:39:19.257948 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/af7822ef-5b14-477a-9fdd-5ef1fac67e23-host-var-lib-kubelet\") pod \"multus-v4pnk\" (UID: \"af7822ef-5b14-477a-9fdd-5ef1fac67e23\") " pod="openshift-multus/multus-v4pnk" Jul 06 02:39:19.259771 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:39:19.257965 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/efcf5e11-1e5d-4e2a-9b38-303a8f0787c4-host-cni-netd\") pod \"ovnkube-node-cw2f7\" (UID: \"efcf5e11-1e5d-4e2a-9b38-303a8f0787c4\") " pod="openshift-ovn-kubernetes/ovnkube-node-cw2f7" Jul 06 02:39:19.259771 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:39:19.257973 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/3c23051e-1536-45e2-8a89-dc077a09d982-socket-dir\") pod \"aws-ebs-csi-driver-node-84mq4\" (UID: \"3c23051e-1536-45e2-8a89-dc077a09d982\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-84mq4" Jul 06 02:39:19.259771 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:39:19.258000 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kpq6h\" (UniqueName: \"kubernetes.io/projected/efcf5e11-1e5d-4e2a-9b38-303a8f0787c4-kube-api-access-kpq6h\") pod \"ovnkube-node-cw2f7\" (UID: \"efcf5e11-1e5d-4e2a-9b38-303a8f0787c4\") " pod="openshift-ovn-kubernetes/ovnkube-node-cw2f7" Jul 06 02:39:19.259771 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:39:19.258014 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/b6d9cfba-1e8c-48ce-a3ed-20464e26c44e-etc-sysconfig\") pod \"tuned-fxqsb\" (UID: \"b6d9cfba-1e8c-48ce-a3ed-20464e26c44e\") " pod="openshift-cluster-node-tuning-operator/tuned-fxqsb" Jul 06 02:39:19.259771 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:39:19.258013 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/af7822ef-5b14-477a-9fdd-5ef1fac67e23-cni-binary-copy\") pod \"multus-v4pnk\" (UID: \"af7822ef-5b14-477a-9fdd-5ef1fac67e23\") " pod="openshift-multus/multus-v4pnk" Jul 06 02:39:19.259771 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:39:19.258028 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/1b70e00f-2790-4bb2-8fc6-4ce6fb7e5ab1-iptables-alerter-script\") pod \"iptables-alerter-hgnzv\" (UID: \"1b70e00f-2790-4bb2-8fc6-4ce6fb7e5ab1\") " pod="openshift-network-operator/iptables-alerter-hgnzv" Jul 06 02:39:19.259771 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:39:19.258053 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/00981958-29f3-4942-8ba6-0be9f03348a4-agent-certs\") pod \"konnectivity-agent-twqqj\" (UID: \"00981958-29f3-4942-8ba6-0be9f03348a4\") " pod="kube-system/konnectivity-agent-twqqj" Jul 06 02:39:19.259771 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:39:19.258063 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/3c23051e-1536-45e2-8a89-dc077a09d982-sys-fs\") pod \"aws-ebs-csi-driver-node-84mq4\" (UID: \"3c23051e-1536-45e2-8a89-dc077a09d982\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-84mq4" Jul 06 02:39:19.259771 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:39:19.258035 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/efcf5e11-1e5d-4e2a-9b38-303a8f0787c4-host-run-netns\") pod \"ovnkube-node-cw2f7\" (UID: \"efcf5e11-1e5d-4e2a-9b38-303a8f0787c4\") " pod="openshift-ovn-kubernetes/ovnkube-node-cw2f7" Jul 06 02:39:19.259771 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:39:19.258082 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wm5x6\" (UniqueName: \"kubernetes.io/projected/29184621-7845-4a4a-9b81-04b439571ca3-kube-api-access-wm5x6\") pod \"multus-additional-cni-plugins-7nlb2\" (UID: \"29184621-7845-4a4a-9b81-04b439571ca3\") " pod="openshift-multus/multus-additional-cni-plugins-7nlb2" Jul 06 02:39:19.259771 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:39:19.258107 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/b6d9cfba-1e8c-48ce-a3ed-20464e26c44e-lib-modules\") pod \"tuned-fxqsb\" (UID: \"b6d9cfba-1e8c-48ce-a3ed-20464e26c44e\") " pod="openshift-cluster-node-tuning-operator/tuned-fxqsb" Jul 06 02:39:19.259771 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:39:19.258130 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/efcf5e11-1e5d-4e2a-9b38-303a8f0787c4-host-kubelet\") pod \"ovnkube-node-cw2f7\" (UID: \"efcf5e11-1e5d-4e2a-9b38-303a8f0787c4\") " pod="openshift-ovn-kubernetes/ovnkube-node-cw2f7" Jul 06 02:39:19.259771 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:39:19.258150 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/af7822ef-5b14-477a-9fdd-5ef1fac67e23-host-var-lib-cni-multus\") pod \"multus-v4pnk\" (UID: \"af7822ef-5b14-477a-9fdd-5ef1fac67e23\") " pod="openshift-multus/multus-v4pnk" Jul 06 02:39:19.259771 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:39:19.258155 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/b6d9cfba-1e8c-48ce-a3ed-20464e26c44e-var-lib-kubelet\") pod \"tuned-fxqsb\" (UID: \"b6d9cfba-1e8c-48ce-a3ed-20464e26c44e\") " pod="openshift-cluster-node-tuning-operator/tuned-fxqsb" Jul 06 02:39:19.260605 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:39:19.258167 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/29184621-7845-4a4a-9b81-04b439571ca3-tuning-conf-dir\") pod \"multus-additional-cni-plugins-7nlb2\" (UID: \"29184621-7845-4a4a-9b81-04b439571ca3\") " pod="openshift-multus/multus-additional-cni-plugins-7nlb2" Jul 06 02:39:19.260605 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:39:19.258197 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/af7822ef-5b14-477a-9fdd-5ef1fac67e23-host-run-multus-certs\") pod \"multus-v4pnk\" (UID: \"af7822ef-5b14-477a-9fdd-5ef1fac67e23\") " pod="openshift-multus/multus-v4pnk" Jul 06 02:39:19.260605 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:39:19.258202 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/af7822ef-5b14-477a-9fdd-5ef1fac67e23-multus-cni-dir\") pod \"multus-v4pnk\" (UID: \"af7822ef-5b14-477a-9fdd-5ef1fac67e23\") " pod="openshift-multus/multus-v4pnk" Jul 06 02:39:19.260605 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:39:19.258212 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/efcf5e11-1e5d-4e2a-9b38-303a8f0787c4-run-systemd\") pod \"ovnkube-node-cw2f7\" (UID: \"efcf5e11-1e5d-4e2a-9b38-303a8f0787c4\") " pod="openshift-ovn-kubernetes/ovnkube-node-cw2f7" Jul 06 02:39:19.260605 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:39:19.258204 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/af7822ef-5b14-477a-9fdd-5ef1fac67e23-multus-conf-dir\") pod \"multus-v4pnk\" (UID: \"af7822ef-5b14-477a-9fdd-5ef1fac67e23\") " pod="openshift-multus/multus-v4pnk" Jul 06 02:39:19.260605 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:39:19.257395 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/efcf5e11-1e5d-4e2a-9b38-303a8f0787c4-env-overrides\") pod \"ovnkube-node-cw2f7\" (UID: \"efcf5e11-1e5d-4e2a-9b38-303a8f0787c4\") " pod="openshift-ovn-kubernetes/ovnkube-node-cw2f7" Jul 06 02:39:19.260605 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:39:19.258267 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/b6d9cfba-1e8c-48ce-a3ed-20464e26c44e-etc-tuned\") pod \"tuned-fxqsb\" (UID: \"b6d9cfba-1e8c-48ce-a3ed-20464e26c44e\") " pod="openshift-cluster-node-tuning-operator/tuned-fxqsb" Jul 06 02:39:19.260605 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:39:19.258272 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/af7822ef-5b14-477a-9fdd-5ef1fac67e23-multus-socket-dir-parent\") pod \"multus-v4pnk\" (UID: \"af7822ef-5b14-477a-9fdd-5ef1fac67e23\") " pod="openshift-multus/multus-v4pnk" Jul 06 02:39:19.260605 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:39:19.258276 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/3c23051e-1536-45e2-8a89-dc077a09d982-kubelet-dir\") pod \"aws-ebs-csi-driver-node-84mq4\" (UID: \"3c23051e-1536-45e2-8a89-dc077a09d982\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-84mq4" Jul 06 02:39:19.260605 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:39:19.258300 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/b6d9cfba-1e8c-48ce-a3ed-20464e26c44e-var-lib-kubelet\") pod \"tuned-fxqsb\" (UID: \"b6d9cfba-1e8c-48ce-a3ed-20464e26c44e\") " pod="openshift-cluster-node-tuning-operator/tuned-fxqsb" Jul 06 02:39:19.260605 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:39:19.258310 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/af7822ef-5b14-477a-9fdd-5ef1fac67e23-host-var-lib-kubelet\") pod \"multus-v4pnk\" (UID: \"af7822ef-5b14-477a-9fdd-5ef1fac67e23\") " pod="openshift-multus/multus-v4pnk" Jul 06 02:39:19.260605 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:39:19.258333 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/3c23051e-1536-45e2-8a89-dc077a09d982-device-dir\") pod \"aws-ebs-csi-driver-node-84mq4\" (UID: \"3c23051e-1536-45e2-8a89-dc077a09d982\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-84mq4" Jul 06 02:39:19.260605 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:39:19.258369 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/b6d9cfba-1e8c-48ce-a3ed-20464e26c44e-sys\") pod \"tuned-fxqsb\" (UID: \"b6d9cfba-1e8c-48ce-a3ed-20464e26c44e\") " pod="openshift-cluster-node-tuning-operator/tuned-fxqsb" Jul 06 02:39:19.260605 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:39:19.258378 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/5b80db07-d76f-4470-a3d5-3176fdf63aa2-host\") pod \"node-ca-lwmh8\" (UID: \"5b80db07-d76f-4470-a3d5-3176fdf63aa2\") " pod="openshift-image-registry/node-ca-lwmh8" Jul 06 02:39:19.260605 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:39:19.258404 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/efcf5e11-1e5d-4e2a-9b38-303a8f0787c4-host-slash\") pod \"ovnkube-node-cw2f7\" (UID: \"efcf5e11-1e5d-4e2a-9b38-303a8f0787c4\") " pod="openshift-ovn-kubernetes/ovnkube-node-cw2f7" Jul 06 02:39:19.260605 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:39:19.258439 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/5b80db07-d76f-4470-a3d5-3176fdf63aa2-host\") pod \"node-ca-lwmh8\" (UID: \"5b80db07-d76f-4470-a3d5-3176fdf63aa2\") " pod="openshift-image-registry/node-ca-lwmh8" Jul 06 02:39:19.260605 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:39:19.258442 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/efcf5e11-1e5d-4e2a-9b38-303a8f0787c4-host-kubelet\") pod \"ovnkube-node-cw2f7\" (UID: \"efcf5e11-1e5d-4e2a-9b38-303a8f0787c4\") " pod="openshift-ovn-kubernetes/ovnkube-node-cw2f7" Jul 06 02:39:19.260605 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:39:19.258454 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/efcf5e11-1e5d-4e2a-9b38-303a8f0787c4-ovnkube-config\") pod \"ovnkube-node-cw2f7\" (UID: \"efcf5e11-1e5d-4e2a-9b38-303a8f0787c4\") " pod="openshift-ovn-kubernetes/ovnkube-node-cw2f7" Jul 06 02:39:19.261472 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:39:19.258479 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/dd0a7167-560c-4195-857e-b69abeede707-hosts-file\") pod \"node-resolver-bhp46\" (UID: \"dd0a7167-560c-4195-857e-b69abeede707\") " pod="openshift-dns/node-resolver-bhp46" Jul 06 02:39:19.261472 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:39:19.258504 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/efcf5e11-1e5d-4e2a-9b38-303a8f0787c4-host-slash\") pod \"ovnkube-node-cw2f7\" (UID: \"efcf5e11-1e5d-4e2a-9b38-303a8f0787c4\") " pod="openshift-ovn-kubernetes/ovnkube-node-cw2f7" Jul 06 02:39:19.261472 ip-10-0-139-206 kubenswrapper[2557]: E0706 02:39:19.258518 2557 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 06 02:39:19.261472 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:39:19.258523 2557 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory" Jul 06 02:39:19.261472 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:39:19.258549 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/dd0a7167-560c-4195-857e-b69abeede707-hosts-file\") pod \"node-resolver-bhp46\" (UID: \"dd0a7167-560c-4195-857e-b69abeede707\") " pod="openshift-dns/node-resolver-bhp46" Jul 06 02:39:19.261472 ip-10-0-139-206 kubenswrapper[2557]: E0706 02:39:19.258589 2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8f9fee4a-de24-49dc-b667-3dbc852d7ae3-original-pull-secret podName:8f9fee4a-de24-49dc-b667-3dbc852d7ae3 nodeName:}" failed. No retries permitted until 2026-07-06 02:39:19.758574314 +0000 UTC m=+3.564860366 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/8f9fee4a-de24-49dc-b667-3dbc852d7ae3-original-pull-secret") pod "global-pull-secret-syncer-xjlbh" (UID: "8f9fee4a-de24-49dc-b667-3dbc852d7ae3") : object "kube-system"/"original-pull-secret" not registered Jul 06 02:39:19.261472 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:39:19.258593 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/af7822ef-5b14-477a-9fdd-5ef1fac67e23-etc-kubernetes\") pod \"multus-v4pnk\" (UID: \"af7822ef-5b14-477a-9fdd-5ef1fac67e23\") " pod="openshift-multus/multus-v4pnk" Jul 06 02:39:19.261472 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:39:19.258629 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/8f9fee4a-de24-49dc-b667-3dbc852d7ae3-dbus\") pod \"global-pull-secret-syncer-xjlbh\" (UID: \"8f9fee4a-de24-49dc-b667-3dbc852d7ae3\") " pod="kube-system/global-pull-secret-syncer-xjlbh" Jul 06 02:39:19.261472 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:39:19.258672 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/29184621-7845-4a4a-9b81-04b439571ca3-system-cni-dir\") pod \"multus-additional-cni-plugins-7nlb2\" (UID: \"29184621-7845-4a4a-9b81-04b439571ca3\") " pod="openshift-multus/multus-additional-cni-plugins-7nlb2" Jul 06 02:39:19.261472 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:39:19.258635 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/29184621-7845-4a4a-9b81-04b439571ca3-system-cni-dir\") pod \"multus-additional-cni-plugins-7nlb2\" (UID: \"29184621-7845-4a4a-9b81-04b439571ca3\") " pod="openshift-multus/multus-additional-cni-plugins-7nlb2" Jul 06 02:39:19.261472 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:39:19.258727 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/29184621-7845-4a4a-9b81-04b439571ca3-cni-binary-copy\") pod \"multus-additional-cni-plugins-7nlb2\" (UID: \"29184621-7845-4a4a-9b81-04b439571ca3\") " pod="openshift-multus/multus-additional-cni-plugins-7nlb2" Jul 06 02:39:19.261472 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:39:19.258749 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/b6d9cfba-1e8c-48ce-a3ed-20464e26c44e-run\") pod \"tuned-fxqsb\" (UID: \"b6d9cfba-1e8c-48ce-a3ed-20464e26c44e\") " pod="openshift-cluster-node-tuning-operator/tuned-fxqsb" Jul 06 02:39:19.261472 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:39:19.258756 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/3c23051e-1536-45e2-8a89-dc077a09d982-socket-dir\") pod \"aws-ebs-csi-driver-node-84mq4\" (UID: \"3c23051e-1536-45e2-8a89-dc077a09d982\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-84mq4" Jul 06 02:39:19.261472 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:39:19.258769 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/af7822ef-5b14-477a-9fdd-5ef1fac67e23-host-run-netns\") pod \"multus-v4pnk\" (UID: \"af7822ef-5b14-477a-9fdd-5ef1fac67e23\") " pod="openshift-multus/multus-v4pnk" Jul 06 02:39:19.261472 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:39:19.258791 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-grjn2\" (UniqueName: \"kubernetes.io/projected/3c23051e-1536-45e2-8a89-dc077a09d982-kube-api-access-grjn2\") pod \"aws-ebs-csi-driver-node-84mq4\" (UID: \"3c23051e-1536-45e2-8a89-dc077a09d982\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-84mq4" Jul 06 02:39:19.261472 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:39:19.258813 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/efcf5e11-1e5d-4e2a-9b38-303a8f0787c4-etc-openvswitch\") pod \"ovnkube-node-cw2f7\" (UID: \"efcf5e11-1e5d-4e2a-9b38-303a8f0787c4\") " pod="openshift-ovn-kubernetes/ovnkube-node-cw2f7" Jul 06 02:39:19.261472 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:39:19.258827 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/b6d9cfba-1e8c-48ce-a3ed-20464e26c44e-run\") pod \"tuned-fxqsb\" (UID: \"b6d9cfba-1e8c-48ce-a3ed-20464e26c44e\") " pod="openshift-cluster-node-tuning-operator/tuned-fxqsb" Jul 06 02:39:19.261472 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:39:19.258836 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/efcf5e11-1e5d-4e2a-9b38-303a8f0787c4-run-ovn\") pod \"ovnkube-node-cw2f7\" (UID: \"efcf5e11-1e5d-4e2a-9b38-303a8f0787c4\") " pod="openshift-ovn-kubernetes/ovnkube-node-cw2f7" Jul 06 02:39:19.262424 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:39:19.258858 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/b6d9cfba-1e8c-48ce-a3ed-20464e26c44e-lib-modules\") pod \"tuned-fxqsb\" (UID: \"b6d9cfba-1e8c-48ce-a3ed-20464e26c44e\") " pod="openshift-cluster-node-tuning-operator/tuned-fxqsb" Jul 06 02:39:19.262424 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:39:19.258877 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/efcf5e11-1e5d-4e2a-9b38-303a8f0787c4-run-ovn\") pod \"ovnkube-node-cw2f7\" (UID: \"efcf5e11-1e5d-4e2a-9b38-303a8f0787c4\") " pod="openshift-ovn-kubernetes/ovnkube-node-cw2f7" Jul 06 02:39:19.262424 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:39:19.258909 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/af7822ef-5b14-477a-9fdd-5ef1fac67e23-host-run-netns\") pod \"multus-v4pnk\" (UID: \"af7822ef-5b14-477a-9fdd-5ef1fac67e23\") " pod="openshift-multus/multus-v4pnk" Jul 06 02:39:19.262424 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:39:19.258920 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/efcf5e11-1e5d-4e2a-9b38-303a8f0787c4-etc-openvswitch\") pod \"ovnkube-node-cw2f7\" (UID: \"efcf5e11-1e5d-4e2a-9b38-303a8f0787c4\") " pod="openshift-ovn-kubernetes/ovnkube-node-cw2f7" Jul 06 02:39:19.262424 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:39:19.258957 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/29184621-7845-4a4a-9b81-04b439571ca3-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-7nlb2\" (UID: \"29184621-7845-4a4a-9b81-04b439571ca3\") " pod="openshift-multus/multus-additional-cni-plugins-7nlb2" Jul 06 02:39:19.262424 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:39:19.258982 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/b6d9cfba-1e8c-48ce-a3ed-20464e26c44e-etc-modprobe-d\") pod \"tuned-fxqsb\" (UID: \"b6d9cfba-1e8c-48ce-a3ed-20464e26c44e\") " pod="openshift-cluster-node-tuning-operator/tuned-fxqsb" Jul 06 02:39:19.262424 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:39:19.258992 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/efcf5e11-1e5d-4e2a-9b38-303a8f0787c4-ovnkube-config\") pod \"ovnkube-node-cw2f7\" (UID: \"efcf5e11-1e5d-4e2a-9b38-303a8f0787c4\") " pod="openshift-ovn-kubernetes/ovnkube-node-cw2f7" Jul 06 02:39:19.262424 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:39:19.259005 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/b6d9cfba-1e8c-48ce-a3ed-20464e26c44e-etc-sysctl-d\") pod \"tuned-fxqsb\" (UID: \"b6d9cfba-1e8c-48ce-a3ed-20464e26c44e\") " pod="openshift-cluster-node-tuning-operator/tuned-fxqsb" Jul 06 02:39:19.262424 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:39:19.259028 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/af7822ef-5b14-477a-9fdd-5ef1fac67e23-host-run-k8s-cni-cncf-io\") pod \"multus-v4pnk\" (UID: \"af7822ef-5b14-477a-9fdd-5ef1fac67e23\") " pod="openshift-multus/multus-v4pnk" Jul 06 02:39:19.262424 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:39:19.259053 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/af7822ef-5b14-477a-9fdd-5ef1fac67e23-multus-daemon-config\") pod \"multus-v4pnk\" (UID: \"af7822ef-5b14-477a-9fdd-5ef1fac67e23\") " pod="openshift-multus/multus-v4pnk" Jul 06 02:39:19.262424 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:39:19.259101 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/b6d9cfba-1e8c-48ce-a3ed-20464e26c44e-etc-modprobe-d\") pod \"tuned-fxqsb\" (UID: \"b6d9cfba-1e8c-48ce-a3ed-20464e26c44e\") " pod="openshift-cluster-node-tuning-operator/tuned-fxqsb" Jul 06 02:39:19.262424 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:39:19.259154 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/af7822ef-5b14-477a-9fdd-5ef1fac67e23-host-run-k8s-cni-cncf-io\") pod \"multus-v4pnk\" (UID: \"af7822ef-5b14-477a-9fdd-5ef1fac67e23\") " pod="openshift-multus/multus-v4pnk" Jul 06 02:39:19.262424 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:39:19.259176 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/b6d9cfba-1e8c-48ce-a3ed-20464e26c44e-etc-sysctl-d\") pod \"tuned-fxqsb\" (UID: \"b6d9cfba-1e8c-48ce-a3ed-20464e26c44e\") " pod="openshift-cluster-node-tuning-operator/tuned-fxqsb" Jul 06 02:39:19.262424 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:39:19.259199 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/29184621-7845-4a4a-9b81-04b439571ca3-cni-binary-copy\") pod \"multus-additional-cni-plugins-7nlb2\" (UID: \"29184621-7845-4a4a-9b81-04b439571ca3\") " pod="openshift-multus/multus-additional-cni-plugins-7nlb2" Jul 06 02:39:19.262424 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:39:19.259213 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/efcf5e11-1e5d-4e2a-9b38-303a8f0787c4-ovn-node-metrics-cert\") pod \"ovnkube-node-cw2f7\" (UID: \"efcf5e11-1e5d-4e2a-9b38-303a8f0787c4\") " pod="openshift-ovn-kubernetes/ovnkube-node-cw2f7" Jul 06 02:39:19.262424 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:39:19.259242 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/1b70e00f-2790-4bb2-8fc6-4ce6fb7e5ab1-host-slash\") pod \"iptables-alerter-hgnzv\" (UID: \"1b70e00f-2790-4bb2-8fc6-4ce6fb7e5ab1\") " pod="openshift-network-operator/iptables-alerter-hgnzv" Jul 06 02:39:19.262424 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:39:19.259284 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rsmfh\" (UniqueName: \"kubernetes.io/projected/26f69a47-e949-4e18-9d9c-6648a959e281-kube-api-access-rsmfh\") pod \"network-metrics-daemon-756n2\" (UID: \"26f69a47-e949-4e18-9d9c-6648a959e281\") " pod="openshift-multus/network-metrics-daemon-756n2" Jul 06 02:39:19.263141 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:39:19.259378 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/3c23051e-1536-45e2-8a89-dc077a09d982-registration-dir\") pod \"aws-ebs-csi-driver-node-84mq4\" (UID: \"3c23051e-1536-45e2-8a89-dc077a09d982\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-84mq4" Jul 06 02:39:19.263141 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:39:19.259428 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/efcf5e11-1e5d-4e2a-9b38-303a8f0787c4-systemd-units\") pod \"ovnkube-node-cw2f7\" (UID: \"efcf5e11-1e5d-4e2a-9b38-303a8f0787c4\") " pod="openshift-ovn-kubernetes/ovnkube-node-cw2f7" Jul 06 02:39:19.263141 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:39:19.259456 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/efcf5e11-1e5d-4e2a-9b38-303a8f0787c4-host-cni-bin\") pod \"ovnkube-node-cw2f7\" (UID: \"efcf5e11-1e5d-4e2a-9b38-303a8f0787c4\") " pod="openshift-ovn-kubernetes/ovnkube-node-cw2f7" Jul 06 02:39:19.263141 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:39:19.259471 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/3c23051e-1536-45e2-8a89-dc077a09d982-registration-dir\") pod \"aws-ebs-csi-driver-node-84mq4\" (UID: \"3c23051e-1536-45e2-8a89-dc077a09d982\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-84mq4" Jul 06 02:39:19.263141 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:39:19.259472 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/29184621-7845-4a4a-9b81-04b439571ca3-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-7nlb2\" (UID: \"29184621-7845-4a4a-9b81-04b439571ca3\") " pod="openshift-multus/multus-additional-cni-plugins-7nlb2" Jul 06 02:39:19.263141 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:39:19.259497 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/efcf5e11-1e5d-4e2a-9b38-303a8f0787c4-systemd-units\") pod \"ovnkube-node-cw2f7\" (UID: \"efcf5e11-1e5d-4e2a-9b38-303a8f0787c4\") " pod="openshift-ovn-kubernetes/ovnkube-node-cw2f7" Jul 06 02:39:19.263141 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:39:19.259479 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/8f9fee4a-de24-49dc-b667-3dbc852d7ae3-kubelet-config\") pod \"global-pull-secret-syncer-xjlbh\" (UID: \"8f9fee4a-de24-49dc-b667-3dbc852d7ae3\") " pod="kube-system/global-pull-secret-syncer-xjlbh" Jul 06 02:39:19.263141 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:39:19.259546 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/26f69a47-e949-4e18-9d9c-6648a959e281-metrics-certs\") pod \"network-metrics-daemon-756n2\" (UID: \"26f69a47-e949-4e18-9d9c-6648a959e281\") " pod="openshift-multus/network-metrics-daemon-756n2" Jul 06 02:39:19.263141 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:39:19.259574 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/b6d9cfba-1e8c-48ce-a3ed-20464e26c44e-etc-systemd\") pod \"tuned-fxqsb\" (UID: \"b6d9cfba-1e8c-48ce-a3ed-20464e26c44e\") " pod="openshift-cluster-node-tuning-operator/tuned-fxqsb" Jul 06 02:39:19.263141 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:39:19.259500 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/af7822ef-5b14-477a-9fdd-5ef1fac67e23-multus-daemon-config\") pod \"multus-v4pnk\" (UID: \"af7822ef-5b14-477a-9fdd-5ef1fac67e23\") " pod="openshift-multus/multus-v4pnk" Jul 06 02:39:19.263141 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:39:19.259518 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/efcf5e11-1e5d-4e2a-9b38-303a8f0787c4-host-cni-bin\") pod \"ovnkube-node-cw2f7\" (UID: \"efcf5e11-1e5d-4e2a-9b38-303a8f0787c4\") " pod="openshift-ovn-kubernetes/ovnkube-node-cw2f7" Jul 06 02:39:19.263141 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:39:19.259518 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/8f9fee4a-de24-49dc-b667-3dbc852d7ae3-kubelet-config\") pod \"global-pull-secret-syncer-xjlbh\" (UID: \"8f9fee4a-de24-49dc-b667-3dbc852d7ae3\") " pod="kube-system/global-pull-secret-syncer-xjlbh" Jul 06 02:39:19.263141 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:39:19.259628 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/b6d9cfba-1e8c-48ce-a3ed-20464e26c44e-etc-systemd\") pod \"tuned-fxqsb\" (UID: \"b6d9cfba-1e8c-48ce-a3ed-20464e26c44e\") " pod="openshift-cluster-node-tuning-operator/tuned-fxqsb" Jul 06 02:39:19.263141 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:39:19.259659 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/af7822ef-5b14-477a-9fdd-5ef1fac67e23-cnibin\") pod \"multus-v4pnk\" (UID: \"af7822ef-5b14-477a-9fdd-5ef1fac67e23\") " pod="openshift-multus/multus-v4pnk" Jul 06 02:39:19.263141 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:39:19.259688 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/af7822ef-5b14-477a-9fdd-5ef1fac67e23-host-var-lib-cni-bin\") pod \"multus-v4pnk\" (UID: \"af7822ef-5b14-477a-9fdd-5ef1fac67e23\") " pod="openshift-multus/multus-v4pnk" Jul 06 02:39:19.263141 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:39:19.259713 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/af7822ef-5b14-477a-9fdd-5ef1fac67e23-hostroot\") pod \"multus-v4pnk\" (UID: \"af7822ef-5b14-477a-9fdd-5ef1fac67e23\") " pod="openshift-multus/multus-v4pnk" Jul 06 02:39:19.263141 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:39:19.259767 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/af7822ef-5b14-477a-9fdd-5ef1fac67e23-host-var-lib-cni-bin\") pod \"multus-v4pnk\" (UID: \"af7822ef-5b14-477a-9fdd-5ef1fac67e23\") " pod="openshift-multus/multus-v4pnk" Jul 06 02:39:19.263946 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:39:19.259771 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/af7822ef-5b14-477a-9fdd-5ef1fac67e23-cnibin\") pod \"multus-v4pnk\" (UID: \"af7822ef-5b14-477a-9fdd-5ef1fac67e23\") " pod="openshift-multus/multus-v4pnk" Jul 06 02:39:19.263946 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:39:19.259778 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/af7822ef-5b14-477a-9fdd-5ef1fac67e23-hostroot\") pod \"multus-v4pnk\" (UID: \"af7822ef-5b14-477a-9fdd-5ef1fac67e23\") " pod="openshift-multus/multus-v4pnk" Jul 06 02:39:19.263946 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:39:19.263110 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/b6d9cfba-1e8c-48ce-a3ed-20464e26c44e-tmp\") pod \"tuned-fxqsb\" (UID: \"b6d9cfba-1e8c-48ce-a3ed-20464e26c44e\") " pod="openshift-cluster-node-tuning-operator/tuned-fxqsb" Jul 06 02:39:19.263946 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:39:19.263259 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/efcf5e11-1e5d-4e2a-9b38-303a8f0787c4-ovn-node-metrics-cert\") pod \"ovnkube-node-cw2f7\" (UID: \"efcf5e11-1e5d-4e2a-9b38-303a8f0787c4\") " pod="openshift-ovn-kubernetes/ovnkube-node-cw2f7" Jul 06 02:39:19.263946 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:39:19.263458 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/b6d9cfba-1e8c-48ce-a3ed-20464e26c44e-etc-tuned\") pod \"tuned-fxqsb\" (UID: \"b6d9cfba-1e8c-48ce-a3ed-20464e26c44e\") " pod="openshift-cluster-node-tuning-operator/tuned-fxqsb" Jul 06 02:39:19.263946 ip-10-0-139-206 kubenswrapper[2557]: E0706 02:39:19.263695 2557 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 06 02:39:19.263946 ip-10-0-139-206 kubenswrapper[2557]: E0706 02:39:19.263718 2557 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 06 02:39:19.263946 ip-10-0-139-206 kubenswrapper[2557]: E0706 02:39:19.263732 2557 projected.go:196] Error preparing data for projected volume kube-api-access-t4wsh for pod openshift-network-diagnostics/network-check-target-l2btb: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 06 02:39:19.263946 ip-10-0-139-206 kubenswrapper[2557]: E0706 02:39:19.263805 2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/c3ee201f-4897-4a18-bf37-f6c327d25e1e-kube-api-access-t4wsh podName:c3ee201f-4897-4a18-bf37-f6c327d25e1e nodeName:}" failed. No retries permitted until 2026-07-06 02:39:19.763786267 +0000 UTC m=+3.570072323 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-t4wsh" (UniqueName: "kubernetes.io/projected/c3ee201f-4897-4a18-bf37-f6c327d25e1e-kube-api-access-t4wsh") pod "network-check-target-l2btb" (UID: "c3ee201f-4897-4a18-bf37-f6c327d25e1e") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 06 02:39:19.266110 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:39:19.266061 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-hx2xl\" (UniqueName: \"kubernetes.io/projected/af7822ef-5b14-477a-9fdd-5ef1fac67e23-kube-api-access-hx2xl\") pod \"multus-v4pnk\" (UID: \"af7822ef-5b14-477a-9fdd-5ef1fac67e23\") " pod="openshift-multus/multus-v4pnk" Jul 06 02:39:19.267659 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:39:19.267632 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-r55mb\" (UniqueName: \"kubernetes.io/projected/dd0a7167-560c-4195-857e-b69abeede707-kube-api-access-r55mb\") pod \"node-resolver-bhp46\" (UID: \"dd0a7167-560c-4195-857e-b69abeede707\") " pod="openshift-dns/node-resolver-bhp46" Jul 06 02:39:19.268266 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:39:19.268226 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-c64jj\" (UniqueName: \"kubernetes.io/projected/5b80db07-d76f-4470-a3d5-3176fdf63aa2-kube-api-access-c64jj\") pod \"node-ca-lwmh8\" (UID: \"5b80db07-d76f-4470-a3d5-3176fdf63aa2\") " pod="openshift-image-registry/node-ca-lwmh8" Jul 06 02:39:19.268519 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:39:19.268458 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-grjn2\" (UniqueName: \"kubernetes.io/projected/3c23051e-1536-45e2-8a89-dc077a09d982-kube-api-access-grjn2\") pod \"aws-ebs-csi-driver-node-84mq4\" (UID: \"3c23051e-1536-45e2-8a89-dc077a09d982\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-84mq4" Jul 06 02:39:19.268673 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:39:19.268628 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-kpq6h\" (UniqueName: \"kubernetes.io/projected/efcf5e11-1e5d-4e2a-9b38-303a8f0787c4-kube-api-access-kpq6h\") pod \"ovnkube-node-cw2f7\" (UID: \"efcf5e11-1e5d-4e2a-9b38-303a8f0787c4\") " pod="openshift-ovn-kubernetes/ovnkube-node-cw2f7" Jul 06 02:39:19.269116 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:39:19.269074 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-wm5x6\" (UniqueName: \"kubernetes.io/projected/29184621-7845-4a4a-9b81-04b439571ca3-kube-api-access-wm5x6\") pod \"multus-additional-cni-plugins-7nlb2\" (UID: \"29184621-7845-4a4a-9b81-04b439571ca3\") " pod="openshift-multus/multus-additional-cni-plugins-7nlb2" Jul 06 02:39:19.270046 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:39:19.270018 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-ncwnw\" (UniqueName: \"kubernetes.io/projected/b6d9cfba-1e8c-48ce-a3ed-20464e26c44e-kube-api-access-ncwnw\") pod \"tuned-fxqsb\" (UID: \"b6d9cfba-1e8c-48ce-a3ed-20464e26c44e\") " pod="openshift-cluster-node-tuning-operator/tuned-fxqsb" Jul 06 02:39:19.360707 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:39:19.360666 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/1b70e00f-2790-4bb2-8fc6-4ce6fb7e5ab1-iptables-alerter-script\") pod \"iptables-alerter-hgnzv\" (UID: \"1b70e00f-2790-4bb2-8fc6-4ce6fb7e5ab1\") " pod="openshift-network-operator/iptables-alerter-hgnzv" Jul 06 02:39:19.360707 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:39:19.360714 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/00981958-29f3-4942-8ba6-0be9f03348a4-agent-certs\") pod \"konnectivity-agent-twqqj\" (UID: \"00981958-29f3-4942-8ba6-0be9f03348a4\") " pod="kube-system/konnectivity-agent-twqqj" Jul 06 02:39:19.360951 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:39:19.360757 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/1b70e00f-2790-4bb2-8fc6-4ce6fb7e5ab1-host-slash\") pod \"iptables-alerter-hgnzv\" (UID: \"1b70e00f-2790-4bb2-8fc6-4ce6fb7e5ab1\") " pod="openshift-network-operator/iptables-alerter-hgnzv" Jul 06 02:39:19.360951 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:39:19.360782 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rsmfh\" (UniqueName: \"kubernetes.io/projected/26f69a47-e949-4e18-9d9c-6648a959e281-kube-api-access-rsmfh\") pod \"network-metrics-daemon-756n2\" (UID: \"26f69a47-e949-4e18-9d9c-6648a959e281\") " pod="openshift-multus/network-metrics-daemon-756n2" Jul 06 02:39:19.360951 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:39:19.360807 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/26f69a47-e949-4e18-9d9c-6648a959e281-metrics-certs\") pod \"network-metrics-daemon-756n2\" (UID: \"26f69a47-e949-4e18-9d9c-6648a959e281\") " pod="openshift-multus/network-metrics-daemon-756n2" Jul 06 02:39:19.360951 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:39:19.360851 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-l5g9n\" (UniqueName: \"kubernetes.io/projected/1b70e00f-2790-4bb2-8fc6-4ce6fb7e5ab1-kube-api-access-l5g9n\") pod \"iptables-alerter-hgnzv\" (UID: \"1b70e00f-2790-4bb2-8fc6-4ce6fb7e5ab1\") " pod="openshift-network-operator/iptables-alerter-hgnzv" Jul 06 02:39:19.360951 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:39:19.360881 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/00981958-29f3-4942-8ba6-0be9f03348a4-konnectivity-ca\") pod \"konnectivity-agent-twqqj\" (UID: \"00981958-29f3-4942-8ba6-0be9f03348a4\") " pod="kube-system/konnectivity-agent-twqqj" Jul 06 02:39:19.360951 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:39:19.360878 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/1b70e00f-2790-4bb2-8fc6-4ce6fb7e5ab1-host-slash\") pod \"iptables-alerter-hgnzv\" (UID: \"1b70e00f-2790-4bb2-8fc6-4ce6fb7e5ab1\") " pod="openshift-network-operator/iptables-alerter-hgnzv" Jul 06 02:39:19.361245 ip-10-0-139-206 kubenswrapper[2557]: E0706 02:39:19.361012 2557 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 06 02:39:19.361245 ip-10-0-139-206 kubenswrapper[2557]: E0706 02:39:19.361068 2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/26f69a47-e949-4e18-9d9c-6648a959e281-metrics-certs podName:26f69a47-e949-4e18-9d9c-6648a959e281 nodeName:}" failed. No retries permitted until 2026-07-06 02:39:19.861050509 +0000 UTC m=+3.667336550 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/26f69a47-e949-4e18-9d9c-6648a959e281-metrics-certs") pod "network-metrics-daemon-756n2" (UID: "26f69a47-e949-4e18-9d9c-6648a959e281") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 06 02:39:19.361425 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:39:19.361382 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/1b70e00f-2790-4bb2-8fc6-4ce6fb7e5ab1-iptables-alerter-script\") pod \"iptables-alerter-hgnzv\" (UID: \"1b70e00f-2790-4bb2-8fc6-4ce6fb7e5ab1\") " pod="openshift-network-operator/iptables-alerter-hgnzv" Jul 06 02:39:19.361637 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:39:19.361615 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/00981958-29f3-4942-8ba6-0be9f03348a4-konnectivity-ca\") pod \"konnectivity-agent-twqqj\" (UID: \"00981958-29f3-4942-8ba6-0be9f03348a4\") " pod="kube-system/konnectivity-agent-twqqj" Jul 06 02:39:19.363670 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:39:19.363638 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/00981958-29f3-4942-8ba6-0be9f03348a4-agent-certs\") pod \"konnectivity-agent-twqqj\" (UID: \"00981958-29f3-4942-8ba6-0be9f03348a4\") " pod="kube-system/konnectivity-agent-twqqj" Jul 06 02:39:19.370013 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:39:19.369973 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-l5g9n\" (UniqueName: \"kubernetes.io/projected/1b70e00f-2790-4bb2-8fc6-4ce6fb7e5ab1-kube-api-access-l5g9n\") pod \"iptables-alerter-hgnzv\" (UID: \"1b70e00f-2790-4bb2-8fc6-4ce6fb7e5ab1\") " pod="openshift-network-operator/iptables-alerter-hgnzv" Jul 06 02:39:19.370195 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:39:19.370148 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-rsmfh\" (UniqueName: \"kubernetes.io/projected/26f69a47-e949-4e18-9d9c-6648a959e281-kube-api-access-rsmfh\") pod \"network-metrics-daemon-756n2\" (UID: \"26f69a47-e949-4e18-9d9c-6648a959e281\") " pod="openshift-multus/network-metrics-daemon-756n2" Jul 06 02:39:19.451232 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:39:19.451196 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-v4pnk" Jul 06 02:39:19.458102 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:39:19.458079 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-84mq4" Jul 06 02:39:19.467759 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:39:19.467732 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-bhp46" Jul 06 02:39:19.472441 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:39:19.472422 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-lwmh8" Jul 06 02:39:19.478119 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:39:19.478098 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-cw2f7" Jul 06 02:39:19.484670 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:39:19.484644 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-7nlb2" Jul 06 02:39:19.492425 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:39:19.492398 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-fxqsb" Jul 06 02:39:19.499715 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:39:19.499693 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-twqqj" Jul 06 02:39:19.508085 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:39:19.508064 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-hgnzv" Jul 06 02:39:19.763944 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:39:19.763870 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-t4wsh\" (UniqueName: \"kubernetes.io/projected/c3ee201f-4897-4a18-bf37-f6c327d25e1e-kube-api-access-t4wsh\") pod \"network-check-target-l2btb\" (UID: \"c3ee201f-4897-4a18-bf37-f6c327d25e1e\") " pod="openshift-network-diagnostics/network-check-target-l2btb" Jul 06 02:39:19.763944 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:39:19.763901 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/8f9fee4a-de24-49dc-b667-3dbc852d7ae3-original-pull-secret\") pod \"global-pull-secret-syncer-xjlbh\" (UID: \"8f9fee4a-de24-49dc-b667-3dbc852d7ae3\") " pod="kube-system/global-pull-secret-syncer-xjlbh" Jul 06 02:39:19.764426 ip-10-0-139-206 kubenswrapper[2557]: E0706 02:39:19.764015 2557 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 06 02:39:19.764426 ip-10-0-139-206 kubenswrapper[2557]: E0706 02:39:19.764031 2557 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 06 02:39:19.764426 ip-10-0-139-206 kubenswrapper[2557]: E0706 02:39:19.764047 2557 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 06 02:39:19.764426 ip-10-0-139-206 kubenswrapper[2557]: E0706 02:39:19.764058 2557 projected.go:196] Error preparing data for projected volume kube-api-access-t4wsh for pod openshift-network-diagnostics/network-check-target-l2btb: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 06 02:39:19.764426 ip-10-0-139-206 kubenswrapper[2557]: E0706 02:39:19.764082 2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8f9fee4a-de24-49dc-b667-3dbc852d7ae3-original-pull-secret podName:8f9fee4a-de24-49dc-b667-3dbc852d7ae3 nodeName:}" failed. No retries permitted until 2026-07-06 02:39:20.764063733 +0000 UTC m=+4.570349794 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/8f9fee4a-de24-49dc-b667-3dbc852d7ae3-original-pull-secret") pod "global-pull-secret-syncer-xjlbh" (UID: "8f9fee4a-de24-49dc-b667-3dbc852d7ae3") : object "kube-system"/"original-pull-secret" not registered Jul 06 02:39:19.764426 ip-10-0-139-206 kubenswrapper[2557]: E0706 02:39:19.764099 2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/c3ee201f-4897-4a18-bf37-f6c327d25e1e-kube-api-access-t4wsh podName:c3ee201f-4897-4a18-bf37-f6c327d25e1e nodeName:}" failed. No retries permitted until 2026-07-06 02:39:20.764090571 +0000 UTC m=+4.570376609 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-t4wsh" (UniqueName: "kubernetes.io/projected/c3ee201f-4897-4a18-bf37-f6c327d25e1e-kube-api-access-t4wsh") pod "network-check-target-l2btb" (UID: "c3ee201f-4897-4a18-bf37-f6c327d25e1e") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 06 02:39:19.864310 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:39:19.864277 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/26f69a47-e949-4e18-9d9c-6648a959e281-metrics-certs\") pod \"network-metrics-daemon-756n2\" (UID: \"26f69a47-e949-4e18-9d9c-6648a959e281\") " pod="openshift-multus/network-metrics-daemon-756n2" Jul 06 02:39:19.864516 ip-10-0-139-206 kubenswrapper[2557]: E0706 02:39:19.864403 2557 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 06 02:39:19.864516 ip-10-0-139-206 kubenswrapper[2557]: E0706 02:39:19.864464 2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/26f69a47-e949-4e18-9d9c-6648a959e281-metrics-certs podName:26f69a47-e949-4e18-9d9c-6648a959e281 nodeName:}" failed. No retries permitted until 2026-07-06 02:39:20.864445089 +0000 UTC m=+4.670731128 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/26f69a47-e949-4e18-9d9c-6648a959e281-metrics-certs") pod "network-metrics-daemon-756n2" (UID: "26f69a47-e949-4e18-9d9c-6648a959e281") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 06 02:39:20.093275 ip-10-0-139-206 kubenswrapper[2557]: W0706 02:39:20.093250 2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod00981958_29f3_4942_8ba6_0be9f03348a4.slice/crio-82e75f51ff049e697241555ecba7052a1568f5d1bff6259121c228b3ae97ac55 WatchSource:0}: Error finding container 82e75f51ff049e697241555ecba7052a1568f5d1bff6259121c228b3ae97ac55: Status 404 returned error can't find the container with id 82e75f51ff049e697241555ecba7052a1568f5d1bff6259121c228b3ae97ac55 Jul 06 02:39:20.097904 ip-10-0-139-206 kubenswrapper[2557]: W0706 02:39:20.097874 2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod29184621_7845_4a4a_9b81_04b439571ca3.slice/crio-8cd6d3a22071722abefa898a71668d5aa77742338e228b47966d0a3d8893eda3 WatchSource:0}: Error finding container 8cd6d3a22071722abefa898a71668d5aa77742338e228b47966d0a3d8893eda3: Status 404 returned error can't find the container with id 8cd6d3a22071722abefa898a71668d5aa77742338e228b47966d0a3d8893eda3 Jul 06 02:39:20.098569 ip-10-0-139-206 kubenswrapper[2557]: W0706 02:39:20.098548 2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podaf7822ef_5b14_477a_9fdd_5ef1fac67e23.slice/crio-a1f4dead7332792342c40105b56b64f6d1693832f4ac227d1b7a0eaf7b10901b WatchSource:0}: Error finding container a1f4dead7332792342c40105b56b64f6d1693832f4ac227d1b7a0eaf7b10901b: Status 404 returned error can't find the container with id a1f4dead7332792342c40105b56b64f6d1693832f4ac227d1b7a0eaf7b10901b Jul 06 02:39:20.184999 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:39:20.184937 2557 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-05 02:34:19 +0000 UTC" deadline="2028-03-28 10:15:27.468736754 +0000 UTC" Jul 06 02:39:20.184999 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:39:20.184995 2557 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="15151h36m7.283744041s" Jul 06 02:39:20.305069 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:39:20.305028 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-twqqj" event={"ID":"00981958-29f3-4942-8ba6-0be9f03348a4","Type":"ContainerStarted","Data":"82e75f51ff049e697241555ecba7052a1568f5d1bff6259121c228b3ae97ac55"} Jul 06 02:39:20.307124 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:39:20.307091 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-139-206.ec2.internal" event={"ID":"2e5d7d54b6363f2418e74312edd961df","Type":"ContainerStarted","Data":"d606d8b7f2d6489101db45568c200d19df95caae4318a7d05298d9be580c43fa"} Jul 06 02:39:20.308535 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:39:20.308508 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-fxqsb" event={"ID":"b6d9cfba-1e8c-48ce-a3ed-20464e26c44e","Type":"ContainerStarted","Data":"900fa95b6f47b7e2db4567568347b4cf27b783e61785414b0979a7a001421216"} Jul 06 02:39:20.309951 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:39:20.309923 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-lwmh8" event={"ID":"5b80db07-d76f-4470-a3d5-3176fdf63aa2","Type":"ContainerStarted","Data":"70bd3548c6f7be8e910b443258f9719c8a51c5a5ad3715d3fb041fc6be3f1b67"} Jul 06 02:39:20.311694 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:39:20.311665 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-7nlb2" event={"ID":"29184621-7845-4a4a-9b81-04b439571ca3","Type":"ContainerStarted","Data":"8cd6d3a22071722abefa898a71668d5aa77742338e228b47966d0a3d8893eda3"} Jul 06 02:39:20.312783 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:39:20.312762 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-bhp46" event={"ID":"dd0a7167-560c-4195-857e-b69abeede707","Type":"ContainerStarted","Data":"e7ac6c9d95dbce4f30e7a7ff72c22f978f3b3a225b1c51e7952db71fddf3fa91"} Jul 06 02:39:20.313946 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:39:20.313920 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-hgnzv" event={"ID":"1b70e00f-2790-4bb2-8fc6-4ce6fb7e5ab1","Type":"ContainerStarted","Data":"be23efd7bead443b6ec15d26efa0d23f7e0210152c53e00665d5a1e5b6a05d33"} Jul 06 02:39:20.315278 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:39:20.315253 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-cw2f7" event={"ID":"efcf5e11-1e5d-4e2a-9b38-303a8f0787c4","Type":"ContainerStarted","Data":"2a68fe828854199e5c6afe73ebd688da1337bed35b2fd5c35ba7e9299ed5e0a1"} Jul 06 02:39:20.316394 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:39:20.316371 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-84mq4" event={"ID":"3c23051e-1536-45e2-8a89-dc077a09d982","Type":"ContainerStarted","Data":"d2980d23a7243d404c51ba8f477be8ef9830c115b3388cd9e4a3072c49c521ac"} Jul 06 02:39:20.317437 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:39:20.317416 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-v4pnk" event={"ID":"af7822ef-5b14-477a-9fdd-5ef1fac67e23","Type":"ContainerStarted","Data":"a1f4dead7332792342c40105b56b64f6d1693832f4ac227d1b7a0eaf7b10901b"} Jul 06 02:39:20.319808 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:39:20.319769 2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-139-206.ec2.internal" podStartSLOduration=2.31975777 podStartE2EDuration="2.31975777s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-06 02:39:18 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 02:39:20.319375783 +0000 UTC m=+4.125661840" watchObservedRunningTime="2026-07-06 02:39:20.31975777 +0000 UTC m=+4.126043827" Jul 06 02:39:20.771493 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:39:20.771454 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-t4wsh\" (UniqueName: \"kubernetes.io/projected/c3ee201f-4897-4a18-bf37-f6c327d25e1e-kube-api-access-t4wsh\") pod \"network-check-target-l2btb\" (UID: \"c3ee201f-4897-4a18-bf37-f6c327d25e1e\") " pod="openshift-network-diagnostics/network-check-target-l2btb" Jul 06 02:39:20.772104 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:39:20.771511 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/8f9fee4a-de24-49dc-b667-3dbc852d7ae3-original-pull-secret\") pod \"global-pull-secret-syncer-xjlbh\" (UID: \"8f9fee4a-de24-49dc-b667-3dbc852d7ae3\") " pod="kube-system/global-pull-secret-syncer-xjlbh" Jul 06 02:39:20.772104 ip-10-0-139-206 kubenswrapper[2557]: E0706 02:39:20.771681 2557 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 06 02:39:20.772104 ip-10-0-139-206 kubenswrapper[2557]: E0706 02:39:20.771751 2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8f9fee4a-de24-49dc-b667-3dbc852d7ae3-original-pull-secret podName:8f9fee4a-de24-49dc-b667-3dbc852d7ae3 nodeName:}" failed. No retries permitted until 2026-07-06 02:39:22.7717327 +0000 UTC m=+6.578018739 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/8f9fee4a-de24-49dc-b667-3dbc852d7ae3-original-pull-secret") pod "global-pull-secret-syncer-xjlbh" (UID: "8f9fee4a-de24-49dc-b667-3dbc852d7ae3") : object "kube-system"/"original-pull-secret" not registered Jul 06 02:39:20.772293 ip-10-0-139-206 kubenswrapper[2557]: E0706 02:39:20.772175 2557 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 06 02:39:20.772293 ip-10-0-139-206 kubenswrapper[2557]: E0706 02:39:20.772192 2557 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 06 02:39:20.772293 ip-10-0-139-206 kubenswrapper[2557]: E0706 02:39:20.772205 2557 projected.go:196] Error preparing data for projected volume kube-api-access-t4wsh for pod openshift-network-diagnostics/network-check-target-l2btb: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 06 02:39:20.772293 ip-10-0-139-206 kubenswrapper[2557]: E0706 02:39:20.772249 2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/c3ee201f-4897-4a18-bf37-f6c327d25e1e-kube-api-access-t4wsh podName:c3ee201f-4897-4a18-bf37-f6c327d25e1e nodeName:}" failed. No retries permitted until 2026-07-06 02:39:22.772234209 +0000 UTC m=+6.578520260 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-t4wsh" (UniqueName: "kubernetes.io/projected/c3ee201f-4897-4a18-bf37-f6c327d25e1e-kube-api-access-t4wsh") pod "network-check-target-l2btb" (UID: "c3ee201f-4897-4a18-bf37-f6c327d25e1e") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 06 02:39:20.872625 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:39:20.872587 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/26f69a47-e949-4e18-9d9c-6648a959e281-metrics-certs\") pod \"network-metrics-daemon-756n2\" (UID: \"26f69a47-e949-4e18-9d9c-6648a959e281\") " pod="openshift-multus/network-metrics-daemon-756n2" Jul 06 02:39:20.872814 ip-10-0-139-206 kubenswrapper[2557]: E0706 02:39:20.872750 2557 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 06 02:39:20.872884 ip-10-0-139-206 kubenswrapper[2557]: E0706 02:39:20.872818 2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/26f69a47-e949-4e18-9d9c-6648a959e281-metrics-certs podName:26f69a47-e949-4e18-9d9c-6648a959e281 nodeName:}" failed. No retries permitted until 2026-07-06 02:39:22.872798499 +0000 UTC m=+6.679084540 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/26f69a47-e949-4e18-9d9c-6648a959e281-metrics-certs") pod "network-metrics-daemon-756n2" (UID: "26f69a47-e949-4e18-9d9c-6648a959e281") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 06 02:39:21.185431 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:39:21.185357 2557 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-05 02:34:19 +0000 UTC" deadline="2027-12-08 15:42:32.538685661 +0000 UTC" Jul 06 02:39:21.185431 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:39:21.185397 2557 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="12493h3m11.353292769s" Jul 06 02:39:21.298602 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:39:21.298567 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-xjlbh" Jul 06 02:39:21.298767 ip-10-0-139-206 kubenswrapper[2557]: E0706 02:39:21.298706 2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-xjlbh" podUID="8f9fee4a-de24-49dc-b667-3dbc852d7ae3" Jul 06 02:39:21.299109 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:39:21.299089 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-756n2" Jul 06 02:39:21.299224 ip-10-0-139-206 kubenswrapper[2557]: E0706 02:39:21.299200 2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-756n2" podUID="26f69a47-e949-4e18-9d9c-6648a959e281" Jul 06 02:39:21.299288 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:39:21.299275 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-l2btb" Jul 06 02:39:21.299386 ip-10-0-139-206 kubenswrapper[2557]: E0706 02:39:21.299365 2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-l2btb" podUID="c3ee201f-4897-4a18-bf37-f6c327d25e1e" Jul 06 02:39:21.363338 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:39:21.363300 2557 generic.go:356] "Generic (PLEG): container finished" podID="d4d0c49eb31a9149fde1244abdabe200" containerID="9a788b6c9d6bf080953969865549689cebe61622c6f848b4e2224bd81dae6e1c" exitCode=0 Jul 06 02:39:21.363535 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:39:21.363452 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-206.ec2.internal" event={"ID":"d4d0c49eb31a9149fde1244abdabe200","Type":"ContainerDied","Data":"9a788b6c9d6bf080953969865549689cebe61622c6f848b4e2224bd81dae6e1c"} Jul 06 02:39:22.376724 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:39:22.376679 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-206.ec2.internal" event={"ID":"d4d0c49eb31a9149fde1244abdabe200","Type":"ContainerStarted","Data":"c3219e0fcb79ce50ed7afb2c05f2bc9233f4852f579db18b7cfb48c915d1a972"} Jul 06 02:39:22.789608 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:39:22.789508 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-t4wsh\" (UniqueName: \"kubernetes.io/projected/c3ee201f-4897-4a18-bf37-f6c327d25e1e-kube-api-access-t4wsh\") pod \"network-check-target-l2btb\" (UID: \"c3ee201f-4897-4a18-bf37-f6c327d25e1e\") " pod="openshift-network-diagnostics/network-check-target-l2btb" Jul 06 02:39:22.789608 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:39:22.789569 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/8f9fee4a-de24-49dc-b667-3dbc852d7ae3-original-pull-secret\") pod \"global-pull-secret-syncer-xjlbh\" (UID: \"8f9fee4a-de24-49dc-b667-3dbc852d7ae3\") " pod="kube-system/global-pull-secret-syncer-xjlbh" Jul 06 02:39:22.789833 ip-10-0-139-206 kubenswrapper[2557]: E0706 02:39:22.789721 2557 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 06 02:39:22.789833 ip-10-0-139-206 kubenswrapper[2557]: E0706 02:39:22.789787 2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8f9fee4a-de24-49dc-b667-3dbc852d7ae3-original-pull-secret podName:8f9fee4a-de24-49dc-b667-3dbc852d7ae3 nodeName:}" failed. No retries permitted until 2026-07-06 02:39:26.789768921 +0000 UTC m=+10.596054962 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/8f9fee4a-de24-49dc-b667-3dbc852d7ae3-original-pull-secret") pod "global-pull-secret-syncer-xjlbh" (UID: "8f9fee4a-de24-49dc-b667-3dbc852d7ae3") : object "kube-system"/"original-pull-secret" not registered Jul 06 02:39:22.790235 ip-10-0-139-206 kubenswrapper[2557]: E0706 02:39:22.790211 2557 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 06 02:39:22.790235 ip-10-0-139-206 kubenswrapper[2557]: E0706 02:39:22.790238 2557 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 06 02:39:22.790404 ip-10-0-139-206 kubenswrapper[2557]: E0706 02:39:22.790251 2557 projected.go:196] Error preparing data for projected volume kube-api-access-t4wsh for pod openshift-network-diagnostics/network-check-target-l2btb: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 06 02:39:22.790404 ip-10-0-139-206 kubenswrapper[2557]: E0706 02:39:22.790300 2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/c3ee201f-4897-4a18-bf37-f6c327d25e1e-kube-api-access-t4wsh podName:c3ee201f-4897-4a18-bf37-f6c327d25e1e nodeName:}" failed. No retries permitted until 2026-07-06 02:39:26.790284586 +0000 UTC m=+10.596570623 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-t4wsh" (UniqueName: "kubernetes.io/projected/c3ee201f-4897-4a18-bf37-f6c327d25e1e-kube-api-access-t4wsh") pod "network-check-target-l2btb" (UID: "c3ee201f-4897-4a18-bf37-f6c327d25e1e") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 06 02:39:22.891517 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:39:22.890881 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/26f69a47-e949-4e18-9d9c-6648a959e281-metrics-certs\") pod \"network-metrics-daemon-756n2\" (UID: \"26f69a47-e949-4e18-9d9c-6648a959e281\") " pod="openshift-multus/network-metrics-daemon-756n2" Jul 06 02:39:22.891517 ip-10-0-139-206 kubenswrapper[2557]: E0706 02:39:22.891084 2557 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 06 02:39:22.891517 ip-10-0-139-206 kubenswrapper[2557]: E0706 02:39:22.891149 2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/26f69a47-e949-4e18-9d9c-6648a959e281-metrics-certs podName:26f69a47-e949-4e18-9d9c-6648a959e281 nodeName:}" failed. No retries permitted until 2026-07-06 02:39:26.891131976 +0000 UTC m=+10.697418016 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/26f69a47-e949-4e18-9d9c-6648a959e281-metrics-certs") pod "network-metrics-daemon-756n2" (UID: "26f69a47-e949-4e18-9d9c-6648a959e281") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 06 02:39:23.296419 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:39:23.296360 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-xjlbh" Jul 06 02:39:23.296599 ip-10-0-139-206 kubenswrapper[2557]: E0706 02:39:23.296492 2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-xjlbh" podUID="8f9fee4a-de24-49dc-b667-3dbc852d7ae3" Jul 06 02:39:23.296599 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:39:23.296360 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-756n2" Jul 06 02:39:23.296599 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:39:23.296557 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-l2btb" Jul 06 02:39:23.296748 ip-10-0-139-206 kubenswrapper[2557]: E0706 02:39:23.296659 2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-756n2" podUID="26f69a47-e949-4e18-9d9c-6648a959e281" Jul 06 02:39:23.296748 ip-10-0-139-206 kubenswrapper[2557]: E0706 02:39:23.296731 2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-l2btb" podUID="c3ee201f-4897-4a18-bf37-f6c327d25e1e" Jul 06 02:39:25.300338 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:39:25.300281 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-l2btb" Jul 06 02:39:25.300794 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:39:25.300409 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-xjlbh" Jul 06 02:39:25.300794 ip-10-0-139-206 kubenswrapper[2557]: E0706 02:39:25.300420 2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-l2btb" podUID="c3ee201f-4897-4a18-bf37-f6c327d25e1e" Jul 06 02:39:25.300794 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:39:25.300471 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-756n2" Jul 06 02:39:25.300794 ip-10-0-139-206 kubenswrapper[2557]: E0706 02:39:25.300567 2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-xjlbh" podUID="8f9fee4a-de24-49dc-b667-3dbc852d7ae3" Jul 06 02:39:25.300794 ip-10-0-139-206 kubenswrapper[2557]: E0706 02:39:25.300656 2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-756n2" podUID="26f69a47-e949-4e18-9d9c-6648a959e281" Jul 06 02:39:26.824331 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:39:26.824292 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-t4wsh\" (UniqueName: \"kubernetes.io/projected/c3ee201f-4897-4a18-bf37-f6c327d25e1e-kube-api-access-t4wsh\") pod \"network-check-target-l2btb\" (UID: \"c3ee201f-4897-4a18-bf37-f6c327d25e1e\") " pod="openshift-network-diagnostics/network-check-target-l2btb" Jul 06 02:39:26.824822 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:39:26.824358 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/8f9fee4a-de24-49dc-b667-3dbc852d7ae3-original-pull-secret\") pod \"global-pull-secret-syncer-xjlbh\" (UID: \"8f9fee4a-de24-49dc-b667-3dbc852d7ae3\") " pod="kube-system/global-pull-secret-syncer-xjlbh" Jul 06 02:39:26.824822 ip-10-0-139-206 kubenswrapper[2557]: E0706 02:39:26.824553 2557 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 06 02:39:26.824822 ip-10-0-139-206 kubenswrapper[2557]: E0706 02:39:26.824619 2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8f9fee4a-de24-49dc-b667-3dbc852d7ae3-original-pull-secret podName:8f9fee4a-de24-49dc-b667-3dbc852d7ae3 nodeName:}" failed. No retries permitted until 2026-07-06 02:39:34.824600927 +0000 UTC m=+18.630886962 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/8f9fee4a-de24-49dc-b667-3dbc852d7ae3-original-pull-secret") pod "global-pull-secret-syncer-xjlbh" (UID: "8f9fee4a-de24-49dc-b667-3dbc852d7ae3") : object "kube-system"/"original-pull-secret" not registered Jul 06 02:39:26.825087 ip-10-0-139-206 kubenswrapper[2557]: E0706 02:39:26.825061 2557 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 06 02:39:26.825087 ip-10-0-139-206 kubenswrapper[2557]: E0706 02:39:26.825088 2557 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 06 02:39:26.825300 ip-10-0-139-206 kubenswrapper[2557]: E0706 02:39:26.825104 2557 projected.go:196] Error preparing data for projected volume kube-api-access-t4wsh for pod openshift-network-diagnostics/network-check-target-l2btb: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 06 02:39:26.825300 ip-10-0-139-206 kubenswrapper[2557]: E0706 02:39:26.825155 2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/c3ee201f-4897-4a18-bf37-f6c327d25e1e-kube-api-access-t4wsh podName:c3ee201f-4897-4a18-bf37-f6c327d25e1e nodeName:}" failed. No retries permitted until 2026-07-06 02:39:34.82514 +0000 UTC m=+18.631426049 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-t4wsh" (UniqueName: "kubernetes.io/projected/c3ee201f-4897-4a18-bf37-f6c327d25e1e-kube-api-access-t4wsh") pod "network-check-target-l2btb" (UID: "c3ee201f-4897-4a18-bf37-f6c327d25e1e") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 06 02:39:26.925306 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:39:26.925249 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/26f69a47-e949-4e18-9d9c-6648a959e281-metrics-certs\") pod \"network-metrics-daemon-756n2\" (UID: \"26f69a47-e949-4e18-9d9c-6648a959e281\") " pod="openshift-multus/network-metrics-daemon-756n2" Jul 06 02:39:26.925497 ip-10-0-139-206 kubenswrapper[2557]: E0706 02:39:26.925420 2557 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 06 02:39:26.925497 ip-10-0-139-206 kubenswrapper[2557]: E0706 02:39:26.925497 2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/26f69a47-e949-4e18-9d9c-6648a959e281-metrics-certs podName:26f69a47-e949-4e18-9d9c-6648a959e281 nodeName:}" failed. No retries permitted until 2026-07-06 02:39:34.925476718 +0000 UTC m=+18.731762781 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/26f69a47-e949-4e18-9d9c-6648a959e281-metrics-certs") pod "network-metrics-daemon-756n2" (UID: "26f69a47-e949-4e18-9d9c-6648a959e281") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 06 02:39:27.297431 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:39:27.296939 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-756n2" Jul 06 02:39:27.297431 ip-10-0-139-206 kubenswrapper[2557]: E0706 02:39:27.297084 2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-756n2" podUID="26f69a47-e949-4e18-9d9c-6648a959e281" Jul 06 02:39:27.298079 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:39:27.297758 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-xjlbh" Jul 06 02:39:27.298079 ip-10-0-139-206 kubenswrapper[2557]: E0706 02:39:27.297881 2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-xjlbh" podUID="8f9fee4a-de24-49dc-b667-3dbc852d7ae3" Jul 06 02:39:27.298079 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:39:27.297957 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-l2btb" Jul 06 02:39:27.298079 ip-10-0-139-206 kubenswrapper[2557]: E0706 02:39:27.298032 2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-l2btb" podUID="c3ee201f-4897-4a18-bf37-f6c327d25e1e" Jul 06 02:39:29.296552 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:39:29.296520 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-xjlbh" Jul 06 02:39:29.297042 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:39:29.296524 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-756n2" Jul 06 02:39:29.297042 ip-10-0-139-206 kubenswrapper[2557]: E0706 02:39:29.296654 2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-xjlbh" podUID="8f9fee4a-de24-49dc-b667-3dbc852d7ae3" Jul 06 02:39:29.297042 ip-10-0-139-206 kubenswrapper[2557]: E0706 02:39:29.296740 2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-756n2" podUID="26f69a47-e949-4e18-9d9c-6648a959e281" Jul 06 02:39:29.297042 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:39:29.296520 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-l2btb" Jul 06 02:39:29.297042 ip-10-0-139-206 kubenswrapper[2557]: E0706 02:39:29.296843 2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-l2btb" podUID="c3ee201f-4897-4a18-bf37-f6c327d25e1e" Jul 06 02:39:31.296606 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:39:31.296569 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-756n2" Jul 06 02:39:31.297039 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:39:31.296573 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-xjlbh" Jul 06 02:39:31.297039 ip-10-0-139-206 kubenswrapper[2557]: E0706 02:39:31.296716 2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-756n2" podUID="26f69a47-e949-4e18-9d9c-6648a959e281" Jul 06 02:39:31.297039 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:39:31.296569 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-l2btb" Jul 06 02:39:31.297039 ip-10-0-139-206 kubenswrapper[2557]: E0706 02:39:31.296783 2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-xjlbh" podUID="8f9fee4a-de24-49dc-b667-3dbc852d7ae3" Jul 06 02:39:31.297039 ip-10-0-139-206 kubenswrapper[2557]: E0706 02:39:31.296904 2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-l2btb" podUID="c3ee201f-4897-4a18-bf37-f6c327d25e1e" Jul 06 02:39:33.296512 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:39:33.296475 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-xjlbh" Jul 06 02:39:33.296512 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:39:33.296505 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-l2btb" Jul 06 02:39:33.297041 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:39:33.296475 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-756n2" Jul 06 02:39:33.297041 ip-10-0-139-206 kubenswrapper[2557]: E0706 02:39:33.296597 2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-xjlbh" podUID="8f9fee4a-de24-49dc-b667-3dbc852d7ae3" Jul 06 02:39:33.297041 ip-10-0-139-206 kubenswrapper[2557]: E0706 02:39:33.296678 2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-756n2" podUID="26f69a47-e949-4e18-9d9c-6648a959e281" Jul 06 02:39:33.297041 ip-10-0-139-206 kubenswrapper[2557]: E0706 02:39:33.296751 2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-l2btb" podUID="c3ee201f-4897-4a18-bf37-f6c327d25e1e" Jul 06 02:39:34.886710 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:39:34.886674 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-t4wsh\" (UniqueName: \"kubernetes.io/projected/c3ee201f-4897-4a18-bf37-f6c327d25e1e-kube-api-access-t4wsh\") pod \"network-check-target-l2btb\" (UID: \"c3ee201f-4897-4a18-bf37-f6c327d25e1e\") " pod="openshift-network-diagnostics/network-check-target-l2btb" Jul 06 02:39:34.887215 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:39:34.886720 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/8f9fee4a-de24-49dc-b667-3dbc852d7ae3-original-pull-secret\") pod \"global-pull-secret-syncer-xjlbh\" (UID: \"8f9fee4a-de24-49dc-b667-3dbc852d7ae3\") " pod="kube-system/global-pull-secret-syncer-xjlbh" Jul 06 02:39:34.887215 ip-10-0-139-206 kubenswrapper[2557]: E0706 02:39:34.886821 2557 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 06 02:39:34.887215 ip-10-0-139-206 kubenswrapper[2557]: E0706 02:39:34.886870 2557 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 06 02:39:34.887215 ip-10-0-139-206 kubenswrapper[2557]: E0706 02:39:34.886890 2557 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 06 02:39:34.887215 ip-10-0-139-206 kubenswrapper[2557]: E0706 02:39:34.886901 2557 projected.go:196] Error preparing data for projected volume kube-api-access-t4wsh for pod openshift-network-diagnostics/network-check-target-l2btb: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 06 02:39:34.887215 ip-10-0-139-206 kubenswrapper[2557]: E0706 02:39:34.886877 2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8f9fee4a-de24-49dc-b667-3dbc852d7ae3-original-pull-secret podName:8f9fee4a-de24-49dc-b667-3dbc852d7ae3 nodeName:}" failed. No retries permitted until 2026-07-06 02:39:50.886862785 +0000 UTC m=+34.693148821 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/8f9fee4a-de24-49dc-b667-3dbc852d7ae3-original-pull-secret") pod "global-pull-secret-syncer-xjlbh" (UID: "8f9fee4a-de24-49dc-b667-3dbc852d7ae3") : object "kube-system"/"original-pull-secret" not registered Jul 06 02:39:34.887215 ip-10-0-139-206 kubenswrapper[2557]: E0706 02:39:34.886968 2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/c3ee201f-4897-4a18-bf37-f6c327d25e1e-kube-api-access-t4wsh podName:c3ee201f-4897-4a18-bf37-f6c327d25e1e nodeName:}" failed. No retries permitted until 2026-07-06 02:39:50.886946734 +0000 UTC m=+34.693232784 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-t4wsh" (UniqueName: "kubernetes.io/projected/c3ee201f-4897-4a18-bf37-f6c327d25e1e-kube-api-access-t4wsh") pod "network-check-target-l2btb" (UID: "c3ee201f-4897-4a18-bf37-f6c327d25e1e") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 06 02:39:34.987130 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:39:34.987095 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/26f69a47-e949-4e18-9d9c-6648a959e281-metrics-certs\") pod \"network-metrics-daemon-756n2\" (UID: \"26f69a47-e949-4e18-9d9c-6648a959e281\") " pod="openshift-multus/network-metrics-daemon-756n2" Jul 06 02:39:34.987301 ip-10-0-139-206 kubenswrapper[2557]: E0706 02:39:34.987243 2557 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 06 02:39:34.987371 ip-10-0-139-206 kubenswrapper[2557]: E0706 02:39:34.987305 2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/26f69a47-e949-4e18-9d9c-6648a959e281-metrics-certs podName:26f69a47-e949-4e18-9d9c-6648a959e281 nodeName:}" failed. No retries permitted until 2026-07-06 02:39:50.987289829 +0000 UTC m=+34.793575872 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/26f69a47-e949-4e18-9d9c-6648a959e281-metrics-certs") pod "network-metrics-daemon-756n2" (UID: "26f69a47-e949-4e18-9d9c-6648a959e281") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 06 02:39:35.296112 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:39:35.296024 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-756n2" Jul 06 02:39:35.296112 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:39:35.296072 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-l2btb" Jul 06 02:39:35.296387 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:39:35.296024 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-xjlbh" Jul 06 02:39:35.296387 ip-10-0-139-206 kubenswrapper[2557]: E0706 02:39:35.296163 2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-756n2" podUID="26f69a47-e949-4e18-9d9c-6648a959e281" Jul 06 02:39:35.296387 ip-10-0-139-206 kubenswrapper[2557]: E0706 02:39:35.296239 2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-xjlbh" podUID="8f9fee4a-de24-49dc-b667-3dbc852d7ae3" Jul 06 02:39:35.296387 ip-10-0-139-206 kubenswrapper[2557]: E0706 02:39:35.296289 2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-l2btb" podUID="c3ee201f-4897-4a18-bf37-f6c327d25e1e" Jul 06 02:39:37.296718 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:39:37.296678 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-xjlbh" Jul 06 02:39:37.297200 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:39:37.296788 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-756n2" Jul 06 02:39:37.297200 ip-10-0-139-206 kubenswrapper[2557]: E0706 02:39:37.296805 2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-xjlbh" podUID="8f9fee4a-de24-49dc-b667-3dbc852d7ae3" Jul 06 02:39:37.297200 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:39:37.296822 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-l2btb" Jul 06 02:39:37.297200 ip-10-0-139-206 kubenswrapper[2557]: E0706 02:39:37.296908 2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-756n2" podUID="26f69a47-e949-4e18-9d9c-6648a959e281" Jul 06 02:39:37.297200 ip-10-0-139-206 kubenswrapper[2557]: E0706 02:39:37.296990 2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-l2btb" podUID="c3ee201f-4897-4a18-bf37-f6c327d25e1e" Jul 06 02:39:39.296680 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:39:39.296499 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-756n2" Jul 06 02:39:39.297481 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:39:39.296499 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-xjlbh" Jul 06 02:39:39.297481 ip-10-0-139-206 kubenswrapper[2557]: E0706 02:39:39.296775 2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-756n2" podUID="26f69a47-e949-4e18-9d9c-6648a959e281" Jul 06 02:39:39.297481 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:39:39.296561 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-l2btb" Jul 06 02:39:39.297481 ip-10-0-139-206 kubenswrapper[2557]: E0706 02:39:39.296854 2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-xjlbh" podUID="8f9fee4a-de24-49dc-b667-3dbc852d7ae3" Jul 06 02:39:39.297481 ip-10-0-139-206 kubenswrapper[2557]: E0706 02:39:39.296908 2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-l2btb" podUID="c3ee201f-4897-4a18-bf37-f6c327d25e1e" Jul 06 02:39:39.412669 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:39:39.412465 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-lwmh8" event={"ID":"5b80db07-d76f-4470-a3d5-3176fdf63aa2","Type":"ContainerStarted","Data":"6e5f03028a47d71c9c90ac2e7bc234f284af898f591a09451e0f2c1acdd6217d"} Jul 06 02:39:39.413943 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:39:39.413918 2557 generic.go:356] "Generic (PLEG): container finished" podID="29184621-7845-4a4a-9b81-04b439571ca3" containerID="65b4bae5c47f37b647a7ed448d452c3b46c8129af0b62327717af68a297807f3" exitCode=0 Jul 06 02:39:39.414068 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:39:39.414032 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-7nlb2" event={"ID":"29184621-7845-4a4a-9b81-04b439571ca3","Type":"ContainerDied","Data":"65b4bae5c47f37b647a7ed448d452c3b46c8129af0b62327717af68a297807f3"} Jul 06 02:39:39.415509 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:39:39.415485 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-bhp46" event={"ID":"dd0a7167-560c-4195-857e-b69abeede707","Type":"ContainerStarted","Data":"f2c33c417f74137e1e6c21b034c27c30ed6a596f9b2e62e28093c5658619ae2d"} Jul 06 02:39:39.417316 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:39:39.417299 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-cw2f7_efcf5e11-1e5d-4e2a-9b38-303a8f0787c4/ovn-acl-logging/0.log" Jul 06 02:39:39.417799 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:39:39.417775 2557 generic.go:356] "Generic (PLEG): container finished" podID="efcf5e11-1e5d-4e2a-9b38-303a8f0787c4" containerID="8f15dc449d509317d0b451f4fa4dfa5e0a57c9753d4b12592511b5c0528fb880" exitCode=1 Jul 06 02:39:39.417879 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:39:39.417846 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-cw2f7" event={"ID":"efcf5e11-1e5d-4e2a-9b38-303a8f0787c4","Type":"ContainerStarted","Data":"e61cd7feb7e5a51bd297471373451772154f75aac5d95aed5d379fe2b7408426"} Jul 06 02:39:39.417931 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:39:39.417923 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-cw2f7" event={"ID":"efcf5e11-1e5d-4e2a-9b38-303a8f0787c4","Type":"ContainerDied","Data":"8f15dc449d509317d0b451f4fa4dfa5e0a57c9753d4b12592511b5c0528fb880"} Jul 06 02:39:39.417986 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:39:39.417954 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-cw2f7" event={"ID":"efcf5e11-1e5d-4e2a-9b38-303a8f0787c4","Type":"ContainerStarted","Data":"900b847adcf8349b3864b9b13b26f2599c94a665d33b8704e75624bb8319bd5e"} Jul 06 02:39:39.419507 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:39:39.419475 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-84mq4" event={"ID":"3c23051e-1536-45e2-8a89-dc077a09d982","Type":"ContainerStarted","Data":"4fde32665f255e5e619999e6352123c8676140b97eab1c67e2fcdfaae919b9dd"} Jul 06 02:39:39.421167 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:39:39.421139 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-v4pnk" event={"ID":"af7822ef-5b14-477a-9fdd-5ef1fac67e23","Type":"ContainerStarted","Data":"5154431ea4602e66c597cc70a07c8acce011041c5841bb1bdeaa7bfd7821b954"} Jul 06 02:39:39.422777 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:39:39.422749 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-twqqj" event={"ID":"00981958-29f3-4942-8ba6-0be9f03348a4","Type":"ContainerStarted","Data":"04ad227c4d7d74a1b5fb34cd688e1d24d814ce726c7e6a9f91066cf412a45d49"} Jul 06 02:39:39.424317 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:39:39.424295 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-fxqsb" event={"ID":"b6d9cfba-1e8c-48ce-a3ed-20464e26c44e","Type":"ContainerStarted","Data":"0a3af428c418d0d1f6f76ee6280dc6e5328ba187b309cf9f12e9da8640cfa34c"} Jul 06 02:39:39.426314 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:39:39.426271 2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/node-ca-lwmh8" podStartSLOduration=3.741213805 podStartE2EDuration="22.426259768s" totalImagesPullingTime="18.685045963s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-06 02:39:17 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 02:39:39.426165249 +0000 UTC m=+23.232451308" watchObservedRunningTime="2026-07-06 02:39:39.426259768 +0000 UTC m=+23.232545825" Jul 06 02:39:39.426436 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:39:39.426367 2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-206.ec2.internal" podStartSLOduration=21.426360774 podStartE2EDuration="21.426360774s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-06 02:39:18 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 02:39:22.392164846 +0000 UTC m=+6.198450906" watchObservedRunningTime="2026-07-06 02:39:39.426360774 +0000 UTC m=+23.232646827" Jul 06 02:39:39.444673 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:39:39.444628 2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-fxqsb" podStartSLOduration=3.757503597 podStartE2EDuration="22.444608835s" totalImagesPullingTime="18.687105238s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-06 02:39:17 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 02:39:39.4445419 +0000 UTC m=+23.250827959" watchObservedRunningTime="2026-07-06 02:39:39.444608835 +0000 UTC m=+23.250894895" Jul 06 02:39:39.457657 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:39:39.457603 2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/konnectivity-agent-twqqj" podStartSLOduration=7.9995213750000005 podStartE2EDuration="22.457583355s" totalImagesPullingTime="14.45806198s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 02:39:17 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 02:39:39.457238262 +0000 UTC m=+23.263524319" watchObservedRunningTime="2026-07-06 02:39:39.457583355 +0000 UTC m=+23.263869414" Jul 06 02:39:39.470704 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:39:39.470661 2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/node-resolver-bhp46" podStartSLOduration=3.791289885 podStartE2EDuration="22.470643293s" totalImagesPullingTime="18.679353408s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-06 02:39:17 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 02:39:39.470514464 +0000 UTC m=+23.276800526" watchObservedRunningTime="2026-07-06 02:39:39.470643293 +0000 UTC m=+23.276929350" Jul 06 02:39:39.505527 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:39:39.505323 2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-v4pnk" podStartSLOduration=3.781760747 podStartE2EDuration="22.505307152s" totalImagesPullingTime="18.723546405s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-06 02:39:17 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 02:39:39.504772863 +0000 UTC m=+23.311058924" watchObservedRunningTime="2026-07-06 02:39:39.505307152 +0000 UTC m=+23.311593210" Jul 06 02:39:40.246611 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:39:40.246584 2557 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock" Jul 06 02:39:40.428580 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:39:40.428488 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-hgnzv" event={"ID":"1b70e00f-2790-4bb2-8fc6-4ce6fb7e5ab1","Type":"ContainerStarted","Data":"6259efc2926aaff062ced7c775bfa4eb44f94a019696678c1cd9e32f70ef2887"} Jul 06 02:39:40.431439 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:39:40.431415 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-cw2f7_efcf5e11-1e5d-4e2a-9b38-303a8f0787c4/ovn-acl-logging/0.log" Jul 06 02:39:40.431806 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:39:40.431784 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-cw2f7" event={"ID":"efcf5e11-1e5d-4e2a-9b38-303a8f0787c4","Type":"ContainerStarted","Data":"422f34db75157742e64a8c6161f6a2fd5912b44d0ba2cf54399757d0a95e355e"} Jul 06 02:39:40.431867 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:39:40.431854 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-cw2f7" event={"ID":"efcf5e11-1e5d-4e2a-9b38-303a8f0787c4","Type":"ContainerStarted","Data":"d43b1c6af740f65a48bc42e724a59e00d4ea9635f8e4e47afbbb192478a7b78d"} Jul 06 02:39:40.431919 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:39:40.431907 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-cw2f7" event={"ID":"efcf5e11-1e5d-4e2a-9b38-303a8f0787c4","Type":"ContainerStarted","Data":"49b44125be166ef89cb12caa053e595a5d7fced8ed64cc0d4f2d1da45cd7018c"} Jul 06 02:39:40.433401 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:39:40.433375 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-84mq4" event={"ID":"3c23051e-1536-45e2-8a89-dc077a09d982","Type":"ContainerStarted","Data":"064641c5acff13c713320135a5be62a668e38d26f47141b0309c5ac5c2171c36"} Jul 06 02:39:40.442014 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:39:40.441966 2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-hgnzv" podStartSLOduration=4.758154998 podStartE2EDuration="23.441949832s" totalImagesPullingTime="18.683794834s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-06 02:39:17 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 02:39:40.441618672 +0000 UTC m=+24.247904741" watchObservedRunningTime="2026-07-06 02:39:40.441949832 +0000 UTC m=+24.248235890" Jul 06 02:39:41.218523 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:39:41.218410 2557 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-07-06T02:39:40.24660449Z","UUID":"c83c8327-80ed-414b-a975-537a6691ed35","Handler":null,"Name":"","Endpoint":""} Jul 06 02:39:41.220010 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:39:41.219988 2557 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0 Jul 06 02:39:41.220142 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:39:41.220032 2557 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock Jul 06 02:39:41.296154 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:39:41.296111 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-xjlbh" Jul 06 02:39:41.296419 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:39:41.296161 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-l2btb" Jul 06 02:39:41.296419 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:39:41.296233 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-756n2" Jul 06 02:39:41.296419 ip-10-0-139-206 kubenswrapper[2557]: E0706 02:39:41.296250 2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-xjlbh" podUID="8f9fee4a-de24-49dc-b667-3dbc852d7ae3" Jul 06 02:39:41.296419 ip-10-0-139-206 kubenswrapper[2557]: E0706 02:39:41.296342 2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-756n2" podUID="26f69a47-e949-4e18-9d9c-6648a959e281" Jul 06 02:39:41.296636 ip-10-0-139-206 kubenswrapper[2557]: E0706 02:39:41.296451 2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-l2btb" podUID="c3ee201f-4897-4a18-bf37-f6c327d25e1e" Jul 06 02:39:42.442575 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:39:42.442398 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-cw2f7_efcf5e11-1e5d-4e2a-9b38-303a8f0787c4/ovn-acl-logging/0.log" Jul 06 02:39:42.443108 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:39:42.443022 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-cw2f7" event={"ID":"efcf5e11-1e5d-4e2a-9b38-303a8f0787c4","Type":"ContainerStarted","Data":"f6772fcefd6b55f8bb7fcfa3673f2a05c84fb5bcc144f21375175c3993421057"} Jul 06 02:39:42.445271 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:39:42.445245 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-84mq4" event={"ID":"3c23051e-1536-45e2-8a89-dc077a09d982","Type":"ContainerStarted","Data":"28c00c19249b92ee00b2ccf37bae849d77575eb3ed2d49ac85f1656045bed9c1"} Jul 06 02:39:42.460189 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:39:42.460160 2557 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-twqqj" Jul 06 02:39:42.460866 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:39:42.460837 2557 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-twqqj" Jul 06 02:39:42.461766 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:39:42.461725 2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-84mq4" podStartSLOduration=4.5487150960000005 podStartE2EDuration="25.461713659s" totalImagesPullingTime="20.912998563s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-06 02:39:17 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 02:39:42.461679717 +0000 UTC m=+26.267965775" watchObservedRunningTime="2026-07-06 02:39:42.461713659 +0000 UTC m=+26.267999699" Jul 06 02:39:43.295802 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:39:43.295767 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-756n2" Jul 06 02:39:43.296042 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:39:43.295767 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-l2btb" Jul 06 02:39:43.296042 ip-10-0-139-206 kubenswrapper[2557]: E0706 02:39:43.295934 2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-756n2" podUID="26f69a47-e949-4e18-9d9c-6648a959e281" Jul 06 02:39:43.296042 ip-10-0-139-206 kubenswrapper[2557]: E0706 02:39:43.295990 2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-l2btb" podUID="c3ee201f-4897-4a18-bf37-f6c327d25e1e" Jul 06 02:39:43.296042 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:39:43.295777 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-xjlbh" Jul 06 02:39:43.296237 ip-10-0-139-206 kubenswrapper[2557]: E0706 02:39:43.296118 2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-xjlbh" podUID="8f9fee4a-de24-49dc-b667-3dbc852d7ae3" Jul 06 02:39:43.448739 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:39:43.448706 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-twqqj" Jul 06 02:39:43.449219 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:39:43.449013 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-twqqj" Jul 06 02:39:44.453924 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:39:44.453695 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-cw2f7_efcf5e11-1e5d-4e2a-9b38-303a8f0787c4/ovn-acl-logging/0.log" Jul 06 02:39:44.454721 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:39:44.454218 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-cw2f7" event={"ID":"efcf5e11-1e5d-4e2a-9b38-303a8f0787c4","Type":"ContainerStarted","Data":"145aa03054fe1f24e07303820b37b8c031e0c6c382101c4960231b6ad8d3337d"} Jul 06 02:39:44.454721 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:39:44.454508 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-cw2f7" Jul 06 02:39:44.454721 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:39:44.454713 2557 scope.go:117] "RemoveContainer" containerID="8f15dc449d509317d0b451f4fa4dfa5e0a57c9753d4b12592511b5c0528fb880" Jul 06 02:39:44.456211 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:39:44.456185 2557 generic.go:356] "Generic (PLEG): container finished" podID="29184621-7845-4a4a-9b81-04b439571ca3" containerID="b85257aadd9624d27412dab26727b48d42034039da6f64136fb189274ed6fabd" exitCode=0 Jul 06 02:39:44.456336 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:39:44.456259 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-7nlb2" event={"ID":"29184621-7845-4a4a-9b81-04b439571ca3","Type":"ContainerDied","Data":"b85257aadd9624d27412dab26727b48d42034039da6f64136fb189274ed6fabd"} Jul 06 02:39:44.470858 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:39:44.470832 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-cw2f7" Jul 06 02:39:45.295936 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:39:45.295863 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-xjlbh" Jul 06 02:39:45.296111 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:39:45.295864 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-756n2" Jul 06 02:39:45.296111 ip-10-0-139-206 kubenswrapper[2557]: E0706 02:39:45.296005 2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-xjlbh" podUID="8f9fee4a-de24-49dc-b667-3dbc852d7ae3" Jul 06 02:39:45.296111 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:39:45.295870 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-l2btb" Jul 06 02:39:45.296111 ip-10-0-139-206 kubenswrapper[2557]: E0706 02:39:45.296054 2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-756n2" podUID="26f69a47-e949-4e18-9d9c-6648a959e281" Jul 06 02:39:45.296275 ip-10-0-139-206 kubenswrapper[2557]: E0706 02:39:45.296134 2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-l2btb" podUID="c3ee201f-4897-4a18-bf37-f6c327d25e1e" Jul 06 02:39:45.465677 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:39:45.465652 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-cw2f7_efcf5e11-1e5d-4e2a-9b38-303a8f0787c4/ovn-acl-logging/0.log" Jul 06 02:39:45.466164 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:39:45.466103 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-cw2f7" event={"ID":"efcf5e11-1e5d-4e2a-9b38-303a8f0787c4","Type":"ContainerStarted","Data":"38d4dddad27d73b479a652964a2b09c503b6fdc4686f163a3c541910de29a2ef"} Jul 06 02:39:45.466488 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:39:45.466469 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-cw2f7" Jul 06 02:39:45.466604 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:39:45.466528 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-cw2f7" Jul 06 02:39:45.482818 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:39:45.482791 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-cw2f7" Jul 06 02:39:45.492618 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:39:45.492568 2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-cw2f7" podStartSLOduration=9.607840088 podStartE2EDuration="28.492552272s" totalImagesPullingTime="18.884712184s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-06 02:39:17 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 02:39:45.492232886 +0000 UTC m=+29.298518944" watchObservedRunningTime="2026-07-06 02:39:45.492552272 +0000 UTC m=+29.298838653" Jul 06 02:39:45.948494 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:39:45.948167 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-l2btb"] Jul 06 02:39:45.948628 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:39:45.948560 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-l2btb" Jul 06 02:39:45.948684 ip-10-0-139-206 kubenswrapper[2557]: E0706 02:39:45.948667 2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-l2btb" podUID="c3ee201f-4897-4a18-bf37-f6c327d25e1e" Jul 06 02:39:45.951493 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:39:45.951468 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-xjlbh"] Jul 06 02:39:45.951630 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:39:45.951580 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-xjlbh" Jul 06 02:39:45.951716 ip-10-0-139-206 kubenswrapper[2557]: E0706 02:39:45.951696 2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-xjlbh" podUID="8f9fee4a-de24-49dc-b667-3dbc852d7ae3" Jul 06 02:39:45.952166 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:39:45.952144 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-756n2"] Jul 06 02:39:45.952263 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:39:45.952250 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-756n2" Jul 06 02:39:45.952413 ip-10-0-139-206 kubenswrapper[2557]: E0706 02:39:45.952390 2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-756n2" podUID="26f69a47-e949-4e18-9d9c-6648a959e281" Jul 06 02:39:46.470233 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:39:46.470198 2557 generic.go:356] "Generic (PLEG): container finished" podID="29184621-7845-4a4a-9b81-04b439571ca3" containerID="72f3acb88b36bebc973c477b054e82af3244963d35df5e9f389432822956b216" exitCode=0 Jul 06 02:39:46.470872 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:39:46.470254 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-7nlb2" event={"ID":"29184621-7845-4a4a-9b81-04b439571ca3","Type":"ContainerDied","Data":"72f3acb88b36bebc973c477b054e82af3244963d35df5e9f389432822956b216"} Jul 06 02:39:47.296960 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:39:47.296931 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-xjlbh" Jul 06 02:39:47.296960 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:39:47.296962 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-756n2" Jul 06 02:39:47.297222 ip-10-0-139-206 kubenswrapper[2557]: E0706 02:39:47.297012 2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-xjlbh" podUID="8f9fee4a-de24-49dc-b667-3dbc852d7ae3" Jul 06 02:39:47.297222 ip-10-0-139-206 kubenswrapper[2557]: E0706 02:39:47.297057 2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-756n2" podUID="26f69a47-e949-4e18-9d9c-6648a959e281" Jul 06 02:39:47.297222 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:39:47.297096 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-l2btb" Jul 06 02:39:47.297222 ip-10-0-139-206 kubenswrapper[2557]: E0706 02:39:47.297195 2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-l2btb" podUID="c3ee201f-4897-4a18-bf37-f6c327d25e1e" Jul 06 02:39:48.478052 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:39:48.478020 2557 generic.go:356] "Generic (PLEG): container finished" podID="29184621-7845-4a4a-9b81-04b439571ca3" containerID="3deb7207de32912e793f1d93e32b09e833607bed54c9ebf321b4cf2941901819" exitCode=0 Jul 06 02:39:48.478711 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:39:48.478086 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-7nlb2" event={"ID":"29184621-7845-4a4a-9b81-04b439571ca3","Type":"ContainerDied","Data":"3deb7207de32912e793f1d93e32b09e833607bed54c9ebf321b4cf2941901819"} Jul 06 02:39:49.296414 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:39:49.296374 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-xjlbh" Jul 06 02:39:49.296605 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:39:49.296374 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-756n2" Jul 06 02:39:49.296605 ip-10-0-139-206 kubenswrapper[2557]: E0706 02:39:49.296508 2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-xjlbh" podUID="8f9fee4a-de24-49dc-b667-3dbc852d7ae3" Jul 06 02:39:49.296605 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:39:49.296384 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-l2btb" Jul 06 02:39:49.296605 ip-10-0-139-206 kubenswrapper[2557]: E0706 02:39:49.296593 2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-756n2" podUID="26f69a47-e949-4e18-9d9c-6648a959e281" Jul 06 02:39:49.296814 ip-10-0-139-206 kubenswrapper[2557]: E0706 02:39:49.296664 2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-l2btb" podUID="c3ee201f-4897-4a18-bf37-f6c327d25e1e" Jul 06 02:39:50.900159 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:39:50.900066 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/8f9fee4a-de24-49dc-b667-3dbc852d7ae3-original-pull-secret\") pod \"global-pull-secret-syncer-xjlbh\" (UID: \"8f9fee4a-de24-49dc-b667-3dbc852d7ae3\") " pod="kube-system/global-pull-secret-syncer-xjlbh" Jul 06 02:39:50.900767 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:39:50.900166 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-t4wsh\" (UniqueName: \"kubernetes.io/projected/c3ee201f-4897-4a18-bf37-f6c327d25e1e-kube-api-access-t4wsh\") pod \"network-check-target-l2btb\" (UID: \"c3ee201f-4897-4a18-bf37-f6c327d25e1e\") " pod="openshift-network-diagnostics/network-check-target-l2btb" Jul 06 02:39:50.900767 ip-10-0-139-206 kubenswrapper[2557]: E0706 02:39:50.900238 2557 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 06 02:39:50.900767 ip-10-0-139-206 kubenswrapper[2557]: E0706 02:39:50.900277 2557 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 06 02:39:50.900767 ip-10-0-139-206 kubenswrapper[2557]: E0706 02:39:50.900292 2557 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 06 02:39:50.900767 ip-10-0-139-206 kubenswrapper[2557]: E0706 02:39:50.900305 2557 projected.go:196] Error preparing data for projected volume kube-api-access-t4wsh for pod openshift-network-diagnostics/network-check-target-l2btb: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 06 02:39:50.900767 ip-10-0-139-206 kubenswrapper[2557]: E0706 02:39:50.900316 2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8f9fee4a-de24-49dc-b667-3dbc852d7ae3-original-pull-secret podName:8f9fee4a-de24-49dc-b667-3dbc852d7ae3 nodeName:}" failed. No retries permitted until 2026-07-06 02:40:22.900295278 +0000 UTC m=+66.706581313 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/8f9fee4a-de24-49dc-b667-3dbc852d7ae3-original-pull-secret") pod "global-pull-secret-syncer-xjlbh" (UID: "8f9fee4a-de24-49dc-b667-3dbc852d7ae3") : object "kube-system"/"original-pull-secret" not registered Jul 06 02:39:50.900767 ip-10-0-139-206 kubenswrapper[2557]: E0706 02:39:50.900368 2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/c3ee201f-4897-4a18-bf37-f6c327d25e1e-kube-api-access-t4wsh podName:c3ee201f-4897-4a18-bf37-f6c327d25e1e nodeName:}" failed. No retries permitted until 2026-07-06 02:40:22.900339499 +0000 UTC m=+66.706625540 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-t4wsh" (UniqueName: "kubernetes.io/projected/c3ee201f-4897-4a18-bf37-f6c327d25e1e-kube-api-access-t4wsh") pod "network-check-target-l2btb" (UID: "c3ee201f-4897-4a18-bf37-f6c327d25e1e") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 06 02:39:51.001023 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:39:51.000989 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/26f69a47-e949-4e18-9d9c-6648a959e281-metrics-certs\") pod \"network-metrics-daemon-756n2\" (UID: \"26f69a47-e949-4e18-9d9c-6648a959e281\") " pod="openshift-multus/network-metrics-daemon-756n2" Jul 06 02:39:51.001302 ip-10-0-139-206 kubenswrapper[2557]: E0706 02:39:51.001279 2557 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 06 02:39:51.001515 ip-10-0-139-206 kubenswrapper[2557]: E0706 02:39:51.001467 2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/26f69a47-e949-4e18-9d9c-6648a959e281-metrics-certs podName:26f69a47-e949-4e18-9d9c-6648a959e281 nodeName:}" failed. No retries permitted until 2026-07-06 02:40:23.001432317 +0000 UTC m=+66.807718367 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/26f69a47-e949-4e18-9d9c-6648a959e281-metrics-certs") pod "network-metrics-daemon-756n2" (UID: "26f69a47-e949-4e18-9d9c-6648a959e281") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 06 02:39:51.296669 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:39:51.296579 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-756n2" Jul 06 02:39:51.296669 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:39:51.296579 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-l2btb" Jul 06 02:39:51.296890 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:39:51.296709 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-xjlbh" Jul 06 02:39:51.296890 ip-10-0-139-206 kubenswrapper[2557]: E0706 02:39:51.296722 2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-756n2" podUID="26f69a47-e949-4e18-9d9c-6648a959e281" Jul 06 02:39:51.296890 ip-10-0-139-206 kubenswrapper[2557]: E0706 02:39:51.296806 2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-l2btb" podUID="c3ee201f-4897-4a18-bf37-f6c327d25e1e" Jul 06 02:39:51.296890 ip-10-0-139-206 kubenswrapper[2557]: E0706 02:39:51.296875 2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-xjlbh" podUID="8f9fee4a-de24-49dc-b667-3dbc852d7ae3" Jul 06 02:39:52.475032 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:39:52.474850 2557 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-206.ec2.internal" event="NodeReady" Jul 06 02:39:52.475488 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:39:52.475142 2557 kubelet_node_status.go:489] "Fast updating node status as it just became ready" Jul 06 02:39:52.518277 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:39:52.518249 2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-8jcmv"] Jul 06 02:39:52.520749 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:39:52.520721 2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-qkmzt"] Jul 06 02:39:52.520901 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:39:52.520857 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-8jcmv" Jul 06 02:39:52.522872 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:39:52.522829 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-qkmzt" Jul 06 02:39:52.523136 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:39:52.523032 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\"" Jul 06 02:39:52.523422 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:39:52.523402 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\"" Jul 06 02:39:52.523514 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:39:52.523450 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-lsrhl\"" Jul 06 02:39:52.524879 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:39:52.524846 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\"" Jul 06 02:39:52.524989 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:39:52.524884 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\"" Jul 06 02:39:52.525737 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:39:52.525634 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-sfslh\"" Jul 06 02:39:52.525737 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:39:52.525697 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\"" Jul 06 02:39:52.531699 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:39:52.531675 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-8jcmv"] Jul 06 02:39:52.534009 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:39:52.533972 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-qkmzt"] Jul 06 02:39:52.612789 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:39:52.612672 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/974aff39-ecbd-4788-a159-d35d2a73edb6-metrics-tls\") pod \"dns-default-8jcmv\" (UID: \"974aff39-ecbd-4788-a159-d35d2a73edb6\") " pod="openshift-dns/dns-default-8jcmv" Jul 06 02:39:52.612789 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:39:52.612710 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hkbwz\" (UniqueName: \"kubernetes.io/projected/faa07f8b-6058-4e7e-af53-4fc206f18730-kube-api-access-hkbwz\") pod \"ingress-canary-qkmzt\" (UID: \"faa07f8b-6058-4e7e-af53-4fc206f18730\") " pod="openshift-ingress-canary/ingress-canary-qkmzt" Jul 06 02:39:52.612789 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:39:52.612756 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/974aff39-ecbd-4788-a159-d35d2a73edb6-tmp-dir\") pod \"dns-default-8jcmv\" (UID: \"974aff39-ecbd-4788-a159-d35d2a73edb6\") " pod="openshift-dns/dns-default-8jcmv" Jul 06 02:39:52.613045 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:39:52.612807 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/974aff39-ecbd-4788-a159-d35d2a73edb6-config-volume\") pod \"dns-default-8jcmv\" (UID: \"974aff39-ecbd-4788-a159-d35d2a73edb6\") " pod="openshift-dns/dns-default-8jcmv" Jul 06 02:39:52.613045 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:39:52.612869 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xvrts\" (UniqueName: \"kubernetes.io/projected/974aff39-ecbd-4788-a159-d35d2a73edb6-kube-api-access-xvrts\") pod \"dns-default-8jcmv\" (UID: \"974aff39-ecbd-4788-a159-d35d2a73edb6\") " pod="openshift-dns/dns-default-8jcmv" Jul 06 02:39:52.613045 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:39:52.612934 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/faa07f8b-6058-4e7e-af53-4fc206f18730-cert\") pod \"ingress-canary-qkmzt\" (UID: \"faa07f8b-6058-4e7e-af53-4fc206f18730\") " pod="openshift-ingress-canary/ingress-canary-qkmzt" Jul 06 02:39:52.714229 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:39:52.714184 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xvrts\" (UniqueName: \"kubernetes.io/projected/974aff39-ecbd-4788-a159-d35d2a73edb6-kube-api-access-xvrts\") pod \"dns-default-8jcmv\" (UID: \"974aff39-ecbd-4788-a159-d35d2a73edb6\") " pod="openshift-dns/dns-default-8jcmv" Jul 06 02:39:52.714451 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:39:52.714254 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/faa07f8b-6058-4e7e-af53-4fc206f18730-cert\") pod \"ingress-canary-qkmzt\" (UID: \"faa07f8b-6058-4e7e-af53-4fc206f18730\") " pod="openshift-ingress-canary/ingress-canary-qkmzt" Jul 06 02:39:52.714451 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:39:52.714295 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/974aff39-ecbd-4788-a159-d35d2a73edb6-metrics-tls\") pod \"dns-default-8jcmv\" (UID: \"974aff39-ecbd-4788-a159-d35d2a73edb6\") " pod="openshift-dns/dns-default-8jcmv" Jul 06 02:39:52.714451 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:39:52.714313 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hkbwz\" (UniqueName: \"kubernetes.io/projected/faa07f8b-6058-4e7e-af53-4fc206f18730-kube-api-access-hkbwz\") pod \"ingress-canary-qkmzt\" (UID: \"faa07f8b-6058-4e7e-af53-4fc206f18730\") " pod="openshift-ingress-canary/ingress-canary-qkmzt" Jul 06 02:39:52.714451 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:39:52.714340 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/974aff39-ecbd-4788-a159-d35d2a73edb6-tmp-dir\") pod \"dns-default-8jcmv\" (UID: \"974aff39-ecbd-4788-a159-d35d2a73edb6\") " pod="openshift-dns/dns-default-8jcmv" Jul 06 02:39:52.714451 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:39:52.714388 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/974aff39-ecbd-4788-a159-d35d2a73edb6-config-volume\") pod \"dns-default-8jcmv\" (UID: \"974aff39-ecbd-4788-a159-d35d2a73edb6\") " pod="openshift-dns/dns-default-8jcmv" Jul 06 02:39:52.714451 ip-10-0-139-206 kubenswrapper[2557]: E0706 02:39:52.714446 2557 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 06 02:39:52.714752 ip-10-0-139-206 kubenswrapper[2557]: E0706 02:39:52.714514 2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/974aff39-ecbd-4788-a159-d35d2a73edb6-metrics-tls podName:974aff39-ecbd-4788-a159-d35d2a73edb6 nodeName:}" failed. No retries permitted until 2026-07-06 02:39:53.21449516 +0000 UTC m=+37.020781214 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/974aff39-ecbd-4788-a159-d35d2a73edb6-metrics-tls") pod "dns-default-8jcmv" (UID: "974aff39-ecbd-4788-a159-d35d2a73edb6") : secret "dns-default-metrics-tls" not found Jul 06 02:39:52.714870 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:39:52.714847 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/974aff39-ecbd-4788-a159-d35d2a73edb6-tmp-dir\") pod \"dns-default-8jcmv\" (UID: \"974aff39-ecbd-4788-a159-d35d2a73edb6\") " pod="openshift-dns/dns-default-8jcmv" Jul 06 02:39:52.715049 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:39:52.715029 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/974aff39-ecbd-4788-a159-d35d2a73edb6-config-volume\") pod \"dns-default-8jcmv\" (UID: \"974aff39-ecbd-4788-a159-d35d2a73edb6\") " pod="openshift-dns/dns-default-8jcmv" Jul 06 02:39:52.719558 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:39:52.719533 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/faa07f8b-6058-4e7e-af53-4fc206f18730-cert\") pod \"ingress-canary-qkmzt\" (UID: \"faa07f8b-6058-4e7e-af53-4fc206f18730\") " pod="openshift-ingress-canary/ingress-canary-qkmzt" Jul 06 02:39:52.722993 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:39:52.722961 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-xvrts\" (UniqueName: \"kubernetes.io/projected/974aff39-ecbd-4788-a159-d35d2a73edb6-kube-api-access-xvrts\") pod \"dns-default-8jcmv\" (UID: \"974aff39-ecbd-4788-a159-d35d2a73edb6\") " pod="openshift-dns/dns-default-8jcmv" Jul 06 02:39:52.723132 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:39:52.723051 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-hkbwz\" (UniqueName: \"kubernetes.io/projected/faa07f8b-6058-4e7e-af53-4fc206f18730-kube-api-access-hkbwz\") pod \"ingress-canary-qkmzt\" (UID: \"faa07f8b-6058-4e7e-af53-4fc206f18730\") " pod="openshift-ingress-canary/ingress-canary-qkmzt" Jul 06 02:39:52.840301 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:39:52.840262 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-qkmzt" Jul 06 02:39:53.162811 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:39:53.162779 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-qkmzt"] Jul 06 02:39:53.166979 ip-10-0-139-206 kubenswrapper[2557]: W0706 02:39:53.166942 2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfaa07f8b_6058_4e7e_af53_4fc206f18730.slice/crio-6d3b0e5b6b6b908c07ea7e0e9d8a81d938c2225b0ccf15fcc034ad664765dfe5 WatchSource:0}: Error finding container 6d3b0e5b6b6b908c07ea7e0e9d8a81d938c2225b0ccf15fcc034ad664765dfe5: Status 404 returned error can't find the container with id 6d3b0e5b6b6b908c07ea7e0e9d8a81d938c2225b0ccf15fcc034ad664765dfe5 Jul 06 02:39:53.219095 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:39:53.219056 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/974aff39-ecbd-4788-a159-d35d2a73edb6-metrics-tls\") pod \"dns-default-8jcmv\" (UID: \"974aff39-ecbd-4788-a159-d35d2a73edb6\") " pod="openshift-dns/dns-default-8jcmv" Jul 06 02:39:53.219197 ip-10-0-139-206 kubenswrapper[2557]: E0706 02:39:53.219181 2557 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 06 02:39:53.219249 ip-10-0-139-206 kubenswrapper[2557]: E0706 02:39:53.219234 2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/974aff39-ecbd-4788-a159-d35d2a73edb6-metrics-tls podName:974aff39-ecbd-4788-a159-d35d2a73edb6 nodeName:}" failed. No retries permitted until 2026-07-06 02:39:54.219220858 +0000 UTC m=+38.025506894 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/974aff39-ecbd-4788-a159-d35d2a73edb6-metrics-tls") pod "dns-default-8jcmv" (UID: "974aff39-ecbd-4788-a159-d35d2a73edb6") : secret "dns-default-metrics-tls" not found Jul 06 02:39:53.296119 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:39:53.296080 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-xjlbh" Jul 06 02:39:53.296313 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:39:53.296293 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-756n2" Jul 06 02:39:53.296452 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:39:53.296432 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-l2btb" Jul 06 02:39:53.298772 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:39:53.298733 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" Jul 06 02:39:53.298925 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:39:53.298790 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Jul 06 02:39:53.299038 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:39:53.299023 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Jul 06 02:39:53.299130 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:39:53.299111 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-68hwf\"" Jul 06 02:39:53.299773 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:39:53.299761 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-n2bvg\"" Jul 06 02:39:53.299820 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:39:53.299777 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" Jul 06 02:39:53.494700 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:39:53.494665 2557 generic.go:356] "Generic (PLEG): container finished" podID="29184621-7845-4a4a-9b81-04b439571ca3" containerID="7019ab2b28aff3ab60fccb65b7e4048b282477d2d6db7394b02591b2683f4414" exitCode=0 Jul 06 02:39:53.495174 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:39:53.494762 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-7nlb2" event={"ID":"29184621-7845-4a4a-9b81-04b439571ca3","Type":"ContainerDied","Data":"7019ab2b28aff3ab60fccb65b7e4048b282477d2d6db7394b02591b2683f4414"} Jul 06 02:39:53.496160 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:39:53.496135 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-qkmzt" event={"ID":"faa07f8b-6058-4e7e-af53-4fc206f18730","Type":"ContainerStarted","Data":"6d3b0e5b6b6b908c07ea7e0e9d8a81d938c2225b0ccf15fcc034ad664765dfe5"} Jul 06 02:39:54.225967 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:39:54.225763 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/974aff39-ecbd-4788-a159-d35d2a73edb6-metrics-tls\") pod \"dns-default-8jcmv\" (UID: \"974aff39-ecbd-4788-a159-d35d2a73edb6\") " pod="openshift-dns/dns-default-8jcmv" Jul 06 02:39:54.226158 ip-10-0-139-206 kubenswrapper[2557]: E0706 02:39:54.225945 2557 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 06 02:39:54.226158 ip-10-0-139-206 kubenswrapper[2557]: E0706 02:39:54.226113 2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/974aff39-ecbd-4788-a159-d35d2a73edb6-metrics-tls podName:974aff39-ecbd-4788-a159-d35d2a73edb6 nodeName:}" failed. No retries permitted until 2026-07-06 02:39:56.226091523 +0000 UTC m=+40.032377563 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/974aff39-ecbd-4788-a159-d35d2a73edb6-metrics-tls") pod "dns-default-8jcmv" (UID: "974aff39-ecbd-4788-a159-d35d2a73edb6") : secret "dns-default-metrics-tls" not found Jul 06 02:39:54.503507 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:39:54.503424 2557 generic.go:356] "Generic (PLEG): container finished" podID="29184621-7845-4a4a-9b81-04b439571ca3" containerID="828a2bbad8a6e075fb853ab85c9c1fd0140d36f7e460fff8e6c09b709592874e" exitCode=0 Jul 06 02:39:54.503907 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:39:54.503492 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-7nlb2" event={"ID":"29184621-7845-4a4a-9b81-04b439571ca3","Type":"ContainerDied","Data":"828a2bbad8a6e075fb853ab85c9c1fd0140d36f7e460fff8e6c09b709592874e"} Jul 06 02:39:55.509711 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:39:55.509656 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-7nlb2" event={"ID":"29184621-7845-4a4a-9b81-04b439571ca3","Type":"ContainerStarted","Data":"3f18eb29fa5b767ae2abddc5621493b66ff2b14ff9fcab61a91a4a0779c64b80"} Jul 06 02:39:55.533359 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:39:55.533286 2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-7nlb2" podStartSLOduration=8.320104172 podStartE2EDuration="38.533263499s" totalImagesPullingTime="30.213159327s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-06 02:39:17 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 02:39:55.53176798 +0000 UTC m=+39.338054039" watchObservedRunningTime="2026-07-06 02:39:55.533263499 +0000 UTC m=+39.339549558" Jul 06 02:39:56.242817 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:39:56.242721 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/974aff39-ecbd-4788-a159-d35d2a73edb6-metrics-tls\") pod \"dns-default-8jcmv\" (UID: \"974aff39-ecbd-4788-a159-d35d2a73edb6\") " pod="openshift-dns/dns-default-8jcmv" Jul 06 02:39:56.242955 ip-10-0-139-206 kubenswrapper[2557]: E0706 02:39:56.242849 2557 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 06 02:39:56.242955 ip-10-0-139-206 kubenswrapper[2557]: E0706 02:39:56.242903 2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/974aff39-ecbd-4788-a159-d35d2a73edb6-metrics-tls podName:974aff39-ecbd-4788-a159-d35d2a73edb6 nodeName:}" failed. No retries permitted until 2026-07-06 02:40:00.242890063 +0000 UTC m=+44.049176099 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/974aff39-ecbd-4788-a159-d35d2a73edb6-metrics-tls") pod "dns-default-8jcmv" (UID: "974aff39-ecbd-4788-a159-d35d2a73edb6") : secret "dns-default-metrics-tls" not found Jul 06 02:39:56.513245 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:39:56.513149 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-qkmzt" event={"ID":"faa07f8b-6058-4e7e-af53-4fc206f18730","Type":"ContainerStarted","Data":"126608f01a13147bfbfe3f8d05a9cbe055d2b0f4e5789b5584c73a8651d4f79f"} Jul 06 02:39:56.527815 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:39:56.527756 2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-qkmzt" podStartSLOduration=1.969296916 podStartE2EDuration="4.527737213s" totalImagesPullingTime="2.558440297s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 02:39:52 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 02:39:56.526925142 +0000 UTC m=+40.333211191" watchObservedRunningTime="2026-07-06 02:39:56.527737213 +0000 UTC m=+40.334023271" Jul 06 02:40:00.274720 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:40:00.274669 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/974aff39-ecbd-4788-a159-d35d2a73edb6-metrics-tls\") pod \"dns-default-8jcmv\" (UID: \"974aff39-ecbd-4788-a159-d35d2a73edb6\") " pod="openshift-dns/dns-default-8jcmv" Jul 06 02:40:00.275125 ip-10-0-139-206 kubenswrapper[2557]: E0706 02:40:00.274836 2557 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 06 02:40:00.275125 ip-10-0-139-206 kubenswrapper[2557]: E0706 02:40:00.274904 2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/974aff39-ecbd-4788-a159-d35d2a73edb6-metrics-tls podName:974aff39-ecbd-4788-a159-d35d2a73edb6 nodeName:}" failed. No retries permitted until 2026-07-06 02:40:08.274888513 +0000 UTC m=+52.081174548 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/974aff39-ecbd-4788-a159-d35d2a73edb6-metrics-tls") pod "dns-default-8jcmv" (UID: "974aff39-ecbd-4788-a159-d35d2a73edb6") : secret "dns-default-metrics-tls" not found Jul 06 02:40:08.329307 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:40:08.329259 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/974aff39-ecbd-4788-a159-d35d2a73edb6-metrics-tls\") pod \"dns-default-8jcmv\" (UID: \"974aff39-ecbd-4788-a159-d35d2a73edb6\") " pod="openshift-dns/dns-default-8jcmv" Jul 06 02:40:08.329826 ip-10-0-139-206 kubenswrapper[2557]: E0706 02:40:08.329445 2557 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 06 02:40:08.329826 ip-10-0-139-206 kubenswrapper[2557]: E0706 02:40:08.329527 2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/974aff39-ecbd-4788-a159-d35d2a73edb6-metrics-tls podName:974aff39-ecbd-4788-a159-d35d2a73edb6 nodeName:}" failed. No retries permitted until 2026-07-06 02:40:24.329508089 +0000 UTC m=+68.135794141 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/974aff39-ecbd-4788-a159-d35d2a73edb6-metrics-tls") pod "dns-default-8jcmv" (UID: "974aff39-ecbd-4788-a159-d35d2a73edb6") : secret "dns-default-metrics-tls" not found Jul 06 02:40:17.491988 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:40:17.491955 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-cw2f7" Jul 06 02:40:22.932471 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:40:22.932420 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-t4wsh\" (UniqueName: \"kubernetes.io/projected/c3ee201f-4897-4a18-bf37-f6c327d25e1e-kube-api-access-t4wsh\") pod \"network-check-target-l2btb\" (UID: \"c3ee201f-4897-4a18-bf37-f6c327d25e1e\") " pod="openshift-network-diagnostics/network-check-target-l2btb" Jul 06 02:40:22.932471 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:40:22.932481 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/8f9fee4a-de24-49dc-b667-3dbc852d7ae3-original-pull-secret\") pod \"global-pull-secret-syncer-xjlbh\" (UID: \"8f9fee4a-de24-49dc-b667-3dbc852d7ae3\") " pod="kube-system/global-pull-secret-syncer-xjlbh" Jul 06 02:40:22.935300 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:40:22.935273 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" Jul 06 02:40:22.935450 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:40:22.935316 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Jul 06 02:40:22.945140 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:40:22.945112 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" Jul 06 02:40:22.946632 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:40:22.946602 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/8f9fee4a-de24-49dc-b667-3dbc852d7ae3-original-pull-secret\") pod \"global-pull-secret-syncer-xjlbh\" (UID: \"8f9fee4a-de24-49dc-b667-3dbc852d7ae3\") " pod="kube-system/global-pull-secret-syncer-xjlbh" Jul 06 02:40:22.956217 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:40:22.956192 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-t4wsh\" (UniqueName: \"kubernetes.io/projected/c3ee201f-4897-4a18-bf37-f6c327d25e1e-kube-api-access-t4wsh\") pod \"network-check-target-l2btb\" (UID: \"c3ee201f-4897-4a18-bf37-f6c327d25e1e\") " pod="openshift-network-diagnostics/network-check-target-l2btb" Jul 06 02:40:23.017781 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:40:23.017737 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-xjlbh" Jul 06 02:40:23.031224 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:40:23.031198 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-68hwf\"" Jul 06 02:40:23.033214 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:40:23.033187 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/26f69a47-e949-4e18-9d9c-6648a959e281-metrics-certs\") pod \"network-metrics-daemon-756n2\" (UID: \"26f69a47-e949-4e18-9d9c-6648a959e281\") " pod="openshift-multus/network-metrics-daemon-756n2" Jul 06 02:40:23.035137 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:40:23.035114 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Jul 06 02:40:23.038863 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:40:23.038839 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-l2btb" Jul 06 02:40:23.045496 ip-10-0-139-206 kubenswrapper[2557]: E0706 02:40:23.045452 2557 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found Jul 06 02:40:23.045617 ip-10-0-139-206 kubenswrapper[2557]: E0706 02:40:23.045548 2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/26f69a47-e949-4e18-9d9c-6648a959e281-metrics-certs podName:26f69a47-e949-4e18-9d9c-6648a959e281 nodeName:}" failed. No retries permitted until 2026-07-06 02:41:27.045521958 +0000 UTC m=+130.851808014 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/26f69a47-e949-4e18-9d9c-6648a959e281-metrics-certs") pod "network-metrics-daemon-756n2" (UID: "26f69a47-e949-4e18-9d9c-6648a959e281") : secret "metrics-daemon-secret" not found Jul 06 02:40:23.156963 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:40:23.156863 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-xjlbh"] Jul 06 02:40:23.170902 ip-10-0-139-206 kubenswrapper[2557]: W0706 02:40:23.170874 2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8f9fee4a_de24_49dc_b667_3dbc852d7ae3.slice/crio-c92aa7db311379f4b49069924aab0cfdb50c6996d3ff0801654b1e2e5909fd08 WatchSource:0}: Error finding container c92aa7db311379f4b49069924aab0cfdb50c6996d3ff0801654b1e2e5909fd08: Status 404 returned error can't find the container with id c92aa7db311379f4b49069924aab0cfdb50c6996d3ff0801654b1e2e5909fd08 Jul 06 02:40:23.185881 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:40:23.184368 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-l2btb"] Jul 06 02:40:23.190838 ip-10-0-139-206 kubenswrapper[2557]: W0706 02:40:23.190812 2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc3ee201f_4897_4a18_bf37_f6c327d25e1e.slice/crio-472478c186719387bd673ae34c53adb8f180284047006e9fe4c6dda80834a7e2 WatchSource:0}: Error finding container 472478c186719387bd673ae34c53adb8f180284047006e9fe4c6dda80834a7e2: Status 404 returned error can't find the container with id 472478c186719387bd673ae34c53adb8f180284047006e9fe4c6dda80834a7e2 Jul 06 02:40:23.583408 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:40:23.583311 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-xjlbh" event={"ID":"8f9fee4a-de24-49dc-b667-3dbc852d7ae3","Type":"ContainerStarted","Data":"c92aa7db311379f4b49069924aab0cfdb50c6996d3ff0801654b1e2e5909fd08"} Jul 06 02:40:23.584646 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:40:23.584605 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-l2btb" event={"ID":"c3ee201f-4897-4a18-bf37-f6c327d25e1e","Type":"ContainerStarted","Data":"472478c186719387bd673ae34c53adb8f180284047006e9fe4c6dda80834a7e2"} Jul 06 02:40:24.343800 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:40:24.343758 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/974aff39-ecbd-4788-a159-d35d2a73edb6-metrics-tls\") pod \"dns-default-8jcmv\" (UID: \"974aff39-ecbd-4788-a159-d35d2a73edb6\") " pod="openshift-dns/dns-default-8jcmv" Jul 06 02:40:24.344211 ip-10-0-139-206 kubenswrapper[2557]: E0706 02:40:24.343931 2557 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 06 02:40:24.344211 ip-10-0-139-206 kubenswrapper[2557]: E0706 02:40:24.344017 2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/974aff39-ecbd-4788-a159-d35d2a73edb6-metrics-tls podName:974aff39-ecbd-4788-a159-d35d2a73edb6 nodeName:}" failed. No retries permitted until 2026-07-06 02:40:56.343993204 +0000 UTC m=+100.150279268 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/974aff39-ecbd-4788-a159-d35d2a73edb6-metrics-tls") pod "dns-default-8jcmv" (UID: "974aff39-ecbd-4788-a159-d35d2a73edb6") : secret "dns-default-metrics-tls" not found Jul 06 02:40:28.599148 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:40:28.599102 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-l2btb" event={"ID":"c3ee201f-4897-4a18-bf37-f6c327d25e1e","Type":"ContainerStarted","Data":"6ed7e7dbcdd79f602d86af47abcdfea1ef3b51b078043eaa6b8d11997b4a81ee"} Jul 06 02:40:28.599696 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:40:28.599204 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-l2btb" Jul 06 02:40:28.600517 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:40:28.600495 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-xjlbh" event={"ID":"8f9fee4a-de24-49dc-b667-3dbc852d7ae3","Type":"ContainerStarted","Data":"beacfbf40ca3e16380cd77ed38f4e063d4e16712b46ef3be3b04fe033920f86d"} Jul 06 02:40:28.620440 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:40:28.620387 2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-l2btb" podStartSLOduration=66.934197902 podStartE2EDuration="1m11.620372751s" totalImagesPullingTime="4.686174849s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 02:39:17 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 02:40:28.619184026 +0000 UTC m=+72.425470083" watchObservedRunningTime="2026-07-06 02:40:28.620372751 +0000 UTC m=+72.426658802" Jul 06 02:40:28.636332 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:40:28.636278 2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-xjlbh" podStartSLOduration=66.925642427 podStartE2EDuration="1m11.636260348s" totalImagesPullingTime="4.710617921s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-06 02:39:17 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 02:40:28.635407428 +0000 UTC m=+72.441693482" watchObservedRunningTime="2026-07-06 02:40:28.636260348 +0000 UTC m=+72.442546406" Jul 06 02:40:55.679809 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:40:55.679774 2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-5b44787fb-rl8cj"] Jul 06 02:40:55.682247 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:40:55.682228 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-5b44787fb-rl8cj" Jul 06 02:40:55.685787 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:40:55.685750 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"open-cluster-management-image-pull-credentials\"" Jul 06 02:40:55.685963 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:40:55.685827 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"kube-root-ca.crt\"" Jul 06 02:40:55.685963 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:40:55.685764 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"managed-serviceaccount-hub-kubeconfig\"" Jul 06 02:40:55.685963 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:40:55.685872 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"managed-serviceaccount-dockercfg-qshds\"" Jul 06 02:40:55.685963 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:40:55.685961 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"openshift-service-ca.crt\"" Jul 06 02:40:55.688389 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:40:55.688368 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-5b44787fb-rl8cj"] Jul 06 02:40:55.698774 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:40:55.698747 2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7f946cb44b-w5gkh"] Jul 06 02:40:55.700547 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:40:55.700531 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7f946cb44b-w5gkh" Jul 06 02:40:55.702953 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:40:55.702928 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-hub-kubeconfig\"" Jul 06 02:40:55.703086 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:40:55.702930 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-open-cluster-management.io-proxy-agent-signer-client-cert\"" Jul 06 02:40:55.703086 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:40:55.702982 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-ca\"" Jul 06 02:40:55.703086 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:40:55.702986 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-service-proxy-server-certificates\"" Jul 06 02:40:55.711207 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:40:55.711187 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7f946cb44b-w5gkh"] Jul 06 02:40:55.755123 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:40:55.755023 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"additional-service-ca\" (UniqueName: \"kubernetes.io/configmap/60871947-d3a3-44d5-afdb-c8e0c5aa9654-additional-service-ca\") pod \"cluster-proxy-proxy-agent-7f946cb44b-w5gkh\" (UID: \"60871947-d3a3-44d5-afdb-c8e0c5aa9654\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7f946cb44b-w5gkh" Jul 06 02:40:55.755123 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:40:55.755066 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/6efb58f8-d736-4d66-ada0-9e160f26a88e-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-5b44787fb-rl8cj\" (UID: \"6efb58f8-d736-4d66-ada0-9e160f26a88e\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-5b44787fb-rl8cj" Jul 06 02:40:55.755123 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:40:55.755088 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca\" (UniqueName: \"kubernetes.io/secret/60871947-d3a3-44d5-afdb-c8e0c5aa9654-ca\") pod \"cluster-proxy-proxy-agent-7f946cb44b-w5gkh\" (UID: \"60871947-d3a3-44d5-afdb-c8e0c5aa9654\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7f946cb44b-w5gkh" Jul 06 02:40:55.755404 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:40:55.755166 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hub\" (UniqueName: \"kubernetes.io/secret/60871947-d3a3-44d5-afdb-c8e0c5aa9654-hub\") pod \"cluster-proxy-proxy-agent-7f946cb44b-w5gkh\" (UID: \"60871947-d3a3-44d5-afdb-c8e0c5aa9654\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7f946cb44b-w5gkh" Jul 06 02:40:55.755404 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:40:55.755201 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-proxy-server-cert\" (UniqueName: \"kubernetes.io/secret/60871947-d3a3-44d5-afdb-c8e0c5aa9654-service-proxy-server-cert\") pod \"cluster-proxy-proxy-agent-7f946cb44b-w5gkh\" (UID: \"60871947-d3a3-44d5-afdb-c8e0c5aa9654\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7f946cb44b-w5gkh" Jul 06 02:40:55.755404 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:40:55.755223 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gp9s2\" (UniqueName: \"kubernetes.io/projected/6efb58f8-d736-4d66-ada0-9e160f26a88e-kube-api-access-gp9s2\") pod \"managed-serviceaccount-addon-agent-5b44787fb-rl8cj\" (UID: \"6efb58f8-d736-4d66-ada0-9e160f26a88e\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-5b44787fb-rl8cj" Jul 06 02:40:55.755404 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:40:55.755307 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4dmmb\" (UniqueName: \"kubernetes.io/projected/60871947-d3a3-44d5-afdb-c8e0c5aa9654-kube-api-access-4dmmb\") pod \"cluster-proxy-proxy-agent-7f946cb44b-w5gkh\" (UID: \"60871947-d3a3-44d5-afdb-c8e0c5aa9654\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7f946cb44b-w5gkh" Jul 06 02:40:55.755404 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:40:55.755339 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/60871947-d3a3-44d5-afdb-c8e0c5aa9654-hub-kubeconfig\") pod \"cluster-proxy-proxy-agent-7f946cb44b-w5gkh\" (UID: \"60871947-d3a3-44d5-afdb-c8e0c5aa9654\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7f946cb44b-w5gkh" Jul 06 02:40:55.856281 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:40:55.856227 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hub\" (UniqueName: \"kubernetes.io/secret/60871947-d3a3-44d5-afdb-c8e0c5aa9654-hub\") pod \"cluster-proxy-proxy-agent-7f946cb44b-w5gkh\" (UID: \"60871947-d3a3-44d5-afdb-c8e0c5aa9654\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7f946cb44b-w5gkh" Jul 06 02:40:55.856281 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:40:55.856279 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-proxy-server-cert\" (UniqueName: \"kubernetes.io/secret/60871947-d3a3-44d5-afdb-c8e0c5aa9654-service-proxy-server-cert\") pod \"cluster-proxy-proxy-agent-7f946cb44b-w5gkh\" (UID: \"60871947-d3a3-44d5-afdb-c8e0c5aa9654\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7f946cb44b-w5gkh" Jul 06 02:40:55.856522 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:40:55.856303 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gp9s2\" (UniqueName: \"kubernetes.io/projected/6efb58f8-d736-4d66-ada0-9e160f26a88e-kube-api-access-gp9s2\") pod \"managed-serviceaccount-addon-agent-5b44787fb-rl8cj\" (UID: \"6efb58f8-d736-4d66-ada0-9e160f26a88e\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-5b44787fb-rl8cj" Jul 06 02:40:55.856522 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:40:55.856375 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4dmmb\" (UniqueName: \"kubernetes.io/projected/60871947-d3a3-44d5-afdb-c8e0c5aa9654-kube-api-access-4dmmb\") pod \"cluster-proxy-proxy-agent-7f946cb44b-w5gkh\" (UID: \"60871947-d3a3-44d5-afdb-c8e0c5aa9654\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7f946cb44b-w5gkh" Jul 06 02:40:55.856522 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:40:55.856407 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/60871947-d3a3-44d5-afdb-c8e0c5aa9654-hub-kubeconfig\") pod \"cluster-proxy-proxy-agent-7f946cb44b-w5gkh\" (UID: \"60871947-d3a3-44d5-afdb-c8e0c5aa9654\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7f946cb44b-w5gkh" Jul 06 02:40:55.856522 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:40:55.856476 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"additional-service-ca\" (UniqueName: \"kubernetes.io/configmap/60871947-d3a3-44d5-afdb-c8e0c5aa9654-additional-service-ca\") pod \"cluster-proxy-proxy-agent-7f946cb44b-w5gkh\" (UID: \"60871947-d3a3-44d5-afdb-c8e0c5aa9654\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7f946cb44b-w5gkh" Jul 06 02:40:55.856522 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:40:55.856515 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/6efb58f8-d736-4d66-ada0-9e160f26a88e-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-5b44787fb-rl8cj\" (UID: \"6efb58f8-d736-4d66-ada0-9e160f26a88e\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-5b44787fb-rl8cj" Jul 06 02:40:55.856735 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:40:55.856532 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca\" (UniqueName: \"kubernetes.io/secret/60871947-d3a3-44d5-afdb-c8e0c5aa9654-ca\") pod \"cluster-proxy-proxy-agent-7f946cb44b-w5gkh\" (UID: \"60871947-d3a3-44d5-afdb-c8e0c5aa9654\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7f946cb44b-w5gkh" Jul 06 02:40:55.857474 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:40:55.857446 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"additional-service-ca\" (UniqueName: \"kubernetes.io/configmap/60871947-d3a3-44d5-afdb-c8e0c5aa9654-additional-service-ca\") pod \"cluster-proxy-proxy-agent-7f946cb44b-w5gkh\" (UID: \"60871947-d3a3-44d5-afdb-c8e0c5aa9654\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7f946cb44b-w5gkh" Jul 06 02:40:55.859033 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:40:55.859004 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ca\" (UniqueName: \"kubernetes.io/secret/60871947-d3a3-44d5-afdb-c8e0c5aa9654-ca\") pod \"cluster-proxy-proxy-agent-7f946cb44b-w5gkh\" (UID: \"60871947-d3a3-44d5-afdb-c8e0c5aa9654\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7f946cb44b-w5gkh" Jul 06 02:40:55.859158 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:40:55.859131 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hub\" (UniqueName: \"kubernetes.io/secret/60871947-d3a3-44d5-afdb-c8e0c5aa9654-hub\") pod \"cluster-proxy-proxy-agent-7f946cb44b-w5gkh\" (UID: \"60871947-d3a3-44d5-afdb-c8e0c5aa9654\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7f946cb44b-w5gkh" Jul 06 02:40:55.859219 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:40:55.859155 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/6efb58f8-d736-4d66-ada0-9e160f26a88e-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-5b44787fb-rl8cj\" (UID: \"6efb58f8-d736-4d66-ada0-9e160f26a88e\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-5b44787fb-rl8cj" Jul 06 02:40:55.859325 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:40:55.859306 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-proxy-server-cert\" (UniqueName: \"kubernetes.io/secret/60871947-d3a3-44d5-afdb-c8e0c5aa9654-service-proxy-server-cert\") pod \"cluster-proxy-proxy-agent-7f946cb44b-w5gkh\" (UID: \"60871947-d3a3-44d5-afdb-c8e0c5aa9654\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7f946cb44b-w5gkh" Jul 06 02:40:55.859399 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:40:55.859370 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/60871947-d3a3-44d5-afdb-c8e0c5aa9654-hub-kubeconfig\") pod \"cluster-proxy-proxy-agent-7f946cb44b-w5gkh\" (UID: \"60871947-d3a3-44d5-afdb-c8e0c5aa9654\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7f946cb44b-w5gkh" Jul 06 02:40:55.864040 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:40:55.864018 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-gp9s2\" (UniqueName: \"kubernetes.io/projected/6efb58f8-d736-4d66-ada0-9e160f26a88e-kube-api-access-gp9s2\") pod \"managed-serviceaccount-addon-agent-5b44787fb-rl8cj\" (UID: \"6efb58f8-d736-4d66-ada0-9e160f26a88e\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-5b44787fb-rl8cj" Jul 06 02:40:55.864169 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:40:55.864096 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-4dmmb\" (UniqueName: \"kubernetes.io/projected/60871947-d3a3-44d5-afdb-c8e0c5aa9654-kube-api-access-4dmmb\") pod \"cluster-proxy-proxy-agent-7f946cb44b-w5gkh\" (UID: \"60871947-d3a3-44d5-afdb-c8e0c5aa9654\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7f946cb44b-w5gkh" Jul 06 02:40:55.999336 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:40:55.999289 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-5b44787fb-rl8cj" Jul 06 02:40:56.008923 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:40:56.008895 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7f946cb44b-w5gkh" Jul 06 02:40:56.133515 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:40:56.133483 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-5b44787fb-rl8cj"] Jul 06 02:40:56.136830 ip-10-0-139-206 kubenswrapper[2557]: W0706 02:40:56.136801 2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod6efb58f8_d736_4d66_ada0_9e160f26a88e.slice/crio-9d9c9c514797f8aea364173fdabcc5357558f9d7bc6fff1dfdd0043f78fbd96e WatchSource:0}: Error finding container 9d9c9c514797f8aea364173fdabcc5357558f9d7bc6fff1dfdd0043f78fbd96e: Status 404 returned error can't find the container with id 9d9c9c514797f8aea364173fdabcc5357558f9d7bc6fff1dfdd0043f78fbd96e Jul 06 02:40:56.151622 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:40:56.151592 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7f946cb44b-w5gkh"] Jul 06 02:40:56.156307 ip-10-0-139-206 kubenswrapper[2557]: W0706 02:40:56.156278 2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod60871947_d3a3_44d5_afdb_c8e0c5aa9654.slice/crio-c1691a26ecaacb2e4fcd8a0c32ad7a9e3a7892383729426e70a2e1462b3edddc WatchSource:0}: Error finding container c1691a26ecaacb2e4fcd8a0c32ad7a9e3a7892383729426e70a2e1462b3edddc: Status 404 returned error can't find the container with id c1691a26ecaacb2e4fcd8a0c32ad7a9e3a7892383729426e70a2e1462b3edddc Jul 06 02:40:56.362110 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:40:56.362002 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/974aff39-ecbd-4788-a159-d35d2a73edb6-metrics-tls\") pod \"dns-default-8jcmv\" (UID: \"974aff39-ecbd-4788-a159-d35d2a73edb6\") " pod="openshift-dns/dns-default-8jcmv" Jul 06 02:40:56.362290 ip-10-0-139-206 kubenswrapper[2557]: E0706 02:40:56.362166 2557 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 06 02:40:56.362290 ip-10-0-139-206 kubenswrapper[2557]: E0706 02:40:56.362248 2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/974aff39-ecbd-4788-a159-d35d2a73edb6-metrics-tls podName:974aff39-ecbd-4788-a159-d35d2a73edb6 nodeName:}" failed. No retries permitted until 2026-07-06 02:42:00.362226928 +0000 UTC m=+164.168512989 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/974aff39-ecbd-4788-a159-d35d2a73edb6-metrics-tls") pod "dns-default-8jcmv" (UID: "974aff39-ecbd-4788-a159-d35d2a73edb6") : secret "dns-default-metrics-tls" not found Jul 06 02:40:56.670280 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:40:56.670189 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7f946cb44b-w5gkh" event={"ID":"60871947-d3a3-44d5-afdb-c8e0c5aa9654","Type":"ContainerStarted","Data":"c1691a26ecaacb2e4fcd8a0c32ad7a9e3a7892383729426e70a2e1462b3edddc"} Jul 06 02:40:56.671226 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:40:56.671202 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-5b44787fb-rl8cj" event={"ID":"6efb58f8-d736-4d66-ada0-9e160f26a88e","Type":"ContainerStarted","Data":"9d9c9c514797f8aea364173fdabcc5357558f9d7bc6fff1dfdd0043f78fbd96e"} Jul 06 02:40:59.605574 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:40:59.605545 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-l2btb" Jul 06 02:40:59.682539 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:40:59.682505 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-5b44787fb-rl8cj" event={"ID":"6efb58f8-d736-4d66-ada0-9e160f26a88e","Type":"ContainerStarted","Data":"83ce5479d2d6303539663c172990661724d45e775521be57a48b44e6a425f187"} Jul 06 02:40:59.696405 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:40:59.696326 2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-5b44787fb-rl8cj" podStartSLOduration=1.427350413 podStartE2EDuration="4.696300294s" totalImagesPullingTime="3.268949881s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 02:40:55 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 02:40:59.695913033 +0000 UTC m=+103.502199091" watchObservedRunningTime="2026-07-06 02:40:59.696300294 +0000 UTC m=+103.502586353" Jul 06 02:41:02.692576 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:41:02.692540 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7f946cb44b-w5gkh" event={"ID":"60871947-d3a3-44d5-afdb-c8e0c5aa9654","Type":"ContainerStarted","Data":"c72206c92b1b663c38ec3c6f14c7c0c02395e5f18e43acbd7311e2cd38bd58c7"} Jul 06 02:41:02.692576 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:41:02.692583 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7f946cb44b-w5gkh" event={"ID":"60871947-d3a3-44d5-afdb-c8e0c5aa9654","Type":"ContainerStarted","Data":"b2caee014a3b00c12d6f15f83bf08f93eaca8c011a8afaaca9068c6c25e8166e"} Jul 06 02:41:02.693139 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:41:02.692595 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7f946cb44b-w5gkh" event={"ID":"60871947-d3a3-44d5-afdb-c8e0c5aa9654","Type":"ContainerStarted","Data":"2a2e65c9f499d17a83c59c9e2430d089ccbd09d341df56de4ccaab8644cd7a80"} Jul 06 02:41:02.711486 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:41:02.711432 2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7f946cb44b-w5gkh" podStartSLOduration=2.107067792 podStartE2EDuration="7.711413571s" totalImagesPullingTime="5.604345779s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 02:40:55 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 02:41:02.710016658 +0000 UTC m=+106.516302729" watchObservedRunningTime="2026-07-06 02:41:02.711413571 +0000 UTC m=+106.517699629" Jul 06 02:41:27.094392 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:41:27.094324 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/26f69a47-e949-4e18-9d9c-6648a959e281-metrics-certs\") pod \"network-metrics-daemon-756n2\" (UID: \"26f69a47-e949-4e18-9d9c-6648a959e281\") " pod="openshift-multus/network-metrics-daemon-756n2" Jul 06 02:41:27.094986 ip-10-0-139-206 kubenswrapper[2557]: E0706 02:41:27.094513 2557 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found Jul 06 02:41:27.094986 ip-10-0-139-206 kubenswrapper[2557]: E0706 02:41:27.094601 2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/26f69a47-e949-4e18-9d9c-6648a959e281-metrics-certs podName:26f69a47-e949-4e18-9d9c-6648a959e281 nodeName:}" failed. No retries permitted until 2026-07-06 02:43:29.094578682 +0000 UTC m=+252.900864735 (durationBeforeRetry 2m2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/26f69a47-e949-4e18-9d9c-6648a959e281-metrics-certs") pod "network-metrics-daemon-756n2" (UID: "26f69a47-e949-4e18-9d9c-6648a959e281") : secret "metrics-daemon-secret" not found Jul 06 02:41:55.534107 ip-10-0-139-206 kubenswrapper[2557]: E0706 02:41:55.534057 2557 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[metrics-tls], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-dns/dns-default-8jcmv" podUID="974aff39-ecbd-4788-a159-d35d2a73edb6" Jul 06 02:41:55.845913 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:41:55.845826 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-8jcmv" Jul 06 02:41:56.323778 ip-10-0-139-206 kubenswrapper[2557]: E0706 02:41:56.323728 2557 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[metrics-certs], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-multus/network-metrics-daemon-756n2" podUID="26f69a47-e949-4e18-9d9c-6648a959e281" Jul 06 02:42:00.427391 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:42:00.427321 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/974aff39-ecbd-4788-a159-d35d2a73edb6-metrics-tls\") pod \"dns-default-8jcmv\" (UID: \"974aff39-ecbd-4788-a159-d35d2a73edb6\") " pod="openshift-dns/dns-default-8jcmv" Jul 06 02:42:00.427795 ip-10-0-139-206 kubenswrapper[2557]: E0706 02:42:00.427481 2557 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 06 02:42:00.427795 ip-10-0-139-206 kubenswrapper[2557]: E0706 02:42:00.427549 2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/974aff39-ecbd-4788-a159-d35d2a73edb6-metrics-tls podName:974aff39-ecbd-4788-a159-d35d2a73edb6 nodeName:}" failed. No retries permitted until 2026-07-06 02:44:02.427532689 +0000 UTC m=+286.233818724 (durationBeforeRetry 2m2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/974aff39-ecbd-4788-a159-d35d2a73edb6-metrics-tls") pod "dns-default-8jcmv" (UID: "974aff39-ecbd-4788-a159-d35d2a73edb6") : secret "dns-default-metrics-tls" not found Jul 06 02:42:10.296341 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:42:10.296262 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-756n2" Jul 06 02:43:00.031390 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:43:00.031342 2557 generic.go:356] "Generic (PLEG): container finished" podID="6efb58f8-d736-4d66-ada0-9e160f26a88e" containerID="83ce5479d2d6303539663c172990661724d45e775521be57a48b44e6a425f187" exitCode=255 Jul 06 02:43:00.031802 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:43:00.031416 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-5b44787fb-rl8cj" event={"ID":"6efb58f8-d736-4d66-ada0-9e160f26a88e","Type":"ContainerDied","Data":"83ce5479d2d6303539663c172990661724d45e775521be57a48b44e6a425f187"} Jul 06 02:43:00.031802 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:43:00.031718 2557 scope.go:117] "RemoveContainer" containerID="83ce5479d2d6303539663c172990661724d45e775521be57a48b44e6a425f187" Jul 06 02:43:01.036705 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:43:01.036670 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-5b44787fb-rl8cj" event={"ID":"6efb58f8-d736-4d66-ada0-9e160f26a88e","Type":"ContainerStarted","Data":"f90cb9e24ec0e6798b5518a108d07c31cc6ce6b57b4701018041194763c14af2"} Jul 06 02:43:23.227255 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:43:23.227220 2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-g65v5"] Jul 06 02:43:23.229163 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:43:23.229145 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-g65v5" Jul 06 02:43:23.231448 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:43:23.231411 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\"" Jul 06 02:43:23.231630 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:43:23.231487 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\"" Jul 06 02:43:23.232380 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:43:23.232358 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\"" Jul 06 02:43:23.232495 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:43:23.232385 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-zj59f\"" Jul 06 02:43:23.232495 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:43:23.232360 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\"" Jul 06 02:43:23.244597 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:43:23.244566 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-g65v5"] Jul 06 02:43:23.397063 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:43:23.397026 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/77908dbd-b4f5-498a-8ccf-731614f02373-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-g65v5\" (UID: \"77908dbd-b4f5-498a-8ccf-731614f02373\") " pod="openshift-insights/insights-runtime-extractor-g65v5" Jul 06 02:43:23.397238 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:43:23.397083 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kdgvm\" (UniqueName: \"kubernetes.io/projected/77908dbd-b4f5-498a-8ccf-731614f02373-kube-api-access-kdgvm\") pod \"insights-runtime-extractor-g65v5\" (UID: \"77908dbd-b4f5-498a-8ccf-731614f02373\") " pod="openshift-insights/insights-runtime-extractor-g65v5" Jul 06 02:43:23.397238 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:43:23.397125 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/77908dbd-b4f5-498a-8ccf-731614f02373-data-volume\") pod \"insights-runtime-extractor-g65v5\" (UID: \"77908dbd-b4f5-498a-8ccf-731614f02373\") " pod="openshift-insights/insights-runtime-extractor-g65v5" Jul 06 02:43:23.397238 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:43:23.397146 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/77908dbd-b4f5-498a-8ccf-731614f02373-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-g65v5\" (UID: \"77908dbd-b4f5-498a-8ccf-731614f02373\") " pod="openshift-insights/insights-runtime-extractor-g65v5" Jul 06 02:43:23.397238 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:43:23.397183 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/77908dbd-b4f5-498a-8ccf-731614f02373-crio-socket\") pod \"insights-runtime-extractor-g65v5\" (UID: \"77908dbd-b4f5-498a-8ccf-731614f02373\") " pod="openshift-insights/insights-runtime-extractor-g65v5" Jul 06 02:43:23.497548 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:43:23.497464 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/77908dbd-b4f5-498a-8ccf-731614f02373-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-g65v5\" (UID: \"77908dbd-b4f5-498a-8ccf-731614f02373\") " pod="openshift-insights/insights-runtime-extractor-g65v5" Jul 06 02:43:23.497548 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:43:23.497520 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kdgvm\" (UniqueName: \"kubernetes.io/projected/77908dbd-b4f5-498a-8ccf-731614f02373-kube-api-access-kdgvm\") pod \"insights-runtime-extractor-g65v5\" (UID: \"77908dbd-b4f5-498a-8ccf-731614f02373\") " pod="openshift-insights/insights-runtime-extractor-g65v5" Jul 06 02:43:23.497769 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:43:23.497559 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/77908dbd-b4f5-498a-8ccf-731614f02373-data-volume\") pod \"insights-runtime-extractor-g65v5\" (UID: \"77908dbd-b4f5-498a-8ccf-731614f02373\") " pod="openshift-insights/insights-runtime-extractor-g65v5" Jul 06 02:43:23.497769 ip-10-0-139-206 kubenswrapper[2557]: E0706 02:43:23.497598 2557 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 06 02:43:23.497769 ip-10-0-139-206 kubenswrapper[2557]: E0706 02:43:23.497672 2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/77908dbd-b4f5-498a-8ccf-731614f02373-kube-rbac-proxy-cm podName:77908dbd-b4f5-498a-8ccf-731614f02373 nodeName:}" failed. No retries permitted until 2026-07-06 02:43:23.997653549 +0000 UTC m=+247.803939605 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/77908dbd-b4f5-498a-8ccf-731614f02373-kube-rbac-proxy-cm") pod "insights-runtime-extractor-g65v5" (UID: "77908dbd-b4f5-498a-8ccf-731614f02373") : configmap "kube-rbac-proxy" not found Jul 06 02:43:23.497769 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:43:23.497715 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/77908dbd-b4f5-498a-8ccf-731614f02373-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-g65v5\" (UID: \"77908dbd-b4f5-498a-8ccf-731614f02373\") " pod="openshift-insights/insights-runtime-extractor-g65v5" Jul 06 02:43:23.497769 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:43:23.497758 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/77908dbd-b4f5-498a-8ccf-731614f02373-crio-socket\") pod \"insights-runtime-extractor-g65v5\" (UID: \"77908dbd-b4f5-498a-8ccf-731614f02373\") " pod="openshift-insights/insights-runtime-extractor-g65v5" Jul 06 02:43:23.498004 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:43:23.497858 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/77908dbd-b4f5-498a-8ccf-731614f02373-crio-socket\") pod \"insights-runtime-extractor-g65v5\" (UID: \"77908dbd-b4f5-498a-8ccf-731614f02373\") " pod="openshift-insights/insights-runtime-extractor-g65v5" Jul 06 02:43:23.498004 ip-10-0-139-206 kubenswrapper[2557]: E0706 02:43:23.497874 2557 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 06 02:43:23.498004 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:43:23.497881 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/77908dbd-b4f5-498a-8ccf-731614f02373-data-volume\") pod \"insights-runtime-extractor-g65v5\" (UID: \"77908dbd-b4f5-498a-8ccf-731614f02373\") " pod="openshift-insights/insights-runtime-extractor-g65v5" Jul 06 02:43:23.498004 ip-10-0-139-206 kubenswrapper[2557]: E0706 02:43:23.497945 2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/77908dbd-b4f5-498a-8ccf-731614f02373-insights-runtime-extractor-tls podName:77908dbd-b4f5-498a-8ccf-731614f02373 nodeName:}" failed. No retries permitted until 2026-07-06 02:43:23.997926062 +0000 UTC m=+247.804212106 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/77908dbd-b4f5-498a-8ccf-731614f02373-insights-runtime-extractor-tls") pod "insights-runtime-extractor-g65v5" (UID: "77908dbd-b4f5-498a-8ccf-731614f02373") : secret "insights-runtime-extractor-tls" not found Jul 06 02:43:23.505221 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:43:23.505198 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-kdgvm\" (UniqueName: \"kubernetes.io/projected/77908dbd-b4f5-498a-8ccf-731614f02373-kube-api-access-kdgvm\") pod \"insights-runtime-extractor-g65v5\" (UID: \"77908dbd-b4f5-498a-8ccf-731614f02373\") " pod="openshift-insights/insights-runtime-extractor-g65v5" Jul 06 02:43:24.002222 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:43:24.002177 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/77908dbd-b4f5-498a-8ccf-731614f02373-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-g65v5\" (UID: \"77908dbd-b4f5-498a-8ccf-731614f02373\") " pod="openshift-insights/insights-runtime-extractor-g65v5" Jul 06 02:43:24.002436 ip-10-0-139-206 kubenswrapper[2557]: E0706 02:43:24.002328 2557 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 06 02:43:24.002436 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:43:24.002373 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/77908dbd-b4f5-498a-8ccf-731614f02373-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-g65v5\" (UID: \"77908dbd-b4f5-498a-8ccf-731614f02373\") " pod="openshift-insights/insights-runtime-extractor-g65v5" Jul 06 02:43:24.002436 ip-10-0-139-206 kubenswrapper[2557]: E0706 02:43:24.002414 2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/77908dbd-b4f5-498a-8ccf-731614f02373-insights-runtime-extractor-tls podName:77908dbd-b4f5-498a-8ccf-731614f02373 nodeName:}" failed. No retries permitted until 2026-07-06 02:43:25.002398955 +0000 UTC m=+248.808684995 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/77908dbd-b4f5-498a-8ccf-731614f02373-insights-runtime-extractor-tls") pod "insights-runtime-extractor-g65v5" (UID: "77908dbd-b4f5-498a-8ccf-731614f02373") : secret "insights-runtime-extractor-tls" not found Jul 06 02:43:24.002548 ip-10-0-139-206 kubenswrapper[2557]: E0706 02:43:24.002441 2557 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 06 02:43:24.002548 ip-10-0-139-206 kubenswrapper[2557]: E0706 02:43:24.002516 2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/77908dbd-b4f5-498a-8ccf-731614f02373-kube-rbac-proxy-cm podName:77908dbd-b4f5-498a-8ccf-731614f02373 nodeName:}" failed. No retries permitted until 2026-07-06 02:43:25.002503239 +0000 UTC m=+248.808789275 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/77908dbd-b4f5-498a-8ccf-731614f02373-kube-rbac-proxy-cm") pod "insights-runtime-extractor-g65v5" (UID: "77908dbd-b4f5-498a-8ccf-731614f02373") : configmap "kube-rbac-proxy" not found Jul 06 02:43:25.004489 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:43:25.004454 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-bhp46_dd0a7167-560c-4195-857e-b69abeede707/dns-node-resolver/0.log" Jul 06 02:43:25.008801 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:43:25.008756 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/77908dbd-b4f5-498a-8ccf-731614f02373-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-g65v5\" (UID: \"77908dbd-b4f5-498a-8ccf-731614f02373\") " pod="openshift-insights/insights-runtime-extractor-g65v5" Jul 06 02:43:25.008938 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:43:25.008866 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/77908dbd-b4f5-498a-8ccf-731614f02373-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-g65v5\" (UID: \"77908dbd-b4f5-498a-8ccf-731614f02373\") " pod="openshift-insights/insights-runtime-extractor-g65v5" Jul 06 02:43:25.008981 ip-10-0-139-206 kubenswrapper[2557]: E0706 02:43:25.008940 2557 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 06 02:43:25.009014 ip-10-0-139-206 kubenswrapper[2557]: E0706 02:43:25.008976 2557 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 06 02:43:25.009047 ip-10-0-139-206 kubenswrapper[2557]: E0706 02:43:25.009023 2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/77908dbd-b4f5-498a-8ccf-731614f02373-insights-runtime-extractor-tls podName:77908dbd-b4f5-498a-8ccf-731614f02373 nodeName:}" failed. No retries permitted until 2026-07-06 02:43:27.009002373 +0000 UTC m=+250.815288430 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/77908dbd-b4f5-498a-8ccf-731614f02373-insights-runtime-extractor-tls") pod "insights-runtime-extractor-g65v5" (UID: "77908dbd-b4f5-498a-8ccf-731614f02373") : secret "insights-runtime-extractor-tls" not found Jul 06 02:43:25.009047 ip-10-0-139-206 kubenswrapper[2557]: E0706 02:43:25.009041 2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/77908dbd-b4f5-498a-8ccf-731614f02373-kube-rbac-proxy-cm podName:77908dbd-b4f5-498a-8ccf-731614f02373 nodeName:}" failed. No retries permitted until 2026-07-06 02:43:27.00903384 +0000 UTC m=+250.815319880 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/77908dbd-b4f5-498a-8ccf-731614f02373-kube-rbac-proxy-cm") pod "insights-runtime-extractor-g65v5" (UID: "77908dbd-b4f5-498a-8ccf-731614f02373") : configmap "kube-rbac-proxy" not found Jul 06 02:43:26.407418 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:43:26.407387 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-lwmh8_5b80db07-d76f-4470-a3d5-3176fdf63aa2/node-ca/0.log" Jul 06 02:43:27.024276 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:43:27.024222 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/77908dbd-b4f5-498a-8ccf-731614f02373-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-g65v5\" (UID: \"77908dbd-b4f5-498a-8ccf-731614f02373\") " pod="openshift-insights/insights-runtime-extractor-g65v5" Jul 06 02:43:27.024491 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:43:27.024314 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/77908dbd-b4f5-498a-8ccf-731614f02373-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-g65v5\" (UID: \"77908dbd-b4f5-498a-8ccf-731614f02373\") " pod="openshift-insights/insights-runtime-extractor-g65v5" Jul 06 02:43:27.024491 ip-10-0-139-206 kubenswrapper[2557]: E0706 02:43:27.024376 2557 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 06 02:43:27.024491 ip-10-0-139-206 kubenswrapper[2557]: E0706 02:43:27.024450 2557 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 06 02:43:27.024491 ip-10-0-139-206 kubenswrapper[2557]: E0706 02:43:27.024461 2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/77908dbd-b4f5-498a-8ccf-731614f02373-kube-rbac-proxy-cm podName:77908dbd-b4f5-498a-8ccf-731614f02373 nodeName:}" failed. No retries permitted until 2026-07-06 02:43:31.024441167 +0000 UTC m=+254.830727214 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/77908dbd-b4f5-498a-8ccf-731614f02373-kube-rbac-proxy-cm") pod "insights-runtime-extractor-g65v5" (UID: "77908dbd-b4f5-498a-8ccf-731614f02373") : configmap "kube-rbac-proxy" not found Jul 06 02:43:27.024491 ip-10-0-139-206 kubenswrapper[2557]: E0706 02:43:27.024488 2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/77908dbd-b4f5-498a-8ccf-731614f02373-insights-runtime-extractor-tls podName:77908dbd-b4f5-498a-8ccf-731614f02373 nodeName:}" failed. No retries permitted until 2026-07-06 02:43:31.024477769 +0000 UTC m=+254.830763810 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/77908dbd-b4f5-498a-8ccf-731614f02373-insights-runtime-extractor-tls") pod "insights-runtime-extractor-g65v5" (UID: "77908dbd-b4f5-498a-8ccf-731614f02373") : secret "insights-runtime-extractor-tls" not found Jul 06 02:43:27.204860 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:43:27.204830 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-qkmzt_faa07f8b-6058-4e7e-af53-4fc206f18730/serve-healthcheck-canary/0.log" Jul 06 02:43:29.139990 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:43:29.139950 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/26f69a47-e949-4e18-9d9c-6648a959e281-metrics-certs\") pod \"network-metrics-daemon-756n2\" (UID: \"26f69a47-e949-4e18-9d9c-6648a959e281\") " pod="openshift-multus/network-metrics-daemon-756n2" Jul 06 02:43:29.140448 ip-10-0-139-206 kubenswrapper[2557]: E0706 02:43:29.140102 2557 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found Jul 06 02:43:29.140448 ip-10-0-139-206 kubenswrapper[2557]: E0706 02:43:29.140171 2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/26f69a47-e949-4e18-9d9c-6648a959e281-metrics-certs podName:26f69a47-e949-4e18-9d9c-6648a959e281 nodeName:}" failed. No retries permitted until 2026-07-06 02:45:31.140155821 +0000 UTC m=+374.946441858 (durationBeforeRetry 2m2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/26f69a47-e949-4e18-9d9c-6648a959e281-metrics-certs") pod "network-metrics-daemon-756n2" (UID: "26f69a47-e949-4e18-9d9c-6648a959e281") : secret "metrics-daemon-secret" not found Jul 06 02:43:31.055201 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:43:31.055163 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/77908dbd-b4f5-498a-8ccf-731614f02373-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-g65v5\" (UID: \"77908dbd-b4f5-498a-8ccf-731614f02373\") " pod="openshift-insights/insights-runtime-extractor-g65v5" Jul 06 02:43:31.055201 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:43:31.055213 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/77908dbd-b4f5-498a-8ccf-731614f02373-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-g65v5\" (UID: \"77908dbd-b4f5-498a-8ccf-731614f02373\") " pod="openshift-insights/insights-runtime-extractor-g65v5" Jul 06 02:43:31.055658 ip-10-0-139-206 kubenswrapper[2557]: E0706 02:43:31.055288 2557 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 06 02:43:31.055658 ip-10-0-139-206 kubenswrapper[2557]: E0706 02:43:31.055311 2557 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 06 02:43:31.055658 ip-10-0-139-206 kubenswrapper[2557]: E0706 02:43:31.055400 2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/77908dbd-b4f5-498a-8ccf-731614f02373-insights-runtime-extractor-tls podName:77908dbd-b4f5-498a-8ccf-731614f02373 nodeName:}" failed. No retries permitted until 2026-07-06 02:43:39.05536616 +0000 UTC m=+262.861652213 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/77908dbd-b4f5-498a-8ccf-731614f02373-insights-runtime-extractor-tls") pod "insights-runtime-extractor-g65v5" (UID: "77908dbd-b4f5-498a-8ccf-731614f02373") : secret "insights-runtime-extractor-tls" not found Jul 06 02:43:31.055658 ip-10-0-139-206 kubenswrapper[2557]: E0706 02:43:31.055418 2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/77908dbd-b4f5-498a-8ccf-731614f02373-kube-rbac-proxy-cm podName:77908dbd-b4f5-498a-8ccf-731614f02373 nodeName:}" failed. No retries permitted until 2026-07-06 02:43:39.055408997 +0000 UTC m=+262.861695036 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/77908dbd-b4f5-498a-8ccf-731614f02373-kube-rbac-proxy-cm") pod "insights-runtime-extractor-g65v5" (UID: "77908dbd-b4f5-498a-8ccf-731614f02373") : configmap "kube-rbac-proxy" not found Jul 06 02:43:39.117132 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:43:39.117032 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/77908dbd-b4f5-498a-8ccf-731614f02373-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-g65v5\" (UID: \"77908dbd-b4f5-498a-8ccf-731614f02373\") " pod="openshift-insights/insights-runtime-extractor-g65v5" Jul 06 02:43:39.117132 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:43:39.117097 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/77908dbd-b4f5-498a-8ccf-731614f02373-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-g65v5\" (UID: \"77908dbd-b4f5-498a-8ccf-731614f02373\") " pod="openshift-insights/insights-runtime-extractor-g65v5" Jul 06 02:43:39.117796 ip-10-0-139-206 kubenswrapper[2557]: E0706 02:43:39.117242 2557 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 06 02:43:39.117796 ip-10-0-139-206 kubenswrapper[2557]: E0706 02:43:39.117319 2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/77908dbd-b4f5-498a-8ccf-731614f02373-insights-runtime-extractor-tls podName:77908dbd-b4f5-498a-8ccf-731614f02373 nodeName:}" failed. No retries permitted until 2026-07-06 02:43:55.117300409 +0000 UTC m=+278.923586460 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/77908dbd-b4f5-498a-8ccf-731614f02373-insights-runtime-extractor-tls") pod "insights-runtime-extractor-g65v5" (UID: "77908dbd-b4f5-498a-8ccf-731614f02373") : secret "insights-runtime-extractor-tls" not found Jul 06 02:43:39.117796 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:43:39.117596 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/77908dbd-b4f5-498a-8ccf-731614f02373-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-g65v5\" (UID: \"77908dbd-b4f5-498a-8ccf-731614f02373\") " pod="openshift-insights/insights-runtime-extractor-g65v5" Jul 06 02:43:46.010562 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:43:46.010510 2557 prober.go:120] "Probe failed" probeType="Liveness" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7f946cb44b-w5gkh" podUID="60871947-d3a3-44d5-afdb-c8e0c5aa9654" containerName="service-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 500" Jul 06 02:43:53.041858 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:43:53.041821 2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-d559d66d5-l22rc"] Jul 06 02:43:53.043734 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:43:53.043718 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-admission-webhook-d559d66d5-l22rc" Jul 06 02:43:53.046023 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:43:53.046000 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-admission-webhook-tls\"" Jul 06 02:43:53.046154 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:43:53.046134 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-admission-webhook-dockercfg-z2lfn\"" Jul 06 02:43:53.051969 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:43:53.051946 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-d559d66d5-l22rc"] Jul 06 02:43:53.106640 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:43:53.106600 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/f05e1cca-7bfa-449b-8ba2-efe0296386b8-tls-certificates\") pod \"prometheus-operator-admission-webhook-d559d66d5-l22rc\" (UID: \"f05e1cca-7bfa-449b-8ba2-efe0296386b8\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-d559d66d5-l22rc" Jul 06 02:43:53.207655 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:43:53.207613 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/f05e1cca-7bfa-449b-8ba2-efe0296386b8-tls-certificates\") pod \"prometheus-operator-admission-webhook-d559d66d5-l22rc\" (UID: \"f05e1cca-7bfa-449b-8ba2-efe0296386b8\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-d559d66d5-l22rc" Jul 06 02:43:53.210064 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:43:53.210042 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/f05e1cca-7bfa-449b-8ba2-efe0296386b8-tls-certificates\") pod \"prometheus-operator-admission-webhook-d559d66d5-l22rc\" (UID: \"f05e1cca-7bfa-449b-8ba2-efe0296386b8\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-d559d66d5-l22rc" Jul 06 02:43:53.353546 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:43:53.353443 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-admission-webhook-d559d66d5-l22rc" Jul 06 02:43:53.477156 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:43:53.477119 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-d559d66d5-l22rc"] Jul 06 02:43:53.480150 ip-10-0-139-206 kubenswrapper[2557]: W0706 02:43:53.480124 2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf05e1cca_7bfa_449b_8ba2_efe0296386b8.slice/crio-bbe82fd14d6a0ee49640c05800306e4c8c6f2680bacabaaa812aa0d451bac04a WatchSource:0}: Error finding container bbe82fd14d6a0ee49640c05800306e4c8c6f2680bacabaaa812aa0d451bac04a: Status 404 returned error can't find the container with id bbe82fd14d6a0ee49640c05800306e4c8c6f2680bacabaaa812aa0d451bac04a Jul 06 02:43:54.187833 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:43:54.187789 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-admission-webhook-d559d66d5-l22rc" event={"ID":"f05e1cca-7bfa-449b-8ba2-efe0296386b8","Type":"ContainerStarted","Data":"bbe82fd14d6a0ee49640c05800306e4c8c6f2680bacabaaa812aa0d451bac04a"} Jul 06 02:43:55.122158 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:43:55.122124 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/77908dbd-b4f5-498a-8ccf-731614f02373-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-g65v5\" (UID: \"77908dbd-b4f5-498a-8ccf-731614f02373\") " pod="openshift-insights/insights-runtime-extractor-g65v5" Jul 06 02:43:55.124394 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:43:55.124373 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/77908dbd-b4f5-498a-8ccf-731614f02373-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-g65v5\" (UID: \"77908dbd-b4f5-498a-8ccf-731614f02373\") " pod="openshift-insights/insights-runtime-extractor-g65v5" Jul 06 02:43:55.192215 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:43:55.192181 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-admission-webhook-d559d66d5-l22rc" event={"ID":"f05e1cca-7bfa-449b-8ba2-efe0296386b8","Type":"ContainerStarted","Data":"6074f9b3cc964c33f90207f41a59beb2d28c3346a37c4fb37b6d9a917ec76d5f"} Jul 06 02:43:55.192664 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:43:55.192378 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-operator-admission-webhook-d559d66d5-l22rc" Jul 06 02:43:55.196888 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:43:55.196863 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-operator-admission-webhook-d559d66d5-l22rc" Jul 06 02:43:55.206060 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:43:55.206014 2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-operator-admission-webhook-d559d66d5-l22rc" podStartSLOduration=1.087660169 podStartE2EDuration="2.205998872s" totalImagesPullingTime="1.118338703s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 02:43:53 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 02:43:55.205446241 +0000 UTC m=+279.011732301" watchObservedRunningTime="2026-07-06 02:43:55.205998872 +0000 UTC m=+279.012284931" Jul 06 02:43:55.338902 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:43:55.338863 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-g65v5" Jul 06 02:43:55.468168 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:43:55.468132 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-g65v5"] Jul 06 02:43:55.471830 ip-10-0-139-206 kubenswrapper[2557]: W0706 02:43:55.471804 2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod77908dbd_b4f5_498a_8ccf_731614f02373.slice/crio-fbf3f27cb4cdfd78def3b4872343bf01b713f6afaf89a00334fba1e0ab50b73e WatchSource:0}: Error finding container fbf3f27cb4cdfd78def3b4872343bf01b713f6afaf89a00334fba1e0ab50b73e: Status 404 returned error can't find the container with id fbf3f27cb4cdfd78def3b4872343bf01b713f6afaf89a00334fba1e0ab50b73e Jul 06 02:43:56.013826 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:43:56.013697 2557 prober.go:120] "Probe failed" probeType="Liveness" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7f946cb44b-w5gkh" podUID="60871947-d3a3-44d5-afdb-c8e0c5aa9654" containerName="service-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 500" Jul 06 02:43:56.200310 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:43:56.200260 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-g65v5" event={"ID":"77908dbd-b4f5-498a-8ccf-731614f02373","Type":"ContainerStarted","Data":"fbf3f27cb4cdfd78def3b4872343bf01b713f6afaf89a00334fba1e0ab50b73e"} Jul 06 02:43:58.285867 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:43:58.285823 2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-6c88fdb975-qx6tg"] Jul 06 02:43:58.289918 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:43:58.289894 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6c88fdb975-qx6tg" Jul 06 02:43:58.292155 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:43:58.292126 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-dockercfg-ffz2r\"" Jul 06 02:43:58.292287 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:43:58.292127 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"console-config\"" Jul 06 02:43:58.292376 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:43:58.292292 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"service-ca\"" Jul 06 02:43:58.293271 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:43:58.293116 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"openshift-service-ca.crt\"" Jul 06 02:43:58.293271 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:43:58.293150 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-oauth-config\"" Jul 06 02:43:58.293271 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:43:58.293220 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-serving-cert\"" Jul 06 02:43:58.293617 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:43:58.293596 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"kube-root-ca.crt\"" Jul 06 02:43:58.293735 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:43:58.293714 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"oauth-serving-cert\"" Jul 06 02:43:58.298177 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:43:58.298153 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"trusted-ca-bundle\"" Jul 06 02:43:58.298265 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:43:58.298213 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-6c88fdb975-qx6tg"] Jul 06 02:43:58.345083 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:43:58.345048 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/7652727c-1327-4bd3-a642-56242557c416-service-ca\") pod \"console-6c88fdb975-qx6tg\" (UID: \"7652727c-1327-4bd3-a642-56242557c416\") " pod="openshift-console/console-6c88fdb975-qx6tg" Jul 06 02:43:58.345280 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:43:58.345087 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/7652727c-1327-4bd3-a642-56242557c416-console-oauth-config\") pod \"console-6c88fdb975-qx6tg\" (UID: \"7652727c-1327-4bd3-a642-56242557c416\") " pod="openshift-console/console-6c88fdb975-qx6tg" Jul 06 02:43:58.345280 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:43:58.345177 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/7652727c-1327-4bd3-a642-56242557c416-console-config\") pod \"console-6c88fdb975-qx6tg\" (UID: \"7652727c-1327-4bd3-a642-56242557c416\") " pod="openshift-console/console-6c88fdb975-qx6tg" Jul 06 02:43:58.345280 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:43:58.345207 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-znktq\" (UniqueName: \"kubernetes.io/projected/7652727c-1327-4bd3-a642-56242557c416-kube-api-access-znktq\") pod \"console-6c88fdb975-qx6tg\" (UID: \"7652727c-1327-4bd3-a642-56242557c416\") " pod="openshift-console/console-6c88fdb975-qx6tg" Jul 06 02:43:58.345280 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:43:58.345223 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/7652727c-1327-4bd3-a642-56242557c416-console-serving-cert\") pod \"console-6c88fdb975-qx6tg\" (UID: \"7652727c-1327-4bd3-a642-56242557c416\") " pod="openshift-console/console-6c88fdb975-qx6tg" Jul 06 02:43:58.345280 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:43:58.345245 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/7652727c-1327-4bd3-a642-56242557c416-trusted-ca-bundle\") pod \"console-6c88fdb975-qx6tg\" (UID: \"7652727c-1327-4bd3-a642-56242557c416\") " pod="openshift-console/console-6c88fdb975-qx6tg" Jul 06 02:43:58.345280 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:43:58.345269 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/7652727c-1327-4bd3-a642-56242557c416-oauth-serving-cert\") pod \"console-6c88fdb975-qx6tg\" (UID: \"7652727c-1327-4bd3-a642-56242557c416\") " pod="openshift-console/console-6c88fdb975-qx6tg" Jul 06 02:43:58.446238 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:43:58.446205 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/7652727c-1327-4bd3-a642-56242557c416-console-config\") pod \"console-6c88fdb975-qx6tg\" (UID: \"7652727c-1327-4bd3-a642-56242557c416\") " pod="openshift-console/console-6c88fdb975-qx6tg" Jul 06 02:43:58.446426 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:43:58.446254 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-znktq\" (UniqueName: \"kubernetes.io/projected/7652727c-1327-4bd3-a642-56242557c416-kube-api-access-znktq\") pod \"console-6c88fdb975-qx6tg\" (UID: \"7652727c-1327-4bd3-a642-56242557c416\") " pod="openshift-console/console-6c88fdb975-qx6tg" Jul 06 02:43:58.446426 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:43:58.446288 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/7652727c-1327-4bd3-a642-56242557c416-console-serving-cert\") pod \"console-6c88fdb975-qx6tg\" (UID: \"7652727c-1327-4bd3-a642-56242557c416\") " pod="openshift-console/console-6c88fdb975-qx6tg" Jul 06 02:43:58.446426 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:43:58.446308 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/7652727c-1327-4bd3-a642-56242557c416-trusted-ca-bundle\") pod \"console-6c88fdb975-qx6tg\" (UID: \"7652727c-1327-4bd3-a642-56242557c416\") " pod="openshift-console/console-6c88fdb975-qx6tg" Jul 06 02:43:58.446426 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:43:58.446323 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/7652727c-1327-4bd3-a642-56242557c416-oauth-serving-cert\") pod \"console-6c88fdb975-qx6tg\" (UID: \"7652727c-1327-4bd3-a642-56242557c416\") " pod="openshift-console/console-6c88fdb975-qx6tg" Jul 06 02:43:58.446426 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:43:58.446360 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/7652727c-1327-4bd3-a642-56242557c416-service-ca\") pod \"console-6c88fdb975-qx6tg\" (UID: \"7652727c-1327-4bd3-a642-56242557c416\") " pod="openshift-console/console-6c88fdb975-qx6tg" Jul 06 02:43:58.446426 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:43:58.446386 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/7652727c-1327-4bd3-a642-56242557c416-console-oauth-config\") pod \"console-6c88fdb975-qx6tg\" (UID: \"7652727c-1327-4bd3-a642-56242557c416\") " pod="openshift-console/console-6c88fdb975-qx6tg" Jul 06 02:43:58.447065 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:43:58.447029 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/7652727c-1327-4bd3-a642-56242557c416-console-config\") pod \"console-6c88fdb975-qx6tg\" (UID: \"7652727c-1327-4bd3-a642-56242557c416\") " pod="openshift-console/console-6c88fdb975-qx6tg" Jul 06 02:43:58.447202 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:43:58.447097 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/7652727c-1327-4bd3-a642-56242557c416-service-ca\") pod \"console-6c88fdb975-qx6tg\" (UID: \"7652727c-1327-4bd3-a642-56242557c416\") " pod="openshift-console/console-6c88fdb975-qx6tg" Jul 06 02:43:58.447202 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:43:58.447097 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/7652727c-1327-4bd3-a642-56242557c416-oauth-serving-cert\") pod \"console-6c88fdb975-qx6tg\" (UID: \"7652727c-1327-4bd3-a642-56242557c416\") " pod="openshift-console/console-6c88fdb975-qx6tg" Jul 06 02:43:58.447445 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:43:58.447417 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/7652727c-1327-4bd3-a642-56242557c416-trusted-ca-bundle\") pod \"console-6c88fdb975-qx6tg\" (UID: \"7652727c-1327-4bd3-a642-56242557c416\") " pod="openshift-console/console-6c88fdb975-qx6tg" Jul 06 02:43:58.449097 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:43:58.449073 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/7652727c-1327-4bd3-a642-56242557c416-console-oauth-config\") pod \"console-6c88fdb975-qx6tg\" (UID: \"7652727c-1327-4bd3-a642-56242557c416\") " pod="openshift-console/console-6c88fdb975-qx6tg" Jul 06 02:43:58.449487 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:43:58.449465 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/7652727c-1327-4bd3-a642-56242557c416-console-serving-cert\") pod \"console-6c88fdb975-qx6tg\" (UID: \"7652727c-1327-4bd3-a642-56242557c416\") " pod="openshift-console/console-6c88fdb975-qx6tg" Jul 06 02:43:58.454868 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:43:58.454834 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-znktq\" (UniqueName: \"kubernetes.io/projected/7652727c-1327-4bd3-a642-56242557c416-kube-api-access-znktq\") pod \"console-6c88fdb975-qx6tg\" (UID: \"7652727c-1327-4bd3-a642-56242557c416\") " pod="openshift-console/console-6c88fdb975-qx6tg" Jul 06 02:43:58.601784 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:43:58.601692 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6c88fdb975-qx6tg" Jul 06 02:43:58.846364 ip-10-0-139-206 kubenswrapper[2557]: E0706 02:43:58.846304 2557 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[metrics-tls], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-dns/dns-default-8jcmv" podUID="974aff39-ecbd-4788-a159-d35d2a73edb6" Jul 06 02:43:59.190173 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:43:59.190142 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-6c88fdb975-qx6tg"] Jul 06 02:43:59.194091 ip-10-0-139-206 kubenswrapper[2557]: W0706 02:43:59.194063 2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7652727c_1327_4bd3_a642_56242557c416.slice/crio-1ddb757f7691e8a9c89d0f6b1142798d4cb3ad16837aa6cf4a2d050112b69633 WatchSource:0}: Error finding container 1ddb757f7691e8a9c89d0f6b1142798d4cb3ad16837aa6cf4a2d050112b69633: Status 404 returned error can't find the container with id 1ddb757f7691e8a9c89d0f6b1142798d4cb3ad16837aa6cf4a2d050112b69633 Jul 06 02:43:59.211741 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:43:59.211710 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-g65v5" event={"ID":"77908dbd-b4f5-498a-8ccf-731614f02373","Type":"ContainerStarted","Data":"3e88e7c581fbf03b09c66de3bb3396d1d9865aa0b2ed28b6d4582f4b50505b5b"} Jul 06 02:43:59.212808 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:43:59.212790 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-8jcmv" Jul 06 02:43:59.212884 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:43:59.212789 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6c88fdb975-qx6tg" event={"ID":"7652727c-1327-4bd3-a642-56242557c416","Type":"ContainerStarted","Data":"1ddb757f7691e8a9c89d0f6b1142798d4cb3ad16837aa6cf4a2d050112b69633"} Jul 06 02:44:00.218623 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:44:00.218540 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-g65v5" event={"ID":"77908dbd-b4f5-498a-8ccf-731614f02373","Type":"ContainerStarted","Data":"57225efa86f3ff211513011720c6895e7576288ebdb7b77ee98369628ce1a126"} Jul 06 02:44:02.476641 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:44:02.476601 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/974aff39-ecbd-4788-a159-d35d2a73edb6-metrics-tls\") pod \"dns-default-8jcmv\" (UID: \"974aff39-ecbd-4788-a159-d35d2a73edb6\") " pod="openshift-dns/dns-default-8jcmv" Jul 06 02:44:02.479277 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:44:02.479245 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/974aff39-ecbd-4788-a159-d35d2a73edb6-metrics-tls\") pod \"dns-default-8jcmv\" (UID: \"974aff39-ecbd-4788-a159-d35d2a73edb6\") " pod="openshift-dns/dns-default-8jcmv" Jul 06 02:44:02.515496 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:44:02.515465 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-lsrhl\"" Jul 06 02:44:02.523443 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:44:02.523411 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-8jcmv" Jul 06 02:44:03.098945 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:44:03.098912 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-8jcmv"] Jul 06 02:44:03.102082 ip-10-0-139-206 kubenswrapper[2557]: W0706 02:44:03.102053 2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod974aff39_ecbd_4788_a159_d35d2a73edb6.slice/crio-f0598c5d86180269f4969f5278828f3a52b196dc0e4e048689be40d6f391a2da WatchSource:0}: Error finding container f0598c5d86180269f4969f5278828f3a52b196dc0e4e048689be40d6f391a2da: Status 404 returned error can't find the container with id f0598c5d86180269f4969f5278828f3a52b196dc0e4e048689be40d6f391a2da Jul 06 02:44:03.231638 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:44:03.231548 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-g65v5" event={"ID":"77908dbd-b4f5-498a-8ccf-731614f02373","Type":"ContainerStarted","Data":"c78b499fc86985a5e2fb85b9fc32e55c04aee60c1b27fee3a8c396412a571e79"} Jul 06 02:44:03.233011 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:44:03.232984 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6c88fdb975-qx6tg" event={"ID":"7652727c-1327-4bd3-a642-56242557c416","Type":"ContainerStarted","Data":"eef1c1fa74bdc8e2834f2bc69218995687ddd8903dec9a205085d89f2319dcbd"} Jul 06 02:44:03.234050 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:44:03.234028 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-8jcmv" event={"ID":"974aff39-ecbd-4788-a159-d35d2a73edb6","Type":"ContainerStarted","Data":"f0598c5d86180269f4969f5278828f3a52b196dc0e4e048689be40d6f391a2da"} Jul 06 02:44:03.248420 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:44:03.248368 2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-g65v5" podStartSLOduration=32.887448829 podStartE2EDuration="40.248332432s" totalImagesPullingTime="7.360883603s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-06 02:43:23 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 02:44:03.247481487 +0000 UTC m=+287.053767544" watchObservedRunningTime="2026-07-06 02:44:03.248332432 +0000 UTC m=+287.054618471" Jul 06 02:44:03.264976 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:44:03.264921 2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-6c88fdb975-qx6tg" podStartSLOduration=1.4883913149999999 podStartE2EDuration="5.264903494s" totalImagesPullingTime="3.776512179s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 02:43:58 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 02:44:03.264080931 +0000 UTC m=+287.070366990" watchObservedRunningTime="2026-07-06 02:44:03.264903494 +0000 UTC m=+287.071189543" Jul 06 02:44:05.243173 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:44:05.243136 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-8jcmv" event={"ID":"974aff39-ecbd-4788-a159-d35d2a73edb6","Type":"ContainerStarted","Data":"a32a1cde3e16d52fadea8f999d16f7c9f277c6269d4ee133eaa3abb10e09f367"} Jul 06 02:44:05.243619 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:44:05.243190 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-8jcmv" event={"ID":"974aff39-ecbd-4788-a159-d35d2a73edb6","Type":"ContainerStarted","Data":"947574df8d47d1c85a3ca594511e2f3e6c12dd5819e83a1a3d6e42142bc72c2d"} Jul 06 02:44:05.243619 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:44:05.243297 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-8jcmv" Jul 06 02:44:05.259534 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:44:05.259477 2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/dns-default-8jcmv" podStartSLOduration=251.582269715 podStartE2EDuration="4m13.259458656s" totalImagesPullingTime="1.677188941s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 02:39:52 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 02:44:05.257623034 +0000 UTC m=+289.063909092" watchObservedRunningTime="2026-07-06 02:44:05.259458656 +0000 UTC m=+289.065744713" Jul 06 02:44:06.011006 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:44:06.010966 2557 prober.go:120] "Probe failed" probeType="Liveness" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7f946cb44b-w5gkh" podUID="60871947-d3a3-44d5-afdb-c8e0c5aa9654" containerName="service-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 500" Jul 06 02:44:06.011173 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:44:06.011038 2557 kubelet.go:2654] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7f946cb44b-w5gkh" Jul 06 02:44:06.011613 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:44:06.011582 2557 kuberuntime_manager.go:1192] "Message for Container of pod" containerName="service-proxy" containerStatusID={"Type":"cri-o","ID":"c72206c92b1b663c38ec3c6f14c7c0c02395e5f18e43acbd7311e2cd38bd58c7"} pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7f946cb44b-w5gkh" containerMessage="Container service-proxy failed liveness probe, will be restarted" Jul 06 02:44:06.011669 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:44:06.011638 2557 kuberuntime_container.go:872] "Killing container with a grace period" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7f946cb44b-w5gkh" podUID="60871947-d3a3-44d5-afdb-c8e0c5aa9654" containerName="service-proxy" containerID="cri-o://c72206c92b1b663c38ec3c6f14c7c0c02395e5f18e43acbd7311e2cd38bd58c7" gracePeriod=30 Jul 06 02:44:06.248600 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:44:06.248558 2557 generic.go:356] "Generic (PLEG): container finished" podID="60871947-d3a3-44d5-afdb-c8e0c5aa9654" containerID="c72206c92b1b663c38ec3c6f14c7c0c02395e5f18e43acbd7311e2cd38bd58c7" exitCode=2 Jul 06 02:44:06.248977 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:44:06.248628 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7f946cb44b-w5gkh" event={"ID":"60871947-d3a3-44d5-afdb-c8e0c5aa9654","Type":"ContainerDied","Data":"c72206c92b1b663c38ec3c6f14c7c0c02395e5f18e43acbd7311e2cd38bd58c7"} Jul 06 02:44:06.248977 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:44:06.248671 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7f946cb44b-w5gkh" event={"ID":"60871947-d3a3-44d5-afdb-c8e0c5aa9654","Type":"ContainerStarted","Data":"456afde41eb066cf08a51455350d9a47b73b5a33e0c81ef885891bd9fd7d0c77"} Jul 06 02:44:08.602550 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:44:08.602501 2557 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-6c88fdb975-qx6tg" Jul 06 02:44:08.602966 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:44:08.602572 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-6c88fdb975-qx6tg" Jul 06 02:44:08.607397 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:44:08.607369 2557 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-6c88fdb975-qx6tg" Jul 06 02:44:09.262858 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:44:09.262826 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-6c88fdb975-qx6tg" Jul 06 02:44:10.463524 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:44:10.463484 2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/kube-state-metrics-7c9dbb8b7d-nt64l"] Jul 06 02:44:10.466541 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:44:10.466514 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-nt64l" Jul 06 02:44:10.466957 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:44:10.466936 2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-5ktdz"] Jul 06 02:44:10.470609 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:44:10.470579 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\"" Jul 06 02:44:10.470741 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:44:10.470670 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-custom-resource-state-configmap\"" Jul 06 02:44:10.471008 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:44:10.470987 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\"" Jul 06 02:44:10.471084 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:44:10.471063 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\"" Jul 06 02:44:10.471288 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:44:10.471270 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-tls\"" Jul 06 02:44:10.471450 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:44:10.471430 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-dockercfg-vhcq8\"" Jul 06 02:44:10.471632 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:44:10.471615 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-kube-rbac-proxy-config\"" Jul 06 02:44:10.472933 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:44:10.472910 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-5ktdz" Jul 06 02:44:10.475271 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:44:10.474978 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\"" Jul 06 02:44:10.475271 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:44:10.475250 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-6wxmd\"" Jul 06 02:44:10.475489 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:44:10.475470 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\"" Jul 06 02:44:10.475680 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:44:10.475663 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\"" Jul 06 02:44:10.477521 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:44:10.477502 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/kube-state-metrics-7c9dbb8b7d-nt64l"] Jul 06 02:44:10.538498 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:44:10.538457 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/e45d2d95-4d84-4c3a-8809-e4e1b3809bf3-node-exporter-textfile\") pod \"node-exporter-5ktdz\" (UID: \"e45d2d95-4d84-4c3a-8809-e4e1b3809bf3\") " pod="openshift-monitoring/node-exporter-5ktdz" Jul 06 02:44:10.538498 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:44:10.538494 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/e45d2d95-4d84-4c3a-8809-e4e1b3809bf3-sys\") pod \"node-exporter-5ktdz\" (UID: \"e45d2d95-4d84-4c3a-8809-e4e1b3809bf3\") " pod="openshift-monitoring/node-exporter-5ktdz" Jul 06 02:44:10.538727 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:44:10.538512 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/e45d2d95-4d84-4c3a-8809-e4e1b3809bf3-root\") pod \"node-exporter-5ktdz\" (UID: \"e45d2d95-4d84-4c3a-8809-e4e1b3809bf3\") " pod="openshift-monitoring/node-exporter-5ktdz" Jul 06 02:44:10.538727 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:44:10.538530 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/e45d2d95-4d84-4c3a-8809-e4e1b3809bf3-node-exporter-tls\") pod \"node-exporter-5ktdz\" (UID: \"e45d2d95-4d84-4c3a-8809-e4e1b3809bf3\") " pod="openshift-monitoring/node-exporter-5ktdz" Jul 06 02:44:10.538727 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:44:10.538555 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lx5bb\" (UniqueName: \"kubernetes.io/projected/e45d2d95-4d84-4c3a-8809-e4e1b3809bf3-kube-api-access-lx5bb\") pod \"node-exporter-5ktdz\" (UID: \"e45d2d95-4d84-4c3a-8809-e4e1b3809bf3\") " pod="openshift-monitoring/node-exporter-5ktdz" Jul 06 02:44:10.538727 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:44:10.538578 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vg95r\" (UniqueName: \"kubernetes.io/projected/23df5184-9bbf-49ba-82fa-e9f025937b4e-kube-api-access-vg95r\") pod \"kube-state-metrics-7c9dbb8b7d-nt64l\" (UID: \"23df5184-9bbf-49ba-82fa-e9f025937b4e\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-nt64l" Jul 06 02:44:10.538727 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:44:10.538599 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/23df5184-9bbf-49ba-82fa-e9f025937b4e-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-7c9dbb8b7d-nt64l\" (UID: \"23df5184-9bbf-49ba-82fa-e9f025937b4e\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-nt64l" Jul 06 02:44:10.538727 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:44:10.538623 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/e45d2d95-4d84-4c3a-8809-e4e1b3809bf3-metrics-client-ca\") pod \"node-exporter-5ktdz\" (UID: \"e45d2d95-4d84-4c3a-8809-e4e1b3809bf3\") " pod="openshift-monitoring/node-exporter-5ktdz" Jul 06 02:44:10.538727 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:44:10.538669 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/23df5184-9bbf-49ba-82fa-e9f025937b4e-volume-directive-shadow\") pod \"kube-state-metrics-7c9dbb8b7d-nt64l\" (UID: \"23df5184-9bbf-49ba-82fa-e9f025937b4e\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-nt64l" Jul 06 02:44:10.538727 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:44:10.538718 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/23df5184-9bbf-49ba-82fa-e9f025937b4e-kube-state-metrics-tls\") pod \"kube-state-metrics-7c9dbb8b7d-nt64l\" (UID: \"23df5184-9bbf-49ba-82fa-e9f025937b4e\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-nt64l" Jul 06 02:44:10.539071 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:44:10.538739 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/e45d2d95-4d84-4c3a-8809-e4e1b3809bf3-node-exporter-wtmp\") pod \"node-exporter-5ktdz\" (UID: \"e45d2d95-4d84-4c3a-8809-e4e1b3809bf3\") " pod="openshift-monitoring/node-exporter-5ktdz" Jul 06 02:44:10.539071 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:44:10.538799 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/e45d2d95-4d84-4c3a-8809-e4e1b3809bf3-node-exporter-accelerators-collector-config\") pod \"node-exporter-5ktdz\" (UID: \"e45d2d95-4d84-4c3a-8809-e4e1b3809bf3\") " pod="openshift-monitoring/node-exporter-5ktdz" Jul 06 02:44:10.539071 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:44:10.538844 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/23df5184-9bbf-49ba-82fa-e9f025937b4e-metrics-client-ca\") pod \"kube-state-metrics-7c9dbb8b7d-nt64l\" (UID: \"23df5184-9bbf-49ba-82fa-e9f025937b4e\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-nt64l" Jul 06 02:44:10.539071 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:44:10.538872 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/e45d2d95-4d84-4c3a-8809-e4e1b3809bf3-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-5ktdz\" (UID: \"e45d2d95-4d84-4c3a-8809-e4e1b3809bf3\") " pod="openshift-monitoring/node-exporter-5ktdz" Jul 06 02:44:10.539071 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:44:10.538914 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/23df5184-9bbf-49ba-82fa-e9f025937b4e-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-7c9dbb8b7d-nt64l\" (UID: \"23df5184-9bbf-49ba-82fa-e9f025937b4e\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-nt64l" Jul 06 02:44:10.639817 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:44:10.639772 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/23df5184-9bbf-49ba-82fa-e9f025937b4e-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-7c9dbb8b7d-nt64l\" (UID: \"23df5184-9bbf-49ba-82fa-e9f025937b4e\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-nt64l" Jul 06 02:44:10.639817 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:44:10.639822 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/e45d2d95-4d84-4c3a-8809-e4e1b3809bf3-node-exporter-textfile\") pod \"node-exporter-5ktdz\" (UID: \"e45d2d95-4d84-4c3a-8809-e4e1b3809bf3\") " pod="openshift-monitoring/node-exporter-5ktdz" Jul 06 02:44:10.640067 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:44:10.639849 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/e45d2d95-4d84-4c3a-8809-e4e1b3809bf3-sys\") pod \"node-exporter-5ktdz\" (UID: \"e45d2d95-4d84-4c3a-8809-e4e1b3809bf3\") " pod="openshift-monitoring/node-exporter-5ktdz" Jul 06 02:44:10.640067 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:44:10.639872 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/e45d2d95-4d84-4c3a-8809-e4e1b3809bf3-root\") pod \"node-exporter-5ktdz\" (UID: \"e45d2d95-4d84-4c3a-8809-e4e1b3809bf3\") " pod="openshift-monitoring/node-exporter-5ktdz" Jul 06 02:44:10.640067 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:44:10.639897 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/e45d2d95-4d84-4c3a-8809-e4e1b3809bf3-node-exporter-tls\") pod \"node-exporter-5ktdz\" (UID: \"e45d2d95-4d84-4c3a-8809-e4e1b3809bf3\") " pod="openshift-monitoring/node-exporter-5ktdz" Jul 06 02:44:10.640067 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:44:10.639921 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lx5bb\" (UniqueName: \"kubernetes.io/projected/e45d2d95-4d84-4c3a-8809-e4e1b3809bf3-kube-api-access-lx5bb\") pod \"node-exporter-5ktdz\" (UID: \"e45d2d95-4d84-4c3a-8809-e4e1b3809bf3\") " pod="openshift-monitoring/node-exporter-5ktdz" Jul 06 02:44:10.640067 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:44:10.639981 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/e45d2d95-4d84-4c3a-8809-e4e1b3809bf3-root\") pod \"node-exporter-5ktdz\" (UID: \"e45d2d95-4d84-4c3a-8809-e4e1b3809bf3\") " pod="openshift-monitoring/node-exporter-5ktdz" Jul 06 02:44:10.640067 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:44:10.639993 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/e45d2d95-4d84-4c3a-8809-e4e1b3809bf3-sys\") pod \"node-exporter-5ktdz\" (UID: \"e45d2d95-4d84-4c3a-8809-e4e1b3809bf3\") " pod="openshift-monitoring/node-exporter-5ktdz" Jul 06 02:44:10.640067 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:44:10.640032 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vg95r\" (UniqueName: \"kubernetes.io/projected/23df5184-9bbf-49ba-82fa-e9f025937b4e-kube-api-access-vg95r\") pod \"kube-state-metrics-7c9dbb8b7d-nt64l\" (UID: \"23df5184-9bbf-49ba-82fa-e9f025937b4e\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-nt64l" Jul 06 02:44:10.640067 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:44:10.640068 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/23df5184-9bbf-49ba-82fa-e9f025937b4e-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-7c9dbb8b7d-nt64l\" (UID: \"23df5184-9bbf-49ba-82fa-e9f025937b4e\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-nt64l" Jul 06 02:44:10.640471 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:44:10.640086 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/e45d2d95-4d84-4c3a-8809-e4e1b3809bf3-metrics-client-ca\") pod \"node-exporter-5ktdz\" (UID: \"e45d2d95-4d84-4c3a-8809-e4e1b3809bf3\") " pod="openshift-monitoring/node-exporter-5ktdz" Jul 06 02:44:10.640471 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:44:10.640108 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/23df5184-9bbf-49ba-82fa-e9f025937b4e-volume-directive-shadow\") pod \"kube-state-metrics-7c9dbb8b7d-nt64l\" (UID: \"23df5184-9bbf-49ba-82fa-e9f025937b4e\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-nt64l" Jul 06 02:44:10.640471 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:44:10.640142 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/23df5184-9bbf-49ba-82fa-e9f025937b4e-kube-state-metrics-tls\") pod \"kube-state-metrics-7c9dbb8b7d-nt64l\" (UID: \"23df5184-9bbf-49ba-82fa-e9f025937b4e\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-nt64l" Jul 06 02:44:10.640471 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:44:10.640162 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/e45d2d95-4d84-4c3a-8809-e4e1b3809bf3-node-exporter-wtmp\") pod \"node-exporter-5ktdz\" (UID: \"e45d2d95-4d84-4c3a-8809-e4e1b3809bf3\") " pod="openshift-monitoring/node-exporter-5ktdz" Jul 06 02:44:10.640471 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:44:10.640181 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/e45d2d95-4d84-4c3a-8809-e4e1b3809bf3-node-exporter-accelerators-collector-config\") pod \"node-exporter-5ktdz\" (UID: \"e45d2d95-4d84-4c3a-8809-e4e1b3809bf3\") " pod="openshift-monitoring/node-exporter-5ktdz" Jul 06 02:44:10.640471 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:44:10.640197 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/23df5184-9bbf-49ba-82fa-e9f025937b4e-metrics-client-ca\") pod \"kube-state-metrics-7c9dbb8b7d-nt64l\" (UID: \"23df5184-9bbf-49ba-82fa-e9f025937b4e\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-nt64l" Jul 06 02:44:10.640471 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:44:10.640215 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/e45d2d95-4d84-4c3a-8809-e4e1b3809bf3-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-5ktdz\" (UID: \"e45d2d95-4d84-4c3a-8809-e4e1b3809bf3\") " pod="openshift-monitoring/node-exporter-5ktdz" Jul 06 02:44:10.640471 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:44:10.640251 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/e45d2d95-4d84-4c3a-8809-e4e1b3809bf3-node-exporter-textfile\") pod \"node-exporter-5ktdz\" (UID: \"e45d2d95-4d84-4c3a-8809-e4e1b3809bf3\") " pod="openshift-monitoring/node-exporter-5ktdz" Jul 06 02:44:10.640863 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:44:10.640548 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/e45d2d95-4d84-4c3a-8809-e4e1b3809bf3-node-exporter-wtmp\") pod \"node-exporter-5ktdz\" (UID: \"e45d2d95-4d84-4c3a-8809-e4e1b3809bf3\") " pod="openshift-monitoring/node-exporter-5ktdz" Jul 06 02:44:10.640863 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:44:10.640598 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/23df5184-9bbf-49ba-82fa-e9f025937b4e-volume-directive-shadow\") pod \"kube-state-metrics-7c9dbb8b7d-nt64l\" (UID: \"23df5184-9bbf-49ba-82fa-e9f025937b4e\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-nt64l" Jul 06 02:44:10.640863 ip-10-0-139-206 kubenswrapper[2557]: E0706 02:44:10.640640 2557 secret.go:189] Couldn't get secret openshift-monitoring/kube-state-metrics-tls: secret "kube-state-metrics-tls" not found Jul 06 02:44:10.640863 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:44:10.640646 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/23df5184-9bbf-49ba-82fa-e9f025937b4e-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-7c9dbb8b7d-nt64l\" (UID: \"23df5184-9bbf-49ba-82fa-e9f025937b4e\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-nt64l" Jul 06 02:44:10.640863 ip-10-0-139-206 kubenswrapper[2557]: E0706 02:44:10.640707 2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/23df5184-9bbf-49ba-82fa-e9f025937b4e-kube-state-metrics-tls podName:23df5184-9bbf-49ba-82fa-e9f025937b4e nodeName:}" failed. No retries permitted until 2026-07-06 02:44:11.140690653 +0000 UTC m=+294.946976696 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-state-metrics-tls" (UniqueName: "kubernetes.io/secret/23df5184-9bbf-49ba-82fa-e9f025937b4e-kube-state-metrics-tls") pod "kube-state-metrics-7c9dbb8b7d-nt64l" (UID: "23df5184-9bbf-49ba-82fa-e9f025937b4e") : secret "kube-state-metrics-tls" not found Jul 06 02:44:10.641130 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:44:10.640886 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/e45d2d95-4d84-4c3a-8809-e4e1b3809bf3-metrics-client-ca\") pod \"node-exporter-5ktdz\" (UID: \"e45d2d95-4d84-4c3a-8809-e4e1b3809bf3\") " pod="openshift-monitoring/node-exporter-5ktdz" Jul 06 02:44:10.641130 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:44:10.641097 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/e45d2d95-4d84-4c3a-8809-e4e1b3809bf3-node-exporter-accelerators-collector-config\") pod \"node-exporter-5ktdz\" (UID: \"e45d2d95-4d84-4c3a-8809-e4e1b3809bf3\") " pod="openshift-monitoring/node-exporter-5ktdz" Jul 06 02:44:10.641407 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:44:10.641385 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/23df5184-9bbf-49ba-82fa-e9f025937b4e-metrics-client-ca\") pod \"kube-state-metrics-7c9dbb8b7d-nt64l\" (UID: \"23df5184-9bbf-49ba-82fa-e9f025937b4e\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-nt64l" Jul 06 02:44:10.642635 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:44:10.642616 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/e45d2d95-4d84-4c3a-8809-e4e1b3809bf3-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-5ktdz\" (UID: \"e45d2d95-4d84-4c3a-8809-e4e1b3809bf3\") " pod="openshift-monitoring/node-exporter-5ktdz" Jul 06 02:44:10.642863 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:44:10.642847 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/23df5184-9bbf-49ba-82fa-e9f025937b4e-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-7c9dbb8b7d-nt64l\" (UID: \"23df5184-9bbf-49ba-82fa-e9f025937b4e\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-nt64l" Jul 06 02:44:10.642900 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:44:10.642866 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/e45d2d95-4d84-4c3a-8809-e4e1b3809bf3-node-exporter-tls\") pod \"node-exporter-5ktdz\" (UID: \"e45d2d95-4d84-4c3a-8809-e4e1b3809bf3\") " pod="openshift-monitoring/node-exporter-5ktdz" Jul 06 02:44:10.648476 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:44:10.648447 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-lx5bb\" (UniqueName: \"kubernetes.io/projected/e45d2d95-4d84-4c3a-8809-e4e1b3809bf3-kube-api-access-lx5bb\") pod \"node-exporter-5ktdz\" (UID: \"e45d2d95-4d84-4c3a-8809-e4e1b3809bf3\") " pod="openshift-monitoring/node-exporter-5ktdz" Jul 06 02:44:10.648680 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:44:10.648661 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-vg95r\" (UniqueName: \"kubernetes.io/projected/23df5184-9bbf-49ba-82fa-e9f025937b4e-kube-api-access-vg95r\") pod \"kube-state-metrics-7c9dbb8b7d-nt64l\" (UID: \"23df5184-9bbf-49ba-82fa-e9f025937b4e\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-nt64l" Jul 06 02:44:10.784806 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:44:10.784664 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-5ktdz" Jul 06 02:44:10.797933 ip-10-0-139-206 kubenswrapper[2557]: W0706 02:44:10.797906 2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode45d2d95_4d84_4c3a_8809_e4e1b3809bf3.slice/crio-e2b8c1f66e6c32147af05233e08ab002080ffbc26932139a368526b09e20e9d3 WatchSource:0}: Error finding container e2b8c1f66e6c32147af05233e08ab002080ffbc26932139a368526b09e20e9d3: Status 404 returned error can't find the container with id e2b8c1f66e6c32147af05233e08ab002080ffbc26932139a368526b09e20e9d3 Jul 06 02:44:11.144816 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:44:11.144723 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/23df5184-9bbf-49ba-82fa-e9f025937b4e-kube-state-metrics-tls\") pod \"kube-state-metrics-7c9dbb8b7d-nt64l\" (UID: \"23df5184-9bbf-49ba-82fa-e9f025937b4e\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-nt64l" Jul 06 02:44:11.147243 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:44:11.147216 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/23df5184-9bbf-49ba-82fa-e9f025937b4e-kube-state-metrics-tls\") pod \"kube-state-metrics-7c9dbb8b7d-nt64l\" (UID: \"23df5184-9bbf-49ba-82fa-e9f025937b4e\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-nt64l" Jul 06 02:44:11.266402 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:44:11.266363 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-5ktdz" event={"ID":"e45d2d95-4d84-4c3a-8809-e4e1b3809bf3","Type":"ContainerStarted","Data":"e2b8c1f66e6c32147af05233e08ab002080ffbc26932139a368526b09e20e9d3"} Jul 06 02:44:11.379525 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:44:11.379482 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-nt64l" Jul 06 02:44:11.539877 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:44:11.539842 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/kube-state-metrics-7c9dbb8b7d-nt64l"] Jul 06 02:44:11.544473 ip-10-0-139-206 kubenswrapper[2557]: W0706 02:44:11.544436 2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod23df5184_9bbf_49ba_82fa_e9f025937b4e.slice/crio-598f7c7d61b648f51c3c30a6951607182db09ff663f6b9b5dff18cf2596fe001 WatchSource:0}: Error finding container 598f7c7d61b648f51c3c30a6951607182db09ff663f6b9b5dff18cf2596fe001: Status 404 returned error can't find the container with id 598f7c7d61b648f51c3c30a6951607182db09ff663f6b9b5dff18cf2596fe001 Jul 06 02:44:12.273918 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:44:12.273875 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-nt64l" event={"ID":"23df5184-9bbf-49ba-82fa-e9f025937b4e","Type":"ContainerStarted","Data":"598f7c7d61b648f51c3c30a6951607182db09ff663f6b9b5dff18cf2596fe001"} Jul 06 02:44:12.275667 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:44:12.275635 2557 generic.go:356] "Generic (PLEG): container finished" podID="e45d2d95-4d84-4c3a-8809-e4e1b3809bf3" containerID="7bdf6142da94e0fc51ddb7f3968a0214e7603210efe8f2fb93b2bb1908cdf5c7" exitCode=0 Jul 06 02:44:12.275823 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:44:12.275715 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-5ktdz" event={"ID":"e45d2d95-4d84-4c3a-8809-e4e1b3809bf3","Type":"ContainerDied","Data":"7bdf6142da94e0fc51ddb7f3968a0214e7603210efe8f2fb93b2bb1908cdf5c7"} Jul 06 02:44:12.456866 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:44:12.456831 2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/thanos-querier-78bfcd7f6-p2s9h"] Jul 06 02:44:12.460644 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:44:12.460616 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/thanos-querier-78bfcd7f6-p2s9h" Jul 06 02:44:12.464494 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:44:12.464466 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-grpc-tls-400csfcuev6jl\"" Jul 06 02:44:12.465189 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:44:12.465168 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-tls\"" Jul 06 02:44:12.465490 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:44:12.465470 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-web\"" Jul 06 02:44:12.465899 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:44:12.465883 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy\"" Jul 06 02:44:12.465976 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:44:12.465959 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-rules\"" Jul 06 02:44:12.466046 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:44:12.465993 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-dockercfg-2nqtl\"" Jul 06 02:44:12.466431 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:44:12.466409 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-metrics\"" Jul 06 02:44:12.482418 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:44:12.482382 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/thanos-querier-78bfcd7f6-p2s9h"] Jul 06 02:44:12.557492 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:44:12.557404 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/985e1aa6-6348-4f70-85bc-0b47fe3c68c0-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-78bfcd7f6-p2s9h\" (UID: \"985e1aa6-6348-4f70-85bc-0b47fe3c68c0\") " pod="openshift-monitoring/thanos-querier-78bfcd7f6-p2s9h" Jul 06 02:44:12.557492 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:44:12.557456 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qgxqz\" (UniqueName: \"kubernetes.io/projected/985e1aa6-6348-4f70-85bc-0b47fe3c68c0-kube-api-access-qgxqz\") pod \"thanos-querier-78bfcd7f6-p2s9h\" (UID: \"985e1aa6-6348-4f70-85bc-0b47fe3c68c0\") " pod="openshift-monitoring/thanos-querier-78bfcd7f6-p2s9h" Jul 06 02:44:12.557492 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:44:12.557492 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/985e1aa6-6348-4f70-85bc-0b47fe3c68c0-secret-thanos-querier-tls\") pod \"thanos-querier-78bfcd7f6-p2s9h\" (UID: \"985e1aa6-6348-4f70-85bc-0b47fe3c68c0\") " pod="openshift-monitoring/thanos-querier-78bfcd7f6-p2s9h" Jul 06 02:44:12.558093 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:44:12.557516 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/985e1aa6-6348-4f70-85bc-0b47fe3c68c0-secret-grpc-tls\") pod \"thanos-querier-78bfcd7f6-p2s9h\" (UID: \"985e1aa6-6348-4f70-85bc-0b47fe3c68c0\") " pod="openshift-monitoring/thanos-querier-78bfcd7f6-p2s9h" Jul 06 02:44:12.558093 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:44:12.557587 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/985e1aa6-6348-4f70-85bc-0b47fe3c68c0-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-78bfcd7f6-p2s9h\" (UID: \"985e1aa6-6348-4f70-85bc-0b47fe3c68c0\") " pod="openshift-monitoring/thanos-querier-78bfcd7f6-p2s9h" Jul 06 02:44:12.558093 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:44:12.557642 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/985e1aa6-6348-4f70-85bc-0b47fe3c68c0-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-78bfcd7f6-p2s9h\" (UID: \"985e1aa6-6348-4f70-85bc-0b47fe3c68c0\") " pod="openshift-monitoring/thanos-querier-78bfcd7f6-p2s9h" Jul 06 02:44:12.558093 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:44:12.557664 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/985e1aa6-6348-4f70-85bc-0b47fe3c68c0-metrics-client-ca\") pod \"thanos-querier-78bfcd7f6-p2s9h\" (UID: \"985e1aa6-6348-4f70-85bc-0b47fe3c68c0\") " pod="openshift-monitoring/thanos-querier-78bfcd7f6-p2s9h" Jul 06 02:44:12.558093 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:44:12.557761 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/985e1aa6-6348-4f70-85bc-0b47fe3c68c0-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-78bfcd7f6-p2s9h\" (UID: \"985e1aa6-6348-4f70-85bc-0b47fe3c68c0\") " pod="openshift-monitoring/thanos-querier-78bfcd7f6-p2s9h" Jul 06 02:44:12.658399 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:44:12.658343 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/985e1aa6-6348-4f70-85bc-0b47fe3c68c0-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-78bfcd7f6-p2s9h\" (UID: \"985e1aa6-6348-4f70-85bc-0b47fe3c68c0\") " pod="openshift-monitoring/thanos-querier-78bfcd7f6-p2s9h" Jul 06 02:44:12.658606 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:44:12.658423 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qgxqz\" (UniqueName: \"kubernetes.io/projected/985e1aa6-6348-4f70-85bc-0b47fe3c68c0-kube-api-access-qgxqz\") pod \"thanos-querier-78bfcd7f6-p2s9h\" (UID: \"985e1aa6-6348-4f70-85bc-0b47fe3c68c0\") " pod="openshift-monitoring/thanos-querier-78bfcd7f6-p2s9h" Jul 06 02:44:12.658606 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:44:12.658456 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/985e1aa6-6348-4f70-85bc-0b47fe3c68c0-secret-thanos-querier-tls\") pod \"thanos-querier-78bfcd7f6-p2s9h\" (UID: \"985e1aa6-6348-4f70-85bc-0b47fe3c68c0\") " pod="openshift-monitoring/thanos-querier-78bfcd7f6-p2s9h" Jul 06 02:44:12.658606 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:44:12.658480 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/985e1aa6-6348-4f70-85bc-0b47fe3c68c0-secret-grpc-tls\") pod \"thanos-querier-78bfcd7f6-p2s9h\" (UID: \"985e1aa6-6348-4f70-85bc-0b47fe3c68c0\") " pod="openshift-monitoring/thanos-querier-78bfcd7f6-p2s9h" Jul 06 02:44:12.658606 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:44:12.658528 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/985e1aa6-6348-4f70-85bc-0b47fe3c68c0-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-78bfcd7f6-p2s9h\" (UID: \"985e1aa6-6348-4f70-85bc-0b47fe3c68c0\") " pod="openshift-monitoring/thanos-querier-78bfcd7f6-p2s9h" Jul 06 02:44:12.658606 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:44:12.658564 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/985e1aa6-6348-4f70-85bc-0b47fe3c68c0-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-78bfcd7f6-p2s9h\" (UID: \"985e1aa6-6348-4f70-85bc-0b47fe3c68c0\") " pod="openshift-monitoring/thanos-querier-78bfcd7f6-p2s9h" Jul 06 02:44:12.658606 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:44:12.658586 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/985e1aa6-6348-4f70-85bc-0b47fe3c68c0-metrics-client-ca\") pod \"thanos-querier-78bfcd7f6-p2s9h\" (UID: \"985e1aa6-6348-4f70-85bc-0b47fe3c68c0\") " pod="openshift-monitoring/thanos-querier-78bfcd7f6-p2s9h" Jul 06 02:44:12.658925 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:44:12.658682 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/985e1aa6-6348-4f70-85bc-0b47fe3c68c0-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-78bfcd7f6-p2s9h\" (UID: \"985e1aa6-6348-4f70-85bc-0b47fe3c68c0\") " pod="openshift-monitoring/thanos-querier-78bfcd7f6-p2s9h" Jul 06 02:44:12.659597 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:44:12.659563 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/985e1aa6-6348-4f70-85bc-0b47fe3c68c0-metrics-client-ca\") pod \"thanos-querier-78bfcd7f6-p2s9h\" (UID: \"985e1aa6-6348-4f70-85bc-0b47fe3c68c0\") " pod="openshift-monitoring/thanos-querier-78bfcd7f6-p2s9h" Jul 06 02:44:12.661468 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:44:12.661442 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/985e1aa6-6348-4f70-85bc-0b47fe3c68c0-secret-thanos-querier-tls\") pod \"thanos-querier-78bfcd7f6-p2s9h\" (UID: \"985e1aa6-6348-4f70-85bc-0b47fe3c68c0\") " pod="openshift-monitoring/thanos-querier-78bfcd7f6-p2s9h" Jul 06 02:44:12.661587 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:44:12.661482 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/985e1aa6-6348-4f70-85bc-0b47fe3c68c0-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-78bfcd7f6-p2s9h\" (UID: \"985e1aa6-6348-4f70-85bc-0b47fe3c68c0\") " pod="openshift-monitoring/thanos-querier-78bfcd7f6-p2s9h" Jul 06 02:44:12.661771 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:44:12.661747 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/985e1aa6-6348-4f70-85bc-0b47fe3c68c0-secret-grpc-tls\") pod \"thanos-querier-78bfcd7f6-p2s9h\" (UID: \"985e1aa6-6348-4f70-85bc-0b47fe3c68c0\") " pod="openshift-monitoring/thanos-querier-78bfcd7f6-p2s9h" Jul 06 02:44:12.662160 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:44:12.662130 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/985e1aa6-6348-4f70-85bc-0b47fe3c68c0-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-78bfcd7f6-p2s9h\" (UID: \"985e1aa6-6348-4f70-85bc-0b47fe3c68c0\") " pod="openshift-monitoring/thanos-querier-78bfcd7f6-p2s9h" Jul 06 02:44:12.662427 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:44:12.662404 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/985e1aa6-6348-4f70-85bc-0b47fe3c68c0-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-78bfcd7f6-p2s9h\" (UID: \"985e1aa6-6348-4f70-85bc-0b47fe3c68c0\") " pod="openshift-monitoring/thanos-querier-78bfcd7f6-p2s9h" Jul 06 02:44:12.662673 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:44:12.662655 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/985e1aa6-6348-4f70-85bc-0b47fe3c68c0-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-78bfcd7f6-p2s9h\" (UID: \"985e1aa6-6348-4f70-85bc-0b47fe3c68c0\") " pod="openshift-monitoring/thanos-querier-78bfcd7f6-p2s9h" Jul 06 02:44:12.667470 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:44:12.667443 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-qgxqz\" (UniqueName: \"kubernetes.io/projected/985e1aa6-6348-4f70-85bc-0b47fe3c68c0-kube-api-access-qgxqz\") pod \"thanos-querier-78bfcd7f6-p2s9h\" (UID: \"985e1aa6-6348-4f70-85bc-0b47fe3c68c0\") " pod="openshift-monitoring/thanos-querier-78bfcd7f6-p2s9h" Jul 06 02:44:12.771199 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:44:12.771159 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/thanos-querier-78bfcd7f6-p2s9h" Jul 06 02:44:12.938093 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:44:12.937958 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/thanos-querier-78bfcd7f6-p2s9h"] Jul 06 02:44:12.944050 ip-10-0-139-206 kubenswrapper[2557]: W0706 02:44:12.944005 2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod985e1aa6_6348_4f70_85bc_0b47fe3c68c0.slice/crio-310d201f8b8a67d9aff4df745e8e603c79d8af7803ffd86e2de45758e9b697e9 WatchSource:0}: Error finding container 310d201f8b8a67d9aff4df745e8e603c79d8af7803ffd86e2de45758e9b697e9: Status 404 returned error can't find the container with id 310d201f8b8a67d9aff4df745e8e603c79d8af7803ffd86e2de45758e9b697e9 Jul 06 02:44:13.283835 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:44:13.283793 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-78bfcd7f6-p2s9h" event={"ID":"985e1aa6-6348-4f70-85bc-0b47fe3c68c0","Type":"ContainerStarted","Data":"310d201f8b8a67d9aff4df745e8e603c79d8af7803ffd86e2de45758e9b697e9"} Jul 06 02:44:13.285698 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:44:13.285669 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-nt64l" event={"ID":"23df5184-9bbf-49ba-82fa-e9f025937b4e","Type":"ContainerStarted","Data":"06ac9c9978af648d86eefab7d656ac725a9c58f793411253157289842710c6eb"} Jul 06 02:44:13.285835 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:44:13.285734 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-nt64l" event={"ID":"23df5184-9bbf-49ba-82fa-e9f025937b4e","Type":"ContainerStarted","Data":"590ddef09b86bd67f9a8d68134cbaf18203381624626679b4046fcb828fbd1b6"} Jul 06 02:44:13.285835 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:44:13.285764 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-nt64l" event={"ID":"23df5184-9bbf-49ba-82fa-e9f025937b4e","Type":"ContainerStarted","Data":"5b6aa253e664d2a4f12e5af76ba52a580d41d37ba778d3be7998fee8e109f2a1"} Jul 06 02:44:13.287652 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:44:13.287628 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-5ktdz" event={"ID":"e45d2d95-4d84-4c3a-8809-e4e1b3809bf3","Type":"ContainerStarted","Data":"91e55650e086b017100999c7fcdd81c6738524053ce559ce7145b8469cc372c7"} Jul 06 02:44:13.287755 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:44:13.287673 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-5ktdz" event={"ID":"e45d2d95-4d84-4c3a-8809-e4e1b3809bf3","Type":"ContainerStarted","Data":"53587bef813be2f0e6c353cb5bce53d3ed1253a40aa25e23d90d1666409ff2cf"} Jul 06 02:44:13.296778 ip-10-0-139-206 kubenswrapper[2557]: E0706 02:44:13.296746 2557 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[metrics-certs], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-multus/network-metrics-daemon-756n2" podUID="26f69a47-e949-4e18-9d9c-6648a959e281" Jul 06 02:44:13.302051 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:44:13.302003 2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-nt64l" podStartSLOduration=2.069190497 podStartE2EDuration="3.301990187s" totalImagesPullingTime="1.23279969s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 02:44:10 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 02:44:13.301125744 +0000 UTC m=+297.107411805" watchObservedRunningTime="2026-07-06 02:44:13.301990187 +0000 UTC m=+297.108276246" Jul 06 02:44:13.317814 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:44:13.317762 2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-5ktdz" podStartSLOduration=2.207523777 podStartE2EDuration="3.317744713s" totalImagesPullingTime="1.045362552s" totalInitContainerRuntime="64.858384ms" isStatefulPod=true podCreationTimestamp="2026-07-06 02:44:10 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 02:44:13.317037852 +0000 UTC m=+297.123323910" watchObservedRunningTime="2026-07-06 02:44:13.317744713 +0000 UTC m=+297.124030793" Jul 06 02:44:15.251034 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:44:15.251003 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-8jcmv" Jul 06 02:44:15.302277 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:44:15.302110 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-78bfcd7f6-p2s9h" event={"ID":"985e1aa6-6348-4f70-85bc-0b47fe3c68c0","Type":"ContainerStarted","Data":"ec065a4a940ab17b4cf97d9289ff64faa68a1ff34bab1e05316692559b0febf4"} Jul 06 02:44:15.302277 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:44:15.302201 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-78bfcd7f6-p2s9h" event={"ID":"985e1aa6-6348-4f70-85bc-0b47fe3c68c0","Type":"ContainerStarted","Data":"c39d409934998a87a43fcfc2d522d04e44994b4e5c2a8f239884ed29f56337db"} Jul 06 02:44:15.302277 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:44:15.302239 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-78bfcd7f6-p2s9h" event={"ID":"985e1aa6-6348-4f70-85bc-0b47fe3c68c0","Type":"ContainerStarted","Data":"2f8be983ca4870d3f84029aff9a3e37792fd767b0c2120a59481d8d5aba740fe"} Jul 06 02:44:16.303657 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:44:16.303626 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-78bfcd7f6-p2s9h" event={"ID":"985e1aa6-6348-4f70-85bc-0b47fe3c68c0","Type":"ContainerStarted","Data":"be640306a4975355dd0179163559b396b2449284bce1984127b79a8ddc2ce10a"} Jul 06 02:44:17.162867 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:44:17.162833 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-cw2f7_efcf5e11-1e5d-4e2a-9b38-303a8f0787c4/ovn-acl-logging/0.log" Jul 06 02:44:17.164229 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:44:17.164203 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-cw2f7_efcf5e11-1e5d-4e2a-9b38-303a8f0787c4/ovn-acl-logging/0.log" Jul 06 02:44:17.169617 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:44:17.169596 2557 kubelet.go:1648] "Image garbage collection succeeded" Jul 06 02:44:17.309835 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:44:17.309664 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-78bfcd7f6-p2s9h" event={"ID":"985e1aa6-6348-4f70-85bc-0b47fe3c68c0","Type":"ContainerStarted","Data":"dbcf5950a0988d684f8c7a09a5c56163e5a01b2825f4260f4f6b8052219c8677"} Jul 06 02:44:17.309835 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:44:17.309735 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-78bfcd7f6-p2s9h" event={"ID":"985e1aa6-6348-4f70-85bc-0b47fe3c68c0","Type":"ContainerStarted","Data":"57120490113e42a1ea0e92da7b3fc49d0c677bb8f3cf7bc6358c95b6bf24371b"} Jul 06 02:44:17.309835 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:44:17.309779 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/thanos-querier-78bfcd7f6-p2s9h" Jul 06 02:44:17.331078 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:44:17.331023 2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/thanos-querier-78bfcd7f6-p2s9h" podStartSLOduration=2.287764782 podStartE2EDuration="5.330999957s" totalImagesPullingTime="3.043235175s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 02:44:12 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 02:44:17.329182353 +0000 UTC m=+301.135468410" watchObservedRunningTime="2026-07-06 02:44:17.330999957 +0000 UTC m=+301.137286016" Jul 06 02:44:23.319604 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:44:23.319573 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/thanos-querier-78bfcd7f6-p2s9h" Jul 06 02:44:27.298163 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:44:27.298127 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-756n2" Jul 06 02:44:31.312606 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:44:31.312572 2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-6c88fdb975-qx6tg"] Jul 06 02:44:56.333207 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:44:56.333164 2557 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-console/console-6c88fdb975-qx6tg" podUID="7652727c-1327-4bd3-a642-56242557c416" containerName="console" containerID="cri-o://eef1c1fa74bdc8e2834f2bc69218995687ddd8903dec9a205085d89f2319dcbd" gracePeriod=15 Jul 06 02:44:56.446792 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:44:56.446762 2557 generic.go:356] "Generic (PLEG): container finished" podID="7652727c-1327-4bd3-a642-56242557c416" containerID="eef1c1fa74bdc8e2834f2bc69218995687ddd8903dec9a205085d89f2319dcbd" exitCode=0 Jul 06 02:44:56.446953 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:44:56.446832 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6c88fdb975-qx6tg" event={"ID":"7652727c-1327-4bd3-a642-56242557c416","Type":"ContainerDied","Data":"eef1c1fa74bdc8e2834f2bc69218995687ddd8903dec9a205085d89f2319dcbd"} Jul 06 02:44:56.571126 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:44:56.571100 2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6c88fdb975-qx6tg" Jul 06 02:44:56.630080 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:44:56.629982 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/7652727c-1327-4bd3-a642-56242557c416-console-config\") pod \"7652727c-1327-4bd3-a642-56242557c416\" (UID: \"7652727c-1327-4bd3-a642-56242557c416\") " Jul 06 02:44:56.630080 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:44:56.630031 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/7652727c-1327-4bd3-a642-56242557c416-console-oauth-config\") pod \"7652727c-1327-4bd3-a642-56242557c416\" (UID: \"7652727c-1327-4bd3-a642-56242557c416\") " Jul 06 02:44:56.630080 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:44:56.630071 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/7652727c-1327-4bd3-a642-56242557c416-trusted-ca-bundle\") pod \"7652727c-1327-4bd3-a642-56242557c416\" (UID: \"7652727c-1327-4bd3-a642-56242557c416\") " Jul 06 02:44:56.630378 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:44:56.630183 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/7652727c-1327-4bd3-a642-56242557c416-service-ca\") pod \"7652727c-1327-4bd3-a642-56242557c416\" (UID: \"7652727c-1327-4bd3-a642-56242557c416\") " Jul 06 02:44:56.630378 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:44:56.630237 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-znktq\" (UniqueName: \"kubernetes.io/projected/7652727c-1327-4bd3-a642-56242557c416-kube-api-access-znktq\") pod \"7652727c-1327-4bd3-a642-56242557c416\" (UID: \"7652727c-1327-4bd3-a642-56242557c416\") " Jul 06 02:44:56.630378 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:44:56.630290 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/7652727c-1327-4bd3-a642-56242557c416-console-serving-cert\") pod \"7652727c-1327-4bd3-a642-56242557c416\" (UID: \"7652727c-1327-4bd3-a642-56242557c416\") " Jul 06 02:44:56.630378 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:44:56.630315 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/7652727c-1327-4bd3-a642-56242557c416-oauth-serving-cert\") pod \"7652727c-1327-4bd3-a642-56242557c416\" (UID: \"7652727c-1327-4bd3-a642-56242557c416\") " Jul 06 02:44:56.630579 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:44:56.630484 2557 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7652727c-1327-4bd3-a642-56242557c416-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "7652727c-1327-4bd3-a642-56242557c416" (UID: "7652727c-1327-4bd3-a642-56242557c416"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 06 02:44:56.630579 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:44:56.630492 2557 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7652727c-1327-4bd3-a642-56242557c416-console-config" (OuterVolumeSpecName: "console-config") pod "7652727c-1327-4bd3-a642-56242557c416" (UID: "7652727c-1327-4bd3-a642-56242557c416"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 06 02:44:56.630688 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:44:56.630584 2557 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7652727c-1327-4bd3-a642-56242557c416-service-ca" (OuterVolumeSpecName: "service-ca") pod "7652727c-1327-4bd3-a642-56242557c416" (UID: "7652727c-1327-4bd3-a642-56242557c416"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 06 02:44:56.630688 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:44:56.630605 2557 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/7652727c-1327-4bd3-a642-56242557c416-trusted-ca-bundle\") on node \"ip-10-0-139-206.ec2.internal\" DevicePath \"\"" Jul 06 02:44:56.630688 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:44:56.630620 2557 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/7652727c-1327-4bd3-a642-56242557c416-console-config\") on node \"ip-10-0-139-206.ec2.internal\" DevicePath \"\"" Jul 06 02:44:56.630828 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:44:56.630789 2557 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7652727c-1327-4bd3-a642-56242557c416-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "7652727c-1327-4bd3-a642-56242557c416" (UID: "7652727c-1327-4bd3-a642-56242557c416"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 06 02:44:56.632510 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:44:56.632481 2557 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7652727c-1327-4bd3-a642-56242557c416-kube-api-access-znktq" (OuterVolumeSpecName: "kube-api-access-znktq") pod "7652727c-1327-4bd3-a642-56242557c416" (UID: "7652727c-1327-4bd3-a642-56242557c416"). InnerVolumeSpecName "kube-api-access-znktq". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 06 02:44:56.632625 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:44:56.632508 2557 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/7652727c-1327-4bd3-a642-56242557c416-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "7652727c-1327-4bd3-a642-56242557c416" (UID: "7652727c-1327-4bd3-a642-56242557c416"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 06 02:44:56.632625 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:44:56.632487 2557 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/7652727c-1327-4bd3-a642-56242557c416-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "7652727c-1327-4bd3-a642-56242557c416" (UID: "7652727c-1327-4bd3-a642-56242557c416"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 06 02:44:56.731978 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:44:56.731939 2557 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-znktq\" (UniqueName: \"kubernetes.io/projected/7652727c-1327-4bd3-a642-56242557c416-kube-api-access-znktq\") on node \"ip-10-0-139-206.ec2.internal\" DevicePath \"\"" Jul 06 02:44:56.731978 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:44:56.731968 2557 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/7652727c-1327-4bd3-a642-56242557c416-console-serving-cert\") on node \"ip-10-0-139-206.ec2.internal\" DevicePath \"\"" Jul 06 02:44:56.731978 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:44:56.731977 2557 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/7652727c-1327-4bd3-a642-56242557c416-oauth-serving-cert\") on node \"ip-10-0-139-206.ec2.internal\" DevicePath \"\"" Jul 06 02:44:56.731978 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:44:56.731985 2557 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/7652727c-1327-4bd3-a642-56242557c416-console-oauth-config\") on node \"ip-10-0-139-206.ec2.internal\" DevicePath \"\"" Jul 06 02:44:56.731978 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:44:56.731995 2557 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/7652727c-1327-4bd3-a642-56242557c416-service-ca\") on node \"ip-10-0-139-206.ec2.internal\" DevicePath \"\"" Jul 06 02:44:57.451688 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:44:57.451661 2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6c88fdb975-qx6tg" Jul 06 02:44:57.451688 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:44:57.451672 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6c88fdb975-qx6tg" event={"ID":"7652727c-1327-4bd3-a642-56242557c416","Type":"ContainerDied","Data":"1ddb757f7691e8a9c89d0f6b1142798d4cb3ad16837aa6cf4a2d050112b69633"} Jul 06 02:44:57.452216 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:44:57.451740 2557 scope.go:117] "RemoveContainer" containerID="eef1c1fa74bdc8e2834f2bc69218995687ddd8903dec9a205085d89f2319dcbd" Jul 06 02:44:57.469040 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:44:57.468990 2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-6c88fdb975-qx6tg"] Jul 06 02:44:57.474827 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:44:57.474800 2557 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-6c88fdb975-qx6tg"] Jul 06 02:44:59.301075 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:44:59.301042 2557 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="7652727c-1327-4bd3-a642-56242557c416" path="/var/lib/kubelet/pods/7652727c-1327-4bd3-a642-56242557c416/volumes" Jul 06 02:45:31.196146 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:45:31.196096 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/26f69a47-e949-4e18-9d9c-6648a959e281-metrics-certs\") pod \"network-metrics-daemon-756n2\" (UID: \"26f69a47-e949-4e18-9d9c-6648a959e281\") " pod="openshift-multus/network-metrics-daemon-756n2" Jul 06 02:45:31.198564 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:45:31.198536 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/26f69a47-e949-4e18-9d9c-6648a959e281-metrics-certs\") pod \"network-metrics-daemon-756n2\" (UID: \"26f69a47-e949-4e18-9d9c-6648a959e281\") " pod="openshift-multus/network-metrics-daemon-756n2" Jul 06 02:45:31.201268 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:45:31.201243 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-n2bvg\"" Jul 06 02:45:31.208756 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:45:31.208726 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-756n2" Jul 06 02:45:31.336614 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:45:31.336578 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-756n2"] Jul 06 02:45:31.340709 ip-10-0-139-206 kubenswrapper[2557]: W0706 02:45:31.340678 2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod26f69a47_e949_4e18_9d9c_6648a959e281.slice/crio-9a364a51be3077ea6474062227d0fa86cde2484e64b9bb8e17f06a4152ed8f0c WatchSource:0}: Error finding container 9a364a51be3077ea6474062227d0fa86cde2484e64b9bb8e17f06a4152ed8f0c: Status 404 returned error can't find the container with id 9a364a51be3077ea6474062227d0fa86cde2484e64b9bb8e17f06a4152ed8f0c Jul 06 02:45:31.342990 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:45:31.342972 2557 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 06 02:45:31.570044 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:45:31.569955 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-756n2" event={"ID":"26f69a47-e949-4e18-9d9c-6648a959e281","Type":"ContainerStarted","Data":"9a364a51be3077ea6474062227d0fa86cde2484e64b9bb8e17f06a4152ed8f0c"} Jul 06 02:45:32.574866 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:45:32.574763 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-756n2" event={"ID":"26f69a47-e949-4e18-9d9c-6648a959e281","Type":"ContainerStarted","Data":"536c92f33ab7c81280c27109318a4ab7fb662698afa525695a554a37fed5da57"} Jul 06 02:45:32.574866 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:45:32.574835 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-756n2" event={"ID":"26f69a47-e949-4e18-9d9c-6648a959e281","Type":"ContainerStarted","Data":"ccd100e7b40f2bd701f0221217dad93eb4b22094e943e7280bdc4ee60597a87c"} Jul 06 02:45:32.592052 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:45:32.592003 2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-756n2" podStartSLOduration=374.752584387 podStartE2EDuration="6m15.591986187s" totalImagesPullingTime="839.4018ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 02:39:17 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 02:45:32.590814041 +0000 UTC m=+376.397100100" watchObservedRunningTime="2026-07-06 02:45:32.591986187 +0000 UTC m=+376.398272244" Jul 06 02:45:41.985660 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:45:41.985616 2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-7f77647fb6-hr58q"] Jul 06 02:45:41.986168 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:45:41.986000 2557 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7652727c-1327-4bd3-a642-56242557c416" containerName="console" Jul 06 02:45:41.986168 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:45:41.986018 2557 state_mem.go:107] "Deleted CPUSet assignment" podUID="7652727c-1327-4bd3-a642-56242557c416" containerName="console" Jul 06 02:45:41.986168 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:45:41.986085 2557 memory_manager.go:368] "RemoveStaleState removing state" podUID="7652727c-1327-4bd3-a642-56242557c416" containerName="console" Jul 06 02:45:41.987903 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:45:41.987881 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7f77647fb6-hr58q" Jul 06 02:45:41.990189 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:45:41.990166 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"openshift-service-ca.crt\"" Jul 06 02:45:41.990356 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:45:41.990259 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-dockercfg-ffz2r\"" Jul 06 02:45:41.991222 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:45:41.991196 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"console-config\"" Jul 06 02:45:41.991222 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:45:41.991201 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"service-ca\"" Jul 06 02:45:41.991421 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:45:41.991402 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"oauth-serving-cert\"" Jul 06 02:45:41.991529 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:45:41.991509 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"kube-root-ca.crt\"" Jul 06 02:45:41.991529 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:45:41.991524 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-serving-cert\"" Jul 06 02:45:41.992339 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:45:41.992322 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-oauth-config\"" Jul 06 02:45:41.995795 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:45:41.995776 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"trusted-ca-bundle\"" Jul 06 02:45:41.999710 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:45:41.999690 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-7f77647fb6-hr58q"] Jul 06 02:45:42.075798 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:45:42.075754 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/d3c097d6-0b2a-4d50-9ba7-bf7d5336bd65-console-serving-cert\") pod \"console-7f77647fb6-hr58q\" (UID: \"d3c097d6-0b2a-4d50-9ba7-bf7d5336bd65\") " pod="openshift-console/console-7f77647fb6-hr58q" Jul 06 02:45:42.075798 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:45:42.075796 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lz4jr\" (UniqueName: \"kubernetes.io/projected/d3c097d6-0b2a-4d50-9ba7-bf7d5336bd65-kube-api-access-lz4jr\") pod \"console-7f77647fb6-hr58q\" (UID: \"d3c097d6-0b2a-4d50-9ba7-bf7d5336bd65\") " pod="openshift-console/console-7f77647fb6-hr58q" Jul 06 02:45:42.076014 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:45:42.075820 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d3c097d6-0b2a-4d50-9ba7-bf7d5336bd65-trusted-ca-bundle\") pod \"console-7f77647fb6-hr58q\" (UID: \"d3c097d6-0b2a-4d50-9ba7-bf7d5336bd65\") " pod="openshift-console/console-7f77647fb6-hr58q" Jul 06 02:45:42.076014 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:45:42.075881 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/d3c097d6-0b2a-4d50-9ba7-bf7d5336bd65-oauth-serving-cert\") pod \"console-7f77647fb6-hr58q\" (UID: \"d3c097d6-0b2a-4d50-9ba7-bf7d5336bd65\") " pod="openshift-console/console-7f77647fb6-hr58q" Jul 06 02:45:42.076014 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:45:42.075915 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/d3c097d6-0b2a-4d50-9ba7-bf7d5336bd65-service-ca\") pod \"console-7f77647fb6-hr58q\" (UID: \"d3c097d6-0b2a-4d50-9ba7-bf7d5336bd65\") " pod="openshift-console/console-7f77647fb6-hr58q" Jul 06 02:45:42.076014 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:45:42.075940 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/d3c097d6-0b2a-4d50-9ba7-bf7d5336bd65-console-oauth-config\") pod \"console-7f77647fb6-hr58q\" (UID: \"d3c097d6-0b2a-4d50-9ba7-bf7d5336bd65\") " pod="openshift-console/console-7f77647fb6-hr58q" Jul 06 02:45:42.076014 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:45:42.075962 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/d3c097d6-0b2a-4d50-9ba7-bf7d5336bd65-console-config\") pod \"console-7f77647fb6-hr58q\" (UID: \"d3c097d6-0b2a-4d50-9ba7-bf7d5336bd65\") " pod="openshift-console/console-7f77647fb6-hr58q" Jul 06 02:45:42.176510 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:45:42.176465 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/d3c097d6-0b2a-4d50-9ba7-bf7d5336bd65-console-config\") pod \"console-7f77647fb6-hr58q\" (UID: \"d3c097d6-0b2a-4d50-9ba7-bf7d5336bd65\") " pod="openshift-console/console-7f77647fb6-hr58q" Jul 06 02:45:42.176710 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:45:42.176531 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/d3c097d6-0b2a-4d50-9ba7-bf7d5336bd65-console-serving-cert\") pod \"console-7f77647fb6-hr58q\" (UID: \"d3c097d6-0b2a-4d50-9ba7-bf7d5336bd65\") " pod="openshift-console/console-7f77647fb6-hr58q" Jul 06 02:45:42.176710 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:45:42.176556 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lz4jr\" (UniqueName: \"kubernetes.io/projected/d3c097d6-0b2a-4d50-9ba7-bf7d5336bd65-kube-api-access-lz4jr\") pod \"console-7f77647fb6-hr58q\" (UID: \"d3c097d6-0b2a-4d50-9ba7-bf7d5336bd65\") " pod="openshift-console/console-7f77647fb6-hr58q" Jul 06 02:45:42.176710 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:45:42.176585 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d3c097d6-0b2a-4d50-9ba7-bf7d5336bd65-trusted-ca-bundle\") pod \"console-7f77647fb6-hr58q\" (UID: \"d3c097d6-0b2a-4d50-9ba7-bf7d5336bd65\") " pod="openshift-console/console-7f77647fb6-hr58q" Jul 06 02:45:42.176710 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:45:42.176623 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/d3c097d6-0b2a-4d50-9ba7-bf7d5336bd65-oauth-serving-cert\") pod \"console-7f77647fb6-hr58q\" (UID: \"d3c097d6-0b2a-4d50-9ba7-bf7d5336bd65\") " pod="openshift-console/console-7f77647fb6-hr58q" Jul 06 02:45:42.176710 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:45:42.176647 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/d3c097d6-0b2a-4d50-9ba7-bf7d5336bd65-service-ca\") pod \"console-7f77647fb6-hr58q\" (UID: \"d3c097d6-0b2a-4d50-9ba7-bf7d5336bd65\") " pod="openshift-console/console-7f77647fb6-hr58q" Jul 06 02:45:42.176710 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:45:42.176674 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/d3c097d6-0b2a-4d50-9ba7-bf7d5336bd65-console-oauth-config\") pod \"console-7f77647fb6-hr58q\" (UID: \"d3c097d6-0b2a-4d50-9ba7-bf7d5336bd65\") " pod="openshift-console/console-7f77647fb6-hr58q" Jul 06 02:45:42.177371 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:45:42.177320 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/d3c097d6-0b2a-4d50-9ba7-bf7d5336bd65-console-config\") pod \"console-7f77647fb6-hr58q\" (UID: \"d3c097d6-0b2a-4d50-9ba7-bf7d5336bd65\") " pod="openshift-console/console-7f77647fb6-hr58q" Jul 06 02:45:42.177496 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:45:42.177393 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/d3c097d6-0b2a-4d50-9ba7-bf7d5336bd65-service-ca\") pod \"console-7f77647fb6-hr58q\" (UID: \"d3c097d6-0b2a-4d50-9ba7-bf7d5336bd65\") " pod="openshift-console/console-7f77647fb6-hr58q" Jul 06 02:45:42.177496 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:45:42.177436 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d3c097d6-0b2a-4d50-9ba7-bf7d5336bd65-trusted-ca-bundle\") pod \"console-7f77647fb6-hr58q\" (UID: \"d3c097d6-0b2a-4d50-9ba7-bf7d5336bd65\") " pod="openshift-console/console-7f77647fb6-hr58q" Jul 06 02:45:42.177496 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:45:42.177437 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/d3c097d6-0b2a-4d50-9ba7-bf7d5336bd65-oauth-serving-cert\") pod \"console-7f77647fb6-hr58q\" (UID: \"d3c097d6-0b2a-4d50-9ba7-bf7d5336bd65\") " pod="openshift-console/console-7f77647fb6-hr58q" Jul 06 02:45:42.179161 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:45:42.179132 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/d3c097d6-0b2a-4d50-9ba7-bf7d5336bd65-console-oauth-config\") pod \"console-7f77647fb6-hr58q\" (UID: \"d3c097d6-0b2a-4d50-9ba7-bf7d5336bd65\") " pod="openshift-console/console-7f77647fb6-hr58q" Jul 06 02:45:42.179242 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:45:42.179178 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/d3c097d6-0b2a-4d50-9ba7-bf7d5336bd65-console-serving-cert\") pod \"console-7f77647fb6-hr58q\" (UID: \"d3c097d6-0b2a-4d50-9ba7-bf7d5336bd65\") " pod="openshift-console/console-7f77647fb6-hr58q" Jul 06 02:45:42.184408 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:45:42.184388 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-lz4jr\" (UniqueName: \"kubernetes.io/projected/d3c097d6-0b2a-4d50-9ba7-bf7d5336bd65-kube-api-access-lz4jr\") pod \"console-7f77647fb6-hr58q\" (UID: \"d3c097d6-0b2a-4d50-9ba7-bf7d5336bd65\") " pod="openshift-console/console-7f77647fb6-hr58q" Jul 06 02:45:42.297590 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:45:42.297496 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7f77647fb6-hr58q" Jul 06 02:45:42.425633 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:45:42.425601 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-7f77647fb6-hr58q"] Jul 06 02:45:42.428670 ip-10-0-139-206 kubenswrapper[2557]: W0706 02:45:42.428641 2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd3c097d6_0b2a_4d50_9ba7_bf7d5336bd65.slice/crio-8dc21ee92322b3a66d3be120e02ba5eb0fe174e41ee5323c12340b33887fc216 WatchSource:0}: Error finding container 8dc21ee92322b3a66d3be120e02ba5eb0fe174e41ee5323c12340b33887fc216: Status 404 returned error can't find the container with id 8dc21ee92322b3a66d3be120e02ba5eb0fe174e41ee5323c12340b33887fc216 Jul 06 02:45:42.611599 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:45:42.611504 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7f77647fb6-hr58q" event={"ID":"d3c097d6-0b2a-4d50-9ba7-bf7d5336bd65","Type":"ContainerStarted","Data":"00cf1feb70aa5c0e10972ffb17fb981ec6a17086345c2e69f0b3258be0cec8e0"} Jul 06 02:45:42.611599 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:45:42.611557 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7f77647fb6-hr58q" event={"ID":"d3c097d6-0b2a-4d50-9ba7-bf7d5336bd65","Type":"ContainerStarted","Data":"8dc21ee92322b3a66d3be120e02ba5eb0fe174e41ee5323c12340b33887fc216"} Jul 06 02:45:42.628529 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:45:42.628473 2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-7f77647fb6-hr58q" podStartSLOduration=1.628453909 podStartE2EDuration="1.628453909s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 02:45:41 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 02:45:42.627409263 +0000 UTC m=+386.433695322" watchObservedRunningTime="2026-07-06 02:45:42.628453909 +0000 UTC m=+386.434739967" Jul 06 02:45:52.298124 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:45:52.298088 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-7f77647fb6-hr58q" Jul 06 02:45:52.298617 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:45:52.298153 2557 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-7f77647fb6-hr58q" Jul 06 02:45:52.303135 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:45:52.303107 2557 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-7f77647fb6-hr58q" Jul 06 02:45:52.651864 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:45:52.651784 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-7f77647fb6-hr58q" Jul 06 02:46:49.972326 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:46:49.972244 2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-55b4d954d5-9t997"] Jul 06 02:46:49.974361 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:46:49.974323 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-55b4d954d5-9t997" Jul 06 02:46:49.976738 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:46:49.976717 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"openshift-service-ca.crt\"" Jul 06 02:46:49.977495 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:46:49.977481 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"kube-root-ca.crt\"" Jul 06 02:46:49.977548 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:46:49.977499 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-dockercfg-wsfgc\"" Jul 06 02:46:49.984768 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:46:49.984742 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-55b4d954d5-9t997"] Jul 06 02:46:50.094192 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:46:50.094149 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/46aad1fb-9f19-4eae-9839-84111f4c9775-bound-sa-token\") pod \"cert-manager-55b4d954d5-9t997\" (UID: \"46aad1fb-9f19-4eae-9839-84111f4c9775\") " pod="cert-manager/cert-manager-55b4d954d5-9t997" Jul 06 02:46:50.094414 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:46:50.094220 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4z8kr\" (UniqueName: \"kubernetes.io/projected/46aad1fb-9f19-4eae-9839-84111f4c9775-kube-api-access-4z8kr\") pod \"cert-manager-55b4d954d5-9t997\" (UID: \"46aad1fb-9f19-4eae-9839-84111f4c9775\") " pod="cert-manager/cert-manager-55b4d954d5-9t997" Jul 06 02:46:50.195448 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:46:50.195402 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/46aad1fb-9f19-4eae-9839-84111f4c9775-bound-sa-token\") pod \"cert-manager-55b4d954d5-9t997\" (UID: \"46aad1fb-9f19-4eae-9839-84111f4c9775\") " pod="cert-manager/cert-manager-55b4d954d5-9t997" Jul 06 02:46:50.195606 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:46:50.195476 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4z8kr\" (UniqueName: \"kubernetes.io/projected/46aad1fb-9f19-4eae-9839-84111f4c9775-kube-api-access-4z8kr\") pod \"cert-manager-55b4d954d5-9t997\" (UID: \"46aad1fb-9f19-4eae-9839-84111f4c9775\") " pod="cert-manager/cert-manager-55b4d954d5-9t997" Jul 06 02:46:50.203674 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:46:50.203636 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/46aad1fb-9f19-4eae-9839-84111f4c9775-bound-sa-token\") pod \"cert-manager-55b4d954d5-9t997\" (UID: \"46aad1fb-9f19-4eae-9839-84111f4c9775\") " pod="cert-manager/cert-manager-55b4d954d5-9t997" Jul 06 02:46:50.203816 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:46:50.203760 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-4z8kr\" (UniqueName: \"kubernetes.io/projected/46aad1fb-9f19-4eae-9839-84111f4c9775-kube-api-access-4z8kr\") pod \"cert-manager-55b4d954d5-9t997\" (UID: \"46aad1fb-9f19-4eae-9839-84111f4c9775\") " pod="cert-manager/cert-manager-55b4d954d5-9t997" Jul 06 02:46:50.283843 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:46:50.283744 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-55b4d954d5-9t997" Jul 06 02:46:50.411696 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:46:50.411655 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-55b4d954d5-9t997"] Jul 06 02:46:50.415787 ip-10-0-139-206 kubenswrapper[2557]: W0706 02:46:50.415758 2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod46aad1fb_9f19_4eae_9839_84111f4c9775.slice/crio-cda4839ab5bfd4529cdcc71e2dd4a97023202510b7891336f494615eefc53738 WatchSource:0}: Error finding container cda4839ab5bfd4529cdcc71e2dd4a97023202510b7891336f494615eefc53738: Status 404 returned error can't find the container with id cda4839ab5bfd4529cdcc71e2dd4a97023202510b7891336f494615eefc53738 Jul 06 02:46:50.855889 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:46:50.855843 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-55b4d954d5-9t997" event={"ID":"46aad1fb-9f19-4eae-9839-84111f4c9775","Type":"ContainerStarted","Data":"cda4839ab5bfd4529cdcc71e2dd4a97023202510b7891336f494615eefc53738"} Jul 06 02:46:53.868862 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:46:53.868824 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-55b4d954d5-9t997" event={"ID":"46aad1fb-9f19-4eae-9839-84111f4c9775","Type":"ContainerStarted","Data":"78050e71838e74656a56e1cb6a0d3566b867557ba897e02e57f069487ac13e97"} Jul 06 02:46:53.884410 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:46:53.884343 2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager/cert-manager-55b4d954d5-9t997" podStartSLOduration=1.9208371990000002 podStartE2EDuration="4.884326019s" totalImagesPullingTime="2.96348882s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 02:46:49 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 02:46:53.883101275 +0000 UTC m=+457.689387332" watchObservedRunningTime="2026-07-06 02:46:53.884326019 +0000 UTC m=+457.690612077" Jul 06 02:47:03.709316 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:47:03.709278 2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-lws-operator/openshift-lws-operator-fd8ccff4c-2f9nz"] Jul 06 02:47:03.711487 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:47:03.711464 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-2f9nz" Jul 06 02:47:03.713671 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:47:03.713647 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"openshift-service-ca.crt\"" Jul 06 02:47:03.713777 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:47:03.713691 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"kube-root-ca.crt\"" Jul 06 02:47:03.713996 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:47:03.713983 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"openshift-lws-operator-dockercfg-2t6vr\"" Jul 06 02:47:03.721115 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:47:03.721091 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/openshift-lws-operator-fd8ccff4c-2f9nz"] Jul 06 02:47:03.807047 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:47:03.807010 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/8240eeaf-2841-4478-898a-0c76a803bb0b-tmp\") pod \"openshift-lws-operator-fd8ccff4c-2f9nz\" (UID: \"8240eeaf-2841-4478-898a-0c76a803bb0b\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-2f9nz" Jul 06 02:47:03.807047 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:47:03.807049 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mpspg\" (UniqueName: \"kubernetes.io/projected/8240eeaf-2841-4478-898a-0c76a803bb0b-kube-api-access-mpspg\") pod \"openshift-lws-operator-fd8ccff4c-2f9nz\" (UID: \"8240eeaf-2841-4478-898a-0c76a803bb0b\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-2f9nz" Jul 06 02:47:03.907977 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:47:03.907939 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/8240eeaf-2841-4478-898a-0c76a803bb0b-tmp\") pod \"openshift-lws-operator-fd8ccff4c-2f9nz\" (UID: \"8240eeaf-2841-4478-898a-0c76a803bb0b\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-2f9nz" Jul 06 02:47:03.908193 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:47:03.907982 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mpspg\" (UniqueName: \"kubernetes.io/projected/8240eeaf-2841-4478-898a-0c76a803bb0b-kube-api-access-mpspg\") pod \"openshift-lws-operator-fd8ccff4c-2f9nz\" (UID: \"8240eeaf-2841-4478-898a-0c76a803bb0b\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-2f9nz" Jul 06 02:47:03.908386 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:47:03.908342 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/8240eeaf-2841-4478-898a-0c76a803bb0b-tmp\") pod \"openshift-lws-operator-fd8ccff4c-2f9nz\" (UID: \"8240eeaf-2841-4478-898a-0c76a803bb0b\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-2f9nz" Jul 06 02:47:03.916015 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:47:03.915986 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-mpspg\" (UniqueName: \"kubernetes.io/projected/8240eeaf-2841-4478-898a-0c76a803bb0b-kube-api-access-mpspg\") pod \"openshift-lws-operator-fd8ccff4c-2f9nz\" (UID: \"8240eeaf-2841-4478-898a-0c76a803bb0b\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-2f9nz" Jul 06 02:47:04.020916 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:47:04.020812 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-2f9nz" Jul 06 02:47:04.151185 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:47:04.151152 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/openshift-lws-operator-fd8ccff4c-2f9nz"] Jul 06 02:47:04.154636 ip-10-0-139-206 kubenswrapper[2557]: W0706 02:47:04.154604 2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod8240eeaf_2841_4478_898a_0c76a803bb0b.slice/crio-347afe6551d8fdec5bfba32e69577aca1d00fadc11bf8f24e6d19078382ac919 WatchSource:0}: Error finding container 347afe6551d8fdec5bfba32e69577aca1d00fadc11bf8f24e6d19078382ac919: Status 404 returned error can't find the container with id 347afe6551d8fdec5bfba32e69577aca1d00fadc11bf8f24e6d19078382ac919 Jul 06 02:47:04.911762 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:47:04.911727 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-2f9nz" event={"ID":"8240eeaf-2841-4478-898a-0c76a803bb0b","Type":"ContainerStarted","Data":"347afe6551d8fdec5bfba32e69577aca1d00fadc11bf8f24e6d19078382ac919"} Jul 06 02:47:08.930409 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:47:08.930370 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-2f9nz" event={"ID":"8240eeaf-2841-4478-898a-0c76a803bb0b","Type":"ContainerStarted","Data":"33dd36de95b8fc8ef7071ac29980f66eaeb906de4f85344dff0343314a54d1c4"} Jul 06 02:47:08.946249 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:47:08.946198 2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-2f9nz" podStartSLOduration=1.6361966319999999 podStartE2EDuration="5.946179993s" totalImagesPullingTime="4.309983361s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 02:47:03 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 02:47:08.944845504 +0000 UTC m=+472.751131562" watchObservedRunningTime="2026-07-06 02:47:08.946179993 +0000 UTC m=+472.752466051" Jul 06 02:47:24.591366 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:47:24.591308 2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-6d49d64447-dpntf"] Jul 06 02:47:24.598320 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:47:24.598290 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-6d49d64447-dpntf" Jul 06 02:47:24.601156 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:47:24.601130 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-service-cert\"" Jul 06 02:47:24.601156 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:47:24.601148 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\"" Jul 06 02:47:24.601328 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:47:24.601161 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\"" Jul 06 02:47:24.601328 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:47:24.601173 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-webhook-cert\"" Jul 06 02:47:24.601328 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:47:24.601134 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-dockercfg-22csz\"" Jul 06 02:47:24.611925 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:47:24.611892 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-6d49d64447-dpntf"] Jul 06 02:47:24.667769 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:47:24.667735 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/5adfd973-3482-4802-b49e-8a5151e3cf88-manifests\") pod \"opendatahub-operator-controller-manager-6d49d64447-dpntf\" (UID: \"5adfd973-3482-4802-b49e-8a5151e3cf88\") " pod="opendatahub/opendatahub-operator-controller-manager-6d49d64447-dpntf" Jul 06 02:47:24.667769 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:47:24.667780 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/5adfd973-3482-4802-b49e-8a5151e3cf88-tmp\") pod \"opendatahub-operator-controller-manager-6d49d64447-dpntf\" (UID: \"5adfd973-3482-4802-b49e-8a5151e3cf88\") " pod="opendatahub/opendatahub-operator-controller-manager-6d49d64447-dpntf" Jul 06 02:47:24.667997 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:47:24.667799 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/5adfd973-3482-4802-b49e-8a5151e3cf88-apiservice-cert\") pod \"opendatahub-operator-controller-manager-6d49d64447-dpntf\" (UID: \"5adfd973-3482-4802-b49e-8a5151e3cf88\") " pod="opendatahub/opendatahub-operator-controller-manager-6d49d64447-dpntf" Jul 06 02:47:24.667997 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:47:24.667888 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4vthc\" (UniqueName: \"kubernetes.io/projected/5adfd973-3482-4802-b49e-8a5151e3cf88-kube-api-access-4vthc\") pod \"opendatahub-operator-controller-manager-6d49d64447-dpntf\" (UID: \"5adfd973-3482-4802-b49e-8a5151e3cf88\") " pod="opendatahub/opendatahub-operator-controller-manager-6d49d64447-dpntf" Jul 06 02:47:24.667997 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:47:24.667915 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/5adfd973-3482-4802-b49e-8a5151e3cf88-webhook-cert\") pod \"opendatahub-operator-controller-manager-6d49d64447-dpntf\" (UID: \"5adfd973-3482-4802-b49e-8a5151e3cf88\") " pod="opendatahub/opendatahub-operator-controller-manager-6d49d64447-dpntf" Jul 06 02:47:24.768289 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:47:24.768249 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/5adfd973-3482-4802-b49e-8a5151e3cf88-manifests\") pod \"opendatahub-operator-controller-manager-6d49d64447-dpntf\" (UID: \"5adfd973-3482-4802-b49e-8a5151e3cf88\") " pod="opendatahub/opendatahub-operator-controller-manager-6d49d64447-dpntf" Jul 06 02:47:24.768289 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:47:24.768293 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/5adfd973-3482-4802-b49e-8a5151e3cf88-tmp\") pod \"opendatahub-operator-controller-manager-6d49d64447-dpntf\" (UID: \"5adfd973-3482-4802-b49e-8a5151e3cf88\") " pod="opendatahub/opendatahub-operator-controller-manager-6d49d64447-dpntf" Jul 06 02:47:24.768580 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:47:24.768311 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/5adfd973-3482-4802-b49e-8a5151e3cf88-apiservice-cert\") pod \"opendatahub-operator-controller-manager-6d49d64447-dpntf\" (UID: \"5adfd973-3482-4802-b49e-8a5151e3cf88\") " pod="opendatahub/opendatahub-operator-controller-manager-6d49d64447-dpntf" Jul 06 02:47:24.768580 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:47:24.768370 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4vthc\" (UniqueName: \"kubernetes.io/projected/5adfd973-3482-4802-b49e-8a5151e3cf88-kube-api-access-4vthc\") pod \"opendatahub-operator-controller-manager-6d49d64447-dpntf\" (UID: \"5adfd973-3482-4802-b49e-8a5151e3cf88\") " pod="opendatahub/opendatahub-operator-controller-manager-6d49d64447-dpntf" Jul 06 02:47:24.768580 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:47:24.768389 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/5adfd973-3482-4802-b49e-8a5151e3cf88-webhook-cert\") pod \"opendatahub-operator-controller-manager-6d49d64447-dpntf\" (UID: \"5adfd973-3482-4802-b49e-8a5151e3cf88\") " pod="opendatahub/opendatahub-operator-controller-manager-6d49d64447-dpntf" Jul 06 02:47:24.768778 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:47:24.768746 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/5adfd973-3482-4802-b49e-8a5151e3cf88-manifests\") pod \"opendatahub-operator-controller-manager-6d49d64447-dpntf\" (UID: \"5adfd973-3482-4802-b49e-8a5151e3cf88\") " pod="opendatahub/opendatahub-operator-controller-manager-6d49d64447-dpntf" Jul 06 02:47:24.770706 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:47:24.770678 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/5adfd973-3482-4802-b49e-8a5151e3cf88-tmp\") pod \"opendatahub-operator-controller-manager-6d49d64447-dpntf\" (UID: \"5adfd973-3482-4802-b49e-8a5151e3cf88\") " pod="opendatahub/opendatahub-operator-controller-manager-6d49d64447-dpntf" Jul 06 02:47:24.770914 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:47:24.770893 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/5adfd973-3482-4802-b49e-8a5151e3cf88-webhook-cert\") pod \"opendatahub-operator-controller-manager-6d49d64447-dpntf\" (UID: \"5adfd973-3482-4802-b49e-8a5151e3cf88\") " pod="opendatahub/opendatahub-operator-controller-manager-6d49d64447-dpntf" Jul 06 02:47:24.770976 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:47:24.770921 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/5adfd973-3482-4802-b49e-8a5151e3cf88-apiservice-cert\") pod \"opendatahub-operator-controller-manager-6d49d64447-dpntf\" (UID: \"5adfd973-3482-4802-b49e-8a5151e3cf88\") " pod="opendatahub/opendatahub-operator-controller-manager-6d49d64447-dpntf" Jul 06 02:47:24.776900 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:47:24.776873 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-4vthc\" (UniqueName: \"kubernetes.io/projected/5adfd973-3482-4802-b49e-8a5151e3cf88-kube-api-access-4vthc\") pod \"opendatahub-operator-controller-manager-6d49d64447-dpntf\" (UID: \"5adfd973-3482-4802-b49e-8a5151e3cf88\") " pod="opendatahub/opendatahub-operator-controller-manager-6d49d64447-dpntf" Jul 06 02:47:24.908379 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:47:24.908266 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-6d49d64447-dpntf" Jul 06 02:47:25.074525 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:47:25.074491 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-6d49d64447-dpntf"] Jul 06 02:47:25.079119 ip-10-0-139-206 kubenswrapper[2557]: W0706 02:47:25.079086 2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5adfd973_3482_4802_b49e_8a5151e3cf88.slice/crio-5fe211b669b12b4e516f8881c990981981b38f00dc806e501eb341cb8aefb6e2 WatchSource:0}: Error finding container 5fe211b669b12b4e516f8881c990981981b38f00dc806e501eb341cb8aefb6e2: Status 404 returned error can't find the container with id 5fe211b669b12b4e516f8881c990981981b38f00dc806e501eb341cb8aefb6e2 Jul 06 02:47:25.998576 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:47:25.998531 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-6d49d64447-dpntf" event={"ID":"5adfd973-3482-4802-b49e-8a5151e3cf88","Type":"ContainerStarted","Data":"5fe211b669b12b4e516f8881c990981981b38f00dc806e501eb341cb8aefb6e2"} Jul 06 02:47:29.019492 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:47:29.019455 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-6d49d64447-dpntf" event={"ID":"5adfd973-3482-4802-b49e-8a5151e3cf88","Type":"ContainerStarted","Data":"84ed924090ff65ba8b477f3649559a83c5dff6857c68835751a9403143978f48"} Jul 06 02:47:30.025547 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:47:30.025514 2557 generic.go:356] "Generic (PLEG): container finished" podID="5adfd973-3482-4802-b49e-8a5151e3cf88" containerID="84ed924090ff65ba8b477f3649559a83c5dff6857c68835751a9403143978f48" exitCode=0 Jul 06 02:47:30.026032 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:47:30.025604 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-6d49d64447-dpntf" event={"ID":"5adfd973-3482-4802-b49e-8a5151e3cf88","Type":"ContainerDied","Data":"84ed924090ff65ba8b477f3649559a83c5dff6857c68835751a9403143978f48"} Jul 06 02:47:31.031801 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:47:31.031765 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-6d49d64447-dpntf" event={"ID":"5adfd973-3482-4802-b49e-8a5151e3cf88","Type":"ContainerStarted","Data":"cee15d0ac79bd5c76ed24ebce67f0ab492d52dee81f0ca6420ceca3dea6ceae4"} Jul 06 02:47:31.032190 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:47:31.031888 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/opendatahub-operator-controller-manager-6d49d64447-dpntf" Jul 06 02:47:31.053333 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:47:31.053276 2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/opendatahub-operator-controller-manager-6d49d64447-dpntf" podStartSLOduration=2.607095253 podStartE2EDuration="7.053257251s" totalImagesPullingTime="3.429249526s" totalInitContainerRuntime="1.016912472s" isStatefulPod=false podCreationTimestamp="2026-07-06 02:47:24 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 02:47:31.051334175 +0000 UTC m=+494.857620234" watchObservedRunningTime="2026-07-06 02:47:31.053257251 +0000 UTC m=+494.859543308" Jul 06 02:47:42.038421 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:47:42.038386 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/opendatahub-operator-controller-manager-6d49d64447-dpntf" Jul 06 02:47:44.230242 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:47:44.230186 2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/kube-auth-proxy-59cc9f6b7-lxz9f"] Jul 06 02:47:44.232593 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:47:44.232571 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-59cc9f6b7-lxz9f" Jul 06 02:47:44.235483 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:47:44.235384 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-creds\"" Jul 06 02:47:44.235483 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:47:44.235392 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-tls\"" Jul 06 02:47:44.236418 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:47:44.236397 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\"" Jul 06 02:47:44.236578 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:47:44.236562 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-dockercfg-8dpd9\"" Jul 06 02:47:44.236652 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:47:44.236565 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\"" Jul 06 02:47:44.246530 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:47:44.246506 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-59cc9f6b7-lxz9f"] Jul 06 02:47:44.317392 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:47:44.317333 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f699fa9e-d2a2-4e64-a4ee-7423fc3391d1-tls-certs\") pod \"kube-auth-proxy-59cc9f6b7-lxz9f\" (UID: \"f699fa9e-d2a2-4e64-a4ee-7423fc3391d1\") " pod="openshift-ingress/kube-auth-proxy-59cc9f6b7-lxz9f" Jul 06 02:47:44.317580 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:47:44.317428 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/f699fa9e-d2a2-4e64-a4ee-7423fc3391d1-tmp\") pod \"kube-auth-proxy-59cc9f6b7-lxz9f\" (UID: \"f699fa9e-d2a2-4e64-a4ee-7423fc3391d1\") " pod="openshift-ingress/kube-auth-proxy-59cc9f6b7-lxz9f" Jul 06 02:47:44.317580 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:47:44.317473 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xkzjx\" (UniqueName: \"kubernetes.io/projected/f699fa9e-d2a2-4e64-a4ee-7423fc3391d1-kube-api-access-xkzjx\") pod \"kube-auth-proxy-59cc9f6b7-lxz9f\" (UID: \"f699fa9e-d2a2-4e64-a4ee-7423fc3391d1\") " pod="openshift-ingress/kube-auth-proxy-59cc9f6b7-lxz9f" Jul 06 02:47:44.418385 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:47:44.418334 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f699fa9e-d2a2-4e64-a4ee-7423fc3391d1-tls-certs\") pod \"kube-auth-proxy-59cc9f6b7-lxz9f\" (UID: \"f699fa9e-d2a2-4e64-a4ee-7423fc3391d1\") " pod="openshift-ingress/kube-auth-proxy-59cc9f6b7-lxz9f" Jul 06 02:47:44.418577 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:47:44.418397 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/f699fa9e-d2a2-4e64-a4ee-7423fc3391d1-tmp\") pod \"kube-auth-proxy-59cc9f6b7-lxz9f\" (UID: \"f699fa9e-d2a2-4e64-a4ee-7423fc3391d1\") " pod="openshift-ingress/kube-auth-proxy-59cc9f6b7-lxz9f" Jul 06 02:47:44.418577 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:47:44.418449 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xkzjx\" (UniqueName: \"kubernetes.io/projected/f699fa9e-d2a2-4e64-a4ee-7423fc3391d1-kube-api-access-xkzjx\") pod \"kube-auth-proxy-59cc9f6b7-lxz9f\" (UID: \"f699fa9e-d2a2-4e64-a4ee-7423fc3391d1\") " pod="openshift-ingress/kube-auth-proxy-59cc9f6b7-lxz9f" Jul 06 02:47:44.420710 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:47:44.420685 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/f699fa9e-d2a2-4e64-a4ee-7423fc3391d1-tmp\") pod \"kube-auth-proxy-59cc9f6b7-lxz9f\" (UID: \"f699fa9e-d2a2-4e64-a4ee-7423fc3391d1\") " pod="openshift-ingress/kube-auth-proxy-59cc9f6b7-lxz9f" Jul 06 02:47:44.420963 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:47:44.420944 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f699fa9e-d2a2-4e64-a4ee-7423fc3391d1-tls-certs\") pod \"kube-auth-proxy-59cc9f6b7-lxz9f\" (UID: \"f699fa9e-d2a2-4e64-a4ee-7423fc3391d1\") " pod="openshift-ingress/kube-auth-proxy-59cc9f6b7-lxz9f" Jul 06 02:47:44.440534 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:47:44.440508 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-xkzjx\" (UniqueName: \"kubernetes.io/projected/f699fa9e-d2a2-4e64-a4ee-7423fc3391d1-kube-api-access-xkzjx\") pod \"kube-auth-proxy-59cc9f6b7-lxz9f\" (UID: \"f699fa9e-d2a2-4e64-a4ee-7423fc3391d1\") " pod="openshift-ingress/kube-auth-proxy-59cc9f6b7-lxz9f" Jul 06 02:47:44.543204 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:47:44.543110 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-59cc9f6b7-lxz9f" Jul 06 02:47:44.674811 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:47:44.674679 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-59cc9f6b7-lxz9f"] Jul 06 02:47:44.677304 ip-10-0-139-206 kubenswrapper[2557]: W0706 02:47:44.677280 2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf699fa9e_d2a2_4e64_a4ee_7423fc3391d1.slice/crio-f7505f12901297146f7f146d077e59ae44685bc1c064dbf8500515f85f5fe8aa WatchSource:0}: Error finding container f7505f12901297146f7f146d077e59ae44685bc1c064dbf8500515f85f5fe8aa: Status 404 returned error can't find the container with id f7505f12901297146f7f146d077e59ae44685bc1c064dbf8500515f85f5fe8aa Jul 06 02:47:45.090437 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:47:45.090402 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-59cc9f6b7-lxz9f" event={"ID":"f699fa9e-d2a2-4e64-a4ee-7423fc3391d1","Type":"ContainerStarted","Data":"f7505f12901297146f7f146d077e59ae44685bc1c064dbf8500515f85f5fe8aa"} Jul 06 02:47:48.106073 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:47:48.106037 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-59cc9f6b7-lxz9f" event={"ID":"f699fa9e-d2a2-4e64-a4ee-7423fc3391d1","Type":"ContainerStarted","Data":"aff8ea5af8df0349b8585345d34bd7b77dfba45d6779a8a3188fbf93bf1e515b"} Jul 06 02:47:48.121825 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:47:48.121769 2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/kube-auth-proxy-59cc9f6b7-lxz9f" podStartSLOduration=1.572664621 podStartE2EDuration="4.121745687s" totalImagesPullingTime="2.549081066s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 02:47:44 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 02:47:48.12080246 +0000 UTC m=+511.927088520" watchObservedRunningTime="2026-07-06 02:47:48.121745687 +0000 UTC m=+511.928031745" Jul 06 02:47:53.900438 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:47:53.900395 2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/istiod-openshift-gateway-94bb8fbfd-6pv67"] Jul 06 02:47:53.902837 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:47:53.902812 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-6pv67" Jul 06 02:47:53.905410 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:47:53.905382 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"cacerts\"" Jul 06 02:47:53.905410 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:47:53.905396 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\"" Jul 06 02:47:53.905410 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:47:53.905382 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istio-kubeconfig\"" Jul 06 02:47:53.905653 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:47:53.905628 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-tls\"" Jul 06 02:47:53.905725 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:47:53.905706 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-openshift-gateway-dockercfg-f2tgm\"" Jul 06 02:47:53.915926 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:47:53.915901 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-94bb8fbfd-6pv67"] Jul 06 02:47:53.996745 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:47:53.996705 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d6173166-e488-42f6-ab20-a96414f5e3e3-istio-token\") pod \"istiod-openshift-gateway-94bb8fbfd-6pv67\" (UID: \"d6173166-e488-42f6-ab20-a96414f5e3e3\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-6pv67" Jul 06 02:47:53.996745 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:47:53.996752 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/d6173166-e488-42f6-ab20-a96414f5e3e3-cacerts\") pod \"istiod-openshift-gateway-94bb8fbfd-6pv67\" (UID: \"d6173166-e488-42f6-ab20-a96414f5e3e3\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-6pv67" Jul 06 02:47:53.996969 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:47:53.996812 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/d6173166-e488-42f6-ab20-a96414f5e3e3-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-94bb8fbfd-6pv67\" (UID: \"d6173166-e488-42f6-ab20-a96414f5e3e3\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-6pv67" Jul 06 02:47:53.996969 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:47:53.996876 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/d6173166-e488-42f6-ab20-a96414f5e3e3-istio-kubeconfig\") pod \"istiod-openshift-gateway-94bb8fbfd-6pv67\" (UID: \"d6173166-e488-42f6-ab20-a96414f5e3e3\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-6pv67" Jul 06 02:47:53.996969 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:47:53.996927 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/d6173166-e488-42f6-ab20-a96414f5e3e3-local-certs\") pod \"istiod-openshift-gateway-94bb8fbfd-6pv67\" (UID: \"d6173166-e488-42f6-ab20-a96414f5e3e3\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-6pv67" Jul 06 02:47:53.996969 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:47:53.996943 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rfg26\" (UniqueName: \"kubernetes.io/projected/d6173166-e488-42f6-ab20-a96414f5e3e3-kube-api-access-rfg26\") pod \"istiod-openshift-gateway-94bb8fbfd-6pv67\" (UID: \"d6173166-e488-42f6-ab20-a96414f5e3e3\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-6pv67" Jul 06 02:47:53.997091 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:47:53.996984 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/d6173166-e488-42f6-ab20-a96414f5e3e3-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-94bb8fbfd-6pv67\" (UID: \"d6173166-e488-42f6-ab20-a96414f5e3e3\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-6pv67" Jul 06 02:47:54.097527 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:47:54.097472 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d6173166-e488-42f6-ab20-a96414f5e3e3-istio-token\") pod \"istiod-openshift-gateway-94bb8fbfd-6pv67\" (UID: \"d6173166-e488-42f6-ab20-a96414f5e3e3\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-6pv67" Jul 06 02:47:54.097527 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:47:54.097533 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/d6173166-e488-42f6-ab20-a96414f5e3e3-cacerts\") pod \"istiod-openshift-gateway-94bb8fbfd-6pv67\" (UID: \"d6173166-e488-42f6-ab20-a96414f5e3e3\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-6pv67" Jul 06 02:47:54.097760 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:47:54.097553 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/d6173166-e488-42f6-ab20-a96414f5e3e3-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-94bb8fbfd-6pv67\" (UID: \"d6173166-e488-42f6-ab20-a96414f5e3e3\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-6pv67" Jul 06 02:47:54.097760 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:47:54.097721 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/d6173166-e488-42f6-ab20-a96414f5e3e3-istio-kubeconfig\") pod \"istiod-openshift-gateway-94bb8fbfd-6pv67\" (UID: \"d6173166-e488-42f6-ab20-a96414f5e3e3\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-6pv67" Jul 06 02:47:54.097838 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:47:54.097788 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/d6173166-e488-42f6-ab20-a96414f5e3e3-local-certs\") pod \"istiod-openshift-gateway-94bb8fbfd-6pv67\" (UID: \"d6173166-e488-42f6-ab20-a96414f5e3e3\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-6pv67" Jul 06 02:47:54.097838 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:47:54.097814 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rfg26\" (UniqueName: \"kubernetes.io/projected/d6173166-e488-42f6-ab20-a96414f5e3e3-kube-api-access-rfg26\") pod \"istiod-openshift-gateway-94bb8fbfd-6pv67\" (UID: \"d6173166-e488-42f6-ab20-a96414f5e3e3\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-6pv67" Jul 06 02:47:54.097918 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:47:54.097863 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/d6173166-e488-42f6-ab20-a96414f5e3e3-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-94bb8fbfd-6pv67\" (UID: \"d6173166-e488-42f6-ab20-a96414f5e3e3\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-6pv67" Jul 06 02:47:54.098655 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:47:54.098623 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/d6173166-e488-42f6-ab20-a96414f5e3e3-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-94bb8fbfd-6pv67\" (UID: \"d6173166-e488-42f6-ab20-a96414f5e3e3\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-6pv67" Jul 06 02:47:54.099989 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:47:54.099960 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/d6173166-e488-42f6-ab20-a96414f5e3e3-cacerts\") pod \"istiod-openshift-gateway-94bb8fbfd-6pv67\" (UID: \"d6173166-e488-42f6-ab20-a96414f5e3e3\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-6pv67" Jul 06 02:47:54.099989 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:47:54.099977 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/d6173166-e488-42f6-ab20-a96414f5e3e3-local-certs\") pod \"istiod-openshift-gateway-94bb8fbfd-6pv67\" (UID: \"d6173166-e488-42f6-ab20-a96414f5e3e3\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-6pv67" Jul 06 02:47:54.100135 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:47:54.099970 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/d6173166-e488-42f6-ab20-a96414f5e3e3-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-94bb8fbfd-6pv67\" (UID: \"d6173166-e488-42f6-ab20-a96414f5e3e3\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-6pv67" Jul 06 02:47:54.100135 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:47:54.100054 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/d6173166-e488-42f6-ab20-a96414f5e3e3-istio-kubeconfig\") pod \"istiod-openshift-gateway-94bb8fbfd-6pv67\" (UID: \"d6173166-e488-42f6-ab20-a96414f5e3e3\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-6pv67" Jul 06 02:47:54.113785 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:47:54.113745 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-rfg26\" (UniqueName: \"kubernetes.io/projected/d6173166-e488-42f6-ab20-a96414f5e3e3-kube-api-access-rfg26\") pod \"istiod-openshift-gateway-94bb8fbfd-6pv67\" (UID: \"d6173166-e488-42f6-ab20-a96414f5e3e3\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-6pv67" Jul 06 02:47:54.114968 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:47:54.114947 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d6173166-e488-42f6-ab20-a96414f5e3e3-istio-token\") pod \"istiod-openshift-gateway-94bb8fbfd-6pv67\" (UID: \"d6173166-e488-42f6-ab20-a96414f5e3e3\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-6pv67" Jul 06 02:47:54.212021 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:47:54.211983 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-6pv67" Jul 06 02:47:54.372427 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:47:54.372387 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-94bb8fbfd-6pv67"] Jul 06 02:47:54.375523 ip-10-0-139-206 kubenswrapper[2557]: W0706 02:47:54.375491 2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd6173166_e488_42f6_ab20_a96414f5e3e3.slice/crio-cb90fe81ca738aa2304431bd9972d216b07755dfc4059febbee2b0f656f1b9d6 WatchSource:0}: Error finding container cb90fe81ca738aa2304431bd9972d216b07755dfc4059febbee2b0f656f1b9d6: Status 404 returned error can't find the container with id cb90fe81ca738aa2304431bd9972d216b07755dfc4059febbee2b0f656f1b9d6 Jul 06 02:47:55.146446 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:47:55.146385 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-6pv67" event={"ID":"d6173166-e488-42f6-ab20-a96414f5e3e3","Type":"ContainerStarted","Data":"cb90fe81ca738aa2304431bd9972d216b07755dfc4059febbee2b0f656f1b9d6"} Jul 06 02:47:55.781892 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:47:55.781798 2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/llmisvc-controller-manager-777c6bd7d5-qj8g5"] Jul 06 02:47:55.784342 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:47:55.784315 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-qj8g5" Jul 06 02:47:55.786831 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:47:55.786809 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"llmisvc-controller-manager-dockercfg-8kqz8\"" Jul 06 02:47:55.787055 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:47:55.787037 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"llmisvc-webhook-server-cert\"" Jul 06 02:47:55.797812 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:47:55.797781 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/llmisvc-controller-manager-777c6bd7d5-qj8g5"] Jul 06 02:47:55.921836 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:47:55.921789 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sxqqc\" (UniqueName: \"kubernetes.io/projected/8e1bab8d-a366-4520-b989-6b3ba7942c30-kube-api-access-sxqqc\") pod \"llmisvc-controller-manager-777c6bd7d5-qj8g5\" (UID: \"8e1bab8d-a366-4520-b989-6b3ba7942c30\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-qj8g5" Jul 06 02:47:55.922029 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:47:55.921884 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/8e1bab8d-a366-4520-b989-6b3ba7942c30-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-qj8g5\" (UID: \"8e1bab8d-a366-4520-b989-6b3ba7942c30\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-qj8g5" Jul 06 02:47:56.022531 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:47:56.022499 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/8e1bab8d-a366-4520-b989-6b3ba7942c30-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-qj8g5\" (UID: \"8e1bab8d-a366-4520-b989-6b3ba7942c30\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-qj8g5" Jul 06 02:47:56.022713 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:47:56.022580 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sxqqc\" (UniqueName: \"kubernetes.io/projected/8e1bab8d-a366-4520-b989-6b3ba7942c30-kube-api-access-sxqqc\") pod \"llmisvc-controller-manager-777c6bd7d5-qj8g5\" (UID: \"8e1bab8d-a366-4520-b989-6b3ba7942c30\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-qj8g5" Jul 06 02:47:56.025546 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:47:56.025515 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/8e1bab8d-a366-4520-b989-6b3ba7942c30-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-qj8g5\" (UID: \"8e1bab8d-a366-4520-b989-6b3ba7942c30\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-qj8g5" Jul 06 02:47:56.039455 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:47:56.039379 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-sxqqc\" (UniqueName: \"kubernetes.io/projected/8e1bab8d-a366-4520-b989-6b3ba7942c30-kube-api-access-sxqqc\") pod \"llmisvc-controller-manager-777c6bd7d5-qj8g5\" (UID: \"8e1bab8d-a366-4520-b989-6b3ba7942c30\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-qj8g5" Jul 06 02:47:56.097737 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:47:56.097705 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-qj8g5" Jul 06 02:47:56.233031 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:47:56.232997 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/llmisvc-controller-manager-777c6bd7d5-qj8g5"] Jul 06 02:47:56.235659 ip-10-0-139-206 kubenswrapper[2557]: W0706 02:47:56.235628 2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-pod8e1bab8d_a366_4520_b989_6b3ba7942c30.slice/crio-b2ba0d943f15c50bb9b9c3b8e409cf0e3b97f8f8a3644b1c4a51618241c4efde WatchSource:0}: Error finding container b2ba0d943f15c50bb9b9c3b8e409cf0e3b97f8f8a3644b1c4a51618241c4efde: Status 404 returned error can't find the container with id b2ba0d943f15c50bb9b9c3b8e409cf0e3b97f8f8a3644b1c4a51618241c4efde Jul 06 02:47:57.159986 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:47:57.159939 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-qj8g5" event={"ID":"8e1bab8d-a366-4520-b989-6b3ba7942c30","Type":"ContainerStarted","Data":"b2ba0d943f15c50bb9b9c3b8e409cf0e3b97f8f8a3644b1c4a51618241c4efde"} Jul 06 02:47:59.400668 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:47:59.400627 2557 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 06 02:47:59.400928 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:47:59.400698 2557 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 06 02:48:00.175843 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:48:00.175801 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-qj8g5" event={"ID":"8e1bab8d-a366-4520-b989-6b3ba7942c30","Type":"ContainerStarted","Data":"6b08843ebd699346691890fdfb19e80f382f2090c93ac5f8aba26d74eda483d7"} Jul 06 02:48:00.176044 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:48:00.175988 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-qj8g5" Jul 06 02:48:00.177667 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:48:00.177638 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-6pv67" event={"ID":"d6173166-e488-42f6-ab20-a96414f5e3e3","Type":"ContainerStarted","Data":"81193e81943b2020e4ed8453a9eb801f92aae176ff8d28ffb3e8b48ff6cb9a2e"} Jul 06 02:48:00.177888 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:48:00.177843 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-6pv67" Jul 06 02:48:00.179694 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:48:00.179669 2557 patch_prober.go:28] interesting pod/istiod-openshift-gateway-94bb8fbfd-6pv67 container/discovery namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body= Jul 06 02:48:00.179786 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:48:00.179739 2557 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-6pv67" podUID="d6173166-e488-42f6-ab20-a96414f5e3e3" containerName="discovery" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 06 02:48:00.190843 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:48:00.190780 2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-qj8g5" podStartSLOduration=2.0267731429999998 podStartE2EDuration="5.190763637s" totalImagesPullingTime="3.163990494s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 02:47:55 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 02:48:00.190421495 +0000 UTC m=+523.996707553" watchObservedRunningTime="2026-07-06 02:48:00.190763637 +0000 UTC m=+523.997049695" Jul 06 02:48:00.210813 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:48:00.210747 2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-6pv67" podStartSLOduration=2.187851864 podStartE2EDuration="7.210731511s" totalImagesPullingTime="5.022879647s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 02:47:53 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 02:48:00.209155944 +0000 UTC m=+524.015442011" watchObservedRunningTime="2026-07-06 02:48:00.210731511 +0000 UTC m=+524.017017568" Jul 06 02:48:01.186873 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:48:01.186840 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-6pv67" Jul 06 02:48:31.185951 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:48:31.185870 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-qj8g5" Jul 06 02:49:17.188120 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:49:17.188092 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-cw2f7_efcf5e11-1e5d-4e2a-9b38-303a8f0787c4/ovn-acl-logging/0.log" Jul 06 02:49:17.188684 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:49:17.188540 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-cw2f7_efcf5e11-1e5d-4e2a-9b38-303a8f0787c4/ovn-acl-logging/0.log" Jul 06 02:49:20.805420 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:49:20.805379 2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-operator-75cc8d5cb7-2r27p"] Jul 06 02:49:20.808635 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:49:20.808618 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-operator-75cc8d5cb7-2r27p" Jul 06 02:49:20.810951 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:49:20.810929 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\"" Jul 06 02:49:20.811087 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:49:20.811030 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-operator-dockercfg-5z7s9\"" Jul 06 02:49:20.811087 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:49:20.811058 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\"" Jul 06 02:49:20.817085 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:49:20.817059 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-operator-75cc8d5cb7-2r27p"] Jul 06 02:49:20.823642 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:49:20.823615 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gs75t\" (UniqueName: \"kubernetes.io/projected/0e7f6526-6634-4615-8f64-402d99502020-kube-api-access-gs75t\") pod \"authorino-operator-75cc8d5cb7-2r27p\" (UID: \"0e7f6526-6634-4615-8f64-402d99502020\") " pod="kuadrant-system/authorino-operator-75cc8d5cb7-2r27p" Jul 06 02:49:20.924600 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:49:20.924545 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gs75t\" (UniqueName: \"kubernetes.io/projected/0e7f6526-6634-4615-8f64-402d99502020-kube-api-access-gs75t\") pod \"authorino-operator-75cc8d5cb7-2r27p\" (UID: \"0e7f6526-6634-4615-8f64-402d99502020\") " pod="kuadrant-system/authorino-operator-75cc8d5cb7-2r27p" Jul 06 02:49:20.933304 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:49:20.933263 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-gs75t\" (UniqueName: \"kubernetes.io/projected/0e7f6526-6634-4615-8f64-402d99502020-kube-api-access-gs75t\") pod \"authorino-operator-75cc8d5cb7-2r27p\" (UID: \"0e7f6526-6634-4615-8f64-402d99502020\") " pod="kuadrant-system/authorino-operator-75cc8d5cb7-2r27p" Jul 06 02:49:21.119063 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:49:21.118966 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-operator-75cc8d5cb7-2r27p" Jul 06 02:49:21.253860 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:49:21.253832 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-operator-75cc8d5cb7-2r27p"] Jul 06 02:49:21.256514 ip-10-0-139-206 kubenswrapper[2557]: W0706 02:49:21.256487 2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0e7f6526_6634_4615_8f64_402d99502020.slice/crio-6f72add3bd12496a88c18d9e5d32f604c39b739165b6d23a8f7e7d27fc914dac WatchSource:0}: Error finding container 6f72add3bd12496a88c18d9e5d32f604c39b739165b6d23a8f7e7d27fc914dac: Status 404 returned error can't find the container with id 6f72add3bd12496a88c18d9e5d32f604c39b739165b6d23a8f7e7d27fc914dac Jul 06 02:49:21.522253 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:49:21.522210 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-operator-75cc8d5cb7-2r27p" event={"ID":"0e7f6526-6634-4615-8f64-402d99502020","Type":"ContainerStarted","Data":"6f72add3bd12496a88c18d9e5d32f604c39b739165b6d23a8f7e7d27fc914dac"} Jul 06 02:49:23.533623 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:49:23.533585 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-operator-75cc8d5cb7-2r27p" event={"ID":"0e7f6526-6634-4615-8f64-402d99502020","Type":"ContainerStarted","Data":"d3f820b316c710d6f47d0a9b6d73bca36658b97b77fc2f97c6e9d56ea2186afd"} Jul 06 02:49:23.534080 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:49:23.533714 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/authorino-operator-75cc8d5cb7-2r27p" Jul 06 02:49:23.548064 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:49:23.547965 2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-operator-75cc8d5cb7-2r27p" podStartSLOduration=1.364602227 podStartE2EDuration="3.547943945s" totalImagesPullingTime="2.183341718s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 02:49:20 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 02:49:23.547769679 +0000 UTC m=+607.354055738" watchObservedRunningTime="2026-07-06 02:49:23.547943945 +0000 UTC m=+607.354230006" Jul 06 02:49:24.068067 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:49:24.068036 2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-bbdd4588c-jj7xp"] Jul 06 02:49:24.071303 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:49:24.071276 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-bbdd4588c-jj7xp" Jul 06 02:49:24.086992 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:49:24.086961 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-bbdd4588c-jj7xp"] Jul 06 02:49:24.155683 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:49:24.155645 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/24c60978-daaf-41e7-8bc6-165a62f7e728-trusted-ca-bundle\") pod \"console-bbdd4588c-jj7xp\" (UID: \"24c60978-daaf-41e7-8bc6-165a62f7e728\") " pod="openshift-console/console-bbdd4588c-jj7xp" Jul 06 02:49:24.155860 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:49:24.155711 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/24c60978-daaf-41e7-8bc6-165a62f7e728-console-oauth-config\") pod \"console-bbdd4588c-jj7xp\" (UID: \"24c60978-daaf-41e7-8bc6-165a62f7e728\") " pod="openshift-console/console-bbdd4588c-jj7xp" Jul 06 02:49:24.155860 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:49:24.155737 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/24c60978-daaf-41e7-8bc6-165a62f7e728-oauth-serving-cert\") pod \"console-bbdd4588c-jj7xp\" (UID: \"24c60978-daaf-41e7-8bc6-165a62f7e728\") " pod="openshift-console/console-bbdd4588c-jj7xp" Jul 06 02:49:24.155860 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:49:24.155771 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/24c60978-daaf-41e7-8bc6-165a62f7e728-console-config\") pod \"console-bbdd4588c-jj7xp\" (UID: \"24c60978-daaf-41e7-8bc6-165a62f7e728\") " pod="openshift-console/console-bbdd4588c-jj7xp" Jul 06 02:49:24.155860 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:49:24.155803 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4q82r\" (UniqueName: \"kubernetes.io/projected/24c60978-daaf-41e7-8bc6-165a62f7e728-kube-api-access-4q82r\") pod \"console-bbdd4588c-jj7xp\" (UID: \"24c60978-daaf-41e7-8bc6-165a62f7e728\") " pod="openshift-console/console-bbdd4588c-jj7xp" Jul 06 02:49:24.155994 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:49:24.155876 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/24c60978-daaf-41e7-8bc6-165a62f7e728-console-serving-cert\") pod \"console-bbdd4588c-jj7xp\" (UID: \"24c60978-daaf-41e7-8bc6-165a62f7e728\") " pod="openshift-console/console-bbdd4588c-jj7xp" Jul 06 02:49:24.155994 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:49:24.155895 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/24c60978-daaf-41e7-8bc6-165a62f7e728-service-ca\") pod \"console-bbdd4588c-jj7xp\" (UID: \"24c60978-daaf-41e7-8bc6-165a62f7e728\") " pod="openshift-console/console-bbdd4588c-jj7xp" Jul 06 02:49:24.256498 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:49:24.256461 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/24c60978-daaf-41e7-8bc6-165a62f7e728-console-serving-cert\") pod \"console-bbdd4588c-jj7xp\" (UID: \"24c60978-daaf-41e7-8bc6-165a62f7e728\") " pod="openshift-console/console-bbdd4588c-jj7xp" Jul 06 02:49:24.256498 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:49:24.256499 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/24c60978-daaf-41e7-8bc6-165a62f7e728-service-ca\") pod \"console-bbdd4588c-jj7xp\" (UID: \"24c60978-daaf-41e7-8bc6-165a62f7e728\") " pod="openshift-console/console-bbdd4588c-jj7xp" Jul 06 02:49:24.256739 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:49:24.256529 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/24c60978-daaf-41e7-8bc6-165a62f7e728-trusted-ca-bundle\") pod \"console-bbdd4588c-jj7xp\" (UID: \"24c60978-daaf-41e7-8bc6-165a62f7e728\") " pod="openshift-console/console-bbdd4588c-jj7xp" Jul 06 02:49:24.256739 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:49:24.256571 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/24c60978-daaf-41e7-8bc6-165a62f7e728-console-oauth-config\") pod \"console-bbdd4588c-jj7xp\" (UID: \"24c60978-daaf-41e7-8bc6-165a62f7e728\") " pod="openshift-console/console-bbdd4588c-jj7xp" Jul 06 02:49:24.256739 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:49:24.256587 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/24c60978-daaf-41e7-8bc6-165a62f7e728-oauth-serving-cert\") pod \"console-bbdd4588c-jj7xp\" (UID: \"24c60978-daaf-41e7-8bc6-165a62f7e728\") " pod="openshift-console/console-bbdd4588c-jj7xp" Jul 06 02:49:24.256739 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:49:24.256607 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/24c60978-daaf-41e7-8bc6-165a62f7e728-console-config\") pod \"console-bbdd4588c-jj7xp\" (UID: \"24c60978-daaf-41e7-8bc6-165a62f7e728\") " pod="openshift-console/console-bbdd4588c-jj7xp" Jul 06 02:49:24.256739 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:49:24.256631 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4q82r\" (UniqueName: \"kubernetes.io/projected/24c60978-daaf-41e7-8bc6-165a62f7e728-kube-api-access-4q82r\") pod \"console-bbdd4588c-jj7xp\" (UID: \"24c60978-daaf-41e7-8bc6-165a62f7e728\") " pod="openshift-console/console-bbdd4588c-jj7xp" Jul 06 02:49:24.257408 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:49:24.257375 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/24c60978-daaf-41e7-8bc6-165a62f7e728-service-ca\") pod \"console-bbdd4588c-jj7xp\" (UID: \"24c60978-daaf-41e7-8bc6-165a62f7e728\") " pod="openshift-console/console-bbdd4588c-jj7xp" Jul 06 02:49:24.257408 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:49:24.257396 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/24c60978-daaf-41e7-8bc6-165a62f7e728-oauth-serving-cert\") pod \"console-bbdd4588c-jj7xp\" (UID: \"24c60978-daaf-41e7-8bc6-165a62f7e728\") " pod="openshift-console/console-bbdd4588c-jj7xp" Jul 06 02:49:24.257599 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:49:24.257400 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/24c60978-daaf-41e7-8bc6-165a62f7e728-console-config\") pod \"console-bbdd4588c-jj7xp\" (UID: \"24c60978-daaf-41e7-8bc6-165a62f7e728\") " pod="openshift-console/console-bbdd4588c-jj7xp" Jul 06 02:49:24.257599 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:49:24.257584 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/24c60978-daaf-41e7-8bc6-165a62f7e728-trusted-ca-bundle\") pod \"console-bbdd4588c-jj7xp\" (UID: \"24c60978-daaf-41e7-8bc6-165a62f7e728\") " pod="openshift-console/console-bbdd4588c-jj7xp" Jul 06 02:49:24.259116 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:49:24.259088 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/24c60978-daaf-41e7-8bc6-165a62f7e728-console-serving-cert\") pod \"console-bbdd4588c-jj7xp\" (UID: \"24c60978-daaf-41e7-8bc6-165a62f7e728\") " pod="openshift-console/console-bbdd4588c-jj7xp" Jul 06 02:49:24.259217 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:49:24.259128 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/24c60978-daaf-41e7-8bc6-165a62f7e728-console-oauth-config\") pod \"console-bbdd4588c-jj7xp\" (UID: \"24c60978-daaf-41e7-8bc6-165a62f7e728\") " pod="openshift-console/console-bbdd4588c-jj7xp" Jul 06 02:49:24.269518 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:49:24.269491 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-4q82r\" (UniqueName: \"kubernetes.io/projected/24c60978-daaf-41e7-8bc6-165a62f7e728-kube-api-access-4q82r\") pod \"console-bbdd4588c-jj7xp\" (UID: \"24c60978-daaf-41e7-8bc6-165a62f7e728\") " pod="openshift-console/console-bbdd4588c-jj7xp" Jul 06 02:49:24.380878 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:49:24.380790 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-bbdd4588c-jj7xp" Jul 06 02:49:24.546018 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:49:24.545857 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-bbdd4588c-jj7xp"] Jul 06 02:49:24.548511 ip-10-0-139-206 kubenswrapper[2557]: W0706 02:49:24.548481 2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod24c60978_daaf_41e7_8bc6_165a62f7e728.slice/crio-9657595aa1dc6802d3c7515b213f8899c44f7e694790a606b5452a9ea57adfaa WatchSource:0}: Error finding container 9657595aa1dc6802d3c7515b213f8899c44f7e694790a606b5452a9ea57adfaa: Status 404 returned error can't find the container with id 9657595aa1dc6802d3c7515b213f8899c44f7e694790a606b5452a9ea57adfaa Jul 06 02:49:25.544284 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:49:25.544246 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-bbdd4588c-jj7xp" event={"ID":"24c60978-daaf-41e7-8bc6-165a62f7e728","Type":"ContainerStarted","Data":"a26a47718d0dda22e3e746e69086481a1db1c7770f4c9efcc8608e5558f45442"} Jul 06 02:49:25.544459 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:49:25.544298 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-bbdd4588c-jj7xp" event={"ID":"24c60978-daaf-41e7-8bc6-165a62f7e728","Type":"ContainerStarted","Data":"9657595aa1dc6802d3c7515b213f8899c44f7e694790a606b5452a9ea57adfaa"} Jul 06 02:49:25.574886 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:49:25.574834 2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-bbdd4588c-jj7xp" podStartSLOduration=1.574817721 podStartE2EDuration="1.574817721s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 02:49:24 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 02:49:25.573341546 +0000 UTC m=+609.379627616" watchObservedRunningTime="2026-07-06 02:49:25.574817721 +0000 UTC m=+609.381103780" Jul 06 02:49:34.381505 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:49:34.381455 2557 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-bbdd4588c-jj7xp" Jul 06 02:49:34.382129 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:49:34.381572 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-bbdd4588c-jj7xp" Jul 06 02:49:34.386362 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:49:34.386328 2557 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-bbdd4588c-jj7xp" Jul 06 02:49:34.540968 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:49:34.540936 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/authorino-operator-75cc8d5cb7-2r27p" Jul 06 02:49:34.591111 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:49:34.591081 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-bbdd4588c-jj7xp" Jul 06 02:49:34.683921 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:49:34.683894 2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-7f77647fb6-hr58q"] Jul 06 02:49:45.694543 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:49:45.694468 2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-7rxqd"] Jul 06 02:49:45.697533 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:49:45.697516 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-7rxqd" Jul 06 02:49:45.702809 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:49:45.702783 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-controller-manager-dockercfg-f7szv\"" Jul 06 02:49:45.724488 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:49:45.724453 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-7rxqd"] Jul 06 02:49:45.851326 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:49:45.851286 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/13a00771-4978-4c4e-95da-9eac9dd18eee-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-6fb7879bc9-7rxqd\" (UID: \"13a00771-4978-4c4e-95da-9eac9dd18eee\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-7rxqd" Jul 06 02:49:45.851538 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:49:45.851383 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-f2j2p\" (UniqueName: \"kubernetes.io/projected/13a00771-4978-4c4e-95da-9eac9dd18eee-kube-api-access-f2j2p\") pod \"kuadrant-operator-controller-manager-6fb7879bc9-7rxqd\" (UID: \"13a00771-4978-4c4e-95da-9eac9dd18eee\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-7rxqd" Jul 06 02:49:45.952248 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:49:45.952154 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/13a00771-4978-4c4e-95da-9eac9dd18eee-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-6fb7879bc9-7rxqd\" (UID: \"13a00771-4978-4c4e-95da-9eac9dd18eee\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-7rxqd" Jul 06 02:49:45.952248 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:49:45.952236 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-f2j2p\" (UniqueName: \"kubernetes.io/projected/13a00771-4978-4c4e-95da-9eac9dd18eee-kube-api-access-f2j2p\") pod \"kuadrant-operator-controller-manager-6fb7879bc9-7rxqd\" (UID: \"13a00771-4978-4c4e-95da-9eac9dd18eee\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-7rxqd" Jul 06 02:49:45.952648 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:49:45.952627 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/13a00771-4978-4c4e-95da-9eac9dd18eee-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-6fb7879bc9-7rxqd\" (UID: \"13a00771-4978-4c4e-95da-9eac9dd18eee\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-7rxqd" Jul 06 02:49:45.960329 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:49:45.960301 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-f2j2p\" (UniqueName: \"kubernetes.io/projected/13a00771-4978-4c4e-95da-9eac9dd18eee-kube-api-access-f2j2p\") pod \"kuadrant-operator-controller-manager-6fb7879bc9-7rxqd\" (UID: \"13a00771-4978-4c4e-95da-9eac9dd18eee\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-7rxqd" Jul 06 02:49:46.007951 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:49:46.007914 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-7rxqd" Jul 06 02:49:46.151732 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:49:46.151703 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-7rxqd"] Jul 06 02:49:46.154645 ip-10-0-139-206 kubenswrapper[2557]: W0706 02:49:46.154616 2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod13a00771_4978_4c4e_95da_9eac9dd18eee.slice/crio-c56db57150f693aec53282e47c4895bd8503135c1fb87fba4bd20110cd75f3e7 WatchSource:0}: Error finding container c56db57150f693aec53282e47c4895bd8503135c1fb87fba4bd20110cd75f3e7: Status 404 returned error can't find the container with id c56db57150f693aec53282e47c4895bd8503135c1fb87fba4bd20110cd75f3e7 Jul 06 02:49:46.354225 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:49:46.354131 2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-7rxqd"] Jul 06 02:49:46.356171 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:49:46.356146 2557 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-7rxqd"] Jul 06 02:49:46.379303 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:49:46.379263 2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-btvd7"] Jul 06 02:49:46.383779 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:49:46.383756 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-btvd7" Jul 06 02:49:46.396920 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:49:46.396893 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-btvd7"] Jul 06 02:49:46.456688 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:49:46.456656 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-m7qnx\" (UniqueName: \"kubernetes.io/projected/7cae4e45-ce36-4c91-b712-d5eec6febcc5-kube-api-access-m7qnx\") pod \"kuadrant-operator-controller-manager-6fb7879bc9-btvd7\" (UID: \"7cae4e45-ce36-4c91-b712-d5eec6febcc5\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-btvd7" Jul 06 02:49:46.456865 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:49:46.456703 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/7cae4e45-ce36-4c91-b712-d5eec6febcc5-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-6fb7879bc9-btvd7\" (UID: \"7cae4e45-ce36-4c91-b712-d5eec6febcc5\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-btvd7" Jul 06 02:49:46.558114 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:49:46.558076 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/7cae4e45-ce36-4c91-b712-d5eec6febcc5-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-6fb7879bc9-btvd7\" (UID: \"7cae4e45-ce36-4c91-b712-d5eec6febcc5\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-btvd7" Jul 06 02:49:46.558300 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:49:46.558170 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-m7qnx\" (UniqueName: \"kubernetes.io/projected/7cae4e45-ce36-4c91-b712-d5eec6febcc5-kube-api-access-m7qnx\") pod \"kuadrant-operator-controller-manager-6fb7879bc9-btvd7\" (UID: \"7cae4e45-ce36-4c91-b712-d5eec6febcc5\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-btvd7" Jul 06 02:49:46.558543 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:49:46.558510 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/7cae4e45-ce36-4c91-b712-d5eec6febcc5-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-6fb7879bc9-btvd7\" (UID: \"7cae4e45-ce36-4c91-b712-d5eec6febcc5\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-btvd7" Jul 06 02:49:46.572829 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:49:46.572803 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-m7qnx\" (UniqueName: \"kubernetes.io/projected/7cae4e45-ce36-4c91-b712-d5eec6febcc5-kube-api-access-m7qnx\") pod \"kuadrant-operator-controller-manager-6fb7879bc9-btvd7\" (UID: \"7cae4e45-ce36-4c91-b712-d5eec6febcc5\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-btvd7" Jul 06 02:49:46.699613 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:49:46.699578 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-btvd7" Jul 06 02:49:47.161709 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:49:47.135823 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-btvd7"] Jul 06 02:49:47.204379 ip-10-0-139-206 kubenswrapper[2557]: W0706 02:49:47.204334 2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7cae4e45_ce36_4c91_b712_d5eec6febcc5.slice/crio-3643db21cb3ecad96cfaf7f5e9ab2b0c0185dc9a901d37807556a91e71621fbb WatchSource:0}: Error finding container 3643db21cb3ecad96cfaf7f5e9ab2b0c0185dc9a901d37807556a91e71621fbb: Status 404 returned error can't find the container with id 3643db21cb3ecad96cfaf7f5e9ab2b0c0185dc9a901d37807556a91e71621fbb Jul 06 02:49:47.646466 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:49:47.646425 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-btvd7" event={"ID":"7cae4e45-ce36-4c91-b712-d5eec6febcc5","Type":"ContainerStarted","Data":"3643db21cb3ecad96cfaf7f5e9ab2b0c0185dc9a901d37807556a91e71621fbb"} Jul 06 02:49:51.667744 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:49:51.667704 2557 kuberuntime_container.go:872] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-7rxqd" podUID="13a00771-4978-4c4e-95da-9eac9dd18eee" containerName="manager" containerID="cri-o://46f15a54c9dc5bf9643c6adfeeb0722d056a7c037c4fd0cc80314c35061b81b1" gracePeriod=2 Jul 06 02:49:51.669272 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:49:51.669246 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-btvd7" event={"ID":"7cae4e45-ce36-4c91-b712-d5eec6febcc5","Type":"ContainerStarted","Data":"1f8996f5ebb064befb72933c326323e8d55a66485e42bc109e093902662a6385"} Jul 06 02:49:51.669497 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:49:51.669480 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-btvd7" Jul 06 02:49:51.689545 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:49:51.689489 2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-btvd7" podStartSLOduration=2.179360198 podStartE2EDuration="5.689470546s" totalImagesPullingTime="3.510110348s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 02:49:46 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 02:49:51.688359433 +0000 UTC m=+635.494645482" watchObservedRunningTime="2026-07-06 02:49:51.689470546 +0000 UTC m=+635.495756608" Jul 06 02:49:51.913201 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:49:51.913175 2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-7rxqd" Jul 06 02:49:51.915321 ip-10-0-139-206 kubenswrapper[2557]: E0706 02:49:51.915294 2557 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-6fb7879bc9-7rxqd\" is forbidden: User \"system:node:ip-10-0-139-206.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-139-206.ec2.internal' and this object" podUID="13a00771-4978-4c4e-95da-9eac9dd18eee" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-7rxqd" Jul 06 02:49:52.005606 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:49:52.005567 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-f2j2p\" (UniqueName: \"kubernetes.io/projected/13a00771-4978-4c4e-95da-9eac9dd18eee-kube-api-access-f2j2p\") pod \"13a00771-4978-4c4e-95da-9eac9dd18eee\" (UID: \"13a00771-4978-4c4e-95da-9eac9dd18eee\") " Jul 06 02:49:52.005782 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:49:52.005627 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/13a00771-4978-4c4e-95da-9eac9dd18eee-extensions-socket-volume\") pod \"13a00771-4978-4c4e-95da-9eac9dd18eee\" (UID: \"13a00771-4978-4c4e-95da-9eac9dd18eee\") " Jul 06 02:49:52.005927 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:49:52.005897 2557 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/13a00771-4978-4c4e-95da-9eac9dd18eee-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "13a00771-4978-4c4e-95da-9eac9dd18eee" (UID: "13a00771-4978-4c4e-95da-9eac9dd18eee"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 02:49:52.007872 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:49:52.007849 2557 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/13a00771-4978-4c4e-95da-9eac9dd18eee-kube-api-access-f2j2p" (OuterVolumeSpecName: "kube-api-access-f2j2p") pod "13a00771-4978-4c4e-95da-9eac9dd18eee" (UID: "13a00771-4978-4c4e-95da-9eac9dd18eee"). InnerVolumeSpecName "kube-api-access-f2j2p". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 06 02:49:52.106829 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:49:52.106791 2557 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-f2j2p\" (UniqueName: \"kubernetes.io/projected/13a00771-4978-4c4e-95da-9eac9dd18eee-kube-api-access-f2j2p\") on node \"ip-10-0-139-206.ec2.internal\" DevicePath \"\"" Jul 06 02:49:52.106829 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:49:52.106823 2557 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/13a00771-4978-4c4e-95da-9eac9dd18eee-extensions-socket-volume\") on node \"ip-10-0-139-206.ec2.internal\" DevicePath \"\"" Jul 06 02:49:52.674941 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:49:52.674908 2557 generic.go:356] "Generic (PLEG): container finished" podID="13a00771-4978-4c4e-95da-9eac9dd18eee" containerID="46f15a54c9dc5bf9643c6adfeeb0722d056a7c037c4fd0cc80314c35061b81b1" exitCode=2 Jul 06 02:49:52.675372 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:49:52.674952 2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-7rxqd" Jul 06 02:49:52.675372 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:49:52.674999 2557 scope.go:117] "RemoveContainer" containerID="46f15a54c9dc5bf9643c6adfeeb0722d056a7c037c4fd0cc80314c35061b81b1" Jul 06 02:49:52.677341 ip-10-0-139-206 kubenswrapper[2557]: E0706 02:49:52.677309 2557 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-6fb7879bc9-7rxqd\" is forbidden: User \"system:node:ip-10-0-139-206.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-139-206.ec2.internal' and this object" podUID="13a00771-4978-4c4e-95da-9eac9dd18eee" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-7rxqd" Jul 06 02:49:52.684920 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:49:52.684903 2557 scope.go:117] "RemoveContainer" containerID="46f15a54c9dc5bf9643c6adfeeb0722d056a7c037c4fd0cc80314c35061b81b1" Jul 06 02:49:52.685176 ip-10-0-139-206 kubenswrapper[2557]: E0706 02:49:52.685150 2557 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"46f15a54c9dc5bf9643c6adfeeb0722d056a7c037c4fd0cc80314c35061b81b1\": container with ID starting with 46f15a54c9dc5bf9643c6adfeeb0722d056a7c037c4fd0cc80314c35061b81b1 not found: ID does not exist" containerID="46f15a54c9dc5bf9643c6adfeeb0722d056a7c037c4fd0cc80314c35061b81b1" Jul 06 02:49:52.685240 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:49:52.685189 2557 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"46f15a54c9dc5bf9643c6adfeeb0722d056a7c037c4fd0cc80314c35061b81b1"} err="failed to get container status \"46f15a54c9dc5bf9643c6adfeeb0722d056a7c037c4fd0cc80314c35061b81b1\": rpc error: code = NotFound desc = could not find container \"46f15a54c9dc5bf9643c6adfeeb0722d056a7c037c4fd0cc80314c35061b81b1\": container with ID starting with 46f15a54c9dc5bf9643c6adfeeb0722d056a7c037c4fd0cc80314c35061b81b1 not found: ID does not exist" Jul 06 02:49:52.685422 ip-10-0-139-206 kubenswrapper[2557]: E0706 02:49:52.685401 2557 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-6fb7879bc9-7rxqd\" is forbidden: User \"system:node:ip-10-0-139-206.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-139-206.ec2.internal' and this object" podUID="13a00771-4978-4c4e-95da-9eac9dd18eee" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-7rxqd" Jul 06 02:49:53.301505 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:49:53.301471 2557 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="13a00771-4978-4c4e-95da-9eac9dd18eee" path="/var/lib/kubelet/pods/13a00771-4978-4c4e-95da-9eac9dd18eee/volumes" Jul 06 02:49:59.704427 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:49:59.704389 2557 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-console/console-7f77647fb6-hr58q" podUID="d3c097d6-0b2a-4d50-9ba7-bf7d5336bd65" containerName="console" containerID="cri-o://00cf1feb70aa5c0e10972ffb17fb981ec6a17086345c2e69f0b3258be0cec8e0" gracePeriod=15 Jul 06 02:49:59.954233 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:49:59.954202 2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7f77647fb6-hr58q" Jul 06 02:50:00.071010 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:50:00.070922 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/d3c097d6-0b2a-4d50-9ba7-bf7d5336bd65-console-oauth-config\") pod \"d3c097d6-0b2a-4d50-9ba7-bf7d5336bd65\" (UID: \"d3c097d6-0b2a-4d50-9ba7-bf7d5336bd65\") " Jul 06 02:50:00.071010 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:50:00.070980 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/d3c097d6-0b2a-4d50-9ba7-bf7d5336bd65-console-serving-cert\") pod \"d3c097d6-0b2a-4d50-9ba7-bf7d5336bd65\" (UID: \"d3c097d6-0b2a-4d50-9ba7-bf7d5336bd65\") " Jul 06 02:50:00.071010 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:50:00.071009 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d3c097d6-0b2a-4d50-9ba7-bf7d5336bd65-trusted-ca-bundle\") pod \"d3c097d6-0b2a-4d50-9ba7-bf7d5336bd65\" (UID: \"d3c097d6-0b2a-4d50-9ba7-bf7d5336bd65\") " Jul 06 02:50:00.071260 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:50:00.071033 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/d3c097d6-0b2a-4d50-9ba7-bf7d5336bd65-service-ca\") pod \"d3c097d6-0b2a-4d50-9ba7-bf7d5336bd65\" (UID: \"d3c097d6-0b2a-4d50-9ba7-bf7d5336bd65\") " Jul 06 02:50:00.071260 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:50:00.071065 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-lz4jr\" (UniqueName: \"kubernetes.io/projected/d3c097d6-0b2a-4d50-9ba7-bf7d5336bd65-kube-api-access-lz4jr\") pod \"d3c097d6-0b2a-4d50-9ba7-bf7d5336bd65\" (UID: \"d3c097d6-0b2a-4d50-9ba7-bf7d5336bd65\") " Jul 06 02:50:00.071260 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:50:00.071084 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/d3c097d6-0b2a-4d50-9ba7-bf7d5336bd65-console-config\") pod \"d3c097d6-0b2a-4d50-9ba7-bf7d5336bd65\" (UID: \"d3c097d6-0b2a-4d50-9ba7-bf7d5336bd65\") " Jul 06 02:50:00.071260 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:50:00.071130 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/d3c097d6-0b2a-4d50-9ba7-bf7d5336bd65-oauth-serving-cert\") pod \"d3c097d6-0b2a-4d50-9ba7-bf7d5336bd65\" (UID: \"d3c097d6-0b2a-4d50-9ba7-bf7d5336bd65\") " Jul 06 02:50:00.071516 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:50:00.071459 2557 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d3c097d6-0b2a-4d50-9ba7-bf7d5336bd65-service-ca" (OuterVolumeSpecName: "service-ca") pod "d3c097d6-0b2a-4d50-9ba7-bf7d5336bd65" (UID: "d3c097d6-0b2a-4d50-9ba7-bf7d5336bd65"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 06 02:50:00.071571 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:50:00.071507 2557 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d3c097d6-0b2a-4d50-9ba7-bf7d5336bd65-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "d3c097d6-0b2a-4d50-9ba7-bf7d5336bd65" (UID: "d3c097d6-0b2a-4d50-9ba7-bf7d5336bd65"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 06 02:50:00.071671 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:50:00.071642 2557 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d3c097d6-0b2a-4d50-9ba7-bf7d5336bd65-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "d3c097d6-0b2a-4d50-9ba7-bf7d5336bd65" (UID: "d3c097d6-0b2a-4d50-9ba7-bf7d5336bd65"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 06 02:50:00.071767 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:50:00.071697 2557 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d3c097d6-0b2a-4d50-9ba7-bf7d5336bd65-console-config" (OuterVolumeSpecName: "console-config") pod "d3c097d6-0b2a-4d50-9ba7-bf7d5336bd65" (UID: "d3c097d6-0b2a-4d50-9ba7-bf7d5336bd65"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 06 02:50:00.073269 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:50:00.073245 2557 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d3c097d6-0b2a-4d50-9ba7-bf7d5336bd65-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "d3c097d6-0b2a-4d50-9ba7-bf7d5336bd65" (UID: "d3c097d6-0b2a-4d50-9ba7-bf7d5336bd65"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 06 02:50:00.073334 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:50:00.073250 2557 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d3c097d6-0b2a-4d50-9ba7-bf7d5336bd65-kube-api-access-lz4jr" (OuterVolumeSpecName: "kube-api-access-lz4jr") pod "d3c097d6-0b2a-4d50-9ba7-bf7d5336bd65" (UID: "d3c097d6-0b2a-4d50-9ba7-bf7d5336bd65"). InnerVolumeSpecName "kube-api-access-lz4jr". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 06 02:50:00.073334 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:50:00.073293 2557 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d3c097d6-0b2a-4d50-9ba7-bf7d5336bd65-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "d3c097d6-0b2a-4d50-9ba7-bf7d5336bd65" (UID: "d3c097d6-0b2a-4d50-9ba7-bf7d5336bd65"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 06 02:50:00.172263 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:50:00.172224 2557 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/d3c097d6-0b2a-4d50-9ba7-bf7d5336bd65-console-oauth-config\") on node \"ip-10-0-139-206.ec2.internal\" DevicePath \"\"" Jul 06 02:50:00.172263 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:50:00.172251 2557 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/d3c097d6-0b2a-4d50-9ba7-bf7d5336bd65-console-serving-cert\") on node \"ip-10-0-139-206.ec2.internal\" DevicePath \"\"" Jul 06 02:50:00.172263 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:50:00.172263 2557 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d3c097d6-0b2a-4d50-9ba7-bf7d5336bd65-trusted-ca-bundle\") on node \"ip-10-0-139-206.ec2.internal\" DevicePath \"\"" Jul 06 02:50:00.172263 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:50:00.172273 2557 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/d3c097d6-0b2a-4d50-9ba7-bf7d5336bd65-service-ca\") on node \"ip-10-0-139-206.ec2.internal\" DevicePath \"\"" Jul 06 02:50:00.172553 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:50:00.172281 2557 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-lz4jr\" (UniqueName: \"kubernetes.io/projected/d3c097d6-0b2a-4d50-9ba7-bf7d5336bd65-kube-api-access-lz4jr\") on node \"ip-10-0-139-206.ec2.internal\" DevicePath \"\"" Jul 06 02:50:00.172553 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:50:00.172290 2557 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/d3c097d6-0b2a-4d50-9ba7-bf7d5336bd65-console-config\") on node \"ip-10-0-139-206.ec2.internal\" DevicePath \"\"" Jul 06 02:50:00.172553 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:50:00.172298 2557 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/d3c097d6-0b2a-4d50-9ba7-bf7d5336bd65-oauth-serving-cert\") on node \"ip-10-0-139-206.ec2.internal\" DevicePath \"\"" Jul 06 02:50:00.711703 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:50:00.711665 2557 generic.go:356] "Generic (PLEG): container finished" podID="d3c097d6-0b2a-4d50-9ba7-bf7d5336bd65" containerID="00cf1feb70aa5c0e10972ffb17fb981ec6a17086345c2e69f0b3258be0cec8e0" exitCode=0 Jul 06 02:50:00.712115 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:50:00.711717 2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7f77647fb6-hr58q" Jul 06 02:50:00.712115 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:50:00.711752 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7f77647fb6-hr58q" event={"ID":"d3c097d6-0b2a-4d50-9ba7-bf7d5336bd65","Type":"ContainerDied","Data":"00cf1feb70aa5c0e10972ffb17fb981ec6a17086345c2e69f0b3258be0cec8e0"} Jul 06 02:50:00.712115 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:50:00.711809 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7f77647fb6-hr58q" event={"ID":"d3c097d6-0b2a-4d50-9ba7-bf7d5336bd65","Type":"ContainerDied","Data":"8dc21ee92322b3a66d3be120e02ba5eb0fe174e41ee5323c12340b33887fc216"} Jul 06 02:50:00.712115 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:50:00.711832 2557 scope.go:117] "RemoveContainer" containerID="00cf1feb70aa5c0e10972ffb17fb981ec6a17086345c2e69f0b3258be0cec8e0" Jul 06 02:50:00.720662 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:50:00.720640 2557 scope.go:117] "RemoveContainer" containerID="00cf1feb70aa5c0e10972ffb17fb981ec6a17086345c2e69f0b3258be0cec8e0" Jul 06 02:50:00.720925 ip-10-0-139-206 kubenswrapper[2557]: E0706 02:50:00.720906 2557 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"00cf1feb70aa5c0e10972ffb17fb981ec6a17086345c2e69f0b3258be0cec8e0\": container with ID starting with 00cf1feb70aa5c0e10972ffb17fb981ec6a17086345c2e69f0b3258be0cec8e0 not found: ID does not exist" containerID="00cf1feb70aa5c0e10972ffb17fb981ec6a17086345c2e69f0b3258be0cec8e0" Jul 06 02:50:00.720976 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:50:00.720935 2557 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"00cf1feb70aa5c0e10972ffb17fb981ec6a17086345c2e69f0b3258be0cec8e0"} err="failed to get container status \"00cf1feb70aa5c0e10972ffb17fb981ec6a17086345c2e69f0b3258be0cec8e0\": rpc error: code = NotFound desc = could not find container \"00cf1feb70aa5c0e10972ffb17fb981ec6a17086345c2e69f0b3258be0cec8e0\": container with ID starting with 00cf1feb70aa5c0e10972ffb17fb981ec6a17086345c2e69f0b3258be0cec8e0 not found: ID does not exist" Jul 06 02:50:00.732208 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:50:00.732186 2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-7f77647fb6-hr58q"] Jul 06 02:50:00.736235 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:50:00.736214 2557 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-7f77647fb6-hr58q"] Jul 06 02:50:01.301140 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:50:01.301105 2557 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d3c097d6-0b2a-4d50-9ba7-bf7d5336bd65" path="/var/lib/kubelet/pods/d3c097d6-0b2a-4d50-9ba7-bf7d5336bd65/volumes" Jul 06 02:50:02.677617 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:50:02.677584 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-btvd7" Jul 06 02:50:02.913412 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:50:02.913377 2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-2jh7p"] Jul 06 02:50:02.913688 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:50:02.913675 2557 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="13a00771-4978-4c4e-95da-9eac9dd18eee" containerName="manager" Jul 06 02:50:02.913746 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:50:02.913689 2557 state_mem.go:107] "Deleted CPUSet assignment" podUID="13a00771-4978-4c4e-95da-9eac9dd18eee" containerName="manager" Jul 06 02:50:02.913746 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:50:02.913698 2557 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d3c097d6-0b2a-4d50-9ba7-bf7d5336bd65" containerName="console" Jul 06 02:50:02.913746 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:50:02.913704 2557 state_mem.go:107] "Deleted CPUSet assignment" podUID="d3c097d6-0b2a-4d50-9ba7-bf7d5336bd65" containerName="console" Jul 06 02:50:02.913863 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:50:02.913788 2557 memory_manager.go:368] "RemoveStaleState removing state" podUID="d3c097d6-0b2a-4d50-9ba7-bf7d5336bd65" containerName="console" Jul 06 02:50:02.913863 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:50:02.913800 2557 memory_manager.go:368] "RemoveStaleState removing state" podUID="13a00771-4978-4c4e-95da-9eac9dd18eee" containerName="manager" Jul 06 02:50:02.918652 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:50:02.918635 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-2jh7p" Jul 06 02:50:02.930663 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:50:02.930639 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-2jh7p"] Jul 06 02:50:03.095716 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:50:03.095675 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/6baccf3d-a514-4e4d-bd64-39be01777fc5-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-2jh7p\" (UID: \"6baccf3d-a514-4e4d-bd64-39be01777fc5\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-2jh7p" Jul 06 02:50:03.095868 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:50:03.095796 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jqrdr\" (UniqueName: \"kubernetes.io/projected/6baccf3d-a514-4e4d-bd64-39be01777fc5-kube-api-access-jqrdr\") pod \"kuadrant-operator-controller-manager-86878c769f-2jh7p\" (UID: \"6baccf3d-a514-4e4d-bd64-39be01777fc5\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-2jh7p" Jul 06 02:50:03.197209 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:50:03.197123 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jqrdr\" (UniqueName: \"kubernetes.io/projected/6baccf3d-a514-4e4d-bd64-39be01777fc5-kube-api-access-jqrdr\") pod \"kuadrant-operator-controller-manager-86878c769f-2jh7p\" (UID: \"6baccf3d-a514-4e4d-bd64-39be01777fc5\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-2jh7p" Jul 06 02:50:03.197209 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:50:03.197178 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/6baccf3d-a514-4e4d-bd64-39be01777fc5-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-2jh7p\" (UID: \"6baccf3d-a514-4e4d-bd64-39be01777fc5\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-2jh7p" Jul 06 02:50:03.197527 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:50:03.197512 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/6baccf3d-a514-4e4d-bd64-39be01777fc5-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-2jh7p\" (UID: \"6baccf3d-a514-4e4d-bd64-39be01777fc5\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-2jh7p" Jul 06 02:50:03.204865 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:50:03.204844 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-jqrdr\" (UniqueName: \"kubernetes.io/projected/6baccf3d-a514-4e4d-bd64-39be01777fc5-kube-api-access-jqrdr\") pod \"kuadrant-operator-controller-manager-86878c769f-2jh7p\" (UID: \"6baccf3d-a514-4e4d-bd64-39be01777fc5\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-2jh7p" Jul 06 02:50:03.228461 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:50:03.228433 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-2jh7p" Jul 06 02:50:03.362416 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:50:03.362376 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-2jh7p"] Jul 06 02:50:03.365422 ip-10-0-139-206 kubenswrapper[2557]: W0706 02:50:03.365397 2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6baccf3d_a514_4e4d_bd64_39be01777fc5.slice/crio-4bb6a0a332581f3445764755495900faaa2b6fe176fb6c8f06ed601b80f4ebb6 WatchSource:0}: Error finding container 4bb6a0a332581f3445764755495900faaa2b6fe176fb6c8f06ed601b80f4ebb6: Status 404 returned error can't find the container with id 4bb6a0a332581f3445764755495900faaa2b6fe176fb6c8f06ed601b80f4ebb6 Jul 06 02:50:03.729397 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:50:03.729323 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-2jh7p" event={"ID":"6baccf3d-a514-4e4d-bd64-39be01777fc5","Type":"ContainerStarted","Data":"7921a4e1484fc36414507862f48e6bff76a7b62dcfb1a760d176b6eac7a1a506"} Jul 06 02:50:03.729813 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:50:03.729410 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-2jh7p" event={"ID":"6baccf3d-a514-4e4d-bd64-39be01777fc5","Type":"ContainerStarted","Data":"4bb6a0a332581f3445764755495900faaa2b6fe176fb6c8f06ed601b80f4ebb6"} Jul 06 02:50:03.729813 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:50:03.729531 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-2jh7p" Jul 06 02:50:03.749238 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:50:03.749182 2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-2jh7p" podStartSLOduration=1.749165835 podStartE2EDuration="1.749165835s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 02:50:02 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 02:50:03.747142146 +0000 UTC m=+647.553428206" watchObservedRunningTime="2026-07-06 02:50:03.749165835 +0000 UTC m=+647.555451892" Jul 06 02:50:14.736778 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:50:14.736732 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-2jh7p" Jul 06 02:50:14.781142 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:50:14.781110 2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-btvd7"] Jul 06 02:50:14.781447 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:50:14.781419 2557 kuberuntime_container.go:872] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-btvd7" podUID="7cae4e45-ce36-4c91-b712-d5eec6febcc5" containerName="manager" containerID="cri-o://1f8996f5ebb064befb72933c326323e8d55a66485e42bc109e093902662a6385" gracePeriod=10 Jul 06 02:50:15.028743 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:50:15.028717 2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-btvd7" Jul 06 02:50:15.093657 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:50:15.093624 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-m7qnx\" (UniqueName: \"kubernetes.io/projected/7cae4e45-ce36-4c91-b712-d5eec6febcc5-kube-api-access-m7qnx\") pod \"7cae4e45-ce36-4c91-b712-d5eec6febcc5\" (UID: \"7cae4e45-ce36-4c91-b712-d5eec6febcc5\") " Jul 06 02:50:15.093843 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:50:15.093679 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/7cae4e45-ce36-4c91-b712-d5eec6febcc5-extensions-socket-volume\") pod \"7cae4e45-ce36-4c91-b712-d5eec6febcc5\" (UID: \"7cae4e45-ce36-4c91-b712-d5eec6febcc5\") " Jul 06 02:50:15.094168 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:50:15.094145 2557 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7cae4e45-ce36-4c91-b712-d5eec6febcc5-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "7cae4e45-ce36-4c91-b712-d5eec6febcc5" (UID: "7cae4e45-ce36-4c91-b712-d5eec6febcc5"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 02:50:15.095790 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:50:15.095760 2557 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7cae4e45-ce36-4c91-b712-d5eec6febcc5-kube-api-access-m7qnx" (OuterVolumeSpecName: "kube-api-access-m7qnx") pod "7cae4e45-ce36-4c91-b712-d5eec6febcc5" (UID: "7cae4e45-ce36-4c91-b712-d5eec6febcc5"). InnerVolumeSpecName "kube-api-access-m7qnx". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 06 02:50:15.194648 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:50:15.194615 2557 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-m7qnx\" (UniqueName: \"kubernetes.io/projected/7cae4e45-ce36-4c91-b712-d5eec6febcc5-kube-api-access-m7qnx\") on node \"ip-10-0-139-206.ec2.internal\" DevicePath \"\"" Jul 06 02:50:15.194648 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:50:15.194644 2557 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/7cae4e45-ce36-4c91-b712-d5eec6febcc5-extensions-socket-volume\") on node \"ip-10-0-139-206.ec2.internal\" DevicePath \"\"" Jul 06 02:50:15.793408 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:50:15.793374 2557 generic.go:356] "Generic (PLEG): container finished" podID="7cae4e45-ce36-4c91-b712-d5eec6febcc5" containerID="1f8996f5ebb064befb72933c326323e8d55a66485e42bc109e093902662a6385" exitCode=0 Jul 06 02:50:15.793408 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:50:15.793414 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-btvd7" event={"ID":"7cae4e45-ce36-4c91-b712-d5eec6febcc5","Type":"ContainerDied","Data":"1f8996f5ebb064befb72933c326323e8d55a66485e42bc109e093902662a6385"} Jul 06 02:50:15.793930 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:50:15.793457 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-btvd7" event={"ID":"7cae4e45-ce36-4c91-b712-d5eec6febcc5","Type":"ContainerDied","Data":"3643db21cb3ecad96cfaf7f5e9ab2b0c0185dc9a901d37807556a91e71621fbb"} Jul 06 02:50:15.793930 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:50:15.793481 2557 scope.go:117] "RemoveContainer" containerID="1f8996f5ebb064befb72933c326323e8d55a66485e42bc109e093902662a6385" Jul 06 02:50:15.793930 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:50:15.793415 2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-btvd7" Jul 06 02:50:15.802030 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:50:15.802012 2557 scope.go:117] "RemoveContainer" containerID="1f8996f5ebb064befb72933c326323e8d55a66485e42bc109e093902662a6385" Jul 06 02:50:15.802311 ip-10-0-139-206 kubenswrapper[2557]: E0706 02:50:15.802291 2557 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1f8996f5ebb064befb72933c326323e8d55a66485e42bc109e093902662a6385\": container with ID starting with 1f8996f5ebb064befb72933c326323e8d55a66485e42bc109e093902662a6385 not found: ID does not exist" containerID="1f8996f5ebb064befb72933c326323e8d55a66485e42bc109e093902662a6385" Jul 06 02:50:15.802385 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:50:15.802320 2557 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1f8996f5ebb064befb72933c326323e8d55a66485e42bc109e093902662a6385"} err="failed to get container status \"1f8996f5ebb064befb72933c326323e8d55a66485e42bc109e093902662a6385\": rpc error: code = NotFound desc = could not find container \"1f8996f5ebb064befb72933c326323e8d55a66485e42bc109e093902662a6385\": container with ID starting with 1f8996f5ebb064befb72933c326323e8d55a66485e42bc109e093902662a6385 not found: ID does not exist" Jul 06 02:50:15.817966 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:50:15.817935 2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-btvd7"] Jul 06 02:50:15.823960 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:50:15.823933 2557 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-btvd7"] Jul 06 02:50:17.303034 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:50:17.303000 2557 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="7cae4e45-ce36-4c91-b712-d5eec6febcc5" path="/var/lib/kubelet/pods/7cae4e45-ce36-4c91-b712-d5eec6febcc5/volumes" Jul 06 02:50:21.376307 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:50:21.376268 2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-srzrr"] Jul 06 02:50:21.376697 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:50:21.376623 2557 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7cae4e45-ce36-4c91-b712-d5eec6febcc5" containerName="manager" Jul 06 02:50:21.376697 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:50:21.376637 2557 state_mem.go:107] "Deleted CPUSet assignment" podUID="7cae4e45-ce36-4c91-b712-d5eec6febcc5" containerName="manager" Jul 06 02:50:21.376775 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:50:21.376712 2557 memory_manager.go:368] "RemoveStaleState removing state" podUID="7cae4e45-ce36-4c91-b712-d5eec6febcc5" containerName="manager" Jul 06 02:50:21.383432 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:50:21.383408 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-54d6d988d5-srzrr" Jul 06 02:50:21.386685 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:50:21.386659 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"limitador-limits-config-limitador\"" Jul 06 02:50:21.386880 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:50:21.386692 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"default-dockercfg-fszpk\"" Jul 06 02:50:21.387573 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:50:21.387550 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-srzrr"] Jul 06 02:50:21.415747 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:50:21.415713 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-srzrr"] Jul 06 02:50:21.450398 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:50:21.450333 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/6ffdd30b-5859-42e4-bdf0-335d7179ce8c-config-file\") pod \"limitador-limitador-54d6d988d5-srzrr\" (UID: \"6ffdd30b-5859-42e4-bdf0-335d7179ce8c\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-srzrr" Jul 06 02:50:21.450590 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:50:21.450424 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-26psj\" (UniqueName: \"kubernetes.io/projected/6ffdd30b-5859-42e4-bdf0-335d7179ce8c-kube-api-access-26psj\") pod \"limitador-limitador-54d6d988d5-srzrr\" (UID: \"6ffdd30b-5859-42e4-bdf0-335d7179ce8c\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-srzrr" Jul 06 02:50:21.551493 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:50:21.551460 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/6ffdd30b-5859-42e4-bdf0-335d7179ce8c-config-file\") pod \"limitador-limitador-54d6d988d5-srzrr\" (UID: \"6ffdd30b-5859-42e4-bdf0-335d7179ce8c\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-srzrr" Jul 06 02:50:21.551668 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:50:21.551509 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-26psj\" (UniqueName: \"kubernetes.io/projected/6ffdd30b-5859-42e4-bdf0-335d7179ce8c-kube-api-access-26psj\") pod \"limitador-limitador-54d6d988d5-srzrr\" (UID: \"6ffdd30b-5859-42e4-bdf0-335d7179ce8c\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-srzrr" Jul 06 02:50:21.552103 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:50:21.552084 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/6ffdd30b-5859-42e4-bdf0-335d7179ce8c-config-file\") pod \"limitador-limitador-54d6d988d5-srzrr\" (UID: \"6ffdd30b-5859-42e4-bdf0-335d7179ce8c\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-srzrr" Jul 06 02:50:21.559149 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:50:21.559120 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-26psj\" (UniqueName: \"kubernetes.io/projected/6ffdd30b-5859-42e4-bdf0-335d7179ce8c-kube-api-access-26psj\") pod \"limitador-limitador-54d6d988d5-srzrr\" (UID: \"6ffdd30b-5859-42e4-bdf0-335d7179ce8c\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-srzrr" Jul 06 02:50:21.695049 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:50:21.695017 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-54d6d988d5-srzrr" Jul 06 02:50:21.826458 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:50:21.826433 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-srzrr"] Jul 06 02:50:21.830628 ip-10-0-139-206 kubenswrapper[2557]: W0706 02:50:21.830596 2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6ffdd30b_5859_42e4_bdf0_335d7179ce8c.slice/crio-5ef8bb9d54dcc266a017559bde520ea5c6a7810c102c88ac0aec5e6defba6893 WatchSource:0}: Error finding container 5ef8bb9d54dcc266a017559bde520ea5c6a7810c102c88ac0aec5e6defba6893: Status 404 returned error can't find the container with id 5ef8bb9d54dcc266a017559bde520ea5c6a7810c102c88ac0aec5e6defba6893 Jul 06 02:50:22.828599 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:50:22.828561 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-54d6d988d5-srzrr" event={"ID":"6ffdd30b-5859-42e4-bdf0-335d7179ce8c","Type":"ContainerStarted","Data":"5ef8bb9d54dcc266a017559bde520ea5c6a7810c102c88ac0aec5e6defba6893"} Jul 06 02:50:25.844772 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:50:25.844736 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-54d6d988d5-srzrr" event={"ID":"6ffdd30b-5859-42e4-bdf0-335d7179ce8c","Type":"ContainerStarted","Data":"b87f84e799437461b7f7a3786648c28efd1f7ff7129fa8fbff0854355969668f"} Jul 06 02:50:25.845184 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:50:25.844809 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-limitador-54d6d988d5-srzrr" Jul 06 02:50:25.860768 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:50:25.860712 2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-limitador-54d6d988d5-srzrr" podStartSLOduration=1.865097004 podStartE2EDuration="4.860693983s" totalImagesPullingTime="2.995596979s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 02:50:21 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 02:50:25.859402106 +0000 UTC m=+669.665688166" watchObservedRunningTime="2026-07-06 02:50:25.860693983 +0000 UTC m=+669.666980042" Jul 06 02:50:36.850605 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:50:36.850572 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-limitador-54d6d988d5-srzrr" Jul 06 02:51:38.588232 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:51:38.588155 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-srzrr"] Jul 06 02:51:57.673183 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:51:57.673143 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-srzrr"] Jul 06 02:52:13.063529 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:52:13.063490 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-srzrr"] Jul 06 02:52:19.358836 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:52:19.358798 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-srzrr"] Jul 06 02:52:45.568062 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:52:45.567982 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-srzrr"] Jul 06 02:52:53.664967 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:52:53.664924 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-srzrr"] Jul 06 02:53:10.161232 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:53:10.161190 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-srzrr"] Jul 06 02:54:17.223560 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:54:17.223481 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-cw2f7_efcf5e11-1e5d-4e2a-9b38-303a8f0787c4/ovn-acl-logging/0.log" Jul 06 02:54:17.224566 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:54:17.224542 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-cw2f7_efcf5e11-1e5d-4e2a-9b38-303a8f0787c4/ovn-acl-logging/0.log" Jul 06 02:54:49.062391 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:54:49.062336 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-srzrr"] Jul 06 02:54:59.467360 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:54:59.467310 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-srzrr"] Jul 06 02:55:10.059667 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:55:10.059627 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-srzrr"] Jul 06 02:55:18.461513 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:55:18.461469 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-srzrr"] Jul 06 02:55:29.362811 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:55:29.362771 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-srzrr"] Jul 06 02:55:56.663689 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:55:56.663649 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-srzrr"] Jul 06 02:56:09.364877 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:56:09.364836 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-srzrr"] Jul 06 02:56:42.256259 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:56:42.256220 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-srzrr"] Jul 06 02:56:51.761888 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:56:51.761845 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-srzrr"] Jul 06 02:57:00.353111 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:57:00.353073 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-srzrr"] Jul 06 02:57:08.860082 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:57:08.860000 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-srzrr"] Jul 06 02:57:20.068217 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:57:20.068177 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-srzrr"] Jul 06 02:57:28.860547 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:57:28.860510 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-srzrr"] Jul 06 02:57:40.958987 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:57:40.958949 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-srzrr"] Jul 06 02:57:50.363878 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:57:50.363839 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-srzrr"] Jul 06 02:58:13.460936 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:58:13.460900 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-srzrr"] Jul 06 02:58:24.863541 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:58:24.863502 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-srzrr"] Jul 06 02:58:48.465681 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:58:48.465597 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-srzrr"] Jul 06 02:58:58.568001 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:58:58.567961 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-srzrr"] Jul 06 02:59:13.365153 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:59:13.365119 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-srzrr"] Jul 06 02:59:17.251318 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:59:17.251286 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-cw2f7_efcf5e11-1e5d-4e2a-9b38-303a8f0787c4/ovn-acl-logging/0.log" Jul 06 02:59:17.253005 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:59:17.252978 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-cw2f7_efcf5e11-1e5d-4e2a-9b38-303a8f0787c4/ovn-acl-logging/0.log" Jul 06 02:59:22.074227 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:59:22.074190 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-srzrr"] Jul 06 02:59:31.859342 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:59:31.859308 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-srzrr"] Jul 06 02:59:41.159269 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:59:41.159231 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-srzrr"] Jul 06 02:59:50.665094 ip-10-0-139-206 kubenswrapper[2557]: I0706 02:59:50.665053 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-srzrr"] Jul 06 03:00:00.143493 ip-10-0-139-206 kubenswrapper[2557]: I0706 03:00:00.143457 2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-api-key-cleanup-29721780-lgpzb"] Jul 06 03:00:00.146621 ip-10-0-139-206 kubenswrapper[2557]: I0706 03:00:00.146603 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-key-cleanup-29721780-lgpzb" Jul 06 03:00:00.148823 ip-10-0-139-206 kubenswrapper[2557]: I0706 03:00:00.148799 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-api-dockercfg-mg4mh\"" Jul 06 03:00:00.162054 ip-10-0-139-206 kubenswrapper[2557]: I0706 03:00:00.162027 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-key-cleanup-29721780-lgpzb"] Jul 06 03:00:00.220596 ip-10-0-139-206 kubenswrapper[2557]: I0706 03:00:00.220562 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2cttm\" (UniqueName: \"kubernetes.io/projected/652718cb-fc9e-4323-a37e-3c49c4096123-kube-api-access-2cttm\") pod \"maas-api-key-cleanup-29721780-lgpzb\" (UID: \"652718cb-fc9e-4323-a37e-3c49c4096123\") " pod="opendatahub/maas-api-key-cleanup-29721780-lgpzb" Jul 06 03:00:00.321214 ip-10-0-139-206 kubenswrapper[2557]: I0706 03:00:00.321179 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2cttm\" (UniqueName: \"kubernetes.io/projected/652718cb-fc9e-4323-a37e-3c49c4096123-kube-api-access-2cttm\") pod \"maas-api-key-cleanup-29721780-lgpzb\" (UID: \"652718cb-fc9e-4323-a37e-3c49c4096123\") " pod="opendatahub/maas-api-key-cleanup-29721780-lgpzb" Jul 06 03:00:00.329544 ip-10-0-139-206 kubenswrapper[2557]: I0706 03:00:00.329508 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-2cttm\" (UniqueName: \"kubernetes.io/projected/652718cb-fc9e-4323-a37e-3c49c4096123-kube-api-access-2cttm\") pod \"maas-api-key-cleanup-29721780-lgpzb\" (UID: \"652718cb-fc9e-4323-a37e-3c49c4096123\") " pod="opendatahub/maas-api-key-cleanup-29721780-lgpzb" Jul 06 03:00:00.456866 ip-10-0-139-206 kubenswrapper[2557]: I0706 03:00:00.456828 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-key-cleanup-29721780-lgpzb" Jul 06 03:00:00.591555 ip-10-0-139-206 kubenswrapper[2557]: I0706 03:00:00.591528 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-key-cleanup-29721780-lgpzb"] Jul 06 03:00:00.595822 ip-10-0-139-206 kubenswrapper[2557]: W0706 03:00:00.595783 2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod652718cb_fc9e_4323_a37e_3c49c4096123.slice/crio-66eea01fa1787eba0a476ea62948d76a39236064ac6895b2792418b0f6ac0087 WatchSource:0}: Error finding container 66eea01fa1787eba0a476ea62948d76a39236064ac6895b2792418b0f6ac0087: Status 404 returned error can't find the container with id 66eea01fa1787eba0a476ea62948d76a39236064ac6895b2792418b0f6ac0087 Jul 06 03:00:00.598205 ip-10-0-139-206 kubenswrapper[2557]: I0706 03:00:00.598177 2557 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 06 03:00:01.404056 ip-10-0-139-206 kubenswrapper[2557]: I0706 03:00:01.404015 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-key-cleanup-29721780-lgpzb" event={"ID":"652718cb-fc9e-4323-a37e-3c49c4096123","Type":"ContainerStarted","Data":"66eea01fa1787eba0a476ea62948d76a39236064ac6895b2792418b0f6ac0087"} Jul 06 03:00:06.429491 ip-10-0-139-206 kubenswrapper[2557]: I0706 03:00:06.429452 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-key-cleanup-29721780-lgpzb" event={"ID":"652718cb-fc9e-4323-a37e-3c49c4096123","Type":"ContainerStarted","Data":"c09339fee0855b9b305ac772cf0981d5fdc492af6df709666e233bd02d132a06"} Jul 06 03:00:06.444710 ip-10-0-139-206 kubenswrapper[2557]: I0706 03:00:06.444663 2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-api-key-cleanup-29721780-lgpzb" podStartSLOduration=1.015419533 podStartE2EDuration="6.444645791s" totalImagesPullingTime="5.429226258s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 03:00:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 03:00:06.443503684 +0000 UTC m=+1250.249789742" watchObservedRunningTime="2026-07-06 03:00:06.444645791 +0000 UTC m=+1250.250931896" Jul 06 03:00:07.434973 ip-10-0-139-206 kubenswrapper[2557]: I0706 03:00:07.434932 2557 generic.go:356] "Generic (PLEG): container finished" podID="652718cb-fc9e-4323-a37e-3c49c4096123" containerID="c09339fee0855b9b305ac772cf0981d5fdc492af6df709666e233bd02d132a06" exitCode=0 Jul 06 03:00:07.435478 ip-10-0-139-206 kubenswrapper[2557]: I0706 03:00:07.434998 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-key-cleanup-29721780-lgpzb" event={"ID":"652718cb-fc9e-4323-a37e-3c49c4096123","Type":"ContainerDied","Data":"c09339fee0855b9b305ac772cf0981d5fdc492af6df709666e233bd02d132a06"} Jul 06 03:00:08.569737 ip-10-0-139-206 kubenswrapper[2557]: I0706 03:00:08.569713 2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-key-cleanup-29721780-lgpzb" Jul 06 03:00:08.697969 ip-10-0-139-206 kubenswrapper[2557]: I0706 03:00:08.697877 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-2cttm\" (UniqueName: \"kubernetes.io/projected/652718cb-fc9e-4323-a37e-3c49c4096123-kube-api-access-2cttm\") pod \"652718cb-fc9e-4323-a37e-3c49c4096123\" (UID: \"652718cb-fc9e-4323-a37e-3c49c4096123\") " Jul 06 03:00:08.700017 ip-10-0-139-206 kubenswrapper[2557]: I0706 03:00:08.699990 2557 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/652718cb-fc9e-4323-a37e-3c49c4096123-kube-api-access-2cttm" (OuterVolumeSpecName: "kube-api-access-2cttm") pod "652718cb-fc9e-4323-a37e-3c49c4096123" (UID: "652718cb-fc9e-4323-a37e-3c49c4096123"). InnerVolumeSpecName "kube-api-access-2cttm". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 06 03:00:08.798639 ip-10-0-139-206 kubenswrapper[2557]: I0706 03:00:08.798601 2557 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-2cttm\" (UniqueName: \"kubernetes.io/projected/652718cb-fc9e-4323-a37e-3c49c4096123-kube-api-access-2cttm\") on node \"ip-10-0-139-206.ec2.internal\" DevicePath \"\"" Jul 06 03:00:09.444651 ip-10-0-139-206 kubenswrapper[2557]: I0706 03:00:09.444618 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-key-cleanup-29721780-lgpzb" event={"ID":"652718cb-fc9e-4323-a37e-3c49c4096123","Type":"ContainerDied","Data":"66eea01fa1787eba0a476ea62948d76a39236064ac6895b2792418b0f6ac0087"} Jul 06 03:00:09.444651 ip-10-0-139-206 kubenswrapper[2557]: I0706 03:00:09.444639 2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-key-cleanup-29721780-lgpzb" Jul 06 03:00:09.444854 ip-10-0-139-206 kubenswrapper[2557]: I0706 03:00:09.444664 2557 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="66eea01fa1787eba0a476ea62948d76a39236064ac6895b2792418b0f6ac0087" Jul 06 03:00:16.761148 ip-10-0-139-206 kubenswrapper[2557]: I0706 03:00:16.761112 2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-2jh7p"] Jul 06 03:00:16.761663 ip-10-0-139-206 kubenswrapper[2557]: I0706 03:00:16.761321 2557 kuberuntime_container.go:872] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-2jh7p" podUID="6baccf3d-a514-4e4d-bd64-39be01777fc5" containerName="manager" containerID="cri-o://7921a4e1484fc36414507862f48e6bff76a7b62dcfb1a760d176b6eac7a1a506" gracePeriod=10 Jul 06 03:00:19.496627 ip-10-0-139-206 kubenswrapper[2557]: I0706 03:00:19.496588 2557 generic.go:356] "Generic (PLEG): container finished" podID="6baccf3d-a514-4e4d-bd64-39be01777fc5" containerID="7921a4e1484fc36414507862f48e6bff76a7b62dcfb1a760d176b6eac7a1a506" exitCode=0 Jul 06 03:00:19.497087 ip-10-0-139-206 kubenswrapper[2557]: I0706 03:00:19.496671 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-2jh7p" event={"ID":"6baccf3d-a514-4e4d-bd64-39be01777fc5","Type":"ContainerDied","Data":"7921a4e1484fc36414507862f48e6bff76a7b62dcfb1a760d176b6eac7a1a506"} Jul 06 03:00:19.509374 ip-10-0-139-206 kubenswrapper[2557]: I0706 03:00:19.509319 2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-2jh7p" Jul 06 03:00:19.596808 ip-10-0-139-206 kubenswrapper[2557]: I0706 03:00:19.596714 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-jqrdr\" (UniqueName: \"kubernetes.io/projected/6baccf3d-a514-4e4d-bd64-39be01777fc5-kube-api-access-jqrdr\") pod \"6baccf3d-a514-4e4d-bd64-39be01777fc5\" (UID: \"6baccf3d-a514-4e4d-bd64-39be01777fc5\") " Jul 06 03:00:19.596808 ip-10-0-139-206 kubenswrapper[2557]: I0706 03:00:19.596781 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/6baccf3d-a514-4e4d-bd64-39be01777fc5-extensions-socket-volume\") pod \"6baccf3d-a514-4e4d-bd64-39be01777fc5\" (UID: \"6baccf3d-a514-4e4d-bd64-39be01777fc5\") " Jul 06 03:00:19.597158 ip-10-0-139-206 kubenswrapper[2557]: I0706 03:00:19.597117 2557 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6baccf3d-a514-4e4d-bd64-39be01777fc5-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "6baccf3d-a514-4e4d-bd64-39be01777fc5" (UID: "6baccf3d-a514-4e4d-bd64-39be01777fc5"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 03:00:19.598840 ip-10-0-139-206 kubenswrapper[2557]: I0706 03:00:19.598819 2557 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6baccf3d-a514-4e4d-bd64-39be01777fc5-kube-api-access-jqrdr" (OuterVolumeSpecName: "kube-api-access-jqrdr") pod "6baccf3d-a514-4e4d-bd64-39be01777fc5" (UID: "6baccf3d-a514-4e4d-bd64-39be01777fc5"). InnerVolumeSpecName "kube-api-access-jqrdr". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 06 03:00:19.697316 ip-10-0-139-206 kubenswrapper[2557]: I0706 03:00:19.697260 2557 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-jqrdr\" (UniqueName: \"kubernetes.io/projected/6baccf3d-a514-4e4d-bd64-39be01777fc5-kube-api-access-jqrdr\") on node \"ip-10-0-139-206.ec2.internal\" DevicePath \"\"" Jul 06 03:00:19.697316 ip-10-0-139-206 kubenswrapper[2557]: I0706 03:00:19.697306 2557 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/6baccf3d-a514-4e4d-bd64-39be01777fc5-extensions-socket-volume\") on node \"ip-10-0-139-206.ec2.internal\" DevicePath \"\"" Jul 06 03:00:20.502743 ip-10-0-139-206 kubenswrapper[2557]: I0706 03:00:20.502711 2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-2jh7p" Jul 06 03:00:20.502743 ip-10-0-139-206 kubenswrapper[2557]: I0706 03:00:20.502721 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-2jh7p" event={"ID":"6baccf3d-a514-4e4d-bd64-39be01777fc5","Type":"ContainerDied","Data":"4bb6a0a332581f3445764755495900faaa2b6fe176fb6c8f06ed601b80f4ebb6"} Jul 06 03:00:20.503259 ip-10-0-139-206 kubenswrapper[2557]: I0706 03:00:20.502779 2557 scope.go:117] "RemoveContainer" containerID="7921a4e1484fc36414507862f48e6bff76a7b62dcfb1a760d176b6eac7a1a506" Jul 06 03:00:20.523609 ip-10-0-139-206 kubenswrapper[2557]: I0706 03:00:20.523580 2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-2jh7p"] Jul 06 03:00:20.528078 ip-10-0-139-206 kubenswrapper[2557]: I0706 03:00:20.528056 2557 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-2jh7p"] Jul 06 03:00:21.301716 ip-10-0-139-206 kubenswrapper[2557]: I0706 03:00:21.301682 2557 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="6baccf3d-a514-4e4d-bd64-39be01777fc5" path="/var/lib/kubelet/pods/6baccf3d-a514-4e4d-bd64-39be01777fc5/volumes" Jul 06 03:01:17.912312 ip-10-0-139-206 kubenswrapper[2557]: I0706 03:01:17.912269 2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-s4kxn"] Jul 06 03:01:17.912905 ip-10-0-139-206 kubenswrapper[2557]: I0706 03:01:17.912736 2557 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6baccf3d-a514-4e4d-bd64-39be01777fc5" containerName="manager" Jul 06 03:01:17.912905 ip-10-0-139-206 kubenswrapper[2557]: I0706 03:01:17.912755 2557 state_mem.go:107] "Deleted CPUSet assignment" podUID="6baccf3d-a514-4e4d-bd64-39be01777fc5" containerName="manager" Jul 06 03:01:17.912905 ip-10-0-139-206 kubenswrapper[2557]: I0706 03:01:17.912782 2557 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="652718cb-fc9e-4323-a37e-3c49c4096123" containerName="cleanup" Jul 06 03:01:17.912905 ip-10-0-139-206 kubenswrapper[2557]: I0706 03:01:17.912790 2557 state_mem.go:107] "Deleted CPUSet assignment" podUID="652718cb-fc9e-4323-a37e-3c49c4096123" containerName="cleanup" Jul 06 03:01:17.912905 ip-10-0-139-206 kubenswrapper[2557]: I0706 03:01:17.912874 2557 memory_manager.go:368] "RemoveStaleState removing state" podUID="652718cb-fc9e-4323-a37e-3c49c4096123" containerName="cleanup" Jul 06 03:01:17.912905 ip-10-0-139-206 kubenswrapper[2557]: I0706 03:01:17.912888 2557 memory_manager.go:368] "RemoveStaleState removing state" podUID="6baccf3d-a514-4e4d-bd64-39be01777fc5" containerName="manager" Jul 06 03:01:17.916036 ip-10-0-139-206 kubenswrapper[2557]: I0706 03:01:17.916015 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-s4kxn" Jul 06 03:01:17.919317 ip-10-0-139-206 kubenswrapper[2557]: I0706 03:01:17.919297 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-controller-manager-dockercfg-f7szv\"" Jul 06 03:01:17.925696 ip-10-0-139-206 kubenswrapper[2557]: I0706 03:01:17.925671 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-s4kxn"] Jul 06 03:01:17.983680 ip-10-0-139-206 kubenswrapper[2557]: I0706 03:01:17.983645 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rlvgj\" (UniqueName: \"kubernetes.io/projected/c42a9b7b-33ec-48d8-b744-764aad3ce2e9-kube-api-access-rlvgj\") pod \"kuadrant-operator-controller-manager-86878c769f-s4kxn\" (UID: \"c42a9b7b-33ec-48d8-b744-764aad3ce2e9\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-s4kxn" Jul 06 03:01:17.983872 ip-10-0-139-206 kubenswrapper[2557]: I0706 03:01:17.983690 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/c42a9b7b-33ec-48d8-b744-764aad3ce2e9-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-s4kxn\" (UID: \"c42a9b7b-33ec-48d8-b744-764aad3ce2e9\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-s4kxn" Jul 06 03:01:18.084674 ip-10-0-139-206 kubenswrapper[2557]: I0706 03:01:18.084631 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rlvgj\" (UniqueName: \"kubernetes.io/projected/c42a9b7b-33ec-48d8-b744-764aad3ce2e9-kube-api-access-rlvgj\") pod \"kuadrant-operator-controller-manager-86878c769f-s4kxn\" (UID: \"c42a9b7b-33ec-48d8-b744-764aad3ce2e9\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-s4kxn" Jul 06 03:01:18.084674 ip-10-0-139-206 kubenswrapper[2557]: I0706 03:01:18.084675 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/c42a9b7b-33ec-48d8-b744-764aad3ce2e9-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-s4kxn\" (UID: \"c42a9b7b-33ec-48d8-b744-764aad3ce2e9\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-s4kxn" Jul 06 03:01:18.085009 ip-10-0-139-206 kubenswrapper[2557]: I0706 03:01:18.084994 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/c42a9b7b-33ec-48d8-b744-764aad3ce2e9-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-s4kxn\" (UID: \"c42a9b7b-33ec-48d8-b744-764aad3ce2e9\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-s4kxn" Jul 06 03:01:18.095454 ip-10-0-139-206 kubenswrapper[2557]: I0706 03:01:18.095430 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-rlvgj\" (UniqueName: \"kubernetes.io/projected/c42a9b7b-33ec-48d8-b744-764aad3ce2e9-kube-api-access-rlvgj\") pod \"kuadrant-operator-controller-manager-86878c769f-s4kxn\" (UID: \"c42a9b7b-33ec-48d8-b744-764aad3ce2e9\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-s4kxn" Jul 06 03:01:18.226462 ip-10-0-139-206 kubenswrapper[2557]: I0706 03:01:18.226421 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-s4kxn" Jul 06 03:01:18.368544 ip-10-0-139-206 kubenswrapper[2557]: I0706 03:01:18.368511 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-s4kxn"] Jul 06 03:01:18.373252 ip-10-0-139-206 kubenswrapper[2557]: W0706 03:01:18.373227 2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc42a9b7b_33ec_48d8_b744_764aad3ce2e9.slice/crio-9133fd3977ea071c66f53c9fb008d2c7c49a599b07a22b1facd93740d8af1bae WatchSource:0}: Error finding container 9133fd3977ea071c66f53c9fb008d2c7c49a599b07a22b1facd93740d8af1bae: Status 404 returned error can't find the container with id 9133fd3977ea071c66f53c9fb008d2c7c49a599b07a22b1facd93740d8af1bae Jul 06 03:01:18.762734 ip-10-0-139-206 kubenswrapper[2557]: I0706 03:01:18.762701 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-s4kxn" event={"ID":"c42a9b7b-33ec-48d8-b744-764aad3ce2e9","Type":"ContainerStarted","Data":"d02dd8961f7beb80f5abca37cfda75058aca65978a881ba601081b85b19a4eed"} Jul 06 03:01:18.762943 ip-10-0-139-206 kubenswrapper[2557]: I0706 03:01:18.762756 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-s4kxn" event={"ID":"c42a9b7b-33ec-48d8-b744-764aad3ce2e9","Type":"ContainerStarted","Data":"9133fd3977ea071c66f53c9fb008d2c7c49a599b07a22b1facd93740d8af1bae"} Jul 06 03:01:18.762943 ip-10-0-139-206 kubenswrapper[2557]: I0706 03:01:18.762789 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-s4kxn" Jul 06 03:01:18.783343 ip-10-0-139-206 kubenswrapper[2557]: I0706 03:01:18.783287 2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-s4kxn" podStartSLOduration=1.783266214 podStartE2EDuration="1.783266214s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 03:01:17 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 03:01:18.782251583 +0000 UTC m=+1322.588537642" watchObservedRunningTime="2026-07-06 03:01:18.783266214 +0000 UTC m=+1322.589552275" Jul 06 03:01:29.770375 ip-10-0-139-206 kubenswrapper[2557]: I0706 03:01:29.770317 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-s4kxn" Jul 06 03:01:38.963236 ip-10-0-139-206 kubenswrapper[2557]: I0706 03:01:38.963143 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-srzrr"] Jul 06 03:01:47.761329 ip-10-0-139-206 kubenswrapper[2557]: I0706 03:01:47.761292 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-srzrr"] Jul 06 03:01:59.965717 ip-10-0-139-206 kubenswrapper[2557]: I0706 03:01:59.965685 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-srzrr"] Jul 06 03:02:09.770811 ip-10-0-139-206 kubenswrapper[2557]: I0706 03:02:09.770769 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-srzrr"] Jul 06 03:02:20.859627 ip-10-0-139-206 kubenswrapper[2557]: I0706 03:02:20.859584 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-srzrr"] Jul 06 03:02:30.958864 ip-10-0-139-206 kubenswrapper[2557]: I0706 03:02:30.958824 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-srzrr"] Jul 06 03:02:40.967951 ip-10-0-139-206 kubenswrapper[2557]: I0706 03:02:40.967910 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-srzrr"] Jul 06 03:02:51.065334 ip-10-0-139-206 kubenswrapper[2557]: I0706 03:02:51.065288 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-srzrr"] Jul 06 03:03:00.262258 ip-10-0-139-206 kubenswrapper[2557]: I0706 03:03:00.262222 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-srzrr"] Jul 06 03:03:08.273235 ip-10-0-139-206 kubenswrapper[2557]: I0706 03:03:08.273198 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-srzrr"] Jul 06 03:03:17.364337 ip-10-0-139-206 kubenswrapper[2557]: I0706 03:03:17.364299 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-srzrr"] Jul 06 03:03:25.168240 ip-10-0-139-206 kubenswrapper[2557]: I0706 03:03:25.168204 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-srzrr"] Jul 06 03:03:44.079645 ip-10-0-139-206 kubenswrapper[2557]: I0706 03:03:44.079609 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-srzrr"] Jul 06 03:03:53.474557 ip-10-0-139-206 kubenswrapper[2557]: I0706 03:03:53.474512 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-srzrr"] Jul 06 03:04:02.377449 ip-10-0-139-206 kubenswrapper[2557]: I0706 03:04:02.377413 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-srzrr"] Jul 06 03:04:14.379227 ip-10-0-139-206 kubenswrapper[2557]: I0706 03:04:14.379188 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-srzrr"] Jul 06 03:04:17.279064 ip-10-0-139-206 kubenswrapper[2557]: I0706 03:04:17.279037 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-cw2f7_efcf5e11-1e5d-4e2a-9b38-303a8f0787c4/ovn-acl-logging/0.log" Jul 06 03:04:17.280545 ip-10-0-139-206 kubenswrapper[2557]: I0706 03:04:17.280524 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-cw2f7_efcf5e11-1e5d-4e2a-9b38-303a8f0787c4/ovn-acl-logging/0.log" Jul 06 03:04:30.776840 ip-10-0-139-206 kubenswrapper[2557]: I0706 03:04:30.776805 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-srzrr"] Jul 06 03:04:40.285528 ip-10-0-139-206 kubenswrapper[2557]: I0706 03:04:40.285453 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-srzrr"] Jul 06 03:04:49.393270 ip-10-0-139-206 kubenswrapper[2557]: I0706 03:04:49.393235 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-srzrr"] Jul 06 03:04:59.378172 ip-10-0-139-206 kubenswrapper[2557]: I0706 03:04:59.378134 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-srzrr"] Jul 06 03:05:08.383177 ip-10-0-139-206 kubenswrapper[2557]: I0706 03:05:08.383143 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-srzrr"] Jul 06 03:05:17.477425 ip-10-0-139-206 kubenswrapper[2557]: I0706 03:05:17.477396 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-srzrr"] Jul 06 03:05:30.492312 ip-10-0-139-206 kubenswrapper[2557]: I0706 03:05:30.492277 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-srzrr"] Jul 06 03:05:39.373216 ip-10-0-139-206 kubenswrapper[2557]: I0706 03:05:39.373179 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-srzrr"] Jul 06 03:05:49.369180 ip-10-0-139-206 kubenswrapper[2557]: I0706 03:05:49.369139 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-srzrr"] Jul 06 03:06:01.163841 ip-10-0-139-206 kubenswrapper[2557]: I0706 03:06:01.163805 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-srzrr"] Jul 06 03:06:13.968275 ip-10-0-139-206 kubenswrapper[2557]: I0706 03:06:13.968057 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-srzrr"] Jul 06 03:06:22.565941 ip-10-0-139-206 kubenswrapper[2557]: I0706 03:06:22.565904 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-srzrr"] Jul 06 03:06:35.370366 ip-10-0-139-206 kubenswrapper[2557]: I0706 03:06:35.370322 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-srzrr"] Jul 06 03:06:46.972205 ip-10-0-139-206 kubenswrapper[2557]: I0706 03:06:46.972170 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-srzrr"] Jul 06 03:06:57.675986 ip-10-0-139-206 kubenswrapper[2557]: I0706 03:06:57.675948 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-srzrr"] Jul 06 03:07:06.362640 ip-10-0-139-206 kubenswrapper[2557]: I0706 03:07:06.362604 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-srzrr"] Jul 06 03:07:20.066638 ip-10-0-139-206 kubenswrapper[2557]: I0706 03:07:20.066599 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-srzrr"] Jul 06 03:07:28.284367 ip-10-0-139-206 kubenswrapper[2557]: I0706 03:07:28.284318 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-srzrr"] Jul 06 03:07:38.566573 ip-10-0-139-206 kubenswrapper[2557]: I0706 03:07:38.566489 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-srzrr"] Jul 06 03:07:45.779080 ip-10-0-139-206 kubenswrapper[2557]: I0706 03:07:45.779019 2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-ait-3fa4e34a-openshift-default-565ccd99d9-6chnx"] Jul 06 03:07:45.783050 ip-10-0-139-206 kubenswrapper[2557]: I0706 03:07:45.783031 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-3fa4e34a-openshift-default-565ccd99d9-6chnx" Jul 06 03:07:45.785597 ip-10-0-139-206 kubenswrapper[2557]: I0706 03:07:45.785576 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-ait-3fa4e34a-openshift-default-dockercfg-gnq64\"" Jul 06 03:07:45.794674 ip-10-0-139-206 kubenswrapper[2557]: I0706 03:07:45.794649 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-3fa4e34a-openshift-default-565ccd99d9-6chnx"] Jul 06 03:07:45.913212 ip-10-0-139-206 kubenswrapper[2557]: I0706 03:07:45.913168 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/64624a00-3881-4496-a485-7dfb24a4e564-istio-podinfo\") pod \"e2e-ait-3fa4e34a-openshift-default-565ccd99d9-6chnx\" (UID: \"64624a00-3881-4496-a485-7dfb24a4e564\") " pod="openshift-ingress/e2e-ait-3fa4e34a-openshift-default-565ccd99d9-6chnx" Jul 06 03:07:45.913212 ip-10-0-139-206 kubenswrapper[2557]: I0706 03:07:45.913210 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/64624a00-3881-4496-a485-7dfb24a4e564-credential-socket\") pod \"e2e-ait-3fa4e34a-openshift-default-565ccd99d9-6chnx\" (UID: \"64624a00-3881-4496-a485-7dfb24a4e564\") " pod="openshift-ingress/e2e-ait-3fa4e34a-openshift-default-565ccd99d9-6chnx" Jul 06 03:07:45.913505 ip-10-0-139-206 kubenswrapper[2557]: I0706 03:07:45.913249 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/64624a00-3881-4496-a485-7dfb24a4e564-istio-token\") pod \"e2e-ait-3fa4e34a-openshift-default-565ccd99d9-6chnx\" (UID: \"64624a00-3881-4496-a485-7dfb24a4e564\") " pod="openshift-ingress/e2e-ait-3fa4e34a-openshift-default-565ccd99d9-6chnx" Jul 06 03:07:45.913505 ip-10-0-139-206 kubenswrapper[2557]: I0706 03:07:45.913298 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/64624a00-3881-4496-a485-7dfb24a4e564-workload-socket\") pod \"e2e-ait-3fa4e34a-openshift-default-565ccd99d9-6chnx\" (UID: \"64624a00-3881-4496-a485-7dfb24a4e564\") " pod="openshift-ingress/e2e-ait-3fa4e34a-openshift-default-565ccd99d9-6chnx" Jul 06 03:07:45.913505 ip-10-0-139-206 kubenswrapper[2557]: I0706 03:07:45.913313 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/64624a00-3881-4496-a485-7dfb24a4e564-istio-envoy\") pod \"e2e-ait-3fa4e34a-openshift-default-565ccd99d9-6chnx\" (UID: \"64624a00-3881-4496-a485-7dfb24a4e564\") " pod="openshift-ingress/e2e-ait-3fa4e34a-openshift-default-565ccd99d9-6chnx" Jul 06 03:07:45.913505 ip-10-0-139-206 kubenswrapper[2557]: I0706 03:07:45.913337 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/64624a00-3881-4496-a485-7dfb24a4e564-istiod-ca-cert\") pod \"e2e-ait-3fa4e34a-openshift-default-565ccd99d9-6chnx\" (UID: \"64624a00-3881-4496-a485-7dfb24a4e564\") " pod="openshift-ingress/e2e-ait-3fa4e34a-openshift-default-565ccd99d9-6chnx" Jul 06 03:07:45.913505 ip-10-0-139-206 kubenswrapper[2557]: I0706 03:07:45.913378 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/64624a00-3881-4496-a485-7dfb24a4e564-workload-certs\") pod \"e2e-ait-3fa4e34a-openshift-default-565ccd99d9-6chnx\" (UID: \"64624a00-3881-4496-a485-7dfb24a4e564\") " pod="openshift-ingress/e2e-ait-3fa4e34a-openshift-default-565ccd99d9-6chnx" Jul 06 03:07:45.913505 ip-10-0-139-206 kubenswrapper[2557]: I0706 03:07:45.913419 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/64624a00-3881-4496-a485-7dfb24a4e564-istio-data\") pod \"e2e-ait-3fa4e34a-openshift-default-565ccd99d9-6chnx\" (UID: \"64624a00-3881-4496-a485-7dfb24a4e564\") " pod="openshift-ingress/e2e-ait-3fa4e34a-openshift-default-565ccd99d9-6chnx" Jul 06 03:07:45.913505 ip-10-0-139-206 kubenswrapper[2557]: I0706 03:07:45.913496 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-62f6b\" (UniqueName: \"kubernetes.io/projected/64624a00-3881-4496-a485-7dfb24a4e564-kube-api-access-62f6b\") pod \"e2e-ait-3fa4e34a-openshift-default-565ccd99d9-6chnx\" (UID: \"64624a00-3881-4496-a485-7dfb24a4e564\") " pod="openshift-ingress/e2e-ait-3fa4e34a-openshift-default-565ccd99d9-6chnx" Jul 06 03:07:46.014826 ip-10-0-139-206 kubenswrapper[2557]: I0706 03:07:46.014792 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/64624a00-3881-4496-a485-7dfb24a4e564-istiod-ca-cert\") pod \"e2e-ait-3fa4e34a-openshift-default-565ccd99d9-6chnx\" (UID: \"64624a00-3881-4496-a485-7dfb24a4e564\") " pod="openshift-ingress/e2e-ait-3fa4e34a-openshift-default-565ccd99d9-6chnx" Jul 06 03:07:46.014826 ip-10-0-139-206 kubenswrapper[2557]: I0706 03:07:46.014829 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/64624a00-3881-4496-a485-7dfb24a4e564-workload-certs\") pod \"e2e-ait-3fa4e34a-openshift-default-565ccd99d9-6chnx\" (UID: \"64624a00-3881-4496-a485-7dfb24a4e564\") " pod="openshift-ingress/e2e-ait-3fa4e34a-openshift-default-565ccd99d9-6chnx" Jul 06 03:07:46.015074 ip-10-0-139-206 kubenswrapper[2557]: I0706 03:07:46.014847 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/64624a00-3881-4496-a485-7dfb24a4e564-istio-data\") pod \"e2e-ait-3fa4e34a-openshift-default-565ccd99d9-6chnx\" (UID: \"64624a00-3881-4496-a485-7dfb24a4e564\") " pod="openshift-ingress/e2e-ait-3fa4e34a-openshift-default-565ccd99d9-6chnx" Jul 06 03:07:46.015074 ip-10-0-139-206 kubenswrapper[2557]: I0706 03:07:46.014968 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-62f6b\" (UniqueName: \"kubernetes.io/projected/64624a00-3881-4496-a485-7dfb24a4e564-kube-api-access-62f6b\") pod \"e2e-ait-3fa4e34a-openshift-default-565ccd99d9-6chnx\" (UID: \"64624a00-3881-4496-a485-7dfb24a4e564\") " pod="openshift-ingress/e2e-ait-3fa4e34a-openshift-default-565ccd99d9-6chnx" Jul 06 03:07:46.015074 ip-10-0-139-206 kubenswrapper[2557]: I0706 03:07:46.015038 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/64624a00-3881-4496-a485-7dfb24a4e564-istio-podinfo\") pod \"e2e-ait-3fa4e34a-openshift-default-565ccd99d9-6chnx\" (UID: \"64624a00-3881-4496-a485-7dfb24a4e564\") " pod="openshift-ingress/e2e-ait-3fa4e34a-openshift-default-565ccd99d9-6chnx" Jul 06 03:07:46.015074 ip-10-0-139-206 kubenswrapper[2557]: I0706 03:07:46.015063 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/64624a00-3881-4496-a485-7dfb24a4e564-credential-socket\") pod \"e2e-ait-3fa4e34a-openshift-default-565ccd99d9-6chnx\" (UID: \"64624a00-3881-4496-a485-7dfb24a4e564\") " pod="openshift-ingress/e2e-ait-3fa4e34a-openshift-default-565ccd99d9-6chnx" Jul 06 03:07:46.015281 ip-10-0-139-206 kubenswrapper[2557]: I0706 03:07:46.015115 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/64624a00-3881-4496-a485-7dfb24a4e564-istio-token\") pod \"e2e-ait-3fa4e34a-openshift-default-565ccd99d9-6chnx\" (UID: \"64624a00-3881-4496-a485-7dfb24a4e564\") " pod="openshift-ingress/e2e-ait-3fa4e34a-openshift-default-565ccd99d9-6chnx" Jul 06 03:07:46.015281 ip-10-0-139-206 kubenswrapper[2557]: I0706 03:07:46.015157 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/64624a00-3881-4496-a485-7dfb24a4e564-workload-socket\") pod \"e2e-ait-3fa4e34a-openshift-default-565ccd99d9-6chnx\" (UID: \"64624a00-3881-4496-a485-7dfb24a4e564\") " pod="openshift-ingress/e2e-ait-3fa4e34a-openshift-default-565ccd99d9-6chnx" Jul 06 03:07:46.015281 ip-10-0-139-206 kubenswrapper[2557]: I0706 03:07:46.015180 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/64624a00-3881-4496-a485-7dfb24a4e564-istio-envoy\") pod \"e2e-ait-3fa4e34a-openshift-default-565ccd99d9-6chnx\" (UID: \"64624a00-3881-4496-a485-7dfb24a4e564\") " pod="openshift-ingress/e2e-ait-3fa4e34a-openshift-default-565ccd99d9-6chnx" Jul 06 03:07:46.015281 ip-10-0-139-206 kubenswrapper[2557]: I0706 03:07:46.015198 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/64624a00-3881-4496-a485-7dfb24a4e564-istio-data\") pod \"e2e-ait-3fa4e34a-openshift-default-565ccd99d9-6chnx\" (UID: \"64624a00-3881-4496-a485-7dfb24a4e564\") " pod="openshift-ingress/e2e-ait-3fa4e34a-openshift-default-565ccd99d9-6chnx" Jul 06 03:07:46.015281 ip-10-0-139-206 kubenswrapper[2557]: I0706 03:07:46.015230 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/64624a00-3881-4496-a485-7dfb24a4e564-workload-certs\") pod \"e2e-ait-3fa4e34a-openshift-default-565ccd99d9-6chnx\" (UID: \"64624a00-3881-4496-a485-7dfb24a4e564\") " pod="openshift-ingress/e2e-ait-3fa4e34a-openshift-default-565ccd99d9-6chnx" Jul 06 03:07:46.015571 ip-10-0-139-206 kubenswrapper[2557]: I0706 03:07:46.015526 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/64624a00-3881-4496-a485-7dfb24a4e564-workload-socket\") pod \"e2e-ait-3fa4e34a-openshift-default-565ccd99d9-6chnx\" (UID: \"64624a00-3881-4496-a485-7dfb24a4e564\") " pod="openshift-ingress/e2e-ait-3fa4e34a-openshift-default-565ccd99d9-6chnx" Jul 06 03:07:46.015648 ip-10-0-139-206 kubenswrapper[2557]: I0706 03:07:46.015629 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/64624a00-3881-4496-a485-7dfb24a4e564-istiod-ca-cert\") pod \"e2e-ait-3fa4e34a-openshift-default-565ccd99d9-6chnx\" (UID: \"64624a00-3881-4496-a485-7dfb24a4e564\") " pod="openshift-ingress/e2e-ait-3fa4e34a-openshift-default-565ccd99d9-6chnx" Jul 06 03:07:46.015703 ip-10-0-139-206 kubenswrapper[2557]: I0706 03:07:46.015645 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/64624a00-3881-4496-a485-7dfb24a4e564-credential-socket\") pod \"e2e-ait-3fa4e34a-openshift-default-565ccd99d9-6chnx\" (UID: \"64624a00-3881-4496-a485-7dfb24a4e564\") " pod="openshift-ingress/e2e-ait-3fa4e34a-openshift-default-565ccd99d9-6chnx" Jul 06 03:07:46.017459 ip-10-0-139-206 kubenswrapper[2557]: I0706 03:07:46.017437 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/64624a00-3881-4496-a485-7dfb24a4e564-istio-envoy\") pod \"e2e-ait-3fa4e34a-openshift-default-565ccd99d9-6chnx\" (UID: \"64624a00-3881-4496-a485-7dfb24a4e564\") " pod="openshift-ingress/e2e-ait-3fa4e34a-openshift-default-565ccd99d9-6chnx" Jul 06 03:07:46.017716 ip-10-0-139-206 kubenswrapper[2557]: I0706 03:07:46.017677 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/64624a00-3881-4496-a485-7dfb24a4e564-istio-podinfo\") pod \"e2e-ait-3fa4e34a-openshift-default-565ccd99d9-6chnx\" (UID: \"64624a00-3881-4496-a485-7dfb24a4e564\") " pod="openshift-ingress/e2e-ait-3fa4e34a-openshift-default-565ccd99d9-6chnx" Jul 06 03:07:46.022764 ip-10-0-139-206 kubenswrapper[2557]: I0706 03:07:46.022736 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-62f6b\" (UniqueName: \"kubernetes.io/projected/64624a00-3881-4496-a485-7dfb24a4e564-kube-api-access-62f6b\") pod \"e2e-ait-3fa4e34a-openshift-default-565ccd99d9-6chnx\" (UID: \"64624a00-3881-4496-a485-7dfb24a4e564\") " pod="openshift-ingress/e2e-ait-3fa4e34a-openshift-default-565ccd99d9-6chnx" Jul 06 03:07:46.023410 ip-10-0-139-206 kubenswrapper[2557]: I0706 03:07:46.023389 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/64624a00-3881-4496-a485-7dfb24a4e564-istio-token\") pod \"e2e-ait-3fa4e34a-openshift-default-565ccd99d9-6chnx\" (UID: \"64624a00-3881-4496-a485-7dfb24a4e564\") " pod="openshift-ingress/e2e-ait-3fa4e34a-openshift-default-565ccd99d9-6chnx" Jul 06 03:07:46.094657 ip-10-0-139-206 kubenswrapper[2557]: I0706 03:07:46.094574 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-3fa4e34a-openshift-default-565ccd99d9-6chnx" Jul 06 03:07:46.231200 ip-10-0-139-206 kubenswrapper[2557]: I0706 03:07:46.231176 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-3fa4e34a-openshift-default-565ccd99d9-6chnx"] Jul 06 03:07:46.233215 ip-10-0-139-206 kubenswrapper[2557]: W0706 03:07:46.233190 2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod64624a00_3881_4496_a485_7dfb24a4e564.slice/crio-928d2ad185e35ceb9442323831aa6a865f9f79cf7b9b71b99dca9a088a023981 WatchSource:0}: Error finding container 928d2ad185e35ceb9442323831aa6a865f9f79cf7b9b71b99dca9a088a023981: Status 404 returned error can't find the container with id 928d2ad185e35ceb9442323831aa6a865f9f79cf7b9b71b99dca9a088a023981 Jul 06 03:07:46.235337 ip-10-0-139-206 kubenswrapper[2557]: I0706 03:07:46.235321 2557 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 06 03:07:46.505748 ip-10-0-139-206 kubenswrapper[2557]: I0706 03:07:46.505713 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-3fa4e34a-openshift-default-565ccd99d9-6chnx" event={"ID":"64624a00-3881-4496-a485-7dfb24a4e564","Type":"ContainerStarted","Data":"928d2ad185e35ceb9442323831aa6a865f9f79cf7b9b71b99dca9a088a023981"} Jul 06 03:07:48.363320 ip-10-0-139-206 kubenswrapper[2557]: I0706 03:07:48.363284 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-srzrr"] Jul 06 03:07:58.510298 ip-10-0-139-206 kubenswrapper[2557]: I0706 03:07:58.510260 2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-3fa4e34a-openshift-default-565ccd99d9-6chnx"] Jul 06 03:08:01.909463 ip-10-0-139-206 kubenswrapper[2557]: I0706 03:08:01.909417 2557 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 06 03:08:01.909714 ip-10-0-139-206 kubenswrapper[2557]: I0706 03:08:01.909514 2557 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 06 03:08:01.909714 ip-10-0-139-206 kubenswrapper[2557]: I0706 03:08:01.909556 2557 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 06 03:08:02.583029 ip-10-0-139-206 kubenswrapper[2557]: I0706 03:08:02.582986 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-3fa4e34a-openshift-default-565ccd99d9-6chnx" event={"ID":"64624a00-3881-4496-a485-7dfb24a4e564","Type":"ContainerStarted","Data":"cbec3ad7ac85f9cf4e8eec79988dc4bbc70bf0f77ec97fe5f772a050675764ce"} Jul 06 03:08:02.583238 ip-10-0-139-206 kubenswrapper[2557]: I0706 03:08:02.583043 2557 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-ingress/e2e-ait-3fa4e34a-openshift-default-565ccd99d9-6chnx" podUID="64624a00-3881-4496-a485-7dfb24a4e564" containerName="istio-proxy" containerID="cri-o://cbec3ad7ac85f9cf4e8eec79988dc4bbc70bf0f77ec97fe5f772a050675764ce" gracePeriod=30 Jul 06 03:08:02.603626 ip-10-0-139-206 kubenswrapper[2557]: I0706 03:08:02.603573 2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-ait-3fa4e34a-openshift-default-565ccd99d9-6chnx" podStartSLOduration=1.929989987 podStartE2EDuration="17.603556857s" totalImagesPullingTime="15.67356687s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 03:07:45 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 03:08:02.602364138 +0000 UTC m=+1726.408650205" watchObservedRunningTime="2026-07-06 03:08:02.603556857 +0000 UTC m=+1726.409842917" Jul 06 03:08:07.840647 ip-10-0-139-206 kubenswrapper[2557]: I0706 03:08:07.840580 2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-3fa4e34a-openshift-default-565ccd99d9-6chnx" Jul 06 03:08:07.922208 ip-10-0-139-206 kubenswrapper[2557]: I0706 03:08:07.922168 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/64624a00-3881-4496-a485-7dfb24a4e564-istiod-ca-cert\") pod \"64624a00-3881-4496-a485-7dfb24a4e564\" (UID: \"64624a00-3881-4496-a485-7dfb24a4e564\") " Jul 06 03:08:07.922410 ip-10-0-139-206 kubenswrapper[2557]: I0706 03:08:07.922229 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/64624a00-3881-4496-a485-7dfb24a4e564-istio-envoy\") pod \"64624a00-3881-4496-a485-7dfb24a4e564\" (UID: \"64624a00-3881-4496-a485-7dfb24a4e564\") " Jul 06 03:08:07.922410 ip-10-0-139-206 kubenswrapper[2557]: I0706 03:08:07.922247 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/64624a00-3881-4496-a485-7dfb24a4e564-istio-data\") pod \"64624a00-3881-4496-a485-7dfb24a4e564\" (UID: \"64624a00-3881-4496-a485-7dfb24a4e564\") " Jul 06 03:08:07.922503 ip-10-0-139-206 kubenswrapper[2557]: I0706 03:08:07.922408 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/64624a00-3881-4496-a485-7dfb24a4e564-istio-podinfo\") pod \"64624a00-3881-4496-a485-7dfb24a4e564\" (UID: \"64624a00-3881-4496-a485-7dfb24a4e564\") " Jul 06 03:08:07.922503 ip-10-0-139-206 kubenswrapper[2557]: I0706 03:08:07.922466 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-62f6b\" (UniqueName: \"kubernetes.io/projected/64624a00-3881-4496-a485-7dfb24a4e564-kube-api-access-62f6b\") pod \"64624a00-3881-4496-a485-7dfb24a4e564\" (UID: \"64624a00-3881-4496-a485-7dfb24a4e564\") " Jul 06 03:08:07.922590 ip-10-0-139-206 kubenswrapper[2557]: I0706 03:08:07.922501 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/64624a00-3881-4496-a485-7dfb24a4e564-workload-socket\") pod \"64624a00-3881-4496-a485-7dfb24a4e564\" (UID: \"64624a00-3881-4496-a485-7dfb24a4e564\") " Jul 06 03:08:07.922590 ip-10-0-139-206 kubenswrapper[2557]: I0706 03:08:07.922561 2557 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/64624a00-3881-4496-a485-7dfb24a4e564-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "64624a00-3881-4496-a485-7dfb24a4e564" (UID: "64624a00-3881-4496-a485-7dfb24a4e564"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 06 03:08:07.922698 ip-10-0-139-206 kubenswrapper[2557]: I0706 03:08:07.922563 2557 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/64624a00-3881-4496-a485-7dfb24a4e564-istio-data" (OuterVolumeSpecName: "istio-data") pod "64624a00-3881-4496-a485-7dfb24a4e564" (UID: "64624a00-3881-4496-a485-7dfb24a4e564"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 03:08:07.922698 ip-10-0-139-206 kubenswrapper[2557]: I0706 03:08:07.922569 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/64624a00-3881-4496-a485-7dfb24a4e564-credential-socket\") pod \"64624a00-3881-4496-a485-7dfb24a4e564\" (UID: \"64624a00-3881-4496-a485-7dfb24a4e564\") " Jul 06 03:08:07.922698 ip-10-0-139-206 kubenswrapper[2557]: I0706 03:08:07.922647 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/64624a00-3881-4496-a485-7dfb24a4e564-istio-token\") pod \"64624a00-3881-4496-a485-7dfb24a4e564\" (UID: \"64624a00-3881-4496-a485-7dfb24a4e564\") " Jul 06 03:08:07.922698 ip-10-0-139-206 kubenswrapper[2557]: I0706 03:08:07.922684 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/64624a00-3881-4496-a485-7dfb24a4e564-workload-certs\") pod \"64624a00-3881-4496-a485-7dfb24a4e564\" (UID: \"64624a00-3881-4496-a485-7dfb24a4e564\") " Jul 06 03:08:07.922917 ip-10-0-139-206 kubenswrapper[2557]: I0706 03:08:07.922761 2557 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/64624a00-3881-4496-a485-7dfb24a4e564-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "64624a00-3881-4496-a485-7dfb24a4e564" (UID: "64624a00-3881-4496-a485-7dfb24a4e564"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 03:08:07.922917 ip-10-0-139-206 kubenswrapper[2557]: I0706 03:08:07.922784 2557 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/64624a00-3881-4496-a485-7dfb24a4e564-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "64624a00-3881-4496-a485-7dfb24a4e564" (UID: "64624a00-3881-4496-a485-7dfb24a4e564"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 03:08:07.923030 ip-10-0-139-206 kubenswrapper[2557]: I0706 03:08:07.923004 2557 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/64624a00-3881-4496-a485-7dfb24a4e564-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "64624a00-3881-4496-a485-7dfb24a4e564" (UID: "64624a00-3881-4496-a485-7dfb24a4e564"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 03:08:07.923030 ip-10-0-139-206 kubenswrapper[2557]: I0706 03:08:07.923012 2557 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/64624a00-3881-4496-a485-7dfb24a4e564-credential-socket\") on node \"ip-10-0-139-206.ec2.internal\" DevicePath \"\"" Jul 06 03:08:07.923133 ip-10-0-139-206 kubenswrapper[2557]: I0706 03:08:07.923032 2557 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/64624a00-3881-4496-a485-7dfb24a4e564-istiod-ca-cert\") on node \"ip-10-0-139-206.ec2.internal\" DevicePath \"\"" Jul 06 03:08:07.923133 ip-10-0-139-206 kubenswrapper[2557]: I0706 03:08:07.923046 2557 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/64624a00-3881-4496-a485-7dfb24a4e564-istio-data\") on node \"ip-10-0-139-206.ec2.internal\" DevicePath \"\"" Jul 06 03:08:07.923133 ip-10-0-139-206 kubenswrapper[2557]: I0706 03:08:07.923058 2557 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/64624a00-3881-4496-a485-7dfb24a4e564-workload-socket\") on node \"ip-10-0-139-206.ec2.internal\" DevicePath \"\"" Jul 06 03:08:07.924651 ip-10-0-139-206 kubenswrapper[2557]: I0706 03:08:07.924619 2557 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/64624a00-3881-4496-a485-7dfb24a4e564-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "64624a00-3881-4496-a485-7dfb24a4e564" (UID: "64624a00-3881-4496-a485-7dfb24a4e564"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 03:08:07.924771 ip-10-0-139-206 kubenswrapper[2557]: I0706 03:08:07.924647 2557 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/64624a00-3881-4496-a485-7dfb24a4e564-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "64624a00-3881-4496-a485-7dfb24a4e564" (UID: "64624a00-3881-4496-a485-7dfb24a4e564"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 06 03:08:07.924771 ip-10-0-139-206 kubenswrapper[2557]: I0706 03:08:07.924702 2557 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/64624a00-3881-4496-a485-7dfb24a4e564-istio-token" (OuterVolumeSpecName: "istio-token") pod "64624a00-3881-4496-a485-7dfb24a4e564" (UID: "64624a00-3881-4496-a485-7dfb24a4e564"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 06 03:08:07.924771 ip-10-0-139-206 kubenswrapper[2557]: I0706 03:08:07.924711 2557 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/64624a00-3881-4496-a485-7dfb24a4e564-kube-api-access-62f6b" (OuterVolumeSpecName: "kube-api-access-62f6b") pod "64624a00-3881-4496-a485-7dfb24a4e564" (UID: "64624a00-3881-4496-a485-7dfb24a4e564"). InnerVolumeSpecName "kube-api-access-62f6b". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 06 03:08:08.023816 ip-10-0-139-206 kubenswrapper[2557]: I0706 03:08:08.023778 2557 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/64624a00-3881-4496-a485-7dfb24a4e564-istio-envoy\") on node \"ip-10-0-139-206.ec2.internal\" DevicePath \"\"" Jul 06 03:08:08.023816 ip-10-0-139-206 kubenswrapper[2557]: I0706 03:08:08.023809 2557 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/64624a00-3881-4496-a485-7dfb24a4e564-istio-podinfo\") on node \"ip-10-0-139-206.ec2.internal\" DevicePath \"\"" Jul 06 03:08:08.023816 ip-10-0-139-206 kubenswrapper[2557]: I0706 03:08:08.023818 2557 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-62f6b\" (UniqueName: \"kubernetes.io/projected/64624a00-3881-4496-a485-7dfb24a4e564-kube-api-access-62f6b\") on node \"ip-10-0-139-206.ec2.internal\" DevicePath \"\"" Jul 06 03:08:08.023816 ip-10-0-139-206 kubenswrapper[2557]: I0706 03:08:08.023828 2557 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/64624a00-3881-4496-a485-7dfb24a4e564-istio-token\") on node \"ip-10-0-139-206.ec2.internal\" DevicePath \"\"" Jul 06 03:08:08.024082 ip-10-0-139-206 kubenswrapper[2557]: I0706 03:08:08.023836 2557 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/64624a00-3881-4496-a485-7dfb24a4e564-workload-certs\") on node \"ip-10-0-139-206.ec2.internal\" DevicePath \"\"" Jul 06 03:08:08.272801 ip-10-0-139-206 kubenswrapper[2557]: I0706 03:08:08.272767 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-srzrr"] Jul 06 03:08:08.613302 ip-10-0-139-206 kubenswrapper[2557]: I0706 03:08:08.613207 2557 generic.go:356] "Generic (PLEG): container finished" podID="64624a00-3881-4496-a485-7dfb24a4e564" containerID="cbec3ad7ac85f9cf4e8eec79988dc4bbc70bf0f77ec97fe5f772a050675764ce" exitCode=0 Jul 06 03:08:08.613302 ip-10-0-139-206 kubenswrapper[2557]: I0706 03:08:08.613248 2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-3fa4e34a-openshift-default-565ccd99d9-6chnx" Jul 06 03:08:08.613302 ip-10-0-139-206 kubenswrapper[2557]: I0706 03:08:08.613290 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-3fa4e34a-openshift-default-565ccd99d9-6chnx" event={"ID":"64624a00-3881-4496-a485-7dfb24a4e564","Type":"ContainerDied","Data":"cbec3ad7ac85f9cf4e8eec79988dc4bbc70bf0f77ec97fe5f772a050675764ce"} Jul 06 03:08:08.613584 ip-10-0-139-206 kubenswrapper[2557]: I0706 03:08:08.613363 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-3fa4e34a-openshift-default-565ccd99d9-6chnx" event={"ID":"64624a00-3881-4496-a485-7dfb24a4e564","Type":"ContainerDied","Data":"928d2ad185e35ceb9442323831aa6a865f9f79cf7b9b71b99dca9a088a023981"} Jul 06 03:08:08.613584 ip-10-0-139-206 kubenswrapper[2557]: I0706 03:08:08.613407 2557 scope.go:117] "RemoveContainer" containerID="cbec3ad7ac85f9cf4e8eec79988dc4bbc70bf0f77ec97fe5f772a050675764ce" Jul 06 03:08:08.622890 ip-10-0-139-206 kubenswrapper[2557]: I0706 03:08:08.622873 2557 scope.go:117] "RemoveContainer" containerID="cbec3ad7ac85f9cf4e8eec79988dc4bbc70bf0f77ec97fe5f772a050675764ce" Jul 06 03:08:08.623188 ip-10-0-139-206 kubenswrapper[2557]: E0706 03:08:08.623172 2557 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"cbec3ad7ac85f9cf4e8eec79988dc4bbc70bf0f77ec97fe5f772a050675764ce\": container with ID starting with cbec3ad7ac85f9cf4e8eec79988dc4bbc70bf0f77ec97fe5f772a050675764ce not found: ID does not exist" containerID="cbec3ad7ac85f9cf4e8eec79988dc4bbc70bf0f77ec97fe5f772a050675764ce" Jul 06 03:08:08.623232 ip-10-0-139-206 kubenswrapper[2557]: I0706 03:08:08.623197 2557 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"cbec3ad7ac85f9cf4e8eec79988dc4bbc70bf0f77ec97fe5f772a050675764ce"} err="failed to get container status \"cbec3ad7ac85f9cf4e8eec79988dc4bbc70bf0f77ec97fe5f772a050675764ce\": rpc error: code = NotFound desc = could not find container \"cbec3ad7ac85f9cf4e8eec79988dc4bbc70bf0f77ec97fe5f772a050675764ce\": container with ID starting with cbec3ad7ac85f9cf4e8eec79988dc4bbc70bf0f77ec97fe5f772a050675764ce not found: ID does not exist" Jul 06 03:08:08.634764 ip-10-0-139-206 kubenswrapper[2557]: I0706 03:08:08.634727 2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-3fa4e34a-openshift-default-565ccd99d9-6chnx"] Jul 06 03:08:08.640855 ip-10-0-139-206 kubenswrapper[2557]: I0706 03:08:08.640826 2557 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-ait-3fa4e34a-openshift-default-565ccd99d9-6chnx"] Jul 06 03:08:09.302358 ip-10-0-139-206 kubenswrapper[2557]: I0706 03:08:09.302323 2557 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="64624a00-3881-4496-a485-7dfb24a4e564" path="/var/lib/kubelet/pods/64624a00-3881-4496-a485-7dfb24a4e564/volumes" Jul 06 03:08:16.773803 ip-10-0-139-206 kubenswrapper[2557]: I0706 03:08:16.773731 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-srzrr"] Jul 06 03:09:17.306004 ip-10-0-139-206 kubenswrapper[2557]: I0706 03:09:17.305975 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-cw2f7_efcf5e11-1e5d-4e2a-9b38-303a8f0787c4/ovn-acl-logging/0.log" Jul 06 03:09:17.308384 ip-10-0-139-206 kubenswrapper[2557]: I0706 03:09:17.308360 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-cw2f7_efcf5e11-1e5d-4e2a-9b38-303a8f0787c4/ovn-acl-logging/0.log" Jul 06 03:09:38.072185 ip-10-0-139-206 kubenswrapper[2557]: I0706 03:09:38.072148 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-srzrr"] Jul 06 03:09:46.966953 ip-10-0-139-206 kubenswrapper[2557]: I0706 03:09:46.966913 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-srzrr"] Jul 06 03:09:51.029983 ip-10-0-139-206 kubenswrapper[2557]: I0706 03:09:51.029945 2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-mt-ebf514bc-openshift-default-599b766dc5-ndd7k"] Jul 06 03:09:51.030433 ip-10-0-139-206 kubenswrapper[2557]: I0706 03:09:51.030264 2557 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="64624a00-3881-4496-a485-7dfb24a4e564" containerName="istio-proxy" Jul 06 03:09:51.030433 ip-10-0-139-206 kubenswrapper[2557]: I0706 03:09:51.030274 2557 state_mem.go:107] "Deleted CPUSet assignment" podUID="64624a00-3881-4496-a485-7dfb24a4e564" containerName="istio-proxy" Jul 06 03:09:51.030433 ip-10-0-139-206 kubenswrapper[2557]: I0706 03:09:51.030327 2557 memory_manager.go:368] "RemoveStaleState removing state" podUID="64624a00-3881-4496-a485-7dfb24a4e564" containerName="istio-proxy" Jul 06 03:09:51.033364 ip-10-0-139-206 kubenswrapper[2557]: I0706 03:09:51.033333 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-ebf514bc-openshift-default-599b766dc5-ndd7k" Jul 06 03:09:51.035855 ip-10-0-139-206 kubenswrapper[2557]: I0706 03:09:51.035834 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-mt-ebf514bc-openshift-default-dockercfg-fxzmn\"" Jul 06 03:09:51.059618 ip-10-0-139-206 kubenswrapper[2557]: I0706 03:09:51.059587 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-ebf514bc-openshift-default-599b766dc5-ndd7k"] Jul 06 03:09:51.116184 ip-10-0-139-206 kubenswrapper[2557]: I0706 03:09:51.116151 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/fdb03c1e-1cd2-4749-9a7d-84ceda278f48-workload-socket\") pod \"e2e-mt-ebf514bc-openshift-default-599b766dc5-ndd7k\" (UID: \"fdb03c1e-1cd2-4749-9a7d-84ceda278f48\") " pod="openshift-ingress/e2e-mt-ebf514bc-openshift-default-599b766dc5-ndd7k" Jul 06 03:09:51.116184 ip-10-0-139-206 kubenswrapper[2557]: I0706 03:09:51.116184 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/fdb03c1e-1cd2-4749-9a7d-84ceda278f48-istiod-ca-cert\") pod \"e2e-mt-ebf514bc-openshift-default-599b766dc5-ndd7k\" (UID: \"fdb03c1e-1cd2-4749-9a7d-84ceda278f48\") " pod="openshift-ingress/e2e-mt-ebf514bc-openshift-default-599b766dc5-ndd7k" Jul 06 03:09:51.116419 ip-10-0-139-206 kubenswrapper[2557]: I0706 03:09:51.116205 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/fdb03c1e-1cd2-4749-9a7d-84ceda278f48-credential-socket\") pod \"e2e-mt-ebf514bc-openshift-default-599b766dc5-ndd7k\" (UID: \"fdb03c1e-1cd2-4749-9a7d-84ceda278f48\") " pod="openshift-ingress/e2e-mt-ebf514bc-openshift-default-599b766dc5-ndd7k" Jul 06 03:09:51.116419 ip-10-0-139-206 kubenswrapper[2557]: I0706 03:09:51.116253 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/fdb03c1e-1cd2-4749-9a7d-84ceda278f48-istio-podinfo\") pod \"e2e-mt-ebf514bc-openshift-default-599b766dc5-ndd7k\" (UID: \"fdb03c1e-1cd2-4749-9a7d-84ceda278f48\") " pod="openshift-ingress/e2e-mt-ebf514bc-openshift-default-599b766dc5-ndd7k" Jul 06 03:09:51.116419 ip-10-0-139-206 kubenswrapper[2557]: I0706 03:09:51.116298 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/fdb03c1e-1cd2-4749-9a7d-84ceda278f48-istio-envoy\") pod \"e2e-mt-ebf514bc-openshift-default-599b766dc5-ndd7k\" (UID: \"fdb03c1e-1cd2-4749-9a7d-84ceda278f48\") " pod="openshift-ingress/e2e-mt-ebf514bc-openshift-default-599b766dc5-ndd7k" Jul 06 03:09:51.116419 ip-10-0-139-206 kubenswrapper[2557]: I0706 03:09:51.116373 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/fdb03c1e-1cd2-4749-9a7d-84ceda278f48-workload-certs\") pod \"e2e-mt-ebf514bc-openshift-default-599b766dc5-ndd7k\" (UID: \"fdb03c1e-1cd2-4749-9a7d-84ceda278f48\") " pod="openshift-ingress/e2e-mt-ebf514bc-openshift-default-599b766dc5-ndd7k" Jul 06 03:09:51.116566 ip-10-0-139-206 kubenswrapper[2557]: I0706 03:09:51.116458 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/fdb03c1e-1cd2-4749-9a7d-84ceda278f48-istio-token\") pod \"e2e-mt-ebf514bc-openshift-default-599b766dc5-ndd7k\" (UID: \"fdb03c1e-1cd2-4749-9a7d-84ceda278f48\") " pod="openshift-ingress/e2e-mt-ebf514bc-openshift-default-599b766dc5-ndd7k" Jul 06 03:09:51.116566 ip-10-0-139-206 kubenswrapper[2557]: I0706 03:09:51.116480 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mwwp7\" (UniqueName: \"kubernetes.io/projected/fdb03c1e-1cd2-4749-9a7d-84ceda278f48-kube-api-access-mwwp7\") pod \"e2e-mt-ebf514bc-openshift-default-599b766dc5-ndd7k\" (UID: \"fdb03c1e-1cd2-4749-9a7d-84ceda278f48\") " pod="openshift-ingress/e2e-mt-ebf514bc-openshift-default-599b766dc5-ndd7k" Jul 06 03:09:51.116566 ip-10-0-139-206 kubenswrapper[2557]: I0706 03:09:51.116507 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/fdb03c1e-1cd2-4749-9a7d-84ceda278f48-istio-data\") pod \"e2e-mt-ebf514bc-openshift-default-599b766dc5-ndd7k\" (UID: \"fdb03c1e-1cd2-4749-9a7d-84ceda278f48\") " pod="openshift-ingress/e2e-mt-ebf514bc-openshift-default-599b766dc5-ndd7k" Jul 06 03:09:51.217975 ip-10-0-139-206 kubenswrapper[2557]: I0706 03:09:51.217937 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/fdb03c1e-1cd2-4749-9a7d-84ceda278f48-workload-socket\") pod \"e2e-mt-ebf514bc-openshift-default-599b766dc5-ndd7k\" (UID: \"fdb03c1e-1cd2-4749-9a7d-84ceda278f48\") " pod="openshift-ingress/e2e-mt-ebf514bc-openshift-default-599b766dc5-ndd7k" Jul 06 03:09:51.217975 ip-10-0-139-206 kubenswrapper[2557]: I0706 03:09:51.217975 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/fdb03c1e-1cd2-4749-9a7d-84ceda278f48-istiod-ca-cert\") pod \"e2e-mt-ebf514bc-openshift-default-599b766dc5-ndd7k\" (UID: \"fdb03c1e-1cd2-4749-9a7d-84ceda278f48\") " pod="openshift-ingress/e2e-mt-ebf514bc-openshift-default-599b766dc5-ndd7k" Jul 06 03:09:51.218193 ip-10-0-139-206 kubenswrapper[2557]: I0706 03:09:51.217994 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/fdb03c1e-1cd2-4749-9a7d-84ceda278f48-credential-socket\") pod \"e2e-mt-ebf514bc-openshift-default-599b766dc5-ndd7k\" (UID: \"fdb03c1e-1cd2-4749-9a7d-84ceda278f48\") " pod="openshift-ingress/e2e-mt-ebf514bc-openshift-default-599b766dc5-ndd7k" Jul 06 03:09:51.218193 ip-10-0-139-206 kubenswrapper[2557]: I0706 03:09:51.218111 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/fdb03c1e-1cd2-4749-9a7d-84ceda278f48-istio-podinfo\") pod \"e2e-mt-ebf514bc-openshift-default-599b766dc5-ndd7k\" (UID: \"fdb03c1e-1cd2-4749-9a7d-84ceda278f48\") " pod="openshift-ingress/e2e-mt-ebf514bc-openshift-default-599b766dc5-ndd7k" Jul 06 03:09:51.218193 ip-10-0-139-206 kubenswrapper[2557]: I0706 03:09:51.218161 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/fdb03c1e-1cd2-4749-9a7d-84ceda278f48-istio-envoy\") pod \"e2e-mt-ebf514bc-openshift-default-599b766dc5-ndd7k\" (UID: \"fdb03c1e-1cd2-4749-9a7d-84ceda278f48\") " pod="openshift-ingress/e2e-mt-ebf514bc-openshift-default-599b766dc5-ndd7k" Jul 06 03:09:51.218193 ip-10-0-139-206 kubenswrapper[2557]: I0706 03:09:51.218190 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/fdb03c1e-1cd2-4749-9a7d-84ceda278f48-workload-certs\") pod \"e2e-mt-ebf514bc-openshift-default-599b766dc5-ndd7k\" (UID: \"fdb03c1e-1cd2-4749-9a7d-84ceda278f48\") " pod="openshift-ingress/e2e-mt-ebf514bc-openshift-default-599b766dc5-ndd7k" Jul 06 03:09:51.218412 ip-10-0-139-206 kubenswrapper[2557]: I0706 03:09:51.218265 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/fdb03c1e-1cd2-4749-9a7d-84ceda278f48-istio-token\") pod \"e2e-mt-ebf514bc-openshift-default-599b766dc5-ndd7k\" (UID: \"fdb03c1e-1cd2-4749-9a7d-84ceda278f48\") " pod="openshift-ingress/e2e-mt-ebf514bc-openshift-default-599b766dc5-ndd7k" Jul 06 03:09:51.218479 ip-10-0-139-206 kubenswrapper[2557]: I0706 03:09:51.218454 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/fdb03c1e-1cd2-4749-9a7d-84ceda278f48-workload-socket\") pod \"e2e-mt-ebf514bc-openshift-default-599b766dc5-ndd7k\" (UID: \"fdb03c1e-1cd2-4749-9a7d-84ceda278f48\") " pod="openshift-ingress/e2e-mt-ebf514bc-openshift-default-599b766dc5-ndd7k" Jul 06 03:09:51.218539 ip-10-0-139-206 kubenswrapper[2557]: I0706 03:09:51.218506 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mwwp7\" (UniqueName: \"kubernetes.io/projected/fdb03c1e-1cd2-4749-9a7d-84ceda278f48-kube-api-access-mwwp7\") pod \"e2e-mt-ebf514bc-openshift-default-599b766dc5-ndd7k\" (UID: \"fdb03c1e-1cd2-4749-9a7d-84ceda278f48\") " pod="openshift-ingress/e2e-mt-ebf514bc-openshift-default-599b766dc5-ndd7k" Jul 06 03:09:51.218599 ip-10-0-139-206 kubenswrapper[2557]: I0706 03:09:51.218558 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/fdb03c1e-1cd2-4749-9a7d-84ceda278f48-istio-data\") pod \"e2e-mt-ebf514bc-openshift-default-599b766dc5-ndd7k\" (UID: \"fdb03c1e-1cd2-4749-9a7d-84ceda278f48\") " pod="openshift-ingress/e2e-mt-ebf514bc-openshift-default-599b766dc5-ndd7k" Jul 06 03:09:51.218662 ip-10-0-139-206 kubenswrapper[2557]: I0706 03:09:51.218506 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/fdb03c1e-1cd2-4749-9a7d-84ceda278f48-credential-socket\") pod \"e2e-mt-ebf514bc-openshift-default-599b766dc5-ndd7k\" (UID: \"fdb03c1e-1cd2-4749-9a7d-84ceda278f48\") " pod="openshift-ingress/e2e-mt-ebf514bc-openshift-default-599b766dc5-ndd7k" Jul 06 03:09:51.218783 ip-10-0-139-206 kubenswrapper[2557]: I0706 03:09:51.218739 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/fdb03c1e-1cd2-4749-9a7d-84ceda278f48-workload-certs\") pod \"e2e-mt-ebf514bc-openshift-default-599b766dc5-ndd7k\" (UID: \"fdb03c1e-1cd2-4749-9a7d-84ceda278f48\") " pod="openshift-ingress/e2e-mt-ebf514bc-openshift-default-599b766dc5-ndd7k" Jul 06 03:09:51.218984 ip-10-0-139-206 kubenswrapper[2557]: I0706 03:09:51.218813 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/fdb03c1e-1cd2-4749-9a7d-84ceda278f48-istiod-ca-cert\") pod \"e2e-mt-ebf514bc-openshift-default-599b766dc5-ndd7k\" (UID: \"fdb03c1e-1cd2-4749-9a7d-84ceda278f48\") " pod="openshift-ingress/e2e-mt-ebf514bc-openshift-default-599b766dc5-ndd7k" Jul 06 03:09:51.218984 ip-10-0-139-206 kubenswrapper[2557]: I0706 03:09:51.218877 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/fdb03c1e-1cd2-4749-9a7d-84ceda278f48-istio-data\") pod \"e2e-mt-ebf514bc-openshift-default-599b766dc5-ndd7k\" (UID: \"fdb03c1e-1cd2-4749-9a7d-84ceda278f48\") " pod="openshift-ingress/e2e-mt-ebf514bc-openshift-default-599b766dc5-ndd7k" Jul 06 03:09:51.220553 ip-10-0-139-206 kubenswrapper[2557]: I0706 03:09:51.220534 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/fdb03c1e-1cd2-4749-9a7d-84ceda278f48-istio-envoy\") pod \"e2e-mt-ebf514bc-openshift-default-599b766dc5-ndd7k\" (UID: \"fdb03c1e-1cd2-4749-9a7d-84ceda278f48\") " pod="openshift-ingress/e2e-mt-ebf514bc-openshift-default-599b766dc5-ndd7k" Jul 06 03:09:51.220622 ip-10-0-139-206 kubenswrapper[2557]: I0706 03:09:51.220609 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/fdb03c1e-1cd2-4749-9a7d-84ceda278f48-istio-podinfo\") pod \"e2e-mt-ebf514bc-openshift-default-599b766dc5-ndd7k\" (UID: \"fdb03c1e-1cd2-4749-9a7d-84ceda278f48\") " pod="openshift-ingress/e2e-mt-ebf514bc-openshift-default-599b766dc5-ndd7k" Jul 06 03:09:51.226134 ip-10-0-139-206 kubenswrapper[2557]: I0706 03:09:51.226112 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/fdb03c1e-1cd2-4749-9a7d-84ceda278f48-istio-token\") pod \"e2e-mt-ebf514bc-openshift-default-599b766dc5-ndd7k\" (UID: \"fdb03c1e-1cd2-4749-9a7d-84ceda278f48\") " pod="openshift-ingress/e2e-mt-ebf514bc-openshift-default-599b766dc5-ndd7k" Jul 06 03:09:51.226557 ip-10-0-139-206 kubenswrapper[2557]: I0706 03:09:51.226541 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-mwwp7\" (UniqueName: \"kubernetes.io/projected/fdb03c1e-1cd2-4749-9a7d-84ceda278f48-kube-api-access-mwwp7\") pod \"e2e-mt-ebf514bc-openshift-default-599b766dc5-ndd7k\" (UID: \"fdb03c1e-1cd2-4749-9a7d-84ceda278f48\") " pod="openshift-ingress/e2e-mt-ebf514bc-openshift-default-599b766dc5-ndd7k" Jul 06 03:09:51.343663 ip-10-0-139-206 kubenswrapper[2557]: I0706 03:09:51.343571 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-ebf514bc-openshift-default-599b766dc5-ndd7k" Jul 06 03:09:51.482958 ip-10-0-139-206 kubenswrapper[2557]: I0706 03:09:51.482932 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-ebf514bc-openshift-default-599b766dc5-ndd7k"] Jul 06 03:09:51.485543 ip-10-0-139-206 kubenswrapper[2557]: W0706 03:09:51.485518 2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfdb03c1e_1cd2_4749_9a7d_84ceda278f48.slice/crio-7ffc921cad389267b5378d5ef0556bbc42dc1c01ce1ef8728584fc55b50d3fe9 WatchSource:0}: Error finding container 7ffc921cad389267b5378d5ef0556bbc42dc1c01ce1ef8728584fc55b50d3fe9: Status 404 returned error can't find the container with id 7ffc921cad389267b5378d5ef0556bbc42dc1c01ce1ef8728584fc55b50d3fe9 Jul 06 03:09:51.488138 ip-10-0-139-206 kubenswrapper[2557]: I0706 03:09:51.488103 2557 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 06 03:09:51.488222 ip-10-0-139-206 kubenswrapper[2557]: I0706 03:09:51.488171 2557 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 06 03:09:51.488222 ip-10-0-139-206 kubenswrapper[2557]: I0706 03:09:51.488200 2557 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 06 03:09:52.075558 ip-10-0-139-206 kubenswrapper[2557]: I0706 03:09:52.075519 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-ebf514bc-openshift-default-599b766dc5-ndd7k" event={"ID":"fdb03c1e-1cd2-4749-9a7d-84ceda278f48","Type":"ContainerStarted","Data":"90bc3dd14fd540012c7f42e259e6c049106a253d2d4dac96795ef7b686fef8c0"} Jul 06 03:09:52.075954 ip-10-0-139-206 kubenswrapper[2557]: I0706 03:09:52.075583 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-ebf514bc-openshift-default-599b766dc5-ndd7k" event={"ID":"fdb03c1e-1cd2-4749-9a7d-84ceda278f48","Type":"ContainerStarted","Data":"7ffc921cad389267b5378d5ef0556bbc42dc1c01ce1ef8728584fc55b50d3fe9"} Jul 06 03:09:52.095988 ip-10-0-139-206 kubenswrapper[2557]: I0706 03:09:52.095933 2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-mt-ebf514bc-openshift-default-599b766dc5-ndd7k" podStartSLOduration=1.095914207 podStartE2EDuration="1.095914207s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 03:09:51 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 03:09:52.094819156 +0000 UTC m=+1835.901105215" watchObservedRunningTime="2026-07-06 03:09:52.095914207 +0000 UTC m=+1835.902200264" Jul 06 03:09:52.344497 ip-10-0-139-206 kubenswrapper[2557]: I0706 03:09:52.344403 2557 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-ebf514bc-openshift-default-599b766dc5-ndd7k" Jul 06 03:09:52.349471 ip-10-0-139-206 kubenswrapper[2557]: I0706 03:09:52.349443 2557 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-ebf514bc-openshift-default-599b766dc5-ndd7k" Jul 06 03:09:53.080861 ip-10-0-139-206 kubenswrapper[2557]: I0706 03:09:53.080832 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-ebf514bc-openshift-default-599b766dc5-ndd7k" Jul 06 03:09:53.081988 ip-10-0-139-206 kubenswrapper[2557]: I0706 03:09:53.081968 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-ebf514bc-openshift-default-599b766dc5-ndd7k" Jul 06 03:09:59.164878 ip-10-0-139-206 kubenswrapper[2557]: I0706 03:09:59.164839 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-srzrr"] Jul 06 03:10:07.157911 ip-10-0-139-206 kubenswrapper[2557]: I0706 03:10:07.157791 2557 generic.go:356] "Generic (PLEG): container finished" podID="fdb03c1e-1cd2-4749-9a7d-84ceda278f48" containerID="90bc3dd14fd540012c7f42e259e6c049106a253d2d4dac96795ef7b686fef8c0" exitCode=0 Jul 06 03:10:07.157911 ip-10-0-139-206 kubenswrapper[2557]: I0706 03:10:07.157872 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-ebf514bc-openshift-default-599b766dc5-ndd7k" event={"ID":"fdb03c1e-1cd2-4749-9a7d-84ceda278f48","Type":"ContainerDied","Data":"90bc3dd14fd540012c7f42e259e6c049106a253d2d4dac96795ef7b686fef8c0"} Jul 06 03:10:07.158519 ip-10-0-139-206 kubenswrapper[2557]: I0706 03:10:07.158453 2557 scope.go:117] "RemoveContainer" containerID="90bc3dd14fd540012c7f42e259e6c049106a253d2d4dac96795ef7b686fef8c0" Jul 06 03:10:07.159455 ip-10-0-139-206 kubenswrapper[2557]: I0706 03:10:07.159427 2557 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 06 03:10:07.159557 ip-10-0-139-206 kubenswrapper[2557]: I0706 03:10:07.159495 2557 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 06 03:10:07.159557 ip-10-0-139-206 kubenswrapper[2557]: I0706 03:10:07.159532 2557 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 06 03:10:08.081628 ip-10-0-139-206 kubenswrapper[2557]: I0706 03:10:08.081581 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-ebf514bc-openshift-default-599b766dc5-ndd7k" Jul 06 03:10:08.164107 ip-10-0-139-206 kubenswrapper[2557]: I0706 03:10:08.164075 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-ebf514bc-openshift-default-599b766dc5-ndd7k" event={"ID":"fdb03c1e-1cd2-4749-9a7d-84ceda278f48","Type":"ContainerStarted","Data":"9a328ebecccd88c4aef7071a1e8b839b05245c0704fe981cae52e4d6fdb27a3b"} Jul 06 03:10:08.344278 ip-10-0-139-206 kubenswrapper[2557]: I0706 03:10:08.344182 2557 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-ebf514bc-openshift-default-599b766dc5-ndd7k" Jul 06 03:10:08.345436 ip-10-0-139-206 kubenswrapper[2557]: I0706 03:10:08.345412 2557 patch_prober.go:28] interesting pod/e2e-mt-ebf514bc-openshift-default-599b766dc5-ndd7k container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.31:15021/healthz/ready\": dial tcp 10.133.0.31:15021: connect: connection refused" start-of-body= Jul 06 03:10:08.345499 ip-10-0-139-206 kubenswrapper[2557]: I0706 03:10:08.345470 2557 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-ebf514bc-openshift-default-599b766dc5-ndd7k" podUID="fdb03c1e-1cd2-4749-9a7d-84ceda278f48" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.31:15021/healthz/ready\": dial tcp 10.133.0.31:15021: connect: connection refused" Jul 06 03:10:09.344203 ip-10-0-139-206 kubenswrapper[2557]: I0706 03:10:09.344147 2557 patch_prober.go:28] interesting pod/e2e-mt-ebf514bc-openshift-default-599b766dc5-ndd7k container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.31:15021/healthz/ready\": dial tcp 10.133.0.31:15021: connect: connection refused" start-of-body= Jul 06 03:10:09.344728 ip-10-0-139-206 kubenswrapper[2557]: I0706 03:10:09.344217 2557 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-ebf514bc-openshift-default-599b766dc5-ndd7k" podUID="fdb03c1e-1cd2-4749-9a7d-84ceda278f48" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.31:15021/healthz/ready\": dial tcp 10.133.0.31:15021: connect: connection refused" Jul 06 03:10:09.563203 ip-10-0-139-206 kubenswrapper[2557]: I0706 03:10:09.563166 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-srzrr"] Jul 06 03:10:10.344777 ip-10-0-139-206 kubenswrapper[2557]: I0706 03:10:10.344735 2557 patch_prober.go:28] interesting pod/e2e-mt-ebf514bc-openshift-default-599b766dc5-ndd7k container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.31:15021/healthz/ready\": dial tcp 10.133.0.31:15021: connect: connection refused" start-of-body= Jul 06 03:10:10.345285 ip-10-0-139-206 kubenswrapper[2557]: I0706 03:10:10.344820 2557 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-ebf514bc-openshift-default-599b766dc5-ndd7k" podUID="fdb03c1e-1cd2-4749-9a7d-84ceda278f48" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.31:15021/healthz/ready\": dial tcp 10.133.0.31:15021: connect: connection refused" Jul 06 03:10:11.344212 ip-10-0-139-206 kubenswrapper[2557]: I0706 03:10:11.344177 2557 patch_prober.go:28] interesting pod/e2e-mt-ebf514bc-openshift-default-599b766dc5-ndd7k container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.31:15021/healthz/ready\": dial tcp 10.133.0.31:15021: connect: connection refused" start-of-body= Jul 06 03:10:11.344423 ip-10-0-139-206 kubenswrapper[2557]: I0706 03:10:11.344234 2557 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-ebf514bc-openshift-default-599b766dc5-ndd7k" podUID="fdb03c1e-1cd2-4749-9a7d-84ceda278f48" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.31:15021/healthz/ready\": dial tcp 10.133.0.31:15021: connect: connection refused" Jul 06 03:10:12.347922 ip-10-0-139-206 kubenswrapper[2557]: I0706 03:10:12.347885 2557 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-ebf514bc-openshift-default-599b766dc5-ndd7k" Jul 06 03:10:12.348599 ip-10-0-139-206 kubenswrapper[2557]: I0706 03:10:12.348173 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-ebf514bc-openshift-default-599b766dc5-ndd7k" Jul 06 03:10:12.348878 ip-10-0-139-206 kubenswrapper[2557]: I0706 03:10:12.348860 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-ebf514bc-openshift-default-599b766dc5-ndd7k" Jul 06 03:10:12.754028 ip-10-0-139-206 kubenswrapper[2557]: I0706 03:10:12.753990 2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-ebf514bc-openshift-default-599b766dc5-ndd7k"] Jul 06 03:10:13.188973 ip-10-0-139-206 kubenswrapper[2557]: I0706 03:10:13.188944 2557 kubelet_pods.go:1082] "Unable to retrieve pull secret, the image pull may not succeed." pod="openshift-ingress/e2e-mt-ebf514bc-openshift-default-599b766dc5-ndd7k" secret="" err="secret \"e2e-mt-ebf514bc-openshift-default-dockercfg-fxzmn\" not found" Jul 06 03:10:14.193006 ip-10-0-139-206 kubenswrapper[2557]: I0706 03:10:14.192963 2557 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-ingress/e2e-mt-ebf514bc-openshift-default-599b766dc5-ndd7k" podUID="fdb03c1e-1cd2-4749-9a7d-84ceda278f48" containerName="istio-proxy" containerID="cri-o://9a328ebecccd88c4aef7071a1e8b839b05245c0704fe981cae52e4d6fdb27a3b" gracePeriod=30 Jul 06 03:10:19.448501 ip-10-0-139-206 kubenswrapper[2557]: I0706 03:10:19.448469 2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-ebf514bc-openshift-default-599b766dc5-ndd7k" Jul 06 03:10:19.579303 ip-10-0-139-206 kubenswrapper[2557]: I0706 03:10:19.579204 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/fdb03c1e-1cd2-4749-9a7d-84ceda278f48-istio-token\") pod \"fdb03c1e-1cd2-4749-9a7d-84ceda278f48\" (UID: \"fdb03c1e-1cd2-4749-9a7d-84ceda278f48\") " Jul 06 03:10:19.579303 ip-10-0-139-206 kubenswrapper[2557]: I0706 03:10:19.579256 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/fdb03c1e-1cd2-4749-9a7d-84ceda278f48-istio-podinfo\") pod \"fdb03c1e-1cd2-4749-9a7d-84ceda278f48\" (UID: \"fdb03c1e-1cd2-4749-9a7d-84ceda278f48\") " Jul 06 03:10:19.579303 ip-10-0-139-206 kubenswrapper[2557]: I0706 03:10:19.579296 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/fdb03c1e-1cd2-4749-9a7d-84ceda278f48-istiod-ca-cert\") pod \"fdb03c1e-1cd2-4749-9a7d-84ceda278f48\" (UID: \"fdb03c1e-1cd2-4749-9a7d-84ceda278f48\") " Jul 06 03:10:19.579616 ip-10-0-139-206 kubenswrapper[2557]: I0706 03:10:19.579332 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/fdb03c1e-1cd2-4749-9a7d-84ceda278f48-istio-envoy\") pod \"fdb03c1e-1cd2-4749-9a7d-84ceda278f48\" (UID: \"fdb03c1e-1cd2-4749-9a7d-84ceda278f48\") " Jul 06 03:10:19.579616 ip-10-0-139-206 kubenswrapper[2557]: I0706 03:10:19.579406 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-mwwp7\" (UniqueName: \"kubernetes.io/projected/fdb03c1e-1cd2-4749-9a7d-84ceda278f48-kube-api-access-mwwp7\") pod \"fdb03c1e-1cd2-4749-9a7d-84ceda278f48\" (UID: \"fdb03c1e-1cd2-4749-9a7d-84ceda278f48\") " Jul 06 03:10:19.579616 ip-10-0-139-206 kubenswrapper[2557]: I0706 03:10:19.579448 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/fdb03c1e-1cd2-4749-9a7d-84ceda278f48-workload-socket\") pod \"fdb03c1e-1cd2-4749-9a7d-84ceda278f48\" (UID: \"fdb03c1e-1cd2-4749-9a7d-84ceda278f48\") " Jul 06 03:10:19.579616 ip-10-0-139-206 kubenswrapper[2557]: I0706 03:10:19.579483 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/fdb03c1e-1cd2-4749-9a7d-84ceda278f48-workload-certs\") pod \"fdb03c1e-1cd2-4749-9a7d-84ceda278f48\" (UID: \"fdb03c1e-1cd2-4749-9a7d-84ceda278f48\") " Jul 06 03:10:19.579616 ip-10-0-139-206 kubenswrapper[2557]: I0706 03:10:19.579512 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/fdb03c1e-1cd2-4749-9a7d-84ceda278f48-istio-data\") pod \"fdb03c1e-1cd2-4749-9a7d-84ceda278f48\" (UID: \"fdb03c1e-1cd2-4749-9a7d-84ceda278f48\") " Jul 06 03:10:19.579862 ip-10-0-139-206 kubenswrapper[2557]: I0706 03:10:19.579781 2557 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fdb03c1e-1cd2-4749-9a7d-84ceda278f48-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "fdb03c1e-1cd2-4749-9a7d-84ceda278f48" (UID: "fdb03c1e-1cd2-4749-9a7d-84ceda278f48"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 03:10:19.579862 ip-10-0-139-206 kubenswrapper[2557]: I0706 03:10:19.579788 2557 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fdb03c1e-1cd2-4749-9a7d-84ceda278f48-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "fdb03c1e-1cd2-4749-9a7d-84ceda278f48" (UID: "fdb03c1e-1cd2-4749-9a7d-84ceda278f48"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 03:10:19.580343 ip-10-0-139-206 kubenswrapper[2557]: I0706 03:10:19.579545 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/fdb03c1e-1cd2-4749-9a7d-84ceda278f48-credential-socket\") pod \"fdb03c1e-1cd2-4749-9a7d-84ceda278f48\" (UID: \"fdb03c1e-1cd2-4749-9a7d-84ceda278f48\") " Jul 06 03:10:19.580343 ip-10-0-139-206 kubenswrapper[2557]: I0706 03:10:19.580244 2557 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/fdb03c1e-1cd2-4749-9a7d-84ceda278f48-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "fdb03c1e-1cd2-4749-9a7d-84ceda278f48" (UID: "fdb03c1e-1cd2-4749-9a7d-84ceda278f48"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 06 03:10:19.580343 ip-10-0-139-206 kubenswrapper[2557]: I0706 03:10:19.580304 2557 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/fdb03c1e-1cd2-4749-9a7d-84ceda278f48-workload-socket\") on node \"ip-10-0-139-206.ec2.internal\" DevicePath \"\"" Jul 06 03:10:19.580343 ip-10-0-139-206 kubenswrapper[2557]: I0706 03:10:19.580322 2557 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/fdb03c1e-1cd2-4749-9a7d-84ceda278f48-workload-certs\") on node \"ip-10-0-139-206.ec2.internal\" DevicePath \"\"" Jul 06 03:10:19.580583 ip-10-0-139-206 kubenswrapper[2557]: I0706 03:10:19.580378 2557 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fdb03c1e-1cd2-4749-9a7d-84ceda278f48-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "fdb03c1e-1cd2-4749-9a7d-84ceda278f48" (UID: "fdb03c1e-1cd2-4749-9a7d-84ceda278f48"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 03:10:19.580583 ip-10-0-139-206 kubenswrapper[2557]: I0706 03:10:19.580456 2557 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fdb03c1e-1cd2-4749-9a7d-84ceda278f48-istio-data" (OuterVolumeSpecName: "istio-data") pod "fdb03c1e-1cd2-4749-9a7d-84ceda278f48" (UID: "fdb03c1e-1cd2-4749-9a7d-84ceda278f48"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 03:10:19.581708 ip-10-0-139-206 kubenswrapper[2557]: I0706 03:10:19.581684 2557 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/fdb03c1e-1cd2-4749-9a7d-84ceda278f48-istio-token" (OuterVolumeSpecName: "istio-token") pod "fdb03c1e-1cd2-4749-9a7d-84ceda278f48" (UID: "fdb03c1e-1cd2-4749-9a7d-84ceda278f48"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 06 03:10:19.581807 ip-10-0-139-206 kubenswrapper[2557]: I0706 03:10:19.581706 2557 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/fdb03c1e-1cd2-4749-9a7d-84ceda278f48-kube-api-access-mwwp7" (OuterVolumeSpecName: "kube-api-access-mwwp7") pod "fdb03c1e-1cd2-4749-9a7d-84ceda278f48" (UID: "fdb03c1e-1cd2-4749-9a7d-84ceda278f48"). InnerVolumeSpecName "kube-api-access-mwwp7". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 06 03:10:19.581903 ip-10-0-139-206 kubenswrapper[2557]: I0706 03:10:19.581871 2557 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fdb03c1e-1cd2-4749-9a7d-84ceda278f48-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "fdb03c1e-1cd2-4749-9a7d-84ceda278f48" (UID: "fdb03c1e-1cd2-4749-9a7d-84ceda278f48"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 03:10:19.581979 ip-10-0-139-206 kubenswrapper[2557]: I0706 03:10:19.581960 2557 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/fdb03c1e-1cd2-4749-9a7d-84ceda278f48-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "fdb03c1e-1cd2-4749-9a7d-84ceda278f48" (UID: "fdb03c1e-1cd2-4749-9a7d-84ceda278f48"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 06 03:10:19.681094 ip-10-0-139-206 kubenswrapper[2557]: I0706 03:10:19.681056 2557 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/fdb03c1e-1cd2-4749-9a7d-84ceda278f48-istio-data\") on node \"ip-10-0-139-206.ec2.internal\" DevicePath \"\"" Jul 06 03:10:19.681094 ip-10-0-139-206 kubenswrapper[2557]: I0706 03:10:19.681087 2557 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/fdb03c1e-1cd2-4749-9a7d-84ceda278f48-credential-socket\") on node \"ip-10-0-139-206.ec2.internal\" DevicePath \"\"" Jul 06 03:10:19.681094 ip-10-0-139-206 kubenswrapper[2557]: I0706 03:10:19.681098 2557 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/fdb03c1e-1cd2-4749-9a7d-84ceda278f48-istio-token\") on node \"ip-10-0-139-206.ec2.internal\" DevicePath \"\"" Jul 06 03:10:19.681094 ip-10-0-139-206 kubenswrapper[2557]: I0706 03:10:19.681107 2557 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/fdb03c1e-1cd2-4749-9a7d-84ceda278f48-istio-podinfo\") on node \"ip-10-0-139-206.ec2.internal\" DevicePath \"\"" Jul 06 03:10:19.681395 ip-10-0-139-206 kubenswrapper[2557]: I0706 03:10:19.681116 2557 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/fdb03c1e-1cd2-4749-9a7d-84ceda278f48-istiod-ca-cert\") on node \"ip-10-0-139-206.ec2.internal\" DevicePath \"\"" Jul 06 03:10:19.681395 ip-10-0-139-206 kubenswrapper[2557]: I0706 03:10:19.681125 2557 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/fdb03c1e-1cd2-4749-9a7d-84ceda278f48-istio-envoy\") on node \"ip-10-0-139-206.ec2.internal\" DevicePath \"\"" Jul 06 03:10:19.681395 ip-10-0-139-206 kubenswrapper[2557]: I0706 03:10:19.681133 2557 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-mwwp7\" (UniqueName: \"kubernetes.io/projected/fdb03c1e-1cd2-4749-9a7d-84ceda278f48-kube-api-access-mwwp7\") on node \"ip-10-0-139-206.ec2.internal\" DevicePath \"\"" Jul 06 03:10:20.222979 ip-10-0-139-206 kubenswrapper[2557]: I0706 03:10:20.222943 2557 generic.go:356] "Generic (PLEG): container finished" podID="fdb03c1e-1cd2-4749-9a7d-84ceda278f48" containerID="9a328ebecccd88c4aef7071a1e8b839b05245c0704fe981cae52e4d6fdb27a3b" exitCode=0 Jul 06 03:10:20.223154 ip-10-0-139-206 kubenswrapper[2557]: I0706 03:10:20.222998 2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-ebf514bc-openshift-default-599b766dc5-ndd7k" Jul 06 03:10:20.223154 ip-10-0-139-206 kubenswrapper[2557]: I0706 03:10:20.223032 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-ebf514bc-openshift-default-599b766dc5-ndd7k" event={"ID":"fdb03c1e-1cd2-4749-9a7d-84ceda278f48","Type":"ContainerDied","Data":"9a328ebecccd88c4aef7071a1e8b839b05245c0704fe981cae52e4d6fdb27a3b"} Jul 06 03:10:20.223154 ip-10-0-139-206 kubenswrapper[2557]: I0706 03:10:20.223090 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-ebf514bc-openshift-default-599b766dc5-ndd7k" event={"ID":"fdb03c1e-1cd2-4749-9a7d-84ceda278f48","Type":"ContainerDied","Data":"7ffc921cad389267b5378d5ef0556bbc42dc1c01ce1ef8728584fc55b50d3fe9"} Jul 06 03:10:20.223154 ip-10-0-139-206 kubenswrapper[2557]: I0706 03:10:20.223113 2557 scope.go:117] "RemoveContainer" containerID="9a328ebecccd88c4aef7071a1e8b839b05245c0704fe981cae52e4d6fdb27a3b" Jul 06 03:10:20.232240 ip-10-0-139-206 kubenswrapper[2557]: I0706 03:10:20.232220 2557 scope.go:117] "RemoveContainer" containerID="90bc3dd14fd540012c7f42e259e6c049106a253d2d4dac96795ef7b686fef8c0" Jul 06 03:10:20.240538 ip-10-0-139-206 kubenswrapper[2557]: I0706 03:10:20.240513 2557 scope.go:117] "RemoveContainer" containerID="9a328ebecccd88c4aef7071a1e8b839b05245c0704fe981cae52e4d6fdb27a3b" Jul 06 03:10:20.240810 ip-10-0-139-206 kubenswrapper[2557]: E0706 03:10:20.240787 2557 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9a328ebecccd88c4aef7071a1e8b839b05245c0704fe981cae52e4d6fdb27a3b\": container with ID starting with 9a328ebecccd88c4aef7071a1e8b839b05245c0704fe981cae52e4d6fdb27a3b not found: ID does not exist" containerID="9a328ebecccd88c4aef7071a1e8b839b05245c0704fe981cae52e4d6fdb27a3b" Jul 06 03:10:20.240852 ip-10-0-139-206 kubenswrapper[2557]: I0706 03:10:20.240826 2557 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9a328ebecccd88c4aef7071a1e8b839b05245c0704fe981cae52e4d6fdb27a3b"} err="failed to get container status \"9a328ebecccd88c4aef7071a1e8b839b05245c0704fe981cae52e4d6fdb27a3b\": rpc error: code = NotFound desc = could not find container \"9a328ebecccd88c4aef7071a1e8b839b05245c0704fe981cae52e4d6fdb27a3b\": container with ID starting with 9a328ebecccd88c4aef7071a1e8b839b05245c0704fe981cae52e4d6fdb27a3b not found: ID does not exist" Jul 06 03:10:20.240852 ip-10-0-139-206 kubenswrapper[2557]: I0706 03:10:20.240847 2557 scope.go:117] "RemoveContainer" containerID="90bc3dd14fd540012c7f42e259e6c049106a253d2d4dac96795ef7b686fef8c0" Jul 06 03:10:20.241060 ip-10-0-139-206 kubenswrapper[2557]: E0706 03:10:20.241045 2557 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"90bc3dd14fd540012c7f42e259e6c049106a253d2d4dac96795ef7b686fef8c0\": container with ID starting with 90bc3dd14fd540012c7f42e259e6c049106a253d2d4dac96795ef7b686fef8c0 not found: ID does not exist" containerID="90bc3dd14fd540012c7f42e259e6c049106a253d2d4dac96795ef7b686fef8c0" Jul 06 03:10:20.241092 ip-10-0-139-206 kubenswrapper[2557]: I0706 03:10:20.241064 2557 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"90bc3dd14fd540012c7f42e259e6c049106a253d2d4dac96795ef7b686fef8c0"} err="failed to get container status \"90bc3dd14fd540012c7f42e259e6c049106a253d2d4dac96795ef7b686fef8c0\": rpc error: code = NotFound desc = could not find container \"90bc3dd14fd540012c7f42e259e6c049106a253d2d4dac96795ef7b686fef8c0\": container with ID starting with 90bc3dd14fd540012c7f42e259e6c049106a253d2d4dac96795ef7b686fef8c0 not found: ID does not exist" Jul 06 03:10:20.244433 ip-10-0-139-206 kubenswrapper[2557]: I0706 03:10:20.244408 2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-ebf514bc-openshift-default-599b766dc5-ndd7k"] Jul 06 03:10:20.248174 ip-10-0-139-206 kubenswrapper[2557]: I0706 03:10:20.248149 2557 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-mt-ebf514bc-openshift-default-599b766dc5-ndd7k"] Jul 06 03:10:21.302444 ip-10-0-139-206 kubenswrapper[2557]: I0706 03:10:21.302411 2557 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="fdb03c1e-1cd2-4749-9a7d-84ceda278f48" path="/var/lib/kubelet/pods/fdb03c1e-1cd2-4749-9a7d-84ceda278f48/volumes" Jul 06 03:10:32.116472 ip-10-0-139-206 kubenswrapper[2557]: I0706 03:10:32.116434 2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-mt-7b6582bb-openshift-default-757b56899-hbm2j"] Jul 06 03:10:32.116956 ip-10-0-139-206 kubenswrapper[2557]: I0706 03:10:32.116749 2557 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="fdb03c1e-1cd2-4749-9a7d-84ceda278f48" containerName="istio-proxy" Jul 06 03:10:32.116956 ip-10-0-139-206 kubenswrapper[2557]: I0706 03:10:32.116760 2557 state_mem.go:107] "Deleted CPUSet assignment" podUID="fdb03c1e-1cd2-4749-9a7d-84ceda278f48" containerName="istio-proxy" Jul 06 03:10:32.116956 ip-10-0-139-206 kubenswrapper[2557]: I0706 03:10:32.116774 2557 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="fdb03c1e-1cd2-4749-9a7d-84ceda278f48" containerName="istio-proxy" Jul 06 03:10:32.116956 ip-10-0-139-206 kubenswrapper[2557]: I0706 03:10:32.116779 2557 state_mem.go:107] "Deleted CPUSet assignment" podUID="fdb03c1e-1cd2-4749-9a7d-84ceda278f48" containerName="istio-proxy" Jul 06 03:10:32.116956 ip-10-0-139-206 kubenswrapper[2557]: I0706 03:10:32.116839 2557 memory_manager.go:368] "RemoveStaleState removing state" podUID="fdb03c1e-1cd2-4749-9a7d-84ceda278f48" containerName="istio-proxy" Jul 06 03:10:32.121303 ip-10-0-139-206 kubenswrapper[2557]: I0706 03:10:32.121282 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-7b6582bb-openshift-default-757b56899-hbm2j" Jul 06 03:10:32.123840 ip-10-0-139-206 kubenswrapper[2557]: I0706 03:10:32.123818 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-mt-7b6582bb-openshift-default-dockercfg-8kt6q\"" Jul 06 03:10:32.134554 ip-10-0-139-206 kubenswrapper[2557]: I0706 03:10:32.134523 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-7b6582bb-openshift-default-757b56899-hbm2j"] Jul 06 03:10:32.175691 ip-10-0-139-206 kubenswrapper[2557]: I0706 03:10:32.175654 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/4c18e7e2-bde7-407c-bf1d-8aa3884e69a6-credential-socket\") pod \"e2e-mt-7b6582bb-openshift-default-757b56899-hbm2j\" (UID: \"4c18e7e2-bde7-407c-bf1d-8aa3884e69a6\") " pod="openshift-ingress/e2e-mt-7b6582bb-openshift-default-757b56899-hbm2j" Jul 06 03:10:32.175903 ip-10-0-139-206 kubenswrapper[2557]: I0706 03:10:32.175699 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/4c18e7e2-bde7-407c-bf1d-8aa3884e69a6-istio-podinfo\") pod \"e2e-mt-7b6582bb-openshift-default-757b56899-hbm2j\" (UID: \"4c18e7e2-bde7-407c-bf1d-8aa3884e69a6\") " pod="openshift-ingress/e2e-mt-7b6582bb-openshift-default-757b56899-hbm2j" Jul 06 03:10:32.175903 ip-10-0-139-206 kubenswrapper[2557]: I0706 03:10:32.175769 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/4c18e7e2-bde7-407c-bf1d-8aa3884e69a6-workload-certs\") pod \"e2e-mt-7b6582bb-openshift-default-757b56899-hbm2j\" (UID: \"4c18e7e2-bde7-407c-bf1d-8aa3884e69a6\") " pod="openshift-ingress/e2e-mt-7b6582bb-openshift-default-757b56899-hbm2j" Jul 06 03:10:32.175903 ip-10-0-139-206 kubenswrapper[2557]: I0706 03:10:32.175802 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/4c18e7e2-bde7-407c-bf1d-8aa3884e69a6-workload-socket\") pod \"e2e-mt-7b6582bb-openshift-default-757b56899-hbm2j\" (UID: \"4c18e7e2-bde7-407c-bf1d-8aa3884e69a6\") " pod="openshift-ingress/e2e-mt-7b6582bb-openshift-default-757b56899-hbm2j" Jul 06 03:10:32.175903 ip-10-0-139-206 kubenswrapper[2557]: I0706 03:10:32.175823 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-t7lhc\" (UniqueName: \"kubernetes.io/projected/4c18e7e2-bde7-407c-bf1d-8aa3884e69a6-kube-api-access-t7lhc\") pod \"e2e-mt-7b6582bb-openshift-default-757b56899-hbm2j\" (UID: \"4c18e7e2-bde7-407c-bf1d-8aa3884e69a6\") " pod="openshift-ingress/e2e-mt-7b6582bb-openshift-default-757b56899-hbm2j" Jul 06 03:10:32.175903 ip-10-0-139-206 kubenswrapper[2557]: I0706 03:10:32.175858 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/4c18e7e2-bde7-407c-bf1d-8aa3884e69a6-istio-envoy\") pod \"e2e-mt-7b6582bb-openshift-default-757b56899-hbm2j\" (UID: \"4c18e7e2-bde7-407c-bf1d-8aa3884e69a6\") " pod="openshift-ingress/e2e-mt-7b6582bb-openshift-default-757b56899-hbm2j" Jul 06 03:10:32.176099 ip-10-0-139-206 kubenswrapper[2557]: I0706 03:10:32.175920 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/4c18e7e2-bde7-407c-bf1d-8aa3884e69a6-istiod-ca-cert\") pod \"e2e-mt-7b6582bb-openshift-default-757b56899-hbm2j\" (UID: \"4c18e7e2-bde7-407c-bf1d-8aa3884e69a6\") " pod="openshift-ingress/e2e-mt-7b6582bb-openshift-default-757b56899-hbm2j" Jul 06 03:10:32.176099 ip-10-0-139-206 kubenswrapper[2557]: I0706 03:10:32.175959 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/4c18e7e2-bde7-407c-bf1d-8aa3884e69a6-istio-data\") pod \"e2e-mt-7b6582bb-openshift-default-757b56899-hbm2j\" (UID: \"4c18e7e2-bde7-407c-bf1d-8aa3884e69a6\") " pod="openshift-ingress/e2e-mt-7b6582bb-openshift-default-757b56899-hbm2j" Jul 06 03:10:32.176099 ip-10-0-139-206 kubenswrapper[2557]: I0706 03:10:32.175979 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/4c18e7e2-bde7-407c-bf1d-8aa3884e69a6-istio-token\") pod \"e2e-mt-7b6582bb-openshift-default-757b56899-hbm2j\" (UID: \"4c18e7e2-bde7-407c-bf1d-8aa3884e69a6\") " pod="openshift-ingress/e2e-mt-7b6582bb-openshift-default-757b56899-hbm2j" Jul 06 03:10:32.276569 ip-10-0-139-206 kubenswrapper[2557]: I0706 03:10:32.276536 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/4c18e7e2-bde7-407c-bf1d-8aa3884e69a6-workload-certs\") pod \"e2e-mt-7b6582bb-openshift-default-757b56899-hbm2j\" (UID: \"4c18e7e2-bde7-407c-bf1d-8aa3884e69a6\") " pod="openshift-ingress/e2e-mt-7b6582bb-openshift-default-757b56899-hbm2j" Jul 06 03:10:32.276784 ip-10-0-139-206 kubenswrapper[2557]: I0706 03:10:32.276582 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/4c18e7e2-bde7-407c-bf1d-8aa3884e69a6-workload-socket\") pod \"e2e-mt-7b6582bb-openshift-default-757b56899-hbm2j\" (UID: \"4c18e7e2-bde7-407c-bf1d-8aa3884e69a6\") " pod="openshift-ingress/e2e-mt-7b6582bb-openshift-default-757b56899-hbm2j" Jul 06 03:10:32.276784 ip-10-0-139-206 kubenswrapper[2557]: I0706 03:10:32.276607 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-t7lhc\" (UniqueName: \"kubernetes.io/projected/4c18e7e2-bde7-407c-bf1d-8aa3884e69a6-kube-api-access-t7lhc\") pod \"e2e-mt-7b6582bb-openshift-default-757b56899-hbm2j\" (UID: \"4c18e7e2-bde7-407c-bf1d-8aa3884e69a6\") " pod="openshift-ingress/e2e-mt-7b6582bb-openshift-default-757b56899-hbm2j" Jul 06 03:10:32.276784 ip-10-0-139-206 kubenswrapper[2557]: I0706 03:10:32.276636 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/4c18e7e2-bde7-407c-bf1d-8aa3884e69a6-istio-envoy\") pod \"e2e-mt-7b6582bb-openshift-default-757b56899-hbm2j\" (UID: \"4c18e7e2-bde7-407c-bf1d-8aa3884e69a6\") " pod="openshift-ingress/e2e-mt-7b6582bb-openshift-default-757b56899-hbm2j" Jul 06 03:10:32.276784 ip-10-0-139-206 kubenswrapper[2557]: I0706 03:10:32.276655 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/4c18e7e2-bde7-407c-bf1d-8aa3884e69a6-istiod-ca-cert\") pod \"e2e-mt-7b6582bb-openshift-default-757b56899-hbm2j\" (UID: \"4c18e7e2-bde7-407c-bf1d-8aa3884e69a6\") " pod="openshift-ingress/e2e-mt-7b6582bb-openshift-default-757b56899-hbm2j" Jul 06 03:10:32.276784 ip-10-0-139-206 kubenswrapper[2557]: I0706 03:10:32.276689 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/4c18e7e2-bde7-407c-bf1d-8aa3884e69a6-istio-data\") pod \"e2e-mt-7b6582bb-openshift-default-757b56899-hbm2j\" (UID: \"4c18e7e2-bde7-407c-bf1d-8aa3884e69a6\") " pod="openshift-ingress/e2e-mt-7b6582bb-openshift-default-757b56899-hbm2j" Jul 06 03:10:32.276784 ip-10-0-139-206 kubenswrapper[2557]: I0706 03:10:32.276716 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/4c18e7e2-bde7-407c-bf1d-8aa3884e69a6-istio-token\") pod \"e2e-mt-7b6582bb-openshift-default-757b56899-hbm2j\" (UID: \"4c18e7e2-bde7-407c-bf1d-8aa3884e69a6\") " pod="openshift-ingress/e2e-mt-7b6582bb-openshift-default-757b56899-hbm2j" Jul 06 03:10:32.276784 ip-10-0-139-206 kubenswrapper[2557]: I0706 03:10:32.276751 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/4c18e7e2-bde7-407c-bf1d-8aa3884e69a6-credential-socket\") pod \"e2e-mt-7b6582bb-openshift-default-757b56899-hbm2j\" (UID: \"4c18e7e2-bde7-407c-bf1d-8aa3884e69a6\") " pod="openshift-ingress/e2e-mt-7b6582bb-openshift-default-757b56899-hbm2j" Jul 06 03:10:32.276784 ip-10-0-139-206 kubenswrapper[2557]: I0706 03:10:32.276786 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/4c18e7e2-bde7-407c-bf1d-8aa3884e69a6-istio-podinfo\") pod \"e2e-mt-7b6582bb-openshift-default-757b56899-hbm2j\" (UID: \"4c18e7e2-bde7-407c-bf1d-8aa3884e69a6\") " pod="openshift-ingress/e2e-mt-7b6582bb-openshift-default-757b56899-hbm2j" Jul 06 03:10:32.277171 ip-10-0-139-206 kubenswrapper[2557]: I0706 03:10:32.277010 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/4c18e7e2-bde7-407c-bf1d-8aa3884e69a6-workload-socket\") pod \"e2e-mt-7b6582bb-openshift-default-757b56899-hbm2j\" (UID: \"4c18e7e2-bde7-407c-bf1d-8aa3884e69a6\") " pod="openshift-ingress/e2e-mt-7b6582bb-openshift-default-757b56899-hbm2j" Jul 06 03:10:32.277215 ip-10-0-139-206 kubenswrapper[2557]: I0706 03:10:32.277154 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/4c18e7e2-bde7-407c-bf1d-8aa3884e69a6-istio-data\") pod \"e2e-mt-7b6582bb-openshift-default-757b56899-hbm2j\" (UID: \"4c18e7e2-bde7-407c-bf1d-8aa3884e69a6\") " pod="openshift-ingress/e2e-mt-7b6582bb-openshift-default-757b56899-hbm2j" Jul 06 03:10:32.277254 ip-10-0-139-206 kubenswrapper[2557]: I0706 03:10:32.277210 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/4c18e7e2-bde7-407c-bf1d-8aa3884e69a6-credential-socket\") pod \"e2e-mt-7b6582bb-openshift-default-757b56899-hbm2j\" (UID: \"4c18e7e2-bde7-407c-bf1d-8aa3884e69a6\") " pod="openshift-ingress/e2e-mt-7b6582bb-openshift-default-757b56899-hbm2j" Jul 06 03:10:32.277436 ip-10-0-139-206 kubenswrapper[2557]: I0706 03:10:32.277413 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/4c18e7e2-bde7-407c-bf1d-8aa3884e69a6-workload-certs\") pod \"e2e-mt-7b6582bb-openshift-default-757b56899-hbm2j\" (UID: \"4c18e7e2-bde7-407c-bf1d-8aa3884e69a6\") " pod="openshift-ingress/e2e-mt-7b6582bb-openshift-default-757b56899-hbm2j" Jul 06 03:10:32.277526 ip-10-0-139-206 kubenswrapper[2557]: I0706 03:10:32.277417 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/4c18e7e2-bde7-407c-bf1d-8aa3884e69a6-istiod-ca-cert\") pod \"e2e-mt-7b6582bb-openshift-default-757b56899-hbm2j\" (UID: \"4c18e7e2-bde7-407c-bf1d-8aa3884e69a6\") " pod="openshift-ingress/e2e-mt-7b6582bb-openshift-default-757b56899-hbm2j" Jul 06 03:10:32.279426 ip-10-0-139-206 kubenswrapper[2557]: I0706 03:10:32.279401 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/4c18e7e2-bde7-407c-bf1d-8aa3884e69a6-istio-envoy\") pod \"e2e-mt-7b6582bb-openshift-default-757b56899-hbm2j\" (UID: \"4c18e7e2-bde7-407c-bf1d-8aa3884e69a6\") " pod="openshift-ingress/e2e-mt-7b6582bb-openshift-default-757b56899-hbm2j" Jul 06 03:10:32.279712 ip-10-0-139-206 kubenswrapper[2557]: I0706 03:10:32.279693 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/4c18e7e2-bde7-407c-bf1d-8aa3884e69a6-istio-podinfo\") pod \"e2e-mt-7b6582bb-openshift-default-757b56899-hbm2j\" (UID: \"4c18e7e2-bde7-407c-bf1d-8aa3884e69a6\") " pod="openshift-ingress/e2e-mt-7b6582bb-openshift-default-757b56899-hbm2j" Jul 06 03:10:32.284966 ip-10-0-139-206 kubenswrapper[2557]: I0706 03:10:32.284936 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-t7lhc\" (UniqueName: \"kubernetes.io/projected/4c18e7e2-bde7-407c-bf1d-8aa3884e69a6-kube-api-access-t7lhc\") pod \"e2e-mt-7b6582bb-openshift-default-757b56899-hbm2j\" (UID: \"4c18e7e2-bde7-407c-bf1d-8aa3884e69a6\") " pod="openshift-ingress/e2e-mt-7b6582bb-openshift-default-757b56899-hbm2j" Jul 06 03:10:32.285154 ip-10-0-139-206 kubenswrapper[2557]: I0706 03:10:32.285133 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/4c18e7e2-bde7-407c-bf1d-8aa3884e69a6-istio-token\") pod \"e2e-mt-7b6582bb-openshift-default-757b56899-hbm2j\" (UID: \"4c18e7e2-bde7-407c-bf1d-8aa3884e69a6\") " pod="openshift-ingress/e2e-mt-7b6582bb-openshift-default-757b56899-hbm2j" Jul 06 03:10:32.431958 ip-10-0-139-206 kubenswrapper[2557]: I0706 03:10:32.431925 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-7b6582bb-openshift-default-757b56899-hbm2j" Jul 06 03:10:32.574723 ip-10-0-139-206 kubenswrapper[2557]: I0706 03:10:32.574690 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-7b6582bb-openshift-default-757b56899-hbm2j"] Jul 06 03:10:32.577741 ip-10-0-139-206 kubenswrapper[2557]: W0706 03:10:32.577708 2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4c18e7e2_bde7_407c_bf1d_8aa3884e69a6.slice/crio-60daafce3fe9b7d50cfae5fc9a17ed431e7d89bcd7ca2494ad4b14ddd90cdd62 WatchSource:0}: Error finding container 60daafce3fe9b7d50cfae5fc9a17ed431e7d89bcd7ca2494ad4b14ddd90cdd62: Status 404 returned error can't find the container with id 60daafce3fe9b7d50cfae5fc9a17ed431e7d89bcd7ca2494ad4b14ddd90cdd62 Jul 06 03:10:32.579931 ip-10-0-139-206 kubenswrapper[2557]: I0706 03:10:32.579897 2557 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 06 03:10:32.580010 ip-10-0-139-206 kubenswrapper[2557]: I0706 03:10:32.579984 2557 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 06 03:10:32.580050 ip-10-0-139-206 kubenswrapper[2557]: I0706 03:10:32.580015 2557 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 06 03:10:33.285528 ip-10-0-139-206 kubenswrapper[2557]: I0706 03:10:33.285487 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-7b6582bb-openshift-default-757b56899-hbm2j" event={"ID":"4c18e7e2-bde7-407c-bf1d-8aa3884e69a6","Type":"ContainerStarted","Data":"c5d981688451e338a2c180066a6c7d59b30631b6af6093a129b9b0c13201323b"} Jul 06 03:10:33.286018 ip-10-0-139-206 kubenswrapper[2557]: I0706 03:10:33.285559 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-7b6582bb-openshift-default-757b56899-hbm2j" event={"ID":"4c18e7e2-bde7-407c-bf1d-8aa3884e69a6","Type":"ContainerStarted","Data":"60daafce3fe9b7d50cfae5fc9a17ed431e7d89bcd7ca2494ad4b14ddd90cdd62"} Jul 06 03:10:33.307977 ip-10-0-139-206 kubenswrapper[2557]: I0706 03:10:33.307918 2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-mt-7b6582bb-openshift-default-757b56899-hbm2j" podStartSLOduration=1.3078977250000001 podStartE2EDuration="1.307897725s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 03:10:32 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 03:10:33.306829043 +0000 UTC m=+1877.113115099" watchObservedRunningTime="2026-07-06 03:10:33.307897725 +0000 UTC m=+1877.114183784" Jul 06 03:10:33.432557 ip-10-0-139-206 kubenswrapper[2557]: I0706 03:10:33.432523 2557 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-7b6582bb-openshift-default-757b56899-hbm2j" Jul 06 03:10:33.437697 ip-10-0-139-206 kubenswrapper[2557]: I0706 03:10:33.437670 2557 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-7b6582bb-openshift-default-757b56899-hbm2j" Jul 06 03:10:34.289731 ip-10-0-139-206 kubenswrapper[2557]: I0706 03:10:34.289702 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-7b6582bb-openshift-default-757b56899-hbm2j" Jul 06 03:10:34.290691 ip-10-0-139-206 kubenswrapper[2557]: I0706 03:10:34.290675 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-7b6582bb-openshift-default-757b56899-hbm2j" Jul 06 03:10:41.970971 ip-10-0-139-206 kubenswrapper[2557]: I0706 03:10:41.970891 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-srzrr"] Jul 06 03:10:49.755892 ip-10-0-139-206 kubenswrapper[2557]: I0706 03:10:49.755591 2557 patch_prober.go:28] interesting pod/e2e-mt-7b6582bb-openshift-default-757b56899-hbm2j container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.133.0.32:15021/healthz/ready\": read tcp 10.133.0.2:58980->10.133.0.32:15021: read: connection reset by peer" start-of-body= Jul 06 03:10:49.755892 ip-10-0-139-206 kubenswrapper[2557]: I0706 03:10:49.755674 2557 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-mt-7b6582bb-openshift-default-757b56899-hbm2j" podUID="4c18e7e2-bde7-407c-bf1d-8aa3884e69a6" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.32:15021/healthz/ready\": read tcp 10.133.0.2:58980->10.133.0.32:15021: read: connection reset by peer" Jul 06 03:10:50.369192 ip-10-0-139-206 kubenswrapper[2557]: I0706 03:10:50.369157 2557 generic.go:356] "Generic (PLEG): container finished" podID="4c18e7e2-bde7-407c-bf1d-8aa3884e69a6" containerID="c5d981688451e338a2c180066a6c7d59b30631b6af6093a129b9b0c13201323b" exitCode=0 Jul 06 03:10:50.369386 ip-10-0-139-206 kubenswrapper[2557]: I0706 03:10:50.369237 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-7b6582bb-openshift-default-757b56899-hbm2j" event={"ID":"4c18e7e2-bde7-407c-bf1d-8aa3884e69a6","Type":"ContainerDied","Data":"c5d981688451e338a2c180066a6c7d59b30631b6af6093a129b9b0c13201323b"} Jul 06 03:10:50.369824 ip-10-0-139-206 kubenswrapper[2557]: I0706 03:10:50.369807 2557 scope.go:117] "RemoveContainer" containerID="c5d981688451e338a2c180066a6c7d59b30631b6af6093a129b9b0c13201323b" Jul 06 03:10:50.370926 ip-10-0-139-206 kubenswrapper[2557]: I0706 03:10:50.370901 2557 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 06 03:10:50.370987 ip-10-0-139-206 kubenswrapper[2557]: I0706 03:10:50.370971 2557 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 06 03:10:50.371025 ip-10-0-139-206 kubenswrapper[2557]: I0706 03:10:50.371010 2557 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 06 03:10:51.375768 ip-10-0-139-206 kubenswrapper[2557]: I0706 03:10:51.375733 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-7b6582bb-openshift-default-757b56899-hbm2j" event={"ID":"4c18e7e2-bde7-407c-bf1d-8aa3884e69a6","Type":"ContainerStarted","Data":"91a2b2d3fe754802d5cdfded4814a9203589507e075de3c4d07c658b03e2d740"} Jul 06 03:10:51.432438 ip-10-0-139-206 kubenswrapper[2557]: I0706 03:10:51.432390 2557 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-7b6582bb-openshift-default-757b56899-hbm2j" Jul 06 03:10:51.432866 ip-10-0-139-206 kubenswrapper[2557]: I0706 03:10:51.432837 2557 patch_prober.go:28] interesting pod/e2e-mt-7b6582bb-openshift-default-757b56899-hbm2j container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.32:15021/healthz/ready\": dial tcp 10.133.0.32:15021: connect: connection refused" start-of-body= Jul 06 03:10:51.433109 ip-10-0-139-206 kubenswrapper[2557]: I0706 03:10:51.433075 2557 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-7b6582bb-openshift-default-757b56899-hbm2j" podUID="4c18e7e2-bde7-407c-bf1d-8aa3884e69a6" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.32:15021/healthz/ready\": dial tcp 10.133.0.32:15021: connect: connection refused" Jul 06 03:10:52.433410 ip-10-0-139-206 kubenswrapper[2557]: I0706 03:10:52.433367 2557 patch_prober.go:28] interesting pod/e2e-mt-7b6582bb-openshift-default-757b56899-hbm2j container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.32:15021/healthz/ready\": dial tcp 10.133.0.32:15021: connect: connection refused" start-of-body= Jul 06 03:10:52.433819 ip-10-0-139-206 kubenswrapper[2557]: I0706 03:10:52.433429 2557 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-7b6582bb-openshift-default-757b56899-hbm2j" podUID="4c18e7e2-bde7-407c-bf1d-8aa3884e69a6" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.32:15021/healthz/ready\": dial tcp 10.133.0.32:15021: connect: connection refused" Jul 06 03:10:53.432767 ip-10-0-139-206 kubenswrapper[2557]: I0706 03:10:53.432727 2557 patch_prober.go:28] interesting pod/e2e-mt-7b6582bb-openshift-default-757b56899-hbm2j container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.32:15021/healthz/ready\": dial tcp 10.133.0.32:15021: connect: connection refused" start-of-body= Jul 06 03:10:53.432978 ip-10-0-139-206 kubenswrapper[2557]: I0706 03:10:53.432785 2557 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-7b6582bb-openshift-default-757b56899-hbm2j" podUID="4c18e7e2-bde7-407c-bf1d-8aa3884e69a6" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.32:15021/healthz/ready\": dial tcp 10.133.0.32:15021: connect: connection refused" Jul 06 03:10:54.432803 ip-10-0-139-206 kubenswrapper[2557]: I0706 03:10:54.432754 2557 patch_prober.go:28] interesting pod/e2e-mt-7b6582bb-openshift-default-757b56899-hbm2j container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.32:15021/healthz/ready\": dial tcp 10.133.0.32:15021: connect: connection refused" start-of-body= Jul 06 03:10:54.433311 ip-10-0-139-206 kubenswrapper[2557]: I0706 03:10:54.432826 2557 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-7b6582bb-openshift-default-757b56899-hbm2j" podUID="4c18e7e2-bde7-407c-bf1d-8aa3884e69a6" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.32:15021/healthz/ready\": dial tcp 10.133.0.32:15021: connect: connection refused" Jul 06 03:10:55.436176 ip-10-0-139-206 kubenswrapper[2557]: I0706 03:10:55.436146 2557 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-7b6582bb-openshift-default-757b56899-hbm2j" Jul 06 03:10:55.436694 ip-10-0-139-206 kubenswrapper[2557]: I0706 03:10:55.436651 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-7b6582bb-openshift-default-757b56899-hbm2j" Jul 06 03:10:55.437287 ip-10-0-139-206 kubenswrapper[2557]: I0706 03:10:55.437268 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-7b6582bb-openshift-default-757b56899-hbm2j" Jul 06 03:11:03.667551 ip-10-0-139-206 kubenswrapper[2557]: I0706 03:11:03.667509 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-srzrr"] Jul 06 03:11:19.872735 ip-10-0-139-206 kubenswrapper[2557]: I0706 03:11:19.872700 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-srzrr"] Jul 06 03:11:33.772554 ip-10-0-139-206 kubenswrapper[2557]: I0706 03:11:33.772515 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-srzrr"] Jul 06 03:12:13.159572 ip-10-0-139-206 kubenswrapper[2557]: I0706 03:12:13.159490 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-srzrr"] Jul 06 03:12:16.291568 ip-10-0-139-206 kubenswrapper[2557]: I0706 03:12:16.291530 2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-7b6582bb-openshift-default-757b56899-hbm2j"] Jul 06 03:12:16.292060 ip-10-0-139-206 kubenswrapper[2557]: I0706 03:12:16.291721 2557 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-ingress/e2e-mt-7b6582bb-openshift-default-757b56899-hbm2j" podUID="4c18e7e2-bde7-407c-bf1d-8aa3884e69a6" containerName="istio-proxy" containerID="cri-o://91a2b2d3fe754802d5cdfded4814a9203589507e075de3c4d07c658b03e2d740" gracePeriod=30 Jul 06 03:12:21.546548 ip-10-0-139-206 kubenswrapper[2557]: I0706 03:12:21.546518 2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-7b6582bb-openshift-default-757b56899-hbm2j" Jul 06 03:12:21.630264 ip-10-0-139-206 kubenswrapper[2557]: I0706 03:12:21.630169 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/4c18e7e2-bde7-407c-bf1d-8aa3884e69a6-credential-socket\") pod \"4c18e7e2-bde7-407c-bf1d-8aa3884e69a6\" (UID: \"4c18e7e2-bde7-407c-bf1d-8aa3884e69a6\") " Jul 06 03:12:21.630264 ip-10-0-139-206 kubenswrapper[2557]: I0706 03:12:21.630204 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/4c18e7e2-bde7-407c-bf1d-8aa3884e69a6-istio-token\") pod \"4c18e7e2-bde7-407c-bf1d-8aa3884e69a6\" (UID: \"4c18e7e2-bde7-407c-bf1d-8aa3884e69a6\") " Jul 06 03:12:21.630264 ip-10-0-139-206 kubenswrapper[2557]: I0706 03:12:21.630225 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/4c18e7e2-bde7-407c-bf1d-8aa3884e69a6-istio-envoy\") pod \"4c18e7e2-bde7-407c-bf1d-8aa3884e69a6\" (UID: \"4c18e7e2-bde7-407c-bf1d-8aa3884e69a6\") " Jul 06 03:12:21.630264 ip-10-0-139-206 kubenswrapper[2557]: I0706 03:12:21.630249 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/4c18e7e2-bde7-407c-bf1d-8aa3884e69a6-istio-data\") pod \"4c18e7e2-bde7-407c-bf1d-8aa3884e69a6\" (UID: \"4c18e7e2-bde7-407c-bf1d-8aa3884e69a6\") " Jul 06 03:12:21.630639 ip-10-0-139-206 kubenswrapper[2557]: I0706 03:12:21.630278 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/4c18e7e2-bde7-407c-bf1d-8aa3884e69a6-istiod-ca-cert\") pod \"4c18e7e2-bde7-407c-bf1d-8aa3884e69a6\" (UID: \"4c18e7e2-bde7-407c-bf1d-8aa3884e69a6\") " Jul 06 03:12:21.630639 ip-10-0-139-206 kubenswrapper[2557]: I0706 03:12:21.630443 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/4c18e7e2-bde7-407c-bf1d-8aa3884e69a6-istio-podinfo\") pod \"4c18e7e2-bde7-407c-bf1d-8aa3884e69a6\" (UID: \"4c18e7e2-bde7-407c-bf1d-8aa3884e69a6\") " Jul 06 03:12:21.630639 ip-10-0-139-206 kubenswrapper[2557]: I0706 03:12:21.630522 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/4c18e7e2-bde7-407c-bf1d-8aa3884e69a6-workload-certs\") pod \"4c18e7e2-bde7-407c-bf1d-8aa3884e69a6\" (UID: \"4c18e7e2-bde7-407c-bf1d-8aa3884e69a6\") " Jul 06 03:12:21.630639 ip-10-0-139-206 kubenswrapper[2557]: I0706 03:12:21.630575 2557 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4c18e7e2-bde7-407c-bf1d-8aa3884e69a6-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "4c18e7e2-bde7-407c-bf1d-8aa3884e69a6" (UID: "4c18e7e2-bde7-407c-bf1d-8aa3884e69a6"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 03:12:21.630639 ip-10-0-139-206 kubenswrapper[2557]: I0706 03:12:21.630627 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/4c18e7e2-bde7-407c-bf1d-8aa3884e69a6-workload-socket\") pod \"4c18e7e2-bde7-407c-bf1d-8aa3884e69a6\" (UID: \"4c18e7e2-bde7-407c-bf1d-8aa3884e69a6\") " Jul 06 03:12:21.630890 ip-10-0-139-206 kubenswrapper[2557]: I0706 03:12:21.630658 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-t7lhc\" (UniqueName: \"kubernetes.io/projected/4c18e7e2-bde7-407c-bf1d-8aa3884e69a6-kube-api-access-t7lhc\") pod \"4c18e7e2-bde7-407c-bf1d-8aa3884e69a6\" (UID: \"4c18e7e2-bde7-407c-bf1d-8aa3884e69a6\") " Jul 06 03:12:21.630890 ip-10-0-139-206 kubenswrapper[2557]: I0706 03:12:21.630763 2557 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4c18e7e2-bde7-407c-bf1d-8aa3884e69a6-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "4c18e7e2-bde7-407c-bf1d-8aa3884e69a6" (UID: "4c18e7e2-bde7-407c-bf1d-8aa3884e69a6"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 03:12:21.630890 ip-10-0-139-206 kubenswrapper[2557]: I0706 03:12:21.630878 2557 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4c18e7e2-bde7-407c-bf1d-8aa3884e69a6-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "4c18e7e2-bde7-407c-bf1d-8aa3884e69a6" (UID: "4c18e7e2-bde7-407c-bf1d-8aa3884e69a6"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 03:12:21.631090 ip-10-0-139-206 kubenswrapper[2557]: I0706 03:12:21.630969 2557 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/4c18e7e2-bde7-407c-bf1d-8aa3884e69a6-workload-socket\") on node \"ip-10-0-139-206.ec2.internal\" DevicePath \"\"" Jul 06 03:12:21.631090 ip-10-0-139-206 kubenswrapper[2557]: I0706 03:12:21.630988 2557 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/4c18e7e2-bde7-407c-bf1d-8aa3884e69a6-credential-socket\") on node \"ip-10-0-139-206.ec2.internal\" DevicePath \"\"" Jul 06 03:12:21.631090 ip-10-0-139-206 kubenswrapper[2557]: I0706 03:12:21.631051 2557 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/4c18e7e2-bde7-407c-bf1d-8aa3884e69a6-workload-certs\") on node \"ip-10-0-139-206.ec2.internal\" DevicePath \"\"" Jul 06 03:12:21.631240 ip-10-0-139-206 kubenswrapper[2557]: I0706 03:12:21.631150 2557 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4c18e7e2-bde7-407c-bf1d-8aa3884e69a6-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "4c18e7e2-bde7-407c-bf1d-8aa3884e69a6" (UID: "4c18e7e2-bde7-407c-bf1d-8aa3884e69a6"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 06 03:12:21.631324 ip-10-0-139-206 kubenswrapper[2557]: I0706 03:12:21.631295 2557 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4c18e7e2-bde7-407c-bf1d-8aa3884e69a6-istio-data" (OuterVolumeSpecName: "istio-data") pod "4c18e7e2-bde7-407c-bf1d-8aa3884e69a6" (UID: "4c18e7e2-bde7-407c-bf1d-8aa3884e69a6"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 03:12:21.632672 ip-10-0-139-206 kubenswrapper[2557]: I0706 03:12:21.632642 2557 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4c18e7e2-bde7-407c-bf1d-8aa3884e69a6-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "4c18e7e2-bde7-407c-bf1d-8aa3884e69a6" (UID: "4c18e7e2-bde7-407c-bf1d-8aa3884e69a6"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 03:12:21.632672 ip-10-0-139-206 kubenswrapper[2557]: I0706 03:12:21.632656 2557 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4c18e7e2-bde7-407c-bf1d-8aa3884e69a6-istio-token" (OuterVolumeSpecName: "istio-token") pod "4c18e7e2-bde7-407c-bf1d-8aa3884e69a6" (UID: "4c18e7e2-bde7-407c-bf1d-8aa3884e69a6"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 06 03:12:21.632828 ip-10-0-139-206 kubenswrapper[2557]: I0706 03:12:21.632742 2557 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/4c18e7e2-bde7-407c-bf1d-8aa3884e69a6-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "4c18e7e2-bde7-407c-bf1d-8aa3884e69a6" (UID: "4c18e7e2-bde7-407c-bf1d-8aa3884e69a6"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 06 03:12:21.632828 ip-10-0-139-206 kubenswrapper[2557]: I0706 03:12:21.632747 2557 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4c18e7e2-bde7-407c-bf1d-8aa3884e69a6-kube-api-access-t7lhc" (OuterVolumeSpecName: "kube-api-access-t7lhc") pod "4c18e7e2-bde7-407c-bf1d-8aa3884e69a6" (UID: "4c18e7e2-bde7-407c-bf1d-8aa3884e69a6"). InnerVolumeSpecName "kube-api-access-t7lhc". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 06 03:12:21.732221 ip-10-0-139-206 kubenswrapper[2557]: I0706 03:12:21.732186 2557 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/4c18e7e2-bde7-407c-bf1d-8aa3884e69a6-istio-data\") on node \"ip-10-0-139-206.ec2.internal\" DevicePath \"\"" Jul 06 03:12:21.732221 ip-10-0-139-206 kubenswrapper[2557]: I0706 03:12:21.732213 2557 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/4c18e7e2-bde7-407c-bf1d-8aa3884e69a6-istiod-ca-cert\") on node \"ip-10-0-139-206.ec2.internal\" DevicePath \"\"" Jul 06 03:12:21.732221 ip-10-0-139-206 kubenswrapper[2557]: I0706 03:12:21.732223 2557 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/4c18e7e2-bde7-407c-bf1d-8aa3884e69a6-istio-podinfo\") on node \"ip-10-0-139-206.ec2.internal\" DevicePath \"\"" Jul 06 03:12:21.732221 ip-10-0-139-206 kubenswrapper[2557]: I0706 03:12:21.732233 2557 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-t7lhc\" (UniqueName: \"kubernetes.io/projected/4c18e7e2-bde7-407c-bf1d-8aa3884e69a6-kube-api-access-t7lhc\") on node \"ip-10-0-139-206.ec2.internal\" DevicePath \"\"" Jul 06 03:12:21.732532 ip-10-0-139-206 kubenswrapper[2557]: I0706 03:12:21.732242 2557 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/4c18e7e2-bde7-407c-bf1d-8aa3884e69a6-istio-token\") on node \"ip-10-0-139-206.ec2.internal\" DevicePath \"\"" Jul 06 03:12:21.732532 ip-10-0-139-206 kubenswrapper[2557]: I0706 03:12:21.732249 2557 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/4c18e7e2-bde7-407c-bf1d-8aa3884e69a6-istio-envoy\") on node \"ip-10-0-139-206.ec2.internal\" DevicePath \"\"" Jul 06 03:12:21.788810 ip-10-0-139-206 kubenswrapper[2557]: I0706 03:12:21.788776 2557 generic.go:356] "Generic (PLEG): container finished" podID="4c18e7e2-bde7-407c-bf1d-8aa3884e69a6" containerID="91a2b2d3fe754802d5cdfded4814a9203589507e075de3c4d07c658b03e2d740" exitCode=0 Jul 06 03:12:21.788990 ip-10-0-139-206 kubenswrapper[2557]: I0706 03:12:21.788826 2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-7b6582bb-openshift-default-757b56899-hbm2j" Jul 06 03:12:21.788990 ip-10-0-139-206 kubenswrapper[2557]: I0706 03:12:21.788844 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-7b6582bb-openshift-default-757b56899-hbm2j" event={"ID":"4c18e7e2-bde7-407c-bf1d-8aa3884e69a6","Type":"ContainerDied","Data":"91a2b2d3fe754802d5cdfded4814a9203589507e075de3c4d07c658b03e2d740"} Jul 06 03:12:21.788990 ip-10-0-139-206 kubenswrapper[2557]: I0706 03:12:21.788887 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-7b6582bb-openshift-default-757b56899-hbm2j" event={"ID":"4c18e7e2-bde7-407c-bf1d-8aa3884e69a6","Type":"ContainerDied","Data":"60daafce3fe9b7d50cfae5fc9a17ed431e7d89bcd7ca2494ad4b14ddd90cdd62"} Jul 06 03:12:21.788990 ip-10-0-139-206 kubenswrapper[2557]: I0706 03:12:21.788907 2557 scope.go:117] "RemoveContainer" containerID="91a2b2d3fe754802d5cdfded4814a9203589507e075de3c4d07c658b03e2d740" Jul 06 03:12:21.797866 ip-10-0-139-206 kubenswrapper[2557]: I0706 03:12:21.797846 2557 scope.go:117] "RemoveContainer" containerID="c5d981688451e338a2c180066a6c7d59b30631b6af6093a129b9b0c13201323b" Jul 06 03:12:21.805744 ip-10-0-139-206 kubenswrapper[2557]: I0706 03:12:21.805723 2557 scope.go:117] "RemoveContainer" containerID="91a2b2d3fe754802d5cdfded4814a9203589507e075de3c4d07c658b03e2d740" Jul 06 03:12:21.806055 ip-10-0-139-206 kubenswrapper[2557]: E0706 03:12:21.806033 2557 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"91a2b2d3fe754802d5cdfded4814a9203589507e075de3c4d07c658b03e2d740\": container with ID starting with 91a2b2d3fe754802d5cdfded4814a9203589507e075de3c4d07c658b03e2d740 not found: ID does not exist" containerID="91a2b2d3fe754802d5cdfded4814a9203589507e075de3c4d07c658b03e2d740" Jul 06 03:12:21.806136 ip-10-0-139-206 kubenswrapper[2557]: I0706 03:12:21.806068 2557 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"91a2b2d3fe754802d5cdfded4814a9203589507e075de3c4d07c658b03e2d740"} err="failed to get container status \"91a2b2d3fe754802d5cdfded4814a9203589507e075de3c4d07c658b03e2d740\": rpc error: code = NotFound desc = could not find container \"91a2b2d3fe754802d5cdfded4814a9203589507e075de3c4d07c658b03e2d740\": container with ID starting with 91a2b2d3fe754802d5cdfded4814a9203589507e075de3c4d07c658b03e2d740 not found: ID does not exist" Jul 06 03:12:21.806136 ip-10-0-139-206 kubenswrapper[2557]: I0706 03:12:21.806095 2557 scope.go:117] "RemoveContainer" containerID="c5d981688451e338a2c180066a6c7d59b30631b6af6093a129b9b0c13201323b" Jul 06 03:12:21.806358 ip-10-0-139-206 kubenswrapper[2557]: E0706 03:12:21.806329 2557 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c5d981688451e338a2c180066a6c7d59b30631b6af6093a129b9b0c13201323b\": container with ID starting with c5d981688451e338a2c180066a6c7d59b30631b6af6093a129b9b0c13201323b not found: ID does not exist" containerID="c5d981688451e338a2c180066a6c7d59b30631b6af6093a129b9b0c13201323b" Jul 06 03:12:21.806424 ip-10-0-139-206 kubenswrapper[2557]: I0706 03:12:21.806368 2557 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c5d981688451e338a2c180066a6c7d59b30631b6af6093a129b9b0c13201323b"} err="failed to get container status \"c5d981688451e338a2c180066a6c7d59b30631b6af6093a129b9b0c13201323b\": rpc error: code = NotFound desc = could not find container \"c5d981688451e338a2c180066a6c7d59b30631b6af6093a129b9b0c13201323b\": container with ID starting with c5d981688451e338a2c180066a6c7d59b30631b6af6093a129b9b0c13201323b not found: ID does not exist" Jul 06 03:12:21.809689 ip-10-0-139-206 kubenswrapper[2557]: I0706 03:12:21.809664 2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-7b6582bb-openshift-default-757b56899-hbm2j"] Jul 06 03:12:21.813229 ip-10-0-139-206 kubenswrapper[2557]: I0706 03:12:21.813207 2557 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-mt-7b6582bb-openshift-default-757b56899-hbm2j"] Jul 06 03:12:23.302938 ip-10-0-139-206 kubenswrapper[2557]: I0706 03:12:23.302904 2557 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="4c18e7e2-bde7-407c-bf1d-8aa3884e69a6" path="/var/lib/kubelet/pods/4c18e7e2-bde7-407c-bf1d-8aa3884e69a6/volumes" Jul 06 03:13:10.864395 ip-10-0-139-206 kubenswrapper[2557]: I0706 03:13:10.864335 2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-inf-a-2236bf-openshift-default-bf8bf7b95-sf8lf"] Jul 06 03:13:10.864831 ip-10-0-139-206 kubenswrapper[2557]: I0706 03:13:10.864705 2557 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4c18e7e2-bde7-407c-bf1d-8aa3884e69a6" containerName="istio-proxy" Jul 06 03:13:10.864831 ip-10-0-139-206 kubenswrapper[2557]: I0706 03:13:10.864717 2557 state_mem.go:107] "Deleted CPUSet assignment" podUID="4c18e7e2-bde7-407c-bf1d-8aa3884e69a6" containerName="istio-proxy" Jul 06 03:13:10.864831 ip-10-0-139-206 kubenswrapper[2557]: I0706 03:13:10.864777 2557 memory_manager.go:368] "RemoveStaleState removing state" podUID="4c18e7e2-bde7-407c-bf1d-8aa3884e69a6" containerName="istio-proxy" Jul 06 03:13:10.864831 ip-10-0-139-206 kubenswrapper[2557]: I0706 03:13:10.864786 2557 memory_manager.go:368] "RemoveStaleState removing state" podUID="fdb03c1e-1cd2-4749-9a7d-84ceda278f48" containerName="istio-proxy" Jul 06 03:13:10.864831 ip-10-0-139-206 kubenswrapper[2557]: I0706 03:13:10.864793 2557 memory_manager.go:368] "RemoveStaleState removing state" podUID="4c18e7e2-bde7-407c-bf1d-8aa3884e69a6" containerName="istio-proxy" Jul 06 03:13:10.867830 ip-10-0-139-206 kubenswrapper[2557]: I0706 03:13:10.867811 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-2236bf-openshift-default-bf8bf7b95-sf8lf" Jul 06 03:13:10.870152 ip-10-0-139-206 kubenswrapper[2557]: I0706 03:13:10.870131 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-inf-a-2236bf-openshift-default-dockercfg-rgvg5\"" Jul 06 03:13:10.879009 ip-10-0-139-206 kubenswrapper[2557]: I0706 03:13:10.878982 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-a-2236bf-openshift-default-bf8bf7b95-sf8lf"] Jul 06 03:13:10.948538 ip-10-0-139-206 kubenswrapper[2557]: I0706 03:13:10.948498 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/60a08f89-4663-40d2-971e-1360f8c7712f-istio-token\") pod \"e2e-inf-a-2236bf-openshift-default-bf8bf7b95-sf8lf\" (UID: \"60a08f89-4663-40d2-971e-1360f8c7712f\") " pod="openshift-ingress/e2e-inf-a-2236bf-openshift-default-bf8bf7b95-sf8lf" Jul 06 03:13:10.948752 ip-10-0-139-206 kubenswrapper[2557]: I0706 03:13:10.948553 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/60a08f89-4663-40d2-971e-1360f8c7712f-credential-socket\") pod \"e2e-inf-a-2236bf-openshift-default-bf8bf7b95-sf8lf\" (UID: \"60a08f89-4663-40d2-971e-1360f8c7712f\") " pod="openshift-ingress/e2e-inf-a-2236bf-openshift-default-bf8bf7b95-sf8lf" Jul 06 03:13:10.948752 ip-10-0-139-206 kubenswrapper[2557]: I0706 03:13:10.948590 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/60a08f89-4663-40d2-971e-1360f8c7712f-istio-podinfo\") pod \"e2e-inf-a-2236bf-openshift-default-bf8bf7b95-sf8lf\" (UID: \"60a08f89-4663-40d2-971e-1360f8c7712f\") " pod="openshift-ingress/e2e-inf-a-2236bf-openshift-default-bf8bf7b95-sf8lf" Jul 06 03:13:10.948752 ip-10-0-139-206 kubenswrapper[2557]: I0706 03:13:10.948669 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/60a08f89-4663-40d2-971e-1360f8c7712f-istiod-ca-cert\") pod \"e2e-inf-a-2236bf-openshift-default-bf8bf7b95-sf8lf\" (UID: \"60a08f89-4663-40d2-971e-1360f8c7712f\") " pod="openshift-ingress/e2e-inf-a-2236bf-openshift-default-bf8bf7b95-sf8lf" Jul 06 03:13:10.948752 ip-10-0-139-206 kubenswrapper[2557]: I0706 03:13:10.948719 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/60a08f89-4663-40d2-971e-1360f8c7712f-workload-socket\") pod \"e2e-inf-a-2236bf-openshift-default-bf8bf7b95-sf8lf\" (UID: \"60a08f89-4663-40d2-971e-1360f8c7712f\") " pod="openshift-ingress/e2e-inf-a-2236bf-openshift-default-bf8bf7b95-sf8lf" Jul 06 03:13:10.948752 ip-10-0-139-206 kubenswrapper[2557]: I0706 03:13:10.948760 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/60a08f89-4663-40d2-971e-1360f8c7712f-istio-data\") pod \"e2e-inf-a-2236bf-openshift-default-bf8bf7b95-sf8lf\" (UID: \"60a08f89-4663-40d2-971e-1360f8c7712f\") " pod="openshift-ingress/e2e-inf-a-2236bf-openshift-default-bf8bf7b95-sf8lf" Jul 06 03:13:10.948752 ip-10-0-139-206 kubenswrapper[2557]: I0706 03:13:10.948791 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/60a08f89-4663-40d2-971e-1360f8c7712f-workload-certs\") pod \"e2e-inf-a-2236bf-openshift-default-bf8bf7b95-sf8lf\" (UID: \"60a08f89-4663-40d2-971e-1360f8c7712f\") " pod="openshift-ingress/e2e-inf-a-2236bf-openshift-default-bf8bf7b95-sf8lf" Jul 06 03:13:10.948752 ip-10-0-139-206 kubenswrapper[2557]: I0706 03:13:10.948820 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/60a08f89-4663-40d2-971e-1360f8c7712f-istio-envoy\") pod \"e2e-inf-a-2236bf-openshift-default-bf8bf7b95-sf8lf\" (UID: \"60a08f89-4663-40d2-971e-1360f8c7712f\") " pod="openshift-ingress/e2e-inf-a-2236bf-openshift-default-bf8bf7b95-sf8lf" Jul 06 03:13:10.949274 ip-10-0-139-206 kubenswrapper[2557]: I0706 03:13:10.948849 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zl555\" (UniqueName: \"kubernetes.io/projected/60a08f89-4663-40d2-971e-1360f8c7712f-kube-api-access-zl555\") pod \"e2e-inf-a-2236bf-openshift-default-bf8bf7b95-sf8lf\" (UID: \"60a08f89-4663-40d2-971e-1360f8c7712f\") " pod="openshift-ingress/e2e-inf-a-2236bf-openshift-default-bf8bf7b95-sf8lf" Jul 06 03:13:11.050112 ip-10-0-139-206 kubenswrapper[2557]: I0706 03:13:11.050074 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/60a08f89-4663-40d2-971e-1360f8c7712f-credential-socket\") pod \"e2e-inf-a-2236bf-openshift-default-bf8bf7b95-sf8lf\" (UID: \"60a08f89-4663-40d2-971e-1360f8c7712f\") " pod="openshift-ingress/e2e-inf-a-2236bf-openshift-default-bf8bf7b95-sf8lf" Jul 06 03:13:11.050112 ip-10-0-139-206 kubenswrapper[2557]: I0706 03:13:11.050117 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/60a08f89-4663-40d2-971e-1360f8c7712f-istio-podinfo\") pod \"e2e-inf-a-2236bf-openshift-default-bf8bf7b95-sf8lf\" (UID: \"60a08f89-4663-40d2-971e-1360f8c7712f\") " pod="openshift-ingress/e2e-inf-a-2236bf-openshift-default-bf8bf7b95-sf8lf" Jul 06 03:13:11.050382 ip-10-0-139-206 kubenswrapper[2557]: I0706 03:13:11.050136 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/60a08f89-4663-40d2-971e-1360f8c7712f-istiod-ca-cert\") pod \"e2e-inf-a-2236bf-openshift-default-bf8bf7b95-sf8lf\" (UID: \"60a08f89-4663-40d2-971e-1360f8c7712f\") " pod="openshift-ingress/e2e-inf-a-2236bf-openshift-default-bf8bf7b95-sf8lf" Jul 06 03:13:11.050382 ip-10-0-139-206 kubenswrapper[2557]: I0706 03:13:11.050180 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/60a08f89-4663-40d2-971e-1360f8c7712f-workload-socket\") pod \"e2e-inf-a-2236bf-openshift-default-bf8bf7b95-sf8lf\" (UID: \"60a08f89-4663-40d2-971e-1360f8c7712f\") " pod="openshift-ingress/e2e-inf-a-2236bf-openshift-default-bf8bf7b95-sf8lf" Jul 06 03:13:11.050382 ip-10-0-139-206 kubenswrapper[2557]: I0706 03:13:11.050220 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/60a08f89-4663-40d2-971e-1360f8c7712f-istio-data\") pod \"e2e-inf-a-2236bf-openshift-default-bf8bf7b95-sf8lf\" (UID: \"60a08f89-4663-40d2-971e-1360f8c7712f\") " pod="openshift-ingress/e2e-inf-a-2236bf-openshift-default-bf8bf7b95-sf8lf" Jul 06 03:13:11.050382 ip-10-0-139-206 kubenswrapper[2557]: I0706 03:13:11.050250 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/60a08f89-4663-40d2-971e-1360f8c7712f-workload-certs\") pod \"e2e-inf-a-2236bf-openshift-default-bf8bf7b95-sf8lf\" (UID: \"60a08f89-4663-40d2-971e-1360f8c7712f\") " pod="openshift-ingress/e2e-inf-a-2236bf-openshift-default-bf8bf7b95-sf8lf" Jul 06 03:13:11.050382 ip-10-0-139-206 kubenswrapper[2557]: I0706 03:13:11.050281 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/60a08f89-4663-40d2-971e-1360f8c7712f-istio-envoy\") pod \"e2e-inf-a-2236bf-openshift-default-bf8bf7b95-sf8lf\" (UID: \"60a08f89-4663-40d2-971e-1360f8c7712f\") " pod="openshift-ingress/e2e-inf-a-2236bf-openshift-default-bf8bf7b95-sf8lf" Jul 06 03:13:11.050382 ip-10-0-139-206 kubenswrapper[2557]: I0706 03:13:11.050307 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zl555\" (UniqueName: \"kubernetes.io/projected/60a08f89-4663-40d2-971e-1360f8c7712f-kube-api-access-zl555\") pod \"e2e-inf-a-2236bf-openshift-default-bf8bf7b95-sf8lf\" (UID: \"60a08f89-4663-40d2-971e-1360f8c7712f\") " pod="openshift-ingress/e2e-inf-a-2236bf-openshift-default-bf8bf7b95-sf8lf" Jul 06 03:13:11.050689 ip-10-0-139-206 kubenswrapper[2557]: I0706 03:13:11.050390 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/60a08f89-4663-40d2-971e-1360f8c7712f-istio-token\") pod \"e2e-inf-a-2236bf-openshift-default-bf8bf7b95-sf8lf\" (UID: \"60a08f89-4663-40d2-971e-1360f8c7712f\") " pod="openshift-ingress/e2e-inf-a-2236bf-openshift-default-bf8bf7b95-sf8lf" Jul 06 03:13:11.050689 ip-10-0-139-206 kubenswrapper[2557]: I0706 03:13:11.050628 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/60a08f89-4663-40d2-971e-1360f8c7712f-credential-socket\") pod \"e2e-inf-a-2236bf-openshift-default-bf8bf7b95-sf8lf\" (UID: \"60a08f89-4663-40d2-971e-1360f8c7712f\") " pod="openshift-ingress/e2e-inf-a-2236bf-openshift-default-bf8bf7b95-sf8lf" Jul 06 03:13:11.050689 ip-10-0-139-206 kubenswrapper[2557]: I0706 03:13:11.050678 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/60a08f89-4663-40d2-971e-1360f8c7712f-workload-certs\") pod \"e2e-inf-a-2236bf-openshift-default-bf8bf7b95-sf8lf\" (UID: \"60a08f89-4663-40d2-971e-1360f8c7712f\") " pod="openshift-ingress/e2e-inf-a-2236bf-openshift-default-bf8bf7b95-sf8lf" Jul 06 03:13:11.050806 ip-10-0-139-206 kubenswrapper[2557]: I0706 03:13:11.050691 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/60a08f89-4663-40d2-971e-1360f8c7712f-istio-data\") pod \"e2e-inf-a-2236bf-openshift-default-bf8bf7b95-sf8lf\" (UID: \"60a08f89-4663-40d2-971e-1360f8c7712f\") " pod="openshift-ingress/e2e-inf-a-2236bf-openshift-default-bf8bf7b95-sf8lf" Jul 06 03:13:11.050806 ip-10-0-139-206 kubenswrapper[2557]: I0706 03:13:11.050722 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/60a08f89-4663-40d2-971e-1360f8c7712f-workload-socket\") pod \"e2e-inf-a-2236bf-openshift-default-bf8bf7b95-sf8lf\" (UID: \"60a08f89-4663-40d2-971e-1360f8c7712f\") " pod="openshift-ingress/e2e-inf-a-2236bf-openshift-default-bf8bf7b95-sf8lf" Jul 06 03:13:11.051177 ip-10-0-139-206 kubenswrapper[2557]: I0706 03:13:11.051159 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/60a08f89-4663-40d2-971e-1360f8c7712f-istiod-ca-cert\") pod \"e2e-inf-a-2236bf-openshift-default-bf8bf7b95-sf8lf\" (UID: \"60a08f89-4663-40d2-971e-1360f8c7712f\") " pod="openshift-ingress/e2e-inf-a-2236bf-openshift-default-bf8bf7b95-sf8lf" Jul 06 03:13:11.052662 ip-10-0-139-206 kubenswrapper[2557]: I0706 03:13:11.052645 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/60a08f89-4663-40d2-971e-1360f8c7712f-istio-envoy\") pod \"e2e-inf-a-2236bf-openshift-default-bf8bf7b95-sf8lf\" (UID: \"60a08f89-4663-40d2-971e-1360f8c7712f\") " pod="openshift-ingress/e2e-inf-a-2236bf-openshift-default-bf8bf7b95-sf8lf" Jul 06 03:13:11.052881 ip-10-0-139-206 kubenswrapper[2557]: I0706 03:13:11.052864 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/60a08f89-4663-40d2-971e-1360f8c7712f-istio-podinfo\") pod \"e2e-inf-a-2236bf-openshift-default-bf8bf7b95-sf8lf\" (UID: \"60a08f89-4663-40d2-971e-1360f8c7712f\") " pod="openshift-ingress/e2e-inf-a-2236bf-openshift-default-bf8bf7b95-sf8lf" Jul 06 03:13:11.063236 ip-10-0-139-206 kubenswrapper[2557]: I0706 03:13:11.063211 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/60a08f89-4663-40d2-971e-1360f8c7712f-istio-token\") pod \"e2e-inf-a-2236bf-openshift-default-bf8bf7b95-sf8lf\" (UID: \"60a08f89-4663-40d2-971e-1360f8c7712f\") " pod="openshift-ingress/e2e-inf-a-2236bf-openshift-default-bf8bf7b95-sf8lf" Jul 06 03:13:11.070067 ip-10-0-139-206 kubenswrapper[2557]: I0706 03:13:11.070042 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-zl555\" (UniqueName: \"kubernetes.io/projected/60a08f89-4663-40d2-971e-1360f8c7712f-kube-api-access-zl555\") pod \"e2e-inf-a-2236bf-openshift-default-bf8bf7b95-sf8lf\" (UID: \"60a08f89-4663-40d2-971e-1360f8c7712f\") " pod="openshift-ingress/e2e-inf-a-2236bf-openshift-default-bf8bf7b95-sf8lf" Jul 06 03:13:11.178146 ip-10-0-139-206 kubenswrapper[2557]: I0706 03:13:11.178049 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-2236bf-openshift-default-bf8bf7b95-sf8lf" Jul 06 03:13:11.328662 ip-10-0-139-206 kubenswrapper[2557]: I0706 03:13:11.328635 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-a-2236bf-openshift-default-bf8bf7b95-sf8lf"] Jul 06 03:13:11.331094 ip-10-0-139-206 kubenswrapper[2557]: W0706 03:13:11.331065 2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod60a08f89_4663_40d2_971e_1360f8c7712f.slice/crio-ad455e75d0afebf8ec068f2556380256c27b414d982d146f1e6340a4d2bd41b1 WatchSource:0}: Error finding container ad455e75d0afebf8ec068f2556380256c27b414d982d146f1e6340a4d2bd41b1: Status 404 returned error can't find the container with id ad455e75d0afebf8ec068f2556380256c27b414d982d146f1e6340a4d2bd41b1 Jul 06 03:13:11.333260 ip-10-0-139-206 kubenswrapper[2557]: I0706 03:13:11.333224 2557 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 06 03:13:11.333377 ip-10-0-139-206 kubenswrapper[2557]: I0706 03:13:11.333297 2557 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 06 03:13:11.333377 ip-10-0-139-206 kubenswrapper[2557]: I0706 03:13:11.333326 2557 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 06 03:13:12.021341 ip-10-0-139-206 kubenswrapper[2557]: I0706 03:13:12.021308 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-2236bf-openshift-default-bf8bf7b95-sf8lf" event={"ID":"60a08f89-4663-40d2-971e-1360f8c7712f","Type":"ContainerStarted","Data":"2836a1fdfaea33863207aab3c17daa157128e279d353784617aff4a79c8f9fa2"} Jul 06 03:13:12.021726 ip-10-0-139-206 kubenswrapper[2557]: I0706 03:13:12.021375 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-2236bf-openshift-default-bf8bf7b95-sf8lf" event={"ID":"60a08f89-4663-40d2-971e-1360f8c7712f","Type":"ContainerStarted","Data":"ad455e75d0afebf8ec068f2556380256c27b414d982d146f1e6340a4d2bd41b1"} Jul 06 03:13:12.042105 ip-10-0-139-206 kubenswrapper[2557]: I0706 03:13:12.042053 2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-inf-a-2236bf-openshift-default-bf8bf7b95-sf8lf" podStartSLOduration=2.042035759 podStartE2EDuration="2.042035759s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 03:13:10 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 03:13:12.039981233 +0000 UTC m=+2035.846267291" watchObservedRunningTime="2026-07-06 03:13:12.042035759 +0000 UTC m=+2035.848321795" Jul 06 03:13:12.178609 ip-10-0-139-206 kubenswrapper[2557]: I0706 03:13:12.178563 2557 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-a-2236bf-openshift-default-bf8bf7b95-sf8lf" Jul 06 03:13:12.183685 ip-10-0-139-206 kubenswrapper[2557]: I0706 03:13:12.183653 2557 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-a-2236bf-openshift-default-bf8bf7b95-sf8lf" Jul 06 03:13:13.026937 ip-10-0-139-206 kubenswrapper[2557]: I0706 03:13:13.026901 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-a-2236bf-openshift-default-bf8bf7b95-sf8lf" Jul 06 03:13:13.027837 ip-10-0-139-206 kubenswrapper[2557]: I0706 03:13:13.027818 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-a-2236bf-openshift-default-bf8bf7b95-sf8lf" Jul 06 03:13:21.866423 ip-10-0-139-206 kubenswrapper[2557]: I0706 03:13:21.866380 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-srzrr"] Jul 06 03:13:29.068519 ip-10-0-139-206 kubenswrapper[2557]: I0706 03:13:29.068470 2557 patch_prober.go:28] interesting pod/e2e-inf-a-2236bf-openshift-default-bf8bf7b95-sf8lf container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.133.0.33:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body= Jul 06 03:13:29.069192 ip-10-0-139-206 kubenswrapper[2557]: I0706 03:13:29.068565 2557 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-inf-a-2236bf-openshift-default-bf8bf7b95-sf8lf" podUID="60a08f89-4663-40d2-971e-1360f8c7712f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.33:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" Jul 06 03:13:30.106594 ip-10-0-139-206 kubenswrapper[2557]: I0706 03:13:30.106552 2557 generic.go:356] "Generic (PLEG): container finished" podID="60a08f89-4663-40d2-971e-1360f8c7712f" containerID="2836a1fdfaea33863207aab3c17daa157128e279d353784617aff4a79c8f9fa2" exitCode=0 Jul 06 03:13:30.107117 ip-10-0-139-206 kubenswrapper[2557]: I0706 03:13:30.106606 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-2236bf-openshift-default-bf8bf7b95-sf8lf" event={"ID":"60a08f89-4663-40d2-971e-1360f8c7712f","Type":"ContainerDied","Data":"2836a1fdfaea33863207aab3c17daa157128e279d353784617aff4a79c8f9fa2"} Jul 06 03:13:30.107117 ip-10-0-139-206 kubenswrapper[2557]: I0706 03:13:30.107018 2557 scope.go:117] "RemoveContainer" containerID="2836a1fdfaea33863207aab3c17daa157128e279d353784617aff4a79c8f9fa2" Jul 06 03:13:30.108175 ip-10-0-139-206 kubenswrapper[2557]: I0706 03:13:30.108145 2557 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 06 03:13:30.108246 ip-10-0-139-206 kubenswrapper[2557]: I0706 03:13:30.108229 2557 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 06 03:13:30.108298 ip-10-0-139-206 kubenswrapper[2557]: I0706 03:13:30.108280 2557 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 06 03:13:31.116794 ip-10-0-139-206 kubenswrapper[2557]: I0706 03:13:31.116753 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-2236bf-openshift-default-bf8bf7b95-sf8lf" event={"ID":"60a08f89-4663-40d2-971e-1360f8c7712f","Type":"ContainerStarted","Data":"d6198599dce011023b1179c44a06789938f289ac3a4630b07509d65db5006fe5"} Jul 06 03:13:31.178664 ip-10-0-139-206 kubenswrapper[2557]: I0706 03:13:31.178620 2557 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-a-2236bf-openshift-default-bf8bf7b95-sf8lf" Jul 06 03:13:31.178931 ip-10-0-139-206 kubenswrapper[2557]: I0706 03:13:31.178903 2557 patch_prober.go:28] interesting pod/e2e-inf-a-2236bf-openshift-default-bf8bf7b95-sf8lf container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.33:15021/healthz/ready\": dial tcp 10.133.0.33:15021: connect: connection refused" start-of-body= Jul 06 03:13:31.179000 ip-10-0-139-206 kubenswrapper[2557]: I0706 03:13:31.178962 2557 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-2236bf-openshift-default-bf8bf7b95-sf8lf" podUID="60a08f89-4663-40d2-971e-1360f8c7712f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.33:15021/healthz/ready\": dial tcp 10.133.0.33:15021: connect: connection refused" Jul 06 03:13:32.179311 ip-10-0-139-206 kubenswrapper[2557]: I0706 03:13:32.179276 2557 patch_prober.go:28] interesting pod/e2e-inf-a-2236bf-openshift-default-bf8bf7b95-sf8lf container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.33:15021/healthz/ready\": dial tcp 10.133.0.33:15021: connect: connection refused" start-of-body= Jul 06 03:13:32.179755 ip-10-0-139-206 kubenswrapper[2557]: I0706 03:13:32.179330 2557 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-2236bf-openshift-default-bf8bf7b95-sf8lf" podUID="60a08f89-4663-40d2-971e-1360f8c7712f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.33:15021/healthz/ready\": dial tcp 10.133.0.33:15021: connect: connection refused" Jul 06 03:13:33.179362 ip-10-0-139-206 kubenswrapper[2557]: I0706 03:13:33.179321 2557 patch_prober.go:28] interesting pod/e2e-inf-a-2236bf-openshift-default-bf8bf7b95-sf8lf container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.33:15021/healthz/ready\": dial tcp 10.133.0.33:15021: connect: connection refused" start-of-body= Jul 06 03:13:33.179750 ip-10-0-139-206 kubenswrapper[2557]: I0706 03:13:33.179411 2557 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-2236bf-openshift-default-bf8bf7b95-sf8lf" podUID="60a08f89-4663-40d2-971e-1360f8c7712f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.33:15021/healthz/ready\": dial tcp 10.133.0.33:15021: connect: connection refused" Jul 06 03:13:34.179286 ip-10-0-139-206 kubenswrapper[2557]: I0706 03:13:34.179243 2557 patch_prober.go:28] interesting pod/e2e-inf-a-2236bf-openshift-default-bf8bf7b95-sf8lf container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.33:15021/healthz/ready\": dial tcp 10.133.0.33:15021: connect: connection refused" start-of-body= Jul 06 03:13:34.179482 ip-10-0-139-206 kubenswrapper[2557]: I0706 03:13:34.179316 2557 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-2236bf-openshift-default-bf8bf7b95-sf8lf" podUID="60a08f89-4663-40d2-971e-1360f8c7712f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.33:15021/healthz/ready\": dial tcp 10.133.0.33:15021: connect: connection refused" Jul 06 03:13:35.169614 ip-10-0-139-206 kubenswrapper[2557]: I0706 03:13:35.169577 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-srzrr"] Jul 06 03:13:35.182645 ip-10-0-139-206 kubenswrapper[2557]: I0706 03:13:35.182617 2557 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-a-2236bf-openshift-default-bf8bf7b95-sf8lf" Jul 06 03:13:35.183019 ip-10-0-139-206 kubenswrapper[2557]: I0706 03:13:35.182870 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-a-2236bf-openshift-default-bf8bf7b95-sf8lf" Jul 06 03:13:35.183642 ip-10-0-139-206 kubenswrapper[2557]: I0706 03:13:35.183625 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-a-2236bf-openshift-default-bf8bf7b95-sf8lf" Jul 06 03:13:49.361664 ip-10-0-139-206 kubenswrapper[2557]: I0706 03:13:49.361575 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-srzrr"] Jul 06 03:14:13.470360 ip-10-0-139-206 kubenswrapper[2557]: I0706 03:14:13.470309 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-srzrr"] Jul 06 03:14:17.334357 ip-10-0-139-206 kubenswrapper[2557]: I0706 03:14:17.334325 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-cw2f7_efcf5e11-1e5d-4e2a-9b38-303a8f0787c4/ovn-acl-logging/0.log" Jul 06 03:14:17.338959 ip-10-0-139-206 kubenswrapper[2557]: I0706 03:14:17.338934 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-cw2f7_efcf5e11-1e5d-4e2a-9b38-303a8f0787c4/ovn-acl-logging/0.log" Jul 06 03:14:29.665097 ip-10-0-139-206 kubenswrapper[2557]: I0706 03:14:29.665058 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-srzrr"] Jul 06 03:14:41.358591 ip-10-0-139-206 kubenswrapper[2557]: I0706 03:14:41.358555 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-srzrr"] Jul 06 03:15:00.133907 ip-10-0-139-206 kubenswrapper[2557]: I0706 03:15:00.133872 2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-api-key-cleanup-29721795-mqpf9"] Jul 06 03:15:00.134324 ip-10-0-139-206 kubenswrapper[2557]: I0706 03:15:00.134205 2557 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4c18e7e2-bde7-407c-bf1d-8aa3884e69a6" containerName="istio-proxy" Jul 06 03:15:00.134324 ip-10-0-139-206 kubenswrapper[2557]: I0706 03:15:00.134215 2557 state_mem.go:107] "Deleted CPUSet assignment" podUID="4c18e7e2-bde7-407c-bf1d-8aa3884e69a6" containerName="istio-proxy" Jul 06 03:15:00.137197 ip-10-0-139-206 kubenswrapper[2557]: I0706 03:15:00.137176 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-key-cleanup-29721795-mqpf9" Jul 06 03:15:00.139451 ip-10-0-139-206 kubenswrapper[2557]: I0706 03:15:00.139427 2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-api-key-cleanup-e2e-inf-b-dd35c6-29721795-hhgnv"] Jul 06 03:15:00.139736 ip-10-0-139-206 kubenswrapper[2557]: I0706 03:15:00.139718 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-api-dockercfg-mg4mh\"" Jul 06 03:15:00.143403 ip-10-0-139-206 kubenswrapper[2557]: I0706 03:15:00.143384 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-key-cleanup-29721795-mqpf9"] Jul 06 03:15:00.143497 ip-10-0-139-206 kubenswrapper[2557]: I0706 03:15:00.143479 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-key-cleanup-e2e-inf-b-dd35c6-29721795-hhgnv" Jul 06 03:15:00.150202 ip-10-0-139-206 kubenswrapper[2557]: I0706 03:15:00.150178 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-key-cleanup-e2e-inf-b-dd35c6-29721795-hhgnv"] Jul 06 03:15:00.216416 ip-10-0-139-206 kubenswrapper[2557]: I0706 03:15:00.216375 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pkz5z\" (UniqueName: \"kubernetes.io/projected/64155502-de10-48c3-8654-991f9c66ba81-kube-api-access-pkz5z\") pod \"maas-api-key-cleanup-e2e-inf-b-dd35c6-29721795-hhgnv\" (UID: \"64155502-de10-48c3-8654-991f9c66ba81\") " pod="opendatahub/maas-api-key-cleanup-e2e-inf-b-dd35c6-29721795-hhgnv" Jul 06 03:15:00.216604 ip-10-0-139-206 kubenswrapper[2557]: I0706 03:15:00.216423 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-drr92\" (UniqueName: \"kubernetes.io/projected/956ad158-3f0d-4690-8ec2-8f6ea1815ca1-kube-api-access-drr92\") pod \"maas-api-key-cleanup-29721795-mqpf9\" (UID: \"956ad158-3f0d-4690-8ec2-8f6ea1815ca1\") " pod="opendatahub/maas-api-key-cleanup-29721795-mqpf9" Jul 06 03:15:00.317447 ip-10-0-139-206 kubenswrapper[2557]: I0706 03:15:00.317402 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pkz5z\" (UniqueName: \"kubernetes.io/projected/64155502-de10-48c3-8654-991f9c66ba81-kube-api-access-pkz5z\") pod \"maas-api-key-cleanup-e2e-inf-b-dd35c6-29721795-hhgnv\" (UID: \"64155502-de10-48c3-8654-991f9c66ba81\") " pod="opendatahub/maas-api-key-cleanup-e2e-inf-b-dd35c6-29721795-hhgnv" Jul 06 03:15:00.317447 ip-10-0-139-206 kubenswrapper[2557]: I0706 03:15:00.317447 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-drr92\" (UniqueName: \"kubernetes.io/projected/956ad158-3f0d-4690-8ec2-8f6ea1815ca1-kube-api-access-drr92\") pod \"maas-api-key-cleanup-29721795-mqpf9\" (UID: \"956ad158-3f0d-4690-8ec2-8f6ea1815ca1\") " pod="opendatahub/maas-api-key-cleanup-29721795-mqpf9" Jul 06 03:15:00.327231 ip-10-0-139-206 kubenswrapper[2557]: I0706 03:15:00.327199 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-drr92\" (UniqueName: \"kubernetes.io/projected/956ad158-3f0d-4690-8ec2-8f6ea1815ca1-kube-api-access-drr92\") pod \"maas-api-key-cleanup-29721795-mqpf9\" (UID: \"956ad158-3f0d-4690-8ec2-8f6ea1815ca1\") " pod="opendatahub/maas-api-key-cleanup-29721795-mqpf9" Jul 06 03:15:00.327366 ip-10-0-139-206 kubenswrapper[2557]: I0706 03:15:00.327306 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-pkz5z\" (UniqueName: \"kubernetes.io/projected/64155502-de10-48c3-8654-991f9c66ba81-kube-api-access-pkz5z\") pod \"maas-api-key-cleanup-e2e-inf-b-dd35c6-29721795-hhgnv\" (UID: \"64155502-de10-48c3-8654-991f9c66ba81\") " pod="opendatahub/maas-api-key-cleanup-e2e-inf-b-dd35c6-29721795-hhgnv" Jul 06 03:15:00.453432 ip-10-0-139-206 kubenswrapper[2557]: I0706 03:15:00.453400 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-key-cleanup-29721795-mqpf9" Jul 06 03:15:00.459431 ip-10-0-139-206 kubenswrapper[2557]: I0706 03:15:00.459412 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-key-cleanup-e2e-inf-b-dd35c6-29721795-hhgnv" Jul 06 03:15:00.590547 ip-10-0-139-206 kubenswrapper[2557]: I0706 03:15:00.590518 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-key-cleanup-29721795-mqpf9"] Jul 06 03:15:00.593592 ip-10-0-139-206 kubenswrapper[2557]: W0706 03:15:00.593545 2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod956ad158_3f0d_4690_8ec2_8f6ea1815ca1.slice/crio-899a42d394fa918a7c3c11d9855afe0c5e6199736c11e6b84f060848a814f4b5 WatchSource:0}: Error finding container 899a42d394fa918a7c3c11d9855afe0c5e6199736c11e6b84f060848a814f4b5: Status 404 returned error can't find the container with id 899a42d394fa918a7c3c11d9855afe0c5e6199736c11e6b84f060848a814f4b5 Jul 06 03:15:00.629489 ip-10-0-139-206 kubenswrapper[2557]: I0706 03:15:00.629463 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-key-cleanup-e2e-inf-b-dd35c6-29721795-hhgnv"] Jul 06 03:15:00.638575 ip-10-0-139-206 kubenswrapper[2557]: W0706 03:15:00.638550 2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod64155502_de10_48c3_8654_991f9c66ba81.slice/crio-d564d0597eafae7c7a828f8562009bc0a2b08b0c69940f0c3d91ed0f61b9ed8b WatchSource:0}: Error finding container d564d0597eafae7c7a828f8562009bc0a2b08b0c69940f0c3d91ed0f61b9ed8b: Status 404 returned error can't find the container with id d564d0597eafae7c7a828f8562009bc0a2b08b0c69940f0c3d91ed0f61b9ed8b Jul 06 03:15:01.537626 ip-10-0-139-206 kubenswrapper[2557]: I0706 03:15:01.537597 2557 generic.go:356] "Generic (PLEG): container finished" podID="64155502-de10-48c3-8654-991f9c66ba81" containerID="c2b6606a8c1f8980a662c09fe583962782c11493ef8b3029c738bba7934a512c" exitCode=0 Jul 06 03:15:01.538041 ip-10-0-139-206 kubenswrapper[2557]: I0706 03:15:01.537674 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-key-cleanup-e2e-inf-b-dd35c6-29721795-hhgnv" event={"ID":"64155502-de10-48c3-8654-991f9c66ba81","Type":"ContainerDied","Data":"c2b6606a8c1f8980a662c09fe583962782c11493ef8b3029c738bba7934a512c"} Jul 06 03:15:01.538041 ip-10-0-139-206 kubenswrapper[2557]: I0706 03:15:01.537727 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-key-cleanup-e2e-inf-b-dd35c6-29721795-hhgnv" event={"ID":"64155502-de10-48c3-8654-991f9c66ba81","Type":"ContainerStarted","Data":"d564d0597eafae7c7a828f8562009bc0a2b08b0c69940f0c3d91ed0f61b9ed8b"} Jul 06 03:15:01.539121 ip-10-0-139-206 kubenswrapper[2557]: I0706 03:15:01.539099 2557 generic.go:356] "Generic (PLEG): container finished" podID="956ad158-3f0d-4690-8ec2-8f6ea1815ca1" containerID="98c716f76e965c5b65af8d5ce4ebd02645a3dfd191e8f569d804ab929daaec6f" exitCode=0 Jul 06 03:15:01.539227 ip-10-0-139-206 kubenswrapper[2557]: I0706 03:15:01.539144 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-key-cleanup-29721795-mqpf9" event={"ID":"956ad158-3f0d-4690-8ec2-8f6ea1815ca1","Type":"ContainerDied","Data":"98c716f76e965c5b65af8d5ce4ebd02645a3dfd191e8f569d804ab929daaec6f"} Jul 06 03:15:01.539227 ip-10-0-139-206 kubenswrapper[2557]: I0706 03:15:01.539180 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-key-cleanup-29721795-mqpf9" event={"ID":"956ad158-3f0d-4690-8ec2-8f6ea1815ca1","Type":"ContainerStarted","Data":"899a42d394fa918a7c3c11d9855afe0c5e6199736c11e6b84f060848a814f4b5"} Jul 06 03:15:02.695619 ip-10-0-139-206 kubenswrapper[2557]: I0706 03:15:02.695591 2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-key-cleanup-e2e-inf-b-dd35c6-29721795-hhgnv" Jul 06 03:15:02.698269 ip-10-0-139-206 kubenswrapper[2557]: I0706 03:15:02.698247 2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-key-cleanup-29721795-mqpf9" Jul 06 03:15:02.738172 ip-10-0-139-206 kubenswrapper[2557]: I0706 03:15:02.738137 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-pkz5z\" (UniqueName: \"kubernetes.io/projected/64155502-de10-48c3-8654-991f9c66ba81-kube-api-access-pkz5z\") pod \"64155502-de10-48c3-8654-991f9c66ba81\" (UID: \"64155502-de10-48c3-8654-991f9c66ba81\") " Jul 06 03:15:02.740272 ip-10-0-139-206 kubenswrapper[2557]: I0706 03:15:02.740237 2557 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/64155502-de10-48c3-8654-991f9c66ba81-kube-api-access-pkz5z" (OuterVolumeSpecName: "kube-api-access-pkz5z") pod "64155502-de10-48c3-8654-991f9c66ba81" (UID: "64155502-de10-48c3-8654-991f9c66ba81"). InnerVolumeSpecName "kube-api-access-pkz5z". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 06 03:15:02.839821 ip-10-0-139-206 kubenswrapper[2557]: I0706 03:15:02.839721 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-drr92\" (UniqueName: \"kubernetes.io/projected/956ad158-3f0d-4690-8ec2-8f6ea1815ca1-kube-api-access-drr92\") pod \"956ad158-3f0d-4690-8ec2-8f6ea1815ca1\" (UID: \"956ad158-3f0d-4690-8ec2-8f6ea1815ca1\") " Jul 06 03:15:02.839971 ip-10-0-139-206 kubenswrapper[2557]: I0706 03:15:02.839931 2557 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-pkz5z\" (UniqueName: \"kubernetes.io/projected/64155502-de10-48c3-8654-991f9c66ba81-kube-api-access-pkz5z\") on node \"ip-10-0-139-206.ec2.internal\" DevicePath \"\"" Jul 06 03:15:02.841845 ip-10-0-139-206 kubenswrapper[2557]: I0706 03:15:02.841815 2557 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/956ad158-3f0d-4690-8ec2-8f6ea1815ca1-kube-api-access-drr92" (OuterVolumeSpecName: "kube-api-access-drr92") pod "956ad158-3f0d-4690-8ec2-8f6ea1815ca1" (UID: "956ad158-3f0d-4690-8ec2-8f6ea1815ca1"). InnerVolumeSpecName "kube-api-access-drr92". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 06 03:15:02.940426 ip-10-0-139-206 kubenswrapper[2557]: I0706 03:15:02.940381 2557 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-drr92\" (UniqueName: \"kubernetes.io/projected/956ad158-3f0d-4690-8ec2-8f6ea1815ca1-kube-api-access-drr92\") on node \"ip-10-0-139-206.ec2.internal\" DevicePath \"\"" Jul 06 03:15:03.549580 ip-10-0-139-206 kubenswrapper[2557]: I0706 03:15:03.549550 2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-key-cleanup-e2e-inf-b-dd35c6-29721795-hhgnv" Jul 06 03:15:03.549580 ip-10-0-139-206 kubenswrapper[2557]: I0706 03:15:03.549560 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-key-cleanup-e2e-inf-b-dd35c6-29721795-hhgnv" event={"ID":"64155502-de10-48c3-8654-991f9c66ba81","Type":"ContainerDied","Data":"d564d0597eafae7c7a828f8562009bc0a2b08b0c69940f0c3d91ed0f61b9ed8b"} Jul 06 03:15:03.549812 ip-10-0-139-206 kubenswrapper[2557]: I0706 03:15:03.549613 2557 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="d564d0597eafae7c7a828f8562009bc0a2b08b0c69940f0c3d91ed0f61b9ed8b" Jul 06 03:15:03.551000 ip-10-0-139-206 kubenswrapper[2557]: I0706 03:15:03.550972 2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-key-cleanup-29721795-mqpf9" Jul 06 03:15:03.551000 ip-10-0-139-206 kubenswrapper[2557]: I0706 03:15:03.550984 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-key-cleanup-29721795-mqpf9" event={"ID":"956ad158-3f0d-4690-8ec2-8f6ea1815ca1","Type":"ContainerDied","Data":"899a42d394fa918a7c3c11d9855afe0c5e6199736c11e6b84f060848a814f4b5"} Jul 06 03:15:03.551205 ip-10-0-139-206 kubenswrapper[2557]: I0706 03:15:03.551019 2557 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="899a42d394fa918a7c3c11d9855afe0c5e6199736c11e6b84f060848a814f4b5" Jul 06 03:15:13.993017 ip-10-0-139-206 kubenswrapper[2557]: I0706 03:15:13.992926 2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-a-2236bf-openshift-default-bf8bf7b95-sf8lf"] Jul 06 03:15:13.993518 ip-10-0-139-206 kubenswrapper[2557]: I0706 03:15:13.993170 2557 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-ingress/e2e-inf-a-2236bf-openshift-default-bf8bf7b95-sf8lf" podUID="60a08f89-4663-40d2-971e-1360f8c7712f" containerName="istio-proxy" containerID="cri-o://d6198599dce011023b1179c44a06789938f289ac3a4630b07509d65db5006fe5" gracePeriod=30 Jul 06 03:15:15.145804 ip-10-0-139-206 kubenswrapper[2557]: I0706 03:15:15.145764 2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-api-key-cleanup-e2e-inf-b-dd35c6-29721795-hhgnv"] Jul 06 03:15:15.152371 ip-10-0-139-206 kubenswrapper[2557]: I0706 03:15:15.152314 2557 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-api-key-cleanup-e2e-inf-b-dd35c6-29721795-hhgnv"] Jul 06 03:15:15.302296 ip-10-0-139-206 kubenswrapper[2557]: I0706 03:15:15.302263 2557 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="64155502-de10-48c3-8654-991f9c66ba81" path="/var/lib/kubelet/pods/64155502-de10-48c3-8654-991f9c66ba81/volumes" Jul 06 03:15:18.267639 ip-10-0-139-206 kubenswrapper[2557]: I0706 03:15:18.267599 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-srzrr"] Jul 06 03:15:19.253872 ip-10-0-139-206 kubenswrapper[2557]: I0706 03:15:19.253808 2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-2236bf-openshift-default-bf8bf7b95-sf8lf" Jul 06 03:15:19.386622 ip-10-0-139-206 kubenswrapper[2557]: I0706 03:15:19.386589 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/60a08f89-4663-40d2-971e-1360f8c7712f-istio-podinfo\") pod \"60a08f89-4663-40d2-971e-1360f8c7712f\" (UID: \"60a08f89-4663-40d2-971e-1360f8c7712f\") " Jul 06 03:15:19.386622 ip-10-0-139-206 kubenswrapper[2557]: I0706 03:15:19.386638 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/60a08f89-4663-40d2-971e-1360f8c7712f-istio-envoy\") pod \"60a08f89-4663-40d2-971e-1360f8c7712f\" (UID: \"60a08f89-4663-40d2-971e-1360f8c7712f\") " Jul 06 03:15:19.387108 ip-10-0-139-206 kubenswrapper[2557]: I0706 03:15:19.386667 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/60a08f89-4663-40d2-971e-1360f8c7712f-istio-data\") pod \"60a08f89-4663-40d2-971e-1360f8c7712f\" (UID: \"60a08f89-4663-40d2-971e-1360f8c7712f\") " Jul 06 03:15:19.387108 ip-10-0-139-206 kubenswrapper[2557]: I0706 03:15:19.386690 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/60a08f89-4663-40d2-971e-1360f8c7712f-istiod-ca-cert\") pod \"60a08f89-4663-40d2-971e-1360f8c7712f\" (UID: \"60a08f89-4663-40d2-971e-1360f8c7712f\") " Jul 06 03:15:19.387108 ip-10-0-139-206 kubenswrapper[2557]: I0706 03:15:19.386861 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/60a08f89-4663-40d2-971e-1360f8c7712f-workload-certs\") pod \"60a08f89-4663-40d2-971e-1360f8c7712f\" (UID: \"60a08f89-4663-40d2-971e-1360f8c7712f\") " Jul 06 03:15:19.387108 ip-10-0-139-206 kubenswrapper[2557]: I0706 03:15:19.386945 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-zl555\" (UniqueName: \"kubernetes.io/projected/60a08f89-4663-40d2-971e-1360f8c7712f-kube-api-access-zl555\") pod \"60a08f89-4663-40d2-971e-1360f8c7712f\" (UID: \"60a08f89-4663-40d2-971e-1360f8c7712f\") " Jul 06 03:15:19.387108 ip-10-0-139-206 kubenswrapper[2557]: I0706 03:15:19.386991 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/60a08f89-4663-40d2-971e-1360f8c7712f-workload-socket\") pod \"60a08f89-4663-40d2-971e-1360f8c7712f\" (UID: \"60a08f89-4663-40d2-971e-1360f8c7712f\") " Jul 06 03:15:19.387108 ip-10-0-139-206 kubenswrapper[2557]: I0706 03:15:19.387038 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/60a08f89-4663-40d2-971e-1360f8c7712f-istio-token\") pod \"60a08f89-4663-40d2-971e-1360f8c7712f\" (UID: \"60a08f89-4663-40d2-971e-1360f8c7712f\") " Jul 06 03:15:19.387108 ip-10-0-139-206 kubenswrapper[2557]: I0706 03:15:19.387060 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/60a08f89-4663-40d2-971e-1360f8c7712f-credential-socket\") pod \"60a08f89-4663-40d2-971e-1360f8c7712f\" (UID: \"60a08f89-4663-40d2-971e-1360f8c7712f\") " Jul 06 03:15:19.387108 ip-10-0-139-206 kubenswrapper[2557]: I0706 03:15:19.387097 2557 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/60a08f89-4663-40d2-971e-1360f8c7712f-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "60a08f89-4663-40d2-971e-1360f8c7712f" (UID: "60a08f89-4663-40d2-971e-1360f8c7712f"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 03:15:19.387108 ip-10-0-139-206 kubenswrapper[2557]: I0706 03:15:19.387110 2557 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/60a08f89-4663-40d2-971e-1360f8c7712f-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "60a08f89-4663-40d2-971e-1360f8c7712f" (UID: "60a08f89-4663-40d2-971e-1360f8c7712f"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 06 03:15:19.387622 ip-10-0-139-206 kubenswrapper[2557]: I0706 03:15:19.387333 2557 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/60a08f89-4663-40d2-971e-1360f8c7712f-workload-certs\") on node \"ip-10-0-139-206.ec2.internal\" DevicePath \"\"" Jul 06 03:15:19.387622 ip-10-0-139-206 kubenswrapper[2557]: I0706 03:15:19.387384 2557 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/60a08f89-4663-40d2-971e-1360f8c7712f-istiod-ca-cert\") on node \"ip-10-0-139-206.ec2.internal\" DevicePath \"\"" Jul 06 03:15:19.387622 ip-10-0-139-206 kubenswrapper[2557]: I0706 03:15:19.387385 2557 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/60a08f89-4663-40d2-971e-1360f8c7712f-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "60a08f89-4663-40d2-971e-1360f8c7712f" (UID: "60a08f89-4663-40d2-971e-1360f8c7712f"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 03:15:19.387622 ip-10-0-139-206 kubenswrapper[2557]: I0706 03:15:19.387547 2557 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/60a08f89-4663-40d2-971e-1360f8c7712f-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "60a08f89-4663-40d2-971e-1360f8c7712f" (UID: "60a08f89-4663-40d2-971e-1360f8c7712f"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 03:15:19.387829 ip-10-0-139-206 kubenswrapper[2557]: I0706 03:15:19.387683 2557 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/60a08f89-4663-40d2-971e-1360f8c7712f-istio-data" (OuterVolumeSpecName: "istio-data") pod "60a08f89-4663-40d2-971e-1360f8c7712f" (UID: "60a08f89-4663-40d2-971e-1360f8c7712f"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 03:15:19.389036 ip-10-0-139-206 kubenswrapper[2557]: I0706 03:15:19.389014 2557 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/60a08f89-4663-40d2-971e-1360f8c7712f-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "60a08f89-4663-40d2-971e-1360f8c7712f" (UID: "60a08f89-4663-40d2-971e-1360f8c7712f"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 06 03:15:19.389145 ip-10-0-139-206 kubenswrapper[2557]: I0706 03:15:19.389014 2557 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/60a08f89-4663-40d2-971e-1360f8c7712f-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "60a08f89-4663-40d2-971e-1360f8c7712f" (UID: "60a08f89-4663-40d2-971e-1360f8c7712f"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 03:15:19.389145 ip-10-0-139-206 kubenswrapper[2557]: I0706 03:15:19.389049 2557 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/60a08f89-4663-40d2-971e-1360f8c7712f-istio-token" (OuterVolumeSpecName: "istio-token") pod "60a08f89-4663-40d2-971e-1360f8c7712f" (UID: "60a08f89-4663-40d2-971e-1360f8c7712f"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 06 03:15:19.389224 ip-10-0-139-206 kubenswrapper[2557]: I0706 03:15:19.389196 2557 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/60a08f89-4663-40d2-971e-1360f8c7712f-kube-api-access-zl555" (OuterVolumeSpecName: "kube-api-access-zl555") pod "60a08f89-4663-40d2-971e-1360f8c7712f" (UID: "60a08f89-4663-40d2-971e-1360f8c7712f"). InnerVolumeSpecName "kube-api-access-zl555". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 06 03:15:19.488263 ip-10-0-139-206 kubenswrapper[2557]: I0706 03:15:19.488213 2557 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/60a08f89-4663-40d2-971e-1360f8c7712f-istio-podinfo\") on node \"ip-10-0-139-206.ec2.internal\" DevicePath \"\"" Jul 06 03:15:19.488263 ip-10-0-139-206 kubenswrapper[2557]: I0706 03:15:19.488256 2557 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/60a08f89-4663-40d2-971e-1360f8c7712f-istio-envoy\") on node \"ip-10-0-139-206.ec2.internal\" DevicePath \"\"" Jul 06 03:15:19.488263 ip-10-0-139-206 kubenswrapper[2557]: I0706 03:15:19.488265 2557 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/60a08f89-4663-40d2-971e-1360f8c7712f-istio-data\") on node \"ip-10-0-139-206.ec2.internal\" DevicePath \"\"" Jul 06 03:15:19.488263 ip-10-0-139-206 kubenswrapper[2557]: I0706 03:15:19.488273 2557 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-zl555\" (UniqueName: \"kubernetes.io/projected/60a08f89-4663-40d2-971e-1360f8c7712f-kube-api-access-zl555\") on node \"ip-10-0-139-206.ec2.internal\" DevicePath \"\"" Jul 06 03:15:19.488578 ip-10-0-139-206 kubenswrapper[2557]: I0706 03:15:19.488283 2557 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/60a08f89-4663-40d2-971e-1360f8c7712f-workload-socket\") on node \"ip-10-0-139-206.ec2.internal\" DevicePath \"\"" Jul 06 03:15:19.488578 ip-10-0-139-206 kubenswrapper[2557]: I0706 03:15:19.488292 2557 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/60a08f89-4663-40d2-971e-1360f8c7712f-istio-token\") on node \"ip-10-0-139-206.ec2.internal\" DevicePath \"\"" Jul 06 03:15:19.488578 ip-10-0-139-206 kubenswrapper[2557]: I0706 03:15:19.488301 2557 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/60a08f89-4663-40d2-971e-1360f8c7712f-credential-socket\") on node \"ip-10-0-139-206.ec2.internal\" DevicePath \"\"" Jul 06 03:15:19.630767 ip-10-0-139-206 kubenswrapper[2557]: I0706 03:15:19.630667 2557 generic.go:356] "Generic (PLEG): container finished" podID="60a08f89-4663-40d2-971e-1360f8c7712f" containerID="d6198599dce011023b1179c44a06789938f289ac3a4630b07509d65db5006fe5" exitCode=0 Jul 06 03:15:19.630767 ip-10-0-139-206 kubenswrapper[2557]: I0706 03:15:19.630724 2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-2236bf-openshift-default-bf8bf7b95-sf8lf" Jul 06 03:15:19.630767 ip-10-0-139-206 kubenswrapper[2557]: I0706 03:15:19.630744 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-2236bf-openshift-default-bf8bf7b95-sf8lf" event={"ID":"60a08f89-4663-40d2-971e-1360f8c7712f","Type":"ContainerDied","Data":"d6198599dce011023b1179c44a06789938f289ac3a4630b07509d65db5006fe5"} Jul 06 03:15:19.631048 ip-10-0-139-206 kubenswrapper[2557]: I0706 03:15:19.630792 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-2236bf-openshift-default-bf8bf7b95-sf8lf" event={"ID":"60a08f89-4663-40d2-971e-1360f8c7712f","Type":"ContainerDied","Data":"ad455e75d0afebf8ec068f2556380256c27b414d982d146f1e6340a4d2bd41b1"} Jul 06 03:15:19.631048 ip-10-0-139-206 kubenswrapper[2557]: I0706 03:15:19.630816 2557 scope.go:117] "RemoveContainer" containerID="d6198599dce011023b1179c44a06789938f289ac3a4630b07509d65db5006fe5" Jul 06 03:15:19.642600 ip-10-0-139-206 kubenswrapper[2557]: I0706 03:15:19.642580 2557 scope.go:117] "RemoveContainer" containerID="2836a1fdfaea33863207aab3c17daa157128e279d353784617aff4a79c8f9fa2" Jul 06 03:15:19.651597 ip-10-0-139-206 kubenswrapper[2557]: I0706 03:15:19.651567 2557 scope.go:117] "RemoveContainer" containerID="d6198599dce011023b1179c44a06789938f289ac3a4630b07509d65db5006fe5" Jul 06 03:15:19.652323 ip-10-0-139-206 kubenswrapper[2557]: E0706 03:15:19.652059 2557 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d6198599dce011023b1179c44a06789938f289ac3a4630b07509d65db5006fe5\": container with ID starting with d6198599dce011023b1179c44a06789938f289ac3a4630b07509d65db5006fe5 not found: ID does not exist" containerID="d6198599dce011023b1179c44a06789938f289ac3a4630b07509d65db5006fe5" Jul 06 03:15:19.652323 ip-10-0-139-206 kubenswrapper[2557]: I0706 03:15:19.652099 2557 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d6198599dce011023b1179c44a06789938f289ac3a4630b07509d65db5006fe5"} err="failed to get container status \"d6198599dce011023b1179c44a06789938f289ac3a4630b07509d65db5006fe5\": rpc error: code = NotFound desc = could not find container \"d6198599dce011023b1179c44a06789938f289ac3a4630b07509d65db5006fe5\": container with ID starting with d6198599dce011023b1179c44a06789938f289ac3a4630b07509d65db5006fe5 not found: ID does not exist" Jul 06 03:15:19.652323 ip-10-0-139-206 kubenswrapper[2557]: I0706 03:15:19.652126 2557 scope.go:117] "RemoveContainer" containerID="2836a1fdfaea33863207aab3c17daa157128e279d353784617aff4a79c8f9fa2" Jul 06 03:15:19.652568 ip-10-0-139-206 kubenswrapper[2557]: E0706 03:15:19.652468 2557 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2836a1fdfaea33863207aab3c17daa157128e279d353784617aff4a79c8f9fa2\": container with ID starting with 2836a1fdfaea33863207aab3c17daa157128e279d353784617aff4a79c8f9fa2 not found: ID does not exist" containerID="2836a1fdfaea33863207aab3c17daa157128e279d353784617aff4a79c8f9fa2" Jul 06 03:15:19.652568 ip-10-0-139-206 kubenswrapper[2557]: I0706 03:15:19.652496 2557 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2836a1fdfaea33863207aab3c17daa157128e279d353784617aff4a79c8f9fa2"} err="failed to get container status \"2836a1fdfaea33863207aab3c17daa157128e279d353784617aff4a79c8f9fa2\": rpc error: code = NotFound desc = could not find container \"2836a1fdfaea33863207aab3c17daa157128e279d353784617aff4a79c8f9fa2\": container with ID starting with 2836a1fdfaea33863207aab3c17daa157128e279d353784617aff4a79c8f9fa2 not found: ID does not exist" Jul 06 03:15:19.653841 ip-10-0-139-206 kubenswrapper[2557]: I0706 03:15:19.653819 2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-a-2236bf-openshift-default-bf8bf7b95-sf8lf"] Jul 06 03:15:19.657040 ip-10-0-139-206 kubenswrapper[2557]: I0706 03:15:19.657017 2557 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-inf-a-2236bf-openshift-default-bf8bf7b95-sf8lf"] Jul 06 03:15:21.302035 ip-10-0-139-206 kubenswrapper[2557]: I0706 03:15:21.302004 2557 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="60a08f89-4663-40d2-971e-1360f8c7712f" path="/var/lib/kubelet/pods/60a08f89-4663-40d2-971e-1360f8c7712f/volumes" Jul 06 03:15:21.642305 ip-10-0-139-206 kubenswrapper[2557]: I0706 03:15:21.642234 2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-api-a-a20636-openshift-default-5f46b5b487-fzp4k"] Jul 06 03:15:21.642571 ip-10-0-139-206 kubenswrapper[2557]: I0706 03:15:21.642557 2557 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="60a08f89-4663-40d2-971e-1360f8c7712f" containerName="istio-proxy" Jul 06 03:15:21.642623 ip-10-0-139-206 kubenswrapper[2557]: I0706 03:15:21.642573 2557 state_mem.go:107] "Deleted CPUSet assignment" podUID="60a08f89-4663-40d2-971e-1360f8c7712f" containerName="istio-proxy" Jul 06 03:15:21.642623 ip-10-0-139-206 kubenswrapper[2557]: I0706 03:15:21.642580 2557 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="60a08f89-4663-40d2-971e-1360f8c7712f" containerName="istio-proxy" Jul 06 03:15:21.642623 ip-10-0-139-206 kubenswrapper[2557]: I0706 03:15:21.642586 2557 state_mem.go:107] "Deleted CPUSet assignment" podUID="60a08f89-4663-40d2-971e-1360f8c7712f" containerName="istio-proxy" Jul 06 03:15:21.642623 ip-10-0-139-206 kubenswrapper[2557]: I0706 03:15:21.642597 2557 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="64155502-de10-48c3-8654-991f9c66ba81" containerName="cleanup" Jul 06 03:15:21.642623 ip-10-0-139-206 kubenswrapper[2557]: I0706 03:15:21.642603 2557 state_mem.go:107] "Deleted CPUSet assignment" podUID="64155502-de10-48c3-8654-991f9c66ba81" containerName="cleanup" Jul 06 03:15:21.642623 ip-10-0-139-206 kubenswrapper[2557]: I0706 03:15:21.642616 2557 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="956ad158-3f0d-4690-8ec2-8f6ea1815ca1" containerName="cleanup" Jul 06 03:15:21.642623 ip-10-0-139-206 kubenswrapper[2557]: I0706 03:15:21.642621 2557 state_mem.go:107] "Deleted CPUSet assignment" podUID="956ad158-3f0d-4690-8ec2-8f6ea1815ca1" containerName="cleanup" Jul 06 03:15:21.642822 ip-10-0-139-206 kubenswrapper[2557]: I0706 03:15:21.642667 2557 memory_manager.go:368] "RemoveStaleState removing state" podUID="60a08f89-4663-40d2-971e-1360f8c7712f" containerName="istio-proxy" Jul 06 03:15:21.642822 ip-10-0-139-206 kubenswrapper[2557]: I0706 03:15:21.642676 2557 memory_manager.go:368] "RemoveStaleState removing state" podUID="64155502-de10-48c3-8654-991f9c66ba81" containerName="cleanup" Jul 06 03:15:21.642822 ip-10-0-139-206 kubenswrapper[2557]: I0706 03:15:21.642683 2557 memory_manager.go:368] "RemoveStaleState removing state" podUID="956ad158-3f0d-4690-8ec2-8f6ea1815ca1" containerName="cleanup" Jul 06 03:15:21.645544 ip-10-0-139-206 kubenswrapper[2557]: I0706 03:15:21.645527 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-a20636-openshift-default-5f46b5b487-fzp4k" Jul 06 03:15:21.647833 ip-10-0-139-206 kubenswrapper[2557]: I0706 03:15:21.647809 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-api-a-a20636-openshift-default-dockercfg-nv8tq\"" Jul 06 03:15:21.656141 ip-10-0-139-206 kubenswrapper[2557]: I0706 03:15:21.656115 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-a-a20636-openshift-default-5f46b5b487-fzp4k"] Jul 06 03:15:21.809693 ip-10-0-139-206 kubenswrapper[2557]: I0706 03:15:21.809654 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1f1dc8e1-28c7-4004-bf18-49a9ce850c0a-istio-token\") pod \"e2e-api-a-a20636-openshift-default-5f46b5b487-fzp4k\" (UID: \"1f1dc8e1-28c7-4004-bf18-49a9ce850c0a\") " pod="openshift-ingress/e2e-api-a-a20636-openshift-default-5f46b5b487-fzp4k" Jul 06 03:15:21.809874 ip-10-0-139-206 kubenswrapper[2557]: I0706 03:15:21.809716 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1f1dc8e1-28c7-4004-bf18-49a9ce850c0a-istio-podinfo\") pod \"e2e-api-a-a20636-openshift-default-5f46b5b487-fzp4k\" (UID: \"1f1dc8e1-28c7-4004-bf18-49a9ce850c0a\") " pod="openshift-ingress/e2e-api-a-a20636-openshift-default-5f46b5b487-fzp4k" Jul 06 03:15:21.809874 ip-10-0-139-206 kubenswrapper[2557]: I0706 03:15:21.809793 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1f1dc8e1-28c7-4004-bf18-49a9ce850c0a-workload-certs\") pod \"e2e-api-a-a20636-openshift-default-5f46b5b487-fzp4k\" (UID: \"1f1dc8e1-28c7-4004-bf18-49a9ce850c0a\") " pod="openshift-ingress/e2e-api-a-a20636-openshift-default-5f46b5b487-fzp4k" Jul 06 03:15:21.809874 ip-10-0-139-206 kubenswrapper[2557]: I0706 03:15:21.809848 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1f1dc8e1-28c7-4004-bf18-49a9ce850c0a-workload-socket\") pod \"e2e-api-a-a20636-openshift-default-5f46b5b487-fzp4k\" (UID: \"1f1dc8e1-28c7-4004-bf18-49a9ce850c0a\") " pod="openshift-ingress/e2e-api-a-a20636-openshift-default-5f46b5b487-fzp4k" Jul 06 03:15:21.809982 ip-10-0-139-206 kubenswrapper[2557]: I0706 03:15:21.809882 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1f1dc8e1-28c7-4004-bf18-49a9ce850c0a-credential-socket\") pod \"e2e-api-a-a20636-openshift-default-5f46b5b487-fzp4k\" (UID: \"1f1dc8e1-28c7-4004-bf18-49a9ce850c0a\") " pod="openshift-ingress/e2e-api-a-a20636-openshift-default-5f46b5b487-fzp4k" Jul 06 03:15:21.810028 ip-10-0-139-206 kubenswrapper[2557]: I0706 03:15:21.809991 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1f1dc8e1-28c7-4004-bf18-49a9ce850c0a-istio-envoy\") pod \"e2e-api-a-a20636-openshift-default-5f46b5b487-fzp4k\" (UID: \"1f1dc8e1-28c7-4004-bf18-49a9ce850c0a\") " pod="openshift-ingress/e2e-api-a-a20636-openshift-default-5f46b5b487-fzp4k" Jul 06 03:15:21.810062 ip-10-0-139-206 kubenswrapper[2557]: I0706 03:15:21.810040 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tzflw\" (UniqueName: \"kubernetes.io/projected/1f1dc8e1-28c7-4004-bf18-49a9ce850c0a-kube-api-access-tzflw\") pod \"e2e-api-a-a20636-openshift-default-5f46b5b487-fzp4k\" (UID: \"1f1dc8e1-28c7-4004-bf18-49a9ce850c0a\") " pod="openshift-ingress/e2e-api-a-a20636-openshift-default-5f46b5b487-fzp4k" Jul 06 03:15:21.810109 ip-10-0-139-206 kubenswrapper[2557]: I0706 03:15:21.810072 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1f1dc8e1-28c7-4004-bf18-49a9ce850c0a-istio-data\") pod \"e2e-api-a-a20636-openshift-default-5f46b5b487-fzp4k\" (UID: \"1f1dc8e1-28c7-4004-bf18-49a9ce850c0a\") " pod="openshift-ingress/e2e-api-a-a20636-openshift-default-5f46b5b487-fzp4k" Jul 06 03:15:21.810109 ip-10-0-139-206 kubenswrapper[2557]: I0706 03:15:21.810098 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1f1dc8e1-28c7-4004-bf18-49a9ce850c0a-istiod-ca-cert\") pod \"e2e-api-a-a20636-openshift-default-5f46b5b487-fzp4k\" (UID: \"1f1dc8e1-28c7-4004-bf18-49a9ce850c0a\") " pod="openshift-ingress/e2e-api-a-a20636-openshift-default-5f46b5b487-fzp4k" Jul 06 03:15:21.910589 ip-10-0-139-206 kubenswrapper[2557]: I0706 03:15:21.910504 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1f1dc8e1-28c7-4004-bf18-49a9ce850c0a-istio-data\") pod \"e2e-api-a-a20636-openshift-default-5f46b5b487-fzp4k\" (UID: \"1f1dc8e1-28c7-4004-bf18-49a9ce850c0a\") " pod="openshift-ingress/e2e-api-a-a20636-openshift-default-5f46b5b487-fzp4k" Jul 06 03:15:21.910589 ip-10-0-139-206 kubenswrapper[2557]: I0706 03:15:21.910541 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1f1dc8e1-28c7-4004-bf18-49a9ce850c0a-istiod-ca-cert\") pod \"e2e-api-a-a20636-openshift-default-5f46b5b487-fzp4k\" (UID: \"1f1dc8e1-28c7-4004-bf18-49a9ce850c0a\") " pod="openshift-ingress/e2e-api-a-a20636-openshift-default-5f46b5b487-fzp4k" Jul 06 03:15:21.910589 ip-10-0-139-206 kubenswrapper[2557]: I0706 03:15:21.910591 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1f1dc8e1-28c7-4004-bf18-49a9ce850c0a-istio-token\") pod \"e2e-api-a-a20636-openshift-default-5f46b5b487-fzp4k\" (UID: \"1f1dc8e1-28c7-4004-bf18-49a9ce850c0a\") " pod="openshift-ingress/e2e-api-a-a20636-openshift-default-5f46b5b487-fzp4k" Jul 06 03:15:21.910896 ip-10-0-139-206 kubenswrapper[2557]: I0706 03:15:21.910618 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1f1dc8e1-28c7-4004-bf18-49a9ce850c0a-istio-podinfo\") pod \"e2e-api-a-a20636-openshift-default-5f46b5b487-fzp4k\" (UID: \"1f1dc8e1-28c7-4004-bf18-49a9ce850c0a\") " pod="openshift-ingress/e2e-api-a-a20636-openshift-default-5f46b5b487-fzp4k" Jul 06 03:15:21.910896 ip-10-0-139-206 kubenswrapper[2557]: I0706 03:15:21.910645 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1f1dc8e1-28c7-4004-bf18-49a9ce850c0a-workload-certs\") pod \"e2e-api-a-a20636-openshift-default-5f46b5b487-fzp4k\" (UID: \"1f1dc8e1-28c7-4004-bf18-49a9ce850c0a\") " pod="openshift-ingress/e2e-api-a-a20636-openshift-default-5f46b5b487-fzp4k" Jul 06 03:15:21.910896 ip-10-0-139-206 kubenswrapper[2557]: I0706 03:15:21.910683 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1f1dc8e1-28c7-4004-bf18-49a9ce850c0a-workload-socket\") pod \"e2e-api-a-a20636-openshift-default-5f46b5b487-fzp4k\" (UID: \"1f1dc8e1-28c7-4004-bf18-49a9ce850c0a\") " pod="openshift-ingress/e2e-api-a-a20636-openshift-default-5f46b5b487-fzp4k" Jul 06 03:15:21.910896 ip-10-0-139-206 kubenswrapper[2557]: I0706 03:15:21.910715 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1f1dc8e1-28c7-4004-bf18-49a9ce850c0a-credential-socket\") pod \"e2e-api-a-a20636-openshift-default-5f46b5b487-fzp4k\" (UID: \"1f1dc8e1-28c7-4004-bf18-49a9ce850c0a\") " pod="openshift-ingress/e2e-api-a-a20636-openshift-default-5f46b5b487-fzp4k" Jul 06 03:15:21.910896 ip-10-0-139-206 kubenswrapper[2557]: I0706 03:15:21.910768 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1f1dc8e1-28c7-4004-bf18-49a9ce850c0a-istio-envoy\") pod \"e2e-api-a-a20636-openshift-default-5f46b5b487-fzp4k\" (UID: \"1f1dc8e1-28c7-4004-bf18-49a9ce850c0a\") " pod="openshift-ingress/e2e-api-a-a20636-openshift-default-5f46b5b487-fzp4k" Jul 06 03:15:21.910896 ip-10-0-139-206 kubenswrapper[2557]: I0706 03:15:21.910800 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tzflw\" (UniqueName: \"kubernetes.io/projected/1f1dc8e1-28c7-4004-bf18-49a9ce850c0a-kube-api-access-tzflw\") pod \"e2e-api-a-a20636-openshift-default-5f46b5b487-fzp4k\" (UID: \"1f1dc8e1-28c7-4004-bf18-49a9ce850c0a\") " pod="openshift-ingress/e2e-api-a-a20636-openshift-default-5f46b5b487-fzp4k" Jul 06 03:15:21.911187 ip-10-0-139-206 kubenswrapper[2557]: I0706 03:15:21.911009 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1f1dc8e1-28c7-4004-bf18-49a9ce850c0a-istio-data\") pod \"e2e-api-a-a20636-openshift-default-5f46b5b487-fzp4k\" (UID: \"1f1dc8e1-28c7-4004-bf18-49a9ce850c0a\") " pod="openshift-ingress/e2e-api-a-a20636-openshift-default-5f46b5b487-fzp4k" Jul 06 03:15:21.911187 ip-10-0-139-206 kubenswrapper[2557]: I0706 03:15:21.911030 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1f1dc8e1-28c7-4004-bf18-49a9ce850c0a-workload-socket\") pod \"e2e-api-a-a20636-openshift-default-5f46b5b487-fzp4k\" (UID: \"1f1dc8e1-28c7-4004-bf18-49a9ce850c0a\") " pod="openshift-ingress/e2e-api-a-a20636-openshift-default-5f46b5b487-fzp4k" Jul 06 03:15:21.911279 ip-10-0-139-206 kubenswrapper[2557]: I0706 03:15:21.911250 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1f1dc8e1-28c7-4004-bf18-49a9ce850c0a-workload-certs\") pod \"e2e-api-a-a20636-openshift-default-5f46b5b487-fzp4k\" (UID: \"1f1dc8e1-28c7-4004-bf18-49a9ce850c0a\") " pod="openshift-ingress/e2e-api-a-a20636-openshift-default-5f46b5b487-fzp4k" Jul 06 03:15:21.911540 ip-10-0-139-206 kubenswrapper[2557]: I0706 03:15:21.911511 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1f1dc8e1-28c7-4004-bf18-49a9ce850c0a-istiod-ca-cert\") pod \"e2e-api-a-a20636-openshift-default-5f46b5b487-fzp4k\" (UID: \"1f1dc8e1-28c7-4004-bf18-49a9ce850c0a\") " pod="openshift-ingress/e2e-api-a-a20636-openshift-default-5f46b5b487-fzp4k" Jul 06 03:15:21.911641 ip-10-0-139-206 kubenswrapper[2557]: I0706 03:15:21.911537 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1f1dc8e1-28c7-4004-bf18-49a9ce850c0a-credential-socket\") pod \"e2e-api-a-a20636-openshift-default-5f46b5b487-fzp4k\" (UID: \"1f1dc8e1-28c7-4004-bf18-49a9ce850c0a\") " pod="openshift-ingress/e2e-api-a-a20636-openshift-default-5f46b5b487-fzp4k" Jul 06 03:15:21.913808 ip-10-0-139-206 kubenswrapper[2557]: I0706 03:15:21.913778 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1f1dc8e1-28c7-4004-bf18-49a9ce850c0a-istio-envoy\") pod \"e2e-api-a-a20636-openshift-default-5f46b5b487-fzp4k\" (UID: \"1f1dc8e1-28c7-4004-bf18-49a9ce850c0a\") " pod="openshift-ingress/e2e-api-a-a20636-openshift-default-5f46b5b487-fzp4k" Jul 06 03:15:21.913944 ip-10-0-139-206 kubenswrapper[2557]: I0706 03:15:21.913926 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1f1dc8e1-28c7-4004-bf18-49a9ce850c0a-istio-podinfo\") pod \"e2e-api-a-a20636-openshift-default-5f46b5b487-fzp4k\" (UID: \"1f1dc8e1-28c7-4004-bf18-49a9ce850c0a\") " pod="openshift-ingress/e2e-api-a-a20636-openshift-default-5f46b5b487-fzp4k" Jul 06 03:15:21.919212 ip-10-0-139-206 kubenswrapper[2557]: I0706 03:15:21.919175 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-tzflw\" (UniqueName: \"kubernetes.io/projected/1f1dc8e1-28c7-4004-bf18-49a9ce850c0a-kube-api-access-tzflw\") pod \"e2e-api-a-a20636-openshift-default-5f46b5b487-fzp4k\" (UID: \"1f1dc8e1-28c7-4004-bf18-49a9ce850c0a\") " pod="openshift-ingress/e2e-api-a-a20636-openshift-default-5f46b5b487-fzp4k" Jul 06 03:15:21.919328 ip-10-0-139-206 kubenswrapper[2557]: I0706 03:15:21.919318 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1f1dc8e1-28c7-4004-bf18-49a9ce850c0a-istio-token\") pod \"e2e-api-a-a20636-openshift-default-5f46b5b487-fzp4k\" (UID: \"1f1dc8e1-28c7-4004-bf18-49a9ce850c0a\") " pod="openshift-ingress/e2e-api-a-a20636-openshift-default-5f46b5b487-fzp4k" Jul 06 03:15:21.956216 ip-10-0-139-206 kubenswrapper[2557]: I0706 03:15:21.956177 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-a20636-openshift-default-5f46b5b487-fzp4k" Jul 06 03:15:22.098984 ip-10-0-139-206 kubenswrapper[2557]: I0706 03:15:22.098946 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-a-a20636-openshift-default-5f46b5b487-fzp4k"] Jul 06 03:15:22.102167 ip-10-0-139-206 kubenswrapper[2557]: W0706 03:15:22.102135 2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1f1dc8e1_28c7_4004_bf18_49a9ce850c0a.slice/crio-64f6b2b53c3c92f1525ba8eaf3361dda6f3914d2b732900d9f2c56c586fafa4c WatchSource:0}: Error finding container 64f6b2b53c3c92f1525ba8eaf3361dda6f3914d2b732900d9f2c56c586fafa4c: Status 404 returned error can't find the container with id 64f6b2b53c3c92f1525ba8eaf3361dda6f3914d2b732900d9f2c56c586fafa4c Jul 06 03:15:22.104179 ip-10-0-139-206 kubenswrapper[2557]: I0706 03:15:22.104146 2557 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 06 03:15:22.104262 ip-10-0-139-206 kubenswrapper[2557]: I0706 03:15:22.104215 2557 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 06 03:15:22.104262 ip-10-0-139-206 kubenswrapper[2557]: I0706 03:15:22.104245 2557 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 06 03:15:22.648053 ip-10-0-139-206 kubenswrapper[2557]: I0706 03:15:22.648000 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-a20636-openshift-default-5f46b5b487-fzp4k" event={"ID":"1f1dc8e1-28c7-4004-bf18-49a9ce850c0a","Type":"ContainerStarted","Data":"9368f5cc12a4503974c1e0a244f1603afca1eea5598fb675e757723ab36baa11"} Jul 06 03:15:22.648483 ip-10-0-139-206 kubenswrapper[2557]: I0706 03:15:22.648069 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-a20636-openshift-default-5f46b5b487-fzp4k" event={"ID":"1f1dc8e1-28c7-4004-bf18-49a9ce850c0a","Type":"ContainerStarted","Data":"64f6b2b53c3c92f1525ba8eaf3361dda6f3914d2b732900d9f2c56c586fafa4c"} Jul 06 03:15:22.667758 ip-10-0-139-206 kubenswrapper[2557]: I0706 03:15:22.667706 2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-api-a-a20636-openshift-default-5f46b5b487-fzp4k" podStartSLOduration=1.6676887200000001 podStartE2EDuration="1.66768872s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 03:15:21 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 03:15:22.665687006 +0000 UTC m=+2166.471973066" watchObservedRunningTime="2026-07-06 03:15:22.66768872 +0000 UTC m=+2166.473974756" Jul 06 03:15:22.956638 ip-10-0-139-206 kubenswrapper[2557]: I0706 03:15:22.956602 2557 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-a-a20636-openshift-default-5f46b5b487-fzp4k" Jul 06 03:15:23.962713 ip-10-0-139-206 kubenswrapper[2557]: I0706 03:15:23.962679 2557 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-a-a20636-openshift-default-5f46b5b487-fzp4k" Jul 06 03:15:24.657827 ip-10-0-139-206 kubenswrapper[2557]: I0706 03:15:24.657795 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-a-a20636-openshift-default-5f46b5b487-fzp4k" Jul 06 03:15:24.658724 ip-10-0-139-206 kubenswrapper[2557]: I0706 03:15:24.658701 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-a-a20636-openshift-default-5f46b5b487-fzp4k" Jul 06 03:15:30.670918 ip-10-0-139-206 kubenswrapper[2557]: I0706 03:15:30.670878 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-srzrr"] Jul 06 03:15:39.659040 ip-10-0-139-206 kubenswrapper[2557]: I0706 03:15:39.658999 2557 patch_prober.go:28] interesting pod/e2e-api-a-a20636-openshift-default-5f46b5b487-fzp4k container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.133.0.36:15021/healthz/ready\": dial tcp 10.133.0.36:15021: connect: connection refused" start-of-body= Jul 06 03:15:39.659488 ip-10-0-139-206 kubenswrapper[2557]: I0706 03:15:39.659076 2557 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-api-a-a20636-openshift-default-5f46b5b487-fzp4k" podUID="1f1dc8e1-28c7-4004-bf18-49a9ce850c0a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.36:15021/healthz/ready\": dial tcp 10.133.0.36:15021: connect: connection refused" Jul 06 03:15:39.733922 ip-10-0-139-206 kubenswrapper[2557]: I0706 03:15:39.733888 2557 generic.go:356] "Generic (PLEG): container finished" podID="1f1dc8e1-28c7-4004-bf18-49a9ce850c0a" containerID="9368f5cc12a4503974c1e0a244f1603afca1eea5598fb675e757723ab36baa11" exitCode=0 Jul 06 03:15:39.734089 ip-10-0-139-206 kubenswrapper[2557]: I0706 03:15:39.733965 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-a20636-openshift-default-5f46b5b487-fzp4k" event={"ID":"1f1dc8e1-28c7-4004-bf18-49a9ce850c0a","Type":"ContainerDied","Data":"9368f5cc12a4503974c1e0a244f1603afca1eea5598fb675e757723ab36baa11"} Jul 06 03:15:39.734420 ip-10-0-139-206 kubenswrapper[2557]: I0706 03:15:39.734400 2557 scope.go:117] "RemoveContainer" containerID="9368f5cc12a4503974c1e0a244f1603afca1eea5598fb675e757723ab36baa11" Jul 06 03:15:39.735431 ip-10-0-139-206 kubenswrapper[2557]: I0706 03:15:39.735398 2557 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 06 03:15:39.735505 ip-10-0-139-206 kubenswrapper[2557]: I0706 03:15:39.735473 2557 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 06 03:15:39.735544 ip-10-0-139-206 kubenswrapper[2557]: I0706 03:15:39.735510 2557 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 06 03:15:40.741118 ip-10-0-139-206 kubenswrapper[2557]: I0706 03:15:40.741079 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-a20636-openshift-default-5f46b5b487-fzp4k" event={"ID":"1f1dc8e1-28c7-4004-bf18-49a9ce850c0a","Type":"ContainerStarted","Data":"ff53f1fe292d130688920f9571157ba83063b35ae3fc877fabcb146dda3838ff"} Jul 06 03:15:40.957163 ip-10-0-139-206 kubenswrapper[2557]: I0706 03:15:40.957122 2557 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-a-a20636-openshift-default-5f46b5b487-fzp4k" Jul 06 03:15:40.957505 ip-10-0-139-206 kubenswrapper[2557]: I0706 03:15:40.957481 2557 patch_prober.go:28] interesting pod/e2e-api-a-a20636-openshift-default-5f46b5b487-fzp4k container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.36:15021/healthz/ready\": dial tcp 10.133.0.36:15021: connect: connection refused" start-of-body= Jul 06 03:15:40.957590 ip-10-0-139-206 kubenswrapper[2557]: I0706 03:15:40.957568 2557 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-a20636-openshift-default-5f46b5b487-fzp4k" podUID="1f1dc8e1-28c7-4004-bf18-49a9ce850c0a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.36:15021/healthz/ready\": dial tcp 10.133.0.36:15021: connect: connection refused" Jul 06 03:15:41.956809 ip-10-0-139-206 kubenswrapper[2557]: I0706 03:15:41.956769 2557 patch_prober.go:28] interesting pod/e2e-api-a-a20636-openshift-default-5f46b5b487-fzp4k container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.36:15021/healthz/ready\": dial tcp 10.133.0.36:15021: connect: connection refused" start-of-body= Jul 06 03:15:41.957220 ip-10-0-139-206 kubenswrapper[2557]: I0706 03:15:41.956828 2557 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-a20636-openshift-default-5f46b5b487-fzp4k" podUID="1f1dc8e1-28c7-4004-bf18-49a9ce850c0a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.36:15021/healthz/ready\": dial tcp 10.133.0.36:15021: connect: connection refused" Jul 06 03:15:42.957114 ip-10-0-139-206 kubenswrapper[2557]: I0706 03:15:42.957072 2557 patch_prober.go:28] interesting pod/e2e-api-a-a20636-openshift-default-5f46b5b487-fzp4k container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.36:15021/healthz/ready\": dial tcp 10.133.0.36:15021: connect: connection refused" start-of-body= Jul 06 03:15:42.957533 ip-10-0-139-206 kubenswrapper[2557]: I0706 03:15:42.957139 2557 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-a20636-openshift-default-5f46b5b487-fzp4k" podUID="1f1dc8e1-28c7-4004-bf18-49a9ce850c0a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.36:15021/healthz/ready\": dial tcp 10.133.0.36:15021: connect: connection refused" Jul 06 03:15:43.957532 ip-10-0-139-206 kubenswrapper[2557]: I0706 03:15:43.957498 2557 patch_prober.go:28] interesting pod/e2e-api-a-a20636-openshift-default-5f46b5b487-fzp4k container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.36:15021/healthz/ready\": dial tcp 10.133.0.36:15021: connect: connection refused" start-of-body= Jul 06 03:15:43.957940 ip-10-0-139-206 kubenswrapper[2557]: I0706 03:15:43.957559 2557 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-a20636-openshift-default-5f46b5b487-fzp4k" podUID="1f1dc8e1-28c7-4004-bf18-49a9ce850c0a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.36:15021/healthz/ready\": dial tcp 10.133.0.36:15021: connect: connection refused" Jul 06 03:15:44.960832 ip-10-0-139-206 kubenswrapper[2557]: I0706 03:15:44.960804 2557 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-a-a20636-openshift-default-5f46b5b487-fzp4k" Jul 06 03:15:44.961230 ip-10-0-139-206 kubenswrapper[2557]: I0706 03:15:44.961072 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-a-a20636-openshift-default-5f46b5b487-fzp4k" Jul 06 03:15:44.961680 ip-10-0-139-206 kubenswrapper[2557]: I0706 03:15:44.961659 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-a-a20636-openshift-default-5f46b5b487-fzp4k" Jul 06 03:15:45.067706 ip-10-0-139-206 kubenswrapper[2557]: I0706 03:15:45.067673 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-srzrr"] Jul 06 03:15:51.581336 ip-10-0-139-206 kubenswrapper[2557]: I0706 03:15:51.581300 2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-a-a20636-openshift-default-5f46b5b487-fzp4k"] Jul 06 03:15:51.581772 ip-10-0-139-206 kubenswrapper[2557]: I0706 03:15:51.581527 2557 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-ingress/e2e-api-a-a20636-openshift-default-5f46b5b487-fzp4k" podUID="1f1dc8e1-28c7-4004-bf18-49a9ce850c0a" containerName="istio-proxy" containerID="cri-o://ff53f1fe292d130688920f9571157ba83063b35ae3fc877fabcb146dda3838ff" gracePeriod=30 Jul 06 03:15:54.859576 ip-10-0-139-206 kubenswrapper[2557]: I0706 03:15:54.859538 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-srzrr"] Jul 06 03:15:56.821332 ip-10-0-139-206 kubenswrapper[2557]: I0706 03:15:56.821295 2557 generic.go:356] "Generic (PLEG): container finished" podID="1f1dc8e1-28c7-4004-bf18-49a9ce850c0a" containerID="ff53f1fe292d130688920f9571157ba83063b35ae3fc877fabcb146dda3838ff" exitCode=0 Jul 06 03:15:56.821725 ip-10-0-139-206 kubenswrapper[2557]: I0706 03:15:56.821373 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-a20636-openshift-default-5f46b5b487-fzp4k" event={"ID":"1f1dc8e1-28c7-4004-bf18-49a9ce850c0a","Type":"ContainerDied","Data":"ff53f1fe292d130688920f9571157ba83063b35ae3fc877fabcb146dda3838ff"} Jul 06 03:15:56.821725 ip-10-0-139-206 kubenswrapper[2557]: I0706 03:15:56.821463 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-a20636-openshift-default-5f46b5b487-fzp4k" event={"ID":"1f1dc8e1-28c7-4004-bf18-49a9ce850c0a","Type":"ContainerDied","Data":"64f6b2b53c3c92f1525ba8eaf3361dda6f3914d2b732900d9f2c56c586fafa4c"} Jul 06 03:15:56.821725 ip-10-0-139-206 kubenswrapper[2557]: I0706 03:15:56.821483 2557 scope.go:117] "RemoveContainer" containerID="9368f5cc12a4503974c1e0a244f1603afca1eea5598fb675e757723ab36baa11" Jul 06 03:15:56.821725 ip-10-0-139-206 kubenswrapper[2557]: I0706 03:15:56.821486 2557 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="64f6b2b53c3c92f1525ba8eaf3361dda6f3914d2b732900d9f2c56c586fafa4c" Jul 06 03:15:56.824008 ip-10-0-139-206 kubenswrapper[2557]: I0706 03:15:56.823988 2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-a20636-openshift-default-5f46b5b487-fzp4k" Jul 06 03:15:56.922174 ip-10-0-139-206 kubenswrapper[2557]: I0706 03:15:56.922140 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1f1dc8e1-28c7-4004-bf18-49a9ce850c0a-istio-data\") pod \"1f1dc8e1-28c7-4004-bf18-49a9ce850c0a\" (UID: \"1f1dc8e1-28c7-4004-bf18-49a9ce850c0a\") " Jul 06 03:15:56.922174 ip-10-0-139-206 kubenswrapper[2557]: I0706 03:15:56.922178 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1f1dc8e1-28c7-4004-bf18-49a9ce850c0a-workload-certs\") pod \"1f1dc8e1-28c7-4004-bf18-49a9ce850c0a\" (UID: \"1f1dc8e1-28c7-4004-bf18-49a9ce850c0a\") " Jul 06 03:15:56.922472 ip-10-0-139-206 kubenswrapper[2557]: I0706 03:15:56.922196 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1f1dc8e1-28c7-4004-bf18-49a9ce850c0a-istio-envoy\") pod \"1f1dc8e1-28c7-4004-bf18-49a9ce850c0a\" (UID: \"1f1dc8e1-28c7-4004-bf18-49a9ce850c0a\") " Jul 06 03:15:56.922472 ip-10-0-139-206 kubenswrapper[2557]: I0706 03:15:56.922226 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1f1dc8e1-28c7-4004-bf18-49a9ce850c0a-credential-socket\") pod \"1f1dc8e1-28c7-4004-bf18-49a9ce850c0a\" (UID: \"1f1dc8e1-28c7-4004-bf18-49a9ce850c0a\") " Jul 06 03:15:56.922472 ip-10-0-139-206 kubenswrapper[2557]: I0706 03:15:56.922266 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1f1dc8e1-28c7-4004-bf18-49a9ce850c0a-istio-podinfo\") pod \"1f1dc8e1-28c7-4004-bf18-49a9ce850c0a\" (UID: \"1f1dc8e1-28c7-4004-bf18-49a9ce850c0a\") " Jul 06 03:15:56.922472 ip-10-0-139-206 kubenswrapper[2557]: I0706 03:15:56.922292 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-tzflw\" (UniqueName: \"kubernetes.io/projected/1f1dc8e1-28c7-4004-bf18-49a9ce850c0a-kube-api-access-tzflw\") pod \"1f1dc8e1-28c7-4004-bf18-49a9ce850c0a\" (UID: \"1f1dc8e1-28c7-4004-bf18-49a9ce850c0a\") " Jul 06 03:15:56.922472 ip-10-0-139-206 kubenswrapper[2557]: I0706 03:15:56.922327 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1f1dc8e1-28c7-4004-bf18-49a9ce850c0a-istio-token\") pod \"1f1dc8e1-28c7-4004-bf18-49a9ce850c0a\" (UID: \"1f1dc8e1-28c7-4004-bf18-49a9ce850c0a\") " Jul 06 03:15:56.922472 ip-10-0-139-206 kubenswrapper[2557]: I0706 03:15:56.922375 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1f1dc8e1-28c7-4004-bf18-49a9ce850c0a-istiod-ca-cert\") pod \"1f1dc8e1-28c7-4004-bf18-49a9ce850c0a\" (UID: \"1f1dc8e1-28c7-4004-bf18-49a9ce850c0a\") " Jul 06 03:15:56.922472 ip-10-0-139-206 kubenswrapper[2557]: I0706 03:15:56.922455 2557 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1f1dc8e1-28c7-4004-bf18-49a9ce850c0a-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "1f1dc8e1-28c7-4004-bf18-49a9ce850c0a" (UID: "1f1dc8e1-28c7-4004-bf18-49a9ce850c0a"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 03:15:56.922840 ip-10-0-139-206 kubenswrapper[2557]: I0706 03:15:56.922501 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1f1dc8e1-28c7-4004-bf18-49a9ce850c0a-workload-socket\") pod \"1f1dc8e1-28c7-4004-bf18-49a9ce850c0a\" (UID: \"1f1dc8e1-28c7-4004-bf18-49a9ce850c0a\") " Jul 06 03:15:56.922840 ip-10-0-139-206 kubenswrapper[2557]: I0706 03:15:56.922700 2557 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1f1dc8e1-28c7-4004-bf18-49a9ce850c0a-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "1f1dc8e1-28c7-4004-bf18-49a9ce850c0a" (UID: "1f1dc8e1-28c7-4004-bf18-49a9ce850c0a"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 03:15:56.922840 ip-10-0-139-206 kubenswrapper[2557]: I0706 03:15:56.922786 2557 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1f1dc8e1-28c7-4004-bf18-49a9ce850c0a-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "1f1dc8e1-28c7-4004-bf18-49a9ce850c0a" (UID: "1f1dc8e1-28c7-4004-bf18-49a9ce850c0a"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 06 03:15:56.923003 ip-10-0-139-206 kubenswrapper[2557]: I0706 03:15:56.922876 2557 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1f1dc8e1-28c7-4004-bf18-49a9ce850c0a-workload-certs\") on node \"ip-10-0-139-206.ec2.internal\" DevicePath \"\"" Jul 06 03:15:56.923003 ip-10-0-139-206 kubenswrapper[2557]: I0706 03:15:56.922893 2557 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1f1dc8e1-28c7-4004-bf18-49a9ce850c0a-credential-socket\") on node \"ip-10-0-139-206.ec2.internal\" DevicePath \"\"" Jul 06 03:15:56.923003 ip-10-0-139-206 kubenswrapper[2557]: I0706 03:15:56.922896 2557 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1f1dc8e1-28c7-4004-bf18-49a9ce850c0a-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "1f1dc8e1-28c7-4004-bf18-49a9ce850c0a" (UID: "1f1dc8e1-28c7-4004-bf18-49a9ce850c0a"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 03:15:56.923003 ip-10-0-139-206 kubenswrapper[2557]: I0706 03:15:56.922906 2557 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1f1dc8e1-28c7-4004-bf18-49a9ce850c0a-istiod-ca-cert\") on node \"ip-10-0-139-206.ec2.internal\" DevicePath \"\"" Jul 06 03:15:56.923206 ip-10-0-139-206 kubenswrapper[2557]: I0706 03:15:56.923145 2557 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1f1dc8e1-28c7-4004-bf18-49a9ce850c0a-istio-data" (OuterVolumeSpecName: "istio-data") pod "1f1dc8e1-28c7-4004-bf18-49a9ce850c0a" (UID: "1f1dc8e1-28c7-4004-bf18-49a9ce850c0a"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 03:15:56.924636 ip-10-0-139-206 kubenswrapper[2557]: I0706 03:15:56.924597 2557 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1f1dc8e1-28c7-4004-bf18-49a9ce850c0a-istio-token" (OuterVolumeSpecName: "istio-token") pod "1f1dc8e1-28c7-4004-bf18-49a9ce850c0a" (UID: "1f1dc8e1-28c7-4004-bf18-49a9ce850c0a"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 06 03:15:56.924749 ip-10-0-139-206 kubenswrapper[2557]: I0706 03:15:56.924645 2557 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1f1dc8e1-28c7-4004-bf18-49a9ce850c0a-kube-api-access-tzflw" (OuterVolumeSpecName: "kube-api-access-tzflw") pod "1f1dc8e1-28c7-4004-bf18-49a9ce850c0a" (UID: "1f1dc8e1-28c7-4004-bf18-49a9ce850c0a"). InnerVolumeSpecName "kube-api-access-tzflw". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 06 03:15:56.924749 ip-10-0-139-206 kubenswrapper[2557]: I0706 03:15:56.924668 2557 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/1f1dc8e1-28c7-4004-bf18-49a9ce850c0a-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "1f1dc8e1-28c7-4004-bf18-49a9ce850c0a" (UID: "1f1dc8e1-28c7-4004-bf18-49a9ce850c0a"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 06 03:15:56.924749 ip-10-0-139-206 kubenswrapper[2557]: I0706 03:15:56.924676 2557 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1f1dc8e1-28c7-4004-bf18-49a9ce850c0a-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "1f1dc8e1-28c7-4004-bf18-49a9ce850c0a" (UID: "1f1dc8e1-28c7-4004-bf18-49a9ce850c0a"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 03:15:57.024087 ip-10-0-139-206 kubenswrapper[2557]: I0706 03:15:57.024001 2557 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1f1dc8e1-28c7-4004-bf18-49a9ce850c0a-istio-token\") on node \"ip-10-0-139-206.ec2.internal\" DevicePath \"\"" Jul 06 03:15:57.024087 ip-10-0-139-206 kubenswrapper[2557]: I0706 03:15:57.024033 2557 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1f1dc8e1-28c7-4004-bf18-49a9ce850c0a-workload-socket\") on node \"ip-10-0-139-206.ec2.internal\" DevicePath \"\"" Jul 06 03:15:57.024087 ip-10-0-139-206 kubenswrapper[2557]: I0706 03:15:57.024042 2557 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1f1dc8e1-28c7-4004-bf18-49a9ce850c0a-istio-data\") on node \"ip-10-0-139-206.ec2.internal\" DevicePath \"\"" Jul 06 03:15:57.024087 ip-10-0-139-206 kubenswrapper[2557]: I0706 03:15:57.024050 2557 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1f1dc8e1-28c7-4004-bf18-49a9ce850c0a-istio-envoy\") on node \"ip-10-0-139-206.ec2.internal\" DevicePath \"\"" Jul 06 03:15:57.024087 ip-10-0-139-206 kubenswrapper[2557]: I0706 03:15:57.024058 2557 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1f1dc8e1-28c7-4004-bf18-49a9ce850c0a-istio-podinfo\") on node \"ip-10-0-139-206.ec2.internal\" DevicePath \"\"" Jul 06 03:15:57.024087 ip-10-0-139-206 kubenswrapper[2557]: I0706 03:15:57.024066 2557 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-tzflw\" (UniqueName: \"kubernetes.io/projected/1f1dc8e1-28c7-4004-bf18-49a9ce850c0a-kube-api-access-tzflw\") on node \"ip-10-0-139-206.ec2.internal\" DevicePath \"\"" Jul 06 03:15:57.827529 ip-10-0-139-206 kubenswrapper[2557]: I0706 03:15:57.827504 2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-a20636-openshift-default-5f46b5b487-fzp4k" Jul 06 03:15:57.858424 ip-10-0-139-206 kubenswrapper[2557]: I0706 03:15:57.858389 2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-a-a20636-openshift-default-5f46b5b487-fzp4k"] Jul 06 03:15:57.862227 ip-10-0-139-206 kubenswrapper[2557]: I0706 03:15:57.862201 2557 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-api-a-a20636-openshift-default-5f46b5b487-fzp4k"] Jul 06 03:15:59.142496 ip-10-0-139-206 kubenswrapper[2557]: I0706 03:15:59.142461 2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-shared-a-8edd65-openshift-default-75f7b769c9-bwvv8"] Jul 06 03:15:59.142991 ip-10-0-139-206 kubenswrapper[2557]: I0706 03:15:59.142806 2557 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1f1dc8e1-28c7-4004-bf18-49a9ce850c0a" containerName="istio-proxy" Jul 06 03:15:59.142991 ip-10-0-139-206 kubenswrapper[2557]: I0706 03:15:59.142817 2557 state_mem.go:107] "Deleted CPUSet assignment" podUID="1f1dc8e1-28c7-4004-bf18-49a9ce850c0a" containerName="istio-proxy" Jul 06 03:15:59.142991 ip-10-0-139-206 kubenswrapper[2557]: I0706 03:15:59.142878 2557 memory_manager.go:368] "RemoveStaleState removing state" podUID="1f1dc8e1-28c7-4004-bf18-49a9ce850c0a" containerName="istio-proxy" Jul 06 03:15:59.142991 ip-10-0-139-206 kubenswrapper[2557]: I0706 03:15:59.142888 2557 memory_manager.go:368] "RemoveStaleState removing state" podUID="1f1dc8e1-28c7-4004-bf18-49a9ce850c0a" containerName="istio-proxy" Jul 06 03:15:59.142991 ip-10-0-139-206 kubenswrapper[2557]: I0706 03:15:59.142894 2557 memory_manager.go:368] "RemoveStaleState removing state" podUID="60a08f89-4663-40d2-971e-1360f8c7712f" containerName="istio-proxy" Jul 06 03:15:59.146936 ip-10-0-139-206 kubenswrapper[2557]: I0706 03:15:59.146916 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-8edd65-openshift-default-75f7b769c9-bwvv8" Jul 06 03:15:59.149605 ip-10-0-139-206 kubenswrapper[2557]: I0706 03:15:59.149587 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-shared-a-8edd65-openshift-default-dockercfg-tt9zp\"" Jul 06 03:15:59.159240 ip-10-0-139-206 kubenswrapper[2557]: I0706 03:15:59.159217 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-a-8edd65-openshift-default-75f7b769c9-bwvv8"] Jul 06 03:15:59.245004 ip-10-0-139-206 kubenswrapper[2557]: I0706 03:15:59.243747 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/2ed27de3-d9c7-4657-b7e4-37b71941db77-istiod-ca-cert\") pod \"e2e-shared-a-8edd65-openshift-default-75f7b769c9-bwvv8\" (UID: \"2ed27de3-d9c7-4657-b7e4-37b71941db77\") " pod="openshift-ingress/e2e-shared-a-8edd65-openshift-default-75f7b769c9-bwvv8" Jul 06 03:15:59.245004 ip-10-0-139-206 kubenswrapper[2557]: I0706 03:15:59.243814 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/2ed27de3-d9c7-4657-b7e4-37b71941db77-workload-socket\") pod \"e2e-shared-a-8edd65-openshift-default-75f7b769c9-bwvv8\" (UID: \"2ed27de3-d9c7-4657-b7e4-37b71941db77\") " pod="openshift-ingress/e2e-shared-a-8edd65-openshift-default-75f7b769c9-bwvv8" Jul 06 03:15:59.245004 ip-10-0-139-206 kubenswrapper[2557]: I0706 03:15:59.243851 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/2ed27de3-d9c7-4657-b7e4-37b71941db77-workload-certs\") pod \"e2e-shared-a-8edd65-openshift-default-75f7b769c9-bwvv8\" (UID: \"2ed27de3-d9c7-4657-b7e4-37b71941db77\") " pod="openshift-ingress/e2e-shared-a-8edd65-openshift-default-75f7b769c9-bwvv8" Jul 06 03:15:59.245004 ip-10-0-139-206 kubenswrapper[2557]: I0706 03:15:59.243892 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/2ed27de3-d9c7-4657-b7e4-37b71941db77-credential-socket\") pod \"e2e-shared-a-8edd65-openshift-default-75f7b769c9-bwvv8\" (UID: \"2ed27de3-d9c7-4657-b7e4-37b71941db77\") " pod="openshift-ingress/e2e-shared-a-8edd65-openshift-default-75f7b769c9-bwvv8" Jul 06 03:15:59.245004 ip-10-0-139-206 kubenswrapper[2557]: I0706 03:15:59.243939 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/2ed27de3-d9c7-4657-b7e4-37b71941db77-istio-data\") pod \"e2e-shared-a-8edd65-openshift-default-75f7b769c9-bwvv8\" (UID: \"2ed27de3-d9c7-4657-b7e4-37b71941db77\") " pod="openshift-ingress/e2e-shared-a-8edd65-openshift-default-75f7b769c9-bwvv8" Jul 06 03:15:59.245004 ip-10-0-139-206 kubenswrapper[2557]: I0706 03:15:59.243993 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/2ed27de3-d9c7-4657-b7e4-37b71941db77-istio-podinfo\") pod \"e2e-shared-a-8edd65-openshift-default-75f7b769c9-bwvv8\" (UID: \"2ed27de3-d9c7-4657-b7e4-37b71941db77\") " pod="openshift-ingress/e2e-shared-a-8edd65-openshift-default-75f7b769c9-bwvv8" Jul 06 03:15:59.245004 ip-10-0-139-206 kubenswrapper[2557]: I0706 03:15:59.244038 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lgb7r\" (UniqueName: \"kubernetes.io/projected/2ed27de3-d9c7-4657-b7e4-37b71941db77-kube-api-access-lgb7r\") pod \"e2e-shared-a-8edd65-openshift-default-75f7b769c9-bwvv8\" (UID: \"2ed27de3-d9c7-4657-b7e4-37b71941db77\") " pod="openshift-ingress/e2e-shared-a-8edd65-openshift-default-75f7b769c9-bwvv8" Jul 06 03:15:59.245004 ip-10-0-139-206 kubenswrapper[2557]: I0706 03:15:59.244079 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/2ed27de3-d9c7-4657-b7e4-37b71941db77-istio-envoy\") pod \"e2e-shared-a-8edd65-openshift-default-75f7b769c9-bwvv8\" (UID: \"2ed27de3-d9c7-4657-b7e4-37b71941db77\") " pod="openshift-ingress/e2e-shared-a-8edd65-openshift-default-75f7b769c9-bwvv8" Jul 06 03:15:59.245004 ip-10-0-139-206 kubenswrapper[2557]: I0706 03:15:59.244109 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/2ed27de3-d9c7-4657-b7e4-37b71941db77-istio-token\") pod \"e2e-shared-a-8edd65-openshift-default-75f7b769c9-bwvv8\" (UID: \"2ed27de3-d9c7-4657-b7e4-37b71941db77\") " pod="openshift-ingress/e2e-shared-a-8edd65-openshift-default-75f7b769c9-bwvv8" Jul 06 03:15:59.302329 ip-10-0-139-206 kubenswrapper[2557]: I0706 03:15:59.302297 2557 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="1f1dc8e1-28c7-4004-bf18-49a9ce850c0a" path="/var/lib/kubelet/pods/1f1dc8e1-28c7-4004-bf18-49a9ce850c0a/volumes" Jul 06 03:15:59.344973 ip-10-0-139-206 kubenswrapper[2557]: I0706 03:15:59.344941 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/2ed27de3-d9c7-4657-b7e4-37b71941db77-istio-envoy\") pod \"e2e-shared-a-8edd65-openshift-default-75f7b769c9-bwvv8\" (UID: \"2ed27de3-d9c7-4657-b7e4-37b71941db77\") " pod="openshift-ingress/e2e-shared-a-8edd65-openshift-default-75f7b769c9-bwvv8" Jul 06 03:15:59.344973 ip-10-0-139-206 kubenswrapper[2557]: I0706 03:15:59.344974 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/2ed27de3-d9c7-4657-b7e4-37b71941db77-istio-token\") pod \"e2e-shared-a-8edd65-openshift-default-75f7b769c9-bwvv8\" (UID: \"2ed27de3-d9c7-4657-b7e4-37b71941db77\") " pod="openshift-ingress/e2e-shared-a-8edd65-openshift-default-75f7b769c9-bwvv8" Jul 06 03:15:59.345282 ip-10-0-139-206 kubenswrapper[2557]: I0706 03:15:59.344995 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/2ed27de3-d9c7-4657-b7e4-37b71941db77-istiod-ca-cert\") pod \"e2e-shared-a-8edd65-openshift-default-75f7b769c9-bwvv8\" (UID: \"2ed27de3-d9c7-4657-b7e4-37b71941db77\") " pod="openshift-ingress/e2e-shared-a-8edd65-openshift-default-75f7b769c9-bwvv8" Jul 06 03:15:59.345282 ip-10-0-139-206 kubenswrapper[2557]: I0706 03:15:59.345023 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/2ed27de3-d9c7-4657-b7e4-37b71941db77-workload-socket\") pod \"e2e-shared-a-8edd65-openshift-default-75f7b769c9-bwvv8\" (UID: \"2ed27de3-d9c7-4657-b7e4-37b71941db77\") " pod="openshift-ingress/e2e-shared-a-8edd65-openshift-default-75f7b769c9-bwvv8" Jul 06 03:15:59.345282 ip-10-0-139-206 kubenswrapper[2557]: I0706 03:15:59.345048 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/2ed27de3-d9c7-4657-b7e4-37b71941db77-workload-certs\") pod \"e2e-shared-a-8edd65-openshift-default-75f7b769c9-bwvv8\" (UID: \"2ed27de3-d9c7-4657-b7e4-37b71941db77\") " pod="openshift-ingress/e2e-shared-a-8edd65-openshift-default-75f7b769c9-bwvv8" Jul 06 03:15:59.345282 ip-10-0-139-206 kubenswrapper[2557]: I0706 03:15:59.345087 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/2ed27de3-d9c7-4657-b7e4-37b71941db77-credential-socket\") pod \"e2e-shared-a-8edd65-openshift-default-75f7b769c9-bwvv8\" (UID: \"2ed27de3-d9c7-4657-b7e4-37b71941db77\") " pod="openshift-ingress/e2e-shared-a-8edd65-openshift-default-75f7b769c9-bwvv8" Jul 06 03:15:59.345282 ip-10-0-139-206 kubenswrapper[2557]: I0706 03:15:59.345127 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/2ed27de3-d9c7-4657-b7e4-37b71941db77-istio-data\") pod \"e2e-shared-a-8edd65-openshift-default-75f7b769c9-bwvv8\" (UID: \"2ed27de3-d9c7-4657-b7e4-37b71941db77\") " pod="openshift-ingress/e2e-shared-a-8edd65-openshift-default-75f7b769c9-bwvv8" Jul 06 03:15:59.345282 ip-10-0-139-206 kubenswrapper[2557]: I0706 03:15:59.345176 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/2ed27de3-d9c7-4657-b7e4-37b71941db77-istio-podinfo\") pod \"e2e-shared-a-8edd65-openshift-default-75f7b769c9-bwvv8\" (UID: \"2ed27de3-d9c7-4657-b7e4-37b71941db77\") " pod="openshift-ingress/e2e-shared-a-8edd65-openshift-default-75f7b769c9-bwvv8" Jul 06 03:15:59.345282 ip-10-0-139-206 kubenswrapper[2557]: I0706 03:15:59.345223 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lgb7r\" (UniqueName: \"kubernetes.io/projected/2ed27de3-d9c7-4657-b7e4-37b71941db77-kube-api-access-lgb7r\") pod \"e2e-shared-a-8edd65-openshift-default-75f7b769c9-bwvv8\" (UID: \"2ed27de3-d9c7-4657-b7e4-37b71941db77\") " pod="openshift-ingress/e2e-shared-a-8edd65-openshift-default-75f7b769c9-bwvv8" Jul 06 03:15:59.345731 ip-10-0-139-206 kubenswrapper[2557]: I0706 03:15:59.345484 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/2ed27de3-d9c7-4657-b7e4-37b71941db77-workload-socket\") pod \"e2e-shared-a-8edd65-openshift-default-75f7b769c9-bwvv8\" (UID: \"2ed27de3-d9c7-4657-b7e4-37b71941db77\") " pod="openshift-ingress/e2e-shared-a-8edd65-openshift-default-75f7b769c9-bwvv8" Jul 06 03:15:59.345731 ip-10-0-139-206 kubenswrapper[2557]: I0706 03:15:59.345582 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/2ed27de3-d9c7-4657-b7e4-37b71941db77-credential-socket\") pod \"e2e-shared-a-8edd65-openshift-default-75f7b769c9-bwvv8\" (UID: \"2ed27de3-d9c7-4657-b7e4-37b71941db77\") " pod="openshift-ingress/e2e-shared-a-8edd65-openshift-default-75f7b769c9-bwvv8" Jul 06 03:15:59.345731 ip-10-0-139-206 kubenswrapper[2557]: I0706 03:15:59.345636 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/2ed27de3-d9c7-4657-b7e4-37b71941db77-istio-data\") pod \"e2e-shared-a-8edd65-openshift-default-75f7b769c9-bwvv8\" (UID: \"2ed27de3-d9c7-4657-b7e4-37b71941db77\") " pod="openshift-ingress/e2e-shared-a-8edd65-openshift-default-75f7b769c9-bwvv8" Jul 06 03:15:59.345863 ip-10-0-139-206 kubenswrapper[2557]: I0706 03:15:59.345794 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/2ed27de3-d9c7-4657-b7e4-37b71941db77-istiod-ca-cert\") pod \"e2e-shared-a-8edd65-openshift-default-75f7b769c9-bwvv8\" (UID: \"2ed27de3-d9c7-4657-b7e4-37b71941db77\") " pod="openshift-ingress/e2e-shared-a-8edd65-openshift-default-75f7b769c9-bwvv8" Jul 06 03:15:59.345863 ip-10-0-139-206 kubenswrapper[2557]: I0706 03:15:59.345819 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/2ed27de3-d9c7-4657-b7e4-37b71941db77-workload-certs\") pod \"e2e-shared-a-8edd65-openshift-default-75f7b769c9-bwvv8\" (UID: \"2ed27de3-d9c7-4657-b7e4-37b71941db77\") " pod="openshift-ingress/e2e-shared-a-8edd65-openshift-default-75f7b769c9-bwvv8" Jul 06 03:15:59.347295 ip-10-0-139-206 kubenswrapper[2557]: I0706 03:15:59.347272 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/2ed27de3-d9c7-4657-b7e4-37b71941db77-istio-envoy\") pod \"e2e-shared-a-8edd65-openshift-default-75f7b769c9-bwvv8\" (UID: \"2ed27de3-d9c7-4657-b7e4-37b71941db77\") " pod="openshift-ingress/e2e-shared-a-8edd65-openshift-default-75f7b769c9-bwvv8" Jul 06 03:15:59.347681 ip-10-0-139-206 kubenswrapper[2557]: I0706 03:15:59.347662 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/2ed27de3-d9c7-4657-b7e4-37b71941db77-istio-podinfo\") pod \"e2e-shared-a-8edd65-openshift-default-75f7b769c9-bwvv8\" (UID: \"2ed27de3-d9c7-4657-b7e4-37b71941db77\") " pod="openshift-ingress/e2e-shared-a-8edd65-openshift-default-75f7b769c9-bwvv8" Jul 06 03:15:59.352143 ip-10-0-139-206 kubenswrapper[2557]: I0706 03:15:59.352120 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/2ed27de3-d9c7-4657-b7e4-37b71941db77-istio-token\") pod \"e2e-shared-a-8edd65-openshift-default-75f7b769c9-bwvv8\" (UID: \"2ed27de3-d9c7-4657-b7e4-37b71941db77\") " pod="openshift-ingress/e2e-shared-a-8edd65-openshift-default-75f7b769c9-bwvv8" Jul 06 03:15:59.352304 ip-10-0-139-206 kubenswrapper[2557]: I0706 03:15:59.352281 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-lgb7r\" (UniqueName: \"kubernetes.io/projected/2ed27de3-d9c7-4657-b7e4-37b71941db77-kube-api-access-lgb7r\") pod \"e2e-shared-a-8edd65-openshift-default-75f7b769c9-bwvv8\" (UID: \"2ed27de3-d9c7-4657-b7e4-37b71941db77\") " pod="openshift-ingress/e2e-shared-a-8edd65-openshift-default-75f7b769c9-bwvv8" Jul 06 03:15:59.456868 ip-10-0-139-206 kubenswrapper[2557]: I0706 03:15:59.456836 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-8edd65-openshift-default-75f7b769c9-bwvv8" Jul 06 03:15:59.596282 ip-10-0-139-206 kubenswrapper[2557]: I0706 03:15:59.596251 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-a-8edd65-openshift-default-75f7b769c9-bwvv8"] Jul 06 03:15:59.598287 ip-10-0-139-206 kubenswrapper[2557]: W0706 03:15:59.598258 2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2ed27de3_d9c7_4657_b7e4_37b71941db77.slice/crio-fbf8dc9bebe4aecea4f93b7a2decc667e37c855ca76d7270d1d0fd735461fd2c WatchSource:0}: Error finding container fbf8dc9bebe4aecea4f93b7a2decc667e37c855ca76d7270d1d0fd735461fd2c: Status 404 returned error can't find the container with id fbf8dc9bebe4aecea4f93b7a2decc667e37c855ca76d7270d1d0fd735461fd2c Jul 06 03:15:59.603421 ip-10-0-139-206 kubenswrapper[2557]: I0706 03:15:59.603323 2557 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 06 03:15:59.603533 ip-10-0-139-206 kubenswrapper[2557]: I0706 03:15:59.603468 2557 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 06 03:15:59.603533 ip-10-0-139-206 kubenswrapper[2557]: I0706 03:15:59.603507 2557 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 06 03:15:59.844726 ip-10-0-139-206 kubenswrapper[2557]: I0706 03:15:59.844625 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-8edd65-openshift-default-75f7b769c9-bwvv8" event={"ID":"2ed27de3-d9c7-4657-b7e4-37b71941db77","Type":"ContainerStarted","Data":"ae8dac24113dda4d508f213c5230c23bf936ac2f89293a8beea4735c291fa2f9"} Jul 06 03:15:59.844726 ip-10-0-139-206 kubenswrapper[2557]: I0706 03:15:59.844721 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-8edd65-openshift-default-75f7b769c9-bwvv8" event={"ID":"2ed27de3-d9c7-4657-b7e4-37b71941db77","Type":"ContainerStarted","Data":"fbf8dc9bebe4aecea4f93b7a2decc667e37c855ca76d7270d1d0fd735461fd2c"} Jul 06 03:15:59.865053 ip-10-0-139-206 kubenswrapper[2557]: I0706 03:15:59.864999 2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-shared-a-8edd65-openshift-default-75f7b769c9-bwvv8" podStartSLOduration=0.864984241 podStartE2EDuration="864.984241ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 03:15:59 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 03:15:59.862866171 +0000 UTC m=+2203.669152228" watchObservedRunningTime="2026-07-06 03:15:59.864984241 +0000 UTC m=+2203.671270290" Jul 06 03:16:00.457852 ip-10-0-139-206 kubenswrapper[2557]: I0706 03:16:00.457809 2557 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-a-8edd65-openshift-default-75f7b769c9-bwvv8" Jul 06 03:16:00.462733 ip-10-0-139-206 kubenswrapper[2557]: I0706 03:16:00.462710 2557 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-a-8edd65-openshift-default-75f7b769c9-bwvv8" Jul 06 03:16:00.849078 ip-10-0-139-206 kubenswrapper[2557]: I0706 03:16:00.848984 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-a-8edd65-openshift-default-75f7b769c9-bwvv8" Jul 06 03:16:00.849948 ip-10-0-139-206 kubenswrapper[2557]: I0706 03:16:00.849928 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-a-8edd65-openshift-default-75f7b769c9-bwvv8" Jul 06 03:16:09.362644 ip-10-0-139-206 kubenswrapper[2557]: I0706 03:16:09.362601 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-srzrr"] Jul 06 03:16:16.890551 ip-10-0-139-206 kubenswrapper[2557]: I0706 03:16:16.890501 2557 patch_prober.go:28] interesting pod/e2e-shared-a-8edd65-openshift-default-75f7b769c9-bwvv8 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.133.0.37:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body= Jul 06 03:16:16.891083 ip-10-0-139-206 kubenswrapper[2557]: I0706 03:16:16.890581 2557 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-shared-a-8edd65-openshift-default-75f7b769c9-bwvv8" podUID="2ed27de3-d9c7-4657-b7e4-37b71941db77" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.37:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" Jul 06 03:16:17.935646 ip-10-0-139-206 kubenswrapper[2557]: I0706 03:16:17.935610 2557 generic.go:356] "Generic (PLEG): container finished" podID="2ed27de3-d9c7-4657-b7e4-37b71941db77" containerID="ae8dac24113dda4d508f213c5230c23bf936ac2f89293a8beea4735c291fa2f9" exitCode=0 Jul 06 03:16:17.936185 ip-10-0-139-206 kubenswrapper[2557]: I0706 03:16:17.935687 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-8edd65-openshift-default-75f7b769c9-bwvv8" event={"ID":"2ed27de3-d9c7-4657-b7e4-37b71941db77","Type":"ContainerDied","Data":"ae8dac24113dda4d508f213c5230c23bf936ac2f89293a8beea4735c291fa2f9"} Jul 06 03:16:17.936260 ip-10-0-139-206 kubenswrapper[2557]: I0706 03:16:17.936197 2557 scope.go:117] "RemoveContainer" containerID="ae8dac24113dda4d508f213c5230c23bf936ac2f89293a8beea4735c291fa2f9" Jul 06 03:16:17.937245 ip-10-0-139-206 kubenswrapper[2557]: I0706 03:16:17.937216 2557 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 06 03:16:17.937307 ip-10-0-139-206 kubenswrapper[2557]: I0706 03:16:17.937288 2557 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 06 03:16:17.937364 ip-10-0-139-206 kubenswrapper[2557]: I0706 03:16:17.937318 2557 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 06 03:16:18.947977 ip-10-0-139-206 kubenswrapper[2557]: I0706 03:16:18.947941 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-8edd65-openshift-default-75f7b769c9-bwvv8" event={"ID":"2ed27de3-d9c7-4657-b7e4-37b71941db77","Type":"ContainerStarted","Data":"7aaddc22614f8d603030ab23ee7200b2a51283a76c47a98edecac489468e9ebe"} Jul 06 03:16:19.457553 ip-10-0-139-206 kubenswrapper[2557]: I0706 03:16:19.457515 2557 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-a-8edd65-openshift-default-75f7b769c9-bwvv8" Jul 06 03:16:19.457758 ip-10-0-139-206 kubenswrapper[2557]: I0706 03:16:19.457734 2557 patch_prober.go:28] interesting pod/e2e-shared-a-8edd65-openshift-default-75f7b769c9-bwvv8 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.37:15021/healthz/ready\": dial tcp 10.133.0.37:15021: connect: connection refused" start-of-body= Jul 06 03:16:19.457816 ip-10-0-139-206 kubenswrapper[2557]: I0706 03:16:19.457788 2557 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-8edd65-openshift-default-75f7b769c9-bwvv8" podUID="2ed27de3-d9c7-4657-b7e4-37b71941db77" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.37:15021/healthz/ready\": dial tcp 10.133.0.37:15021: connect: connection refused" Jul 06 03:16:20.457767 ip-10-0-139-206 kubenswrapper[2557]: I0706 03:16:20.457725 2557 patch_prober.go:28] interesting pod/e2e-shared-a-8edd65-openshift-default-75f7b769c9-bwvv8 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.37:15021/healthz/ready\": dial tcp 10.133.0.37:15021: connect: connection refused" start-of-body= Jul 06 03:16:20.458202 ip-10-0-139-206 kubenswrapper[2557]: I0706 03:16:20.457784 2557 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-8edd65-openshift-default-75f7b769c9-bwvv8" podUID="2ed27de3-d9c7-4657-b7e4-37b71941db77" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.37:15021/healthz/ready\": dial tcp 10.133.0.37:15021: connect: connection refused" Jul 06 03:16:21.458101 ip-10-0-139-206 kubenswrapper[2557]: I0706 03:16:21.458058 2557 patch_prober.go:28] interesting pod/e2e-shared-a-8edd65-openshift-default-75f7b769c9-bwvv8 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.37:15021/healthz/ready\": dial tcp 10.133.0.37:15021: connect: connection refused" start-of-body= Jul 06 03:16:21.458618 ip-10-0-139-206 kubenswrapper[2557]: I0706 03:16:21.458126 2557 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-8edd65-openshift-default-75f7b769c9-bwvv8" podUID="2ed27de3-d9c7-4657-b7e4-37b71941db77" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.37:15021/healthz/ready\": dial tcp 10.133.0.37:15021: connect: connection refused" Jul 06 03:16:22.457466 ip-10-0-139-206 kubenswrapper[2557]: I0706 03:16:22.457428 2557 patch_prober.go:28] interesting pod/e2e-shared-a-8edd65-openshift-default-75f7b769c9-bwvv8 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.37:15021/healthz/ready\": dial tcp 10.133.0.37:15021: connect: connection refused" start-of-body= Jul 06 03:16:22.457661 ip-10-0-139-206 kubenswrapper[2557]: I0706 03:16:22.457483 2557 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-8edd65-openshift-default-75f7b769c9-bwvv8" podUID="2ed27de3-d9c7-4657-b7e4-37b71941db77" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.37:15021/healthz/ready\": dial tcp 10.133.0.37:15021: connect: connection refused" Jul 06 03:16:23.461065 ip-10-0-139-206 kubenswrapper[2557]: I0706 03:16:23.461027 2557 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-a-8edd65-openshift-default-75f7b769c9-bwvv8" Jul 06 03:16:23.461462 ip-10-0-139-206 kubenswrapper[2557]: I0706 03:16:23.461359 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-a-8edd65-openshift-default-75f7b769c9-bwvv8" Jul 06 03:16:23.461934 ip-10-0-139-206 kubenswrapper[2557]: I0706 03:16:23.461910 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-a-8edd65-openshift-default-75f7b769c9-bwvv8" Jul 06 03:16:23.771251 ip-10-0-139-206 kubenswrapper[2557]: I0706 03:16:23.771150 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-srzrr"] Jul 06 03:16:37.577307 ip-10-0-139-206 kubenswrapper[2557]: I0706 03:16:37.577267 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-srzrr"] Jul 06 03:17:01.068478 ip-10-0-139-206 kubenswrapper[2557]: I0706 03:17:01.068446 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-srzrr"] Jul 06 03:17:21.464879 ip-10-0-139-206 kubenswrapper[2557]: I0706 03:17:21.464841 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-srzrr"] Jul 06 03:17:39.269219 ip-10-0-139-206 kubenswrapper[2557]: I0706 03:17:39.269186 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-srzrr"] Jul 06 03:17:53.767706 ip-10-0-139-206 kubenswrapper[2557]: I0706 03:17:53.767674 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-srzrr"] Jul 06 03:18:11.066854 ip-10-0-139-206 kubenswrapper[2557]: I0706 03:18:11.066766 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-srzrr"] Jul 06 03:18:27.767589 ip-10-0-139-206 kubenswrapper[2557]: I0706 03:18:27.767552 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-srzrr"] Jul 06 03:18:44.871621 ip-10-0-139-206 kubenswrapper[2557]: I0706 03:18:44.867205 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-srzrr"] Jul 06 03:19:02.063828 ip-10-0-139-206 kubenswrapper[2557]: I0706 03:19:02.063787 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-srzrr"] Jul 06 03:19:08.668962 ip-10-0-139-206 kubenswrapper[2557]: I0706 03:19:08.668931 2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-a-8edd65/sub-test-model-8edd65-kserve-659cb4fc9-v4cwq"] Jul 06 03:19:08.669450 ip-10-0-139-206 kubenswrapper[2557]: I0706 03:19:08.669418 2557 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1f1dc8e1-28c7-4004-bf18-49a9ce850c0a" containerName="istio-proxy" Jul 06 03:19:08.669450 ip-10-0-139-206 kubenswrapper[2557]: I0706 03:19:08.669436 2557 state_mem.go:107] "Deleted CPUSet assignment" podUID="1f1dc8e1-28c7-4004-bf18-49a9ce850c0a" containerName="istio-proxy" Jul 06 03:19:08.672597 ip-10-0-139-206 kubenswrapper[2557]: I0706 03:19:08.672575 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-8edd65/sub-test-model-8edd65-kserve-659cb4fc9-v4cwq" Jul 06 03:19:08.674988 ip-10-0-139-206 kubenswrapper[2557]: I0706 03:19:08.674967 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-8edd65\"/\"default-dockercfg-trrx2\"" Jul 06 03:19:08.675319 ip-10-0-139-206 kubenswrapper[2557]: I0706 03:19:08.675299 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-8edd65\"/\"kube-root-ca.crt\"" Jul 06 03:19:08.676006 ip-10-0-139-206 kubenswrapper[2557]: I0706 03:19:08.675990 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-8edd65\"/\"openshift-service-ca.crt\"" Jul 06 03:19:08.676101 ip-10-0-139-206 kubenswrapper[2557]: I0706 03:19:08.676087 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-8edd65\"/\"sub-test-model-8edd65-kserve-self-signed-certs\"" Jul 06 03:19:08.681924 ip-10-0-139-206 kubenswrapper[2557]: I0706 03:19:08.681900 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-8edd65/sub-test-model-8edd65-kserve-659cb4fc9-v4cwq"] Jul 06 03:19:08.711764 ip-10-0-139-206 kubenswrapper[2557]: I0706 03:19:08.711724 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f1f937ba-6535-4e96-a104-f73e6762e206-tls-certs\") pod \"sub-test-model-8edd65-kserve-659cb4fc9-v4cwq\" (UID: \"f1f937ba-6535-4e96-a104-f73e6762e206\") " pod="ai-tenant-e2e-shared-a-8edd65/sub-test-model-8edd65-kserve-659cb4fc9-v4cwq" Jul 06 03:19:08.711966 ip-10-0-139-206 kubenswrapper[2557]: I0706 03:19:08.711790 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f1f937ba-6535-4e96-a104-f73e6762e206-home\") pod \"sub-test-model-8edd65-kserve-659cb4fc9-v4cwq\" (UID: \"f1f937ba-6535-4e96-a104-f73e6762e206\") " pod="ai-tenant-e2e-shared-a-8edd65/sub-test-model-8edd65-kserve-659cb4fc9-v4cwq" Jul 06 03:19:08.711966 ip-10-0-139-206 kubenswrapper[2557]: I0706 03:19:08.711861 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f1f937ba-6535-4e96-a104-f73e6762e206-dshm\") pod \"sub-test-model-8edd65-kserve-659cb4fc9-v4cwq\" (UID: \"f1f937ba-6535-4e96-a104-f73e6762e206\") " pod="ai-tenant-e2e-shared-a-8edd65/sub-test-model-8edd65-kserve-659cb4fc9-v4cwq" Jul 06 03:19:08.711966 ip-10-0-139-206 kubenswrapper[2557]: I0706 03:19:08.711890 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f1f937ba-6535-4e96-a104-f73e6762e206-model-cache\") pod \"sub-test-model-8edd65-kserve-659cb4fc9-v4cwq\" (UID: \"f1f937ba-6535-4e96-a104-f73e6762e206\") " pod="ai-tenant-e2e-shared-a-8edd65/sub-test-model-8edd65-kserve-659cb4fc9-v4cwq" Jul 06 03:19:08.711966 ip-10-0-139-206 kubenswrapper[2557]: I0706 03:19:08.711923 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f1f937ba-6535-4e96-a104-f73e6762e206-tmp-dir\") pod \"sub-test-model-8edd65-kserve-659cb4fc9-v4cwq\" (UID: \"f1f937ba-6535-4e96-a104-f73e6762e206\") " pod="ai-tenant-e2e-shared-a-8edd65/sub-test-model-8edd65-kserve-659cb4fc9-v4cwq" Jul 06 03:19:08.711966 ip-10-0-139-206 kubenswrapper[2557]: I0706 03:19:08.711950 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kqrm4\" (UniqueName: \"kubernetes.io/projected/f1f937ba-6535-4e96-a104-f73e6762e206-kube-api-access-kqrm4\") pod \"sub-test-model-8edd65-kserve-659cb4fc9-v4cwq\" (UID: \"f1f937ba-6535-4e96-a104-f73e6762e206\") " pod="ai-tenant-e2e-shared-a-8edd65/sub-test-model-8edd65-kserve-659cb4fc9-v4cwq" Jul 06 03:19:08.812634 ip-10-0-139-206 kubenswrapper[2557]: I0706 03:19:08.812596 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f1f937ba-6535-4e96-a104-f73e6762e206-home\") pod \"sub-test-model-8edd65-kserve-659cb4fc9-v4cwq\" (UID: \"f1f937ba-6535-4e96-a104-f73e6762e206\") " pod="ai-tenant-e2e-shared-a-8edd65/sub-test-model-8edd65-kserve-659cb4fc9-v4cwq" Jul 06 03:19:08.812857 ip-10-0-139-206 kubenswrapper[2557]: I0706 03:19:08.812672 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f1f937ba-6535-4e96-a104-f73e6762e206-dshm\") pod \"sub-test-model-8edd65-kserve-659cb4fc9-v4cwq\" (UID: \"f1f937ba-6535-4e96-a104-f73e6762e206\") " pod="ai-tenant-e2e-shared-a-8edd65/sub-test-model-8edd65-kserve-659cb4fc9-v4cwq" Jul 06 03:19:08.812857 ip-10-0-139-206 kubenswrapper[2557]: I0706 03:19:08.812703 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f1f937ba-6535-4e96-a104-f73e6762e206-model-cache\") pod \"sub-test-model-8edd65-kserve-659cb4fc9-v4cwq\" (UID: \"f1f937ba-6535-4e96-a104-f73e6762e206\") " pod="ai-tenant-e2e-shared-a-8edd65/sub-test-model-8edd65-kserve-659cb4fc9-v4cwq" Jul 06 03:19:08.812857 ip-10-0-139-206 kubenswrapper[2557]: I0706 03:19:08.812726 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f1f937ba-6535-4e96-a104-f73e6762e206-tmp-dir\") pod \"sub-test-model-8edd65-kserve-659cb4fc9-v4cwq\" (UID: \"f1f937ba-6535-4e96-a104-f73e6762e206\") " pod="ai-tenant-e2e-shared-a-8edd65/sub-test-model-8edd65-kserve-659cb4fc9-v4cwq" Jul 06 03:19:08.812857 ip-10-0-139-206 kubenswrapper[2557]: I0706 03:19:08.812752 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kqrm4\" (UniqueName: \"kubernetes.io/projected/f1f937ba-6535-4e96-a104-f73e6762e206-kube-api-access-kqrm4\") pod \"sub-test-model-8edd65-kserve-659cb4fc9-v4cwq\" (UID: \"f1f937ba-6535-4e96-a104-f73e6762e206\") " pod="ai-tenant-e2e-shared-a-8edd65/sub-test-model-8edd65-kserve-659cb4fc9-v4cwq" Jul 06 03:19:08.813076 ip-10-0-139-206 kubenswrapper[2557]: I0706 03:19:08.812876 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f1f937ba-6535-4e96-a104-f73e6762e206-tls-certs\") pod \"sub-test-model-8edd65-kserve-659cb4fc9-v4cwq\" (UID: \"f1f937ba-6535-4e96-a104-f73e6762e206\") " pod="ai-tenant-e2e-shared-a-8edd65/sub-test-model-8edd65-kserve-659cb4fc9-v4cwq" Jul 06 03:19:08.813137 ip-10-0-139-206 kubenswrapper[2557]: I0706 03:19:08.813102 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f1f937ba-6535-4e96-a104-f73e6762e206-home\") pod \"sub-test-model-8edd65-kserve-659cb4fc9-v4cwq\" (UID: \"f1f937ba-6535-4e96-a104-f73e6762e206\") " pod="ai-tenant-e2e-shared-a-8edd65/sub-test-model-8edd65-kserve-659cb4fc9-v4cwq" Jul 06 03:19:08.813225 ip-10-0-139-206 kubenswrapper[2557]: I0706 03:19:08.813205 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f1f937ba-6535-4e96-a104-f73e6762e206-tmp-dir\") pod \"sub-test-model-8edd65-kserve-659cb4fc9-v4cwq\" (UID: \"f1f937ba-6535-4e96-a104-f73e6762e206\") " pod="ai-tenant-e2e-shared-a-8edd65/sub-test-model-8edd65-kserve-659cb4fc9-v4cwq" Jul 06 03:19:08.813414 ip-10-0-139-206 kubenswrapper[2557]: I0706 03:19:08.813387 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f1f937ba-6535-4e96-a104-f73e6762e206-model-cache\") pod \"sub-test-model-8edd65-kserve-659cb4fc9-v4cwq\" (UID: \"f1f937ba-6535-4e96-a104-f73e6762e206\") " pod="ai-tenant-e2e-shared-a-8edd65/sub-test-model-8edd65-kserve-659cb4fc9-v4cwq" Jul 06 03:19:08.815055 ip-10-0-139-206 kubenswrapper[2557]: I0706 03:19:08.815022 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f1f937ba-6535-4e96-a104-f73e6762e206-dshm\") pod \"sub-test-model-8edd65-kserve-659cb4fc9-v4cwq\" (UID: \"f1f937ba-6535-4e96-a104-f73e6762e206\") " pod="ai-tenant-e2e-shared-a-8edd65/sub-test-model-8edd65-kserve-659cb4fc9-v4cwq" Jul 06 03:19:08.815378 ip-10-0-139-206 kubenswrapper[2557]: I0706 03:19:08.815339 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f1f937ba-6535-4e96-a104-f73e6762e206-tls-certs\") pod \"sub-test-model-8edd65-kserve-659cb4fc9-v4cwq\" (UID: \"f1f937ba-6535-4e96-a104-f73e6762e206\") " pod="ai-tenant-e2e-shared-a-8edd65/sub-test-model-8edd65-kserve-659cb4fc9-v4cwq" Jul 06 03:19:08.820523 ip-10-0-139-206 kubenswrapper[2557]: I0706 03:19:08.820495 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-kqrm4\" (UniqueName: \"kubernetes.io/projected/f1f937ba-6535-4e96-a104-f73e6762e206-kube-api-access-kqrm4\") pod \"sub-test-model-8edd65-kserve-659cb4fc9-v4cwq\" (UID: \"f1f937ba-6535-4e96-a104-f73e6762e206\") " pod="ai-tenant-e2e-shared-a-8edd65/sub-test-model-8edd65-kserve-659cb4fc9-v4cwq" Jul 06 03:19:08.982537 ip-10-0-139-206 kubenswrapper[2557]: I0706 03:19:08.982500 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-8edd65/sub-test-model-8edd65-kserve-659cb4fc9-v4cwq" Jul 06 03:19:09.124165 ip-10-0-139-206 kubenswrapper[2557]: I0706 03:19:09.124072 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-8edd65/sub-test-model-8edd65-kserve-659cb4fc9-v4cwq"] Jul 06 03:19:09.126505 ip-10-0-139-206 kubenswrapper[2557]: W0706 03:19:09.126477 2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podf1f937ba_6535_4e96_a104_f73e6762e206.slice/crio-66286f1a455b58d47b34131a1c5ee3f36cb2b7cb80d0f9043f6f9b759eb25124 WatchSource:0}: Error finding container 66286f1a455b58d47b34131a1c5ee3f36cb2b7cb80d0f9043f6f9b759eb25124: Status 404 returned error can't find the container with id 66286f1a455b58d47b34131a1c5ee3f36cb2b7cb80d0f9043f6f9b759eb25124 Jul 06 03:19:09.128620 ip-10-0-139-206 kubenswrapper[2557]: I0706 03:19:09.128600 2557 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 06 03:19:09.759064 ip-10-0-139-206 kubenswrapper[2557]: I0706 03:19:09.759030 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-8edd65/sub-test-model-8edd65-kserve-659cb4fc9-v4cwq" event={"ID":"f1f937ba-6535-4e96-a104-f73e6762e206","Type":"ContainerStarted","Data":"66286f1a455b58d47b34131a1c5ee3f36cb2b7cb80d0f9043f6f9b759eb25124"} Jul 06 03:19:10.766616 ip-10-0-139-206 kubenswrapper[2557]: I0706 03:19:10.766578 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-8edd65/sub-test-model-8edd65-kserve-659cb4fc9-v4cwq" event={"ID":"f1f937ba-6535-4e96-a104-f73e6762e206","Type":"ContainerStarted","Data":"6e9e57c4e377aa89d74ffba1130af44995c48602c65b4f3a094dac2a3fbf672f"} Jul 06 03:19:10.783966 ip-10-0-139-206 kubenswrapper[2557]: I0706 03:19:10.783916 2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-a-8edd65/sub-test-model-8edd65-kserve-659cb4fc9-v4cwq" podStartSLOduration=1.657144116 podStartE2EDuration="2.783899256s" totalImagesPullingTime="1.12675514s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 03:19:08 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 03:19:10.782554658 +0000 UTC m=+2394.588840741" watchObservedRunningTime="2026-07-06 03:19:10.783899256 +0000 UTC m=+2394.590185313" Jul 06 03:19:17.371474 ip-10-0-139-206 kubenswrapper[2557]: I0706 03:19:17.371447 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-cw2f7_efcf5e11-1e5d-4e2a-9b38-303a8f0787c4/ovn-acl-logging/0.log" Jul 06 03:19:17.374625 ip-10-0-139-206 kubenswrapper[2557]: I0706 03:19:17.374602 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-cw2f7_efcf5e11-1e5d-4e2a-9b38-303a8f0787c4/ovn-acl-logging/0.log" Jul 06 03:19:18.983097 ip-10-0-139-206 kubenswrapper[2557]: I0706 03:19:18.983055 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-a-8edd65/sub-test-model-8edd65-kserve-659cb4fc9-v4cwq" Jul 06 03:19:18.983534 ip-10-0-139-206 kubenswrapper[2557]: I0706 03:19:18.983112 2557 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-a-8edd65/sub-test-model-8edd65-kserve-659cb4fc9-v4cwq" Jul 06 03:19:18.995963 ip-10-0-139-206 kubenswrapper[2557]: I0706 03:19:18.995934 2557 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-a-8edd65/sub-test-model-8edd65-kserve-659cb4fc9-v4cwq" Jul 06 03:19:19.822677 ip-10-0-139-206 kubenswrapper[2557]: I0706 03:19:19.822650 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-a-8edd65/sub-test-model-8edd65-kserve-659cb4fc9-v4cwq" Jul 06 03:19:21.058449 ip-10-0-139-206 kubenswrapper[2557]: I0706 03:19:21.058413 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-srzrr"] Jul 06 03:19:43.767394 ip-10-0-139-206 kubenswrapper[2557]: I0706 03:19:43.767286 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-srzrr"] Jul 06 03:20:03.264064 ip-10-0-139-206 kubenswrapper[2557]: I0706 03:20:03.264030 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-srzrr"] Jul 06 03:20:19.172536 ip-10-0-139-206 kubenswrapper[2557]: I0706 03:20:19.172497 2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-8edd65/sub-test-model-8edd65-kserve-659cb4fc9-v4cwq"] Jul 06 03:20:23.665595 ip-10-0-139-206 kubenswrapper[2557]: I0706 03:20:23.665552 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-srzrr"] Jul 06 03:20:34.192858 ip-10-0-139-206 kubenswrapper[2557]: I0706 03:20:34.192791 2557 kuberuntime_container.go:872] "Killing container with a grace period" pod="ai-tenant-e2e-shared-a-8edd65/sub-test-model-8edd65-kserve-659cb4fc9-v4cwq" podUID="f1f937ba-6535-4e96-a104-f73e6762e206" containerName="main" containerID="cri-o://6e9e57c4e377aa89d74ffba1130af44995c48602c65b4f3a094dac2a3fbf672f" gracePeriod=45 Jul 06 03:20:34.442682 ip-10-0-139-206 kubenswrapper[2557]: I0706 03:20:34.442658 2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-8edd65/sub-test-model-8edd65-kserve-659cb4fc9-v4cwq" Jul 06 03:20:34.474440 ip-10-0-139-206 kubenswrapper[2557]: I0706 03:20:34.474337 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f1f937ba-6535-4e96-a104-f73e6762e206-tls-certs\") pod \"f1f937ba-6535-4e96-a104-f73e6762e206\" (UID: \"f1f937ba-6535-4e96-a104-f73e6762e206\") " Jul 06 03:20:34.474440 ip-10-0-139-206 kubenswrapper[2557]: I0706 03:20:34.474383 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f1f937ba-6535-4e96-a104-f73e6762e206-tmp-dir\") pod \"f1f937ba-6535-4e96-a104-f73e6762e206\" (UID: \"f1f937ba-6535-4e96-a104-f73e6762e206\") " Jul 06 03:20:34.474440 ip-10-0-139-206 kubenswrapper[2557]: I0706 03:20:34.474415 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f1f937ba-6535-4e96-a104-f73e6762e206-home\") pod \"f1f937ba-6535-4e96-a104-f73e6762e206\" (UID: \"f1f937ba-6535-4e96-a104-f73e6762e206\") " Jul 06 03:20:34.474718 ip-10-0-139-206 kubenswrapper[2557]: I0706 03:20:34.474468 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f1f937ba-6535-4e96-a104-f73e6762e206-dshm\") pod \"f1f937ba-6535-4e96-a104-f73e6762e206\" (UID: \"f1f937ba-6535-4e96-a104-f73e6762e206\") " Jul 06 03:20:34.474718 ip-10-0-139-206 kubenswrapper[2557]: I0706 03:20:34.474532 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-kqrm4\" (UniqueName: \"kubernetes.io/projected/f1f937ba-6535-4e96-a104-f73e6762e206-kube-api-access-kqrm4\") pod \"f1f937ba-6535-4e96-a104-f73e6762e206\" (UID: \"f1f937ba-6535-4e96-a104-f73e6762e206\") " Jul 06 03:20:34.474718 ip-10-0-139-206 kubenswrapper[2557]: I0706 03:20:34.474550 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f1f937ba-6535-4e96-a104-f73e6762e206-model-cache\") pod \"f1f937ba-6535-4e96-a104-f73e6762e206\" (UID: \"f1f937ba-6535-4e96-a104-f73e6762e206\") " Jul 06 03:20:34.474718 ip-10-0-139-206 kubenswrapper[2557]: I0706 03:20:34.474626 2557 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f1f937ba-6535-4e96-a104-f73e6762e206-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "f1f937ba-6535-4e96-a104-f73e6762e206" (UID: "f1f937ba-6535-4e96-a104-f73e6762e206"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 03:20:34.474937 ip-10-0-139-206 kubenswrapper[2557]: I0706 03:20:34.474736 2557 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f1f937ba-6535-4e96-a104-f73e6762e206-home" (OuterVolumeSpecName: "home") pod "f1f937ba-6535-4e96-a104-f73e6762e206" (UID: "f1f937ba-6535-4e96-a104-f73e6762e206"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 03:20:34.474937 ip-10-0-139-206 kubenswrapper[2557]: I0706 03:20:34.474820 2557 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f1f937ba-6535-4e96-a104-f73e6762e206-tmp-dir\") on node \"ip-10-0-139-206.ec2.internal\" DevicePath \"\"" Jul 06 03:20:34.474937 ip-10-0-139-206 kubenswrapper[2557]: I0706 03:20:34.474841 2557 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f1f937ba-6535-4e96-a104-f73e6762e206-home\") on node \"ip-10-0-139-206.ec2.internal\" DevicePath \"\"" Jul 06 03:20:34.474937 ip-10-0-139-206 kubenswrapper[2557]: I0706 03:20:34.474849 2557 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f1f937ba-6535-4e96-a104-f73e6762e206-model-cache" (OuterVolumeSpecName: "model-cache") pod "f1f937ba-6535-4e96-a104-f73e6762e206" (UID: "f1f937ba-6535-4e96-a104-f73e6762e206"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 03:20:34.476640 ip-10-0-139-206 kubenswrapper[2557]: I0706 03:20:34.476608 2557 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f1f937ba-6535-4e96-a104-f73e6762e206-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "f1f937ba-6535-4e96-a104-f73e6762e206" (UID: "f1f937ba-6535-4e96-a104-f73e6762e206"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 06 03:20:34.476779 ip-10-0-139-206 kubenswrapper[2557]: I0706 03:20:34.476655 2557 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f1f937ba-6535-4e96-a104-f73e6762e206-kube-api-access-kqrm4" (OuterVolumeSpecName: "kube-api-access-kqrm4") pod "f1f937ba-6535-4e96-a104-f73e6762e206" (UID: "f1f937ba-6535-4e96-a104-f73e6762e206"). InnerVolumeSpecName "kube-api-access-kqrm4". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 06 03:20:34.476779 ip-10-0-139-206 kubenswrapper[2557]: I0706 03:20:34.476736 2557 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f1f937ba-6535-4e96-a104-f73e6762e206-dshm" (OuterVolumeSpecName: "dshm") pod "f1f937ba-6535-4e96-a104-f73e6762e206" (UID: "f1f937ba-6535-4e96-a104-f73e6762e206"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 03:20:34.575787 ip-10-0-139-206 kubenswrapper[2557]: I0706 03:20:34.575750 2557 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f1f937ba-6535-4e96-a104-f73e6762e206-dshm\") on node \"ip-10-0-139-206.ec2.internal\" DevicePath \"\"" Jul 06 03:20:34.575787 ip-10-0-139-206 kubenswrapper[2557]: I0706 03:20:34.575779 2557 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-kqrm4\" (UniqueName: \"kubernetes.io/projected/f1f937ba-6535-4e96-a104-f73e6762e206-kube-api-access-kqrm4\") on node \"ip-10-0-139-206.ec2.internal\" DevicePath \"\"" Jul 06 03:20:34.575787 ip-10-0-139-206 kubenswrapper[2557]: I0706 03:20:34.575791 2557 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f1f937ba-6535-4e96-a104-f73e6762e206-model-cache\") on node \"ip-10-0-139-206.ec2.internal\" DevicePath \"\"" Jul 06 03:20:34.575787 ip-10-0-139-206 kubenswrapper[2557]: I0706 03:20:34.575800 2557 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f1f937ba-6535-4e96-a104-f73e6762e206-tls-certs\") on node \"ip-10-0-139-206.ec2.internal\" DevicePath \"\"" Jul 06 03:20:35.176951 ip-10-0-139-206 kubenswrapper[2557]: I0706 03:20:35.176911 2557 generic.go:356] "Generic (PLEG): container finished" podID="f1f937ba-6535-4e96-a104-f73e6762e206" containerID="6e9e57c4e377aa89d74ffba1130af44995c48602c65b4f3a094dac2a3fbf672f" exitCode=0 Jul 06 03:20:35.177137 ip-10-0-139-206 kubenswrapper[2557]: I0706 03:20:35.176975 2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-8edd65/sub-test-model-8edd65-kserve-659cb4fc9-v4cwq" Jul 06 03:20:35.177137 ip-10-0-139-206 kubenswrapper[2557]: I0706 03:20:35.176998 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-8edd65/sub-test-model-8edd65-kserve-659cb4fc9-v4cwq" event={"ID":"f1f937ba-6535-4e96-a104-f73e6762e206","Type":"ContainerDied","Data":"6e9e57c4e377aa89d74ffba1130af44995c48602c65b4f3a094dac2a3fbf672f"} Jul 06 03:20:35.177137 ip-10-0-139-206 kubenswrapper[2557]: I0706 03:20:35.177044 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-8edd65/sub-test-model-8edd65-kserve-659cb4fc9-v4cwq" event={"ID":"f1f937ba-6535-4e96-a104-f73e6762e206","Type":"ContainerDied","Data":"66286f1a455b58d47b34131a1c5ee3f36cb2b7cb80d0f9043f6f9b759eb25124"} Jul 06 03:20:35.177137 ip-10-0-139-206 kubenswrapper[2557]: I0706 03:20:35.177062 2557 scope.go:117] "RemoveContainer" containerID="6e9e57c4e377aa89d74ffba1130af44995c48602c65b4f3a094dac2a3fbf672f" Jul 06 03:20:35.186120 ip-10-0-139-206 kubenswrapper[2557]: I0706 03:20:35.186103 2557 scope.go:117] "RemoveContainer" containerID="6e9e57c4e377aa89d74ffba1130af44995c48602c65b4f3a094dac2a3fbf672f" Jul 06 03:20:35.186417 ip-10-0-139-206 kubenswrapper[2557]: E0706 03:20:35.186399 2557 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6e9e57c4e377aa89d74ffba1130af44995c48602c65b4f3a094dac2a3fbf672f\": container with ID starting with 6e9e57c4e377aa89d74ffba1130af44995c48602c65b4f3a094dac2a3fbf672f not found: ID does not exist" containerID="6e9e57c4e377aa89d74ffba1130af44995c48602c65b4f3a094dac2a3fbf672f" Jul 06 03:20:35.186475 ip-10-0-139-206 kubenswrapper[2557]: I0706 03:20:35.186423 2557 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6e9e57c4e377aa89d74ffba1130af44995c48602c65b4f3a094dac2a3fbf672f"} err="failed to get container status \"6e9e57c4e377aa89d74ffba1130af44995c48602c65b4f3a094dac2a3fbf672f\": rpc error: code = NotFound desc = could not find container \"6e9e57c4e377aa89d74ffba1130af44995c48602c65b4f3a094dac2a3fbf672f\": container with ID starting with 6e9e57c4e377aa89d74ffba1130af44995c48602c65b4f3a094dac2a3fbf672f not found: ID does not exist" Jul 06 03:20:35.197243 ip-10-0-139-206 kubenswrapper[2557]: I0706 03:20:35.197215 2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-8edd65/sub-test-model-8edd65-kserve-659cb4fc9-v4cwq"] Jul 06 03:20:35.200704 ip-10-0-139-206 kubenswrapper[2557]: I0706 03:20:35.200677 2557 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-a-8edd65/sub-test-model-8edd65-kserve-659cb4fc9-v4cwq"] Jul 06 03:20:35.302973 ip-10-0-139-206 kubenswrapper[2557]: I0706 03:20:35.302934 2557 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f1f937ba-6535-4e96-a104-f73e6762e206" path="/var/lib/kubelet/pods/f1f937ba-6535-4e96-a104-f73e6762e206/volumes" Jul 06 03:20:38.765537 ip-10-0-139-206 kubenswrapper[2557]: I0706 03:20:38.765500 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-srzrr"] Jul 06 03:21:01.564899 ip-10-0-139-206 kubenswrapper[2557]: I0706 03:21:01.564823 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-srzrr"] Jul 06 03:21:17.444803 ip-10-0-139-206 kubenswrapper[2557]: I0706 03:21:17.444774 2557 scope.go:117] "RemoveContainer" containerID="c2b6606a8c1f8980a662c09fe583962782c11493ef8b3029c738bba7934a512c" Jul 06 03:21:21.216228 ip-10-0-139-206 kubenswrapper[2557]: I0706 03:21:21.216190 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-srzrr"] Jul 06 03:21:35.363650 ip-10-0-139-206 kubenswrapper[2557]: I0706 03:21:35.363619 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-srzrr"] Jul 06 03:21:48.661752 ip-10-0-139-206 kubenswrapper[2557]: I0706 03:21:48.661712 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-srzrr"] Jul 06 03:22:17.468119 ip-10-0-139-206 kubenswrapper[2557]: I0706 03:22:17.468089 2557 scope.go:117] "RemoveContainer" containerID="ff53f1fe292d130688920f9571157ba83063b35ae3fc877fabcb146dda3838ff" Jul 06 03:22:25.400296 ip-10-0-139-206 kubenswrapper[2557]: I0706 03:22:25.400261 2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-a-8edd65-openshift-default-75f7b769c9-bwvv8"] Jul 06 03:22:25.400841 ip-10-0-139-206 kubenswrapper[2557]: I0706 03:22:25.400526 2557 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-ingress/e2e-shared-a-8edd65-openshift-default-75f7b769c9-bwvv8" podUID="2ed27de3-d9c7-4657-b7e4-37b71941db77" containerName="istio-proxy" containerID="cri-o://7aaddc22614f8d603030ab23ee7200b2a51283a76c47a98edecac489468e9ebe" gracePeriod=30 Jul 06 03:22:29.461984 ip-10-0-139-206 kubenswrapper[2557]: I0706 03:22:29.461947 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-srzrr"] Jul 06 03:22:30.654299 ip-10-0-139-206 kubenswrapper[2557]: I0706 03:22:30.654274 2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-8edd65-openshift-default-75f7b769c9-bwvv8" Jul 06 03:22:30.728923 ip-10-0-139-206 kubenswrapper[2557]: I0706 03:22:30.728884 2557 generic.go:356] "Generic (PLEG): container finished" podID="2ed27de3-d9c7-4657-b7e4-37b71941db77" containerID="7aaddc22614f8d603030ab23ee7200b2a51283a76c47a98edecac489468e9ebe" exitCode=0 Jul 06 03:22:30.728923 ip-10-0-139-206 kubenswrapper[2557]: I0706 03:22:30.728929 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-8edd65-openshift-default-75f7b769c9-bwvv8" event={"ID":"2ed27de3-d9c7-4657-b7e4-37b71941db77","Type":"ContainerDied","Data":"7aaddc22614f8d603030ab23ee7200b2a51283a76c47a98edecac489468e9ebe"} Jul 06 03:22:30.729135 ip-10-0-139-206 kubenswrapper[2557]: I0706 03:22:30.728937 2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-8edd65-openshift-default-75f7b769c9-bwvv8" Jul 06 03:22:30.729135 ip-10-0-139-206 kubenswrapper[2557]: I0706 03:22:30.728973 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-8edd65-openshift-default-75f7b769c9-bwvv8" event={"ID":"2ed27de3-d9c7-4657-b7e4-37b71941db77","Type":"ContainerDied","Data":"fbf8dc9bebe4aecea4f93b7a2decc667e37c855ca76d7270d1d0fd735461fd2c"} Jul 06 03:22:30.729135 ip-10-0-139-206 kubenswrapper[2557]: I0706 03:22:30.729004 2557 scope.go:117] "RemoveContainer" containerID="7aaddc22614f8d603030ab23ee7200b2a51283a76c47a98edecac489468e9ebe" Jul 06 03:22:30.737567 ip-10-0-139-206 kubenswrapper[2557]: I0706 03:22:30.737544 2557 scope.go:117] "RemoveContainer" containerID="ae8dac24113dda4d508f213c5230c23bf936ac2f89293a8beea4735c291fa2f9" Jul 06 03:22:30.745860 ip-10-0-139-206 kubenswrapper[2557]: I0706 03:22:30.745842 2557 scope.go:117] "RemoveContainer" containerID="7aaddc22614f8d603030ab23ee7200b2a51283a76c47a98edecac489468e9ebe" Jul 06 03:22:30.746091 ip-10-0-139-206 kubenswrapper[2557]: E0706 03:22:30.746073 2557 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7aaddc22614f8d603030ab23ee7200b2a51283a76c47a98edecac489468e9ebe\": container with ID starting with 7aaddc22614f8d603030ab23ee7200b2a51283a76c47a98edecac489468e9ebe not found: ID does not exist" containerID="7aaddc22614f8d603030ab23ee7200b2a51283a76c47a98edecac489468e9ebe" Jul 06 03:22:30.746133 ip-10-0-139-206 kubenswrapper[2557]: I0706 03:22:30.746101 2557 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7aaddc22614f8d603030ab23ee7200b2a51283a76c47a98edecac489468e9ebe"} err="failed to get container status \"7aaddc22614f8d603030ab23ee7200b2a51283a76c47a98edecac489468e9ebe\": rpc error: code = NotFound desc = could not find container \"7aaddc22614f8d603030ab23ee7200b2a51283a76c47a98edecac489468e9ebe\": container with ID starting with 7aaddc22614f8d603030ab23ee7200b2a51283a76c47a98edecac489468e9ebe not found: ID does not exist" Jul 06 03:22:30.746133 ip-10-0-139-206 kubenswrapper[2557]: I0706 03:22:30.746124 2557 scope.go:117] "RemoveContainer" containerID="ae8dac24113dda4d508f213c5230c23bf936ac2f89293a8beea4735c291fa2f9" Jul 06 03:22:30.746299 ip-10-0-139-206 kubenswrapper[2557]: E0706 03:22:30.746284 2557 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ae8dac24113dda4d508f213c5230c23bf936ac2f89293a8beea4735c291fa2f9\": container with ID starting with ae8dac24113dda4d508f213c5230c23bf936ac2f89293a8beea4735c291fa2f9 not found: ID does not exist" containerID="ae8dac24113dda4d508f213c5230c23bf936ac2f89293a8beea4735c291fa2f9" Jul 06 03:22:30.746365 ip-10-0-139-206 kubenswrapper[2557]: I0706 03:22:30.746303 2557 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ae8dac24113dda4d508f213c5230c23bf936ac2f89293a8beea4735c291fa2f9"} err="failed to get container status \"ae8dac24113dda4d508f213c5230c23bf936ac2f89293a8beea4735c291fa2f9\": rpc error: code = NotFound desc = could not find container \"ae8dac24113dda4d508f213c5230c23bf936ac2f89293a8beea4735c291fa2f9\": container with ID starting with ae8dac24113dda4d508f213c5230c23bf936ac2f89293a8beea4735c291fa2f9 not found: ID does not exist" Jul 06 03:22:30.787742 ip-10-0-139-206 kubenswrapper[2557]: I0706 03:22:30.787705 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/2ed27de3-d9c7-4657-b7e4-37b71941db77-credential-socket\") pod \"2ed27de3-d9c7-4657-b7e4-37b71941db77\" (UID: \"2ed27de3-d9c7-4657-b7e4-37b71941db77\") " Jul 06 03:22:30.787742 ip-10-0-139-206 kubenswrapper[2557]: I0706 03:22:30.787742 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/2ed27de3-d9c7-4657-b7e4-37b71941db77-istio-data\") pod \"2ed27de3-d9c7-4657-b7e4-37b71941db77\" (UID: \"2ed27de3-d9c7-4657-b7e4-37b71941db77\") " Jul 06 03:22:30.787993 ip-10-0-139-206 kubenswrapper[2557]: I0706 03:22:30.787775 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-lgb7r\" (UniqueName: \"kubernetes.io/projected/2ed27de3-d9c7-4657-b7e4-37b71941db77-kube-api-access-lgb7r\") pod \"2ed27de3-d9c7-4657-b7e4-37b71941db77\" (UID: \"2ed27de3-d9c7-4657-b7e4-37b71941db77\") " Jul 06 03:22:30.787993 ip-10-0-139-206 kubenswrapper[2557]: I0706 03:22:30.787802 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/2ed27de3-d9c7-4657-b7e4-37b71941db77-istio-podinfo\") pod \"2ed27de3-d9c7-4657-b7e4-37b71941db77\" (UID: \"2ed27de3-d9c7-4657-b7e4-37b71941db77\") " Jul 06 03:22:30.787993 ip-10-0-139-206 kubenswrapper[2557]: I0706 03:22:30.787832 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/2ed27de3-d9c7-4657-b7e4-37b71941db77-istiod-ca-cert\") pod \"2ed27de3-d9c7-4657-b7e4-37b71941db77\" (UID: \"2ed27de3-d9c7-4657-b7e4-37b71941db77\") " Jul 06 03:22:30.787993 ip-10-0-139-206 kubenswrapper[2557]: I0706 03:22:30.787865 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/2ed27de3-d9c7-4657-b7e4-37b71941db77-istio-token\") pod \"2ed27de3-d9c7-4657-b7e4-37b71941db77\" (UID: \"2ed27de3-d9c7-4657-b7e4-37b71941db77\") " Jul 06 03:22:30.787993 ip-10-0-139-206 kubenswrapper[2557]: I0706 03:22:30.787891 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/2ed27de3-d9c7-4657-b7e4-37b71941db77-workload-socket\") pod \"2ed27de3-d9c7-4657-b7e4-37b71941db77\" (UID: \"2ed27de3-d9c7-4657-b7e4-37b71941db77\") " Jul 06 03:22:30.787993 ip-10-0-139-206 kubenswrapper[2557]: I0706 03:22:30.787919 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/2ed27de3-d9c7-4657-b7e4-37b71941db77-workload-certs\") pod \"2ed27de3-d9c7-4657-b7e4-37b71941db77\" (UID: \"2ed27de3-d9c7-4657-b7e4-37b71941db77\") " Jul 06 03:22:30.787993 ip-10-0-139-206 kubenswrapper[2557]: I0706 03:22:30.787991 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/2ed27de3-d9c7-4657-b7e4-37b71941db77-istio-envoy\") pod \"2ed27de3-d9c7-4657-b7e4-37b71941db77\" (UID: \"2ed27de3-d9c7-4657-b7e4-37b71941db77\") " Jul 06 03:22:30.788376 ip-10-0-139-206 kubenswrapper[2557]: I0706 03:22:30.788038 2557 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2ed27de3-d9c7-4657-b7e4-37b71941db77-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "2ed27de3-d9c7-4657-b7e4-37b71941db77" (UID: "2ed27de3-d9c7-4657-b7e4-37b71941db77"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 03:22:30.788376 ip-10-0-139-206 kubenswrapper[2557]: I0706 03:22:30.788208 2557 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2ed27de3-d9c7-4657-b7e4-37b71941db77-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "2ed27de3-d9c7-4657-b7e4-37b71941db77" (UID: "2ed27de3-d9c7-4657-b7e4-37b71941db77"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 03:22:30.788376 ip-10-0-139-206 kubenswrapper[2557]: I0706 03:22:30.788243 2557 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/2ed27de3-d9c7-4657-b7e4-37b71941db77-credential-socket\") on node \"ip-10-0-139-206.ec2.internal\" DevicePath \"\"" Jul 06 03:22:30.788376 ip-10-0-139-206 kubenswrapper[2557]: I0706 03:22:30.788274 2557 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2ed27de3-d9c7-4657-b7e4-37b71941db77-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "2ed27de3-d9c7-4657-b7e4-37b71941db77" (UID: "2ed27de3-d9c7-4657-b7e4-37b71941db77"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 03:22:30.788587 ip-10-0-139-206 kubenswrapper[2557]: I0706 03:22:30.788539 2557 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2ed27de3-d9c7-4657-b7e4-37b71941db77-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "2ed27de3-d9c7-4657-b7e4-37b71941db77" (UID: "2ed27de3-d9c7-4657-b7e4-37b71941db77"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 06 03:22:30.788703 ip-10-0-139-206 kubenswrapper[2557]: I0706 03:22:30.788681 2557 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2ed27de3-d9c7-4657-b7e4-37b71941db77-istio-data" (OuterVolumeSpecName: "istio-data") pod "2ed27de3-d9c7-4657-b7e4-37b71941db77" (UID: "2ed27de3-d9c7-4657-b7e4-37b71941db77"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 03:22:30.790113 ip-10-0-139-206 kubenswrapper[2557]: I0706 03:22:30.790077 2557 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/2ed27de3-d9c7-4657-b7e4-37b71941db77-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "2ed27de3-d9c7-4657-b7e4-37b71941db77" (UID: "2ed27de3-d9c7-4657-b7e4-37b71941db77"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 06 03:22:30.790235 ip-10-0-139-206 kubenswrapper[2557]: I0706 03:22:30.790170 2557 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2ed27de3-d9c7-4657-b7e4-37b71941db77-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "2ed27de3-d9c7-4657-b7e4-37b71941db77" (UID: "2ed27de3-d9c7-4657-b7e4-37b71941db77"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 03:22:30.790417 ip-10-0-139-206 kubenswrapper[2557]: I0706 03:22:30.790395 2557 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2ed27de3-d9c7-4657-b7e4-37b71941db77-kube-api-access-lgb7r" (OuterVolumeSpecName: "kube-api-access-lgb7r") pod "2ed27de3-d9c7-4657-b7e4-37b71941db77" (UID: "2ed27de3-d9c7-4657-b7e4-37b71941db77"). InnerVolumeSpecName "kube-api-access-lgb7r". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 06 03:22:30.790502 ip-10-0-139-206 kubenswrapper[2557]: I0706 03:22:30.790482 2557 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2ed27de3-d9c7-4657-b7e4-37b71941db77-istio-token" (OuterVolumeSpecName: "istio-token") pod "2ed27de3-d9c7-4657-b7e4-37b71941db77" (UID: "2ed27de3-d9c7-4657-b7e4-37b71941db77"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 06 03:22:30.889578 ip-10-0-139-206 kubenswrapper[2557]: I0706 03:22:30.889540 2557 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/2ed27de3-d9c7-4657-b7e4-37b71941db77-istio-envoy\") on node \"ip-10-0-139-206.ec2.internal\" DevicePath \"\"" Jul 06 03:22:30.889578 ip-10-0-139-206 kubenswrapper[2557]: I0706 03:22:30.889572 2557 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/2ed27de3-d9c7-4657-b7e4-37b71941db77-istio-data\") on node \"ip-10-0-139-206.ec2.internal\" DevicePath \"\"" Jul 06 03:22:30.889578 ip-10-0-139-206 kubenswrapper[2557]: I0706 03:22:30.889581 2557 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-lgb7r\" (UniqueName: \"kubernetes.io/projected/2ed27de3-d9c7-4657-b7e4-37b71941db77-kube-api-access-lgb7r\") on node \"ip-10-0-139-206.ec2.internal\" DevicePath \"\"" Jul 06 03:22:30.889578 ip-10-0-139-206 kubenswrapper[2557]: I0706 03:22:30.889590 2557 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/2ed27de3-d9c7-4657-b7e4-37b71941db77-istio-podinfo\") on node \"ip-10-0-139-206.ec2.internal\" DevicePath \"\"" Jul 06 03:22:30.889878 ip-10-0-139-206 kubenswrapper[2557]: I0706 03:22:30.889599 2557 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/2ed27de3-d9c7-4657-b7e4-37b71941db77-istiod-ca-cert\") on node \"ip-10-0-139-206.ec2.internal\" DevicePath \"\"" Jul 06 03:22:30.889878 ip-10-0-139-206 kubenswrapper[2557]: I0706 03:22:30.889607 2557 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/2ed27de3-d9c7-4657-b7e4-37b71941db77-istio-token\") on node \"ip-10-0-139-206.ec2.internal\" DevicePath \"\"" Jul 06 03:22:30.889878 ip-10-0-139-206 kubenswrapper[2557]: I0706 03:22:30.889615 2557 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/2ed27de3-d9c7-4657-b7e4-37b71941db77-workload-socket\") on node \"ip-10-0-139-206.ec2.internal\" DevicePath \"\"" Jul 06 03:22:30.889878 ip-10-0-139-206 kubenswrapper[2557]: I0706 03:22:30.889623 2557 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/2ed27de3-d9c7-4657-b7e4-37b71941db77-workload-certs\") on node \"ip-10-0-139-206.ec2.internal\" DevicePath \"\"" Jul 06 03:22:31.050185 ip-10-0-139-206 kubenswrapper[2557]: I0706 03:22:31.050146 2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-a-8edd65-openshift-default-75f7b769c9-bwvv8"] Jul 06 03:22:31.054315 ip-10-0-139-206 kubenswrapper[2557]: I0706 03:22:31.054287 2557 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-shared-a-8edd65-openshift-default-75f7b769c9-bwvv8"] Jul 06 03:22:31.307385 ip-10-0-139-206 kubenswrapper[2557]: I0706 03:22:31.307262 2557 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="2ed27de3-d9c7-4657-b7e4-37b71941db77" path="/var/lib/kubelet/pods/2ed27de3-d9c7-4657-b7e4-37b71941db77/volumes" Jul 06 03:22:37.655329 ip-10-0-139-206 kubenswrapper[2557]: I0706 03:22:37.655292 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_llmisvc-controller-manager-777c6bd7d5-qj8g5_8e1bab8d-a366-4520-b989-6b3ba7942c30/manager/0.log" Jul 06 03:22:37.888536 ip-10-0-139-206 kubenswrapper[2557]: I0706 03:22:37.888509 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-api-key-cleanup-29721780-lgpzb_652718cb-fc9e-4323-a37e-3c49c4096123/cleanup/0.log" Jul 06 03:22:38.001225 ip-10-0-139-206 kubenswrapper[2557]: I0706 03:22:38.001199 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-api-key-cleanup-29721795-mqpf9_956ad158-3f0d-4690-8ec2-8f6ea1815ca1/cleanup/0.log" Jul 06 03:22:38.472104 ip-10-0-139-206 kubenswrapper[2557]: I0706 03:22:38.472026 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-6d49d64447-dpntf_5adfd973-3482-4802-b49e-8a5151e3cf88/manager/0.log" Jul 06 03:22:38.477686 ip-10-0-139-206 kubenswrapper[2557]: I0706 03:22:38.477665 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-6d49d64447-dpntf_5adfd973-3482-4802-b49e-8a5151e3cf88/copy-manifests/0.log" Jul 06 03:22:40.230604 ip-10-0-139-206 kubenswrapper[2557]: I0706 03:22:40.230569 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-operator-75cc8d5cb7-2r27p_0e7f6526-6634-4615-8f64-402d99502020/manager/0.log" Jul 06 03:22:40.718477 ip-10-0-139-206 kubenswrapper[2557]: I0706 03:22:40.718440 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-controller-manager-86878c769f-s4kxn_c42a9b7b-33ec-48d8-b744-764aad3ce2e9/manager/0.log" Jul 06 03:22:40.848099 ip-10-0-139-206 kubenswrapper[2557]: I0706 03:22:40.848068 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-limitador-54d6d988d5-srzrr_6ffdd30b-5859-42e4-bdf0-335d7179ce8c/limitador/0.log" Jul 06 03:22:41.449272 ip-10-0-139-206 kubenswrapper[2557]: I0706 03:22:41.449227 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-94bb8fbfd-6pv67_d6173166-e488-42f6-ab20-a96414f5e3e3/discovery/0.log" Jul 06 03:22:41.683077 ip-10-0-139-206 kubenswrapper[2557]: I0706 03:22:41.683040 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-59cc9f6b7-lxz9f_f699fa9e-d2a2-4e64-a4ee-7423fc3391d1/kube-auth-proxy/0.log" Jul 06 03:22:50.164382 ip-10-0-139-206 kubenswrapper[2557]: I0706 03:22:50.164336 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-xjlbh_8f9fee4a-de24-49dc-b667-3dbc852d7ae3/global-pull-secret-syncer/0.log" Jul 06 03:22:50.286358 ip-10-0-139-206 kubenswrapper[2557]: I0706 03:22:50.286314 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-twqqj_00981958-29f3-4942-8ba6-0be9f03348a4/konnectivity-agent/0.log" Jul 06 03:22:50.330321 ip-10-0-139-206 kubenswrapper[2557]: I0706 03:22:50.330280 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-139-206.ec2.internal_2e5d7d54b6363f2418e74312edd961df/haproxy/0.log" Jul 06 03:22:55.269475 ip-10-0-139-206 kubenswrapper[2557]: I0706 03:22:55.269440 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-operator-75cc8d5cb7-2r27p_0e7f6526-6634-4615-8f64-402d99502020/manager/0.log" Jul 06 03:22:55.504544 ip-10-0-139-206 kubenswrapper[2557]: I0706 03:22:55.504506 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-controller-manager-86878c769f-s4kxn_c42a9b7b-33ec-48d8-b744-764aad3ce2e9/manager/0.log" Jul 06 03:22:55.568057 ip-10-0-139-206 kubenswrapper[2557]: I0706 03:22:55.567970 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-limitador-54d6d988d5-srzrr_6ffdd30b-5859-42e4-bdf0-335d7179ce8c/limitador/0.log" Jul 06 03:22:57.536554 ip-10-0-139-206 kubenswrapper[2557]: I0706 03:22:57.536456 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-7c9dbb8b7d-nt64l_23df5184-9bbf-49ba-82fa-e9f025937b4e/kube-state-metrics/0.log" Jul 06 03:22:57.557229 ip-10-0-139-206 kubenswrapper[2557]: I0706 03:22:57.557180 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-7c9dbb8b7d-nt64l_23df5184-9bbf-49ba-82fa-e9f025937b4e/kube-rbac-proxy-main/0.log" Jul 06 03:22:57.581399 ip-10-0-139-206 kubenswrapper[2557]: I0706 03:22:57.581368 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-7c9dbb8b7d-nt64l_23df5184-9bbf-49ba-82fa-e9f025937b4e/kube-rbac-proxy-self/0.log" Jul 06 03:22:57.672769 ip-10-0-139-206 kubenswrapper[2557]: I0706 03:22:57.672736 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-5ktdz_e45d2d95-4d84-4c3a-8809-e4e1b3809bf3/node-exporter/0.log" Jul 06 03:22:57.709333 ip-10-0-139-206 kubenswrapper[2557]: I0706 03:22:57.709309 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-5ktdz_e45d2d95-4d84-4c3a-8809-e4e1b3809bf3/kube-rbac-proxy/0.log" Jul 06 03:22:57.734077 ip-10-0-139-206 kubenswrapper[2557]: I0706 03:22:57.734054 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-5ktdz_e45d2d95-4d84-4c3a-8809-e4e1b3809bf3/init-textfile/0.log" Jul 06 03:22:58.219576 ip-10-0-139-206 kubenswrapper[2557]: I0706 03:22:58.219547 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-admission-webhook-d559d66d5-l22rc_f05e1cca-7bfa-449b-8ba2-efe0296386b8/prometheus-operator-admission-webhook/0.log" Jul 06 03:22:58.322845 ip-10-0-139-206 kubenswrapper[2557]: I0706 03:22:58.322819 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-78bfcd7f6-p2s9h_985e1aa6-6348-4f70-85bc-0b47fe3c68c0/thanos-query/0.log" Jul 06 03:22:58.344702 ip-10-0-139-206 kubenswrapper[2557]: I0706 03:22:58.344672 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-78bfcd7f6-p2s9h_985e1aa6-6348-4f70-85bc-0b47fe3c68c0/kube-rbac-proxy-web/0.log" Jul 06 03:22:58.365252 ip-10-0-139-206 kubenswrapper[2557]: I0706 03:22:58.365224 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-78bfcd7f6-p2s9h_985e1aa6-6348-4f70-85bc-0b47fe3c68c0/kube-rbac-proxy/0.log" Jul 06 03:22:58.386259 ip-10-0-139-206 kubenswrapper[2557]: I0706 03:22:58.386226 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-78bfcd7f6-p2s9h_985e1aa6-6348-4f70-85bc-0b47fe3c68c0/prom-label-proxy/0.log" Jul 06 03:22:58.407062 ip-10-0-139-206 kubenswrapper[2557]: I0706 03:22:58.407032 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-78bfcd7f6-p2s9h_985e1aa6-6348-4f70-85bc-0b47fe3c68c0/kube-rbac-proxy-rules/0.log" Jul 06 03:22:58.428373 ip-10-0-139-206 kubenswrapper[2557]: I0706 03:22:58.428317 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-78bfcd7f6-p2s9h_985e1aa6-6348-4f70-85bc-0b47fe3c68c0/kube-rbac-proxy-metrics/0.log" Jul 06 03:23:00.460503 ip-10-0-139-206 kubenswrapper[2557]: I0706 03:23:00.460471 2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-ftgk5/perf-node-gather-daemonset-7mm5f"] Jul 06 03:23:00.460910 ip-10-0-139-206 kubenswrapper[2557]: I0706 03:23:00.460783 2557 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2ed27de3-d9c7-4657-b7e4-37b71941db77" containerName="istio-proxy" Jul 06 03:23:00.460910 ip-10-0-139-206 kubenswrapper[2557]: I0706 03:23:00.460794 2557 state_mem.go:107] "Deleted CPUSet assignment" podUID="2ed27de3-d9c7-4657-b7e4-37b71941db77" containerName="istio-proxy" Jul 06 03:23:00.460910 ip-10-0-139-206 kubenswrapper[2557]: I0706 03:23:00.460815 2557 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f1f937ba-6535-4e96-a104-f73e6762e206" containerName="main" Jul 06 03:23:00.460910 ip-10-0-139-206 kubenswrapper[2557]: I0706 03:23:00.460821 2557 state_mem.go:107] "Deleted CPUSet assignment" podUID="f1f937ba-6535-4e96-a104-f73e6762e206" containerName="main" Jul 06 03:23:00.460910 ip-10-0-139-206 kubenswrapper[2557]: I0706 03:23:00.460872 2557 memory_manager.go:368] "RemoveStaleState removing state" podUID="2ed27de3-d9c7-4657-b7e4-37b71941db77" containerName="istio-proxy" Jul 06 03:23:00.460910 ip-10-0-139-206 kubenswrapper[2557]: I0706 03:23:00.460880 2557 memory_manager.go:368] "RemoveStaleState removing state" podUID="2ed27de3-d9c7-4657-b7e4-37b71941db77" containerName="istio-proxy" Jul 06 03:23:00.460910 ip-10-0-139-206 kubenswrapper[2557]: I0706 03:23:00.460886 2557 memory_manager.go:368] "RemoveStaleState removing state" podUID="f1f937ba-6535-4e96-a104-f73e6762e206" containerName="main" Jul 06 03:23:00.464027 ip-10-0-139-206 kubenswrapper[2557]: I0706 03:23:00.464004 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-ftgk5/perf-node-gather-daemonset-7mm5f" Jul 06 03:23:00.466499 ip-10-0-139-206 kubenswrapper[2557]: I0706 03:23:00.466470 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-ftgk5\"/\"default-dockercfg-lnmn7\"" Jul 06 03:23:00.466717 ip-10-0-139-206 kubenswrapper[2557]: I0706 03:23:00.466701 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-ftgk5\"/\"openshift-service-ca.crt\"" Jul 06 03:23:00.467428 ip-10-0-139-206 kubenswrapper[2557]: I0706 03:23:00.467397 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-ftgk5\"/\"kube-root-ca.crt\"" Jul 06 03:23:00.473294 ip-10-0-139-206 kubenswrapper[2557]: I0706 03:23:00.473272 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-ftgk5/perf-node-gather-daemonset-7mm5f"] Jul 06 03:23:00.534407 ip-10-0-139-206 kubenswrapper[2557]: I0706 03:23:00.534334 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/4df31e27-cba6-437b-a9a4-53f3315ac20e-proc\") pod \"perf-node-gather-daemonset-7mm5f\" (UID: \"4df31e27-cba6-437b-a9a4-53f3315ac20e\") " pod="openshift-must-gather-ftgk5/perf-node-gather-daemonset-7mm5f" Jul 06 03:23:00.534585 ip-10-0-139-206 kubenswrapper[2557]: I0706 03:23:00.534431 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/4df31e27-cba6-437b-a9a4-53f3315ac20e-sys\") pod \"perf-node-gather-daemonset-7mm5f\" (UID: \"4df31e27-cba6-437b-a9a4-53f3315ac20e\") " pod="openshift-must-gather-ftgk5/perf-node-gather-daemonset-7mm5f" Jul 06 03:23:00.534585 ip-10-0-139-206 kubenswrapper[2557]: I0706 03:23:00.534447 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/4df31e27-cba6-437b-a9a4-53f3315ac20e-lib-modules\") pod \"perf-node-gather-daemonset-7mm5f\" (UID: \"4df31e27-cba6-437b-a9a4-53f3315ac20e\") " pod="openshift-must-gather-ftgk5/perf-node-gather-daemonset-7mm5f" Jul 06 03:23:00.534585 ip-10-0-139-206 kubenswrapper[2557]: I0706 03:23:00.534537 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-d8vph\" (UniqueName: \"kubernetes.io/projected/4df31e27-cba6-437b-a9a4-53f3315ac20e-kube-api-access-d8vph\") pod \"perf-node-gather-daemonset-7mm5f\" (UID: \"4df31e27-cba6-437b-a9a4-53f3315ac20e\") " pod="openshift-must-gather-ftgk5/perf-node-gather-daemonset-7mm5f" Jul 06 03:23:00.534585 ip-10-0-139-206 kubenswrapper[2557]: I0706 03:23:00.534579 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/4df31e27-cba6-437b-a9a4-53f3315ac20e-podres\") pod \"perf-node-gather-daemonset-7mm5f\" (UID: \"4df31e27-cba6-437b-a9a4-53f3315ac20e\") " pod="openshift-must-gather-ftgk5/perf-node-gather-daemonset-7mm5f" Jul 06 03:23:00.635301 ip-10-0-139-206 kubenswrapper[2557]: I0706 03:23:00.635246 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/4df31e27-cba6-437b-a9a4-53f3315ac20e-sys\") pod \"perf-node-gather-daemonset-7mm5f\" (UID: \"4df31e27-cba6-437b-a9a4-53f3315ac20e\") " pod="openshift-must-gather-ftgk5/perf-node-gather-daemonset-7mm5f" Jul 06 03:23:00.635301 ip-10-0-139-206 kubenswrapper[2557]: I0706 03:23:00.635299 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/4df31e27-cba6-437b-a9a4-53f3315ac20e-lib-modules\") pod \"perf-node-gather-daemonset-7mm5f\" (UID: \"4df31e27-cba6-437b-a9a4-53f3315ac20e\") " pod="openshift-must-gather-ftgk5/perf-node-gather-daemonset-7mm5f" Jul 06 03:23:00.635570 ip-10-0-139-206 kubenswrapper[2557]: I0706 03:23:00.635379 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-d8vph\" (UniqueName: \"kubernetes.io/projected/4df31e27-cba6-437b-a9a4-53f3315ac20e-kube-api-access-d8vph\") pod \"perf-node-gather-daemonset-7mm5f\" (UID: \"4df31e27-cba6-437b-a9a4-53f3315ac20e\") " pod="openshift-must-gather-ftgk5/perf-node-gather-daemonset-7mm5f" Jul 06 03:23:00.635570 ip-10-0-139-206 kubenswrapper[2557]: I0706 03:23:00.635402 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/4df31e27-cba6-437b-a9a4-53f3315ac20e-sys\") pod \"perf-node-gather-daemonset-7mm5f\" (UID: \"4df31e27-cba6-437b-a9a4-53f3315ac20e\") " pod="openshift-must-gather-ftgk5/perf-node-gather-daemonset-7mm5f" Jul 06 03:23:00.635570 ip-10-0-139-206 kubenswrapper[2557]: I0706 03:23:00.635410 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/4df31e27-cba6-437b-a9a4-53f3315ac20e-podres\") pod \"perf-node-gather-daemonset-7mm5f\" (UID: \"4df31e27-cba6-437b-a9a4-53f3315ac20e\") " pod="openshift-must-gather-ftgk5/perf-node-gather-daemonset-7mm5f" Jul 06 03:23:00.635570 ip-10-0-139-206 kubenswrapper[2557]: I0706 03:23:00.635472 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/4df31e27-cba6-437b-a9a4-53f3315ac20e-proc\") pod \"perf-node-gather-daemonset-7mm5f\" (UID: \"4df31e27-cba6-437b-a9a4-53f3315ac20e\") " pod="openshift-must-gather-ftgk5/perf-node-gather-daemonset-7mm5f" Jul 06 03:23:00.635570 ip-10-0-139-206 kubenswrapper[2557]: I0706 03:23:00.635483 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/4df31e27-cba6-437b-a9a4-53f3315ac20e-podres\") pod \"perf-node-gather-daemonset-7mm5f\" (UID: \"4df31e27-cba6-437b-a9a4-53f3315ac20e\") " pod="openshift-must-gather-ftgk5/perf-node-gather-daemonset-7mm5f" Jul 06 03:23:00.635570 ip-10-0-139-206 kubenswrapper[2557]: I0706 03:23:00.635488 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/4df31e27-cba6-437b-a9a4-53f3315ac20e-lib-modules\") pod \"perf-node-gather-daemonset-7mm5f\" (UID: \"4df31e27-cba6-437b-a9a4-53f3315ac20e\") " pod="openshift-must-gather-ftgk5/perf-node-gather-daemonset-7mm5f" Jul 06 03:23:00.635570 ip-10-0-139-206 kubenswrapper[2557]: I0706 03:23:00.635551 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/4df31e27-cba6-437b-a9a4-53f3315ac20e-proc\") pod \"perf-node-gather-daemonset-7mm5f\" (UID: \"4df31e27-cba6-437b-a9a4-53f3315ac20e\") " pod="openshift-must-gather-ftgk5/perf-node-gather-daemonset-7mm5f" Jul 06 03:23:00.643590 ip-10-0-139-206 kubenswrapper[2557]: I0706 03:23:00.643560 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-d8vph\" (UniqueName: \"kubernetes.io/projected/4df31e27-cba6-437b-a9a4-53f3315ac20e-kube-api-access-d8vph\") pod \"perf-node-gather-daemonset-7mm5f\" (UID: \"4df31e27-cba6-437b-a9a4-53f3315ac20e\") " pod="openshift-must-gather-ftgk5/perf-node-gather-daemonset-7mm5f" Jul 06 03:23:00.739190 ip-10-0-139-206 kubenswrapper[2557]: I0706 03:23:00.739110 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_console-bbdd4588c-jj7xp_24c60978-daaf-41e7-8bc6-165a62f7e728/console/0.log" Jul 06 03:23:00.774372 ip-10-0-139-206 kubenswrapper[2557]: I0706 03:23:00.774314 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-ftgk5/perf-node-gather-daemonset-7mm5f" Jul 06 03:23:00.904471 ip-10-0-139-206 kubenswrapper[2557]: I0706 03:23:00.904439 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-ftgk5/perf-node-gather-daemonset-7mm5f"] Jul 06 03:23:00.908140 ip-10-0-139-206 kubenswrapper[2557]: W0706 03:23:00.908107 2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-pod4df31e27_cba6_437b_a9a4_53f3315ac20e.slice/crio-34a663c2cf3123ce4d67eb55b44419f6b20a9d8bcb68a53bbb3fc69cdafbcdfc WatchSource:0}: Error finding container 34a663c2cf3123ce4d67eb55b44419f6b20a9d8bcb68a53bbb3fc69cdafbcdfc: Status 404 returned error can't find the container with id 34a663c2cf3123ce4d67eb55b44419f6b20a9d8bcb68a53bbb3fc69cdafbcdfc Jul 06 03:23:01.872618 ip-10-0-139-206 kubenswrapper[2557]: I0706 03:23:01.872581 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-ftgk5/perf-node-gather-daemonset-7mm5f" event={"ID":"4df31e27-cba6-437b-a9a4-53f3315ac20e","Type":"ContainerStarted","Data":"e238e4127ecbca9ea797836824fcfebbe919db14faf337681113c8ba6bd3088f"} Jul 06 03:23:01.872992 ip-10-0-139-206 kubenswrapper[2557]: I0706 03:23:01.872635 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-ftgk5/perf-node-gather-daemonset-7mm5f" event={"ID":"4df31e27-cba6-437b-a9a4-53f3315ac20e","Type":"ContainerStarted","Data":"34a663c2cf3123ce4d67eb55b44419f6b20a9d8bcb68a53bbb3fc69cdafbcdfc"} Jul 06 03:23:01.872992 ip-10-0-139-206 kubenswrapper[2557]: I0706 03:23:01.872657 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-ftgk5/perf-node-gather-daemonset-7mm5f" Jul 06 03:23:01.888156 ip-10-0-139-206 kubenswrapper[2557]: I0706 03:23:01.888113 2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-ftgk5/perf-node-gather-daemonset-7mm5f" podStartSLOduration=1.8880985689999998 podStartE2EDuration="1.888098569s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-06 03:23:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 03:23:01.886572364 +0000 UTC m=+2625.692858425" watchObservedRunningTime="2026-07-06 03:23:01.888098569 +0000 UTC m=+2625.694384605" Jul 06 03:23:02.440865 ip-10-0-139-206 kubenswrapper[2557]: I0706 03:23:02.440837 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-8jcmv_974aff39-ecbd-4788-a159-d35d2a73edb6/dns/0.log" Jul 06 03:23:02.462276 ip-10-0-139-206 kubenswrapper[2557]: I0706 03:23:02.462242 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-8jcmv_974aff39-ecbd-4788-a159-d35d2a73edb6/kube-rbac-proxy/0.log" Jul 06 03:23:02.528013 ip-10-0-139-206 kubenswrapper[2557]: I0706 03:23:02.527985 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-bhp46_dd0a7167-560c-4195-857e-b69abeede707/dns-node-resolver/0.log" Jul 06 03:23:03.163150 ip-10-0-139-206 kubenswrapper[2557]: I0706 03:23:03.163122 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-lwmh8_5b80db07-d76f-4470-a3d5-3176fdf63aa2/node-ca/0.log" Jul 06 03:23:04.215391 ip-10-0-139-206 kubenswrapper[2557]: I0706 03:23:04.215335 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-94bb8fbfd-6pv67_d6173166-e488-42f6-ab20-a96414f5e3e3/discovery/0.log" Jul 06 03:23:04.264034 ip-10-0-139-206 kubenswrapper[2557]: I0706 03:23:04.264004 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-59cc9f6b7-lxz9f_f699fa9e-d2a2-4e64-a4ee-7423fc3391d1/kube-auth-proxy/0.log" Jul 06 03:23:05.180123 ip-10-0-139-206 kubenswrapper[2557]: I0706 03:23:05.180087 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-qkmzt_faa07f8b-6058-4e7e-af53-4fc206f18730/serve-healthcheck-canary/0.log" Jul 06 03:23:05.703109 ip-10-0-139-206 kubenswrapper[2557]: I0706 03:23:05.703088 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-g65v5_77908dbd-b4f5-498a-8ccf-731614f02373/kube-rbac-proxy/0.log" Jul 06 03:23:05.724891 ip-10-0-139-206 kubenswrapper[2557]: I0706 03:23:05.724859 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-g65v5_77908dbd-b4f5-498a-8ccf-731614f02373/exporter/0.log" Jul 06 03:23:05.746922 ip-10-0-139-206 kubenswrapper[2557]: I0706 03:23:05.746897 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-g65v5_77908dbd-b4f5-498a-8ccf-731614f02373/extractor/0.log" Jul 06 03:23:07.888182 ip-10-0-139-206 kubenswrapper[2557]: I0706 03:23:07.888156 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-ftgk5/perf-node-gather-daemonset-7mm5f" Jul 06 03:23:08.126515 ip-10-0-139-206 kubenswrapper[2557]: I0706 03:23:08.126479 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_llmisvc-controller-manager-777c6bd7d5-qj8g5_8e1bab8d-a366-4520-b989-6b3ba7942c30/manager/0.log" Jul 06 03:23:08.202749 ip-10-0-139-206 kubenswrapper[2557]: I0706 03:23:08.202719 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-api-key-cleanup-29721780-lgpzb_652718cb-fc9e-4323-a37e-3c49c4096123/cleanup/0.log" Jul 06 03:23:08.228294 ip-10-0-139-206 kubenswrapper[2557]: I0706 03:23:08.228265 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-api-key-cleanup-29721795-mqpf9_956ad158-3f0d-4690-8ec2-8f6ea1815ca1/cleanup/0.log" Jul 06 03:23:08.527030 ip-10-0-139-206 kubenswrapper[2557]: I0706 03:23:08.526950 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-6d49d64447-dpntf_5adfd973-3482-4802-b49e-8a5151e3cf88/manager/0.log" Jul 06 03:23:08.548083 ip-10-0-139-206 kubenswrapper[2557]: I0706 03:23:08.548055 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-6d49d64447-dpntf_5adfd973-3482-4802-b49e-8a5151e3cf88/copy-manifests/0.log" Jul 06 03:23:10.316234 ip-10-0-139-206 kubenswrapper[2557]: I0706 03:23:10.316206 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-lws-operator_openshift-lws-operator-fd8ccff4c-2f9nz_8240eeaf-2841-4478-898a-0c76a803bb0b/openshift-lws-operator/0.log" Jul 06 03:23:10.606874 ip-10-0-139-206 kubenswrapper[2557]: I0706 03:23:10.606847 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-94bb8fbfd-6pv67_d6173166-e488-42f6-ab20-a96414f5e3e3/discovery/0.log" Jul 06 03:23:10.635933 ip-10-0-139-206 kubenswrapper[2557]: I0706 03:23:10.635905 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-59cc9f6b7-lxz9f_f699fa9e-d2a2-4e64-a4ee-7423fc3391d1/kube-auth-proxy/0.log" Jul 06 03:23:12.032947 ip-10-0-139-206 kubenswrapper[2557]: I0706 03:23:12.032859 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-94bb8fbfd-6pv67_d6173166-e488-42f6-ab20-a96414f5e3e3/discovery/0.log" Jul 06 03:23:12.065512 ip-10-0-139-206 kubenswrapper[2557]: I0706 03:23:12.065483 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-59cc9f6b7-lxz9f_f699fa9e-d2a2-4e64-a4ee-7423fc3391d1/kube-auth-proxy/0.log" Jul 06 03:23:16.538433 ip-10-0-139-206 kubenswrapper[2557]: I0706 03:23:16.538398 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-7nlb2_29184621-7845-4a4a-9b81-04b439571ca3/kube-multus-additional-cni-plugins/0.log" Jul 06 03:23:16.564752 ip-10-0-139-206 kubenswrapper[2557]: I0706 03:23:16.564728 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-7nlb2_29184621-7845-4a4a-9b81-04b439571ca3/egress-router-binary-copy/0.log" Jul 06 03:23:16.587831 ip-10-0-139-206 kubenswrapper[2557]: I0706 03:23:16.587800 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-7nlb2_29184621-7845-4a4a-9b81-04b439571ca3/cni-plugins/0.log" Jul 06 03:23:16.609056 ip-10-0-139-206 kubenswrapper[2557]: I0706 03:23:16.609030 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-7nlb2_29184621-7845-4a4a-9b81-04b439571ca3/bond-cni-plugin/0.log" Jul 06 03:23:16.631192 ip-10-0-139-206 kubenswrapper[2557]: I0706 03:23:16.631165 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-7nlb2_29184621-7845-4a4a-9b81-04b439571ca3/routeoverride-cni/0.log" Jul 06 03:23:16.652525 ip-10-0-139-206 kubenswrapper[2557]: I0706 03:23:16.652496 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-7nlb2_29184621-7845-4a4a-9b81-04b439571ca3/whereabouts-cni-bincopy/0.log" Jul 06 03:23:16.674764 ip-10-0-139-206 kubenswrapper[2557]: I0706 03:23:16.674734 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-7nlb2_29184621-7845-4a4a-9b81-04b439571ca3/whereabouts-cni/0.log" Jul 06 03:23:17.150261 ip-10-0-139-206 kubenswrapper[2557]: I0706 03:23:17.150230 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-v4pnk_af7822ef-5b14-477a-9fdd-5ef1fac67e23/kube-multus/0.log" Jul 06 03:23:17.215578 ip-10-0-139-206 kubenswrapper[2557]: I0706 03:23:17.215547 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-756n2_26f69a47-e949-4e18-9d9c-6648a959e281/network-metrics-daemon/0.log" Jul 06 03:23:17.235533 ip-10-0-139-206 kubenswrapper[2557]: I0706 03:23:17.235506 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-756n2_26f69a47-e949-4e18-9d9c-6648a959e281/kube-rbac-proxy/0.log"