Jul 07 11:15:55.566738 ip-10-0-142-112 systemd[1]: Starting Kubernetes Kubelet... Jul 07 11:15:56.038958 ip-10-0-142-112 kubenswrapper[2578]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 07 11:15:56.038958 ip-10-0-142-112 kubenswrapper[2578]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version. Jul 07 11:15:56.038958 ip-10-0-142-112 kubenswrapper[2578]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 07 11:15:56.038958 ip-10-0-142-112 kubenswrapper[2578]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI. Jul 07 11:15:56.038958 ip-10-0-142-112 kubenswrapper[2578]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 07 11:15:56.040052 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:15:56.039963 2578 server.go:213] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime" Jul 07 11:15:56.047678 ip-10-0-142-112 kubenswrapper[2578]: W0707 11:15:56.047652 2578 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 07 11:15:56.047678 ip-10-0-142-112 kubenswrapper[2578]: W0707 11:15:56.047672 2578 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 07 11:15:56.047678 ip-10-0-142-112 kubenswrapper[2578]: W0707 11:15:56.047677 2578 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 07 11:15:56.047678 ip-10-0-142-112 kubenswrapper[2578]: W0707 11:15:56.047681 2578 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 07 11:15:56.047678 ip-10-0-142-112 kubenswrapper[2578]: W0707 11:15:56.047685 2578 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 07 11:15:56.047678 ip-10-0-142-112 kubenswrapper[2578]: W0707 11:15:56.047689 2578 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 07 11:15:56.048007 ip-10-0-142-112 kubenswrapper[2578]: W0707 11:15:56.047693 2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 07 11:15:56.048007 ip-10-0-142-112 kubenswrapper[2578]: W0707 11:15:56.047697 2578 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 07 11:15:56.048007 ip-10-0-142-112 kubenswrapper[2578]: W0707 11:15:56.047701 2578 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 07 11:15:56.048007 ip-10-0-142-112 kubenswrapper[2578]: W0707 11:15:56.047704 2578 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 07 11:15:56.048007 ip-10-0-142-112 kubenswrapper[2578]: W0707 11:15:56.047708 2578 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 07 11:15:56.048007 ip-10-0-142-112 kubenswrapper[2578]: W0707 11:15:56.047712 2578 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 07 11:15:56.048007 ip-10-0-142-112 kubenswrapper[2578]: W0707 11:15:56.047715 2578 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 07 11:15:56.048007 ip-10-0-142-112 kubenswrapper[2578]: W0707 11:15:56.047719 2578 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 07 11:15:56.048007 ip-10-0-142-112 kubenswrapper[2578]: W0707 11:15:56.047722 2578 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 07 11:15:56.048007 ip-10-0-142-112 kubenswrapper[2578]: W0707 11:15:56.047726 2578 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 07 11:15:56.048007 ip-10-0-142-112 kubenswrapper[2578]: W0707 11:15:56.047730 2578 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 07 11:15:56.048007 ip-10-0-142-112 kubenswrapper[2578]: W0707 11:15:56.047734 2578 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 07 11:15:56.048007 ip-10-0-142-112 kubenswrapper[2578]: W0707 11:15:56.047738 2578 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 07 11:15:56.048007 ip-10-0-142-112 kubenswrapper[2578]: W0707 11:15:56.047741 2578 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 07 11:15:56.048007 ip-10-0-142-112 kubenswrapper[2578]: W0707 11:15:56.047745 2578 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 07 11:15:56.048007 ip-10-0-142-112 kubenswrapper[2578]: W0707 11:15:56.047749 2578 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 07 11:15:56.048007 ip-10-0-142-112 kubenswrapper[2578]: W0707 11:15:56.047753 2578 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 07 11:15:56.048007 ip-10-0-142-112 kubenswrapper[2578]: W0707 11:15:56.047758 2578 feature_gate.go:328] unrecognized feature gate: Example Jul 07 11:15:56.048007 ip-10-0-142-112 kubenswrapper[2578]: W0707 11:15:56.047762 2578 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 07 11:15:56.048007 ip-10-0-142-112 kubenswrapper[2578]: W0707 11:15:56.047766 2578 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 07 11:15:56.048820 ip-10-0-142-112 kubenswrapper[2578]: W0707 11:15:56.047771 2578 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 07 11:15:56.048820 ip-10-0-142-112 kubenswrapper[2578]: W0707 11:15:56.047778 2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 07 11:15:56.048820 ip-10-0-142-112 kubenswrapper[2578]: W0707 11:15:56.047782 2578 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 07 11:15:56.048820 ip-10-0-142-112 kubenswrapper[2578]: W0707 11:15:56.047786 2578 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 07 11:15:56.048820 ip-10-0-142-112 kubenswrapper[2578]: W0707 11:15:56.047791 2578 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 07 11:15:56.048820 ip-10-0-142-112 kubenswrapper[2578]: W0707 11:15:56.047795 2578 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 07 11:15:56.048820 ip-10-0-142-112 kubenswrapper[2578]: W0707 11:15:56.047815 2578 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 07 11:15:56.048820 ip-10-0-142-112 kubenswrapper[2578]: W0707 11:15:56.047821 2578 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 07 11:15:56.048820 ip-10-0-142-112 kubenswrapper[2578]: W0707 11:15:56.047825 2578 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 07 11:15:56.048820 ip-10-0-142-112 kubenswrapper[2578]: W0707 11:15:56.047829 2578 feature_gate.go:328] unrecognized feature gate: Example2 Jul 07 11:15:56.048820 ip-10-0-142-112 kubenswrapper[2578]: W0707 11:15:56.047833 2578 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 07 11:15:56.048820 ip-10-0-142-112 kubenswrapper[2578]: W0707 11:15:56.047838 2578 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 07 11:15:56.048820 ip-10-0-142-112 kubenswrapper[2578]: W0707 11:15:56.047842 2578 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 07 11:15:56.048820 ip-10-0-142-112 kubenswrapper[2578]: W0707 11:15:56.047847 2578 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 07 11:15:56.048820 ip-10-0-142-112 kubenswrapper[2578]: W0707 11:15:56.047852 2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 07 11:15:56.048820 ip-10-0-142-112 kubenswrapper[2578]: W0707 11:15:56.047856 2578 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 07 11:15:56.048820 ip-10-0-142-112 kubenswrapper[2578]: W0707 11:15:56.047860 2578 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 07 11:15:56.048820 ip-10-0-142-112 kubenswrapper[2578]: W0707 11:15:56.047864 2578 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 07 11:15:56.048820 ip-10-0-142-112 kubenswrapper[2578]: W0707 11:15:56.047869 2578 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 07 11:15:56.049633 ip-10-0-142-112 kubenswrapper[2578]: W0707 11:15:56.047873 2578 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 07 11:15:56.049633 ip-10-0-142-112 kubenswrapper[2578]: W0707 11:15:56.047877 2578 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 07 11:15:56.049633 ip-10-0-142-112 kubenswrapper[2578]: W0707 11:15:56.047881 2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 07 11:15:56.049633 ip-10-0-142-112 kubenswrapper[2578]: W0707 11:15:56.047886 2578 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 07 11:15:56.049633 ip-10-0-142-112 kubenswrapper[2578]: W0707 11:15:56.047890 2578 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 07 11:15:56.049633 ip-10-0-142-112 kubenswrapper[2578]: W0707 11:15:56.047894 2578 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 07 11:15:56.049633 ip-10-0-142-112 kubenswrapper[2578]: W0707 11:15:56.047899 2578 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 07 11:15:56.049633 ip-10-0-142-112 kubenswrapper[2578]: W0707 11:15:56.047905 2578 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 07 11:15:56.049633 ip-10-0-142-112 kubenswrapper[2578]: W0707 11:15:56.047910 2578 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 07 11:15:56.049633 ip-10-0-142-112 kubenswrapper[2578]: W0707 11:15:56.047914 2578 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 07 11:15:56.049633 ip-10-0-142-112 kubenswrapper[2578]: W0707 11:15:56.047918 2578 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 07 11:15:56.049633 ip-10-0-142-112 kubenswrapper[2578]: W0707 11:15:56.047922 2578 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 07 11:15:56.049633 ip-10-0-142-112 kubenswrapper[2578]: W0707 11:15:56.047926 2578 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 07 11:15:56.049633 ip-10-0-142-112 kubenswrapper[2578]: W0707 11:15:56.047930 2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 07 11:15:56.049633 ip-10-0-142-112 kubenswrapper[2578]: W0707 11:15:56.047935 2578 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 07 11:15:56.049633 ip-10-0-142-112 kubenswrapper[2578]: W0707 11:15:56.047939 2578 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 07 11:15:56.049633 ip-10-0-142-112 kubenswrapper[2578]: W0707 11:15:56.047943 2578 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 07 11:15:56.049633 ip-10-0-142-112 kubenswrapper[2578]: W0707 11:15:56.047947 2578 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 07 11:15:56.049633 ip-10-0-142-112 kubenswrapper[2578]: W0707 11:15:56.047952 2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 07 11:15:56.049633 ip-10-0-142-112 kubenswrapper[2578]: W0707 11:15:56.047956 2578 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 07 11:15:56.050438 ip-10-0-142-112 kubenswrapper[2578]: W0707 11:15:56.047960 2578 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 07 11:15:56.050438 ip-10-0-142-112 kubenswrapper[2578]: W0707 11:15:56.047967 2578 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 07 11:15:56.050438 ip-10-0-142-112 kubenswrapper[2578]: W0707 11:15:56.047975 2578 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 07 11:15:56.050438 ip-10-0-142-112 kubenswrapper[2578]: W0707 11:15:56.047980 2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 07 11:15:56.050438 ip-10-0-142-112 kubenswrapper[2578]: W0707 11:15:56.047984 2578 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 07 11:15:56.050438 ip-10-0-142-112 kubenswrapper[2578]: W0707 11:15:56.047989 2578 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 07 11:15:56.050438 ip-10-0-142-112 kubenswrapper[2578]: W0707 11:15:56.047994 2578 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 07 11:15:56.050438 ip-10-0-142-112 kubenswrapper[2578]: W0707 11:15:56.047998 2578 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 07 11:15:56.050438 ip-10-0-142-112 kubenswrapper[2578]: W0707 11:15:56.048002 2578 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 07 11:15:56.050438 ip-10-0-142-112 kubenswrapper[2578]: W0707 11:15:56.048006 2578 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 07 11:15:56.050438 ip-10-0-142-112 kubenswrapper[2578]: W0707 11:15:56.048010 2578 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 07 11:15:56.050438 ip-10-0-142-112 kubenswrapper[2578]: W0707 11:15:56.048014 2578 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 07 11:15:56.050438 ip-10-0-142-112 kubenswrapper[2578]: W0707 11:15:56.048018 2578 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 07 11:15:56.050438 ip-10-0-142-112 kubenswrapper[2578]: W0707 11:15:56.048022 2578 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 07 11:15:56.050438 ip-10-0-142-112 kubenswrapper[2578]: W0707 11:15:56.048026 2578 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 07 11:15:56.050438 ip-10-0-142-112 kubenswrapper[2578]: W0707 11:15:56.048030 2578 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 07 11:15:56.050438 ip-10-0-142-112 kubenswrapper[2578]: W0707 11:15:56.048034 2578 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 07 11:15:56.050438 ip-10-0-142-112 kubenswrapper[2578]: W0707 11:15:56.048038 2578 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 07 11:15:56.050438 ip-10-0-142-112 kubenswrapper[2578]: W0707 11:15:56.048045 2578 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 07 11:15:56.050438 ip-10-0-142-112 kubenswrapper[2578]: W0707 11:15:56.048050 2578 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 07 11:15:56.051273 ip-10-0-142-112 kubenswrapper[2578]: W0707 11:15:56.048054 2578 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 07 11:15:56.051273 ip-10-0-142-112 kubenswrapper[2578]: W0707 11:15:56.048058 2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 07 11:15:56.051273 ip-10-0-142-112 kubenswrapper[2578]: W0707 11:15:56.048062 2578 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 07 11:15:56.051273 ip-10-0-142-112 kubenswrapper[2578]: W0707 11:15:56.048066 2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 07 11:15:56.051273 ip-10-0-142-112 kubenswrapper[2578]: W0707 11:15:56.048070 2578 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 07 11:15:56.051273 ip-10-0-142-112 kubenswrapper[2578]: W0707 11:15:56.048074 2578 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 07 11:15:56.051273 ip-10-0-142-112 kubenswrapper[2578]: W0707 11:15:56.048078 2578 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 07 11:15:56.051273 ip-10-0-142-112 kubenswrapper[2578]: W0707 11:15:56.048083 2578 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 07 11:15:56.051273 ip-10-0-142-112 kubenswrapper[2578]: W0707 11:15:56.048087 2578 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 07 11:15:56.051273 ip-10-0-142-112 kubenswrapper[2578]: W0707 11:15:56.048094 2578 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 07 11:15:56.051273 ip-10-0-142-112 kubenswrapper[2578]: W0707 11:15:56.048100 2578 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 07 11:15:56.051273 ip-10-0-142-112 kubenswrapper[2578]: W0707 11:15:56.048105 2578 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 07 11:15:56.051273 ip-10-0-142-112 kubenswrapper[2578]: W0707 11:15:56.048110 2578 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 07 11:15:56.051273 ip-10-0-142-112 kubenswrapper[2578]: W0707 11:15:56.048114 2578 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 07 11:15:56.051273 ip-10-0-142-112 kubenswrapper[2578]: W0707 11:15:56.049320 2578 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 07 11:15:56.051273 ip-10-0-142-112 kubenswrapper[2578]: W0707 11:15:56.049331 2578 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 07 11:15:56.051273 ip-10-0-142-112 kubenswrapper[2578]: W0707 11:15:56.049336 2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 07 11:15:56.051273 ip-10-0-142-112 kubenswrapper[2578]: W0707 11:15:56.049341 2578 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 07 11:15:56.051273 ip-10-0-142-112 kubenswrapper[2578]: W0707 11:15:56.049347 2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 07 11:15:56.052083 ip-10-0-142-112 kubenswrapper[2578]: W0707 11:15:56.049352 2578 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 07 11:15:56.052083 ip-10-0-142-112 kubenswrapper[2578]: W0707 11:15:56.049359 2578 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 07 11:15:56.052083 ip-10-0-142-112 kubenswrapper[2578]: W0707 11:15:56.049366 2578 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 07 11:15:56.052083 ip-10-0-142-112 kubenswrapper[2578]: W0707 11:15:56.049371 2578 feature_gate.go:328] unrecognized feature gate: Example Jul 07 11:15:56.052083 ip-10-0-142-112 kubenswrapper[2578]: W0707 11:15:56.049375 2578 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 07 11:15:56.052083 ip-10-0-142-112 kubenswrapper[2578]: W0707 11:15:56.049379 2578 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 07 11:15:56.052083 ip-10-0-142-112 kubenswrapper[2578]: W0707 11:15:56.049384 2578 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 07 11:15:56.052083 ip-10-0-142-112 kubenswrapper[2578]: W0707 11:15:56.049389 2578 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 07 11:15:56.052083 ip-10-0-142-112 kubenswrapper[2578]: W0707 11:15:56.049393 2578 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 07 11:15:56.052083 ip-10-0-142-112 kubenswrapper[2578]: W0707 11:15:56.049397 2578 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 07 11:15:56.052083 ip-10-0-142-112 kubenswrapper[2578]: W0707 11:15:56.049401 2578 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 07 11:15:56.052083 ip-10-0-142-112 kubenswrapper[2578]: W0707 11:15:56.049406 2578 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 07 11:15:56.052083 ip-10-0-142-112 kubenswrapper[2578]: W0707 11:15:56.049412 2578 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 07 11:15:56.052083 ip-10-0-142-112 kubenswrapper[2578]: W0707 11:15:56.049416 2578 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 07 11:15:56.052083 ip-10-0-142-112 kubenswrapper[2578]: W0707 11:15:56.049442 2578 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 07 11:15:56.052083 ip-10-0-142-112 kubenswrapper[2578]: W0707 11:15:56.049447 2578 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 07 11:15:56.052083 ip-10-0-142-112 kubenswrapper[2578]: W0707 11:15:56.049451 2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 07 11:15:56.052083 ip-10-0-142-112 kubenswrapper[2578]: W0707 11:15:56.049458 2578 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 07 11:15:56.052083 ip-10-0-142-112 kubenswrapper[2578]: W0707 11:15:56.049463 2578 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 07 11:15:56.052822 ip-10-0-142-112 kubenswrapper[2578]: W0707 11:15:56.049467 2578 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 07 11:15:56.052822 ip-10-0-142-112 kubenswrapper[2578]: W0707 11:15:56.049472 2578 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 07 11:15:56.052822 ip-10-0-142-112 kubenswrapper[2578]: W0707 11:15:56.049477 2578 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 07 11:15:56.052822 ip-10-0-142-112 kubenswrapper[2578]: W0707 11:15:56.049482 2578 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 07 11:15:56.052822 ip-10-0-142-112 kubenswrapper[2578]: W0707 11:15:56.049486 2578 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 07 11:15:56.052822 ip-10-0-142-112 kubenswrapper[2578]: W0707 11:15:56.049491 2578 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 07 11:15:56.052822 ip-10-0-142-112 kubenswrapper[2578]: W0707 11:15:56.049495 2578 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 07 11:15:56.052822 ip-10-0-142-112 kubenswrapper[2578]: W0707 11:15:56.049499 2578 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 07 11:15:56.052822 ip-10-0-142-112 kubenswrapper[2578]: W0707 11:15:56.049504 2578 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 07 11:15:56.052822 ip-10-0-142-112 kubenswrapper[2578]: W0707 11:15:56.049508 2578 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 07 11:15:56.052822 ip-10-0-142-112 kubenswrapper[2578]: W0707 11:15:56.049513 2578 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 07 11:15:56.052822 ip-10-0-142-112 kubenswrapper[2578]: W0707 11:15:56.049516 2578 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 07 11:15:56.052822 ip-10-0-142-112 kubenswrapper[2578]: W0707 11:15:56.049522 2578 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 07 11:15:56.052822 ip-10-0-142-112 kubenswrapper[2578]: W0707 11:15:56.049527 2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 07 11:15:56.052822 ip-10-0-142-112 kubenswrapper[2578]: W0707 11:15:56.049532 2578 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 07 11:15:56.052822 ip-10-0-142-112 kubenswrapper[2578]: W0707 11:15:56.049536 2578 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 07 11:15:56.052822 ip-10-0-142-112 kubenswrapper[2578]: W0707 11:15:56.049540 2578 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 07 11:15:56.052822 ip-10-0-142-112 kubenswrapper[2578]: W0707 11:15:56.049545 2578 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 07 11:15:56.052822 ip-10-0-142-112 kubenswrapper[2578]: W0707 11:15:56.049548 2578 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 07 11:15:56.053306 ip-10-0-142-112 kubenswrapper[2578]: W0707 11:15:56.049553 2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 07 11:15:56.053306 ip-10-0-142-112 kubenswrapper[2578]: W0707 11:15:56.049557 2578 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 07 11:15:56.053306 ip-10-0-142-112 kubenswrapper[2578]: W0707 11:15:56.049561 2578 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 07 11:15:56.053306 ip-10-0-142-112 kubenswrapper[2578]: W0707 11:15:56.049565 2578 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 07 11:15:56.053306 ip-10-0-142-112 kubenswrapper[2578]: W0707 11:15:56.049569 2578 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 07 11:15:56.053306 ip-10-0-142-112 kubenswrapper[2578]: W0707 11:15:56.049574 2578 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 07 11:15:56.053306 ip-10-0-142-112 kubenswrapper[2578]: W0707 11:15:56.049578 2578 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 07 11:15:56.053306 ip-10-0-142-112 kubenswrapper[2578]: W0707 11:15:56.049583 2578 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 07 11:15:56.053306 ip-10-0-142-112 kubenswrapper[2578]: W0707 11:15:56.049587 2578 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 07 11:15:56.053306 ip-10-0-142-112 kubenswrapper[2578]: W0707 11:15:56.049591 2578 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 07 11:15:56.053306 ip-10-0-142-112 kubenswrapper[2578]: W0707 11:15:56.049595 2578 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 07 11:15:56.053306 ip-10-0-142-112 kubenswrapper[2578]: W0707 11:15:56.049599 2578 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 07 11:15:56.053306 ip-10-0-142-112 kubenswrapper[2578]: W0707 11:15:56.049604 2578 feature_gate.go:328] unrecognized feature gate: Example2 Jul 07 11:15:56.053306 ip-10-0-142-112 kubenswrapper[2578]: W0707 11:15:56.049608 2578 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 07 11:15:56.053306 ip-10-0-142-112 kubenswrapper[2578]: W0707 11:15:56.049612 2578 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 07 11:15:56.053306 ip-10-0-142-112 kubenswrapper[2578]: W0707 11:15:56.049616 2578 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 07 11:15:56.053306 ip-10-0-142-112 kubenswrapper[2578]: W0707 11:15:56.049621 2578 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 07 11:15:56.053306 ip-10-0-142-112 kubenswrapper[2578]: W0707 11:15:56.049625 2578 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 07 11:15:56.053306 ip-10-0-142-112 kubenswrapper[2578]: W0707 11:15:56.049629 2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 07 11:15:56.053306 ip-10-0-142-112 kubenswrapper[2578]: W0707 11:15:56.049633 2578 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 07 11:15:56.053995 ip-10-0-142-112 kubenswrapper[2578]: W0707 11:15:56.049639 2578 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 07 11:15:56.053995 ip-10-0-142-112 kubenswrapper[2578]: W0707 11:15:56.049644 2578 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 07 11:15:56.053995 ip-10-0-142-112 kubenswrapper[2578]: W0707 11:15:56.049648 2578 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 07 11:15:56.053995 ip-10-0-142-112 kubenswrapper[2578]: W0707 11:15:56.049653 2578 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 07 11:15:56.053995 ip-10-0-142-112 kubenswrapper[2578]: W0707 11:15:56.049657 2578 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 07 11:15:56.053995 ip-10-0-142-112 kubenswrapper[2578]: W0707 11:15:56.049662 2578 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 07 11:15:56.053995 ip-10-0-142-112 kubenswrapper[2578]: W0707 11:15:56.049668 2578 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 07 11:15:56.053995 ip-10-0-142-112 kubenswrapper[2578]: W0707 11:15:56.049674 2578 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 07 11:15:56.053995 ip-10-0-142-112 kubenswrapper[2578]: W0707 11:15:56.049678 2578 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 07 11:15:56.053995 ip-10-0-142-112 kubenswrapper[2578]: W0707 11:15:56.049683 2578 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 07 11:15:56.053995 ip-10-0-142-112 kubenswrapper[2578]: W0707 11:15:56.049687 2578 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 07 11:15:56.053995 ip-10-0-142-112 kubenswrapper[2578]: W0707 11:15:56.049691 2578 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 07 11:15:56.053995 ip-10-0-142-112 kubenswrapper[2578]: W0707 11:15:56.049695 2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 07 11:15:56.053995 ip-10-0-142-112 kubenswrapper[2578]: W0707 11:15:56.049699 2578 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 07 11:15:56.053995 ip-10-0-142-112 kubenswrapper[2578]: W0707 11:15:56.049704 2578 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 07 11:15:56.053995 ip-10-0-142-112 kubenswrapper[2578]: W0707 11:15:56.049708 2578 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 07 11:15:56.053995 ip-10-0-142-112 kubenswrapper[2578]: W0707 11:15:56.049713 2578 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 07 11:15:56.053995 ip-10-0-142-112 kubenswrapper[2578]: W0707 11:15:56.049717 2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 07 11:15:56.053995 ip-10-0-142-112 kubenswrapper[2578]: W0707 11:15:56.049721 2578 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 07 11:15:56.054486 ip-10-0-142-112 kubenswrapper[2578]: W0707 11:15:56.049725 2578 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 07 11:15:56.054486 ip-10-0-142-112 kubenswrapper[2578]: W0707 11:15:56.049731 2578 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 07 11:15:56.054486 ip-10-0-142-112 kubenswrapper[2578]: W0707 11:15:56.049735 2578 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 07 11:15:56.054486 ip-10-0-142-112 kubenswrapper[2578]: W0707 11:15:56.049740 2578 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 07 11:15:56.054486 ip-10-0-142-112 kubenswrapper[2578]: W0707 11:15:56.049744 2578 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 07 11:15:56.054486 ip-10-0-142-112 kubenswrapper[2578]: W0707 11:15:56.049749 2578 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 07 11:15:56.054486 ip-10-0-142-112 kubenswrapper[2578]: W0707 11:15:56.049753 2578 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 07 11:15:56.054486 ip-10-0-142-112 kubenswrapper[2578]: W0707 11:15:56.049757 2578 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 07 11:15:56.054486 ip-10-0-142-112 kubenswrapper[2578]: W0707 11:15:56.049762 2578 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 07 11:15:56.054486 ip-10-0-142-112 kubenswrapper[2578]: W0707 11:15:56.049766 2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 07 11:15:56.054486 ip-10-0-142-112 kubenswrapper[2578]: W0707 11:15:56.049770 2578 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 07 11:15:56.054486 ip-10-0-142-112 kubenswrapper[2578]: W0707 11:15:56.049774 2578 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 07 11:15:56.054486 ip-10-0-142-112 kubenswrapper[2578]: W0707 11:15:56.049779 2578 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 07 11:15:56.054486 ip-10-0-142-112 kubenswrapper[2578]: W0707 11:15:56.049783 2578 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 07 11:15:56.054486 ip-10-0-142-112 kubenswrapper[2578]: W0707 11:15:56.049788 2578 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 07 11:15:56.054486 ip-10-0-142-112 kubenswrapper[2578]: W0707 11:15:56.049792 2578 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 07 11:15:56.054486 ip-10-0-142-112 kubenswrapper[2578]: W0707 11:15:56.049797 2578 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 07 11:15:56.054486 ip-10-0-142-112 kubenswrapper[2578]: W0707 11:15:56.050452 2578 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 07 11:15:56.054486 ip-10-0-142-112 kubenswrapper[2578]: W0707 11:15:56.050463 2578 feature_gate.go:328] unrecognized feature gate: Example Jul 07 11:15:56.054486 ip-10-0-142-112 kubenswrapper[2578]: W0707 11:15:56.050468 2578 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 07 11:15:56.054960 ip-10-0-142-112 kubenswrapper[2578]: W0707 11:15:56.050473 2578 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 07 11:15:56.054960 ip-10-0-142-112 kubenswrapper[2578]: W0707 11:15:56.050477 2578 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 07 11:15:56.054960 ip-10-0-142-112 kubenswrapper[2578]: W0707 11:15:56.050483 2578 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 07 11:15:56.054960 ip-10-0-142-112 kubenswrapper[2578]: W0707 11:15:56.050488 2578 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 07 11:15:56.054960 ip-10-0-142-112 kubenswrapper[2578]: W0707 11:15:56.050492 2578 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 07 11:15:56.054960 ip-10-0-142-112 kubenswrapper[2578]: W0707 11:15:56.050497 2578 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 07 11:15:56.054960 ip-10-0-142-112 kubenswrapper[2578]: W0707 11:15:56.050501 2578 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 07 11:15:56.054960 ip-10-0-142-112 kubenswrapper[2578]: W0707 11:15:56.050505 2578 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 07 11:15:56.054960 ip-10-0-142-112 kubenswrapper[2578]: W0707 11:15:56.050510 2578 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 07 11:15:56.054960 ip-10-0-142-112 kubenswrapper[2578]: W0707 11:15:56.050514 2578 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 07 11:15:56.054960 ip-10-0-142-112 kubenswrapper[2578]: W0707 11:15:56.050518 2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 07 11:15:56.054960 ip-10-0-142-112 kubenswrapper[2578]: W0707 11:15:56.050522 2578 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 07 11:15:56.054960 ip-10-0-142-112 kubenswrapper[2578]: W0707 11:15:56.050527 2578 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 07 11:15:56.054960 ip-10-0-142-112 kubenswrapper[2578]: W0707 11:15:56.050532 2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 07 11:15:56.054960 ip-10-0-142-112 kubenswrapper[2578]: W0707 11:15:56.050537 2578 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 07 11:15:56.054960 ip-10-0-142-112 kubenswrapper[2578]: W0707 11:15:56.050541 2578 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 07 11:15:56.054960 ip-10-0-142-112 kubenswrapper[2578]: W0707 11:15:56.050546 2578 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 07 11:15:56.054960 ip-10-0-142-112 kubenswrapper[2578]: W0707 11:15:56.050549 2578 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 07 11:15:56.054960 ip-10-0-142-112 kubenswrapper[2578]: W0707 11:15:56.050554 2578 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 07 11:15:56.054960 ip-10-0-142-112 kubenswrapper[2578]: W0707 11:15:56.050558 2578 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 07 11:15:56.055574 ip-10-0-142-112 kubenswrapper[2578]: W0707 11:15:56.050562 2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 07 11:15:56.055574 ip-10-0-142-112 kubenswrapper[2578]: W0707 11:15:56.050566 2578 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 07 11:15:56.055574 ip-10-0-142-112 kubenswrapper[2578]: W0707 11:15:56.050570 2578 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 07 11:15:56.055574 ip-10-0-142-112 kubenswrapper[2578]: W0707 11:15:56.050574 2578 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 07 11:15:56.055574 ip-10-0-142-112 kubenswrapper[2578]: W0707 11:15:56.050578 2578 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 07 11:15:56.055574 ip-10-0-142-112 kubenswrapper[2578]: W0707 11:15:56.050583 2578 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 07 11:15:56.055574 ip-10-0-142-112 kubenswrapper[2578]: W0707 11:15:56.050587 2578 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 07 11:15:56.055574 ip-10-0-142-112 kubenswrapper[2578]: W0707 11:15:56.050591 2578 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 07 11:15:56.055574 ip-10-0-142-112 kubenswrapper[2578]: W0707 11:15:56.050595 2578 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 07 11:15:56.055574 ip-10-0-142-112 kubenswrapper[2578]: W0707 11:15:56.050599 2578 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 07 11:15:56.055574 ip-10-0-142-112 kubenswrapper[2578]: W0707 11:15:56.050603 2578 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 07 11:15:56.055574 ip-10-0-142-112 kubenswrapper[2578]: W0707 11:15:56.050608 2578 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 07 11:15:56.055574 ip-10-0-142-112 kubenswrapper[2578]: W0707 11:15:56.050613 2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 07 11:15:56.055574 ip-10-0-142-112 kubenswrapper[2578]: W0707 11:15:56.050617 2578 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 07 11:15:56.055574 ip-10-0-142-112 kubenswrapper[2578]: W0707 11:15:56.050621 2578 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 07 11:15:56.055574 ip-10-0-142-112 kubenswrapper[2578]: W0707 11:15:56.050625 2578 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 07 11:15:56.055574 ip-10-0-142-112 kubenswrapper[2578]: W0707 11:15:56.050629 2578 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 07 11:15:56.055574 ip-10-0-142-112 kubenswrapper[2578]: W0707 11:15:56.050633 2578 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 07 11:15:56.055574 ip-10-0-142-112 kubenswrapper[2578]: W0707 11:15:56.050637 2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 07 11:15:56.055574 ip-10-0-142-112 kubenswrapper[2578]: W0707 11:15:56.050641 2578 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 07 11:15:56.056067 ip-10-0-142-112 kubenswrapper[2578]: W0707 11:15:56.050645 2578 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 07 11:15:56.056067 ip-10-0-142-112 kubenswrapper[2578]: W0707 11:15:56.050649 2578 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 07 11:15:56.056067 ip-10-0-142-112 kubenswrapper[2578]: W0707 11:15:56.050654 2578 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 07 11:15:56.056067 ip-10-0-142-112 kubenswrapper[2578]: W0707 11:15:56.050658 2578 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 07 11:15:56.056067 ip-10-0-142-112 kubenswrapper[2578]: W0707 11:15:56.050662 2578 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 07 11:15:56.056067 ip-10-0-142-112 kubenswrapper[2578]: W0707 11:15:56.050667 2578 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 07 11:15:56.056067 ip-10-0-142-112 kubenswrapper[2578]: W0707 11:15:56.050672 2578 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 07 11:15:56.056067 ip-10-0-142-112 kubenswrapper[2578]: W0707 11:15:56.050676 2578 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 07 11:15:56.056067 ip-10-0-142-112 kubenswrapper[2578]: W0707 11:15:56.050680 2578 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 07 11:15:56.056067 ip-10-0-142-112 kubenswrapper[2578]: W0707 11:15:56.050684 2578 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 07 11:15:56.056067 ip-10-0-142-112 kubenswrapper[2578]: W0707 11:15:56.050688 2578 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 07 11:15:56.056067 ip-10-0-142-112 kubenswrapper[2578]: W0707 11:15:56.050692 2578 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 07 11:15:56.056067 ip-10-0-142-112 kubenswrapper[2578]: W0707 11:15:56.050696 2578 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 07 11:15:56.056067 ip-10-0-142-112 kubenswrapper[2578]: W0707 11:15:56.050702 2578 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 07 11:15:56.056067 ip-10-0-142-112 kubenswrapper[2578]: W0707 11:15:56.050707 2578 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 07 11:15:56.056067 ip-10-0-142-112 kubenswrapper[2578]: W0707 11:15:56.050711 2578 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 07 11:15:56.056067 ip-10-0-142-112 kubenswrapper[2578]: W0707 11:15:56.050719 2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 07 11:15:56.056067 ip-10-0-142-112 kubenswrapper[2578]: W0707 11:15:56.050723 2578 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 07 11:15:56.056067 ip-10-0-142-112 kubenswrapper[2578]: W0707 11:15:56.050728 2578 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 07 11:15:56.056067 ip-10-0-142-112 kubenswrapper[2578]: W0707 11:15:56.050732 2578 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 07 11:15:56.056592 ip-10-0-142-112 kubenswrapper[2578]: W0707 11:15:56.050736 2578 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 07 11:15:56.056592 ip-10-0-142-112 kubenswrapper[2578]: W0707 11:15:56.050743 2578 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 07 11:15:56.056592 ip-10-0-142-112 kubenswrapper[2578]: W0707 11:15:56.050750 2578 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 07 11:15:56.056592 ip-10-0-142-112 kubenswrapper[2578]: W0707 11:15:56.050755 2578 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 07 11:15:56.056592 ip-10-0-142-112 kubenswrapper[2578]: W0707 11:15:56.050760 2578 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 07 11:15:56.056592 ip-10-0-142-112 kubenswrapper[2578]: W0707 11:15:56.050764 2578 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 07 11:15:56.056592 ip-10-0-142-112 kubenswrapper[2578]: W0707 11:15:56.050773 2578 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 07 11:15:56.056592 ip-10-0-142-112 kubenswrapper[2578]: W0707 11:15:56.050779 2578 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 07 11:15:56.056592 ip-10-0-142-112 kubenswrapper[2578]: W0707 11:15:56.050784 2578 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 07 11:15:56.056592 ip-10-0-142-112 kubenswrapper[2578]: W0707 11:15:56.050788 2578 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 07 11:15:56.056592 ip-10-0-142-112 kubenswrapper[2578]: W0707 11:15:56.050793 2578 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 07 11:15:56.056592 ip-10-0-142-112 kubenswrapper[2578]: W0707 11:15:56.050797 2578 feature_gate.go:328] unrecognized feature gate: Example2 Jul 07 11:15:56.056592 ip-10-0-142-112 kubenswrapper[2578]: W0707 11:15:56.050801 2578 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 07 11:15:56.056592 ip-10-0-142-112 kubenswrapper[2578]: W0707 11:15:56.050805 2578 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 07 11:15:56.056592 ip-10-0-142-112 kubenswrapper[2578]: W0707 11:15:56.050810 2578 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 07 11:15:56.056592 ip-10-0-142-112 kubenswrapper[2578]: W0707 11:15:56.050816 2578 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 07 11:15:56.056592 ip-10-0-142-112 kubenswrapper[2578]: W0707 11:15:56.050822 2578 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 07 11:15:56.056592 ip-10-0-142-112 kubenswrapper[2578]: W0707 11:15:56.050826 2578 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 07 11:15:56.057014 ip-10-0-142-112 kubenswrapper[2578]: W0707 11:15:56.050831 2578 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 07 11:15:56.057014 ip-10-0-142-112 kubenswrapper[2578]: W0707 11:15:56.050836 2578 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 07 11:15:56.057014 ip-10-0-142-112 kubenswrapper[2578]: W0707 11:15:56.050840 2578 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 07 11:15:56.057014 ip-10-0-142-112 kubenswrapper[2578]: W0707 11:15:56.050845 2578 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 07 11:15:56.057014 ip-10-0-142-112 kubenswrapper[2578]: W0707 11:15:56.050849 2578 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 07 11:15:56.057014 ip-10-0-142-112 kubenswrapper[2578]: W0707 11:15:56.050853 2578 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 07 11:15:56.057014 ip-10-0-142-112 kubenswrapper[2578]: W0707 11:15:56.050858 2578 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 07 11:15:56.057014 ip-10-0-142-112 kubenswrapper[2578]: W0707 11:15:56.050862 2578 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 07 11:15:56.057014 ip-10-0-142-112 kubenswrapper[2578]: W0707 11:15:56.050866 2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 07 11:15:56.057014 ip-10-0-142-112 kubenswrapper[2578]: W0707 11:15:56.050871 2578 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 07 11:15:56.057014 ip-10-0-142-112 kubenswrapper[2578]: W0707 11:15:56.050876 2578 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 07 11:15:56.057014 ip-10-0-142-112 kubenswrapper[2578]: W0707 11:15:56.050880 2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 07 11:15:56.057014 ip-10-0-142-112 kubenswrapper[2578]: W0707 11:15:56.050884 2578 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 07 11:15:56.057014 ip-10-0-142-112 kubenswrapper[2578]: W0707 11:15:56.050889 2578 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 07 11:15:56.057014 ip-10-0-142-112 kubenswrapper[2578]: W0707 11:15:56.050893 2578 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 07 11:15:56.057014 ip-10-0-142-112 kubenswrapper[2578]: W0707 11:15:56.050899 2578 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 07 11:15:56.057014 ip-10-0-142-112 kubenswrapper[2578]: W0707 11:15:56.050904 2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 07 11:15:56.057014 ip-10-0-142-112 kubenswrapper[2578]: W0707 11:15:56.050908 2578 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 07 11:15:56.057014 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:15:56.051009 2578 flags.go:64] FLAG: --address="0.0.0.0" Jul 07 11:15:56.057014 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:15:56.051020 2578 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]" Jul 07 11:15:56.057501 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:15:56.051029 2578 flags.go:64] FLAG: --anonymous-auth="true" Jul 07 11:15:56.057501 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:15:56.051037 2578 flags.go:64] FLAG: --application-metrics-count-limit="100" Jul 07 11:15:56.057501 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:15:56.051044 2578 flags.go:64] FLAG: --authentication-token-webhook="false" Jul 07 11:15:56.057501 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:15:56.051049 2578 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s" Jul 07 11:15:56.057501 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:15:56.051055 2578 flags.go:64] FLAG: --authorization-mode="AlwaysAllow" Jul 07 11:15:56.057501 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:15:56.051062 2578 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s" Jul 07 11:15:56.057501 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:15:56.051067 2578 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s" Jul 07 11:15:56.057501 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:15:56.051072 2578 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id" Jul 07 11:15:56.057501 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:15:56.051078 2578 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig" Jul 07 11:15:56.057501 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:15:56.051084 2578 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki" Jul 07 11:15:56.057501 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:15:56.051089 2578 flags.go:64] FLAG: --cgroup-driver="cgroupfs" Jul 07 11:15:56.057501 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:15:56.051094 2578 flags.go:64] FLAG: --cgroup-root="" Jul 07 11:15:56.057501 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:15:56.051098 2578 flags.go:64] FLAG: --cgroups-per-qos="true" Jul 07 11:15:56.057501 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:15:56.051103 2578 flags.go:64] FLAG: --client-ca-file="" Jul 07 11:15:56.057501 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:15:56.051108 2578 flags.go:64] FLAG: --cloud-provider="external" Jul 07 11:15:56.057501 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:15:56.051113 2578 flags.go:64] FLAG: --cluster-dns="[]" Jul 07 11:15:56.057501 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:15:56.051121 2578 flags.go:64] FLAG: --cluster-domain="" Jul 07 11:15:56.057501 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:15:56.051126 2578 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf" Jul 07 11:15:56.057501 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:15:56.051131 2578 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d" Jul 07 11:15:56.057501 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:15:56.051136 2578 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json" Jul 07 11:15:56.057501 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:15:56.051142 2578 flags.go:64] FLAG: --container-log-max-files="5" Jul 07 11:15:56.057501 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:15:56.051148 2578 flags.go:64] FLAG: --container-log-max-size="10Mi" Jul 07 11:15:56.057501 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:15:56.051154 2578 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock" Jul 07 11:15:56.058100 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:15:56.051159 2578 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock" Jul 07 11:15:56.058100 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:15:56.051164 2578 flags.go:64] FLAG: --containerd-namespace="k8s.io" Jul 07 11:15:56.058100 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:15:56.051169 2578 flags.go:64] FLAG: --contention-profiling="false" Jul 07 11:15:56.058100 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:15:56.051174 2578 flags.go:64] FLAG: --cpu-cfs-quota="true" Jul 07 11:15:56.058100 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:15:56.051180 2578 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms" Jul 07 11:15:56.058100 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:15:56.051185 2578 flags.go:64] FLAG: --cpu-manager-policy="none" Jul 07 11:15:56.058100 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:15:56.051190 2578 flags.go:64] FLAG: --cpu-manager-policy-options="" Jul 07 11:15:56.058100 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:15:56.051196 2578 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s" Jul 07 11:15:56.058100 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:15:56.051201 2578 flags.go:64] FLAG: --enable-controller-attach-detach="true" Jul 07 11:15:56.058100 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:15:56.051206 2578 flags.go:64] FLAG: --enable-debugging-handlers="true" Jul 07 11:15:56.058100 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:15:56.051211 2578 flags.go:64] FLAG: --enable-load-reader="false" Jul 07 11:15:56.058100 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:15:56.051216 2578 flags.go:64] FLAG: --enable-server="true" Jul 07 11:15:56.058100 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:15:56.051220 2578 flags.go:64] FLAG: --enforce-node-allocatable="[pods]" Jul 07 11:15:56.058100 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:15:56.051228 2578 flags.go:64] FLAG: --event-burst="100" Jul 07 11:15:56.058100 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:15:56.051232 2578 flags.go:64] FLAG: --event-qps="50" Jul 07 11:15:56.058100 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:15:56.051237 2578 flags.go:64] FLAG: --event-storage-age-limit="default=0" Jul 07 11:15:56.058100 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:15:56.051242 2578 flags.go:64] FLAG: --event-storage-event-limit="default=0" Jul 07 11:15:56.058100 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:15:56.051247 2578 flags.go:64] FLAG: --eviction-hard="" Jul 07 11:15:56.058100 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:15:56.051253 2578 flags.go:64] FLAG: --eviction-max-pod-grace-period="0" Jul 07 11:15:56.058100 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:15:56.051258 2578 flags.go:64] FLAG: --eviction-minimum-reclaim="" Jul 07 11:15:56.058100 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:15:56.051263 2578 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s" Jul 07 11:15:56.058100 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:15:56.051268 2578 flags.go:64] FLAG: --eviction-soft="" Jul 07 11:15:56.058100 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:15:56.051272 2578 flags.go:64] FLAG: --eviction-soft-grace-period="" Jul 07 11:15:56.058100 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:15:56.051277 2578 flags.go:64] FLAG: --exit-on-lock-contention="false" Jul 07 11:15:56.058100 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:15:56.051282 2578 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false" Jul 07 11:15:56.058763 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:15:56.051286 2578 flags.go:64] FLAG: --experimental-mounter-path="" Jul 07 11:15:56.058763 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:15:56.051291 2578 flags.go:64] FLAG: --fail-cgroupv1="false" Jul 07 11:15:56.058763 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:15:56.051295 2578 flags.go:64] FLAG: --fail-swap-on="true" Jul 07 11:15:56.058763 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:15:56.051299 2578 flags.go:64] FLAG: --feature-gates="" Jul 07 11:15:56.058763 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:15:56.051313 2578 flags.go:64] FLAG: --file-check-frequency="20s" Jul 07 11:15:56.058763 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:15:56.051319 2578 flags.go:64] FLAG: --global-housekeeping-interval="1m0s" Jul 07 11:15:56.058763 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:15:56.051326 2578 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge" Jul 07 11:15:56.058763 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:15:56.051331 2578 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1" Jul 07 11:15:56.058763 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:15:56.051336 2578 flags.go:64] FLAG: --healthz-port="10248" Jul 07 11:15:56.058763 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:15:56.051341 2578 flags.go:64] FLAG: --help="false" Jul 07 11:15:56.058763 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:15:56.051345 2578 flags.go:64] FLAG: --hostname-override="ip-10-0-142-112.ec2.internal" Jul 07 11:15:56.058763 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:15:56.051352 2578 flags.go:64] FLAG: --housekeeping-interval="10s" Jul 07 11:15:56.058763 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:15:56.051357 2578 flags.go:64] FLAG: --http-check-frequency="20s" Jul 07 11:15:56.058763 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:15:56.051362 2578 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins" Jul 07 11:15:56.058763 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:15:56.051367 2578 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml" Jul 07 11:15:56.058763 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:15:56.051373 2578 flags.go:64] FLAG: --image-gc-high-threshold="85" Jul 07 11:15:56.058763 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:15:56.051377 2578 flags.go:64] FLAG: --image-gc-low-threshold="80" Jul 07 11:15:56.058763 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:15:56.051382 2578 flags.go:64] FLAG: --image-service-endpoint="" Jul 07 11:15:56.058763 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:15:56.051386 2578 flags.go:64] FLAG: --kernel-memcg-notification="false" Jul 07 11:15:56.058763 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:15:56.051391 2578 flags.go:64] FLAG: --kube-api-burst="100" Jul 07 11:15:56.058763 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:15:56.051395 2578 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf" Jul 07 11:15:56.058763 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:15:56.051401 2578 flags.go:64] FLAG: --kube-api-qps="50" Jul 07 11:15:56.058763 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:15:56.051405 2578 flags.go:64] FLAG: --kube-reserved="" Jul 07 11:15:56.058763 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:15:56.051410 2578 flags.go:64] FLAG: --kube-reserved-cgroup="" Jul 07 11:15:56.059346 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:15:56.051414 2578 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig" Jul 07 11:15:56.059346 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:15:56.051435 2578 flags.go:64] FLAG: --kubelet-cgroups="" Jul 07 11:15:56.059346 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:15:56.051440 2578 flags.go:64] FLAG: --local-storage-capacity-isolation="true" Jul 07 11:15:56.059346 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:15:56.051445 2578 flags.go:64] FLAG: --lock-file="" Jul 07 11:15:56.059346 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:15:56.051449 2578 flags.go:64] FLAG: --log-cadvisor-usage="false" Jul 07 11:15:56.059346 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:15:56.051453 2578 flags.go:64] FLAG: --log-flush-frequency="5s" Jul 07 11:15:56.059346 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:15:56.051459 2578 flags.go:64] FLAG: --log-json-info-buffer-size="0" Jul 07 11:15:56.059346 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:15:56.051468 2578 flags.go:64] FLAG: --log-json-split-stream="false" Jul 07 11:15:56.059346 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:15:56.051474 2578 flags.go:64] FLAG: --log-text-info-buffer-size="0" Jul 07 11:15:56.059346 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:15:56.051479 2578 flags.go:64] FLAG: --log-text-split-stream="false" Jul 07 11:15:56.059346 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:15:56.051483 2578 flags.go:64] FLAG: --logging-format="text" Jul 07 11:15:56.059346 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:15:56.051488 2578 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id" Jul 07 11:15:56.059346 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:15:56.051493 2578 flags.go:64] FLAG: --make-iptables-util-chains="true" Jul 07 11:15:56.059346 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:15:56.051518 2578 flags.go:64] FLAG: --manifest-url="" Jul 07 11:15:56.059346 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:15:56.051526 2578 flags.go:64] FLAG: --manifest-url-header="" Jul 07 11:15:56.059346 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:15:56.051534 2578 flags.go:64] FLAG: --max-housekeeping-interval="15s" Jul 07 11:15:56.059346 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:15:56.051540 2578 flags.go:64] FLAG: --max-open-files="1000000" Jul 07 11:15:56.059346 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:15:56.051547 2578 flags.go:64] FLAG: --max-pods="110" Jul 07 11:15:56.059346 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:15:56.051552 2578 flags.go:64] FLAG: --maximum-dead-containers="-1" Jul 07 11:15:56.059346 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:15:56.051559 2578 flags.go:64] FLAG: --maximum-dead-containers-per-container="1" Jul 07 11:15:56.059346 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:15:56.051564 2578 flags.go:64] FLAG: --memory-manager-policy="None" Jul 07 11:15:56.059346 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:15:56.051568 2578 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s" Jul 07 11:15:56.059346 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:15:56.051574 2578 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s" Jul 07 11:15:56.059346 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:15:56.051579 2578 flags.go:64] FLAG: --node-ip="0.0.0.0" Jul 07 11:15:56.059346 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:15:56.051583 2578 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel" Jul 07 11:15:56.059958 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:15:56.051596 2578 flags.go:64] FLAG: --node-status-max-images="50" Jul 07 11:15:56.059958 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:15:56.051600 2578 flags.go:64] FLAG: --node-status-update-frequency="10s" Jul 07 11:15:56.059958 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:15:56.051606 2578 flags.go:64] FLAG: --oom-score-adj="-999" Jul 07 11:15:56.059958 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:15:56.051611 2578 flags.go:64] FLAG: --pod-cidr="" Jul 07 11:15:56.059958 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:15:56.051615 2578 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:ce93f6b4197b9c62a26a3131f8057ffe9c9379c5079a4f62e5919b75151394e5" Jul 07 11:15:56.059958 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:15:56.051623 2578 flags.go:64] FLAG: --pod-manifest-path="" Jul 07 11:15:56.059958 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:15:56.051628 2578 flags.go:64] FLAG: --pod-max-pids="-1" Jul 07 11:15:56.059958 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:15:56.051633 2578 flags.go:64] FLAG: --pods-per-core="0" Jul 07 11:15:56.059958 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:15:56.051637 2578 flags.go:64] FLAG: --port="10250" Jul 07 11:15:56.059958 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:15:56.051642 2578 flags.go:64] FLAG: --protect-kernel-defaults="false" Jul 07 11:15:56.059958 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:15:56.051647 2578 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-05fda285c54212a53" Jul 07 11:15:56.059958 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:15:56.051652 2578 flags.go:64] FLAG: --qos-reserved="" Jul 07 11:15:56.059958 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:15:56.051656 2578 flags.go:64] FLAG: --read-only-port="10255" Jul 07 11:15:56.059958 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:15:56.051661 2578 flags.go:64] FLAG: --register-node="true" Jul 07 11:15:56.059958 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:15:56.051666 2578 flags.go:64] FLAG: --register-with-taints="" Jul 07 11:15:56.059958 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:15:56.051672 2578 flags.go:64] FLAG: --registry-burst="10" Jul 07 11:15:56.059958 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:15:56.051677 2578 flags.go:64] FLAG: --registry-qps="5" Jul 07 11:15:56.059958 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:15:56.051682 2578 flags.go:64] FLAG: --reserved-cpus="" Jul 07 11:15:56.059958 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:15:56.051686 2578 flags.go:64] FLAG: --reserved-memory="" Jul 07 11:15:56.059958 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:15:56.051696 2578 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf" Jul 07 11:15:56.059958 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:15:56.051701 2578 flags.go:64] FLAG: --root-dir="/var/lib/kubelet" Jul 07 11:15:56.059958 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:15:56.051706 2578 flags.go:64] FLAG: --rotate-certificates="false" Jul 07 11:15:56.059958 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:15:56.051712 2578 flags.go:64] FLAG: --rotate-server-certificates="false" Jul 07 11:15:56.059958 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:15:56.051717 2578 flags.go:64] FLAG: --runonce="false" Jul 07 11:15:56.059958 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:15:56.051721 2578 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service" Jul 07 11:15:56.060561 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:15:56.051726 2578 flags.go:64] FLAG: --runtime-request-timeout="2m0s" Jul 07 11:15:56.060561 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:15:56.051733 2578 flags.go:64] FLAG: --seccomp-default="false" Jul 07 11:15:56.060561 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:15:56.051738 2578 flags.go:64] FLAG: --serialize-image-pulls="true" Jul 07 11:15:56.060561 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:15:56.051743 2578 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s" Jul 07 11:15:56.060561 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:15:56.051748 2578 flags.go:64] FLAG: --storage-driver-db="cadvisor" Jul 07 11:15:56.060561 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:15:56.051753 2578 flags.go:64] FLAG: --storage-driver-host="localhost:8086" Jul 07 11:15:56.060561 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:15:56.051758 2578 flags.go:64] FLAG: --storage-driver-password="root" Jul 07 11:15:56.060561 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:15:56.051762 2578 flags.go:64] FLAG: --storage-driver-secure="false" Jul 07 11:15:56.060561 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:15:56.051767 2578 flags.go:64] FLAG: --storage-driver-table="stats" Jul 07 11:15:56.060561 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:15:56.051772 2578 flags.go:64] FLAG: --storage-driver-user="root" Jul 07 11:15:56.060561 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:15:56.051777 2578 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s" Jul 07 11:15:56.060561 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:15:56.051782 2578 flags.go:64] FLAG: --sync-frequency="1m0s" Jul 07 11:15:56.060561 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:15:56.051787 2578 flags.go:64] FLAG: --system-cgroups="" Jul 07 11:15:56.060561 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:15:56.051791 2578 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi" Jul 07 11:15:56.060561 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:15:56.051800 2578 flags.go:64] FLAG: --system-reserved-cgroup="" Jul 07 11:15:56.060561 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:15:56.051805 2578 flags.go:64] FLAG: --tls-cert-file="" Jul 07 11:15:56.060561 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:15:56.051809 2578 flags.go:64] FLAG: --tls-cipher-suites="[]" Jul 07 11:15:56.060561 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:15:56.051818 2578 flags.go:64] FLAG: --tls-min-version="" Jul 07 11:15:56.060561 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:15:56.051823 2578 flags.go:64] FLAG: --tls-private-key-file="" Jul 07 11:15:56.060561 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:15:56.051827 2578 flags.go:64] FLAG: --topology-manager-policy="none" Jul 07 11:15:56.060561 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:15:56.051832 2578 flags.go:64] FLAG: --topology-manager-policy-options="" Jul 07 11:15:56.060561 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:15:56.051836 2578 flags.go:64] FLAG: --topology-manager-scope="container" Jul 07 11:15:56.060561 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:15:56.051841 2578 flags.go:64] FLAG: --v="2" Jul 07 11:15:56.060561 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:15:56.051847 2578 flags.go:64] FLAG: --version="false" Jul 07 11:15:56.060561 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:15:56.051854 2578 flags.go:64] FLAG: --vmodule="" Jul 07 11:15:56.061162 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:15:56.051860 2578 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec" Jul 07 11:15:56.061162 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:15:56.051866 2578 flags.go:64] FLAG: --volume-stats-agg-period="1m0s" Jul 07 11:15:56.061162 ip-10-0-142-112 kubenswrapper[2578]: W0707 11:15:56.052046 2578 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 07 11:15:56.061162 ip-10-0-142-112 kubenswrapper[2578]: W0707 11:15:56.052054 2578 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 07 11:15:56.061162 ip-10-0-142-112 kubenswrapper[2578]: W0707 11:15:56.052062 2578 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 07 11:15:56.061162 ip-10-0-142-112 kubenswrapper[2578]: W0707 11:15:56.052066 2578 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 07 11:15:56.061162 ip-10-0-142-112 kubenswrapper[2578]: W0707 11:15:56.052071 2578 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 07 11:15:56.061162 ip-10-0-142-112 kubenswrapper[2578]: W0707 11:15:56.052075 2578 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 07 11:15:56.061162 ip-10-0-142-112 kubenswrapper[2578]: W0707 11:15:56.052084 2578 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 07 11:15:56.061162 ip-10-0-142-112 kubenswrapper[2578]: W0707 11:15:56.052089 2578 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 07 11:15:56.061162 ip-10-0-142-112 kubenswrapper[2578]: W0707 11:15:56.052094 2578 feature_gate.go:328] unrecognized feature gate: Example2 Jul 07 11:15:56.061162 ip-10-0-142-112 kubenswrapper[2578]: W0707 11:15:56.052099 2578 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 07 11:15:56.061162 ip-10-0-142-112 kubenswrapper[2578]: W0707 11:15:56.052103 2578 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 07 11:15:56.061162 ip-10-0-142-112 kubenswrapper[2578]: W0707 11:15:56.052107 2578 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 07 11:15:56.061162 ip-10-0-142-112 kubenswrapper[2578]: W0707 11:15:56.052112 2578 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 07 11:15:56.061162 ip-10-0-142-112 kubenswrapper[2578]: W0707 11:15:56.052117 2578 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 07 11:15:56.061162 ip-10-0-142-112 kubenswrapper[2578]: W0707 11:15:56.052121 2578 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 07 11:15:56.061162 ip-10-0-142-112 kubenswrapper[2578]: W0707 11:15:56.052125 2578 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 07 11:15:56.061162 ip-10-0-142-112 kubenswrapper[2578]: W0707 11:15:56.052130 2578 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 07 11:15:56.061672 ip-10-0-142-112 kubenswrapper[2578]: W0707 11:15:56.052134 2578 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 07 11:15:56.061672 ip-10-0-142-112 kubenswrapper[2578]: W0707 11:15:56.052138 2578 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 07 11:15:56.061672 ip-10-0-142-112 kubenswrapper[2578]: W0707 11:15:56.052143 2578 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 07 11:15:56.061672 ip-10-0-142-112 kubenswrapper[2578]: W0707 11:15:56.052147 2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 07 11:15:56.061672 ip-10-0-142-112 kubenswrapper[2578]: W0707 11:15:56.052151 2578 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 07 11:15:56.061672 ip-10-0-142-112 kubenswrapper[2578]: W0707 11:15:56.052155 2578 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 07 11:15:56.061672 ip-10-0-142-112 kubenswrapper[2578]: W0707 11:15:56.052159 2578 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 07 11:15:56.061672 ip-10-0-142-112 kubenswrapper[2578]: W0707 11:15:56.052164 2578 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 07 11:15:56.061672 ip-10-0-142-112 kubenswrapper[2578]: W0707 11:15:56.052168 2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 07 11:15:56.061672 ip-10-0-142-112 kubenswrapper[2578]: W0707 11:15:56.052172 2578 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 07 11:15:56.061672 ip-10-0-142-112 kubenswrapper[2578]: W0707 11:15:56.052176 2578 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 07 11:15:56.061672 ip-10-0-142-112 kubenswrapper[2578]: W0707 11:15:56.052181 2578 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 07 11:15:56.061672 ip-10-0-142-112 kubenswrapper[2578]: W0707 11:15:56.052185 2578 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 07 11:15:56.061672 ip-10-0-142-112 kubenswrapper[2578]: W0707 11:15:56.052190 2578 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 07 11:15:56.061672 ip-10-0-142-112 kubenswrapper[2578]: W0707 11:15:56.052194 2578 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 07 11:15:56.061672 ip-10-0-142-112 kubenswrapper[2578]: W0707 11:15:56.052198 2578 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 07 11:15:56.061672 ip-10-0-142-112 kubenswrapper[2578]: W0707 11:15:56.052202 2578 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 07 11:15:56.061672 ip-10-0-142-112 kubenswrapper[2578]: W0707 11:15:56.052209 2578 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 07 11:15:56.061672 ip-10-0-142-112 kubenswrapper[2578]: W0707 11:15:56.052213 2578 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 07 11:15:56.061672 ip-10-0-142-112 kubenswrapper[2578]: W0707 11:15:56.052217 2578 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 07 11:15:56.062139 ip-10-0-142-112 kubenswrapper[2578]: W0707 11:15:56.052222 2578 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 07 11:15:56.062139 ip-10-0-142-112 kubenswrapper[2578]: W0707 11:15:56.052228 2578 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 07 11:15:56.062139 ip-10-0-142-112 kubenswrapper[2578]: W0707 11:15:56.052231 2578 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 07 11:15:56.062139 ip-10-0-142-112 kubenswrapper[2578]: W0707 11:15:56.052236 2578 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 07 11:15:56.062139 ip-10-0-142-112 kubenswrapper[2578]: W0707 11:15:56.052240 2578 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 07 11:15:56.062139 ip-10-0-142-112 kubenswrapper[2578]: W0707 11:15:56.052245 2578 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 07 11:15:56.062139 ip-10-0-142-112 kubenswrapper[2578]: W0707 11:15:56.052249 2578 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 07 11:15:56.062139 ip-10-0-142-112 kubenswrapper[2578]: W0707 11:15:56.052253 2578 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 07 11:15:56.062139 ip-10-0-142-112 kubenswrapper[2578]: W0707 11:15:56.052257 2578 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 07 11:15:56.062139 ip-10-0-142-112 kubenswrapper[2578]: W0707 11:15:56.052261 2578 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 07 11:15:56.062139 ip-10-0-142-112 kubenswrapper[2578]: W0707 11:15:56.052265 2578 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 07 11:15:56.062139 ip-10-0-142-112 kubenswrapper[2578]: W0707 11:15:56.052269 2578 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 07 11:15:56.062139 ip-10-0-142-112 kubenswrapper[2578]: W0707 11:15:56.052273 2578 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 07 11:15:56.062139 ip-10-0-142-112 kubenswrapper[2578]: W0707 11:15:56.052277 2578 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 07 11:15:56.062139 ip-10-0-142-112 kubenswrapper[2578]: W0707 11:15:56.052281 2578 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 07 11:15:56.062139 ip-10-0-142-112 kubenswrapper[2578]: W0707 11:15:56.052285 2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 07 11:15:56.062139 ip-10-0-142-112 kubenswrapper[2578]: W0707 11:15:56.052289 2578 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 07 11:15:56.062139 ip-10-0-142-112 kubenswrapper[2578]: W0707 11:15:56.052293 2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 07 11:15:56.062139 ip-10-0-142-112 kubenswrapper[2578]: W0707 11:15:56.052298 2578 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 07 11:15:56.062676 ip-10-0-142-112 kubenswrapper[2578]: W0707 11:15:56.052302 2578 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 07 11:15:56.062676 ip-10-0-142-112 kubenswrapper[2578]: W0707 11:15:56.052306 2578 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 07 11:15:56.062676 ip-10-0-142-112 kubenswrapper[2578]: W0707 11:15:56.052310 2578 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 07 11:15:56.062676 ip-10-0-142-112 kubenswrapper[2578]: W0707 11:15:56.052315 2578 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 07 11:15:56.062676 ip-10-0-142-112 kubenswrapper[2578]: W0707 11:15:56.052318 2578 feature_gate.go:328] unrecognized feature gate: Example Jul 07 11:15:56.062676 ip-10-0-142-112 kubenswrapper[2578]: W0707 11:15:56.052323 2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 07 11:15:56.062676 ip-10-0-142-112 kubenswrapper[2578]: W0707 11:15:56.052327 2578 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 07 11:15:56.062676 ip-10-0-142-112 kubenswrapper[2578]: W0707 11:15:56.052332 2578 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 07 11:15:56.062676 ip-10-0-142-112 kubenswrapper[2578]: W0707 11:15:56.052336 2578 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 07 11:15:56.062676 ip-10-0-142-112 kubenswrapper[2578]: W0707 11:15:56.052340 2578 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 07 11:15:56.062676 ip-10-0-142-112 kubenswrapper[2578]: W0707 11:15:56.052346 2578 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 07 11:15:56.062676 ip-10-0-142-112 kubenswrapper[2578]: W0707 11:15:56.052350 2578 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 07 11:15:56.062676 ip-10-0-142-112 kubenswrapper[2578]: W0707 11:15:56.052354 2578 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 07 11:15:56.062676 ip-10-0-142-112 kubenswrapper[2578]: W0707 11:15:56.052358 2578 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 07 11:15:56.062676 ip-10-0-142-112 kubenswrapper[2578]: W0707 11:15:56.052364 2578 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 07 11:15:56.062676 ip-10-0-142-112 kubenswrapper[2578]: W0707 11:15:56.052371 2578 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 07 11:15:56.062676 ip-10-0-142-112 kubenswrapper[2578]: W0707 11:15:56.052376 2578 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 07 11:15:56.062676 ip-10-0-142-112 kubenswrapper[2578]: W0707 11:15:56.052381 2578 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 07 11:15:56.062676 ip-10-0-142-112 kubenswrapper[2578]: W0707 11:15:56.052387 2578 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 07 11:15:56.063133 ip-10-0-142-112 kubenswrapper[2578]: W0707 11:15:56.052392 2578 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 07 11:15:56.063133 ip-10-0-142-112 kubenswrapper[2578]: W0707 11:15:56.052397 2578 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 07 11:15:56.063133 ip-10-0-142-112 kubenswrapper[2578]: W0707 11:15:56.052401 2578 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 07 11:15:56.063133 ip-10-0-142-112 kubenswrapper[2578]: W0707 11:15:56.052406 2578 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 07 11:15:56.063133 ip-10-0-142-112 kubenswrapper[2578]: W0707 11:15:56.052411 2578 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 07 11:15:56.063133 ip-10-0-142-112 kubenswrapper[2578]: W0707 11:15:56.052415 2578 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 07 11:15:56.063133 ip-10-0-142-112 kubenswrapper[2578]: W0707 11:15:56.052436 2578 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 07 11:15:56.063133 ip-10-0-142-112 kubenswrapper[2578]: W0707 11:15:56.052441 2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 07 11:15:56.063133 ip-10-0-142-112 kubenswrapper[2578]: W0707 11:15:56.052445 2578 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 07 11:15:56.063133 ip-10-0-142-112 kubenswrapper[2578]: W0707 11:15:56.052449 2578 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 07 11:15:56.063133 ip-10-0-142-112 kubenswrapper[2578]: W0707 11:15:56.052453 2578 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 07 11:15:56.063133 ip-10-0-142-112 kubenswrapper[2578]: W0707 11:15:56.052458 2578 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 07 11:15:56.063133 ip-10-0-142-112 kubenswrapper[2578]: W0707 11:15:56.052463 2578 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 07 11:15:56.063133 ip-10-0-142-112 kubenswrapper[2578]: W0707 11:15:56.052467 2578 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 07 11:15:56.063133 ip-10-0-142-112 kubenswrapper[2578]: W0707 11:15:56.052472 2578 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 07 11:15:56.063133 ip-10-0-142-112 kubenswrapper[2578]: W0707 11:15:56.052476 2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 07 11:15:56.063133 ip-10-0-142-112 kubenswrapper[2578]: W0707 11:15:56.052480 2578 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 07 11:15:56.063133 ip-10-0-142-112 kubenswrapper[2578]: W0707 11:15:56.052484 2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 07 11:15:56.063133 ip-10-0-142-112 kubenswrapper[2578]: W0707 11:15:56.052489 2578 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 07 11:15:56.063133 ip-10-0-142-112 kubenswrapper[2578]: W0707 11:15:56.052493 2578 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 07 11:15:56.063629 ip-10-0-142-112 kubenswrapper[2578]: W0707 11:15:56.052497 2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 07 11:15:56.063629 ip-10-0-142-112 kubenswrapper[2578]: W0707 11:15:56.052502 2578 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 07 11:15:56.063629 ip-10-0-142-112 kubenswrapper[2578]: W0707 11:15:56.052507 2578 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 07 11:15:56.063629 ip-10-0-142-112 kubenswrapper[2578]: W0707 11:15:56.052513 2578 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 07 11:15:56.063629 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:15:56.052522 2578 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 07 11:15:56.063629 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:15:56.059654 2578 server.go:529] "Kubelet version" kubeletVersion="v1.34.8" Jul 07 11:15:56.063629 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:15:56.059670 2578 server.go:531] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK="" Jul 07 11:15:56.063629 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:15:56.061474 2578 watchdog_linux.go:95] "Systemd watchdog is not enabled" Jul 07 11:15:56.063629 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:15:56.061483 2578 watchdog_linux.go:137] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started." Jul 07 11:15:56.063629 ip-10-0-142-112 kubenswrapper[2578]: W0707 11:15:56.061542 2578 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 07 11:15:56.063629 ip-10-0-142-112 kubenswrapper[2578]: W0707 11:15:56.061547 2578 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 07 11:15:56.063629 ip-10-0-142-112 kubenswrapper[2578]: W0707 11:15:56.061550 2578 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 07 11:15:56.063629 ip-10-0-142-112 kubenswrapper[2578]: W0707 11:15:56.061554 2578 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 07 11:15:56.063629 ip-10-0-142-112 kubenswrapper[2578]: W0707 11:15:56.061557 2578 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 07 11:15:56.063629 ip-10-0-142-112 kubenswrapper[2578]: W0707 11:15:56.061561 2578 feature_gate.go:328] unrecognized feature gate: Example2 Jul 07 11:15:56.063997 ip-10-0-142-112 kubenswrapper[2578]: W0707 11:15:56.061563 2578 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 07 11:15:56.063997 ip-10-0-142-112 kubenswrapper[2578]: W0707 11:15:56.061567 2578 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 07 11:15:56.063997 ip-10-0-142-112 kubenswrapper[2578]: W0707 11:15:56.061570 2578 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 07 11:15:56.063997 ip-10-0-142-112 kubenswrapper[2578]: W0707 11:15:56.061573 2578 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 07 11:15:56.063997 ip-10-0-142-112 kubenswrapper[2578]: W0707 11:15:56.061576 2578 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 07 11:15:56.063997 ip-10-0-142-112 kubenswrapper[2578]: W0707 11:15:56.061578 2578 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 07 11:15:56.063997 ip-10-0-142-112 kubenswrapper[2578]: W0707 11:15:56.061581 2578 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 07 11:15:56.063997 ip-10-0-142-112 kubenswrapper[2578]: W0707 11:15:56.061584 2578 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 07 11:15:56.063997 ip-10-0-142-112 kubenswrapper[2578]: W0707 11:15:56.061586 2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 07 11:15:56.063997 ip-10-0-142-112 kubenswrapper[2578]: W0707 11:15:56.061589 2578 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 07 11:15:56.063997 ip-10-0-142-112 kubenswrapper[2578]: W0707 11:15:56.061592 2578 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 07 11:15:56.063997 ip-10-0-142-112 kubenswrapper[2578]: W0707 11:15:56.061594 2578 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 07 11:15:56.063997 ip-10-0-142-112 kubenswrapper[2578]: W0707 11:15:56.061597 2578 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 07 11:15:56.063997 ip-10-0-142-112 kubenswrapper[2578]: W0707 11:15:56.061599 2578 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 07 11:15:56.063997 ip-10-0-142-112 kubenswrapper[2578]: W0707 11:15:56.061602 2578 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 07 11:15:56.063997 ip-10-0-142-112 kubenswrapper[2578]: W0707 11:15:56.061605 2578 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 07 11:15:56.063997 ip-10-0-142-112 kubenswrapper[2578]: W0707 11:15:56.061607 2578 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 07 11:15:56.063997 ip-10-0-142-112 kubenswrapper[2578]: W0707 11:15:56.061610 2578 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 07 11:15:56.063997 ip-10-0-142-112 kubenswrapper[2578]: W0707 11:15:56.061613 2578 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 07 11:15:56.063997 ip-10-0-142-112 kubenswrapper[2578]: W0707 11:15:56.061615 2578 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 07 11:15:56.064493 ip-10-0-142-112 kubenswrapper[2578]: W0707 11:15:56.061618 2578 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 07 11:15:56.064493 ip-10-0-142-112 kubenswrapper[2578]: W0707 11:15:56.061621 2578 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 07 11:15:56.064493 ip-10-0-142-112 kubenswrapper[2578]: W0707 11:15:56.061623 2578 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 07 11:15:56.064493 ip-10-0-142-112 kubenswrapper[2578]: W0707 11:15:56.061626 2578 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 07 11:15:56.064493 ip-10-0-142-112 kubenswrapper[2578]: W0707 11:15:56.061629 2578 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 07 11:15:56.064493 ip-10-0-142-112 kubenswrapper[2578]: W0707 11:15:56.061632 2578 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 07 11:15:56.064493 ip-10-0-142-112 kubenswrapper[2578]: W0707 11:15:56.061635 2578 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 07 11:15:56.064493 ip-10-0-142-112 kubenswrapper[2578]: W0707 11:15:56.061637 2578 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 07 11:15:56.064493 ip-10-0-142-112 kubenswrapper[2578]: W0707 11:15:56.061640 2578 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 07 11:15:56.064493 ip-10-0-142-112 kubenswrapper[2578]: W0707 11:15:56.061643 2578 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 07 11:15:56.064493 ip-10-0-142-112 kubenswrapper[2578]: W0707 11:15:56.061645 2578 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 07 11:15:56.064493 ip-10-0-142-112 kubenswrapper[2578]: W0707 11:15:56.061648 2578 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 07 11:15:56.064493 ip-10-0-142-112 kubenswrapper[2578]: W0707 11:15:56.061650 2578 feature_gate.go:328] unrecognized feature gate: Example Jul 07 11:15:56.064493 ip-10-0-142-112 kubenswrapper[2578]: W0707 11:15:56.061653 2578 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 07 11:15:56.064493 ip-10-0-142-112 kubenswrapper[2578]: W0707 11:15:56.061655 2578 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 07 11:15:56.064493 ip-10-0-142-112 kubenswrapper[2578]: W0707 11:15:56.061658 2578 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 07 11:15:56.064493 ip-10-0-142-112 kubenswrapper[2578]: W0707 11:15:56.061661 2578 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 07 11:15:56.064493 ip-10-0-142-112 kubenswrapper[2578]: W0707 11:15:56.061663 2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 07 11:15:56.064493 ip-10-0-142-112 kubenswrapper[2578]: W0707 11:15:56.061666 2578 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 07 11:15:56.064493 ip-10-0-142-112 kubenswrapper[2578]: W0707 11:15:56.061668 2578 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 07 11:15:56.064965 ip-10-0-142-112 kubenswrapper[2578]: W0707 11:15:56.061670 2578 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 07 11:15:56.064965 ip-10-0-142-112 kubenswrapper[2578]: W0707 11:15:56.061673 2578 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 07 11:15:56.064965 ip-10-0-142-112 kubenswrapper[2578]: W0707 11:15:56.061676 2578 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 07 11:15:56.064965 ip-10-0-142-112 kubenswrapper[2578]: W0707 11:15:56.061678 2578 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 07 11:15:56.064965 ip-10-0-142-112 kubenswrapper[2578]: W0707 11:15:56.061681 2578 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 07 11:15:56.064965 ip-10-0-142-112 kubenswrapper[2578]: W0707 11:15:56.061683 2578 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 07 11:15:56.064965 ip-10-0-142-112 kubenswrapper[2578]: W0707 11:15:56.061686 2578 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 07 11:15:56.064965 ip-10-0-142-112 kubenswrapper[2578]: W0707 11:15:56.061689 2578 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 07 11:15:56.064965 ip-10-0-142-112 kubenswrapper[2578]: W0707 11:15:56.061691 2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 07 11:15:56.064965 ip-10-0-142-112 kubenswrapper[2578]: W0707 11:15:56.061695 2578 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 07 11:15:56.064965 ip-10-0-142-112 kubenswrapper[2578]: W0707 11:15:56.061699 2578 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 07 11:15:56.064965 ip-10-0-142-112 kubenswrapper[2578]: W0707 11:15:56.061702 2578 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 07 11:15:56.064965 ip-10-0-142-112 kubenswrapper[2578]: W0707 11:15:56.061705 2578 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 07 11:15:56.064965 ip-10-0-142-112 kubenswrapper[2578]: W0707 11:15:56.061707 2578 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 07 11:15:56.064965 ip-10-0-142-112 kubenswrapper[2578]: W0707 11:15:56.061710 2578 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 07 11:15:56.064965 ip-10-0-142-112 kubenswrapper[2578]: W0707 11:15:56.061714 2578 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 07 11:15:56.064965 ip-10-0-142-112 kubenswrapper[2578]: W0707 11:15:56.061716 2578 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 07 11:15:56.064965 ip-10-0-142-112 kubenswrapper[2578]: W0707 11:15:56.061719 2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 07 11:15:56.064965 ip-10-0-142-112 kubenswrapper[2578]: W0707 11:15:56.061722 2578 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 07 11:15:56.065433 ip-10-0-142-112 kubenswrapper[2578]: W0707 11:15:56.061726 2578 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 07 11:15:56.065433 ip-10-0-142-112 kubenswrapper[2578]: W0707 11:15:56.061730 2578 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 07 11:15:56.065433 ip-10-0-142-112 kubenswrapper[2578]: W0707 11:15:56.061733 2578 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 07 11:15:56.065433 ip-10-0-142-112 kubenswrapper[2578]: W0707 11:15:56.061736 2578 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 07 11:15:56.065433 ip-10-0-142-112 kubenswrapper[2578]: W0707 11:15:56.061738 2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 07 11:15:56.065433 ip-10-0-142-112 kubenswrapper[2578]: W0707 11:15:56.061741 2578 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 07 11:15:56.065433 ip-10-0-142-112 kubenswrapper[2578]: W0707 11:15:56.061744 2578 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 07 11:15:56.065433 ip-10-0-142-112 kubenswrapper[2578]: W0707 11:15:56.061747 2578 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 07 11:15:56.065433 ip-10-0-142-112 kubenswrapper[2578]: W0707 11:15:56.061750 2578 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 07 11:15:56.065433 ip-10-0-142-112 kubenswrapper[2578]: W0707 11:15:56.061753 2578 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 07 11:15:56.065433 ip-10-0-142-112 kubenswrapper[2578]: W0707 11:15:56.061755 2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 07 11:15:56.065433 ip-10-0-142-112 kubenswrapper[2578]: W0707 11:15:56.061759 2578 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 07 11:15:56.065433 ip-10-0-142-112 kubenswrapper[2578]: W0707 11:15:56.061761 2578 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 07 11:15:56.065433 ip-10-0-142-112 kubenswrapper[2578]: W0707 11:15:56.061764 2578 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 07 11:15:56.065433 ip-10-0-142-112 kubenswrapper[2578]: W0707 11:15:56.061767 2578 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 07 11:15:56.065433 ip-10-0-142-112 kubenswrapper[2578]: W0707 11:15:56.061769 2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 07 11:15:56.065433 ip-10-0-142-112 kubenswrapper[2578]: W0707 11:15:56.061772 2578 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 07 11:15:56.065433 ip-10-0-142-112 kubenswrapper[2578]: W0707 11:15:56.061775 2578 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 07 11:15:56.065433 ip-10-0-142-112 kubenswrapper[2578]: W0707 11:15:56.061777 2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 07 11:15:56.065886 ip-10-0-142-112 kubenswrapper[2578]: W0707 11:15:56.061780 2578 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 07 11:15:56.065886 ip-10-0-142-112 kubenswrapper[2578]: W0707 11:15:56.061782 2578 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 07 11:15:56.065886 ip-10-0-142-112 kubenswrapper[2578]: W0707 11:15:56.061785 2578 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 07 11:15:56.065886 ip-10-0-142-112 kubenswrapper[2578]: W0707 11:15:56.061788 2578 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 07 11:15:56.065886 ip-10-0-142-112 kubenswrapper[2578]: W0707 11:15:56.061791 2578 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 07 11:15:56.065886 ip-10-0-142-112 kubenswrapper[2578]: W0707 11:15:56.061794 2578 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 07 11:15:56.065886 ip-10-0-142-112 kubenswrapper[2578]: W0707 11:15:56.061797 2578 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 07 11:15:56.065886 ip-10-0-142-112 kubenswrapper[2578]: W0707 11:15:56.061800 2578 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 07 11:15:56.065886 ip-10-0-142-112 kubenswrapper[2578]: W0707 11:15:56.061802 2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 07 11:15:56.065886 ip-10-0-142-112 kubenswrapper[2578]: W0707 11:15:56.061805 2578 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 07 11:15:56.065886 ip-10-0-142-112 kubenswrapper[2578]: W0707 11:15:56.061808 2578 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 07 11:15:56.065886 ip-10-0-142-112 kubenswrapper[2578]: W0707 11:15:56.061810 2578 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 07 11:15:56.065886 ip-10-0-142-112 kubenswrapper[2578]: W0707 11:15:56.061815 2578 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 07 11:15:56.065886 ip-10-0-142-112 kubenswrapper[2578]: W0707 11:15:56.061818 2578 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 07 11:15:56.065886 ip-10-0-142-112 kubenswrapper[2578]: W0707 11:15:56.061821 2578 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 07 11:15:56.066338 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:15:56.061826 2578 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 07 11:15:56.066338 ip-10-0-142-112 kubenswrapper[2578]: W0707 11:15:56.061935 2578 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 07 11:15:56.066338 ip-10-0-142-112 kubenswrapper[2578]: W0707 11:15:56.061939 2578 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 07 11:15:56.066338 ip-10-0-142-112 kubenswrapper[2578]: W0707 11:15:56.061942 2578 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 07 11:15:56.066338 ip-10-0-142-112 kubenswrapper[2578]: W0707 11:15:56.061945 2578 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 07 11:15:56.066338 ip-10-0-142-112 kubenswrapper[2578]: W0707 11:15:56.061948 2578 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 07 11:15:56.066338 ip-10-0-142-112 kubenswrapper[2578]: W0707 11:15:56.061951 2578 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 07 11:15:56.066338 ip-10-0-142-112 kubenswrapper[2578]: W0707 11:15:56.061954 2578 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 07 11:15:56.066338 ip-10-0-142-112 kubenswrapper[2578]: W0707 11:15:56.061957 2578 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 07 11:15:56.066338 ip-10-0-142-112 kubenswrapper[2578]: W0707 11:15:56.061959 2578 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 07 11:15:56.066338 ip-10-0-142-112 kubenswrapper[2578]: W0707 11:15:56.061963 2578 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 07 11:15:56.066338 ip-10-0-142-112 kubenswrapper[2578]: W0707 11:15:56.061966 2578 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 07 11:15:56.066338 ip-10-0-142-112 kubenswrapper[2578]: W0707 11:15:56.061969 2578 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 07 11:15:56.066338 ip-10-0-142-112 kubenswrapper[2578]: W0707 11:15:56.061973 2578 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 07 11:15:56.066711 ip-10-0-142-112 kubenswrapper[2578]: W0707 11:15:56.061977 2578 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 07 11:15:56.066711 ip-10-0-142-112 kubenswrapper[2578]: W0707 11:15:56.061980 2578 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 07 11:15:56.066711 ip-10-0-142-112 kubenswrapper[2578]: W0707 11:15:56.061983 2578 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 07 11:15:56.066711 ip-10-0-142-112 kubenswrapper[2578]: W0707 11:15:56.061985 2578 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 07 11:15:56.066711 ip-10-0-142-112 kubenswrapper[2578]: W0707 11:15:56.061989 2578 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 07 11:15:56.066711 ip-10-0-142-112 kubenswrapper[2578]: W0707 11:15:56.061991 2578 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 07 11:15:56.066711 ip-10-0-142-112 kubenswrapper[2578]: W0707 11:15:56.061994 2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 07 11:15:56.066711 ip-10-0-142-112 kubenswrapper[2578]: W0707 11:15:56.061997 2578 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 07 11:15:56.066711 ip-10-0-142-112 kubenswrapper[2578]: W0707 11:15:56.062000 2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 07 11:15:56.066711 ip-10-0-142-112 kubenswrapper[2578]: W0707 11:15:56.062003 2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 07 11:15:56.066711 ip-10-0-142-112 kubenswrapper[2578]: W0707 11:15:56.062006 2578 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 07 11:15:56.066711 ip-10-0-142-112 kubenswrapper[2578]: W0707 11:15:56.062008 2578 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 07 11:15:56.066711 ip-10-0-142-112 kubenswrapper[2578]: W0707 11:15:56.062011 2578 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 07 11:15:56.066711 ip-10-0-142-112 kubenswrapper[2578]: W0707 11:15:56.062014 2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 07 11:15:56.066711 ip-10-0-142-112 kubenswrapper[2578]: W0707 11:15:56.062017 2578 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 07 11:15:56.066711 ip-10-0-142-112 kubenswrapper[2578]: W0707 11:15:56.062020 2578 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 07 11:15:56.066711 ip-10-0-142-112 kubenswrapper[2578]: W0707 11:15:56.062022 2578 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 07 11:15:56.066711 ip-10-0-142-112 kubenswrapper[2578]: W0707 11:15:56.062025 2578 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 07 11:15:56.066711 ip-10-0-142-112 kubenswrapper[2578]: W0707 11:15:56.062028 2578 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 07 11:15:56.066711 ip-10-0-142-112 kubenswrapper[2578]: W0707 11:15:56.062030 2578 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 07 11:15:56.067197 ip-10-0-142-112 kubenswrapper[2578]: W0707 11:15:56.062033 2578 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 07 11:15:56.067197 ip-10-0-142-112 kubenswrapper[2578]: W0707 11:15:56.062035 2578 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 07 11:15:56.067197 ip-10-0-142-112 kubenswrapper[2578]: W0707 11:15:56.062038 2578 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 07 11:15:56.067197 ip-10-0-142-112 kubenswrapper[2578]: W0707 11:15:56.062040 2578 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 07 11:15:56.067197 ip-10-0-142-112 kubenswrapper[2578]: W0707 11:15:56.062043 2578 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 07 11:15:56.067197 ip-10-0-142-112 kubenswrapper[2578]: W0707 11:15:56.062046 2578 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 07 11:15:56.067197 ip-10-0-142-112 kubenswrapper[2578]: W0707 11:15:56.062048 2578 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 07 11:15:56.067197 ip-10-0-142-112 kubenswrapper[2578]: W0707 11:15:56.062051 2578 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 07 11:15:56.067197 ip-10-0-142-112 kubenswrapper[2578]: W0707 11:15:56.062053 2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 07 11:15:56.067197 ip-10-0-142-112 kubenswrapper[2578]: W0707 11:15:56.062056 2578 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 07 11:15:56.067197 ip-10-0-142-112 kubenswrapper[2578]: W0707 11:15:56.062058 2578 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 07 11:15:56.067197 ip-10-0-142-112 kubenswrapper[2578]: W0707 11:15:56.062061 2578 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 07 11:15:56.067197 ip-10-0-142-112 kubenswrapper[2578]: W0707 11:15:56.062064 2578 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 07 11:15:56.067197 ip-10-0-142-112 kubenswrapper[2578]: W0707 11:15:56.062066 2578 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 07 11:15:56.067197 ip-10-0-142-112 kubenswrapper[2578]: W0707 11:15:56.062069 2578 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 07 11:15:56.067197 ip-10-0-142-112 kubenswrapper[2578]: W0707 11:15:56.062072 2578 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 07 11:15:56.067197 ip-10-0-142-112 kubenswrapper[2578]: W0707 11:15:56.062074 2578 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 07 11:15:56.067197 ip-10-0-142-112 kubenswrapper[2578]: W0707 11:15:56.062077 2578 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 07 11:15:56.067197 ip-10-0-142-112 kubenswrapper[2578]: W0707 11:15:56.062079 2578 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 07 11:15:56.067197 ip-10-0-142-112 kubenswrapper[2578]: W0707 11:15:56.062082 2578 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 07 11:15:56.067743 ip-10-0-142-112 kubenswrapper[2578]: W0707 11:15:56.062084 2578 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 07 11:15:56.067743 ip-10-0-142-112 kubenswrapper[2578]: W0707 11:15:56.062088 2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 07 11:15:56.067743 ip-10-0-142-112 kubenswrapper[2578]: W0707 11:15:56.062090 2578 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 07 11:15:56.067743 ip-10-0-142-112 kubenswrapper[2578]: W0707 11:15:56.062093 2578 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 07 11:15:56.067743 ip-10-0-142-112 kubenswrapper[2578]: W0707 11:15:56.062097 2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 07 11:15:56.067743 ip-10-0-142-112 kubenswrapper[2578]: W0707 11:15:56.062099 2578 feature_gate.go:328] unrecognized feature gate: Example Jul 07 11:15:56.067743 ip-10-0-142-112 kubenswrapper[2578]: W0707 11:15:56.062102 2578 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 07 11:15:56.067743 ip-10-0-142-112 kubenswrapper[2578]: W0707 11:15:56.062105 2578 feature_gate.go:328] unrecognized feature gate: Example2 Jul 07 11:15:56.067743 ip-10-0-142-112 kubenswrapper[2578]: W0707 11:15:56.062107 2578 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 07 11:15:56.067743 ip-10-0-142-112 kubenswrapper[2578]: W0707 11:15:56.062110 2578 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 07 11:15:56.067743 ip-10-0-142-112 kubenswrapper[2578]: W0707 11:15:56.062112 2578 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 07 11:15:56.067743 ip-10-0-142-112 kubenswrapper[2578]: W0707 11:15:56.062115 2578 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 07 11:15:56.067743 ip-10-0-142-112 kubenswrapper[2578]: W0707 11:15:56.062118 2578 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 07 11:15:56.067743 ip-10-0-142-112 kubenswrapper[2578]: W0707 11:15:56.062120 2578 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 07 11:15:56.067743 ip-10-0-142-112 kubenswrapper[2578]: W0707 11:15:56.062123 2578 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 07 11:15:56.067743 ip-10-0-142-112 kubenswrapper[2578]: W0707 11:15:56.062125 2578 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 07 11:15:56.067743 ip-10-0-142-112 kubenswrapper[2578]: W0707 11:15:56.062128 2578 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 07 11:15:56.067743 ip-10-0-142-112 kubenswrapper[2578]: W0707 11:15:56.062130 2578 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 07 11:15:56.067743 ip-10-0-142-112 kubenswrapper[2578]: W0707 11:15:56.062133 2578 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 07 11:15:56.067743 ip-10-0-142-112 kubenswrapper[2578]: W0707 11:15:56.062135 2578 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 07 11:15:56.068212 ip-10-0-142-112 kubenswrapper[2578]: W0707 11:15:56.062138 2578 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 07 11:15:56.068212 ip-10-0-142-112 kubenswrapper[2578]: W0707 11:15:56.062141 2578 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 07 11:15:56.068212 ip-10-0-142-112 kubenswrapper[2578]: W0707 11:15:56.062143 2578 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 07 11:15:56.068212 ip-10-0-142-112 kubenswrapper[2578]: W0707 11:15:56.062146 2578 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 07 11:15:56.068212 ip-10-0-142-112 kubenswrapper[2578]: W0707 11:15:56.062149 2578 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 07 11:15:56.068212 ip-10-0-142-112 kubenswrapper[2578]: W0707 11:15:56.062152 2578 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 07 11:15:56.068212 ip-10-0-142-112 kubenswrapper[2578]: W0707 11:15:56.062154 2578 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 07 11:15:56.068212 ip-10-0-142-112 kubenswrapper[2578]: W0707 11:15:56.062157 2578 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 07 11:15:56.068212 ip-10-0-142-112 kubenswrapper[2578]: W0707 11:15:56.062161 2578 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 07 11:15:56.068212 ip-10-0-142-112 kubenswrapper[2578]: W0707 11:15:56.062164 2578 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 07 11:15:56.068212 ip-10-0-142-112 kubenswrapper[2578]: W0707 11:15:56.062166 2578 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 07 11:15:56.068212 ip-10-0-142-112 kubenswrapper[2578]: W0707 11:15:56.062169 2578 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 07 11:15:56.068212 ip-10-0-142-112 kubenswrapper[2578]: W0707 11:15:56.062172 2578 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 07 11:15:56.068212 ip-10-0-142-112 kubenswrapper[2578]: W0707 11:15:56.062175 2578 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 07 11:15:56.068212 ip-10-0-142-112 kubenswrapper[2578]: W0707 11:15:56.062178 2578 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 07 11:15:56.068212 ip-10-0-142-112 kubenswrapper[2578]: W0707 11:15:56.062180 2578 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 07 11:15:56.068212 ip-10-0-142-112 kubenswrapper[2578]: W0707 11:15:56.062184 2578 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 07 11:15:56.068212 ip-10-0-142-112 kubenswrapper[2578]: W0707 11:15:56.062187 2578 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 07 11:15:56.068212 ip-10-0-142-112 kubenswrapper[2578]: W0707 11:15:56.062189 2578 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 07 11:15:56.068678 ip-10-0-142-112 kubenswrapper[2578]: W0707 11:15:56.062192 2578 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 07 11:15:56.068678 ip-10-0-142-112 kubenswrapper[2578]: W0707 11:15:56.062194 2578 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 07 11:15:56.068678 ip-10-0-142-112 kubenswrapper[2578]: W0707 11:15:56.062197 2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 07 11:15:56.068678 ip-10-0-142-112 kubenswrapper[2578]: W0707 11:15:56.062200 2578 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 07 11:15:56.068678 ip-10-0-142-112 kubenswrapper[2578]: W0707 11:15:56.062203 2578 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 07 11:15:56.068678 ip-10-0-142-112 kubenswrapper[2578]: W0707 11:15:56.062205 2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 07 11:15:56.068678 ip-10-0-142-112 kubenswrapper[2578]: W0707 11:15:56.062208 2578 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 07 11:15:56.068678 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:15:56.062213 2578 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 07 11:15:56.068678 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:15:56.062319 2578 server.go:962] "Client rotation is on, will bootstrap in background" Jul 07 11:15:56.068678 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:15:56.064474 2578 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir" Jul 07 11:15:56.068678 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:15:56.066021 2578 server.go:1019] "Starting client certificate rotation" Jul 07 11:15:56.068678 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:15:56.066122 2578 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet" Jul 07 11:15:56.068678 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:15:56.066161 2578 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet" Jul 07 11:15:56.069651 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:15:56.069633 2578 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Jul 07 11:15:56.070858 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:15:56.070834 2578 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Jul 07 11:15:56.085156 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:15:56.085137 2578 log.go:25] "Validated CRI v1 runtime API" Jul 07 11:15:56.087214 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:15:56.087199 2578 log.go:25] "Validated CRI v1 image API" Jul 07 11:15:56.088044 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:15:56.088029 2578 server.go:1429] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd" Jul 07 11:15:56.092558 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:15:56.092540 2578 fs.go:135] Filesystem UUIDs: map[1afb72ac-db05-411e-8ce7-e81e73628468:/dev/nvme0n1p3 7B77-95E7:/dev/nvme0n1p2 859f7864-5d0e-42b8-9b6e-566176e080a3:/dev/nvme0n1p4] Jul 07 11:15:56.092632 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:15:56.092558 2578 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}] Jul 07 11:15:56.094801 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:15:56.094783 2578 reflector.go:439] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Jul 07 11:15:56.097819 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:15:56.097704 2578 manager.go:216] Machine: {Timestamp:2026-07-07 11:15:56.09624353 +0000 UTC m=+0.392393110 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3099941 MemoryCapacity:33164492800 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec2ab458a2aeeafa2d2542aa4915863c SystemUUID:ec2ab458-a2ae-eafa-2d25-42aa4915863c BootID:09625dfb-2e50-4ab8-88dc-ea6032a9025a Filesystems:[{Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16582246400 Type:vfs Inodes:4048400 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6632898560 Type:vfs Inodes:819200 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6107136 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16582246400 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:d5:35:3f:f0:7f Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:d5:35:3f:f0:7f Speed:0 Mtu:9001} {Name:ovs-system MacAddress:5e:40:dc:50:29:b2 Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:33164492800 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:37486592 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None} Jul 07 11:15:56.097819 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:15:56.097814 2578 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available. Jul 07 11:15:56.097957 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:15:56.097894 2578 manager.go:232] Version: {KernelVersion:5.14.0-570.125.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260630-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:} Jul 07 11:15:56.101380 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:15:56.101355 2578 container_manager_linux.go:273] "Container manager verified user specified cgroup-root exists" cgroupRoot=[] Jul 07 11:15:56.101566 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:15:56.101382 2578 container_manager_linux.go:278] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-142-112.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2} Jul 07 11:15:56.102897 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:15:56.102885 2578 topology_manager.go:138] "Creating topology manager with none policy" Jul 07 11:15:56.102962 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:15:56.102901 2578 container_manager_linux.go:309] "Creating device plugin manager" Jul 07 11:15:56.102962 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:15:56.102914 2578 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock" Jul 07 11:15:56.104049 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:15:56.104036 2578 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock" Jul 07 11:15:56.105612 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:15:56.105602 2578 container_manager_linux.go:318] "Creating Dynamic Resource Allocation (DRA) manager" Jul 07 11:15:56.109574 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:15:56.109561 2578 state_mem.go:36] "Initialized new in-memory state store" Jul 07 11:15:56.109702 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:15:56.109691 2578 server.go:1240] "Using root directory" path="/var/lib/kubelet" Jul 07 11:15:56.109755 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:15:56.109728 2578 kubelet.go:486] "Attempting to sync node with API server" Jul 07 11:15:56.109755 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:15:56.109743 2578 kubelet.go:387] "Adding static pod path" path="/etc/kubernetes/manifests" Jul 07 11:15:56.109853 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:15:56.109761 2578 file.go:69] "Watching path" path="/etc/kubernetes/manifests" Jul 07 11:15:56.109853 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:15:56.109777 2578 kubelet.go:398] "Adding apiserver pod source" Jul 07 11:15:56.109853 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:15:56.109789 2578 apiserver.go:42] "Waiting for node sync before watching apiserver pods" Jul 07 11:15:56.110268 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:15:56.110254 2578 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Jul 07 11:15:56.110268 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:15:56.110277 2578 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Jul 07 11:15:56.112740 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:15:56.112721 2578 kuberuntime_manager.go:301] "Container runtime initialized" containerRuntime="cri-o" version="1.34.8-3.rhaos4.21.git91b4b0b.el9" apiVersion="v1" Jul 07 11:15:56.115029 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:15:56.115014 2578 kubelet.go:960] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled" Jul 07 11:15:56.115087 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:15:56.115051 2578 kubelet.go:984] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled" Jul 07 11:15:56.116166 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:15:56.116154 2578 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume" Jul 07 11:15:56.116203 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:15:56.116172 2578 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir" Jul 07 11:15:56.116203 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:15:56.116178 2578 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo" Jul 07 11:15:56.116203 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:15:56.116184 2578 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path" Jul 07 11:15:56.116203 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:15:56.116192 2578 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs" Jul 07 11:15:56.116203 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:15:56.116197 2578 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret" Jul 07 11:15:56.116203 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:15:56.116204 2578 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi" Jul 07 11:15:56.116367 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:15:56.116210 2578 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api" Jul 07 11:15:56.116367 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:15:56.116216 2578 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc" Jul 07 11:15:56.116367 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:15:56.116222 2578 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap" Jul 07 11:15:56.116367 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:15:56.116230 2578 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected" Jul 07 11:15:56.116367 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:15:56.116236 2578 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume" Jul 07 11:15:56.117442 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:15:56.117416 2578 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi" Jul 07 11:15:56.117478 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:15:56.117444 2578 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image" Jul 07 11:15:56.118616 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:15:56.118598 2578 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-7bbm6" Jul 07 11:15:56.119985 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:15:56.119968 2578 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-142-112.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope Jul 07 11:15:56.120063 ip-10-0-142-112 kubenswrapper[2578]: E0707 11:15:56.119979 2578 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service" Jul 07 11:15:56.120063 ip-10-0-142-112 kubenswrapper[2578]: E0707 11:15:56.119964 2578 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-142-112.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node" Jul 07 11:15:56.121101 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:15:56.121069 2578 server.go:1268] "Started kubelet" Jul 07 11:15:56.121202 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:15:56.121164 2578 server.go:180] "Starting to listen" address="0.0.0.0" port=10250 Jul 07 11:15:56.121244 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:15:56.121209 2578 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10 Jul 07 11:15:56.121280 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:15:56.121268 2578 server_v1.go:49] "podresources" method="list" useActivePods=true Jul 07 11:15:56.125611 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:15:56.125593 2578 server.go:249] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock" Jul 07 11:15:56.126237 ip-10-0-142-112 systemd[1]: Started Kubernetes Kubelet. Jul 07 11:15:56.126812 ip-10-0-142-112 kubenswrapper[2578]: E0707 11:15:56.126590 2578 kubelet.go:1638] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache" Jul 07 11:15:56.127435 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:15:56.127093 2578 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-7bbm6" Jul 07 11:15:56.128616 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:15:56.128600 2578 server.go:310] "Adding debug handlers to kubelet server" Jul 07 11:15:56.131521 ip-10-0-142-112 kubenswrapper[2578]: E0707 11:15:56.130536 2578 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-142-112.ec2.internal.18bffdbac7dee7df default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-142-112.ec2.internal,UID:ip-10-0-142-112.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-142-112.ec2.internal,},FirstTimestamp:2026-07-07 11:15:56.121024479 +0000 UTC m=+0.417174057,LastTimestamp:2026-07-07 11:15:56.121024479 +0000 UTC m=+0.417174057,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-142-112.ec2.internal,}" Jul 07 11:15:56.133504 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:15:56.133483 2578 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer" Jul 07 11:15:56.133504 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:15:56.133497 2578 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving" Jul 07 11:15:56.133658 ip-10-0-142-112 kubenswrapper[2578]: E0707 11:15:56.133594 2578 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-142-112.ec2.internal\" not found" Jul 07 11:15:56.134276 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:15:56.134253 2578 desired_state_of_world_populator.go:146] "Desired state populator starts to run" Jul 07 11:15:56.134402 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:15:56.134387 2578 volume_manager.go:311] "The desired_state_of_world populator starts" Jul 07 11:15:56.134402 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:15:56.134404 2578 volume_manager.go:313] "Starting Kubelet Volume Manager" Jul 07 11:15:56.134544 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:15:56.134500 2578 reconstruct.go:97] "Volume reconstruction finished" Jul 07 11:15:56.134544 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:15:56.134510 2578 reconciler.go:29] "Reconciler: start to sync state" Jul 07 11:15:56.136809 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:15:56.136788 2578 factory.go:55] Registering systemd factory Jul 07 11:15:56.136809 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:15:56.136807 2578 factory.go:223] Registration of the systemd container factory successfully Jul 07 11:15:56.137028 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:15:56.137014 2578 factory.go:153] Registering CRI-O factory Jul 07 11:15:56.137064 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:15:56.137034 2578 factory.go:223] Registration of the crio container factory successfully Jul 07 11:15:56.137107 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:15:56.137093 2578 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory Jul 07 11:15:56.137148 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:15:56.137136 2578 factory.go:103] Registering Raw factory Jul 07 11:15:56.137180 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:15:56.137160 2578 manager.go:1196] Started watching for new ooms in manager Jul 07 11:15:56.137878 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:15:56.137864 2578 manager.go:349] Starting recovery of all containers Jul 07 11:15:56.144203 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:15:56.144181 2578 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160" Jul 07 11:15:56.147987 ip-10-0-142-112 kubenswrapper[2578]: E0707 11:15:56.147875 2578 nodelease.go:49] "Failed to get node when trying to set owner ref to the node lease" err="nodes \"ip-10-0-142-112.ec2.internal\" not found" node="ip-10-0-142-112.ec2.internal" Jul 07 11:15:56.147987 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:15:56.147919 2578 manager.go:354] Recovery completed Jul 07 11:15:56.153286 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:15:56.153275 2578 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 07 11:15:56.154919 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:15:56.154906 2578 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-112.ec2.internal" event="NodeHasSufficientMemory" Jul 07 11:15:56.154972 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:15:56.154933 2578 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-112.ec2.internal" event="NodeHasNoDiskPressure" Jul 07 11:15:56.154972 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:15:56.154944 2578 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-112.ec2.internal" event="NodeHasSufficientPID" Jul 07 11:15:56.155449 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:15:56.155437 2578 cpu_manager.go:222] "Starting CPU manager" policy="none" Jul 07 11:15:56.155490 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:15:56.155447 2578 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 07 11:15:56.155525 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:15:56.155448 2578 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s" Jul 07 11:15:56.155525 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:15:56.155509 2578 state_mem.go:36] "Initialized new in-memory state store" Jul 07 11:15:56.156715 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:15:56.156698 2578 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-112.ec2.internal" event="NodeHasSufficientMemory" Jul 07 11:15:56.156751 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:15:56.156734 2578 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-112.ec2.internal" event="NodeHasNoDiskPressure" Jul 07 11:15:56.156751 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:15:56.156748 2578 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-112.ec2.internal" event="NodeHasSufficientPID" Jul 07 11:15:56.157020 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:15:56.157007 2578 policy_none.go:49] "None policy: Start" Jul 07 11:15:56.157051 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:15:56.157023 2578 memory_manager.go:187] "Starting memorymanager" policy="None" Jul 07 11:15:56.157051 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:15:56.157034 2578 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint" Jul 07 11:15:56.158326 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:15:56.158316 2578 policy_none.go:47] "Start" Jul 07 11:15:56.188108 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:15:56.188085 2578 manager.go:337] "Starting Device Plugin manager" Jul 07 11:15:56.188244 ip-10-0-142-112 kubenswrapper[2578]: E0707 11:15:56.188120 2578 manager.go:513] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint" Jul 07 11:15:56.188244 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:15:56.188132 2578 server.go:87] "Starting device plugin registration server" Jul 07 11:15:56.188414 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:15:56.188401 2578 eviction_manager.go:189] "Eviction manager: starting control loop" Jul 07 11:15:56.188466 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:15:56.188432 2578 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s" Jul 07 11:15:56.188582 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:15:56.188559 2578 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry" Jul 07 11:15:56.188689 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:15:56.188648 2578 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts" Jul 07 11:15:56.188689 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:15:56.188657 2578 plugin_manager.go:118] "Starting Kubelet Plugin Manager" Jul 07 11:15:56.189183 ip-10-0-142-112 kubenswrapper[2578]: E0707 11:15:56.189164 2578 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\"" Jul 07 11:15:56.189248 ip-10-0-142-112 kubenswrapper[2578]: E0707 11:15:56.189212 2578 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-142-112.ec2.internal\" not found" Jul 07 11:15:56.288931 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:15:56.288896 2578 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 07 11:15:56.290352 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:15:56.290048 2578 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-112.ec2.internal" event="NodeHasSufficientMemory" Jul 07 11:15:56.290352 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:15:56.290085 2578 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-112.ec2.internal" event="NodeHasNoDiskPressure" Jul 07 11:15:56.290352 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:15:56.290097 2578 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-112.ec2.internal" event="NodeHasSufficientPID" Jul 07 11:15:56.290352 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:15:56.290126 2578 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-142-112.ec2.internal" Jul 07 11:15:56.298697 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:15:56.298680 2578 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-142-112.ec2.internal" Jul 07 11:15:56.298799 ip-10-0-142-112 kubenswrapper[2578]: E0707 11:15:56.298703 2578 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-142-112.ec2.internal\": node \"ip-10-0-142-112.ec2.internal\" not found" Jul 07 11:15:56.299736 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:15:56.299709 2578 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4" Jul 07 11:15:56.300945 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:15:56.300924 2578 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6" Jul 07 11:15:56.301045 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:15:56.300957 2578 status_manager.go:244] "Starting to sync pod status with apiserver" Jul 07 11:15:56.301045 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:15:56.300977 2578 kubelet.go:2448] "Starting kubelet main sync loop" Jul 07 11:15:56.301045 ip-10-0-142-112 kubenswrapper[2578]: E0707 11:15:56.301004 2578 kubelet.go:2472] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful" Jul 07 11:15:56.302901 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:15:56.302884 2578 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Jul 07 11:15:56.346076 ip-10-0-142-112 kubenswrapper[2578]: E0707 11:15:56.346053 2578 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-142-112.ec2.internal\" not found" Jul 07 11:15:56.401100 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:15:56.401064 2578 kubelet.go:2535] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-112.ec2.internal","kube-system/kube-apiserver-proxy-ip-10-0-142-112.ec2.internal"] Jul 07 11:15:56.401224 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:15:56.401158 2578 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 07 11:15:56.402758 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:15:56.402742 2578 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-112.ec2.internal" event="NodeHasSufficientMemory" Jul 07 11:15:56.402843 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:15:56.402773 2578 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-112.ec2.internal" event="NodeHasNoDiskPressure" Jul 07 11:15:56.402843 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:15:56.402783 2578 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-112.ec2.internal" event="NodeHasSufficientPID" Jul 07 11:15:56.404170 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:15:56.404158 2578 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 07 11:15:56.404302 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:15:56.404287 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-112.ec2.internal" Jul 07 11:15:56.404342 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:15:56.404321 2578 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 07 11:15:56.404887 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:15:56.404869 2578 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-112.ec2.internal" event="NodeHasSufficientMemory" Jul 07 11:15:56.404970 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:15:56.404894 2578 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-112.ec2.internal" event="NodeHasNoDiskPressure" Jul 07 11:15:56.404970 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:15:56.404903 2578 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-112.ec2.internal" event="NodeHasSufficientPID" Jul 07 11:15:56.404970 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:15:56.404873 2578 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-112.ec2.internal" event="NodeHasSufficientMemory" Jul 07 11:15:56.404970 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:15:56.404956 2578 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-112.ec2.internal" event="NodeHasNoDiskPressure" Jul 07 11:15:56.404970 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:15:56.404965 2578 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-112.ec2.internal" event="NodeHasSufficientPID" Jul 07 11:15:56.406152 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:15:56.406137 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-142-112.ec2.internal" Jul 07 11:15:56.406227 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:15:56.406163 2578 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 07 11:15:56.406803 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:15:56.406783 2578 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-112.ec2.internal" event="NodeHasSufficientMemory" Jul 07 11:15:56.406803 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:15:56.406802 2578 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-112.ec2.internal" event="NodeHasNoDiskPressure" Jul 07 11:15:56.406915 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:15:56.406811 2578 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-112.ec2.internal" event="NodeHasSufficientPID" Jul 07 11:15:56.434029 ip-10-0-142-112 kubenswrapper[2578]: E0707 11:15:56.434003 2578 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-142-112.ec2.internal\" not found" node="ip-10-0-142-112.ec2.internal" Jul 07 11:15:56.434734 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:15:56.434718 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/8f3aa81f595d6303175e12307ce9a955-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-142-112.ec2.internal\" (UID: \"8f3aa81f595d6303175e12307ce9a955\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-112.ec2.internal" Jul 07 11:15:56.434786 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:15:56.434742 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/8f3aa81f595d6303175e12307ce9a955-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-142-112.ec2.internal\" (UID: \"8f3aa81f595d6303175e12307ce9a955\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-112.ec2.internal" Jul 07 11:15:56.434786 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:15:56.434765 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/48c63b94b1c5e4397a6ecdc3e9151f62-config\") pod \"kube-apiserver-proxy-ip-10-0-142-112.ec2.internal\" (UID: \"48c63b94b1c5e4397a6ecdc3e9151f62\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-142-112.ec2.internal" Jul 07 11:15:56.438307 ip-10-0-142-112 kubenswrapper[2578]: E0707 11:15:56.438292 2578 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-142-112.ec2.internal\" not found" node="ip-10-0-142-112.ec2.internal" Jul 07 11:15:56.446351 ip-10-0-142-112 kubenswrapper[2578]: E0707 11:15:56.446335 2578 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-142-112.ec2.internal\" not found" Jul 07 11:15:56.534904 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:15:56.534866 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/48c63b94b1c5e4397a6ecdc3e9151f62-config\") pod \"kube-apiserver-proxy-ip-10-0-142-112.ec2.internal\" (UID: \"48c63b94b1c5e4397a6ecdc3e9151f62\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-142-112.ec2.internal" Jul 07 11:15:56.534904 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:15:56.534903 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/8f3aa81f595d6303175e12307ce9a955-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-142-112.ec2.internal\" (UID: \"8f3aa81f595d6303175e12307ce9a955\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-112.ec2.internal" Jul 07 11:15:56.535111 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:15:56.534918 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/8f3aa81f595d6303175e12307ce9a955-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-142-112.ec2.internal\" (UID: \"8f3aa81f595d6303175e12307ce9a955\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-112.ec2.internal" Jul 07 11:15:56.535111 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:15:56.534974 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/48c63b94b1c5e4397a6ecdc3e9151f62-config\") pod \"kube-apiserver-proxy-ip-10-0-142-112.ec2.internal\" (UID: \"48c63b94b1c5e4397a6ecdc3e9151f62\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-142-112.ec2.internal" Jul 07 11:15:56.535111 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:15:56.535003 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/8f3aa81f595d6303175e12307ce9a955-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-142-112.ec2.internal\" (UID: \"8f3aa81f595d6303175e12307ce9a955\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-112.ec2.internal" Jul 07 11:15:56.535111 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:15:56.534974 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/8f3aa81f595d6303175e12307ce9a955-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-142-112.ec2.internal\" (UID: \"8f3aa81f595d6303175e12307ce9a955\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-112.ec2.internal" Jul 07 11:15:56.546985 ip-10-0-142-112 kubenswrapper[2578]: E0707 11:15:56.546945 2578 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-142-112.ec2.internal\" not found" Jul 07 11:15:56.647677 ip-10-0-142-112 kubenswrapper[2578]: E0707 11:15:56.647638 2578 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-142-112.ec2.internal\" not found" Jul 07 11:15:56.737108 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:15:56.737071 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-112.ec2.internal" Jul 07 11:15:56.743260 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:15:56.743240 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-142-112.ec2.internal" Jul 07 11:15:56.748103 ip-10-0-142-112 kubenswrapper[2578]: E0707 11:15:56.748086 2578 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-142-112.ec2.internal\" not found" Jul 07 11:15:56.848688 ip-10-0-142-112 kubenswrapper[2578]: E0707 11:15:56.848590 2578 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-142-112.ec2.internal\" not found" Jul 07 11:15:56.949141 ip-10-0-142-112 kubenswrapper[2578]: E0707 11:15:56.949111 2578 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-142-112.ec2.internal\" not found" Jul 07 11:15:57.049632 ip-10-0-142-112 kubenswrapper[2578]: E0707 11:15:57.049587 2578 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-142-112.ec2.internal\" not found" Jul 07 11:15:57.065773 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:15:57.065740 2578 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials" Jul 07 11:15:57.065954 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:15:57.065887 2578 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received" Jul 07 11:15:57.065954 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:15:57.065910 2578 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received" Jul 07 11:15:57.131630 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:15:57.131544 2578 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-07-06 11:10:56 +0000 UTC" deadline="2028-02-17 03:13:30.327805278 +0000 UTC" Jul 07 11:15:57.131630 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:15:57.131576 2578 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="14151h57m33.196232722s" Jul 07 11:15:57.150209 ip-10-0-142-112 kubenswrapper[2578]: E0707 11:15:57.150180 2578 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-142-112.ec2.internal\" not found" Jul 07 11:15:57.250812 ip-10-0-142-112 kubenswrapper[2578]: E0707 11:15:57.250774 2578 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-142-112.ec2.internal\" not found" Jul 07 11:15:57.297209 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:15:57.297190 2578 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160" Jul 07 11:15:57.300893 ip-10-0-142-112 kubenswrapper[2578]: W0707 11:15:57.300865 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8f3aa81f595d6303175e12307ce9a955.slice/crio-2c6565a53ced996b30377e55ba7957bb4550d8ec250b9da6712e06134c622c80 WatchSource:0}: Error finding container 2c6565a53ced996b30377e55ba7957bb4550d8ec250b9da6712e06134c622c80: Status 404 returned error can't find the container with id 2c6565a53ced996b30377e55ba7957bb4550d8ec250b9da6712e06134c622c80 Jul 07 11:15:57.301178 ip-10-0-142-112 kubenswrapper[2578]: W0707 11:15:57.301166 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod48c63b94b1c5e4397a6ecdc3e9151f62.slice/crio-c62300deb8bb7a7003385dc5b749c33566af036fd2232c20155e06fa7951a2fb WatchSource:0}: Error finding container c62300deb8bb7a7003385dc5b749c33566af036fd2232c20155e06fa7951a2fb: Status 404 returned error can't find the container with id c62300deb8bb7a7003385dc5b749c33566af036fd2232c20155e06fa7951a2fb Jul 07 11:15:57.307109 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:15:57.307091 2578 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 07 11:15:57.334511 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:15:57.334485 2578 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-112.ec2.internal" Jul 07 11:15:57.349518 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:15:57.349497 2578 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Jul 07 11:15:57.351408 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:15:57.351394 2578 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-142-112.ec2.internal" Jul 07 11:15:57.360042 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:15:57.360023 2578 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Jul 07 11:15:57.588724 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:15:57.588698 2578 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160" Jul 07 11:15:57.949913 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:15:57.949821 2578 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Jul 07 11:15:58.110836 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:15:58.110809 2578 apiserver.go:52] "Watching apiserver" Jul 07 11:15:58.119629 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:15:58.119439 2578 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66" Jul 07 11:15:58.122519 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:15:58.122489 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-multus/network-metrics-daemon-xhlzx","openshift-ovn-kubernetes/ovnkube-node-ndwrc","kube-system/global-pull-secret-syncer-8vbr5","kube-system/konnectivity-agent-qvc9d","kube-system/kube-apiserver-proxy-ip-10-0-142-112.ec2.internal","openshift-cluster-node-tuning-operator/tuned-ds6m5","openshift-image-registry/node-ca-rms7l","openshift-multus/multus-additional-cni-plugins-rtggm","openshift-network-diagnostics/network-check-target-tjprx","openshift-network-operator/iptables-alerter-sdhzh","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-ckhn2","openshift-dns/node-resolver-djzsf","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-112.ec2.internal","openshift-multus/multus-5ntjm"] Jul 07 11:15:58.124160 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:15:58.124138 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-rms7l" Jul 07 11:15:58.125416 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:15:58.125379 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-rtggm" Jul 07 11:15:58.127365 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:15:58.127342 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\"" Jul 07 11:15:58.127594 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:15:58.127577 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\"" Jul 07 11:15:58.127690 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:15:58.127615 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\"" Jul 07 11:15:58.127788 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:15:58.127769 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\"" Jul 07 11:15:58.127855 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:15:58.127815 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\"" Jul 07 11:15:58.127996 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:15:58.127980 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\"" Jul 07 11:15:58.128348 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:15:58.128064 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-w2frx\"" Jul 07 11:15:58.128348 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:15:58.128161 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-pbvtp\"" Jul 07 11:15:58.128348 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:15:58.128224 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\"" Jul 07 11:15:58.128348 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:15:58.128257 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\"" Jul 07 11:15:58.128348 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:15:58.128351 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-8vbr5" Jul 07 11:15:58.128661 ip-10-0-142-112 kubenswrapper[2578]: E0707 11:15:58.128440 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-8vbr5" podUID="3bd399c7-6314-4991-9984-8d2ff9677352" Jul 07 11:15:58.128661 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:15:58.128485 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-qvc9d" Jul 07 11:15:58.129635 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:15:58.129522 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-xhlzx" Jul 07 11:15:58.129635 ip-10-0-142-112 kubenswrapper[2578]: E0707 11:15:58.129587 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-xhlzx" podUID="1dc0212e-9bf8-4b06-b984-c694479a14ad" Jul 07 11:15:58.130499 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:15:58.130273 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-fpz7v\"" Jul 07 11:15:58.130499 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:15:58.130368 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\"" Jul 07 11:15:58.130709 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:15:58.130551 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\"" Jul 07 11:15:58.130829 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:15:58.130811 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-ds6m5" Jul 07 11:15:58.132551 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:15:58.132444 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-tjprx" Jul 07 11:15:58.132551 ip-10-0-142-112 kubenswrapper[2578]: E0707 11:15:58.132501 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-tjprx" podUID="5467fdde-6375-4364-ac0a-f5c6125c0807" Jul 07 11:15:58.132705 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:15:58.132660 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\"" Jul 07 11:15:58.132882 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:15:58.132865 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-2nqkf\"" Jul 07 11:15:58.132986 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:15:58.132927 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\"" Jul 07 11:15:58.133656 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:15:58.133641 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-5ntjm" Jul 07 11:15:58.133875 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:15:58.133857 2578 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving" Jul 07 11:15:58.135326 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:15:58.135289 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-ndwrc" Jul 07 11:15:58.136016 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:15:58.135998 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-tcpsg\"" Jul 07 11:15:58.136105 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:15:58.136021 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\"" Jul 07 11:15:58.137195 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:15:58.137149 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-sdhzh" Jul 07 11:15:58.137660 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:15:58.137645 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\"" Jul 07 11:15:58.137866 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:15:58.137847 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\"" Jul 07 11:15:58.138622 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:15:58.137874 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\"" Jul 07 11:15:58.138710 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:15:58.137893 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\"" Jul 07 11:15:58.138767 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:15:58.137917 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-hj7dq\"" Jul 07 11:15:58.138821 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:15:58.137952 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\"" Jul 07 11:15:58.138821 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:15:58.138072 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\"" Jul 07 11:15:58.138915 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:15:58.138209 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-ckhn2" Jul 07 11:15:58.139578 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:15:58.139563 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-djzsf" Jul 07 11:15:58.139980 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:15:58.139888 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\"" Jul 07 11:15:58.140272 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:15:58.140073 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\"" Jul 07 11:15:58.140272 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:15:58.140079 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\"" Jul 07 11:15:58.140272 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:15:58.140178 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-dh2cr\"" Jul 07 11:15:58.141017 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:15:58.140813 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\"" Jul 07 11:15:58.143065 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:15:58.143038 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\"" Jul 07 11:15:58.143164 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:15:58.143087 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/08ea46cf-197f-4fa4-b752-08df27dbdfee-tuning-conf-dir\") pod \"multus-additional-cni-plugins-rtggm\" (UID: \"08ea46cf-197f-4fa4-b752-08df27dbdfee\") " pod="openshift-multus/multus-additional-cni-plugins-rtggm" Jul 07 11:15:58.143164 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:15:58.143118 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/08ea46cf-197f-4fa4-b752-08df27dbdfee-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-rtggm\" (UID: \"08ea46cf-197f-4fa4-b752-08df27dbdfee\") " pod="openshift-multus/multus-additional-cni-plugins-rtggm" Jul 07 11:15:58.143164 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:15:58.143146 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/3bd399c7-6314-4991-9984-8d2ff9677352-original-pull-secret\") pod \"global-pull-secret-syncer-8vbr5\" (UID: \"3bd399c7-6314-4991-9984-8d2ff9677352\") " pod="kube-system/global-pull-secret-syncer-8vbr5" Jul 07 11:15:58.143350 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:15:58.143171 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/58b487ae-26a9-4f42-a945-cd1389584821-host-kubelet\") pod \"ovnkube-node-ndwrc\" (UID: \"58b487ae-26a9-4f42-a945-cd1389584821\") " pod="openshift-ovn-kubernetes/ovnkube-node-ndwrc" Jul 07 11:15:58.143350 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:15:58.143050 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\"" Jul 07 11:15:58.143350 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:15:58.143197 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/f03343fc-0662-41c1-9904-3ece548a3f42-serviceca\") pod \"node-ca-rms7l\" (UID: \"f03343fc-0662-41c1-9904-3ece548a3f42\") " pod="openshift-image-registry/node-ca-rms7l" Jul 07 11:15:58.143350 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:15:58.143221 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7tpkn\" (UniqueName: \"kubernetes.io/projected/f03343fc-0662-41c1-9904-3ece548a3f42-kube-api-access-7tpkn\") pod \"node-ca-rms7l\" (UID: \"f03343fc-0662-41c1-9904-3ece548a3f42\") " pod="openshift-image-registry/node-ca-rms7l" Jul 07 11:15:58.143350 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:15:58.143243 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/9ac335b6-28f5-4d6a-979c-02587fbd8e95-agent-certs\") pod \"konnectivity-agent-qvc9d\" (UID: \"9ac335b6-28f5-4d6a-979c-02587fbd8e95\") " pod="kube-system/konnectivity-agent-qvc9d" Jul 07 11:15:58.143350 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:15:58.143264 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/3bd399c7-6314-4991-9984-8d2ff9677352-dbus\") pod \"global-pull-secret-syncer-8vbr5\" (UID: \"3bd399c7-6314-4991-9984-8d2ff9677352\") " pod="kube-system/global-pull-secret-syncer-8vbr5" Jul 07 11:15:58.143350 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:15:58.143285 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/a60168c4-c743-44e4-9047-df74d47da598-host-run-netns\") pod \"multus-5ntjm\" (UID: \"a60168c4-c743-44e4-9047-df74d47da598\") " pod="openshift-multus/multus-5ntjm" Jul 07 11:15:58.143350 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:15:58.143318 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/58b487ae-26a9-4f42-a945-cd1389584821-host-run-netns\") pod \"ovnkube-node-ndwrc\" (UID: \"58b487ae-26a9-4f42-a945-cd1389584821\") " pod="openshift-ovn-kubernetes/ovnkube-node-ndwrc" Jul 07 11:15:58.143350 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:15:58.143347 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/f03343fc-0662-41c1-9904-3ece548a3f42-host\") pod \"node-ca-rms7l\" (UID: \"f03343fc-0662-41c1-9904-3ece548a3f42\") " pod="openshift-image-registry/node-ca-rms7l" Jul 07 11:15:58.143777 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:15:58.143384 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/08ea46cf-197f-4fa4-b752-08df27dbdfee-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-rtggm\" (UID: \"08ea46cf-197f-4fa4-b752-08df27dbdfee\") " pod="openshift-multus/multus-additional-cni-plugins-rtggm" Jul 07 11:15:58.143777 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:15:58.143413 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/58b487ae-26a9-4f42-a945-cd1389584821-env-overrides\") pod \"ovnkube-node-ndwrc\" (UID: \"58b487ae-26a9-4f42-a945-cd1389584821\") " pod="openshift-ovn-kubernetes/ovnkube-node-ndwrc" Jul 07 11:15:58.143777 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:15:58.143455 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/58b487ae-26a9-4f42-a945-cd1389584821-ovnkube-script-lib\") pod \"ovnkube-node-ndwrc\" (UID: \"58b487ae-26a9-4f42-a945-cd1389584821\") " pod="openshift-ovn-kubernetes/ovnkube-node-ndwrc" Jul 07 11:15:58.143777 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:15:58.143481 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/d009277d-d0d0-4163-9a6b-3ed9d1ecffa5-etc-sysconfig\") pod \"tuned-ds6m5\" (UID: \"d009277d-d0d0-4163-9a6b-3ed9d1ecffa5\") " pod="openshift-cluster-node-tuning-operator/tuned-ds6m5" Jul 07 11:15:58.143777 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:15:58.143502 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/d009277d-d0d0-4163-9a6b-3ed9d1ecffa5-run\") pod \"tuned-ds6m5\" (UID: \"d009277d-d0d0-4163-9a6b-3ed9d1ecffa5\") " pod="openshift-cluster-node-tuning-operator/tuned-ds6m5" Jul 07 11:15:58.143777 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:15:58.143525 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/58b487ae-26a9-4f42-a945-cd1389584821-host-cni-bin\") pod \"ovnkube-node-ndwrc\" (UID: \"58b487ae-26a9-4f42-a945-cd1389584821\") " pod="openshift-ovn-kubernetes/ovnkube-node-ndwrc" Jul 07 11:15:58.143777 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:15:58.143546 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/a60168c4-c743-44e4-9047-df74d47da598-cnibin\") pod \"multus-5ntjm\" (UID: \"a60168c4-c743-44e4-9047-df74d47da598\") " pod="openshift-multus/multus-5ntjm" Jul 07 11:15:58.143777 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:15:58.143573 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-czc2n\" (UniqueName: \"kubernetes.io/projected/08ea46cf-197f-4fa4-b752-08df27dbdfee-kube-api-access-czc2n\") pod \"multus-additional-cni-plugins-rtggm\" (UID: \"08ea46cf-197f-4fa4-b752-08df27dbdfee\") " pod="openshift-multus/multus-additional-cni-plugins-rtggm" Jul 07 11:15:58.143777 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:15:58.143594 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/d009277d-d0d0-4163-9a6b-3ed9d1ecffa5-etc-modprobe-d\") pod \"tuned-ds6m5\" (UID: \"d009277d-d0d0-4163-9a6b-3ed9d1ecffa5\") " pod="openshift-cluster-node-tuning-operator/tuned-ds6m5" Jul 07 11:15:58.143777 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:15:58.143615 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/d009277d-d0d0-4163-9a6b-3ed9d1ecffa5-etc-sysctl-d\") pod \"tuned-ds6m5\" (UID: \"d009277d-d0d0-4163-9a6b-3ed9d1ecffa5\") " pod="openshift-cluster-node-tuning-operator/tuned-ds6m5" Jul 07 11:15:58.143777 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:15:58.143635 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/d009277d-d0d0-4163-9a6b-3ed9d1ecffa5-etc-sysctl-conf\") pod \"tuned-ds6m5\" (UID: \"d009277d-d0d0-4163-9a6b-3ed9d1ecffa5\") " pod="openshift-cluster-node-tuning-operator/tuned-ds6m5" Jul 07 11:15:58.143777 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:15:58.143662 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/a60168c4-c743-44e4-9047-df74d47da598-cni-binary-copy\") pod \"multus-5ntjm\" (UID: \"a60168c4-c743-44e4-9047-df74d47da598\") " pod="openshift-multus/multus-5ntjm" Jul 07 11:15:58.143777 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:15:58.143726 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/58b487ae-26a9-4f42-a945-cd1389584821-var-lib-openvswitch\") pod \"ovnkube-node-ndwrc\" (UID: \"58b487ae-26a9-4f42-a945-cd1389584821\") " pod="openshift-ovn-kubernetes/ovnkube-node-ndwrc" Jul 07 11:15:58.143777 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:15:58.143753 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/08ea46cf-197f-4fa4-b752-08df27dbdfee-os-release\") pod \"multus-additional-cni-plugins-rtggm\" (UID: \"08ea46cf-197f-4fa4-b752-08df27dbdfee\") " pod="openshift-multus/multus-additional-cni-plugins-rtggm" Jul 07 11:15:58.143777 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:15:58.143777 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/1dc0212e-9bf8-4b06-b984-c694479a14ad-metrics-certs\") pod \"network-metrics-daemon-xhlzx\" (UID: \"1dc0212e-9bf8-4b06-b984-c694479a14ad\") " pod="openshift-multus/network-metrics-daemon-xhlzx" Jul 07 11:15:58.144385 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:15:58.143799 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/d009277d-d0d0-4163-9a6b-3ed9d1ecffa5-lib-modules\") pod \"tuned-ds6m5\" (UID: \"d009277d-d0d0-4163-9a6b-3ed9d1ecffa5\") " pod="openshift-cluster-node-tuning-operator/tuned-ds6m5" Jul 07 11:15:58.144385 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:15:58.143839 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5h5r9\" (UniqueName: \"kubernetes.io/projected/5467fdde-6375-4364-ac0a-f5c6125c0807-kube-api-access-5h5r9\") pod \"network-check-target-tjprx\" (UID: \"5467fdde-6375-4364-ac0a-f5c6125c0807\") " pod="openshift-network-diagnostics/network-check-target-tjprx" Jul 07 11:15:58.144385 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:15:58.143862 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-65lxp\" (UniqueName: \"kubernetes.io/projected/58b487ae-26a9-4f42-a945-cd1389584821-kube-api-access-65lxp\") pod \"ovnkube-node-ndwrc\" (UID: \"58b487ae-26a9-4f42-a945-cd1389584821\") " pod="openshift-ovn-kubernetes/ovnkube-node-ndwrc" Jul 07 11:15:58.144385 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:15:58.143882 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/d009277d-d0d0-4163-9a6b-3ed9d1ecffa5-host\") pod \"tuned-ds6m5\" (UID: \"d009277d-d0d0-4163-9a6b-3ed9d1ecffa5\") " pod="openshift-cluster-node-tuning-operator/tuned-ds6m5" Jul 07 11:15:58.144385 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:15:58.143920 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/a60168c4-c743-44e4-9047-df74d47da598-host-var-lib-cni-multus\") pod \"multus-5ntjm\" (UID: \"a60168c4-c743-44e4-9047-df74d47da598\") " pod="openshift-multus/multus-5ntjm" Jul 07 11:15:58.144385 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:15:58.143941 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/58b487ae-26a9-4f42-a945-cd1389584821-log-socket\") pod \"ovnkube-node-ndwrc\" (UID: \"58b487ae-26a9-4f42-a945-cd1389584821\") " pod="openshift-ovn-kubernetes/ovnkube-node-ndwrc" Jul 07 11:15:58.144870 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:15:58.144849 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/a60168c4-c743-44e4-9047-df74d47da598-multus-cni-dir\") pod \"multus-5ntjm\" (UID: \"a60168c4-c743-44e4-9047-df74d47da598\") " pod="openshift-multus/multus-5ntjm" Jul 07 11:15:58.144953 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:15:58.144903 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/a60168c4-c743-44e4-9047-df74d47da598-multus-socket-dir-parent\") pod \"multus-5ntjm\" (UID: \"a60168c4-c743-44e4-9047-df74d47da598\") " pod="openshift-multus/multus-5ntjm" Jul 07 11:15:58.144953 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:15:58.144932 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/a60168c4-c743-44e4-9047-df74d47da598-host-var-lib-cni-bin\") pod \"multus-5ntjm\" (UID: \"a60168c4-c743-44e4-9047-df74d47da598\") " pod="openshift-multus/multus-5ntjm" Jul 07 11:15:58.145058 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:15:58.144964 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/58b487ae-26a9-4f42-a945-cd1389584821-host-cni-netd\") pod \"ovnkube-node-ndwrc\" (UID: \"58b487ae-26a9-4f42-a945-cd1389584821\") " pod="openshift-ovn-kubernetes/ovnkube-node-ndwrc" Jul 07 11:15:58.145058 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:15:58.145010 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/d009277d-d0d0-4163-9a6b-3ed9d1ecffa5-etc-tuned\") pod \"tuned-ds6m5\" (UID: \"d009277d-d0d0-4163-9a6b-3ed9d1ecffa5\") " pod="openshift-cluster-node-tuning-operator/tuned-ds6m5" Jul 07 11:15:58.145154 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:15:58.145055 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/a60168c4-c743-44e4-9047-df74d47da598-host-var-lib-kubelet\") pod \"multus-5ntjm\" (UID: \"a60168c4-c743-44e4-9047-df74d47da598\") " pod="openshift-multus/multus-5ntjm" Jul 07 11:15:58.145154 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:15:58.145087 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/58b487ae-26a9-4f42-a945-cd1389584821-run-systemd\") pod \"ovnkube-node-ndwrc\" (UID: \"58b487ae-26a9-4f42-a945-cd1389584821\") " pod="openshift-ovn-kubernetes/ovnkube-node-ndwrc" Jul 07 11:15:58.145154 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:15:58.145115 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/58b487ae-26a9-4f42-a945-cd1389584821-ovn-node-metrics-cert\") pod \"ovnkube-node-ndwrc\" (UID: \"58b487ae-26a9-4f42-a945-cd1389584821\") " pod="openshift-ovn-kubernetes/ovnkube-node-ndwrc" Jul 07 11:15:58.145282 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:15:58.145150 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/9ac335b6-28f5-4d6a-979c-02587fbd8e95-konnectivity-ca\") pod \"konnectivity-agent-qvc9d\" (UID: \"9ac335b6-28f5-4d6a-979c-02587fbd8e95\") " pod="kube-system/konnectivity-agent-qvc9d" Jul 07 11:15:58.145282 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:15:58.145185 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/58b487ae-26a9-4f42-a945-cd1389584821-etc-openvswitch\") pod \"ovnkube-node-ndwrc\" (UID: \"58b487ae-26a9-4f42-a945-cd1389584821\") " pod="openshift-ovn-kubernetes/ovnkube-node-ndwrc" Jul 07 11:15:58.145282 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:15:58.145215 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/58b487ae-26a9-4f42-a945-cd1389584821-host-run-ovn-kubernetes\") pod \"ovnkube-node-ndwrc\" (UID: \"58b487ae-26a9-4f42-a945-cd1389584821\") " pod="openshift-ovn-kubernetes/ovnkube-node-ndwrc" Jul 07 11:15:58.145282 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:15:58.145262 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/08ea46cf-197f-4fa4-b752-08df27dbdfee-cni-binary-copy\") pod \"multus-additional-cni-plugins-rtggm\" (UID: \"08ea46cf-197f-4fa4-b752-08df27dbdfee\") " pod="openshift-multus/multus-additional-cni-plugins-rtggm" Jul 07 11:15:58.145470 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:15:58.145288 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vnslv\" (UniqueName: \"kubernetes.io/projected/1dc0212e-9bf8-4b06-b984-c694479a14ad-kube-api-access-vnslv\") pod \"network-metrics-daemon-xhlzx\" (UID: \"1dc0212e-9bf8-4b06-b984-c694479a14ad\") " pod="openshift-multus/network-metrics-daemon-xhlzx" Jul 07 11:15:58.145470 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:15:58.145317 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/d009277d-d0d0-4163-9a6b-3ed9d1ecffa5-etc-kubernetes\") pod \"tuned-ds6m5\" (UID: \"d009277d-d0d0-4163-9a6b-3ed9d1ecffa5\") " pod="openshift-cluster-node-tuning-operator/tuned-ds6m5" Jul 07 11:15:58.145470 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:15:58.145349 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/a60168c4-c743-44e4-9047-df74d47da598-host-run-k8s-cni-cncf-io\") pod \"multus-5ntjm\" (UID: \"a60168c4-c743-44e4-9047-df74d47da598\") " pod="openshift-multus/multus-5ntjm" Jul 07 11:15:58.145470 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:15:58.145380 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/58b487ae-26a9-4f42-a945-cd1389584821-ovnkube-config\") pod \"ovnkube-node-ndwrc\" (UID: \"58b487ae-26a9-4f42-a945-cd1389584821\") " pod="openshift-ovn-kubernetes/ovnkube-node-ndwrc" Jul 07 11:15:58.145470 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:15:58.145411 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/08ea46cf-197f-4fa4-b752-08df27dbdfee-system-cni-dir\") pod \"multus-additional-cni-plugins-rtggm\" (UID: \"08ea46cf-197f-4fa4-b752-08df27dbdfee\") " pod="openshift-multus/multus-additional-cni-plugins-rtggm" Jul 07 11:15:58.145470 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:15:58.145461 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/a60168c4-c743-44e4-9047-df74d47da598-multus-conf-dir\") pod \"multus-5ntjm\" (UID: \"a60168c4-c743-44e4-9047-df74d47da598\") " pod="openshift-multus/multus-5ntjm" Jul 07 11:15:58.145722 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:15:58.145494 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/58b487ae-26a9-4f42-a945-cd1389584821-host-slash\") pod \"ovnkube-node-ndwrc\" (UID: \"58b487ae-26a9-4f42-a945-cd1389584821\") " pod="openshift-ovn-kubernetes/ovnkube-node-ndwrc" Jul 07 11:15:58.145722 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:15:58.145522 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/58b487ae-26a9-4f42-a945-cd1389584821-node-log\") pod \"ovnkube-node-ndwrc\" (UID: \"58b487ae-26a9-4f42-a945-cd1389584821\") " pod="openshift-ovn-kubernetes/ovnkube-node-ndwrc" Jul 07 11:15:58.145722 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:15:58.145556 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/08ea46cf-197f-4fa4-b752-08df27dbdfee-cnibin\") pod \"multus-additional-cni-plugins-rtggm\" (UID: \"08ea46cf-197f-4fa4-b752-08df27dbdfee\") " pod="openshift-multus/multus-additional-cni-plugins-rtggm" Jul 07 11:15:58.145722 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:15:58.145587 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/d009277d-d0d0-4163-9a6b-3ed9d1ecffa5-var-lib-kubelet\") pod \"tuned-ds6m5\" (UID: \"d009277d-d0d0-4163-9a6b-3ed9d1ecffa5\") " pod="openshift-cluster-node-tuning-operator/tuned-ds6m5" Jul 07 11:15:58.145722 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:15:58.145611 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-47t8g\" (UniqueName: \"kubernetes.io/projected/d009277d-d0d0-4163-9a6b-3ed9d1ecffa5-kube-api-access-47t8g\") pod \"tuned-ds6m5\" (UID: \"d009277d-d0d0-4163-9a6b-3ed9d1ecffa5\") " pod="openshift-cluster-node-tuning-operator/tuned-ds6m5" Jul 07 11:15:58.145722 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:15:58.145642 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/a60168c4-c743-44e4-9047-df74d47da598-multus-daemon-config\") pod \"multus-5ntjm\" (UID: \"a60168c4-c743-44e4-9047-df74d47da598\") " pod="openshift-multus/multus-5ntjm" Jul 07 11:15:58.145722 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:15:58.145672 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/a60168c4-c743-44e4-9047-df74d47da598-host-run-multus-certs\") pod \"multus-5ntjm\" (UID: \"a60168c4-c743-44e4-9047-df74d47da598\") " pod="openshift-multus/multus-5ntjm" Jul 07 11:15:58.145722 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:15:58.145679 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\"" Jul 07 11:15:58.145722 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:15:58.145704 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nr5pg\" (UniqueName: \"kubernetes.io/projected/a60168c4-c743-44e4-9047-df74d47da598-kube-api-access-nr5pg\") pod \"multus-5ntjm\" (UID: \"a60168c4-c743-44e4-9047-df74d47da598\") " pod="openshift-multus/multus-5ntjm" Jul 07 11:15:58.146128 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:15:58.145734 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/58b487ae-26a9-4f42-a945-cd1389584821-systemd-units\") pod \"ovnkube-node-ndwrc\" (UID: \"58b487ae-26a9-4f42-a945-cd1389584821\") " pod="openshift-ovn-kubernetes/ovnkube-node-ndwrc" Jul 07 11:15:58.146128 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:15:58.145745 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\"" Jul 07 11:15:58.146128 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:15:58.145767 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/58b487ae-26a9-4f42-a945-cd1389584821-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-ndwrc\" (UID: \"58b487ae-26a9-4f42-a945-cd1389584821\") " pod="openshift-ovn-kubernetes/ovnkube-node-ndwrc" Jul 07 11:15:58.146128 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:15:58.145799 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/d009277d-d0d0-4163-9a6b-3ed9d1ecffa5-sys\") pod \"tuned-ds6m5\" (UID: \"d009277d-d0d0-4163-9a6b-3ed9d1ecffa5\") " pod="openshift-cluster-node-tuning-operator/tuned-ds6m5" Jul 07 11:15:58.146128 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:15:58.145829 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/a60168c4-c743-44e4-9047-df74d47da598-os-release\") pod \"multus-5ntjm\" (UID: \"a60168c4-c743-44e4-9047-df74d47da598\") " pod="openshift-multus/multus-5ntjm" Jul 07 11:15:58.146128 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:15:58.145857 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/a60168c4-c743-44e4-9047-df74d47da598-hostroot\") pod \"multus-5ntjm\" (UID: \"a60168c4-c743-44e4-9047-df74d47da598\") " pod="openshift-multus/multus-5ntjm" Jul 07 11:15:58.146128 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:15:58.145892 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/a60168c4-c743-44e4-9047-df74d47da598-etc-kubernetes\") pod \"multus-5ntjm\" (UID: \"a60168c4-c743-44e4-9047-df74d47da598\") " pod="openshift-multus/multus-5ntjm" Jul 07 11:15:58.146128 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:15:58.145919 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/58b487ae-26a9-4f42-a945-cd1389584821-run-ovn\") pod \"ovnkube-node-ndwrc\" (UID: \"58b487ae-26a9-4f42-a945-cd1389584821\") " pod="openshift-ovn-kubernetes/ovnkube-node-ndwrc" Jul 07 11:15:58.146128 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:15:58.145944 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/3bd399c7-6314-4991-9984-8d2ff9677352-kubelet-config\") pod \"global-pull-secret-syncer-8vbr5\" (UID: \"3bd399c7-6314-4991-9984-8d2ff9677352\") " pod="kube-system/global-pull-secret-syncer-8vbr5" Jul 07 11:15:58.146128 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:15:58.146085 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/d009277d-d0d0-4163-9a6b-3ed9d1ecffa5-etc-systemd\") pod \"tuned-ds6m5\" (UID: \"d009277d-d0d0-4163-9a6b-3ed9d1ecffa5\") " pod="openshift-cluster-node-tuning-operator/tuned-ds6m5" Jul 07 11:15:58.146128 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:15:58.146112 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/d009277d-d0d0-4163-9a6b-3ed9d1ecffa5-tmp\") pod \"tuned-ds6m5\" (UID: \"d009277d-d0d0-4163-9a6b-3ed9d1ecffa5\") " pod="openshift-cluster-node-tuning-operator/tuned-ds6m5" Jul 07 11:15:58.146681 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:15:58.146142 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/a60168c4-c743-44e4-9047-df74d47da598-system-cni-dir\") pod \"multus-5ntjm\" (UID: \"a60168c4-c743-44e4-9047-df74d47da598\") " pod="openshift-multus/multus-5ntjm" Jul 07 11:15:58.146681 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:15:58.146169 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/58b487ae-26a9-4f42-a945-cd1389584821-run-openvswitch\") pod \"ovnkube-node-ndwrc\" (UID: \"58b487ae-26a9-4f42-a945-cd1389584821\") " pod="openshift-ovn-kubernetes/ovnkube-node-ndwrc" Jul 07 11:15:58.146681 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:15:58.146220 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-ljtwf\"" Jul 07 11:15:58.146834 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:15:58.146689 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-xw8q6\"" Jul 07 11:15:58.155880 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:15:58.155857 2578 reflector.go:439] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Jul 07 11:15:58.170228 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:15:58.170200 2578 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-phmzm" Jul 07 11:15:58.178040 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:15:58.178019 2578 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-phmzm" Jul 07 11:15:58.235315 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:15:58.235283 2578 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world" Jul 07 11:15:58.246703 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:15:58.246677 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/58b487ae-26a9-4f42-a945-cd1389584821-etc-openvswitch\") pod \"ovnkube-node-ndwrc\" (UID: \"58b487ae-26a9-4f42-a945-cd1389584821\") " pod="openshift-ovn-kubernetes/ovnkube-node-ndwrc" Jul 07 11:15:58.246860 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:15:58.246709 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/58b487ae-26a9-4f42-a945-cd1389584821-host-run-ovn-kubernetes\") pod \"ovnkube-node-ndwrc\" (UID: \"58b487ae-26a9-4f42-a945-cd1389584821\") " pod="openshift-ovn-kubernetes/ovnkube-node-ndwrc" Jul 07 11:15:58.246860 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:15:58.246732 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/08ea46cf-197f-4fa4-b752-08df27dbdfee-cni-binary-copy\") pod \"multus-additional-cni-plugins-rtggm\" (UID: \"08ea46cf-197f-4fa4-b752-08df27dbdfee\") " pod="openshift-multus/multus-additional-cni-plugins-rtggm" Jul 07 11:15:58.246860 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:15:58.246759 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vnslv\" (UniqueName: \"kubernetes.io/projected/1dc0212e-9bf8-4b06-b984-c694479a14ad-kube-api-access-vnslv\") pod \"network-metrics-daemon-xhlzx\" (UID: \"1dc0212e-9bf8-4b06-b984-c694479a14ad\") " pod="openshift-multus/network-metrics-daemon-xhlzx" Jul 07 11:15:58.246860 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:15:58.246786 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/d009277d-d0d0-4163-9a6b-3ed9d1ecffa5-etc-kubernetes\") pod \"tuned-ds6m5\" (UID: \"d009277d-d0d0-4163-9a6b-3ed9d1ecffa5\") " pod="openshift-cluster-node-tuning-operator/tuned-ds6m5" Jul 07 11:15:58.246860 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:15:58.246804 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/58b487ae-26a9-4f42-a945-cd1389584821-host-run-ovn-kubernetes\") pod \"ovnkube-node-ndwrc\" (UID: \"58b487ae-26a9-4f42-a945-cd1389584821\") " pod="openshift-ovn-kubernetes/ovnkube-node-ndwrc" Jul 07 11:15:58.246860 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:15:58.246812 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/6fcad21b-4b48-4fa1-9f65-92ed08e86703-socket-dir\") pod \"aws-ebs-csi-driver-node-ckhn2\" (UID: \"6fcad21b-4b48-4fa1-9f65-92ed08e86703\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-ckhn2" Jul 07 11:15:58.246860 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:15:58.246803 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/58b487ae-26a9-4f42-a945-cd1389584821-etc-openvswitch\") pod \"ovnkube-node-ndwrc\" (UID: \"58b487ae-26a9-4f42-a945-cd1389584821\") " pod="openshift-ovn-kubernetes/ovnkube-node-ndwrc" Jul 07 11:15:58.246860 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:15:58.246840 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/a60168c4-c743-44e4-9047-df74d47da598-host-run-k8s-cni-cncf-io\") pod \"multus-5ntjm\" (UID: \"a60168c4-c743-44e4-9047-df74d47da598\") " pod="openshift-multus/multus-5ntjm" Jul 07 11:15:58.246860 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:15:58.246857 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/d009277d-d0d0-4163-9a6b-3ed9d1ecffa5-etc-kubernetes\") pod \"tuned-ds6m5\" (UID: \"d009277d-d0d0-4163-9a6b-3ed9d1ecffa5\") " pod="openshift-cluster-node-tuning-operator/tuned-ds6m5" Jul 07 11:15:58.247298 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:15:58.246863 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/58b487ae-26a9-4f42-a945-cd1389584821-ovnkube-config\") pod \"ovnkube-node-ndwrc\" (UID: \"58b487ae-26a9-4f42-a945-cd1389584821\") " pod="openshift-ovn-kubernetes/ovnkube-node-ndwrc" Jul 07 11:15:58.247298 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:15:58.246888 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/a60168c4-c743-44e4-9047-df74d47da598-host-run-k8s-cni-cncf-io\") pod \"multus-5ntjm\" (UID: \"a60168c4-c743-44e4-9047-df74d47da598\") " pod="openshift-multus/multus-5ntjm" Jul 07 11:15:58.247298 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:15:58.246910 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/08ea46cf-197f-4fa4-b752-08df27dbdfee-system-cni-dir\") pod \"multus-additional-cni-plugins-rtggm\" (UID: \"08ea46cf-197f-4fa4-b752-08df27dbdfee\") " pod="openshift-multus/multus-additional-cni-plugins-rtggm" Jul 07 11:15:58.247298 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:15:58.246936 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/a60168c4-c743-44e4-9047-df74d47da598-multus-conf-dir\") pod \"multus-5ntjm\" (UID: \"a60168c4-c743-44e4-9047-df74d47da598\") " pod="openshift-multus/multus-5ntjm" Jul 07 11:15:58.247298 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:15:58.246959 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/58b487ae-26a9-4f42-a945-cd1389584821-host-slash\") pod \"ovnkube-node-ndwrc\" (UID: \"58b487ae-26a9-4f42-a945-cd1389584821\") " pod="openshift-ovn-kubernetes/ovnkube-node-ndwrc" Jul 07 11:15:58.247298 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:15:58.246980 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/08ea46cf-197f-4fa4-b752-08df27dbdfee-system-cni-dir\") pod \"multus-additional-cni-plugins-rtggm\" (UID: \"08ea46cf-197f-4fa4-b752-08df27dbdfee\") " pod="openshift-multus/multus-additional-cni-plugins-rtggm" Jul 07 11:15:58.247298 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:15:58.246985 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/58b487ae-26a9-4f42-a945-cd1389584821-node-log\") pod \"ovnkube-node-ndwrc\" (UID: \"58b487ae-26a9-4f42-a945-cd1389584821\") " pod="openshift-ovn-kubernetes/ovnkube-node-ndwrc" Jul 07 11:15:58.247298 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:15:58.247004 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/a60168c4-c743-44e4-9047-df74d47da598-multus-conf-dir\") pod \"multus-5ntjm\" (UID: \"a60168c4-c743-44e4-9047-df74d47da598\") " pod="openshift-multus/multus-5ntjm" Jul 07 11:15:58.247298 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:15:58.247027 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/08ea46cf-197f-4fa4-b752-08df27dbdfee-cnibin\") pod \"multus-additional-cni-plugins-rtggm\" (UID: \"08ea46cf-197f-4fa4-b752-08df27dbdfee\") " pod="openshift-multus/multus-additional-cni-plugins-rtggm" Jul 07 11:15:58.247298 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:15:58.247036 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/58b487ae-26a9-4f42-a945-cd1389584821-host-slash\") pod \"ovnkube-node-ndwrc\" (UID: \"58b487ae-26a9-4f42-a945-cd1389584821\") " pod="openshift-ovn-kubernetes/ovnkube-node-ndwrc" Jul 07 11:15:58.247298 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:15:58.247055 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/58b487ae-26a9-4f42-a945-cd1389584821-node-log\") pod \"ovnkube-node-ndwrc\" (UID: \"58b487ae-26a9-4f42-a945-cd1389584821\") " pod="openshift-ovn-kubernetes/ovnkube-node-ndwrc" Jul 07 11:15:58.247298 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:15:58.247059 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/d009277d-d0d0-4163-9a6b-3ed9d1ecffa5-var-lib-kubelet\") pod \"tuned-ds6m5\" (UID: \"d009277d-d0d0-4163-9a6b-3ed9d1ecffa5\") " pod="openshift-cluster-node-tuning-operator/tuned-ds6m5" Jul 07 11:15:58.247298 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:15:58.247090 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/08ea46cf-197f-4fa4-b752-08df27dbdfee-cnibin\") pod \"multus-additional-cni-plugins-rtggm\" (UID: \"08ea46cf-197f-4fa4-b752-08df27dbdfee\") " pod="openshift-multus/multus-additional-cni-plugins-rtggm" Jul 07 11:15:58.247298 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:15:58.247101 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/d009277d-d0d0-4163-9a6b-3ed9d1ecffa5-var-lib-kubelet\") pod \"tuned-ds6m5\" (UID: \"d009277d-d0d0-4163-9a6b-3ed9d1ecffa5\") " pod="openshift-cluster-node-tuning-operator/tuned-ds6m5" Jul 07 11:15:58.247298 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:15:58.247104 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-47t8g\" (UniqueName: \"kubernetes.io/projected/d009277d-d0d0-4163-9a6b-3ed9d1ecffa5-kube-api-access-47t8g\") pod \"tuned-ds6m5\" (UID: \"d009277d-d0d0-4163-9a6b-3ed9d1ecffa5\") " pod="openshift-cluster-node-tuning-operator/tuned-ds6m5" Jul 07 11:15:58.247298 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:15:58.247133 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/6fcad21b-4b48-4fa1-9f65-92ed08e86703-kubelet-dir\") pod \"aws-ebs-csi-driver-node-ckhn2\" (UID: \"6fcad21b-4b48-4fa1-9f65-92ed08e86703\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-ckhn2" Jul 07 11:15:58.247298 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:15:58.247164 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/a60168c4-c743-44e4-9047-df74d47da598-multus-daemon-config\") pod \"multus-5ntjm\" (UID: \"a60168c4-c743-44e4-9047-df74d47da598\") " pod="openshift-multus/multus-5ntjm" Jul 07 11:15:58.248099 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:15:58.247207 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/a60168c4-c743-44e4-9047-df74d47da598-host-run-multus-certs\") pod \"multus-5ntjm\" (UID: \"a60168c4-c743-44e4-9047-df74d47da598\") " pod="openshift-multus/multus-5ntjm" Jul 07 11:15:58.248099 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:15:58.247231 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nr5pg\" (UniqueName: \"kubernetes.io/projected/a60168c4-c743-44e4-9047-df74d47da598-kube-api-access-nr5pg\") pod \"multus-5ntjm\" (UID: \"a60168c4-c743-44e4-9047-df74d47da598\") " pod="openshift-multus/multus-5ntjm" Jul 07 11:15:58.248099 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:15:58.247255 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/58b487ae-26a9-4f42-a945-cd1389584821-systemd-units\") pod \"ovnkube-node-ndwrc\" (UID: \"58b487ae-26a9-4f42-a945-cd1389584821\") " pod="openshift-ovn-kubernetes/ovnkube-node-ndwrc" Jul 07 11:15:58.248099 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:15:58.247281 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/58b487ae-26a9-4f42-a945-cd1389584821-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-ndwrc\" (UID: \"58b487ae-26a9-4f42-a945-cd1389584821\") " pod="openshift-ovn-kubernetes/ovnkube-node-ndwrc" Jul 07 11:15:58.248099 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:15:58.247313 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/d009277d-d0d0-4163-9a6b-3ed9d1ecffa5-sys\") pod \"tuned-ds6m5\" (UID: \"d009277d-d0d0-4163-9a6b-3ed9d1ecffa5\") " pod="openshift-cluster-node-tuning-operator/tuned-ds6m5" Jul 07 11:15:58.248099 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:15:58.247336 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/a60168c4-c743-44e4-9047-df74d47da598-os-release\") pod \"multus-5ntjm\" (UID: \"a60168c4-c743-44e4-9047-df74d47da598\") " pod="openshift-multus/multus-5ntjm" Jul 07 11:15:58.248099 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:15:58.247359 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/a60168c4-c743-44e4-9047-df74d47da598-hostroot\") pod \"multus-5ntjm\" (UID: \"a60168c4-c743-44e4-9047-df74d47da598\") " pod="openshift-multus/multus-5ntjm" Jul 07 11:15:58.248099 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:15:58.247393 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/08ea46cf-197f-4fa4-b752-08df27dbdfee-cni-binary-copy\") pod \"multus-additional-cni-plugins-rtggm\" (UID: \"08ea46cf-197f-4fa4-b752-08df27dbdfee\") " pod="openshift-multus/multus-additional-cni-plugins-rtggm" Jul 07 11:15:58.248099 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:15:58.247448 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/a60168c4-c743-44e4-9047-df74d47da598-etc-kubernetes\") pod \"multus-5ntjm\" (UID: \"a60168c4-c743-44e4-9047-df74d47da598\") " pod="openshift-multus/multus-5ntjm" Jul 07 11:15:58.248099 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:15:58.247396 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/a60168c4-c743-44e4-9047-df74d47da598-etc-kubernetes\") pod \"multus-5ntjm\" (UID: \"a60168c4-c743-44e4-9047-df74d47da598\") " pod="openshift-multus/multus-5ntjm" Jul 07 11:15:58.248099 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:15:58.247452 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/58b487ae-26a9-4f42-a945-cd1389584821-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-ndwrc\" (UID: \"58b487ae-26a9-4f42-a945-cd1389584821\") " pod="openshift-ovn-kubernetes/ovnkube-node-ndwrc" Jul 07 11:15:58.248099 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:15:58.247491 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/a60168c4-c743-44e4-9047-df74d47da598-host-run-multus-certs\") pod \"multus-5ntjm\" (UID: \"a60168c4-c743-44e4-9047-df74d47da598\") " pod="openshift-multus/multus-5ntjm" Jul 07 11:15:58.248099 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:15:58.247500 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/58b487ae-26a9-4f42-a945-cd1389584821-ovnkube-config\") pod \"ovnkube-node-ndwrc\" (UID: \"58b487ae-26a9-4f42-a945-cd1389584821\") " pod="openshift-ovn-kubernetes/ovnkube-node-ndwrc" Jul 07 11:15:58.248099 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:15:58.247519 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/a60168c4-c743-44e4-9047-df74d47da598-os-release\") pod \"multus-5ntjm\" (UID: \"a60168c4-c743-44e4-9047-df74d47da598\") " pod="openshift-multus/multus-5ntjm" Jul 07 11:15:58.248099 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:15:58.247537 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/d009277d-d0d0-4163-9a6b-3ed9d1ecffa5-sys\") pod \"tuned-ds6m5\" (UID: \"d009277d-d0d0-4163-9a6b-3ed9d1ecffa5\") " pod="openshift-cluster-node-tuning-operator/tuned-ds6m5" Jul 07 11:15:58.248099 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:15:58.247546 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/58b487ae-26a9-4f42-a945-cd1389584821-run-ovn\") pod \"ovnkube-node-ndwrc\" (UID: \"58b487ae-26a9-4f42-a945-cd1389584821\") " pod="openshift-ovn-kubernetes/ovnkube-node-ndwrc" Jul 07 11:15:58.248099 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:15:58.247551 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/a60168c4-c743-44e4-9047-df74d47da598-hostroot\") pod \"multus-5ntjm\" (UID: \"a60168c4-c743-44e4-9047-df74d47da598\") " pod="openshift-multus/multus-5ntjm" Jul 07 11:15:58.248099 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:15:58.247572 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/3bd399c7-6314-4991-9984-8d2ff9677352-kubelet-config\") pod \"global-pull-secret-syncer-8vbr5\" (UID: \"3bd399c7-6314-4991-9984-8d2ff9677352\") " pod="kube-system/global-pull-secret-syncer-8vbr5" Jul 07 11:15:58.248945 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:15:58.247577 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/58b487ae-26a9-4f42-a945-cd1389584821-systemd-units\") pod \"ovnkube-node-ndwrc\" (UID: \"58b487ae-26a9-4f42-a945-cd1389584821\") " pod="openshift-ovn-kubernetes/ovnkube-node-ndwrc" Jul 07 11:15:58.248945 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:15:58.247587 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/58b487ae-26a9-4f42-a945-cd1389584821-run-ovn\") pod \"ovnkube-node-ndwrc\" (UID: \"58b487ae-26a9-4f42-a945-cd1389584821\") " pod="openshift-ovn-kubernetes/ovnkube-node-ndwrc" Jul 07 11:15:58.248945 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:15:58.247611 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/d009277d-d0d0-4163-9a6b-3ed9d1ecffa5-etc-systemd\") pod \"tuned-ds6m5\" (UID: \"d009277d-d0d0-4163-9a6b-3ed9d1ecffa5\") " pod="openshift-cluster-node-tuning-operator/tuned-ds6m5" Jul 07 11:15:58.248945 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:15:58.247635 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/3bd399c7-6314-4991-9984-8d2ff9677352-kubelet-config\") pod \"global-pull-secret-syncer-8vbr5\" (UID: \"3bd399c7-6314-4991-9984-8d2ff9677352\") " pod="kube-system/global-pull-secret-syncer-8vbr5" Jul 07 11:15:58.248945 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:15:58.247636 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/d009277d-d0d0-4163-9a6b-3ed9d1ecffa5-tmp\") pod \"tuned-ds6m5\" (UID: \"d009277d-d0d0-4163-9a6b-3ed9d1ecffa5\") " pod="openshift-cluster-node-tuning-operator/tuned-ds6m5" Jul 07 11:15:58.248945 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:15:58.247670 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/6fcad21b-4b48-4fa1-9f65-92ed08e86703-registration-dir\") pod \"aws-ebs-csi-driver-node-ckhn2\" (UID: \"6fcad21b-4b48-4fa1-9f65-92ed08e86703\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-ckhn2" Jul 07 11:15:58.248945 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:15:58.247691 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/d009277d-d0d0-4163-9a6b-3ed9d1ecffa5-etc-systemd\") pod \"tuned-ds6m5\" (UID: \"d009277d-d0d0-4163-9a6b-3ed9d1ecffa5\") " pod="openshift-cluster-node-tuning-operator/tuned-ds6m5" Jul 07 11:15:58.248945 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:15:58.247695 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/a60168c4-c743-44e4-9047-df74d47da598-system-cni-dir\") pod \"multus-5ntjm\" (UID: \"a60168c4-c743-44e4-9047-df74d47da598\") " pod="openshift-multus/multus-5ntjm" Jul 07 11:15:58.248945 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:15:58.247728 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/58b487ae-26a9-4f42-a945-cd1389584821-run-openvswitch\") pod \"ovnkube-node-ndwrc\" (UID: \"58b487ae-26a9-4f42-a945-cd1389584821\") " pod="openshift-ovn-kubernetes/ovnkube-node-ndwrc" Jul 07 11:15:58.248945 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:15:58.247731 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/a60168c4-c743-44e4-9047-df74d47da598-system-cni-dir\") pod \"multus-5ntjm\" (UID: \"a60168c4-c743-44e4-9047-df74d47da598\") " pod="openshift-multus/multus-5ntjm" Jul 07 11:15:58.248945 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:15:58.247757 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/08ea46cf-197f-4fa4-b752-08df27dbdfee-tuning-conf-dir\") pod \"multus-additional-cni-plugins-rtggm\" (UID: \"08ea46cf-197f-4fa4-b752-08df27dbdfee\") " pod="openshift-multus/multus-additional-cni-plugins-rtggm" Jul 07 11:15:58.248945 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:15:58.247782 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/08ea46cf-197f-4fa4-b752-08df27dbdfee-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-rtggm\" (UID: \"08ea46cf-197f-4fa4-b752-08df27dbdfee\") " pod="openshift-multus/multus-additional-cni-plugins-rtggm" Jul 07 11:15:58.248945 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:15:58.247791 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/a60168c4-c743-44e4-9047-df74d47da598-multus-daemon-config\") pod \"multus-5ntjm\" (UID: \"a60168c4-c743-44e4-9047-df74d47da598\") " pod="openshift-multus/multus-5ntjm" Jul 07 11:15:58.248945 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:15:58.247824 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/58b487ae-26a9-4f42-a945-cd1389584821-run-openvswitch\") pod \"ovnkube-node-ndwrc\" (UID: \"58b487ae-26a9-4f42-a945-cd1389584821\") " pod="openshift-ovn-kubernetes/ovnkube-node-ndwrc" Jul 07 11:15:58.248945 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:15:58.247829 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/3bd399c7-6314-4991-9984-8d2ff9677352-original-pull-secret\") pod \"global-pull-secret-syncer-8vbr5\" (UID: \"3bd399c7-6314-4991-9984-8d2ff9677352\") " pod="kube-system/global-pull-secret-syncer-8vbr5" Jul 07 11:15:58.248945 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:15:58.247867 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/76df1ef6-134a-4d3e-a323-f1a904534e74-host-slash\") pod \"iptables-alerter-sdhzh\" (UID: \"76df1ef6-134a-4d3e-a323-f1a904534e74\") " pod="openshift-network-operator/iptables-alerter-sdhzh" Jul 07 11:15:58.248945 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:15:58.247889 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/08ea46cf-197f-4fa4-b752-08df27dbdfee-tuning-conf-dir\") pod \"multus-additional-cni-plugins-rtggm\" (UID: \"08ea46cf-197f-4fa4-b752-08df27dbdfee\") " pod="openshift-multus/multus-additional-cni-plugins-rtggm" Jul 07 11:15:58.249802 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:15:58.247892 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/6fcad21b-4b48-4fa1-9f65-92ed08e86703-device-dir\") pod \"aws-ebs-csi-driver-node-ckhn2\" (UID: \"6fcad21b-4b48-4fa1-9f65-92ed08e86703\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-ckhn2" Jul 07 11:15:58.249802 ip-10-0-142-112 kubenswrapper[2578]: E0707 11:15:58.247907 2578 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 07 11:15:58.249802 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:15:58.247930 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/58b487ae-26a9-4f42-a945-cd1389584821-host-kubelet\") pod \"ovnkube-node-ndwrc\" (UID: \"58b487ae-26a9-4f42-a945-cd1389584821\") " pod="openshift-ovn-kubernetes/ovnkube-node-ndwrc" Jul 07 11:15:58.249802 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:15:58.247934 2578 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory" Jul 07 11:15:58.249802 ip-10-0-142-112 kubenswrapper[2578]: E0707 11:15:58.247991 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3bd399c7-6314-4991-9984-8d2ff9677352-original-pull-secret podName:3bd399c7-6314-4991-9984-8d2ff9677352 nodeName:}" failed. No retries permitted until 2026-07-07 11:15:58.747957974 +0000 UTC m=+3.044107539 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/3bd399c7-6314-4991-9984-8d2ff9677352-original-pull-secret") pod "global-pull-secret-syncer-8vbr5" (UID: "3bd399c7-6314-4991-9984-8d2ff9677352") : object "kube-system"/"original-pull-secret" not registered Jul 07 11:15:58.249802 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:15:58.247993 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/58b487ae-26a9-4f42-a945-cd1389584821-host-kubelet\") pod \"ovnkube-node-ndwrc\" (UID: \"58b487ae-26a9-4f42-a945-cd1389584821\") " pod="openshift-ovn-kubernetes/ovnkube-node-ndwrc" Jul 07 11:15:58.249802 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:15:58.248015 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/f03343fc-0662-41c1-9904-3ece548a3f42-serviceca\") pod \"node-ca-rms7l\" (UID: \"f03343fc-0662-41c1-9904-3ece548a3f42\") " pod="openshift-image-registry/node-ca-rms7l" Jul 07 11:15:58.249802 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:15:58.248046 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7tpkn\" (UniqueName: \"kubernetes.io/projected/f03343fc-0662-41c1-9904-3ece548a3f42-kube-api-access-7tpkn\") pod \"node-ca-rms7l\" (UID: \"f03343fc-0662-41c1-9904-3ece548a3f42\") " pod="openshift-image-registry/node-ca-rms7l" Jul 07 11:15:58.249802 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:15:58.248071 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/9ac335b6-28f5-4d6a-979c-02587fbd8e95-agent-certs\") pod \"konnectivity-agent-qvc9d\" (UID: \"9ac335b6-28f5-4d6a-979c-02587fbd8e95\") " pod="kube-system/konnectivity-agent-qvc9d" Jul 07 11:15:58.249802 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:15:58.248097 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/3bd399c7-6314-4991-9984-8d2ff9677352-dbus\") pod \"global-pull-secret-syncer-8vbr5\" (UID: \"3bd399c7-6314-4991-9984-8d2ff9677352\") " pod="kube-system/global-pull-secret-syncer-8vbr5" Jul 07 11:15:58.249802 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:15:58.248177 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/3bd399c7-6314-4991-9984-8d2ff9677352-dbus\") pod \"global-pull-secret-syncer-8vbr5\" (UID: \"3bd399c7-6314-4991-9984-8d2ff9677352\") " pod="kube-system/global-pull-secret-syncer-8vbr5" Jul 07 11:15:58.249802 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:15:58.248224 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/08ea46cf-197f-4fa4-b752-08df27dbdfee-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-rtggm\" (UID: \"08ea46cf-197f-4fa4-b752-08df27dbdfee\") " pod="openshift-multus/multus-additional-cni-plugins-rtggm" Jul 07 11:15:58.249802 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:15:58.248234 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/6fcad21b-4b48-4fa1-9f65-92ed08e86703-sys-fs\") pod \"aws-ebs-csi-driver-node-ckhn2\" (UID: \"6fcad21b-4b48-4fa1-9f65-92ed08e86703\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-ckhn2" Jul 07 11:15:58.249802 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:15:58.248268 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/109be65e-ec91-4d45-b856-334d021f900c-hosts-file\") pod \"node-resolver-djzsf\" (UID: \"109be65e-ec91-4d45-b856-334d021f900c\") " pod="openshift-dns/node-resolver-djzsf" Jul 07 11:15:58.249802 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:15:58.248294 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8z6s2\" (UniqueName: \"kubernetes.io/projected/109be65e-ec91-4d45-b856-334d021f900c-kube-api-access-8z6s2\") pod \"node-resolver-djzsf\" (UID: \"109be65e-ec91-4d45-b856-334d021f900c\") " pod="openshift-dns/node-resolver-djzsf" Jul 07 11:15:58.249802 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:15:58.248339 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/a60168c4-c743-44e4-9047-df74d47da598-host-run-netns\") pod \"multus-5ntjm\" (UID: \"a60168c4-c743-44e4-9047-df74d47da598\") " pod="openshift-multus/multus-5ntjm" Jul 07 11:15:58.249802 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:15:58.248357 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/58b487ae-26a9-4f42-a945-cd1389584821-host-run-netns\") pod \"ovnkube-node-ndwrc\" (UID: \"58b487ae-26a9-4f42-a945-cd1389584821\") " pod="openshift-ovn-kubernetes/ovnkube-node-ndwrc" Jul 07 11:15:58.250535 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:15:58.248373 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/f03343fc-0662-41c1-9904-3ece548a3f42-host\") pod \"node-ca-rms7l\" (UID: \"f03343fc-0662-41c1-9904-3ece548a3f42\") " pod="openshift-image-registry/node-ca-rms7l" Jul 07 11:15:58.250535 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:15:58.248389 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/08ea46cf-197f-4fa4-b752-08df27dbdfee-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-rtggm\" (UID: \"08ea46cf-197f-4fa4-b752-08df27dbdfee\") " pod="openshift-multus/multus-additional-cni-plugins-rtggm" Jul 07 11:15:58.250535 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:15:58.248389 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/a60168c4-c743-44e4-9047-df74d47da598-host-run-netns\") pod \"multus-5ntjm\" (UID: \"a60168c4-c743-44e4-9047-df74d47da598\") " pod="openshift-multus/multus-5ntjm" Jul 07 11:15:58.250535 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:15:58.248404 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/109be65e-ec91-4d45-b856-334d021f900c-tmp-dir\") pod \"node-resolver-djzsf\" (UID: \"109be65e-ec91-4d45-b856-334d021f900c\") " pod="openshift-dns/node-resolver-djzsf" Jul 07 11:15:58.250535 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:15:58.248470 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/f03343fc-0662-41c1-9904-3ece548a3f42-host\") pod \"node-ca-rms7l\" (UID: \"f03343fc-0662-41c1-9904-3ece548a3f42\") " pod="openshift-image-registry/node-ca-rms7l" Jul 07 11:15:58.250535 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:15:58.248478 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/58b487ae-26a9-4f42-a945-cd1389584821-host-run-netns\") pod \"ovnkube-node-ndwrc\" (UID: \"58b487ae-26a9-4f42-a945-cd1389584821\") " pod="openshift-ovn-kubernetes/ovnkube-node-ndwrc" Jul 07 11:15:58.250535 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:15:58.248471 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/f03343fc-0662-41c1-9904-3ece548a3f42-serviceca\") pod \"node-ca-rms7l\" (UID: \"f03343fc-0662-41c1-9904-3ece548a3f42\") " pod="openshift-image-registry/node-ca-rms7l" Jul 07 11:15:58.250535 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:15:58.248511 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/58b487ae-26a9-4f42-a945-cd1389584821-env-overrides\") pod \"ovnkube-node-ndwrc\" (UID: \"58b487ae-26a9-4f42-a945-cd1389584821\") " pod="openshift-ovn-kubernetes/ovnkube-node-ndwrc" Jul 07 11:15:58.250535 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:15:58.248535 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/58b487ae-26a9-4f42-a945-cd1389584821-ovnkube-script-lib\") pod \"ovnkube-node-ndwrc\" (UID: \"58b487ae-26a9-4f42-a945-cd1389584821\") " pod="openshift-ovn-kubernetes/ovnkube-node-ndwrc" Jul 07 11:15:58.250535 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:15:58.248557 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/d009277d-d0d0-4163-9a6b-3ed9d1ecffa5-etc-sysconfig\") pod \"tuned-ds6m5\" (UID: \"d009277d-d0d0-4163-9a6b-3ed9d1ecffa5\") " pod="openshift-cluster-node-tuning-operator/tuned-ds6m5" Jul 07 11:15:58.250535 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:15:58.248578 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/d009277d-d0d0-4163-9a6b-3ed9d1ecffa5-run\") pod \"tuned-ds6m5\" (UID: \"d009277d-d0d0-4163-9a6b-3ed9d1ecffa5\") " pod="openshift-cluster-node-tuning-operator/tuned-ds6m5" Jul 07 11:15:58.250535 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:15:58.248604 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sqsnn\" (UniqueName: \"kubernetes.io/projected/76df1ef6-134a-4d3e-a323-f1a904534e74-kube-api-access-sqsnn\") pod \"iptables-alerter-sdhzh\" (UID: \"76df1ef6-134a-4d3e-a323-f1a904534e74\") " pod="openshift-network-operator/iptables-alerter-sdhzh" Jul 07 11:15:58.250535 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:15:58.248609 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/d009277d-d0d0-4163-9a6b-3ed9d1ecffa5-etc-sysconfig\") pod \"tuned-ds6m5\" (UID: \"d009277d-d0d0-4163-9a6b-3ed9d1ecffa5\") " pod="openshift-cluster-node-tuning-operator/tuned-ds6m5" Jul 07 11:15:58.250535 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:15:58.248646 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-w5r2l\" (UniqueName: \"kubernetes.io/projected/6fcad21b-4b48-4fa1-9f65-92ed08e86703-kube-api-access-w5r2l\") pod \"aws-ebs-csi-driver-node-ckhn2\" (UID: \"6fcad21b-4b48-4fa1-9f65-92ed08e86703\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-ckhn2" Jul 07 11:15:58.250535 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:15:58.248653 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/d009277d-d0d0-4163-9a6b-3ed9d1ecffa5-run\") pod \"tuned-ds6m5\" (UID: \"d009277d-d0d0-4163-9a6b-3ed9d1ecffa5\") " pod="openshift-cluster-node-tuning-operator/tuned-ds6m5" Jul 07 11:15:58.250535 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:15:58.248672 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/58b487ae-26a9-4f42-a945-cd1389584821-host-cni-bin\") pod \"ovnkube-node-ndwrc\" (UID: \"58b487ae-26a9-4f42-a945-cd1389584821\") " pod="openshift-ovn-kubernetes/ovnkube-node-ndwrc" Jul 07 11:15:58.250535 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:15:58.248693 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/a60168c4-c743-44e4-9047-df74d47da598-cnibin\") pod \"multus-5ntjm\" (UID: \"a60168c4-c743-44e4-9047-df74d47da598\") " pod="openshift-multus/multus-5ntjm" Jul 07 11:15:58.250535 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:15:58.248702 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/58b487ae-26a9-4f42-a945-cd1389584821-host-cni-bin\") pod \"ovnkube-node-ndwrc\" (UID: \"58b487ae-26a9-4f42-a945-cd1389584821\") " pod="openshift-ovn-kubernetes/ovnkube-node-ndwrc" Jul 07 11:15:58.251339 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:15:58.248713 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-czc2n\" (UniqueName: \"kubernetes.io/projected/08ea46cf-197f-4fa4-b752-08df27dbdfee-kube-api-access-czc2n\") pod \"multus-additional-cni-plugins-rtggm\" (UID: \"08ea46cf-197f-4fa4-b752-08df27dbdfee\") " pod="openshift-multus/multus-additional-cni-plugins-rtggm" Jul 07 11:15:58.251339 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:15:58.248733 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/d009277d-d0d0-4163-9a6b-3ed9d1ecffa5-etc-modprobe-d\") pod \"tuned-ds6m5\" (UID: \"d009277d-d0d0-4163-9a6b-3ed9d1ecffa5\") " pod="openshift-cluster-node-tuning-operator/tuned-ds6m5" Jul 07 11:15:58.251339 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:15:58.248746 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/a60168c4-c743-44e4-9047-df74d47da598-cnibin\") pod \"multus-5ntjm\" (UID: \"a60168c4-c743-44e4-9047-df74d47da598\") " pod="openshift-multus/multus-5ntjm" Jul 07 11:15:58.251339 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:15:58.248759 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/d009277d-d0d0-4163-9a6b-3ed9d1ecffa5-etc-sysctl-d\") pod \"tuned-ds6m5\" (UID: \"d009277d-d0d0-4163-9a6b-3ed9d1ecffa5\") " pod="openshift-cluster-node-tuning-operator/tuned-ds6m5" Jul 07 11:15:58.251339 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:15:58.248778 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/d009277d-d0d0-4163-9a6b-3ed9d1ecffa5-etc-sysctl-conf\") pod \"tuned-ds6m5\" (UID: \"d009277d-d0d0-4163-9a6b-3ed9d1ecffa5\") " pod="openshift-cluster-node-tuning-operator/tuned-ds6m5" Jul 07 11:15:58.251339 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:15:58.248798 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/6fcad21b-4b48-4fa1-9f65-92ed08e86703-etc-selinux\") pod \"aws-ebs-csi-driver-node-ckhn2\" (UID: \"6fcad21b-4b48-4fa1-9f65-92ed08e86703\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-ckhn2" Jul 07 11:15:58.251339 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:15:58.248822 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/a60168c4-c743-44e4-9047-df74d47da598-cni-binary-copy\") pod \"multus-5ntjm\" (UID: \"a60168c4-c743-44e4-9047-df74d47da598\") " pod="openshift-multus/multus-5ntjm" Jul 07 11:15:58.251339 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:15:58.248845 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/58b487ae-26a9-4f42-a945-cd1389584821-var-lib-openvswitch\") pod \"ovnkube-node-ndwrc\" (UID: \"58b487ae-26a9-4f42-a945-cd1389584821\") " pod="openshift-ovn-kubernetes/ovnkube-node-ndwrc" Jul 07 11:15:58.251339 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:15:58.248869 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/08ea46cf-197f-4fa4-b752-08df27dbdfee-os-release\") pod \"multus-additional-cni-plugins-rtggm\" (UID: \"08ea46cf-197f-4fa4-b752-08df27dbdfee\") " pod="openshift-multus/multus-additional-cni-plugins-rtggm" Jul 07 11:15:58.251339 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:15:58.248891 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/1dc0212e-9bf8-4b06-b984-c694479a14ad-metrics-certs\") pod \"network-metrics-daemon-xhlzx\" (UID: \"1dc0212e-9bf8-4b06-b984-c694479a14ad\") " pod="openshift-multus/network-metrics-daemon-xhlzx" Jul 07 11:15:58.251339 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:15:58.248916 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/d009277d-d0d0-4163-9a6b-3ed9d1ecffa5-lib-modules\") pod \"tuned-ds6m5\" (UID: \"d009277d-d0d0-4163-9a6b-3ed9d1ecffa5\") " pod="openshift-cluster-node-tuning-operator/tuned-ds6m5" Jul 07 11:15:58.251339 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:15:58.248933 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/08ea46cf-197f-4fa4-b752-08df27dbdfee-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-rtggm\" (UID: \"08ea46cf-197f-4fa4-b752-08df27dbdfee\") " pod="openshift-multus/multus-additional-cni-plugins-rtggm" Jul 07 11:15:58.251339 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:15:58.248939 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5h5r9\" (UniqueName: \"kubernetes.io/projected/5467fdde-6375-4364-ac0a-f5c6125c0807-kube-api-access-5h5r9\") pod \"network-check-target-tjprx\" (UID: \"5467fdde-6375-4364-ac0a-f5c6125c0807\") " pod="openshift-network-diagnostics/network-check-target-tjprx" Jul 07 11:15:58.251339 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:15:58.248965 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/58b487ae-26a9-4f42-a945-cd1389584821-env-overrides\") pod \"ovnkube-node-ndwrc\" (UID: \"58b487ae-26a9-4f42-a945-cd1389584821\") " pod="openshift-ovn-kubernetes/ovnkube-node-ndwrc" Jul 07 11:15:58.251339 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:15:58.248995 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/58b487ae-26a9-4f42-a945-cd1389584821-ovnkube-script-lib\") pod \"ovnkube-node-ndwrc\" (UID: \"58b487ae-26a9-4f42-a945-cd1389584821\") " pod="openshift-ovn-kubernetes/ovnkube-node-ndwrc" Jul 07 11:15:58.251339 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:15:58.249000 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-65lxp\" (UniqueName: \"kubernetes.io/projected/58b487ae-26a9-4f42-a945-cd1389584821-kube-api-access-65lxp\") pod \"ovnkube-node-ndwrc\" (UID: \"58b487ae-26a9-4f42-a945-cd1389584821\") " pod="openshift-ovn-kubernetes/ovnkube-node-ndwrc" Jul 07 11:15:58.251339 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:15:58.248985 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/58b487ae-26a9-4f42-a945-cd1389584821-var-lib-openvswitch\") pod \"ovnkube-node-ndwrc\" (UID: \"58b487ae-26a9-4f42-a945-cd1389584821\") " pod="openshift-ovn-kubernetes/ovnkube-node-ndwrc" Jul 07 11:15:58.252008 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:15:58.249037 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/d009277d-d0d0-4163-9a6b-3ed9d1ecffa5-host\") pod \"tuned-ds6m5\" (UID: \"d009277d-d0d0-4163-9a6b-3ed9d1ecffa5\") " pod="openshift-cluster-node-tuning-operator/tuned-ds6m5" Jul 07 11:15:58.252008 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:15:58.249047 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/d009277d-d0d0-4163-9a6b-3ed9d1ecffa5-etc-modprobe-d\") pod \"tuned-ds6m5\" (UID: \"d009277d-d0d0-4163-9a6b-3ed9d1ecffa5\") " pod="openshift-cluster-node-tuning-operator/tuned-ds6m5" Jul 07 11:15:58.252008 ip-10-0-142-112 kubenswrapper[2578]: E0707 11:15:58.249057 2578 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 07 11:15:58.252008 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:15:58.249059 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/a60168c4-c743-44e4-9047-df74d47da598-host-var-lib-cni-multus\") pod \"multus-5ntjm\" (UID: \"a60168c4-c743-44e4-9047-df74d47da598\") " pod="openshift-multus/multus-5ntjm" Jul 07 11:15:58.252008 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:15:58.249086 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/58b487ae-26a9-4f42-a945-cd1389584821-log-socket\") pod \"ovnkube-node-ndwrc\" (UID: \"58b487ae-26a9-4f42-a945-cd1389584821\") " pod="openshift-ovn-kubernetes/ovnkube-node-ndwrc" Jul 07 11:15:58.252008 ip-10-0-142-112 kubenswrapper[2578]: E0707 11:15:58.249119 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1dc0212e-9bf8-4b06-b984-c694479a14ad-metrics-certs podName:1dc0212e-9bf8-4b06-b984-c694479a14ad nodeName:}" failed. No retries permitted until 2026-07-07 11:15:58.749100298 +0000 UTC m=+3.045249881 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/1dc0212e-9bf8-4b06-b984-c694479a14ad-metrics-certs") pod "network-metrics-daemon-xhlzx" (UID: "1dc0212e-9bf8-4b06-b984-c694479a14ad") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 07 11:15:58.252008 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:15:58.249140 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/58b487ae-26a9-4f42-a945-cd1389584821-log-socket\") pod \"ovnkube-node-ndwrc\" (UID: \"58b487ae-26a9-4f42-a945-cd1389584821\") " pod="openshift-ovn-kubernetes/ovnkube-node-ndwrc" Jul 07 11:15:58.252008 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:15:58.249119 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/d009277d-d0d0-4163-9a6b-3ed9d1ecffa5-lib-modules\") pod \"tuned-ds6m5\" (UID: \"d009277d-d0d0-4163-9a6b-3ed9d1ecffa5\") " pod="openshift-cluster-node-tuning-operator/tuned-ds6m5" Jul 07 11:15:58.252008 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:15:58.249141 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/76df1ef6-134a-4d3e-a323-f1a904534e74-iptables-alerter-script\") pod \"iptables-alerter-sdhzh\" (UID: \"76df1ef6-134a-4d3e-a323-f1a904534e74\") " pod="openshift-network-operator/iptables-alerter-sdhzh" Jul 07 11:15:58.252008 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:15:58.249169 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/08ea46cf-197f-4fa4-b752-08df27dbdfee-os-release\") pod \"multus-additional-cni-plugins-rtggm\" (UID: \"08ea46cf-197f-4fa4-b752-08df27dbdfee\") " pod="openshift-multus/multus-additional-cni-plugins-rtggm" Jul 07 11:15:58.252008 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:15:58.249272 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/a60168c4-c743-44e4-9047-df74d47da598-host-var-lib-cni-multus\") pod \"multus-5ntjm\" (UID: \"a60168c4-c743-44e4-9047-df74d47da598\") " pod="openshift-multus/multus-5ntjm" Jul 07 11:15:58.252008 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:15:58.249273 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/d009277d-d0d0-4163-9a6b-3ed9d1ecffa5-host\") pod \"tuned-ds6m5\" (UID: \"d009277d-d0d0-4163-9a6b-3ed9d1ecffa5\") " pod="openshift-cluster-node-tuning-operator/tuned-ds6m5" Jul 07 11:15:58.252008 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:15:58.249317 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/a60168c4-c743-44e4-9047-df74d47da598-multus-cni-dir\") pod \"multus-5ntjm\" (UID: \"a60168c4-c743-44e4-9047-df74d47da598\") " pod="openshift-multus/multus-5ntjm" Jul 07 11:15:58.252008 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:15:58.249340 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/d009277d-d0d0-4163-9a6b-3ed9d1ecffa5-etc-sysctl-conf\") pod \"tuned-ds6m5\" (UID: \"d009277d-d0d0-4163-9a6b-3ed9d1ecffa5\") " pod="openshift-cluster-node-tuning-operator/tuned-ds6m5" Jul 07 11:15:58.252008 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:15:58.249363 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/a60168c4-c743-44e4-9047-df74d47da598-multus-socket-dir-parent\") pod \"multus-5ntjm\" (UID: \"a60168c4-c743-44e4-9047-df74d47da598\") " pod="openshift-multus/multus-5ntjm" Jul 07 11:15:58.252008 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:15:58.249401 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/a60168c4-c743-44e4-9047-df74d47da598-host-var-lib-cni-bin\") pod \"multus-5ntjm\" (UID: \"a60168c4-c743-44e4-9047-df74d47da598\") " pod="openshift-multus/multus-5ntjm" Jul 07 11:15:58.252008 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:15:58.249403 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/a60168c4-c743-44e4-9047-df74d47da598-multus-socket-dir-parent\") pod \"multus-5ntjm\" (UID: \"a60168c4-c743-44e4-9047-df74d47da598\") " pod="openshift-multus/multus-5ntjm" Jul 07 11:15:58.252699 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:15:58.249446 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/a60168c4-c743-44e4-9047-df74d47da598-multus-cni-dir\") pod \"multus-5ntjm\" (UID: \"a60168c4-c743-44e4-9047-df74d47da598\") " pod="openshift-multus/multus-5ntjm" Jul 07 11:15:58.252699 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:15:58.249461 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/a60168c4-c743-44e4-9047-df74d47da598-host-var-lib-cni-bin\") pod \"multus-5ntjm\" (UID: \"a60168c4-c743-44e4-9047-df74d47da598\") " pod="openshift-multus/multus-5ntjm" Jul 07 11:15:58.252699 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:15:58.249502 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/58b487ae-26a9-4f42-a945-cd1389584821-host-cni-netd\") pod \"ovnkube-node-ndwrc\" (UID: \"58b487ae-26a9-4f42-a945-cd1389584821\") " pod="openshift-ovn-kubernetes/ovnkube-node-ndwrc" Jul 07 11:15:58.252699 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:15:58.249539 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/58b487ae-26a9-4f42-a945-cd1389584821-host-cni-netd\") pod \"ovnkube-node-ndwrc\" (UID: \"58b487ae-26a9-4f42-a945-cd1389584821\") " pod="openshift-ovn-kubernetes/ovnkube-node-ndwrc" Jul 07 11:15:58.252699 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:15:58.249550 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/d009277d-d0d0-4163-9a6b-3ed9d1ecffa5-etc-tuned\") pod \"tuned-ds6m5\" (UID: \"d009277d-d0d0-4163-9a6b-3ed9d1ecffa5\") " pod="openshift-cluster-node-tuning-operator/tuned-ds6m5" Jul 07 11:15:58.252699 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:15:58.249570 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/a60168c4-c743-44e4-9047-df74d47da598-cni-binary-copy\") pod \"multus-5ntjm\" (UID: \"a60168c4-c743-44e4-9047-df74d47da598\") " pod="openshift-multus/multus-5ntjm" Jul 07 11:15:58.252699 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:15:58.249580 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/a60168c4-c743-44e4-9047-df74d47da598-host-var-lib-kubelet\") pod \"multus-5ntjm\" (UID: \"a60168c4-c743-44e4-9047-df74d47da598\") " pod="openshift-multus/multus-5ntjm" Jul 07 11:15:58.252699 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:15:58.249617 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/a60168c4-c743-44e4-9047-df74d47da598-host-var-lib-kubelet\") pod \"multus-5ntjm\" (UID: \"a60168c4-c743-44e4-9047-df74d47da598\") " pod="openshift-multus/multus-5ntjm" Jul 07 11:15:58.252699 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:15:58.249661 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/58b487ae-26a9-4f42-a945-cd1389584821-run-systemd\") pod \"ovnkube-node-ndwrc\" (UID: \"58b487ae-26a9-4f42-a945-cd1389584821\") " pod="openshift-ovn-kubernetes/ovnkube-node-ndwrc" Jul 07 11:15:58.252699 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:15:58.249704 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/58b487ae-26a9-4f42-a945-cd1389584821-ovn-node-metrics-cert\") pod \"ovnkube-node-ndwrc\" (UID: \"58b487ae-26a9-4f42-a945-cd1389584821\") " pod="openshift-ovn-kubernetes/ovnkube-node-ndwrc" Jul 07 11:15:58.252699 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:15:58.249728 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/9ac335b6-28f5-4d6a-979c-02587fbd8e95-konnectivity-ca\") pod \"konnectivity-agent-qvc9d\" (UID: \"9ac335b6-28f5-4d6a-979c-02587fbd8e95\") " pod="kube-system/konnectivity-agent-qvc9d" Jul 07 11:15:58.252699 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:15:58.249754 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/d009277d-d0d0-4163-9a6b-3ed9d1ecffa5-etc-sysctl-d\") pod \"tuned-ds6m5\" (UID: \"d009277d-d0d0-4163-9a6b-3ed9d1ecffa5\") " pod="openshift-cluster-node-tuning-operator/tuned-ds6m5" Jul 07 11:15:58.252699 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:15:58.249786 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/58b487ae-26a9-4f42-a945-cd1389584821-run-systemd\") pod \"ovnkube-node-ndwrc\" (UID: \"58b487ae-26a9-4f42-a945-cd1389584821\") " pod="openshift-ovn-kubernetes/ovnkube-node-ndwrc" Jul 07 11:15:58.252699 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:15:58.250785 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/9ac335b6-28f5-4d6a-979c-02587fbd8e95-konnectivity-ca\") pod \"konnectivity-agent-qvc9d\" (UID: \"9ac335b6-28f5-4d6a-979c-02587fbd8e95\") " pod="kube-system/konnectivity-agent-qvc9d" Jul 07 11:15:58.252699 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:15:58.251484 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/d009277d-d0d0-4163-9a6b-3ed9d1ecffa5-tmp\") pod \"tuned-ds6m5\" (UID: \"d009277d-d0d0-4163-9a6b-3ed9d1ecffa5\") " pod="openshift-cluster-node-tuning-operator/tuned-ds6m5" Jul 07 11:15:58.252699 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:15:58.251753 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/9ac335b6-28f5-4d6a-979c-02587fbd8e95-agent-certs\") pod \"konnectivity-agent-qvc9d\" (UID: \"9ac335b6-28f5-4d6a-979c-02587fbd8e95\") " pod="kube-system/konnectivity-agent-qvc9d" Jul 07 11:15:58.252699 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:15:58.252082 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/58b487ae-26a9-4f42-a945-cd1389584821-ovn-node-metrics-cert\") pod \"ovnkube-node-ndwrc\" (UID: \"58b487ae-26a9-4f42-a945-cd1389584821\") " pod="openshift-ovn-kubernetes/ovnkube-node-ndwrc" Jul 07 11:15:58.253460 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:15:58.252824 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/d009277d-d0d0-4163-9a6b-3ed9d1ecffa5-etc-tuned\") pod \"tuned-ds6m5\" (UID: \"d009277d-d0d0-4163-9a6b-3ed9d1ecffa5\") " pod="openshift-cluster-node-tuning-operator/tuned-ds6m5" Jul 07 11:15:58.255035 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:15:58.255012 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-47t8g\" (UniqueName: \"kubernetes.io/projected/d009277d-d0d0-4163-9a6b-3ed9d1ecffa5-kube-api-access-47t8g\") pod \"tuned-ds6m5\" (UID: \"d009277d-d0d0-4163-9a6b-3ed9d1ecffa5\") " pod="openshift-cluster-node-tuning-operator/tuned-ds6m5" Jul 07 11:15:58.255169 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:15:58.255148 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-vnslv\" (UniqueName: \"kubernetes.io/projected/1dc0212e-9bf8-4b06-b984-c694479a14ad-kube-api-access-vnslv\") pod \"network-metrics-daemon-xhlzx\" (UID: \"1dc0212e-9bf8-4b06-b984-c694479a14ad\") " pod="openshift-multus/network-metrics-daemon-xhlzx" Jul 07 11:15:58.256723 ip-10-0-142-112 kubenswrapper[2578]: E0707 11:15:58.256692 2578 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 07 11:15:58.256723 ip-10-0-142-112 kubenswrapper[2578]: E0707 11:15:58.256718 2578 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 07 11:15:58.256928 ip-10-0-142-112 kubenswrapper[2578]: E0707 11:15:58.256731 2578 projected.go:196] Error preparing data for projected volume kube-api-access-5h5r9 for pod openshift-network-diagnostics/network-check-target-tjprx: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 07 11:15:58.256928 ip-10-0-142-112 kubenswrapper[2578]: E0707 11:15:58.256790 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/5467fdde-6375-4364-ac0a-f5c6125c0807-kube-api-access-5h5r9 podName:5467fdde-6375-4364-ac0a-f5c6125c0807 nodeName:}" failed. No retries permitted until 2026-07-07 11:15:58.756772225 +0000 UTC m=+3.052921795 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-5h5r9" (UniqueName: "kubernetes.io/projected/5467fdde-6375-4364-ac0a-f5c6125c0807-kube-api-access-5h5r9") pod "network-check-target-tjprx" (UID: "5467fdde-6375-4364-ac0a-f5c6125c0807") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 07 11:15:58.257859 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:15:58.257673 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-nr5pg\" (UniqueName: \"kubernetes.io/projected/a60168c4-c743-44e4-9047-df74d47da598-kube-api-access-nr5pg\") pod \"multus-5ntjm\" (UID: \"a60168c4-c743-44e4-9047-df74d47da598\") " pod="openshift-multus/multus-5ntjm" Jul 07 11:15:58.257859 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:15:58.257793 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-7tpkn\" (UniqueName: \"kubernetes.io/projected/f03343fc-0662-41c1-9904-3ece548a3f42-kube-api-access-7tpkn\") pod \"node-ca-rms7l\" (UID: \"f03343fc-0662-41c1-9904-3ece548a3f42\") " pod="openshift-image-registry/node-ca-rms7l" Jul 07 11:15:58.257859 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:15:58.257802 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-65lxp\" (UniqueName: \"kubernetes.io/projected/58b487ae-26a9-4f42-a945-cd1389584821-kube-api-access-65lxp\") pod \"ovnkube-node-ndwrc\" (UID: \"58b487ae-26a9-4f42-a945-cd1389584821\") " pod="openshift-ovn-kubernetes/ovnkube-node-ndwrc" Jul 07 11:15:58.258040 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:15:58.257889 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-czc2n\" (UniqueName: \"kubernetes.io/projected/08ea46cf-197f-4fa4-b752-08df27dbdfee-kube-api-access-czc2n\") pod \"multus-additional-cni-plugins-rtggm\" (UID: \"08ea46cf-197f-4fa4-b752-08df27dbdfee\") " pod="openshift-multus/multus-additional-cni-plugins-rtggm" Jul 07 11:15:58.305439 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:15:58.305336 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-142-112.ec2.internal" event={"ID":"48c63b94b1c5e4397a6ecdc3e9151f62","Type":"ContainerStarted","Data":"c62300deb8bb7a7003385dc5b749c33566af036fd2232c20155e06fa7951a2fb"} Jul 07 11:15:58.306536 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:15:58.306504 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-112.ec2.internal" event={"ID":"8f3aa81f595d6303175e12307ce9a955","Type":"ContainerStarted","Data":"2c6565a53ced996b30377e55ba7957bb4550d8ec250b9da6712e06134c622c80"} Jul 07 11:15:58.350808 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:15:58.350773 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/6fcad21b-4b48-4fa1-9f65-92ed08e86703-etc-selinux\") pod \"aws-ebs-csi-driver-node-ckhn2\" (UID: \"6fcad21b-4b48-4fa1-9f65-92ed08e86703\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-ckhn2" Jul 07 11:15:58.350962 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:15:58.350828 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/76df1ef6-134a-4d3e-a323-f1a904534e74-iptables-alerter-script\") pod \"iptables-alerter-sdhzh\" (UID: \"76df1ef6-134a-4d3e-a323-f1a904534e74\") " pod="openshift-network-operator/iptables-alerter-sdhzh" Jul 07 11:15:58.350962 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:15:58.350859 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/6fcad21b-4b48-4fa1-9f65-92ed08e86703-socket-dir\") pod \"aws-ebs-csi-driver-node-ckhn2\" (UID: \"6fcad21b-4b48-4fa1-9f65-92ed08e86703\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-ckhn2" Jul 07 11:15:58.350962 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:15:58.350885 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/6fcad21b-4b48-4fa1-9f65-92ed08e86703-kubelet-dir\") pod \"aws-ebs-csi-driver-node-ckhn2\" (UID: \"6fcad21b-4b48-4fa1-9f65-92ed08e86703\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-ckhn2" Jul 07 11:15:58.350962 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:15:58.350912 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/6fcad21b-4b48-4fa1-9f65-92ed08e86703-registration-dir\") pod \"aws-ebs-csi-driver-node-ckhn2\" (UID: \"6fcad21b-4b48-4fa1-9f65-92ed08e86703\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-ckhn2" Jul 07 11:15:58.350962 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:15:58.350938 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/76df1ef6-134a-4d3e-a323-f1a904534e74-host-slash\") pod \"iptables-alerter-sdhzh\" (UID: \"76df1ef6-134a-4d3e-a323-f1a904534e74\") " pod="openshift-network-operator/iptables-alerter-sdhzh" Jul 07 11:15:58.350962 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:15:58.350961 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/6fcad21b-4b48-4fa1-9f65-92ed08e86703-device-dir\") pod \"aws-ebs-csi-driver-node-ckhn2\" (UID: \"6fcad21b-4b48-4fa1-9f65-92ed08e86703\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-ckhn2" Jul 07 11:15:58.351242 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:15:58.350958 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/6fcad21b-4b48-4fa1-9f65-92ed08e86703-etc-selinux\") pod \"aws-ebs-csi-driver-node-ckhn2\" (UID: \"6fcad21b-4b48-4fa1-9f65-92ed08e86703\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-ckhn2" Jul 07 11:15:58.351242 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:15:58.351007 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/6fcad21b-4b48-4fa1-9f65-92ed08e86703-sys-fs\") pod \"aws-ebs-csi-driver-node-ckhn2\" (UID: \"6fcad21b-4b48-4fa1-9f65-92ed08e86703\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-ckhn2" Jul 07 11:15:58.351242 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:15:58.351025 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/6fcad21b-4b48-4fa1-9f65-92ed08e86703-kubelet-dir\") pod \"aws-ebs-csi-driver-node-ckhn2\" (UID: \"6fcad21b-4b48-4fa1-9f65-92ed08e86703\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-ckhn2" Jul 07 11:15:58.351242 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:15:58.351045 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/109be65e-ec91-4d45-b856-334d021f900c-hosts-file\") pod \"node-resolver-djzsf\" (UID: \"109be65e-ec91-4d45-b856-334d021f900c\") " pod="openshift-dns/node-resolver-djzsf" Jul 07 11:15:58.351242 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:15:58.351039 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/6fcad21b-4b48-4fa1-9f65-92ed08e86703-socket-dir\") pod \"aws-ebs-csi-driver-node-ckhn2\" (UID: \"6fcad21b-4b48-4fa1-9f65-92ed08e86703\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-ckhn2" Jul 07 11:15:58.351242 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:15:58.351066 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/6fcad21b-4b48-4fa1-9f65-92ed08e86703-device-dir\") pod \"aws-ebs-csi-driver-node-ckhn2\" (UID: \"6fcad21b-4b48-4fa1-9f65-92ed08e86703\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-ckhn2" Jul 07 11:15:58.351242 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:15:58.351093 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/6fcad21b-4b48-4fa1-9f65-92ed08e86703-registration-dir\") pod \"aws-ebs-csi-driver-node-ckhn2\" (UID: \"6fcad21b-4b48-4fa1-9f65-92ed08e86703\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-ckhn2" Jul 07 11:15:58.351242 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:15:58.351102 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/6fcad21b-4b48-4fa1-9f65-92ed08e86703-sys-fs\") pod \"aws-ebs-csi-driver-node-ckhn2\" (UID: \"6fcad21b-4b48-4fa1-9f65-92ed08e86703\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-ckhn2" Jul 07 11:15:58.351242 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:15:58.351093 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/76df1ef6-134a-4d3e-a323-f1a904534e74-host-slash\") pod \"iptables-alerter-sdhzh\" (UID: \"76df1ef6-134a-4d3e-a323-f1a904534e74\") " pod="openshift-network-operator/iptables-alerter-sdhzh" Jul 07 11:15:58.351242 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:15:58.351125 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8z6s2\" (UniqueName: \"kubernetes.io/projected/109be65e-ec91-4d45-b856-334d021f900c-kube-api-access-8z6s2\") pod \"node-resolver-djzsf\" (UID: \"109be65e-ec91-4d45-b856-334d021f900c\") " pod="openshift-dns/node-resolver-djzsf" Jul 07 11:15:58.351242 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:15:58.351174 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/109be65e-ec91-4d45-b856-334d021f900c-hosts-file\") pod \"node-resolver-djzsf\" (UID: \"109be65e-ec91-4d45-b856-334d021f900c\") " pod="openshift-dns/node-resolver-djzsf" Jul 07 11:15:58.351763 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:15:58.351264 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/109be65e-ec91-4d45-b856-334d021f900c-tmp-dir\") pod \"node-resolver-djzsf\" (UID: \"109be65e-ec91-4d45-b856-334d021f900c\") " pod="openshift-dns/node-resolver-djzsf" Jul 07 11:15:58.351763 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:15:58.351302 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sqsnn\" (UniqueName: \"kubernetes.io/projected/76df1ef6-134a-4d3e-a323-f1a904534e74-kube-api-access-sqsnn\") pod \"iptables-alerter-sdhzh\" (UID: \"76df1ef6-134a-4d3e-a323-f1a904534e74\") " pod="openshift-network-operator/iptables-alerter-sdhzh" Jul 07 11:15:58.351763 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:15:58.351339 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-w5r2l\" (UniqueName: \"kubernetes.io/projected/6fcad21b-4b48-4fa1-9f65-92ed08e86703-kube-api-access-w5r2l\") pod \"aws-ebs-csi-driver-node-ckhn2\" (UID: \"6fcad21b-4b48-4fa1-9f65-92ed08e86703\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-ckhn2" Jul 07 11:15:58.351763 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:15:58.351382 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/76df1ef6-134a-4d3e-a323-f1a904534e74-iptables-alerter-script\") pod \"iptables-alerter-sdhzh\" (UID: \"76df1ef6-134a-4d3e-a323-f1a904534e74\") " pod="openshift-network-operator/iptables-alerter-sdhzh" Jul 07 11:15:58.351763 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:15:58.351594 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/109be65e-ec91-4d45-b856-334d021f900c-tmp-dir\") pod \"node-resolver-djzsf\" (UID: \"109be65e-ec91-4d45-b856-334d021f900c\") " pod="openshift-dns/node-resolver-djzsf" Jul 07 11:15:58.360325 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:15:58.360296 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-sqsnn\" (UniqueName: \"kubernetes.io/projected/76df1ef6-134a-4d3e-a323-f1a904534e74-kube-api-access-sqsnn\") pod \"iptables-alerter-sdhzh\" (UID: \"76df1ef6-134a-4d3e-a323-f1a904534e74\") " pod="openshift-network-operator/iptables-alerter-sdhzh" Jul 07 11:15:58.360599 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:15:58.360579 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-8z6s2\" (UniqueName: \"kubernetes.io/projected/109be65e-ec91-4d45-b856-334d021f900c-kube-api-access-8z6s2\") pod \"node-resolver-djzsf\" (UID: \"109be65e-ec91-4d45-b856-334d021f900c\") " pod="openshift-dns/node-resolver-djzsf" Jul 07 11:15:58.360723 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:15:58.360687 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-w5r2l\" (UniqueName: \"kubernetes.io/projected/6fcad21b-4b48-4fa1-9f65-92ed08e86703-kube-api-access-w5r2l\") pod \"aws-ebs-csi-driver-node-ckhn2\" (UID: \"6fcad21b-4b48-4fa1-9f65-92ed08e86703\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-ckhn2" Jul 07 11:15:58.388829 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:15:58.388800 2578 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160" Jul 07 11:15:58.440149 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:15:58.440119 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-rms7l" Jul 07 11:15:58.452612 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:15:58.452556 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-rtggm" Jul 07 11:15:58.459850 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:15:58.459826 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-qvc9d" Jul 07 11:15:58.466192 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:15:58.466169 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-ds6m5" Jul 07 11:15:58.473730 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:15:58.473709 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-5ntjm" Jul 07 11:15:58.481984 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:15:58.481961 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-ndwrc" Jul 07 11:15:58.497607 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:15:58.497583 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-sdhzh" Jul 07 11:15:58.504384 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:15:58.504364 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-ckhn2" Jul 07 11:15:58.509504 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:15:58.509484 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-djzsf" Jul 07 11:15:58.754711 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:15:58.754629 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/1dc0212e-9bf8-4b06-b984-c694479a14ad-metrics-certs\") pod \"network-metrics-daemon-xhlzx\" (UID: \"1dc0212e-9bf8-4b06-b984-c694479a14ad\") " pod="openshift-multus/network-metrics-daemon-xhlzx" Jul 07 11:15:58.754711 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:15:58.754698 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/3bd399c7-6314-4991-9984-8d2ff9677352-original-pull-secret\") pod \"global-pull-secret-syncer-8vbr5\" (UID: \"3bd399c7-6314-4991-9984-8d2ff9677352\") " pod="kube-system/global-pull-secret-syncer-8vbr5" Jul 07 11:15:58.754869 ip-10-0-142-112 kubenswrapper[2578]: E0707 11:15:58.754785 2578 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 07 11:15:58.754869 ip-10-0-142-112 kubenswrapper[2578]: E0707 11:15:58.754850 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1dc0212e-9bf8-4b06-b984-c694479a14ad-metrics-certs podName:1dc0212e-9bf8-4b06-b984-c694479a14ad nodeName:}" failed. No retries permitted until 2026-07-07 11:15:59.754835802 +0000 UTC m=+4.050985371 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/1dc0212e-9bf8-4b06-b984-c694479a14ad-metrics-certs") pod "network-metrics-daemon-xhlzx" (UID: "1dc0212e-9bf8-4b06-b984-c694479a14ad") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 07 11:15:58.754869 ip-10-0-142-112 kubenswrapper[2578]: E0707 11:15:58.754784 2578 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 07 11:15:58.754966 ip-10-0-142-112 kubenswrapper[2578]: E0707 11:15:58.754897 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3bd399c7-6314-4991-9984-8d2ff9677352-original-pull-secret podName:3bd399c7-6314-4991-9984-8d2ff9677352 nodeName:}" failed. No retries permitted until 2026-07-07 11:15:59.754888217 +0000 UTC m=+4.051037790 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/3bd399c7-6314-4991-9984-8d2ff9677352-original-pull-secret") pod "global-pull-secret-syncer-8vbr5" (UID: "3bd399c7-6314-4991-9984-8d2ff9677352") : object "kube-system"/"original-pull-secret" not registered Jul 07 11:15:58.855008 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:15:58.854973 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5h5r9\" (UniqueName: \"kubernetes.io/projected/5467fdde-6375-4364-ac0a-f5c6125c0807-kube-api-access-5h5r9\") pod \"network-check-target-tjprx\" (UID: \"5467fdde-6375-4364-ac0a-f5c6125c0807\") " pod="openshift-network-diagnostics/network-check-target-tjprx" Jul 07 11:15:58.855184 ip-10-0-142-112 kubenswrapper[2578]: E0707 11:15:58.855127 2578 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 07 11:15:58.855184 ip-10-0-142-112 kubenswrapper[2578]: E0707 11:15:58.855149 2578 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 07 11:15:58.855184 ip-10-0-142-112 kubenswrapper[2578]: E0707 11:15:58.855158 2578 projected.go:196] Error preparing data for projected volume kube-api-access-5h5r9 for pod openshift-network-diagnostics/network-check-target-tjprx: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 07 11:15:58.855352 ip-10-0-142-112 kubenswrapper[2578]: E0707 11:15:58.855210 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/5467fdde-6375-4364-ac0a-f5c6125c0807-kube-api-access-5h5r9 podName:5467fdde-6375-4364-ac0a-f5c6125c0807 nodeName:}" failed. No retries permitted until 2026-07-07 11:15:59.855196636 +0000 UTC m=+4.151346201 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-5h5r9" (UniqueName: "kubernetes.io/projected/5467fdde-6375-4364-ac0a-f5c6125c0807-kube-api-access-5h5r9") pod "network-check-target-tjprx" (UID: "5467fdde-6375-4364-ac0a-f5c6125c0807") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 07 11:15:59.095607 ip-10-0-142-112 kubenswrapper[2578]: W0707 11:15:59.095581 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6fcad21b_4b48_4fa1_9f65_92ed08e86703.slice/crio-e1c915a2471b13ab512ab1f8001199f319bcc23aaee0fa1722e5d5dbd8825081 WatchSource:0}: Error finding container e1c915a2471b13ab512ab1f8001199f319bcc23aaee0fa1722e5d5dbd8825081: Status 404 returned error can't find the container with id e1c915a2471b13ab512ab1f8001199f319bcc23aaee0fa1722e5d5dbd8825081 Jul 07 11:15:59.098916 ip-10-0-142-112 kubenswrapper[2578]: W0707 11:15:59.098887 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod08ea46cf_197f_4fa4_b752_08df27dbdfee.slice/crio-4428b8d543d71cb583e2795453bea2e70bf3107e23185e3f4b502f03f45a4b50 WatchSource:0}: Error finding container 4428b8d543d71cb583e2795453bea2e70bf3107e23185e3f4b502f03f45a4b50: Status 404 returned error can't find the container with id 4428b8d543d71cb583e2795453bea2e70bf3107e23185e3f4b502f03f45a4b50 Jul 07 11:15:59.100973 ip-10-0-142-112 kubenswrapper[2578]: W0707 11:15:59.100883 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd009277d_d0d0_4163_9a6b_3ed9d1ecffa5.slice/crio-d6ed11a030889e4df8d8b6a35920894c81220eed1c23312697d1d479dd144428 WatchSource:0}: Error finding container d6ed11a030889e4df8d8b6a35920894c81220eed1c23312697d1d479dd144428: Status 404 returned error can't find the container with id d6ed11a030889e4df8d8b6a35920894c81220eed1c23312697d1d479dd144428 Jul 07 11:15:59.101637 ip-10-0-142-112 kubenswrapper[2578]: W0707 11:15:59.101618 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf03343fc_0662_41c1_9904_3ece548a3f42.slice/crio-c04e12b1a61c3cc24a643a750afc2148ed2448b8af0abf2f1b2b8bf336bc79e0 WatchSource:0}: Error finding container c04e12b1a61c3cc24a643a750afc2148ed2448b8af0abf2f1b2b8bf336bc79e0: Status 404 returned error can't find the container with id c04e12b1a61c3cc24a643a750afc2148ed2448b8af0abf2f1b2b8bf336bc79e0 Jul 07 11:15:59.102378 ip-10-0-142-112 kubenswrapper[2578]: W0707 11:15:59.102277 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod58b487ae_26a9_4f42_a945_cd1389584821.slice/crio-ffad39043cb4fde0ed68e7c8d71a96bc94445f50f31c2390d34fe8756f765be7 WatchSource:0}: Error finding container ffad39043cb4fde0ed68e7c8d71a96bc94445f50f31c2390d34fe8756f765be7: Status 404 returned error can't find the container with id ffad39043cb4fde0ed68e7c8d71a96bc94445f50f31c2390d34fe8756f765be7 Jul 07 11:15:59.104840 ip-10-0-142-112 kubenswrapper[2578]: W0707 11:15:59.104814 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod76df1ef6_134a_4d3e_a323_f1a904534e74.slice/crio-fe5db443e1884fa96b13502a3c13d1a9050132250e3a4e7727e981f26f02e46c WatchSource:0}: Error finding container fe5db443e1884fa96b13502a3c13d1a9050132250e3a4e7727e981f26f02e46c: Status 404 returned error can't find the container with id fe5db443e1884fa96b13502a3c13d1a9050132250e3a4e7727e981f26f02e46c Jul 07 11:15:59.105467 ip-10-0-142-112 kubenswrapper[2578]: W0707 11:15:59.105446 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod109be65e_ec91_4d45_b856_334d021f900c.slice/crio-eb5c43ff14aba16bb377414009a74c0bccee9c4e6f4b783fcbb01b7c73dc33ff WatchSource:0}: Error finding container eb5c43ff14aba16bb377414009a74c0bccee9c4e6f4b783fcbb01b7c73dc33ff: Status 404 returned error can't find the container with id eb5c43ff14aba16bb377414009a74c0bccee9c4e6f4b783fcbb01b7c73dc33ff Jul 07 11:15:59.106735 ip-10-0-142-112 kubenswrapper[2578]: W0707 11:15:59.106607 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda60168c4_c743_44e4_9047_df74d47da598.slice/crio-ff3097bc809f7100b7c0fb56e33e3b3eb2e2141740b501e2ddab2b5da44b73f7 WatchSource:0}: Error finding container ff3097bc809f7100b7c0fb56e33e3b3eb2e2141740b501e2ddab2b5da44b73f7: Status 404 returned error can't find the container with id ff3097bc809f7100b7c0fb56e33e3b3eb2e2141740b501e2ddab2b5da44b73f7 Jul 07 11:15:59.107116 ip-10-0-142-112 kubenswrapper[2578]: W0707 11:15:59.107048 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9ac335b6_28f5_4d6a_979c_02587fbd8e95.slice/crio-965e4e273272018ed1e0f21a0cc25a1f6395f4ad9592cf1296efa7c4ed8e957f WatchSource:0}: Error finding container 965e4e273272018ed1e0f21a0cc25a1f6395f4ad9592cf1296efa7c4ed8e957f: Status 404 returned error can't find the container with id 965e4e273272018ed1e0f21a0cc25a1f6395f4ad9592cf1296efa7c4ed8e957f Jul 07 11:15:59.180847 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:15:59.180680 2578 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-06 11:10:58 +0000 UTC" deadline="2027-12-26 00:32:45.258793676 +0000 UTC" Jul 07 11:15:59.180847 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:15:59.180845 2578 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="12877h16m46.077951643s" Jul 07 11:15:59.301254 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:15:59.301228 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-8vbr5" Jul 07 11:15:59.301461 ip-10-0-142-112 kubenswrapper[2578]: E0707 11:15:59.301327 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-8vbr5" podUID="3bd399c7-6314-4991-9984-8d2ff9677352" Jul 07 11:15:59.310149 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:15:59.310120 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-qvc9d" event={"ID":"9ac335b6-28f5-4d6a-979c-02587fbd8e95","Type":"ContainerStarted","Data":"965e4e273272018ed1e0f21a0cc25a1f6395f4ad9592cf1296efa7c4ed8e957f"} Jul 07 11:15:59.311231 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:15:59.311209 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-5ntjm" event={"ID":"a60168c4-c743-44e4-9047-df74d47da598","Type":"ContainerStarted","Data":"ff3097bc809f7100b7c0fb56e33e3b3eb2e2141740b501e2ddab2b5da44b73f7"} Jul 07 11:15:59.312321 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:15:59.312299 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-djzsf" event={"ID":"109be65e-ec91-4d45-b856-334d021f900c","Type":"ContainerStarted","Data":"eb5c43ff14aba16bb377414009a74c0bccee9c4e6f4b783fcbb01b7c73dc33ff"} Jul 07 11:15:59.313567 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:15:59.313547 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-ndwrc" event={"ID":"58b487ae-26a9-4f42-a945-cd1389584821","Type":"ContainerStarted","Data":"ffad39043cb4fde0ed68e7c8d71a96bc94445f50f31c2390d34fe8756f765be7"} Jul 07 11:15:59.314710 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:15:59.314691 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-ds6m5" event={"ID":"d009277d-d0d0-4163-9a6b-3ed9d1ecffa5","Type":"ContainerStarted","Data":"d6ed11a030889e4df8d8b6a35920894c81220eed1c23312697d1d479dd144428"} Jul 07 11:15:59.315728 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:15:59.315694 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-ckhn2" event={"ID":"6fcad21b-4b48-4fa1-9f65-92ed08e86703","Type":"ContainerStarted","Data":"e1c915a2471b13ab512ab1f8001199f319bcc23aaee0fa1722e5d5dbd8825081"} Jul 07 11:15:59.316845 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:15:59.316825 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-sdhzh" event={"ID":"76df1ef6-134a-4d3e-a323-f1a904534e74","Type":"ContainerStarted","Data":"fe5db443e1884fa96b13502a3c13d1a9050132250e3a4e7727e981f26f02e46c"} Jul 07 11:15:59.317853 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:15:59.317815 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-rms7l" event={"ID":"f03343fc-0662-41c1-9904-3ece548a3f42","Type":"ContainerStarted","Data":"c04e12b1a61c3cc24a643a750afc2148ed2448b8af0abf2f1b2b8bf336bc79e0"} Jul 07 11:15:59.319281 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:15:59.319264 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-rtggm" event={"ID":"08ea46cf-197f-4fa4-b752-08df27dbdfee","Type":"ContainerStarted","Data":"4428b8d543d71cb583e2795453bea2e70bf3107e23185e3f4b502f03f45a4b50"} Jul 07 11:15:59.321947 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:15:59.321905 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-142-112.ec2.internal" event={"ID":"48c63b94b1c5e4397a6ecdc3e9151f62","Type":"ContainerStarted","Data":"492bdf158b2c98e156490ce04cb58d760c37bbaf02637d84e1e1be3c5b595822"} Jul 07 11:15:59.338763 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:15:59.338705 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-142-112.ec2.internal" podStartSLOduration=2.338692072 podStartE2EDuration="2.338692072s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-07 11:15:57 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 11:15:59.338557905 +0000 UTC m=+3.634707492" watchObservedRunningTime="2026-07-07 11:15:59.338692072 +0000 UTC m=+3.634841660" Jul 07 11:15:59.763488 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:15:59.763377 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/3bd399c7-6314-4991-9984-8d2ff9677352-original-pull-secret\") pod \"global-pull-secret-syncer-8vbr5\" (UID: \"3bd399c7-6314-4991-9984-8d2ff9677352\") " pod="kube-system/global-pull-secret-syncer-8vbr5" Jul 07 11:15:59.763488 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:15:59.763459 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/1dc0212e-9bf8-4b06-b984-c694479a14ad-metrics-certs\") pod \"network-metrics-daemon-xhlzx\" (UID: \"1dc0212e-9bf8-4b06-b984-c694479a14ad\") " pod="openshift-multus/network-metrics-daemon-xhlzx" Jul 07 11:15:59.763699 ip-10-0-142-112 kubenswrapper[2578]: E0707 11:15:59.763597 2578 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 07 11:15:59.763699 ip-10-0-142-112 kubenswrapper[2578]: E0707 11:15:59.763660 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1dc0212e-9bf8-4b06-b984-c694479a14ad-metrics-certs podName:1dc0212e-9bf8-4b06-b984-c694479a14ad nodeName:}" failed. No retries permitted until 2026-07-07 11:16:01.763641011 +0000 UTC m=+6.059790577 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/1dc0212e-9bf8-4b06-b984-c694479a14ad-metrics-certs") pod "network-metrics-daemon-xhlzx" (UID: "1dc0212e-9bf8-4b06-b984-c694479a14ad") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 07 11:15:59.764088 ip-10-0-142-112 kubenswrapper[2578]: E0707 11:15:59.764065 2578 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 07 11:15:59.764185 ip-10-0-142-112 kubenswrapper[2578]: E0707 11:15:59.764123 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3bd399c7-6314-4991-9984-8d2ff9677352-original-pull-secret podName:3bd399c7-6314-4991-9984-8d2ff9677352 nodeName:}" failed. No retries permitted until 2026-07-07 11:16:01.764108329 +0000 UTC m=+6.060257911 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/3bd399c7-6314-4991-9984-8d2ff9677352-original-pull-secret") pod "global-pull-secret-syncer-8vbr5" (UID: "3bd399c7-6314-4991-9984-8d2ff9677352") : object "kube-system"/"original-pull-secret" not registered Jul 07 11:15:59.864577 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:15:59.864542 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5h5r9\" (UniqueName: \"kubernetes.io/projected/5467fdde-6375-4364-ac0a-f5c6125c0807-kube-api-access-5h5r9\") pod \"network-check-target-tjprx\" (UID: \"5467fdde-6375-4364-ac0a-f5c6125c0807\") " pod="openshift-network-diagnostics/network-check-target-tjprx" Jul 07 11:15:59.864740 ip-10-0-142-112 kubenswrapper[2578]: E0707 11:15:59.864700 2578 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 07 11:15:59.864740 ip-10-0-142-112 kubenswrapper[2578]: E0707 11:15:59.864720 2578 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 07 11:15:59.864740 ip-10-0-142-112 kubenswrapper[2578]: E0707 11:15:59.864732 2578 projected.go:196] Error preparing data for projected volume kube-api-access-5h5r9 for pod openshift-network-diagnostics/network-check-target-tjprx: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 07 11:15:59.864894 ip-10-0-142-112 kubenswrapper[2578]: E0707 11:15:59.864791 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/5467fdde-6375-4364-ac0a-f5c6125c0807-kube-api-access-5h5r9 podName:5467fdde-6375-4364-ac0a-f5c6125c0807 nodeName:}" failed. No retries permitted until 2026-07-07 11:16:01.864771818 +0000 UTC m=+6.160921408 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-5h5r9" (UniqueName: "kubernetes.io/projected/5467fdde-6375-4364-ac0a-f5c6125c0807-kube-api-access-5h5r9") pod "network-check-target-tjprx" (UID: "5467fdde-6375-4364-ac0a-f5c6125c0807") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 07 11:16:00.182010 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:16:00.181932 2578 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-06 11:10:58 +0000 UTC" deadline="2027-12-31 12:18:29.741982309 +0000 UTC" Jul 07 11:16:00.182010 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:16:00.181973 2578 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="13009h2m29.560014665s" Jul 07 11:16:00.303350 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:16:00.302554 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-xhlzx" Jul 07 11:16:00.303350 ip-10-0-142-112 kubenswrapper[2578]: E0707 11:16:00.302848 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-xhlzx" podUID="1dc0212e-9bf8-4b06-b984-c694479a14ad" Jul 07 11:16:00.303950 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:16:00.303725 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-tjprx" Jul 07 11:16:00.303950 ip-10-0-142-112 kubenswrapper[2578]: E0707 11:16:00.303888 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-tjprx" podUID="5467fdde-6375-4364-ac0a-f5c6125c0807" Jul 07 11:16:00.336467 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:16:00.336416 2578 generic.go:356] "Generic (PLEG): container finished" podID="8f3aa81f595d6303175e12307ce9a955" containerID="63e9639ba9cf73b6b81b309593f8a09652e9b5df32f9e18e2153ed6bcaa5f0e0" exitCode=0 Jul 07 11:16:00.337964 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:16:00.337691 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-112.ec2.internal" event={"ID":"8f3aa81f595d6303175e12307ce9a955","Type":"ContainerDied","Data":"63e9639ba9cf73b6b81b309593f8a09652e9b5df32f9e18e2153ed6bcaa5f0e0"} Jul 07 11:16:01.301457 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:16:01.301403 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-8vbr5" Jul 07 11:16:01.301980 ip-10-0-142-112 kubenswrapper[2578]: E0707 11:16:01.301561 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-8vbr5" podUID="3bd399c7-6314-4991-9984-8d2ff9677352" Jul 07 11:16:01.368792 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:16:01.368753 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-112.ec2.internal" event={"ID":"8f3aa81f595d6303175e12307ce9a955","Type":"ContainerStarted","Data":"1506c1d08f781e18e2b29ee3d461525e78cce72bff7fa879f635aaea7604daa8"} Jul 07 11:16:01.388748 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:16:01.388671 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-112.ec2.internal" podStartSLOduration=4.388647532 podStartE2EDuration="4.388647532s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-07 11:15:57 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 11:16:01.388252468 +0000 UTC m=+5.684402057" watchObservedRunningTime="2026-07-07 11:16:01.388647532 +0000 UTC m=+5.684797123" Jul 07 11:16:01.781283 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:16:01.781196 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/3bd399c7-6314-4991-9984-8d2ff9677352-original-pull-secret\") pod \"global-pull-secret-syncer-8vbr5\" (UID: \"3bd399c7-6314-4991-9984-8d2ff9677352\") " pod="kube-system/global-pull-secret-syncer-8vbr5" Jul 07 11:16:01.781283 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:16:01.781265 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/1dc0212e-9bf8-4b06-b984-c694479a14ad-metrics-certs\") pod \"network-metrics-daemon-xhlzx\" (UID: \"1dc0212e-9bf8-4b06-b984-c694479a14ad\") " pod="openshift-multus/network-metrics-daemon-xhlzx" Jul 07 11:16:01.781585 ip-10-0-142-112 kubenswrapper[2578]: E0707 11:16:01.781403 2578 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 07 11:16:01.781585 ip-10-0-142-112 kubenswrapper[2578]: E0707 11:16:01.781478 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1dc0212e-9bf8-4b06-b984-c694479a14ad-metrics-certs podName:1dc0212e-9bf8-4b06-b984-c694479a14ad nodeName:}" failed. No retries permitted until 2026-07-07 11:16:05.781459777 +0000 UTC m=+10.077609362 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/1dc0212e-9bf8-4b06-b984-c694479a14ad-metrics-certs") pod "network-metrics-daemon-xhlzx" (UID: "1dc0212e-9bf8-4b06-b984-c694479a14ad") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 07 11:16:01.781585 ip-10-0-142-112 kubenswrapper[2578]: E0707 11:16:01.781545 2578 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 07 11:16:01.781585 ip-10-0-142-112 kubenswrapper[2578]: E0707 11:16:01.781575 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3bd399c7-6314-4991-9984-8d2ff9677352-original-pull-secret podName:3bd399c7-6314-4991-9984-8d2ff9677352 nodeName:}" failed. No retries permitted until 2026-07-07 11:16:05.781564808 +0000 UTC m=+10.077714389 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/3bd399c7-6314-4991-9984-8d2ff9677352-original-pull-secret") pod "global-pull-secret-syncer-8vbr5" (UID: "3bd399c7-6314-4991-9984-8d2ff9677352") : object "kube-system"/"original-pull-secret" not registered Jul 07 11:16:01.882465 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:16:01.882412 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5h5r9\" (UniqueName: \"kubernetes.io/projected/5467fdde-6375-4364-ac0a-f5c6125c0807-kube-api-access-5h5r9\") pod \"network-check-target-tjprx\" (UID: \"5467fdde-6375-4364-ac0a-f5c6125c0807\") " pod="openshift-network-diagnostics/network-check-target-tjprx" Jul 07 11:16:01.882665 ip-10-0-142-112 kubenswrapper[2578]: E0707 11:16:01.882644 2578 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 07 11:16:01.882737 ip-10-0-142-112 kubenswrapper[2578]: E0707 11:16:01.882676 2578 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 07 11:16:01.882737 ip-10-0-142-112 kubenswrapper[2578]: E0707 11:16:01.882689 2578 projected.go:196] Error preparing data for projected volume kube-api-access-5h5r9 for pod openshift-network-diagnostics/network-check-target-tjprx: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 07 11:16:01.882838 ip-10-0-142-112 kubenswrapper[2578]: E0707 11:16:01.882748 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/5467fdde-6375-4364-ac0a-f5c6125c0807-kube-api-access-5h5r9 podName:5467fdde-6375-4364-ac0a-f5c6125c0807 nodeName:}" failed. No retries permitted until 2026-07-07 11:16:05.882729756 +0000 UTC m=+10.178879326 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-5h5r9" (UniqueName: "kubernetes.io/projected/5467fdde-6375-4364-ac0a-f5c6125c0807-kube-api-access-5h5r9") pod "network-check-target-tjprx" (UID: "5467fdde-6375-4364-ac0a-f5c6125c0807") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 07 11:16:02.302194 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:16:02.302160 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-xhlzx" Jul 07 11:16:02.302701 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:16:02.302211 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-tjprx" Jul 07 11:16:02.302701 ip-10-0-142-112 kubenswrapper[2578]: E0707 11:16:02.302317 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-xhlzx" podUID="1dc0212e-9bf8-4b06-b984-c694479a14ad" Jul 07 11:16:02.302701 ip-10-0-142-112 kubenswrapper[2578]: E0707 11:16:02.302438 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-tjprx" podUID="5467fdde-6375-4364-ac0a-f5c6125c0807" Jul 07 11:16:03.302168 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:16:03.302131 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-8vbr5" Jul 07 11:16:03.302369 ip-10-0-142-112 kubenswrapper[2578]: E0707 11:16:03.302279 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-8vbr5" podUID="3bd399c7-6314-4991-9984-8d2ff9677352" Jul 07 11:16:04.301966 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:16:04.301933 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-xhlzx" Jul 07 11:16:04.302146 ip-10-0-142-112 kubenswrapper[2578]: E0707 11:16:04.302074 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-xhlzx" podUID="1dc0212e-9bf8-4b06-b984-c694479a14ad" Jul 07 11:16:04.302486 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:16:04.302465 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-tjprx" Jul 07 11:16:04.302879 ip-10-0-142-112 kubenswrapper[2578]: E0707 11:16:04.302562 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-tjprx" podUID="5467fdde-6375-4364-ac0a-f5c6125c0807" Jul 07 11:16:05.301342 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:16:05.301305 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-8vbr5" Jul 07 11:16:05.301546 ip-10-0-142-112 kubenswrapper[2578]: E0707 11:16:05.301468 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-8vbr5" podUID="3bd399c7-6314-4991-9984-8d2ff9677352" Jul 07 11:16:05.816559 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:16:05.816521 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/1dc0212e-9bf8-4b06-b984-c694479a14ad-metrics-certs\") pod \"network-metrics-daemon-xhlzx\" (UID: \"1dc0212e-9bf8-4b06-b984-c694479a14ad\") " pod="openshift-multus/network-metrics-daemon-xhlzx" Jul 07 11:16:05.817015 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:16:05.816621 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/3bd399c7-6314-4991-9984-8d2ff9677352-original-pull-secret\") pod \"global-pull-secret-syncer-8vbr5\" (UID: \"3bd399c7-6314-4991-9984-8d2ff9677352\") " pod="kube-system/global-pull-secret-syncer-8vbr5" Jul 07 11:16:05.817015 ip-10-0-142-112 kubenswrapper[2578]: E0707 11:16:05.816761 2578 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 07 11:16:05.817015 ip-10-0-142-112 kubenswrapper[2578]: E0707 11:16:05.816828 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3bd399c7-6314-4991-9984-8d2ff9677352-original-pull-secret podName:3bd399c7-6314-4991-9984-8d2ff9677352 nodeName:}" failed. No retries permitted until 2026-07-07 11:16:13.816808273 +0000 UTC m=+18.112957842 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/3bd399c7-6314-4991-9984-8d2ff9677352-original-pull-secret") pod "global-pull-secret-syncer-8vbr5" (UID: "3bd399c7-6314-4991-9984-8d2ff9677352") : object "kube-system"/"original-pull-secret" not registered Jul 07 11:16:05.817179 ip-10-0-142-112 kubenswrapper[2578]: E0707 11:16:05.817027 2578 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 07 11:16:05.817179 ip-10-0-142-112 kubenswrapper[2578]: E0707 11:16:05.817076 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1dc0212e-9bf8-4b06-b984-c694479a14ad-metrics-certs podName:1dc0212e-9bf8-4b06-b984-c694479a14ad nodeName:}" failed. No retries permitted until 2026-07-07 11:16:13.817062343 +0000 UTC m=+18.113211919 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/1dc0212e-9bf8-4b06-b984-c694479a14ad-metrics-certs") pod "network-metrics-daemon-xhlzx" (UID: "1dc0212e-9bf8-4b06-b984-c694479a14ad") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 07 11:16:05.917254 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:16:05.917210 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5h5r9\" (UniqueName: \"kubernetes.io/projected/5467fdde-6375-4364-ac0a-f5c6125c0807-kube-api-access-5h5r9\") pod \"network-check-target-tjprx\" (UID: \"5467fdde-6375-4364-ac0a-f5c6125c0807\") " pod="openshift-network-diagnostics/network-check-target-tjprx" Jul 07 11:16:05.917458 ip-10-0-142-112 kubenswrapper[2578]: E0707 11:16:05.917374 2578 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 07 11:16:05.917458 ip-10-0-142-112 kubenswrapper[2578]: E0707 11:16:05.917396 2578 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 07 11:16:05.917458 ip-10-0-142-112 kubenswrapper[2578]: E0707 11:16:05.917411 2578 projected.go:196] Error preparing data for projected volume kube-api-access-5h5r9 for pod openshift-network-diagnostics/network-check-target-tjprx: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 07 11:16:05.917630 ip-10-0-142-112 kubenswrapper[2578]: E0707 11:16:05.917490 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/5467fdde-6375-4364-ac0a-f5c6125c0807-kube-api-access-5h5r9 podName:5467fdde-6375-4364-ac0a-f5c6125c0807 nodeName:}" failed. No retries permitted until 2026-07-07 11:16:13.917469477 +0000 UTC m=+18.213619049 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-5h5r9" (UniqueName: "kubernetes.io/projected/5467fdde-6375-4364-ac0a-f5c6125c0807-kube-api-access-5h5r9") pod "network-check-target-tjprx" (UID: "5467fdde-6375-4364-ac0a-f5c6125c0807") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 07 11:16:06.302036 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:16:06.301545 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-xhlzx" Jul 07 11:16:06.302036 ip-10-0-142-112 kubenswrapper[2578]: E0707 11:16:06.301691 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-xhlzx" podUID="1dc0212e-9bf8-4b06-b984-c694479a14ad" Jul 07 11:16:06.302036 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:16:06.302029 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-tjprx" Jul 07 11:16:06.303075 ip-10-0-142-112 kubenswrapper[2578]: E0707 11:16:06.303036 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-tjprx" podUID="5467fdde-6375-4364-ac0a-f5c6125c0807" Jul 07 11:16:07.301667 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:16:07.301580 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-8vbr5" Jul 07 11:16:07.302129 ip-10-0-142-112 kubenswrapper[2578]: E0707 11:16:07.301719 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-8vbr5" podUID="3bd399c7-6314-4991-9984-8d2ff9677352" Jul 07 11:16:08.305219 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:16:08.305185 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-tjprx" Jul 07 11:16:08.305706 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:16:08.305185 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-xhlzx" Jul 07 11:16:08.305706 ip-10-0-142-112 kubenswrapper[2578]: E0707 11:16:08.305304 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-tjprx" podUID="5467fdde-6375-4364-ac0a-f5c6125c0807" Jul 07 11:16:08.305706 ip-10-0-142-112 kubenswrapper[2578]: E0707 11:16:08.305398 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-xhlzx" podUID="1dc0212e-9bf8-4b06-b984-c694479a14ad" Jul 07 11:16:09.301136 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:16:09.301104 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-8vbr5" Jul 07 11:16:09.301276 ip-10-0-142-112 kubenswrapper[2578]: E0707 11:16:09.301201 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-8vbr5" podUID="3bd399c7-6314-4991-9984-8d2ff9677352" Jul 07 11:16:10.301816 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:16:10.301781 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-tjprx" Jul 07 11:16:10.302250 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:16:10.301789 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-xhlzx" Jul 07 11:16:10.302250 ip-10-0-142-112 kubenswrapper[2578]: E0707 11:16:10.301933 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-tjprx" podUID="5467fdde-6375-4364-ac0a-f5c6125c0807" Jul 07 11:16:10.302250 ip-10-0-142-112 kubenswrapper[2578]: E0707 11:16:10.301977 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-xhlzx" podUID="1dc0212e-9bf8-4b06-b984-c694479a14ad" Jul 07 11:16:11.301585 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:16:11.301554 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-8vbr5" Jul 07 11:16:11.301774 ip-10-0-142-112 kubenswrapper[2578]: E0707 11:16:11.301674 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-8vbr5" podUID="3bd399c7-6314-4991-9984-8d2ff9677352" Jul 07 11:16:12.302253 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:16:12.302216 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-xhlzx" Jul 07 11:16:12.302670 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:16:12.302216 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-tjprx" Jul 07 11:16:12.302670 ip-10-0-142-112 kubenswrapper[2578]: E0707 11:16:12.302358 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-xhlzx" podUID="1dc0212e-9bf8-4b06-b984-c694479a14ad" Jul 07 11:16:12.302670 ip-10-0-142-112 kubenswrapper[2578]: E0707 11:16:12.302435 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-tjprx" podUID="5467fdde-6375-4364-ac0a-f5c6125c0807" Jul 07 11:16:13.302182 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:16:13.302144 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-8vbr5" Jul 07 11:16:13.302385 ip-10-0-142-112 kubenswrapper[2578]: E0707 11:16:13.302252 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-8vbr5" podUID="3bd399c7-6314-4991-9984-8d2ff9677352" Jul 07 11:16:13.875658 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:16:13.875621 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/3bd399c7-6314-4991-9984-8d2ff9677352-original-pull-secret\") pod \"global-pull-secret-syncer-8vbr5\" (UID: \"3bd399c7-6314-4991-9984-8d2ff9677352\") " pod="kube-system/global-pull-secret-syncer-8vbr5" Jul 07 11:16:13.875956 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:16:13.875685 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/1dc0212e-9bf8-4b06-b984-c694479a14ad-metrics-certs\") pod \"network-metrics-daemon-xhlzx\" (UID: \"1dc0212e-9bf8-4b06-b984-c694479a14ad\") " pod="openshift-multus/network-metrics-daemon-xhlzx" Jul 07 11:16:13.875956 ip-10-0-142-112 kubenswrapper[2578]: E0707 11:16:13.875782 2578 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 07 11:16:13.875956 ip-10-0-142-112 kubenswrapper[2578]: E0707 11:16:13.875792 2578 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 07 11:16:13.875956 ip-10-0-142-112 kubenswrapper[2578]: E0707 11:16:13.875844 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3bd399c7-6314-4991-9984-8d2ff9677352-original-pull-secret podName:3bd399c7-6314-4991-9984-8d2ff9677352 nodeName:}" failed. No retries permitted until 2026-07-07 11:16:29.875830426 +0000 UTC m=+34.171979991 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/3bd399c7-6314-4991-9984-8d2ff9677352-original-pull-secret") pod "global-pull-secret-syncer-8vbr5" (UID: "3bd399c7-6314-4991-9984-8d2ff9677352") : object "kube-system"/"original-pull-secret" not registered Jul 07 11:16:13.875956 ip-10-0-142-112 kubenswrapper[2578]: E0707 11:16:13.875861 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1dc0212e-9bf8-4b06-b984-c694479a14ad-metrics-certs podName:1dc0212e-9bf8-4b06-b984-c694479a14ad nodeName:}" failed. No retries permitted until 2026-07-07 11:16:29.875853503 +0000 UTC m=+34.172003069 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/1dc0212e-9bf8-4b06-b984-c694479a14ad-metrics-certs") pod "network-metrics-daemon-xhlzx" (UID: "1dc0212e-9bf8-4b06-b984-c694479a14ad") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 07 11:16:13.976867 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:16:13.976831 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5h5r9\" (UniqueName: \"kubernetes.io/projected/5467fdde-6375-4364-ac0a-f5c6125c0807-kube-api-access-5h5r9\") pod \"network-check-target-tjprx\" (UID: \"5467fdde-6375-4364-ac0a-f5c6125c0807\") " pod="openshift-network-diagnostics/network-check-target-tjprx" Jul 07 11:16:13.977031 ip-10-0-142-112 kubenswrapper[2578]: E0707 11:16:13.976961 2578 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 07 11:16:13.977031 ip-10-0-142-112 kubenswrapper[2578]: E0707 11:16:13.976978 2578 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 07 11:16:13.977031 ip-10-0-142-112 kubenswrapper[2578]: E0707 11:16:13.976992 2578 projected.go:196] Error preparing data for projected volume kube-api-access-5h5r9 for pod openshift-network-diagnostics/network-check-target-tjprx: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 07 11:16:13.977169 ip-10-0-142-112 kubenswrapper[2578]: E0707 11:16:13.977046 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/5467fdde-6375-4364-ac0a-f5c6125c0807-kube-api-access-5h5r9 podName:5467fdde-6375-4364-ac0a-f5c6125c0807 nodeName:}" failed. No retries permitted until 2026-07-07 11:16:29.977033092 +0000 UTC m=+34.273182659 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-5h5r9" (UniqueName: "kubernetes.io/projected/5467fdde-6375-4364-ac0a-f5c6125c0807-kube-api-access-5h5r9") pod "network-check-target-tjprx" (UID: "5467fdde-6375-4364-ac0a-f5c6125c0807") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 07 11:16:14.301314 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:16:14.301277 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-xhlzx" Jul 07 11:16:14.301510 ip-10-0-142-112 kubenswrapper[2578]: E0707 11:16:14.301418 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-xhlzx" podUID="1dc0212e-9bf8-4b06-b984-c694479a14ad" Jul 07 11:16:14.301510 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:16:14.301453 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-tjprx" Jul 07 11:16:14.301633 ip-10-0-142-112 kubenswrapper[2578]: E0707 11:16:14.301538 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-tjprx" podUID="5467fdde-6375-4364-ac0a-f5c6125c0807" Jul 07 11:16:15.301510 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:16:15.301479 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-8vbr5" Jul 07 11:16:15.301936 ip-10-0-142-112 kubenswrapper[2578]: E0707 11:16:15.301587 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-8vbr5" podUID="3bd399c7-6314-4991-9984-8d2ff9677352" Jul 07 11:16:16.302818 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:16:16.302716 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-tjprx" Jul 07 11:16:16.302818 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:16:16.302795 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-xhlzx" Jul 07 11:16:16.303331 ip-10-0-142-112 kubenswrapper[2578]: E0707 11:16:16.302820 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-tjprx" podUID="5467fdde-6375-4364-ac0a-f5c6125c0807" Jul 07 11:16:16.303331 ip-10-0-142-112 kubenswrapper[2578]: E0707 11:16:16.302918 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-xhlzx" podUID="1dc0212e-9bf8-4b06-b984-c694479a14ad" Jul 07 11:16:17.301312 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:16:17.301276 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-8vbr5" Jul 07 11:16:17.301493 ip-10-0-142-112 kubenswrapper[2578]: E0707 11:16:17.301409 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-8vbr5" podUID="3bd399c7-6314-4991-9984-8d2ff9677352" Jul 07 11:16:18.302055 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:16:18.301711 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-xhlzx" Jul 07 11:16:18.302852 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:16:18.301756 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-tjprx" Jul 07 11:16:18.302852 ip-10-0-142-112 kubenswrapper[2578]: E0707 11:16:18.302180 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-xhlzx" podUID="1dc0212e-9bf8-4b06-b984-c694479a14ad" Jul 07 11:16:18.302852 ip-10-0-142-112 kubenswrapper[2578]: E0707 11:16:18.302210 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-tjprx" podUID="5467fdde-6375-4364-ac0a-f5c6125c0807" Jul 07 11:16:18.407293 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:16:18.407261 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-djzsf" event={"ID":"109be65e-ec91-4d45-b856-334d021f900c","Type":"ContainerStarted","Data":"99dc7647e915765e6259de347af230b5124beca16a899af26d0186cb03f73f9b"} Jul 07 11:16:18.409826 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:16:18.409800 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-ndwrc" event={"ID":"58b487ae-26a9-4f42-a945-cd1389584821","Type":"ContainerStarted","Data":"b27947da0294aa8b4b79c3b90a0746a8ee23ece8a7018b072bf8ddf212f265ab"} Jul 07 11:16:18.409920 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:16:18.409872 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-ndwrc" event={"ID":"58b487ae-26a9-4f42-a945-cd1389584821","Type":"ContainerStarted","Data":"d2ec5183be4c8d8258df739f0a28dbb5fb95bf59d2da0b17600cd056dbf938b9"} Jul 07 11:16:18.409997 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:16:18.409924 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-ndwrc" event={"ID":"58b487ae-26a9-4f42-a945-cd1389584821","Type":"ContainerStarted","Data":"3f758536cdfd84917cd7972f5e78210b5c0c00d6bf4a5d56e043a5c86ad4bd4d"} Jul 07 11:16:18.411362 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:16:18.411331 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-ds6m5" event={"ID":"d009277d-d0d0-4163-9a6b-3ed9d1ecffa5","Type":"ContainerStarted","Data":"8f4e3a0a3a8a3749df1eebfe3aee965d99d0012485a0c1864ef4110b0697f935"} Jul 07 11:16:18.412803 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:16:18.412783 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-ckhn2" event={"ID":"6fcad21b-4b48-4fa1-9f65-92ed08e86703","Type":"ContainerStarted","Data":"af3c9ffe116afcc3b31fa1d5b808356438496385b351649b1a86a328a819f8c1"} Jul 07 11:16:18.414185 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:16:18.414163 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-rms7l" event={"ID":"f03343fc-0662-41c1-9904-3ece548a3f42","Type":"ContainerStarted","Data":"65a8ec8fecdc983677f22b803c8f9a2c8f43c7c80746e845db225248aaa720d2"} Jul 07 11:16:18.415816 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:16:18.415792 2578 generic.go:356] "Generic (PLEG): container finished" podID="08ea46cf-197f-4fa4-b752-08df27dbdfee" containerID="97ba93f9bcb2ae84b764c53fb62d158ff428599e9b12b6466e7eb25f2a7105e6" exitCode=0 Jul 07 11:16:18.415906 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:16:18.415872 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-rtggm" event={"ID":"08ea46cf-197f-4fa4-b752-08df27dbdfee","Type":"ContainerDied","Data":"97ba93f9bcb2ae84b764c53fb62d158ff428599e9b12b6466e7eb25f2a7105e6"} Jul 07 11:16:18.417356 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:16:18.417331 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-qvc9d" event={"ID":"9ac335b6-28f5-4d6a-979c-02587fbd8e95","Type":"ContainerStarted","Data":"4b72bd7bf185c98d0e68a2a3cb68c1fe5bb261c3cf503c966e21f6d37d372fa2"} Jul 07 11:16:18.418800 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:16:18.418779 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-5ntjm" event={"ID":"a60168c4-c743-44e4-9047-df74d47da598","Type":"ContainerStarted","Data":"eea57ba96b82923bc15e0be1240c7c09698fc2cef59603b72b96997242365c88"} Jul 07 11:16:18.421924 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:16:18.421889 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/node-resolver-djzsf" podStartSLOduration=3.619400742 podStartE2EDuration="22.421878295s" totalImagesPullingTime="18.802477553s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-07 11:15:56 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 11:16:18.421532244 +0000 UTC m=+22.717681832" watchObservedRunningTime="2026-07-07 11:16:18.421878295 +0000 UTC m=+22.718027883" Jul 07 11:16:18.434862 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:16:18.434818 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/node-ca-rms7l" podStartSLOduration=3.627521304 podStartE2EDuration="22.43479547s" totalImagesPullingTime="18.807274166s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-07 11:15:56 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 11:16:18.434711803 +0000 UTC m=+22.730861391" watchObservedRunningTime="2026-07-07 11:16:18.43479547 +0000 UTC m=+22.730945060" Jul 07 11:16:18.470405 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:16:18.470365 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/konnectivity-agent-qvc9d" podStartSLOduration=3.701316993 podStartE2EDuration="22.47035044s" totalImagesPullingTime="18.769033447s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 11:15:56 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 11:16:18.469801692 +0000 UTC m=+22.765951280" watchObservedRunningTime="2026-07-07 11:16:18.47035044 +0000 UTC m=+22.766500028" Jul 07 11:16:18.484997 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:16:18.484954 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-ds6m5" podStartSLOduration=3.70942914 podStartE2EDuration="22.484939998s" totalImagesPullingTime="18.775510858s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-07 11:15:56 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 11:16:18.484737867 +0000 UTC m=+22.780887456" watchObservedRunningTime="2026-07-07 11:16:18.484939998 +0000 UTC m=+22.781089586" Jul 07 11:16:18.501441 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:16:18.501386 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-5ntjm" podStartSLOduration=3.525519912 podStartE2EDuration="22.501371615s" totalImagesPullingTime="18.975851703s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-07 11:15:56 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 11:16:18.50093181 +0000 UTC m=+22.797081397" watchObservedRunningTime="2026-07-07 11:16:18.501371615 +0000 UTC m=+22.797521206" Jul 07 11:16:18.863089 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:16:18.863055 2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-qvc9d" Jul 07 11:16:18.863739 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:16:18.863714 2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-qvc9d" Jul 07 11:16:19.302028 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:16:19.302001 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-8vbr5" Jul 07 11:16:19.302206 ip-10-0-142-112 kubenswrapper[2578]: E0707 11:16:19.302122 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-8vbr5" podUID="3bd399c7-6314-4991-9984-8d2ff9677352" Jul 07 11:16:19.317946 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:16:19.317924 2578 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock" Jul 07 11:16:19.423096 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:16:19.423014 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-ckhn2" event={"ID":"6fcad21b-4b48-4fa1-9f65-92ed08e86703","Type":"ContainerStarted","Data":"007e6968cf2d1a0c013f02dca27dd896dc182ef31259095793fd0f8af01a03ea"} Jul 07 11:16:19.424669 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:16:19.424643 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-sdhzh" event={"ID":"76df1ef6-134a-4d3e-a323-f1a904534e74","Type":"ContainerStarted","Data":"d39243aea2eece79d4affcc2352bc7708ece9570a4a9a9a9993c70bd797de58f"} Jul 07 11:16:19.428723 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:16:19.428149 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-ndwrc" event={"ID":"58b487ae-26a9-4f42-a945-cd1389584821","Type":"ContainerStarted","Data":"92a4b86f52e07277653b80cff503d5a1df4b804214feb93a4793473252095c49"} Jul 07 11:16:19.428723 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:16:19.428225 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-ndwrc" event={"ID":"58b487ae-26a9-4f42-a945-cd1389584821","Type":"ContainerStarted","Data":"dfa87b026b5387a1a8e7d42ba707b88759cb4698e01c72d9f0804d8a5ffe1ba7"} Jul 07 11:16:19.428723 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:16:19.428282 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-ndwrc" event={"ID":"58b487ae-26a9-4f42-a945-cd1389584821","Type":"ContainerStarted","Data":"974a9435885f60b63e3159648ec38cad3b1ab4b57517db268ed7b4b9451850b9"} Jul 07 11:16:19.440150 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:16:19.440108 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-sdhzh" podStartSLOduration=4.668792006 podStartE2EDuration="23.440095965s" totalImagesPullingTime="18.771303959s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-07 11:15:56 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 11:16:19.439840686 +0000 UTC m=+23.735990276" watchObservedRunningTime="2026-07-07 11:16:19.440095965 +0000 UTC m=+23.736245567" Jul 07 11:16:20.204169 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:16:20.204031 2578 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-07-07T11:16:19.317940612Z","UUID":"2308533e-7cda-45de-b0d3-9feefebbbe2f","Handler":null,"Name":"","Endpoint":""} Jul 07 11:16:20.206110 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:16:20.205999 2578 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0 Jul 07 11:16:20.206110 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:16:20.206047 2578 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock Jul 07 11:16:20.302200 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:16:20.302168 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-xhlzx" Jul 07 11:16:20.302668 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:16:20.302167 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-tjprx" Jul 07 11:16:20.302668 ip-10-0-142-112 kubenswrapper[2578]: E0707 11:16:20.302304 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-xhlzx" podUID="1dc0212e-9bf8-4b06-b984-c694479a14ad" Jul 07 11:16:20.302668 ip-10-0-142-112 kubenswrapper[2578]: E0707 11:16:20.302367 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-tjprx" podUID="5467fdde-6375-4364-ac0a-f5c6125c0807" Jul 07 11:16:20.430687 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:16:20.430657 2578 prober_manager.go:312] "Failed to trigger a manual run" probe="Readiness" Jul 07 11:16:21.301759 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:16:21.301730 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-8vbr5" Jul 07 11:16:21.301934 ip-10-0-142-112 kubenswrapper[2578]: E0707 11:16:21.301833 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-8vbr5" podUID="3bd399c7-6314-4991-9984-8d2ff9677352" Jul 07 11:16:21.437129 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:16:21.437092 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-ndwrc" event={"ID":"58b487ae-26a9-4f42-a945-cd1389584821","Type":"ContainerStarted","Data":"1c60e481afd887230382964ac4978c8e39305819ba9d179946508918f579dcaa"} Jul 07 11:16:21.439379 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:16:21.439355 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-ckhn2" event={"ID":"6fcad21b-4b48-4fa1-9f65-92ed08e86703","Type":"ContainerStarted","Data":"8386e9608869d1b3da3487b56cfbe0a6cc6b8196fb684dda7fc95130a827b2b0"} Jul 07 11:16:21.464205 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:16:21.464148 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-ckhn2" podStartSLOduration=4.536343449 podStartE2EDuration="25.464133568s" totalImagesPullingTime="20.927790119s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-07 11:15:56 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 11:16:21.461208324 +0000 UTC m=+25.757357913" watchObservedRunningTime="2026-07-07 11:16:21.464133568 +0000 UTC m=+25.760283155" Jul 07 11:16:22.302043 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:16:22.301858 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-xhlzx" Jul 07 11:16:22.302255 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:16:22.301854 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-tjprx" Jul 07 11:16:22.302255 ip-10-0-142-112 kubenswrapper[2578]: E0707 11:16:22.302138 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-xhlzx" podUID="1dc0212e-9bf8-4b06-b984-c694479a14ad" Jul 07 11:16:22.302255 ip-10-0-142-112 kubenswrapper[2578]: E0707 11:16:22.302228 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-tjprx" podUID="5467fdde-6375-4364-ac0a-f5c6125c0807" Jul 07 11:16:23.301919 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:16:23.301736 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-8vbr5" Jul 07 11:16:23.302702 ip-10-0-142-112 kubenswrapper[2578]: E0707 11:16:23.302021 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-8vbr5" podUID="3bd399c7-6314-4991-9984-8d2ff9677352" Jul 07 11:16:23.447230 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:16:23.447107 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-ndwrc" event={"ID":"58b487ae-26a9-4f42-a945-cd1389584821","Type":"ContainerStarted","Data":"4d61000a6b64c387aeddfe34da16d93139df9b0810359ceeaf542b36a95db08b"} Jul 07 11:16:23.447586 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:16:23.447321 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-ndwrc" Jul 07 11:16:23.449065 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:16:23.449039 2578 generic.go:356] "Generic (PLEG): container finished" podID="08ea46cf-197f-4fa4-b752-08df27dbdfee" containerID="c8fc682efb420c61fee86aabc6053f947e9a7e2d4a25d9710c1f8ca73979697c" exitCode=0 Jul 07 11:16:23.449207 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:16:23.449077 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-rtggm" event={"ID":"08ea46cf-197f-4fa4-b752-08df27dbdfee","Type":"ContainerDied","Data":"c8fc682efb420c61fee86aabc6053f947e9a7e2d4a25d9710c1f8ca73979697c"} Jul 07 11:16:23.462697 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:16:23.462678 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-ndwrc" Jul 07 11:16:23.477751 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:16:23.477710 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-ndwrc" podStartSLOduration=8.621155848 podStartE2EDuration="27.477697355s" totalImagesPullingTime="18.856541507s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-07 11:15:56 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 11:16:23.477500232 +0000 UTC m=+27.773649820" watchObservedRunningTime="2026-07-07 11:16:23.477697355 +0000 UTC m=+27.773846943" Jul 07 11:16:24.301510 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:16:24.301479 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-xhlzx" Jul 07 11:16:24.301747 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:16:24.301479 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-tjprx" Jul 07 11:16:24.301747 ip-10-0-142-112 kubenswrapper[2578]: E0707 11:16:24.301607 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-xhlzx" podUID="1dc0212e-9bf8-4b06-b984-c694479a14ad" Jul 07 11:16:24.301747 ip-10-0-142-112 kubenswrapper[2578]: E0707 11:16:24.301693 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-tjprx" podUID="5467fdde-6375-4364-ac0a-f5c6125c0807" Jul 07 11:16:24.451375 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:16:24.451346 2578 prober_manager.go:312] "Failed to trigger a manual run" probe="Readiness" Jul 07 11:16:24.451824 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:16:24.451800 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-ndwrc" Jul 07 11:16:24.466979 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:16:24.466957 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-ndwrc" Jul 07 11:16:25.301757 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:16:25.301718 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-8vbr5" Jul 07 11:16:25.301934 ip-10-0-142-112 kubenswrapper[2578]: E0707 11:16:25.301827 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-8vbr5" podUID="3bd399c7-6314-4991-9984-8d2ff9677352" Jul 07 11:16:25.372761 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:16:25.372727 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-qvc9d" Jul 07 11:16:25.372908 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:16:25.372866 2578 prober_manager.go:312] "Failed to trigger a manual run" probe="Readiness" Jul 07 11:16:25.373373 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:16:25.373359 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-qvc9d" Jul 07 11:16:25.455114 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:16:25.455080 2578 generic.go:356] "Generic (PLEG): container finished" podID="08ea46cf-197f-4fa4-b752-08df27dbdfee" containerID="d7b03654257a90d42b5a58977ee91c41109f64d30af7304681e1396c7b721a0e" exitCode=0 Jul 07 11:16:25.455522 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:16:25.455163 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-rtggm" event={"ID":"08ea46cf-197f-4fa4-b752-08df27dbdfee","Type":"ContainerDied","Data":"d7b03654257a90d42b5a58977ee91c41109f64d30af7304681e1396c7b721a0e"} Jul 07 11:16:25.455522 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:16:25.455267 2578 prober_manager.go:312] "Failed to trigger a manual run" probe="Readiness" Jul 07 11:16:25.945883 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:16:25.945847 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-ndwrc" Jul 07 11:16:26.302852 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:16:26.302819 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-xhlzx" Jul 07 11:16:26.303019 ip-10-0-142-112 kubenswrapper[2578]: E0707 11:16:26.302913 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-xhlzx" podUID="1dc0212e-9bf8-4b06-b984-c694479a14ad" Jul 07 11:16:26.303019 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:16:26.302966 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-tjprx" Jul 07 11:16:26.303096 ip-10-0-142-112 kubenswrapper[2578]: E0707 11:16:26.303073 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-tjprx" podUID="5467fdde-6375-4364-ac0a-f5c6125c0807" Jul 07 11:16:27.301833 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:16:27.301808 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-8vbr5" Jul 07 11:16:27.302137 ip-10-0-142-112 kubenswrapper[2578]: E0707 11:16:27.301904 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-8vbr5" podUID="3bd399c7-6314-4991-9984-8d2ff9677352" Jul 07 11:16:27.462736 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:16:27.462707 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-rtggm" event={"ID":"08ea46cf-197f-4fa4-b752-08df27dbdfee","Type":"ContainerStarted","Data":"647b659a516ad3f3957c54849f6ebf9edd0213cee54871f6ea6865ca5068e315"} Jul 07 11:16:27.478971 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:16:27.478935 2578 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ovn-kubernetes/ovnkube-node-ndwrc" podUID="58b487ae-26a9-4f42-a945-cd1389584821" containerName="ovnkube-controller" probeResult="failure" output="" Jul 07 11:16:28.304483 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:16:28.304454 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-xhlzx" Jul 07 11:16:28.304871 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:16:28.304454 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-tjprx" Jul 07 11:16:28.304871 ip-10-0-142-112 kubenswrapper[2578]: E0707 11:16:28.304560 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-xhlzx" podUID="1dc0212e-9bf8-4b06-b984-c694479a14ad" Jul 07 11:16:28.304871 ip-10-0-142-112 kubenswrapper[2578]: E0707 11:16:28.304651 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-tjprx" podUID="5467fdde-6375-4364-ac0a-f5c6125c0807" Jul 07 11:16:28.468658 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:16:28.468627 2578 generic.go:356] "Generic (PLEG): container finished" podID="08ea46cf-197f-4fa4-b752-08df27dbdfee" containerID="647b659a516ad3f3957c54849f6ebf9edd0213cee54871f6ea6865ca5068e315" exitCode=0 Jul 07 11:16:28.468825 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:16:28.468666 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-rtggm" event={"ID":"08ea46cf-197f-4fa4-b752-08df27dbdfee","Type":"ContainerDied","Data":"647b659a516ad3f3957c54849f6ebf9edd0213cee54871f6ea6865ca5068e315"} Jul 07 11:16:29.301978 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:16:29.301947 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-8vbr5" Jul 07 11:16:29.302156 ip-10-0-142-112 kubenswrapper[2578]: E0707 11:16:29.302063 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-8vbr5" podUID="3bd399c7-6314-4991-9984-8d2ff9677352" Jul 07 11:16:29.898300 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:16:29.898260 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/3bd399c7-6314-4991-9984-8d2ff9677352-original-pull-secret\") pod \"global-pull-secret-syncer-8vbr5\" (UID: \"3bd399c7-6314-4991-9984-8d2ff9677352\") " pod="kube-system/global-pull-secret-syncer-8vbr5" Jul 07 11:16:29.898720 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:16:29.898334 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/1dc0212e-9bf8-4b06-b984-c694479a14ad-metrics-certs\") pod \"network-metrics-daemon-xhlzx\" (UID: \"1dc0212e-9bf8-4b06-b984-c694479a14ad\") " pod="openshift-multus/network-metrics-daemon-xhlzx" Jul 07 11:16:29.898720 ip-10-0-142-112 kubenswrapper[2578]: E0707 11:16:29.898455 2578 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 07 11:16:29.898720 ip-10-0-142-112 kubenswrapper[2578]: E0707 11:16:29.898484 2578 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 07 11:16:29.898720 ip-10-0-142-112 kubenswrapper[2578]: E0707 11:16:29.898541 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3bd399c7-6314-4991-9984-8d2ff9677352-original-pull-secret podName:3bd399c7-6314-4991-9984-8d2ff9677352 nodeName:}" failed. No retries permitted until 2026-07-07 11:17:01.89851707 +0000 UTC m=+66.194666659 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/3bd399c7-6314-4991-9984-8d2ff9677352-original-pull-secret") pod "global-pull-secret-syncer-8vbr5" (UID: "3bd399c7-6314-4991-9984-8d2ff9677352") : object "kube-system"/"original-pull-secret" not registered Jul 07 11:16:29.898720 ip-10-0-142-112 kubenswrapper[2578]: E0707 11:16:29.898561 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1dc0212e-9bf8-4b06-b984-c694479a14ad-metrics-certs podName:1dc0212e-9bf8-4b06-b984-c694479a14ad nodeName:}" failed. No retries permitted until 2026-07-07 11:17:01.898551615 +0000 UTC m=+66.194701184 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/1dc0212e-9bf8-4b06-b984-c694479a14ad-metrics-certs") pod "network-metrics-daemon-xhlzx" (UID: "1dc0212e-9bf8-4b06-b984-c694479a14ad") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 07 11:16:29.999182 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:16:29.999150 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5h5r9\" (UniqueName: \"kubernetes.io/projected/5467fdde-6375-4364-ac0a-f5c6125c0807-kube-api-access-5h5r9\") pod \"network-check-target-tjprx\" (UID: \"5467fdde-6375-4364-ac0a-f5c6125c0807\") " pod="openshift-network-diagnostics/network-check-target-tjprx" Jul 07 11:16:29.999359 ip-10-0-142-112 kubenswrapper[2578]: E0707 11:16:29.999292 2578 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 07 11:16:29.999359 ip-10-0-142-112 kubenswrapper[2578]: E0707 11:16:29.999315 2578 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 07 11:16:29.999359 ip-10-0-142-112 kubenswrapper[2578]: E0707 11:16:29.999328 2578 projected.go:196] Error preparing data for projected volume kube-api-access-5h5r9 for pod openshift-network-diagnostics/network-check-target-tjprx: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 07 11:16:29.999525 ip-10-0-142-112 kubenswrapper[2578]: E0707 11:16:29.999385 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/5467fdde-6375-4364-ac0a-f5c6125c0807-kube-api-access-5h5r9 podName:5467fdde-6375-4364-ac0a-f5c6125c0807 nodeName:}" failed. No retries permitted until 2026-07-07 11:17:01.999368117 +0000 UTC m=+66.295517683 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-5h5r9" (UniqueName: "kubernetes.io/projected/5467fdde-6375-4364-ac0a-f5c6125c0807-kube-api-access-5h5r9") pod "network-check-target-tjprx" (UID: "5467fdde-6375-4364-ac0a-f5c6125c0807") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 07 11:16:30.305394 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:16:30.305192 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-tjprx" Jul 07 11:16:30.305597 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:16:30.305255 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-xhlzx" Jul 07 11:16:30.305597 ip-10-0-142-112 kubenswrapper[2578]: E0707 11:16:30.305509 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-tjprx" podUID="5467fdde-6375-4364-ac0a-f5c6125c0807" Jul 07 11:16:30.305597 ip-10-0-142-112 kubenswrapper[2578]: E0707 11:16:30.305574 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-xhlzx" podUID="1dc0212e-9bf8-4b06-b984-c694479a14ad" Jul 07 11:16:30.355807 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:16:30.355718 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-tjprx"] Jul 07 11:16:30.360042 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:16:30.360016 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-xhlzx"] Jul 07 11:16:30.379586 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:16:30.379556 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-8vbr5"] Jul 07 11:16:30.379755 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:16:30.379664 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-8vbr5" Jul 07 11:16:30.379815 ip-10-0-142-112 kubenswrapper[2578]: E0707 11:16:30.379766 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-8vbr5" podUID="3bd399c7-6314-4991-9984-8d2ff9677352" Jul 07 11:16:30.474297 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:16:30.474215 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-xhlzx" Jul 07 11:16:30.474470 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:16:30.474215 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-tjprx" Jul 07 11:16:30.474470 ip-10-0-142-112 kubenswrapper[2578]: E0707 11:16:30.474366 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-xhlzx" podUID="1dc0212e-9bf8-4b06-b984-c694479a14ad" Jul 07 11:16:30.474470 ip-10-0-142-112 kubenswrapper[2578]: E0707 11:16:30.474417 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-tjprx" podUID="5467fdde-6375-4364-ac0a-f5c6125c0807" Jul 07 11:16:32.305122 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:16:32.305089 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-tjprx" Jul 07 11:16:32.305677 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:16:32.305174 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-xhlzx" Jul 07 11:16:32.305677 ip-10-0-142-112 kubenswrapper[2578]: E0707 11:16:32.305184 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-tjprx" podUID="5467fdde-6375-4364-ac0a-f5c6125c0807" Jul 07 11:16:32.305677 ip-10-0-142-112 kubenswrapper[2578]: E0707 11:16:32.305262 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-xhlzx" podUID="1dc0212e-9bf8-4b06-b984-c694479a14ad" Jul 07 11:16:32.305677 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:16:32.305298 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-8vbr5" Jul 07 11:16:32.305677 ip-10-0-142-112 kubenswrapper[2578]: E0707 11:16:32.305360 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-8vbr5" podUID="3bd399c7-6314-4991-9984-8d2ff9677352" Jul 07 11:16:32.481582 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:16:32.481546 2578 generic.go:356] "Generic (PLEG): container finished" podID="08ea46cf-197f-4fa4-b752-08df27dbdfee" containerID="de0f9e420034e29aeb45512efcce2e2199c17f0ad216b6e87e5ee1fdde9b0b8e" exitCode=0 Jul 07 11:16:32.481582 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:16:32.481586 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-rtggm" event={"ID":"08ea46cf-197f-4fa4-b752-08df27dbdfee","Type":"ContainerDied","Data":"de0f9e420034e29aeb45512efcce2e2199c17f0ad216b6e87e5ee1fdde9b0b8e"} Jul 07 11:16:33.486348 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:16:33.486314 2578 generic.go:356] "Generic (PLEG): container finished" podID="08ea46cf-197f-4fa4-b752-08df27dbdfee" containerID="2b0a4f7ececc38ca1596ec41dba5108bcf0ed6c6d1a0014177f37be6c291ad6f" exitCode=0 Jul 07 11:16:33.486747 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:16:33.486356 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-rtggm" event={"ID":"08ea46cf-197f-4fa4-b752-08df27dbdfee","Type":"ContainerDied","Data":"2b0a4f7ececc38ca1596ec41dba5108bcf0ed6c6d1a0014177f37be6c291ad6f"} Jul 07 11:16:34.302232 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:16:34.302198 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-xhlzx" Jul 07 11:16:34.302232 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:16:34.302229 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-8vbr5" Jul 07 11:16:34.302456 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:16:34.302198 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-tjprx" Jul 07 11:16:34.302456 ip-10-0-142-112 kubenswrapper[2578]: E0707 11:16:34.302319 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-xhlzx" podUID="1dc0212e-9bf8-4b06-b984-c694479a14ad" Jul 07 11:16:34.302456 ip-10-0-142-112 kubenswrapper[2578]: E0707 11:16:34.302398 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-tjprx" podUID="5467fdde-6375-4364-ac0a-f5c6125c0807" Jul 07 11:16:34.302570 ip-10-0-142-112 kubenswrapper[2578]: E0707 11:16:34.302490 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-8vbr5" podUID="3bd399c7-6314-4991-9984-8d2ff9677352" Jul 07 11:16:34.492691 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:16:34.492666 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-rtggm" event={"ID":"08ea46cf-197f-4fa4-b752-08df27dbdfee","Type":"ContainerStarted","Data":"649f66d4e7bb228d4f5b8e552da6991d600d383ec628146708b51b28b7a044c6"} Jul 07 11:16:34.994193 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:16:34.994165 2578 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-112.ec2.internal" event="NodeReady" Jul 07 11:16:34.994360 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:16:34.994279 2578 kubelet_node_status.go:489] "Fast updating node status as it just became ready" Jul 07 11:16:35.079653 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:16:35.079597 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-rtggm" podStartSLOduration=8.602244839 podStartE2EDuration="39.079579779s" totalImagesPullingTime="30.47733494s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-07 11:15:56 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 11:16:34.528028153 +0000 UTC m=+38.824177732" watchObservedRunningTime="2026-07-07 11:16:35.079579779 +0000 UTC m=+39.375729369" Jul 07 11:16:35.079972 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:16:35.079959 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-lgwzn"] Jul 07 11:16:35.091528 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:16:35.091501 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-9t57z"] Jul 07 11:16:35.091643 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:16:35.091610 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-lgwzn" Jul 07 11:16:35.097299 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:16:35.097269 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\"" Jul 07 11:16:35.097875 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:16:35.097843 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\"" Jul 07 11:16:35.102386 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:16:35.102366 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-lgwzn"] Jul 07 11:16:35.102509 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:16:35.102494 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-9t57z" Jul 07 11:16:35.106893 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:16:35.106859 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\"" Jul 07 11:16:35.107045 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:16:35.107030 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-qdxb4\"" Jul 07 11:16:35.107114 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:16:35.107072 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\"" Jul 07 11:16:35.107185 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:16:35.107170 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\"" Jul 07 11:16:35.107619 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:16:35.107599 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\"" Jul 07 11:16:35.107708 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:16:35.107606 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-hgd79\"" Jul 07 11:16:35.111477 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:16:35.111457 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-9t57z"] Jul 07 11:16:35.119743 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:16:35.119724 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-kdmpx"] Jul 07 11:16:35.124816 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:16:35.124798 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-kdmpx" Jul 07 11:16:35.127709 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:16:35.127690 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\"" Jul 07 11:16:35.127836 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:16:35.127820 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\"" Jul 07 11:16:35.128035 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:16:35.127912 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\"" Jul 07 11:16:35.129613 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:16:35.129573 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-hr2xm\"" Jul 07 11:16:35.136985 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:16:35.136964 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-kdmpx"] Jul 07 11:16:35.237980 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:16:35.237943 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/bcd0ff4f-9b04-41d6-afef-018180f7dd2a-data-volume\") pod \"insights-runtime-extractor-9t57z\" (UID: \"bcd0ff4f-9b04-41d6-afef-018180f7dd2a\") " pod="openshift-insights/insights-runtime-extractor-9t57z" Jul 07 11:16:35.237980 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:16:35.237979 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/fba5ce69-3377-4a37-ba7c-6b3adb88b2f3-cert\") pod \"ingress-canary-kdmpx\" (UID: \"fba5ce69-3377-4a37-ba7c-6b3adb88b2f3\") " pod="openshift-ingress-canary/ingress-canary-kdmpx" Jul 07 11:16:35.238194 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:16:35.238002 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/12b4908e-0a68-467c-966f-a1900491e26c-config-volume\") pod \"dns-default-lgwzn\" (UID: \"12b4908e-0a68-467c-966f-a1900491e26c\") " pod="openshift-dns/dns-default-lgwzn" Jul 07 11:16:35.238194 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:16:35.238072 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/12b4908e-0a68-467c-966f-a1900491e26c-metrics-tls\") pod \"dns-default-lgwzn\" (UID: \"12b4908e-0a68-467c-966f-a1900491e26c\") " pod="openshift-dns/dns-default-lgwzn" Jul 07 11:16:35.238194 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:16:35.238099 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/12b4908e-0a68-467c-966f-a1900491e26c-tmp-dir\") pod \"dns-default-lgwzn\" (UID: \"12b4908e-0a68-467c-966f-a1900491e26c\") " pod="openshift-dns/dns-default-lgwzn" Jul 07 11:16:35.238194 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:16:35.238155 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/bcd0ff4f-9b04-41d6-afef-018180f7dd2a-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-9t57z\" (UID: \"bcd0ff4f-9b04-41d6-afef-018180f7dd2a\") " pod="openshift-insights/insights-runtime-extractor-9t57z" Jul 07 11:16:35.238194 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:16:35.238174 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-j5ppc\" (UniqueName: \"kubernetes.io/projected/bcd0ff4f-9b04-41d6-afef-018180f7dd2a-kube-api-access-j5ppc\") pod \"insights-runtime-extractor-9t57z\" (UID: \"bcd0ff4f-9b04-41d6-afef-018180f7dd2a\") " pod="openshift-insights/insights-runtime-extractor-9t57z" Jul 07 11:16:35.238364 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:16:35.238202 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/bcd0ff4f-9b04-41d6-afef-018180f7dd2a-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-9t57z\" (UID: \"bcd0ff4f-9b04-41d6-afef-018180f7dd2a\") " pod="openshift-insights/insights-runtime-extractor-9t57z" Jul 07 11:16:35.238364 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:16:35.238253 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-twx2q\" (UniqueName: \"kubernetes.io/projected/12b4908e-0a68-467c-966f-a1900491e26c-kube-api-access-twx2q\") pod \"dns-default-lgwzn\" (UID: \"12b4908e-0a68-467c-966f-a1900491e26c\") " pod="openshift-dns/dns-default-lgwzn" Jul 07 11:16:35.238364 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:16:35.238268 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vxvd9\" (UniqueName: \"kubernetes.io/projected/fba5ce69-3377-4a37-ba7c-6b3adb88b2f3-kube-api-access-vxvd9\") pod \"ingress-canary-kdmpx\" (UID: \"fba5ce69-3377-4a37-ba7c-6b3adb88b2f3\") " pod="openshift-ingress-canary/ingress-canary-kdmpx" Jul 07 11:16:35.238364 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:16:35.238310 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/bcd0ff4f-9b04-41d6-afef-018180f7dd2a-crio-socket\") pod \"insights-runtime-extractor-9t57z\" (UID: \"bcd0ff4f-9b04-41d6-afef-018180f7dd2a\") " pod="openshift-insights/insights-runtime-extractor-9t57z" Jul 07 11:16:35.339656 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:16:35.339616 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/bcd0ff4f-9b04-41d6-afef-018180f7dd2a-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-9t57z\" (UID: \"bcd0ff4f-9b04-41d6-afef-018180f7dd2a\") " pod="openshift-insights/insights-runtime-extractor-9t57z" Jul 07 11:16:35.339656 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:16:35.339652 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-j5ppc\" (UniqueName: \"kubernetes.io/projected/bcd0ff4f-9b04-41d6-afef-018180f7dd2a-kube-api-access-j5ppc\") pod \"insights-runtime-extractor-9t57z\" (UID: \"bcd0ff4f-9b04-41d6-afef-018180f7dd2a\") " pod="openshift-insights/insights-runtime-extractor-9t57z" Jul 07 11:16:35.339903 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:16:35.339673 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/bcd0ff4f-9b04-41d6-afef-018180f7dd2a-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-9t57z\" (UID: \"bcd0ff4f-9b04-41d6-afef-018180f7dd2a\") " pod="openshift-insights/insights-runtime-extractor-9t57z" Jul 07 11:16:35.339903 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:16:35.339712 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-twx2q\" (UniqueName: \"kubernetes.io/projected/12b4908e-0a68-467c-966f-a1900491e26c-kube-api-access-twx2q\") pod \"dns-default-lgwzn\" (UID: \"12b4908e-0a68-467c-966f-a1900491e26c\") " pod="openshift-dns/dns-default-lgwzn" Jul 07 11:16:35.339903 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:16:35.339728 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vxvd9\" (UniqueName: \"kubernetes.io/projected/fba5ce69-3377-4a37-ba7c-6b3adb88b2f3-kube-api-access-vxvd9\") pod \"ingress-canary-kdmpx\" (UID: \"fba5ce69-3377-4a37-ba7c-6b3adb88b2f3\") " pod="openshift-ingress-canary/ingress-canary-kdmpx" Jul 07 11:16:35.339903 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:16:35.339744 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/bcd0ff4f-9b04-41d6-afef-018180f7dd2a-crio-socket\") pod \"insights-runtime-extractor-9t57z\" (UID: \"bcd0ff4f-9b04-41d6-afef-018180f7dd2a\") " pod="openshift-insights/insights-runtime-extractor-9t57z" Jul 07 11:16:35.339903 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:16:35.339761 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/bcd0ff4f-9b04-41d6-afef-018180f7dd2a-data-volume\") pod \"insights-runtime-extractor-9t57z\" (UID: \"bcd0ff4f-9b04-41d6-afef-018180f7dd2a\") " pod="openshift-insights/insights-runtime-extractor-9t57z" Jul 07 11:16:35.339903 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:16:35.339782 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/fba5ce69-3377-4a37-ba7c-6b3adb88b2f3-cert\") pod \"ingress-canary-kdmpx\" (UID: \"fba5ce69-3377-4a37-ba7c-6b3adb88b2f3\") " pod="openshift-ingress-canary/ingress-canary-kdmpx" Jul 07 11:16:35.340169 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:16:35.340090 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/bcd0ff4f-9b04-41d6-afef-018180f7dd2a-data-volume\") pod \"insights-runtime-extractor-9t57z\" (UID: \"bcd0ff4f-9b04-41d6-afef-018180f7dd2a\") " pod="openshift-insights/insights-runtime-extractor-9t57z" Jul 07 11:16:35.340169 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:16:35.340130 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/12b4908e-0a68-467c-966f-a1900491e26c-config-volume\") pod \"dns-default-lgwzn\" (UID: \"12b4908e-0a68-467c-966f-a1900491e26c\") " pod="openshift-dns/dns-default-lgwzn" Jul 07 11:16:35.340169 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:16:35.340149 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/bcd0ff4f-9b04-41d6-afef-018180f7dd2a-crio-socket\") pod \"insights-runtime-extractor-9t57z\" (UID: \"bcd0ff4f-9b04-41d6-afef-018180f7dd2a\") " pod="openshift-insights/insights-runtime-extractor-9t57z" Jul 07 11:16:35.340286 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:16:35.340176 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/12b4908e-0a68-467c-966f-a1900491e26c-metrics-tls\") pod \"dns-default-lgwzn\" (UID: \"12b4908e-0a68-467c-966f-a1900491e26c\") " pod="openshift-dns/dns-default-lgwzn" Jul 07 11:16:35.340286 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:16:35.340211 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/12b4908e-0a68-467c-966f-a1900491e26c-tmp-dir\") pod \"dns-default-lgwzn\" (UID: \"12b4908e-0a68-467c-966f-a1900491e26c\") " pod="openshift-dns/dns-default-lgwzn" Jul 07 11:16:35.340286 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:16:35.340265 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/bcd0ff4f-9b04-41d6-afef-018180f7dd2a-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-9t57z\" (UID: \"bcd0ff4f-9b04-41d6-afef-018180f7dd2a\") " pod="openshift-insights/insights-runtime-extractor-9t57z" Jul 07 11:16:35.340561 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:16:35.340542 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/12b4908e-0a68-467c-966f-a1900491e26c-tmp-dir\") pod \"dns-default-lgwzn\" (UID: \"12b4908e-0a68-467c-966f-a1900491e26c\") " pod="openshift-dns/dns-default-lgwzn" Jul 07 11:16:35.340675 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:16:35.340660 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/12b4908e-0a68-467c-966f-a1900491e26c-config-volume\") pod \"dns-default-lgwzn\" (UID: \"12b4908e-0a68-467c-966f-a1900491e26c\") " pod="openshift-dns/dns-default-lgwzn" Jul 07 11:16:35.344027 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:16:35.343894 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/12b4908e-0a68-467c-966f-a1900491e26c-metrics-tls\") pod \"dns-default-lgwzn\" (UID: \"12b4908e-0a68-467c-966f-a1900491e26c\") " pod="openshift-dns/dns-default-lgwzn" Jul 07 11:16:35.344136 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:16:35.343904 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/fba5ce69-3377-4a37-ba7c-6b3adb88b2f3-cert\") pod \"ingress-canary-kdmpx\" (UID: \"fba5ce69-3377-4a37-ba7c-6b3adb88b2f3\") " pod="openshift-ingress-canary/ingress-canary-kdmpx" Jul 07 11:16:35.344136 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:16:35.343909 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/bcd0ff4f-9b04-41d6-afef-018180f7dd2a-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-9t57z\" (UID: \"bcd0ff4f-9b04-41d6-afef-018180f7dd2a\") " pod="openshift-insights/insights-runtime-extractor-9t57z" Jul 07 11:16:35.353682 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:16:35.353657 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-j5ppc\" (UniqueName: \"kubernetes.io/projected/bcd0ff4f-9b04-41d6-afef-018180f7dd2a-kube-api-access-j5ppc\") pod \"insights-runtime-extractor-9t57z\" (UID: \"bcd0ff4f-9b04-41d6-afef-018180f7dd2a\") " pod="openshift-insights/insights-runtime-extractor-9t57z" Jul 07 11:16:35.353800 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:16:35.353704 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-vxvd9\" (UniqueName: \"kubernetes.io/projected/fba5ce69-3377-4a37-ba7c-6b3adb88b2f3-kube-api-access-vxvd9\") pod \"ingress-canary-kdmpx\" (UID: \"fba5ce69-3377-4a37-ba7c-6b3adb88b2f3\") " pod="openshift-ingress-canary/ingress-canary-kdmpx" Jul 07 11:16:35.355775 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:16:35.355754 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-twx2q\" (UniqueName: \"kubernetes.io/projected/12b4908e-0a68-467c-966f-a1900491e26c-kube-api-access-twx2q\") pod \"dns-default-lgwzn\" (UID: \"12b4908e-0a68-467c-966f-a1900491e26c\") " pod="openshift-dns/dns-default-lgwzn" Jul 07 11:16:35.401493 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:16:35.401456 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-lgwzn" Jul 07 11:16:35.410621 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:16:35.410594 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-9t57z" Jul 07 11:16:35.432756 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:16:35.432729 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-kdmpx" Jul 07 11:16:35.634294 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:16:35.634264 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-9t57z"] Jul 07 11:16:35.637812 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:16:35.637789 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-lgwzn"] Jul 07 11:16:35.638477 ip-10-0-142-112 kubenswrapper[2578]: W0707 11:16:35.638452 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbcd0ff4f_9b04_41d6_afef_018180f7dd2a.slice/crio-4b811ce2beaf7793edee3281b5ea0b098223834b77ddd60d044d14f14370a9d1 WatchSource:0}: Error finding container 4b811ce2beaf7793edee3281b5ea0b098223834b77ddd60d044d14f14370a9d1: Status 404 returned error can't find the container with id 4b811ce2beaf7793edee3281b5ea0b098223834b77ddd60d044d14f14370a9d1 Jul 07 11:16:35.640571 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:16:35.640505 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-kdmpx"] Jul 07 11:16:35.641318 ip-10-0-142-112 kubenswrapper[2578]: W0707 11:16:35.641294 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod12b4908e_0a68_467c_966f_a1900491e26c.slice/crio-aa3de94de11acd196383503d25b727ce3b0ed0241923d2f278a04239376e850e WatchSource:0}: Error finding container aa3de94de11acd196383503d25b727ce3b0ed0241923d2f278a04239376e850e: Status 404 returned error can't find the container with id aa3de94de11acd196383503d25b727ce3b0ed0241923d2f278a04239376e850e Jul 07 11:16:35.645825 ip-10-0-142-112 kubenswrapper[2578]: W0707 11:16:35.645808 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfba5ce69_3377_4a37_ba7c_6b3adb88b2f3.slice/crio-d929b0732dd2a9b16c0bf3e13cfd00325b4948119b71da340e303720e707c396 WatchSource:0}: Error finding container d929b0732dd2a9b16c0bf3e13cfd00325b4948119b71da340e303720e707c396: Status 404 returned error can't find the container with id d929b0732dd2a9b16c0bf3e13cfd00325b4948119b71da340e303720e707c396 Jul 07 11:16:36.305482 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:16:36.305450 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-tjprx" Jul 07 11:16:36.305714 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:16:36.305552 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-xhlzx" Jul 07 11:16:36.305794 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:16:36.305738 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-8vbr5" Jul 07 11:16:36.308001 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:16:36.307982 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Jul 07 11:16:36.308118 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:16:36.308016 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-42fbv\"" Jul 07 11:16:36.308770 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:16:36.308733 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" Jul 07 11:16:36.308877 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:16:36.308842 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" Jul 07 11:16:36.308942 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:16:36.308877 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Jul 07 11:16:36.308992 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:16:36.308957 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-89jkb\"" Jul 07 11:16:36.499821 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:16:36.499782 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-kdmpx" event={"ID":"fba5ce69-3377-4a37-ba7c-6b3adb88b2f3","Type":"ContainerStarted","Data":"d929b0732dd2a9b16c0bf3e13cfd00325b4948119b71da340e303720e707c396"} Jul 07 11:16:36.502650 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:16:36.502618 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-9t57z" event={"ID":"bcd0ff4f-9b04-41d6-afef-018180f7dd2a","Type":"ContainerStarted","Data":"4b811ce2beaf7793edee3281b5ea0b098223834b77ddd60d044d14f14370a9d1"} Jul 07 11:16:36.503996 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:16:36.503971 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-lgwzn" event={"ID":"12b4908e-0a68-467c-966f-a1900491e26c","Type":"ContainerStarted","Data":"aa3de94de11acd196383503d25b727ce3b0ed0241923d2f278a04239376e850e"} Jul 07 11:16:41.424921 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:16:41.424883 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-operator-567f7b8b7c-lbgcs"] Jul 07 11:16:41.453164 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:16:41.453132 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-567f7b8b7c-lbgcs"] Jul 07 11:16:41.453352 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:16:41.453275 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-567f7b8b7c-lbgcs" Jul 07 11:16:41.455819 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:16:41.455795 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-tls\"" Jul 07 11:16:41.455979 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:16:41.455962 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-dockercfg-j64cr\"" Jul 07 11:16:41.456056 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:16:41.456023 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\"" Jul 07 11:16:41.456056 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:16:41.456024 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-kube-rbac-proxy-config\"" Jul 07 11:16:41.456713 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:16:41.456693 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\"" Jul 07 11:16:41.456830 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:16:41.456732 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\"" Jul 07 11:16:41.588045 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:16:41.588021 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/eb34d7f1-0c6a-4b0a-92f6-e03dcf0a989d-prometheus-operator-tls\") pod \"prometheus-operator-567f7b8b7c-lbgcs\" (UID: \"eb34d7f1-0c6a-4b0a-92f6-e03dcf0a989d\") " pod="openshift-monitoring/prometheus-operator-567f7b8b7c-lbgcs" Jul 07 11:16:41.588190 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:16:41.588050 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/eb34d7f1-0c6a-4b0a-92f6-e03dcf0a989d-metrics-client-ca\") pod \"prometheus-operator-567f7b8b7c-lbgcs\" (UID: \"eb34d7f1-0c6a-4b0a-92f6-e03dcf0a989d\") " pod="openshift-monitoring/prometheus-operator-567f7b8b7c-lbgcs" Jul 07 11:16:41.588190 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:16:41.588069 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/eb34d7f1-0c6a-4b0a-92f6-e03dcf0a989d-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-567f7b8b7c-lbgcs\" (UID: \"eb34d7f1-0c6a-4b0a-92f6-e03dcf0a989d\") " pod="openshift-monitoring/prometheus-operator-567f7b8b7c-lbgcs" Jul 07 11:16:41.588190 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:16:41.588085 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7c8x4\" (UniqueName: \"kubernetes.io/projected/eb34d7f1-0c6a-4b0a-92f6-e03dcf0a989d-kube-api-access-7c8x4\") pod \"prometheus-operator-567f7b8b7c-lbgcs\" (UID: \"eb34d7f1-0c6a-4b0a-92f6-e03dcf0a989d\") " pod="openshift-monitoring/prometheus-operator-567f7b8b7c-lbgcs" Jul 07 11:16:41.688775 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:16:41.688748 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/eb34d7f1-0c6a-4b0a-92f6-e03dcf0a989d-prometheus-operator-tls\") pod \"prometheus-operator-567f7b8b7c-lbgcs\" (UID: \"eb34d7f1-0c6a-4b0a-92f6-e03dcf0a989d\") " pod="openshift-monitoring/prometheus-operator-567f7b8b7c-lbgcs" Jul 07 11:16:41.688864 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:16:41.688798 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/eb34d7f1-0c6a-4b0a-92f6-e03dcf0a989d-metrics-client-ca\") pod \"prometheus-operator-567f7b8b7c-lbgcs\" (UID: \"eb34d7f1-0c6a-4b0a-92f6-e03dcf0a989d\") " pod="openshift-monitoring/prometheus-operator-567f7b8b7c-lbgcs" Jul 07 11:16:41.688864 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:16:41.688829 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/eb34d7f1-0c6a-4b0a-92f6-e03dcf0a989d-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-567f7b8b7c-lbgcs\" (UID: \"eb34d7f1-0c6a-4b0a-92f6-e03dcf0a989d\") " pod="openshift-monitoring/prometheus-operator-567f7b8b7c-lbgcs" Jul 07 11:16:41.688864 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:16:41.688856 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7c8x4\" (UniqueName: \"kubernetes.io/projected/eb34d7f1-0c6a-4b0a-92f6-e03dcf0a989d-kube-api-access-7c8x4\") pod \"prometheus-operator-567f7b8b7c-lbgcs\" (UID: \"eb34d7f1-0c6a-4b0a-92f6-e03dcf0a989d\") " pod="openshift-monitoring/prometheus-operator-567f7b8b7c-lbgcs" Jul 07 11:16:41.689603 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:16:41.689557 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/eb34d7f1-0c6a-4b0a-92f6-e03dcf0a989d-metrics-client-ca\") pod \"prometheus-operator-567f7b8b7c-lbgcs\" (UID: \"eb34d7f1-0c6a-4b0a-92f6-e03dcf0a989d\") " pod="openshift-monitoring/prometheus-operator-567f7b8b7c-lbgcs" Jul 07 11:16:41.692699 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:16:41.692674 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/eb34d7f1-0c6a-4b0a-92f6-e03dcf0a989d-prometheus-operator-tls\") pod \"prometheus-operator-567f7b8b7c-lbgcs\" (UID: \"eb34d7f1-0c6a-4b0a-92f6-e03dcf0a989d\") " pod="openshift-monitoring/prometheus-operator-567f7b8b7c-lbgcs" Jul 07 11:16:41.692797 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:16:41.692674 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/eb34d7f1-0c6a-4b0a-92f6-e03dcf0a989d-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-567f7b8b7c-lbgcs\" (UID: \"eb34d7f1-0c6a-4b0a-92f6-e03dcf0a989d\") " pod="openshift-monitoring/prometheus-operator-567f7b8b7c-lbgcs" Jul 07 11:16:41.696934 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:16:41.696915 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-7c8x4\" (UniqueName: \"kubernetes.io/projected/eb34d7f1-0c6a-4b0a-92f6-e03dcf0a989d-kube-api-access-7c8x4\") pod \"prometheus-operator-567f7b8b7c-lbgcs\" (UID: \"eb34d7f1-0c6a-4b0a-92f6-e03dcf0a989d\") " pod="openshift-monitoring/prometheus-operator-567f7b8b7c-lbgcs" Jul 07 11:16:41.763905 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:16:41.763874 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-567f7b8b7c-lbgcs" Jul 07 11:16:42.031531 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:16:42.031505 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-567f7b8b7c-lbgcs"] Jul 07 11:16:42.035240 ip-10-0-142-112 kubenswrapper[2578]: W0707 11:16:42.035213 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podeb34d7f1_0c6a_4b0a_92f6_e03dcf0a989d.slice/crio-a57c2950e299a4d65181ab389e0d3a7a03bd468c65478844547f5ecc5990c7a3 WatchSource:0}: Error finding container a57c2950e299a4d65181ab389e0d3a7a03bd468c65478844547f5ecc5990c7a3: Status 404 returned error can't find the container with id a57c2950e299a4d65181ab389e0d3a7a03bd468c65478844547f5ecc5990c7a3 Jul 07 11:16:42.522090 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:16:42.522056 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-567f7b8b7c-lbgcs" event={"ID":"eb34d7f1-0c6a-4b0a-92f6-e03dcf0a989d","Type":"ContainerStarted","Data":"a57c2950e299a4d65181ab389e0d3a7a03bd468c65478844547f5ecc5990c7a3"} Jul 07 11:16:42.523579 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:16:42.523556 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-kdmpx" event={"ID":"fba5ce69-3377-4a37-ba7c-6b3adb88b2f3","Type":"ContainerStarted","Data":"58edff2a65308b418bf6150e72cde18780f771056030028a7ab4dfacb1455883"} Jul 07 11:16:42.525320 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:16:42.525286 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-lgwzn" event={"ID":"12b4908e-0a68-467c-966f-a1900491e26c","Type":"ContainerStarted","Data":"084981ebdeb474ff95ab9bf0d847b1c5d450c007e588f6bf25f8eed54ae248ff"} Jul 07 11:16:42.525414 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:16:42.525330 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-lgwzn" event={"ID":"12b4908e-0a68-467c-966f-a1900491e26c","Type":"ContainerStarted","Data":"d44655985c9a6189921ecaa14397da0417670d225af979b45ab24716a80b16da"} Jul 07 11:16:42.525414 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:16:42.525363 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-lgwzn" Jul 07 11:16:42.526668 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:16:42.526649 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-9t57z" event={"ID":"bcd0ff4f-9b04-41d6-afef-018180f7dd2a","Type":"ContainerStarted","Data":"dbd4252d37657106bc4bf099b27946bc91088cb184488e8bc699970d60c4e365"} Jul 07 11:16:42.539997 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:16:42.539811 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-kdmpx" podStartSLOduration=1.60848323 podStartE2EDuration="7.539794429s" totalImagesPullingTime="5.931311199s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 11:16:35 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 11:16:42.539317156 +0000 UTC m=+46.835466744" watchObservedRunningTime="2026-07-07 11:16:42.539794429 +0000 UTC m=+46.835944017" Jul 07 11:16:42.557218 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:16:42.557170 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/dns-default-lgwzn" podStartSLOduration=1.621812935 podStartE2EDuration="7.557155299s" totalImagesPullingTime="5.935342364s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 11:16:35 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 11:16:42.555999111 +0000 UTC m=+46.852148701" watchObservedRunningTime="2026-07-07 11:16:42.557155299 +0000 UTC m=+46.853304917" Jul 07 11:16:43.432436 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:16:43.432392 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-86b7c49ddc-xg9tg"] Jul 07 11:16:43.453684 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:16:43.453657 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-86b7c49ddc-xg9tg"] Jul 07 11:16:43.453867 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:16:43.453772 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-86b7c49ddc-xg9tg" Jul 07 11:16:43.457165 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:16:43.457131 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"service-ca\"" Jul 07 11:16:43.457441 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:16:43.457402 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"console-config\"" Jul 07 11:16:43.457554 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:16:43.457498 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"openshift-service-ca.crt\"" Jul 07 11:16:43.462494 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:16:43.457941 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-oauth-config\"" Jul 07 11:16:43.462494 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:16:43.458236 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"oauth-serving-cert\"" Jul 07 11:16:43.462944 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:16:43.462914 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-serving-cert\"" Jul 07 11:16:43.463042 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:16:43.463000 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-dockercfg-mdb5z\"" Jul 07 11:16:43.464094 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:16:43.462926 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"kube-root-ca.crt\"" Jul 07 11:16:43.604442 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:16:43.604390 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/f52d638f-d46b-465b-a454-f34152acb425-console-serving-cert\") pod \"console-86b7c49ddc-xg9tg\" (UID: \"f52d638f-d46b-465b-a454-f34152acb425\") " pod="openshift-console/console-86b7c49ddc-xg9tg" Jul 07 11:16:43.604863 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:16:43.604452 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/f52d638f-d46b-465b-a454-f34152acb425-console-config\") pod \"console-86b7c49ddc-xg9tg\" (UID: \"f52d638f-d46b-465b-a454-f34152acb425\") " pod="openshift-console/console-86b7c49ddc-xg9tg" Jul 07 11:16:43.604863 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:16:43.604478 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/f52d638f-d46b-465b-a454-f34152acb425-oauth-serving-cert\") pod \"console-86b7c49ddc-xg9tg\" (UID: \"f52d638f-d46b-465b-a454-f34152acb425\") " pod="openshift-console/console-86b7c49ddc-xg9tg" Jul 07 11:16:43.604863 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:16:43.604507 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/f52d638f-d46b-465b-a454-f34152acb425-console-oauth-config\") pod \"console-86b7c49ddc-xg9tg\" (UID: \"f52d638f-d46b-465b-a454-f34152acb425\") " pod="openshift-console/console-86b7c49ddc-xg9tg" Jul 07 11:16:43.604863 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:16:43.604631 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-b4d7l\" (UniqueName: \"kubernetes.io/projected/f52d638f-d46b-465b-a454-f34152acb425-kube-api-access-b4d7l\") pod \"console-86b7c49ddc-xg9tg\" (UID: \"f52d638f-d46b-465b-a454-f34152acb425\") " pod="openshift-console/console-86b7c49ddc-xg9tg" Jul 07 11:16:43.604863 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:16:43.604686 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/f52d638f-d46b-465b-a454-f34152acb425-service-ca\") pod \"console-86b7c49ddc-xg9tg\" (UID: \"f52d638f-d46b-465b-a454-f34152acb425\") " pod="openshift-console/console-86b7c49ddc-xg9tg" Jul 07 11:16:43.705283 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:16:43.705263 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/f52d638f-d46b-465b-a454-f34152acb425-console-serving-cert\") pod \"console-86b7c49ddc-xg9tg\" (UID: \"f52d638f-d46b-465b-a454-f34152acb425\") " pod="openshift-console/console-86b7c49ddc-xg9tg" Jul 07 11:16:43.705337 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:16:43.705293 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/f52d638f-d46b-465b-a454-f34152acb425-console-config\") pod \"console-86b7c49ddc-xg9tg\" (UID: \"f52d638f-d46b-465b-a454-f34152acb425\") " pod="openshift-console/console-86b7c49ddc-xg9tg" Jul 07 11:16:43.705337 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:16:43.705314 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/f52d638f-d46b-465b-a454-f34152acb425-oauth-serving-cert\") pod \"console-86b7c49ddc-xg9tg\" (UID: \"f52d638f-d46b-465b-a454-f34152acb425\") " pod="openshift-console/console-86b7c49ddc-xg9tg" Jul 07 11:16:43.705444 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:16:43.705359 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/f52d638f-d46b-465b-a454-f34152acb425-console-oauth-config\") pod \"console-86b7c49ddc-xg9tg\" (UID: \"f52d638f-d46b-465b-a454-f34152acb425\") " pod="openshift-console/console-86b7c49ddc-xg9tg" Jul 07 11:16:43.705624 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:16:43.705603 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-b4d7l\" (UniqueName: \"kubernetes.io/projected/f52d638f-d46b-465b-a454-f34152acb425-kube-api-access-b4d7l\") pod \"console-86b7c49ddc-xg9tg\" (UID: \"f52d638f-d46b-465b-a454-f34152acb425\") " pod="openshift-console/console-86b7c49ddc-xg9tg" Jul 07 11:16:43.705725 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:16:43.705701 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/f52d638f-d46b-465b-a454-f34152acb425-service-ca\") pod \"console-86b7c49ddc-xg9tg\" (UID: \"f52d638f-d46b-465b-a454-f34152acb425\") " pod="openshift-console/console-86b7c49ddc-xg9tg" Jul 07 11:16:43.706124 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:16:43.706089 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/f52d638f-d46b-465b-a454-f34152acb425-console-config\") pod \"console-86b7c49ddc-xg9tg\" (UID: \"f52d638f-d46b-465b-a454-f34152acb425\") " pod="openshift-console/console-86b7c49ddc-xg9tg" Jul 07 11:16:43.706220 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:16:43.706199 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/f52d638f-d46b-465b-a454-f34152acb425-oauth-serving-cert\") pod \"console-86b7c49ddc-xg9tg\" (UID: \"f52d638f-d46b-465b-a454-f34152acb425\") " pod="openshift-console/console-86b7c49ddc-xg9tg" Jul 07 11:16:43.706398 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:16:43.706367 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/f52d638f-d46b-465b-a454-f34152acb425-service-ca\") pod \"console-86b7c49ddc-xg9tg\" (UID: \"f52d638f-d46b-465b-a454-f34152acb425\") " pod="openshift-console/console-86b7c49ddc-xg9tg" Jul 07 11:16:43.707932 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:16:43.707909 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/f52d638f-d46b-465b-a454-f34152acb425-console-serving-cert\") pod \"console-86b7c49ddc-xg9tg\" (UID: \"f52d638f-d46b-465b-a454-f34152acb425\") " pod="openshift-console/console-86b7c49ddc-xg9tg" Jul 07 11:16:43.708321 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:16:43.708304 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/f52d638f-d46b-465b-a454-f34152acb425-console-oauth-config\") pod \"console-86b7c49ddc-xg9tg\" (UID: \"f52d638f-d46b-465b-a454-f34152acb425\") " pod="openshift-console/console-86b7c49ddc-xg9tg" Jul 07 11:16:43.715103 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:16:43.715079 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-b4d7l\" (UniqueName: \"kubernetes.io/projected/f52d638f-d46b-465b-a454-f34152acb425-kube-api-access-b4d7l\") pod \"console-86b7c49ddc-xg9tg\" (UID: \"f52d638f-d46b-465b-a454-f34152acb425\") " pod="openshift-console/console-86b7c49ddc-xg9tg" Jul 07 11:16:43.771244 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:16:43.771202 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-86b7c49ddc-xg9tg" Jul 07 11:16:44.051062 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:16:44.051033 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-86b7c49ddc-xg9tg"] Jul 07 11:16:44.055080 ip-10-0-142-112 kubenswrapper[2578]: W0707 11:16:44.055053 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf52d638f_d46b_465b_a454_f34152acb425.slice/crio-079d0a4ea8427e831aca84566e16b8aacac87627f531ad2448a2871a2ee6566d WatchSource:0}: Error finding container 079d0a4ea8427e831aca84566e16b8aacac87627f531ad2448a2871a2ee6566d: Status 404 returned error can't find the container with id 079d0a4ea8427e831aca84566e16b8aacac87627f531ad2448a2871a2ee6566d Jul 07 11:16:44.535044 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:16:44.534996 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-567f7b8b7c-lbgcs" event={"ID":"eb34d7f1-0c6a-4b0a-92f6-e03dcf0a989d","Type":"ContainerStarted","Data":"67a49f4fc840dee6097138a2088202475d2c43619c9b7d2a26844eaa669f7e12"} Jul 07 11:16:44.535230 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:16:44.535073 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-567f7b8b7c-lbgcs" event={"ID":"eb34d7f1-0c6a-4b0a-92f6-e03dcf0a989d","Type":"ContainerStarted","Data":"73506528aa1134f38269b7f4287abff24e20f9d94c19f4c0c10988d51f04969b"} Jul 07 11:16:44.536227 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:16:44.536200 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-86b7c49ddc-xg9tg" event={"ID":"f52d638f-d46b-465b-a454-f34152acb425","Type":"ContainerStarted","Data":"079d0a4ea8427e831aca84566e16b8aacac87627f531ad2448a2871a2ee6566d"} Jul 07 11:16:44.538128 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:16:44.538107 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-9t57z" event={"ID":"bcd0ff4f-9b04-41d6-afef-018180f7dd2a","Type":"ContainerStarted","Data":"82843173f040943dad184233f68d61f7bf116a28fbf70e993ce7a111571c6e6f"} Jul 07 11:16:44.554438 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:16:44.554377 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-operator-567f7b8b7c-lbgcs" podStartSLOduration=1.66209014 podStartE2EDuration="3.554363491s" totalImagesPullingTime="1.892273351s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 11:16:41 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 11:16:44.553374471 +0000 UTC m=+48.849524060" watchObservedRunningTime="2026-07-07 11:16:44.554363491 +0000 UTC m=+48.850513079" Jul 07 11:16:47.121569 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:16:47.121484 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-98hzz"] Jul 07 11:16:47.138599 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:16:47.138571 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/kube-state-metrics-7c9dbb8b7d-rqcp8"] Jul 07 11:16:47.138728 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:16:47.138677 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-98hzz" Jul 07 11:16:47.142729 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:16:47.142683 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-hdwhd\"" Jul 07 11:16:47.142916 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:16:47.142895 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\"" Jul 07 11:16:47.143082 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:16:47.143067 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\"" Jul 07 11:16:47.143189 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:16:47.143167 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\"" Jul 07 11:16:47.161641 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:16:47.161616 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/kube-state-metrics-7c9dbb8b7d-rqcp8"] Jul 07 11:16:47.161757 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:16:47.161730 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-rqcp8" Jul 07 11:16:47.166033 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:16:47.166013 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-tls\"" Jul 07 11:16:47.166272 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:16:47.166252 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-kube-rbac-proxy-config\"" Jul 07 11:16:47.166520 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:16:47.166499 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-custom-resource-state-configmap\"" Jul 07 11:16:47.166675 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:16:47.166494 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-dockercfg-8s5zr\"" Jul 07 11:16:47.237817 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:16:47.237775 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/6ef2378c-fd55-4ba4-b260-86c732ed130b-node-exporter-wtmp\") pod \"node-exporter-98hzz\" (UID: \"6ef2378c-fd55-4ba4-b260-86c732ed130b\") " pod="openshift-monitoring/node-exporter-98hzz" Jul 07 11:16:47.237817 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:16:47.237808 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/6ef2378c-fd55-4ba4-b260-86c732ed130b-metrics-client-ca\") pod \"node-exporter-98hzz\" (UID: \"6ef2378c-fd55-4ba4-b260-86c732ed130b\") " pod="openshift-monitoring/node-exporter-98hzz" Jul 07 11:16:47.238063 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:16:47.237841 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/6ef2378c-fd55-4ba4-b260-86c732ed130b-node-exporter-accelerators-collector-config\") pod \"node-exporter-98hzz\" (UID: \"6ef2378c-fd55-4ba4-b260-86c732ed130b\") " pod="openshift-monitoring/node-exporter-98hzz" Jul 07 11:16:47.238063 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:16:47.237912 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4dnwm\" (UniqueName: \"kubernetes.io/projected/6ef2378c-fd55-4ba4-b260-86c732ed130b-kube-api-access-4dnwm\") pod \"node-exporter-98hzz\" (UID: \"6ef2378c-fd55-4ba4-b260-86c732ed130b\") " pod="openshift-monitoring/node-exporter-98hzz" Jul 07 11:16:47.238063 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:16:47.237944 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/6ef2378c-fd55-4ba4-b260-86c732ed130b-node-exporter-tls\") pod \"node-exporter-98hzz\" (UID: \"6ef2378c-fd55-4ba4-b260-86c732ed130b\") " pod="openshift-monitoring/node-exporter-98hzz" Jul 07 11:16:47.238063 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:16:47.237975 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/6ef2378c-fd55-4ba4-b260-86c732ed130b-node-exporter-textfile\") pod \"node-exporter-98hzz\" (UID: \"6ef2378c-fd55-4ba4-b260-86c732ed130b\") " pod="openshift-monitoring/node-exporter-98hzz" Jul 07 11:16:47.238063 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:16:47.238030 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/6ef2378c-fd55-4ba4-b260-86c732ed130b-root\") pod \"node-exporter-98hzz\" (UID: \"6ef2378c-fd55-4ba4-b260-86c732ed130b\") " pod="openshift-monitoring/node-exporter-98hzz" Jul 07 11:16:47.238489 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:16:47.238066 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/6ef2378c-fd55-4ba4-b260-86c732ed130b-sys\") pod \"node-exporter-98hzz\" (UID: \"6ef2378c-fd55-4ba4-b260-86c732ed130b\") " pod="openshift-monitoring/node-exporter-98hzz" Jul 07 11:16:47.238489 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:16:47.238099 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/6ef2378c-fd55-4ba4-b260-86c732ed130b-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-98hzz\" (UID: \"6ef2378c-fd55-4ba4-b260-86c732ed130b\") " pod="openshift-monitoring/node-exporter-98hzz" Jul 07 11:16:47.339310 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:16:47.339267 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/c82e9536-2e0e-47a3-9573-d7acd8ade54a-volume-directive-shadow\") pod \"kube-state-metrics-7c9dbb8b7d-rqcp8\" (UID: \"c82e9536-2e0e-47a3-9573-d7acd8ade54a\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-rqcp8" Jul 07 11:16:47.339310 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:16:47.339321 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4dnwm\" (UniqueName: \"kubernetes.io/projected/6ef2378c-fd55-4ba4-b260-86c732ed130b-kube-api-access-4dnwm\") pod \"node-exporter-98hzz\" (UID: \"6ef2378c-fd55-4ba4-b260-86c732ed130b\") " pod="openshift-monitoring/node-exporter-98hzz" Jul 07 11:16:47.339579 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:16:47.339413 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/6ef2378c-fd55-4ba4-b260-86c732ed130b-node-exporter-tls\") pod \"node-exporter-98hzz\" (UID: \"6ef2378c-fd55-4ba4-b260-86c732ed130b\") " pod="openshift-monitoring/node-exporter-98hzz" Jul 07 11:16:47.339579 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:16:47.339462 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/c82e9536-2e0e-47a3-9573-d7acd8ade54a-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-7c9dbb8b7d-rqcp8\" (UID: \"c82e9536-2e0e-47a3-9573-d7acd8ade54a\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-rqcp8" Jul 07 11:16:47.339579 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:16:47.339496 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/6ef2378c-fd55-4ba4-b260-86c732ed130b-node-exporter-textfile\") pod \"node-exporter-98hzz\" (UID: \"6ef2378c-fd55-4ba4-b260-86c732ed130b\") " pod="openshift-monitoring/node-exporter-98hzz" Jul 07 11:16:47.339579 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:16:47.339553 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7fs7r\" (UniqueName: \"kubernetes.io/projected/c82e9536-2e0e-47a3-9573-d7acd8ade54a-kube-api-access-7fs7r\") pod \"kube-state-metrics-7c9dbb8b7d-rqcp8\" (UID: \"c82e9536-2e0e-47a3-9573-d7acd8ade54a\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-rqcp8" Jul 07 11:16:47.339788 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:16:47.339587 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/6ef2378c-fd55-4ba4-b260-86c732ed130b-root\") pod \"node-exporter-98hzz\" (UID: \"6ef2378c-fd55-4ba4-b260-86c732ed130b\") " pod="openshift-monitoring/node-exporter-98hzz" Jul 07 11:16:47.339788 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:16:47.339668 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/6ef2378c-fd55-4ba4-b260-86c732ed130b-sys\") pod \"node-exporter-98hzz\" (UID: \"6ef2378c-fd55-4ba4-b260-86c732ed130b\") " pod="openshift-monitoring/node-exporter-98hzz" Jul 07 11:16:47.339788 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:16:47.339679 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/6ef2378c-fd55-4ba4-b260-86c732ed130b-root\") pod \"node-exporter-98hzz\" (UID: \"6ef2378c-fd55-4ba4-b260-86c732ed130b\") " pod="openshift-monitoring/node-exporter-98hzz" Jul 07 11:16:47.339788 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:16:47.339722 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/c82e9536-2e0e-47a3-9573-d7acd8ade54a-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-7c9dbb8b7d-rqcp8\" (UID: \"c82e9536-2e0e-47a3-9573-d7acd8ade54a\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-rqcp8" Jul 07 11:16:47.339788 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:16:47.339731 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/6ef2378c-fd55-4ba4-b260-86c732ed130b-sys\") pod \"node-exporter-98hzz\" (UID: \"6ef2378c-fd55-4ba4-b260-86c732ed130b\") " pod="openshift-monitoring/node-exporter-98hzz" Jul 07 11:16:47.339788 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:16:47.339765 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/6ef2378c-fd55-4ba4-b260-86c732ed130b-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-98hzz\" (UID: \"6ef2378c-fd55-4ba4-b260-86c732ed130b\") " pod="openshift-monitoring/node-exporter-98hzz" Jul 07 11:16:47.340072 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:16:47.339794 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/c82e9536-2e0e-47a3-9573-d7acd8ade54a-metrics-client-ca\") pod \"kube-state-metrics-7c9dbb8b7d-rqcp8\" (UID: \"c82e9536-2e0e-47a3-9573-d7acd8ade54a\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-rqcp8" Jul 07 11:16:47.340072 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:16:47.339815 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/6ef2378c-fd55-4ba4-b260-86c732ed130b-node-exporter-wtmp\") pod \"node-exporter-98hzz\" (UID: \"6ef2378c-fd55-4ba4-b260-86c732ed130b\") " pod="openshift-monitoring/node-exporter-98hzz" Jul 07 11:16:47.340072 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:16:47.339828 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/6ef2378c-fd55-4ba4-b260-86c732ed130b-node-exporter-textfile\") pod \"node-exporter-98hzz\" (UID: \"6ef2378c-fd55-4ba4-b260-86c732ed130b\") " pod="openshift-monitoring/node-exporter-98hzz" Jul 07 11:16:47.340072 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:16:47.339836 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/6ef2378c-fd55-4ba4-b260-86c732ed130b-metrics-client-ca\") pod \"node-exporter-98hzz\" (UID: \"6ef2378c-fd55-4ba4-b260-86c732ed130b\") " pod="openshift-monitoring/node-exporter-98hzz" Jul 07 11:16:47.340072 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:16:47.339891 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/6ef2378c-fd55-4ba4-b260-86c732ed130b-node-exporter-accelerators-collector-config\") pod \"node-exporter-98hzz\" (UID: \"6ef2378c-fd55-4ba4-b260-86c732ed130b\") " pod="openshift-monitoring/node-exporter-98hzz" Jul 07 11:16:47.340072 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:16:47.339917 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/c82e9536-2e0e-47a3-9573-d7acd8ade54a-kube-state-metrics-tls\") pod \"kube-state-metrics-7c9dbb8b7d-rqcp8\" (UID: \"c82e9536-2e0e-47a3-9573-d7acd8ade54a\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-rqcp8" Jul 07 11:16:47.340072 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:16:47.339986 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/6ef2378c-fd55-4ba4-b260-86c732ed130b-node-exporter-wtmp\") pod \"node-exporter-98hzz\" (UID: \"6ef2378c-fd55-4ba4-b260-86c732ed130b\") " pod="openshift-monitoring/node-exporter-98hzz" Jul 07 11:16:47.340477 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:16:47.340457 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/6ef2378c-fd55-4ba4-b260-86c732ed130b-metrics-client-ca\") pod \"node-exporter-98hzz\" (UID: \"6ef2378c-fd55-4ba4-b260-86c732ed130b\") " pod="openshift-monitoring/node-exporter-98hzz" Jul 07 11:16:47.340546 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:16:47.340496 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/6ef2378c-fd55-4ba4-b260-86c732ed130b-node-exporter-accelerators-collector-config\") pod \"node-exporter-98hzz\" (UID: \"6ef2378c-fd55-4ba4-b260-86c732ed130b\") " pod="openshift-monitoring/node-exporter-98hzz" Jul 07 11:16:47.342196 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:16:47.342174 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/6ef2378c-fd55-4ba4-b260-86c732ed130b-node-exporter-tls\") pod \"node-exporter-98hzz\" (UID: \"6ef2378c-fd55-4ba4-b260-86c732ed130b\") " pod="openshift-monitoring/node-exporter-98hzz" Jul 07 11:16:47.342196 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:16:47.342185 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/6ef2378c-fd55-4ba4-b260-86c732ed130b-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-98hzz\" (UID: \"6ef2378c-fd55-4ba4-b260-86c732ed130b\") " pod="openshift-monitoring/node-exporter-98hzz" Jul 07 11:16:47.348486 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:16:47.348459 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-4dnwm\" (UniqueName: \"kubernetes.io/projected/6ef2378c-fd55-4ba4-b260-86c732ed130b-kube-api-access-4dnwm\") pod \"node-exporter-98hzz\" (UID: \"6ef2378c-fd55-4ba4-b260-86c732ed130b\") " pod="openshift-monitoring/node-exporter-98hzz" Jul 07 11:16:47.440953 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:16:47.440862 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7fs7r\" (UniqueName: \"kubernetes.io/projected/c82e9536-2e0e-47a3-9573-d7acd8ade54a-kube-api-access-7fs7r\") pod \"kube-state-metrics-7c9dbb8b7d-rqcp8\" (UID: \"c82e9536-2e0e-47a3-9573-d7acd8ade54a\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-rqcp8" Jul 07 11:16:47.441118 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:16:47.440951 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/c82e9536-2e0e-47a3-9573-d7acd8ade54a-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-7c9dbb8b7d-rqcp8\" (UID: \"c82e9536-2e0e-47a3-9573-d7acd8ade54a\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-rqcp8" Jul 07 11:16:47.441118 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:16:47.440988 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/c82e9536-2e0e-47a3-9573-d7acd8ade54a-metrics-client-ca\") pod \"kube-state-metrics-7c9dbb8b7d-rqcp8\" (UID: \"c82e9536-2e0e-47a3-9573-d7acd8ade54a\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-rqcp8" Jul 07 11:16:47.441118 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:16:47.441025 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/c82e9536-2e0e-47a3-9573-d7acd8ade54a-kube-state-metrics-tls\") pod \"kube-state-metrics-7c9dbb8b7d-rqcp8\" (UID: \"c82e9536-2e0e-47a3-9573-d7acd8ade54a\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-rqcp8" Jul 07 11:16:47.441118 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:16:47.441060 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/c82e9536-2e0e-47a3-9573-d7acd8ade54a-volume-directive-shadow\") pod \"kube-state-metrics-7c9dbb8b7d-rqcp8\" (UID: \"c82e9536-2e0e-47a3-9573-d7acd8ade54a\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-rqcp8" Jul 07 11:16:47.441118 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:16:47.441106 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/c82e9536-2e0e-47a3-9573-d7acd8ade54a-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-7c9dbb8b7d-rqcp8\" (UID: \"c82e9536-2e0e-47a3-9573-d7acd8ade54a\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-rqcp8" Jul 07 11:16:47.441714 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:16:47.441679 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/c82e9536-2e0e-47a3-9573-d7acd8ade54a-volume-directive-shadow\") pod \"kube-state-metrics-7c9dbb8b7d-rqcp8\" (UID: \"c82e9536-2e0e-47a3-9573-d7acd8ade54a\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-rqcp8" Jul 07 11:16:47.441837 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:16:47.441784 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/c82e9536-2e0e-47a3-9573-d7acd8ade54a-metrics-client-ca\") pod \"kube-state-metrics-7c9dbb8b7d-rqcp8\" (UID: \"c82e9536-2e0e-47a3-9573-d7acd8ade54a\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-rqcp8" Jul 07 11:16:47.441948 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:16:47.441844 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/c82e9536-2e0e-47a3-9573-d7acd8ade54a-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-7c9dbb8b7d-rqcp8\" (UID: \"c82e9536-2e0e-47a3-9573-d7acd8ade54a\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-rqcp8" Jul 07 11:16:47.443878 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:16:47.443858 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/c82e9536-2e0e-47a3-9573-d7acd8ade54a-kube-state-metrics-tls\") pod \"kube-state-metrics-7c9dbb8b7d-rqcp8\" (UID: \"c82e9536-2e0e-47a3-9573-d7acd8ade54a\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-rqcp8" Jul 07 11:16:47.443977 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:16:47.443933 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/c82e9536-2e0e-47a3-9573-d7acd8ade54a-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-7c9dbb8b7d-rqcp8\" (UID: \"c82e9536-2e0e-47a3-9573-d7acd8ade54a\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-rqcp8" Jul 07 11:16:47.449585 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:16:47.449532 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-98hzz" Jul 07 11:16:47.450719 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:16:47.450697 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-7fs7r\" (UniqueName: \"kubernetes.io/projected/c82e9536-2e0e-47a3-9573-d7acd8ade54a-kube-api-access-7fs7r\") pod \"kube-state-metrics-7c9dbb8b7d-rqcp8\" (UID: \"c82e9536-2e0e-47a3-9573-d7acd8ade54a\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-rqcp8" Jul 07 11:16:47.469923 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:16:47.469901 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-rqcp8" Jul 07 11:16:47.550912 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:16:47.550875 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-9t57z" event={"ID":"bcd0ff4f-9b04-41d6-afef-018180f7dd2a","Type":"ContainerStarted","Data":"ecbc2977947ab9cb15b70f01e0ece4817a87985fb73fc0c72cf668275c0bcdb5"} Jul 07 11:16:47.591561 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:16:47.590742 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-9t57z" podStartSLOduration=1.6680344809999998 podStartE2EDuration="12.590719497s" totalImagesPullingTime="10.922685016s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-07 11:16:35 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 11:16:47.589041275 +0000 UTC m=+51.885190863" watchObservedRunningTime="2026-07-07 11:16:47.590719497 +0000 UTC m=+51.886869089" Jul 07 11:16:48.287854 ip-10-0-142-112 kubenswrapper[2578]: W0707 11:16:48.287823 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6ef2378c_fd55_4ba4_b260_86c732ed130b.slice/crio-2ec274ba626c1f6c53663ba413fe0c7a81ec0671a4578189ddda2d21eaedce42 WatchSource:0}: Error finding container 2ec274ba626c1f6c53663ba413fe0c7a81ec0671a4578189ddda2d21eaedce42: Status 404 returned error can't find the container with id 2ec274ba626c1f6c53663ba413fe0c7a81ec0671a4578189ddda2d21eaedce42 Jul 07 11:16:48.406270 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:16:48.406246 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/kube-state-metrics-7c9dbb8b7d-rqcp8"] Jul 07 11:16:48.408362 ip-10-0-142-112 kubenswrapper[2578]: W0707 11:16:48.408336 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc82e9536_2e0e_47a3_9573_d7acd8ade54a.slice/crio-9b83571e70d36dace8991c1654a37c15a4e689ff5e0c78b0a27d320669f87274 WatchSource:0}: Error finding container 9b83571e70d36dace8991c1654a37c15a4e689ff5e0c78b0a27d320669f87274: Status 404 returned error can't find the container with id 9b83571e70d36dace8991c1654a37c15a4e689ff5e0c78b0a27d320669f87274 Jul 07 11:16:48.554702 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:16:48.554667 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-rqcp8" event={"ID":"c82e9536-2e0e-47a3-9573-d7acd8ade54a","Type":"ContainerStarted","Data":"9b83571e70d36dace8991c1654a37c15a4e689ff5e0c78b0a27d320669f87274"} Jul 07 11:16:48.555753 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:16:48.555724 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-98hzz" event={"ID":"6ef2378c-fd55-4ba4-b260-86c732ed130b","Type":"ContainerStarted","Data":"2ec274ba626c1f6c53663ba413fe0c7a81ec0671a4578189ddda2d21eaedce42"} Jul 07 11:16:49.099936 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:16:49.099901 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/thanos-querier-7b5486658-ncz65"] Jul 07 11:16:49.119619 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:16:49.119583 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/thanos-querier-7b5486658-ncz65"] Jul 07 11:16:49.119769 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:16:49.119700 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/thanos-querier-7b5486658-ncz65" Jul 07 11:16:49.122194 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:16:49.122144 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-rules\"" Jul 07 11:16:49.122319 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:16:49.122195 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-metrics\"" Jul 07 11:16:49.122319 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:16:49.122201 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-dockercfg-vllfr\"" Jul 07 11:16:49.122319 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:16:49.122217 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy\"" Jul 07 11:16:49.122319 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:16:49.122213 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-grpc-tls-e2382s2ubqttb\"" Jul 07 11:16:49.122858 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:16:49.122843 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-tls\"" Jul 07 11:16:49.122899 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:16:49.122843 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-web\"" Jul 07 11:16:49.256111 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:16:49.256071 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/7a646b22-d6b9-448c-a618-78a5edf13a17-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-7b5486658-ncz65\" (UID: \"7a646b22-d6b9-448c-a618-78a5edf13a17\") " pod="openshift-monitoring/thanos-querier-7b5486658-ncz65" Jul 07 11:16:49.256298 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:16:49.256121 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ls5sz\" (UniqueName: \"kubernetes.io/projected/7a646b22-d6b9-448c-a618-78a5edf13a17-kube-api-access-ls5sz\") pod \"thanos-querier-7b5486658-ncz65\" (UID: \"7a646b22-d6b9-448c-a618-78a5edf13a17\") " pod="openshift-monitoring/thanos-querier-7b5486658-ncz65" Jul 07 11:16:49.256298 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:16:49.256204 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/7a646b22-d6b9-448c-a618-78a5edf13a17-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-7b5486658-ncz65\" (UID: \"7a646b22-d6b9-448c-a618-78a5edf13a17\") " pod="openshift-monitoring/thanos-querier-7b5486658-ncz65" Jul 07 11:16:49.256298 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:16:49.256233 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/7a646b22-d6b9-448c-a618-78a5edf13a17-secret-thanos-querier-tls\") pod \"thanos-querier-7b5486658-ncz65\" (UID: \"7a646b22-d6b9-448c-a618-78a5edf13a17\") " pod="openshift-monitoring/thanos-querier-7b5486658-ncz65" Jul 07 11:16:49.256298 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:16:49.256291 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/7a646b22-d6b9-448c-a618-78a5edf13a17-secret-grpc-tls\") pod \"thanos-querier-7b5486658-ncz65\" (UID: \"7a646b22-d6b9-448c-a618-78a5edf13a17\") " pod="openshift-monitoring/thanos-querier-7b5486658-ncz65" Jul 07 11:16:49.256525 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:16:49.256340 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/7a646b22-d6b9-448c-a618-78a5edf13a17-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-7b5486658-ncz65\" (UID: \"7a646b22-d6b9-448c-a618-78a5edf13a17\") " pod="openshift-monitoring/thanos-querier-7b5486658-ncz65" Jul 07 11:16:49.256525 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:16:49.256371 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/7a646b22-d6b9-448c-a618-78a5edf13a17-metrics-client-ca\") pod \"thanos-querier-7b5486658-ncz65\" (UID: \"7a646b22-d6b9-448c-a618-78a5edf13a17\") " pod="openshift-monitoring/thanos-querier-7b5486658-ncz65" Jul 07 11:16:49.256525 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:16:49.256400 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/7a646b22-d6b9-448c-a618-78a5edf13a17-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-7b5486658-ncz65\" (UID: \"7a646b22-d6b9-448c-a618-78a5edf13a17\") " pod="openshift-monitoring/thanos-querier-7b5486658-ncz65" Jul 07 11:16:49.357575 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:16:49.357474 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/7a646b22-d6b9-448c-a618-78a5edf13a17-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-7b5486658-ncz65\" (UID: \"7a646b22-d6b9-448c-a618-78a5edf13a17\") " pod="openshift-monitoring/thanos-querier-7b5486658-ncz65" Jul 07 11:16:49.357575 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:16:49.357532 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/7a646b22-d6b9-448c-a618-78a5edf13a17-secret-thanos-querier-tls\") pod \"thanos-querier-7b5486658-ncz65\" (UID: \"7a646b22-d6b9-448c-a618-78a5edf13a17\") " pod="openshift-monitoring/thanos-querier-7b5486658-ncz65" Jul 07 11:16:49.357575 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:16:49.357568 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/7a646b22-d6b9-448c-a618-78a5edf13a17-secret-grpc-tls\") pod \"thanos-querier-7b5486658-ncz65\" (UID: \"7a646b22-d6b9-448c-a618-78a5edf13a17\") " pod="openshift-monitoring/thanos-querier-7b5486658-ncz65" Jul 07 11:16:49.358105 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:16:49.357612 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/7a646b22-d6b9-448c-a618-78a5edf13a17-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-7b5486658-ncz65\" (UID: \"7a646b22-d6b9-448c-a618-78a5edf13a17\") " pod="openshift-monitoring/thanos-querier-7b5486658-ncz65" Jul 07 11:16:49.358105 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:16:49.357646 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/7a646b22-d6b9-448c-a618-78a5edf13a17-metrics-client-ca\") pod \"thanos-querier-7b5486658-ncz65\" (UID: \"7a646b22-d6b9-448c-a618-78a5edf13a17\") " pod="openshift-monitoring/thanos-querier-7b5486658-ncz65" Jul 07 11:16:49.358105 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:16:49.357682 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/7a646b22-d6b9-448c-a618-78a5edf13a17-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-7b5486658-ncz65\" (UID: \"7a646b22-d6b9-448c-a618-78a5edf13a17\") " pod="openshift-monitoring/thanos-querier-7b5486658-ncz65" Jul 07 11:16:49.358105 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:16:49.357749 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/7a646b22-d6b9-448c-a618-78a5edf13a17-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-7b5486658-ncz65\" (UID: \"7a646b22-d6b9-448c-a618-78a5edf13a17\") " pod="openshift-monitoring/thanos-querier-7b5486658-ncz65" Jul 07 11:16:49.358105 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:16:49.357773 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ls5sz\" (UniqueName: \"kubernetes.io/projected/7a646b22-d6b9-448c-a618-78a5edf13a17-kube-api-access-ls5sz\") pod \"thanos-querier-7b5486658-ncz65\" (UID: \"7a646b22-d6b9-448c-a618-78a5edf13a17\") " pod="openshift-monitoring/thanos-querier-7b5486658-ncz65" Jul 07 11:16:49.359407 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:16:49.359320 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/7a646b22-d6b9-448c-a618-78a5edf13a17-metrics-client-ca\") pod \"thanos-querier-7b5486658-ncz65\" (UID: \"7a646b22-d6b9-448c-a618-78a5edf13a17\") " pod="openshift-monitoring/thanos-querier-7b5486658-ncz65" Jul 07 11:16:49.361241 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:16:49.361195 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/7a646b22-d6b9-448c-a618-78a5edf13a17-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-7b5486658-ncz65\" (UID: \"7a646b22-d6b9-448c-a618-78a5edf13a17\") " pod="openshift-monitoring/thanos-querier-7b5486658-ncz65" Jul 07 11:16:49.362102 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:16:49.362063 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/7a646b22-d6b9-448c-a618-78a5edf13a17-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-7b5486658-ncz65\" (UID: \"7a646b22-d6b9-448c-a618-78a5edf13a17\") " pod="openshift-monitoring/thanos-querier-7b5486658-ncz65" Jul 07 11:16:49.362309 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:16:49.362287 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/7a646b22-d6b9-448c-a618-78a5edf13a17-secret-grpc-tls\") pod \"thanos-querier-7b5486658-ncz65\" (UID: \"7a646b22-d6b9-448c-a618-78a5edf13a17\") " pod="openshift-monitoring/thanos-querier-7b5486658-ncz65" Jul 07 11:16:49.362573 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:16:49.362533 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/7a646b22-d6b9-448c-a618-78a5edf13a17-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-7b5486658-ncz65\" (UID: \"7a646b22-d6b9-448c-a618-78a5edf13a17\") " pod="openshift-monitoring/thanos-querier-7b5486658-ncz65" Jul 07 11:16:49.363220 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:16:49.363200 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/7a646b22-d6b9-448c-a618-78a5edf13a17-secret-thanos-querier-tls\") pod \"thanos-querier-7b5486658-ncz65\" (UID: \"7a646b22-d6b9-448c-a618-78a5edf13a17\") " pod="openshift-monitoring/thanos-querier-7b5486658-ncz65" Jul 07 11:16:49.363630 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:16:49.363577 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/7a646b22-d6b9-448c-a618-78a5edf13a17-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-7b5486658-ncz65\" (UID: \"7a646b22-d6b9-448c-a618-78a5edf13a17\") " pod="openshift-monitoring/thanos-querier-7b5486658-ncz65" Jul 07 11:16:49.366785 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:16:49.366763 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-ls5sz\" (UniqueName: \"kubernetes.io/projected/7a646b22-d6b9-448c-a618-78a5edf13a17-kube-api-access-ls5sz\") pod \"thanos-querier-7b5486658-ncz65\" (UID: \"7a646b22-d6b9-448c-a618-78a5edf13a17\") " pod="openshift-monitoring/thanos-querier-7b5486658-ncz65" Jul 07 11:16:49.428633 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:16:49.428586 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/thanos-querier-7b5486658-ncz65" Jul 07 11:16:49.561652 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:16:49.561574 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-86b7c49ddc-xg9tg" event={"ID":"f52d638f-d46b-465b-a454-f34152acb425","Type":"ContainerStarted","Data":"f444cd29359a51a26ea3285a565412ae16e675287bce7c9ed50c1c9273a60599"} Jul 07 11:16:49.580710 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:16:49.580618 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-86b7c49ddc-xg9tg" podStartSLOduration=2.044908511 podStartE2EDuration="6.580597382s" totalImagesPullingTime="4.535688871s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 11:16:43 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 11:16:49.579270722 +0000 UTC m=+53.875420311" watchObservedRunningTime="2026-07-07 11:16:49.580597382 +0000 UTC m=+53.876746973" Jul 07 11:16:49.630580 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:16:49.630498 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/thanos-querier-7b5486658-ncz65"] Jul 07 11:16:49.634396 ip-10-0-142-112 kubenswrapper[2578]: W0707 11:16:49.634367 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7a646b22_d6b9_448c_a618_78a5edf13a17.slice/crio-cc5f91842153172f81ef96d133526493d694085959c228280120d74e11ba9423 WatchSource:0}: Error finding container cc5f91842153172f81ef96d133526493d694085959c228280120d74e11ba9423: Status 404 returned error can't find the container with id cc5f91842153172f81ef96d133526493d694085959c228280120d74e11ba9423 Jul 07 11:16:50.566536 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:16:50.566498 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-7b5486658-ncz65" event={"ID":"7a646b22-d6b9-448c-a618-78a5edf13a17","Type":"ContainerStarted","Data":"cc5f91842153172f81ef96d133526493d694085959c228280120d74e11ba9423"} Jul 07 11:16:50.568403 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:16:50.568370 2578 generic.go:356] "Generic (PLEG): container finished" podID="6ef2378c-fd55-4ba4-b260-86c732ed130b" containerID="a7be40447804b2c5cf287a3b52852d90a9a8373de4e8f4efe52440ec65ed9c73" exitCode=0 Jul 07 11:16:50.568545 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:16:50.568461 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-98hzz" event={"ID":"6ef2378c-fd55-4ba4-b260-86c732ed130b","Type":"ContainerDied","Data":"a7be40447804b2c5cf287a3b52852d90a9a8373de4e8f4efe52440ec65ed9c73"} Jul 07 11:16:51.573900 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:16:51.573862 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-rqcp8" event={"ID":"c82e9536-2e0e-47a3-9573-d7acd8ade54a","Type":"ContainerStarted","Data":"d8cf5be4668212d9cdf5b3a4f8194d5619808f2c2d2749c7bb8077b87e1dcafc"} Jul 07 11:16:51.574488 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:16:51.573939 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-rqcp8" event={"ID":"c82e9536-2e0e-47a3-9573-d7acd8ade54a","Type":"ContainerStarted","Data":"a882851a83ebc877f4b10deb8622a3431e0f17694b0f63388f4bf056473b2a8d"} Jul 07 11:16:51.574488 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:16:51.573969 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-rqcp8" event={"ID":"c82e9536-2e0e-47a3-9573-d7acd8ade54a","Type":"ContainerStarted","Data":"cfaa3090845fcd119f02788c28c5e7ea37a1f46b5d66f127cc1ba9fe5e14d533"} Jul 07 11:16:51.576317 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:16:51.576290 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-98hzz" event={"ID":"6ef2378c-fd55-4ba4-b260-86c732ed130b","Type":"ContainerStarted","Data":"6e5edafcc7df2fabb483ef184a6f80de777cd54928548a14ca2f9df6e8084627"} Jul 07 11:16:51.576451 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:16:51.576341 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-98hzz" event={"ID":"6ef2378c-fd55-4ba4-b260-86c732ed130b","Type":"ContainerStarted","Data":"60f1b7c004065529144efdba0ed9008f0af88f5bd5d4a2f162ddd60f8ed737b9"} Jul 07 11:16:51.598671 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:16:51.598623 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-rqcp8" podStartSLOduration=2.522719345 podStartE2EDuration="4.598607976s" totalImagesPullingTime="2.075888631s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 11:16:47 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 11:16:51.596833325 +0000 UTC m=+55.892982933" watchObservedRunningTime="2026-07-07 11:16:51.598607976 +0000 UTC m=+55.894757563" Jul 07 11:16:51.629098 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:16:51.629040 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-98hzz" podStartSLOduration=2.609635876 podStartE2EDuration="4.629016317s" totalImagesPullingTime="1.92141195s" totalInitContainerRuntime="97.968491ms" isStatefulPod=true podCreationTimestamp="2026-07-07 11:16:47 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 11:16:51.628646953 +0000 UTC m=+55.924796542" watchObservedRunningTime="2026-07-07 11:16:51.629016317 +0000 UTC m=+55.925165905" Jul 07 11:16:51.665321 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:16:51.665288 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/monitoring-plugin-d5dcff7cf-tn4c9"] Jul 07 11:16:51.667870 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:16:51.667847 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/monitoring-plugin-d5dcff7cf-tn4c9" Jul 07 11:16:51.670513 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:16:51.670306 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"monitoring-plugin-cert\"" Jul 07 11:16:51.670513 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:16:51.670403 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"monitoring-plugin-dockercfg-l4t9d\"" Jul 07 11:16:51.681924 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:16:51.681904 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/monitoring-plugin-d5dcff7cf-tn4c9"] Jul 07 11:16:51.776353 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:16:51.776322 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cvptg\" (UniqueName: \"kubernetes.io/projected/5719f623-3be7-4a90-ae42-ac58bc2e1614-kube-api-access-cvptg\") pod \"monitoring-plugin-d5dcff7cf-tn4c9\" (UID: \"5719f623-3be7-4a90-ae42-ac58bc2e1614\") " pod="openshift-monitoring/monitoring-plugin-d5dcff7cf-tn4c9" Jul 07 11:16:51.776542 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:16:51.776365 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/5719f623-3be7-4a90-ae42-ac58bc2e1614-monitoring-plugin-cert\") pod \"monitoring-plugin-d5dcff7cf-tn4c9\" (UID: \"5719f623-3be7-4a90-ae42-ac58bc2e1614\") " pod="openshift-monitoring/monitoring-plugin-d5dcff7cf-tn4c9" Jul 07 11:16:51.877544 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:16:51.877469 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cvptg\" (UniqueName: \"kubernetes.io/projected/5719f623-3be7-4a90-ae42-ac58bc2e1614-kube-api-access-cvptg\") pod \"monitoring-plugin-d5dcff7cf-tn4c9\" (UID: \"5719f623-3be7-4a90-ae42-ac58bc2e1614\") " pod="openshift-monitoring/monitoring-plugin-d5dcff7cf-tn4c9" Jul 07 11:16:51.877544 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:16:51.877509 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/5719f623-3be7-4a90-ae42-ac58bc2e1614-monitoring-plugin-cert\") pod \"monitoring-plugin-d5dcff7cf-tn4c9\" (UID: \"5719f623-3be7-4a90-ae42-ac58bc2e1614\") " pod="openshift-monitoring/monitoring-plugin-d5dcff7cf-tn4c9" Jul 07 11:16:51.880408 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:16:51.880382 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/5719f623-3be7-4a90-ae42-ac58bc2e1614-monitoring-plugin-cert\") pod \"monitoring-plugin-d5dcff7cf-tn4c9\" (UID: \"5719f623-3be7-4a90-ae42-ac58bc2e1614\") " pod="openshift-monitoring/monitoring-plugin-d5dcff7cf-tn4c9" Jul 07 11:16:51.888500 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:16:51.888477 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-cvptg\" (UniqueName: \"kubernetes.io/projected/5719f623-3be7-4a90-ae42-ac58bc2e1614-kube-api-access-cvptg\") pod \"monitoring-plugin-d5dcff7cf-tn4c9\" (UID: \"5719f623-3be7-4a90-ae42-ac58bc2e1614\") " pod="openshift-monitoring/monitoring-plugin-d5dcff7cf-tn4c9" Jul 07 11:16:51.978563 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:16:51.978530 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/monitoring-plugin-d5dcff7cf-tn4c9" Jul 07 11:16:52.038241 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:16:52.038211 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-79c45cc49-fh26k"] Jul 07 11:16:52.040853 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:16:52.040832 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-79c45cc49-fh26k" Jul 07 11:16:52.051801 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:16:52.050605 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"trusted-ca-bundle\"" Jul 07 11:16:52.056712 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:16:52.056678 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-79c45cc49-fh26k"] Jul 07 11:16:52.079091 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:16:52.079064 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-x5j4h\" (UniqueName: \"kubernetes.io/projected/a5225789-7bf3-4087-88cc-f8569798678c-kube-api-access-x5j4h\") pod \"console-79c45cc49-fh26k\" (UID: \"a5225789-7bf3-4087-88cc-f8569798678c\") " pod="openshift-console/console-79c45cc49-fh26k" Jul 07 11:16:52.079217 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:16:52.079119 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/a5225789-7bf3-4087-88cc-f8569798678c-service-ca\") pod \"console-79c45cc49-fh26k\" (UID: \"a5225789-7bf3-4087-88cc-f8569798678c\") " pod="openshift-console/console-79c45cc49-fh26k" Jul 07 11:16:52.079217 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:16:52.079174 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/a5225789-7bf3-4087-88cc-f8569798678c-console-serving-cert\") pod \"console-79c45cc49-fh26k\" (UID: \"a5225789-7bf3-4087-88cc-f8569798678c\") " pod="openshift-console/console-79c45cc49-fh26k" Jul 07 11:16:52.079327 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:16:52.079245 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/a5225789-7bf3-4087-88cc-f8569798678c-oauth-serving-cert\") pod \"console-79c45cc49-fh26k\" (UID: \"a5225789-7bf3-4087-88cc-f8569798678c\") " pod="openshift-console/console-79c45cc49-fh26k" Jul 07 11:16:52.079327 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:16:52.079316 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a5225789-7bf3-4087-88cc-f8569798678c-trusted-ca-bundle\") pod \"console-79c45cc49-fh26k\" (UID: \"a5225789-7bf3-4087-88cc-f8569798678c\") " pod="openshift-console/console-79c45cc49-fh26k" Jul 07 11:16:52.079407 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:16:52.079350 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/a5225789-7bf3-4087-88cc-f8569798678c-console-oauth-config\") pod \"console-79c45cc49-fh26k\" (UID: \"a5225789-7bf3-4087-88cc-f8569798678c\") " pod="openshift-console/console-79c45cc49-fh26k" Jul 07 11:16:52.079407 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:16:52.079368 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/a5225789-7bf3-4087-88cc-f8569798678c-console-config\") pod \"console-79c45cc49-fh26k\" (UID: \"a5225789-7bf3-4087-88cc-f8569798678c\") " pod="openshift-console/console-79c45cc49-fh26k" Jul 07 11:16:52.121804 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:16:52.121711 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/monitoring-plugin-d5dcff7cf-tn4c9"] Jul 07 11:16:52.125808 ip-10-0-142-112 kubenswrapper[2578]: W0707 11:16:52.125775 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5719f623_3be7_4a90_ae42_ac58bc2e1614.slice/crio-126239d4ff676bdcf91cdc4796983930652eb881565fe78931637159324cc268 WatchSource:0}: Error finding container 126239d4ff676bdcf91cdc4796983930652eb881565fe78931637159324cc268: Status 404 returned error can't find the container with id 126239d4ff676bdcf91cdc4796983930652eb881565fe78931637159324cc268 Jul 07 11:16:52.179703 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:16:52.179668 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/a5225789-7bf3-4087-88cc-f8569798678c-oauth-serving-cert\") pod \"console-79c45cc49-fh26k\" (UID: \"a5225789-7bf3-4087-88cc-f8569798678c\") " pod="openshift-console/console-79c45cc49-fh26k" Jul 07 11:16:52.179703 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:16:52.179701 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a5225789-7bf3-4087-88cc-f8569798678c-trusted-ca-bundle\") pod \"console-79c45cc49-fh26k\" (UID: \"a5225789-7bf3-4087-88cc-f8569798678c\") " pod="openshift-console/console-79c45cc49-fh26k" Jul 07 11:16:52.179861 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:16:52.179724 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/a5225789-7bf3-4087-88cc-f8569798678c-console-oauth-config\") pod \"console-79c45cc49-fh26k\" (UID: \"a5225789-7bf3-4087-88cc-f8569798678c\") " pod="openshift-console/console-79c45cc49-fh26k" Jul 07 11:16:52.179944 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:16:52.179926 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/a5225789-7bf3-4087-88cc-f8569798678c-console-config\") pod \"console-79c45cc49-fh26k\" (UID: \"a5225789-7bf3-4087-88cc-f8569798678c\") " pod="openshift-console/console-79c45cc49-fh26k" Jul 07 11:16:52.180069 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:16:52.180026 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-x5j4h\" (UniqueName: \"kubernetes.io/projected/a5225789-7bf3-4087-88cc-f8569798678c-kube-api-access-x5j4h\") pod \"console-79c45cc49-fh26k\" (UID: \"a5225789-7bf3-4087-88cc-f8569798678c\") " pod="openshift-console/console-79c45cc49-fh26k" Jul 07 11:16:52.180190 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:16:52.180083 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/a5225789-7bf3-4087-88cc-f8569798678c-service-ca\") pod \"console-79c45cc49-fh26k\" (UID: \"a5225789-7bf3-4087-88cc-f8569798678c\") " pod="openshift-console/console-79c45cc49-fh26k" Jul 07 11:16:52.180190 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:16:52.180111 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/a5225789-7bf3-4087-88cc-f8569798678c-console-serving-cert\") pod \"console-79c45cc49-fh26k\" (UID: \"a5225789-7bf3-4087-88cc-f8569798678c\") " pod="openshift-console/console-79c45cc49-fh26k" Jul 07 11:16:52.180677 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:16:52.180654 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/a5225789-7bf3-4087-88cc-f8569798678c-oauth-serving-cert\") pod \"console-79c45cc49-fh26k\" (UID: \"a5225789-7bf3-4087-88cc-f8569798678c\") " pod="openshift-console/console-79c45cc49-fh26k" Jul 07 11:16:52.180798 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:16:52.180738 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/a5225789-7bf3-4087-88cc-f8569798678c-console-config\") pod \"console-79c45cc49-fh26k\" (UID: \"a5225789-7bf3-4087-88cc-f8569798678c\") " pod="openshift-console/console-79c45cc49-fh26k" Jul 07 11:16:52.180902 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:16:52.180826 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a5225789-7bf3-4087-88cc-f8569798678c-trusted-ca-bundle\") pod \"console-79c45cc49-fh26k\" (UID: \"a5225789-7bf3-4087-88cc-f8569798678c\") " pod="openshift-console/console-79c45cc49-fh26k" Jul 07 11:16:52.180970 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:16:52.180954 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/a5225789-7bf3-4087-88cc-f8569798678c-service-ca\") pod \"console-79c45cc49-fh26k\" (UID: \"a5225789-7bf3-4087-88cc-f8569798678c\") " pod="openshift-console/console-79c45cc49-fh26k" Jul 07 11:16:52.182813 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:16:52.182590 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/a5225789-7bf3-4087-88cc-f8569798678c-console-oauth-config\") pod \"console-79c45cc49-fh26k\" (UID: \"a5225789-7bf3-4087-88cc-f8569798678c\") " pod="openshift-console/console-79c45cc49-fh26k" Jul 07 11:16:52.183143 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:16:52.183101 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/a5225789-7bf3-4087-88cc-f8569798678c-console-serving-cert\") pod \"console-79c45cc49-fh26k\" (UID: \"a5225789-7bf3-4087-88cc-f8569798678c\") " pod="openshift-console/console-79c45cc49-fh26k" Jul 07 11:16:52.190951 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:16:52.190920 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-x5j4h\" (UniqueName: \"kubernetes.io/projected/a5225789-7bf3-4087-88cc-f8569798678c-kube-api-access-x5j4h\") pod \"console-79c45cc49-fh26k\" (UID: \"a5225789-7bf3-4087-88cc-f8569798678c\") " pod="openshift-console/console-79c45cc49-fh26k" Jul 07 11:16:52.364913 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:16:52.364874 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-79c45cc49-fh26k" Jul 07 11:16:52.486306 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:16:52.486277 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-79c45cc49-fh26k"] Jul 07 11:16:52.490143 ip-10-0-142-112 kubenswrapper[2578]: W0707 11:16:52.490109 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda5225789_7bf3_4087_88cc_f8569798678c.slice/crio-65be6c5fee76009211de5e29009e4e3ee4d7ecfc90e42cfaa6c56953e9ad2722 WatchSource:0}: Error finding container 65be6c5fee76009211de5e29009e4e3ee4d7ecfc90e42cfaa6c56953e9ad2722: Status 404 returned error can't find the container with id 65be6c5fee76009211de5e29009e4e3ee4d7ecfc90e42cfaa6c56953e9ad2722 Jul 07 11:16:52.532019 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:16:52.531995 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-lgwzn" Jul 07 11:16:52.583752 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:16:52.583717 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-7b5486658-ncz65" event={"ID":"7a646b22-d6b9-448c-a618-78a5edf13a17","Type":"ContainerStarted","Data":"d73e7497e3538b3b2f9b79acee317b977b3c1628d3a10c16d37608814ec85209"} Jul 07 11:16:52.584210 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:16:52.583790 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-7b5486658-ncz65" event={"ID":"7a646b22-d6b9-448c-a618-78a5edf13a17","Type":"ContainerStarted","Data":"8008f0797f5dabd4e764cbffffa7d8090e835859dfa0f6959ece86c7a0915754"} Jul 07 11:16:52.584210 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:16:52.583828 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-7b5486658-ncz65" event={"ID":"7a646b22-d6b9-448c-a618-78a5edf13a17","Type":"ContainerStarted","Data":"26fa0b98973b1b88d749faa0cb4972818d1b0115fae4fe64b1aafa8af354da7e"} Jul 07 11:16:52.585465 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:16:52.585435 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-79c45cc49-fh26k" event={"ID":"a5225789-7bf3-4087-88cc-f8569798678c","Type":"ContainerStarted","Data":"c8613cae23dfc1f5f2e3ac63034d30d7d6ebbe897e79f089a893f3d14ce3e59f"} Jul 07 11:16:52.585623 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:16:52.585482 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-79c45cc49-fh26k" event={"ID":"a5225789-7bf3-4087-88cc-f8569798678c","Type":"ContainerStarted","Data":"65be6c5fee76009211de5e29009e4e3ee4d7ecfc90e42cfaa6c56953e9ad2722"} Jul 07 11:16:52.586791 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:16:52.586769 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/monitoring-plugin-d5dcff7cf-tn4c9" event={"ID":"5719f623-3be7-4a90-ae42-ac58bc2e1614","Type":"ContainerStarted","Data":"126239d4ff676bdcf91cdc4796983930652eb881565fe78931637159324cc268"} Jul 07 11:16:52.606354 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:16:52.606287 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-79c45cc49-fh26k" podStartSLOduration=0.606266631 podStartE2EDuration="606.266631ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 11:16:52 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 11:16:52.604069752 +0000 UTC m=+56.900219347" watchObservedRunningTime="2026-07-07 11:16:52.606266631 +0000 UTC m=+56.902416219" Jul 07 11:16:53.374959 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:16:53.374885 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jul 07 11:16:53.382160 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:16:53.381248 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" Jul 07 11:16:53.388845 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:16:53.388345 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-client-certs\"" Jul 07 11:16:53.388845 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:16:53.388445 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-rbac-proxy\"" Jul 07 11:16:53.388845 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:16:53.388772 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls-assets-0\"" Jul 07 11:16:53.396416 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:16:53.389158 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-kube-rbac-proxy-web\"" Jul 07 11:16:53.396416 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:16:53.389589 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s\"" Jul 07 11:16:53.396416 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:16:53.390158 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-prometheus-http-client-file\"" Jul 07 11:16:53.396416 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:16:53.390288 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-1\"" Jul 07 11:16:53.396416 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:16:53.390391 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls\"" Jul 07 11:16:53.396416 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:16:53.390639 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-web-config\"" Jul 07 11:16:53.396416 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:16:53.391122 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kubelet-serving-ca-bundle\"" Jul 07 11:16:53.396416 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:16:53.390374 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-2\"" Jul 07 11:16:53.396416 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:16:53.392235 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"serving-certs-ca-bundle\"" Jul 07 11:16:53.396416 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:16:53.393017 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/850ea628-4496-41f7-839c-5233a8831455-config\") pod \"prometheus-k8s-0\" (UID: \"850ea628-4496-41f7-839c-5233a8831455\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 11:16:53.396416 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:16:53.393238 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/850ea628-4496-41f7-839c-5233a8831455-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"850ea628-4496-41f7-839c-5233a8831455\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 11:16:53.396416 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:16:53.393291 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/850ea628-4496-41f7-839c-5233a8831455-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"850ea628-4496-41f7-839c-5233a8831455\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 11:16:53.396416 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:16:53.393333 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/850ea628-4496-41f7-839c-5233a8831455-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"850ea628-4496-41f7-839c-5233a8831455\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 11:16:53.396416 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:16:53.393565 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/850ea628-4496-41f7-839c-5233a8831455-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"850ea628-4496-41f7-839c-5233a8831455\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 11:16:53.396416 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:16:53.393609 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/850ea628-4496-41f7-839c-5233a8831455-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"850ea628-4496-41f7-839c-5233a8831455\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 11:16:53.396416 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:16:53.393641 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/850ea628-4496-41f7-839c-5233a8831455-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"850ea628-4496-41f7-839c-5233a8831455\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 11:16:53.396416 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:16:53.393754 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/850ea628-4496-41f7-839c-5233a8831455-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"850ea628-4496-41f7-839c-5233a8831455\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 11:16:53.396416 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:16:53.393794 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/850ea628-4496-41f7-839c-5233a8831455-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"850ea628-4496-41f7-839c-5233a8831455\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 11:16:53.396416 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:16:53.393818 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/850ea628-4496-41f7-839c-5233a8831455-web-config\") pod \"prometheus-k8s-0\" (UID: \"850ea628-4496-41f7-839c-5233a8831455\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 11:16:53.396416 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:16:53.393889 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/850ea628-4496-41f7-839c-5233a8831455-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"850ea628-4496-41f7-839c-5233a8831455\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 11:16:53.396416 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:16:53.393923 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/850ea628-4496-41f7-839c-5233a8831455-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"850ea628-4496-41f7-839c-5233a8831455\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 11:16:53.397505 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:16:53.393987 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/850ea628-4496-41f7-839c-5233a8831455-config-out\") pod \"prometheus-k8s-0\" (UID: \"850ea628-4496-41f7-839c-5233a8831455\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 11:16:53.397505 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:16:53.394118 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/850ea628-4496-41f7-839c-5233a8831455-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"850ea628-4496-41f7-839c-5233a8831455\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 11:16:53.397505 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:16:53.394162 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/850ea628-4496-41f7-839c-5233a8831455-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"850ea628-4496-41f7-839c-5233a8831455\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 11:16:53.397505 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:16:53.394338 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/850ea628-4496-41f7-839c-5233a8831455-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"850ea628-4496-41f7-839c-5233a8831455\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 11:16:53.397505 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:16:53.394390 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/850ea628-4496-41f7-839c-5233a8831455-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"850ea628-4496-41f7-839c-5233a8831455\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 11:16:53.397505 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:16:53.394416 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/850ea628-4496-41f7-839c-5233a8831455-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"850ea628-4496-41f7-839c-5233a8831455\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 11:16:53.397505 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:16:53.394559 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/850ea628-4496-41f7-839c-5233a8831455-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"850ea628-4496-41f7-839c-5233a8831455\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 11:16:53.397505 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:16:53.394685 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lxd49\" (UniqueName: \"kubernetes.io/projected/850ea628-4496-41f7-839c-5233a8831455-kube-api-access-lxd49\") pod \"prometheus-k8s-0\" (UID: \"850ea628-4496-41f7-839c-5233a8831455\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 11:16:53.397505 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:16:53.395654 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-grpc-tls-ch72mp8e5duvf\"" Jul 07 11:16:53.397505 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:16:53.396100 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-dockercfg-qk6v2\"" Jul 07 11:16:53.397505 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:16:53.396971 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-sidecar-tls\"" Jul 07 11:16:53.399134 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:16:53.398949 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-0\"" Jul 07 11:16:53.402261 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:16:53.402243 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-trusted-ca-bundle\"" Jul 07 11:16:53.413859 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:16:53.412927 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jul 07 11:16:53.496655 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:16:53.496620 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/850ea628-4496-41f7-839c-5233a8831455-config\") pod \"prometheus-k8s-0\" (UID: \"850ea628-4496-41f7-839c-5233a8831455\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 11:16:53.496855 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:16:53.496666 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/850ea628-4496-41f7-839c-5233a8831455-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"850ea628-4496-41f7-839c-5233a8831455\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 11:16:53.496855 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:16:53.496795 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/850ea628-4496-41f7-839c-5233a8831455-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"850ea628-4496-41f7-839c-5233a8831455\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 11:16:53.496855 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:16:53.496849 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/850ea628-4496-41f7-839c-5233a8831455-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"850ea628-4496-41f7-839c-5233a8831455\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 11:16:53.497023 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:16:53.496890 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/850ea628-4496-41f7-839c-5233a8831455-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"850ea628-4496-41f7-839c-5233a8831455\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 11:16:53.497023 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:16:53.496919 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/850ea628-4496-41f7-839c-5233a8831455-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"850ea628-4496-41f7-839c-5233a8831455\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 11:16:53.497023 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:16:53.496950 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/850ea628-4496-41f7-839c-5233a8831455-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"850ea628-4496-41f7-839c-5233a8831455\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 11:16:53.497023 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:16:53.496990 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/850ea628-4496-41f7-839c-5233a8831455-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"850ea628-4496-41f7-839c-5233a8831455\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 11:16:53.497023 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:16:53.497015 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/850ea628-4496-41f7-839c-5233a8831455-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"850ea628-4496-41f7-839c-5233a8831455\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 11:16:53.497266 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:16:53.497041 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/850ea628-4496-41f7-839c-5233a8831455-web-config\") pod \"prometheus-k8s-0\" (UID: \"850ea628-4496-41f7-839c-5233a8831455\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 11:16:53.497266 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:16:53.497066 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/850ea628-4496-41f7-839c-5233a8831455-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"850ea628-4496-41f7-839c-5233a8831455\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 11:16:53.497266 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:16:53.497095 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/850ea628-4496-41f7-839c-5233a8831455-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"850ea628-4496-41f7-839c-5233a8831455\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 11:16:53.497266 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:16:53.497156 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/850ea628-4496-41f7-839c-5233a8831455-config-out\") pod \"prometheus-k8s-0\" (UID: \"850ea628-4496-41f7-839c-5233a8831455\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 11:16:53.497266 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:16:53.497229 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/850ea628-4496-41f7-839c-5233a8831455-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"850ea628-4496-41f7-839c-5233a8831455\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 11:16:53.497540 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:16:53.497267 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/850ea628-4496-41f7-839c-5233a8831455-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"850ea628-4496-41f7-839c-5233a8831455\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 11:16:53.497540 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:16:53.497298 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/850ea628-4496-41f7-839c-5233a8831455-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"850ea628-4496-41f7-839c-5233a8831455\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 11:16:53.497540 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:16:53.497332 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/850ea628-4496-41f7-839c-5233a8831455-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"850ea628-4496-41f7-839c-5233a8831455\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 11:16:53.497540 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:16:53.497387 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/850ea628-4496-41f7-839c-5233a8831455-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"850ea628-4496-41f7-839c-5233a8831455\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 11:16:53.497540 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:16:53.497416 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/850ea628-4496-41f7-839c-5233a8831455-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"850ea628-4496-41f7-839c-5233a8831455\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 11:16:53.497540 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:16:53.497473 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lxd49\" (UniqueName: \"kubernetes.io/projected/850ea628-4496-41f7-839c-5233a8831455-kube-api-access-lxd49\") pod \"prometheus-k8s-0\" (UID: \"850ea628-4496-41f7-839c-5233a8831455\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 11:16:53.498857 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:16:53.498827 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/850ea628-4496-41f7-839c-5233a8831455-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"850ea628-4496-41f7-839c-5233a8831455\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 11:16:53.499903 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:16:53.499882 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/850ea628-4496-41f7-839c-5233a8831455-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"850ea628-4496-41f7-839c-5233a8831455\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 11:16:53.500392 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:16:53.500372 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/850ea628-4496-41f7-839c-5233a8831455-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"850ea628-4496-41f7-839c-5233a8831455\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 11:16:53.501095 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:16:53.501005 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/850ea628-4496-41f7-839c-5233a8831455-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"850ea628-4496-41f7-839c-5233a8831455\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 11:16:53.501564 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:16:53.501504 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/850ea628-4496-41f7-839c-5233a8831455-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"850ea628-4496-41f7-839c-5233a8831455\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 11:16:53.502013 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:16:53.501958 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/850ea628-4496-41f7-839c-5233a8831455-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"850ea628-4496-41f7-839c-5233a8831455\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 11:16:53.502159 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:16:53.501989 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/850ea628-4496-41f7-839c-5233a8831455-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"850ea628-4496-41f7-839c-5233a8831455\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 11:16:53.502713 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:16:53.502554 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/secret/850ea628-4496-41f7-839c-5233a8831455-config\") pod \"prometheus-k8s-0\" (UID: \"850ea628-4496-41f7-839c-5233a8831455\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 11:16:53.502713 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:16:53.502679 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/850ea628-4496-41f7-839c-5233a8831455-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"850ea628-4496-41f7-839c-5233a8831455\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 11:16:53.503028 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:16:53.503004 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/850ea628-4496-41f7-839c-5233a8831455-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"850ea628-4496-41f7-839c-5233a8831455\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 11:16:53.503104 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:16:53.503044 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/850ea628-4496-41f7-839c-5233a8831455-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"850ea628-4496-41f7-839c-5233a8831455\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 11:16:53.503104 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:16:53.503078 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/850ea628-4496-41f7-839c-5233a8831455-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"850ea628-4496-41f7-839c-5233a8831455\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 11:16:53.504130 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:16:53.504086 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/850ea628-4496-41f7-839c-5233a8831455-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"850ea628-4496-41f7-839c-5233a8831455\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 11:16:53.504617 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:16:53.504564 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/850ea628-4496-41f7-839c-5233a8831455-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"850ea628-4496-41f7-839c-5233a8831455\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 11:16:53.504766 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:16:53.504729 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/850ea628-4496-41f7-839c-5233a8831455-web-config\") pod \"prometheus-k8s-0\" (UID: \"850ea628-4496-41f7-839c-5233a8831455\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 11:16:53.505061 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:16:53.505036 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/850ea628-4496-41f7-839c-5233a8831455-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"850ea628-4496-41f7-839c-5233a8831455\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 11:16:53.505198 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:16:53.505177 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/850ea628-4496-41f7-839c-5233a8831455-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"850ea628-4496-41f7-839c-5233a8831455\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 11:16:53.505643 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:16:53.505620 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/850ea628-4496-41f7-839c-5233a8831455-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"850ea628-4496-41f7-839c-5233a8831455\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 11:16:53.506519 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:16:53.506498 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/850ea628-4496-41f7-839c-5233a8831455-config-out\") pod \"prometheus-k8s-0\" (UID: \"850ea628-4496-41f7-839c-5233a8831455\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 11:16:53.507304 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:16:53.507286 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-lxd49\" (UniqueName: \"kubernetes.io/projected/850ea628-4496-41f7-839c-5233a8831455-kube-api-access-lxd49\") pod \"prometheus-k8s-0\" (UID: \"850ea628-4496-41f7-839c-5233a8831455\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 11:16:53.596789 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:16:53.596746 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-7b5486658-ncz65" event={"ID":"7a646b22-d6b9-448c-a618-78a5edf13a17","Type":"ContainerStarted","Data":"6fbab3200185a2b0ae42c1eb2810287863f974981ee5c583cadfa0362894db38"} Jul 07 11:16:53.597233 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:16:53.596837 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-7b5486658-ncz65" event={"ID":"7a646b22-d6b9-448c-a618-78a5edf13a17","Type":"ContainerStarted","Data":"ac287de0a767866ee33340338f09f62f3636e8f8d1aa45fac119d27d5478c8d2"} Jul 07 11:16:53.597233 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:16:53.596881 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-7b5486658-ncz65" event={"ID":"7a646b22-d6b9-448c-a618-78a5edf13a17","Type":"ContainerStarted","Data":"55a649fbfe91e1d9beaada53a273f2ec9a3f3aa02bb10c49937d0c2b49ecc228"} Jul 07 11:16:53.626963 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:16:53.625314 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/thanos-querier-7b5486658-ncz65" podStartSLOduration=1.37973163 podStartE2EDuration="4.62529028s" totalImagesPullingTime="3.24555865s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 11:16:49 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 11:16:53.624283852 +0000 UTC m=+57.920433442" watchObservedRunningTime="2026-07-07 11:16:53.62529028 +0000 UTC m=+57.921439869" Jul 07 11:16:53.701260 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:16:53.701231 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" Jul 07 11:16:53.771775 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:16:53.771404 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-86b7c49ddc-xg9tg" Jul 07 11:16:53.772654 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:16:53.772165 2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-86b7c49ddc-xg9tg" Jul 07 11:16:53.780246 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:16:53.780223 2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-86b7c49ddc-xg9tg" Jul 07 11:16:53.879296 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:16:53.879124 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jul 07 11:16:53.885879 ip-10-0-142-112 kubenswrapper[2578]: W0707 11:16:53.885848 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod850ea628_4496_41f7_839c_5233a8831455.slice/crio-3849121e3669cb53faab41e1d0e381d46e9ad3c5ed3e664c81d85417f9bcec6e WatchSource:0}: Error finding container 3849121e3669cb53faab41e1d0e381d46e9ad3c5ed3e664c81d85417f9bcec6e: Status 404 returned error can't find the container with id 3849121e3669cb53faab41e1d0e381d46e9ad3c5ed3e664c81d85417f9bcec6e Jul 07 11:16:54.603344 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:16:54.603297 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/monitoring-plugin-d5dcff7cf-tn4c9" event={"ID":"5719f623-3be7-4a90-ae42-ac58bc2e1614","Type":"ContainerStarted","Data":"6a9256f76014fdbc3b0489e0a5515f23acc55a9af005d9297ebdc7e091ac48c5"} Jul 07 11:16:54.604121 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:16:54.604099 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/monitoring-plugin-d5dcff7cf-tn4c9" Jul 07 11:16:54.607722 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:16:54.607697 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"850ea628-4496-41f7-839c-5233a8831455","Type":"ContainerStarted","Data":"3849121e3669cb53faab41e1d0e381d46e9ad3c5ed3e664c81d85417f9bcec6e"} Jul 07 11:16:54.608263 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:16:54.608240 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/thanos-querier-7b5486658-ncz65" Jul 07 11:16:54.610809 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:16:54.610788 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/monitoring-plugin-d5dcff7cf-tn4c9" Jul 07 11:16:54.612442 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:16:54.612401 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-86b7c49ddc-xg9tg" Jul 07 11:16:54.639219 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:16:54.639160 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/monitoring-plugin-d5dcff7cf-tn4c9" podStartSLOduration=2.037552682 podStartE2EDuration="3.639131548s" totalImagesPullingTime="1.601578866s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 11:16:51 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 11:16:54.620798171 +0000 UTC m=+58.916947760" watchObservedRunningTime="2026-07-07 11:16:54.639131548 +0000 UTC m=+58.935281117" Jul 07 11:16:55.611640 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:16:55.611606 2578 generic.go:356] "Generic (PLEG): container finished" podID="850ea628-4496-41f7-839c-5233a8831455" containerID="a6be01a6cfd01c0d23c702625286f20cfa8f27c6d83f547ac992f392660ec0cc" exitCode=0 Jul 07 11:16:55.612088 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:16:55.611690 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"850ea628-4496-41f7-839c-5233a8831455","Type":"ContainerDied","Data":"a6be01a6cfd01c0d23c702625286f20cfa8f27c6d83f547ac992f392660ec0cc"} Jul 07 11:16:57.475009 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:16:57.474978 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-ndwrc" Jul 07 11:16:59.630618 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:16:59.630579 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"850ea628-4496-41f7-839c-5233a8831455","Type":"ContainerStarted","Data":"cdde943e1ddcdbb5dcd233e7963037b127015ff2f70530bed844906e7a52db04"} Jul 07 11:16:59.630997 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:16:59.630663 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"850ea628-4496-41f7-839c-5233a8831455","Type":"ContainerStarted","Data":"c532e02e1b8ba45be092c065567941d63731e3096a62eec267ab457b17456975"} Jul 07 11:16:59.630997 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:16:59.630702 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"850ea628-4496-41f7-839c-5233a8831455","Type":"ContainerStarted","Data":"d65b0d8c8c320a51488aac9d021ce7d2de30c1173a3b0f200c26f19f110bf83b"} Jul 07 11:16:59.634776 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:16:59.631118 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"850ea628-4496-41f7-839c-5233a8831455","Type":"ContainerStarted","Data":"453fb6d8710dbd3680fce0cd3b52b3305f3f9cca5b5d58341bdf259c65a433a4"} Jul 07 11:16:59.634776 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:16:59.631275 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"850ea628-4496-41f7-839c-5233a8831455","Type":"ContainerStarted","Data":"c418aa245eca303145bb03fbcb9a31fa1de8856257ab73a75674887b720ab31d"} Jul 07 11:16:59.634776 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:16:59.631363 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"850ea628-4496-41f7-839c-5233a8831455","Type":"ContainerStarted","Data":"f4571a2743a77df32f8a41cb9a23bb5ac17fffbf0b52b30957bc5b9565a745f1"} Jul 07 11:16:59.680263 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:16:59.680189 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-k8s-0" podStartSLOduration=2.015072627 podStartE2EDuration="6.680166475s" totalImagesPullingTime="4.64579946s" totalInitContainerRuntime="19.294388ms" isStatefulPod=false podCreationTimestamp="2026-07-07 11:16:53 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 11:16:59.678214021 +0000 UTC m=+63.974363609" watchObservedRunningTime="2026-07-07 11:16:59.680166475 +0000 UTC m=+63.976316091" Jul 07 11:17:00.617572 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:17:00.617547 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/thanos-querier-7b5486658-ncz65" Jul 07 11:17:01.977891 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:17:01.977851 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/3bd399c7-6314-4991-9984-8d2ff9677352-original-pull-secret\") pod \"global-pull-secret-syncer-8vbr5\" (UID: \"3bd399c7-6314-4991-9984-8d2ff9677352\") " pod="kube-system/global-pull-secret-syncer-8vbr5" Jul 07 11:17:01.978242 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:17:01.977902 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/1dc0212e-9bf8-4b06-b984-c694479a14ad-metrics-certs\") pod \"network-metrics-daemon-xhlzx\" (UID: \"1dc0212e-9bf8-4b06-b984-c694479a14ad\") " pod="openshift-multus/network-metrics-daemon-xhlzx" Jul 07 11:17:01.980562 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:17:01.980542 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Jul 07 11:17:01.980609 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:17:01.980540 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Jul 07 11:17:01.991100 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:17:01.991072 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/3bd399c7-6314-4991-9984-8d2ff9677352-original-pull-secret\") pod \"global-pull-secret-syncer-8vbr5\" (UID: \"3bd399c7-6314-4991-9984-8d2ff9677352\") " pod="kube-system/global-pull-secret-syncer-8vbr5" Jul 07 11:17:01.991100 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:17:01.991087 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/1dc0212e-9bf8-4b06-b984-c694479a14ad-metrics-certs\") pod \"network-metrics-daemon-xhlzx\" (UID: \"1dc0212e-9bf8-4b06-b984-c694479a14ad\") " pod="openshift-multus/network-metrics-daemon-xhlzx" Jul 07 11:17:02.078751 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:17:02.078709 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5h5r9\" (UniqueName: \"kubernetes.io/projected/5467fdde-6375-4364-ac0a-f5c6125c0807-kube-api-access-5h5r9\") pod \"network-check-target-tjprx\" (UID: \"5467fdde-6375-4364-ac0a-f5c6125c0807\") " pod="openshift-network-diagnostics/network-check-target-tjprx" Jul 07 11:17:02.081591 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:17:02.081573 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" Jul 07 11:17:02.092156 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:17:02.092136 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" Jul 07 11:17:02.102638 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:17:02.102613 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-5h5r9\" (UniqueName: \"kubernetes.io/projected/5467fdde-6375-4364-ac0a-f5c6125c0807-kube-api-access-5h5r9\") pod \"network-check-target-tjprx\" (UID: \"5467fdde-6375-4364-ac0a-f5c6125c0807\") " pod="openshift-network-diagnostics/network-check-target-tjprx" Jul 07 11:17:02.121865 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:17:02.121844 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-42fbv\"" Jul 07 11:17:02.130408 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:17:02.130388 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-tjprx" Jul 07 11:17:02.130660 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:17:02.130647 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-89jkb\"" Jul 07 11:17:02.134512 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:17:02.134499 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-8vbr5" Jul 07 11:17:02.138568 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:17:02.138553 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-xhlzx" Jul 07 11:17:02.284637 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:17:02.284286 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-tjprx"] Jul 07 11:17:02.286366 ip-10-0-142-112 kubenswrapper[2578]: W0707 11:17:02.286341 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5467fdde_6375_4364_ac0a_f5c6125c0807.slice/crio-6f6125fa8bc146acb5646fbeebfdbaa588df32f8f1c990b3bb85be127f91acdf WatchSource:0}: Error finding container 6f6125fa8bc146acb5646fbeebfdbaa588df32f8f1c990b3bb85be127f91acdf: Status 404 returned error can't find the container with id 6f6125fa8bc146acb5646fbeebfdbaa588df32f8f1c990b3bb85be127f91acdf Jul 07 11:17:02.365176 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:17:02.365136 2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-79c45cc49-fh26k" Jul 07 11:17:02.365342 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:17:02.365206 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-79c45cc49-fh26k" Jul 07 11:17:02.369987 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:17:02.369966 2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-79c45cc49-fh26k" Jul 07 11:17:02.518236 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:17:02.518159 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-8vbr5"] Jul 07 11:17:02.521255 ip-10-0-142-112 kubenswrapper[2578]: W0707 11:17:02.521230 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3bd399c7_6314_4991_9984_8d2ff9677352.slice/crio-647bace583b2149f5cf67cb392e872e0d51b90d870fef01f0dab5ddda27e3822 WatchSource:0}: Error finding container 647bace583b2149f5cf67cb392e872e0d51b90d870fef01f0dab5ddda27e3822: Status 404 returned error can't find the container with id 647bace583b2149f5cf67cb392e872e0d51b90d870fef01f0dab5ddda27e3822 Jul 07 11:17:02.538005 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:17:02.537977 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-xhlzx"] Jul 07 11:17:02.540894 ip-10-0-142-112 kubenswrapper[2578]: W0707 11:17:02.540867 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1dc0212e_9bf8_4b06_b984_c694479a14ad.slice/crio-d1e1c619ddb6960569245b754dfcee4d2ce4b7bf8a18bc3bb97f274b3495edad WatchSource:0}: Error finding container d1e1c619ddb6960569245b754dfcee4d2ce4b7bf8a18bc3bb97f274b3495edad: Status 404 returned error can't find the container with id d1e1c619ddb6960569245b754dfcee4d2ce4b7bf8a18bc3bb97f274b3495edad Jul 07 11:17:02.644973 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:17:02.644940 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-xhlzx" event={"ID":"1dc0212e-9bf8-4b06-b984-c694479a14ad","Type":"ContainerStarted","Data":"d1e1c619ddb6960569245b754dfcee4d2ce4b7bf8a18bc3bb97f274b3495edad"} Jul 07 11:17:02.646119 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:17:02.646098 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-tjprx" event={"ID":"5467fdde-6375-4364-ac0a-f5c6125c0807","Type":"ContainerStarted","Data":"6f6125fa8bc146acb5646fbeebfdbaa588df32f8f1c990b3bb85be127f91acdf"} Jul 07 11:17:02.647321 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:17:02.647297 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-8vbr5" event={"ID":"3bd399c7-6314-4991-9984-8d2ff9677352","Type":"ContainerStarted","Data":"647bace583b2149f5cf67cb392e872e0d51b90d870fef01f0dab5ddda27e3822"} Jul 07 11:17:02.651170 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:17:02.651152 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-79c45cc49-fh26k" Jul 07 11:17:02.779945 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:17:02.779871 2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-86b7c49ddc-xg9tg"] Jul 07 11:17:03.702076 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:17:03.702019 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-k8s-0" Jul 07 11:17:04.661316 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:17:04.661276 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-xhlzx" event={"ID":"1dc0212e-9bf8-4b06-b984-c694479a14ad","Type":"ContainerStarted","Data":"9dca23acd732b1ce3a0c3046690959a675c7f68fe1c87f0dd9ece303bb76eaac"} Jul 07 11:17:04.661542 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:17:04.661342 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-xhlzx" event={"ID":"1dc0212e-9bf8-4b06-b984-c694479a14ad","Type":"ContainerStarted","Data":"8d83492c8cc5ddee07df598e17b7c30cfbcc4219bb3dc93ba45d35a155d294e1"} Jul 07 11:17:04.689777 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:17:04.689713 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-xhlzx" podStartSLOduration=67.295941594 podStartE2EDuration="1m8.689688495s" totalImagesPullingTime="1.393746901s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 11:15:56 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 11:17:04.687290228 +0000 UTC m=+68.983439817" watchObservedRunningTime="2026-07-07 11:17:04.689688495 +0000 UTC m=+68.985838084" Jul 07 11:17:07.677623 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:17:07.677581 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-8vbr5" event={"ID":"3bd399c7-6314-4991-9984-8d2ff9677352","Type":"ContainerStarted","Data":"7710c5900867f7707d96ca4524d16451cba7364c234cfda138e7641e000d935b"} Jul 07 11:17:07.678976 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:17:07.678954 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-tjprx" event={"ID":"5467fdde-6375-4364-ac0a-f5c6125c0807","Type":"ContainerStarted","Data":"ee9de9ef4e7c14cca5da51c5eb2a2b5e47fd6b4a405db176bccf2678542859a0"} Jul 07 11:17:07.679134 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:17:07.679082 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-tjprx" Jul 07 11:17:07.702888 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:17:07.702843 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-8vbr5" podStartSLOduration=67.202182676 podStartE2EDuration="1m11.702828941s" totalImagesPullingTime="4.500646265s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-07 11:15:56 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 11:17:07.70232304 +0000 UTC m=+71.998472639" watchObservedRunningTime="2026-07-07 11:17:07.702828941 +0000 UTC m=+71.998978528" Jul 07 11:17:07.729641 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:17:07.729588 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-tjprx" podStartSLOduration=66.997698997 podStartE2EDuration="1m11.729572569s" totalImagesPullingTime="4.731873572s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 11:15:56 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 11:17:07.728781121 +0000 UTC m=+72.024930711" watchObservedRunningTime="2026-07-07 11:17:07.729572569 +0000 UTC m=+72.025722150" Jul 07 11:17:24.760927 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:17:24.760800 2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/prometheus-k8s-0" Jul 07 11:17:24.782480 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:17:24.782453 2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/prometheus-k8s-0" Jul 07 11:17:25.763108 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:17:25.763079 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-k8s-0" Jul 07 11:17:27.807276 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:17:27.807210 2578 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-console/console-86b7c49ddc-xg9tg" podUID="f52d638f-d46b-465b-a454-f34152acb425" containerName="console" containerID="cri-o://f444cd29359a51a26ea3285a565412ae16e675287bce7c9ed50c1c9273a60599" gracePeriod=15 Jul 07 11:17:28.046033 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:17:28.046008 2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-86b7c49ddc-xg9tg" Jul 07 11:17:28.214232 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:17:28.214142 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/f52d638f-d46b-465b-a454-f34152acb425-console-serving-cert\") pod \"f52d638f-d46b-465b-a454-f34152acb425\" (UID: \"f52d638f-d46b-465b-a454-f34152acb425\") " Jul 07 11:17:28.214232 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:17:28.214184 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/f52d638f-d46b-465b-a454-f34152acb425-service-ca\") pod \"f52d638f-d46b-465b-a454-f34152acb425\" (UID: \"f52d638f-d46b-465b-a454-f34152acb425\") " Jul 07 11:17:28.214232 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:17:28.214225 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/f52d638f-d46b-465b-a454-f34152acb425-console-config\") pod \"f52d638f-d46b-465b-a454-f34152acb425\" (UID: \"f52d638f-d46b-465b-a454-f34152acb425\") " Jul 07 11:17:28.214472 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:17:28.214276 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/f52d638f-d46b-465b-a454-f34152acb425-oauth-serving-cert\") pod \"f52d638f-d46b-465b-a454-f34152acb425\" (UID: \"f52d638f-d46b-465b-a454-f34152acb425\") " Jul 07 11:17:28.214472 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:17:28.214304 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-b4d7l\" (UniqueName: \"kubernetes.io/projected/f52d638f-d46b-465b-a454-f34152acb425-kube-api-access-b4d7l\") pod \"f52d638f-d46b-465b-a454-f34152acb425\" (UID: \"f52d638f-d46b-465b-a454-f34152acb425\") " Jul 07 11:17:28.214472 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:17:28.214370 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/f52d638f-d46b-465b-a454-f34152acb425-console-oauth-config\") pod \"f52d638f-d46b-465b-a454-f34152acb425\" (UID: \"f52d638f-d46b-465b-a454-f34152acb425\") " Jul 07 11:17:28.214748 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:17:28.214715 2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f52d638f-d46b-465b-a454-f34152acb425-console-config" (OuterVolumeSpecName: "console-config") pod "f52d638f-d46b-465b-a454-f34152acb425" (UID: "f52d638f-d46b-465b-a454-f34152acb425"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 07 11:17:28.214873 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:17:28.214717 2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f52d638f-d46b-465b-a454-f34152acb425-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "f52d638f-d46b-465b-a454-f34152acb425" (UID: "f52d638f-d46b-465b-a454-f34152acb425"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 07 11:17:28.214873 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:17:28.214765 2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f52d638f-d46b-465b-a454-f34152acb425-service-ca" (OuterVolumeSpecName: "service-ca") pod "f52d638f-d46b-465b-a454-f34152acb425" (UID: "f52d638f-d46b-465b-a454-f34152acb425"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 07 11:17:28.216689 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:17:28.216660 2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f52d638f-d46b-465b-a454-f34152acb425-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "f52d638f-d46b-465b-a454-f34152acb425" (UID: "f52d638f-d46b-465b-a454-f34152acb425"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 07 11:17:28.216796 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:17:28.216769 2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f52d638f-d46b-465b-a454-f34152acb425-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "f52d638f-d46b-465b-a454-f34152acb425" (UID: "f52d638f-d46b-465b-a454-f34152acb425"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 07 11:17:28.216837 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:17:28.216800 2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f52d638f-d46b-465b-a454-f34152acb425-kube-api-access-b4d7l" (OuterVolumeSpecName: "kube-api-access-b4d7l") pod "f52d638f-d46b-465b-a454-f34152acb425" (UID: "f52d638f-d46b-465b-a454-f34152acb425"). InnerVolumeSpecName "kube-api-access-b4d7l". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 07 11:17:28.315922 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:17:28.315898 2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-b4d7l\" (UniqueName: \"kubernetes.io/projected/f52d638f-d46b-465b-a454-f34152acb425-kube-api-access-b4d7l\") on node \"ip-10-0-142-112.ec2.internal\" DevicePath \"\"" Jul 07 11:17:28.315922 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:17:28.315922 2578 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/f52d638f-d46b-465b-a454-f34152acb425-console-oauth-config\") on node \"ip-10-0-142-112.ec2.internal\" DevicePath \"\"" Jul 07 11:17:28.316069 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:17:28.315931 2578 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/f52d638f-d46b-465b-a454-f34152acb425-console-serving-cert\") on node \"ip-10-0-142-112.ec2.internal\" DevicePath \"\"" Jul 07 11:17:28.316069 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:17:28.315939 2578 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/f52d638f-d46b-465b-a454-f34152acb425-service-ca\") on node \"ip-10-0-142-112.ec2.internal\" DevicePath \"\"" Jul 07 11:17:28.316069 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:17:28.315948 2578 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/f52d638f-d46b-465b-a454-f34152acb425-console-config\") on node \"ip-10-0-142-112.ec2.internal\" DevicePath \"\"" Jul 07 11:17:28.316069 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:17:28.315956 2578 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/f52d638f-d46b-465b-a454-f34152acb425-oauth-serving-cert\") on node \"ip-10-0-142-112.ec2.internal\" DevicePath \"\"" Jul 07 11:17:28.763316 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:17:28.763282 2578 generic.go:356] "Generic (PLEG): container finished" podID="f52d638f-d46b-465b-a454-f34152acb425" containerID="f444cd29359a51a26ea3285a565412ae16e675287bce7c9ed50c1c9273a60599" exitCode=0 Jul 07 11:17:28.763505 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:17:28.763321 2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-86b7c49ddc-xg9tg" Jul 07 11:17:28.763505 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:17:28.763360 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-86b7c49ddc-xg9tg" event={"ID":"f52d638f-d46b-465b-a454-f34152acb425","Type":"ContainerDied","Data":"f444cd29359a51a26ea3285a565412ae16e675287bce7c9ed50c1c9273a60599"} Jul 07 11:17:28.763505 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:17:28.763433 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-86b7c49ddc-xg9tg" event={"ID":"f52d638f-d46b-465b-a454-f34152acb425","Type":"ContainerDied","Data":"079d0a4ea8427e831aca84566e16b8aacac87627f531ad2448a2871a2ee6566d"} Jul 07 11:17:28.763505 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:17:28.763448 2578 scope.go:117] "RemoveContainer" containerID="f444cd29359a51a26ea3285a565412ae16e675287bce7c9ed50c1c9273a60599" Jul 07 11:17:28.772351 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:17:28.772333 2578 scope.go:117] "RemoveContainer" containerID="f444cd29359a51a26ea3285a565412ae16e675287bce7c9ed50c1c9273a60599" Jul 07 11:17:28.772663 ip-10-0-142-112 kubenswrapper[2578]: E0707 11:17:28.772636 2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f444cd29359a51a26ea3285a565412ae16e675287bce7c9ed50c1c9273a60599\": container with ID starting with f444cd29359a51a26ea3285a565412ae16e675287bce7c9ed50c1c9273a60599 not found: ID does not exist" containerID="f444cd29359a51a26ea3285a565412ae16e675287bce7c9ed50c1c9273a60599" Jul 07 11:17:28.772746 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:17:28.772671 2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f444cd29359a51a26ea3285a565412ae16e675287bce7c9ed50c1c9273a60599"} err="failed to get container status \"f444cd29359a51a26ea3285a565412ae16e675287bce7c9ed50c1c9273a60599\": rpc error: code = NotFound desc = could not find container \"f444cd29359a51a26ea3285a565412ae16e675287bce7c9ed50c1c9273a60599\": container with ID starting with f444cd29359a51a26ea3285a565412ae16e675287bce7c9ed50c1c9273a60599 not found: ID does not exist" Jul 07 11:17:28.780357 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:17:28.780335 2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-86b7c49ddc-xg9tg"] Jul 07 11:17:28.783710 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:17:28.783690 2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-86b7c49ddc-xg9tg"] Jul 07 11:17:30.306794 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:17:30.306766 2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f52d638f-d46b-465b-a454-f34152acb425" path="/var/lib/kubelet/pods/f52d638f-d46b-465b-a454-f34152acb425/volumes" Jul 07 11:17:38.685652 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:17:38.685616 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-tjprx" Jul 07 11:17:41.799517 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:17:41.799486 2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jul 07 11:17:41.799902 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:17:41.799829 2578 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="850ea628-4496-41f7-839c-5233a8831455" containerName="prometheus" containerID="cri-o://f4571a2743a77df32f8a41cb9a23bb5ac17fffbf0b52b30957bc5b9565a745f1" gracePeriod=600 Jul 07 11:17:41.799902 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:17:41.799833 2578 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="850ea628-4496-41f7-839c-5233a8831455" containerName="thanos-sidecar" containerID="cri-o://453fb6d8710dbd3680fce0cd3b52b3305f3f9cca5b5d58341bdf259c65a433a4" gracePeriod=600 Jul 07 11:17:41.799902 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:17:41.799835 2578 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="850ea628-4496-41f7-839c-5233a8831455" containerName="config-reloader" containerID="cri-o://c418aa245eca303145bb03fbcb9a31fa1de8856257ab73a75674887b720ab31d" gracePeriod=600 Jul 07 11:17:41.799902 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:17:41.799881 2578 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="850ea628-4496-41f7-839c-5233a8831455" containerName="kube-rbac-proxy" containerID="cri-o://c532e02e1b8ba45be092c065567941d63731e3096a62eec267ab457b17456975" gracePeriod=600 Jul 07 11:17:41.800095 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:17:41.799949 2578 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="850ea628-4496-41f7-839c-5233a8831455" containerName="kube-rbac-proxy-thanos" containerID="cri-o://cdde943e1ddcdbb5dcd233e7963037b127015ff2f70530bed844906e7a52db04" gracePeriod=600 Jul 07 11:17:41.800095 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:17:41.800002 2578 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="850ea628-4496-41f7-839c-5233a8831455" containerName="kube-rbac-proxy-web" containerID="cri-o://d65b0d8c8c320a51488aac9d021ce7d2de30c1173a3b0f200c26f19f110bf83b" gracePeriod=600 Jul 07 11:17:42.822292 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:17:42.822260 2578 generic.go:356] "Generic (PLEG): container finished" podID="850ea628-4496-41f7-839c-5233a8831455" containerID="cdde943e1ddcdbb5dcd233e7963037b127015ff2f70530bed844906e7a52db04" exitCode=0 Jul 07 11:17:42.822292 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:17:42.822286 2578 generic.go:356] "Generic (PLEG): container finished" podID="850ea628-4496-41f7-839c-5233a8831455" containerID="c532e02e1b8ba45be092c065567941d63731e3096a62eec267ab457b17456975" exitCode=0 Jul 07 11:17:42.822292 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:17:42.822295 2578 generic.go:356] "Generic (PLEG): container finished" podID="850ea628-4496-41f7-839c-5233a8831455" containerID="453fb6d8710dbd3680fce0cd3b52b3305f3f9cca5b5d58341bdf259c65a433a4" exitCode=0 Jul 07 11:17:42.822292 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:17:42.822301 2578 generic.go:356] "Generic (PLEG): container finished" podID="850ea628-4496-41f7-839c-5233a8831455" containerID="c418aa245eca303145bb03fbcb9a31fa1de8856257ab73a75674887b720ab31d" exitCode=0 Jul 07 11:17:42.822768 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:17:42.822309 2578 generic.go:356] "Generic (PLEG): container finished" podID="850ea628-4496-41f7-839c-5233a8831455" containerID="f4571a2743a77df32f8a41cb9a23bb5ac17fffbf0b52b30957bc5b9565a745f1" exitCode=0 Jul 07 11:17:42.822768 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:17:42.822327 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"850ea628-4496-41f7-839c-5233a8831455","Type":"ContainerDied","Data":"cdde943e1ddcdbb5dcd233e7963037b127015ff2f70530bed844906e7a52db04"} Jul 07 11:17:42.822768 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:17:42.822387 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"850ea628-4496-41f7-839c-5233a8831455","Type":"ContainerDied","Data":"c532e02e1b8ba45be092c065567941d63731e3096a62eec267ab457b17456975"} Jul 07 11:17:42.822768 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:17:42.822413 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"850ea628-4496-41f7-839c-5233a8831455","Type":"ContainerDied","Data":"453fb6d8710dbd3680fce0cd3b52b3305f3f9cca5b5d58341bdf259c65a433a4"} Jul 07 11:17:42.822768 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:17:42.822459 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"850ea628-4496-41f7-839c-5233a8831455","Type":"ContainerDied","Data":"c418aa245eca303145bb03fbcb9a31fa1de8856257ab73a75674887b720ab31d"} Jul 07 11:17:42.822768 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:17:42.822485 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"850ea628-4496-41f7-839c-5233a8831455","Type":"ContainerDied","Data":"f4571a2743a77df32f8a41cb9a23bb5ac17fffbf0b52b30957bc5b9565a745f1"} Jul 07 11:17:43.039869 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:17:43.039847 2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" Jul 07 11:17:43.134784 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:17:43.134712 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/850ea628-4496-41f7-839c-5233a8831455-config\") pod \"850ea628-4496-41f7-839c-5233a8831455\" (UID: \"850ea628-4496-41f7-839c-5233a8831455\") " Jul 07 11:17:43.134784 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:17:43.134758 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/850ea628-4496-41f7-839c-5233a8831455-configmap-kubelet-serving-ca-bundle\") pod \"850ea628-4496-41f7-839c-5233a8831455\" (UID: \"850ea628-4496-41f7-839c-5233a8831455\") " Jul 07 11:17:43.134951 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:17:43.134792 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/850ea628-4496-41f7-839c-5233a8831455-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"850ea628-4496-41f7-839c-5233a8831455\" (UID: \"850ea628-4496-41f7-839c-5233a8831455\") " Jul 07 11:17:43.134951 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:17:43.134829 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/850ea628-4496-41f7-839c-5233a8831455-secret-metrics-client-certs\") pod \"850ea628-4496-41f7-839c-5233a8831455\" (UID: \"850ea628-4496-41f7-839c-5233a8831455\") " Jul 07 11:17:43.134951 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:17:43.134857 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/850ea628-4496-41f7-839c-5233a8831455-prometheus-k8s-rulefiles-1\") pod \"850ea628-4496-41f7-839c-5233a8831455\" (UID: \"850ea628-4496-41f7-839c-5233a8831455\") " Jul 07 11:17:43.134951 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:17:43.134884 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/850ea628-4496-41f7-839c-5233a8831455-web-config\") pod \"850ea628-4496-41f7-839c-5233a8831455\" (UID: \"850ea628-4496-41f7-839c-5233a8831455\") " Jul 07 11:17:43.134951 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:17:43.134908 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/850ea628-4496-41f7-839c-5233a8831455-thanos-prometheus-http-client-file\") pod \"850ea628-4496-41f7-839c-5233a8831455\" (UID: \"850ea628-4496-41f7-839c-5233a8831455\") " Jul 07 11:17:43.134951 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:17:43.134937 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/850ea628-4496-41f7-839c-5233a8831455-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"850ea628-4496-41f7-839c-5233a8831455\" (UID: \"850ea628-4496-41f7-839c-5233a8831455\") " Jul 07 11:17:43.135237 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:17:43.134961 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/850ea628-4496-41f7-839c-5233a8831455-configmap-serving-certs-ca-bundle\") pod \"850ea628-4496-41f7-839c-5233a8831455\" (UID: \"850ea628-4496-41f7-839c-5233a8831455\") " Jul 07 11:17:43.135237 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:17:43.134997 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/850ea628-4496-41f7-839c-5233a8831455-configmap-metrics-client-ca\") pod \"850ea628-4496-41f7-839c-5233a8831455\" (UID: \"850ea628-4496-41f7-839c-5233a8831455\") " Jul 07 11:17:43.135237 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:17:43.135034 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/850ea628-4496-41f7-839c-5233a8831455-tls-assets\") pod \"850ea628-4496-41f7-839c-5233a8831455\" (UID: \"850ea628-4496-41f7-839c-5233a8831455\") " Jul 07 11:17:43.135237 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:17:43.135066 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/850ea628-4496-41f7-839c-5233a8831455-config-out\") pod \"850ea628-4496-41f7-839c-5233a8831455\" (UID: \"850ea628-4496-41f7-839c-5233a8831455\") " Jul 07 11:17:43.135237 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:17:43.135092 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/850ea628-4496-41f7-839c-5233a8831455-secret-prometheus-k8s-tls\") pod \"850ea628-4496-41f7-839c-5233a8831455\" (UID: \"850ea628-4496-41f7-839c-5233a8831455\") " Jul 07 11:17:43.135237 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:17:43.135124 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/850ea628-4496-41f7-839c-5233a8831455-secret-kube-rbac-proxy\") pod \"850ea628-4496-41f7-839c-5233a8831455\" (UID: \"850ea628-4496-41f7-839c-5233a8831455\") " Jul 07 11:17:43.135237 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:17:43.135175 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/850ea628-4496-41f7-839c-5233a8831455-prometheus-trusted-ca-bundle\") pod \"850ea628-4496-41f7-839c-5233a8831455\" (UID: \"850ea628-4496-41f7-839c-5233a8831455\") " Jul 07 11:17:43.135237 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:17:43.135202 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/850ea628-4496-41f7-839c-5233a8831455-prometheus-k8s-rulefiles-2\") pod \"850ea628-4496-41f7-839c-5233a8831455\" (UID: \"850ea628-4496-41f7-839c-5233a8831455\") " Jul 07 11:17:43.135659 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:17:43.135245 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-lxd49\" (UniqueName: \"kubernetes.io/projected/850ea628-4496-41f7-839c-5233a8831455-kube-api-access-lxd49\") pod \"850ea628-4496-41f7-839c-5233a8831455\" (UID: \"850ea628-4496-41f7-839c-5233a8831455\") " Jul 07 11:17:43.135659 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:17:43.135249 2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/850ea628-4496-41f7-839c-5233a8831455-configmap-kubelet-serving-ca-bundle" (OuterVolumeSpecName: "configmap-kubelet-serving-ca-bundle") pod "850ea628-4496-41f7-839c-5233a8831455" (UID: "850ea628-4496-41f7-839c-5233a8831455"). InnerVolumeSpecName "configmap-kubelet-serving-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 07 11:17:43.135659 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:17:43.135268 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/850ea628-4496-41f7-839c-5233a8831455-secret-grpc-tls\") pod \"850ea628-4496-41f7-839c-5233a8831455\" (UID: \"850ea628-4496-41f7-839c-5233a8831455\") " Jul 07 11:17:43.135659 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:17:43.135338 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/850ea628-4496-41f7-839c-5233a8831455-prometheus-k8s-rulefiles-0\") pod \"850ea628-4496-41f7-839c-5233a8831455\" (UID: \"850ea628-4496-41f7-839c-5233a8831455\") " Jul 07 11:17:43.135659 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:17:43.135388 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/850ea628-4496-41f7-839c-5233a8831455-prometheus-k8s-db\") pod \"850ea628-4496-41f7-839c-5233a8831455\" (UID: \"850ea628-4496-41f7-839c-5233a8831455\") " Jul 07 11:17:43.135908 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:17:43.135780 2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/850ea628-4496-41f7-839c-5233a8831455-prometheus-k8s-rulefiles-1" (OuterVolumeSpecName: "prometheus-k8s-rulefiles-1") pod "850ea628-4496-41f7-839c-5233a8831455" (UID: "850ea628-4496-41f7-839c-5233a8831455"). InnerVolumeSpecName "prometheus-k8s-rulefiles-1". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 07 11:17:43.137174 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:17:43.136529 2578 reconciler_common.go:299] "Volume detached for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/850ea628-4496-41f7-839c-5233a8831455-configmap-kubelet-serving-ca-bundle\") on node \"ip-10-0-142-112.ec2.internal\" DevicePath \"\"" Jul 07 11:17:43.137174 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:17:43.136561 2578 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/850ea628-4496-41f7-839c-5233a8831455-prometheus-k8s-rulefiles-1\") on node \"ip-10-0-142-112.ec2.internal\" DevicePath \"\"" Jul 07 11:17:43.137493 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:17:43.137467 2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/850ea628-4496-41f7-839c-5233a8831455-prometheus-k8s-rulefiles-2" (OuterVolumeSpecName: "prometheus-k8s-rulefiles-2") pod "850ea628-4496-41f7-839c-5233a8831455" (UID: "850ea628-4496-41f7-839c-5233a8831455"). InnerVolumeSpecName "prometheus-k8s-rulefiles-2". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 07 11:17:43.137838 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:17:43.137805 2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/850ea628-4496-41f7-839c-5233a8831455-prometheus-trusted-ca-bundle" (OuterVolumeSpecName: "prometheus-trusted-ca-bundle") pod "850ea628-4496-41f7-839c-5233a8831455" (UID: "850ea628-4496-41f7-839c-5233a8831455"). InnerVolumeSpecName "prometheus-trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 07 11:17:43.138379 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:17:43.138116 2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/850ea628-4496-41f7-839c-5233a8831455-prometheus-k8s-rulefiles-0" (OuterVolumeSpecName: "prometheus-k8s-rulefiles-0") pod "850ea628-4496-41f7-839c-5233a8831455" (UID: "850ea628-4496-41f7-839c-5233a8831455"). InnerVolumeSpecName "prometheus-k8s-rulefiles-0". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 07 11:17:43.138379 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:17:43.138132 2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/850ea628-4496-41f7-839c-5233a8831455-configmap-metrics-client-ca" (OuterVolumeSpecName: "configmap-metrics-client-ca") pod "850ea628-4496-41f7-839c-5233a8831455" (UID: "850ea628-4496-41f7-839c-5233a8831455"). InnerVolumeSpecName "configmap-metrics-client-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 07 11:17:43.138379 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:17:43.138174 2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/850ea628-4496-41f7-839c-5233a8831455-prometheus-k8s-db" (OuterVolumeSpecName: "prometheus-k8s-db") pod "850ea628-4496-41f7-839c-5233a8831455" (UID: "850ea628-4496-41f7-839c-5233a8831455"). InnerVolumeSpecName "prometheus-k8s-db". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 11:17:43.138625 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:17:43.138530 2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/850ea628-4496-41f7-839c-5233a8831455-configmap-serving-certs-ca-bundle" (OuterVolumeSpecName: "configmap-serving-certs-ca-bundle") pod "850ea628-4496-41f7-839c-5233a8831455" (UID: "850ea628-4496-41f7-839c-5233a8831455"). InnerVolumeSpecName "configmap-serving-certs-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 07 11:17:43.140127 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:17:43.140068 2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/850ea628-4496-41f7-839c-5233a8831455-thanos-prometheus-http-client-file" (OuterVolumeSpecName: "thanos-prometheus-http-client-file") pod "850ea628-4496-41f7-839c-5233a8831455" (UID: "850ea628-4496-41f7-839c-5233a8831455"). InnerVolumeSpecName "thanos-prometheus-http-client-file". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 07 11:17:43.140614 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:17:43.140578 2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/850ea628-4496-41f7-839c-5233a8831455-secret-grpc-tls" (OuterVolumeSpecName: "secret-grpc-tls") pod "850ea628-4496-41f7-839c-5233a8831455" (UID: "850ea628-4496-41f7-839c-5233a8831455"). InnerVolumeSpecName "secret-grpc-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 07 11:17:43.142535 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:17:43.141006 2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/850ea628-4496-41f7-839c-5233a8831455-secret-metrics-client-certs" (OuterVolumeSpecName: "secret-metrics-client-certs") pod "850ea628-4496-41f7-839c-5233a8831455" (UID: "850ea628-4496-41f7-839c-5233a8831455"). InnerVolumeSpecName "secret-metrics-client-certs". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 07 11:17:43.142535 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:17:43.141028 2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/850ea628-4496-41f7-839c-5233a8831455-tls-assets" (OuterVolumeSpecName: "tls-assets") pod "850ea628-4496-41f7-839c-5233a8831455" (UID: "850ea628-4496-41f7-839c-5233a8831455"). InnerVolumeSpecName "tls-assets". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 07 11:17:43.142535 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:17:43.141287 2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/850ea628-4496-41f7-839c-5233a8831455-kube-api-access-lxd49" (OuterVolumeSpecName: "kube-api-access-lxd49") pod "850ea628-4496-41f7-839c-5233a8831455" (UID: "850ea628-4496-41f7-839c-5233a8831455"). InnerVolumeSpecName "kube-api-access-lxd49". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 07 11:17:43.142535 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:17:43.141560 2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/850ea628-4496-41f7-839c-5233a8831455-config-out" (OuterVolumeSpecName: "config-out") pod "850ea628-4496-41f7-839c-5233a8831455" (UID: "850ea628-4496-41f7-839c-5233a8831455"). InnerVolumeSpecName "config-out". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 11:17:43.144278 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:17:43.144152 2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/850ea628-4496-41f7-839c-5233a8831455-secret-prometheus-k8s-kube-rbac-proxy-web" (OuterVolumeSpecName: "secret-prometheus-k8s-kube-rbac-proxy-web") pod "850ea628-4496-41f7-839c-5233a8831455" (UID: "850ea628-4496-41f7-839c-5233a8831455"). InnerVolumeSpecName "secret-prometheus-k8s-kube-rbac-proxy-web". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 07 11:17:43.144609 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:17:43.144474 2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/850ea628-4496-41f7-839c-5233a8831455-config" (OuterVolumeSpecName: "config") pod "850ea628-4496-41f7-839c-5233a8831455" (UID: "850ea628-4496-41f7-839c-5233a8831455"). InnerVolumeSpecName "config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 07 11:17:43.145155 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:17:43.144725 2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/850ea628-4496-41f7-839c-5233a8831455-secret-prometheus-k8s-tls" (OuterVolumeSpecName: "secret-prometheus-k8s-tls") pod "850ea628-4496-41f7-839c-5233a8831455" (UID: "850ea628-4496-41f7-839c-5233a8831455"). InnerVolumeSpecName "secret-prometheus-k8s-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 07 11:17:43.145155 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:17:43.144899 2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/850ea628-4496-41f7-839c-5233a8831455-secret-kube-rbac-proxy" (OuterVolumeSpecName: "secret-kube-rbac-proxy") pod "850ea628-4496-41f7-839c-5233a8831455" (UID: "850ea628-4496-41f7-839c-5233a8831455"). InnerVolumeSpecName "secret-kube-rbac-proxy". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 07 11:17:43.145155 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:17:43.145044 2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/850ea628-4496-41f7-839c-5233a8831455-secret-prometheus-k8s-thanos-sidecar-tls" (OuterVolumeSpecName: "secret-prometheus-k8s-thanos-sidecar-tls") pod "850ea628-4496-41f7-839c-5233a8831455" (UID: "850ea628-4496-41f7-839c-5233a8831455"). InnerVolumeSpecName "secret-prometheus-k8s-thanos-sidecar-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 07 11:17:43.154198 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:17:43.154176 2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/850ea628-4496-41f7-839c-5233a8831455-web-config" (OuterVolumeSpecName: "web-config") pod "850ea628-4496-41f7-839c-5233a8831455" (UID: "850ea628-4496-41f7-839c-5233a8831455"). InnerVolumeSpecName "web-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 07 11:17:43.237727 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:17:43.237679 2578 reconciler_common.go:299] "Volume detached for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/850ea628-4496-41f7-839c-5233a8831455-prometheus-trusted-ca-bundle\") on node \"ip-10-0-142-112.ec2.internal\" DevicePath \"\"" Jul 07 11:17:43.237727 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:17:43.237723 2578 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/850ea628-4496-41f7-839c-5233a8831455-prometheus-k8s-rulefiles-2\") on node \"ip-10-0-142-112.ec2.internal\" DevicePath \"\"" Jul 07 11:17:43.237727 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:17:43.237735 2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-lxd49\" (UniqueName: \"kubernetes.io/projected/850ea628-4496-41f7-839c-5233a8831455-kube-api-access-lxd49\") on node \"ip-10-0-142-112.ec2.internal\" DevicePath \"\"" Jul 07 11:17:43.237727 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:17:43.237745 2578 reconciler_common.go:299] "Volume detached for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/850ea628-4496-41f7-839c-5233a8831455-secret-grpc-tls\") on node \"ip-10-0-142-112.ec2.internal\" DevicePath \"\"" Jul 07 11:17:43.237995 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:17:43.237756 2578 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/850ea628-4496-41f7-839c-5233a8831455-prometheus-k8s-rulefiles-0\") on node \"ip-10-0-142-112.ec2.internal\" DevicePath \"\"" Jul 07 11:17:43.237995 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:17:43.237764 2578 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/850ea628-4496-41f7-839c-5233a8831455-prometheus-k8s-db\") on node \"ip-10-0-142-112.ec2.internal\" DevicePath \"\"" Jul 07 11:17:43.237995 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:17:43.237772 2578 reconciler_common.go:299] "Volume detached for volume \"config\" (UniqueName: \"kubernetes.io/secret/850ea628-4496-41f7-839c-5233a8831455-config\") on node \"ip-10-0-142-112.ec2.internal\" DevicePath \"\"" Jul 07 11:17:43.237995 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:17:43.237782 2578 reconciler_common.go:299] "Volume detached for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/850ea628-4496-41f7-839c-5233a8831455-secret-prometheus-k8s-kube-rbac-proxy-web\") on node \"ip-10-0-142-112.ec2.internal\" DevicePath \"\"" Jul 07 11:17:43.237995 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:17:43.237791 2578 reconciler_common.go:299] "Volume detached for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/850ea628-4496-41f7-839c-5233a8831455-secret-metrics-client-certs\") on node \"ip-10-0-142-112.ec2.internal\" DevicePath \"\"" Jul 07 11:17:43.237995 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:17:43.237821 2578 reconciler_common.go:299] "Volume detached for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/850ea628-4496-41f7-839c-5233a8831455-web-config\") on node \"ip-10-0-142-112.ec2.internal\" DevicePath \"\"" Jul 07 11:17:43.237995 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:17:43.237831 2578 reconciler_common.go:299] "Volume detached for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/850ea628-4496-41f7-839c-5233a8831455-thanos-prometheus-http-client-file\") on node \"ip-10-0-142-112.ec2.internal\" DevicePath \"\"" Jul 07 11:17:43.237995 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:17:43.237839 2578 reconciler_common.go:299] "Volume detached for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/850ea628-4496-41f7-839c-5233a8831455-secret-prometheus-k8s-thanos-sidecar-tls\") on node \"ip-10-0-142-112.ec2.internal\" DevicePath \"\"" Jul 07 11:17:43.237995 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:17:43.237848 2578 reconciler_common.go:299] "Volume detached for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/850ea628-4496-41f7-839c-5233a8831455-configmap-serving-certs-ca-bundle\") on node \"ip-10-0-142-112.ec2.internal\" DevicePath \"\"" Jul 07 11:17:43.237995 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:17:43.237856 2578 reconciler_common.go:299] "Volume detached for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/850ea628-4496-41f7-839c-5233a8831455-configmap-metrics-client-ca\") on node \"ip-10-0-142-112.ec2.internal\" DevicePath \"\"" Jul 07 11:17:43.237995 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:17:43.237864 2578 reconciler_common.go:299] "Volume detached for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/850ea628-4496-41f7-839c-5233a8831455-tls-assets\") on node \"ip-10-0-142-112.ec2.internal\" DevicePath \"\"" Jul 07 11:17:43.237995 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:17:43.237871 2578 reconciler_common.go:299] "Volume detached for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/850ea628-4496-41f7-839c-5233a8831455-config-out\") on node \"ip-10-0-142-112.ec2.internal\" DevicePath \"\"" Jul 07 11:17:43.237995 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:17:43.237879 2578 reconciler_common.go:299] "Volume detached for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/850ea628-4496-41f7-839c-5233a8831455-secret-prometheus-k8s-tls\") on node \"ip-10-0-142-112.ec2.internal\" DevicePath \"\"" Jul 07 11:17:43.237995 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:17:43.237887 2578 reconciler_common.go:299] "Volume detached for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/850ea628-4496-41f7-839c-5233a8831455-secret-kube-rbac-proxy\") on node \"ip-10-0-142-112.ec2.internal\" DevicePath \"\"" Jul 07 11:17:43.829073 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:17:43.829037 2578 generic.go:356] "Generic (PLEG): container finished" podID="850ea628-4496-41f7-839c-5233a8831455" containerID="d65b0d8c8c320a51488aac9d021ce7d2de30c1173a3b0f200c26f19f110bf83b" exitCode=0 Jul 07 11:17:43.829543 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:17:43.829120 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"850ea628-4496-41f7-839c-5233a8831455","Type":"ContainerDied","Data":"d65b0d8c8c320a51488aac9d021ce7d2de30c1173a3b0f200c26f19f110bf83b"} Jul 07 11:17:43.829543 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:17:43.829136 2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" Jul 07 11:17:43.829543 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:17:43.829192 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"850ea628-4496-41f7-839c-5233a8831455","Type":"ContainerDied","Data":"3849121e3669cb53faab41e1d0e381d46e9ad3c5ed3e664c81d85417f9bcec6e"} Jul 07 11:17:43.829543 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:17:43.829225 2578 scope.go:117] "RemoveContainer" containerID="cdde943e1ddcdbb5dcd233e7963037b127015ff2f70530bed844906e7a52db04" Jul 07 11:17:43.837755 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:17:43.837732 2578 scope.go:117] "RemoveContainer" containerID="c532e02e1b8ba45be092c065567941d63731e3096a62eec267ab457b17456975" Jul 07 11:17:43.844346 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:17:43.844329 2578 scope.go:117] "RemoveContainer" containerID="d65b0d8c8c320a51488aac9d021ce7d2de30c1173a3b0f200c26f19f110bf83b" Jul 07 11:17:43.850732 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:17:43.850716 2578 scope.go:117] "RemoveContainer" containerID="453fb6d8710dbd3680fce0cd3b52b3305f3f9cca5b5d58341bdf259c65a433a4" Jul 07 11:17:43.855001 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:17:43.854980 2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jul 07 11:17:43.857490 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:17:43.857470 2578 scope.go:117] "RemoveContainer" containerID="c418aa245eca303145bb03fbcb9a31fa1de8856257ab73a75674887b720ab31d" Jul 07 11:17:43.860598 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:17:43.860559 2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jul 07 11:17:43.864487 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:17:43.864467 2578 scope.go:117] "RemoveContainer" containerID="f4571a2743a77df32f8a41cb9a23bb5ac17fffbf0b52b30957bc5b9565a745f1" Jul 07 11:17:43.871352 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:17:43.871334 2578 scope.go:117] "RemoveContainer" containerID="a6be01a6cfd01c0d23c702625286f20cfa8f27c6d83f547ac992f392660ec0cc" Jul 07 11:17:43.877657 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:17:43.877629 2578 scope.go:117] "RemoveContainer" containerID="cdde943e1ddcdbb5dcd233e7963037b127015ff2f70530bed844906e7a52db04" Jul 07 11:17:43.877897 ip-10-0-142-112 kubenswrapper[2578]: E0707 11:17:43.877877 2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"cdde943e1ddcdbb5dcd233e7963037b127015ff2f70530bed844906e7a52db04\": container with ID starting with cdde943e1ddcdbb5dcd233e7963037b127015ff2f70530bed844906e7a52db04 not found: ID does not exist" containerID="cdde943e1ddcdbb5dcd233e7963037b127015ff2f70530bed844906e7a52db04" Jul 07 11:17:43.877950 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:17:43.877905 2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"cdde943e1ddcdbb5dcd233e7963037b127015ff2f70530bed844906e7a52db04"} err="failed to get container status \"cdde943e1ddcdbb5dcd233e7963037b127015ff2f70530bed844906e7a52db04\": rpc error: code = NotFound desc = could not find container \"cdde943e1ddcdbb5dcd233e7963037b127015ff2f70530bed844906e7a52db04\": container with ID starting with cdde943e1ddcdbb5dcd233e7963037b127015ff2f70530bed844906e7a52db04 not found: ID does not exist" Jul 07 11:17:43.877950 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:17:43.877924 2578 scope.go:117] "RemoveContainer" containerID="c532e02e1b8ba45be092c065567941d63731e3096a62eec267ab457b17456975" Jul 07 11:17:43.878166 ip-10-0-142-112 kubenswrapper[2578]: E0707 11:17:43.878148 2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c532e02e1b8ba45be092c065567941d63731e3096a62eec267ab457b17456975\": container with ID starting with c532e02e1b8ba45be092c065567941d63731e3096a62eec267ab457b17456975 not found: ID does not exist" containerID="c532e02e1b8ba45be092c065567941d63731e3096a62eec267ab457b17456975" Jul 07 11:17:43.878225 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:17:43.878178 2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c532e02e1b8ba45be092c065567941d63731e3096a62eec267ab457b17456975"} err="failed to get container status \"c532e02e1b8ba45be092c065567941d63731e3096a62eec267ab457b17456975\": rpc error: code = NotFound desc = could not find container \"c532e02e1b8ba45be092c065567941d63731e3096a62eec267ab457b17456975\": container with ID starting with c532e02e1b8ba45be092c065567941d63731e3096a62eec267ab457b17456975 not found: ID does not exist" Jul 07 11:17:43.878225 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:17:43.878201 2578 scope.go:117] "RemoveContainer" containerID="d65b0d8c8c320a51488aac9d021ce7d2de30c1173a3b0f200c26f19f110bf83b" Jul 07 11:17:43.878494 ip-10-0-142-112 kubenswrapper[2578]: E0707 11:17:43.878468 2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d65b0d8c8c320a51488aac9d021ce7d2de30c1173a3b0f200c26f19f110bf83b\": container with ID starting with d65b0d8c8c320a51488aac9d021ce7d2de30c1173a3b0f200c26f19f110bf83b not found: ID does not exist" containerID="d65b0d8c8c320a51488aac9d021ce7d2de30c1173a3b0f200c26f19f110bf83b" Jul 07 11:17:43.878581 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:17:43.878496 2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d65b0d8c8c320a51488aac9d021ce7d2de30c1173a3b0f200c26f19f110bf83b"} err="failed to get container status \"d65b0d8c8c320a51488aac9d021ce7d2de30c1173a3b0f200c26f19f110bf83b\": rpc error: code = NotFound desc = could not find container \"d65b0d8c8c320a51488aac9d021ce7d2de30c1173a3b0f200c26f19f110bf83b\": container with ID starting with d65b0d8c8c320a51488aac9d021ce7d2de30c1173a3b0f200c26f19f110bf83b not found: ID does not exist" Jul 07 11:17:43.878581 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:17:43.878511 2578 scope.go:117] "RemoveContainer" containerID="453fb6d8710dbd3680fce0cd3b52b3305f3f9cca5b5d58341bdf259c65a433a4" Jul 07 11:17:43.878741 ip-10-0-142-112 kubenswrapper[2578]: E0707 11:17:43.878726 2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"453fb6d8710dbd3680fce0cd3b52b3305f3f9cca5b5d58341bdf259c65a433a4\": container with ID starting with 453fb6d8710dbd3680fce0cd3b52b3305f3f9cca5b5d58341bdf259c65a433a4 not found: ID does not exist" containerID="453fb6d8710dbd3680fce0cd3b52b3305f3f9cca5b5d58341bdf259c65a433a4" Jul 07 11:17:43.878779 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:17:43.878746 2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"453fb6d8710dbd3680fce0cd3b52b3305f3f9cca5b5d58341bdf259c65a433a4"} err="failed to get container status \"453fb6d8710dbd3680fce0cd3b52b3305f3f9cca5b5d58341bdf259c65a433a4\": rpc error: code = NotFound desc = could not find container \"453fb6d8710dbd3680fce0cd3b52b3305f3f9cca5b5d58341bdf259c65a433a4\": container with ID starting with 453fb6d8710dbd3680fce0cd3b52b3305f3f9cca5b5d58341bdf259c65a433a4 not found: ID does not exist" Jul 07 11:17:43.878779 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:17:43.878758 2578 scope.go:117] "RemoveContainer" containerID="c418aa245eca303145bb03fbcb9a31fa1de8856257ab73a75674887b720ab31d" Jul 07 11:17:43.878982 ip-10-0-142-112 kubenswrapper[2578]: E0707 11:17:43.878965 2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c418aa245eca303145bb03fbcb9a31fa1de8856257ab73a75674887b720ab31d\": container with ID starting with c418aa245eca303145bb03fbcb9a31fa1de8856257ab73a75674887b720ab31d not found: ID does not exist" containerID="c418aa245eca303145bb03fbcb9a31fa1de8856257ab73a75674887b720ab31d" Jul 07 11:17:43.879034 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:17:43.878987 2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c418aa245eca303145bb03fbcb9a31fa1de8856257ab73a75674887b720ab31d"} err="failed to get container status \"c418aa245eca303145bb03fbcb9a31fa1de8856257ab73a75674887b720ab31d\": rpc error: code = NotFound desc = could not find container \"c418aa245eca303145bb03fbcb9a31fa1de8856257ab73a75674887b720ab31d\": container with ID starting with c418aa245eca303145bb03fbcb9a31fa1de8856257ab73a75674887b720ab31d not found: ID does not exist" Jul 07 11:17:43.879034 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:17:43.879010 2578 scope.go:117] "RemoveContainer" containerID="f4571a2743a77df32f8a41cb9a23bb5ac17fffbf0b52b30957bc5b9565a745f1" Jul 07 11:17:43.879237 ip-10-0-142-112 kubenswrapper[2578]: E0707 11:17:43.879212 2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f4571a2743a77df32f8a41cb9a23bb5ac17fffbf0b52b30957bc5b9565a745f1\": container with ID starting with f4571a2743a77df32f8a41cb9a23bb5ac17fffbf0b52b30957bc5b9565a745f1 not found: ID does not exist" containerID="f4571a2743a77df32f8a41cb9a23bb5ac17fffbf0b52b30957bc5b9565a745f1" Jul 07 11:17:43.879299 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:17:43.879247 2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f4571a2743a77df32f8a41cb9a23bb5ac17fffbf0b52b30957bc5b9565a745f1"} err="failed to get container status \"f4571a2743a77df32f8a41cb9a23bb5ac17fffbf0b52b30957bc5b9565a745f1\": rpc error: code = NotFound desc = could not find container \"f4571a2743a77df32f8a41cb9a23bb5ac17fffbf0b52b30957bc5b9565a745f1\": container with ID starting with f4571a2743a77df32f8a41cb9a23bb5ac17fffbf0b52b30957bc5b9565a745f1 not found: ID does not exist" Jul 07 11:17:43.879299 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:17:43.879269 2578 scope.go:117] "RemoveContainer" containerID="a6be01a6cfd01c0d23c702625286f20cfa8f27c6d83f547ac992f392660ec0cc" Jul 07 11:17:43.879535 ip-10-0-142-112 kubenswrapper[2578]: E0707 11:17:43.879518 2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a6be01a6cfd01c0d23c702625286f20cfa8f27c6d83f547ac992f392660ec0cc\": container with ID starting with a6be01a6cfd01c0d23c702625286f20cfa8f27c6d83f547ac992f392660ec0cc not found: ID does not exist" containerID="a6be01a6cfd01c0d23c702625286f20cfa8f27c6d83f547ac992f392660ec0cc" Jul 07 11:17:43.879576 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:17:43.879538 2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a6be01a6cfd01c0d23c702625286f20cfa8f27c6d83f547ac992f392660ec0cc"} err="failed to get container status \"a6be01a6cfd01c0d23c702625286f20cfa8f27c6d83f547ac992f392660ec0cc\": rpc error: code = NotFound desc = could not find container \"a6be01a6cfd01c0d23c702625286f20cfa8f27c6d83f547ac992f392660ec0cc\": container with ID starting with a6be01a6cfd01c0d23c702625286f20cfa8f27c6d83f547ac992f392660ec0cc not found: ID does not exist" Jul 07 11:17:43.918028 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:17:43.917995 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jul 07 11:17:43.918273 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:17:43.918262 2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="850ea628-4496-41f7-839c-5233a8831455" containerName="kube-rbac-proxy-web" Jul 07 11:17:43.918309 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:17:43.918275 2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="850ea628-4496-41f7-839c-5233a8831455" containerName="kube-rbac-proxy-web" Jul 07 11:17:43.918309 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:17:43.918284 2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="850ea628-4496-41f7-839c-5233a8831455" containerName="thanos-sidecar" Jul 07 11:17:43.918309 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:17:43.918290 2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="850ea628-4496-41f7-839c-5233a8831455" containerName="thanos-sidecar" Jul 07 11:17:43.918309 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:17:43.918300 2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="850ea628-4496-41f7-839c-5233a8831455" containerName="prometheus" Jul 07 11:17:43.918309 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:17:43.918306 2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="850ea628-4496-41f7-839c-5233a8831455" containerName="prometheus" Jul 07 11:17:43.918483 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:17:43.918313 2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="850ea628-4496-41f7-839c-5233a8831455" containerName="kube-rbac-proxy-thanos" Jul 07 11:17:43.918483 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:17:43.918318 2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="850ea628-4496-41f7-839c-5233a8831455" containerName="kube-rbac-proxy-thanos" Jul 07 11:17:43.918483 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:17:43.918326 2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f52d638f-d46b-465b-a454-f34152acb425" containerName="console" Jul 07 11:17:43.918483 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:17:43.918331 2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="f52d638f-d46b-465b-a454-f34152acb425" containerName="console" Jul 07 11:17:43.918483 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:17:43.918343 2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="850ea628-4496-41f7-839c-5233a8831455" containerName="config-reloader" Jul 07 11:17:43.918483 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:17:43.918349 2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="850ea628-4496-41f7-839c-5233a8831455" containerName="config-reloader" Jul 07 11:17:43.918483 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:17:43.918356 2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="850ea628-4496-41f7-839c-5233a8831455" containerName="init-config-reloader" Jul 07 11:17:43.918483 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:17:43.918362 2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="850ea628-4496-41f7-839c-5233a8831455" containerName="init-config-reloader" Jul 07 11:17:43.918483 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:17:43.918368 2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="850ea628-4496-41f7-839c-5233a8831455" containerName="kube-rbac-proxy" Jul 07 11:17:43.918483 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:17:43.918374 2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="850ea628-4496-41f7-839c-5233a8831455" containerName="kube-rbac-proxy" Jul 07 11:17:43.918483 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:17:43.918414 2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="850ea628-4496-41f7-839c-5233a8831455" containerName="config-reloader" Jul 07 11:17:43.918483 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:17:43.918434 2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="850ea628-4496-41f7-839c-5233a8831455" containerName="kube-rbac-proxy-web" Jul 07 11:17:43.918483 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:17:43.918442 2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="850ea628-4496-41f7-839c-5233a8831455" containerName="prometheus" Jul 07 11:17:43.918483 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:17:43.918448 2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="850ea628-4496-41f7-839c-5233a8831455" containerName="kube-rbac-proxy-thanos" Jul 07 11:17:43.918483 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:17:43.918453 2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="f52d638f-d46b-465b-a454-f34152acb425" containerName="console" Jul 07 11:17:43.918483 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:17:43.918458 2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="850ea628-4496-41f7-839c-5233a8831455" containerName="kube-rbac-proxy" Jul 07 11:17:43.918483 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:17:43.918463 2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="850ea628-4496-41f7-839c-5233a8831455" containerName="thanos-sidecar" Jul 07 11:17:43.922373 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:17:43.922358 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" Jul 07 11:17:43.925910 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:17:43.925887 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-grpc-tls-ch72mp8e5duvf\"" Jul 07 11:17:43.926502 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:17:43.926485 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-prometheus-http-client-file\"" Jul 07 11:17:43.926605 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:17:43.926488 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-client-certs\"" Jul 07 11:17:43.936906 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:17:43.936887 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s\"" Jul 07 11:17:43.937166 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:17:43.937151 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-1\"" Jul 07 11:17:43.937297 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:17:43.937280 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-kube-rbac-proxy-web\"" Jul 07 11:17:43.937379 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:17:43.937300 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-web-config\"" Jul 07 11:17:43.937379 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:17:43.937282 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-2\"" Jul 07 11:17:43.937379 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:17:43.937335 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls-assets-0\"" Jul 07 11:17:43.937567 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:17:43.937452 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-rbac-proxy\"" Jul 07 11:17:43.937703 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:17:43.937686 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"serving-certs-ca-bundle\"" Jul 07 11:17:43.937800 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:17:43.937778 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-sidecar-tls\"" Jul 07 11:17:43.937800 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:17:43.937790 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kubelet-serving-ca-bundle\"" Jul 07 11:17:43.937934 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:17:43.937909 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls\"" Jul 07 11:17:43.938029 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:17:43.938010 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-dockercfg-qk6v2\"" Jul 07 11:17:43.940238 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:17:43.940223 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-0\"" Jul 07 11:17:43.941880 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:17:43.941860 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/fd952068-3307-4f9d-82a0-dd29b894f014-config\") pod \"prometheus-k8s-0\" (UID: \"fd952068-3307-4f9d-82a0-dd29b894f014\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 11:17:43.941976 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:17:43.941895 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/fd952068-3307-4f9d-82a0-dd29b894f014-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"fd952068-3307-4f9d-82a0-dd29b894f014\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 11:17:43.941976 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:17:43.941953 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/fd952068-3307-4f9d-82a0-dd29b894f014-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"fd952068-3307-4f9d-82a0-dd29b894f014\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 11:17:43.942043 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:17:43.941988 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/fd952068-3307-4f9d-82a0-dd29b894f014-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"fd952068-3307-4f9d-82a0-dd29b894f014\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 11:17:43.942043 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:17:43.942013 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/fd952068-3307-4f9d-82a0-dd29b894f014-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"fd952068-3307-4f9d-82a0-dd29b894f014\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 11:17:43.942043 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:17:43.942035 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/fd952068-3307-4f9d-82a0-dd29b894f014-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"fd952068-3307-4f9d-82a0-dd29b894f014\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 11:17:43.942134 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:17:43.942052 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/fd952068-3307-4f9d-82a0-dd29b894f014-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"fd952068-3307-4f9d-82a0-dd29b894f014\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 11:17:43.942134 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:17:43.942068 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/fd952068-3307-4f9d-82a0-dd29b894f014-config-out\") pod \"prometheus-k8s-0\" (UID: \"fd952068-3307-4f9d-82a0-dd29b894f014\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 11:17:43.942134 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:17:43.942091 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/fd952068-3307-4f9d-82a0-dd29b894f014-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"fd952068-3307-4f9d-82a0-dd29b894f014\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 11:17:43.942134 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:17:43.942118 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/fd952068-3307-4f9d-82a0-dd29b894f014-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"fd952068-3307-4f9d-82a0-dd29b894f014\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 11:17:43.942288 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:17:43.942143 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/fd952068-3307-4f9d-82a0-dd29b894f014-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"fd952068-3307-4f9d-82a0-dd29b894f014\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 11:17:43.942288 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:17:43.942186 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/fd952068-3307-4f9d-82a0-dd29b894f014-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"fd952068-3307-4f9d-82a0-dd29b894f014\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 11:17:43.942288 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:17:43.942211 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hnjjb\" (UniqueName: \"kubernetes.io/projected/fd952068-3307-4f9d-82a0-dd29b894f014-kube-api-access-hnjjb\") pod \"prometheus-k8s-0\" (UID: \"fd952068-3307-4f9d-82a0-dd29b894f014\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 11:17:43.942288 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:17:43.942229 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/fd952068-3307-4f9d-82a0-dd29b894f014-web-config\") pod \"prometheus-k8s-0\" (UID: \"fd952068-3307-4f9d-82a0-dd29b894f014\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 11:17:43.942502 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:17:43.942286 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/fd952068-3307-4f9d-82a0-dd29b894f014-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"fd952068-3307-4f9d-82a0-dd29b894f014\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 11:17:43.942502 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:17:43.942325 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/fd952068-3307-4f9d-82a0-dd29b894f014-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"fd952068-3307-4f9d-82a0-dd29b894f014\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 11:17:43.942502 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:17:43.942361 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/fd952068-3307-4f9d-82a0-dd29b894f014-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"fd952068-3307-4f9d-82a0-dd29b894f014\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 11:17:43.942502 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:17:43.942401 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/fd952068-3307-4f9d-82a0-dd29b894f014-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"fd952068-3307-4f9d-82a0-dd29b894f014\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 11:17:43.942502 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:17:43.942445 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/fd952068-3307-4f9d-82a0-dd29b894f014-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"fd952068-3307-4f9d-82a0-dd29b894f014\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 11:17:43.942502 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:17:43.942469 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/fd952068-3307-4f9d-82a0-dd29b894f014-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"fd952068-3307-4f9d-82a0-dd29b894f014\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 11:17:43.945851 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:17:43.945831 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-trusted-ca-bundle\"" Jul 07 11:17:43.984187 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:17:43.984161 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jul 07 11:17:44.043007 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:17:44.042973 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/fd952068-3307-4f9d-82a0-dd29b894f014-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"fd952068-3307-4f9d-82a0-dd29b894f014\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 11:17:44.043007 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:17:44.043006 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/fd952068-3307-4f9d-82a0-dd29b894f014-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"fd952068-3307-4f9d-82a0-dd29b894f014\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 11:17:44.043240 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:17:44.043024 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/fd952068-3307-4f9d-82a0-dd29b894f014-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"fd952068-3307-4f9d-82a0-dd29b894f014\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 11:17:44.043240 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:17:44.043040 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/fd952068-3307-4f9d-82a0-dd29b894f014-config\") pod \"prometheus-k8s-0\" (UID: \"fd952068-3307-4f9d-82a0-dd29b894f014\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 11:17:44.043240 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:17:44.043056 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/fd952068-3307-4f9d-82a0-dd29b894f014-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"fd952068-3307-4f9d-82a0-dd29b894f014\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 11:17:44.043240 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:17:44.043079 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/fd952068-3307-4f9d-82a0-dd29b894f014-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"fd952068-3307-4f9d-82a0-dd29b894f014\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 11:17:44.043240 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:17:44.043107 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/fd952068-3307-4f9d-82a0-dd29b894f014-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"fd952068-3307-4f9d-82a0-dd29b894f014\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 11:17:44.043240 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:17:44.043134 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/fd952068-3307-4f9d-82a0-dd29b894f014-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"fd952068-3307-4f9d-82a0-dd29b894f014\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 11:17:44.043240 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:17:44.043161 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/fd952068-3307-4f9d-82a0-dd29b894f014-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"fd952068-3307-4f9d-82a0-dd29b894f014\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 11:17:44.043240 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:17:44.043191 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/fd952068-3307-4f9d-82a0-dd29b894f014-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"fd952068-3307-4f9d-82a0-dd29b894f014\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 11:17:44.043240 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:17:44.043220 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/fd952068-3307-4f9d-82a0-dd29b894f014-config-out\") pod \"prometheus-k8s-0\" (UID: \"fd952068-3307-4f9d-82a0-dd29b894f014\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 11:17:44.043240 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:17:44.043241 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/fd952068-3307-4f9d-82a0-dd29b894f014-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"fd952068-3307-4f9d-82a0-dd29b894f014\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 11:17:44.043821 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:17:44.043257 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/fd952068-3307-4f9d-82a0-dd29b894f014-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"fd952068-3307-4f9d-82a0-dd29b894f014\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 11:17:44.043821 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:17:44.043276 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/fd952068-3307-4f9d-82a0-dd29b894f014-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"fd952068-3307-4f9d-82a0-dd29b894f014\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 11:17:44.043821 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:17:44.043321 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/fd952068-3307-4f9d-82a0-dd29b894f014-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"fd952068-3307-4f9d-82a0-dd29b894f014\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 11:17:44.043821 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:17:44.043352 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hnjjb\" (UniqueName: \"kubernetes.io/projected/fd952068-3307-4f9d-82a0-dd29b894f014-kube-api-access-hnjjb\") pod \"prometheus-k8s-0\" (UID: \"fd952068-3307-4f9d-82a0-dd29b894f014\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 11:17:44.043821 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:17:44.043377 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/fd952068-3307-4f9d-82a0-dd29b894f014-web-config\") pod \"prometheus-k8s-0\" (UID: \"fd952068-3307-4f9d-82a0-dd29b894f014\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 11:17:44.043821 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:17:44.043401 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/fd952068-3307-4f9d-82a0-dd29b894f014-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"fd952068-3307-4f9d-82a0-dd29b894f014\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 11:17:44.043821 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:17:44.043452 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/fd952068-3307-4f9d-82a0-dd29b894f014-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"fd952068-3307-4f9d-82a0-dd29b894f014\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 11:17:44.043821 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:17:44.043458 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/fd952068-3307-4f9d-82a0-dd29b894f014-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"fd952068-3307-4f9d-82a0-dd29b894f014\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 11:17:44.043821 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:17:44.043488 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/fd952068-3307-4f9d-82a0-dd29b894f014-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"fd952068-3307-4f9d-82a0-dd29b894f014\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 11:17:44.044628 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:17:44.043918 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/fd952068-3307-4f9d-82a0-dd29b894f014-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"fd952068-3307-4f9d-82a0-dd29b894f014\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 11:17:44.044628 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:17:44.043999 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/fd952068-3307-4f9d-82a0-dd29b894f014-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"fd952068-3307-4f9d-82a0-dd29b894f014\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 11:17:44.044628 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:17:44.044004 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/fd952068-3307-4f9d-82a0-dd29b894f014-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"fd952068-3307-4f9d-82a0-dd29b894f014\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 11:17:44.044628 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:17:44.044161 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/fd952068-3307-4f9d-82a0-dd29b894f014-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"fd952068-3307-4f9d-82a0-dd29b894f014\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 11:17:44.044821 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:17:44.044627 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/fd952068-3307-4f9d-82a0-dd29b894f014-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"fd952068-3307-4f9d-82a0-dd29b894f014\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 11:17:44.046621 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:17:44.046596 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/fd952068-3307-4f9d-82a0-dd29b894f014-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"fd952068-3307-4f9d-82a0-dd29b894f014\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 11:17:44.046717 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:17:44.046621 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/fd952068-3307-4f9d-82a0-dd29b894f014-config-out\") pod \"prometheus-k8s-0\" (UID: \"fd952068-3307-4f9d-82a0-dd29b894f014\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 11:17:44.048007 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:17:44.047027 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/secret/fd952068-3307-4f9d-82a0-dd29b894f014-config\") pod \"prometheus-k8s-0\" (UID: \"fd952068-3307-4f9d-82a0-dd29b894f014\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 11:17:44.048007 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:17:44.047220 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/fd952068-3307-4f9d-82a0-dd29b894f014-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"fd952068-3307-4f9d-82a0-dd29b894f014\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 11:17:44.048007 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:17:44.047401 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/fd952068-3307-4f9d-82a0-dd29b894f014-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"fd952068-3307-4f9d-82a0-dd29b894f014\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 11:17:44.048007 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:17:44.047572 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/fd952068-3307-4f9d-82a0-dd29b894f014-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"fd952068-3307-4f9d-82a0-dd29b894f014\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 11:17:44.048007 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:17:44.047968 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/fd952068-3307-4f9d-82a0-dd29b894f014-web-config\") pod \"prometheus-k8s-0\" (UID: \"fd952068-3307-4f9d-82a0-dd29b894f014\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 11:17:44.048007 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:17:44.047980 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/fd952068-3307-4f9d-82a0-dd29b894f014-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"fd952068-3307-4f9d-82a0-dd29b894f014\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 11:17:44.048512 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:17:44.048023 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/fd952068-3307-4f9d-82a0-dd29b894f014-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"fd952068-3307-4f9d-82a0-dd29b894f014\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 11:17:44.048733 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:17:44.048706 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/fd952068-3307-4f9d-82a0-dd29b894f014-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"fd952068-3307-4f9d-82a0-dd29b894f014\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 11:17:44.048955 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:17:44.048915 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/fd952068-3307-4f9d-82a0-dd29b894f014-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"fd952068-3307-4f9d-82a0-dd29b894f014\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 11:17:44.049442 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:17:44.049401 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/fd952068-3307-4f9d-82a0-dd29b894f014-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"fd952068-3307-4f9d-82a0-dd29b894f014\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 11:17:44.049666 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:17:44.049650 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/fd952068-3307-4f9d-82a0-dd29b894f014-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"fd952068-3307-4f9d-82a0-dd29b894f014\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 11:17:44.057806 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:17:44.057778 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-hnjjb\" (UniqueName: \"kubernetes.io/projected/fd952068-3307-4f9d-82a0-dd29b894f014-kube-api-access-hnjjb\") pod \"prometheus-k8s-0\" (UID: \"fd952068-3307-4f9d-82a0-dd29b894f014\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 11:17:44.232155 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:17:44.232074 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" Jul 07 11:17:44.306135 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:17:44.306101 2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="850ea628-4496-41f7-839c-5233a8831455" path="/var/lib/kubelet/pods/850ea628-4496-41f7-839c-5233a8831455/volumes" Jul 07 11:17:44.385909 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:17:44.385870 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jul 07 11:17:44.389260 ip-10-0-142-112 kubenswrapper[2578]: W0707 11:17:44.389237 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfd952068_3307_4f9d_82a0_dd29b894f014.slice/crio-7ae01b30060d7aecbfe8ebf50d89f98afa0c695cf185579e3b74440394f9ebcf WatchSource:0}: Error finding container 7ae01b30060d7aecbfe8ebf50d89f98afa0c695cf185579e3b74440394f9ebcf: Status 404 returned error can't find the container with id 7ae01b30060d7aecbfe8ebf50d89f98afa0c695cf185579e3b74440394f9ebcf Jul 07 11:17:44.839005 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:17:44.838973 2578 generic.go:356] "Generic (PLEG): container finished" podID="fd952068-3307-4f9d-82a0-dd29b894f014" containerID="5100c1695ff02907dd585ac62690dbd97c74dda0f4d004cc22eb3574ed7160a7" exitCode=0 Jul 07 11:17:44.839371 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:17:44.839063 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"fd952068-3307-4f9d-82a0-dd29b894f014","Type":"ContainerDied","Data":"5100c1695ff02907dd585ac62690dbd97c74dda0f4d004cc22eb3574ed7160a7"} Jul 07 11:17:44.839371 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:17:44.839169 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"fd952068-3307-4f9d-82a0-dd29b894f014","Type":"ContainerStarted","Data":"7ae01b30060d7aecbfe8ebf50d89f98afa0c695cf185579e3b74440394f9ebcf"} Jul 07 11:17:45.849444 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:17:45.849391 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"fd952068-3307-4f9d-82a0-dd29b894f014","Type":"ContainerStarted","Data":"1b98a2ecaf2edde393825b85918cf20020fa50c41a8ddb80c3c04c669d8e7434"} Jul 07 11:17:45.849871 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:17:45.849488 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"fd952068-3307-4f9d-82a0-dd29b894f014","Type":"ContainerStarted","Data":"b182b92d9880a25919bdb7bc4e9fcb6ed85d22a8e0ba665fd673c539ab980c44"} Jul 07 11:17:45.849871 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:17:45.849517 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"fd952068-3307-4f9d-82a0-dd29b894f014","Type":"ContainerStarted","Data":"4e08924678094ae32d5b86b131e6aeb82b5d0ee44998dba2e4b92820aeae9a8a"} Jul 07 11:17:45.849871 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:17:45.849559 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"fd952068-3307-4f9d-82a0-dd29b894f014","Type":"ContainerStarted","Data":"56869f996792964eb628aa59efa1485f1a83e43f8c1ed7afec95b1ba1c967148"} Jul 07 11:17:45.849871 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:17:45.849585 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"fd952068-3307-4f9d-82a0-dd29b894f014","Type":"ContainerStarted","Data":"e7d6279532bd6d207bb8203b9ecebee72d20135a7cd4bab457df185f8de06021"} Jul 07 11:17:45.849871 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:17:45.849619 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"fd952068-3307-4f9d-82a0-dd29b894f014","Type":"ContainerStarted","Data":"4c0b741dd83f5e58fefd82e597a594a8a40868e6ea363e54fed2fed6d25459dc"} Jul 07 11:17:46.056877 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:17:46.056824 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-k8s-0" podStartSLOduration=3.039073191 podStartE2EDuration="3.056802434s" totalImagesPullingTime="0s" totalInitContainerRuntime="17.729243ms" isStatefulPod=false podCreationTimestamp="2026-07-07 11:17:43 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 11:17:45.896289045 +0000 UTC m=+110.192438648" watchObservedRunningTime="2026-07-07 11:17:46.056802434 +0000 UTC m=+110.352952029" Jul 07 11:17:46.057064 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:17:46.057048 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-57449f86c4-77zmv"] Jul 07 11:17:46.060153 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:17:46.060133 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-57449f86c4-77zmv" Jul 07 11:17:46.073718 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:17:46.073684 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-57449f86c4-77zmv"] Jul 07 11:17:46.163510 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:17:46.163409 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/937b2d57-4a9f-4325-87b6-baeaba02bf4e-console-serving-cert\") pod \"console-57449f86c4-77zmv\" (UID: \"937b2d57-4a9f-4325-87b6-baeaba02bf4e\") " pod="openshift-console/console-57449f86c4-77zmv" Jul 07 11:17:46.163510 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:17:46.163469 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/937b2d57-4a9f-4325-87b6-baeaba02bf4e-trusted-ca-bundle\") pod \"console-57449f86c4-77zmv\" (UID: \"937b2d57-4a9f-4325-87b6-baeaba02bf4e\") " pod="openshift-console/console-57449f86c4-77zmv" Jul 07 11:17:46.163510 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:17:46.163497 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/937b2d57-4a9f-4325-87b6-baeaba02bf4e-console-config\") pod \"console-57449f86c4-77zmv\" (UID: \"937b2d57-4a9f-4325-87b6-baeaba02bf4e\") " pod="openshift-console/console-57449f86c4-77zmv" Jul 07 11:17:46.163710 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:17:46.163555 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/937b2d57-4a9f-4325-87b6-baeaba02bf4e-service-ca\") pod \"console-57449f86c4-77zmv\" (UID: \"937b2d57-4a9f-4325-87b6-baeaba02bf4e\") " pod="openshift-console/console-57449f86c4-77zmv" Jul 07 11:17:46.163710 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:17:46.163581 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/937b2d57-4a9f-4325-87b6-baeaba02bf4e-oauth-serving-cert\") pod \"console-57449f86c4-77zmv\" (UID: \"937b2d57-4a9f-4325-87b6-baeaba02bf4e\") " pod="openshift-console/console-57449f86c4-77zmv" Jul 07 11:17:46.163710 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:17:46.163659 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/937b2d57-4a9f-4325-87b6-baeaba02bf4e-console-oauth-config\") pod \"console-57449f86c4-77zmv\" (UID: \"937b2d57-4a9f-4325-87b6-baeaba02bf4e\") " pod="openshift-console/console-57449f86c4-77zmv" Jul 07 11:17:46.163799 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:17:46.163711 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-l558g\" (UniqueName: \"kubernetes.io/projected/937b2d57-4a9f-4325-87b6-baeaba02bf4e-kube-api-access-l558g\") pod \"console-57449f86c4-77zmv\" (UID: \"937b2d57-4a9f-4325-87b6-baeaba02bf4e\") " pod="openshift-console/console-57449f86c4-77zmv" Jul 07 11:17:46.265083 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:17:46.265047 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/937b2d57-4a9f-4325-87b6-baeaba02bf4e-service-ca\") pod \"console-57449f86c4-77zmv\" (UID: \"937b2d57-4a9f-4325-87b6-baeaba02bf4e\") " pod="openshift-console/console-57449f86c4-77zmv" Jul 07 11:17:46.265083 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:17:46.265084 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/937b2d57-4a9f-4325-87b6-baeaba02bf4e-oauth-serving-cert\") pod \"console-57449f86c4-77zmv\" (UID: \"937b2d57-4a9f-4325-87b6-baeaba02bf4e\") " pod="openshift-console/console-57449f86c4-77zmv" Jul 07 11:17:46.265330 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:17:46.265110 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/937b2d57-4a9f-4325-87b6-baeaba02bf4e-console-oauth-config\") pod \"console-57449f86c4-77zmv\" (UID: \"937b2d57-4a9f-4325-87b6-baeaba02bf4e\") " pod="openshift-console/console-57449f86c4-77zmv" Jul 07 11:17:46.265330 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:17:46.265157 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-l558g\" (UniqueName: \"kubernetes.io/projected/937b2d57-4a9f-4325-87b6-baeaba02bf4e-kube-api-access-l558g\") pod \"console-57449f86c4-77zmv\" (UID: \"937b2d57-4a9f-4325-87b6-baeaba02bf4e\") " pod="openshift-console/console-57449f86c4-77zmv" Jul 07 11:17:46.265330 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:17:46.265205 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/937b2d57-4a9f-4325-87b6-baeaba02bf4e-console-serving-cert\") pod \"console-57449f86c4-77zmv\" (UID: \"937b2d57-4a9f-4325-87b6-baeaba02bf4e\") " pod="openshift-console/console-57449f86c4-77zmv" Jul 07 11:17:46.265330 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:17:46.265228 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/937b2d57-4a9f-4325-87b6-baeaba02bf4e-trusted-ca-bundle\") pod \"console-57449f86c4-77zmv\" (UID: \"937b2d57-4a9f-4325-87b6-baeaba02bf4e\") " pod="openshift-console/console-57449f86c4-77zmv" Jul 07 11:17:46.265330 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:17:46.265253 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/937b2d57-4a9f-4325-87b6-baeaba02bf4e-console-config\") pod \"console-57449f86c4-77zmv\" (UID: \"937b2d57-4a9f-4325-87b6-baeaba02bf4e\") " pod="openshift-console/console-57449f86c4-77zmv" Jul 07 11:17:46.265955 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:17:46.265923 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/937b2d57-4a9f-4325-87b6-baeaba02bf4e-oauth-serving-cert\") pod \"console-57449f86c4-77zmv\" (UID: \"937b2d57-4a9f-4325-87b6-baeaba02bf4e\") " pod="openshift-console/console-57449f86c4-77zmv" Jul 07 11:17:46.265955 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:17:46.265943 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/937b2d57-4a9f-4325-87b6-baeaba02bf4e-service-ca\") pod \"console-57449f86c4-77zmv\" (UID: \"937b2d57-4a9f-4325-87b6-baeaba02bf4e\") " pod="openshift-console/console-57449f86c4-77zmv" Jul 07 11:17:46.266092 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:17:46.266063 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/937b2d57-4a9f-4325-87b6-baeaba02bf4e-console-config\") pod \"console-57449f86c4-77zmv\" (UID: \"937b2d57-4a9f-4325-87b6-baeaba02bf4e\") " pod="openshift-console/console-57449f86c4-77zmv" Jul 07 11:17:46.266445 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:17:46.266384 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/937b2d57-4a9f-4325-87b6-baeaba02bf4e-trusted-ca-bundle\") pod \"console-57449f86c4-77zmv\" (UID: \"937b2d57-4a9f-4325-87b6-baeaba02bf4e\") " pod="openshift-console/console-57449f86c4-77zmv" Jul 07 11:17:46.267829 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:17:46.267810 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/937b2d57-4a9f-4325-87b6-baeaba02bf4e-console-oauth-config\") pod \"console-57449f86c4-77zmv\" (UID: \"937b2d57-4a9f-4325-87b6-baeaba02bf4e\") " pod="openshift-console/console-57449f86c4-77zmv" Jul 07 11:17:46.267973 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:17:46.267954 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/937b2d57-4a9f-4325-87b6-baeaba02bf4e-console-serving-cert\") pod \"console-57449f86c4-77zmv\" (UID: \"937b2d57-4a9f-4325-87b6-baeaba02bf4e\") " pod="openshift-console/console-57449f86c4-77zmv" Jul 07 11:17:46.273912 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:17:46.273892 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-l558g\" (UniqueName: \"kubernetes.io/projected/937b2d57-4a9f-4325-87b6-baeaba02bf4e-kube-api-access-l558g\") pod \"console-57449f86c4-77zmv\" (UID: \"937b2d57-4a9f-4325-87b6-baeaba02bf4e\") " pod="openshift-console/console-57449f86c4-77zmv" Jul 07 11:17:46.372028 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:17:46.371778 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-57449f86c4-77zmv" Jul 07 11:17:46.512496 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:17:46.512458 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-57449f86c4-77zmv"] Jul 07 11:17:46.517256 ip-10-0-142-112 kubenswrapper[2578]: W0707 11:17:46.517229 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod937b2d57_4a9f_4325_87b6_baeaba02bf4e.slice/crio-fe5ff274fc298d583d3850624b4f019a324f0694b31cbaefe1b45ae0dab272c5 WatchSource:0}: Error finding container fe5ff274fc298d583d3850624b4f019a324f0694b31cbaefe1b45ae0dab272c5: Status 404 returned error can't find the container with id fe5ff274fc298d583d3850624b4f019a324f0694b31cbaefe1b45ae0dab272c5 Jul 07 11:17:46.858115 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:17:46.858068 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-57449f86c4-77zmv" event={"ID":"937b2d57-4a9f-4325-87b6-baeaba02bf4e","Type":"ContainerStarted","Data":"56b0caa415174c2b1896199822a2ea0bbe380ede9657062751e26563d2c37b36"} Jul 07 11:17:46.858602 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:17:46.858129 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-57449f86c4-77zmv" event={"ID":"937b2d57-4a9f-4325-87b6-baeaba02bf4e","Type":"ContainerStarted","Data":"fe5ff274fc298d583d3850624b4f019a324f0694b31cbaefe1b45ae0dab272c5"} Jul 07 11:17:46.891306 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:17:46.891258 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-57449f86c4-77zmv" podStartSLOduration=0.891239754 podStartE2EDuration="891.239754ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 11:17:46 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 11:17:46.891183455 +0000 UTC m=+111.187333044" watchObservedRunningTime="2026-07-07 11:17:46.891239754 +0000 UTC m=+111.187389341" Jul 07 11:17:49.232564 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:17:49.232529 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-k8s-0" Jul 07 11:17:56.372831 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:17:56.372781 2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-57449f86c4-77zmv" Jul 07 11:17:56.373300 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:17:56.372871 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-57449f86c4-77zmv" Jul 07 11:17:56.377802 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:17:56.377780 2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-57449f86c4-77zmv" Jul 07 11:17:56.902679 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:17:56.902418 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-57449f86c4-77zmv" Jul 07 11:17:56.957605 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:17:56.957565 2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-79c45cc49-fh26k"] Jul 07 11:18:21.978778 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:18:21.978729 2578 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-console/console-79c45cc49-fh26k" podUID="a5225789-7bf3-4087-88cc-f8569798678c" containerName="console" containerID="cri-o://c8613cae23dfc1f5f2e3ac63034d30d7d6ebbe897e79f089a893f3d14ce3e59f" gracePeriod=15 Jul 07 11:18:22.221269 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:18:22.221247 2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-79c45cc49-fh26k" Jul 07 11:18:22.277394 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:18:22.277300 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/a5225789-7bf3-4087-88cc-f8569798678c-service-ca\") pod \"a5225789-7bf3-4087-88cc-f8569798678c\" (UID: \"a5225789-7bf3-4087-88cc-f8569798678c\") " Jul 07 11:18:22.277394 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:18:22.277340 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-x5j4h\" (UniqueName: \"kubernetes.io/projected/a5225789-7bf3-4087-88cc-f8569798678c-kube-api-access-x5j4h\") pod \"a5225789-7bf3-4087-88cc-f8569798678c\" (UID: \"a5225789-7bf3-4087-88cc-f8569798678c\") " Jul 07 11:18:22.277394 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:18:22.277371 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/a5225789-7bf3-4087-88cc-f8569798678c-console-serving-cert\") pod \"a5225789-7bf3-4087-88cc-f8569798678c\" (UID: \"a5225789-7bf3-4087-88cc-f8569798678c\") " Jul 07 11:18:22.277394 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:18:22.277396 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a5225789-7bf3-4087-88cc-f8569798678c-trusted-ca-bundle\") pod \"a5225789-7bf3-4087-88cc-f8569798678c\" (UID: \"a5225789-7bf3-4087-88cc-f8569798678c\") " Jul 07 11:18:22.277730 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:18:22.277497 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/a5225789-7bf3-4087-88cc-f8569798678c-console-oauth-config\") pod \"a5225789-7bf3-4087-88cc-f8569798678c\" (UID: \"a5225789-7bf3-4087-88cc-f8569798678c\") " Jul 07 11:18:22.277730 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:18:22.277521 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/a5225789-7bf3-4087-88cc-f8569798678c-console-config\") pod \"a5225789-7bf3-4087-88cc-f8569798678c\" (UID: \"a5225789-7bf3-4087-88cc-f8569798678c\") " Jul 07 11:18:22.277730 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:18:22.277555 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/a5225789-7bf3-4087-88cc-f8569798678c-oauth-serving-cert\") pod \"a5225789-7bf3-4087-88cc-f8569798678c\" (UID: \"a5225789-7bf3-4087-88cc-f8569798678c\") " Jul 07 11:18:22.277881 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:18:22.277845 2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a5225789-7bf3-4087-88cc-f8569798678c-service-ca" (OuterVolumeSpecName: "service-ca") pod "a5225789-7bf3-4087-88cc-f8569798678c" (UID: "a5225789-7bf3-4087-88cc-f8569798678c"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 07 11:18:22.277931 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:18:22.277880 2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a5225789-7bf3-4087-88cc-f8569798678c-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "a5225789-7bf3-4087-88cc-f8569798678c" (UID: "a5225789-7bf3-4087-88cc-f8569798678c"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 07 11:18:22.278006 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:18:22.277903 2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a5225789-7bf3-4087-88cc-f8569798678c-console-config" (OuterVolumeSpecName: "console-config") pod "a5225789-7bf3-4087-88cc-f8569798678c" (UID: "a5225789-7bf3-4087-88cc-f8569798678c"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 07 11:18:22.278135 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:18:22.278085 2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a5225789-7bf3-4087-88cc-f8569798678c-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "a5225789-7bf3-4087-88cc-f8569798678c" (UID: "a5225789-7bf3-4087-88cc-f8569798678c"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 07 11:18:22.279718 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:18:22.279688 2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a5225789-7bf3-4087-88cc-f8569798678c-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "a5225789-7bf3-4087-88cc-f8569798678c" (UID: "a5225789-7bf3-4087-88cc-f8569798678c"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 07 11:18:22.279822 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:18:22.279728 2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a5225789-7bf3-4087-88cc-f8569798678c-kube-api-access-x5j4h" (OuterVolumeSpecName: "kube-api-access-x5j4h") pod "a5225789-7bf3-4087-88cc-f8569798678c" (UID: "a5225789-7bf3-4087-88cc-f8569798678c"). InnerVolumeSpecName "kube-api-access-x5j4h". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 07 11:18:22.279822 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:18:22.279742 2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a5225789-7bf3-4087-88cc-f8569798678c-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "a5225789-7bf3-4087-88cc-f8569798678c" (UID: "a5225789-7bf3-4087-88cc-f8569798678c"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 07 11:18:22.378472 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:18:22.378416 2578 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/a5225789-7bf3-4087-88cc-f8569798678c-console-serving-cert\") on node \"ip-10-0-142-112.ec2.internal\" DevicePath \"\"" Jul 07 11:18:22.378472 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:18:22.378468 2578 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a5225789-7bf3-4087-88cc-f8569798678c-trusted-ca-bundle\") on node \"ip-10-0-142-112.ec2.internal\" DevicePath \"\"" Jul 07 11:18:22.378472 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:18:22.378477 2578 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/a5225789-7bf3-4087-88cc-f8569798678c-console-oauth-config\") on node \"ip-10-0-142-112.ec2.internal\" DevicePath \"\"" Jul 07 11:18:22.378701 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:18:22.378498 2578 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/a5225789-7bf3-4087-88cc-f8569798678c-console-config\") on node \"ip-10-0-142-112.ec2.internal\" DevicePath \"\"" Jul 07 11:18:22.378701 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:18:22.378507 2578 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/a5225789-7bf3-4087-88cc-f8569798678c-oauth-serving-cert\") on node \"ip-10-0-142-112.ec2.internal\" DevicePath \"\"" Jul 07 11:18:22.378701 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:18:22.378516 2578 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/a5225789-7bf3-4087-88cc-f8569798678c-service-ca\") on node \"ip-10-0-142-112.ec2.internal\" DevicePath \"\"" Jul 07 11:18:22.378701 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:18:22.378524 2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-x5j4h\" (UniqueName: \"kubernetes.io/projected/a5225789-7bf3-4087-88cc-f8569798678c-kube-api-access-x5j4h\") on node \"ip-10-0-142-112.ec2.internal\" DevicePath \"\"" Jul 07 11:18:22.993188 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:18:22.993147 2578 generic.go:356] "Generic (PLEG): container finished" podID="a5225789-7bf3-4087-88cc-f8569798678c" containerID="c8613cae23dfc1f5f2e3ac63034d30d7d6ebbe897e79f089a893f3d14ce3e59f" exitCode=0 Jul 07 11:18:22.993188 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:18:22.993190 2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-79c45cc49-fh26k" Jul 07 11:18:22.993686 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:18:22.993203 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-79c45cc49-fh26k" event={"ID":"a5225789-7bf3-4087-88cc-f8569798678c","Type":"ContainerDied","Data":"c8613cae23dfc1f5f2e3ac63034d30d7d6ebbe897e79f089a893f3d14ce3e59f"} Jul 07 11:18:22.993686 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:18:22.993248 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-79c45cc49-fh26k" event={"ID":"a5225789-7bf3-4087-88cc-f8569798678c","Type":"ContainerDied","Data":"65be6c5fee76009211de5e29009e4e3ee4d7ecfc90e42cfaa6c56953e9ad2722"} Jul 07 11:18:22.993686 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:18:22.993268 2578 scope.go:117] "RemoveContainer" containerID="c8613cae23dfc1f5f2e3ac63034d30d7d6ebbe897e79f089a893f3d14ce3e59f" Jul 07 11:18:23.001193 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:18:23.001176 2578 scope.go:117] "RemoveContainer" containerID="c8613cae23dfc1f5f2e3ac63034d30d7d6ebbe897e79f089a893f3d14ce3e59f" Jul 07 11:18:23.001447 ip-10-0-142-112 kubenswrapper[2578]: E0707 11:18:23.001409 2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c8613cae23dfc1f5f2e3ac63034d30d7d6ebbe897e79f089a893f3d14ce3e59f\": container with ID starting with c8613cae23dfc1f5f2e3ac63034d30d7d6ebbe897e79f089a893f3d14ce3e59f not found: ID does not exist" containerID="c8613cae23dfc1f5f2e3ac63034d30d7d6ebbe897e79f089a893f3d14ce3e59f" Jul 07 11:18:23.001517 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:18:23.001457 2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c8613cae23dfc1f5f2e3ac63034d30d7d6ebbe897e79f089a893f3d14ce3e59f"} err="failed to get container status \"c8613cae23dfc1f5f2e3ac63034d30d7d6ebbe897e79f089a893f3d14ce3e59f\": rpc error: code = NotFound desc = could not find container \"c8613cae23dfc1f5f2e3ac63034d30d7d6ebbe897e79f089a893f3d14ce3e59f\": container with ID starting with c8613cae23dfc1f5f2e3ac63034d30d7d6ebbe897e79f089a893f3d14ce3e59f not found: ID does not exist" Jul 07 11:18:23.009958 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:18:23.009931 2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-79c45cc49-fh26k"] Jul 07 11:18:23.016375 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:18:23.016348 2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-79c45cc49-fh26k"] Jul 07 11:18:24.306263 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:18:24.306226 2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a5225789-7bf3-4087-88cc-f8569798678c" path="/var/lib/kubelet/pods/a5225789-7bf3-4087-88cc-f8569798678c/volumes" Jul 07 11:18:44.233305 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:18:44.233261 2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/prometheus-k8s-0" Jul 07 11:18:44.248349 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:18:44.248321 2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/prometheus-k8s-0" Jul 07 11:18:45.085146 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:18:45.085111 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-k8s-0" Jul 07 11:19:08.257839 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:19:08.257806 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-7754b567bf-k222b"] Jul 07 11:19:08.258260 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:19:08.258084 2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a5225789-7bf3-4087-88cc-f8569798678c" containerName="console" Jul 07 11:19:08.258260 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:19:08.258095 2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="a5225789-7bf3-4087-88cc-f8569798678c" containerName="console" Jul 07 11:19:08.258260 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:19:08.258143 2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="a5225789-7bf3-4087-88cc-f8569798678c" containerName="console" Jul 07 11:19:08.260579 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:19:08.260554 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7754b567bf-k222b" Jul 07 11:19:08.276633 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:19:08.276608 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-7754b567bf-k222b"] Jul 07 11:19:08.358755 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:19:08.358720 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/d4726168-e989-42f1-88dc-4dc1c64c1dcc-service-ca\") pod \"console-7754b567bf-k222b\" (UID: \"d4726168-e989-42f1-88dc-4dc1c64c1dcc\") " pod="openshift-console/console-7754b567bf-k222b" Jul 07 11:19:08.358919 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:19:08.358763 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/d4726168-e989-42f1-88dc-4dc1c64c1dcc-console-config\") pod \"console-7754b567bf-k222b\" (UID: \"d4726168-e989-42f1-88dc-4dc1c64c1dcc\") " pod="openshift-console/console-7754b567bf-k222b" Jul 07 11:19:08.358919 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:19:08.358792 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/d4726168-e989-42f1-88dc-4dc1c64c1dcc-console-oauth-config\") pod \"console-7754b567bf-k222b\" (UID: \"d4726168-e989-42f1-88dc-4dc1c64c1dcc\") " pod="openshift-console/console-7754b567bf-k222b" Jul 07 11:19:08.358919 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:19:08.358864 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d4726168-e989-42f1-88dc-4dc1c64c1dcc-trusted-ca-bundle\") pod \"console-7754b567bf-k222b\" (UID: \"d4726168-e989-42f1-88dc-4dc1c64c1dcc\") " pod="openshift-console/console-7754b567bf-k222b" Jul 07 11:19:08.358919 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:19:08.358908 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/d4726168-e989-42f1-88dc-4dc1c64c1dcc-console-serving-cert\") pod \"console-7754b567bf-k222b\" (UID: \"d4726168-e989-42f1-88dc-4dc1c64c1dcc\") " pod="openshift-console/console-7754b567bf-k222b" Jul 07 11:19:08.359051 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:19:08.358957 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/d4726168-e989-42f1-88dc-4dc1c64c1dcc-oauth-serving-cert\") pod \"console-7754b567bf-k222b\" (UID: \"d4726168-e989-42f1-88dc-4dc1c64c1dcc\") " pod="openshift-console/console-7754b567bf-k222b" Jul 07 11:19:08.359051 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:19:08.358977 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-c62vf\" (UniqueName: \"kubernetes.io/projected/d4726168-e989-42f1-88dc-4dc1c64c1dcc-kube-api-access-c62vf\") pod \"console-7754b567bf-k222b\" (UID: \"d4726168-e989-42f1-88dc-4dc1c64c1dcc\") " pod="openshift-console/console-7754b567bf-k222b" Jul 07 11:19:08.460073 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:19:08.460039 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/d4726168-e989-42f1-88dc-4dc1c64c1dcc-console-serving-cert\") pod \"console-7754b567bf-k222b\" (UID: \"d4726168-e989-42f1-88dc-4dc1c64c1dcc\") " pod="openshift-console/console-7754b567bf-k222b" Jul 07 11:19:08.460073 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:19:08.460083 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/d4726168-e989-42f1-88dc-4dc1c64c1dcc-oauth-serving-cert\") pod \"console-7754b567bf-k222b\" (UID: \"d4726168-e989-42f1-88dc-4dc1c64c1dcc\") " pod="openshift-console/console-7754b567bf-k222b" Jul 07 11:19:08.460281 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:19:08.460099 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-c62vf\" (UniqueName: \"kubernetes.io/projected/d4726168-e989-42f1-88dc-4dc1c64c1dcc-kube-api-access-c62vf\") pod \"console-7754b567bf-k222b\" (UID: \"d4726168-e989-42f1-88dc-4dc1c64c1dcc\") " pod="openshift-console/console-7754b567bf-k222b" Jul 07 11:19:08.460281 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:19:08.460217 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/d4726168-e989-42f1-88dc-4dc1c64c1dcc-service-ca\") pod \"console-7754b567bf-k222b\" (UID: \"d4726168-e989-42f1-88dc-4dc1c64c1dcc\") " pod="openshift-console/console-7754b567bf-k222b" Jul 07 11:19:08.460281 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:19:08.460266 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/d4726168-e989-42f1-88dc-4dc1c64c1dcc-console-config\") pod \"console-7754b567bf-k222b\" (UID: \"d4726168-e989-42f1-88dc-4dc1c64c1dcc\") " pod="openshift-console/console-7754b567bf-k222b" Jul 07 11:19:08.460473 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:19:08.460449 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/d4726168-e989-42f1-88dc-4dc1c64c1dcc-console-oauth-config\") pod \"console-7754b567bf-k222b\" (UID: \"d4726168-e989-42f1-88dc-4dc1c64c1dcc\") " pod="openshift-console/console-7754b567bf-k222b" Jul 07 11:19:08.460542 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:19:08.460527 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d4726168-e989-42f1-88dc-4dc1c64c1dcc-trusted-ca-bundle\") pod \"console-7754b567bf-k222b\" (UID: \"d4726168-e989-42f1-88dc-4dc1c64c1dcc\") " pod="openshift-console/console-7754b567bf-k222b" Jul 07 11:19:08.460892 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:19:08.460863 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/d4726168-e989-42f1-88dc-4dc1c64c1dcc-oauth-serving-cert\") pod \"console-7754b567bf-k222b\" (UID: \"d4726168-e989-42f1-88dc-4dc1c64c1dcc\") " pod="openshift-console/console-7754b567bf-k222b" Jul 07 11:19:08.461005 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:19:08.460903 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/d4726168-e989-42f1-88dc-4dc1c64c1dcc-console-config\") pod \"console-7754b567bf-k222b\" (UID: \"d4726168-e989-42f1-88dc-4dc1c64c1dcc\") " pod="openshift-console/console-7754b567bf-k222b" Jul 07 11:19:08.461005 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:19:08.460982 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/d4726168-e989-42f1-88dc-4dc1c64c1dcc-service-ca\") pod \"console-7754b567bf-k222b\" (UID: \"d4726168-e989-42f1-88dc-4dc1c64c1dcc\") " pod="openshift-console/console-7754b567bf-k222b" Jul 07 11:19:08.461252 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:19:08.461236 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d4726168-e989-42f1-88dc-4dc1c64c1dcc-trusted-ca-bundle\") pod \"console-7754b567bf-k222b\" (UID: \"d4726168-e989-42f1-88dc-4dc1c64c1dcc\") " pod="openshift-console/console-7754b567bf-k222b" Jul 07 11:19:08.463045 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:19:08.463024 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/d4726168-e989-42f1-88dc-4dc1c64c1dcc-console-oauth-config\") pod \"console-7754b567bf-k222b\" (UID: \"d4726168-e989-42f1-88dc-4dc1c64c1dcc\") " pod="openshift-console/console-7754b567bf-k222b" Jul 07 11:19:08.463149 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:19:08.463133 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/d4726168-e989-42f1-88dc-4dc1c64c1dcc-console-serving-cert\") pod \"console-7754b567bf-k222b\" (UID: \"d4726168-e989-42f1-88dc-4dc1c64c1dcc\") " pod="openshift-console/console-7754b567bf-k222b" Jul 07 11:19:08.470274 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:19:08.470248 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-c62vf\" (UniqueName: \"kubernetes.io/projected/d4726168-e989-42f1-88dc-4dc1c64c1dcc-kube-api-access-c62vf\") pod \"console-7754b567bf-k222b\" (UID: \"d4726168-e989-42f1-88dc-4dc1c64c1dcc\") " pod="openshift-console/console-7754b567bf-k222b" Jul 07 11:19:08.571139 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:19:08.571105 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7754b567bf-k222b" Jul 07 11:19:08.698483 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:19:08.698452 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-7754b567bf-k222b"] Jul 07 11:19:08.701336 ip-10-0-142-112 kubenswrapper[2578]: W0707 11:19:08.701309 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd4726168_e989_42f1_88dc_4dc1c64c1dcc.slice/crio-1c9e3da57c4df37bed9f4df88998c03674e4c1c0e709406da9f15c918973ed64 WatchSource:0}: Error finding container 1c9e3da57c4df37bed9f4df88998c03674e4c1c0e709406da9f15c918973ed64: Status 404 returned error can't find the container with id 1c9e3da57c4df37bed9f4df88998c03674e4c1c0e709406da9f15c918973ed64 Jul 07 11:19:09.162083 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:19:09.162048 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7754b567bf-k222b" event={"ID":"d4726168-e989-42f1-88dc-4dc1c64c1dcc","Type":"ContainerStarted","Data":"311b69e229244f5190de84738692de5b5069ede4cd9fc0701cfa0702887eb928"} Jul 07 11:19:09.162262 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:19:09.162097 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7754b567bf-k222b" event={"ID":"d4726168-e989-42f1-88dc-4dc1c64c1dcc","Type":"ContainerStarted","Data":"1c9e3da57c4df37bed9f4df88998c03674e4c1c0e709406da9f15c918973ed64"} Jul 07 11:19:09.182249 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:19:09.182197 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-7754b567bf-k222b" podStartSLOduration=1.182178281 podStartE2EDuration="1.182178281s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 11:19:08 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 11:19:09.181504793 +0000 UTC m=+193.477654383" watchObservedRunningTime="2026-07-07 11:19:09.182178281 +0000 UTC m=+193.478327868" Jul 07 11:19:18.571273 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:19:18.571237 2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-7754b567bf-k222b" Jul 07 11:19:18.571764 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:19:18.571301 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-7754b567bf-k222b" Jul 07 11:19:18.576056 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:19:18.576035 2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-7754b567bf-k222b" Jul 07 11:19:19.206575 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:19:19.206549 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-7754b567bf-k222b" Jul 07 11:19:19.253029 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:19:19.252997 2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-57449f86c4-77zmv"] Jul 07 11:19:44.273053 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:19:44.272989 2578 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-console/console-57449f86c4-77zmv" podUID="937b2d57-4a9f-4325-87b6-baeaba02bf4e" containerName="console" containerID="cri-o://56b0caa415174c2b1896199822a2ea0bbe380ede9657062751e26563d2c37b36" gracePeriod=15 Jul 07 11:19:44.515279 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:19:44.515247 2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-57449f86c4-77zmv" Jul 07 11:19:44.631649 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:19:44.631616 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/937b2d57-4a9f-4325-87b6-baeaba02bf4e-console-serving-cert\") pod \"937b2d57-4a9f-4325-87b6-baeaba02bf4e\" (UID: \"937b2d57-4a9f-4325-87b6-baeaba02bf4e\") " Jul 07 11:19:44.631823 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:19:44.631660 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/937b2d57-4a9f-4325-87b6-baeaba02bf4e-trusted-ca-bundle\") pod \"937b2d57-4a9f-4325-87b6-baeaba02bf4e\" (UID: \"937b2d57-4a9f-4325-87b6-baeaba02bf4e\") " Jul 07 11:19:44.631823 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:19:44.631680 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/937b2d57-4a9f-4325-87b6-baeaba02bf4e-console-config\") pod \"937b2d57-4a9f-4325-87b6-baeaba02bf4e\" (UID: \"937b2d57-4a9f-4325-87b6-baeaba02bf4e\") " Jul 07 11:19:44.631823 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:19:44.631703 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/937b2d57-4a9f-4325-87b6-baeaba02bf4e-oauth-serving-cert\") pod \"937b2d57-4a9f-4325-87b6-baeaba02bf4e\" (UID: \"937b2d57-4a9f-4325-87b6-baeaba02bf4e\") " Jul 07 11:19:44.631823 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:19:44.631725 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-l558g\" (UniqueName: \"kubernetes.io/projected/937b2d57-4a9f-4325-87b6-baeaba02bf4e-kube-api-access-l558g\") pod \"937b2d57-4a9f-4325-87b6-baeaba02bf4e\" (UID: \"937b2d57-4a9f-4325-87b6-baeaba02bf4e\") " Jul 07 11:19:44.631823 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:19:44.631790 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/937b2d57-4a9f-4325-87b6-baeaba02bf4e-console-oauth-config\") pod \"937b2d57-4a9f-4325-87b6-baeaba02bf4e\" (UID: \"937b2d57-4a9f-4325-87b6-baeaba02bf4e\") " Jul 07 11:19:44.632081 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:19:44.631827 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/937b2d57-4a9f-4325-87b6-baeaba02bf4e-service-ca\") pod \"937b2d57-4a9f-4325-87b6-baeaba02bf4e\" (UID: \"937b2d57-4a9f-4325-87b6-baeaba02bf4e\") " Jul 07 11:19:44.632136 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:19:44.632104 2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/937b2d57-4a9f-4325-87b6-baeaba02bf4e-console-config" (OuterVolumeSpecName: "console-config") pod "937b2d57-4a9f-4325-87b6-baeaba02bf4e" (UID: "937b2d57-4a9f-4325-87b6-baeaba02bf4e"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 07 11:19:44.632136 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:19:44.632114 2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/937b2d57-4a9f-4325-87b6-baeaba02bf4e-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "937b2d57-4a9f-4325-87b6-baeaba02bf4e" (UID: "937b2d57-4a9f-4325-87b6-baeaba02bf4e"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 07 11:19:44.632356 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:19:44.632328 2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/937b2d57-4a9f-4325-87b6-baeaba02bf4e-service-ca" (OuterVolumeSpecName: "service-ca") pod "937b2d57-4a9f-4325-87b6-baeaba02bf4e" (UID: "937b2d57-4a9f-4325-87b6-baeaba02bf4e"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 07 11:19:44.632459 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:19:44.632376 2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/937b2d57-4a9f-4325-87b6-baeaba02bf4e-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "937b2d57-4a9f-4325-87b6-baeaba02bf4e" (UID: "937b2d57-4a9f-4325-87b6-baeaba02bf4e"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 07 11:19:44.634018 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:19:44.633990 2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/937b2d57-4a9f-4325-87b6-baeaba02bf4e-kube-api-access-l558g" (OuterVolumeSpecName: "kube-api-access-l558g") pod "937b2d57-4a9f-4325-87b6-baeaba02bf4e" (UID: "937b2d57-4a9f-4325-87b6-baeaba02bf4e"). InnerVolumeSpecName "kube-api-access-l558g". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 07 11:19:44.634018 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:19:44.633997 2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/937b2d57-4a9f-4325-87b6-baeaba02bf4e-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "937b2d57-4a9f-4325-87b6-baeaba02bf4e" (UID: "937b2d57-4a9f-4325-87b6-baeaba02bf4e"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 07 11:19:44.634018 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:19:44.634015 2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/937b2d57-4a9f-4325-87b6-baeaba02bf4e-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "937b2d57-4a9f-4325-87b6-baeaba02bf4e" (UID: "937b2d57-4a9f-4325-87b6-baeaba02bf4e"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 07 11:19:44.732778 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:19:44.732741 2578 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/937b2d57-4a9f-4325-87b6-baeaba02bf4e-console-serving-cert\") on node \"ip-10-0-142-112.ec2.internal\" DevicePath \"\"" Jul 07 11:19:44.732778 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:19:44.732770 2578 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/937b2d57-4a9f-4325-87b6-baeaba02bf4e-trusted-ca-bundle\") on node \"ip-10-0-142-112.ec2.internal\" DevicePath \"\"" Jul 07 11:19:44.732778 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:19:44.732779 2578 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/937b2d57-4a9f-4325-87b6-baeaba02bf4e-console-config\") on node \"ip-10-0-142-112.ec2.internal\" DevicePath \"\"" Jul 07 11:19:44.732778 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:19:44.732788 2578 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/937b2d57-4a9f-4325-87b6-baeaba02bf4e-oauth-serving-cert\") on node \"ip-10-0-142-112.ec2.internal\" DevicePath \"\"" Jul 07 11:19:44.733033 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:19:44.732796 2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-l558g\" (UniqueName: \"kubernetes.io/projected/937b2d57-4a9f-4325-87b6-baeaba02bf4e-kube-api-access-l558g\") on node \"ip-10-0-142-112.ec2.internal\" DevicePath \"\"" Jul 07 11:19:44.733033 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:19:44.732805 2578 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/937b2d57-4a9f-4325-87b6-baeaba02bf4e-console-oauth-config\") on node \"ip-10-0-142-112.ec2.internal\" DevicePath \"\"" Jul 07 11:19:44.733033 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:19:44.732813 2578 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/937b2d57-4a9f-4325-87b6-baeaba02bf4e-service-ca\") on node \"ip-10-0-142-112.ec2.internal\" DevicePath \"\"" Jul 07 11:19:45.295470 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:19:45.295436 2578 generic.go:356] "Generic (PLEG): container finished" podID="937b2d57-4a9f-4325-87b6-baeaba02bf4e" containerID="56b0caa415174c2b1896199822a2ea0bbe380ede9657062751e26563d2c37b36" exitCode=0 Jul 07 11:19:45.295849 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:19:45.295485 2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-57449f86c4-77zmv" Jul 07 11:19:45.295849 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:19:45.295510 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-57449f86c4-77zmv" event={"ID":"937b2d57-4a9f-4325-87b6-baeaba02bf4e","Type":"ContainerDied","Data":"56b0caa415174c2b1896199822a2ea0bbe380ede9657062751e26563d2c37b36"} Jul 07 11:19:45.295849 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:19:45.295560 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-57449f86c4-77zmv" event={"ID":"937b2d57-4a9f-4325-87b6-baeaba02bf4e","Type":"ContainerDied","Data":"fe5ff274fc298d583d3850624b4f019a324f0694b31cbaefe1b45ae0dab272c5"} Jul 07 11:19:45.295849 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:19:45.295579 2578 scope.go:117] "RemoveContainer" containerID="56b0caa415174c2b1896199822a2ea0bbe380ede9657062751e26563d2c37b36" Jul 07 11:19:45.303814 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:19:45.303792 2578 scope.go:117] "RemoveContainer" containerID="56b0caa415174c2b1896199822a2ea0bbe380ede9657062751e26563d2c37b36" Jul 07 11:19:45.304074 ip-10-0-142-112 kubenswrapper[2578]: E0707 11:19:45.304056 2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"56b0caa415174c2b1896199822a2ea0bbe380ede9657062751e26563d2c37b36\": container with ID starting with 56b0caa415174c2b1896199822a2ea0bbe380ede9657062751e26563d2c37b36 not found: ID does not exist" containerID="56b0caa415174c2b1896199822a2ea0bbe380ede9657062751e26563d2c37b36" Jul 07 11:19:45.304123 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:19:45.304082 2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"56b0caa415174c2b1896199822a2ea0bbe380ede9657062751e26563d2c37b36"} err="failed to get container status \"56b0caa415174c2b1896199822a2ea0bbe380ede9657062751e26563d2c37b36\": rpc error: code = NotFound desc = could not find container \"56b0caa415174c2b1896199822a2ea0bbe380ede9657062751e26563d2c37b36\": container with ID starting with 56b0caa415174c2b1896199822a2ea0bbe380ede9657062751e26563d2c37b36 not found: ID does not exist" Jul 07 11:19:45.317011 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:19:45.316985 2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-57449f86c4-77zmv"] Jul 07 11:19:45.319173 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:19:45.319151 2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-57449f86c4-77zmv"] Jul 07 11:19:46.307049 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:19:46.307019 2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="937b2d57-4a9f-4325-87b6-baeaba02bf4e" path="/var/lib/kubelet/pods/937b2d57-4a9f-4325-87b6-baeaba02bf4e/volumes" Jul 07 11:20:26.248836 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:20:26.248768 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-55b4d954d5-9r988"] Jul 07 11:20:26.249348 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:20:26.249081 2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="937b2d57-4a9f-4325-87b6-baeaba02bf4e" containerName="console" Jul 07 11:20:26.249348 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:20:26.249096 2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="937b2d57-4a9f-4325-87b6-baeaba02bf4e" containerName="console" Jul 07 11:20:26.249348 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:20:26.249158 2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="937b2d57-4a9f-4325-87b6-baeaba02bf4e" containerName="console" Jul 07 11:20:26.252037 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:20:26.252008 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-55b4d954d5-9r988" Jul 07 11:20:26.254487 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:20:26.254462 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-dockercfg-d9nk7\"" Jul 07 11:20:26.254613 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:20:26.254485 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"kube-root-ca.crt\"" Jul 07 11:20:26.254613 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:20:26.254485 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"openshift-service-ca.crt\"" Jul 07 11:20:26.261660 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:20:26.261633 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-55b4d954d5-9r988"] Jul 07 11:20:26.283475 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:20:26.283408 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8269q\" (UniqueName: \"kubernetes.io/projected/8c663f54-bba2-4a7e-8cbe-1de49a582a21-kube-api-access-8269q\") pod \"cert-manager-55b4d954d5-9r988\" (UID: \"8c663f54-bba2-4a7e-8cbe-1de49a582a21\") " pod="cert-manager/cert-manager-55b4d954d5-9r988" Jul 07 11:20:26.283630 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:20:26.283492 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/8c663f54-bba2-4a7e-8cbe-1de49a582a21-bound-sa-token\") pod \"cert-manager-55b4d954d5-9r988\" (UID: \"8c663f54-bba2-4a7e-8cbe-1de49a582a21\") " pod="cert-manager/cert-manager-55b4d954d5-9r988" Jul 07 11:20:26.384688 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:20:26.384646 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8269q\" (UniqueName: \"kubernetes.io/projected/8c663f54-bba2-4a7e-8cbe-1de49a582a21-kube-api-access-8269q\") pod \"cert-manager-55b4d954d5-9r988\" (UID: \"8c663f54-bba2-4a7e-8cbe-1de49a582a21\") " pod="cert-manager/cert-manager-55b4d954d5-9r988" Jul 07 11:20:26.384877 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:20:26.384711 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/8c663f54-bba2-4a7e-8cbe-1de49a582a21-bound-sa-token\") pod \"cert-manager-55b4d954d5-9r988\" (UID: \"8c663f54-bba2-4a7e-8cbe-1de49a582a21\") " pod="cert-manager/cert-manager-55b4d954d5-9r988" Jul 07 11:20:26.394182 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:20:26.394147 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/8c663f54-bba2-4a7e-8cbe-1de49a582a21-bound-sa-token\") pod \"cert-manager-55b4d954d5-9r988\" (UID: \"8c663f54-bba2-4a7e-8cbe-1de49a582a21\") " pod="cert-manager/cert-manager-55b4d954d5-9r988" Jul 07 11:20:26.394295 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:20:26.394241 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-8269q\" (UniqueName: \"kubernetes.io/projected/8c663f54-bba2-4a7e-8cbe-1de49a582a21-kube-api-access-8269q\") pod \"cert-manager-55b4d954d5-9r988\" (UID: \"8c663f54-bba2-4a7e-8cbe-1de49a582a21\") " pod="cert-manager/cert-manager-55b4d954d5-9r988" Jul 07 11:20:26.569884 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:20:26.569853 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-55b4d954d5-9r988" Jul 07 11:20:26.717013 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:20:26.716977 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-55b4d954d5-9r988"] Jul 07 11:20:26.721316 ip-10-0-142-112 kubenswrapper[2578]: W0707 11:20:26.721280 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod8c663f54_bba2_4a7e_8cbe_1de49a582a21.slice/crio-359b16f876f4bbf80bd11966e6e1414ee5bc29f98f2b04b73078a10e0bfd6c46 WatchSource:0}: Error finding container 359b16f876f4bbf80bd11966e6e1414ee5bc29f98f2b04b73078a10e0bfd6c46: Status 404 returned error can't find the container with id 359b16f876f4bbf80bd11966e6e1414ee5bc29f98f2b04b73078a10e0bfd6c46 Jul 07 11:20:27.459332 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:20:27.459249 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-55b4d954d5-9r988" event={"ID":"8c663f54-bba2-4a7e-8cbe-1de49a582a21","Type":"ContainerStarted","Data":"359b16f876f4bbf80bd11966e6e1414ee5bc29f98f2b04b73078a10e0bfd6c46"} Jul 07 11:20:30.471473 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:20:30.471414 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-55b4d954d5-9r988" event={"ID":"8c663f54-bba2-4a7e-8cbe-1de49a582a21","Type":"ContainerStarted","Data":"3b60e7894b289339a34e629460a4a00b96ef4115d3d66ff6721fe94696d3cd6d"} Jul 07 11:20:30.486654 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:20:30.486605 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager/cert-manager-55b4d954d5-9r988" podStartSLOduration=1.620553589 podStartE2EDuration="4.486589036s" totalImagesPullingTime="2.866035447s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 11:20:26 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 11:20:30.486488853 +0000 UTC m=+274.782638458" watchObservedRunningTime="2026-07-07 11:20:30.486589036 +0000 UTC m=+274.782738618" Jul 07 11:20:51.227466 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:20:51.227416 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-59ddd9ddd8-9mtpg"] Jul 07 11:20:51.230590 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:20:51.230571 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-59ddd9ddd8-9mtpg" Jul 07 11:20:51.233615 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:20:51.233589 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\"" Jul 07 11:20:51.234075 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:20:51.234060 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-service-cert\"" Jul 07 11:20:51.234185 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:20:51.234166 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-webhook-cert\"" Jul 07 11:20:51.234253 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:20:51.234237 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-dockercfg-tp44d\"" Jul 07 11:20:51.246643 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:20:51.246620 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-59ddd9ddd8-9mtpg"] Jul 07 11:20:51.249895 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:20:51.249876 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\"" Jul 07 11:20:51.401605 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:20:51.401564 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/c6075474-f9d8-4cfb-821c-707c5ef167f1-webhook-cert\") pod \"opendatahub-operator-controller-manager-59ddd9ddd8-9mtpg\" (UID: \"c6075474-f9d8-4cfb-821c-707c5ef167f1\") " pod="opendatahub/opendatahub-operator-controller-manager-59ddd9ddd8-9mtpg" Jul 07 11:20:51.401776 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:20:51.401622 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-v52xw\" (UniqueName: \"kubernetes.io/projected/c6075474-f9d8-4cfb-821c-707c5ef167f1-kube-api-access-v52xw\") pod \"opendatahub-operator-controller-manager-59ddd9ddd8-9mtpg\" (UID: \"c6075474-f9d8-4cfb-821c-707c5ef167f1\") " pod="opendatahub/opendatahub-operator-controller-manager-59ddd9ddd8-9mtpg" Jul 07 11:20:51.401776 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:20:51.401702 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/c6075474-f9d8-4cfb-821c-707c5ef167f1-manifests\") pod \"opendatahub-operator-controller-manager-59ddd9ddd8-9mtpg\" (UID: \"c6075474-f9d8-4cfb-821c-707c5ef167f1\") " pod="opendatahub/opendatahub-operator-controller-manager-59ddd9ddd8-9mtpg" Jul 07 11:20:51.401853 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:20:51.401782 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/c6075474-f9d8-4cfb-821c-707c5ef167f1-tmp\") pod \"opendatahub-operator-controller-manager-59ddd9ddd8-9mtpg\" (UID: \"c6075474-f9d8-4cfb-821c-707c5ef167f1\") " pod="opendatahub/opendatahub-operator-controller-manager-59ddd9ddd8-9mtpg" Jul 07 11:20:51.401853 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:20:51.401806 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/c6075474-f9d8-4cfb-821c-707c5ef167f1-apiservice-cert\") pod \"opendatahub-operator-controller-manager-59ddd9ddd8-9mtpg\" (UID: \"c6075474-f9d8-4cfb-821c-707c5ef167f1\") " pod="opendatahub/opendatahub-operator-controller-manager-59ddd9ddd8-9mtpg" Jul 07 11:20:51.502510 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:20:51.502417 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/c6075474-f9d8-4cfb-821c-707c5ef167f1-webhook-cert\") pod \"opendatahub-operator-controller-manager-59ddd9ddd8-9mtpg\" (UID: \"c6075474-f9d8-4cfb-821c-707c5ef167f1\") " pod="opendatahub/opendatahub-operator-controller-manager-59ddd9ddd8-9mtpg" Jul 07 11:20:51.502510 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:20:51.502483 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-v52xw\" (UniqueName: \"kubernetes.io/projected/c6075474-f9d8-4cfb-821c-707c5ef167f1-kube-api-access-v52xw\") pod \"opendatahub-operator-controller-manager-59ddd9ddd8-9mtpg\" (UID: \"c6075474-f9d8-4cfb-821c-707c5ef167f1\") " pod="opendatahub/opendatahub-operator-controller-manager-59ddd9ddd8-9mtpg" Jul 07 11:20:51.502510 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:20:51.502507 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/c6075474-f9d8-4cfb-821c-707c5ef167f1-manifests\") pod \"opendatahub-operator-controller-manager-59ddd9ddd8-9mtpg\" (UID: \"c6075474-f9d8-4cfb-821c-707c5ef167f1\") " pod="opendatahub/opendatahub-operator-controller-manager-59ddd9ddd8-9mtpg" Jul 07 11:20:51.502719 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:20:51.502555 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/c6075474-f9d8-4cfb-821c-707c5ef167f1-tmp\") pod \"opendatahub-operator-controller-manager-59ddd9ddd8-9mtpg\" (UID: \"c6075474-f9d8-4cfb-821c-707c5ef167f1\") " pod="opendatahub/opendatahub-operator-controller-manager-59ddd9ddd8-9mtpg" Jul 07 11:20:51.502719 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:20:51.502661 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/c6075474-f9d8-4cfb-821c-707c5ef167f1-apiservice-cert\") pod \"opendatahub-operator-controller-manager-59ddd9ddd8-9mtpg\" (UID: \"c6075474-f9d8-4cfb-821c-707c5ef167f1\") " pod="opendatahub/opendatahub-operator-controller-manager-59ddd9ddd8-9mtpg" Jul 07 11:20:51.503011 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:20:51.502981 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/c6075474-f9d8-4cfb-821c-707c5ef167f1-manifests\") pod \"opendatahub-operator-controller-manager-59ddd9ddd8-9mtpg\" (UID: \"c6075474-f9d8-4cfb-821c-707c5ef167f1\") " pod="opendatahub/opendatahub-operator-controller-manager-59ddd9ddd8-9mtpg" Jul 07 11:20:51.504733 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:20:51.504704 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/c6075474-f9d8-4cfb-821c-707c5ef167f1-tmp\") pod \"opendatahub-operator-controller-manager-59ddd9ddd8-9mtpg\" (UID: \"c6075474-f9d8-4cfb-821c-707c5ef167f1\") " pod="opendatahub/opendatahub-operator-controller-manager-59ddd9ddd8-9mtpg" Jul 07 11:20:51.504942 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:20:51.504926 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/c6075474-f9d8-4cfb-821c-707c5ef167f1-webhook-cert\") pod \"opendatahub-operator-controller-manager-59ddd9ddd8-9mtpg\" (UID: \"c6075474-f9d8-4cfb-821c-707c5ef167f1\") " pod="opendatahub/opendatahub-operator-controller-manager-59ddd9ddd8-9mtpg" Jul 07 11:20:51.504982 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:20:51.504944 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/c6075474-f9d8-4cfb-821c-707c5ef167f1-apiservice-cert\") pod \"opendatahub-operator-controller-manager-59ddd9ddd8-9mtpg\" (UID: \"c6075474-f9d8-4cfb-821c-707c5ef167f1\") " pod="opendatahub/opendatahub-operator-controller-manager-59ddd9ddd8-9mtpg" Jul 07 11:20:51.511487 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:20:51.511415 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-v52xw\" (UniqueName: \"kubernetes.io/projected/c6075474-f9d8-4cfb-821c-707c5ef167f1-kube-api-access-v52xw\") pod \"opendatahub-operator-controller-manager-59ddd9ddd8-9mtpg\" (UID: \"c6075474-f9d8-4cfb-821c-707c5ef167f1\") " pod="opendatahub/opendatahub-operator-controller-manager-59ddd9ddd8-9mtpg" Jul 07 11:20:51.541397 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:20:51.541370 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-59ddd9ddd8-9mtpg" Jul 07 11:20:51.697948 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:20:51.697916 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-59ddd9ddd8-9mtpg"] Jul 07 11:20:51.702089 ip-10-0-142-112 kubenswrapper[2578]: W0707 11:20:51.702062 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc6075474_f9d8_4cfb_821c_707c5ef167f1.slice/crio-0abe3e8a4ae3571da6d897718386dcf163cedbd3f9ad73002e890ad847d4f7d0 WatchSource:0}: Error finding container 0abe3e8a4ae3571da6d897718386dcf163cedbd3f9ad73002e890ad847d4f7d0: Status 404 returned error can't find the container with id 0abe3e8a4ae3571da6d897718386dcf163cedbd3f9ad73002e890ad847d4f7d0 Jul 07 11:20:52.551174 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:20:52.551131 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-59ddd9ddd8-9mtpg" event={"ID":"c6075474-f9d8-4cfb-821c-707c5ef167f1","Type":"ContainerStarted","Data":"0abe3e8a4ae3571da6d897718386dcf163cedbd3f9ad73002e890ad847d4f7d0"} Jul 07 11:20:55.564489 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:20:55.564453 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-59ddd9ddd8-9mtpg" event={"ID":"c6075474-f9d8-4cfb-821c-707c5ef167f1","Type":"ContainerStarted","Data":"d2ae7893fe4d8407087e457981f51c63019f7f2864009b59d7bde986f8531c1b"} Jul 07 11:20:56.153787 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:20:56.153750 2578 kubelet.go:1648] "Image garbage collection succeeded" Jul 07 11:20:56.569471 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:20:56.569417 2578 generic.go:356] "Generic (PLEG): container finished" podID="c6075474-f9d8-4cfb-821c-707c5ef167f1" containerID="d2ae7893fe4d8407087e457981f51c63019f7f2864009b59d7bde986f8531c1b" exitCode=0 Jul 07 11:20:56.572003 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:20:56.569486 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-59ddd9ddd8-9mtpg" event={"ID":"c6075474-f9d8-4cfb-821c-707c5ef167f1","Type":"ContainerDied","Data":"d2ae7893fe4d8407087e457981f51c63019f7f2864009b59d7bde986f8531c1b"} Jul 07 11:20:57.577480 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:20:57.577418 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-59ddd9ddd8-9mtpg" event={"ID":"c6075474-f9d8-4cfb-821c-707c5ef167f1","Type":"ContainerStarted","Data":"5ac708b2a71e0d2d504ea931af4e339ec6487d17cde0855da8617f6a8a3a0961"} Jul 07 11:20:57.577962 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:20:57.577572 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/opendatahub-operator-controller-manager-59ddd9ddd8-9mtpg" Jul 07 11:20:57.624223 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:20:57.623811 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/opendatahub-operator-controller-manager-59ddd9ddd8-9mtpg" podStartSLOduration=2.348949053 podStartE2EDuration="6.62379091s" totalImagesPullingTime="3.20151124s" totalInitContainerRuntime="1.073330617s" isStatefulPod=false podCreationTimestamp="2026-07-07 11:20:51 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 11:20:57.620511434 +0000 UTC m=+301.916661021" watchObservedRunningTime="2026-07-07 11:20:57.62379091 +0000 UTC m=+301.919940499" Jul 07 11:21:08.583264 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:21:08.583235 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/opendatahub-operator-controller-manager-59ddd9ddd8-9mtpg" Jul 07 11:21:10.926775 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:21:10.926743 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/kube-auth-proxy-785759898c-7pnk6"] Jul 07 11:21:10.933589 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:21:10.933568 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-785759898c-7pnk6" Jul 07 11:21:10.937665 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:21:10.937643 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-creds\"" Jul 07 11:21:10.937958 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:21:10.937941 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-tls\"" Jul 07 11:21:10.938500 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:21:10.938485 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\"" Jul 07 11:21:10.938748 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:21:10.938735 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\"" Jul 07 11:21:10.938794 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:21:10.938731 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-dockercfg-hq5xl\"" Jul 07 11:21:10.955143 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:21:10.955115 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-785759898c-7pnk6"] Jul 07 11:21:10.958073 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:21:10.958050 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/ffe5c6b1-dc1e-4fea-b7b6-a6a527517ec5-tmp\") pod \"kube-auth-proxy-785759898c-7pnk6\" (UID: \"ffe5c6b1-dc1e-4fea-b7b6-a6a527517ec5\") " pod="openshift-ingress/kube-auth-proxy-785759898c-7pnk6" Jul 07 11:21:10.958177 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:21:10.958081 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-d5h5l\" (UniqueName: \"kubernetes.io/projected/ffe5c6b1-dc1e-4fea-b7b6-a6a527517ec5-kube-api-access-d5h5l\") pod \"kube-auth-proxy-785759898c-7pnk6\" (UID: \"ffe5c6b1-dc1e-4fea-b7b6-a6a527517ec5\") " pod="openshift-ingress/kube-auth-proxy-785759898c-7pnk6" Jul 07 11:21:10.958227 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:21:10.958209 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ffe5c6b1-dc1e-4fea-b7b6-a6a527517ec5-tls-certs\") pod \"kube-auth-proxy-785759898c-7pnk6\" (UID: \"ffe5c6b1-dc1e-4fea-b7b6-a6a527517ec5\") " pod="openshift-ingress/kube-auth-proxy-785759898c-7pnk6" Jul 07 11:21:11.059203 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:21:11.059162 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ffe5c6b1-dc1e-4fea-b7b6-a6a527517ec5-tls-certs\") pod \"kube-auth-proxy-785759898c-7pnk6\" (UID: \"ffe5c6b1-dc1e-4fea-b7b6-a6a527517ec5\") " pod="openshift-ingress/kube-auth-proxy-785759898c-7pnk6" Jul 07 11:21:11.059203 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:21:11.059212 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/ffe5c6b1-dc1e-4fea-b7b6-a6a527517ec5-tmp\") pod \"kube-auth-proxy-785759898c-7pnk6\" (UID: \"ffe5c6b1-dc1e-4fea-b7b6-a6a527517ec5\") " pod="openshift-ingress/kube-auth-proxy-785759898c-7pnk6" Jul 07 11:21:11.059494 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:21:11.059229 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-d5h5l\" (UniqueName: \"kubernetes.io/projected/ffe5c6b1-dc1e-4fea-b7b6-a6a527517ec5-kube-api-access-d5h5l\") pod \"kube-auth-proxy-785759898c-7pnk6\" (UID: \"ffe5c6b1-dc1e-4fea-b7b6-a6a527517ec5\") " pod="openshift-ingress/kube-auth-proxy-785759898c-7pnk6" Jul 07 11:21:11.061545 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:21:11.061512 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/ffe5c6b1-dc1e-4fea-b7b6-a6a527517ec5-tmp\") pod \"kube-auth-proxy-785759898c-7pnk6\" (UID: \"ffe5c6b1-dc1e-4fea-b7b6-a6a527517ec5\") " pod="openshift-ingress/kube-auth-proxy-785759898c-7pnk6" Jul 07 11:21:11.061728 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:21:11.061711 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ffe5c6b1-dc1e-4fea-b7b6-a6a527517ec5-tls-certs\") pod \"kube-auth-proxy-785759898c-7pnk6\" (UID: \"ffe5c6b1-dc1e-4fea-b7b6-a6a527517ec5\") " pod="openshift-ingress/kube-auth-proxy-785759898c-7pnk6" Jul 07 11:21:11.087406 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:21:11.087378 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-d5h5l\" (UniqueName: \"kubernetes.io/projected/ffe5c6b1-dc1e-4fea-b7b6-a6a527517ec5-kube-api-access-d5h5l\") pod \"kube-auth-proxy-785759898c-7pnk6\" (UID: \"ffe5c6b1-dc1e-4fea-b7b6-a6a527517ec5\") " pod="openshift-ingress/kube-auth-proxy-785759898c-7pnk6" Jul 07 11:21:11.243565 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:21:11.243478 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-785759898c-7pnk6" Jul 07 11:21:11.370143 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:21:11.370081 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-785759898c-7pnk6"] Jul 07 11:21:11.372769 ip-10-0-142-112 kubenswrapper[2578]: W0707 11:21:11.372741 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podffe5c6b1_dc1e_4fea_b7b6_a6a527517ec5.slice/crio-1641e02baa0828232358fb1a437ce9a2738a9625b6007023b7076878e26a8149 WatchSource:0}: Error finding container 1641e02baa0828232358fb1a437ce9a2738a9625b6007023b7076878e26a8149: Status 404 returned error can't find the container with id 1641e02baa0828232358fb1a437ce9a2738a9625b6007023b7076878e26a8149 Jul 07 11:21:11.374528 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:21:11.374511 2578 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 07 11:21:11.629978 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:21:11.629941 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-785759898c-7pnk6" event={"ID":"ffe5c6b1-dc1e-4fea-b7b6-a6a527517ec5","Type":"ContainerStarted","Data":"1641e02baa0828232358fb1a437ce9a2738a9625b6007023b7076878e26a8149"} Jul 07 11:21:14.642562 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:21:14.642471 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-785759898c-7pnk6" event={"ID":"ffe5c6b1-dc1e-4fea-b7b6-a6a527517ec5","Type":"ContainerStarted","Data":"5ab3f27dcda4922f8ae5b56a7d153c2020121588e496e9d90b780f17b7fb4550"} Jul 07 11:21:14.685910 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:21:14.685804 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/kube-auth-proxy-785759898c-7pnk6" podStartSLOduration=1.697427715 podStartE2EDuration="4.685782274s" totalImagesPullingTime="2.988354559s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 11:21:10 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 11:21:14.684994001 +0000 UTC m=+318.981143588" watchObservedRunningTime="2026-07-07 11:21:14.685782274 +0000 UTC m=+318.981931840" Jul 07 11:21:18.661346 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:21:18.661315 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4q24nh"] Jul 07 11:21:18.664540 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:21:18.664523 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4q24nh" Jul 07 11:21:18.667548 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:21:18.667524 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"data-science-gateway-data-science-gateway-class-dockercfg-7gwg7\"" Jul 07 11:21:18.667779 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:21:18.667754 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\"" Jul 07 11:21:18.683466 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:21:18.683438 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4q24nh"] Jul 07 11:21:18.727507 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:21:18.727470 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5c24eef8-4fc8-4973-a25b-4e93f4badb41-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4q24nh\" (UID: \"5c24eef8-4fc8-4973-a25b-4e93f4badb41\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4q24nh" Jul 07 11:21:18.727507 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:21:18.727506 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5c24eef8-4fc8-4973-a25b-4e93f4badb41-istio-data\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4q24nh\" (UID: \"5c24eef8-4fc8-4973-a25b-4e93f4badb41\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4q24nh" Jul 07 11:21:18.727727 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:21:18.727524 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5c24eef8-4fc8-4973-a25b-4e93f4badb41-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4q24nh\" (UID: \"5c24eef8-4fc8-4973-a25b-4e93f4badb41\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4q24nh" Jul 07 11:21:18.727727 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:21:18.727594 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5c24eef8-4fc8-4973-a25b-4e93f4badb41-istio-token\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4q24nh\" (UID: \"5c24eef8-4fc8-4973-a25b-4e93f4badb41\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4q24nh" Jul 07 11:21:18.727727 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:21:18.727631 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5c24eef8-4fc8-4973-a25b-4e93f4badb41-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4q24nh\" (UID: \"5c24eef8-4fc8-4973-a25b-4e93f4badb41\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4q24nh" Jul 07 11:21:18.727727 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:21:18.727680 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5c24eef8-4fc8-4973-a25b-4e93f4badb41-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4q24nh\" (UID: \"5c24eef8-4fc8-4973-a25b-4e93f4badb41\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4q24nh" Jul 07 11:21:18.727917 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:21:18.727760 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5c24eef8-4fc8-4973-a25b-4e93f4badb41-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4q24nh\" (UID: \"5c24eef8-4fc8-4973-a25b-4e93f4badb41\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4q24nh" Jul 07 11:21:18.727917 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:21:18.727799 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-z4hsj\" (UniqueName: \"kubernetes.io/projected/5c24eef8-4fc8-4973-a25b-4e93f4badb41-kube-api-access-z4hsj\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4q24nh\" (UID: \"5c24eef8-4fc8-4973-a25b-4e93f4badb41\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4q24nh" Jul 07 11:21:18.727917 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:21:18.727838 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5c24eef8-4fc8-4973-a25b-4e93f4badb41-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4q24nh\" (UID: \"5c24eef8-4fc8-4973-a25b-4e93f4badb41\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4q24nh" Jul 07 11:21:18.828551 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:21:18.828511 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5c24eef8-4fc8-4973-a25b-4e93f4badb41-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4q24nh\" (UID: \"5c24eef8-4fc8-4973-a25b-4e93f4badb41\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4q24nh" Jul 07 11:21:18.828551 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:21:18.828549 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5c24eef8-4fc8-4973-a25b-4e93f4badb41-istio-data\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4q24nh\" (UID: \"5c24eef8-4fc8-4973-a25b-4e93f4badb41\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4q24nh" Jul 07 11:21:18.828809 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:21:18.828567 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5c24eef8-4fc8-4973-a25b-4e93f4badb41-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4q24nh\" (UID: \"5c24eef8-4fc8-4973-a25b-4e93f4badb41\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4q24nh" Jul 07 11:21:18.828809 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:21:18.828594 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5c24eef8-4fc8-4973-a25b-4e93f4badb41-istio-token\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4q24nh\" (UID: \"5c24eef8-4fc8-4973-a25b-4e93f4badb41\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4q24nh" Jul 07 11:21:18.828809 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:21:18.828614 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5c24eef8-4fc8-4973-a25b-4e93f4badb41-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4q24nh\" (UID: \"5c24eef8-4fc8-4973-a25b-4e93f4badb41\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4q24nh" Jul 07 11:21:18.828809 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:21:18.828652 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5c24eef8-4fc8-4973-a25b-4e93f4badb41-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4q24nh\" (UID: \"5c24eef8-4fc8-4973-a25b-4e93f4badb41\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4q24nh" Jul 07 11:21:18.828809 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:21:18.828671 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5c24eef8-4fc8-4973-a25b-4e93f4badb41-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4q24nh\" (UID: \"5c24eef8-4fc8-4973-a25b-4e93f4badb41\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4q24nh" Jul 07 11:21:18.828809 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:21:18.828690 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-z4hsj\" (UniqueName: \"kubernetes.io/projected/5c24eef8-4fc8-4973-a25b-4e93f4badb41-kube-api-access-z4hsj\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4q24nh\" (UID: \"5c24eef8-4fc8-4973-a25b-4e93f4badb41\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4q24nh" Jul 07 11:21:18.828809 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:21:18.828721 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5c24eef8-4fc8-4973-a25b-4e93f4badb41-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4q24nh\" (UID: \"5c24eef8-4fc8-4973-a25b-4e93f4badb41\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4q24nh" Jul 07 11:21:18.829161 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:21:18.828998 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5c24eef8-4fc8-4973-a25b-4e93f4badb41-istio-data\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4q24nh\" (UID: \"5c24eef8-4fc8-4973-a25b-4e93f4badb41\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4q24nh" Jul 07 11:21:18.829161 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:21:18.829058 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5c24eef8-4fc8-4973-a25b-4e93f4badb41-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4q24nh\" (UID: \"5c24eef8-4fc8-4973-a25b-4e93f4badb41\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4q24nh" Jul 07 11:21:18.829161 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:21:18.829152 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5c24eef8-4fc8-4973-a25b-4e93f4badb41-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4q24nh\" (UID: \"5c24eef8-4fc8-4973-a25b-4e93f4badb41\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4q24nh" Jul 07 11:21:18.829308 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:21:18.829250 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5c24eef8-4fc8-4973-a25b-4e93f4badb41-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4q24nh\" (UID: \"5c24eef8-4fc8-4973-a25b-4e93f4badb41\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4q24nh" Jul 07 11:21:18.829344 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:21:18.829316 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5c24eef8-4fc8-4973-a25b-4e93f4badb41-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4q24nh\" (UID: \"5c24eef8-4fc8-4973-a25b-4e93f4badb41\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4q24nh" Jul 07 11:21:18.831014 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:21:18.830981 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5c24eef8-4fc8-4973-a25b-4e93f4badb41-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4q24nh\" (UID: \"5c24eef8-4fc8-4973-a25b-4e93f4badb41\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4q24nh" Jul 07 11:21:18.831199 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:21:18.831172 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5c24eef8-4fc8-4973-a25b-4e93f4badb41-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4q24nh\" (UID: \"5c24eef8-4fc8-4973-a25b-4e93f4badb41\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4q24nh" Jul 07 11:21:18.840094 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:21:18.840074 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-z4hsj\" (UniqueName: \"kubernetes.io/projected/5c24eef8-4fc8-4973-a25b-4e93f4badb41-kube-api-access-z4hsj\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4q24nh\" (UID: \"5c24eef8-4fc8-4973-a25b-4e93f4badb41\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4q24nh" Jul 07 11:21:18.848118 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:21:18.848096 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5c24eef8-4fc8-4973-a25b-4e93f4badb41-istio-token\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4q24nh\" (UID: \"5c24eef8-4fc8-4973-a25b-4e93f4badb41\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4q24nh" Jul 07 11:21:18.977341 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:21:18.977246 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4q24nh" Jul 07 11:21:19.114377 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:21:19.114314 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4q24nh"] Jul 07 11:21:19.116963 ip-10-0-142-112 kubenswrapper[2578]: W0707 11:21:19.116938 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5c24eef8_4fc8_4973_a25b_4e93f4badb41.slice/crio-10481f494e0a8b2c407d801d8c52fc210df23a19cb6e7522638ee4dccf5bd9d2 WatchSource:0}: Error finding container 10481f494e0a8b2c407d801d8c52fc210df23a19cb6e7522638ee4dccf5bd9d2: Status 404 returned error can't find the container with id 10481f494e0a8b2c407d801d8c52fc210df23a19cb6e7522638ee4dccf5bd9d2 Jul 07 11:21:19.666492 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:21:19.666439 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4q24nh" event={"ID":"5c24eef8-4fc8-4973-a25b-4e93f4badb41","Type":"ContainerStarted","Data":"10481f494e0a8b2c407d801d8c52fc210df23a19cb6e7522638ee4dccf5bd9d2"} Jul 07 11:21:19.920607 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:21:19.920519 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/istiod-openshift-gateway-94bb8fbfd-qmvqq"] Jul 07 11:21:19.926206 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:21:19.926175 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-qmvqq" Jul 07 11:21:19.929023 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:21:19.928802 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-tls\"" Jul 07 11:21:19.929023 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:21:19.928802 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istio-kubeconfig\"" Jul 07 11:21:19.932371 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:21:19.932181 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"cacerts\"" Jul 07 11:21:19.932371 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:21:19.932299 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-openshift-gateway-dockercfg-pb7sv\"" Jul 07 11:21:19.945537 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:21:19.945498 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-94bb8fbfd-qmvqq"] Jul 07 11:21:20.038605 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:21:20.038569 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/8111ef68-a869-479b-9087-2e5ed23349c0-local-certs\") pod \"istiod-openshift-gateway-94bb8fbfd-qmvqq\" (UID: \"8111ef68-a869-479b-9087-2e5ed23349c0\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-qmvqq" Jul 07 11:21:20.038771 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:21:20.038623 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/8111ef68-a869-479b-9087-2e5ed23349c0-cacerts\") pod \"istiod-openshift-gateway-94bb8fbfd-qmvqq\" (UID: \"8111ef68-a869-479b-9087-2e5ed23349c0\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-qmvqq" Jul 07 11:21:20.038771 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:21:20.038697 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/8111ef68-a869-479b-9087-2e5ed23349c0-istio-kubeconfig\") pod \"istiod-openshift-gateway-94bb8fbfd-qmvqq\" (UID: \"8111ef68-a869-479b-9087-2e5ed23349c0\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-qmvqq" Jul 07 11:21:20.038771 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:21:20.038729 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/8111ef68-a869-479b-9087-2e5ed23349c0-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-94bb8fbfd-qmvqq\" (UID: \"8111ef68-a869-479b-9087-2e5ed23349c0\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-qmvqq" Jul 07 11:21:20.038771 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:21:20.038763 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/8111ef68-a869-479b-9087-2e5ed23349c0-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-94bb8fbfd-qmvqq\" (UID: \"8111ef68-a869-479b-9087-2e5ed23349c0\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-qmvqq" Jul 07 11:21:20.038983 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:21:20.038807 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-s9b7h\" (UniqueName: \"kubernetes.io/projected/8111ef68-a869-479b-9087-2e5ed23349c0-kube-api-access-s9b7h\") pod \"istiod-openshift-gateway-94bb8fbfd-qmvqq\" (UID: \"8111ef68-a869-479b-9087-2e5ed23349c0\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-qmvqq" Jul 07 11:21:20.038983 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:21:20.038852 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8111ef68-a869-479b-9087-2e5ed23349c0-istio-token\") pod \"istiod-openshift-gateway-94bb8fbfd-qmvqq\" (UID: \"8111ef68-a869-479b-9087-2e5ed23349c0\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-qmvqq" Jul 07 11:21:20.140085 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:21:20.140047 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/8111ef68-a869-479b-9087-2e5ed23349c0-istio-kubeconfig\") pod \"istiod-openshift-gateway-94bb8fbfd-qmvqq\" (UID: \"8111ef68-a869-479b-9087-2e5ed23349c0\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-qmvqq" Jul 07 11:21:20.140252 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:21:20.140101 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/8111ef68-a869-479b-9087-2e5ed23349c0-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-94bb8fbfd-qmvqq\" (UID: \"8111ef68-a869-479b-9087-2e5ed23349c0\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-qmvqq" Jul 07 11:21:20.140252 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:21:20.140139 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/8111ef68-a869-479b-9087-2e5ed23349c0-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-94bb8fbfd-qmvqq\" (UID: \"8111ef68-a869-479b-9087-2e5ed23349c0\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-qmvqq" Jul 07 11:21:20.140252 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:21:20.140168 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-s9b7h\" (UniqueName: \"kubernetes.io/projected/8111ef68-a869-479b-9087-2e5ed23349c0-kube-api-access-s9b7h\") pod \"istiod-openshift-gateway-94bb8fbfd-qmvqq\" (UID: \"8111ef68-a869-479b-9087-2e5ed23349c0\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-qmvqq" Jul 07 11:21:20.140252 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:21:20.140214 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8111ef68-a869-479b-9087-2e5ed23349c0-istio-token\") pod \"istiod-openshift-gateway-94bb8fbfd-qmvqq\" (UID: \"8111ef68-a869-479b-9087-2e5ed23349c0\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-qmvqq" Jul 07 11:21:20.140507 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:21:20.140266 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/8111ef68-a869-479b-9087-2e5ed23349c0-local-certs\") pod \"istiod-openshift-gateway-94bb8fbfd-qmvqq\" (UID: \"8111ef68-a869-479b-9087-2e5ed23349c0\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-qmvqq" Jul 07 11:21:20.140507 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:21:20.140301 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/8111ef68-a869-479b-9087-2e5ed23349c0-cacerts\") pod \"istiod-openshift-gateway-94bb8fbfd-qmvqq\" (UID: \"8111ef68-a869-479b-9087-2e5ed23349c0\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-qmvqq" Jul 07 11:21:20.141012 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:21:20.140985 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/8111ef68-a869-479b-9087-2e5ed23349c0-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-94bb8fbfd-qmvqq\" (UID: \"8111ef68-a869-479b-9087-2e5ed23349c0\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-qmvqq" Jul 07 11:21:20.142968 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:21:20.142936 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/8111ef68-a869-479b-9087-2e5ed23349c0-istio-kubeconfig\") pod \"istiod-openshift-gateway-94bb8fbfd-qmvqq\" (UID: \"8111ef68-a869-479b-9087-2e5ed23349c0\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-qmvqq" Jul 07 11:21:20.143062 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:21:20.143016 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/8111ef68-a869-479b-9087-2e5ed23349c0-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-94bb8fbfd-qmvqq\" (UID: \"8111ef68-a869-479b-9087-2e5ed23349c0\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-qmvqq" Jul 07 11:21:20.143121 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:21:20.143057 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/8111ef68-a869-479b-9087-2e5ed23349c0-local-certs\") pod \"istiod-openshift-gateway-94bb8fbfd-qmvqq\" (UID: \"8111ef68-a869-479b-9087-2e5ed23349c0\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-qmvqq" Jul 07 11:21:20.143410 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:21:20.143387 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/8111ef68-a869-479b-9087-2e5ed23349c0-cacerts\") pod \"istiod-openshift-gateway-94bb8fbfd-qmvqq\" (UID: \"8111ef68-a869-479b-9087-2e5ed23349c0\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-qmvqq" Jul 07 11:21:20.168317 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:21:20.168290 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8111ef68-a869-479b-9087-2e5ed23349c0-istio-token\") pod \"istiod-openshift-gateway-94bb8fbfd-qmvqq\" (UID: \"8111ef68-a869-479b-9087-2e5ed23349c0\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-qmvqq" Jul 07 11:21:20.168614 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:21:20.168584 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-s9b7h\" (UniqueName: \"kubernetes.io/projected/8111ef68-a869-479b-9087-2e5ed23349c0-kube-api-access-s9b7h\") pod \"istiod-openshift-gateway-94bb8fbfd-qmvqq\" (UID: \"8111ef68-a869-479b-9087-2e5ed23349c0\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-qmvqq" Jul 07 11:21:20.238542 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:21:20.238464 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-qmvqq" Jul 07 11:21:20.439161 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:21:20.439136 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-94bb8fbfd-qmvqq"] Jul 07 11:21:20.441510 ip-10-0-142-112 kubenswrapper[2578]: W0707 11:21:20.441474 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8111ef68_a869_479b_9087_2e5ed23349c0.slice/crio-aa3787726da014bc679b1f3da56cdf19c0c208ae20be57baaeb858f4ecfeb970 WatchSource:0}: Error finding container aa3787726da014bc679b1f3da56cdf19c0c208ae20be57baaeb858f4ecfeb970: Status 404 returned error can't find the container with id aa3787726da014bc679b1f3da56cdf19c0c208ae20be57baaeb858f4ecfeb970 Jul 07 11:21:20.681696 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:21:20.681653 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-qmvqq" event={"ID":"8111ef68-a869-479b-9087-2e5ed23349c0","Type":"ContainerStarted","Data":"aa3787726da014bc679b1f3da56cdf19c0c208ae20be57baaeb858f4ecfeb970"} Jul 07 11:21:21.540917 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:21:21.540878 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-lws-operator/lws-controller-manager-65f8ccd9c5-8tww2"] Jul 07 11:21:21.552462 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:21:21.552241 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-65f8ccd9c5-8tww2" Jul 07 11:21:21.556387 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:21:21.556361 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"kube-root-ca.crt\"" Jul 07 11:21:21.557576 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:21:21.557535 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"metrics-server-cert\"" Jul 07 11:21:21.557697 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:21:21.557631 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"lws-manager-config\"" Jul 07 11:21:21.557785 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:21:21.557731 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"openshift-service-ca.crt\"" Jul 07 11:21:21.557843 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:21:21.557535 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"webhook-server-cert\"" Jul 07 11:21:21.566478 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:21:21.566450 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-65f8ccd9c5-8tww2"] Jul 07 11:21:21.566999 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:21:21.566977 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"lws-controller-manager-dockercfg-k5prt\"" Jul 07 11:21:21.653867 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:21:21.653829 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/3431aa7e-d46e-4ac2-8c76-b3d62a99eb0e-metrics-cert\") pod \"lws-controller-manager-65f8ccd9c5-8tww2\" (UID: \"3431aa7e-d46e-4ac2-8c76-b3d62a99eb0e\") " pod="openshift-lws-operator/lws-controller-manager-65f8ccd9c5-8tww2" Jul 07 11:21:21.654031 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:21:21.653895 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/3431aa7e-d46e-4ac2-8c76-b3d62a99eb0e-cert\") pod \"lws-controller-manager-65f8ccd9c5-8tww2\" (UID: \"3431aa7e-d46e-4ac2-8c76-b3d62a99eb0e\") " pod="openshift-lws-operator/lws-controller-manager-65f8ccd9c5-8tww2" Jul 07 11:21:21.654031 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:21:21.653918 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5v96s\" (UniqueName: \"kubernetes.io/projected/3431aa7e-d46e-4ac2-8c76-b3d62a99eb0e-kube-api-access-5v96s\") pod \"lws-controller-manager-65f8ccd9c5-8tww2\" (UID: \"3431aa7e-d46e-4ac2-8c76-b3d62a99eb0e\") " pod="openshift-lws-operator/lws-controller-manager-65f8ccd9c5-8tww2" Jul 07 11:21:21.654031 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:21:21.653939 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/3431aa7e-d46e-4ac2-8c76-b3d62a99eb0e-manager-config\") pod \"lws-controller-manager-65f8ccd9c5-8tww2\" (UID: \"3431aa7e-d46e-4ac2-8c76-b3d62a99eb0e\") " pod="openshift-lws-operator/lws-controller-manager-65f8ccd9c5-8tww2" Jul 07 11:21:21.671746 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:21:21.670240 2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 07 11:21:21.671746 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:21:21.670342 2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 07 11:21:21.671746 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:21:21.670394 2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 07 11:21:21.757033 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:21:21.756997 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/3431aa7e-d46e-4ac2-8c76-b3d62a99eb0e-cert\") pod \"lws-controller-manager-65f8ccd9c5-8tww2\" (UID: \"3431aa7e-d46e-4ac2-8c76-b3d62a99eb0e\") " pod="openshift-lws-operator/lws-controller-manager-65f8ccd9c5-8tww2" Jul 07 11:21:21.757515 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:21:21.757046 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5v96s\" (UniqueName: \"kubernetes.io/projected/3431aa7e-d46e-4ac2-8c76-b3d62a99eb0e-kube-api-access-5v96s\") pod \"lws-controller-manager-65f8ccd9c5-8tww2\" (UID: \"3431aa7e-d46e-4ac2-8c76-b3d62a99eb0e\") " pod="openshift-lws-operator/lws-controller-manager-65f8ccd9c5-8tww2" Jul 07 11:21:21.757515 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:21:21.757080 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/3431aa7e-d46e-4ac2-8c76-b3d62a99eb0e-manager-config\") pod \"lws-controller-manager-65f8ccd9c5-8tww2\" (UID: \"3431aa7e-d46e-4ac2-8c76-b3d62a99eb0e\") " pod="openshift-lws-operator/lws-controller-manager-65f8ccd9c5-8tww2" Jul 07 11:21:21.757515 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:21:21.757147 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/3431aa7e-d46e-4ac2-8c76-b3d62a99eb0e-metrics-cert\") pod \"lws-controller-manager-65f8ccd9c5-8tww2\" (UID: \"3431aa7e-d46e-4ac2-8c76-b3d62a99eb0e\") " pod="openshift-lws-operator/lws-controller-manager-65f8ccd9c5-8tww2" Jul 07 11:21:21.758532 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:21:21.758495 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/3431aa7e-d46e-4ac2-8c76-b3d62a99eb0e-manager-config\") pod \"lws-controller-manager-65f8ccd9c5-8tww2\" (UID: \"3431aa7e-d46e-4ac2-8c76-b3d62a99eb0e\") " pod="openshift-lws-operator/lws-controller-manager-65f8ccd9c5-8tww2" Jul 07 11:21:21.760112 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:21:21.760051 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/3431aa7e-d46e-4ac2-8c76-b3d62a99eb0e-metrics-cert\") pod \"lws-controller-manager-65f8ccd9c5-8tww2\" (UID: \"3431aa7e-d46e-4ac2-8c76-b3d62a99eb0e\") " pod="openshift-lws-operator/lws-controller-manager-65f8ccd9c5-8tww2" Jul 07 11:21:21.760283 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:21:21.760261 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/3431aa7e-d46e-4ac2-8c76-b3d62a99eb0e-cert\") pod \"lws-controller-manager-65f8ccd9c5-8tww2\" (UID: \"3431aa7e-d46e-4ac2-8c76-b3d62a99eb0e\") " pod="openshift-lws-operator/lws-controller-manager-65f8ccd9c5-8tww2" Jul 07 11:21:21.766783 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:21:21.766757 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-5v96s\" (UniqueName: \"kubernetes.io/projected/3431aa7e-d46e-4ac2-8c76-b3d62a99eb0e-kube-api-access-5v96s\") pod \"lws-controller-manager-65f8ccd9c5-8tww2\" (UID: \"3431aa7e-d46e-4ac2-8c76-b3d62a99eb0e\") " pod="openshift-lws-operator/lws-controller-manager-65f8ccd9c5-8tww2" Jul 07 11:21:21.874953 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:21:21.874790 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-65f8ccd9c5-8tww2" Jul 07 11:21:22.042811 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:21:22.042767 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-65f8ccd9c5-8tww2"] Jul 07 11:21:22.046982 ip-10-0-142-112 kubenswrapper[2578]: W0707 11:21:22.046953 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3431aa7e_d46e_4ac2_8c76_b3d62a99eb0e.slice/crio-258b3d6995e771076b88ba0e4ce2d0c4e11606dd5d9410d5f3e285a8db4b1076 WatchSource:0}: Error finding container 258b3d6995e771076b88ba0e4ce2d0c4e11606dd5d9410d5f3e285a8db4b1076: Status 404 returned error can't find the container with id 258b3d6995e771076b88ba0e4ce2d0c4e11606dd5d9410d5f3e285a8db4b1076 Jul 07 11:21:22.693578 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:21:22.693542 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-65f8ccd9c5-8tww2" event={"ID":"3431aa7e-d46e-4ac2-8c76-b3d62a99eb0e","Type":"ContainerStarted","Data":"258b3d6995e771076b88ba0e4ce2d0c4e11606dd5d9410d5f3e285a8db4b1076"} Jul 07 11:21:22.695257 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:21:22.695214 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4q24nh" event={"ID":"5c24eef8-4fc8-4973-a25b-4e93f4badb41","Type":"ContainerStarted","Data":"aaad2888b345f2bf4d1b164b0d7d1c01760749da10b0f7c77d69a414359ea25f"} Jul 07 11:21:22.735937 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:21:22.735882 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4q24nh" podStartSLOduration=2.184923269 podStartE2EDuration="4.735840631s" totalImagesPullingTime="2.550917362s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 11:21:18 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 11:21:22.733870368 +0000 UTC m=+327.030019956" watchObservedRunningTime="2026-07-07 11:21:22.735840631 +0000 UTC m=+327.031990221" Jul 07 11:21:22.978412 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:21:22.978325 2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4q24nh" Jul 07 11:21:22.984075 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:21:22.984050 2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4q24nh" Jul 07 11:21:23.458438 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:21:23.458254 2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 07 11:21:23.458438 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:21:23.458332 2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 07 11:21:23.707914 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:21:23.707817 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-qmvqq" event={"ID":"8111ef68-a869-479b-9087-2e5ed23349c0","Type":"ContainerStarted","Data":"3dd52778e6abe838d5e9a3036211525ae47c9b890155e979919ccee95a213d08"} Jul 07 11:21:23.707914 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:21:23.707880 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4q24nh" Jul 07 11:21:23.708126 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:21:23.707922 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-qmvqq" Jul 07 11:21:23.709447 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:21:23.709409 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4q24nh" Jul 07 11:21:23.735744 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:21:23.734691 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-qmvqq" podStartSLOduration=1.720654315 podStartE2EDuration="4.73466614s" totalImagesPullingTime="3.014011825s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 11:21:19 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 11:21:23.732844213 +0000 UTC m=+328.028993801" watchObservedRunningTime="2026-07-07 11:21:23.73466614 +0000 UTC m=+328.030815729" Jul 07 11:21:24.713267 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:21:24.713239 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-qmvqq" Jul 07 11:21:25.716707 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:21:25.716615 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-65f8ccd9c5-8tww2" event={"ID":"3431aa7e-d46e-4ac2-8c76-b3d62a99eb0e","Type":"ContainerStarted","Data":"65d582d547e766ef44b2f4636d5b7fb6ca20704fd063caf89e6c908533803183"} Jul 07 11:21:25.717159 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:21:25.717047 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-lws-operator/lws-controller-manager-65f8ccd9c5-8tww2" Jul 07 11:21:25.736823 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:21:25.736766 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-lws-operator/lws-controller-manager-65f8ccd9c5-8tww2" podStartSLOduration=1.439774648 podStartE2EDuration="4.736746364s" totalImagesPullingTime="3.296971716s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 11:21:21 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 11:21:25.736465039 +0000 UTC m=+330.032614630" watchObservedRunningTime="2026-07-07 11:21:25.736746364 +0000 UTC m=+330.032895955" Jul 07 11:21:36.722757 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:21:36.722724 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-lws-operator/lws-controller-manager-65f8ccd9c5-8tww2" Jul 07 11:22:53.988232 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:22:53.988201 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-6694655d5d-bwrbf"] Jul 07 11:22:53.991800 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:22:53.991776 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6694655d5d-bwrbf" Jul 07 11:22:54.005777 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:22:54.005753 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-6694655d5d-bwrbf"] Jul 07 11:22:54.070722 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:22:54.070685 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/d6fdfe74-5cde-4df0-8203-ef253eae64cf-oauth-serving-cert\") pod \"console-6694655d5d-bwrbf\" (UID: \"d6fdfe74-5cde-4df0-8203-ef253eae64cf\") " pod="openshift-console/console-6694655d5d-bwrbf" Jul 07 11:22:54.070881 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:22:54.070739 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/d6fdfe74-5cde-4df0-8203-ef253eae64cf-service-ca\") pod \"console-6694655d5d-bwrbf\" (UID: \"d6fdfe74-5cde-4df0-8203-ef253eae64cf\") " pod="openshift-console/console-6694655d5d-bwrbf" Jul 07 11:22:54.070881 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:22:54.070760 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d6fdfe74-5cde-4df0-8203-ef253eae64cf-trusted-ca-bundle\") pod \"console-6694655d5d-bwrbf\" (UID: \"d6fdfe74-5cde-4df0-8203-ef253eae64cf\") " pod="openshift-console/console-6694655d5d-bwrbf" Jul 07 11:22:54.070881 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:22:54.070785 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/d6fdfe74-5cde-4df0-8203-ef253eae64cf-console-serving-cert\") pod \"console-6694655d5d-bwrbf\" (UID: \"d6fdfe74-5cde-4df0-8203-ef253eae64cf\") " pod="openshift-console/console-6694655d5d-bwrbf" Jul 07 11:22:54.070881 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:22:54.070807 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/d6fdfe74-5cde-4df0-8203-ef253eae64cf-console-config\") pod \"console-6694655d5d-bwrbf\" (UID: \"d6fdfe74-5cde-4df0-8203-ef253eae64cf\") " pod="openshift-console/console-6694655d5d-bwrbf" Jul 07 11:22:54.070881 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:22:54.070865 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/d6fdfe74-5cde-4df0-8203-ef253eae64cf-console-oauth-config\") pod \"console-6694655d5d-bwrbf\" (UID: \"d6fdfe74-5cde-4df0-8203-ef253eae64cf\") " pod="openshift-console/console-6694655d5d-bwrbf" Jul 07 11:22:54.071100 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:22:54.070884 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-n6mwj\" (UniqueName: \"kubernetes.io/projected/d6fdfe74-5cde-4df0-8203-ef253eae64cf-kube-api-access-n6mwj\") pod \"console-6694655d5d-bwrbf\" (UID: \"d6fdfe74-5cde-4df0-8203-ef253eae64cf\") " pod="openshift-console/console-6694655d5d-bwrbf" Jul 07 11:22:54.171592 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:22:54.171550 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/d6fdfe74-5cde-4df0-8203-ef253eae64cf-console-oauth-config\") pod \"console-6694655d5d-bwrbf\" (UID: \"d6fdfe74-5cde-4df0-8203-ef253eae64cf\") " pod="openshift-console/console-6694655d5d-bwrbf" Jul 07 11:22:54.171793 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:22:54.171602 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-n6mwj\" (UniqueName: \"kubernetes.io/projected/d6fdfe74-5cde-4df0-8203-ef253eae64cf-kube-api-access-n6mwj\") pod \"console-6694655d5d-bwrbf\" (UID: \"d6fdfe74-5cde-4df0-8203-ef253eae64cf\") " pod="openshift-console/console-6694655d5d-bwrbf" Jul 07 11:22:54.171793 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:22:54.171629 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/d6fdfe74-5cde-4df0-8203-ef253eae64cf-oauth-serving-cert\") pod \"console-6694655d5d-bwrbf\" (UID: \"d6fdfe74-5cde-4df0-8203-ef253eae64cf\") " pod="openshift-console/console-6694655d5d-bwrbf" Jul 07 11:22:54.171793 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:22:54.171780 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/d6fdfe74-5cde-4df0-8203-ef253eae64cf-service-ca\") pod \"console-6694655d5d-bwrbf\" (UID: \"d6fdfe74-5cde-4df0-8203-ef253eae64cf\") " pod="openshift-console/console-6694655d5d-bwrbf" Jul 07 11:22:54.171957 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:22:54.171798 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d6fdfe74-5cde-4df0-8203-ef253eae64cf-trusted-ca-bundle\") pod \"console-6694655d5d-bwrbf\" (UID: \"d6fdfe74-5cde-4df0-8203-ef253eae64cf\") " pod="openshift-console/console-6694655d5d-bwrbf" Jul 07 11:22:54.171957 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:22:54.171839 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/d6fdfe74-5cde-4df0-8203-ef253eae64cf-console-serving-cert\") pod \"console-6694655d5d-bwrbf\" (UID: \"d6fdfe74-5cde-4df0-8203-ef253eae64cf\") " pod="openshift-console/console-6694655d5d-bwrbf" Jul 07 11:22:54.171957 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:22:54.171910 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/d6fdfe74-5cde-4df0-8203-ef253eae64cf-console-config\") pod \"console-6694655d5d-bwrbf\" (UID: \"d6fdfe74-5cde-4df0-8203-ef253eae64cf\") " pod="openshift-console/console-6694655d5d-bwrbf" Jul 07 11:22:54.172392 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:22:54.172367 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/d6fdfe74-5cde-4df0-8203-ef253eae64cf-oauth-serving-cert\") pod \"console-6694655d5d-bwrbf\" (UID: \"d6fdfe74-5cde-4df0-8203-ef253eae64cf\") " pod="openshift-console/console-6694655d5d-bwrbf" Jul 07 11:22:54.172553 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:22:54.172504 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/d6fdfe74-5cde-4df0-8203-ef253eae64cf-service-ca\") pod \"console-6694655d5d-bwrbf\" (UID: \"d6fdfe74-5cde-4df0-8203-ef253eae64cf\") " pod="openshift-console/console-6694655d5d-bwrbf" Jul 07 11:22:54.172653 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:22:54.172635 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/d6fdfe74-5cde-4df0-8203-ef253eae64cf-console-config\") pod \"console-6694655d5d-bwrbf\" (UID: \"d6fdfe74-5cde-4df0-8203-ef253eae64cf\") " pod="openshift-console/console-6694655d5d-bwrbf" Jul 07 11:22:54.172758 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:22:54.172740 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d6fdfe74-5cde-4df0-8203-ef253eae64cf-trusted-ca-bundle\") pod \"console-6694655d5d-bwrbf\" (UID: \"d6fdfe74-5cde-4df0-8203-ef253eae64cf\") " pod="openshift-console/console-6694655d5d-bwrbf" Jul 07 11:22:54.174059 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:22:54.174039 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/d6fdfe74-5cde-4df0-8203-ef253eae64cf-console-oauth-config\") pod \"console-6694655d5d-bwrbf\" (UID: \"d6fdfe74-5cde-4df0-8203-ef253eae64cf\") " pod="openshift-console/console-6694655d5d-bwrbf" Jul 07 11:22:54.174232 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:22:54.174217 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/d6fdfe74-5cde-4df0-8203-ef253eae64cf-console-serving-cert\") pod \"console-6694655d5d-bwrbf\" (UID: \"d6fdfe74-5cde-4df0-8203-ef253eae64cf\") " pod="openshift-console/console-6694655d5d-bwrbf" Jul 07 11:22:54.180949 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:22:54.180926 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-n6mwj\" (UniqueName: \"kubernetes.io/projected/d6fdfe74-5cde-4df0-8203-ef253eae64cf-kube-api-access-n6mwj\") pod \"console-6694655d5d-bwrbf\" (UID: \"d6fdfe74-5cde-4df0-8203-ef253eae64cf\") " pod="openshift-console/console-6694655d5d-bwrbf" Jul 07 11:22:54.300331 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:22:54.300302 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6694655d5d-bwrbf" Jul 07 11:22:54.432401 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:22:54.432377 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-6694655d5d-bwrbf"] Jul 07 11:22:54.435052 ip-10-0-142-112 kubenswrapper[2578]: W0707 11:22:54.435022 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd6fdfe74_5cde_4df0_8203_ef253eae64cf.slice/crio-de15eafbf4317d1f169fdfa4fb6cafa8a747fe140bcc6dbfd6187232b3203e85 WatchSource:0}: Error finding container de15eafbf4317d1f169fdfa4fb6cafa8a747fe140bcc6dbfd6187232b3203e85: Status 404 returned error can't find the container with id de15eafbf4317d1f169fdfa4fb6cafa8a747fe140bcc6dbfd6187232b3203e85 Jul 07 11:22:55.103756 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:22:55.103721 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6694655d5d-bwrbf" event={"ID":"d6fdfe74-5cde-4df0-8203-ef253eae64cf","Type":"ContainerStarted","Data":"c49accea1508f26ca853cdddc9e3a6cb23c62135595516b18d406f991f52d98b"} Jul 07 11:22:55.104204 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:22:55.103815 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6694655d5d-bwrbf" event={"ID":"d6fdfe74-5cde-4df0-8203-ef253eae64cf","Type":"ContainerStarted","Data":"de15eafbf4317d1f169fdfa4fb6cafa8a747fe140bcc6dbfd6187232b3203e85"} Jul 07 11:22:55.125442 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:22:55.125378 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-6694655d5d-bwrbf" podStartSLOduration=2.125362631 podStartE2EDuration="2.125362631s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 11:22:53 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 11:22:55.123587195 +0000 UTC m=+419.419736783" watchObservedRunningTime="2026-07-07 11:22:55.125362631 +0000 UTC m=+419.421512219" Jul 07 11:23:04.300896 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:23:04.300842 2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-6694655d5d-bwrbf" Jul 07 11:23:04.301486 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:23:04.300925 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-6694655d5d-bwrbf" Jul 07 11:23:04.306613 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:23:04.306591 2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-6694655d5d-bwrbf" Jul 07 11:23:05.148591 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:23:05.148560 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-6694655d5d-bwrbf" Jul 07 11:23:05.231860 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:23:05.231822 2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-7754b567bf-k222b"] Jul 07 11:23:30.251331 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:23:30.251224 2578 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-console/console-7754b567bf-k222b" podUID="d4726168-e989-42f1-88dc-4dc1c64c1dcc" containerName="console" containerID="cri-o://311b69e229244f5190de84738692de5b5069ede4cd9fc0701cfa0702887eb928" gracePeriod=15 Jul 07 11:23:30.496001 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:23:30.495976 2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7754b567bf-k222b" Jul 07 11:23:30.577352 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:23:30.577323 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d4726168-e989-42f1-88dc-4dc1c64c1dcc-trusted-ca-bundle\") pod \"d4726168-e989-42f1-88dc-4dc1c64c1dcc\" (UID: \"d4726168-e989-42f1-88dc-4dc1c64c1dcc\") " Jul 07 11:23:30.577573 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:23:30.577368 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/d4726168-e989-42f1-88dc-4dc1c64c1dcc-oauth-serving-cert\") pod \"d4726168-e989-42f1-88dc-4dc1c64c1dcc\" (UID: \"d4726168-e989-42f1-88dc-4dc1c64c1dcc\") " Jul 07 11:23:30.577573 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:23:30.577451 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-c62vf\" (UniqueName: \"kubernetes.io/projected/d4726168-e989-42f1-88dc-4dc1c64c1dcc-kube-api-access-c62vf\") pod \"d4726168-e989-42f1-88dc-4dc1c64c1dcc\" (UID: \"d4726168-e989-42f1-88dc-4dc1c64c1dcc\") " Jul 07 11:23:30.577573 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:23:30.577481 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/d4726168-e989-42f1-88dc-4dc1c64c1dcc-service-ca\") pod \"d4726168-e989-42f1-88dc-4dc1c64c1dcc\" (UID: \"d4726168-e989-42f1-88dc-4dc1c64c1dcc\") " Jul 07 11:23:30.577573 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:23:30.577496 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/d4726168-e989-42f1-88dc-4dc1c64c1dcc-console-config\") pod \"d4726168-e989-42f1-88dc-4dc1c64c1dcc\" (UID: \"d4726168-e989-42f1-88dc-4dc1c64c1dcc\") " Jul 07 11:23:30.577795 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:23:30.577629 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/d4726168-e989-42f1-88dc-4dc1c64c1dcc-console-serving-cert\") pod \"d4726168-e989-42f1-88dc-4dc1c64c1dcc\" (UID: \"d4726168-e989-42f1-88dc-4dc1c64c1dcc\") " Jul 07 11:23:30.577795 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:23:30.577721 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/d4726168-e989-42f1-88dc-4dc1c64c1dcc-console-oauth-config\") pod \"d4726168-e989-42f1-88dc-4dc1c64c1dcc\" (UID: \"d4726168-e989-42f1-88dc-4dc1c64c1dcc\") " Jul 07 11:23:30.577896 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:23:30.577845 2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d4726168-e989-42f1-88dc-4dc1c64c1dcc-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "d4726168-e989-42f1-88dc-4dc1c64c1dcc" (UID: "d4726168-e989-42f1-88dc-4dc1c64c1dcc"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 07 11:23:30.577896 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:23:30.577853 2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d4726168-e989-42f1-88dc-4dc1c64c1dcc-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "d4726168-e989-42f1-88dc-4dc1c64c1dcc" (UID: "d4726168-e989-42f1-88dc-4dc1c64c1dcc"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 07 11:23:30.577994 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:23:30.577889 2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d4726168-e989-42f1-88dc-4dc1c64c1dcc-service-ca" (OuterVolumeSpecName: "service-ca") pod "d4726168-e989-42f1-88dc-4dc1c64c1dcc" (UID: "d4726168-e989-42f1-88dc-4dc1c64c1dcc"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 07 11:23:30.578183 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:23:30.578158 2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d4726168-e989-42f1-88dc-4dc1c64c1dcc-console-config" (OuterVolumeSpecName: "console-config") pod "d4726168-e989-42f1-88dc-4dc1c64c1dcc" (UID: "d4726168-e989-42f1-88dc-4dc1c64c1dcc"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 07 11:23:30.578183 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:23:30.578177 2578 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/d4726168-e989-42f1-88dc-4dc1c64c1dcc-oauth-serving-cert\") on node \"ip-10-0-142-112.ec2.internal\" DevicePath \"\"" Jul 07 11:23:30.578354 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:23:30.578196 2578 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/d4726168-e989-42f1-88dc-4dc1c64c1dcc-service-ca\") on node \"ip-10-0-142-112.ec2.internal\" DevicePath \"\"" Jul 07 11:23:30.578354 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:23:30.578206 2578 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d4726168-e989-42f1-88dc-4dc1c64c1dcc-trusted-ca-bundle\") on node \"ip-10-0-142-112.ec2.internal\" DevicePath \"\"" Jul 07 11:23:30.579690 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:23:30.579665 2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d4726168-e989-42f1-88dc-4dc1c64c1dcc-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "d4726168-e989-42f1-88dc-4dc1c64c1dcc" (UID: "d4726168-e989-42f1-88dc-4dc1c64c1dcc"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 07 11:23:30.579766 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:23:30.579688 2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d4726168-e989-42f1-88dc-4dc1c64c1dcc-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "d4726168-e989-42f1-88dc-4dc1c64c1dcc" (UID: "d4726168-e989-42f1-88dc-4dc1c64c1dcc"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 07 11:23:30.579766 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:23:30.579722 2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d4726168-e989-42f1-88dc-4dc1c64c1dcc-kube-api-access-c62vf" (OuterVolumeSpecName: "kube-api-access-c62vf") pod "d4726168-e989-42f1-88dc-4dc1c64c1dcc" (UID: "d4726168-e989-42f1-88dc-4dc1c64c1dcc"). InnerVolumeSpecName "kube-api-access-c62vf". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 07 11:23:30.678808 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:23:30.678775 2578 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/d4726168-e989-42f1-88dc-4dc1c64c1dcc-console-serving-cert\") on node \"ip-10-0-142-112.ec2.internal\" DevicePath \"\"" Jul 07 11:23:30.678808 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:23:30.678802 2578 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/d4726168-e989-42f1-88dc-4dc1c64c1dcc-console-oauth-config\") on node \"ip-10-0-142-112.ec2.internal\" DevicePath \"\"" Jul 07 11:23:30.678808 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:23:30.678813 2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-c62vf\" (UniqueName: \"kubernetes.io/projected/d4726168-e989-42f1-88dc-4dc1c64c1dcc-kube-api-access-c62vf\") on node \"ip-10-0-142-112.ec2.internal\" DevicePath \"\"" Jul 07 11:23:30.679035 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:23:30.678825 2578 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/d4726168-e989-42f1-88dc-4dc1c64c1dcc-console-config\") on node \"ip-10-0-142-112.ec2.internal\" DevicePath \"\"" Jul 07 11:23:31.253449 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:23:31.253395 2578 generic.go:356] "Generic (PLEG): container finished" podID="d4726168-e989-42f1-88dc-4dc1c64c1dcc" containerID="311b69e229244f5190de84738692de5b5069ede4cd9fc0701cfa0702887eb928" exitCode=0 Jul 07 11:23:31.253864 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:23:31.253456 2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7754b567bf-k222b" Jul 07 11:23:31.253864 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:23:31.253487 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7754b567bf-k222b" event={"ID":"d4726168-e989-42f1-88dc-4dc1c64c1dcc","Type":"ContainerDied","Data":"311b69e229244f5190de84738692de5b5069ede4cd9fc0701cfa0702887eb928"} Jul 07 11:23:31.253864 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:23:31.253538 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7754b567bf-k222b" event={"ID":"d4726168-e989-42f1-88dc-4dc1c64c1dcc","Type":"ContainerDied","Data":"1c9e3da57c4df37bed9f4df88998c03674e4c1c0e709406da9f15c918973ed64"} Jul 07 11:23:31.253864 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:23:31.253559 2578 scope.go:117] "RemoveContainer" containerID="311b69e229244f5190de84738692de5b5069ede4cd9fc0701cfa0702887eb928" Jul 07 11:23:31.262188 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:23:31.262173 2578 scope.go:117] "RemoveContainer" containerID="311b69e229244f5190de84738692de5b5069ede4cd9fc0701cfa0702887eb928" Jul 07 11:23:31.262484 ip-10-0-142-112 kubenswrapper[2578]: E0707 11:23:31.262462 2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"311b69e229244f5190de84738692de5b5069ede4cd9fc0701cfa0702887eb928\": container with ID starting with 311b69e229244f5190de84738692de5b5069ede4cd9fc0701cfa0702887eb928 not found: ID does not exist" containerID="311b69e229244f5190de84738692de5b5069ede4cd9fc0701cfa0702887eb928" Jul 07 11:23:31.262544 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:23:31.262493 2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"311b69e229244f5190de84738692de5b5069ede4cd9fc0701cfa0702887eb928"} err="failed to get container status \"311b69e229244f5190de84738692de5b5069ede4cd9fc0701cfa0702887eb928\": rpc error: code = NotFound desc = could not find container \"311b69e229244f5190de84738692de5b5069ede4cd9fc0701cfa0702887eb928\": container with ID starting with 311b69e229244f5190de84738692de5b5069ede4cd9fc0701cfa0702887eb928 not found: ID does not exist" Jul 07 11:23:31.279979 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:23:31.279952 2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-7754b567bf-k222b"] Jul 07 11:23:31.282263 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:23:31.282238 2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-7754b567bf-k222b"] Jul 07 11:23:32.306995 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:23:32.306965 2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d4726168-e989-42f1-88dc-4dc1c64c1dcc" path="/var/lib/kubelet/pods/d4726168-e989-42f1-88dc-4dc1c64c1dcc/volumes" Jul 07 11:26:59.490021 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:26:59.489988 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-74b68447df-flr9c"] Jul 07 11:26:59.490487 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:26:59.490349 2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d4726168-e989-42f1-88dc-4dc1c64c1dcc" containerName="console" Jul 07 11:26:59.490487 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:26:59.490361 2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="d4726168-e989-42f1-88dc-4dc1c64c1dcc" containerName="console" Jul 07 11:26:59.490487 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:26:59.490441 2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="d4726168-e989-42f1-88dc-4dc1c64c1dcc" containerName="console" Jul 07 11:26:59.493267 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:26:59.493248 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-flr9c" Jul 07 11:26:59.495634 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:26:59.495614 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"kube-root-ca.crt\"" Jul 07 11:26:59.495721 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:26:59.495617 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"openshift-service-ca.crt\"" Jul 07 11:26:59.496402 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:26:59.496383 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"default-dockercfg-5gs4g\"" Jul 07 11:26:59.496487 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:26:59.496409 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"facebook-opt-125m-simulated-kserve-self-signed-certs\"" Jul 07 11:26:59.504620 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:26:59.504370 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-74b68447df-flr9c"] Jul 07 11:26:59.607246 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:26:59.607208 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/fb1c7b72-0552-414c-9e00-adc7008c5f9e-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-flr9c\" (UID: \"fb1c7b72-0552-414c-9e00-adc7008c5f9e\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-flr9c" Jul 07 11:26:59.607246 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:26:59.607252 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4nvqs\" (UniqueName: \"kubernetes.io/projected/fb1c7b72-0552-414c-9e00-adc7008c5f9e-kube-api-access-4nvqs\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-flr9c\" (UID: \"fb1c7b72-0552-414c-9e00-adc7008c5f9e\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-flr9c" Jul 07 11:26:59.607500 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:26:59.607293 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/fb1c7b72-0552-414c-9e00-adc7008c5f9e-model-cache\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-flr9c\" (UID: \"fb1c7b72-0552-414c-9e00-adc7008c5f9e\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-flr9c" Jul 07 11:26:59.607500 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:26:59.607325 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/fb1c7b72-0552-414c-9e00-adc7008c5f9e-home\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-flr9c\" (UID: \"fb1c7b72-0552-414c-9e00-adc7008c5f9e\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-flr9c" Jul 07 11:26:59.607500 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:26:59.607339 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/fb1c7b72-0552-414c-9e00-adc7008c5f9e-tmp-dir\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-flr9c\" (UID: \"fb1c7b72-0552-414c-9e00-adc7008c5f9e\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-flr9c" Jul 07 11:26:59.607500 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:26:59.607398 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/fb1c7b72-0552-414c-9e00-adc7008c5f9e-dshm\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-flr9c\" (UID: \"fb1c7b72-0552-414c-9e00-adc7008c5f9e\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-flr9c" Jul 07 11:26:59.708896 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:26:59.708856 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/fb1c7b72-0552-414c-9e00-adc7008c5f9e-model-cache\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-flr9c\" (UID: \"fb1c7b72-0552-414c-9e00-adc7008c5f9e\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-flr9c" Jul 07 11:26:59.709073 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:26:59.708913 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/fb1c7b72-0552-414c-9e00-adc7008c5f9e-home\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-flr9c\" (UID: \"fb1c7b72-0552-414c-9e00-adc7008c5f9e\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-flr9c" Jul 07 11:26:59.709073 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:26:59.708934 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/fb1c7b72-0552-414c-9e00-adc7008c5f9e-tmp-dir\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-flr9c\" (UID: \"fb1c7b72-0552-414c-9e00-adc7008c5f9e\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-flr9c" Jul 07 11:26:59.709073 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:26:59.708974 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/fb1c7b72-0552-414c-9e00-adc7008c5f9e-dshm\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-flr9c\" (UID: \"fb1c7b72-0552-414c-9e00-adc7008c5f9e\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-flr9c" Jul 07 11:26:59.709205 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:26:59.709119 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/fb1c7b72-0552-414c-9e00-adc7008c5f9e-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-flr9c\" (UID: \"fb1c7b72-0552-414c-9e00-adc7008c5f9e\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-flr9c" Jul 07 11:26:59.709205 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:26:59.709171 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4nvqs\" (UniqueName: \"kubernetes.io/projected/fb1c7b72-0552-414c-9e00-adc7008c5f9e-kube-api-access-4nvqs\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-flr9c\" (UID: \"fb1c7b72-0552-414c-9e00-adc7008c5f9e\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-flr9c" Jul 07 11:26:59.709318 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:26:59.709294 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/fb1c7b72-0552-414c-9e00-adc7008c5f9e-model-cache\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-flr9c\" (UID: \"fb1c7b72-0552-414c-9e00-adc7008c5f9e\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-flr9c" Jul 07 11:26:59.709398 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:26:59.709367 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/fb1c7b72-0552-414c-9e00-adc7008c5f9e-home\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-flr9c\" (UID: \"fb1c7b72-0552-414c-9e00-adc7008c5f9e\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-flr9c" Jul 07 11:26:59.709535 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:26:59.709398 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/fb1c7b72-0552-414c-9e00-adc7008c5f9e-tmp-dir\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-flr9c\" (UID: \"fb1c7b72-0552-414c-9e00-adc7008c5f9e\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-flr9c" Jul 07 11:26:59.711307 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:26:59.711285 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/fb1c7b72-0552-414c-9e00-adc7008c5f9e-dshm\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-flr9c\" (UID: \"fb1c7b72-0552-414c-9e00-adc7008c5f9e\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-flr9c" Jul 07 11:26:59.711494 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:26:59.711476 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/fb1c7b72-0552-414c-9e00-adc7008c5f9e-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-flr9c\" (UID: \"fb1c7b72-0552-414c-9e00-adc7008c5f9e\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-flr9c" Jul 07 11:26:59.718218 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:26:59.718195 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-4nvqs\" (UniqueName: \"kubernetes.io/projected/fb1c7b72-0552-414c-9e00-adc7008c5f9e-kube-api-access-4nvqs\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-flr9c\" (UID: \"fb1c7b72-0552-414c-9e00-adc7008c5f9e\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-flr9c" Jul 07 11:26:59.804361 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:26:59.804328 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-flr9c" Jul 07 11:26:59.934908 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:26:59.934876 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-74b68447df-flr9c"] Jul 07 11:26:59.937919 ip-10-0-142-112 kubenswrapper[2578]: W0707 11:26:59.937881 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfb1c7b72_0552_414c_9e00_adc7008c5f9e.slice/crio-ab0a30929bbc4e8cc6689ba72f840f9ad30ed54fd05a7aa1b17b3e5bab67f732 WatchSource:0}: Error finding container ab0a30929bbc4e8cc6689ba72f840f9ad30ed54fd05a7aa1b17b3e5bab67f732: Status 404 returned error can't find the container with id ab0a30929bbc4e8cc6689ba72f840f9ad30ed54fd05a7aa1b17b3e5bab67f732 Jul 07 11:26:59.939086 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:26:59.939071 2578 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 07 11:27:00.118415 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:27:00.118323 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-flr9c" event={"ID":"fb1c7b72-0552-414c-9e00-adc7008c5f9e","Type":"ContainerStarted","Data":"ab0a30929bbc4e8cc6689ba72f840f9ad30ed54fd05a7aa1b17b3e5bab67f732"} Jul 07 11:27:01.124757 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:27:01.124721 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-flr9c" event={"ID":"fb1c7b72-0552-414c-9e00-adc7008c5f9e","Type":"ContainerStarted","Data":"fffc5240702a5a345086ee382ec3addd99ecef7452ef4016f6f3e74b0b136f0d"} Jul 07 11:27:01.150512 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:27:01.150461 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-flr9c" podStartSLOduration=1.119267698 podStartE2EDuration="2.150444385s" totalImagesPullingTime="1.031176687s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 11:26:59 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 11:27:01.149547402 +0000 UTC m=+665.445696990" watchObservedRunningTime="2026-07-07 11:27:01.150444385 +0000 UTC m=+665.446593969" Jul 07 11:27:09.804613 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:27:09.804575 2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-flr9c" Jul 07 11:27:09.805030 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:27:09.804637 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-flr9c" Jul 07 11:27:09.817153 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:27:09.817129 2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-flr9c" Jul 07 11:27:10.174974 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:27:10.174893 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-flr9c" Jul 07 11:41:51.450907 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:41:51.450874 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-ait-d21d182b-openshift-default-7bf5768659-2x9r6"] Jul 07 11:41:51.454146 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:41:51.454130 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-d21d182b-openshift-default-7bf5768659-2x9r6" Jul 07 11:41:51.457342 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:41:51.457326 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-ait-d21d182b-openshift-default-dockercfg-gkr9n\"" Jul 07 11:41:51.492836 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:41:51.492807 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-d21d182b-openshift-default-7bf5768659-2x9r6"] Jul 07 11:41:51.581087 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:41:51.581044 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/eff99700-dbc1-4101-a81c-05232bf02875-istio-podinfo\") pod \"e2e-ait-d21d182b-openshift-default-7bf5768659-2x9r6\" (UID: \"eff99700-dbc1-4101-a81c-05232bf02875\") " pod="openshift-ingress/e2e-ait-d21d182b-openshift-default-7bf5768659-2x9r6" Jul 07 11:41:51.581087 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:41:51.581092 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/eff99700-dbc1-4101-a81c-05232bf02875-workload-certs\") pod \"e2e-ait-d21d182b-openshift-default-7bf5768659-2x9r6\" (UID: \"eff99700-dbc1-4101-a81c-05232bf02875\") " pod="openshift-ingress/e2e-ait-d21d182b-openshift-default-7bf5768659-2x9r6" Jul 07 11:41:51.581283 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:41:51.581161 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/eff99700-dbc1-4101-a81c-05232bf02875-istio-token\") pod \"e2e-ait-d21d182b-openshift-default-7bf5768659-2x9r6\" (UID: \"eff99700-dbc1-4101-a81c-05232bf02875\") " pod="openshift-ingress/e2e-ait-d21d182b-openshift-default-7bf5768659-2x9r6" Jul 07 11:41:51.581283 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:41:51.581180 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/eff99700-dbc1-4101-a81c-05232bf02875-credential-socket\") pod \"e2e-ait-d21d182b-openshift-default-7bf5768659-2x9r6\" (UID: \"eff99700-dbc1-4101-a81c-05232bf02875\") " pod="openshift-ingress/e2e-ait-d21d182b-openshift-default-7bf5768659-2x9r6" Jul 07 11:41:51.581283 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:41:51.581218 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/eff99700-dbc1-4101-a81c-05232bf02875-istio-data\") pod \"e2e-ait-d21d182b-openshift-default-7bf5768659-2x9r6\" (UID: \"eff99700-dbc1-4101-a81c-05232bf02875\") " pod="openshift-ingress/e2e-ait-d21d182b-openshift-default-7bf5768659-2x9r6" Jul 07 11:41:51.581283 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:41:51.581234 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ckx95\" (UniqueName: \"kubernetes.io/projected/eff99700-dbc1-4101-a81c-05232bf02875-kube-api-access-ckx95\") pod \"e2e-ait-d21d182b-openshift-default-7bf5768659-2x9r6\" (UID: \"eff99700-dbc1-4101-a81c-05232bf02875\") " pod="openshift-ingress/e2e-ait-d21d182b-openshift-default-7bf5768659-2x9r6" Jul 07 11:41:51.581405 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:41:51.581300 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/eff99700-dbc1-4101-a81c-05232bf02875-workload-socket\") pod \"e2e-ait-d21d182b-openshift-default-7bf5768659-2x9r6\" (UID: \"eff99700-dbc1-4101-a81c-05232bf02875\") " pod="openshift-ingress/e2e-ait-d21d182b-openshift-default-7bf5768659-2x9r6" Jul 07 11:41:51.581405 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:41:51.581331 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/eff99700-dbc1-4101-a81c-05232bf02875-istio-envoy\") pod \"e2e-ait-d21d182b-openshift-default-7bf5768659-2x9r6\" (UID: \"eff99700-dbc1-4101-a81c-05232bf02875\") " pod="openshift-ingress/e2e-ait-d21d182b-openshift-default-7bf5768659-2x9r6" Jul 07 11:41:51.581405 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:41:51.581352 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/eff99700-dbc1-4101-a81c-05232bf02875-istiod-ca-cert\") pod \"e2e-ait-d21d182b-openshift-default-7bf5768659-2x9r6\" (UID: \"eff99700-dbc1-4101-a81c-05232bf02875\") " pod="openshift-ingress/e2e-ait-d21d182b-openshift-default-7bf5768659-2x9r6" Jul 07 11:41:51.682052 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:41:51.681955 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/eff99700-dbc1-4101-a81c-05232bf02875-workload-socket\") pod \"e2e-ait-d21d182b-openshift-default-7bf5768659-2x9r6\" (UID: \"eff99700-dbc1-4101-a81c-05232bf02875\") " pod="openshift-ingress/e2e-ait-d21d182b-openshift-default-7bf5768659-2x9r6" Jul 07 11:41:51.682052 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:41:51.682001 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/eff99700-dbc1-4101-a81c-05232bf02875-istio-envoy\") pod \"e2e-ait-d21d182b-openshift-default-7bf5768659-2x9r6\" (UID: \"eff99700-dbc1-4101-a81c-05232bf02875\") " pod="openshift-ingress/e2e-ait-d21d182b-openshift-default-7bf5768659-2x9r6" Jul 07 11:41:51.682052 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:41:51.682022 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/eff99700-dbc1-4101-a81c-05232bf02875-istiod-ca-cert\") pod \"e2e-ait-d21d182b-openshift-default-7bf5768659-2x9r6\" (UID: \"eff99700-dbc1-4101-a81c-05232bf02875\") " pod="openshift-ingress/e2e-ait-d21d182b-openshift-default-7bf5768659-2x9r6" Jul 07 11:41:51.682327 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:41:51.682060 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/eff99700-dbc1-4101-a81c-05232bf02875-istio-podinfo\") pod \"e2e-ait-d21d182b-openshift-default-7bf5768659-2x9r6\" (UID: \"eff99700-dbc1-4101-a81c-05232bf02875\") " pod="openshift-ingress/e2e-ait-d21d182b-openshift-default-7bf5768659-2x9r6" Jul 07 11:41:51.682327 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:41:51.682090 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/eff99700-dbc1-4101-a81c-05232bf02875-workload-certs\") pod \"e2e-ait-d21d182b-openshift-default-7bf5768659-2x9r6\" (UID: \"eff99700-dbc1-4101-a81c-05232bf02875\") " pod="openshift-ingress/e2e-ait-d21d182b-openshift-default-7bf5768659-2x9r6" Jul 07 11:41:51.682327 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:41:51.682148 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/eff99700-dbc1-4101-a81c-05232bf02875-istio-token\") pod \"e2e-ait-d21d182b-openshift-default-7bf5768659-2x9r6\" (UID: \"eff99700-dbc1-4101-a81c-05232bf02875\") " pod="openshift-ingress/e2e-ait-d21d182b-openshift-default-7bf5768659-2x9r6" Jul 07 11:41:51.682327 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:41:51.682174 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/eff99700-dbc1-4101-a81c-05232bf02875-credential-socket\") pod \"e2e-ait-d21d182b-openshift-default-7bf5768659-2x9r6\" (UID: \"eff99700-dbc1-4101-a81c-05232bf02875\") " pod="openshift-ingress/e2e-ait-d21d182b-openshift-default-7bf5768659-2x9r6" Jul 07 11:41:51.682327 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:41:51.682203 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/eff99700-dbc1-4101-a81c-05232bf02875-istio-data\") pod \"e2e-ait-d21d182b-openshift-default-7bf5768659-2x9r6\" (UID: \"eff99700-dbc1-4101-a81c-05232bf02875\") " pod="openshift-ingress/e2e-ait-d21d182b-openshift-default-7bf5768659-2x9r6" Jul 07 11:41:51.682327 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:41:51.682229 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ckx95\" (UniqueName: \"kubernetes.io/projected/eff99700-dbc1-4101-a81c-05232bf02875-kube-api-access-ckx95\") pod \"e2e-ait-d21d182b-openshift-default-7bf5768659-2x9r6\" (UID: \"eff99700-dbc1-4101-a81c-05232bf02875\") " pod="openshift-ingress/e2e-ait-d21d182b-openshift-default-7bf5768659-2x9r6" Jul 07 11:41:51.682614 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:41:51.682496 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/eff99700-dbc1-4101-a81c-05232bf02875-workload-socket\") pod \"e2e-ait-d21d182b-openshift-default-7bf5768659-2x9r6\" (UID: \"eff99700-dbc1-4101-a81c-05232bf02875\") " pod="openshift-ingress/e2e-ait-d21d182b-openshift-default-7bf5768659-2x9r6" Jul 07 11:41:51.682810 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:41:51.682784 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/eff99700-dbc1-4101-a81c-05232bf02875-workload-certs\") pod \"e2e-ait-d21d182b-openshift-default-7bf5768659-2x9r6\" (UID: \"eff99700-dbc1-4101-a81c-05232bf02875\") " pod="openshift-ingress/e2e-ait-d21d182b-openshift-default-7bf5768659-2x9r6" Jul 07 11:41:51.682909 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:41:51.682877 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/eff99700-dbc1-4101-a81c-05232bf02875-credential-socket\") pod \"e2e-ait-d21d182b-openshift-default-7bf5768659-2x9r6\" (UID: \"eff99700-dbc1-4101-a81c-05232bf02875\") " pod="openshift-ingress/e2e-ait-d21d182b-openshift-default-7bf5768659-2x9r6" Jul 07 11:41:51.682974 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:41:51.682953 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/eff99700-dbc1-4101-a81c-05232bf02875-istio-data\") pod \"e2e-ait-d21d182b-openshift-default-7bf5768659-2x9r6\" (UID: \"eff99700-dbc1-4101-a81c-05232bf02875\") " pod="openshift-ingress/e2e-ait-d21d182b-openshift-default-7bf5768659-2x9r6" Jul 07 11:41:51.683116 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:41:51.683099 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/eff99700-dbc1-4101-a81c-05232bf02875-istiod-ca-cert\") pod \"e2e-ait-d21d182b-openshift-default-7bf5768659-2x9r6\" (UID: \"eff99700-dbc1-4101-a81c-05232bf02875\") " pod="openshift-ingress/e2e-ait-d21d182b-openshift-default-7bf5768659-2x9r6" Jul 07 11:41:51.684332 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:41:51.684314 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/eff99700-dbc1-4101-a81c-05232bf02875-istio-envoy\") pod \"e2e-ait-d21d182b-openshift-default-7bf5768659-2x9r6\" (UID: \"eff99700-dbc1-4101-a81c-05232bf02875\") " pod="openshift-ingress/e2e-ait-d21d182b-openshift-default-7bf5768659-2x9r6" Jul 07 11:41:51.684586 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:41:51.684570 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/eff99700-dbc1-4101-a81c-05232bf02875-istio-podinfo\") pod \"e2e-ait-d21d182b-openshift-default-7bf5768659-2x9r6\" (UID: \"eff99700-dbc1-4101-a81c-05232bf02875\") " pod="openshift-ingress/e2e-ait-d21d182b-openshift-default-7bf5768659-2x9r6" Jul 07 11:41:51.692612 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:41:51.692594 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/eff99700-dbc1-4101-a81c-05232bf02875-istio-token\") pod \"e2e-ait-d21d182b-openshift-default-7bf5768659-2x9r6\" (UID: \"eff99700-dbc1-4101-a81c-05232bf02875\") " pod="openshift-ingress/e2e-ait-d21d182b-openshift-default-7bf5768659-2x9r6" Jul 07 11:41:51.693770 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:41:51.693753 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-ckx95\" (UniqueName: \"kubernetes.io/projected/eff99700-dbc1-4101-a81c-05232bf02875-kube-api-access-ckx95\") pod \"e2e-ait-d21d182b-openshift-default-7bf5768659-2x9r6\" (UID: \"eff99700-dbc1-4101-a81c-05232bf02875\") " pod="openshift-ingress/e2e-ait-d21d182b-openshift-default-7bf5768659-2x9r6" Jul 07 11:41:51.764467 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:41:51.764409 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-d21d182b-openshift-default-7bf5768659-2x9r6" Jul 07 11:41:51.909003 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:41:51.908977 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-d21d182b-openshift-default-7bf5768659-2x9r6"] Jul 07 11:41:51.911103 ip-10-0-142-112 kubenswrapper[2578]: W0707 11:41:51.911079 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podeff99700_dbc1_4101_a81c_05232bf02875.slice/crio-57920b82dc82e5078ded81b079142d7bf076fcce98c5f0c83183c2366e0e6847 WatchSource:0}: Error finding container 57920b82dc82e5078ded81b079142d7bf076fcce98c5f0c83183c2366e0e6847: Status 404 returned error can't find the container with id 57920b82dc82e5078ded81b079142d7bf076fcce98c5f0c83183c2366e0e6847 Jul 07 11:41:51.913095 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:41:51.913065 2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 07 11:41:51.913148 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:41:51.913125 2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 07 11:41:51.913183 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:41:51.913153 2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 07 11:41:52.912588 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:41:52.912550 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-d21d182b-openshift-default-7bf5768659-2x9r6" event={"ID":"eff99700-dbc1-4101-a81c-05232bf02875","Type":"ContainerStarted","Data":"6f3c58e79168caf8d026898d544874f6fc9291ad28bba1dabd5bbf82b3f5b2c8"} Jul 07 11:41:52.912987 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:41:52.912615 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-d21d182b-openshift-default-7bf5768659-2x9r6" event={"ID":"eff99700-dbc1-4101-a81c-05232bf02875","Type":"ContainerStarted","Data":"57920b82dc82e5078ded81b079142d7bf076fcce98c5f0c83183c2366e0e6847"} Jul 07 11:41:52.948441 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:41:52.943005 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-ait-d21d182b-openshift-default-7bf5768659-2x9r6" podStartSLOduration=1.942982642 podStartE2EDuration="1.942982642s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 11:41:51 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 11:41:52.941640594 +0000 UTC m=+1557.237790182" watchObservedRunningTime="2026-07-07 11:41:52.942982642 +0000 UTC m=+1557.239132231" Jul 07 11:41:53.764780 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:41:53.764739 2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ait-d21d182b-openshift-default-7bf5768659-2x9r6" Jul 07 11:41:53.769287 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:41:53.769263 2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ait-d21d182b-openshift-default-7bf5768659-2x9r6" Jul 07 11:41:53.916799 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:41:53.916766 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-d21d182b-openshift-default-7bf5768659-2x9r6" Jul 07 11:41:53.917747 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:41:53.917732 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ait-d21d182b-openshift-default-7bf5768659-2x9r6" Jul 07 11:41:58.049280 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:41:58.049241 2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-d21d182b-openshift-default-7bf5768659-2x9r6"] Jul 07 11:41:58.049723 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:41:58.049446 2578 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-ingress/e2e-ait-d21d182b-openshift-default-7bf5768659-2x9r6" podUID="eff99700-dbc1-4101-a81c-05232bf02875" containerName="istio-proxy" containerID="cri-o://6f3c58e79168caf8d026898d544874f6fc9291ad28bba1dabd5bbf82b3f5b2c8" gracePeriod=30 Jul 07 11:42:03.694981 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:42:03.694954 2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-d21d182b-openshift-default-7bf5768659-2x9r6" Jul 07 11:42:03.793723 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:42:03.793638 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/eff99700-dbc1-4101-a81c-05232bf02875-workload-socket\") pod \"eff99700-dbc1-4101-a81c-05232bf02875\" (UID: \"eff99700-dbc1-4101-a81c-05232bf02875\") " Jul 07 11:42:03.793723 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:42:03.793716 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/eff99700-dbc1-4101-a81c-05232bf02875-istio-podinfo\") pod \"eff99700-dbc1-4101-a81c-05232bf02875\" (UID: \"eff99700-dbc1-4101-a81c-05232bf02875\") " Jul 07 11:42:03.793936 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:42:03.793746 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/eff99700-dbc1-4101-a81c-05232bf02875-credential-socket\") pod \"eff99700-dbc1-4101-a81c-05232bf02875\" (UID: \"eff99700-dbc1-4101-a81c-05232bf02875\") " Jul 07 11:42:03.793936 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:42:03.793782 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/eff99700-dbc1-4101-a81c-05232bf02875-istiod-ca-cert\") pod \"eff99700-dbc1-4101-a81c-05232bf02875\" (UID: \"eff99700-dbc1-4101-a81c-05232bf02875\") " Jul 07 11:42:03.793936 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:42:03.793805 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/eff99700-dbc1-4101-a81c-05232bf02875-istio-envoy\") pod \"eff99700-dbc1-4101-a81c-05232bf02875\" (UID: \"eff99700-dbc1-4101-a81c-05232bf02875\") " Jul 07 11:42:03.793936 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:42:03.793832 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/eff99700-dbc1-4101-a81c-05232bf02875-istio-data\") pod \"eff99700-dbc1-4101-a81c-05232bf02875\" (UID: \"eff99700-dbc1-4101-a81c-05232bf02875\") " Jul 07 11:42:03.793936 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:42:03.793870 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/eff99700-dbc1-4101-a81c-05232bf02875-istio-token\") pod \"eff99700-dbc1-4101-a81c-05232bf02875\" (UID: \"eff99700-dbc1-4101-a81c-05232bf02875\") " Jul 07 11:42:03.793936 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:42:03.793909 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/eff99700-dbc1-4101-a81c-05232bf02875-workload-certs\") pod \"eff99700-dbc1-4101-a81c-05232bf02875\" (UID: \"eff99700-dbc1-4101-a81c-05232bf02875\") " Jul 07 11:42:03.794187 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:42:03.793944 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-ckx95\" (UniqueName: \"kubernetes.io/projected/eff99700-dbc1-4101-a81c-05232bf02875-kube-api-access-ckx95\") pod \"eff99700-dbc1-4101-a81c-05232bf02875\" (UID: \"eff99700-dbc1-4101-a81c-05232bf02875\") " Jul 07 11:42:03.794187 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:42:03.793976 2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/eff99700-dbc1-4101-a81c-05232bf02875-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "eff99700-dbc1-4101-a81c-05232bf02875" (UID: "eff99700-dbc1-4101-a81c-05232bf02875"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 11:42:03.794187 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:42:03.794090 2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/eff99700-dbc1-4101-a81c-05232bf02875-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "eff99700-dbc1-4101-a81c-05232bf02875" (UID: "eff99700-dbc1-4101-a81c-05232bf02875"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 11:42:03.794303 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:42:03.794193 2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/eff99700-dbc1-4101-a81c-05232bf02875-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "eff99700-dbc1-4101-a81c-05232bf02875" (UID: "eff99700-dbc1-4101-a81c-05232bf02875"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 07 11:42:03.794303 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:42:03.794243 2578 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/eff99700-dbc1-4101-a81c-05232bf02875-credential-socket\") on node \"ip-10-0-142-112.ec2.internal\" DevicePath \"\"" Jul 07 11:42:03.794303 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:42:03.794283 2578 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/eff99700-dbc1-4101-a81c-05232bf02875-workload-socket\") on node \"ip-10-0-142-112.ec2.internal\" DevicePath \"\"" Jul 07 11:42:03.794563 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:42:03.794530 2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/eff99700-dbc1-4101-a81c-05232bf02875-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "eff99700-dbc1-4101-a81c-05232bf02875" (UID: "eff99700-dbc1-4101-a81c-05232bf02875"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 11:42:03.794696 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:42:03.794677 2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/eff99700-dbc1-4101-a81c-05232bf02875-istio-data" (OuterVolumeSpecName: "istio-data") pod "eff99700-dbc1-4101-a81c-05232bf02875" (UID: "eff99700-dbc1-4101-a81c-05232bf02875"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 11:42:03.796303 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:42:03.796272 2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/eff99700-dbc1-4101-a81c-05232bf02875-istio-token" (OuterVolumeSpecName: "istio-token") pod "eff99700-dbc1-4101-a81c-05232bf02875" (UID: "eff99700-dbc1-4101-a81c-05232bf02875"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 07 11:42:03.796389 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:42:03.796308 2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/eff99700-dbc1-4101-a81c-05232bf02875-kube-api-access-ckx95" (OuterVolumeSpecName: "kube-api-access-ckx95") pod "eff99700-dbc1-4101-a81c-05232bf02875" (UID: "eff99700-dbc1-4101-a81c-05232bf02875"). InnerVolumeSpecName "kube-api-access-ckx95". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 07 11:42:03.796389 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:42:03.796362 2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/eff99700-dbc1-4101-a81c-05232bf02875-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "eff99700-dbc1-4101-a81c-05232bf02875" (UID: "eff99700-dbc1-4101-a81c-05232bf02875"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 07 11:42:03.796553 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:42:03.796459 2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/eff99700-dbc1-4101-a81c-05232bf02875-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "eff99700-dbc1-4101-a81c-05232bf02875" (UID: "eff99700-dbc1-4101-a81c-05232bf02875"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 11:42:03.894855 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:42:03.894806 2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-ckx95\" (UniqueName: \"kubernetes.io/projected/eff99700-dbc1-4101-a81c-05232bf02875-kube-api-access-ckx95\") on node \"ip-10-0-142-112.ec2.internal\" DevicePath \"\"" Jul 07 11:42:03.894855 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:42:03.894846 2578 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/eff99700-dbc1-4101-a81c-05232bf02875-istio-podinfo\") on node \"ip-10-0-142-112.ec2.internal\" DevicePath \"\"" Jul 07 11:42:03.894855 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:42:03.894859 2578 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/eff99700-dbc1-4101-a81c-05232bf02875-istiod-ca-cert\") on node \"ip-10-0-142-112.ec2.internal\" DevicePath \"\"" Jul 07 11:42:03.895099 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:42:03.894872 2578 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/eff99700-dbc1-4101-a81c-05232bf02875-istio-envoy\") on node \"ip-10-0-142-112.ec2.internal\" DevicePath \"\"" Jul 07 11:42:03.895099 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:42:03.894882 2578 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/eff99700-dbc1-4101-a81c-05232bf02875-istio-data\") on node \"ip-10-0-142-112.ec2.internal\" DevicePath \"\"" Jul 07 11:42:03.895099 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:42:03.894893 2578 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/eff99700-dbc1-4101-a81c-05232bf02875-istio-token\") on node \"ip-10-0-142-112.ec2.internal\" DevicePath \"\"" Jul 07 11:42:03.895099 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:42:03.894904 2578 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/eff99700-dbc1-4101-a81c-05232bf02875-workload-certs\") on node \"ip-10-0-142-112.ec2.internal\" DevicePath \"\"" Jul 07 11:42:03.959966 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:42:03.959931 2578 generic.go:356] "Generic (PLEG): container finished" podID="eff99700-dbc1-4101-a81c-05232bf02875" containerID="6f3c58e79168caf8d026898d544874f6fc9291ad28bba1dabd5bbf82b3f5b2c8" exitCode=0 Jul 07 11:42:03.960162 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:42:03.959979 2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-d21d182b-openshift-default-7bf5768659-2x9r6" Jul 07 11:42:03.960162 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:42:03.960002 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-d21d182b-openshift-default-7bf5768659-2x9r6" event={"ID":"eff99700-dbc1-4101-a81c-05232bf02875","Type":"ContainerDied","Data":"6f3c58e79168caf8d026898d544874f6fc9291ad28bba1dabd5bbf82b3f5b2c8"} Jul 07 11:42:03.960162 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:42:03.960051 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-d21d182b-openshift-default-7bf5768659-2x9r6" event={"ID":"eff99700-dbc1-4101-a81c-05232bf02875","Type":"ContainerDied","Data":"57920b82dc82e5078ded81b079142d7bf076fcce98c5f0c83183c2366e0e6847"} Jul 07 11:42:03.960162 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:42:03.960081 2578 scope.go:117] "RemoveContainer" containerID="6f3c58e79168caf8d026898d544874f6fc9291ad28bba1dabd5bbf82b3f5b2c8" Jul 07 11:42:03.968956 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:42:03.968938 2578 scope.go:117] "RemoveContainer" containerID="6f3c58e79168caf8d026898d544874f6fc9291ad28bba1dabd5bbf82b3f5b2c8" Jul 07 11:42:03.969190 ip-10-0-142-112 kubenswrapper[2578]: E0707 11:42:03.969169 2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6f3c58e79168caf8d026898d544874f6fc9291ad28bba1dabd5bbf82b3f5b2c8\": container with ID starting with 6f3c58e79168caf8d026898d544874f6fc9291ad28bba1dabd5bbf82b3f5b2c8 not found: ID does not exist" containerID="6f3c58e79168caf8d026898d544874f6fc9291ad28bba1dabd5bbf82b3f5b2c8" Jul 07 11:42:03.969244 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:42:03.969199 2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6f3c58e79168caf8d026898d544874f6fc9291ad28bba1dabd5bbf82b3f5b2c8"} err="failed to get container status \"6f3c58e79168caf8d026898d544874f6fc9291ad28bba1dabd5bbf82b3f5b2c8\": rpc error: code = NotFound desc = could not find container \"6f3c58e79168caf8d026898d544874f6fc9291ad28bba1dabd5bbf82b3f5b2c8\": container with ID starting with 6f3c58e79168caf8d026898d544874f6fc9291ad28bba1dabd5bbf82b3f5b2c8 not found: ID does not exist" Jul 07 11:42:03.982813 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:42:03.982785 2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-d21d182b-openshift-default-7bf5768659-2x9r6"] Jul 07 11:42:03.989963 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:42:03.989942 2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-ait-d21d182b-openshift-default-7bf5768659-2x9r6"] Jul 07 11:42:04.306856 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:42:04.306825 2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="eff99700-dbc1-4101-a81c-05232bf02875" path="/var/lib/kubelet/pods/eff99700-dbc1-4101-a81c-05232bf02875/volumes" Jul 07 11:42:04.957775 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:42:04.957740 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-ait-2813d1c1-openshift-default-675f9f69c-449kb"] Jul 07 11:42:04.958234 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:42:04.958096 2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="eff99700-dbc1-4101-a81c-05232bf02875" containerName="istio-proxy" Jul 07 11:42:04.958234 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:42:04.958109 2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="eff99700-dbc1-4101-a81c-05232bf02875" containerName="istio-proxy" Jul 07 11:42:04.958234 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:42:04.958158 2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="eff99700-dbc1-4101-a81c-05232bf02875" containerName="istio-proxy" Jul 07 11:42:04.961326 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:42:04.961303 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-2813d1c1-openshift-default-675f9f69c-449kb" Jul 07 11:42:04.963783 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:42:04.963758 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-ait-2813d1c1-openshift-default-dockercfg-qd4gm\"" Jul 07 11:42:04.975287 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:42:04.975261 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-2813d1c1-openshift-default-675f9f69c-449kb"] Jul 07 11:42:05.105304 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:42:05.105273 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dd55d\" (UniqueName: \"kubernetes.io/projected/e2c27bec-e55b-4f54-a244-bea51a9a842d-kube-api-access-dd55d\") pod \"e2e-ait-2813d1c1-openshift-default-675f9f69c-449kb\" (UID: \"e2c27bec-e55b-4f54-a244-bea51a9a842d\") " pod="openshift-ingress/e2e-ait-2813d1c1-openshift-default-675f9f69c-449kb" Jul 07 11:42:05.105487 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:42:05.105322 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e2c27bec-e55b-4f54-a244-bea51a9a842d-istio-token\") pod \"e2e-ait-2813d1c1-openshift-default-675f9f69c-449kb\" (UID: \"e2c27bec-e55b-4f54-a244-bea51a9a842d\") " pod="openshift-ingress/e2e-ait-2813d1c1-openshift-default-675f9f69c-449kb" Jul 07 11:42:05.105487 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:42:05.105398 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e2c27bec-e55b-4f54-a244-bea51a9a842d-credential-socket\") pod \"e2e-ait-2813d1c1-openshift-default-675f9f69c-449kb\" (UID: \"e2c27bec-e55b-4f54-a244-bea51a9a842d\") " pod="openshift-ingress/e2e-ait-2813d1c1-openshift-default-675f9f69c-449kb" Jul 07 11:42:05.105487 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:42:05.105448 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e2c27bec-e55b-4f54-a244-bea51a9a842d-istiod-ca-cert\") pod \"e2e-ait-2813d1c1-openshift-default-675f9f69c-449kb\" (UID: \"e2c27bec-e55b-4f54-a244-bea51a9a842d\") " pod="openshift-ingress/e2e-ait-2813d1c1-openshift-default-675f9f69c-449kb" Jul 07 11:42:05.105487 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:42:05.105470 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e2c27bec-e55b-4f54-a244-bea51a9a842d-istio-podinfo\") pod \"e2e-ait-2813d1c1-openshift-default-675f9f69c-449kb\" (UID: \"e2c27bec-e55b-4f54-a244-bea51a9a842d\") " pod="openshift-ingress/e2e-ait-2813d1c1-openshift-default-675f9f69c-449kb" Jul 07 11:42:05.105687 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:42:05.105535 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e2c27bec-e55b-4f54-a244-bea51a9a842d-workload-certs\") pod \"e2e-ait-2813d1c1-openshift-default-675f9f69c-449kb\" (UID: \"e2c27bec-e55b-4f54-a244-bea51a9a842d\") " pod="openshift-ingress/e2e-ait-2813d1c1-openshift-default-675f9f69c-449kb" Jul 07 11:42:05.105687 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:42:05.105600 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e2c27bec-e55b-4f54-a244-bea51a9a842d-istio-envoy\") pod \"e2e-ait-2813d1c1-openshift-default-675f9f69c-449kb\" (UID: \"e2c27bec-e55b-4f54-a244-bea51a9a842d\") " pod="openshift-ingress/e2e-ait-2813d1c1-openshift-default-675f9f69c-449kb" Jul 07 11:42:05.105687 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:42:05.105663 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e2c27bec-e55b-4f54-a244-bea51a9a842d-workload-socket\") pod \"e2e-ait-2813d1c1-openshift-default-675f9f69c-449kb\" (UID: \"e2c27bec-e55b-4f54-a244-bea51a9a842d\") " pod="openshift-ingress/e2e-ait-2813d1c1-openshift-default-675f9f69c-449kb" Jul 07 11:42:05.105811 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:42:05.105701 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e2c27bec-e55b-4f54-a244-bea51a9a842d-istio-data\") pod \"e2e-ait-2813d1c1-openshift-default-675f9f69c-449kb\" (UID: \"e2c27bec-e55b-4f54-a244-bea51a9a842d\") " pod="openshift-ingress/e2e-ait-2813d1c1-openshift-default-675f9f69c-449kb" Jul 07 11:42:05.206653 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:42:05.206616 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e2c27bec-e55b-4f54-a244-bea51a9a842d-workload-socket\") pod \"e2e-ait-2813d1c1-openshift-default-675f9f69c-449kb\" (UID: \"e2c27bec-e55b-4f54-a244-bea51a9a842d\") " pod="openshift-ingress/e2e-ait-2813d1c1-openshift-default-675f9f69c-449kb" Jul 07 11:42:05.206653 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:42:05.206659 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e2c27bec-e55b-4f54-a244-bea51a9a842d-istio-data\") pod \"e2e-ait-2813d1c1-openshift-default-675f9f69c-449kb\" (UID: \"e2c27bec-e55b-4f54-a244-bea51a9a842d\") " pod="openshift-ingress/e2e-ait-2813d1c1-openshift-default-675f9f69c-449kb" Jul 07 11:42:05.206915 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:42:05.206695 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dd55d\" (UniqueName: \"kubernetes.io/projected/e2c27bec-e55b-4f54-a244-bea51a9a842d-kube-api-access-dd55d\") pod \"e2e-ait-2813d1c1-openshift-default-675f9f69c-449kb\" (UID: \"e2c27bec-e55b-4f54-a244-bea51a9a842d\") " pod="openshift-ingress/e2e-ait-2813d1c1-openshift-default-675f9f69c-449kb" Jul 07 11:42:05.206915 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:42:05.206727 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e2c27bec-e55b-4f54-a244-bea51a9a842d-istio-token\") pod \"e2e-ait-2813d1c1-openshift-default-675f9f69c-449kb\" (UID: \"e2c27bec-e55b-4f54-a244-bea51a9a842d\") " pod="openshift-ingress/e2e-ait-2813d1c1-openshift-default-675f9f69c-449kb" Jul 07 11:42:05.206915 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:42:05.206763 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e2c27bec-e55b-4f54-a244-bea51a9a842d-credential-socket\") pod \"e2e-ait-2813d1c1-openshift-default-675f9f69c-449kb\" (UID: \"e2c27bec-e55b-4f54-a244-bea51a9a842d\") " pod="openshift-ingress/e2e-ait-2813d1c1-openshift-default-675f9f69c-449kb" Jul 07 11:42:05.206915 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:42:05.206793 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e2c27bec-e55b-4f54-a244-bea51a9a842d-istiod-ca-cert\") pod \"e2e-ait-2813d1c1-openshift-default-675f9f69c-449kb\" (UID: \"e2c27bec-e55b-4f54-a244-bea51a9a842d\") " pod="openshift-ingress/e2e-ait-2813d1c1-openshift-default-675f9f69c-449kb" Jul 07 11:42:05.206915 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:42:05.206820 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e2c27bec-e55b-4f54-a244-bea51a9a842d-istio-podinfo\") pod \"e2e-ait-2813d1c1-openshift-default-675f9f69c-449kb\" (UID: \"e2c27bec-e55b-4f54-a244-bea51a9a842d\") " pod="openshift-ingress/e2e-ait-2813d1c1-openshift-default-675f9f69c-449kb" Jul 07 11:42:05.206915 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:42:05.206861 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e2c27bec-e55b-4f54-a244-bea51a9a842d-workload-certs\") pod \"e2e-ait-2813d1c1-openshift-default-675f9f69c-449kb\" (UID: \"e2c27bec-e55b-4f54-a244-bea51a9a842d\") " pod="openshift-ingress/e2e-ait-2813d1c1-openshift-default-675f9f69c-449kb" Jul 07 11:42:05.207609 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:42:05.207563 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e2c27bec-e55b-4f54-a244-bea51a9a842d-workload-socket\") pod \"e2e-ait-2813d1c1-openshift-default-675f9f69c-449kb\" (UID: \"e2c27bec-e55b-4f54-a244-bea51a9a842d\") " pod="openshift-ingress/e2e-ait-2813d1c1-openshift-default-675f9f69c-449kb" Jul 07 11:42:05.207727 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:42:05.207607 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e2c27bec-e55b-4f54-a244-bea51a9a842d-istio-data\") pod \"e2e-ait-2813d1c1-openshift-default-675f9f69c-449kb\" (UID: \"e2c27bec-e55b-4f54-a244-bea51a9a842d\") " pod="openshift-ingress/e2e-ait-2813d1c1-openshift-default-675f9f69c-449kb" Jul 07 11:42:05.207727 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:42:05.207651 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e2c27bec-e55b-4f54-a244-bea51a9a842d-credential-socket\") pod \"e2e-ait-2813d1c1-openshift-default-675f9f69c-449kb\" (UID: \"e2c27bec-e55b-4f54-a244-bea51a9a842d\") " pod="openshift-ingress/e2e-ait-2813d1c1-openshift-default-675f9f69c-449kb" Jul 07 11:42:05.207727 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:42:05.207668 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e2c27bec-e55b-4f54-a244-bea51a9a842d-workload-certs\") pod \"e2e-ait-2813d1c1-openshift-default-675f9f69c-449kb\" (UID: \"e2c27bec-e55b-4f54-a244-bea51a9a842d\") " pod="openshift-ingress/e2e-ait-2813d1c1-openshift-default-675f9f69c-449kb" Jul 07 11:42:05.207990 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:42:05.207747 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e2c27bec-e55b-4f54-a244-bea51a9a842d-istio-envoy\") pod \"e2e-ait-2813d1c1-openshift-default-675f9f69c-449kb\" (UID: \"e2c27bec-e55b-4f54-a244-bea51a9a842d\") " pod="openshift-ingress/e2e-ait-2813d1c1-openshift-default-675f9f69c-449kb" Jul 07 11:42:05.213438 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:42:05.208828 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e2c27bec-e55b-4f54-a244-bea51a9a842d-istiod-ca-cert\") pod \"e2e-ait-2813d1c1-openshift-default-675f9f69c-449kb\" (UID: \"e2c27bec-e55b-4f54-a244-bea51a9a842d\") " pod="openshift-ingress/e2e-ait-2813d1c1-openshift-default-675f9f69c-449kb" Jul 07 11:42:05.214400 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:42:05.214382 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e2c27bec-e55b-4f54-a244-bea51a9a842d-istio-envoy\") pod \"e2e-ait-2813d1c1-openshift-default-675f9f69c-449kb\" (UID: \"e2c27bec-e55b-4f54-a244-bea51a9a842d\") " pod="openshift-ingress/e2e-ait-2813d1c1-openshift-default-675f9f69c-449kb" Jul 07 11:42:05.214621 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:42:05.214601 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e2c27bec-e55b-4f54-a244-bea51a9a842d-istio-podinfo\") pod \"e2e-ait-2813d1c1-openshift-default-675f9f69c-449kb\" (UID: \"e2c27bec-e55b-4f54-a244-bea51a9a842d\") " pod="openshift-ingress/e2e-ait-2813d1c1-openshift-default-675f9f69c-449kb" Jul 07 11:42:05.220150 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:42:05.220129 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-dd55d\" (UniqueName: \"kubernetes.io/projected/e2c27bec-e55b-4f54-a244-bea51a9a842d-kube-api-access-dd55d\") pod \"e2e-ait-2813d1c1-openshift-default-675f9f69c-449kb\" (UID: \"e2c27bec-e55b-4f54-a244-bea51a9a842d\") " pod="openshift-ingress/e2e-ait-2813d1c1-openshift-default-675f9f69c-449kb" Jul 07 11:42:05.221683 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:42:05.221664 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e2c27bec-e55b-4f54-a244-bea51a9a842d-istio-token\") pod \"e2e-ait-2813d1c1-openshift-default-675f9f69c-449kb\" (UID: \"e2c27bec-e55b-4f54-a244-bea51a9a842d\") " pod="openshift-ingress/e2e-ait-2813d1c1-openshift-default-675f9f69c-449kb" Jul 07 11:42:05.272777 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:42:05.272721 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-2813d1c1-openshift-default-675f9f69c-449kb" Jul 07 11:42:05.420805 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:42:05.420776 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-2813d1c1-openshift-default-675f9f69c-449kb"] Jul 07 11:42:05.422367 ip-10-0-142-112 kubenswrapper[2578]: W0707 11:42:05.422338 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode2c27bec_e55b_4f54_a244_bea51a9a842d.slice/crio-d92ba0ba1a208b4a98add598564d6052c29a6bb70e1c4df47e7d10a74227c3a2 WatchSource:0}: Error finding container d92ba0ba1a208b4a98add598564d6052c29a6bb70e1c4df47e7d10a74227c3a2: Status 404 returned error can't find the container with id d92ba0ba1a208b4a98add598564d6052c29a6bb70e1c4df47e7d10a74227c3a2 Jul 07 11:42:05.424691 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:42:05.424657 2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 07 11:42:05.424767 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:42:05.424723 2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 07 11:42:05.424767 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:42:05.424749 2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 07 11:42:05.972760 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:42:05.972726 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-2813d1c1-openshift-default-675f9f69c-449kb" event={"ID":"e2c27bec-e55b-4f54-a244-bea51a9a842d","Type":"ContainerStarted","Data":"9c5a44d87a7eafab5e9199125b760ad4f8695d31470e779d61cbde32de4cae9e"} Jul 07 11:42:05.973145 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:42:05.972779 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-2813d1c1-openshift-default-675f9f69c-449kb" event={"ID":"e2c27bec-e55b-4f54-a244-bea51a9a842d","Type":"ContainerStarted","Data":"d92ba0ba1a208b4a98add598564d6052c29a6bb70e1c4df47e7d10a74227c3a2"} Jul 07 11:42:06.003226 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:42:06.003165 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-ait-2813d1c1-openshift-default-675f9f69c-449kb" podStartSLOduration=2.003148394 podStartE2EDuration="2.003148394s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 11:42:04 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 11:42:05.999785674 +0000 UTC m=+1570.295935261" watchObservedRunningTime="2026-07-07 11:42:06.003148394 +0000 UTC m=+1570.299297983" Jul 07 11:42:06.272912 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:42:06.272825 2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ait-2813d1c1-openshift-default-675f9f69c-449kb" Jul 07 11:42:06.278065 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:42:06.278039 2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ait-2813d1c1-openshift-default-675f9f69c-449kb" Jul 07 11:42:06.977955 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:42:06.977924 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-2813d1c1-openshift-default-675f9f69c-449kb" Jul 07 11:42:06.978796 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:42:06.978776 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ait-2813d1c1-openshift-default-675f9f69c-449kb" Jul 07 11:42:17.660371 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:42:17.660333 2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-2813d1c1-openshift-default-675f9f69c-449kb"] Jul 07 11:42:17.660981 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:42:17.660933 2578 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-ingress/e2e-ait-2813d1c1-openshift-default-675f9f69c-449kb" podUID="e2c27bec-e55b-4f54-a244-bea51a9a842d" containerName="istio-proxy" containerID="cri-o://9c5a44d87a7eafab5e9199125b760ad4f8695d31470e779d61cbde32de4cae9e" gracePeriod=30 Jul 07 11:42:21.981028 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:42:21.980991 2578 patch_prober.go:28] interesting pod/e2e-ait-2813d1c1-openshift-default-675f9f69c-449kb container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body= Jul 07 11:42:21.981457 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:42:21.981050 2578 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-ait-2813d1c1-openshift-default-675f9f69c-449kb" podUID="e2c27bec-e55b-4f54-a244-bea51a9a842d" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 07 11:42:22.909599 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:42:22.909573 2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-2813d1c1-openshift-default-675f9f69c-449kb" Jul 07 11:42:22.958333 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:42:22.958257 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e2c27bec-e55b-4f54-a244-bea51a9a842d-istio-token\") pod \"e2c27bec-e55b-4f54-a244-bea51a9a842d\" (UID: \"e2c27bec-e55b-4f54-a244-bea51a9a842d\") " Jul 07 11:42:22.958333 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:42:22.958296 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e2c27bec-e55b-4f54-a244-bea51a9a842d-istio-envoy\") pod \"e2c27bec-e55b-4f54-a244-bea51a9a842d\" (UID: \"e2c27bec-e55b-4f54-a244-bea51a9a842d\") " Jul 07 11:42:22.958333 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:42:22.958321 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e2c27bec-e55b-4f54-a244-bea51a9a842d-istio-podinfo\") pod \"e2c27bec-e55b-4f54-a244-bea51a9a842d\" (UID: \"e2c27bec-e55b-4f54-a244-bea51a9a842d\") " Jul 07 11:42:22.958813 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:42:22.958347 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e2c27bec-e55b-4f54-a244-bea51a9a842d-workload-certs\") pod \"e2c27bec-e55b-4f54-a244-bea51a9a842d\" (UID: \"e2c27bec-e55b-4f54-a244-bea51a9a842d\") " Jul 07 11:42:22.958813 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:42:22.958382 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e2c27bec-e55b-4f54-a244-bea51a9a842d-istio-data\") pod \"e2c27bec-e55b-4f54-a244-bea51a9a842d\" (UID: \"e2c27bec-e55b-4f54-a244-bea51a9a842d\") " Jul 07 11:42:22.958813 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:42:22.958440 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e2c27bec-e55b-4f54-a244-bea51a9a842d-credential-socket\") pod \"e2c27bec-e55b-4f54-a244-bea51a9a842d\" (UID: \"e2c27bec-e55b-4f54-a244-bea51a9a842d\") " Jul 07 11:42:22.958813 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:42:22.958479 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e2c27bec-e55b-4f54-a244-bea51a9a842d-workload-socket\") pod \"e2c27bec-e55b-4f54-a244-bea51a9a842d\" (UID: \"e2c27bec-e55b-4f54-a244-bea51a9a842d\") " Jul 07 11:42:22.958813 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:42:22.958521 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e2c27bec-e55b-4f54-a244-bea51a9a842d-istiod-ca-cert\") pod \"e2c27bec-e55b-4f54-a244-bea51a9a842d\" (UID: \"e2c27bec-e55b-4f54-a244-bea51a9a842d\") " Jul 07 11:42:22.958813 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:42:22.958539 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-dd55d\" (UniqueName: \"kubernetes.io/projected/e2c27bec-e55b-4f54-a244-bea51a9a842d-kube-api-access-dd55d\") pod \"e2c27bec-e55b-4f54-a244-bea51a9a842d\" (UID: \"e2c27bec-e55b-4f54-a244-bea51a9a842d\") " Jul 07 11:42:22.961826 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:42:22.961791 2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e2c27bec-e55b-4f54-a244-bea51a9a842d-istio-token" (OuterVolumeSpecName: "istio-token") pod "e2c27bec-e55b-4f54-a244-bea51a9a842d" (UID: "e2c27bec-e55b-4f54-a244-bea51a9a842d"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 07 11:42:22.962703 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:42:22.962674 2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e2c27bec-e55b-4f54-a244-bea51a9a842d-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "e2c27bec-e55b-4f54-a244-bea51a9a842d" (UID: "e2c27bec-e55b-4f54-a244-bea51a9a842d"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 11:42:22.962852 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:42:22.962816 2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e2c27bec-e55b-4f54-a244-bea51a9a842d-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "e2c27bec-e55b-4f54-a244-bea51a9a842d" (UID: "e2c27bec-e55b-4f54-a244-bea51a9a842d"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 11:42:22.962852 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:42:22.962841 2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e2c27bec-e55b-4f54-a244-bea51a9a842d-istio-data" (OuterVolumeSpecName: "istio-data") pod "e2c27bec-e55b-4f54-a244-bea51a9a842d" (UID: "e2c27bec-e55b-4f54-a244-bea51a9a842d"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 11:42:22.963007 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:42:22.962958 2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e2c27bec-e55b-4f54-a244-bea51a9a842d-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "e2c27bec-e55b-4f54-a244-bea51a9a842d" (UID: "e2c27bec-e55b-4f54-a244-bea51a9a842d"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 11:42:22.963224 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:42:22.963166 2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e2c27bec-e55b-4f54-a244-bea51a9a842d-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "e2c27bec-e55b-4f54-a244-bea51a9a842d" (UID: "e2c27bec-e55b-4f54-a244-bea51a9a842d"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 07 11:42:22.964452 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:42:22.964412 2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e2c27bec-e55b-4f54-a244-bea51a9a842d-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "e2c27bec-e55b-4f54-a244-bea51a9a842d" (UID: "e2c27bec-e55b-4f54-a244-bea51a9a842d"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 11:42:22.964541 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:42:22.964475 2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e2c27bec-e55b-4f54-a244-bea51a9a842d-kube-api-access-dd55d" (OuterVolumeSpecName: "kube-api-access-dd55d") pod "e2c27bec-e55b-4f54-a244-bea51a9a842d" (UID: "e2c27bec-e55b-4f54-a244-bea51a9a842d"). InnerVolumeSpecName "kube-api-access-dd55d". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 07 11:42:22.964895 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:42:22.964876 2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/e2c27bec-e55b-4f54-a244-bea51a9a842d-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "e2c27bec-e55b-4f54-a244-bea51a9a842d" (UID: "e2c27bec-e55b-4f54-a244-bea51a9a842d"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 07 11:42:23.048569 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:42:23.048532 2578 generic.go:356] "Generic (PLEG): container finished" podID="e2c27bec-e55b-4f54-a244-bea51a9a842d" containerID="9c5a44d87a7eafab5e9199125b760ad4f8695d31470e779d61cbde32de4cae9e" exitCode=0 Jul 07 11:42:23.048950 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:42:23.048583 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-2813d1c1-openshift-default-675f9f69c-449kb" event={"ID":"e2c27bec-e55b-4f54-a244-bea51a9a842d","Type":"ContainerDied","Data":"9c5a44d87a7eafab5e9199125b760ad4f8695d31470e779d61cbde32de4cae9e"} Jul 07 11:42:23.048950 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:42:23.048624 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-2813d1c1-openshift-default-675f9f69c-449kb" event={"ID":"e2c27bec-e55b-4f54-a244-bea51a9a842d","Type":"ContainerDied","Data":"d92ba0ba1a208b4a98add598564d6052c29a6bb70e1c4df47e7d10a74227c3a2"} Jul 07 11:42:23.048950 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:42:23.048647 2578 scope.go:117] "RemoveContainer" containerID="9c5a44d87a7eafab5e9199125b760ad4f8695d31470e779d61cbde32de4cae9e" Jul 07 11:42:23.048950 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:42:23.048586 2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-2813d1c1-openshift-default-675f9f69c-449kb" Jul 07 11:42:23.057154 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:42:23.057138 2578 scope.go:117] "RemoveContainer" containerID="9c5a44d87a7eafab5e9199125b760ad4f8695d31470e779d61cbde32de4cae9e" Jul 07 11:42:23.057433 ip-10-0-142-112 kubenswrapper[2578]: E0707 11:42:23.057400 2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9c5a44d87a7eafab5e9199125b760ad4f8695d31470e779d61cbde32de4cae9e\": container with ID starting with 9c5a44d87a7eafab5e9199125b760ad4f8695d31470e779d61cbde32de4cae9e not found: ID does not exist" containerID="9c5a44d87a7eafab5e9199125b760ad4f8695d31470e779d61cbde32de4cae9e" Jul 07 11:42:23.057495 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:42:23.057445 2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9c5a44d87a7eafab5e9199125b760ad4f8695d31470e779d61cbde32de4cae9e"} err="failed to get container status \"9c5a44d87a7eafab5e9199125b760ad4f8695d31470e779d61cbde32de4cae9e\": rpc error: code = NotFound desc = could not find container \"9c5a44d87a7eafab5e9199125b760ad4f8695d31470e779d61cbde32de4cae9e\": container with ID starting with 9c5a44d87a7eafab5e9199125b760ad4f8695d31470e779d61cbde32de4cae9e not found: ID does not exist" Jul 07 11:42:23.059861 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:42:23.059840 2578 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e2c27bec-e55b-4f54-a244-bea51a9a842d-istio-token\") on node \"ip-10-0-142-112.ec2.internal\" DevicePath \"\"" Jul 07 11:42:23.059861 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:42:23.059861 2578 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e2c27bec-e55b-4f54-a244-bea51a9a842d-istio-envoy\") on node \"ip-10-0-142-112.ec2.internal\" DevicePath \"\"" Jul 07 11:42:23.059980 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:42:23.059871 2578 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e2c27bec-e55b-4f54-a244-bea51a9a842d-istio-podinfo\") on node \"ip-10-0-142-112.ec2.internal\" DevicePath \"\"" Jul 07 11:42:23.059980 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:42:23.059880 2578 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e2c27bec-e55b-4f54-a244-bea51a9a842d-workload-certs\") on node \"ip-10-0-142-112.ec2.internal\" DevicePath \"\"" Jul 07 11:42:23.059980 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:42:23.059889 2578 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e2c27bec-e55b-4f54-a244-bea51a9a842d-istio-data\") on node \"ip-10-0-142-112.ec2.internal\" DevicePath \"\"" Jul 07 11:42:23.059980 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:42:23.059897 2578 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e2c27bec-e55b-4f54-a244-bea51a9a842d-credential-socket\") on node \"ip-10-0-142-112.ec2.internal\" DevicePath \"\"" Jul 07 11:42:23.059980 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:42:23.059905 2578 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e2c27bec-e55b-4f54-a244-bea51a9a842d-workload-socket\") on node \"ip-10-0-142-112.ec2.internal\" DevicePath \"\"" Jul 07 11:42:23.059980 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:42:23.059914 2578 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e2c27bec-e55b-4f54-a244-bea51a9a842d-istiod-ca-cert\") on node \"ip-10-0-142-112.ec2.internal\" DevicePath \"\"" Jul 07 11:42:23.059980 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:42:23.059921 2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-dd55d\" (UniqueName: \"kubernetes.io/projected/e2c27bec-e55b-4f54-a244-bea51a9a842d-kube-api-access-dd55d\") on node \"ip-10-0-142-112.ec2.internal\" DevicePath \"\"" Jul 07 11:42:23.072163 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:42:23.072113 2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-2813d1c1-openshift-default-675f9f69c-449kb"] Jul 07 11:42:23.074127 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:42:23.074105 2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-ait-2813d1c1-openshift-default-675f9f69c-449kb"] Jul 07 11:42:24.307489 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:42:24.307451 2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e2c27bec-e55b-4f54-a244-bea51a9a842d" path="/var/lib/kubelet/pods/e2c27bec-e55b-4f54-a244-bea51a9a842d/volumes" Jul 07 11:48:47.599249 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:48:47.599161 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-inf-b-c903aa/test-model-c903aa-kserve-74c5db6cf9-5jbph"] Jul 07 11:48:47.599720 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:48:47.599577 2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e2c27bec-e55b-4f54-a244-bea51a9a842d" containerName="istio-proxy" Jul 07 11:48:47.599720 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:48:47.599590 2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="e2c27bec-e55b-4f54-a244-bea51a9a842d" containerName="istio-proxy" Jul 07 11:48:47.599720 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:48:47.599655 2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="e2c27bec-e55b-4f54-a244-bea51a9a842d" containerName="istio-proxy" Jul 07 11:48:47.602506 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:48:47.602490 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-c903aa/test-model-c903aa-kserve-74c5db6cf9-5jbph" Jul 07 11:48:47.609853 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:48:47.609834 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-b-c903aa\"/\"default-dockercfg-4dxr7\"" Jul 07 11:48:47.610597 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:48:47.610579 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-b-c903aa\"/\"test-model-c903aa-kserve-self-signed-certs\"" Jul 07 11:48:47.611155 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:48:47.611136 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-b-c903aa\"/\"openshift-service-ca.crt\"" Jul 07 11:48:47.611258 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:48:47.611206 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-b-c903aa\"/\"kube-root-ca.crt\"" Jul 07 11:48:47.617160 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:48:47.617135 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-b-c903aa/test-model-c903aa-kserve-74c5db6cf9-5jbph"] Jul 07 11:48:47.687483 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:48:47.687451 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d6554292-b6ce-4b59-9168-8fbba8b09cc6-tmp-dir\") pod \"test-model-c903aa-kserve-74c5db6cf9-5jbph\" (UID: \"d6554292-b6ce-4b59-9168-8fbba8b09cc6\") " pod="ai-tenant-e2e-inf-b-c903aa/test-model-c903aa-kserve-74c5db6cf9-5jbph" Jul 07 11:48:47.687669 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:48:47.687497 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d6554292-b6ce-4b59-9168-8fbba8b09cc6-tls-certs\") pod \"test-model-c903aa-kserve-74c5db6cf9-5jbph\" (UID: \"d6554292-b6ce-4b59-9168-8fbba8b09cc6\") " pod="ai-tenant-e2e-inf-b-c903aa/test-model-c903aa-kserve-74c5db6cf9-5jbph" Jul 07 11:48:47.687669 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:48:47.687623 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d6554292-b6ce-4b59-9168-8fbba8b09cc6-home\") pod \"test-model-c903aa-kserve-74c5db6cf9-5jbph\" (UID: \"d6554292-b6ce-4b59-9168-8fbba8b09cc6\") " pod="ai-tenant-e2e-inf-b-c903aa/test-model-c903aa-kserve-74c5db6cf9-5jbph" Jul 07 11:48:47.687669 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:48:47.687651 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d6554292-b6ce-4b59-9168-8fbba8b09cc6-dshm\") pod \"test-model-c903aa-kserve-74c5db6cf9-5jbph\" (UID: \"d6554292-b6ce-4b59-9168-8fbba8b09cc6\") " pod="ai-tenant-e2e-inf-b-c903aa/test-model-c903aa-kserve-74c5db6cf9-5jbph" Jul 07 11:48:47.687799 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:48:47.687674 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d6554292-b6ce-4b59-9168-8fbba8b09cc6-model-cache\") pod \"test-model-c903aa-kserve-74c5db6cf9-5jbph\" (UID: \"d6554292-b6ce-4b59-9168-8fbba8b09cc6\") " pod="ai-tenant-e2e-inf-b-c903aa/test-model-c903aa-kserve-74c5db6cf9-5jbph" Jul 07 11:48:47.687799 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:48:47.687754 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lkfjq\" (UniqueName: \"kubernetes.io/projected/d6554292-b6ce-4b59-9168-8fbba8b09cc6-kube-api-access-lkfjq\") pod \"test-model-c903aa-kserve-74c5db6cf9-5jbph\" (UID: \"d6554292-b6ce-4b59-9168-8fbba8b09cc6\") " pod="ai-tenant-e2e-inf-b-c903aa/test-model-c903aa-kserve-74c5db6cf9-5jbph" Jul 07 11:48:47.788820 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:48:47.788785 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d6554292-b6ce-4b59-9168-8fbba8b09cc6-home\") pod \"test-model-c903aa-kserve-74c5db6cf9-5jbph\" (UID: \"d6554292-b6ce-4b59-9168-8fbba8b09cc6\") " pod="ai-tenant-e2e-inf-b-c903aa/test-model-c903aa-kserve-74c5db6cf9-5jbph" Jul 07 11:48:47.788820 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:48:47.788819 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d6554292-b6ce-4b59-9168-8fbba8b09cc6-dshm\") pod \"test-model-c903aa-kserve-74c5db6cf9-5jbph\" (UID: \"d6554292-b6ce-4b59-9168-8fbba8b09cc6\") " pod="ai-tenant-e2e-inf-b-c903aa/test-model-c903aa-kserve-74c5db6cf9-5jbph" Jul 07 11:48:47.789047 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:48:47.788837 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d6554292-b6ce-4b59-9168-8fbba8b09cc6-model-cache\") pod \"test-model-c903aa-kserve-74c5db6cf9-5jbph\" (UID: \"d6554292-b6ce-4b59-9168-8fbba8b09cc6\") " pod="ai-tenant-e2e-inf-b-c903aa/test-model-c903aa-kserve-74c5db6cf9-5jbph" Jul 07 11:48:47.789047 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:48:47.788960 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lkfjq\" (UniqueName: \"kubernetes.io/projected/d6554292-b6ce-4b59-9168-8fbba8b09cc6-kube-api-access-lkfjq\") pod \"test-model-c903aa-kserve-74c5db6cf9-5jbph\" (UID: \"d6554292-b6ce-4b59-9168-8fbba8b09cc6\") " pod="ai-tenant-e2e-inf-b-c903aa/test-model-c903aa-kserve-74c5db6cf9-5jbph" Jul 07 11:48:47.789047 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:48:47.789015 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d6554292-b6ce-4b59-9168-8fbba8b09cc6-tmp-dir\") pod \"test-model-c903aa-kserve-74c5db6cf9-5jbph\" (UID: \"d6554292-b6ce-4b59-9168-8fbba8b09cc6\") " pod="ai-tenant-e2e-inf-b-c903aa/test-model-c903aa-kserve-74c5db6cf9-5jbph" Jul 07 11:48:47.789252 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:48:47.789071 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d6554292-b6ce-4b59-9168-8fbba8b09cc6-tls-certs\") pod \"test-model-c903aa-kserve-74c5db6cf9-5jbph\" (UID: \"d6554292-b6ce-4b59-9168-8fbba8b09cc6\") " pod="ai-tenant-e2e-inf-b-c903aa/test-model-c903aa-kserve-74c5db6cf9-5jbph" Jul 07 11:48:47.789252 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:48:47.789210 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d6554292-b6ce-4b59-9168-8fbba8b09cc6-model-cache\") pod \"test-model-c903aa-kserve-74c5db6cf9-5jbph\" (UID: \"d6554292-b6ce-4b59-9168-8fbba8b09cc6\") " pod="ai-tenant-e2e-inf-b-c903aa/test-model-c903aa-kserve-74c5db6cf9-5jbph" Jul 07 11:48:47.789342 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:48:47.789280 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d6554292-b6ce-4b59-9168-8fbba8b09cc6-home\") pod \"test-model-c903aa-kserve-74c5db6cf9-5jbph\" (UID: \"d6554292-b6ce-4b59-9168-8fbba8b09cc6\") " pod="ai-tenant-e2e-inf-b-c903aa/test-model-c903aa-kserve-74c5db6cf9-5jbph" Jul 07 11:48:47.789380 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:48:47.789341 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d6554292-b6ce-4b59-9168-8fbba8b09cc6-tmp-dir\") pod \"test-model-c903aa-kserve-74c5db6cf9-5jbph\" (UID: \"d6554292-b6ce-4b59-9168-8fbba8b09cc6\") " pod="ai-tenant-e2e-inf-b-c903aa/test-model-c903aa-kserve-74c5db6cf9-5jbph" Jul 07 11:48:47.791137 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:48:47.791115 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d6554292-b6ce-4b59-9168-8fbba8b09cc6-dshm\") pod \"test-model-c903aa-kserve-74c5db6cf9-5jbph\" (UID: \"d6554292-b6ce-4b59-9168-8fbba8b09cc6\") " pod="ai-tenant-e2e-inf-b-c903aa/test-model-c903aa-kserve-74c5db6cf9-5jbph" Jul 07 11:48:47.791480 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:48:47.791462 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d6554292-b6ce-4b59-9168-8fbba8b09cc6-tls-certs\") pod \"test-model-c903aa-kserve-74c5db6cf9-5jbph\" (UID: \"d6554292-b6ce-4b59-9168-8fbba8b09cc6\") " pod="ai-tenant-e2e-inf-b-c903aa/test-model-c903aa-kserve-74c5db6cf9-5jbph" Jul 07 11:48:47.798116 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:48:47.798084 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-lkfjq\" (UniqueName: \"kubernetes.io/projected/d6554292-b6ce-4b59-9168-8fbba8b09cc6-kube-api-access-lkfjq\") pod \"test-model-c903aa-kserve-74c5db6cf9-5jbph\" (UID: \"d6554292-b6ce-4b59-9168-8fbba8b09cc6\") " pod="ai-tenant-e2e-inf-b-c903aa/test-model-c903aa-kserve-74c5db6cf9-5jbph" Jul 07 11:48:47.912029 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:48:47.911939 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-c903aa/test-model-c903aa-kserve-74c5db6cf9-5jbph" Jul 07 11:48:48.043506 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:48:48.043484 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-b-c903aa/test-model-c903aa-kserve-74c5db6cf9-5jbph"] Jul 07 11:48:48.046220 ip-10-0-142-112 kubenswrapper[2578]: W0707 11:48:48.046192 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podd6554292_b6ce_4b59_9168_8fbba8b09cc6.slice/crio-14c380da358792b19d1497438857b4c3581cd20f8faaa8b00459fe49aefbd611 WatchSource:0}: Error finding container 14c380da358792b19d1497438857b4c3581cd20f8faaa8b00459fe49aefbd611: Status 404 returned error can't find the container with id 14c380da358792b19d1497438857b4c3581cd20f8faaa8b00459fe49aefbd611 Jul 07 11:48:48.689046 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:48:48.689011 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-c903aa/test-model-c903aa-kserve-74c5db6cf9-5jbph" event={"ID":"d6554292-b6ce-4b59-9168-8fbba8b09cc6","Type":"ContainerStarted","Data":"567485f74efade6c951b53cde7a114cc466001bb6d118fc5a067d1608a7ea4ab"} Jul 07 11:48:48.689046 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:48:48.689054 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-c903aa/test-model-c903aa-kserve-74c5db6cf9-5jbph" event={"ID":"d6554292-b6ce-4b59-9168-8fbba8b09cc6","Type":"ContainerStarted","Data":"14c380da358792b19d1497438857b4c3581cd20f8faaa8b00459fe49aefbd611"} Jul 07 11:48:48.714164 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:48:48.714116 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-inf-b-c903aa/test-model-c903aa-kserve-74c5db6cf9-5jbph" podStartSLOduration=1.7141015419999999 podStartE2EDuration="1.714101542s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 11:48:47 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 11:48:48.713750997 +0000 UTC m=+1973.009900586" watchObservedRunningTime="2026-07-07 11:48:48.714101542 +0000 UTC m=+1973.010251175" Jul 07 11:48:57.913038 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:48:57.912990 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-inf-b-c903aa/test-model-c903aa-kserve-74c5db6cf9-5jbph" Jul 07 11:48:57.913038 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:48:57.913044 2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-inf-b-c903aa/test-model-c903aa-kserve-74c5db6cf9-5jbph" Jul 07 11:48:57.925487 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:48:57.925458 2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-inf-b-c903aa/test-model-c903aa-kserve-74c5db6cf9-5jbph" Jul 07 11:48:58.742076 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:48:58.742043 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-inf-b-c903aa/test-model-c903aa-kserve-74c5db6cf9-5jbph" Jul 07 11:49:14.015496 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:49:14.015458 2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-b-c903aa/test-model-c903aa-kserve-74c5db6cf9-5jbph"] Jul 07 11:49:29.035336 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:49:29.035267 2578 kuberuntime_container.go:872] "Killing container with a grace period" pod="ai-tenant-e2e-inf-b-c903aa/test-model-c903aa-kserve-74c5db6cf9-5jbph" podUID="d6554292-b6ce-4b59-9168-8fbba8b09cc6" containerName="main" containerID="cri-o://567485f74efade6c951b53cde7a114cc466001bb6d118fc5a067d1608a7ea4ab" gracePeriod=45 Jul 07 11:49:29.284809 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:49:29.284785 2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-c903aa/test-model-c903aa-kserve-74c5db6cf9-5jbph" Jul 07 11:49:29.456082 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:49:29.456034 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d6554292-b6ce-4b59-9168-8fbba8b09cc6-model-cache\") pod \"d6554292-b6ce-4b59-9168-8fbba8b09cc6\" (UID: \"d6554292-b6ce-4b59-9168-8fbba8b09cc6\") " Jul 07 11:49:29.456274 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:49:29.456114 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-lkfjq\" (UniqueName: \"kubernetes.io/projected/d6554292-b6ce-4b59-9168-8fbba8b09cc6-kube-api-access-lkfjq\") pod \"d6554292-b6ce-4b59-9168-8fbba8b09cc6\" (UID: \"d6554292-b6ce-4b59-9168-8fbba8b09cc6\") " Jul 07 11:49:29.456274 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:49:29.456139 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d6554292-b6ce-4b59-9168-8fbba8b09cc6-home\") pod \"d6554292-b6ce-4b59-9168-8fbba8b09cc6\" (UID: \"d6554292-b6ce-4b59-9168-8fbba8b09cc6\") " Jul 07 11:49:29.456274 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:49:29.456173 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d6554292-b6ce-4b59-9168-8fbba8b09cc6-tls-certs\") pod \"d6554292-b6ce-4b59-9168-8fbba8b09cc6\" (UID: \"d6554292-b6ce-4b59-9168-8fbba8b09cc6\") " Jul 07 11:49:29.456274 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:49:29.456211 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d6554292-b6ce-4b59-9168-8fbba8b09cc6-dshm\") pod \"d6554292-b6ce-4b59-9168-8fbba8b09cc6\" (UID: \"d6554292-b6ce-4b59-9168-8fbba8b09cc6\") " Jul 07 11:49:29.456274 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:49:29.456239 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d6554292-b6ce-4b59-9168-8fbba8b09cc6-tmp-dir\") pod \"d6554292-b6ce-4b59-9168-8fbba8b09cc6\" (UID: \"d6554292-b6ce-4b59-9168-8fbba8b09cc6\") " Jul 07 11:49:29.456570 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:49:29.456365 2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d6554292-b6ce-4b59-9168-8fbba8b09cc6-model-cache" (OuterVolumeSpecName: "model-cache") pod "d6554292-b6ce-4b59-9168-8fbba8b09cc6" (UID: "d6554292-b6ce-4b59-9168-8fbba8b09cc6"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 11:49:29.456570 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:49:29.456489 2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d6554292-b6ce-4b59-9168-8fbba8b09cc6-home" (OuterVolumeSpecName: "home") pod "d6554292-b6ce-4b59-9168-8fbba8b09cc6" (UID: "d6554292-b6ce-4b59-9168-8fbba8b09cc6"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 11:49:29.456674 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:49:29.456602 2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d6554292-b6ce-4b59-9168-8fbba8b09cc6-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "d6554292-b6ce-4b59-9168-8fbba8b09cc6" (UID: "d6554292-b6ce-4b59-9168-8fbba8b09cc6"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 11:49:29.458355 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:49:29.458331 2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d6554292-b6ce-4b59-9168-8fbba8b09cc6-dshm" (OuterVolumeSpecName: "dshm") pod "d6554292-b6ce-4b59-9168-8fbba8b09cc6" (UID: "d6554292-b6ce-4b59-9168-8fbba8b09cc6"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 11:49:29.458414 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:49:29.458370 2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d6554292-b6ce-4b59-9168-8fbba8b09cc6-kube-api-access-lkfjq" (OuterVolumeSpecName: "kube-api-access-lkfjq") pod "d6554292-b6ce-4b59-9168-8fbba8b09cc6" (UID: "d6554292-b6ce-4b59-9168-8fbba8b09cc6"). InnerVolumeSpecName "kube-api-access-lkfjq". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 07 11:49:29.458475 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:49:29.458409 2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d6554292-b6ce-4b59-9168-8fbba8b09cc6-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "d6554292-b6ce-4b59-9168-8fbba8b09cc6" (UID: "d6554292-b6ce-4b59-9168-8fbba8b09cc6"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 07 11:49:29.557017 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:49:29.556969 2578 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d6554292-b6ce-4b59-9168-8fbba8b09cc6-tls-certs\") on node \"ip-10-0-142-112.ec2.internal\" DevicePath \"\"" Jul 07 11:49:29.557017 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:49:29.557012 2578 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d6554292-b6ce-4b59-9168-8fbba8b09cc6-dshm\") on node \"ip-10-0-142-112.ec2.internal\" DevicePath \"\"" Jul 07 11:49:29.557219 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:49:29.557044 2578 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d6554292-b6ce-4b59-9168-8fbba8b09cc6-tmp-dir\") on node \"ip-10-0-142-112.ec2.internal\" DevicePath \"\"" Jul 07 11:49:29.557219 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:49:29.557052 2578 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d6554292-b6ce-4b59-9168-8fbba8b09cc6-model-cache\") on node \"ip-10-0-142-112.ec2.internal\" DevicePath \"\"" Jul 07 11:49:29.557219 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:49:29.557061 2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-lkfjq\" (UniqueName: \"kubernetes.io/projected/d6554292-b6ce-4b59-9168-8fbba8b09cc6-kube-api-access-lkfjq\") on node \"ip-10-0-142-112.ec2.internal\" DevicePath \"\"" Jul 07 11:49:29.557219 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:49:29.557070 2578 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d6554292-b6ce-4b59-9168-8fbba8b09cc6-home\") on node \"ip-10-0-142-112.ec2.internal\" DevicePath \"\"" Jul 07 11:49:29.862634 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:49:29.862600 2578 generic.go:356] "Generic (PLEG): container finished" podID="d6554292-b6ce-4b59-9168-8fbba8b09cc6" containerID="567485f74efade6c951b53cde7a114cc466001bb6d118fc5a067d1608a7ea4ab" exitCode=0 Jul 07 11:49:29.862834 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:49:29.862641 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-c903aa/test-model-c903aa-kserve-74c5db6cf9-5jbph" event={"ID":"d6554292-b6ce-4b59-9168-8fbba8b09cc6","Type":"ContainerDied","Data":"567485f74efade6c951b53cde7a114cc466001bb6d118fc5a067d1608a7ea4ab"} Jul 07 11:49:29.862834 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:49:29.862668 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-c903aa/test-model-c903aa-kserve-74c5db6cf9-5jbph" event={"ID":"d6554292-b6ce-4b59-9168-8fbba8b09cc6","Type":"ContainerDied","Data":"14c380da358792b19d1497438857b4c3581cd20f8faaa8b00459fe49aefbd611"} Jul 07 11:49:29.862834 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:49:29.862642 2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-c903aa/test-model-c903aa-kserve-74c5db6cf9-5jbph" Jul 07 11:49:29.862834 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:49:29.862688 2578 scope.go:117] "RemoveContainer" containerID="567485f74efade6c951b53cde7a114cc466001bb6d118fc5a067d1608a7ea4ab" Jul 07 11:49:29.871123 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:49:29.871106 2578 scope.go:117] "RemoveContainer" containerID="567485f74efade6c951b53cde7a114cc466001bb6d118fc5a067d1608a7ea4ab" Jul 07 11:49:29.871365 ip-10-0-142-112 kubenswrapper[2578]: E0707 11:49:29.871344 2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"567485f74efade6c951b53cde7a114cc466001bb6d118fc5a067d1608a7ea4ab\": container with ID starting with 567485f74efade6c951b53cde7a114cc466001bb6d118fc5a067d1608a7ea4ab not found: ID does not exist" containerID="567485f74efade6c951b53cde7a114cc466001bb6d118fc5a067d1608a7ea4ab" Jul 07 11:49:29.871455 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:49:29.871377 2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"567485f74efade6c951b53cde7a114cc466001bb6d118fc5a067d1608a7ea4ab"} err="failed to get container status \"567485f74efade6c951b53cde7a114cc466001bb6d118fc5a067d1608a7ea4ab\": rpc error: code = NotFound desc = could not find container \"567485f74efade6c951b53cde7a114cc466001bb6d118fc5a067d1608a7ea4ab\": container with ID starting with 567485f74efade6c951b53cde7a114cc466001bb6d118fc5a067d1608a7ea4ab not found: ID does not exist" Jul 07 11:49:29.882034 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:49:29.881994 2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-b-c903aa/test-model-c903aa-kserve-74c5db6cf9-5jbph"] Jul 07 11:49:29.883965 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:49:29.883946 2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-inf-b-c903aa/test-model-c903aa-kserve-74c5db6cf9-5jbph"] Jul 07 11:49:30.307550 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:49:30.307517 2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d6554292-b6ce-4b59-9168-8fbba8b09cc6" path="/var/lib/kubelet/pods/d6554292-b6ce-4b59-9168-8fbba8b09cc6/volumes" Jul 07 11:51:34.900353 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:51:34.900319 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-b-97eb93/auth-test-model-97eb93-kserve-6db4fc4f87-xsssn"] Jul 07 11:51:34.900953 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:51:34.900648 2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d6554292-b6ce-4b59-9168-8fbba8b09cc6" containerName="main" Jul 07 11:51:34.900953 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:51:34.900660 2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="d6554292-b6ce-4b59-9168-8fbba8b09cc6" containerName="main" Jul 07 11:51:34.900953 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:51:34.900734 2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="d6554292-b6ce-4b59-9168-8fbba8b09cc6" containerName="main" Jul 07 11:51:34.903454 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:51:34.903433 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-97eb93/auth-test-model-97eb93-kserve-6db4fc4f87-xsssn" Jul 07 11:51:34.906491 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:51:34.906468 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-97eb93\"/\"default-dockercfg-ksdwn\"" Jul 07 11:51:34.906491 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:51:34.906476 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-97eb93\"/\"openshift-service-ca.crt\"" Jul 07 11:51:34.906686 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:51:34.906513 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-97eb93\"/\"auth-test-model-97eb93-kserve-self-signed-certs\"" Jul 07 11:51:34.906686 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:51:34.906523 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-97eb93\"/\"kube-root-ca.crt\"" Jul 07 11:51:34.914577 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:51:34.914555 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-97eb93/auth-test-model-97eb93-kserve-6db4fc4f87-xsssn"] Jul 07 11:51:34.946629 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:51:34.946595 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a4498081-d9f8-4267-b53b-72d52a3b9250-dshm\") pod \"auth-test-model-97eb93-kserve-6db4fc4f87-xsssn\" (UID: \"a4498081-d9f8-4267-b53b-72d52a3b9250\") " pod="ai-tenant-e2e-shared-b-97eb93/auth-test-model-97eb93-kserve-6db4fc4f87-xsssn" Jul 07 11:51:34.946629 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:51:34.946626 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a4498081-d9f8-4267-b53b-72d52a3b9250-tls-certs\") pod \"auth-test-model-97eb93-kserve-6db4fc4f87-xsssn\" (UID: \"a4498081-d9f8-4267-b53b-72d52a3b9250\") " pod="ai-tenant-e2e-shared-b-97eb93/auth-test-model-97eb93-kserve-6db4fc4f87-xsssn" Jul 07 11:51:34.946836 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:51:34.946675 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a4498081-d9f8-4267-b53b-72d52a3b9250-tmp-dir\") pod \"auth-test-model-97eb93-kserve-6db4fc4f87-xsssn\" (UID: \"a4498081-d9f8-4267-b53b-72d52a3b9250\") " pod="ai-tenant-e2e-shared-b-97eb93/auth-test-model-97eb93-kserve-6db4fc4f87-xsssn" Jul 07 11:51:34.946836 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:51:34.946695 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lfpq7\" (UniqueName: \"kubernetes.io/projected/a4498081-d9f8-4267-b53b-72d52a3b9250-kube-api-access-lfpq7\") pod \"auth-test-model-97eb93-kserve-6db4fc4f87-xsssn\" (UID: \"a4498081-d9f8-4267-b53b-72d52a3b9250\") " pod="ai-tenant-e2e-shared-b-97eb93/auth-test-model-97eb93-kserve-6db4fc4f87-xsssn" Jul 07 11:51:34.946836 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:51:34.946721 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a4498081-d9f8-4267-b53b-72d52a3b9250-home\") pod \"auth-test-model-97eb93-kserve-6db4fc4f87-xsssn\" (UID: \"a4498081-d9f8-4267-b53b-72d52a3b9250\") " pod="ai-tenant-e2e-shared-b-97eb93/auth-test-model-97eb93-kserve-6db4fc4f87-xsssn" Jul 07 11:51:34.946836 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:51:34.946783 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a4498081-d9f8-4267-b53b-72d52a3b9250-model-cache\") pod \"auth-test-model-97eb93-kserve-6db4fc4f87-xsssn\" (UID: \"a4498081-d9f8-4267-b53b-72d52a3b9250\") " pod="ai-tenant-e2e-shared-b-97eb93/auth-test-model-97eb93-kserve-6db4fc4f87-xsssn" Jul 07 11:51:35.048037 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:51:35.048000 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a4498081-d9f8-4267-b53b-72d52a3b9250-dshm\") pod \"auth-test-model-97eb93-kserve-6db4fc4f87-xsssn\" (UID: \"a4498081-d9f8-4267-b53b-72d52a3b9250\") " pod="ai-tenant-e2e-shared-b-97eb93/auth-test-model-97eb93-kserve-6db4fc4f87-xsssn" Jul 07 11:51:35.048037 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:51:35.048035 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a4498081-d9f8-4267-b53b-72d52a3b9250-tls-certs\") pod \"auth-test-model-97eb93-kserve-6db4fc4f87-xsssn\" (UID: \"a4498081-d9f8-4267-b53b-72d52a3b9250\") " pod="ai-tenant-e2e-shared-b-97eb93/auth-test-model-97eb93-kserve-6db4fc4f87-xsssn" Jul 07 11:51:35.048264 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:51:35.048072 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a4498081-d9f8-4267-b53b-72d52a3b9250-tmp-dir\") pod \"auth-test-model-97eb93-kserve-6db4fc4f87-xsssn\" (UID: \"a4498081-d9f8-4267-b53b-72d52a3b9250\") " pod="ai-tenant-e2e-shared-b-97eb93/auth-test-model-97eb93-kserve-6db4fc4f87-xsssn" Jul 07 11:51:35.048264 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:51:35.048091 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lfpq7\" (UniqueName: \"kubernetes.io/projected/a4498081-d9f8-4267-b53b-72d52a3b9250-kube-api-access-lfpq7\") pod \"auth-test-model-97eb93-kserve-6db4fc4f87-xsssn\" (UID: \"a4498081-d9f8-4267-b53b-72d52a3b9250\") " pod="ai-tenant-e2e-shared-b-97eb93/auth-test-model-97eb93-kserve-6db4fc4f87-xsssn" Jul 07 11:51:35.048264 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:51:35.048117 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a4498081-d9f8-4267-b53b-72d52a3b9250-home\") pod \"auth-test-model-97eb93-kserve-6db4fc4f87-xsssn\" (UID: \"a4498081-d9f8-4267-b53b-72d52a3b9250\") " pod="ai-tenant-e2e-shared-b-97eb93/auth-test-model-97eb93-kserve-6db4fc4f87-xsssn" Jul 07 11:51:35.048264 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:51:35.048140 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a4498081-d9f8-4267-b53b-72d52a3b9250-model-cache\") pod \"auth-test-model-97eb93-kserve-6db4fc4f87-xsssn\" (UID: \"a4498081-d9f8-4267-b53b-72d52a3b9250\") " pod="ai-tenant-e2e-shared-b-97eb93/auth-test-model-97eb93-kserve-6db4fc4f87-xsssn" Jul 07 11:51:35.048509 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:51:35.048486 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a4498081-d9f8-4267-b53b-72d52a3b9250-home\") pod \"auth-test-model-97eb93-kserve-6db4fc4f87-xsssn\" (UID: \"a4498081-d9f8-4267-b53b-72d52a3b9250\") " pod="ai-tenant-e2e-shared-b-97eb93/auth-test-model-97eb93-kserve-6db4fc4f87-xsssn" Jul 07 11:51:35.048572 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:51:35.048528 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a4498081-d9f8-4267-b53b-72d52a3b9250-tmp-dir\") pod \"auth-test-model-97eb93-kserve-6db4fc4f87-xsssn\" (UID: \"a4498081-d9f8-4267-b53b-72d52a3b9250\") " pod="ai-tenant-e2e-shared-b-97eb93/auth-test-model-97eb93-kserve-6db4fc4f87-xsssn" Jul 07 11:51:35.048630 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:51:35.048584 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a4498081-d9f8-4267-b53b-72d52a3b9250-model-cache\") pod \"auth-test-model-97eb93-kserve-6db4fc4f87-xsssn\" (UID: \"a4498081-d9f8-4267-b53b-72d52a3b9250\") " pod="ai-tenant-e2e-shared-b-97eb93/auth-test-model-97eb93-kserve-6db4fc4f87-xsssn" Jul 07 11:51:35.050493 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:51:35.050473 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a4498081-d9f8-4267-b53b-72d52a3b9250-dshm\") pod \"auth-test-model-97eb93-kserve-6db4fc4f87-xsssn\" (UID: \"a4498081-d9f8-4267-b53b-72d52a3b9250\") " pod="ai-tenant-e2e-shared-b-97eb93/auth-test-model-97eb93-kserve-6db4fc4f87-xsssn" Jul 07 11:51:35.050592 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:51:35.050577 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a4498081-d9f8-4267-b53b-72d52a3b9250-tls-certs\") pod \"auth-test-model-97eb93-kserve-6db4fc4f87-xsssn\" (UID: \"a4498081-d9f8-4267-b53b-72d52a3b9250\") " pod="ai-tenant-e2e-shared-b-97eb93/auth-test-model-97eb93-kserve-6db4fc4f87-xsssn" Jul 07 11:51:35.057794 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:51:35.057774 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-lfpq7\" (UniqueName: \"kubernetes.io/projected/a4498081-d9f8-4267-b53b-72d52a3b9250-kube-api-access-lfpq7\") pod \"auth-test-model-97eb93-kserve-6db4fc4f87-xsssn\" (UID: \"a4498081-d9f8-4267-b53b-72d52a3b9250\") " pod="ai-tenant-e2e-shared-b-97eb93/auth-test-model-97eb93-kserve-6db4fc4f87-xsssn" Jul 07 11:51:35.213464 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:51:35.213347 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-97eb93/auth-test-model-97eb93-kserve-6db4fc4f87-xsssn" Jul 07 11:51:35.350374 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:51:35.350342 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-97eb93/auth-test-model-97eb93-kserve-6db4fc4f87-xsssn"] Jul 07 11:51:35.353584 ip-10-0-142-112 kubenswrapper[2578]: W0707 11:51:35.353549 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-poda4498081_d9f8_4267_b53b_72d52a3b9250.slice/crio-4b089793f36cdcee71edc08d76d46782ee29abea0f548254f1b448c6cb8036ed WatchSource:0}: Error finding container 4b089793f36cdcee71edc08d76d46782ee29abea0f548254f1b448c6cb8036ed: Status 404 returned error can't find the container with id 4b089793f36cdcee71edc08d76d46782ee29abea0f548254f1b448c6cb8036ed Jul 07 11:51:35.391629 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:51:35.391600 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-97eb93/auth-test-model-97eb93-kserve-6db4fc4f87-xsssn" event={"ID":"a4498081-d9f8-4267-b53b-72d52a3b9250","Type":"ContainerStarted","Data":"4b089793f36cdcee71edc08d76d46782ee29abea0f548254f1b448c6cb8036ed"} Jul 07 11:51:36.397702 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:51:36.397657 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-97eb93/auth-test-model-97eb93-kserve-6db4fc4f87-xsssn" event={"ID":"a4498081-d9f8-4267-b53b-72d52a3b9250","Type":"ContainerStarted","Data":"6455be92c9ccf0895568f408800fe94e7496beab10f116a41644a016acd1510b"} Jul 07 11:51:36.417286 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:51:36.417235 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-b-97eb93/auth-test-model-97eb93-kserve-6db4fc4f87-xsssn" podStartSLOduration=2.417218809 podStartE2EDuration="2.417218809s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 11:51:34 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 11:51:36.415947282 +0000 UTC m=+2140.712096871" watchObservedRunningTime="2026-07-07 11:51:36.417218809 +0000 UTC m=+2140.713368427" Jul 07 11:51:45.213558 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:51:45.213522 2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-b-97eb93/auth-test-model-97eb93-kserve-6db4fc4f87-xsssn" Jul 07 11:51:45.214001 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:51:45.213593 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-b-97eb93/auth-test-model-97eb93-kserve-6db4fc4f87-xsssn" Jul 07 11:51:45.226065 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:51:45.226041 2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-b-97eb93/auth-test-model-97eb93-kserve-6db4fc4f87-xsssn" Jul 07 11:51:45.448555 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:51:45.448526 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-b-97eb93/auth-test-model-97eb93-kserve-6db4fc4f87-xsssn" Jul 07 11:53:23.076521 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:53:23.076442 2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-97eb93/auth-test-model-97eb93-kserve-6db4fc4f87-xsssn"] Jul 07 11:53:38.096347 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:53:38.096281 2578 kuberuntime_container.go:872] "Killing container with a grace period" pod="ai-tenant-e2e-shared-b-97eb93/auth-test-model-97eb93-kserve-6db4fc4f87-xsssn" podUID="a4498081-d9f8-4267-b53b-72d52a3b9250" containerName="main" containerID="cri-o://6455be92c9ccf0895568f408800fe94e7496beab10f116a41644a016acd1510b" gracePeriod=45 Jul 07 11:53:38.345873 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:53:38.345847 2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-97eb93/auth-test-model-97eb93-kserve-6db4fc4f87-xsssn" Jul 07 11:53:38.413617 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:53:38.413527 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a4498081-d9f8-4267-b53b-72d52a3b9250-dshm\") pod \"a4498081-d9f8-4267-b53b-72d52a3b9250\" (UID: \"a4498081-d9f8-4267-b53b-72d52a3b9250\") " Jul 07 11:53:38.413617 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:53:38.413578 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a4498081-d9f8-4267-b53b-72d52a3b9250-tls-certs\") pod \"a4498081-d9f8-4267-b53b-72d52a3b9250\" (UID: \"a4498081-d9f8-4267-b53b-72d52a3b9250\") " Jul 07 11:53:38.413849 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:53:38.413642 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a4498081-d9f8-4267-b53b-72d52a3b9250-tmp-dir\") pod \"a4498081-d9f8-4267-b53b-72d52a3b9250\" (UID: \"a4498081-d9f8-4267-b53b-72d52a3b9250\") " Jul 07 11:53:38.413849 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:53:38.413668 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a4498081-d9f8-4267-b53b-72d52a3b9250-model-cache\") pod \"a4498081-d9f8-4267-b53b-72d52a3b9250\" (UID: \"a4498081-d9f8-4267-b53b-72d52a3b9250\") " Jul 07 11:53:38.413849 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:53:38.413699 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-lfpq7\" (UniqueName: \"kubernetes.io/projected/a4498081-d9f8-4267-b53b-72d52a3b9250-kube-api-access-lfpq7\") pod \"a4498081-d9f8-4267-b53b-72d52a3b9250\" (UID: \"a4498081-d9f8-4267-b53b-72d52a3b9250\") " Jul 07 11:53:38.413849 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:53:38.413776 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a4498081-d9f8-4267-b53b-72d52a3b9250-home\") pod \"a4498081-d9f8-4267-b53b-72d52a3b9250\" (UID: \"a4498081-d9f8-4267-b53b-72d52a3b9250\") " Jul 07 11:53:38.414050 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:53:38.413947 2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a4498081-d9f8-4267-b53b-72d52a3b9250-model-cache" (OuterVolumeSpecName: "model-cache") pod "a4498081-d9f8-4267-b53b-72d52a3b9250" (UID: "a4498081-d9f8-4267-b53b-72d52a3b9250"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 11:53:38.414050 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:53:38.413961 2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a4498081-d9f8-4267-b53b-72d52a3b9250-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "a4498081-d9f8-4267-b53b-72d52a3b9250" (UID: "a4498081-d9f8-4267-b53b-72d52a3b9250"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 11:53:38.414157 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:53:38.414053 2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a4498081-d9f8-4267-b53b-72d52a3b9250-home" (OuterVolumeSpecName: "home") pod "a4498081-d9f8-4267-b53b-72d52a3b9250" (UID: "a4498081-d9f8-4267-b53b-72d52a3b9250"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 11:53:38.414214 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:53:38.414154 2578 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a4498081-d9f8-4267-b53b-72d52a3b9250-tmp-dir\") on node \"ip-10-0-142-112.ec2.internal\" DevicePath \"\"" Jul 07 11:53:38.414214 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:53:38.414172 2578 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a4498081-d9f8-4267-b53b-72d52a3b9250-model-cache\") on node \"ip-10-0-142-112.ec2.internal\" DevicePath \"\"" Jul 07 11:53:38.414214 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:53:38.414184 2578 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a4498081-d9f8-4267-b53b-72d52a3b9250-home\") on node \"ip-10-0-142-112.ec2.internal\" DevicePath \"\"" Jul 07 11:53:38.415933 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:53:38.415903 2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a4498081-d9f8-4267-b53b-72d52a3b9250-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "a4498081-d9f8-4267-b53b-72d52a3b9250" (UID: "a4498081-d9f8-4267-b53b-72d52a3b9250"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 07 11:53:38.416044 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:53:38.415943 2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a4498081-d9f8-4267-b53b-72d52a3b9250-dshm" (OuterVolumeSpecName: "dshm") pod "a4498081-d9f8-4267-b53b-72d52a3b9250" (UID: "a4498081-d9f8-4267-b53b-72d52a3b9250"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 11:53:38.416044 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:53:38.415952 2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a4498081-d9f8-4267-b53b-72d52a3b9250-kube-api-access-lfpq7" (OuterVolumeSpecName: "kube-api-access-lfpq7") pod "a4498081-d9f8-4267-b53b-72d52a3b9250" (UID: "a4498081-d9f8-4267-b53b-72d52a3b9250"). InnerVolumeSpecName "kube-api-access-lfpq7". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 07 11:53:38.515583 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:53:38.515546 2578 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a4498081-d9f8-4267-b53b-72d52a3b9250-dshm\") on node \"ip-10-0-142-112.ec2.internal\" DevicePath \"\"" Jul 07 11:53:38.515583 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:53:38.515577 2578 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a4498081-d9f8-4267-b53b-72d52a3b9250-tls-certs\") on node \"ip-10-0-142-112.ec2.internal\" DevicePath \"\"" Jul 07 11:53:38.515769 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:53:38.515591 2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-lfpq7\" (UniqueName: \"kubernetes.io/projected/a4498081-d9f8-4267-b53b-72d52a3b9250-kube-api-access-lfpq7\") on node \"ip-10-0-142-112.ec2.internal\" DevicePath \"\"" Jul 07 11:53:38.940143 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:53:38.940107 2578 generic.go:356] "Generic (PLEG): container finished" podID="a4498081-d9f8-4267-b53b-72d52a3b9250" containerID="6455be92c9ccf0895568f408800fe94e7496beab10f116a41644a016acd1510b" exitCode=0 Jul 07 11:53:38.940143 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:53:38.940144 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-97eb93/auth-test-model-97eb93-kserve-6db4fc4f87-xsssn" event={"ID":"a4498081-d9f8-4267-b53b-72d52a3b9250","Type":"ContainerDied","Data":"6455be92c9ccf0895568f408800fe94e7496beab10f116a41644a016acd1510b"} Jul 07 11:53:38.940143 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:53:38.940148 2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-97eb93/auth-test-model-97eb93-kserve-6db4fc4f87-xsssn" Jul 07 11:53:38.940474 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:53:38.940171 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-97eb93/auth-test-model-97eb93-kserve-6db4fc4f87-xsssn" event={"ID":"a4498081-d9f8-4267-b53b-72d52a3b9250","Type":"ContainerDied","Data":"4b089793f36cdcee71edc08d76d46782ee29abea0f548254f1b448c6cb8036ed"} Jul 07 11:53:38.940474 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:53:38.940187 2578 scope.go:117] "RemoveContainer" containerID="6455be92c9ccf0895568f408800fe94e7496beab10f116a41644a016acd1510b" Jul 07 11:53:38.948568 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:53:38.948553 2578 scope.go:117] "RemoveContainer" containerID="6455be92c9ccf0895568f408800fe94e7496beab10f116a41644a016acd1510b" Jul 07 11:53:38.948814 ip-10-0-142-112 kubenswrapper[2578]: E0707 11:53:38.948793 2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6455be92c9ccf0895568f408800fe94e7496beab10f116a41644a016acd1510b\": container with ID starting with 6455be92c9ccf0895568f408800fe94e7496beab10f116a41644a016acd1510b not found: ID does not exist" containerID="6455be92c9ccf0895568f408800fe94e7496beab10f116a41644a016acd1510b" Jul 07 11:53:38.948856 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:53:38.948822 2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6455be92c9ccf0895568f408800fe94e7496beab10f116a41644a016acd1510b"} err="failed to get container status \"6455be92c9ccf0895568f408800fe94e7496beab10f116a41644a016acd1510b\": rpc error: code = NotFound desc = could not find container \"6455be92c9ccf0895568f408800fe94e7496beab10f116a41644a016acd1510b\": container with ID starting with 6455be92c9ccf0895568f408800fe94e7496beab10f116a41644a016acd1510b not found: ID does not exist" Jul 07 11:53:38.965326 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:53:38.965302 2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-97eb93/auth-test-model-97eb93-kserve-6db4fc4f87-xsssn"] Jul 07 11:53:38.974536 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:53:38.974506 2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-b-97eb93/auth-test-model-97eb93-kserve-6db4fc4f87-xsssn"] Jul 07 11:53:40.307385 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:53:40.307351 2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a4498081-d9f8-4267-b53b-72d52a3b9250" path="/var/lib/kubelet/pods/a4498081-d9f8-4267-b53b-72d52a3b9250/volumes" Jul 07 11:54:03.397070 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:54:03.397037 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-b-97eb93/sub-test-model-97eb93-kserve-6cd4bd578f-mqtkc"] Jul 07 11:54:03.397464 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:54:03.397352 2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a4498081-d9f8-4267-b53b-72d52a3b9250" containerName="main" Jul 07 11:54:03.397464 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:54:03.397363 2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="a4498081-d9f8-4267-b53b-72d52a3b9250" containerName="main" Jul 07 11:54:03.397464 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:54:03.397445 2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="a4498081-d9f8-4267-b53b-72d52a3b9250" containerName="main" Jul 07 11:54:03.399923 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:54:03.399906 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-97eb93/sub-test-model-97eb93-kserve-6cd4bd578f-mqtkc" Jul 07 11:54:03.403072 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:54:03.403047 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-97eb93\"/\"sub-test-model-97eb93-kserve-self-signed-certs\"" Jul 07 11:54:03.403072 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:54:03.403060 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-97eb93\"/\"default-dockercfg-ksdwn\"" Jul 07 11:54:03.403072 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:54:03.403070 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-97eb93\"/\"kube-root-ca.crt\"" Jul 07 11:54:03.403445 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:54:03.403069 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-97eb93\"/\"openshift-service-ca.crt\"" Jul 07 11:54:03.409621 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:54:03.409604 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-97eb93/sub-test-model-97eb93-kserve-6cd4bd578f-mqtkc"] Jul 07 11:54:03.417098 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:54:03.417073 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e042a319-8b47-4d47-928f-47724a8af11a-tls-certs\") pod \"sub-test-model-97eb93-kserve-6cd4bd578f-mqtkc\" (UID: \"e042a319-8b47-4d47-928f-47724a8af11a\") " pod="ai-tenant-e2e-shared-b-97eb93/sub-test-model-97eb93-kserve-6cd4bd578f-mqtkc" Jul 07 11:54:03.417247 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:54:03.417112 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e042a319-8b47-4d47-928f-47724a8af11a-model-cache\") pod \"sub-test-model-97eb93-kserve-6cd4bd578f-mqtkc\" (UID: \"e042a319-8b47-4d47-928f-47724a8af11a\") " pod="ai-tenant-e2e-shared-b-97eb93/sub-test-model-97eb93-kserve-6cd4bd578f-mqtkc" Jul 07 11:54:03.417247 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:54:03.417147 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e042a319-8b47-4d47-928f-47724a8af11a-tmp-dir\") pod \"sub-test-model-97eb93-kserve-6cd4bd578f-mqtkc\" (UID: \"e042a319-8b47-4d47-928f-47724a8af11a\") " pod="ai-tenant-e2e-shared-b-97eb93/sub-test-model-97eb93-kserve-6cd4bd578f-mqtkc" Jul 07 11:54:03.417247 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:54:03.417170 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/e042a319-8b47-4d47-928f-47724a8af11a-dshm\") pod \"sub-test-model-97eb93-kserve-6cd4bd578f-mqtkc\" (UID: \"e042a319-8b47-4d47-928f-47724a8af11a\") " pod="ai-tenant-e2e-shared-b-97eb93/sub-test-model-97eb93-kserve-6cd4bd578f-mqtkc" Jul 07 11:54:03.417247 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:54:03.417195 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/e042a319-8b47-4d47-928f-47724a8af11a-home\") pod \"sub-test-model-97eb93-kserve-6cd4bd578f-mqtkc\" (UID: \"e042a319-8b47-4d47-928f-47724a8af11a\") " pod="ai-tenant-e2e-shared-b-97eb93/sub-test-model-97eb93-kserve-6cd4bd578f-mqtkc" Jul 07 11:54:03.417247 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:54:03.417216 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-55p6f\" (UniqueName: \"kubernetes.io/projected/e042a319-8b47-4d47-928f-47724a8af11a-kube-api-access-55p6f\") pod \"sub-test-model-97eb93-kserve-6cd4bd578f-mqtkc\" (UID: \"e042a319-8b47-4d47-928f-47724a8af11a\") " pod="ai-tenant-e2e-shared-b-97eb93/sub-test-model-97eb93-kserve-6cd4bd578f-mqtkc" Jul 07 11:54:03.517624 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:54:03.517579 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e042a319-8b47-4d47-928f-47724a8af11a-tmp-dir\") pod \"sub-test-model-97eb93-kserve-6cd4bd578f-mqtkc\" (UID: \"e042a319-8b47-4d47-928f-47724a8af11a\") " pod="ai-tenant-e2e-shared-b-97eb93/sub-test-model-97eb93-kserve-6cd4bd578f-mqtkc" Jul 07 11:54:03.517800 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:54:03.517627 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/e042a319-8b47-4d47-928f-47724a8af11a-dshm\") pod \"sub-test-model-97eb93-kserve-6cd4bd578f-mqtkc\" (UID: \"e042a319-8b47-4d47-928f-47724a8af11a\") " pod="ai-tenant-e2e-shared-b-97eb93/sub-test-model-97eb93-kserve-6cd4bd578f-mqtkc" Jul 07 11:54:03.517800 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:54:03.517672 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/e042a319-8b47-4d47-928f-47724a8af11a-home\") pod \"sub-test-model-97eb93-kserve-6cd4bd578f-mqtkc\" (UID: \"e042a319-8b47-4d47-928f-47724a8af11a\") " pod="ai-tenant-e2e-shared-b-97eb93/sub-test-model-97eb93-kserve-6cd4bd578f-mqtkc" Jul 07 11:54:03.517800 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:54:03.517694 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-55p6f\" (UniqueName: \"kubernetes.io/projected/e042a319-8b47-4d47-928f-47724a8af11a-kube-api-access-55p6f\") pod \"sub-test-model-97eb93-kserve-6cd4bd578f-mqtkc\" (UID: \"e042a319-8b47-4d47-928f-47724a8af11a\") " pod="ai-tenant-e2e-shared-b-97eb93/sub-test-model-97eb93-kserve-6cd4bd578f-mqtkc" Jul 07 11:54:03.517959 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:54:03.517852 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e042a319-8b47-4d47-928f-47724a8af11a-tls-certs\") pod \"sub-test-model-97eb93-kserve-6cd4bd578f-mqtkc\" (UID: \"e042a319-8b47-4d47-928f-47724a8af11a\") " pod="ai-tenant-e2e-shared-b-97eb93/sub-test-model-97eb93-kserve-6cd4bd578f-mqtkc" Jul 07 11:54:03.517959 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:54:03.517934 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e042a319-8b47-4d47-928f-47724a8af11a-model-cache\") pod \"sub-test-model-97eb93-kserve-6cd4bd578f-mqtkc\" (UID: \"e042a319-8b47-4d47-928f-47724a8af11a\") " pod="ai-tenant-e2e-shared-b-97eb93/sub-test-model-97eb93-kserve-6cd4bd578f-mqtkc" Jul 07 11:54:03.518057 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:54:03.518016 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e042a319-8b47-4d47-928f-47724a8af11a-tmp-dir\") pod \"sub-test-model-97eb93-kserve-6cd4bd578f-mqtkc\" (UID: \"e042a319-8b47-4d47-928f-47724a8af11a\") " pod="ai-tenant-e2e-shared-b-97eb93/sub-test-model-97eb93-kserve-6cd4bd578f-mqtkc" Jul 07 11:54:03.518097 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:54:03.518082 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/e042a319-8b47-4d47-928f-47724a8af11a-home\") pod \"sub-test-model-97eb93-kserve-6cd4bd578f-mqtkc\" (UID: \"e042a319-8b47-4d47-928f-47724a8af11a\") " pod="ai-tenant-e2e-shared-b-97eb93/sub-test-model-97eb93-kserve-6cd4bd578f-mqtkc" Jul 07 11:54:03.518228 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:54:03.518207 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e042a319-8b47-4d47-928f-47724a8af11a-model-cache\") pod \"sub-test-model-97eb93-kserve-6cd4bd578f-mqtkc\" (UID: \"e042a319-8b47-4d47-928f-47724a8af11a\") " pod="ai-tenant-e2e-shared-b-97eb93/sub-test-model-97eb93-kserve-6cd4bd578f-mqtkc" Jul 07 11:54:03.519761 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:54:03.519740 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/e042a319-8b47-4d47-928f-47724a8af11a-dshm\") pod \"sub-test-model-97eb93-kserve-6cd4bd578f-mqtkc\" (UID: \"e042a319-8b47-4d47-928f-47724a8af11a\") " pod="ai-tenant-e2e-shared-b-97eb93/sub-test-model-97eb93-kserve-6cd4bd578f-mqtkc" Jul 07 11:54:03.520086 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:54:03.520071 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e042a319-8b47-4d47-928f-47724a8af11a-tls-certs\") pod \"sub-test-model-97eb93-kserve-6cd4bd578f-mqtkc\" (UID: \"e042a319-8b47-4d47-928f-47724a8af11a\") " pod="ai-tenant-e2e-shared-b-97eb93/sub-test-model-97eb93-kserve-6cd4bd578f-mqtkc" Jul 07 11:54:03.526517 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:54:03.526495 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-55p6f\" (UniqueName: \"kubernetes.io/projected/e042a319-8b47-4d47-928f-47724a8af11a-kube-api-access-55p6f\") pod \"sub-test-model-97eb93-kserve-6cd4bd578f-mqtkc\" (UID: \"e042a319-8b47-4d47-928f-47724a8af11a\") " pod="ai-tenant-e2e-shared-b-97eb93/sub-test-model-97eb93-kserve-6cd4bd578f-mqtkc" Jul 07 11:54:03.710627 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:54:03.710526 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-97eb93/sub-test-model-97eb93-kserve-6cd4bd578f-mqtkc" Jul 07 11:54:03.838178 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:54:03.838152 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-97eb93/sub-test-model-97eb93-kserve-6cd4bd578f-mqtkc"] Jul 07 11:54:03.840866 ip-10-0-142-112 kubenswrapper[2578]: W0707 11:54:03.840840 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pode042a319_8b47_4d47_928f_47724a8af11a.slice/crio-7b34175828290eebaa8401ca9c8c32c3f1cb5b55de629313e6aeed33f8a3221e WatchSource:0}: Error finding container 7b34175828290eebaa8401ca9c8c32c3f1cb5b55de629313e6aeed33f8a3221e: Status 404 returned error can't find the container with id 7b34175828290eebaa8401ca9c8c32c3f1cb5b55de629313e6aeed33f8a3221e Jul 07 11:54:04.045064 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:54:04.044968 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-97eb93/sub-test-model-97eb93-kserve-6cd4bd578f-mqtkc" event={"ID":"e042a319-8b47-4d47-928f-47724a8af11a","Type":"ContainerStarted","Data":"b5d74ce52b4dc64def13cc8aee7b8616d6078fd95d5881cd7a9d6e3c988f5950"} Jul 07 11:54:04.045064 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:54:04.045013 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-97eb93/sub-test-model-97eb93-kserve-6cd4bd578f-mqtkc" event={"ID":"e042a319-8b47-4d47-928f-47724a8af11a","Type":"ContainerStarted","Data":"7b34175828290eebaa8401ca9c8c32c3f1cb5b55de629313e6aeed33f8a3221e"} Jul 07 11:54:04.064174 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:54:04.064126 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-b-97eb93/sub-test-model-97eb93-kserve-6cd4bd578f-mqtkc" podStartSLOduration=1.064112075 podStartE2EDuration="1.064112075s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 11:54:03 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 11:54:04.062879712 +0000 UTC m=+2288.359029300" watchObservedRunningTime="2026-07-07 11:54:04.064112075 +0000 UTC m=+2288.360261663" Jul 07 11:54:13.710872 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:54:13.710839 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-b-97eb93/sub-test-model-97eb93-kserve-6cd4bd578f-mqtkc" Jul 07 11:54:13.711360 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:54:13.711147 2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-b-97eb93/sub-test-model-97eb93-kserve-6cd4bd578f-mqtkc" Jul 07 11:54:13.723513 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:54:13.723490 2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-b-97eb93/sub-test-model-97eb93-kserve-6cd4bd578f-mqtkc" Jul 07 11:54:14.099461 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:54:14.099410 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-b-97eb93/sub-test-model-97eb93-kserve-6cd4bd578f-mqtkc" Jul 07 11:54:37.605483 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:54:37.605448 2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-97eb93/sub-test-model-97eb93-kserve-6cd4bd578f-mqtkc"] Jul 07 11:54:52.624900 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:54:52.624784 2578 kuberuntime_container.go:872] "Killing container with a grace period" pod="ai-tenant-e2e-shared-b-97eb93/sub-test-model-97eb93-kserve-6cd4bd578f-mqtkc" podUID="e042a319-8b47-4d47-928f-47724a8af11a" containerName="main" containerID="cri-o://b5d74ce52b4dc64def13cc8aee7b8616d6078fd95d5881cd7a9d6e3c988f5950" gracePeriod=45 Jul 07 11:54:52.862693 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:54:52.862666 2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-97eb93/sub-test-model-97eb93-kserve-6cd4bd578f-mqtkc" Jul 07 11:54:52.948393 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:54:52.948297 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e042a319-8b47-4d47-928f-47724a8af11a-model-cache\") pod \"e042a319-8b47-4d47-928f-47724a8af11a\" (UID: \"e042a319-8b47-4d47-928f-47724a8af11a\") " Jul 07 11:54:52.948393 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:54:52.948395 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e042a319-8b47-4d47-928f-47724a8af11a-tls-certs\") pod \"e042a319-8b47-4d47-928f-47724a8af11a\" (UID: \"e042a319-8b47-4d47-928f-47724a8af11a\") " Jul 07 11:54:52.948607 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:54:52.948488 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/e042a319-8b47-4d47-928f-47724a8af11a-dshm\") pod \"e042a319-8b47-4d47-928f-47724a8af11a\" (UID: \"e042a319-8b47-4d47-928f-47724a8af11a\") " Jul 07 11:54:52.948607 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:54:52.948524 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-55p6f\" (UniqueName: \"kubernetes.io/projected/e042a319-8b47-4d47-928f-47724a8af11a-kube-api-access-55p6f\") pod \"e042a319-8b47-4d47-928f-47724a8af11a\" (UID: \"e042a319-8b47-4d47-928f-47724a8af11a\") " Jul 07 11:54:52.948607 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:54:52.948567 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e042a319-8b47-4d47-928f-47724a8af11a-tmp-dir\") pod \"e042a319-8b47-4d47-928f-47724a8af11a\" (UID: \"e042a319-8b47-4d47-928f-47724a8af11a\") " Jul 07 11:54:52.948731 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:54:52.948617 2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e042a319-8b47-4d47-928f-47724a8af11a-model-cache" (OuterVolumeSpecName: "model-cache") pod "e042a319-8b47-4d47-928f-47724a8af11a" (UID: "e042a319-8b47-4d47-928f-47724a8af11a"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 11:54:52.948731 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:54:52.948624 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/e042a319-8b47-4d47-928f-47724a8af11a-home\") pod \"e042a319-8b47-4d47-928f-47724a8af11a\" (UID: \"e042a319-8b47-4d47-928f-47724a8af11a\") " Jul 07 11:54:52.948885 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:54:52.948837 2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e042a319-8b47-4d47-928f-47724a8af11a-home" (OuterVolumeSpecName: "home") pod "e042a319-8b47-4d47-928f-47724a8af11a" (UID: "e042a319-8b47-4d47-928f-47724a8af11a"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 11:54:52.949015 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:54:52.948902 2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e042a319-8b47-4d47-928f-47724a8af11a-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "e042a319-8b47-4d47-928f-47724a8af11a" (UID: "e042a319-8b47-4d47-928f-47724a8af11a"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 11:54:52.949015 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:54:52.948998 2578 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e042a319-8b47-4d47-928f-47724a8af11a-tmp-dir\") on node \"ip-10-0-142-112.ec2.internal\" DevicePath \"\"" Jul 07 11:54:52.949092 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:54:52.949019 2578 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/e042a319-8b47-4d47-928f-47724a8af11a-home\") on node \"ip-10-0-142-112.ec2.internal\" DevicePath \"\"" Jul 07 11:54:52.949092 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:54:52.949032 2578 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e042a319-8b47-4d47-928f-47724a8af11a-model-cache\") on node \"ip-10-0-142-112.ec2.internal\" DevicePath \"\"" Jul 07 11:54:52.950475 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:54:52.950449 2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e042a319-8b47-4d47-928f-47724a8af11a-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "e042a319-8b47-4d47-928f-47724a8af11a" (UID: "e042a319-8b47-4d47-928f-47724a8af11a"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 07 11:54:52.950603 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:54:52.950583 2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e042a319-8b47-4d47-928f-47724a8af11a-dshm" (OuterVolumeSpecName: "dshm") pod "e042a319-8b47-4d47-928f-47724a8af11a" (UID: "e042a319-8b47-4d47-928f-47724a8af11a"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 11:54:52.950691 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:54:52.950676 2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e042a319-8b47-4d47-928f-47724a8af11a-kube-api-access-55p6f" (OuterVolumeSpecName: "kube-api-access-55p6f") pod "e042a319-8b47-4d47-928f-47724a8af11a" (UID: "e042a319-8b47-4d47-928f-47724a8af11a"). InnerVolumeSpecName "kube-api-access-55p6f". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 07 11:54:53.050245 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:54:53.050208 2578 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e042a319-8b47-4d47-928f-47724a8af11a-tls-certs\") on node \"ip-10-0-142-112.ec2.internal\" DevicePath \"\"" Jul 07 11:54:53.050245 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:54:53.050240 2578 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/e042a319-8b47-4d47-928f-47724a8af11a-dshm\") on node \"ip-10-0-142-112.ec2.internal\" DevicePath \"\"" Jul 07 11:54:53.050245 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:54:53.050249 2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-55p6f\" (UniqueName: \"kubernetes.io/projected/e042a319-8b47-4d47-928f-47724a8af11a-kube-api-access-55p6f\") on node \"ip-10-0-142-112.ec2.internal\" DevicePath \"\"" Jul 07 11:54:53.256597 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:54:53.256506 2578 generic.go:356] "Generic (PLEG): container finished" podID="e042a319-8b47-4d47-928f-47724a8af11a" containerID="b5d74ce52b4dc64def13cc8aee7b8616d6078fd95d5881cd7a9d6e3c988f5950" exitCode=0 Jul 07 11:54:53.256597 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:54:53.256549 2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-97eb93/sub-test-model-97eb93-kserve-6cd4bd578f-mqtkc" Jul 07 11:54:53.256771 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:54:53.256591 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-97eb93/sub-test-model-97eb93-kserve-6cd4bd578f-mqtkc" event={"ID":"e042a319-8b47-4d47-928f-47724a8af11a","Type":"ContainerDied","Data":"b5d74ce52b4dc64def13cc8aee7b8616d6078fd95d5881cd7a9d6e3c988f5950"} Jul 07 11:54:53.256771 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:54:53.256633 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-97eb93/sub-test-model-97eb93-kserve-6cd4bd578f-mqtkc" event={"ID":"e042a319-8b47-4d47-928f-47724a8af11a","Type":"ContainerDied","Data":"7b34175828290eebaa8401ca9c8c32c3f1cb5b55de629313e6aeed33f8a3221e"} Jul 07 11:54:53.256771 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:54:53.256652 2578 scope.go:117] "RemoveContainer" containerID="b5d74ce52b4dc64def13cc8aee7b8616d6078fd95d5881cd7a9d6e3c988f5950" Jul 07 11:54:53.268769 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:54:53.268644 2578 scope.go:117] "RemoveContainer" containerID="b5d74ce52b4dc64def13cc8aee7b8616d6078fd95d5881cd7a9d6e3c988f5950" Jul 07 11:54:53.269175 ip-10-0-142-112 kubenswrapper[2578]: E0707 11:54:53.269149 2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b5d74ce52b4dc64def13cc8aee7b8616d6078fd95d5881cd7a9d6e3c988f5950\": container with ID starting with b5d74ce52b4dc64def13cc8aee7b8616d6078fd95d5881cd7a9d6e3c988f5950 not found: ID does not exist" containerID="b5d74ce52b4dc64def13cc8aee7b8616d6078fd95d5881cd7a9d6e3c988f5950" Jul 07 11:54:53.269260 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:54:53.269187 2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b5d74ce52b4dc64def13cc8aee7b8616d6078fd95d5881cd7a9d6e3c988f5950"} err="failed to get container status \"b5d74ce52b4dc64def13cc8aee7b8616d6078fd95d5881cd7a9d6e3c988f5950\": rpc error: code = NotFound desc = could not find container \"b5d74ce52b4dc64def13cc8aee7b8616d6078fd95d5881cd7a9d6e3c988f5950\": container with ID starting with b5d74ce52b4dc64def13cc8aee7b8616d6078fd95d5881cd7a9d6e3c988f5950 not found: ID does not exist" Jul 07 11:54:53.282539 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:54:53.282511 2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-97eb93/sub-test-model-97eb93-kserve-6cd4bd578f-mqtkc"] Jul 07 11:54:53.286269 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:54:53.286241 2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-b-97eb93/sub-test-model-97eb93-kserve-6cd4bd578f-mqtkc"] Jul 07 11:54:54.307172 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:54:54.307141 2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e042a319-8b47-4d47-928f-47724a8af11a" path="/var/lib/kubelet/pods/e042a319-8b47-4d47-928f-47724a8af11a/volumes" Jul 07 11:55:27.094744 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:55:27.094711 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-b-97eb93/rate-test-model-97eb93-kserve-7b96477b5f-f99qw"] Jul 07 11:55:27.095327 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:55:27.095072 2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e042a319-8b47-4d47-928f-47724a8af11a" containerName="main" Jul 07 11:55:27.095327 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:55:27.095091 2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="e042a319-8b47-4d47-928f-47724a8af11a" containerName="main" Jul 07 11:55:27.095327 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:55:27.095162 2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="e042a319-8b47-4d47-928f-47724a8af11a" containerName="main" Jul 07 11:55:27.098097 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:55:27.098078 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-97eb93/rate-test-model-97eb93-kserve-7b96477b5f-f99qw" Jul 07 11:55:27.101169 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:55:27.101153 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-97eb93\"/\"kube-root-ca.crt\"" Jul 07 11:55:27.101388 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:55:27.101364 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-97eb93\"/\"default-dockercfg-ksdwn\"" Jul 07 11:55:27.101532 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:55:27.101155 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-97eb93\"/\"rate-test-model-97eb93-kserve-self-signed-certs\"" Jul 07 11:55:27.101753 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:55:27.101730 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-97eb93\"/\"openshift-service-ca.crt\"" Jul 07 11:55:27.106019 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:55:27.105882 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-97eb93/rate-test-model-97eb93-kserve-7b96477b5f-f99qw"] Jul 07 11:55:27.253831 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:55:27.253798 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/9012964d-64f3-4a44-80ea-77f81b086465-home\") pod \"rate-test-model-97eb93-kserve-7b96477b5f-f99qw\" (UID: \"9012964d-64f3-4a44-80ea-77f81b086465\") " pod="ai-tenant-e2e-shared-b-97eb93/rate-test-model-97eb93-kserve-7b96477b5f-f99qw" Jul 07 11:55:27.253831 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:55:27.253842 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/9012964d-64f3-4a44-80ea-77f81b086465-tls-certs\") pod \"rate-test-model-97eb93-kserve-7b96477b5f-f99qw\" (UID: \"9012964d-64f3-4a44-80ea-77f81b086465\") " pod="ai-tenant-e2e-shared-b-97eb93/rate-test-model-97eb93-kserve-7b96477b5f-f99qw" Jul 07 11:55:27.254088 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:55:27.253868 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/9012964d-64f3-4a44-80ea-77f81b086465-tmp-dir\") pod \"rate-test-model-97eb93-kserve-7b96477b5f-f99qw\" (UID: \"9012964d-64f3-4a44-80ea-77f81b086465\") " pod="ai-tenant-e2e-shared-b-97eb93/rate-test-model-97eb93-kserve-7b96477b5f-f99qw" Jul 07 11:55:27.254088 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:55:27.253952 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qw886\" (UniqueName: \"kubernetes.io/projected/9012964d-64f3-4a44-80ea-77f81b086465-kube-api-access-qw886\") pod \"rate-test-model-97eb93-kserve-7b96477b5f-f99qw\" (UID: \"9012964d-64f3-4a44-80ea-77f81b086465\") " pod="ai-tenant-e2e-shared-b-97eb93/rate-test-model-97eb93-kserve-7b96477b5f-f99qw" Jul 07 11:55:27.254088 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:55:27.253991 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/9012964d-64f3-4a44-80ea-77f81b086465-dshm\") pod \"rate-test-model-97eb93-kserve-7b96477b5f-f99qw\" (UID: \"9012964d-64f3-4a44-80ea-77f81b086465\") " pod="ai-tenant-e2e-shared-b-97eb93/rate-test-model-97eb93-kserve-7b96477b5f-f99qw" Jul 07 11:55:27.254088 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:55:27.254058 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/9012964d-64f3-4a44-80ea-77f81b086465-model-cache\") pod \"rate-test-model-97eb93-kserve-7b96477b5f-f99qw\" (UID: \"9012964d-64f3-4a44-80ea-77f81b086465\") " pod="ai-tenant-e2e-shared-b-97eb93/rate-test-model-97eb93-kserve-7b96477b5f-f99qw" Jul 07 11:55:27.355214 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:55:27.355123 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/9012964d-64f3-4a44-80ea-77f81b086465-tls-certs\") pod \"rate-test-model-97eb93-kserve-7b96477b5f-f99qw\" (UID: \"9012964d-64f3-4a44-80ea-77f81b086465\") " pod="ai-tenant-e2e-shared-b-97eb93/rate-test-model-97eb93-kserve-7b96477b5f-f99qw" Jul 07 11:55:27.355214 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:55:27.355171 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/9012964d-64f3-4a44-80ea-77f81b086465-tmp-dir\") pod \"rate-test-model-97eb93-kserve-7b96477b5f-f99qw\" (UID: \"9012964d-64f3-4a44-80ea-77f81b086465\") " pod="ai-tenant-e2e-shared-b-97eb93/rate-test-model-97eb93-kserve-7b96477b5f-f99qw" Jul 07 11:55:27.355214 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:55:27.355204 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qw886\" (UniqueName: \"kubernetes.io/projected/9012964d-64f3-4a44-80ea-77f81b086465-kube-api-access-qw886\") pod \"rate-test-model-97eb93-kserve-7b96477b5f-f99qw\" (UID: \"9012964d-64f3-4a44-80ea-77f81b086465\") " pod="ai-tenant-e2e-shared-b-97eb93/rate-test-model-97eb93-kserve-7b96477b5f-f99qw" Jul 07 11:55:27.355514 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:55:27.355230 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/9012964d-64f3-4a44-80ea-77f81b086465-dshm\") pod \"rate-test-model-97eb93-kserve-7b96477b5f-f99qw\" (UID: \"9012964d-64f3-4a44-80ea-77f81b086465\") " pod="ai-tenant-e2e-shared-b-97eb93/rate-test-model-97eb93-kserve-7b96477b5f-f99qw" Jul 07 11:55:27.355514 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:55:27.355274 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/9012964d-64f3-4a44-80ea-77f81b086465-model-cache\") pod \"rate-test-model-97eb93-kserve-7b96477b5f-f99qw\" (UID: \"9012964d-64f3-4a44-80ea-77f81b086465\") " pod="ai-tenant-e2e-shared-b-97eb93/rate-test-model-97eb93-kserve-7b96477b5f-f99qw" Jul 07 11:55:27.355514 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:55:27.355355 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/9012964d-64f3-4a44-80ea-77f81b086465-home\") pod \"rate-test-model-97eb93-kserve-7b96477b5f-f99qw\" (UID: \"9012964d-64f3-4a44-80ea-77f81b086465\") " pod="ai-tenant-e2e-shared-b-97eb93/rate-test-model-97eb93-kserve-7b96477b5f-f99qw" Jul 07 11:55:27.355698 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:55:27.355673 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/9012964d-64f3-4a44-80ea-77f81b086465-tmp-dir\") pod \"rate-test-model-97eb93-kserve-7b96477b5f-f99qw\" (UID: \"9012964d-64f3-4a44-80ea-77f81b086465\") " pod="ai-tenant-e2e-shared-b-97eb93/rate-test-model-97eb93-kserve-7b96477b5f-f99qw" Jul 07 11:55:27.355769 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:55:27.355742 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/9012964d-64f3-4a44-80ea-77f81b086465-model-cache\") pod \"rate-test-model-97eb93-kserve-7b96477b5f-f99qw\" (UID: \"9012964d-64f3-4a44-80ea-77f81b086465\") " pod="ai-tenant-e2e-shared-b-97eb93/rate-test-model-97eb93-kserve-7b96477b5f-f99qw" Jul 07 11:55:27.355844 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:55:27.355821 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/9012964d-64f3-4a44-80ea-77f81b086465-home\") pod \"rate-test-model-97eb93-kserve-7b96477b5f-f99qw\" (UID: \"9012964d-64f3-4a44-80ea-77f81b086465\") " pod="ai-tenant-e2e-shared-b-97eb93/rate-test-model-97eb93-kserve-7b96477b5f-f99qw" Jul 07 11:55:27.357403 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:55:27.357384 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/9012964d-64f3-4a44-80ea-77f81b086465-dshm\") pod \"rate-test-model-97eb93-kserve-7b96477b5f-f99qw\" (UID: \"9012964d-64f3-4a44-80ea-77f81b086465\") " pod="ai-tenant-e2e-shared-b-97eb93/rate-test-model-97eb93-kserve-7b96477b5f-f99qw" Jul 07 11:55:27.357548 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:55:27.357531 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/9012964d-64f3-4a44-80ea-77f81b086465-tls-certs\") pod \"rate-test-model-97eb93-kserve-7b96477b5f-f99qw\" (UID: \"9012964d-64f3-4a44-80ea-77f81b086465\") " pod="ai-tenant-e2e-shared-b-97eb93/rate-test-model-97eb93-kserve-7b96477b5f-f99qw" Jul 07 11:55:27.363320 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:55:27.363296 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-qw886\" (UniqueName: \"kubernetes.io/projected/9012964d-64f3-4a44-80ea-77f81b086465-kube-api-access-qw886\") pod \"rate-test-model-97eb93-kserve-7b96477b5f-f99qw\" (UID: \"9012964d-64f3-4a44-80ea-77f81b086465\") " pod="ai-tenant-e2e-shared-b-97eb93/rate-test-model-97eb93-kserve-7b96477b5f-f99qw" Jul 07 11:55:27.409125 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:55:27.409094 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-97eb93/rate-test-model-97eb93-kserve-7b96477b5f-f99qw" Jul 07 11:55:27.536678 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:55:27.536640 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-97eb93/rate-test-model-97eb93-kserve-7b96477b5f-f99qw"] Jul 07 11:55:27.538921 ip-10-0-142-112 kubenswrapper[2578]: W0707 11:55:27.538894 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod9012964d_64f3_4a44_80ea_77f81b086465.slice/crio-b8db755ef0bf5d75c6ec8c140ae4d0e0126abbbabd309ee90b1b19de489c3b95 WatchSource:0}: Error finding container b8db755ef0bf5d75c6ec8c140ae4d0e0126abbbabd309ee90b1b19de489c3b95: Status 404 returned error can't find the container with id b8db755ef0bf5d75c6ec8c140ae4d0e0126abbbabd309ee90b1b19de489c3b95 Jul 07 11:55:28.412252 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:55:28.412215 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-97eb93/rate-test-model-97eb93-kserve-7b96477b5f-f99qw" event={"ID":"9012964d-64f3-4a44-80ea-77f81b086465","Type":"ContainerStarted","Data":"977bbf2cc363adbb6fe5b2cdb1270f64f40efc9869fd9e54487c9407ec66ac17"} Jul 07 11:55:28.412650 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:55:28.412259 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-97eb93/rate-test-model-97eb93-kserve-7b96477b5f-f99qw" event={"ID":"9012964d-64f3-4a44-80ea-77f81b086465","Type":"ContainerStarted","Data":"b8db755ef0bf5d75c6ec8c140ae4d0e0126abbbabd309ee90b1b19de489c3b95"} Jul 07 11:55:28.428639 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:55:28.428590 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-b-97eb93/rate-test-model-97eb93-kserve-7b96477b5f-f99qw" podStartSLOduration=1.428574878 podStartE2EDuration="1.428574878s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 11:55:27 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 11:55:28.42777153 +0000 UTC m=+2372.723921118" watchObservedRunningTime="2026-07-07 11:55:28.428574878 +0000 UTC m=+2372.724724466" Jul 07 11:55:37.410059 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:55:37.410009 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-b-97eb93/rate-test-model-97eb93-kserve-7b96477b5f-f99qw" Jul 07 11:55:37.410059 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:55:37.410074 2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-b-97eb93/rate-test-model-97eb93-kserve-7b96477b5f-f99qw" Jul 07 11:55:37.422124 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:55:37.422101 2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-b-97eb93/rate-test-model-97eb93-kserve-7b96477b5f-f99qw" Jul 07 11:55:37.462057 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:55:37.462031 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-b-97eb93/rate-test-model-97eb93-kserve-7b96477b5f-f99qw" Jul 07 11:56:17.797622 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:56:17.797531 2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-97eb93/rate-test-model-97eb93-kserve-7b96477b5f-f99qw"] Jul 07 11:56:32.817686 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:56:32.817633 2578 kuberuntime_container.go:872] "Killing container with a grace period" pod="ai-tenant-e2e-shared-b-97eb93/rate-test-model-97eb93-kserve-7b96477b5f-f99qw" podUID="9012964d-64f3-4a44-80ea-77f81b086465" containerName="main" containerID="cri-o://977bbf2cc363adbb6fe5b2cdb1270f64f40efc9869fd9e54487c9407ec66ac17" gracePeriod=45 Jul 07 11:56:33.072460 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:56:33.072384 2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-97eb93/rate-test-model-97eb93-kserve-7b96477b5f-f99qw" Jul 07 11:56:33.133748 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:56:33.133706 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-qw886\" (UniqueName: \"kubernetes.io/projected/9012964d-64f3-4a44-80ea-77f81b086465-kube-api-access-qw886\") pod \"9012964d-64f3-4a44-80ea-77f81b086465\" (UID: \"9012964d-64f3-4a44-80ea-77f81b086465\") " Jul 07 11:56:33.133955 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:56:33.133793 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/9012964d-64f3-4a44-80ea-77f81b086465-tls-certs\") pod \"9012964d-64f3-4a44-80ea-77f81b086465\" (UID: \"9012964d-64f3-4a44-80ea-77f81b086465\") " Jul 07 11:56:33.133955 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:56:33.133821 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/9012964d-64f3-4a44-80ea-77f81b086465-home\") pod \"9012964d-64f3-4a44-80ea-77f81b086465\" (UID: \"9012964d-64f3-4a44-80ea-77f81b086465\") " Jul 07 11:56:33.133955 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:56:33.133852 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/9012964d-64f3-4a44-80ea-77f81b086465-tmp-dir\") pod \"9012964d-64f3-4a44-80ea-77f81b086465\" (UID: \"9012964d-64f3-4a44-80ea-77f81b086465\") " Jul 07 11:56:33.133955 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:56:33.133886 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/9012964d-64f3-4a44-80ea-77f81b086465-model-cache\") pod \"9012964d-64f3-4a44-80ea-77f81b086465\" (UID: \"9012964d-64f3-4a44-80ea-77f81b086465\") " Jul 07 11:56:33.134150 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:56:33.133960 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/9012964d-64f3-4a44-80ea-77f81b086465-dshm\") pod \"9012964d-64f3-4a44-80ea-77f81b086465\" (UID: \"9012964d-64f3-4a44-80ea-77f81b086465\") " Jul 07 11:56:33.134150 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:56:33.134049 2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9012964d-64f3-4a44-80ea-77f81b086465-home" (OuterVolumeSpecName: "home") pod "9012964d-64f3-4a44-80ea-77f81b086465" (UID: "9012964d-64f3-4a44-80ea-77f81b086465"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 11:56:33.134150 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:56:33.134094 2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9012964d-64f3-4a44-80ea-77f81b086465-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "9012964d-64f3-4a44-80ea-77f81b086465" (UID: "9012964d-64f3-4a44-80ea-77f81b086465"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 11:56:33.134276 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:56:33.134180 2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9012964d-64f3-4a44-80ea-77f81b086465-model-cache" (OuterVolumeSpecName: "model-cache") pod "9012964d-64f3-4a44-80ea-77f81b086465" (UID: "9012964d-64f3-4a44-80ea-77f81b086465"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 11:56:33.134331 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:56:33.134309 2578 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/9012964d-64f3-4a44-80ea-77f81b086465-home\") on node \"ip-10-0-142-112.ec2.internal\" DevicePath \"\"" Jul 07 11:56:33.134331 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:56:33.134324 2578 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/9012964d-64f3-4a44-80ea-77f81b086465-tmp-dir\") on node \"ip-10-0-142-112.ec2.internal\" DevicePath \"\"" Jul 07 11:56:33.134460 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:56:33.134334 2578 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/9012964d-64f3-4a44-80ea-77f81b086465-model-cache\") on node \"ip-10-0-142-112.ec2.internal\" DevicePath \"\"" Jul 07 11:56:33.135942 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:56:33.135916 2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9012964d-64f3-4a44-80ea-77f81b086465-dshm" (OuterVolumeSpecName: "dshm") pod "9012964d-64f3-4a44-80ea-77f81b086465" (UID: "9012964d-64f3-4a44-80ea-77f81b086465"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 11:56:33.136047 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:56:33.135925 2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/9012964d-64f3-4a44-80ea-77f81b086465-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "9012964d-64f3-4a44-80ea-77f81b086465" (UID: "9012964d-64f3-4a44-80ea-77f81b086465"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 07 11:56:33.136047 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:56:33.135961 2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9012964d-64f3-4a44-80ea-77f81b086465-kube-api-access-qw886" (OuterVolumeSpecName: "kube-api-access-qw886") pod "9012964d-64f3-4a44-80ea-77f81b086465" (UID: "9012964d-64f3-4a44-80ea-77f81b086465"). InnerVolumeSpecName "kube-api-access-qw886". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 07 11:56:33.235441 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:56:33.235386 2578 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/9012964d-64f3-4a44-80ea-77f81b086465-tls-certs\") on node \"ip-10-0-142-112.ec2.internal\" DevicePath \"\"" Jul 07 11:56:33.235441 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:56:33.235414 2578 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/9012964d-64f3-4a44-80ea-77f81b086465-dshm\") on node \"ip-10-0-142-112.ec2.internal\" DevicePath \"\"" Jul 07 11:56:33.235642 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:56:33.235462 2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-qw886\" (UniqueName: \"kubernetes.io/projected/9012964d-64f3-4a44-80ea-77f81b086465-kube-api-access-qw886\") on node \"ip-10-0-142-112.ec2.internal\" DevicePath \"\"" Jul 07 11:56:33.692763 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:56:33.692724 2578 generic.go:356] "Generic (PLEG): container finished" podID="9012964d-64f3-4a44-80ea-77f81b086465" containerID="977bbf2cc363adbb6fe5b2cdb1270f64f40efc9869fd9e54487c9407ec66ac17" exitCode=0 Jul 07 11:56:33.692763 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:56:33.692767 2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-97eb93/rate-test-model-97eb93-kserve-7b96477b5f-f99qw" Jul 07 11:56:33.692974 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:56:33.692807 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-97eb93/rate-test-model-97eb93-kserve-7b96477b5f-f99qw" event={"ID":"9012964d-64f3-4a44-80ea-77f81b086465","Type":"ContainerDied","Data":"977bbf2cc363adbb6fe5b2cdb1270f64f40efc9869fd9e54487c9407ec66ac17"} Jul 07 11:56:33.692974 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:56:33.692852 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-97eb93/rate-test-model-97eb93-kserve-7b96477b5f-f99qw" event={"ID":"9012964d-64f3-4a44-80ea-77f81b086465","Type":"ContainerDied","Data":"b8db755ef0bf5d75c6ec8c140ae4d0e0126abbbabd309ee90b1b19de489c3b95"} Jul 07 11:56:33.692974 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:56:33.692870 2578 scope.go:117] "RemoveContainer" containerID="977bbf2cc363adbb6fe5b2cdb1270f64f40efc9869fd9e54487c9407ec66ac17" Jul 07 11:56:33.704271 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:56:33.704253 2578 scope.go:117] "RemoveContainer" containerID="977bbf2cc363adbb6fe5b2cdb1270f64f40efc9869fd9e54487c9407ec66ac17" Jul 07 11:56:33.704573 ip-10-0-142-112 kubenswrapper[2578]: E0707 11:56:33.704552 2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"977bbf2cc363adbb6fe5b2cdb1270f64f40efc9869fd9e54487c9407ec66ac17\": container with ID starting with 977bbf2cc363adbb6fe5b2cdb1270f64f40efc9869fd9e54487c9407ec66ac17 not found: ID does not exist" containerID="977bbf2cc363adbb6fe5b2cdb1270f64f40efc9869fd9e54487c9407ec66ac17" Jul 07 11:56:33.704661 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:56:33.704585 2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"977bbf2cc363adbb6fe5b2cdb1270f64f40efc9869fd9e54487c9407ec66ac17"} err="failed to get container status \"977bbf2cc363adbb6fe5b2cdb1270f64f40efc9869fd9e54487c9407ec66ac17\": rpc error: code = NotFound desc = could not find container \"977bbf2cc363adbb6fe5b2cdb1270f64f40efc9869fd9e54487c9407ec66ac17\": container with ID starting with 977bbf2cc363adbb6fe5b2cdb1270f64f40efc9869fd9e54487c9407ec66ac17 not found: ID does not exist" Jul 07 11:56:33.715701 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:56:33.715673 2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-97eb93/rate-test-model-97eb93-kserve-7b96477b5f-f99qw"] Jul 07 11:56:33.720933 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:56:33.720911 2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-b-97eb93/rate-test-model-97eb93-kserve-7b96477b5f-f99qw"] Jul 07 11:56:34.308849 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:56:34.308813 2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="9012964d-64f3-4a44-80ea-77f81b086465" path="/var/lib/kubelet/pods/9012964d-64f3-4a44-80ea-77f81b086465/volumes" Jul 07 11:57:15.570100 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:57:15.570071 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-59ddd9ddd8-9mtpg_c6075474-f9d8-4cfb-821c-707c5ef167f1/copy-manifests/0.log" Jul 07 11:57:15.576799 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:57:15.576765 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-59ddd9ddd8-9mtpg_c6075474-f9d8-4cfb-821c-707c5ef167f1/manager/0.log" Jul 07 11:57:18.225347 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:57:18.225311 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-6576859cf4q24nh_5c24eef8-4fc8-4973-a25b-4e93f4badb41/istio-proxy/0.log" Jul 07 11:57:18.351118 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:57:18.351088 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-94bb8fbfd-qmvqq_8111ef68-a869-479b-9087-2e5ed23349c0/discovery/0.log" Jul 07 11:57:18.461121 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:57:18.461090 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-785759898c-7pnk6_ffe5c6b1-dc1e-4fea-b7b6-a6a527517ec5/kube-auth-proxy/0.log" Jul 07 11:57:19.833515 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:57:19.833486 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_facebook-opt-125m-simulated-kserve-74b68447df-flr9c_fb1c7b72-0552-414c-9e00-adc7008c5f9e/main/0.log" Jul 07 11:57:27.004657 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:57:27.004624 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-8vbr5_3bd399c7-6314-4991-9984-8d2ff9677352/global-pull-secret-syncer/0.log" Jul 07 11:57:27.138698 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:57:27.138666 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-qvc9d_9ac335b6-28f5-4d6a-979c-02587fbd8e95/konnectivity-agent/0.log" Jul 07 11:57:27.293946 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:57:27.293854 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-142-112.ec2.internal_48c63b94b1c5e4397a6ecdc3e9151f62/haproxy/0.log" Jul 07 11:57:34.956820 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:57:34.956758 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-7c9dbb8b7d-rqcp8_c82e9536-2e0e-47a3-9573-d7acd8ade54a/kube-state-metrics/0.log" Jul 07 11:57:34.982887 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:57:34.982854 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-7c9dbb8b7d-rqcp8_c82e9536-2e0e-47a3-9573-d7acd8ade54a/kube-rbac-proxy-main/0.log" Jul 07 11:57:35.008740 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:57:35.008713 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-7c9dbb8b7d-rqcp8_c82e9536-2e0e-47a3-9573-d7acd8ade54a/kube-rbac-proxy-self/0.log" Jul 07 11:57:35.068817 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:57:35.068791 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_monitoring-plugin-d5dcff7cf-tn4c9_5719f623-3be7-4a90-ae42-ac58bc2e1614/monitoring-plugin/0.log" Jul 07 11:57:35.106779 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:57:35.106695 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-98hzz_6ef2378c-fd55-4ba4-b260-86c732ed130b/node-exporter/0.log" Jul 07 11:57:35.133233 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:57:35.133199 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-98hzz_6ef2378c-fd55-4ba4-b260-86c732ed130b/kube-rbac-proxy/0.log" Jul 07 11:57:35.163712 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:57:35.163686 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-98hzz_6ef2378c-fd55-4ba4-b260-86c732ed130b/init-textfile/0.log" Jul 07 11:57:35.476773 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:57:35.476688 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_fd952068-3307-4f9d-82a0-dd29b894f014/prometheus/0.log" Jul 07 11:57:35.501177 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:57:35.501151 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_fd952068-3307-4f9d-82a0-dd29b894f014/config-reloader/0.log" Jul 07 11:57:35.527411 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:57:35.527375 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_fd952068-3307-4f9d-82a0-dd29b894f014/thanos-sidecar/0.log" Jul 07 11:57:35.551301 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:57:35.551274 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_fd952068-3307-4f9d-82a0-dd29b894f014/kube-rbac-proxy-web/0.log" Jul 07 11:57:35.577900 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:57:35.577876 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_fd952068-3307-4f9d-82a0-dd29b894f014/kube-rbac-proxy/0.log" Jul 07 11:57:35.609358 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:57:35.609330 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_fd952068-3307-4f9d-82a0-dd29b894f014/kube-rbac-proxy-thanos/0.log" Jul 07 11:57:35.631945 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:57:35.631918 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_fd952068-3307-4f9d-82a0-dd29b894f014/init-config-reloader/0.log" Jul 07 11:57:35.662489 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:57:35.662459 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-567f7b8b7c-lbgcs_eb34d7f1-0c6a-4b0a-92f6-e03dcf0a989d/prometheus-operator/0.log" Jul 07 11:57:35.678838 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:57:35.678814 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-567f7b8b7c-lbgcs_eb34d7f1-0c6a-4b0a-92f6-e03dcf0a989d/kube-rbac-proxy/0.log" Jul 07 11:57:35.814244 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:57:35.814217 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-7b5486658-ncz65_7a646b22-d6b9-448c-a618-78a5edf13a17/thanos-query/0.log" Jul 07 11:57:35.838111 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:57:35.838084 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-7b5486658-ncz65_7a646b22-d6b9-448c-a618-78a5edf13a17/kube-rbac-proxy-web/0.log" Jul 07 11:57:35.869836 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:57:35.869813 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-7b5486658-ncz65_7a646b22-d6b9-448c-a618-78a5edf13a17/kube-rbac-proxy/0.log" Jul 07 11:57:35.893148 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:57:35.893120 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-7b5486658-ncz65_7a646b22-d6b9-448c-a618-78a5edf13a17/prom-label-proxy/0.log" Jul 07 11:57:35.916840 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:57:35.916808 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-7b5486658-ncz65_7a646b22-d6b9-448c-a618-78a5edf13a17/kube-rbac-proxy-rules/0.log" Jul 07 11:57:35.937285 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:57:35.937265 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-7b5486658-ncz65_7a646b22-d6b9-448c-a618-78a5edf13a17/kube-rbac-proxy-metrics/0.log" Jul 07 11:57:37.413651 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:57:37.413615 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-k5px7/perf-node-gather-daemonset-lqsps"] Jul 07 11:57:37.414091 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:57:37.414057 2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9012964d-64f3-4a44-80ea-77f81b086465" containerName="main" Jul 07 11:57:37.414091 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:57:37.414077 2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="9012964d-64f3-4a44-80ea-77f81b086465" containerName="main" Jul 07 11:57:37.414226 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:57:37.414162 2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="9012964d-64f3-4a44-80ea-77f81b086465" containerName="main" Jul 07 11:57:37.417133 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:57:37.417114 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-k5px7/perf-node-gather-daemonset-lqsps" Jul 07 11:57:37.419255 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:57:37.419229 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-k5px7\"/\"openshift-service-ca.crt\"" Jul 07 11:57:37.419383 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:57:37.419261 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-k5px7\"/\"kube-root-ca.crt\"" Jul 07 11:57:37.419868 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:57:37.419850 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-k5px7\"/\"default-dockercfg-7s2dd\"" Jul 07 11:57:37.428513 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:57:37.428487 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-k5px7/perf-node-gather-daemonset-lqsps"] Jul 07 11:57:37.492305 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:57:37.492275 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/e4d16b3a-62a0-47d1-8756-e2dbc282b201-proc\") pod \"perf-node-gather-daemonset-lqsps\" (UID: \"e4d16b3a-62a0-47d1-8756-e2dbc282b201\") " pod="openshift-must-gather-k5px7/perf-node-gather-daemonset-lqsps" Jul 07 11:57:37.492305 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:57:37.492308 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/e4d16b3a-62a0-47d1-8756-e2dbc282b201-lib-modules\") pod \"perf-node-gather-daemonset-lqsps\" (UID: \"e4d16b3a-62a0-47d1-8756-e2dbc282b201\") " pod="openshift-must-gather-k5px7/perf-node-gather-daemonset-lqsps" Jul 07 11:57:37.492572 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:57:37.492367 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/e4d16b3a-62a0-47d1-8756-e2dbc282b201-podres\") pod \"perf-node-gather-daemonset-lqsps\" (UID: \"e4d16b3a-62a0-47d1-8756-e2dbc282b201\") " pod="openshift-must-gather-k5px7/perf-node-gather-daemonset-lqsps" Jul 07 11:57:37.492572 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:57:37.492386 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kfzlf\" (UniqueName: \"kubernetes.io/projected/e4d16b3a-62a0-47d1-8756-e2dbc282b201-kube-api-access-kfzlf\") pod \"perf-node-gather-daemonset-lqsps\" (UID: \"e4d16b3a-62a0-47d1-8756-e2dbc282b201\") " pod="openshift-must-gather-k5px7/perf-node-gather-daemonset-lqsps" Jul 07 11:57:37.492572 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:57:37.492406 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/e4d16b3a-62a0-47d1-8756-e2dbc282b201-sys\") pod \"perf-node-gather-daemonset-lqsps\" (UID: \"e4d16b3a-62a0-47d1-8756-e2dbc282b201\") " pod="openshift-must-gather-k5px7/perf-node-gather-daemonset-lqsps" Jul 07 11:57:37.593634 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:57:37.593585 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/e4d16b3a-62a0-47d1-8756-e2dbc282b201-podres\") pod \"perf-node-gather-daemonset-lqsps\" (UID: \"e4d16b3a-62a0-47d1-8756-e2dbc282b201\") " pod="openshift-must-gather-k5px7/perf-node-gather-daemonset-lqsps" Jul 07 11:57:37.593826 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:57:37.593684 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kfzlf\" (UniqueName: \"kubernetes.io/projected/e4d16b3a-62a0-47d1-8756-e2dbc282b201-kube-api-access-kfzlf\") pod \"perf-node-gather-daemonset-lqsps\" (UID: \"e4d16b3a-62a0-47d1-8756-e2dbc282b201\") " pod="openshift-must-gather-k5px7/perf-node-gather-daemonset-lqsps" Jul 07 11:57:37.593826 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:57:37.593727 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/e4d16b3a-62a0-47d1-8756-e2dbc282b201-sys\") pod \"perf-node-gather-daemonset-lqsps\" (UID: \"e4d16b3a-62a0-47d1-8756-e2dbc282b201\") " pod="openshift-must-gather-k5px7/perf-node-gather-daemonset-lqsps" Jul 07 11:57:37.593826 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:57:37.593776 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/e4d16b3a-62a0-47d1-8756-e2dbc282b201-podres\") pod \"perf-node-gather-daemonset-lqsps\" (UID: \"e4d16b3a-62a0-47d1-8756-e2dbc282b201\") " pod="openshift-must-gather-k5px7/perf-node-gather-daemonset-lqsps" Jul 07 11:57:37.594006 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:57:37.593848 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/e4d16b3a-62a0-47d1-8756-e2dbc282b201-proc\") pod \"perf-node-gather-daemonset-lqsps\" (UID: \"e4d16b3a-62a0-47d1-8756-e2dbc282b201\") " pod="openshift-must-gather-k5px7/perf-node-gather-daemonset-lqsps" Jul 07 11:57:37.594006 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:57:37.593851 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/e4d16b3a-62a0-47d1-8756-e2dbc282b201-sys\") pod \"perf-node-gather-daemonset-lqsps\" (UID: \"e4d16b3a-62a0-47d1-8756-e2dbc282b201\") " pod="openshift-must-gather-k5px7/perf-node-gather-daemonset-lqsps" Jul 07 11:57:37.594006 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:57:37.593877 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/e4d16b3a-62a0-47d1-8756-e2dbc282b201-lib-modules\") pod \"perf-node-gather-daemonset-lqsps\" (UID: \"e4d16b3a-62a0-47d1-8756-e2dbc282b201\") " pod="openshift-must-gather-k5px7/perf-node-gather-daemonset-lqsps" Jul 07 11:57:37.594006 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:57:37.593908 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/e4d16b3a-62a0-47d1-8756-e2dbc282b201-proc\") pod \"perf-node-gather-daemonset-lqsps\" (UID: \"e4d16b3a-62a0-47d1-8756-e2dbc282b201\") " pod="openshift-must-gather-k5px7/perf-node-gather-daemonset-lqsps" Jul 07 11:57:37.594205 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:57:37.594077 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/e4d16b3a-62a0-47d1-8756-e2dbc282b201-lib-modules\") pod \"perf-node-gather-daemonset-lqsps\" (UID: \"e4d16b3a-62a0-47d1-8756-e2dbc282b201\") " pod="openshift-must-gather-k5px7/perf-node-gather-daemonset-lqsps" Jul 07 11:57:37.602355 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:57:37.602334 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-kfzlf\" (UniqueName: \"kubernetes.io/projected/e4d16b3a-62a0-47d1-8756-e2dbc282b201-kube-api-access-kfzlf\") pod \"perf-node-gather-daemonset-lqsps\" (UID: \"e4d16b3a-62a0-47d1-8756-e2dbc282b201\") " pod="openshift-must-gather-k5px7/perf-node-gather-daemonset-lqsps" Jul 07 11:57:37.727071 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:57:37.726980 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-k5px7/perf-node-gather-daemonset-lqsps" Jul 07 11:57:37.864020 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:57:37.863985 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-k5px7/perf-node-gather-daemonset-lqsps"] Jul 07 11:57:37.868128 ip-10-0-142-112 kubenswrapper[2578]: W0707 11:57:37.868094 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-pode4d16b3a_62a0_47d1_8756_e2dbc282b201.slice/crio-52b7006cfa20611c9f4b96c09fc574760a6e0b9c5aaba9d3595cdb9c9c7fd0dd WatchSource:0}: Error finding container 52b7006cfa20611c9f4b96c09fc574760a6e0b9c5aaba9d3595cdb9c9c7fd0dd: Status 404 returned error can't find the container with id 52b7006cfa20611c9f4b96c09fc574760a6e0b9c5aaba9d3595cdb9c9c7fd0dd Jul 07 11:57:37.968700 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:57:37.968664 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-k5px7/perf-node-gather-daemonset-lqsps" event={"ID":"e4d16b3a-62a0-47d1-8756-e2dbc282b201","Type":"ContainerStarted","Data":"8a94dde2b4a092b294c227381d38bbad4d3584e1834bce5fd55e5a54187619b1"} Jul 07 11:57:37.968899 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:57:37.968722 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-k5px7/perf-node-gather-daemonset-lqsps" event={"ID":"e4d16b3a-62a0-47d1-8756-e2dbc282b201","Type":"ContainerStarted","Data":"52b7006cfa20611c9f4b96c09fc574760a6e0b9c5aaba9d3595cdb9c9c7fd0dd"} Jul 07 11:57:37.968899 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:57:37.968789 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-k5px7/perf-node-gather-daemonset-lqsps" Jul 07 11:57:37.984233 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:57:37.984122 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-k5px7/perf-node-gather-daemonset-lqsps" podStartSLOduration=0.984104885 podStartE2EDuration="984.104885ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-07 11:57:37 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 11:57:37.983475119 +0000 UTC m=+2502.279624706" watchObservedRunningTime="2026-07-07 11:57:37.984104885 +0000 UTC m=+2502.280254473" Jul 07 11:57:38.252228 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:57:38.252146 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_console-6694655d5d-bwrbf_d6fdfe74-5cde-4df0-8203-ef253eae64cf/console/0.log" Jul 07 11:57:39.933948 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:57:39.933919 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-lgwzn_12b4908e-0a68-467c-966f-a1900491e26c/dns/0.log" Jul 07 11:57:39.955190 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:57:39.955167 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-lgwzn_12b4908e-0a68-467c-966f-a1900491e26c/kube-rbac-proxy/0.log" Jul 07 11:57:40.025648 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:57:40.025622 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-djzsf_109be65e-ec91-4d45-b856-334d021f900c/dns-node-resolver/0.log" Jul 07 11:57:40.695079 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:57:40.695051 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-rms7l_f03343fc-0662-41c1-9904-3ece548a3f42/node-ca/0.log" Jul 07 11:57:41.598878 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:57:41.598849 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-6576859cf4q24nh_5c24eef8-4fc8-4973-a25b-4e93f4badb41/istio-proxy/0.log" Jul 07 11:57:41.790074 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:57:41.790035 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-94bb8fbfd-qmvqq_8111ef68-a869-479b-9087-2e5ed23349c0/discovery/0.log" Jul 07 11:57:41.813071 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:57:41.813047 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-785759898c-7pnk6_ffe5c6b1-dc1e-4fea-b7b6-a6a527517ec5/kube-auth-proxy/0.log" Jul 07 11:57:42.737601 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:57:42.737440 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-kdmpx_fba5ce69-3377-4a37-ba7c-6b3adb88b2f3/serve-healthcheck-canary/0.log" Jul 07 11:57:43.277588 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:57:43.277560 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-9t57z_bcd0ff4f-9b04-41d6-afef-018180f7dd2a/kube-rbac-proxy/0.log" Jul 07 11:57:43.296002 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:57:43.295978 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-9t57z_bcd0ff4f-9b04-41d6-afef-018180f7dd2a/exporter/0.log" Jul 07 11:57:43.314734 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:57:43.314706 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-9t57z_bcd0ff4f-9b04-41d6-afef-018180f7dd2a/extractor/0.log" Jul 07 11:57:43.986307 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:57:43.986267 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-k5px7/perf-node-gather-daemonset-lqsps" Jul 07 11:57:46.432110 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:57:46.432074 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-59ddd9ddd8-9mtpg_c6075474-f9d8-4cfb-821c-707c5ef167f1/manager/0.log" Jul 07 11:57:46.452831 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:57:46.452806 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-59ddd9ddd8-9mtpg_c6075474-f9d8-4cfb-821c-707c5ef167f1/copy-manifests/0.log" Jul 07 11:57:47.913927 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:57:47.913832 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-6576859cf4q24nh_5c24eef8-4fc8-4973-a25b-4e93f4badb41/istio-proxy/0.log" Jul 07 11:57:47.965143 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:57:47.965111 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-lws-operator_lws-controller-manager-65f8ccd9c5-8tww2_3431aa7e-d46e-4ac2-8c76-b3d62a99eb0e/manager/0.log" Jul 07 11:57:48.093483 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:57:48.093449 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-94bb8fbfd-qmvqq_8111ef68-a869-479b-9087-2e5ed23349c0/discovery/0.log" Jul 07 11:57:48.104025 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:57:48.104003 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-785759898c-7pnk6_ffe5c6b1-dc1e-4fea-b7b6-a6a527517ec5/kube-auth-proxy/0.log" Jul 07 11:57:49.534208 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:57:49.534119 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-6576859cf4q24nh_5c24eef8-4fc8-4973-a25b-4e93f4badb41/istio-proxy/0.log" Jul 07 11:57:49.717966 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:57:49.717934 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-94bb8fbfd-qmvqq_8111ef68-a869-479b-9087-2e5ed23349c0/discovery/0.log" Jul 07 11:57:49.730197 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:57:49.730162 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-785759898c-7pnk6_ffe5c6b1-dc1e-4fea-b7b6-a6a527517ec5/kube-auth-proxy/0.log" Jul 07 11:57:54.340497 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:57:54.340459 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-5ntjm_a60168c4-c743-44e4-9047-df74d47da598/kube-multus/0.log" Jul 07 11:57:54.707861 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:57:54.707776 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-rtggm_08ea46cf-197f-4fa4-b752-08df27dbdfee/kube-multus-additional-cni-plugins/0.log" Jul 07 11:57:54.728180 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:57:54.728149 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-rtggm_08ea46cf-197f-4fa4-b752-08df27dbdfee/egress-router-binary-copy/0.log" Jul 07 11:57:54.747730 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:57:54.747709 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-rtggm_08ea46cf-197f-4fa4-b752-08df27dbdfee/cni-plugins/0.log" Jul 07 11:57:54.766790 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:57:54.766767 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-rtggm_08ea46cf-197f-4fa4-b752-08df27dbdfee/bond-cni-plugin/0.log" Jul 07 11:57:54.790039 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:57:54.790021 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-rtggm_08ea46cf-197f-4fa4-b752-08df27dbdfee/routeoverride-cni/0.log" Jul 07 11:57:54.810862 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:57:54.810833 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-rtggm_08ea46cf-197f-4fa4-b752-08df27dbdfee/whereabouts-cni-bincopy/0.log" Jul 07 11:57:54.829120 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:57:54.829086 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-rtggm_08ea46cf-197f-4fa4-b752-08df27dbdfee/whereabouts-cni/0.log" Jul 07 11:57:54.994752 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:57:54.994679 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-xhlzx_1dc0212e-9bf8-4b06-b984-c694479a14ad/network-metrics-daemon/0.log" Jul 07 11:57:55.014145 ip-10-0-142-112 kubenswrapper[2578]: I0707 11:57:55.014122 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-xhlzx_1dc0212e-9bf8-4b06-b984-c694479a14ad/kube-rbac-proxy/0.log"