Jul 16 17:12:20.132908 ip-10-0-132-130 systemd[1]: kubelet.service: Failed to load environment files: No such file or directory Jul 16 17:12:20.132917 ip-10-0-132-130 systemd[1]: kubelet.service: Failed to run 'start-pre' task: No such file or directory Jul 16 17:12:20.132926 ip-10-0-132-130 systemd[1]: kubelet.service: Failed with result 'resources'. Jul 16 17:12:20.133230 ip-10-0-132-130 systemd[1]: Failed to start Kubernetes Kubelet. Jul 16 17:12:30.165506 ip-10-0-132-130 systemd[1]: kubelet.service: Failed to schedule restart job: Unit crio.service not found. Jul 16 17:12:30.165528 ip-10-0-132-130 systemd[1]: kubelet.service: Failed with result 'resources'. -- Boot 901e0ff28fb3452199546aee33186f16 -- Jul 16 17:14:14.236604 ip-10-0-132-130 systemd[1]: Starting Kubernetes Kubelet... Jul 16 17:14:14.687316 ip-10-0-132-130 kubenswrapper[2578]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 16 17:14:14.687316 ip-10-0-132-130 kubenswrapper[2578]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version. Jul 16 17:14:14.687316 ip-10-0-132-130 kubenswrapper[2578]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 16 17:14:14.687316 ip-10-0-132-130 kubenswrapper[2578]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI. Jul 16 17:14:14.687316 ip-10-0-132-130 kubenswrapper[2578]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 16 17:14:14.687316 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:14:14.682324 2578 server.go:213] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime" Jul 16 17:14:14.696340 ip-10-0-132-130 kubenswrapper[2578]: W0716 17:14:14.687743 2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 16 17:14:14.696340 ip-10-0-132-130 kubenswrapper[2578]: W0716 17:14:14.687753 2578 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 16 17:14:14.696340 ip-10-0-132-130 kubenswrapper[2578]: W0716 17:14:14.687758 2578 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 16 17:14:14.696340 ip-10-0-132-130 kubenswrapper[2578]: W0716 17:14:14.687761 2578 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 16 17:14:14.696340 ip-10-0-132-130 kubenswrapper[2578]: W0716 17:14:14.687764 2578 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 16 17:14:14.696340 ip-10-0-132-130 kubenswrapper[2578]: W0716 17:14:14.687767 2578 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 16 17:14:14.696340 ip-10-0-132-130 kubenswrapper[2578]: W0716 17:14:14.687770 2578 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 16 17:14:14.696340 ip-10-0-132-130 kubenswrapper[2578]: W0716 17:14:14.687773 2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 16 17:14:14.696340 ip-10-0-132-130 kubenswrapper[2578]: W0716 17:14:14.687776 2578 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 16 17:14:14.696340 ip-10-0-132-130 kubenswrapper[2578]: W0716 17:14:14.687780 2578 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 16 17:14:14.696340 ip-10-0-132-130 kubenswrapper[2578]: W0716 17:14:14.687784 2578 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 16 17:14:14.696340 ip-10-0-132-130 kubenswrapper[2578]: W0716 17:14:14.687787 2578 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 16 17:14:14.696340 ip-10-0-132-130 kubenswrapper[2578]: W0716 17:14:14.687790 2578 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 16 17:14:14.696340 ip-10-0-132-130 kubenswrapper[2578]: W0716 17:14:14.687793 2578 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 16 17:14:14.696340 ip-10-0-132-130 kubenswrapper[2578]: W0716 17:14:14.687796 2578 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 16 17:14:14.696340 ip-10-0-132-130 kubenswrapper[2578]: W0716 17:14:14.687800 2578 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 16 17:14:14.696340 ip-10-0-132-130 kubenswrapper[2578]: W0716 17:14:14.687803 2578 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 16 17:14:14.696340 ip-10-0-132-130 kubenswrapper[2578]: W0716 17:14:14.687806 2578 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 16 17:14:14.696340 ip-10-0-132-130 kubenswrapper[2578]: W0716 17:14:14.687809 2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 16 17:14:14.696340 ip-10-0-132-130 kubenswrapper[2578]: W0716 17:14:14.687811 2578 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 16 17:14:14.699674 ip-10-0-132-130 kubenswrapper[2578]: W0716 17:14:14.687814 2578 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 16 17:14:14.699674 ip-10-0-132-130 kubenswrapper[2578]: W0716 17:14:14.687817 2578 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 16 17:14:14.699674 ip-10-0-132-130 kubenswrapper[2578]: W0716 17:14:14.687820 2578 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 16 17:14:14.699674 ip-10-0-132-130 kubenswrapper[2578]: W0716 17:14:14.687822 2578 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 16 17:14:14.699674 ip-10-0-132-130 kubenswrapper[2578]: W0716 17:14:14.687825 2578 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 16 17:14:14.699674 ip-10-0-132-130 kubenswrapper[2578]: W0716 17:14:14.687827 2578 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 16 17:14:14.699674 ip-10-0-132-130 kubenswrapper[2578]: W0716 17:14:14.687830 2578 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 16 17:14:14.699674 ip-10-0-132-130 kubenswrapper[2578]: W0716 17:14:14.687832 2578 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 16 17:14:14.699674 ip-10-0-132-130 kubenswrapper[2578]: W0716 17:14:14.687834 2578 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 16 17:14:14.699674 ip-10-0-132-130 kubenswrapper[2578]: W0716 17:14:14.687837 2578 feature_gate.go:328] unrecognized feature gate: Example2 Jul 16 17:14:14.699674 ip-10-0-132-130 kubenswrapper[2578]: W0716 17:14:14.687840 2578 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 16 17:14:14.699674 ip-10-0-132-130 kubenswrapper[2578]: W0716 17:14:14.687843 2578 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 16 17:14:14.699674 ip-10-0-132-130 kubenswrapper[2578]: W0716 17:14:14.687845 2578 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 16 17:14:14.699674 ip-10-0-132-130 kubenswrapper[2578]: W0716 17:14:14.687847 2578 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 16 17:14:14.699674 ip-10-0-132-130 kubenswrapper[2578]: W0716 17:14:14.687850 2578 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 16 17:14:14.699674 ip-10-0-132-130 kubenswrapper[2578]: W0716 17:14:14.687853 2578 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 16 17:14:14.699674 ip-10-0-132-130 kubenswrapper[2578]: W0716 17:14:14.687855 2578 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 16 17:14:14.699674 ip-10-0-132-130 kubenswrapper[2578]: W0716 17:14:14.687858 2578 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 16 17:14:14.699674 ip-10-0-132-130 kubenswrapper[2578]: W0716 17:14:14.687861 2578 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 16 17:14:14.699674 ip-10-0-132-130 kubenswrapper[2578]: W0716 17:14:14.687864 2578 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 16 17:14:14.710429 ip-10-0-132-130 kubenswrapper[2578]: W0716 17:14:14.687866 2578 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 16 17:14:14.710429 ip-10-0-132-130 kubenswrapper[2578]: W0716 17:14:14.687872 2578 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 16 17:14:14.710429 ip-10-0-132-130 kubenswrapper[2578]: W0716 17:14:14.687875 2578 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 16 17:14:14.710429 ip-10-0-132-130 kubenswrapper[2578]: W0716 17:14:14.687877 2578 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 16 17:14:14.710429 ip-10-0-132-130 kubenswrapper[2578]: W0716 17:14:14.687881 2578 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 16 17:14:14.710429 ip-10-0-132-130 kubenswrapper[2578]: W0716 17:14:14.687883 2578 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 16 17:14:14.710429 ip-10-0-132-130 kubenswrapper[2578]: W0716 17:14:14.687886 2578 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 16 17:14:14.710429 ip-10-0-132-130 kubenswrapper[2578]: W0716 17:14:14.687888 2578 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 16 17:14:14.710429 ip-10-0-132-130 kubenswrapper[2578]: W0716 17:14:14.687891 2578 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 16 17:14:14.710429 ip-10-0-132-130 kubenswrapper[2578]: W0716 17:14:14.687894 2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 16 17:14:14.710429 ip-10-0-132-130 kubenswrapper[2578]: W0716 17:14:14.687896 2578 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 16 17:14:14.710429 ip-10-0-132-130 kubenswrapper[2578]: W0716 17:14:14.687899 2578 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 16 17:14:14.710429 ip-10-0-132-130 kubenswrapper[2578]: W0716 17:14:14.687902 2578 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 16 17:14:14.710429 ip-10-0-132-130 kubenswrapper[2578]: W0716 17:14:14.687905 2578 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 16 17:14:14.710429 ip-10-0-132-130 kubenswrapper[2578]: W0716 17:14:14.687907 2578 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 16 17:14:14.710429 ip-10-0-132-130 kubenswrapper[2578]: W0716 17:14:14.687913 2578 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 16 17:14:14.710429 ip-10-0-132-130 kubenswrapper[2578]: W0716 17:14:14.687916 2578 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 16 17:14:14.710429 ip-10-0-132-130 kubenswrapper[2578]: W0716 17:14:14.687919 2578 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 16 17:14:14.710429 ip-10-0-132-130 kubenswrapper[2578]: W0716 17:14:14.687921 2578 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 16 17:14:14.714003 ip-10-0-132-130 kubenswrapper[2578]: W0716 17:14:14.687925 2578 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 16 17:14:14.714003 ip-10-0-132-130 kubenswrapper[2578]: W0716 17:14:14.687928 2578 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 16 17:14:14.714003 ip-10-0-132-130 kubenswrapper[2578]: W0716 17:14:14.687931 2578 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 16 17:14:14.714003 ip-10-0-132-130 kubenswrapper[2578]: W0716 17:14:14.687934 2578 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 16 17:14:14.714003 ip-10-0-132-130 kubenswrapper[2578]: W0716 17:14:14.687937 2578 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 16 17:14:14.714003 ip-10-0-132-130 kubenswrapper[2578]: W0716 17:14:14.687942 2578 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 16 17:14:14.714003 ip-10-0-132-130 kubenswrapper[2578]: W0716 17:14:14.687945 2578 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 16 17:14:14.714003 ip-10-0-132-130 kubenswrapper[2578]: W0716 17:14:14.687948 2578 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 16 17:14:14.714003 ip-10-0-132-130 kubenswrapper[2578]: W0716 17:14:14.687950 2578 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 16 17:14:14.714003 ip-10-0-132-130 kubenswrapper[2578]: W0716 17:14:14.687953 2578 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 16 17:14:14.714003 ip-10-0-132-130 kubenswrapper[2578]: W0716 17:14:14.687955 2578 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 16 17:14:14.714003 ip-10-0-132-130 kubenswrapper[2578]: W0716 17:14:14.687958 2578 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 16 17:14:14.714003 ip-10-0-132-130 kubenswrapper[2578]: W0716 17:14:14.687965 2578 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 16 17:14:14.714003 ip-10-0-132-130 kubenswrapper[2578]: W0716 17:14:14.687969 2578 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 16 17:14:14.714003 ip-10-0-132-130 kubenswrapper[2578]: W0716 17:14:14.687971 2578 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 16 17:14:14.714003 ip-10-0-132-130 kubenswrapper[2578]: W0716 17:14:14.687974 2578 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 16 17:14:14.714003 ip-10-0-132-130 kubenswrapper[2578]: W0716 17:14:14.687976 2578 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 16 17:14:14.714003 ip-10-0-132-130 kubenswrapper[2578]: W0716 17:14:14.687980 2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 16 17:14:14.714003 ip-10-0-132-130 kubenswrapper[2578]: W0716 17:14:14.687986 2578 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 16 17:14:14.725478 ip-10-0-132-130 kubenswrapper[2578]: W0716 17:14:14.687989 2578 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 16 17:14:14.725478 ip-10-0-132-130 kubenswrapper[2578]: W0716 17:14:14.687991 2578 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 16 17:14:14.725478 ip-10-0-132-130 kubenswrapper[2578]: W0716 17:14:14.687994 2578 feature_gate.go:328] unrecognized feature gate: Example Jul 16 17:14:14.725478 ip-10-0-132-130 kubenswrapper[2578]: W0716 17:14:14.687997 2578 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 16 17:14:14.725478 ip-10-0-132-130 kubenswrapper[2578]: W0716 17:14:14.687999 2578 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 16 17:14:14.725478 ip-10-0-132-130 kubenswrapper[2578]: W0716 17:14:14.688006 2578 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 16 17:14:14.725478 ip-10-0-132-130 kubenswrapper[2578]: W0716 17:14:14.688009 2578 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 16 17:14:14.725478 ip-10-0-132-130 kubenswrapper[2578]: W0716 17:14:14.688012 2578 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 16 17:14:14.725478 ip-10-0-132-130 kubenswrapper[2578]: W0716 17:14:14.688015 2578 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 16 17:14:14.725478 ip-10-0-132-130 kubenswrapper[2578]: W0716 17:14:14.688018 2578 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 16 17:14:14.725478 ip-10-0-132-130 kubenswrapper[2578]: W0716 17:14:14.688024 2578 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 16 17:14:14.725478 ip-10-0-132-130 kubenswrapper[2578]: W0716 17:14:14.688027 2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 16 17:14:14.725478 ip-10-0-132-130 kubenswrapper[2578]: W0716 17:14:14.688029 2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 16 17:14:14.725478 ip-10-0-132-130 kubenswrapper[2578]: W0716 17:14:14.688032 2578 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 16 17:14:14.725478 ip-10-0-132-130 kubenswrapper[2578]: W0716 17:14:14.688034 2578 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 16 17:14:14.725478 ip-10-0-132-130 kubenswrapper[2578]: W0716 17:14:14.688037 2578 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 16 17:14:14.725478 ip-10-0-132-130 kubenswrapper[2578]: W0716 17:14:14.688044 2578 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 16 17:14:14.725478 ip-10-0-132-130 kubenswrapper[2578]: W0716 17:14:14.688046 2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 16 17:14:14.725478 ip-10-0-132-130 kubenswrapper[2578]: W0716 17:14:14.688049 2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 16 17:14:14.725478 ip-10-0-132-130 kubenswrapper[2578]: W0716 17:14:14.688051 2578 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 16 17:14:14.729795 ip-10-0-132-130 kubenswrapper[2578]: W0716 17:14:14.688055 2578 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 16 17:14:14.729795 ip-10-0-132-130 kubenswrapper[2578]: W0716 17:14:14.688928 2578 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 16 17:14:14.729795 ip-10-0-132-130 kubenswrapper[2578]: W0716 17:14:14.688936 2578 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 16 17:14:14.729795 ip-10-0-132-130 kubenswrapper[2578]: W0716 17:14:14.688945 2578 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 16 17:14:14.729795 ip-10-0-132-130 kubenswrapper[2578]: W0716 17:14:14.688949 2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 16 17:14:14.729795 ip-10-0-132-130 kubenswrapper[2578]: W0716 17:14:14.688952 2578 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 16 17:14:14.729795 ip-10-0-132-130 kubenswrapper[2578]: W0716 17:14:14.688955 2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 16 17:14:14.729795 ip-10-0-132-130 kubenswrapper[2578]: W0716 17:14:14.688958 2578 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 16 17:14:14.729795 ip-10-0-132-130 kubenswrapper[2578]: W0716 17:14:14.688966 2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 16 17:14:14.729795 ip-10-0-132-130 kubenswrapper[2578]: W0716 17:14:14.688969 2578 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 16 17:14:14.729795 ip-10-0-132-130 kubenswrapper[2578]: W0716 17:14:14.688971 2578 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 16 17:14:14.729795 ip-10-0-132-130 kubenswrapper[2578]: W0716 17:14:14.688974 2578 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 16 17:14:14.729795 ip-10-0-132-130 kubenswrapper[2578]: W0716 17:14:14.688977 2578 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 16 17:14:14.729795 ip-10-0-132-130 kubenswrapper[2578]: W0716 17:14:14.688983 2578 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 16 17:14:14.729795 ip-10-0-132-130 kubenswrapper[2578]: W0716 17:14:14.688986 2578 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 16 17:14:14.729795 ip-10-0-132-130 kubenswrapper[2578]: W0716 17:14:14.688989 2578 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 16 17:14:14.729795 ip-10-0-132-130 kubenswrapper[2578]: W0716 17:14:14.688992 2578 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 16 17:14:14.729795 ip-10-0-132-130 kubenswrapper[2578]: W0716 17:14:14.688994 2578 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 16 17:14:14.729795 ip-10-0-132-130 kubenswrapper[2578]: W0716 17:14:14.688997 2578 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 16 17:14:14.746068 ip-10-0-132-130 kubenswrapper[2578]: W0716 17:14:14.689000 2578 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 16 17:14:14.746068 ip-10-0-132-130 kubenswrapper[2578]: W0716 17:14:14.689007 2578 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 16 17:14:14.746068 ip-10-0-132-130 kubenswrapper[2578]: W0716 17:14:14.689009 2578 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 16 17:14:14.746068 ip-10-0-132-130 kubenswrapper[2578]: W0716 17:14:14.689012 2578 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 16 17:14:14.746068 ip-10-0-132-130 kubenswrapper[2578]: W0716 17:14:14.689014 2578 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 16 17:14:14.746068 ip-10-0-132-130 kubenswrapper[2578]: W0716 17:14:14.689017 2578 feature_gate.go:328] unrecognized feature gate: Example2 Jul 16 17:14:14.746068 ip-10-0-132-130 kubenswrapper[2578]: W0716 17:14:14.689020 2578 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 16 17:14:14.746068 ip-10-0-132-130 kubenswrapper[2578]: W0716 17:14:14.689022 2578 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 16 17:14:14.746068 ip-10-0-132-130 kubenswrapper[2578]: W0716 17:14:14.689025 2578 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 16 17:14:14.746068 ip-10-0-132-130 kubenswrapper[2578]: W0716 17:14:14.689028 2578 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 16 17:14:14.746068 ip-10-0-132-130 kubenswrapper[2578]: W0716 17:14:14.689030 2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 16 17:14:14.746068 ip-10-0-132-130 kubenswrapper[2578]: W0716 17:14:14.689037 2578 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 16 17:14:14.746068 ip-10-0-132-130 kubenswrapper[2578]: W0716 17:14:14.689040 2578 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 16 17:14:14.746068 ip-10-0-132-130 kubenswrapper[2578]: W0716 17:14:14.689042 2578 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 16 17:14:14.746068 ip-10-0-132-130 kubenswrapper[2578]: W0716 17:14:14.689046 2578 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 16 17:14:14.746068 ip-10-0-132-130 kubenswrapper[2578]: W0716 17:14:14.689048 2578 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 16 17:14:14.746068 ip-10-0-132-130 kubenswrapper[2578]: W0716 17:14:14.689050 2578 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 16 17:14:14.746068 ip-10-0-132-130 kubenswrapper[2578]: W0716 17:14:14.689057 2578 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 16 17:14:14.746068 ip-10-0-132-130 kubenswrapper[2578]: W0716 17:14:14.689060 2578 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 16 17:14:14.747897 ip-10-0-132-130 kubenswrapper[2578]: W0716 17:14:14.689064 2578 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 16 17:14:14.747897 ip-10-0-132-130 kubenswrapper[2578]: W0716 17:14:14.689080 2578 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 16 17:14:14.747897 ip-10-0-132-130 kubenswrapper[2578]: W0716 17:14:14.689083 2578 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 16 17:14:14.747897 ip-10-0-132-130 kubenswrapper[2578]: W0716 17:14:14.689086 2578 feature_gate.go:328] unrecognized feature gate: Example Jul 16 17:14:14.747897 ip-10-0-132-130 kubenswrapper[2578]: W0716 17:14:14.689089 2578 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 16 17:14:14.747897 ip-10-0-132-130 kubenswrapper[2578]: W0716 17:14:14.689092 2578 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 16 17:14:14.747897 ip-10-0-132-130 kubenswrapper[2578]: W0716 17:14:14.689101 2578 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 16 17:14:14.747897 ip-10-0-132-130 kubenswrapper[2578]: W0716 17:14:14.689103 2578 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 16 17:14:14.747897 ip-10-0-132-130 kubenswrapper[2578]: W0716 17:14:14.689106 2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 16 17:14:14.747897 ip-10-0-132-130 kubenswrapper[2578]: W0716 17:14:14.689109 2578 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 16 17:14:14.747897 ip-10-0-132-130 kubenswrapper[2578]: W0716 17:14:14.689111 2578 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 16 17:14:14.747897 ip-10-0-132-130 kubenswrapper[2578]: W0716 17:14:14.689118 2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 16 17:14:14.747897 ip-10-0-132-130 kubenswrapper[2578]: W0716 17:14:14.689121 2578 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 16 17:14:14.747897 ip-10-0-132-130 kubenswrapper[2578]: W0716 17:14:14.689124 2578 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 16 17:14:14.747897 ip-10-0-132-130 kubenswrapper[2578]: W0716 17:14:14.689126 2578 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 16 17:14:14.747897 ip-10-0-132-130 kubenswrapper[2578]: W0716 17:14:14.689129 2578 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 16 17:14:14.747897 ip-10-0-132-130 kubenswrapper[2578]: W0716 17:14:14.689135 2578 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 16 17:14:14.747897 ip-10-0-132-130 kubenswrapper[2578]: W0716 17:14:14.689138 2578 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 16 17:14:14.747897 ip-10-0-132-130 kubenswrapper[2578]: W0716 17:14:14.689141 2578 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 16 17:14:14.747897 ip-10-0-132-130 kubenswrapper[2578]: W0716 17:14:14.689143 2578 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 16 17:14:14.770827 ip-10-0-132-130 kubenswrapper[2578]: W0716 17:14:14.689146 2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 16 17:14:14.770827 ip-10-0-132-130 kubenswrapper[2578]: W0716 17:14:14.689149 2578 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 16 17:14:14.770827 ip-10-0-132-130 kubenswrapper[2578]: W0716 17:14:14.689152 2578 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 16 17:14:14.770827 ip-10-0-132-130 kubenswrapper[2578]: W0716 17:14:14.689158 2578 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 16 17:14:14.770827 ip-10-0-132-130 kubenswrapper[2578]: W0716 17:14:14.689161 2578 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 16 17:14:14.770827 ip-10-0-132-130 kubenswrapper[2578]: W0716 17:14:14.689163 2578 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 16 17:14:14.770827 ip-10-0-132-130 kubenswrapper[2578]: W0716 17:14:14.689166 2578 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 16 17:14:14.770827 ip-10-0-132-130 kubenswrapper[2578]: W0716 17:14:14.689169 2578 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 16 17:14:14.770827 ip-10-0-132-130 kubenswrapper[2578]: W0716 17:14:14.689171 2578 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 16 17:14:14.770827 ip-10-0-132-130 kubenswrapper[2578]: W0716 17:14:14.689178 2578 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 16 17:14:14.770827 ip-10-0-132-130 kubenswrapper[2578]: W0716 17:14:14.689181 2578 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 16 17:14:14.770827 ip-10-0-132-130 kubenswrapper[2578]: W0716 17:14:14.689184 2578 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 16 17:14:14.770827 ip-10-0-132-130 kubenswrapper[2578]: W0716 17:14:14.689186 2578 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 16 17:14:14.770827 ip-10-0-132-130 kubenswrapper[2578]: W0716 17:14:14.689188 2578 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 16 17:14:14.770827 ip-10-0-132-130 kubenswrapper[2578]: W0716 17:14:14.689195 2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 16 17:14:14.770827 ip-10-0-132-130 kubenswrapper[2578]: W0716 17:14:14.689198 2578 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 16 17:14:14.770827 ip-10-0-132-130 kubenswrapper[2578]: W0716 17:14:14.689201 2578 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 16 17:14:14.770827 ip-10-0-132-130 kubenswrapper[2578]: W0716 17:14:14.689204 2578 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 16 17:14:14.770827 ip-10-0-132-130 kubenswrapper[2578]: W0716 17:14:14.689230 2578 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 16 17:14:14.766026 ip-10-0-132-130 systemd[1]: Started Kubernetes Kubelet. Jul 16 17:14:14.775759 ip-10-0-132-130 kubenswrapper[2578]: W0716 17:14:14.689240 2578 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 16 17:14:14.775759 ip-10-0-132-130 kubenswrapper[2578]: W0716 17:14:14.689244 2578 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 16 17:14:14.775759 ip-10-0-132-130 kubenswrapper[2578]: W0716 17:14:14.689247 2578 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 16 17:14:14.775759 ip-10-0-132-130 kubenswrapper[2578]: W0716 17:14:14.689249 2578 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 16 17:14:14.775759 ip-10-0-132-130 kubenswrapper[2578]: W0716 17:14:14.689252 2578 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 16 17:14:14.775759 ip-10-0-132-130 kubenswrapper[2578]: W0716 17:14:14.689258 2578 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 16 17:14:14.775759 ip-10-0-132-130 kubenswrapper[2578]: W0716 17:14:14.689261 2578 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 16 17:14:14.775759 ip-10-0-132-130 kubenswrapper[2578]: W0716 17:14:14.689264 2578 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 16 17:14:14.775759 ip-10-0-132-130 kubenswrapper[2578]: W0716 17:14:14.689266 2578 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 16 17:14:14.775759 ip-10-0-132-130 kubenswrapper[2578]: W0716 17:14:14.689269 2578 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 16 17:14:14.775759 ip-10-0-132-130 kubenswrapper[2578]: W0716 17:14:14.689271 2578 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 16 17:14:14.775759 ip-10-0-132-130 kubenswrapper[2578]: W0716 17:14:14.689274 2578 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 16 17:14:14.775759 ip-10-0-132-130 kubenswrapper[2578]: W0716 17:14:14.689280 2578 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 16 17:14:14.775759 ip-10-0-132-130 kubenswrapper[2578]: W0716 17:14:14.689285 2578 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 16 17:14:14.775759 ip-10-0-132-130 kubenswrapper[2578]: W0716 17:14:14.689288 2578 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 16 17:14:14.775759 ip-10-0-132-130 kubenswrapper[2578]: W0716 17:14:14.689291 2578 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 16 17:14:14.775759 ip-10-0-132-130 kubenswrapper[2578]: W0716 17:14:14.689293 2578 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 16 17:14:14.775759 ip-10-0-132-130 kubenswrapper[2578]: W0716 17:14:14.689300 2578 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 16 17:14:14.775759 ip-10-0-132-130 kubenswrapper[2578]: W0716 17:14:14.689302 2578 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 16 17:14:14.775759 ip-10-0-132-130 kubenswrapper[2578]: W0716 17:14:14.689305 2578 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 16 17:14:14.789956 ip-10-0-132-130 kubenswrapper[2578]: W0716 17:14:14.689308 2578 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 16 17:14:14.789956 ip-10-0-132-130 kubenswrapper[2578]: W0716 17:14:14.689310 2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 16 17:14:14.789956 ip-10-0-132-130 kubenswrapper[2578]: W0716 17:14:14.689313 2578 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 16 17:14:14.789956 ip-10-0-132-130 kubenswrapper[2578]: W0716 17:14:14.689724 2578 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 16 17:14:14.789956 ip-10-0-132-130 kubenswrapper[2578]: W0716 17:14:14.689732 2578 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 16 17:14:14.789956 ip-10-0-132-130 kubenswrapper[2578]: W0716 17:14:14.689735 2578 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 16 17:14:14.789956 ip-10-0-132-130 kubenswrapper[2578]: W0716 17:14:14.689738 2578 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 16 17:14:14.789956 ip-10-0-132-130 kubenswrapper[2578]: W0716 17:14:14.689741 2578 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 16 17:14:14.789956 ip-10-0-132-130 kubenswrapper[2578]: W0716 17:14:14.689744 2578 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 16 17:14:14.789956 ip-10-0-132-130 kubenswrapper[2578]: W0716 17:14:14.689747 2578 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 16 17:14:14.789956 ip-10-0-132-130 kubenswrapper[2578]: W0716 17:14:14.689753 2578 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 16 17:14:14.789956 ip-10-0-132-130 kubenswrapper[2578]: W0716 17:14:14.689757 2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 16 17:14:14.789956 ip-10-0-132-130 kubenswrapper[2578]: W0716 17:14:14.689760 2578 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 16 17:14:14.789956 ip-10-0-132-130 kubenswrapper[2578]: W0716 17:14:14.689762 2578 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 16 17:14:14.789956 ip-10-0-132-130 kubenswrapper[2578]: W0716 17:14:14.689765 2578 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 16 17:14:14.789956 ip-10-0-132-130 kubenswrapper[2578]: W0716 17:14:14.689772 2578 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 16 17:14:14.789956 ip-10-0-132-130 kubenswrapper[2578]: W0716 17:14:14.689774 2578 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 16 17:14:14.789956 ip-10-0-132-130 kubenswrapper[2578]: W0716 17:14:14.689777 2578 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 16 17:14:14.789956 ip-10-0-132-130 kubenswrapper[2578]: W0716 17:14:14.689779 2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 16 17:14:14.789956 ip-10-0-132-130 kubenswrapper[2578]: W0716 17:14:14.689782 2578 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 16 17:14:14.794013 ip-10-0-132-130 kubenswrapper[2578]: W0716 17:14:14.689784 2578 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 16 17:14:14.794013 ip-10-0-132-130 kubenswrapper[2578]: W0716 17:14:14.689791 2578 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 16 17:14:14.794013 ip-10-0-132-130 kubenswrapper[2578]: W0716 17:14:14.689794 2578 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 16 17:14:14.794013 ip-10-0-132-130 kubenswrapper[2578]: W0716 17:14:14.689796 2578 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 16 17:14:14.794013 ip-10-0-132-130 kubenswrapper[2578]: W0716 17:14:14.689798 2578 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 16 17:14:14.794013 ip-10-0-132-130 kubenswrapper[2578]: W0716 17:14:14.689802 2578 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 16 17:14:14.794013 ip-10-0-132-130 kubenswrapper[2578]: W0716 17:14:14.689805 2578 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 16 17:14:14.794013 ip-10-0-132-130 kubenswrapper[2578]: W0716 17:14:14.689811 2578 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 16 17:14:14.794013 ip-10-0-132-130 kubenswrapper[2578]: W0716 17:14:14.689814 2578 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 16 17:14:14.794013 ip-10-0-132-130 kubenswrapper[2578]: W0716 17:14:14.689816 2578 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 16 17:14:14.794013 ip-10-0-132-130 kubenswrapper[2578]: W0716 17:14:14.689818 2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 16 17:14:14.794013 ip-10-0-132-130 kubenswrapper[2578]: W0716 17:14:14.689821 2578 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 16 17:14:14.794013 ip-10-0-132-130 kubenswrapper[2578]: W0716 17:14:14.689824 2578 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 16 17:14:14.794013 ip-10-0-132-130 kubenswrapper[2578]: W0716 17:14:14.689830 2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 16 17:14:14.794013 ip-10-0-132-130 kubenswrapper[2578]: W0716 17:14:14.689833 2578 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 16 17:14:14.794013 ip-10-0-132-130 kubenswrapper[2578]: W0716 17:14:14.689837 2578 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 16 17:14:14.794013 ip-10-0-132-130 kubenswrapper[2578]: W0716 17:14:14.689840 2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 16 17:14:14.794013 ip-10-0-132-130 kubenswrapper[2578]: W0716 17:14:14.689843 2578 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 16 17:14:14.794013 ip-10-0-132-130 kubenswrapper[2578]: W0716 17:14:14.689851 2578 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 16 17:14:14.808569 ip-10-0-132-130 kubenswrapper[2578]: W0716 17:14:14.689854 2578 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 16 17:14:14.808569 ip-10-0-132-130 kubenswrapper[2578]: W0716 17:14:14.689858 2578 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 16 17:14:14.808569 ip-10-0-132-130 kubenswrapper[2578]: W0716 17:14:14.689862 2578 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 16 17:14:14.808569 ip-10-0-132-130 kubenswrapper[2578]: W0716 17:14:14.689865 2578 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 16 17:14:14.808569 ip-10-0-132-130 kubenswrapper[2578]: W0716 17:14:14.689873 2578 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 16 17:14:14.808569 ip-10-0-132-130 kubenswrapper[2578]: W0716 17:14:14.689876 2578 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 16 17:14:14.808569 ip-10-0-132-130 kubenswrapper[2578]: W0716 17:14:14.689879 2578 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 16 17:14:14.808569 ip-10-0-132-130 kubenswrapper[2578]: W0716 17:14:14.689881 2578 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 16 17:14:14.808569 ip-10-0-132-130 kubenswrapper[2578]: W0716 17:14:14.689884 2578 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 16 17:14:14.808569 ip-10-0-132-130 kubenswrapper[2578]: W0716 17:14:14.689892 2578 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 16 17:14:14.808569 ip-10-0-132-130 kubenswrapper[2578]: W0716 17:14:14.689895 2578 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 16 17:14:14.808569 ip-10-0-132-130 kubenswrapper[2578]: W0716 17:14:14.689898 2578 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 16 17:14:14.808569 ip-10-0-132-130 kubenswrapper[2578]: W0716 17:14:14.689900 2578 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 16 17:14:14.808569 ip-10-0-132-130 kubenswrapper[2578]: W0716 17:14:14.689903 2578 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 16 17:14:14.808569 ip-10-0-132-130 kubenswrapper[2578]: W0716 17:14:14.689905 2578 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 16 17:14:14.808569 ip-10-0-132-130 kubenswrapper[2578]: W0716 17:14:14.689912 2578 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 16 17:14:14.808569 ip-10-0-132-130 kubenswrapper[2578]: W0716 17:14:14.689914 2578 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 16 17:14:14.808569 ip-10-0-132-130 kubenswrapper[2578]: W0716 17:14:14.689917 2578 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 16 17:14:14.808569 ip-10-0-132-130 kubenswrapper[2578]: W0716 17:14:14.689921 2578 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 16 17:14:14.956330 ip-10-0-132-130 kubenswrapper[2578]: W0716 17:14:14.689923 2578 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 16 17:14:14.956330 ip-10-0-132-130 kubenswrapper[2578]: W0716 17:14:14.689926 2578 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 16 17:14:14.956330 ip-10-0-132-130 kubenswrapper[2578]: W0716 17:14:14.689932 2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 16 17:14:14.956330 ip-10-0-132-130 kubenswrapper[2578]: W0716 17:14:14.689935 2578 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 16 17:14:14.956330 ip-10-0-132-130 kubenswrapper[2578]: W0716 17:14:14.689938 2578 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 16 17:14:14.956330 ip-10-0-132-130 kubenswrapper[2578]: W0716 17:14:14.689940 2578 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 16 17:14:14.956330 ip-10-0-132-130 kubenswrapper[2578]: W0716 17:14:14.689943 2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 16 17:14:14.956330 ip-10-0-132-130 kubenswrapper[2578]: W0716 17:14:14.689946 2578 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 16 17:14:14.956330 ip-10-0-132-130 kubenswrapper[2578]: W0716 17:14:14.689952 2578 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 16 17:14:14.956330 ip-10-0-132-130 kubenswrapper[2578]: W0716 17:14:14.689954 2578 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 16 17:14:14.956330 ip-10-0-132-130 kubenswrapper[2578]: W0716 17:14:14.689957 2578 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 16 17:14:14.956330 ip-10-0-132-130 kubenswrapper[2578]: W0716 17:14:14.689959 2578 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 16 17:14:14.956330 ip-10-0-132-130 kubenswrapper[2578]: W0716 17:14:14.689961 2578 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 16 17:14:14.956330 ip-10-0-132-130 kubenswrapper[2578]: W0716 17:14:14.689964 2578 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 16 17:14:14.956330 ip-10-0-132-130 kubenswrapper[2578]: W0716 17:14:14.689966 2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 16 17:14:14.956330 ip-10-0-132-130 kubenswrapper[2578]: W0716 17:14:14.689970 2578 feature_gate.go:328] unrecognized feature gate: Example Jul 16 17:14:14.956330 ip-10-0-132-130 kubenswrapper[2578]: W0716 17:14:14.689975 2578 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 16 17:14:14.956330 ip-10-0-132-130 kubenswrapper[2578]: W0716 17:14:14.689980 2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 16 17:14:14.956330 ip-10-0-132-130 kubenswrapper[2578]: W0716 17:14:14.689983 2578 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 16 17:14:14.956330 ip-10-0-132-130 kubenswrapper[2578]: W0716 17:14:14.689986 2578 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 16 17:14:14.959042 ip-10-0-132-130 kubenswrapper[2578]: W0716 17:14:14.689988 2578 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 16 17:14:14.959042 ip-10-0-132-130 kubenswrapper[2578]: W0716 17:14:14.689990 2578 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 16 17:14:14.959042 ip-10-0-132-130 kubenswrapper[2578]: W0716 17:14:14.689993 2578 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 16 17:14:14.959042 ip-10-0-132-130 kubenswrapper[2578]: W0716 17:14:14.689995 2578 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 16 17:14:14.959042 ip-10-0-132-130 kubenswrapper[2578]: W0716 17:14:14.689998 2578 feature_gate.go:328] unrecognized feature gate: Example2 Jul 16 17:14:14.959042 ip-10-0-132-130 kubenswrapper[2578]: W0716 17:14:14.690000 2578 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 16 17:14:14.959042 ip-10-0-132-130 kubenswrapper[2578]: W0716 17:14:14.690006 2578 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 16 17:14:14.959042 ip-10-0-132-130 kubenswrapper[2578]: W0716 17:14:14.690009 2578 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 16 17:14:14.959042 ip-10-0-132-130 kubenswrapper[2578]: W0716 17:14:14.690011 2578 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 16 17:14:14.959042 ip-10-0-132-130 kubenswrapper[2578]: W0716 17:14:14.690013 2578 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 16 17:14:14.959042 ip-10-0-132-130 kubenswrapper[2578]: W0716 17:14:14.690016 2578 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 16 17:14:14.959042 ip-10-0-132-130 kubenswrapper[2578]: W0716 17:14:14.690020 2578 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 16 17:14:14.959042 ip-10-0-132-130 kubenswrapper[2578]: W0716 17:14:14.690023 2578 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 16 17:14:14.959042 ip-10-0-132-130 kubenswrapper[2578]: W0716 17:14:14.690029 2578 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 16 17:14:14.959042 ip-10-0-132-130 kubenswrapper[2578]: W0716 17:14:14.690031 2578 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 16 17:14:14.959042 ip-10-0-132-130 kubenswrapper[2578]: W0716 17:14:14.690035 2578 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 16 17:14:14.959042 ip-10-0-132-130 kubenswrapper[2578]: W0716 17:14:14.690037 2578 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 16 17:14:14.959042 ip-10-0-132-130 kubenswrapper[2578]: W0716 17:14:14.690040 2578 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 16 17:14:14.959042 ip-10-0-132-130 kubenswrapper[2578]: W0716 17:14:14.690043 2578 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 16 17:14:14.959042 ip-10-0-132-130 kubenswrapper[2578]: W0716 17:14:14.690049 2578 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 16 17:14:14.959668 ip-10-0-132-130 kubenswrapper[2578]: W0716 17:14:14.690052 2578 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 16 17:14:14.959668 ip-10-0-132-130 kubenswrapper[2578]: W0716 17:14:14.690054 2578 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 16 17:14:14.959668 ip-10-0-132-130 kubenswrapper[2578]: W0716 17:14:14.690057 2578 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 16 17:14:14.959668 ip-10-0-132-130 kubenswrapper[2578]: W0716 17:14:14.690059 2578 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 16 17:14:14.959668 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:14:14.690141 2578 flags.go:64] FLAG: --address="0.0.0.0" Jul 16 17:14:14.959668 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:14:14.690152 2578 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]" Jul 16 17:14:14.959668 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:14:14.690158 2578 flags.go:64] FLAG: --anonymous-auth="true" Jul 16 17:14:14.959668 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:14:14.690162 2578 flags.go:64] FLAG: --application-metrics-count-limit="100" Jul 16 17:14:14.959668 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:14:14.690170 2578 flags.go:64] FLAG: --authentication-token-webhook="false" Jul 16 17:14:14.959668 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:14:14.690175 2578 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s" Jul 16 17:14:14.959668 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:14:14.690178 2578 flags.go:64] FLAG: --authorization-mode="AlwaysAllow" Jul 16 17:14:14.959668 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:14:14.690186 2578 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s" Jul 16 17:14:14.959668 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:14:14.690189 2578 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s" Jul 16 17:14:14.959668 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:14:14.690193 2578 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id" Jul 16 17:14:14.959668 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:14:14.690196 2578 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig" Jul 16 17:14:14.959668 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:14:14.690199 2578 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki" Jul 16 17:14:14.959668 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:14:14.690202 2578 flags.go:64] FLAG: --cgroup-driver="cgroupfs" Jul 16 17:14:14.959668 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:14:14.690223 2578 flags.go:64] FLAG: --cgroup-root="" Jul 16 17:14:14.959668 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:14:14.690230 2578 flags.go:64] FLAG: --cgroups-per-qos="true" Jul 16 17:14:14.959668 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:14:14.690233 2578 flags.go:64] FLAG: --client-ca-file="" Jul 16 17:14:14.959668 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:14:14.690236 2578 flags.go:64] FLAG: --cloud-provider="external" Jul 16 17:14:14.959668 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:14:14.690240 2578 flags.go:64] FLAG: --cluster-dns="[]" Jul 16 17:14:14.959668 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:14:14.690244 2578 flags.go:64] FLAG: --cluster-domain="" Jul 16 17:14:14.962565 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:14:14.690252 2578 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf" Jul 16 17:14:14.962565 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:14:14.690256 2578 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d" Jul 16 17:14:14.962565 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:14:14.690259 2578 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json" Jul 16 17:14:14.962565 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:14:14.690262 2578 flags.go:64] FLAG: --container-log-max-files="5" Jul 16 17:14:14.962565 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:14:14.690265 2578 flags.go:64] FLAG: --container-log-max-size="10Mi" Jul 16 17:14:14.962565 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:14:14.690272 2578 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock" Jul 16 17:14:14.962565 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:14:14.690275 2578 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock" Jul 16 17:14:14.962565 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:14:14.690278 2578 flags.go:64] FLAG: --containerd-namespace="k8s.io" Jul 16 17:14:14.962565 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:14:14.690281 2578 flags.go:64] FLAG: --contention-profiling="false" Jul 16 17:14:14.962565 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:14:14.690284 2578 flags.go:64] FLAG: --cpu-cfs-quota="true" Jul 16 17:14:14.962565 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:14:14.690286 2578 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms" Jul 16 17:14:14.962565 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:14:14.690294 2578 flags.go:64] FLAG: --cpu-manager-policy="none" Jul 16 17:14:14.962565 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:14:14.690297 2578 flags.go:64] FLAG: --cpu-manager-policy-options="" Jul 16 17:14:14.962565 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:14:14.690302 2578 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s" Jul 16 17:14:14.962565 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:14:14.690305 2578 flags.go:64] FLAG: --enable-controller-attach-detach="true" Jul 16 17:14:14.962565 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:14:14.690312 2578 flags.go:64] FLAG: --enable-debugging-handlers="true" Jul 16 17:14:14.962565 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:14:14.690314 2578 flags.go:64] FLAG: --enable-load-reader="false" Jul 16 17:14:14.962565 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:14:14.690318 2578 flags.go:64] FLAG: --enable-server="true" Jul 16 17:14:14.962565 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:14:14.690323 2578 flags.go:64] FLAG: --enforce-node-allocatable="[pods]" Jul 16 17:14:14.962565 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:14:14.690327 2578 flags.go:64] FLAG: --event-burst="100" Jul 16 17:14:14.962565 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:14:14.690333 2578 flags.go:64] FLAG: --event-qps="50" Jul 16 17:14:14.962565 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:14:14.690336 2578 flags.go:64] FLAG: --event-storage-age-limit="default=0" Jul 16 17:14:14.962565 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:14:14.690339 2578 flags.go:64] FLAG: --event-storage-event-limit="default=0" Jul 16 17:14:14.962565 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:14:14.690342 2578 flags.go:64] FLAG: --eviction-hard="" Jul 16 17:14:14.964723 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:14:14.690345 2578 flags.go:64] FLAG: --eviction-max-pod-grace-period="0" Jul 16 17:14:14.964723 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:14:14.690348 2578 flags.go:64] FLAG: --eviction-minimum-reclaim="" Jul 16 17:14:14.964723 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:14:14.690351 2578 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s" Jul 16 17:14:14.964723 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:14:14.690354 2578 flags.go:64] FLAG: --eviction-soft="" Jul 16 17:14:14.964723 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:14:14.690357 2578 flags.go:64] FLAG: --eviction-soft-grace-period="" Jul 16 17:14:14.964723 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:14:14.690363 2578 flags.go:64] FLAG: --exit-on-lock-contention="false" Jul 16 17:14:14.964723 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:14:14.690366 2578 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false" Jul 16 17:14:14.964723 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:14:14.690368 2578 flags.go:64] FLAG: --experimental-mounter-path="" Jul 16 17:14:14.964723 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:14:14.690373 2578 flags.go:64] FLAG: --fail-cgroupv1="false" Jul 16 17:14:14.964723 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:14:14.690376 2578 flags.go:64] FLAG: --fail-swap-on="true" Jul 16 17:14:14.964723 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:14:14.690382 2578 flags.go:64] FLAG: --feature-gates="" Jul 16 17:14:14.964723 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:14:14.690386 2578 flags.go:64] FLAG: --file-check-frequency="20s" Jul 16 17:14:14.964723 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:14:14.690389 2578 flags.go:64] FLAG: --global-housekeeping-interval="1m0s" Jul 16 17:14:14.964723 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:14:14.690392 2578 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge" Jul 16 17:14:14.964723 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:14:14.690395 2578 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1" Jul 16 17:14:14.964723 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:14:14.690401 2578 flags.go:64] FLAG: --healthz-port="10248" Jul 16 17:14:14.964723 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:14:14.690405 2578 flags.go:64] FLAG: --help="false" Jul 16 17:14:14.964723 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:14:14.690407 2578 flags.go:64] FLAG: --hostname-override="ip-10-0-132-130.ec2.internal" Jul 16 17:14:14.964723 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:14:14.690411 2578 flags.go:64] FLAG: --housekeeping-interval="10s" Jul 16 17:14:14.964723 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:14:14.690414 2578 flags.go:64] FLAG: --http-check-frequency="20s" Jul 16 17:14:14.964723 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:14:14.690416 2578 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins" Jul 16 17:14:14.964723 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:14:14.690424 2578 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml" Jul 16 17:14:14.964723 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:14:14.690427 2578 flags.go:64] FLAG: --image-gc-high-threshold="85" Jul 16 17:14:14.993370 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:14:14.690430 2578 flags.go:64] FLAG: --image-gc-low-threshold="80" Jul 16 17:14:14.993370 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:14:14.690433 2578 flags.go:64] FLAG: --image-service-endpoint="" Jul 16 17:14:14.993370 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:14:14.690436 2578 flags.go:64] FLAG: --kernel-memcg-notification="false" Jul 16 17:14:14.993370 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:14:14.690444 2578 flags.go:64] FLAG: --kube-api-burst="100" Jul 16 17:14:14.993370 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:14:14.690447 2578 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf" Jul 16 17:14:14.993370 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:14:14.690450 2578 flags.go:64] FLAG: --kube-api-qps="50" Jul 16 17:14:14.993370 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:14:14.690453 2578 flags.go:64] FLAG: --kube-reserved="" Jul 16 17:14:14.993370 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:14:14.690456 2578 flags.go:64] FLAG: --kube-reserved-cgroup="" Jul 16 17:14:14.993370 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:14:14.690462 2578 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig" Jul 16 17:14:14.993370 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:14:14.690466 2578 flags.go:64] FLAG: --kubelet-cgroups="" Jul 16 17:14:14.993370 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:14:14.690469 2578 flags.go:64] FLAG: --local-storage-capacity-isolation="true" Jul 16 17:14:14.993370 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:14:14.690471 2578 flags.go:64] FLAG: --lock-file="" Jul 16 17:14:14.993370 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:14:14.690474 2578 flags.go:64] FLAG: --log-cadvisor-usage="false" Jul 16 17:14:14.993370 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:14:14.690477 2578 flags.go:64] FLAG: --log-flush-frequency="5s" Jul 16 17:14:14.993370 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:14:14.690484 2578 flags.go:64] FLAG: --log-json-info-buffer-size="0" Jul 16 17:14:14.993370 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:14:14.690489 2578 flags.go:64] FLAG: --log-json-split-stream="false" Jul 16 17:14:14.993370 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:14:14.690492 2578 flags.go:64] FLAG: --log-text-info-buffer-size="0" Jul 16 17:14:14.993370 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:14:14.690496 2578 flags.go:64] FLAG: --log-text-split-stream="false" Jul 16 17:14:14.993370 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:14:14.690498 2578 flags.go:64] FLAG: --logging-format="text" Jul 16 17:14:14.993370 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:14:14.690505 2578 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id" Jul 16 17:14:14.993370 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:14:14.690508 2578 flags.go:64] FLAG: --make-iptables-util-chains="true" Jul 16 17:14:14.993370 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:14:14.690511 2578 flags.go:64] FLAG: --manifest-url="" Jul 16 17:14:14.993370 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:14:14.690514 2578 flags.go:64] FLAG: --manifest-url-header="" Jul 16 17:14:14.993370 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:14:14.690518 2578 flags.go:64] FLAG: --max-housekeeping-interval="15s" Jul 16 17:14:14.993370 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:14:14.690521 2578 flags.go:64] FLAG: --max-open-files="1000000" Jul 16 17:14:14.995863 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:14:14.690529 2578 flags.go:64] FLAG: --max-pods="110" Jul 16 17:14:14.995863 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:14:14.690532 2578 flags.go:64] FLAG: --maximum-dead-containers="-1" Jul 16 17:14:14.995863 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:14:14.690536 2578 flags.go:64] FLAG: --maximum-dead-containers-per-container="1" Jul 16 17:14:14.995863 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:14:14.690538 2578 flags.go:64] FLAG: --memory-manager-policy="None" Jul 16 17:14:14.995863 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:14:14.690541 2578 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s" Jul 16 17:14:14.995863 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:14:14.690548 2578 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s" Jul 16 17:14:14.995863 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:14:14.690551 2578 flags.go:64] FLAG: --node-ip="0.0.0.0" Jul 16 17:14:14.995863 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:14:14.690554 2578 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel" Jul 16 17:14:14.995863 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:14:14.690561 2578 flags.go:64] FLAG: --node-status-max-images="50" Jul 16 17:14:14.995863 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:14:14.690568 2578 flags.go:64] FLAG: --node-status-update-frequency="10s" Jul 16 17:14:14.995863 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:14:14.690572 2578 flags.go:64] FLAG: --oom-score-adj="-999" Jul 16 17:14:14.995863 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:14:14.690575 2578 flags.go:64] FLAG: --pod-cidr="" Jul 16 17:14:14.995863 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:14:14.690579 2578 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:b04c0b1f73af4d021dd8f0b51bfeb302c24482c093de5f2c43654ab56c506926" Jul 16 17:14:14.995863 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:14:14.690587 2578 flags.go:64] FLAG: --pod-manifest-path="" Jul 16 17:14:14.995863 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:14:14.690590 2578 flags.go:64] FLAG: --pod-max-pids="-1" Jul 16 17:14:14.995863 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:14:14.690593 2578 flags.go:64] FLAG: --pods-per-core="0" Jul 16 17:14:14.995863 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:14:14.690596 2578 flags.go:64] FLAG: --port="10250" Jul 16 17:14:14.995863 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:14:14.690599 2578 flags.go:64] FLAG: --protect-kernel-defaults="false" Jul 16 17:14:14.995863 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:14:14.690602 2578 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-00a450734869e0e4d" Jul 16 17:14:14.995863 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:14:14.690609 2578 flags.go:64] FLAG: --qos-reserved="" Jul 16 17:14:14.995863 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:14:14.690611 2578 flags.go:64] FLAG: --read-only-port="10255" Jul 16 17:14:14.995863 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:14:14.690614 2578 flags.go:64] FLAG: --register-node="true" Jul 16 17:14:14.995863 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:14:14.690617 2578 flags.go:64] FLAG: --register-with-taints="" Jul 16 17:14:14.995863 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:14:14.690620 2578 flags.go:64] FLAG: --registry-burst="10" Jul 16 17:14:14.999266 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:14:14.690628 2578 flags.go:64] FLAG: --registry-qps="5" Jul 16 17:14:14.999266 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:14:14.690631 2578 flags.go:64] FLAG: --reserved-cpus="" Jul 16 17:14:14.999266 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:14:14.690634 2578 flags.go:64] FLAG: --reserved-memory="" Jul 16 17:14:14.999266 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:14:14.690637 2578 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf" Jul 16 17:14:14.999266 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:14:14.690640 2578 flags.go:64] FLAG: --root-dir="/var/lib/kubelet" Jul 16 17:14:14.999266 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:14:14.690643 2578 flags.go:64] FLAG: --rotate-certificates="false" Jul 16 17:14:14.999266 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:14:14.690651 2578 flags.go:64] FLAG: --rotate-server-certificates="false" Jul 16 17:14:14.999266 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:14:14.690654 2578 flags.go:64] FLAG: --runonce="false" Jul 16 17:14:14.999266 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:14:14.690657 2578 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service" Jul 16 17:14:14.999266 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:14:14.690660 2578 flags.go:64] FLAG: --runtime-request-timeout="2m0s" Jul 16 17:14:14.999266 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:14:14.690663 2578 flags.go:64] FLAG: --seccomp-default="false" Jul 16 17:14:14.999266 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:14:14.690670 2578 flags.go:64] FLAG: --serialize-image-pulls="true" Jul 16 17:14:14.999266 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:14:14.690672 2578 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s" Jul 16 17:14:14.999266 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:14:14.690675 2578 flags.go:64] FLAG: --storage-driver-db="cadvisor" Jul 16 17:14:14.999266 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:14:14.690678 2578 flags.go:64] FLAG: --storage-driver-host="localhost:8086" Jul 16 17:14:14.999266 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:14:14.690681 2578 flags.go:64] FLAG: --storage-driver-password="root" Jul 16 17:14:14.999266 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:14:14.690684 2578 flags.go:64] FLAG: --storage-driver-secure="false" Jul 16 17:14:14.999266 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:14:14.690690 2578 flags.go:64] FLAG: --storage-driver-table="stats" Jul 16 17:14:14.999266 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:14:14.690693 2578 flags.go:64] FLAG: --storage-driver-user="root" Jul 16 17:14:14.999266 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:14:14.690697 2578 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s" Jul 16 17:14:14.999266 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:14:14.690700 2578 flags.go:64] FLAG: --sync-frequency="1m0s" Jul 16 17:14:14.999266 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:14:14.690704 2578 flags.go:64] FLAG: --system-cgroups="" Jul 16 17:14:14.999266 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:14:14.690706 2578 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi" Jul 16 17:14:14.999266 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:14:14.690711 2578 flags.go:64] FLAG: --system-reserved-cgroup="" Jul 16 17:14:14.999266 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:14:14.690714 2578 flags.go:64] FLAG: --tls-cert-file="" Jul 16 17:14:15.000414 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:14:14.690720 2578 flags.go:64] FLAG: --tls-cipher-suites="[]" Jul 16 17:14:15.000414 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:14:14.690724 2578 flags.go:64] FLAG: --tls-min-version="" Jul 16 17:14:15.000414 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:14:14.690727 2578 flags.go:64] FLAG: --tls-private-key-file="" Jul 16 17:14:15.000414 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:14:14.690730 2578 flags.go:64] FLAG: --topology-manager-policy="none" Jul 16 17:14:15.000414 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:14:14.690733 2578 flags.go:64] FLAG: --topology-manager-policy-options="" Jul 16 17:14:15.000414 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:14:14.690739 2578 flags.go:64] FLAG: --topology-manager-scope="container" Jul 16 17:14:15.000414 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:14:14.690742 2578 flags.go:64] FLAG: --v="2" Jul 16 17:14:15.000414 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:14:14.690747 2578 flags.go:64] FLAG: --version="false" Jul 16 17:14:15.000414 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:14:14.690751 2578 flags.go:64] FLAG: --vmodule="" Jul 16 17:14:15.000414 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:14:14.690759 2578 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec" Jul 16 17:14:15.000414 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:14:14.690762 2578 flags.go:64] FLAG: --volume-stats-agg-period="1m0s" Jul 16 17:14:15.000414 ip-10-0-132-130 kubenswrapper[2578]: W0716 17:14:14.690890 2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 16 17:14:15.000414 ip-10-0-132-130 kubenswrapper[2578]: W0716 17:14:14.690894 2578 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 16 17:14:15.000414 ip-10-0-132-130 kubenswrapper[2578]: W0716 17:14:14.690902 2578 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 16 17:14:15.000414 ip-10-0-132-130 kubenswrapper[2578]: W0716 17:14:14.690904 2578 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 16 17:14:15.000414 ip-10-0-132-130 kubenswrapper[2578]: W0716 17:14:14.690908 2578 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 16 17:14:15.000414 ip-10-0-132-130 kubenswrapper[2578]: W0716 17:14:14.690910 2578 feature_gate.go:328] unrecognized feature gate: Example2 Jul 16 17:14:15.000414 ip-10-0-132-130 kubenswrapper[2578]: W0716 17:14:14.690913 2578 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 16 17:14:15.000414 ip-10-0-132-130 kubenswrapper[2578]: W0716 17:14:14.690920 2578 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 16 17:14:15.000414 ip-10-0-132-130 kubenswrapper[2578]: W0716 17:14:14.690923 2578 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 16 17:14:15.000414 ip-10-0-132-130 kubenswrapper[2578]: W0716 17:14:14.690925 2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 16 17:14:15.000414 ip-10-0-132-130 kubenswrapper[2578]: W0716 17:14:14.690928 2578 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 16 17:14:15.000414 ip-10-0-132-130 kubenswrapper[2578]: W0716 17:14:14.690930 2578 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 16 17:14:15.003060 ip-10-0-132-130 kubenswrapper[2578]: W0716 17:14:14.690933 2578 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 16 17:14:15.003060 ip-10-0-132-130 kubenswrapper[2578]: W0716 17:14:14.690939 2578 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 16 17:14:15.003060 ip-10-0-132-130 kubenswrapper[2578]: W0716 17:14:14.690943 2578 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 16 17:14:15.003060 ip-10-0-132-130 kubenswrapper[2578]: W0716 17:14:14.690947 2578 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 16 17:14:15.003060 ip-10-0-132-130 kubenswrapper[2578]: W0716 17:14:14.690950 2578 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 16 17:14:15.003060 ip-10-0-132-130 kubenswrapper[2578]: W0716 17:14:14.690952 2578 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 16 17:14:15.003060 ip-10-0-132-130 kubenswrapper[2578]: W0716 17:14:14.690955 2578 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 16 17:14:15.003060 ip-10-0-132-130 kubenswrapper[2578]: W0716 17:14:14.690961 2578 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 16 17:14:15.003060 ip-10-0-132-130 kubenswrapper[2578]: W0716 17:14:14.690964 2578 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 16 17:14:15.003060 ip-10-0-132-130 kubenswrapper[2578]: W0716 17:14:14.690967 2578 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 16 17:14:15.003060 ip-10-0-132-130 kubenswrapper[2578]: W0716 17:14:14.690969 2578 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 16 17:14:15.003060 ip-10-0-132-130 kubenswrapper[2578]: W0716 17:14:14.690972 2578 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 16 17:14:15.003060 ip-10-0-132-130 kubenswrapper[2578]: W0716 17:14:14.690974 2578 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 16 17:14:15.003060 ip-10-0-132-130 kubenswrapper[2578]: W0716 17:14:14.690981 2578 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 16 17:14:15.003060 ip-10-0-132-130 kubenswrapper[2578]: W0716 17:14:14.690983 2578 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 16 17:14:15.003060 ip-10-0-132-130 kubenswrapper[2578]: W0716 17:14:14.690985 2578 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 16 17:14:15.003060 ip-10-0-132-130 kubenswrapper[2578]: W0716 17:14:14.690990 2578 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 16 17:14:15.003060 ip-10-0-132-130 kubenswrapper[2578]: W0716 17:14:14.690993 2578 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 16 17:14:15.003060 ip-10-0-132-130 kubenswrapper[2578]: W0716 17:14:14.690999 2578 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 16 17:14:15.003060 ip-10-0-132-130 kubenswrapper[2578]: W0716 17:14:14.691002 2578 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 16 17:14:15.003862 ip-10-0-132-130 kubenswrapper[2578]: W0716 17:14:14.691004 2578 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 16 17:14:15.003862 ip-10-0-132-130 kubenswrapper[2578]: W0716 17:14:14.691007 2578 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 16 17:14:15.003862 ip-10-0-132-130 kubenswrapper[2578]: W0716 17:14:14.691009 2578 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 16 17:14:15.003862 ip-10-0-132-130 kubenswrapper[2578]: W0716 17:14:14.691012 2578 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 16 17:14:15.003862 ip-10-0-132-130 kubenswrapper[2578]: W0716 17:14:14.691018 2578 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 16 17:14:15.003862 ip-10-0-132-130 kubenswrapper[2578]: W0716 17:14:14.691021 2578 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 16 17:14:15.003862 ip-10-0-132-130 kubenswrapper[2578]: W0716 17:14:14.691023 2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 16 17:14:15.003862 ip-10-0-132-130 kubenswrapper[2578]: W0716 17:14:14.691026 2578 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 16 17:14:15.003862 ip-10-0-132-130 kubenswrapper[2578]: W0716 17:14:14.691028 2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 16 17:14:15.003862 ip-10-0-132-130 kubenswrapper[2578]: W0716 17:14:14.691031 2578 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 16 17:14:15.003862 ip-10-0-132-130 kubenswrapper[2578]: W0716 17:14:14.691033 2578 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 16 17:14:15.003862 ip-10-0-132-130 kubenswrapper[2578]: W0716 17:14:14.691036 2578 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 16 17:14:15.003862 ip-10-0-132-130 kubenswrapper[2578]: W0716 17:14:14.691042 2578 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 16 17:14:15.003862 ip-10-0-132-130 kubenswrapper[2578]: W0716 17:14:14.691045 2578 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 16 17:14:15.003862 ip-10-0-132-130 kubenswrapper[2578]: W0716 17:14:14.691047 2578 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 16 17:14:15.003862 ip-10-0-132-130 kubenswrapper[2578]: W0716 17:14:14.691052 2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 16 17:14:15.003862 ip-10-0-132-130 kubenswrapper[2578]: W0716 17:14:14.691054 2578 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 16 17:14:15.003862 ip-10-0-132-130 kubenswrapper[2578]: W0716 17:14:14.691056 2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 16 17:14:15.003862 ip-10-0-132-130 kubenswrapper[2578]: W0716 17:14:14.691059 2578 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 16 17:14:15.003862 ip-10-0-132-130 kubenswrapper[2578]: W0716 17:14:14.691062 2578 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 16 17:14:15.004800 ip-10-0-132-130 kubenswrapper[2578]: W0716 17:14:14.691064 2578 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 16 17:14:15.004800 ip-10-0-132-130 kubenswrapper[2578]: W0716 17:14:14.691067 2578 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 16 17:14:15.004800 ip-10-0-132-130 kubenswrapper[2578]: W0716 17:14:14.691073 2578 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 16 17:14:15.004800 ip-10-0-132-130 kubenswrapper[2578]: W0716 17:14:14.691075 2578 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 16 17:14:15.004800 ip-10-0-132-130 kubenswrapper[2578]: W0716 17:14:14.691078 2578 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 16 17:14:15.004800 ip-10-0-132-130 kubenswrapper[2578]: W0716 17:14:14.691080 2578 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 16 17:14:15.004800 ip-10-0-132-130 kubenswrapper[2578]: W0716 17:14:14.691083 2578 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 16 17:14:15.004800 ip-10-0-132-130 kubenswrapper[2578]: W0716 17:14:14.691085 2578 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 16 17:14:15.004800 ip-10-0-132-130 kubenswrapper[2578]: W0716 17:14:14.691094 2578 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 16 17:14:15.004800 ip-10-0-132-130 kubenswrapper[2578]: W0716 17:14:14.691098 2578 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 16 17:14:15.004800 ip-10-0-132-130 kubenswrapper[2578]: W0716 17:14:14.691100 2578 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 16 17:14:15.004800 ip-10-0-132-130 kubenswrapper[2578]: W0716 17:14:14.691103 2578 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 16 17:14:15.004800 ip-10-0-132-130 kubenswrapper[2578]: W0716 17:14:14.691105 2578 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 16 17:14:15.004800 ip-10-0-132-130 kubenswrapper[2578]: W0716 17:14:14.691107 2578 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 16 17:14:15.004800 ip-10-0-132-130 kubenswrapper[2578]: W0716 17:14:14.691114 2578 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 16 17:14:15.004800 ip-10-0-132-130 kubenswrapper[2578]: W0716 17:14:14.691116 2578 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 16 17:14:15.004800 ip-10-0-132-130 kubenswrapper[2578]: W0716 17:14:14.691119 2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 16 17:14:15.004800 ip-10-0-132-130 kubenswrapper[2578]: W0716 17:14:14.691121 2578 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 16 17:14:15.004800 ip-10-0-132-130 kubenswrapper[2578]: W0716 17:14:14.691124 2578 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 16 17:14:15.005361 ip-10-0-132-130 kubenswrapper[2578]: W0716 17:14:14.691128 2578 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 16 17:14:15.005361 ip-10-0-132-130 kubenswrapper[2578]: W0716 17:14:14.691132 2578 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 16 17:14:15.005361 ip-10-0-132-130 kubenswrapper[2578]: W0716 17:14:14.691134 2578 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 16 17:14:15.005361 ip-10-0-132-130 kubenswrapper[2578]: W0716 17:14:14.691137 2578 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 16 17:14:15.005361 ip-10-0-132-130 kubenswrapper[2578]: W0716 17:14:14.691139 2578 feature_gate.go:328] unrecognized feature gate: Example Jul 16 17:14:15.005361 ip-10-0-132-130 kubenswrapper[2578]: W0716 17:14:14.691141 2578 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 16 17:14:15.005361 ip-10-0-132-130 kubenswrapper[2578]: W0716 17:14:14.691144 2578 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 16 17:14:15.005361 ip-10-0-132-130 kubenswrapper[2578]: W0716 17:14:14.691146 2578 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 16 17:14:15.005361 ip-10-0-132-130 kubenswrapper[2578]: W0716 17:14:14.691150 2578 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 16 17:14:15.005361 ip-10-0-132-130 kubenswrapper[2578]: W0716 17:14:14.691153 2578 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 16 17:14:15.005361 ip-10-0-132-130 kubenswrapper[2578]: W0716 17:14:14.691155 2578 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 16 17:14:15.005361 ip-10-0-132-130 kubenswrapper[2578]: W0716 17:14:14.691158 2578 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 16 17:14:15.005361 ip-10-0-132-130 kubenswrapper[2578]: W0716 17:14:14.691161 2578 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 16 17:14:15.005361 ip-10-0-132-130 kubenswrapper[2578]: W0716 17:14:14.691164 2578 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 16 17:14:15.005361 ip-10-0-132-130 kubenswrapper[2578]: W0716 17:14:14.691166 2578 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 16 17:14:15.005361 ip-10-0-132-130 kubenswrapper[2578]: W0716 17:14:14.691169 2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 16 17:14:15.005361 ip-10-0-132-130 kubenswrapper[2578]: W0716 17:14:14.691171 2578 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 16 17:14:15.005361 ip-10-0-132-130 kubenswrapper[2578]: W0716 17:14:14.691173 2578 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 16 17:14:15.005361 ip-10-0-132-130 kubenswrapper[2578]: W0716 17:14:14.691176 2578 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 16 17:14:15.005361 ip-10-0-132-130 kubenswrapper[2578]: W0716 17:14:14.691178 2578 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 16 17:14:15.005977 ip-10-0-132-130 kubenswrapper[2578]: W0716 17:14:14.691181 2578 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 16 17:14:15.005977 ip-10-0-132-130 kubenswrapper[2578]: W0716 17:14:14.691188 2578 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 16 17:14:15.005977 ip-10-0-132-130 kubenswrapper[2578]: W0716 17:14:14.691191 2578 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 16 17:14:15.005977 ip-10-0-132-130 kubenswrapper[2578]: W0716 17:14:14.691194 2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 16 17:14:15.005977 ip-10-0-132-130 kubenswrapper[2578]: W0716 17:14:14.691196 2578 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 16 17:14:15.005977 ip-10-0-132-130 kubenswrapper[2578]: W0716 17:14:14.691198 2578 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 16 17:14:15.005977 ip-10-0-132-130 kubenswrapper[2578]: W0716 17:14:14.691201 2578 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 16 17:14:15.005977 ip-10-0-132-130 kubenswrapper[2578]: W0716 17:14:14.691205 2578 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 16 17:14:15.005977 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:14:14.691228 2578 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 16 17:14:15.005977 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:14:14.699076 2578 server.go:529] "Kubelet version" kubeletVersion="v1.34.9" Jul 16 17:14:15.005977 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:14:14.699091 2578 server.go:531] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK="" Jul 16 17:14:15.005977 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:14:14.699110 2578 watchdog_linux.go:95] "Systemd watchdog is not enabled" Jul 16 17:14:15.005977 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:14:14.699115 2578 watchdog_linux.go:137] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started." Jul 16 17:14:15.005977 ip-10-0-132-130 kubenswrapper[2578]: W0716 17:14:14.699178 2578 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 16 17:14:15.006672 ip-10-0-132-130 kubenswrapper[2578]: W0716 17:14:14.699184 2578 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 16 17:14:15.006672 ip-10-0-132-130 kubenswrapper[2578]: W0716 17:14:14.699187 2578 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 16 17:14:15.006672 ip-10-0-132-130 kubenswrapper[2578]: W0716 17:14:14.699190 2578 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 16 17:14:15.006672 ip-10-0-132-130 kubenswrapper[2578]: W0716 17:14:14.699193 2578 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 16 17:14:15.006672 ip-10-0-132-130 kubenswrapper[2578]: W0716 17:14:14.699196 2578 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 16 17:14:15.006672 ip-10-0-132-130 kubenswrapper[2578]: W0716 17:14:14.699199 2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 16 17:14:15.006672 ip-10-0-132-130 kubenswrapper[2578]: W0716 17:14:14.699202 2578 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 16 17:14:15.006672 ip-10-0-132-130 kubenswrapper[2578]: W0716 17:14:14.699205 2578 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 16 17:14:15.006672 ip-10-0-132-130 kubenswrapper[2578]: W0716 17:14:14.699223 2578 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 16 17:14:15.006672 ip-10-0-132-130 kubenswrapper[2578]: W0716 17:14:14.699228 2578 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 16 17:14:15.006672 ip-10-0-132-130 kubenswrapper[2578]: W0716 17:14:14.699232 2578 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 16 17:14:15.006672 ip-10-0-132-130 kubenswrapper[2578]: W0716 17:14:14.699235 2578 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 16 17:14:15.006672 ip-10-0-132-130 kubenswrapper[2578]: W0716 17:14:14.699238 2578 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 16 17:14:15.006672 ip-10-0-132-130 kubenswrapper[2578]: W0716 17:14:14.699241 2578 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 16 17:14:15.006672 ip-10-0-132-130 kubenswrapper[2578]: W0716 17:14:14.699244 2578 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 16 17:14:15.006672 ip-10-0-132-130 kubenswrapper[2578]: W0716 17:14:14.699246 2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 16 17:14:15.006672 ip-10-0-132-130 kubenswrapper[2578]: W0716 17:14:14.699249 2578 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 16 17:14:15.006672 ip-10-0-132-130 kubenswrapper[2578]: W0716 17:14:14.699252 2578 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 16 17:14:15.006672 ip-10-0-132-130 kubenswrapper[2578]: W0716 17:14:14.699254 2578 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 16 17:14:15.006672 ip-10-0-132-130 kubenswrapper[2578]: W0716 17:14:14.699256 2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 16 17:14:15.007240 ip-10-0-132-130 kubenswrapper[2578]: W0716 17:14:14.699259 2578 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 16 17:14:15.007240 ip-10-0-132-130 kubenswrapper[2578]: W0716 17:14:14.699262 2578 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 16 17:14:15.007240 ip-10-0-132-130 kubenswrapper[2578]: W0716 17:14:14.699265 2578 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 16 17:14:15.007240 ip-10-0-132-130 kubenswrapper[2578]: W0716 17:14:14.699267 2578 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 16 17:14:15.007240 ip-10-0-132-130 kubenswrapper[2578]: W0716 17:14:14.699270 2578 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 16 17:14:15.007240 ip-10-0-132-130 kubenswrapper[2578]: W0716 17:14:14.699273 2578 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 16 17:14:15.007240 ip-10-0-132-130 kubenswrapper[2578]: W0716 17:14:14.699276 2578 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 16 17:14:15.007240 ip-10-0-132-130 kubenswrapper[2578]: W0716 17:14:14.699278 2578 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 16 17:14:15.007240 ip-10-0-132-130 kubenswrapper[2578]: W0716 17:14:14.699287 2578 feature_gate.go:328] unrecognized feature gate: Example Jul 16 17:14:15.007240 ip-10-0-132-130 kubenswrapper[2578]: W0716 17:14:14.699290 2578 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 16 17:14:15.007240 ip-10-0-132-130 kubenswrapper[2578]: W0716 17:14:14.699293 2578 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 16 17:14:15.007240 ip-10-0-132-130 kubenswrapper[2578]: W0716 17:14:14.699297 2578 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 16 17:14:15.007240 ip-10-0-132-130 kubenswrapper[2578]: W0716 17:14:14.699299 2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 16 17:14:15.007240 ip-10-0-132-130 kubenswrapper[2578]: W0716 17:14:14.699302 2578 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 16 17:14:15.007240 ip-10-0-132-130 kubenswrapper[2578]: W0716 17:14:14.699305 2578 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 16 17:14:15.007240 ip-10-0-132-130 kubenswrapper[2578]: W0716 17:14:14.699307 2578 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 16 17:14:15.007240 ip-10-0-132-130 kubenswrapper[2578]: W0716 17:14:14.699310 2578 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 16 17:14:15.007240 ip-10-0-132-130 kubenswrapper[2578]: W0716 17:14:14.699312 2578 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 16 17:14:15.007240 ip-10-0-132-130 kubenswrapper[2578]: W0716 17:14:14.699314 2578 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 16 17:14:15.007240 ip-10-0-132-130 kubenswrapper[2578]: W0716 17:14:14.699317 2578 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 16 17:14:15.007795 ip-10-0-132-130 kubenswrapper[2578]: W0716 17:14:14.699320 2578 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 16 17:14:15.007795 ip-10-0-132-130 kubenswrapper[2578]: W0716 17:14:14.699322 2578 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 16 17:14:15.007795 ip-10-0-132-130 kubenswrapper[2578]: W0716 17:14:14.699325 2578 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 16 17:14:15.007795 ip-10-0-132-130 kubenswrapper[2578]: W0716 17:14:14.699328 2578 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 16 17:14:15.007795 ip-10-0-132-130 kubenswrapper[2578]: W0716 17:14:14.699330 2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 16 17:14:15.007795 ip-10-0-132-130 kubenswrapper[2578]: W0716 17:14:14.699333 2578 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 16 17:14:15.007795 ip-10-0-132-130 kubenswrapper[2578]: W0716 17:14:14.699335 2578 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 16 17:14:15.007795 ip-10-0-132-130 kubenswrapper[2578]: W0716 17:14:14.699338 2578 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 16 17:14:15.007795 ip-10-0-132-130 kubenswrapper[2578]: W0716 17:14:14.699340 2578 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 16 17:14:15.007795 ip-10-0-132-130 kubenswrapper[2578]: W0716 17:14:14.699343 2578 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 16 17:14:15.007795 ip-10-0-132-130 kubenswrapper[2578]: W0716 17:14:14.699345 2578 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 16 17:14:15.007795 ip-10-0-132-130 kubenswrapper[2578]: W0716 17:14:14.699348 2578 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 16 17:14:15.007795 ip-10-0-132-130 kubenswrapper[2578]: W0716 17:14:14.699350 2578 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 16 17:14:15.007795 ip-10-0-132-130 kubenswrapper[2578]: W0716 17:14:14.699353 2578 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 16 17:14:15.007795 ip-10-0-132-130 kubenswrapper[2578]: W0716 17:14:14.699359 2578 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 16 17:14:15.007795 ip-10-0-132-130 kubenswrapper[2578]: W0716 17:14:14.699362 2578 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 16 17:14:15.007795 ip-10-0-132-130 kubenswrapper[2578]: W0716 17:14:14.699366 2578 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 16 17:14:15.007795 ip-10-0-132-130 kubenswrapper[2578]: W0716 17:14:14.699369 2578 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 16 17:14:15.007795 ip-10-0-132-130 kubenswrapper[2578]: W0716 17:14:14.699372 2578 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 16 17:14:15.007795 ip-10-0-132-130 kubenswrapper[2578]: W0716 17:14:14.699375 2578 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 16 17:14:15.011640 ip-10-0-132-130 kubenswrapper[2578]: W0716 17:14:14.699378 2578 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 16 17:14:15.011640 ip-10-0-132-130 kubenswrapper[2578]: W0716 17:14:14.699387 2578 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 16 17:14:15.011640 ip-10-0-132-130 kubenswrapper[2578]: W0716 17:14:14.699390 2578 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 16 17:14:15.011640 ip-10-0-132-130 kubenswrapper[2578]: W0716 17:14:14.699393 2578 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 16 17:14:15.011640 ip-10-0-132-130 kubenswrapper[2578]: W0716 17:14:14.699395 2578 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 16 17:14:15.011640 ip-10-0-132-130 kubenswrapper[2578]: W0716 17:14:14.699398 2578 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 16 17:14:15.011640 ip-10-0-132-130 kubenswrapper[2578]: W0716 17:14:14.699400 2578 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 16 17:14:15.011640 ip-10-0-132-130 kubenswrapper[2578]: W0716 17:14:14.699402 2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 16 17:14:15.011640 ip-10-0-132-130 kubenswrapper[2578]: W0716 17:14:14.699405 2578 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 16 17:14:15.011640 ip-10-0-132-130 kubenswrapper[2578]: W0716 17:14:14.699407 2578 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 16 17:14:15.011640 ip-10-0-132-130 kubenswrapper[2578]: W0716 17:14:14.699410 2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 16 17:14:15.011640 ip-10-0-132-130 kubenswrapper[2578]: W0716 17:14:14.699412 2578 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 16 17:14:15.011640 ip-10-0-132-130 kubenswrapper[2578]: W0716 17:14:14.699415 2578 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 16 17:14:15.011640 ip-10-0-132-130 kubenswrapper[2578]: W0716 17:14:14.699417 2578 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 16 17:14:15.011640 ip-10-0-132-130 kubenswrapper[2578]: W0716 17:14:14.699420 2578 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 16 17:14:15.011640 ip-10-0-132-130 kubenswrapper[2578]: W0716 17:14:14.699422 2578 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 16 17:14:15.011640 ip-10-0-132-130 kubenswrapper[2578]: W0716 17:14:14.699425 2578 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 16 17:14:15.011640 ip-10-0-132-130 kubenswrapper[2578]: W0716 17:14:14.699427 2578 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 16 17:14:15.011640 ip-10-0-132-130 kubenswrapper[2578]: W0716 17:14:14.699430 2578 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 16 17:14:15.012472 ip-10-0-132-130 kubenswrapper[2578]: W0716 17:14:14.699432 2578 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 16 17:14:15.012472 ip-10-0-132-130 kubenswrapper[2578]: W0716 17:14:14.699434 2578 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 16 17:14:15.012472 ip-10-0-132-130 kubenswrapper[2578]: W0716 17:14:14.699437 2578 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 16 17:14:15.012472 ip-10-0-132-130 kubenswrapper[2578]: W0716 17:14:14.699439 2578 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 16 17:14:15.012472 ip-10-0-132-130 kubenswrapper[2578]: W0716 17:14:14.699441 2578 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 16 17:14:15.012472 ip-10-0-132-130 kubenswrapper[2578]: W0716 17:14:14.699444 2578 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 16 17:14:15.012472 ip-10-0-132-130 kubenswrapper[2578]: W0716 17:14:14.699446 2578 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 16 17:14:15.012472 ip-10-0-132-130 kubenswrapper[2578]: W0716 17:14:14.699449 2578 feature_gate.go:328] unrecognized feature gate: Example2 Jul 16 17:14:15.012472 ip-10-0-132-130 kubenswrapper[2578]: W0716 17:14:14.699451 2578 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 16 17:14:15.012472 ip-10-0-132-130 kubenswrapper[2578]: W0716 17:14:14.699454 2578 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 16 17:14:15.012472 ip-10-0-132-130 kubenswrapper[2578]: W0716 17:14:14.699457 2578 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 16 17:14:15.012472 ip-10-0-132-130 kubenswrapper[2578]: W0716 17:14:14.699460 2578 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 16 17:14:15.012472 ip-10-0-132-130 kubenswrapper[2578]: W0716 17:14:14.699462 2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 16 17:14:15.012472 ip-10-0-132-130 kubenswrapper[2578]: W0716 17:14:14.699465 2578 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 16 17:14:15.012472 ip-10-0-132-130 kubenswrapper[2578]: W0716 17:14:14.699467 2578 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 16 17:14:15.012472 ip-10-0-132-130 kubenswrapper[2578]: W0716 17:14:14.699476 2578 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 16 17:14:15.012472 ip-10-0-132-130 kubenswrapper[2578]: W0716 17:14:14.699480 2578 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 16 17:14:15.012472 ip-10-0-132-130 kubenswrapper[2578]: W0716 17:14:14.699483 2578 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 16 17:14:15.012472 ip-10-0-132-130 kubenswrapper[2578]: W0716 17:14:14.699488 2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 16 17:14:15.126583 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:14:14.699493 2578 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 16 17:14:15.126583 ip-10-0-132-130 kubenswrapper[2578]: W0716 17:14:14.699648 2578 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 16 17:14:15.126583 ip-10-0-132-130 kubenswrapper[2578]: W0716 17:14:14.699653 2578 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 16 17:14:15.126583 ip-10-0-132-130 kubenswrapper[2578]: W0716 17:14:14.699656 2578 feature_gate.go:328] unrecognized feature gate: Example2 Jul 16 17:14:15.126583 ip-10-0-132-130 kubenswrapper[2578]: W0716 17:14:14.699658 2578 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 16 17:14:15.126583 ip-10-0-132-130 kubenswrapper[2578]: W0716 17:14:14.699661 2578 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 16 17:14:15.126583 ip-10-0-132-130 kubenswrapper[2578]: W0716 17:14:14.699664 2578 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 16 17:14:15.126583 ip-10-0-132-130 kubenswrapper[2578]: W0716 17:14:14.699668 2578 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 16 17:14:15.126583 ip-10-0-132-130 kubenswrapper[2578]: W0716 17:14:14.699671 2578 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 16 17:14:15.126583 ip-10-0-132-130 kubenswrapper[2578]: W0716 17:14:14.699674 2578 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 16 17:14:15.126583 ip-10-0-132-130 kubenswrapper[2578]: W0716 17:14:14.699677 2578 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 16 17:14:15.126583 ip-10-0-132-130 kubenswrapper[2578]: W0716 17:14:14.699680 2578 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 16 17:14:15.126583 ip-10-0-132-130 kubenswrapper[2578]: W0716 17:14:14.699683 2578 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 16 17:14:15.126583 ip-10-0-132-130 kubenswrapper[2578]: W0716 17:14:14.699686 2578 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 16 17:14:15.126997 ip-10-0-132-130 kubenswrapper[2578]: W0716 17:14:14.699689 2578 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 16 17:14:15.126997 ip-10-0-132-130 kubenswrapper[2578]: W0716 17:14:14.699692 2578 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 16 17:14:15.126997 ip-10-0-132-130 kubenswrapper[2578]: W0716 17:14:14.699694 2578 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 16 17:14:15.126997 ip-10-0-132-130 kubenswrapper[2578]: W0716 17:14:14.699697 2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 16 17:14:15.126997 ip-10-0-132-130 kubenswrapper[2578]: W0716 17:14:14.699699 2578 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 16 17:14:15.126997 ip-10-0-132-130 kubenswrapper[2578]: W0716 17:14:14.699702 2578 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 16 17:14:15.126997 ip-10-0-132-130 kubenswrapper[2578]: W0716 17:14:14.699704 2578 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 16 17:14:15.126997 ip-10-0-132-130 kubenswrapper[2578]: W0716 17:14:14.699706 2578 feature_gate.go:328] unrecognized feature gate: Example Jul 16 17:14:15.126997 ip-10-0-132-130 kubenswrapper[2578]: W0716 17:14:14.699709 2578 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 16 17:14:15.126997 ip-10-0-132-130 kubenswrapper[2578]: W0716 17:14:14.699711 2578 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 16 17:14:15.126997 ip-10-0-132-130 kubenswrapper[2578]: W0716 17:14:14.699714 2578 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 16 17:14:15.126997 ip-10-0-132-130 kubenswrapper[2578]: W0716 17:14:14.699716 2578 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 16 17:14:15.126997 ip-10-0-132-130 kubenswrapper[2578]: W0716 17:14:14.699719 2578 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 16 17:14:15.126997 ip-10-0-132-130 kubenswrapper[2578]: W0716 17:14:14.699722 2578 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 16 17:14:15.126997 ip-10-0-132-130 kubenswrapper[2578]: W0716 17:14:14.699725 2578 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 16 17:14:15.126997 ip-10-0-132-130 kubenswrapper[2578]: W0716 17:14:14.699733 2578 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 16 17:14:15.126997 ip-10-0-132-130 kubenswrapper[2578]: W0716 17:14:14.699736 2578 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 16 17:14:15.126997 ip-10-0-132-130 kubenswrapper[2578]: W0716 17:14:14.699740 2578 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 16 17:14:15.126997 ip-10-0-132-130 kubenswrapper[2578]: W0716 17:14:14.699742 2578 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 16 17:14:15.126997 ip-10-0-132-130 kubenswrapper[2578]: W0716 17:14:14.699745 2578 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 16 17:14:15.127773 ip-10-0-132-130 kubenswrapper[2578]: W0716 17:14:14.699747 2578 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 16 17:14:15.127773 ip-10-0-132-130 kubenswrapper[2578]: W0716 17:14:14.699750 2578 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 16 17:14:15.127773 ip-10-0-132-130 kubenswrapper[2578]: W0716 17:14:14.699752 2578 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 16 17:14:15.127773 ip-10-0-132-130 kubenswrapper[2578]: W0716 17:14:14.699755 2578 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 16 17:14:15.127773 ip-10-0-132-130 kubenswrapper[2578]: W0716 17:14:14.699757 2578 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 16 17:14:15.127773 ip-10-0-132-130 kubenswrapper[2578]: W0716 17:14:14.699759 2578 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 16 17:14:15.127773 ip-10-0-132-130 kubenswrapper[2578]: W0716 17:14:14.699763 2578 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 16 17:14:15.127773 ip-10-0-132-130 kubenswrapper[2578]: W0716 17:14:14.699766 2578 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 16 17:14:15.127773 ip-10-0-132-130 kubenswrapper[2578]: W0716 17:14:14.699769 2578 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 16 17:14:15.127773 ip-10-0-132-130 kubenswrapper[2578]: W0716 17:14:14.699772 2578 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 16 17:14:15.127773 ip-10-0-132-130 kubenswrapper[2578]: W0716 17:14:14.699775 2578 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 16 17:14:15.127773 ip-10-0-132-130 kubenswrapper[2578]: W0716 17:14:14.699777 2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 16 17:14:15.127773 ip-10-0-132-130 kubenswrapper[2578]: W0716 17:14:14.699780 2578 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 16 17:14:15.127773 ip-10-0-132-130 kubenswrapper[2578]: W0716 17:14:14.699782 2578 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 16 17:14:15.127773 ip-10-0-132-130 kubenswrapper[2578]: W0716 17:14:14.699785 2578 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 16 17:14:15.127773 ip-10-0-132-130 kubenswrapper[2578]: W0716 17:14:14.699788 2578 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 16 17:14:15.127773 ip-10-0-132-130 kubenswrapper[2578]: W0716 17:14:14.699791 2578 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 16 17:14:15.127773 ip-10-0-132-130 kubenswrapper[2578]: W0716 17:14:14.699793 2578 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 16 17:14:15.127773 ip-10-0-132-130 kubenswrapper[2578]: W0716 17:14:14.699796 2578 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 16 17:14:15.127773 ip-10-0-132-130 kubenswrapper[2578]: W0716 17:14:14.699798 2578 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 16 17:14:15.128336 ip-10-0-132-130 kubenswrapper[2578]: W0716 17:14:14.699801 2578 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 16 17:14:15.128336 ip-10-0-132-130 kubenswrapper[2578]: W0716 17:14:14.699804 2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 16 17:14:15.128336 ip-10-0-132-130 kubenswrapper[2578]: W0716 17:14:14.699806 2578 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 16 17:14:15.128336 ip-10-0-132-130 kubenswrapper[2578]: W0716 17:14:14.699809 2578 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 16 17:14:15.128336 ip-10-0-132-130 kubenswrapper[2578]: W0716 17:14:14.699812 2578 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 16 17:14:15.128336 ip-10-0-132-130 kubenswrapper[2578]: W0716 17:14:14.699814 2578 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 16 17:14:15.128336 ip-10-0-132-130 kubenswrapper[2578]: W0716 17:14:14.699817 2578 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 16 17:14:15.128336 ip-10-0-132-130 kubenswrapper[2578]: W0716 17:14:14.699819 2578 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 16 17:14:15.128336 ip-10-0-132-130 kubenswrapper[2578]: W0716 17:14:14.699827 2578 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 16 17:14:15.128336 ip-10-0-132-130 kubenswrapper[2578]: W0716 17:14:14.699831 2578 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 16 17:14:15.128336 ip-10-0-132-130 kubenswrapper[2578]: W0716 17:14:14.699834 2578 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 16 17:14:15.128336 ip-10-0-132-130 kubenswrapper[2578]: W0716 17:14:14.699836 2578 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 16 17:14:15.128336 ip-10-0-132-130 kubenswrapper[2578]: W0716 17:14:14.699839 2578 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 16 17:14:15.128336 ip-10-0-132-130 kubenswrapper[2578]: W0716 17:14:14.699841 2578 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 16 17:14:15.128336 ip-10-0-132-130 kubenswrapper[2578]: W0716 17:14:14.699844 2578 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 16 17:14:15.128336 ip-10-0-132-130 kubenswrapper[2578]: W0716 17:14:14.699846 2578 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 16 17:14:15.128336 ip-10-0-132-130 kubenswrapper[2578]: W0716 17:14:14.699848 2578 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 16 17:14:15.128336 ip-10-0-132-130 kubenswrapper[2578]: W0716 17:14:14.699851 2578 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 16 17:14:15.128336 ip-10-0-132-130 kubenswrapper[2578]: W0716 17:14:14.699853 2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 16 17:14:15.246044 ip-10-0-132-130 kubenswrapper[2578]: W0716 17:14:14.699856 2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 16 17:14:15.246044 ip-10-0-132-130 kubenswrapper[2578]: W0716 17:14:14.699858 2578 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 16 17:14:15.246044 ip-10-0-132-130 kubenswrapper[2578]: W0716 17:14:14.699860 2578 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 16 17:14:15.246044 ip-10-0-132-130 kubenswrapper[2578]: W0716 17:14:14.699863 2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 16 17:14:15.246044 ip-10-0-132-130 kubenswrapper[2578]: W0716 17:14:14.699865 2578 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 16 17:14:15.246044 ip-10-0-132-130 kubenswrapper[2578]: W0716 17:14:14.699867 2578 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 16 17:14:15.246044 ip-10-0-132-130 kubenswrapper[2578]: W0716 17:14:14.699870 2578 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 16 17:14:15.246044 ip-10-0-132-130 kubenswrapper[2578]: W0716 17:14:14.699873 2578 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 16 17:14:15.246044 ip-10-0-132-130 kubenswrapper[2578]: W0716 17:14:14.699875 2578 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 16 17:14:15.246044 ip-10-0-132-130 kubenswrapper[2578]: W0716 17:14:14.699877 2578 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 16 17:14:15.246044 ip-10-0-132-130 kubenswrapper[2578]: W0716 17:14:14.699880 2578 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 16 17:14:15.246044 ip-10-0-132-130 kubenswrapper[2578]: W0716 17:14:14.699882 2578 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 16 17:14:15.246044 ip-10-0-132-130 kubenswrapper[2578]: W0716 17:14:14.699885 2578 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 16 17:14:15.246044 ip-10-0-132-130 kubenswrapper[2578]: W0716 17:14:14.699887 2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 16 17:14:15.246044 ip-10-0-132-130 kubenswrapper[2578]: W0716 17:14:14.699890 2578 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 16 17:14:15.246044 ip-10-0-132-130 kubenswrapper[2578]: W0716 17:14:14.699892 2578 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 16 17:14:15.246044 ip-10-0-132-130 kubenswrapper[2578]: W0716 17:14:14.699895 2578 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 16 17:14:15.246044 ip-10-0-132-130 kubenswrapper[2578]: W0716 17:14:14.699897 2578 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 16 17:14:15.246044 ip-10-0-132-130 kubenswrapper[2578]: W0716 17:14:14.699899 2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 16 17:14:15.246044 ip-10-0-132-130 kubenswrapper[2578]: W0716 17:14:14.699902 2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 16 17:14:15.246691 ip-10-0-132-130 kubenswrapper[2578]: W0716 17:14:14.699904 2578 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 16 17:14:15.246691 ip-10-0-132-130 kubenswrapper[2578]: W0716 17:14:14.699907 2578 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 16 17:14:15.246691 ip-10-0-132-130 kubenswrapper[2578]: W0716 17:14:14.699917 2578 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 16 17:14:15.246691 ip-10-0-132-130 kubenswrapper[2578]: W0716 17:14:14.699919 2578 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 16 17:14:15.246691 ip-10-0-132-130 kubenswrapper[2578]: W0716 17:14:14.699922 2578 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 16 17:14:15.246691 ip-10-0-132-130 kubenswrapper[2578]: W0716 17:14:14.699924 2578 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 16 17:14:15.246691 ip-10-0-132-130 kubenswrapper[2578]: W0716 17:14:14.699927 2578 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 16 17:14:15.246691 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:14:14.699931 2578 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 16 17:14:15.246691 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:14:14.700045 2578 server.go:962] "Client rotation is on, will bootstrap in background" Jul 16 17:14:15.246691 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:14:14.702590 2578 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir" Jul 16 17:14:15.246691 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:14:14.704218 2578 server.go:1019] "Starting client certificate rotation" Jul 16 17:14:15.246691 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:14:14.704338 2578 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet" Jul 16 17:14:15.246691 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:14:14.704382 2578 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet" Jul 16 17:14:15.247132 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:14:14.710762 2578 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Jul 16 17:14:15.247132 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:14:14.711955 2578 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Jul 16 17:14:15.247132 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:14:14.727097 2578 log.go:25] "Validated CRI v1 runtime API" Jul 16 17:14:15.247132 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:14:14.729574 2578 log.go:25] "Validated CRI v1 image API" Jul 16 17:14:15.247132 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:14:14.731484 2578 server.go:1429] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd" Jul 16 17:14:15.247132 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:14:14.734153 2578 fs.go:135] Filesystem UUIDs: map[02e27e3e-5497-439d-b429-ea0f44eae448:/dev/nvme0n1p3 7B77-95E7:/dev/nvme0n1p2 f9257d75-dff5-46a8-b993-a48dc840a048:/dev/nvme0n1p4] Jul 16 17:14:15.247132 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:14:14.734172 2578 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}] Jul 16 17:14:15.247516 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:14:14.740839 2578 manager.go:216] Machine: {Timestamp:2026-07-16 17:14:14.739430471 +0000 UTC m=+0.375753866 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3189970 MemoryCapacity:33164484608 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec23d49d842738236fdcc81f9a4d0fba SystemUUID:ec23d49d-8427-3823-6fdc-c81f9a4d0fba BootID:901e0ff2-8fb3-4521-9954-6aee33186f16 Filesystems:[{Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6632898560 Type:vfs Inodes:819200 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6107136 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16582242304 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true} {Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16582242304 Type:vfs Inodes:4048399 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:75:2c:72:71:cf Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:75:2c:72:71:cf Speed:0 Mtu:9001} {Name:ovs-system MacAddress:ba:65:d2:a6:48:db Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:33164484608 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:37486592 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None} Jul 16 17:14:15.247516 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:14:14.740930 2578 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available. Jul 16 17:14:15.247516 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:14:14.741000 2578 manager.go:232] Version: {KernelVersion:5.14.0-570.126.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260702-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:} Jul 16 17:14:15.247516 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:14:14.744389 2578 container_manager_linux.go:273] "Container manager verified user specified cgroup-root exists" cgroupRoot=[] Jul 16 17:14:15.247516 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:14:14.744414 2578 container_manager_linux.go:278] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-132-130.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2} Jul 16 17:14:15.247516 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:14:14.745223 2578 topology_manager.go:138] "Creating topology manager with none policy" Jul 16 17:14:15.247516 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:14:14.745233 2578 container_manager_linux.go:309] "Creating device plugin manager" Jul 16 17:14:15.247516 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:14:14.745240 2578 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock" Jul 16 17:14:15.247516 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:14:14.745255 2578 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock" Jul 16 17:14:15.247516 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:14:14.746520 2578 container_manager_linux.go:318] "Creating Dynamic Resource Allocation (DRA) manager" Jul 16 17:14:15.247516 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:14:14.751465 2578 reflector.go:439] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Jul 16 17:14:15.247516 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:14:14.751489 2578 state_mem.go:36] "Initialized new in-memory state store" Jul 16 17:14:15.247516 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:14:14.751603 2578 server.go:1240] "Using root directory" path="/var/lib/kubelet" Jul 16 17:14:15.247516 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:14:14.751629 2578 kubelet.go:486] "Attempting to sync node with API server" Jul 16 17:14:15.247516 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:14:14.751641 2578 kubelet.go:387] "Adding static pod path" path="/etc/kubernetes/manifests" Jul 16 17:14:15.247516 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:14:14.751658 2578 file.go:69] "Watching path" path="/etc/kubernetes/manifests" Jul 16 17:14:15.247516 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:14:14.751668 2578 kubelet.go:398] "Adding apiserver pod source" Jul 16 17:14:15.248066 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:14:14.751677 2578 apiserver.go:42] "Waiting for node sync before watching apiserver pods" Jul 16 17:14:15.248066 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:14:14.751990 2578 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Jul 16 17:14:15.248066 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:14:14.752005 2578 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Jul 16 17:14:15.248066 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:14:14.755443 2578 kuberuntime_manager.go:301] "Container runtime initialized" containerRuntime="cri-o" version="1.34.10-2.rhaos4.21.git83ee25b.el9" apiVersion="v1" Jul 16 17:14:15.248066 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:14:14.760400 2578 kubelet.go:960] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled" Jul 16 17:14:15.248066 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:14:14.760428 2578 kubelet.go:984] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled" Jul 16 17:14:15.248066 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:14:14.761251 2578 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume" Jul 16 17:14:15.248066 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:14:14.761264 2578 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir" Jul 16 17:14:15.248066 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:14:14.761270 2578 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo" Jul 16 17:14:15.248066 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:14:14.761277 2578 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path" Jul 16 17:14:15.248066 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:14:14.761285 2578 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs" Jul 16 17:14:15.248066 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:14:14.761290 2578 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret" Jul 16 17:14:15.248066 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:14:14.761298 2578 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi" Jul 16 17:14:15.248066 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:14:14.761304 2578 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api" Jul 16 17:14:15.248066 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:14:14.761311 2578 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc" Jul 16 17:14:15.248066 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:14:14.761317 2578 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap" Jul 16 17:14:15.248066 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:14:14.761324 2578 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected" Jul 16 17:14:15.248066 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:14:14.761329 2578 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume" Jul 16 17:14:15.248066 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:14:14.762257 2578 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi" Jul 16 17:14:15.248066 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:14:14.762264 2578 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image" Jul 16 17:14:15.248066 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:14:14.764980 2578 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-132-130.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope Jul 16 17:14:15.248066 ip-10-0-132-130 kubenswrapper[2578]: E0716 17:14:14.765124 2578 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service" Jul 16 17:14:15.248066 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:14:14.765248 2578 server.go:1268] "Started kubelet" Jul 16 17:14:15.248066 ip-10-0-132-130 kubenswrapper[2578]: E0716 17:14:14.765341 2578 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-132-130.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node" Jul 16 17:14:15.248066 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:14:14.765365 2578 server.go:180] "Starting to listen" address="0.0.0.0" port=10250 Jul 16 17:14:15.248066 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:14:14.765373 2578 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10 Jul 16 17:14:15.248066 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:14:14.765433 2578 server_v1.go:49] "podresources" method="list" useActivePods=true Jul 16 17:14:15.248066 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:14:14.767478 2578 server.go:310] "Adding debug handlers to kubelet server" Jul 16 17:14:15.248066 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:14:14.767493 2578 server.go:249] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock" Jul 16 17:14:15.248066 ip-10-0-132-130 kubenswrapper[2578]: E0716 17:14:14.770635 2578 kubelet.go:1638] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache" Jul 16 17:14:15.248066 ip-10-0-132-130 kubenswrapper[2578]: E0716 17:14:14.770482 2578 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-132-130.ec2.internal.18c2d4816eb2f54c default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-132-130.ec2.internal,UID:ip-10-0-132-130.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-132-130.ec2.internal,},FirstTimestamp:2026-07-16 17:14:14.765229388 +0000 UTC m=+0.401552780,LastTimestamp:2026-07-16 17:14:14.765229388 +0000 UTC m=+0.401552780,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-132-130.ec2.internal,}" Jul 16 17:14:15.248066 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:14:14.775264 2578 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer" Jul 16 17:14:15.248066 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:14:14.775275 2578 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving" Jul 16 17:14:15.248066 ip-10-0-132-130 kubenswrapper[2578]: E0716 17:14:14.775383 2578 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-132-130.ec2.internal\" not found" Jul 16 17:14:15.248066 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:14:14.776025 2578 volume_manager.go:311] "The desired_state_of_world populator starts" Jul 16 17:14:15.248066 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:14:14.776047 2578 volume_manager.go:313] "Starting Kubelet Volume Manager" Jul 16 17:14:15.248066 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:14:14.776032 2578 factory.go:55] Registering systemd factory Jul 16 17:14:15.248066 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:14:14.776086 2578 factory.go:223] Registration of the systemd container factory successfully Jul 16 17:14:15.248066 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:14:14.776145 2578 desired_state_of_world_populator.go:146] "Desired state populator starts to run" Jul 16 17:14:15.248066 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:14:14.776173 2578 reconstruct.go:97] "Volume reconstruction finished" Jul 16 17:14:15.248066 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:14:14.776184 2578 reconciler.go:29] "Reconciler: start to sync state" Jul 16 17:14:15.249245 ip-10-0-132-130 kubenswrapper[2578]: E0716 17:14:14.777110 2578 reflector.go:205] "Failed to watch" err="failed to list *v1.CSIDriver: csidrivers.storage.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"csidrivers\" in API group \"storage.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver" Jul 16 17:14:15.249245 ip-10-0-132-130 kubenswrapper[2578]: E0716 17:14:14.777150 2578 controller.go:145] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-132-130.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="200ms" Jul 16 17:14:15.249245 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:14:14.779698 2578 factory.go:153] Registering CRI-O factory Jul 16 17:14:15.249245 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:14:14.779714 2578 factory.go:223] Registration of the crio container factory successfully Jul 16 17:14:15.249245 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:14:14.779777 2578 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory Jul 16 17:14:15.249245 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:14:14.779805 2578 factory.go:103] Registering Raw factory Jul 16 17:14:15.249245 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:14:14.779823 2578 manager.go:1196] Started watching for new ooms in manager Jul 16 17:14:15.249245 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:14:14.780722 2578 manager.go:349] Starting recovery of all containers Jul 16 17:14:15.249245 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:14:14.787003 2578 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-bxxk7" Jul 16 17:14:15.249245 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:14:14.790935 2578 manager.go:354] Recovery completed Jul 16 17:14:15.249245 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:14:14.797455 2578 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 16 17:14:15.249245 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:14:14.797952 2578 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-bxxk7" Jul 16 17:14:15.249245 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:14:14.799422 2578 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-130.ec2.internal" event="NodeHasSufficientMemory" Jul 16 17:14:15.249245 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:14:14.799452 2578 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-130.ec2.internal" event="NodeHasNoDiskPressure" Jul 16 17:14:15.249245 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:14:14.799467 2578 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-130.ec2.internal" event="NodeHasSufficientPID" Jul 16 17:14:15.249245 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:14:14.800042 2578 cpu_manager.go:222] "Starting CPU manager" policy="none" Jul 16 17:14:15.249245 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:14:14.800053 2578 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s" Jul 16 17:14:15.249245 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:14:14.800069 2578 state_mem.go:36] "Initialized new in-memory state store" Jul 16 17:14:15.249245 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:14:14.800330 2578 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 16 17:14:15.249245 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:14:14.801036 2578 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-130.ec2.internal" event="NodeHasSufficientMemory" Jul 16 17:14:15.249245 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:14:14.801057 2578 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-130.ec2.internal" event="NodeHasNoDiskPressure" Jul 16 17:14:15.249245 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:14:14.801068 2578 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-130.ec2.internal" event="NodeHasSufficientPID" Jul 16 17:14:15.249245 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:14:14.802273 2578 policy_none.go:49] "None policy: Start" Jul 16 17:14:15.249245 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:14:14.802287 2578 memory_manager.go:187] "Starting memorymanager" policy="None" Jul 16 17:14:15.249245 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:14:14.802296 2578 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint" Jul 16 17:14:15.249245 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:14:14.804443 2578 policy_none.go:47] "Start" Jul 16 17:14:15.249245 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:14:14.829395 2578 manager.go:337] "Starting Device Plugin manager" Jul 16 17:14:15.249245 ip-10-0-132-130 kubenswrapper[2578]: E0716 17:14:14.829528 2578 manager.go:513] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint" Jul 16 17:14:15.249245 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:14:14.829542 2578 server.go:87] "Starting device plugin registration server" Jul 16 17:14:15.249245 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:14:14.829710 2578 eviction_manager.go:189] "Eviction manager: starting control loop" Jul 16 17:14:15.249245 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:14:14.829719 2578 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s" Jul 16 17:14:15.249245 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:14:14.829817 2578 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry" Jul 16 17:14:15.249245 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:14:14.829896 2578 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts" Jul 16 17:14:15.249245 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:14:14.829906 2578 plugin_manager.go:118] "Starting Kubelet Plugin Manager" Jul 16 17:14:15.249245 ip-10-0-132-130 kubenswrapper[2578]: E0716 17:14:14.830379 2578 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\"" Jul 16 17:14:15.249245 ip-10-0-132-130 kubenswrapper[2578]: E0716 17:14:14.830424 2578 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-132-130.ec2.internal\" not found" Jul 16 17:14:15.249245 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:14:14.930096 2578 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 16 17:14:15.249245 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:14:14.930925 2578 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-130.ec2.internal" event="NodeHasSufficientMemory" Jul 16 17:14:15.249245 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:14:14.930954 2578 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-130.ec2.internal" event="NodeHasNoDiskPressure" Jul 16 17:14:15.249245 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:14:14.930963 2578 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-130.ec2.internal" event="NodeHasSufficientPID" Jul 16 17:14:15.249245 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:14:14.930987 2578 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-132-130.ec2.internal" Jul 16 17:14:15.249245 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:14:14.942615 2578 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-132-130.ec2.internal" Jul 16 17:14:15.249245 ip-10-0-132-130 kubenswrapper[2578]: E0716 17:14:14.942633 2578 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-132-130.ec2.internal\": node \"ip-10-0-132-130.ec2.internal\" not found" Jul 16 17:14:15.249245 ip-10-0-132-130 kubenswrapper[2578]: E0716 17:14:14.969492 2578 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-132-130.ec2.internal\" not found" Jul 16 17:14:15.249245 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:14:15.057097 2578 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4" Jul 16 17:14:15.250490 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:14:15.058233 2578 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6" Jul 16 17:14:15.250490 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:14:15.058267 2578 status_manager.go:244] "Starting to sync pod status with apiserver" Jul 16 17:14:15.250490 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:14:15.058291 2578 kubelet.go:2448] "Starting kubelet main sync loop" Jul 16 17:14:15.250490 ip-10-0-132-130 kubenswrapper[2578]: E0716 17:14:15.058327 2578 kubelet.go:2472] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful" Jul 16 17:14:15.250490 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:14:15.060753 2578 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Jul 16 17:14:15.250490 ip-10-0-132-130 kubenswrapper[2578]: E0716 17:14:15.069766 2578 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-132-130.ec2.internal\" not found" Jul 16 17:14:15.250490 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:14:15.159247 2578 kubelet.go:2535] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-130.ec2.internal","kube-system/kube-apiserver-proxy-ip-10-0-132-130.ec2.internal"] Jul 16 17:14:15.250490 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:14:15.159318 2578 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 16 17:14:15.250490 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:14:15.160348 2578 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-130.ec2.internal" event="NodeHasSufficientMemory" Jul 16 17:14:15.250490 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:14:15.160373 2578 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-130.ec2.internal" event="NodeHasNoDiskPressure" Jul 16 17:14:15.250490 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:14:15.160385 2578 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-130.ec2.internal" event="NodeHasSufficientPID" Jul 16 17:14:15.250490 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:14:15.162827 2578 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 16 17:14:15.250490 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:14:15.162924 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-130.ec2.internal" Jul 16 17:14:15.250490 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:14:15.162956 2578 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 16 17:14:15.250490 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:14:15.163491 2578 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-130.ec2.internal" event="NodeHasSufficientMemory" Jul 16 17:14:15.250490 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:14:15.163509 2578 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-130.ec2.internal" event="NodeHasNoDiskPressure" Jul 16 17:14:15.250490 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:14:15.163518 2578 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-130.ec2.internal" event="NodeHasSufficientPID" Jul 16 17:14:15.250490 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:14:15.164016 2578 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-130.ec2.internal" event="NodeHasSufficientMemory" Jul 16 17:14:15.250490 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:14:15.164034 2578 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-130.ec2.internal" event="NodeHasNoDiskPressure" Jul 16 17:14:15.250490 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:14:15.164048 2578 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-130.ec2.internal" event="NodeHasSufficientPID" Jul 16 17:14:15.250490 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:14:15.166267 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-132-130.ec2.internal" Jul 16 17:14:15.250490 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:14:15.166288 2578 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 16 17:14:15.250490 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:14:15.168664 2578 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-130.ec2.internal" event="NodeHasSufficientMemory" Jul 16 17:14:15.250490 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:14:15.168692 2578 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-130.ec2.internal" event="NodeHasNoDiskPressure" Jul 16 17:14:15.250490 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:14:15.168706 2578 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-130.ec2.internal" event="NodeHasSufficientPID" Jul 16 17:14:15.250490 ip-10-0-132-130 kubenswrapper[2578]: E0716 17:14:15.170485 2578 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-132-130.ec2.internal\" not found" Jul 16 17:14:15.250490 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:14:15.178852 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/f724a0822cdfaacbec5071cc8e93c31c-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-132-130.ec2.internal\" (UID: \"f724a0822cdfaacbec5071cc8e93c31c\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-130.ec2.internal" Jul 16 17:14:15.250490 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:14:15.178870 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/f724a0822cdfaacbec5071cc8e93c31c-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-132-130.ec2.internal\" (UID: \"f724a0822cdfaacbec5071cc8e93c31c\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-130.ec2.internal" Jul 16 17:14:15.250490 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:14:15.178887 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/7876ab904043bc7f87e7c28015b01ebc-config\") pod \"kube-apiserver-proxy-ip-10-0-132-130.ec2.internal\" (UID: \"7876ab904043bc7f87e7c28015b01ebc\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-132-130.ec2.internal" Jul 16 17:14:15.250490 ip-10-0-132-130 kubenswrapper[2578]: E0716 17:14:15.202760 2578 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-132-130.ec2.internal\" not found" node="ip-10-0-132-130.ec2.internal" Jul 16 17:14:15.250490 ip-10-0-132-130 kubenswrapper[2578]: E0716 17:14:15.208033 2578 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-132-130.ec2.internal\" not found" node="ip-10-0-132-130.ec2.internal" Jul 16 17:14:15.270641 ip-10-0-132-130 kubenswrapper[2578]: E0716 17:14:15.270618 2578 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-132-130.ec2.internal\" not found" Jul 16 17:14:15.279955 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:14:15.279938 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/f724a0822cdfaacbec5071cc8e93c31c-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-132-130.ec2.internal\" (UID: \"f724a0822cdfaacbec5071cc8e93c31c\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-130.ec2.internal" Jul 16 17:14:15.280027 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:14:15.279969 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/f724a0822cdfaacbec5071cc8e93c31c-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-132-130.ec2.internal\" (UID: \"f724a0822cdfaacbec5071cc8e93c31c\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-130.ec2.internal" Jul 16 17:14:15.280027 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:14:15.279986 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/7876ab904043bc7f87e7c28015b01ebc-config\") pod \"kube-apiserver-proxy-ip-10-0-132-130.ec2.internal\" (UID: \"7876ab904043bc7f87e7c28015b01ebc\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-132-130.ec2.internal" Jul 16 17:14:15.280027 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:14:15.280011 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/7876ab904043bc7f87e7c28015b01ebc-config\") pod \"kube-apiserver-proxy-ip-10-0-132-130.ec2.internal\" (UID: \"7876ab904043bc7f87e7c28015b01ebc\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-132-130.ec2.internal" Jul 16 17:14:15.280134 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:14:15.280025 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/f724a0822cdfaacbec5071cc8e93c31c-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-132-130.ec2.internal\" (UID: \"f724a0822cdfaacbec5071cc8e93c31c\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-130.ec2.internal" Jul 16 17:14:15.280134 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:14:15.280050 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/f724a0822cdfaacbec5071cc8e93c31c-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-132-130.ec2.internal\" (UID: \"f724a0822cdfaacbec5071cc8e93c31c\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-130.ec2.internal" Jul 16 17:14:16.522029 ip-10-0-132-130 kubenswrapper[2578]: E0716 17:14:16.522002 2578 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-132-130.ec2.internal\" not found" Jul 16 17:14:16.622574 ip-10-0-132-130 kubenswrapper[2578]: E0716 17:14:16.622541 2578 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-132-130.ec2.internal\" not found" Jul 16 17:14:16.655034 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:14:16.654979 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-130.ec2.internal" Jul 16 17:14:16.660943 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:14:16.660884 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-132-130.ec2.internal" Jul 16 17:14:16.723303 ip-10-0-132-130 kubenswrapper[2578]: E0716 17:14:16.723270 2578 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-132-130.ec2.internal\" not found" Jul 16 17:14:16.823827 ip-10-0-132-130 kubenswrapper[2578]: E0716 17:14:16.823794 2578 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-132-130.ec2.internal\" not found" Jul 16 17:14:16.854252 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:14:16.854231 2578 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials" Jul 16 17:14:16.854726 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:14:16.854374 2578 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received" Jul 16 17:14:16.924088 ip-10-0-132-130 kubenswrapper[2578]: E0716 17:14:16.924029 2578 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-132-130.ec2.internal\" not found" Jul 16 17:14:16.954819 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:14:16.954785 2578 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-07-15 17:09:15 +0000 UTC" deadline="2028-03-21 07:58:30.832778969 +0000 UTC" Jul 16 17:14:16.954819 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:14:16.954814 2578 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="14726h44m13.877969106s" Jul 16 17:14:17.024432 ip-10-0-132-130 kubenswrapper[2578]: E0716 17:14:17.024407 2578 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-132-130.ec2.internal\" not found" Jul 16 17:14:17.121521 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:14:17.121498 2578 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160" Jul 16 17:14:17.126216 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:14:17.126201 2578 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-130.ec2.internal" Jul 16 17:14:17.150392 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:14:17.150372 2578 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Jul 16 17:14:17.151843 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:14:17.151830 2578 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-132-130.ec2.internal" Jul 16 17:14:17.176003 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:14:17.175939 2578 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Jul 16 17:14:17.209071 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:14:17.209045 2578 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160" Jul 16 17:14:17.269012 ip-10-0-132-130 kubenswrapper[2578]: W0716 17:14:17.268967 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7876ab904043bc7f87e7c28015b01ebc.slice/crio-cf0870221f7437ddbfd6db32e5917b8206a0158116c286b3af334bab9b587fef WatchSource:0}: Error finding container cf0870221f7437ddbfd6db32e5917b8206a0158116c286b3af334bab9b587fef: Status 404 returned error can't find the container with id cf0870221f7437ddbfd6db32e5917b8206a0158116c286b3af334bab9b587fef Jul 16 17:14:17.272760 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:14:17.272745 2578 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 16 17:14:17.316297 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:14:17.316268 2578 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160" Jul 16 17:14:17.902757 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:14:17.902732 2578 apiserver.go:52] "Watching apiserver" Jul 16 17:14:17.909495 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:14:17.909475 2578 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66" Jul 16 17:14:17.911673 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:14:17.911651 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-network-operator/iptables-alerter-9mk4h","openshift-ovn-kubernetes/ovnkube-node-h7t4d","kube-system/global-pull-secret-syncer-rg5vf","kube-system/konnectivity-agent-kq8j9","openshift-cluster-node-tuning-operator/tuned-78ctd","openshift-image-registry/node-ca-lfsvs","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-130.ec2.internal","openshift-multus/multus-additional-cni-plugins-6gzcp","openshift-multus/multus-qj6b6","openshift-multus/network-metrics-daemon-kvr2r","kube-system/kube-apiserver-proxy-ip-10-0-132-130.ec2.internal","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-cn7hz","openshift-dns/node-resolver-zw2xf","openshift-network-diagnostics/network-check-target-g4r9z"] Jul 16 17:14:17.915196 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:14:17.915180 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-kvr2r" Jul 16 17:14:17.915282 ip-10-0-132-130 kubenswrapper[2578]: E0716 17:14:17.915258 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-kvr2r" podUID="f90822ad-ab14-4108-8294-2e76c35a8aa5" Jul 16 17:14:17.917238 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:14:17.917222 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-zw2xf" Jul 16 17:14:17.919244 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:14:17.919209 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-rg5vf" Jul 16 17:14:17.919337 ip-10-0-132-130 kubenswrapper[2578]: E0716 17:14:17.919317 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-rg5vf" podUID="4e00566f-55f0-43ec-9d1a-a883e839823e" Jul 16 17:14:17.919644 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:14:17.919599 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\"" Jul 16 17:14:17.919796 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:14:17.919775 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\"" Jul 16 17:14:17.919796 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:14:17.919793 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-gxcn8\"" Jul 16 17:14:17.921261 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:14:17.921246 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-kq8j9" Jul 16 17:14:17.923269 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:14:17.923253 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\"" Jul 16 17:14:17.923371 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:14:17.923350 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-78ctd" Jul 16 17:14:17.923576 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:14:17.923552 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\"" Jul 16 17:14:17.923665 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:14:17.923587 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-7gv4n\"" Jul 16 17:14:17.925546 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:14:17.925527 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-lfsvs" Jul 16 17:14:17.926197 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:14:17.926181 2578 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving" Jul 16 17:14:17.927764 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:14:17.927749 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-qj6b6" Jul 16 17:14:17.928199 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:14:17.928185 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\"" Jul 16 17:14:17.928558 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:14:17.928544 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\"" Jul 16 17:14:17.928629 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:14:17.928570 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-dqx82\"" Jul 16 17:14:17.928714 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:14:17.928628 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\"" Jul 16 17:14:17.928714 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:14:17.928640 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\"" Jul 16 17:14:17.928714 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:14:17.928667 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-qzcvc\"" Jul 16 17:14:17.928917 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:14:17.928765 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\"" Jul 16 17:14:17.929975 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:14:17.929959 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-g4r9z" Jul 16 17:14:17.930055 ip-10-0-132-130 kubenswrapper[2578]: E0716 17:14:17.930018 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-g4r9z" podUID="8232d058-70ea-4e6d-ae51-e5ba4303dd93" Jul 16 17:14:17.930503 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:14:17.930484 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\"" Jul 16 17:14:17.930561 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:14:17.930494 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-pb258\"" Jul 16 17:14:17.930804 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:14:17.930790 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\"" Jul 16 17:14:17.930882 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:14:17.930803 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\"" Jul 16 17:14:17.932253 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:14:17.932237 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-h7t4d" Jul 16 17:14:17.933352 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:14:17.933338 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\"" Jul 16 17:14:17.934471 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:14:17.934455 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-9mk4h" Jul 16 17:14:17.934738 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:14:17.934635 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\"" Jul 16 17:14:17.935021 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:14:17.934917 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\"" Jul 16 17:14:17.935021 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:14:17.934950 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\"" Jul 16 17:14:17.936503 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:14:17.936483 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\"" Jul 16 17:14:17.936798 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:14:17.936777 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\"" Jul 16 17:14:17.936961 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:14:17.936946 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-6gzcp" Jul 16 17:14:17.937572 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:14:17.937553 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f90822ad-ab14-4108-8294-2e76c35a8aa5-metrics-certs\") pod \"network-metrics-daemon-kvr2r\" (UID: \"f90822ad-ab14-4108-8294-2e76c35a8aa5\") " pod="openshift-multus/network-metrics-daemon-kvr2r" Jul 16 17:14:17.937675 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:14:17.937579 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cglr8\" (UniqueName: \"kubernetes.io/projected/dfca47c1-7bfb-4111-a283-7b86affa45a7-kube-api-access-cglr8\") pod \"node-ca-lfsvs\" (UID: \"dfca47c1-7bfb-4111-a283-7b86affa45a7\") " pod="openshift-image-registry/node-ca-lfsvs" Jul 16 17:14:17.937675 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:14:17.937599 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/31abc787-9b8d-4943-a302-b6db28480e19-system-cni-dir\") pod \"multus-qj6b6\" (UID: \"31abc787-9b8d-4943-a302-b6db28480e19\") " pod="openshift-multus/multus-qj6b6" Jul 16 17:14:17.937675 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:14:17.937620 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/31abc787-9b8d-4943-a302-b6db28480e19-cni-binary-copy\") pod \"multus-qj6b6\" (UID: \"31abc787-9b8d-4943-a302-b6db28480e19\") " pod="openshift-multus/multus-qj6b6" Jul 16 17:14:17.937828 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:14:17.937747 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/31abc787-9b8d-4943-a302-b6db28480e19-host-run-k8s-cni-cncf-io\") pod \"multus-qj6b6\" (UID: \"31abc787-9b8d-4943-a302-b6db28480e19\") " pod="openshift-multus/multus-qj6b6" Jul 16 17:14:17.937828 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:14:17.937776 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/31abc787-9b8d-4943-a302-b6db28480e19-host-run-netns\") pod \"multus-qj6b6\" (UID: \"31abc787-9b8d-4943-a302-b6db28480e19\") " pod="openshift-multus/multus-qj6b6" Jul 16 17:14:17.937828 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:14:17.937800 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/4e00566f-55f0-43ec-9d1a-a883e839823e-original-pull-secret\") pod \"global-pull-secret-syncer-rg5vf\" (UID: \"4e00566f-55f0-43ec-9d1a-a883e839823e\") " pod="kube-system/global-pull-secret-syncer-rg5vf" Jul 16 17:14:17.937828 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:14:17.937823 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/31abc787-9b8d-4943-a302-b6db28480e19-multus-socket-dir-parent\") pod \"multus-qj6b6\" (UID: \"31abc787-9b8d-4943-a302-b6db28480e19\") " pod="openshift-multus/multus-qj6b6" Jul 16 17:14:17.937982 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:14:17.937849 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/31abc787-9b8d-4943-a302-b6db28480e19-host-var-lib-cni-bin\") pod \"multus-qj6b6\" (UID: \"31abc787-9b8d-4943-a302-b6db28480e19\") " pod="openshift-multus/multus-qj6b6" Jul 16 17:14:17.937982 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:14:17.937870 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/31abc787-9b8d-4943-a302-b6db28480e19-multus-conf-dir\") pod \"multus-qj6b6\" (UID: \"31abc787-9b8d-4943-a302-b6db28480e19\") " pod="openshift-multus/multus-qj6b6" Jul 16 17:14:17.937982 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:14:17.937951 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/31abc787-9b8d-4943-a302-b6db28480e19-multus-daemon-config\") pod \"multus-qj6b6\" (UID: \"31abc787-9b8d-4943-a302-b6db28480e19\") " pod="openshift-multus/multus-qj6b6" Jul 16 17:14:17.937982 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:14:17.937975 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/a8425d74-5613-43e9-b6ff-3884b419dbf6-konnectivity-ca\") pod \"konnectivity-agent-kq8j9\" (UID: \"a8425d74-5613-43e9-b6ff-3884b419dbf6\") " pod="kube-system/konnectivity-agent-kq8j9" Jul 16 17:14:17.938166 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:14:17.937992 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/7db99718-7ae4-48bd-9c33-900ea9a925ad-host\") pod \"tuned-78ctd\" (UID: \"7db99718-7ae4-48bd-9c33-900ea9a925ad\") " pod="openshift-cluster-node-tuning-operator/tuned-78ctd" Jul 16 17:14:17.938166 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:14:17.938007 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/31abc787-9b8d-4943-a302-b6db28480e19-host-var-lib-kubelet\") pod \"multus-qj6b6\" (UID: \"31abc787-9b8d-4943-a302-b6db28480e19\") " pod="openshift-multus/multus-qj6b6" Jul 16 17:14:17.938166 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:14:17.938042 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/31abc787-9b8d-4943-a302-b6db28480e19-host-run-multus-certs\") pod \"multus-qj6b6\" (UID: \"31abc787-9b8d-4943-a302-b6db28480e19\") " pod="openshift-multus/multus-qj6b6" Jul 16 17:14:17.938166 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:14:17.938085 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/31abc787-9b8d-4943-a302-b6db28480e19-etc-kubernetes\") pod \"multus-qj6b6\" (UID: \"31abc787-9b8d-4943-a302-b6db28480e19\") " pod="openshift-multus/multus-qj6b6" Jul 16 17:14:17.938166 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:14:17.938092 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\"" Jul 16 17:14:17.938166 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:14:17.938113 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-b9pkd\" (UniqueName: \"kubernetes.io/projected/31abc787-9b8d-4943-a302-b6db28480e19-kube-api-access-b9pkd\") pod \"multus-qj6b6\" (UID: \"31abc787-9b8d-4943-a302-b6db28480e19\") " pod="openshift-multus/multus-qj6b6" Jul 16 17:14:17.938166 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:14:17.938133 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-r9ts7\"" Jul 16 17:14:17.938166 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:14:17.938136 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ff79255c-4f85-4600-a7d2-54534f299e29-tmp-dir\") pod \"node-resolver-zw2xf\" (UID: \"ff79255c-4f85-4600-a7d2-54534f299e29\") " pod="openshift-dns/node-resolver-zw2xf" Jul 16 17:14:17.938166 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:14:17.938117 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\"" Jul 16 17:14:17.938166 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:14:17.938169 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\"" Jul 16 17:14:17.938622 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:14:17.938205 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rx9sg\" (UniqueName: \"kubernetes.io/projected/7db99718-7ae4-48bd-9c33-900ea9a925ad-kube-api-access-rx9sg\") pod \"tuned-78ctd\" (UID: \"7db99718-7ae4-48bd-9c33-900ea9a925ad\") " pod="openshift-cluster-node-tuning-operator/tuned-78ctd" Jul 16 17:14:17.938622 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:14:17.938168 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-lfsv5\"" Jul 16 17:14:17.938622 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:14:17.938236 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/dfca47c1-7bfb-4111-a283-7b86affa45a7-serviceca\") pod \"node-ca-lfsvs\" (UID: \"dfca47c1-7bfb-4111-a283-7b86affa45a7\") " pod="openshift-image-registry/node-ca-lfsvs" Jul 16 17:14:17.938622 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:14:17.938261 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\"" Jul 16 17:14:17.938622 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:14:17.938259 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/31abc787-9b8d-4943-a302-b6db28480e19-hostroot\") pod \"multus-qj6b6\" (UID: \"31abc787-9b8d-4943-a302-b6db28480e19\") " pod="openshift-multus/multus-qj6b6" Jul 16 17:14:17.938622 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:14:17.938320 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vh2mm\" (UniqueName: \"kubernetes.io/projected/f90822ad-ab14-4108-8294-2e76c35a8aa5-kube-api-access-vh2mm\") pod \"network-metrics-daemon-kvr2r\" (UID: \"f90822ad-ab14-4108-8294-2e76c35a8aa5\") " pod="openshift-multus/network-metrics-daemon-kvr2r" Jul 16 17:14:17.938622 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:14:17.938343 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/dfca47c1-7bfb-4111-a283-7b86affa45a7-host\") pod \"node-ca-lfsvs\" (UID: \"dfca47c1-7bfb-4111-a283-7b86affa45a7\") " pod="openshift-image-registry/node-ca-lfsvs" Jul 16 17:14:17.938622 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:14:17.938375 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/31abc787-9b8d-4943-a302-b6db28480e19-host-var-lib-cni-multus\") pod \"multus-qj6b6\" (UID: \"31abc787-9b8d-4943-a302-b6db28480e19\") " pod="openshift-multus/multus-qj6b6" Jul 16 17:14:17.938622 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:14:17.938418 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/7db99718-7ae4-48bd-9c33-900ea9a925ad-etc-systemd\") pod \"tuned-78ctd\" (UID: \"7db99718-7ae4-48bd-9c33-900ea9a925ad\") " pod="openshift-cluster-node-tuning-operator/tuned-78ctd" Jul 16 17:14:17.938622 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:14:17.938442 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/7db99718-7ae4-48bd-9c33-900ea9a925ad-tmp\") pod \"tuned-78ctd\" (UID: \"7db99718-7ae4-48bd-9c33-900ea9a925ad\") " pod="openshift-cluster-node-tuning-operator/tuned-78ctd" Jul 16 17:14:17.938622 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:14:17.938467 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/31abc787-9b8d-4943-a302-b6db28480e19-cnibin\") pod \"multus-qj6b6\" (UID: \"31abc787-9b8d-4943-a302-b6db28480e19\") " pod="openshift-multus/multus-qj6b6" Jul 16 17:14:17.938622 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:14:17.938492 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/31abc787-9b8d-4943-a302-b6db28480e19-os-release\") pod \"multus-qj6b6\" (UID: \"31abc787-9b8d-4943-a302-b6db28480e19\") " pod="openshift-multus/multus-qj6b6" Jul 16 17:14:17.938622 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:14:17.938530 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6dhhw\" (UniqueName: \"kubernetes.io/projected/ff79255c-4f85-4600-a7d2-54534f299e29-kube-api-access-6dhhw\") pod \"node-resolver-zw2xf\" (UID: \"ff79255c-4f85-4600-a7d2-54534f299e29\") " pod="openshift-dns/node-resolver-zw2xf" Jul 16 17:14:17.938622 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:14:17.938569 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/a8425d74-5613-43e9-b6ff-3884b419dbf6-agent-certs\") pod \"konnectivity-agent-kq8j9\" (UID: \"a8425d74-5613-43e9-b6ff-3884b419dbf6\") " pod="kube-system/konnectivity-agent-kq8j9" Jul 16 17:14:17.939252 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:14:17.938637 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/4e00566f-55f0-43ec-9d1a-a883e839823e-kubelet-config\") pod \"global-pull-secret-syncer-rg5vf\" (UID: \"4e00566f-55f0-43ec-9d1a-a883e839823e\") " pod="kube-system/global-pull-secret-syncer-rg5vf" Jul 16 17:14:17.939252 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:14:17.938664 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/7db99718-7ae4-48bd-9c33-900ea9a925ad-etc-modprobe-d\") pod \"tuned-78ctd\" (UID: \"7db99718-7ae4-48bd-9c33-900ea9a925ad\") " pod="openshift-cluster-node-tuning-operator/tuned-78ctd" Jul 16 17:14:17.939252 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:14:17.938686 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/7db99718-7ae4-48bd-9c33-900ea9a925ad-etc-sysconfig\") pod \"tuned-78ctd\" (UID: \"7db99718-7ae4-48bd-9c33-900ea9a925ad\") " pod="openshift-cluster-node-tuning-operator/tuned-78ctd" Jul 16 17:14:17.939252 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:14:17.938794 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/7db99718-7ae4-48bd-9c33-900ea9a925ad-etc-sysctl-conf\") pod \"tuned-78ctd\" (UID: \"7db99718-7ae4-48bd-9c33-900ea9a925ad\") " pod="openshift-cluster-node-tuning-operator/tuned-78ctd" Jul 16 17:14:17.939252 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:14:17.938814 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/7db99718-7ae4-48bd-9c33-900ea9a925ad-sys\") pod \"tuned-78ctd\" (UID: \"7db99718-7ae4-48bd-9c33-900ea9a925ad\") " pod="openshift-cluster-node-tuning-operator/tuned-78ctd" Jul 16 17:14:17.939252 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:14:17.938837 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/7db99718-7ae4-48bd-9c33-900ea9a925ad-lib-modules\") pod \"tuned-78ctd\" (UID: \"7db99718-7ae4-48bd-9c33-900ea9a925ad\") " pod="openshift-cluster-node-tuning-operator/tuned-78ctd" Jul 16 17:14:17.939252 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:14:17.938859 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/4e00566f-55f0-43ec-9d1a-a883e839823e-dbus\") pod \"global-pull-secret-syncer-rg5vf\" (UID: \"4e00566f-55f0-43ec-9d1a-a883e839823e\") " pod="kube-system/global-pull-secret-syncer-rg5vf" Jul 16 17:14:17.939252 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:14:17.938880 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/7db99718-7ae4-48bd-9c33-900ea9a925ad-etc-kubernetes\") pod \"tuned-78ctd\" (UID: \"7db99718-7ae4-48bd-9c33-900ea9a925ad\") " pod="openshift-cluster-node-tuning-operator/tuned-78ctd" Jul 16 17:14:17.939252 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:14:17.938904 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/7db99718-7ae4-48bd-9c33-900ea9a925ad-etc-sysctl-d\") pod \"tuned-78ctd\" (UID: \"7db99718-7ae4-48bd-9c33-900ea9a925ad\") " pod="openshift-cluster-node-tuning-operator/tuned-78ctd" Jul 16 17:14:17.939252 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:14:17.938928 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/7db99718-7ae4-48bd-9c33-900ea9a925ad-var-lib-kubelet\") pod \"tuned-78ctd\" (UID: \"7db99718-7ae4-48bd-9c33-900ea9a925ad\") " pod="openshift-cluster-node-tuning-operator/tuned-78ctd" Jul 16 17:14:17.939252 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:14:17.938960 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/ff79255c-4f85-4600-a7d2-54534f299e29-hosts-file\") pod \"node-resolver-zw2xf\" (UID: \"ff79255c-4f85-4600-a7d2-54534f299e29\") " pod="openshift-dns/node-resolver-zw2xf" Jul 16 17:14:17.939252 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:14:17.938996 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/31abc787-9b8d-4943-a302-b6db28480e19-multus-cni-dir\") pod \"multus-qj6b6\" (UID: \"31abc787-9b8d-4943-a302-b6db28480e19\") " pod="openshift-multus/multus-qj6b6" Jul 16 17:14:17.939856 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:14:17.939348 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\"" Jul 16 17:14:17.939856 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:14:17.939021 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/7db99718-7ae4-48bd-9c33-900ea9a925ad-run\") pod \"tuned-78ctd\" (UID: \"7db99718-7ae4-48bd-9c33-900ea9a925ad\") " pod="openshift-cluster-node-tuning-operator/tuned-78ctd" Jul 16 17:14:17.939856 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:14:17.939474 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/7db99718-7ae4-48bd-9c33-900ea9a925ad-etc-tuned\") pod \"tuned-78ctd\" (UID: \"7db99718-7ae4-48bd-9c33-900ea9a925ad\") " pod="openshift-cluster-node-tuning-operator/tuned-78ctd" Jul 16 17:14:17.939856 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:14:17.939596 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-cn7hz" Jul 16 17:14:17.940045 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:14:17.940010 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\"" Jul 16 17:14:17.940764 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:14:17.940706 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-hmzd2\"" Jul 16 17:14:17.943470 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:14:17.943448 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-bg6pc\"" Jul 16 17:14:17.943559 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:14:17.943512 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\"" Jul 16 17:14:17.943621 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:14:17.943447 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\"" Jul 16 17:14:17.943709 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:14:17.943673 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\"" Jul 16 17:14:17.947330 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:14:17.947307 2578 reflector.go:439] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Jul 16 17:14:17.966993 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:14:17.966977 2578 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-48rc5" Jul 16 17:14:17.977259 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:14:17.977247 2578 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-48rc5" Jul 16 17:14:18.027404 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:14:18.027380 2578 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world" Jul 16 17:14:18.039937 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:14:18.039917 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/9a29f379-46c1-4211-81b2-7f3d82d82d01-host-run-ovn-kubernetes\") pod \"ovnkube-node-h7t4d\" (UID: \"9a29f379-46c1-4211-81b2-7f3d82d82d01\") " pod="openshift-ovn-kubernetes/ovnkube-node-h7t4d" Jul 16 17:14:18.040076 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:14:18.039938 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/9a29f379-46c1-4211-81b2-7f3d82d82d01-host-cni-bin\") pod \"ovnkube-node-h7t4d\" (UID: \"9a29f379-46c1-4211-81b2-7f3d82d82d01\") " pod="openshift-ovn-kubernetes/ovnkube-node-h7t4d" Jul 16 17:14:18.040076 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:14:18.039970 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qtzjs\" (UniqueName: \"kubernetes.io/projected/fa2959ee-fcc6-49eb-9205-65cf1a08779f-kube-api-access-qtzjs\") pod \"multus-additional-cni-plugins-6gzcp\" (UID: \"fa2959ee-fcc6-49eb-9205-65cf1a08779f\") " pod="openshift-multus/multus-additional-cni-plugins-6gzcp" Jul 16 17:14:18.040076 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:14:18.039989 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/7db99718-7ae4-48bd-9c33-900ea9a925ad-tmp\") pod \"tuned-78ctd\" (UID: \"7db99718-7ae4-48bd-9c33-900ea9a925ad\") " pod="openshift-cluster-node-tuning-operator/tuned-78ctd" Jul 16 17:14:18.040076 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:14:18.040003 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/31abc787-9b8d-4943-a302-b6db28480e19-cnibin\") pod \"multus-qj6b6\" (UID: \"31abc787-9b8d-4943-a302-b6db28480e19\") " pod="openshift-multus/multus-qj6b6" Jul 16 17:14:18.040076 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:14:18.040037 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jr76l\" (UniqueName: \"kubernetes.io/projected/6659c756-0d21-42ec-aa63-157ef0294e02-kube-api-access-jr76l\") pod \"iptables-alerter-9mk4h\" (UID: \"6659c756-0d21-42ec-aa63-157ef0294e02\") " pod="openshift-network-operator/iptables-alerter-9mk4h" Jul 16 17:14:18.040076 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:14:18.040053 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/31abc787-9b8d-4943-a302-b6db28480e19-cnibin\") pod \"multus-qj6b6\" (UID: \"31abc787-9b8d-4943-a302-b6db28480e19\") " pod="openshift-multus/multus-qj6b6" Jul 16 17:14:18.040076 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:14:18.040065 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/fa2959ee-fcc6-49eb-9205-65cf1a08779f-tuning-conf-dir\") pod \"multus-additional-cni-plugins-6gzcp\" (UID: \"fa2959ee-fcc6-49eb-9205-65cf1a08779f\") " pod="openshift-multus/multus-additional-cni-plugins-6gzcp" Jul 16 17:14:18.040076 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:14:18.040080 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/01e55058-0022-4c74-966c-50605ba63d4e-kubelet-dir\") pod \"aws-ebs-csi-driver-node-cn7hz\" (UID: \"01e55058-0022-4c74-966c-50605ba63d4e\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-cn7hz" Jul 16 17:14:18.040319 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:14:18.040104 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/7db99718-7ae4-48bd-9c33-900ea9a925ad-etc-sysctl-conf\") pod \"tuned-78ctd\" (UID: \"7db99718-7ae4-48bd-9c33-900ea9a925ad\") " pod="openshift-cluster-node-tuning-operator/tuned-78ctd" Jul 16 17:14:18.040319 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:14:18.040124 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/7db99718-7ae4-48bd-9c33-900ea9a925ad-lib-modules\") pod \"tuned-78ctd\" (UID: \"7db99718-7ae4-48bd-9c33-900ea9a925ad\") " pod="openshift-cluster-node-tuning-operator/tuned-78ctd" Jul 16 17:14:18.040319 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:14:18.040197 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/9a29f379-46c1-4211-81b2-7f3d82d82d01-host-cni-netd\") pod \"ovnkube-node-h7t4d\" (UID: \"9a29f379-46c1-4211-81b2-7f3d82d82d01\") " pod="openshift-ovn-kubernetes/ovnkube-node-h7t4d" Jul 16 17:14:18.040319 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:14:18.040205 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/7db99718-7ae4-48bd-9c33-900ea9a925ad-etc-sysctl-conf\") pod \"tuned-78ctd\" (UID: \"7db99718-7ae4-48bd-9c33-900ea9a925ad\") " pod="openshift-cluster-node-tuning-operator/tuned-78ctd" Jul 16 17:14:18.040319 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:14:18.040223 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/9a29f379-46c1-4211-81b2-7f3d82d82d01-ovn-node-metrics-cert\") pod \"ovnkube-node-h7t4d\" (UID: \"9a29f379-46c1-4211-81b2-7f3d82d82d01\") " pod="openshift-ovn-kubernetes/ovnkube-node-h7t4d" Jul 16 17:14:18.040319 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:14:18.040226 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/7db99718-7ae4-48bd-9c33-900ea9a925ad-lib-modules\") pod \"tuned-78ctd\" (UID: \"7db99718-7ae4-48bd-9c33-900ea9a925ad\") " pod="openshift-cluster-node-tuning-operator/tuned-78ctd" Jul 16 17:14:18.040319 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:14:18.040252 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/01e55058-0022-4c74-966c-50605ba63d4e-registration-dir\") pod \"aws-ebs-csi-driver-node-cn7hz\" (UID: \"01e55058-0022-4c74-966c-50605ba63d4e\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-cn7hz" Jul 16 17:14:18.040319 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:14:18.040245 2578 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory" Jul 16 17:14:18.040319 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:14:18.040276 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/31abc787-9b8d-4943-a302-b6db28480e19-host-run-netns\") pod \"multus-qj6b6\" (UID: \"31abc787-9b8d-4943-a302-b6db28480e19\") " pod="openshift-multus/multus-qj6b6" Jul 16 17:14:18.040319 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:14:18.040292 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/9a29f379-46c1-4211-81b2-7f3d82d82d01-systemd-units\") pod \"ovnkube-node-h7t4d\" (UID: \"9a29f379-46c1-4211-81b2-7f3d82d82d01\") " pod="openshift-ovn-kubernetes/ovnkube-node-h7t4d" Jul 16 17:14:18.040319 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:14:18.040305 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/4e00566f-55f0-43ec-9d1a-a883e839823e-dbus\") pod \"global-pull-secret-syncer-rg5vf\" (UID: \"4e00566f-55f0-43ec-9d1a-a883e839823e\") " pod="kube-system/global-pull-secret-syncer-rg5vf" Jul 16 17:14:18.040818 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:14:18.040324 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/7db99718-7ae4-48bd-9c33-900ea9a925ad-etc-sysctl-d\") pod \"tuned-78ctd\" (UID: \"7db99718-7ae4-48bd-9c33-900ea9a925ad\") " pod="openshift-cluster-node-tuning-operator/tuned-78ctd" Jul 16 17:14:18.040818 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:14:18.040346 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/fa2959ee-fcc6-49eb-9205-65cf1a08779f-system-cni-dir\") pod \"multus-additional-cni-plugins-6gzcp\" (UID: \"fa2959ee-fcc6-49eb-9205-65cf1a08779f\") " pod="openshift-multus/multus-additional-cni-plugins-6gzcp" Jul 16 17:14:18.040818 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:14:18.040367 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/7db99718-7ae4-48bd-9c33-900ea9a925ad-etc-tuned\") pod \"tuned-78ctd\" (UID: \"7db99718-7ae4-48bd-9c33-900ea9a925ad\") " pod="openshift-cluster-node-tuning-operator/tuned-78ctd" Jul 16 17:14:18.040818 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:14:18.040387 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f90822ad-ab14-4108-8294-2e76c35a8aa5-metrics-certs\") pod \"network-metrics-daemon-kvr2r\" (UID: \"f90822ad-ab14-4108-8294-2e76c35a8aa5\") " pod="openshift-multus/network-metrics-daemon-kvr2r" Jul 16 17:14:18.040818 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:14:18.040345 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/31abc787-9b8d-4943-a302-b6db28480e19-host-run-netns\") pod \"multus-qj6b6\" (UID: \"31abc787-9b8d-4943-a302-b6db28480e19\") " pod="openshift-multus/multus-qj6b6" Jul 16 17:14:18.040818 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:14:18.040415 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cglr8\" (UniqueName: \"kubernetes.io/projected/dfca47c1-7bfb-4111-a283-7b86affa45a7-kube-api-access-cglr8\") pod \"node-ca-lfsvs\" (UID: \"dfca47c1-7bfb-4111-a283-7b86affa45a7\") " pod="openshift-image-registry/node-ca-lfsvs" Jul 16 17:14:18.040818 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:14:18.040420 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/7db99718-7ae4-48bd-9c33-900ea9a925ad-etc-sysctl-d\") pod \"tuned-78ctd\" (UID: \"7db99718-7ae4-48bd-9c33-900ea9a925ad\") " pod="openshift-cluster-node-tuning-operator/tuned-78ctd" Jul 16 17:14:18.040818 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:14:18.040437 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/31abc787-9b8d-4943-a302-b6db28480e19-system-cni-dir\") pod \"multus-qj6b6\" (UID: \"31abc787-9b8d-4943-a302-b6db28480e19\") " pod="openshift-multus/multus-qj6b6" Jul 16 17:14:18.040818 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:14:18.040459 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/4e00566f-55f0-43ec-9d1a-a883e839823e-dbus\") pod \"global-pull-secret-syncer-rg5vf\" (UID: \"4e00566f-55f0-43ec-9d1a-a883e839823e\") " pod="kube-system/global-pull-secret-syncer-rg5vf" Jul 16 17:14:18.040818 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:14:18.040483 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/31abc787-9b8d-4943-a302-b6db28480e19-host-run-k8s-cni-cncf-io\") pod \"multus-qj6b6\" (UID: \"31abc787-9b8d-4943-a302-b6db28480e19\") " pod="openshift-multus/multus-qj6b6" Jul 16 17:14:18.040818 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:14:18.040491 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/31abc787-9b8d-4943-a302-b6db28480e19-system-cni-dir\") pod \"multus-qj6b6\" (UID: \"31abc787-9b8d-4943-a302-b6db28480e19\") " pod="openshift-multus/multus-qj6b6" Jul 16 17:14:18.040818 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:14:18.040509 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/31abc787-9b8d-4943-a302-b6db28480e19-multus-conf-dir\") pod \"multus-qj6b6\" (UID: \"31abc787-9b8d-4943-a302-b6db28480e19\") " pod="openshift-multus/multus-qj6b6" Jul 16 17:14:18.040818 ip-10-0-132-130 kubenswrapper[2578]: E0716 17:14:18.040517 2578 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 16 17:14:18.040818 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:14:18.040534 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/9a29f379-46c1-4211-81b2-7f3d82d82d01-node-log\") pod \"ovnkube-node-h7t4d\" (UID: \"9a29f379-46c1-4211-81b2-7f3d82d82d01\") " pod="openshift-ovn-kubernetes/ovnkube-node-h7t4d" Jul 16 17:14:18.040818 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:14:18.040534 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/31abc787-9b8d-4943-a302-b6db28480e19-host-run-k8s-cni-cncf-io\") pod \"multus-qj6b6\" (UID: \"31abc787-9b8d-4943-a302-b6db28480e19\") " pod="openshift-multus/multus-qj6b6" Jul 16 17:14:18.040818 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:14:18.040557 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/9a29f379-46c1-4211-81b2-7f3d82d82d01-host-slash\") pod \"ovnkube-node-h7t4d\" (UID: \"9a29f379-46c1-4211-81b2-7f3d82d82d01\") " pod="openshift-ovn-kubernetes/ovnkube-node-h7t4d" Jul 16 17:14:18.040818 ip-10-0-132-130 kubenswrapper[2578]: E0716 17:14:18.040581 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f90822ad-ab14-4108-8294-2e76c35a8aa5-metrics-certs podName:f90822ad-ab14-4108-8294-2e76c35a8aa5 nodeName:}" failed. No retries permitted until 2026-07-16 17:14:18.540561985 +0000 UTC m=+3.026405854 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/f90822ad-ab14-4108-8294-2e76c35a8aa5-metrics-certs") pod "network-metrics-daemon-kvr2r" (UID: "f90822ad-ab14-4108-8294-2e76c35a8aa5") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 16 17:14:18.041415 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:14:18.040594 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/31abc787-9b8d-4943-a302-b6db28480e19-multus-conf-dir\") pod \"multus-qj6b6\" (UID: \"31abc787-9b8d-4943-a302-b6db28480e19\") " pod="openshift-multus/multus-qj6b6" Jul 16 17:14:18.041415 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:14:18.040616 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/9a29f379-46c1-4211-81b2-7f3d82d82d01-run-systemd\") pod \"ovnkube-node-h7t4d\" (UID: \"9a29f379-46c1-4211-81b2-7f3d82d82d01\") " pod="openshift-ovn-kubernetes/ovnkube-node-h7t4d" Jul 16 17:14:18.041415 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:14:18.040655 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/9a29f379-46c1-4211-81b2-7f3d82d82d01-etc-openvswitch\") pod \"ovnkube-node-h7t4d\" (UID: \"9a29f379-46c1-4211-81b2-7f3d82d82d01\") " pod="openshift-ovn-kubernetes/ovnkube-node-h7t4d" Jul 16 17:14:18.041415 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:14:18.040678 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/9a29f379-46c1-4211-81b2-7f3d82d82d01-run-openvswitch\") pod \"ovnkube-node-h7t4d\" (UID: \"9a29f379-46c1-4211-81b2-7f3d82d82d01\") " pod="openshift-ovn-kubernetes/ovnkube-node-h7t4d" Jul 16 17:14:18.041415 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:14:18.040714 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/9a29f379-46c1-4211-81b2-7f3d82d82d01-ovnkube-config\") pod \"ovnkube-node-h7t4d\" (UID: \"9a29f379-46c1-4211-81b2-7f3d82d82d01\") " pod="openshift-ovn-kubernetes/ovnkube-node-h7t4d" Jul 16 17:14:18.041415 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:14:18.040739 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/7db99718-7ae4-48bd-9c33-900ea9a925ad-host\") pod \"tuned-78ctd\" (UID: \"7db99718-7ae4-48bd-9c33-900ea9a925ad\") " pod="openshift-cluster-node-tuning-operator/tuned-78ctd" Jul 16 17:14:18.041415 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:14:18.040765 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/31abc787-9b8d-4943-a302-b6db28480e19-etc-kubernetes\") pod \"multus-qj6b6\" (UID: \"31abc787-9b8d-4943-a302-b6db28480e19\") " pod="openshift-multus/multus-qj6b6" Jul 16 17:14:18.041415 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:14:18.040792 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-b9pkd\" (UniqueName: \"kubernetes.io/projected/31abc787-9b8d-4943-a302-b6db28480e19-kube-api-access-b9pkd\") pod \"multus-qj6b6\" (UID: \"31abc787-9b8d-4943-a302-b6db28480e19\") " pod="openshift-multus/multus-qj6b6" Jul 16 17:14:18.041415 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:14:18.040814 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/9a29f379-46c1-4211-81b2-7f3d82d82d01-host-run-netns\") pod \"ovnkube-node-h7t4d\" (UID: \"9a29f379-46c1-4211-81b2-7f3d82d82d01\") " pod="openshift-ovn-kubernetes/ovnkube-node-h7t4d" Jul 16 17:14:18.041415 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:14:18.040824 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/7db99718-7ae4-48bd-9c33-900ea9a925ad-host\") pod \"tuned-78ctd\" (UID: \"7db99718-7ae4-48bd-9c33-900ea9a925ad\") " pod="openshift-cluster-node-tuning-operator/tuned-78ctd" Jul 16 17:14:18.041415 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:14:18.040858 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/31abc787-9b8d-4943-a302-b6db28480e19-etc-kubernetes\") pod \"multus-qj6b6\" (UID: \"31abc787-9b8d-4943-a302-b6db28480e19\") " pod="openshift-multus/multus-qj6b6" Jul 16 17:14:18.041415 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:14:18.040910 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/9a29f379-46c1-4211-81b2-7f3d82d82d01-var-lib-openvswitch\") pod \"ovnkube-node-h7t4d\" (UID: \"9a29f379-46c1-4211-81b2-7f3d82d82d01\") " pod="openshift-ovn-kubernetes/ovnkube-node-h7t4d" Jul 16 17:14:18.041415 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:14:18.040931 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/9a29f379-46c1-4211-81b2-7f3d82d82d01-run-ovn\") pod \"ovnkube-node-h7t4d\" (UID: \"9a29f379-46c1-4211-81b2-7f3d82d82d01\") " pod="openshift-ovn-kubernetes/ovnkube-node-h7t4d" Jul 16 17:14:18.041415 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:14:18.040954 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/01e55058-0022-4c74-966c-50605ba63d4e-etc-selinux\") pod \"aws-ebs-csi-driver-node-cn7hz\" (UID: \"01e55058-0022-4c74-966c-50605ba63d4e\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-cn7hz" Jul 16 17:14:18.041415 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:14:18.040973 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/dfca47c1-7bfb-4111-a283-7b86affa45a7-serviceca\") pod \"node-ca-lfsvs\" (UID: \"dfca47c1-7bfb-4111-a283-7b86affa45a7\") " pod="openshift-image-registry/node-ca-lfsvs" Jul 16 17:14:18.041415 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:14:18.040992 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/31abc787-9b8d-4943-a302-b6db28480e19-hostroot\") pod \"multus-qj6b6\" (UID: \"31abc787-9b8d-4943-a302-b6db28480e19\") " pod="openshift-multus/multus-qj6b6" Jul 16 17:14:18.041415 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:14:18.041009 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/9a29f379-46c1-4211-81b2-7f3d82d82d01-env-overrides\") pod \"ovnkube-node-h7t4d\" (UID: \"9a29f379-46c1-4211-81b2-7f3d82d82d01\") " pod="openshift-ovn-kubernetes/ovnkube-node-h7t4d" Jul 16 17:14:18.042203 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:14:18.041034 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/fa2959ee-fcc6-49eb-9205-65cf1a08779f-os-release\") pod \"multus-additional-cni-plugins-6gzcp\" (UID: \"fa2959ee-fcc6-49eb-9205-65cf1a08779f\") " pod="openshift-multus/multus-additional-cni-plugins-6gzcp" Jul 16 17:14:18.042203 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:14:18.041056 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vh2mm\" (UniqueName: \"kubernetes.io/projected/f90822ad-ab14-4108-8294-2e76c35a8aa5-kube-api-access-vh2mm\") pod \"network-metrics-daemon-kvr2r\" (UID: \"f90822ad-ab14-4108-8294-2e76c35a8aa5\") " pod="openshift-multus/network-metrics-daemon-kvr2r" Jul 16 17:14:18.042203 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:14:18.041067 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/31abc787-9b8d-4943-a302-b6db28480e19-hostroot\") pod \"multus-qj6b6\" (UID: \"31abc787-9b8d-4943-a302-b6db28480e19\") " pod="openshift-multus/multus-qj6b6" Jul 16 17:14:18.042203 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:14:18.041085 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/9a29f379-46c1-4211-81b2-7f3d82d82d01-ovnkube-script-lib\") pod \"ovnkube-node-h7t4d\" (UID: \"9a29f379-46c1-4211-81b2-7f3d82d82d01\") " pod="openshift-ovn-kubernetes/ovnkube-node-h7t4d" Jul 16 17:14:18.042203 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:14:18.041173 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/6659c756-0d21-42ec-aa63-157ef0294e02-iptables-alerter-script\") pod \"iptables-alerter-9mk4h\" (UID: \"6659c756-0d21-42ec-aa63-157ef0294e02\") " pod="openshift-network-operator/iptables-alerter-9mk4h" Jul 16 17:14:18.042203 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:14:18.041198 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/7db99718-7ae4-48bd-9c33-900ea9a925ad-etc-systemd\") pod \"tuned-78ctd\" (UID: \"7db99718-7ae4-48bd-9c33-900ea9a925ad\") " pod="openshift-cluster-node-tuning-operator/tuned-78ctd" Jul 16 17:14:18.042203 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:14:18.041218 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/31abc787-9b8d-4943-a302-b6db28480e19-os-release\") pod \"multus-qj6b6\" (UID: \"31abc787-9b8d-4943-a302-b6db28480e19\") " pod="openshift-multus/multus-qj6b6" Jul 16 17:14:18.042203 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:14:18.041239 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/9a29f379-46c1-4211-81b2-7f3d82d82d01-log-socket\") pod \"ovnkube-node-h7t4d\" (UID: \"9a29f379-46c1-4211-81b2-7f3d82d82d01\") " pod="openshift-ovn-kubernetes/ovnkube-node-h7t4d" Jul 16 17:14:18.042203 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:14:18.041261 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/31abc787-9b8d-4943-a302-b6db28480e19-multus-socket-dir-parent\") pod \"multus-qj6b6\" (UID: \"31abc787-9b8d-4943-a302-b6db28480e19\") " pod="openshift-multus/multus-qj6b6" Jul 16 17:14:18.042203 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:14:18.041272 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/7db99718-7ae4-48bd-9c33-900ea9a925ad-etc-systemd\") pod \"tuned-78ctd\" (UID: \"7db99718-7ae4-48bd-9c33-900ea9a925ad\") " pod="openshift-cluster-node-tuning-operator/tuned-78ctd" Jul 16 17:14:18.042203 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:14:18.041281 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/9a29f379-46c1-4211-81b2-7f3d82d82d01-host-kubelet\") pod \"ovnkube-node-h7t4d\" (UID: \"9a29f379-46c1-4211-81b2-7f3d82d82d01\") " pod="openshift-ovn-kubernetes/ovnkube-node-h7t4d" Jul 16 17:14:18.042203 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:14:18.041285 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/31abc787-9b8d-4943-a302-b6db28480e19-os-release\") pod \"multus-qj6b6\" (UID: \"31abc787-9b8d-4943-a302-b6db28480e19\") " pod="openshift-multus/multus-qj6b6" Jul 16 17:14:18.042203 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:14:18.041303 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6dhhw\" (UniqueName: \"kubernetes.io/projected/ff79255c-4f85-4600-a7d2-54534f299e29-kube-api-access-6dhhw\") pod \"node-resolver-zw2xf\" (UID: \"ff79255c-4f85-4600-a7d2-54534f299e29\") " pod="openshift-dns/node-resolver-zw2xf" Jul 16 17:14:18.042203 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:14:18.041325 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/a8425d74-5613-43e9-b6ff-3884b419dbf6-agent-certs\") pod \"konnectivity-agent-kq8j9\" (UID: \"a8425d74-5613-43e9-b6ff-3884b419dbf6\") " pod="kube-system/konnectivity-agent-kq8j9" Jul 16 17:14:18.042203 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:14:18.041343 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/31abc787-9b8d-4943-a302-b6db28480e19-multus-socket-dir-parent\") pod \"multus-qj6b6\" (UID: \"31abc787-9b8d-4943-a302-b6db28480e19\") " pod="openshift-multus/multus-qj6b6" Jul 16 17:14:18.042203 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:14:18.041349 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/4e00566f-55f0-43ec-9d1a-a883e839823e-kubelet-config\") pod \"global-pull-secret-syncer-rg5vf\" (UID: \"4e00566f-55f0-43ec-9d1a-a883e839823e\") " pod="kube-system/global-pull-secret-syncer-rg5vf" Jul 16 17:14:18.042203 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:14:18.041378 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/7db99718-7ae4-48bd-9c33-900ea9a925ad-etc-modprobe-d\") pod \"tuned-78ctd\" (UID: \"7db99718-7ae4-48bd-9c33-900ea9a925ad\") " pod="openshift-cluster-node-tuning-operator/tuned-78ctd" Jul 16 17:14:18.042746 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:14:18.041401 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/7db99718-7ae4-48bd-9c33-900ea9a925ad-etc-sysconfig\") pod \"tuned-78ctd\" (UID: \"7db99718-7ae4-48bd-9c33-900ea9a925ad\") " pod="openshift-cluster-node-tuning-operator/tuned-78ctd" Jul 16 17:14:18.042746 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:14:18.041403 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/dfca47c1-7bfb-4111-a283-7b86affa45a7-serviceca\") pod \"node-ca-lfsvs\" (UID: \"dfca47c1-7bfb-4111-a283-7b86affa45a7\") " pod="openshift-image-registry/node-ca-lfsvs" Jul 16 17:14:18.042746 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:14:18.041406 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/4e00566f-55f0-43ec-9d1a-a883e839823e-kubelet-config\") pod \"global-pull-secret-syncer-rg5vf\" (UID: \"4e00566f-55f0-43ec-9d1a-a883e839823e\") " pod="kube-system/global-pull-secret-syncer-rg5vf" Jul 16 17:14:18.042746 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:14:18.041423 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/31abc787-9b8d-4943-a302-b6db28480e19-host-run-multus-certs\") pod \"multus-qj6b6\" (UID: \"31abc787-9b8d-4943-a302-b6db28480e19\") " pod="openshift-multus/multus-qj6b6" Jul 16 17:14:18.042746 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:14:18.041445 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/7db99718-7ae4-48bd-9c33-900ea9a925ad-sys\") pod \"tuned-78ctd\" (UID: \"7db99718-7ae4-48bd-9c33-900ea9a925ad\") " pod="openshift-cluster-node-tuning-operator/tuned-78ctd" Jul 16 17:14:18.042746 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:14:18.041467 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/7db99718-7ae4-48bd-9c33-900ea9a925ad-etc-kubernetes\") pod \"tuned-78ctd\" (UID: \"7db99718-7ae4-48bd-9c33-900ea9a925ad\") " pod="openshift-cluster-node-tuning-operator/tuned-78ctd" Jul 16 17:14:18.042746 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:14:18.041486 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/7db99718-7ae4-48bd-9c33-900ea9a925ad-etc-modprobe-d\") pod \"tuned-78ctd\" (UID: \"7db99718-7ae4-48bd-9c33-900ea9a925ad\") " pod="openshift-cluster-node-tuning-operator/tuned-78ctd" Jul 16 17:14:18.042746 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:14:18.041486 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/7db99718-7ae4-48bd-9c33-900ea9a925ad-var-lib-kubelet\") pod \"tuned-78ctd\" (UID: \"7db99718-7ae4-48bd-9c33-900ea9a925ad\") " pod="openshift-cluster-node-tuning-operator/tuned-78ctd" Jul 16 17:14:18.042746 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:14:18.041524 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/ff79255c-4f85-4600-a7d2-54534f299e29-hosts-file\") pod \"node-resolver-zw2xf\" (UID: \"ff79255c-4f85-4600-a7d2-54534f299e29\") " pod="openshift-dns/node-resolver-zw2xf" Jul 16 17:14:18.042746 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:14:18.041524 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/31abc787-9b8d-4943-a302-b6db28480e19-host-run-multus-certs\") pod \"multus-qj6b6\" (UID: \"31abc787-9b8d-4943-a302-b6db28480e19\") " pod="openshift-multus/multus-qj6b6" Jul 16 17:14:18.042746 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:14:18.041548 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/31abc787-9b8d-4943-a302-b6db28480e19-multus-cni-dir\") pod \"multus-qj6b6\" (UID: \"31abc787-9b8d-4943-a302-b6db28480e19\") " pod="openshift-multus/multus-qj6b6" Jul 16 17:14:18.042746 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:14:18.041565 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/7db99718-7ae4-48bd-9c33-900ea9a925ad-sys\") pod \"tuned-78ctd\" (UID: \"7db99718-7ae4-48bd-9c33-900ea9a925ad\") " pod="openshift-cluster-node-tuning-operator/tuned-78ctd" Jul 16 17:14:18.042746 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:14:18.041566 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/7db99718-7ae4-48bd-9c33-900ea9a925ad-etc-sysconfig\") pod \"tuned-78ctd\" (UID: \"7db99718-7ae4-48bd-9c33-900ea9a925ad\") " pod="openshift-cluster-node-tuning-operator/tuned-78ctd" Jul 16 17:14:18.042746 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:14:18.041575 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/9a29f379-46c1-4211-81b2-7f3d82d82d01-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-h7t4d\" (UID: \"9a29f379-46c1-4211-81b2-7f3d82d82d01\") " pod="openshift-ovn-kubernetes/ovnkube-node-h7t4d" Jul 16 17:14:18.042746 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:14:18.041585 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/ff79255c-4f85-4600-a7d2-54534f299e29-hosts-file\") pod \"node-resolver-zw2xf\" (UID: \"ff79255c-4f85-4600-a7d2-54534f299e29\") " pod="openshift-dns/node-resolver-zw2xf" Jul 16 17:14:18.042746 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:14:18.041528 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/7db99718-7ae4-48bd-9c33-900ea9a925ad-var-lib-kubelet\") pod \"tuned-78ctd\" (UID: \"7db99718-7ae4-48bd-9c33-900ea9a925ad\") " pod="openshift-cluster-node-tuning-operator/tuned-78ctd" Jul 16 17:14:18.042746 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:14:18.041600 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nxtp2\" (UniqueName: \"kubernetes.io/projected/9a29f379-46c1-4211-81b2-7f3d82d82d01-kube-api-access-nxtp2\") pod \"ovnkube-node-h7t4d\" (UID: \"9a29f379-46c1-4211-81b2-7f3d82d82d01\") " pod="openshift-ovn-kubernetes/ovnkube-node-h7t4d" Jul 16 17:14:18.042746 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:14:18.041613 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/7db99718-7ae4-48bd-9c33-900ea9a925ad-etc-kubernetes\") pod \"tuned-78ctd\" (UID: \"7db99718-7ae4-48bd-9c33-900ea9a925ad\") " pod="openshift-cluster-node-tuning-operator/tuned-78ctd" Jul 16 17:14:18.043241 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:14:18.041630 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/7db99718-7ae4-48bd-9c33-900ea9a925ad-run\") pod \"tuned-78ctd\" (UID: \"7db99718-7ae4-48bd-9c33-900ea9a925ad\") " pod="openshift-cluster-node-tuning-operator/tuned-78ctd" Jul 16 17:14:18.043241 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:14:18.041634 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/31abc787-9b8d-4943-a302-b6db28480e19-multus-cni-dir\") pod \"multus-qj6b6\" (UID: \"31abc787-9b8d-4943-a302-b6db28480e19\") " pod="openshift-multus/multus-qj6b6" Jul 16 17:14:18.043241 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:14:18.041655 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/31abc787-9b8d-4943-a302-b6db28480e19-cni-binary-copy\") pod \"multus-qj6b6\" (UID: \"31abc787-9b8d-4943-a302-b6db28480e19\") " pod="openshift-multus/multus-qj6b6" Jul 16 17:14:18.043241 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:14:18.041704 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/6659c756-0d21-42ec-aa63-157ef0294e02-host-slash\") pod \"iptables-alerter-9mk4h\" (UID: \"6659c756-0d21-42ec-aa63-157ef0294e02\") " pod="openshift-network-operator/iptables-alerter-9mk4h" Jul 16 17:14:18.043241 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:14:18.041708 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/7db99718-7ae4-48bd-9c33-900ea9a925ad-run\") pod \"tuned-78ctd\" (UID: \"7db99718-7ae4-48bd-9c33-900ea9a925ad\") " pod="openshift-cluster-node-tuning-operator/tuned-78ctd" Jul 16 17:14:18.043241 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:14:18.041730 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/fa2959ee-fcc6-49eb-9205-65cf1a08779f-cnibin\") pod \"multus-additional-cni-plugins-6gzcp\" (UID: \"fa2959ee-fcc6-49eb-9205-65cf1a08779f\") " pod="openshift-multus/multus-additional-cni-plugins-6gzcp" Jul 16 17:14:18.043241 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:14:18.041754 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/01e55058-0022-4c74-966c-50605ba63d4e-device-dir\") pod \"aws-ebs-csi-driver-node-cn7hz\" (UID: \"01e55058-0022-4c74-966c-50605ba63d4e\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-cn7hz" Jul 16 17:14:18.043241 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:14:18.041781 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/4e00566f-55f0-43ec-9d1a-a883e839823e-original-pull-secret\") pod \"global-pull-secret-syncer-rg5vf\" (UID: \"4e00566f-55f0-43ec-9d1a-a883e839823e\") " pod="kube-system/global-pull-secret-syncer-rg5vf" Jul 16 17:14:18.043241 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:14:18.041804 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/31abc787-9b8d-4943-a302-b6db28480e19-host-var-lib-cni-bin\") pod \"multus-qj6b6\" (UID: \"31abc787-9b8d-4943-a302-b6db28480e19\") " pod="openshift-multus/multus-qj6b6" Jul 16 17:14:18.043241 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:14:18.041827 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/31abc787-9b8d-4943-a302-b6db28480e19-multus-daemon-config\") pod \"multus-qj6b6\" (UID: \"31abc787-9b8d-4943-a302-b6db28480e19\") " pod="openshift-multus/multus-qj6b6" Jul 16 17:14:18.043241 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:14:18.041868 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/fa2959ee-fcc6-49eb-9205-65cf1a08779f-cni-binary-copy\") pod \"multus-additional-cni-plugins-6gzcp\" (UID: \"fa2959ee-fcc6-49eb-9205-65cf1a08779f\") " pod="openshift-multus/multus-additional-cni-plugins-6gzcp" Jul 16 17:14:18.043241 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:14:18.041887 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/31abc787-9b8d-4943-a302-b6db28480e19-host-var-lib-cni-bin\") pod \"multus-qj6b6\" (UID: \"31abc787-9b8d-4943-a302-b6db28480e19\") " pod="openshift-multus/multus-qj6b6" Jul 16 17:14:18.043241 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:14:18.041913 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/fa2959ee-fcc6-49eb-9205-65cf1a08779f-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-6gzcp\" (UID: \"fa2959ee-fcc6-49eb-9205-65cf1a08779f\") " pod="openshift-multus/multus-additional-cni-plugins-6gzcp" Jul 16 17:14:18.043241 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:14:18.041939 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/fa2959ee-fcc6-49eb-9205-65cf1a08779f-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-6gzcp\" (UID: \"fa2959ee-fcc6-49eb-9205-65cf1a08779f\") " pod="openshift-multus/multus-additional-cni-plugins-6gzcp" Jul 16 17:14:18.043241 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:14:18.041962 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/01e55058-0022-4c74-966c-50605ba63d4e-socket-dir\") pod \"aws-ebs-csi-driver-node-cn7hz\" (UID: \"01e55058-0022-4c74-966c-50605ba63d4e\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-cn7hz" Jul 16 17:14:18.043241 ip-10-0-132-130 kubenswrapper[2578]: E0716 17:14:18.041972 2578 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 16 17:14:18.043241 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:14:18.041988 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2q66t\" (UniqueName: \"kubernetes.io/projected/01e55058-0022-4c74-966c-50605ba63d4e-kube-api-access-2q66t\") pod \"aws-ebs-csi-driver-node-cn7hz\" (UID: \"01e55058-0022-4c74-966c-50605ba63d4e\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-cn7hz" Jul 16 17:14:18.043759 ip-10-0-132-130 kubenswrapper[2578]: E0716 17:14:18.042019 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4e00566f-55f0-43ec-9d1a-a883e839823e-original-pull-secret podName:4e00566f-55f0-43ec-9d1a-a883e839823e nodeName:}" failed. No retries permitted until 2026-07-16 17:14:18.542000376 +0000 UTC m=+3.027844244 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/4e00566f-55f0-43ec-9d1a-a883e839823e-original-pull-secret") pod "global-pull-secret-syncer-rg5vf" (UID: "4e00566f-55f0-43ec-9d1a-a883e839823e") : object "kube-system"/"original-pull-secret" not registered Jul 16 17:14:18.043759 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:14:18.042044 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/a8425d74-5613-43e9-b6ff-3884b419dbf6-konnectivity-ca\") pod \"konnectivity-agent-kq8j9\" (UID: \"a8425d74-5613-43e9-b6ff-3884b419dbf6\") " pod="kube-system/konnectivity-agent-kq8j9" Jul 16 17:14:18.043759 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:14:18.042073 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/31abc787-9b8d-4943-a302-b6db28480e19-host-var-lib-kubelet\") pod \"multus-qj6b6\" (UID: \"31abc787-9b8d-4943-a302-b6db28480e19\") " pod="openshift-multus/multus-qj6b6" Jul 16 17:14:18.043759 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:14:18.042099 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-l64xp\" (UniqueName: \"kubernetes.io/projected/8232d058-70ea-4e6d-ae51-e5ba4303dd93-kube-api-access-l64xp\") pod \"network-check-target-g4r9z\" (UID: \"8232d058-70ea-4e6d-ae51-e5ba4303dd93\") " pod="openshift-network-diagnostics/network-check-target-g4r9z" Jul 16 17:14:18.043759 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:14:18.042125 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ff79255c-4f85-4600-a7d2-54534f299e29-tmp-dir\") pod \"node-resolver-zw2xf\" (UID: \"ff79255c-4f85-4600-a7d2-54534f299e29\") " pod="openshift-dns/node-resolver-zw2xf" Jul 16 17:14:18.043759 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:14:18.042137 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/31abc787-9b8d-4943-a302-b6db28480e19-host-var-lib-kubelet\") pod \"multus-qj6b6\" (UID: \"31abc787-9b8d-4943-a302-b6db28480e19\") " pod="openshift-multus/multus-qj6b6" Jul 16 17:14:18.043759 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:14:18.042148 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rx9sg\" (UniqueName: \"kubernetes.io/projected/7db99718-7ae4-48bd-9c33-900ea9a925ad-kube-api-access-rx9sg\") pod \"tuned-78ctd\" (UID: \"7db99718-7ae4-48bd-9c33-900ea9a925ad\") " pod="openshift-cluster-node-tuning-operator/tuned-78ctd" Jul 16 17:14:18.043759 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:14:18.042178 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/01e55058-0022-4c74-966c-50605ba63d4e-sys-fs\") pod \"aws-ebs-csi-driver-node-cn7hz\" (UID: \"01e55058-0022-4c74-966c-50605ba63d4e\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-cn7hz" Jul 16 17:14:18.043759 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:14:18.042203 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/dfca47c1-7bfb-4111-a283-7b86affa45a7-host\") pod \"node-ca-lfsvs\" (UID: \"dfca47c1-7bfb-4111-a283-7b86affa45a7\") " pod="openshift-image-registry/node-ca-lfsvs" Jul 16 17:14:18.043759 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:14:18.042228 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/31abc787-9b8d-4943-a302-b6db28480e19-host-var-lib-cni-multus\") pod \"multus-qj6b6\" (UID: \"31abc787-9b8d-4943-a302-b6db28480e19\") " pod="openshift-multus/multus-qj6b6" Jul 16 17:14:18.043759 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:14:18.042300 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/31abc787-9b8d-4943-a302-b6db28480e19-host-var-lib-cni-multus\") pod \"multus-qj6b6\" (UID: \"31abc787-9b8d-4943-a302-b6db28480e19\") " pod="openshift-multus/multus-qj6b6" Jul 16 17:14:18.043759 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:14:18.042306 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/31abc787-9b8d-4943-a302-b6db28480e19-multus-daemon-config\") pod \"multus-qj6b6\" (UID: \"31abc787-9b8d-4943-a302-b6db28480e19\") " pod="openshift-multus/multus-qj6b6" Jul 16 17:14:18.043759 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:14:18.042353 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/dfca47c1-7bfb-4111-a283-7b86affa45a7-host\") pod \"node-ca-lfsvs\" (UID: \"dfca47c1-7bfb-4111-a283-7b86affa45a7\") " pod="openshift-image-registry/node-ca-lfsvs" Jul 16 17:14:18.043759 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:14:18.042476 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ff79255c-4f85-4600-a7d2-54534f299e29-tmp-dir\") pod \"node-resolver-zw2xf\" (UID: \"ff79255c-4f85-4600-a7d2-54534f299e29\") " pod="openshift-dns/node-resolver-zw2xf" Jul 16 17:14:18.043759 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:14:18.042582 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/a8425d74-5613-43e9-b6ff-3884b419dbf6-konnectivity-ca\") pod \"konnectivity-agent-kq8j9\" (UID: \"a8425d74-5613-43e9-b6ff-3884b419dbf6\") " pod="kube-system/konnectivity-agent-kq8j9" Jul 16 17:14:18.043759 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:14:18.042640 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/31abc787-9b8d-4943-a302-b6db28480e19-cni-binary-copy\") pod \"multus-qj6b6\" (UID: \"31abc787-9b8d-4943-a302-b6db28480e19\") " pod="openshift-multus/multus-qj6b6" Jul 16 17:14:18.044189 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:14:18.043847 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/7db99718-7ae4-48bd-9c33-900ea9a925ad-tmp\") pod \"tuned-78ctd\" (UID: \"7db99718-7ae4-48bd-9c33-900ea9a925ad\") " pod="openshift-cluster-node-tuning-operator/tuned-78ctd" Jul 16 17:14:18.044189 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:14:18.043862 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/7db99718-7ae4-48bd-9c33-900ea9a925ad-etc-tuned\") pod \"tuned-78ctd\" (UID: \"7db99718-7ae4-48bd-9c33-900ea9a925ad\") " pod="openshift-cluster-node-tuning-operator/tuned-78ctd" Jul 16 17:14:18.044189 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:14:18.044010 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/a8425d74-5613-43e9-b6ff-3884b419dbf6-agent-certs\") pod \"konnectivity-agent-kq8j9\" (UID: \"a8425d74-5613-43e9-b6ff-3884b419dbf6\") " pod="kube-system/konnectivity-agent-kq8j9" Jul 16 17:14:18.067354 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:14:18.067327 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vh2mm\" (UniqueName: \"kubernetes.io/projected/f90822ad-ab14-4108-8294-2e76c35a8aa5-kube-api-access-vh2mm\") pod \"network-metrics-daemon-kvr2r\" (UID: \"f90822ad-ab14-4108-8294-2e76c35a8aa5\") " pod="openshift-multus/network-metrics-daemon-kvr2r" Jul 16 17:14:18.068933 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:14:18.068909 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-b9pkd\" (UniqueName: \"kubernetes.io/projected/31abc787-9b8d-4943-a302-b6db28480e19-kube-api-access-b9pkd\") pod \"multus-qj6b6\" (UID: \"31abc787-9b8d-4943-a302-b6db28480e19\") " pod="openshift-multus/multus-qj6b6" Jul 16 17:14:18.069167 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:14:18.069151 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-cglr8\" (UniqueName: \"kubernetes.io/projected/dfca47c1-7bfb-4111-a283-7b86affa45a7-kube-api-access-cglr8\") pod \"node-ca-lfsvs\" (UID: \"dfca47c1-7bfb-4111-a283-7b86affa45a7\") " pod="openshift-image-registry/node-ca-lfsvs" Jul 16 17:14:18.069530 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:14:18.069507 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rx9sg\" (UniqueName: \"kubernetes.io/projected/7db99718-7ae4-48bd-9c33-900ea9a925ad-kube-api-access-rx9sg\") pod \"tuned-78ctd\" (UID: \"7db99718-7ae4-48bd-9c33-900ea9a925ad\") " pod="openshift-cluster-node-tuning-operator/tuned-78ctd" Jul 16 17:14:18.070878 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:14:18.070860 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6dhhw\" (UniqueName: \"kubernetes.io/projected/ff79255c-4f85-4600-a7d2-54534f299e29-kube-api-access-6dhhw\") pod \"node-resolver-zw2xf\" (UID: \"ff79255c-4f85-4600-a7d2-54534f299e29\") " pod="openshift-dns/node-resolver-zw2xf" Jul 16 17:14:18.118585 ip-10-0-132-130 kubenswrapper[2578]: W0716 17:14:18.118561 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf724a0822cdfaacbec5071cc8e93c31c.slice/crio-964122b336ace1f20dd08913039e4c0c219cd329885568a9d2f86a685c5d275c WatchSource:0}: Error finding container 964122b336ace1f20dd08913039e4c0c219cd329885568a9d2f86a685c5d275c: Status 404 returned error can't find the container with id 964122b336ace1f20dd08913039e4c0c219cd329885568a9d2f86a685c5d275c Jul 16 17:14:18.142590 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:14:18.142566 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/01e55058-0022-4c74-966c-50605ba63d4e-etc-selinux\") pod \"aws-ebs-csi-driver-node-cn7hz\" (UID: \"01e55058-0022-4c74-966c-50605ba63d4e\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-cn7hz" Jul 16 17:14:18.142681 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:14:18.142595 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/9a29f379-46c1-4211-81b2-7f3d82d82d01-env-overrides\") pod \"ovnkube-node-h7t4d\" (UID: \"9a29f379-46c1-4211-81b2-7f3d82d82d01\") " pod="openshift-ovn-kubernetes/ovnkube-node-h7t4d" Jul 16 17:14:18.142681 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:14:18.142616 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/fa2959ee-fcc6-49eb-9205-65cf1a08779f-os-release\") pod \"multus-additional-cni-plugins-6gzcp\" (UID: \"fa2959ee-fcc6-49eb-9205-65cf1a08779f\") " pod="openshift-multus/multus-additional-cni-plugins-6gzcp" Jul 16 17:14:18.142681 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:14:18.142633 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/9a29f379-46c1-4211-81b2-7f3d82d82d01-ovnkube-script-lib\") pod \"ovnkube-node-h7t4d\" (UID: \"9a29f379-46c1-4211-81b2-7f3d82d82d01\") " pod="openshift-ovn-kubernetes/ovnkube-node-h7t4d" Jul 16 17:14:18.142883 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:14:18.142685 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/fa2959ee-fcc6-49eb-9205-65cf1a08779f-os-release\") pod \"multus-additional-cni-plugins-6gzcp\" (UID: \"fa2959ee-fcc6-49eb-9205-65cf1a08779f\") " pod="openshift-multus/multus-additional-cni-plugins-6gzcp" Jul 16 17:14:18.142883 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:14:18.142734 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/6659c756-0d21-42ec-aa63-157ef0294e02-iptables-alerter-script\") pod \"iptables-alerter-9mk4h\" (UID: \"6659c756-0d21-42ec-aa63-157ef0294e02\") " pod="openshift-network-operator/iptables-alerter-9mk4h" Jul 16 17:14:18.142883 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:14:18.142750 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/9a29f379-46c1-4211-81b2-7f3d82d82d01-log-socket\") pod \"ovnkube-node-h7t4d\" (UID: \"9a29f379-46c1-4211-81b2-7f3d82d82d01\") " pod="openshift-ovn-kubernetes/ovnkube-node-h7t4d" Jul 16 17:14:18.142883 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:14:18.142791 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/9a29f379-46c1-4211-81b2-7f3d82d82d01-host-kubelet\") pod \"ovnkube-node-h7t4d\" (UID: \"9a29f379-46c1-4211-81b2-7f3d82d82d01\") " pod="openshift-ovn-kubernetes/ovnkube-node-h7t4d" Jul 16 17:14:18.142883 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:14:18.142748 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/01e55058-0022-4c74-966c-50605ba63d4e-etc-selinux\") pod \"aws-ebs-csi-driver-node-cn7hz\" (UID: \"01e55058-0022-4c74-966c-50605ba63d4e\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-cn7hz" Jul 16 17:14:18.142883 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:14:18.142822 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/9a29f379-46c1-4211-81b2-7f3d82d82d01-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-h7t4d\" (UID: \"9a29f379-46c1-4211-81b2-7f3d82d82d01\") " pod="openshift-ovn-kubernetes/ovnkube-node-h7t4d" Jul 16 17:14:18.142883 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:14:18.142836 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/9a29f379-46c1-4211-81b2-7f3d82d82d01-log-socket\") pod \"ovnkube-node-h7t4d\" (UID: \"9a29f379-46c1-4211-81b2-7f3d82d82d01\") " pod="openshift-ovn-kubernetes/ovnkube-node-h7t4d" Jul 16 17:14:18.142883 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:14:18.142845 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nxtp2\" (UniqueName: \"kubernetes.io/projected/9a29f379-46c1-4211-81b2-7f3d82d82d01-kube-api-access-nxtp2\") pod \"ovnkube-node-h7t4d\" (UID: \"9a29f379-46c1-4211-81b2-7f3d82d82d01\") " pod="openshift-ovn-kubernetes/ovnkube-node-h7t4d" Jul 16 17:14:18.142883 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:14:18.142886 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/6659c756-0d21-42ec-aa63-157ef0294e02-host-slash\") pod \"iptables-alerter-9mk4h\" (UID: \"6659c756-0d21-42ec-aa63-157ef0294e02\") " pod="openshift-network-operator/iptables-alerter-9mk4h" Jul 16 17:14:18.143325 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:14:18.142900 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/9a29f379-46c1-4211-81b2-7f3d82d82d01-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-h7t4d\" (UID: \"9a29f379-46c1-4211-81b2-7f3d82d82d01\") " pod="openshift-ovn-kubernetes/ovnkube-node-h7t4d" Jul 16 17:14:18.143325 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:14:18.142918 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/9a29f379-46c1-4211-81b2-7f3d82d82d01-host-kubelet\") pod \"ovnkube-node-h7t4d\" (UID: \"9a29f379-46c1-4211-81b2-7f3d82d82d01\") " pod="openshift-ovn-kubernetes/ovnkube-node-h7t4d" Jul 16 17:14:18.143325 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:14:18.142938 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/fa2959ee-fcc6-49eb-9205-65cf1a08779f-cnibin\") pod \"multus-additional-cni-plugins-6gzcp\" (UID: \"fa2959ee-fcc6-49eb-9205-65cf1a08779f\") " pod="openshift-multus/multus-additional-cni-plugins-6gzcp" Jul 16 17:14:18.143325 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:14:18.142945 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/6659c756-0d21-42ec-aa63-157ef0294e02-host-slash\") pod \"iptables-alerter-9mk4h\" (UID: \"6659c756-0d21-42ec-aa63-157ef0294e02\") " pod="openshift-network-operator/iptables-alerter-9mk4h" Jul 16 17:14:18.143325 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:14:18.142965 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/01e55058-0022-4c74-966c-50605ba63d4e-device-dir\") pod \"aws-ebs-csi-driver-node-cn7hz\" (UID: \"01e55058-0022-4c74-966c-50605ba63d4e\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-cn7hz" Jul 16 17:14:18.143325 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:14:18.142992 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/fa2959ee-fcc6-49eb-9205-65cf1a08779f-cnibin\") pod \"multus-additional-cni-plugins-6gzcp\" (UID: \"fa2959ee-fcc6-49eb-9205-65cf1a08779f\") " pod="openshift-multus/multus-additional-cni-plugins-6gzcp" Jul 16 17:14:18.143325 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:14:18.143068 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/01e55058-0022-4c74-966c-50605ba63d4e-device-dir\") pod \"aws-ebs-csi-driver-node-cn7hz\" (UID: \"01e55058-0022-4c74-966c-50605ba63d4e\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-cn7hz" Jul 16 17:14:18.143325 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:14:18.143092 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/fa2959ee-fcc6-49eb-9205-65cf1a08779f-cni-binary-copy\") pod \"multus-additional-cni-plugins-6gzcp\" (UID: \"fa2959ee-fcc6-49eb-9205-65cf1a08779f\") " pod="openshift-multus/multus-additional-cni-plugins-6gzcp" Jul 16 17:14:18.143325 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:14:18.143126 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/fa2959ee-fcc6-49eb-9205-65cf1a08779f-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-6gzcp\" (UID: \"fa2959ee-fcc6-49eb-9205-65cf1a08779f\") " pod="openshift-multus/multus-additional-cni-plugins-6gzcp" Jul 16 17:14:18.143325 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:14:18.143160 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/fa2959ee-fcc6-49eb-9205-65cf1a08779f-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-6gzcp\" (UID: \"fa2959ee-fcc6-49eb-9205-65cf1a08779f\") " pod="openshift-multus/multus-additional-cni-plugins-6gzcp" Jul 16 17:14:18.143325 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:14:18.143196 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/01e55058-0022-4c74-966c-50605ba63d4e-socket-dir\") pod \"aws-ebs-csi-driver-node-cn7hz\" (UID: \"01e55058-0022-4c74-966c-50605ba63d4e\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-cn7hz" Jul 16 17:14:18.143325 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:14:18.143232 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2q66t\" (UniqueName: \"kubernetes.io/projected/01e55058-0022-4c74-966c-50605ba63d4e-kube-api-access-2q66t\") pod \"aws-ebs-csi-driver-node-cn7hz\" (UID: \"01e55058-0022-4c74-966c-50605ba63d4e\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-cn7hz" Jul 16 17:14:18.143325 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:14:18.143258 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-l64xp\" (UniqueName: \"kubernetes.io/projected/8232d058-70ea-4e6d-ae51-e5ba4303dd93-kube-api-access-l64xp\") pod \"network-check-target-g4r9z\" (UID: \"8232d058-70ea-4e6d-ae51-e5ba4303dd93\") " pod="openshift-network-diagnostics/network-check-target-g4r9z" Jul 16 17:14:18.143325 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:14:18.143303 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/01e55058-0022-4c74-966c-50605ba63d4e-sys-fs\") pod \"aws-ebs-csi-driver-node-cn7hz\" (UID: \"01e55058-0022-4c74-966c-50605ba63d4e\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-cn7hz" Jul 16 17:14:18.143325 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:14:18.143331 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/9a29f379-46c1-4211-81b2-7f3d82d82d01-host-run-ovn-kubernetes\") pod \"ovnkube-node-h7t4d\" (UID: \"9a29f379-46c1-4211-81b2-7f3d82d82d01\") " pod="openshift-ovn-kubernetes/ovnkube-node-h7t4d" Jul 16 17:14:18.144047 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:14:18.143353 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/9a29f379-46c1-4211-81b2-7f3d82d82d01-host-cni-bin\") pod \"ovnkube-node-h7t4d\" (UID: \"9a29f379-46c1-4211-81b2-7f3d82d82d01\") " pod="openshift-ovn-kubernetes/ovnkube-node-h7t4d" Jul 16 17:14:18.144047 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:14:18.143378 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qtzjs\" (UniqueName: \"kubernetes.io/projected/fa2959ee-fcc6-49eb-9205-65cf1a08779f-kube-api-access-qtzjs\") pod \"multus-additional-cni-plugins-6gzcp\" (UID: \"fa2959ee-fcc6-49eb-9205-65cf1a08779f\") " pod="openshift-multus/multus-additional-cni-plugins-6gzcp" Jul 16 17:14:18.144047 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:14:18.143404 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jr76l\" (UniqueName: \"kubernetes.io/projected/6659c756-0d21-42ec-aa63-157ef0294e02-kube-api-access-jr76l\") pod \"iptables-alerter-9mk4h\" (UID: \"6659c756-0d21-42ec-aa63-157ef0294e02\") " pod="openshift-network-operator/iptables-alerter-9mk4h" Jul 16 17:14:18.144047 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:14:18.143428 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/fa2959ee-fcc6-49eb-9205-65cf1a08779f-tuning-conf-dir\") pod \"multus-additional-cni-plugins-6gzcp\" (UID: \"fa2959ee-fcc6-49eb-9205-65cf1a08779f\") " pod="openshift-multus/multus-additional-cni-plugins-6gzcp" Jul 16 17:14:18.144047 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:14:18.143451 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/01e55058-0022-4c74-966c-50605ba63d4e-kubelet-dir\") pod \"aws-ebs-csi-driver-node-cn7hz\" (UID: \"01e55058-0022-4c74-966c-50605ba63d4e\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-cn7hz" Jul 16 17:14:18.144047 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:14:18.143475 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/9a29f379-46c1-4211-81b2-7f3d82d82d01-host-cni-netd\") pod \"ovnkube-node-h7t4d\" (UID: \"9a29f379-46c1-4211-81b2-7f3d82d82d01\") " pod="openshift-ovn-kubernetes/ovnkube-node-h7t4d" Jul 16 17:14:18.144047 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:14:18.143499 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/9a29f379-46c1-4211-81b2-7f3d82d82d01-ovn-node-metrics-cert\") pod \"ovnkube-node-h7t4d\" (UID: \"9a29f379-46c1-4211-81b2-7f3d82d82d01\") " pod="openshift-ovn-kubernetes/ovnkube-node-h7t4d" Jul 16 17:14:18.144047 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:14:18.143520 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/01e55058-0022-4c74-966c-50605ba63d4e-registration-dir\") pod \"aws-ebs-csi-driver-node-cn7hz\" (UID: \"01e55058-0022-4c74-966c-50605ba63d4e\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-cn7hz" Jul 16 17:14:18.144047 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:14:18.143542 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/9a29f379-46c1-4211-81b2-7f3d82d82d01-systemd-units\") pod \"ovnkube-node-h7t4d\" (UID: \"9a29f379-46c1-4211-81b2-7f3d82d82d01\") " pod="openshift-ovn-kubernetes/ovnkube-node-h7t4d" Jul 16 17:14:18.144047 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:14:18.143568 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/fa2959ee-fcc6-49eb-9205-65cf1a08779f-system-cni-dir\") pod \"multus-additional-cni-plugins-6gzcp\" (UID: \"fa2959ee-fcc6-49eb-9205-65cf1a08779f\") " pod="openshift-multus/multus-additional-cni-plugins-6gzcp" Jul 16 17:14:18.144047 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:14:18.143583 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/9a29f379-46c1-4211-81b2-7f3d82d82d01-env-overrides\") pod \"ovnkube-node-h7t4d\" (UID: \"9a29f379-46c1-4211-81b2-7f3d82d82d01\") " pod="openshift-ovn-kubernetes/ovnkube-node-h7t4d" Jul 16 17:14:18.144047 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:14:18.143615 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/9a29f379-46c1-4211-81b2-7f3d82d82d01-node-log\") pod \"ovnkube-node-h7t4d\" (UID: \"9a29f379-46c1-4211-81b2-7f3d82d82d01\") " pod="openshift-ovn-kubernetes/ovnkube-node-h7t4d" Jul 16 17:14:18.144047 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:14:18.143640 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/9a29f379-46c1-4211-81b2-7f3d82d82d01-host-slash\") pod \"ovnkube-node-h7t4d\" (UID: \"9a29f379-46c1-4211-81b2-7f3d82d82d01\") " pod="openshift-ovn-kubernetes/ovnkube-node-h7t4d" Jul 16 17:14:18.144047 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:14:18.143663 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/9a29f379-46c1-4211-81b2-7f3d82d82d01-run-systemd\") pod \"ovnkube-node-h7t4d\" (UID: \"9a29f379-46c1-4211-81b2-7f3d82d82d01\") " pod="openshift-ovn-kubernetes/ovnkube-node-h7t4d" Jul 16 17:14:18.144047 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:14:18.143729 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/9a29f379-46c1-4211-81b2-7f3d82d82d01-etc-openvswitch\") pod \"ovnkube-node-h7t4d\" (UID: \"9a29f379-46c1-4211-81b2-7f3d82d82d01\") " pod="openshift-ovn-kubernetes/ovnkube-node-h7t4d" Jul 16 17:14:18.144047 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:14:18.143753 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/9a29f379-46c1-4211-81b2-7f3d82d82d01-run-openvswitch\") pod \"ovnkube-node-h7t4d\" (UID: \"9a29f379-46c1-4211-81b2-7f3d82d82d01\") " pod="openshift-ovn-kubernetes/ovnkube-node-h7t4d" Jul 16 17:14:18.144047 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:14:18.143754 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/fa2959ee-fcc6-49eb-9205-65cf1a08779f-cni-binary-copy\") pod \"multus-additional-cni-plugins-6gzcp\" (UID: \"fa2959ee-fcc6-49eb-9205-65cf1a08779f\") " pod="openshift-multus/multus-additional-cni-plugins-6gzcp" Jul 16 17:14:18.144561 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:14:18.143759 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/fa2959ee-fcc6-49eb-9205-65cf1a08779f-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-6gzcp\" (UID: \"fa2959ee-fcc6-49eb-9205-65cf1a08779f\") " pod="openshift-multus/multus-additional-cni-plugins-6gzcp" Jul 16 17:14:18.144561 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:14:18.143768 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/9a29f379-46c1-4211-81b2-7f3d82d82d01-ovnkube-script-lib\") pod \"ovnkube-node-h7t4d\" (UID: \"9a29f379-46c1-4211-81b2-7f3d82d82d01\") " pod="openshift-ovn-kubernetes/ovnkube-node-h7t4d" Jul 16 17:14:18.144561 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:14:18.143777 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/9a29f379-46c1-4211-81b2-7f3d82d82d01-ovnkube-config\") pod \"ovnkube-node-h7t4d\" (UID: \"9a29f379-46c1-4211-81b2-7f3d82d82d01\") " pod="openshift-ovn-kubernetes/ovnkube-node-h7t4d" Jul 16 17:14:18.144561 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:14:18.143800 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/fa2959ee-fcc6-49eb-9205-65cf1a08779f-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-6gzcp\" (UID: \"fa2959ee-fcc6-49eb-9205-65cf1a08779f\") " pod="openshift-multus/multus-additional-cni-plugins-6gzcp" Jul 16 17:14:18.144561 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:14:18.143818 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/9a29f379-46c1-4211-81b2-7f3d82d82d01-systemd-units\") pod \"ovnkube-node-h7t4d\" (UID: \"9a29f379-46c1-4211-81b2-7f3d82d82d01\") " pod="openshift-ovn-kubernetes/ovnkube-node-h7t4d" Jul 16 17:14:18.144561 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:14:18.143838 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/fa2959ee-fcc6-49eb-9205-65cf1a08779f-system-cni-dir\") pod \"multus-additional-cni-plugins-6gzcp\" (UID: \"fa2959ee-fcc6-49eb-9205-65cf1a08779f\") " pod="openshift-multus/multus-additional-cni-plugins-6gzcp" Jul 16 17:14:18.144561 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:14:18.143846 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/01e55058-0022-4c74-966c-50605ba63d4e-kubelet-dir\") pod \"aws-ebs-csi-driver-node-cn7hz\" (UID: \"01e55058-0022-4c74-966c-50605ba63d4e\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-cn7hz" Jul 16 17:14:18.144561 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:14:18.143859 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/9a29f379-46c1-4211-81b2-7f3d82d82d01-host-run-ovn-kubernetes\") pod \"ovnkube-node-h7t4d\" (UID: \"9a29f379-46c1-4211-81b2-7f3d82d82d01\") " pod="openshift-ovn-kubernetes/ovnkube-node-h7t4d" Jul 16 17:14:18.144561 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:14:18.143863 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/9a29f379-46c1-4211-81b2-7f3d82d82d01-host-cni-bin\") pod \"ovnkube-node-h7t4d\" (UID: \"9a29f379-46c1-4211-81b2-7f3d82d82d01\") " pod="openshift-ovn-kubernetes/ovnkube-node-h7t4d" Jul 16 17:14:18.144561 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:14:18.143867 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/01e55058-0022-4c74-966c-50605ba63d4e-socket-dir\") pod \"aws-ebs-csi-driver-node-cn7hz\" (UID: \"01e55058-0022-4c74-966c-50605ba63d4e\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-cn7hz" Jul 16 17:14:18.144561 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:14:18.143871 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/9a29f379-46c1-4211-81b2-7f3d82d82d01-host-run-netns\") pod \"ovnkube-node-h7t4d\" (UID: \"9a29f379-46c1-4211-81b2-7f3d82d82d01\") " pod="openshift-ovn-kubernetes/ovnkube-node-h7t4d" Jul 16 17:14:18.144561 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:14:18.143881 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/9a29f379-46c1-4211-81b2-7f3d82d82d01-node-log\") pod \"ovnkube-node-h7t4d\" (UID: \"9a29f379-46c1-4211-81b2-7f3d82d82d01\") " pod="openshift-ovn-kubernetes/ovnkube-node-h7t4d" Jul 16 17:14:18.144561 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:14:18.143897 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/01e55058-0022-4c74-966c-50605ba63d4e-registration-dir\") pod \"aws-ebs-csi-driver-node-cn7hz\" (UID: \"01e55058-0022-4c74-966c-50605ba63d4e\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-cn7hz" Jul 16 17:14:18.144561 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:14:18.143906 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/9a29f379-46c1-4211-81b2-7f3d82d82d01-host-run-netns\") pod \"ovnkube-node-h7t4d\" (UID: \"9a29f379-46c1-4211-81b2-7f3d82d82d01\") " pod="openshift-ovn-kubernetes/ovnkube-node-h7t4d" Jul 16 17:14:18.144561 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:14:18.143901 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/01e55058-0022-4c74-966c-50605ba63d4e-sys-fs\") pod \"aws-ebs-csi-driver-node-cn7hz\" (UID: \"01e55058-0022-4c74-966c-50605ba63d4e\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-cn7hz" Jul 16 17:14:18.144561 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:14:18.143911 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/6659c756-0d21-42ec-aa63-157ef0294e02-iptables-alerter-script\") pod \"iptables-alerter-9mk4h\" (UID: \"6659c756-0d21-42ec-aa63-157ef0294e02\") " pod="openshift-network-operator/iptables-alerter-9mk4h" Jul 16 17:14:18.144561 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:14:18.143939 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/9a29f379-46c1-4211-81b2-7f3d82d82d01-var-lib-openvswitch\") pod \"ovnkube-node-h7t4d\" (UID: \"9a29f379-46c1-4211-81b2-7f3d82d82d01\") " pod="openshift-ovn-kubernetes/ovnkube-node-h7t4d" Jul 16 17:14:18.145036 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:14:18.143972 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/9a29f379-46c1-4211-81b2-7f3d82d82d01-host-slash\") pod \"ovnkube-node-h7t4d\" (UID: \"9a29f379-46c1-4211-81b2-7f3d82d82d01\") " pod="openshift-ovn-kubernetes/ovnkube-node-h7t4d" Jul 16 17:14:18.145036 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:14:18.143976 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/9a29f379-46c1-4211-81b2-7f3d82d82d01-run-ovn\") pod \"ovnkube-node-h7t4d\" (UID: \"9a29f379-46c1-4211-81b2-7f3d82d82d01\") " pod="openshift-ovn-kubernetes/ovnkube-node-h7t4d" Jul 16 17:14:18.145036 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:14:18.144021 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/9a29f379-46c1-4211-81b2-7f3d82d82d01-var-lib-openvswitch\") pod \"ovnkube-node-h7t4d\" (UID: \"9a29f379-46c1-4211-81b2-7f3d82d82d01\") " pod="openshift-ovn-kubernetes/ovnkube-node-h7t4d" Jul 16 17:14:18.145036 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:14:18.144035 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/9a29f379-46c1-4211-81b2-7f3d82d82d01-run-ovn\") pod \"ovnkube-node-h7t4d\" (UID: \"9a29f379-46c1-4211-81b2-7f3d82d82d01\") " pod="openshift-ovn-kubernetes/ovnkube-node-h7t4d" Jul 16 17:14:18.145036 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:14:18.144066 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/fa2959ee-fcc6-49eb-9205-65cf1a08779f-tuning-conf-dir\") pod \"multus-additional-cni-plugins-6gzcp\" (UID: \"fa2959ee-fcc6-49eb-9205-65cf1a08779f\") " pod="openshift-multus/multus-additional-cni-plugins-6gzcp" Jul 16 17:14:18.145036 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:14:18.144084 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/9a29f379-46c1-4211-81b2-7f3d82d82d01-etc-openvswitch\") pod \"ovnkube-node-h7t4d\" (UID: \"9a29f379-46c1-4211-81b2-7f3d82d82d01\") " pod="openshift-ovn-kubernetes/ovnkube-node-h7t4d" Jul 16 17:14:18.145036 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:14:18.144124 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/9a29f379-46c1-4211-81b2-7f3d82d82d01-run-systemd\") pod \"ovnkube-node-h7t4d\" (UID: \"9a29f379-46c1-4211-81b2-7f3d82d82d01\") " pod="openshift-ovn-kubernetes/ovnkube-node-h7t4d" Jul 16 17:14:18.145036 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:14:18.144158 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/9a29f379-46c1-4211-81b2-7f3d82d82d01-host-cni-netd\") pod \"ovnkube-node-h7t4d\" (UID: \"9a29f379-46c1-4211-81b2-7f3d82d82d01\") " pod="openshift-ovn-kubernetes/ovnkube-node-h7t4d" Jul 16 17:14:18.145036 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:14:18.144175 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/9a29f379-46c1-4211-81b2-7f3d82d82d01-run-openvswitch\") pod \"ovnkube-node-h7t4d\" (UID: \"9a29f379-46c1-4211-81b2-7f3d82d82d01\") " pod="openshift-ovn-kubernetes/ovnkube-node-h7t4d" Jul 16 17:14:18.145036 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:14:18.144366 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/9a29f379-46c1-4211-81b2-7f3d82d82d01-ovnkube-config\") pod \"ovnkube-node-h7t4d\" (UID: \"9a29f379-46c1-4211-81b2-7f3d82d82d01\") " pod="openshift-ovn-kubernetes/ovnkube-node-h7t4d" Jul 16 17:14:18.145902 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:14:18.145888 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/9a29f379-46c1-4211-81b2-7f3d82d82d01-ovn-node-metrics-cert\") pod \"ovnkube-node-h7t4d\" (UID: \"9a29f379-46c1-4211-81b2-7f3d82d82d01\") " pod="openshift-ovn-kubernetes/ovnkube-node-h7t4d" Jul 16 17:14:18.157027 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:14:18.156961 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nxtp2\" (UniqueName: \"kubernetes.io/projected/9a29f379-46c1-4211-81b2-7f3d82d82d01-kube-api-access-nxtp2\") pod \"ovnkube-node-h7t4d\" (UID: \"9a29f379-46c1-4211-81b2-7f3d82d82d01\") " pod="openshift-ovn-kubernetes/ovnkube-node-h7t4d" Jul 16 17:14:18.157525 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:14:18.157502 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jr76l\" (UniqueName: \"kubernetes.io/projected/6659c756-0d21-42ec-aa63-157ef0294e02-kube-api-access-jr76l\") pod \"iptables-alerter-9mk4h\" (UID: \"6659c756-0d21-42ec-aa63-157ef0294e02\") " pod="openshift-network-operator/iptables-alerter-9mk4h" Jul 16 17:14:18.160521 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:14:18.158323 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qtzjs\" (UniqueName: \"kubernetes.io/projected/fa2959ee-fcc6-49eb-9205-65cf1a08779f-kube-api-access-qtzjs\") pod \"multus-additional-cni-plugins-6gzcp\" (UID: \"fa2959ee-fcc6-49eb-9205-65cf1a08779f\") " pod="openshift-multus/multus-additional-cni-plugins-6gzcp" Jul 16 17:14:18.161010 ip-10-0-132-130 kubenswrapper[2578]: E0716 17:14:18.160991 2578 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 16 17:14:18.161048 ip-10-0-132-130 kubenswrapper[2578]: E0716 17:14:18.161014 2578 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 16 17:14:18.161048 ip-10-0-132-130 kubenswrapper[2578]: E0716 17:14:18.161025 2578 projected.go:196] Error preparing data for projected volume kube-api-access-l64xp for pod openshift-network-diagnostics/network-check-target-g4r9z: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 16 17:14:18.161127 ip-10-0-132-130 kubenswrapper[2578]: E0716 17:14:18.161074 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/8232d058-70ea-4e6d-ae51-e5ba4303dd93-kube-api-access-l64xp podName:8232d058-70ea-4e6d-ae51-e5ba4303dd93 nodeName:}" failed. No retries permitted until 2026-07-16 17:14:18.661060372 +0000 UTC m=+3.146904238 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-l64xp" (UniqueName: "kubernetes.io/projected/8232d058-70ea-4e6d-ae51-e5ba4303dd93-kube-api-access-l64xp") pod "network-check-target-g4r9z" (UID: "8232d058-70ea-4e6d-ae51-e5ba4303dd93") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 16 17:14:18.162730 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:14:18.162706 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2q66t\" (UniqueName: \"kubernetes.io/projected/01e55058-0022-4c74-966c-50605ba63d4e-kube-api-access-2q66t\") pod \"aws-ebs-csi-driver-node-cn7hz\" (UID: \"01e55058-0022-4c74-966c-50605ba63d4e\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-cn7hz" Jul 16 17:14:18.216166 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:14:18.216129 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-130.ec2.internal" event={"ID":"f724a0822cdfaacbec5071cc8e93c31c","Type":"ContainerStarted","Data":"964122b336ace1f20dd08913039e4c0c219cd329885568a9d2f86a685c5d275c"} Jul 16 17:14:18.217408 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:14:18.217390 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-132-130.ec2.internal" event={"ID":"7876ab904043bc7f87e7c28015b01ebc","Type":"ContainerStarted","Data":"cf0870221f7437ddbfd6db32e5917b8206a0158116c286b3af334bab9b587fef"} Jul 16 17:14:18.226536 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:14:18.226520 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-zw2xf" Jul 16 17:14:18.232114 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:14:18.232098 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-kq8j9" Jul 16 17:14:18.236336 ip-10-0-132-130 kubenswrapper[2578]: W0716 17:14:18.236313 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podff79255c_4f85_4600_a7d2_54534f299e29.slice/crio-0371f04102ddfc1a240117eb2cc8d4024884d1ccd4a9eb7097ff46dffe54596e WatchSource:0}: Error finding container 0371f04102ddfc1a240117eb2cc8d4024884d1ccd4a9eb7097ff46dffe54596e: Status 404 returned error can't find the container with id 0371f04102ddfc1a240117eb2cc8d4024884d1ccd4a9eb7097ff46dffe54596e Jul 16 17:14:18.239184 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:14:18.239168 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-78ctd" Jul 16 17:14:18.242578 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:14:18.242557 2578 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Jul 16 17:14:18.242680 ip-10-0-132-130 kubenswrapper[2578]: W0716 17:14:18.242658 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda8425d74_5613_43e9_b6ff_3884b419dbf6.slice/crio-266ea4646af1b69d4fabb3369c14113ae0e2af907058d2c7b664f70c6df4a191 WatchSource:0}: Error finding container 266ea4646af1b69d4fabb3369c14113ae0e2af907058d2c7b664f70c6df4a191: Status 404 returned error can't find the container with id 266ea4646af1b69d4fabb3369c14113ae0e2af907058d2c7b664f70c6df4a191 Jul 16 17:14:18.246072 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:14:18.246059 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-lfsvs" Jul 16 17:14:18.247146 ip-10-0-132-130 kubenswrapper[2578]: W0716 17:14:18.247129 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7db99718_7ae4_48bd_9c33_900ea9a925ad.slice/crio-ab4e90e0c61475c096ed2fc3106f8777b71e83178b81691dd136a4f43c22aa19 WatchSource:0}: Error finding container ab4e90e0c61475c096ed2fc3106f8777b71e83178b81691dd136a4f43c22aa19: Status 404 returned error can't find the container with id ab4e90e0c61475c096ed2fc3106f8777b71e83178b81691dd136a4f43c22aa19 Jul 16 17:14:18.251561 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:14:18.251547 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-qj6b6" Jul 16 17:14:18.254997 ip-10-0-132-130 kubenswrapper[2578]: W0716 17:14:18.254977 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddfca47c1_7bfb_4111_a283_7b86affa45a7.slice/crio-050ac290e0fc4dab53adba54e1e2e3ea3e513e86765439b40e72fc818f297dc3 WatchSource:0}: Error finding container 050ac290e0fc4dab53adba54e1e2e3ea3e513e86765439b40e72fc818f297dc3: Status 404 returned error can't find the container with id 050ac290e0fc4dab53adba54e1e2e3ea3e513e86765439b40e72fc818f297dc3 Jul 16 17:14:18.256465 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:14:18.256447 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-h7t4d" Jul 16 17:14:18.259413 ip-10-0-132-130 kubenswrapper[2578]: W0716 17:14:18.259387 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod31abc787_9b8d_4943_a302_b6db28480e19.slice/crio-1fc0e7d33d79f57f2bf29ecd6f3ead7945574a34eef44cd9d6ab36465f271f19 WatchSource:0}: Error finding container 1fc0e7d33d79f57f2bf29ecd6f3ead7945574a34eef44cd9d6ab36465f271f19: Status 404 returned error can't find the container with id 1fc0e7d33d79f57f2bf29ecd6f3ead7945574a34eef44cd9d6ab36465f271f19 Jul 16 17:14:18.261821 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:14:18.261759 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-9mk4h" Jul 16 17:14:18.264170 ip-10-0-132-130 kubenswrapper[2578]: W0716 17:14:18.264150 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9a29f379_46c1_4211_81b2_7f3d82d82d01.slice/crio-62e80ae6a9daaaaaa328f035755ebdcfc1f8e0c834b07e53f5a004802825ea5c WatchSource:0}: Error finding container 62e80ae6a9daaaaaa328f035755ebdcfc1f8e0c834b07e53f5a004802825ea5c: Status 404 returned error can't find the container with id 62e80ae6a9daaaaaa328f035755ebdcfc1f8e0c834b07e53f5a004802825ea5c Jul 16 17:14:18.267392 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:14:18.267379 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-6gzcp" Jul 16 17:14:18.269320 ip-10-0-132-130 kubenswrapper[2578]: W0716 17:14:18.269304 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6659c756_0d21_42ec_aa63_157ef0294e02.slice/crio-7241f02b48591eaa70d7c7e44540ef78ab52798a30d1812d1f5fa9766efc1b67 WatchSource:0}: Error finding container 7241f02b48591eaa70d7c7e44540ef78ab52798a30d1812d1f5fa9766efc1b67: Status 404 returned error can't find the container with id 7241f02b48591eaa70d7c7e44540ef78ab52798a30d1812d1f5fa9766efc1b67 Jul 16 17:14:18.272294 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:14:18.272277 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-cn7hz" Jul 16 17:14:18.279777 ip-10-0-132-130 kubenswrapper[2578]: W0716 17:14:18.279749 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfa2959ee_fcc6_49eb_9205_65cf1a08779f.slice/crio-7b60c4b36332aebd884948fb57e49c857bf192fc77cc7c14119959fe359ab3ce WatchSource:0}: Error finding container 7b60c4b36332aebd884948fb57e49c857bf192fc77cc7c14119959fe359ab3ce: Status 404 returned error can't find the container with id 7b60c4b36332aebd884948fb57e49c857bf192fc77cc7c14119959fe359ab3ce Jul 16 17:14:18.282582 ip-10-0-132-130 kubenswrapper[2578]: W0716 17:14:18.282559 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod01e55058_0022_4c74_966c_50605ba63d4e.slice/crio-fcf11eb059341d2a270804605891f134b2550f488010ee5581fff10e8dd9be63 WatchSource:0}: Error finding container fcf11eb059341d2a270804605891f134b2550f488010ee5581fff10e8dd9be63: Status 404 returned error can't find the container with id fcf11eb059341d2a270804605891f134b2550f488010ee5581fff10e8dd9be63 Jul 16 17:14:18.547449 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:14:18.547405 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/4e00566f-55f0-43ec-9d1a-a883e839823e-original-pull-secret\") pod \"global-pull-secret-syncer-rg5vf\" (UID: \"4e00566f-55f0-43ec-9d1a-a883e839823e\") " pod="kube-system/global-pull-secret-syncer-rg5vf" Jul 16 17:14:18.547635 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:14:18.547475 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f90822ad-ab14-4108-8294-2e76c35a8aa5-metrics-certs\") pod \"network-metrics-daemon-kvr2r\" (UID: \"f90822ad-ab14-4108-8294-2e76c35a8aa5\") " pod="openshift-multus/network-metrics-daemon-kvr2r" Jul 16 17:14:18.547635 ip-10-0-132-130 kubenswrapper[2578]: E0716 17:14:18.547621 2578 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 16 17:14:18.547772 ip-10-0-132-130 kubenswrapper[2578]: E0716 17:14:18.547675 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f90822ad-ab14-4108-8294-2e76c35a8aa5-metrics-certs podName:f90822ad-ab14-4108-8294-2e76c35a8aa5 nodeName:}" failed. No retries permitted until 2026-07-16 17:14:19.547657483 +0000 UTC m=+4.033501356 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/f90822ad-ab14-4108-8294-2e76c35a8aa5-metrics-certs") pod "network-metrics-daemon-kvr2r" (UID: "f90822ad-ab14-4108-8294-2e76c35a8aa5") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 16 17:14:18.547838 ip-10-0-132-130 kubenswrapper[2578]: E0716 17:14:18.547798 2578 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 16 17:14:18.547892 ip-10-0-132-130 kubenswrapper[2578]: E0716 17:14:18.547843 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4e00566f-55f0-43ec-9d1a-a883e839823e-original-pull-secret podName:4e00566f-55f0-43ec-9d1a-a883e839823e nodeName:}" failed. No retries permitted until 2026-07-16 17:14:19.547828918 +0000 UTC m=+4.033672797 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/4e00566f-55f0-43ec-9d1a-a883e839823e-original-pull-secret") pod "global-pull-secret-syncer-rg5vf" (UID: "4e00566f-55f0-43ec-9d1a-a883e839823e") : object "kube-system"/"original-pull-secret" not registered Jul 16 17:14:18.749219 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:14:18.748486 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-l64xp\" (UniqueName: \"kubernetes.io/projected/8232d058-70ea-4e6d-ae51-e5ba4303dd93-kube-api-access-l64xp\") pod \"network-check-target-g4r9z\" (UID: \"8232d058-70ea-4e6d-ae51-e5ba4303dd93\") " pod="openshift-network-diagnostics/network-check-target-g4r9z" Jul 16 17:14:18.749219 ip-10-0-132-130 kubenswrapper[2578]: E0716 17:14:18.748743 2578 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 16 17:14:18.749219 ip-10-0-132-130 kubenswrapper[2578]: E0716 17:14:18.748767 2578 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 16 17:14:18.749219 ip-10-0-132-130 kubenswrapper[2578]: E0716 17:14:18.748782 2578 projected.go:196] Error preparing data for projected volume kube-api-access-l64xp for pod openshift-network-diagnostics/network-check-target-g4r9z: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 16 17:14:18.749219 ip-10-0-132-130 kubenswrapper[2578]: E0716 17:14:18.748839 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/8232d058-70ea-4e6d-ae51-e5ba4303dd93-kube-api-access-l64xp podName:8232d058-70ea-4e6d-ae51-e5ba4303dd93 nodeName:}" failed. No retries permitted until 2026-07-16 17:14:19.74882136 +0000 UTC m=+4.234665249 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-l64xp" (UniqueName: "kubernetes.io/projected/8232d058-70ea-4e6d-ae51-e5ba4303dd93-kube-api-access-l64xp") pod "network-check-target-g4r9z" (UID: "8232d058-70ea-4e6d-ae51-e5ba4303dd93") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 16 17:14:18.980943 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:14:18.980815 2578 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-15 17:09:17 +0000 UTC" deadline="2028-03-18 02:28:41.793139619 +0000 UTC" Jul 16 17:14:18.980943 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:14:18.980847 2578 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="14649h14m22.812296497s" Jul 16 17:14:19.209122 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:14:19.209094 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-kvr2r" Jul 16 17:14:19.209334 ip-10-0-132-130 kubenswrapper[2578]: E0716 17:14:19.209229 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-kvr2r" podUID="f90822ad-ab14-4108-8294-2e76c35a8aa5" Jul 16 17:14:19.209632 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:14:19.209613 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-g4r9z" Jul 16 17:14:19.209745 ip-10-0-132-130 kubenswrapper[2578]: E0716 17:14:19.209721 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-g4r9z" podUID="8232d058-70ea-4e6d-ae51-e5ba4303dd93" Jul 16 17:14:19.228619 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:14:19.228592 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-kq8j9" event={"ID":"a8425d74-5613-43e9-b6ff-3884b419dbf6","Type":"ContainerStarted","Data":"266ea4646af1b69d4fabb3369c14113ae0e2af907058d2c7b664f70c6df4a191"} Jul 16 17:14:19.232394 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:14:19.232327 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-zw2xf" event={"ID":"ff79255c-4f85-4600-a7d2-54534f299e29","Type":"ContainerStarted","Data":"0371f04102ddfc1a240117eb2cc8d4024884d1ccd4a9eb7097ff46dffe54596e"} Jul 16 17:14:19.235585 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:14:19.235557 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-cn7hz" event={"ID":"01e55058-0022-4c74-966c-50605ba63d4e","Type":"ContainerStarted","Data":"fcf11eb059341d2a270804605891f134b2550f488010ee5581fff10e8dd9be63"} Jul 16 17:14:19.237421 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:14:19.237372 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-6gzcp" event={"ID":"fa2959ee-fcc6-49eb-9205-65cf1a08779f","Type":"ContainerStarted","Data":"7b60c4b36332aebd884948fb57e49c857bf192fc77cc7c14119959fe359ab3ce"} Jul 16 17:14:19.239087 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:14:19.239059 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-qj6b6" event={"ID":"31abc787-9b8d-4943-a302-b6db28480e19","Type":"ContainerStarted","Data":"1fc0e7d33d79f57f2bf29ecd6f3ead7945574a34eef44cd9d6ab36465f271f19"} Jul 16 17:14:19.241991 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:14:19.241966 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-78ctd" event={"ID":"7db99718-7ae4-48bd-9c33-900ea9a925ad","Type":"ContainerStarted","Data":"ab4e90e0c61475c096ed2fc3106f8777b71e83178b81691dd136a4f43c22aa19"} Jul 16 17:14:19.246352 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:14:19.246294 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-9mk4h" event={"ID":"6659c756-0d21-42ec-aa63-157ef0294e02","Type":"ContainerStarted","Data":"7241f02b48591eaa70d7c7e44540ef78ab52798a30d1812d1f5fa9766efc1b67"} Jul 16 17:14:19.248664 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:14:19.248642 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-h7t4d" event={"ID":"9a29f379-46c1-4211-81b2-7f3d82d82d01","Type":"ContainerStarted","Data":"62e80ae6a9daaaaaa328f035755ebdcfc1f8e0c834b07e53f5a004802825ea5c"} Jul 16 17:14:19.257879 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:14:19.257855 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-lfsvs" event={"ID":"dfca47c1-7bfb-4111-a283-7b86affa45a7","Type":"ContainerStarted","Data":"050ac290e0fc4dab53adba54e1e2e3ea3e513e86765439b40e72fc818f297dc3"} Jul 16 17:14:19.554230 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:14:19.554196 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f90822ad-ab14-4108-8294-2e76c35a8aa5-metrics-certs\") pod \"network-metrics-daemon-kvr2r\" (UID: \"f90822ad-ab14-4108-8294-2e76c35a8aa5\") " pod="openshift-multus/network-metrics-daemon-kvr2r" Jul 16 17:14:19.554414 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:14:19.554301 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/4e00566f-55f0-43ec-9d1a-a883e839823e-original-pull-secret\") pod \"global-pull-secret-syncer-rg5vf\" (UID: \"4e00566f-55f0-43ec-9d1a-a883e839823e\") " pod="kube-system/global-pull-secret-syncer-rg5vf" Jul 16 17:14:19.554472 ip-10-0-132-130 kubenswrapper[2578]: E0716 17:14:19.554437 2578 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 16 17:14:19.554525 ip-10-0-132-130 kubenswrapper[2578]: E0716 17:14:19.554516 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4e00566f-55f0-43ec-9d1a-a883e839823e-original-pull-secret podName:4e00566f-55f0-43ec-9d1a-a883e839823e nodeName:}" failed. No retries permitted until 2026-07-16 17:14:21.554477436 +0000 UTC m=+6.040321305 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/4e00566f-55f0-43ec-9d1a-a883e839823e-original-pull-secret") pod "global-pull-secret-syncer-rg5vf" (UID: "4e00566f-55f0-43ec-9d1a-a883e839823e") : object "kube-system"/"original-pull-secret" not registered Jul 16 17:14:19.554866 ip-10-0-132-130 kubenswrapper[2578]: E0716 17:14:19.554846 2578 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 16 17:14:19.554954 ip-10-0-132-130 kubenswrapper[2578]: E0716 17:14:19.554903 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f90822ad-ab14-4108-8294-2e76c35a8aa5-metrics-certs podName:f90822ad-ab14-4108-8294-2e76c35a8aa5 nodeName:}" failed. No retries permitted until 2026-07-16 17:14:21.554887951 +0000 UTC m=+6.040731835 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/f90822ad-ab14-4108-8294-2e76c35a8aa5-metrics-certs") pod "network-metrics-daemon-kvr2r" (UID: "f90822ad-ab14-4108-8294-2e76c35a8aa5") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 16 17:14:19.755484 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:14:19.755449 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-l64xp\" (UniqueName: \"kubernetes.io/projected/8232d058-70ea-4e6d-ae51-e5ba4303dd93-kube-api-access-l64xp\") pod \"network-check-target-g4r9z\" (UID: \"8232d058-70ea-4e6d-ae51-e5ba4303dd93\") " pod="openshift-network-diagnostics/network-check-target-g4r9z" Jul 16 17:14:19.755683 ip-10-0-132-130 kubenswrapper[2578]: E0716 17:14:19.755657 2578 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 16 17:14:19.755787 ip-10-0-132-130 kubenswrapper[2578]: E0716 17:14:19.755735 2578 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 16 17:14:19.755787 ip-10-0-132-130 kubenswrapper[2578]: E0716 17:14:19.755751 2578 projected.go:196] Error preparing data for projected volume kube-api-access-l64xp for pod openshift-network-diagnostics/network-check-target-g4r9z: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 16 17:14:19.755944 ip-10-0-132-130 kubenswrapper[2578]: E0716 17:14:19.755822 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/8232d058-70ea-4e6d-ae51-e5ba4303dd93-kube-api-access-l64xp podName:8232d058-70ea-4e6d-ae51-e5ba4303dd93 nodeName:}" failed. No retries permitted until 2026-07-16 17:14:21.755791413 +0000 UTC m=+6.241635306 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-l64xp" (UniqueName: "kubernetes.io/projected/8232d058-70ea-4e6d-ae51-e5ba4303dd93-kube-api-access-l64xp") pod "network-check-target-g4r9z" (UID: "8232d058-70ea-4e6d-ae51-e5ba4303dd93") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 16 17:14:19.981594 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:14:19.981486 2578 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-15 17:09:17 +0000 UTC" deadline="2028-03-14 23:44:26.325725916 +0000 UTC" Jul 16 17:14:19.981594 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:14:19.981547 2578 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="14574h30m6.344182538s" Jul 16 17:14:20.211254 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:14:20.211222 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-rg5vf" Jul 16 17:14:20.211449 ip-10-0-132-130 kubenswrapper[2578]: E0716 17:14:20.211351 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-rg5vf" podUID="4e00566f-55f0-43ec-9d1a-a883e839823e" Jul 16 17:14:21.209237 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:14:21.209202 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-kvr2r" Jul 16 17:14:21.209681 ip-10-0-132-130 kubenswrapper[2578]: E0716 17:14:21.209357 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-kvr2r" podUID="f90822ad-ab14-4108-8294-2e76c35a8aa5" Jul 16 17:14:21.209801 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:14:21.209741 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-g4r9z" Jul 16 17:14:21.209868 ip-10-0-132-130 kubenswrapper[2578]: E0716 17:14:21.209824 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-g4r9z" podUID="8232d058-70ea-4e6d-ae51-e5ba4303dd93" Jul 16 17:14:21.574430 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:14:21.573684 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/4e00566f-55f0-43ec-9d1a-a883e839823e-original-pull-secret\") pod \"global-pull-secret-syncer-rg5vf\" (UID: \"4e00566f-55f0-43ec-9d1a-a883e839823e\") " pod="kube-system/global-pull-secret-syncer-rg5vf" Jul 16 17:14:21.574430 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:14:21.573783 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f90822ad-ab14-4108-8294-2e76c35a8aa5-metrics-certs\") pod \"network-metrics-daemon-kvr2r\" (UID: \"f90822ad-ab14-4108-8294-2e76c35a8aa5\") " pod="openshift-multus/network-metrics-daemon-kvr2r" Jul 16 17:14:21.574430 ip-10-0-132-130 kubenswrapper[2578]: E0716 17:14:21.573902 2578 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 16 17:14:21.574430 ip-10-0-132-130 kubenswrapper[2578]: E0716 17:14:21.573962 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f90822ad-ab14-4108-8294-2e76c35a8aa5-metrics-certs podName:f90822ad-ab14-4108-8294-2e76c35a8aa5 nodeName:}" failed. No retries permitted until 2026-07-16 17:14:25.573943865 +0000 UTC m=+10.059787737 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/f90822ad-ab14-4108-8294-2e76c35a8aa5-metrics-certs") pod "network-metrics-daemon-kvr2r" (UID: "f90822ad-ab14-4108-8294-2e76c35a8aa5") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 16 17:14:21.574430 ip-10-0-132-130 kubenswrapper[2578]: E0716 17:14:21.574349 2578 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 16 17:14:21.574430 ip-10-0-132-130 kubenswrapper[2578]: E0716 17:14:21.574398 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4e00566f-55f0-43ec-9d1a-a883e839823e-original-pull-secret podName:4e00566f-55f0-43ec-9d1a-a883e839823e nodeName:}" failed. No retries permitted until 2026-07-16 17:14:25.574381954 +0000 UTC m=+10.060225827 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/4e00566f-55f0-43ec-9d1a-a883e839823e-original-pull-secret") pod "global-pull-secret-syncer-rg5vf" (UID: "4e00566f-55f0-43ec-9d1a-a883e839823e") : object "kube-system"/"original-pull-secret" not registered Jul 16 17:14:21.776299 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:14:21.776260 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-l64xp\" (UniqueName: \"kubernetes.io/projected/8232d058-70ea-4e6d-ae51-e5ba4303dd93-kube-api-access-l64xp\") pod \"network-check-target-g4r9z\" (UID: \"8232d058-70ea-4e6d-ae51-e5ba4303dd93\") " pod="openshift-network-diagnostics/network-check-target-g4r9z" Jul 16 17:14:21.776479 ip-10-0-132-130 kubenswrapper[2578]: E0716 17:14:21.776425 2578 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 16 17:14:21.776479 ip-10-0-132-130 kubenswrapper[2578]: E0716 17:14:21.776450 2578 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 16 17:14:21.776479 ip-10-0-132-130 kubenswrapper[2578]: E0716 17:14:21.776464 2578 projected.go:196] Error preparing data for projected volume kube-api-access-l64xp for pod openshift-network-diagnostics/network-check-target-g4r9z: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 16 17:14:21.776669 ip-10-0-132-130 kubenswrapper[2578]: E0716 17:14:21.776525 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/8232d058-70ea-4e6d-ae51-e5ba4303dd93-kube-api-access-l64xp podName:8232d058-70ea-4e6d-ae51-e5ba4303dd93 nodeName:}" failed. No retries permitted until 2026-07-16 17:14:25.776506915 +0000 UTC m=+10.262350797 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-l64xp" (UniqueName: "kubernetes.io/projected/8232d058-70ea-4e6d-ae51-e5ba4303dd93-kube-api-access-l64xp") pod "network-check-target-g4r9z" (UID: "8232d058-70ea-4e6d-ae51-e5ba4303dd93") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 16 17:14:22.209947 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:14:22.209916 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-rg5vf" Jul 16 17:14:22.210367 ip-10-0-132-130 kubenswrapper[2578]: E0716 17:14:22.210046 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-rg5vf" podUID="4e00566f-55f0-43ec-9d1a-a883e839823e" Jul 16 17:14:23.209593 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:14:23.209554 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-kvr2r" Jul 16 17:14:23.209787 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:14:23.209624 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-g4r9z" Jul 16 17:14:23.209864 ip-10-0-132-130 kubenswrapper[2578]: E0716 17:14:23.209794 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-kvr2r" podUID="f90822ad-ab14-4108-8294-2e76c35a8aa5" Jul 16 17:14:23.209977 ip-10-0-132-130 kubenswrapper[2578]: E0716 17:14:23.209912 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-g4r9z" podUID="8232d058-70ea-4e6d-ae51-e5ba4303dd93" Jul 16 17:14:24.210589 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:14:24.210563 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-rg5vf" Jul 16 17:14:24.211084 ip-10-0-132-130 kubenswrapper[2578]: E0716 17:14:24.210656 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-rg5vf" podUID="4e00566f-55f0-43ec-9d1a-a883e839823e" Jul 16 17:14:25.209717 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:14:25.209237 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-kvr2r" Jul 16 17:14:25.209717 ip-10-0-132-130 kubenswrapper[2578]: E0716 17:14:25.209374 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-kvr2r" podUID="f90822ad-ab14-4108-8294-2e76c35a8aa5" Jul 16 17:14:25.209717 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:14:25.209490 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-g4r9z" Jul 16 17:14:25.209717 ip-10-0-132-130 kubenswrapper[2578]: E0716 17:14:25.209610 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-g4r9z" podUID="8232d058-70ea-4e6d-ae51-e5ba4303dd93" Jul 16 17:14:25.608351 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:14:25.608314 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f90822ad-ab14-4108-8294-2e76c35a8aa5-metrics-certs\") pod \"network-metrics-daemon-kvr2r\" (UID: \"f90822ad-ab14-4108-8294-2e76c35a8aa5\") " pod="openshift-multus/network-metrics-daemon-kvr2r" Jul 16 17:14:25.608828 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:14:25.608382 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/4e00566f-55f0-43ec-9d1a-a883e839823e-original-pull-secret\") pod \"global-pull-secret-syncer-rg5vf\" (UID: \"4e00566f-55f0-43ec-9d1a-a883e839823e\") " pod="kube-system/global-pull-secret-syncer-rg5vf" Jul 16 17:14:25.608828 ip-10-0-132-130 kubenswrapper[2578]: E0716 17:14:25.608455 2578 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 16 17:14:25.608828 ip-10-0-132-130 kubenswrapper[2578]: E0716 17:14:25.608500 2578 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 16 17:14:25.608828 ip-10-0-132-130 kubenswrapper[2578]: E0716 17:14:25.608528 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f90822ad-ab14-4108-8294-2e76c35a8aa5-metrics-certs podName:f90822ad-ab14-4108-8294-2e76c35a8aa5 nodeName:}" failed. No retries permitted until 2026-07-16 17:14:33.608506943 +0000 UTC m=+18.094350826 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/f90822ad-ab14-4108-8294-2e76c35a8aa5-metrics-certs") pod "network-metrics-daemon-kvr2r" (UID: "f90822ad-ab14-4108-8294-2e76c35a8aa5") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 16 17:14:25.608828 ip-10-0-132-130 kubenswrapper[2578]: E0716 17:14:25.608548 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4e00566f-55f0-43ec-9d1a-a883e839823e-original-pull-secret podName:4e00566f-55f0-43ec-9d1a-a883e839823e nodeName:}" failed. No retries permitted until 2026-07-16 17:14:33.608538657 +0000 UTC m=+18.094382529 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/4e00566f-55f0-43ec-9d1a-a883e839823e-original-pull-secret") pod "global-pull-secret-syncer-rg5vf" (UID: "4e00566f-55f0-43ec-9d1a-a883e839823e") : object "kube-system"/"original-pull-secret" not registered Jul 16 17:14:25.810176 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:14:25.810142 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-l64xp\" (UniqueName: \"kubernetes.io/projected/8232d058-70ea-4e6d-ae51-e5ba4303dd93-kube-api-access-l64xp\") pod \"network-check-target-g4r9z\" (UID: \"8232d058-70ea-4e6d-ae51-e5ba4303dd93\") " pod="openshift-network-diagnostics/network-check-target-g4r9z" Jul 16 17:14:25.810349 ip-10-0-132-130 kubenswrapper[2578]: E0716 17:14:25.810289 2578 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 16 17:14:25.810349 ip-10-0-132-130 kubenswrapper[2578]: E0716 17:14:25.810304 2578 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 16 17:14:25.810349 ip-10-0-132-130 kubenswrapper[2578]: E0716 17:14:25.810316 2578 projected.go:196] Error preparing data for projected volume kube-api-access-l64xp for pod openshift-network-diagnostics/network-check-target-g4r9z: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 16 17:14:25.810546 ip-10-0-132-130 kubenswrapper[2578]: E0716 17:14:25.810367 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/8232d058-70ea-4e6d-ae51-e5ba4303dd93-kube-api-access-l64xp podName:8232d058-70ea-4e6d-ae51-e5ba4303dd93 nodeName:}" failed. No retries permitted until 2026-07-16 17:14:33.810350861 +0000 UTC m=+18.296194746 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-l64xp" (UniqueName: "kubernetes.io/projected/8232d058-70ea-4e6d-ae51-e5ba4303dd93-kube-api-access-l64xp") pod "network-check-target-g4r9z" (UID: "8232d058-70ea-4e6d-ae51-e5ba4303dd93") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 16 17:14:26.210752 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:14:26.210265 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-rg5vf" Jul 16 17:14:26.210752 ip-10-0-132-130 kubenswrapper[2578]: E0716 17:14:26.210375 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-rg5vf" podUID="4e00566f-55f0-43ec-9d1a-a883e839823e" Jul 16 17:14:27.209798 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:14:27.209708 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-g4r9z" Jul 16 17:14:27.209798 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:14:27.209715 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-kvr2r" Jul 16 17:14:27.210294 ip-10-0-132-130 kubenswrapper[2578]: E0716 17:14:27.209846 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-g4r9z" podUID="8232d058-70ea-4e6d-ae51-e5ba4303dd93" Jul 16 17:14:27.210294 ip-10-0-132-130 kubenswrapper[2578]: E0716 17:14:27.209959 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-kvr2r" podUID="f90822ad-ab14-4108-8294-2e76c35a8aa5" Jul 16 17:14:28.209777 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:14:28.209745 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-rg5vf" Jul 16 17:14:28.209945 ip-10-0-132-130 kubenswrapper[2578]: E0716 17:14:28.209915 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-rg5vf" podUID="4e00566f-55f0-43ec-9d1a-a883e839823e" Jul 16 17:14:29.209090 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:14:29.209062 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-kvr2r" Jul 16 17:14:29.209241 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:14:29.209093 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-g4r9z" Jul 16 17:14:29.209241 ip-10-0-132-130 kubenswrapper[2578]: E0716 17:14:29.209164 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-kvr2r" podUID="f90822ad-ab14-4108-8294-2e76c35a8aa5" Jul 16 17:14:29.209338 ip-10-0-132-130 kubenswrapper[2578]: E0716 17:14:29.209311 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-g4r9z" podUID="8232d058-70ea-4e6d-ae51-e5ba4303dd93" Jul 16 17:14:30.209905 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:14:30.209871 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-rg5vf" Jul 16 17:14:30.210308 ip-10-0-132-130 kubenswrapper[2578]: E0716 17:14:30.209993 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-rg5vf" podUID="4e00566f-55f0-43ec-9d1a-a883e839823e" Jul 16 17:14:31.209068 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:14:31.209036 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-kvr2r" Jul 16 17:14:31.209262 ip-10-0-132-130 kubenswrapper[2578]: E0716 17:14:31.209163 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-kvr2r" podUID="f90822ad-ab14-4108-8294-2e76c35a8aa5" Jul 16 17:14:31.209262 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:14:31.209228 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-g4r9z" Jul 16 17:14:31.209362 ip-10-0-132-130 kubenswrapper[2578]: E0716 17:14:31.209326 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-g4r9z" podUID="8232d058-70ea-4e6d-ae51-e5ba4303dd93" Jul 16 17:14:32.209797 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:14:32.209766 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-rg5vf" Jul 16 17:14:32.210241 ip-10-0-132-130 kubenswrapper[2578]: E0716 17:14:32.209899 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-rg5vf" podUID="4e00566f-55f0-43ec-9d1a-a883e839823e" Jul 16 17:14:33.209358 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:14:33.209322 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-g4r9z" Jul 16 17:14:33.209593 ip-10-0-132-130 kubenswrapper[2578]: E0716 17:14:33.209437 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-g4r9z" podUID="8232d058-70ea-4e6d-ae51-e5ba4303dd93" Jul 16 17:14:33.209593 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:14:33.209452 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-kvr2r" Jul 16 17:14:33.209593 ip-10-0-132-130 kubenswrapper[2578]: E0716 17:14:33.209575 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-kvr2r" podUID="f90822ad-ab14-4108-8294-2e76c35a8aa5" Jul 16 17:14:33.672244 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:14:33.672207 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/4e00566f-55f0-43ec-9d1a-a883e839823e-original-pull-secret\") pod \"global-pull-secret-syncer-rg5vf\" (UID: \"4e00566f-55f0-43ec-9d1a-a883e839823e\") " pod="kube-system/global-pull-secret-syncer-rg5vf" Jul 16 17:14:33.672720 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:14:33.672285 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f90822ad-ab14-4108-8294-2e76c35a8aa5-metrics-certs\") pod \"network-metrics-daemon-kvr2r\" (UID: \"f90822ad-ab14-4108-8294-2e76c35a8aa5\") " pod="openshift-multus/network-metrics-daemon-kvr2r" Jul 16 17:14:33.672720 ip-10-0-132-130 kubenswrapper[2578]: E0716 17:14:33.672353 2578 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 16 17:14:33.672720 ip-10-0-132-130 kubenswrapper[2578]: E0716 17:14:33.672357 2578 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 16 17:14:33.672720 ip-10-0-132-130 kubenswrapper[2578]: E0716 17:14:33.672433 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4e00566f-55f0-43ec-9d1a-a883e839823e-original-pull-secret podName:4e00566f-55f0-43ec-9d1a-a883e839823e nodeName:}" failed. No retries permitted until 2026-07-16 17:14:49.672411912 +0000 UTC m=+34.158255801 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/4e00566f-55f0-43ec-9d1a-a883e839823e-original-pull-secret") pod "global-pull-secret-syncer-rg5vf" (UID: "4e00566f-55f0-43ec-9d1a-a883e839823e") : object "kube-system"/"original-pull-secret" not registered Jul 16 17:14:33.672720 ip-10-0-132-130 kubenswrapper[2578]: E0716 17:14:33.672491 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f90822ad-ab14-4108-8294-2e76c35a8aa5-metrics-certs podName:f90822ad-ab14-4108-8294-2e76c35a8aa5 nodeName:}" failed. No retries permitted until 2026-07-16 17:14:49.672470615 +0000 UTC m=+34.158314481 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/f90822ad-ab14-4108-8294-2e76c35a8aa5-metrics-certs") pod "network-metrics-daemon-kvr2r" (UID: "f90822ad-ab14-4108-8294-2e76c35a8aa5") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 16 17:14:33.874501 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:14:33.874471 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-l64xp\" (UniqueName: \"kubernetes.io/projected/8232d058-70ea-4e6d-ae51-e5ba4303dd93-kube-api-access-l64xp\") pod \"network-check-target-g4r9z\" (UID: \"8232d058-70ea-4e6d-ae51-e5ba4303dd93\") " pod="openshift-network-diagnostics/network-check-target-g4r9z" Jul 16 17:14:33.874685 ip-10-0-132-130 kubenswrapper[2578]: E0716 17:14:33.874603 2578 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 16 17:14:33.874685 ip-10-0-132-130 kubenswrapper[2578]: E0716 17:14:33.874624 2578 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 16 17:14:33.874685 ip-10-0-132-130 kubenswrapper[2578]: E0716 17:14:33.874636 2578 projected.go:196] Error preparing data for projected volume kube-api-access-l64xp for pod openshift-network-diagnostics/network-check-target-g4r9z: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 16 17:14:33.874881 ip-10-0-132-130 kubenswrapper[2578]: E0716 17:14:33.874710 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/8232d058-70ea-4e6d-ae51-e5ba4303dd93-kube-api-access-l64xp podName:8232d058-70ea-4e6d-ae51-e5ba4303dd93 nodeName:}" failed. No retries permitted until 2026-07-16 17:14:49.874677787 +0000 UTC m=+34.360521670 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-l64xp" (UniqueName: "kubernetes.io/projected/8232d058-70ea-4e6d-ae51-e5ba4303dd93-kube-api-access-l64xp") pod "network-check-target-g4r9z" (UID: "8232d058-70ea-4e6d-ae51-e5ba4303dd93") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 16 17:14:34.210051 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:14:34.210015 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-rg5vf" Jul 16 17:14:34.210223 ip-10-0-132-130 kubenswrapper[2578]: E0716 17:14:34.210146 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-rg5vf" podUID="4e00566f-55f0-43ec-9d1a-a883e839823e" Jul 16 17:14:35.209685 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:14:35.209652 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-kvr2r" Jul 16 17:14:35.209685 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:14:35.209686 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-g4r9z" Jul 16 17:14:35.210202 ip-10-0-132-130 kubenswrapper[2578]: E0716 17:14:35.209779 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-kvr2r" podUID="f90822ad-ab14-4108-8294-2e76c35a8aa5" Jul 16 17:14:35.210202 ip-10-0-132-130 kubenswrapper[2578]: E0716 17:14:35.209893 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-g4r9z" podUID="8232d058-70ea-4e6d-ae51-e5ba4303dd93" Jul 16 17:14:36.210250 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:14:36.210220 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-rg5vf" Jul 16 17:14:36.210631 ip-10-0-132-130 kubenswrapper[2578]: E0716 17:14:36.210336 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-rg5vf" podUID="4e00566f-55f0-43ec-9d1a-a883e839823e" Jul 16 17:14:37.209943 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:14:37.209910 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-kvr2r" Jul 16 17:14:37.210119 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:14:37.209919 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-g4r9z" Jul 16 17:14:37.210119 ip-10-0-132-130 kubenswrapper[2578]: E0716 17:14:37.210009 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-kvr2r" podUID="f90822ad-ab14-4108-8294-2e76c35a8aa5" Jul 16 17:14:37.210119 ip-10-0-132-130 kubenswrapper[2578]: E0716 17:14:37.210087 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-g4r9z" podUID="8232d058-70ea-4e6d-ae51-e5ba4303dd93" Jul 16 17:14:38.209596 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:14:38.209252 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-rg5vf" Jul 16 17:14:38.210194 ip-10-0-132-130 kubenswrapper[2578]: E0716 17:14:38.209675 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-rg5vf" podUID="4e00566f-55f0-43ec-9d1a-a883e839823e" Jul 16 17:14:38.296297 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:14:38.296268 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-lfsvs" event={"ID":"dfca47c1-7bfb-4111-a283-7b86affa45a7","Type":"ContainerStarted","Data":"81dfb0d46f3f4c620e721eee0d95b3657822d2681aae5c9c092fb1047c4a27f6"} Jul 16 17:14:38.297749 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:14:38.297724 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-kq8j9" event={"ID":"a8425d74-5613-43e9-b6ff-3884b419dbf6","Type":"ContainerStarted","Data":"1e7fa3110876e5fed864b29abac050938e8af74ebace896a4a0ba169e6779da4"} Jul 16 17:14:38.299051 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:14:38.298998 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-zw2xf" event={"ID":"ff79255c-4f85-4600-a7d2-54534f299e29","Type":"ContainerStarted","Data":"ae2b98240cafd10943d211b5eee847e2c435c2b41124f8288c3d42bb58c2bf4f"} Jul 16 17:14:38.300706 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:14:38.300664 2578 generic.go:356] "Generic (PLEG): container finished" podID="f724a0822cdfaacbec5071cc8e93c31c" containerID="184e3dbf6b8adfbb6ec6e6b120242543a721fe3227b6a2d546a638bb2801e07d" exitCode=0 Jul 16 17:14:38.300810 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:14:38.300750 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-130.ec2.internal" event={"ID":"f724a0822cdfaacbec5071cc8e93c31c","Type":"ContainerDied","Data":"184e3dbf6b8adfbb6ec6e6b120242543a721fe3227b6a2d546a638bb2801e07d"} Jul 16 17:14:38.302268 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:14:38.302244 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-cn7hz" event={"ID":"01e55058-0022-4c74-966c-50605ba63d4e","Type":"ContainerStarted","Data":"d02df8a2290919a502db68a0dfdee990dfa30fb0541e14b9a8510a8a026aa92b"} Jul 16 17:14:38.303885 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:14:38.303865 2578 generic.go:356] "Generic (PLEG): container finished" podID="fa2959ee-fcc6-49eb-9205-65cf1a08779f" containerID="34a71cc1c02646ec1ecd89061cda585754e42eba93153bbbf6f6388b450d496c" exitCode=0 Jul 16 17:14:38.303989 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:14:38.303932 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-6gzcp" event={"ID":"fa2959ee-fcc6-49eb-9205-65cf1a08779f","Type":"ContainerDied","Data":"34a71cc1c02646ec1ecd89061cda585754e42eba93153bbbf6f6388b450d496c"} Jul 16 17:14:38.305765 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:14:38.305736 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-qj6b6" event={"ID":"31abc787-9b8d-4943-a302-b6db28480e19","Type":"ContainerStarted","Data":"a6dfb9cb6c4703726fa74387a0fb26f178a5f1821ee2908a7bcba04b4aae5235"} Jul 16 17:14:38.307366 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:14:38.307340 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-78ctd" event={"ID":"7db99718-7ae4-48bd-9c33-900ea9a925ad","Type":"ContainerStarted","Data":"7e8f17181cb5a5e25706b7c82e7a96ddd0fcf6f69f66e388aad95f5e53282f6e"} Jul 16 17:14:38.308778 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:14:38.308754 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-132-130.ec2.internal" event={"ID":"7876ab904043bc7f87e7c28015b01ebc","Type":"ContainerStarted","Data":"1be23aa913c6b1710b14697e27ea8aedc35f4c1d346483a9518176f0b27c49d0"} Jul 16 17:14:38.312128 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:14:38.312110 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-h7t4d" event={"ID":"9a29f379-46c1-4211-81b2-7f3d82d82d01","Type":"ContainerStarted","Data":"526fffca8a57317d4682bf73c14f082a573f35ad05758ffefe640d99d6052e55"} Jul 16 17:14:38.312205 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:14:38.312158 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-h7t4d" event={"ID":"9a29f379-46c1-4211-81b2-7f3d82d82d01","Type":"ContainerStarted","Data":"eaf585e5de7ce802b05902c231dba9fad4a1ecafbf9ecf17ad1ce0a84ec522b5"} Jul 16 17:14:38.312205 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:14:38.312186 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-h7t4d" event={"ID":"9a29f379-46c1-4211-81b2-7f3d82d82d01","Type":"ContainerStarted","Data":"5e2d941c227dd3093ee8b215dc91fb2f420e6ec3068c6d472c07fd8f658f5866"} Jul 16 17:14:38.312276 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:14:38.312214 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-h7t4d" event={"ID":"9a29f379-46c1-4211-81b2-7f3d82d82d01","Type":"ContainerStarted","Data":"cf0fdd8bad39030cf84c5d4dbf46b8dab8cc9b3268aa6bc1ad6a7d7dc1db2865"} Jul 16 17:14:38.312276 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:14:38.312240 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-h7t4d" event={"ID":"9a29f379-46c1-4211-81b2-7f3d82d82d01","Type":"ContainerStarted","Data":"af9fa42b514612d818901b6109d5f508402217ca0e75b7d08031c8b5e4941944"} Jul 16 17:14:38.312276 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:14:38.312267 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-h7t4d" event={"ID":"9a29f379-46c1-4211-81b2-7f3d82d82d01","Type":"ContainerStarted","Data":"b823ca3325b442ebabd9cba0344e738011e910fc3516246b26abe94bab63c81b"} Jul 16 17:14:38.366199 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:14:38.366154 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/konnectivity-agent-kq8j9" podStartSLOduration=3.091270456 podStartE2EDuration="22.366138126s" totalImagesPullingTime="19.27486767s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 17:14:16 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 17:14:38.362862717 +0000 UTC m=+22.848706606" watchObservedRunningTime="2026-07-16 17:14:38.366138126 +0000 UTC m=+22.851982014" Jul 16 17:14:38.366340 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:14:38.366316 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/node-ca-lfsvs" podStartSLOduration=3.102720006 podStartE2EDuration="22.366310408s" totalImagesPullingTime="19.263590402s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-16 17:14:16 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 17:14:38.336519267 +0000 UTC m=+22.822363156" watchObservedRunningTime="2026-07-16 17:14:38.366310408 +0000 UTC m=+22.852154298" Jul 16 17:14:38.423184 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:14:38.422811 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-78ctd" podStartSLOduration=3.141975928 podStartE2EDuration="22.422796398s" totalImagesPullingTime="19.28082047s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-16 17:14:16 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 17:14:38.394637605 +0000 UTC m=+22.880481493" watchObservedRunningTime="2026-07-16 17:14:38.422796398 +0000 UTC m=+22.908640328" Jul 16 17:14:38.477820 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:14:38.477795 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-kq8j9_a8425d74-5613-43e9-b6ff-3884b419dbf6/konnectivity-agent/0.log" limit=1024 Jul 16 17:14:38.480957 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:14:38.480924 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-qj6b6" podStartSLOduration=3.163549932 podStartE2EDuration="22.480911032s" totalImagesPullingTime="19.3173611s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-16 17:14:16 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 17:14:38.448200261 +0000 UTC m=+22.934044150" watchObservedRunningTime="2026-07-16 17:14:38.480911032 +0000 UTC m=+22.966754898" Jul 16 17:14:38.499421 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:14:38.499387 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-132-130.ec2.internal" podStartSLOduration=21.499374178 podStartE2EDuration="21.499374178s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-16 17:14:17 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 17:14:38.499256678 +0000 UTC m=+22.985100567" watchObservedRunningTime="2026-07-16 17:14:38.499374178 +0000 UTC m=+22.985218066" Jul 16 17:14:38.556996 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:14:38.556973 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-kq8j9_a8425d74-5613-43e9-b6ff-3884b419dbf6/konnectivity-agent/0.log" limit=1024 Jul 16 17:14:38.709714 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:14:38.709674 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-kq8j9_a8425d74-5613-43e9-b6ff-3884b419dbf6/konnectivity-agent/0.log" limit=1024 Jul 16 17:14:38.796199 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:14:38.796170 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-kq8j9_a8425d74-5613-43e9-b6ff-3884b419dbf6/konnectivity-agent/0.log" limit=1024 Jul 16 17:14:39.037366 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:14:39.037297 2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-kq8j9" Jul 16 17:14:39.037919 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:14:39.037901 2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-kq8j9" Jul 16 17:14:39.060235 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:14:39.060194 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/node-resolver-zw2xf" podStartSLOduration=3.772859563 podStartE2EDuration="23.060175676s" totalImagesPullingTime="19.287316113s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-16 17:14:16 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 17:14:38.520355556 +0000 UTC m=+23.006199443" watchObservedRunningTime="2026-07-16 17:14:39.060175676 +0000 UTC m=+23.546019564" Jul 16 17:14:39.111628 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:14:39.111601 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-kq8j9_a8425d74-5613-43e9-b6ff-3884b419dbf6/konnectivity-agent/0.log" limit=1024 Jul 16 17:14:39.209600 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:14:39.209572 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-kvr2r" Jul 16 17:14:39.210113 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:14:39.209572 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-g4r9z" Jul 16 17:14:39.210113 ip-10-0-132-130 kubenswrapper[2578]: E0716 17:14:39.209727 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-kvr2r" podUID="f90822ad-ab14-4108-8294-2e76c35a8aa5" Jul 16 17:14:39.210113 ip-10-0-132-130 kubenswrapper[2578]: E0716 17:14:39.209797 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-g4r9z" podUID="8232d058-70ea-4e6d-ae51-e5ba4303dd93" Jul 16 17:14:39.213467 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:14:39.213437 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-kq8j9_a8425d74-5613-43e9-b6ff-3884b419dbf6/konnectivity-agent/0.log" limit=1024 Jul 16 17:14:39.315592 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:14:39.315530 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-9mk4h" event={"ID":"6659c756-0d21-42ec-aa63-157ef0294e02","Type":"ContainerStarted","Data":"4e364defc8f3c0b9ffd0b27fa8b202190a17ad880441d34e064cfc7411bfba44"} Jul 16 17:14:39.317627 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:14:39.317588 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-130.ec2.internal" event={"ID":"f724a0822cdfaacbec5071cc8e93c31c","Type":"ContainerStarted","Data":"f11b1b66b2fc297b20220ddb5956c371862d3776ac290ac864946447160a476f"} Jul 16 17:14:39.317808 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:14:39.317782 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-kq8j9" Jul 16 17:14:39.320655 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:14:39.319004 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-kq8j9" Jul 16 17:14:39.357720 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:14:39.357665 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-9mk4h" podStartSLOduration=4.095856583 podStartE2EDuration="23.357651671s" totalImagesPullingTime="19.261795088s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-16 17:14:16 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 17:14:39.336309912 +0000 UTC m=+23.822153799" watchObservedRunningTime="2026-07-16 17:14:39.357651671 +0000 UTC m=+23.843495619" Jul 16 17:14:39.412417 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:14:39.412389 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-kq8j9_a8425d74-5613-43e9-b6ff-3884b419dbf6/konnectivity-agent/0.log" limit=1024 Jul 16 17:14:39.512048 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:14:39.512011 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-kq8j9_a8425d74-5613-43e9-b6ff-3884b419dbf6/konnectivity-agent/0.log" limit=1024 Jul 16 17:14:39.619999 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:14:39.619901 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-kq8j9_a8425d74-5613-43e9-b6ff-3884b419dbf6/konnectivity-agent/0.log" limit=1024 Jul 16 17:14:40.213002 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:14:40.212973 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-rg5vf" Jul 16 17:14:40.213387 ip-10-0-132-130 kubenswrapper[2578]: E0716 17:14:40.213074 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-rg5vf" podUID="4e00566f-55f0-43ec-9d1a-a883e839823e" Jul 16 17:14:40.388118 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:14:40.388093 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-kq8j9_a8425d74-5613-43e9-b6ff-3884b419dbf6/konnectivity-agent/0.log" limit=1024 Jul 16 17:14:40.707720 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:14:40.707671 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-kq8j9_a8425d74-5613-43e9-b6ff-3884b419dbf6/konnectivity-agent/0.log" limit=1024 Jul 16 17:14:40.820950 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:14:40.820915 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-kq8j9_a8425d74-5613-43e9-b6ff-3884b419dbf6/konnectivity-agent/0.log" limit=1024 Jul 16 17:14:41.209602 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:14:41.209526 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-g4r9z" Jul 16 17:14:41.209773 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:14:41.209526 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-kvr2r" Jul 16 17:14:41.209773 ip-10-0-132-130 kubenswrapper[2578]: E0716 17:14:41.209655 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-g4r9z" podUID="8232d058-70ea-4e6d-ae51-e5ba4303dd93" Jul 16 17:14:41.209773 ip-10-0-132-130 kubenswrapper[2578]: E0716 17:14:41.209724 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-kvr2r" podUID="f90822ad-ab14-4108-8294-2e76c35a8aa5" Jul 16 17:14:41.326544 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:14:41.326509 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-h7t4d" event={"ID":"9a29f379-46c1-4211-81b2-7f3d82d82d01","Type":"ContainerStarted","Data":"edc64e681255d89a8e68c8f1a7a69694947b686c82f8f424a984034e3a92887f"} Jul 16 17:14:41.725526 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:14:41.725479 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-kq8j9_a8425d74-5613-43e9-b6ff-3884b419dbf6/konnectivity-agent/0.log" limit=1024 Jul 16 17:14:41.873416 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:14:41.873369 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-kq8j9_a8425d74-5613-43e9-b6ff-3884b419dbf6/konnectivity-agent/0.log" limit=1024 Jul 16 17:14:42.116968 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:14:42.116930 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-kq8j9_a8425d74-5613-43e9-b6ff-3884b419dbf6/konnectivity-agent/0.log" limit=1024 Jul 16 17:14:42.198353 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:14:42.198315 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-kq8j9_a8425d74-5613-43e9-b6ff-3884b419dbf6/konnectivity-agent/0.log" limit=1024 Jul 16 17:14:42.209214 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:14:42.209189 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-rg5vf" Jul 16 17:14:42.209342 ip-10-0-132-130 kubenswrapper[2578]: E0716 17:14:42.209310 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-rg5vf" podUID="4e00566f-55f0-43ec-9d1a-a883e839823e" Jul 16 17:14:42.790447 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:14:42.790231 2578 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock" Jul 16 17:14:42.994825 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:14:42.994737 2578 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-07-16T17:14:42.790442059Z","UUID":"cbd10182-acb3-493d-ae17-d3c36fb8d391","Handler":null,"Name":"","Endpoint":""} Jul 16 17:14:42.996119 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:14:42.996101 2578 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0 Jul 16 17:14:42.996224 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:14:42.996133 2578 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock Jul 16 17:14:43.209959 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:14:43.209929 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-kvr2r" Jul 16 17:14:43.209959 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:14:43.209947 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-g4r9z" Jul 16 17:14:43.210163 ip-10-0-132-130 kubenswrapper[2578]: E0716 17:14:43.210053 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-kvr2r" podUID="f90822ad-ab14-4108-8294-2e76c35a8aa5" Jul 16 17:14:43.210211 ip-10-0-132-130 kubenswrapper[2578]: E0716 17:14:43.210178 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-g4r9z" podUID="8232d058-70ea-4e6d-ae51-e5ba4303dd93" Jul 16 17:14:43.334477 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:14:43.334443 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-h7t4d" event={"ID":"9a29f379-46c1-4211-81b2-7f3d82d82d01","Type":"ContainerStarted","Data":"d01a572c06bac49ebc983e4ddb2c7d5cd0fd8db17c3bd7defaf61b0094784712"} Jul 16 17:14:43.334748 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:14:43.334724 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-h7t4d" Jul 16 17:14:43.334857 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:14:43.334793 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-h7t4d" Jul 16 17:14:43.336342 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:14:43.336321 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-cn7hz" event={"ID":"01e55058-0022-4c74-966c-50605ba63d4e","Type":"ContainerStarted","Data":"d248a83c4f5ca62045139958f5f1b743c830c76c93ccb4f53cd6aa33ce58fe4c"} Jul 16 17:14:43.338037 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:14:43.338013 2578 generic.go:356] "Generic (PLEG): container finished" podID="fa2959ee-fcc6-49eb-9205-65cf1a08779f" containerID="a9d563fcd49a0fcfd97df64502ba957feebc92c8951805c6e31a1eed88b7aedd" exitCode=0 Jul 16 17:14:43.338116 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:14:43.338049 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-6gzcp" event={"ID":"fa2959ee-fcc6-49eb-9205-65cf1a08779f","Type":"ContainerDied","Data":"a9d563fcd49a0fcfd97df64502ba957feebc92c8951805c6e31a1eed88b7aedd"} Jul 16 17:14:43.349983 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:14:43.349962 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-h7t4d" Jul 16 17:14:43.368157 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:14:43.368103 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-130.ec2.internal" podStartSLOduration=26.368090942 podStartE2EDuration="26.368090942s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-16 17:14:17 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 17:14:39.374447262 +0000 UTC m=+23.860291151" watchObservedRunningTime="2026-07-16 17:14:43.368090942 +0000 UTC m=+27.853934831" Jul 16 17:14:43.368537 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:14:43.368509 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-h7t4d" podStartSLOduration=8.045616335 podStartE2EDuration="27.368502508s" totalImagesPullingTime="19.322886173s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-16 17:14:16 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 17:14:43.367274113 +0000 UTC m=+27.853118002" watchObservedRunningTime="2026-07-16 17:14:43.368502508 +0000 UTC m=+27.854346397" Jul 16 17:14:44.209910 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:14:44.209841 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-rg5vf" Jul 16 17:14:44.210415 ip-10-0-132-130 kubenswrapper[2578]: E0716 17:14:44.209939 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-rg5vf" podUID="4e00566f-55f0-43ec-9d1a-a883e839823e" Jul 16 17:14:44.344976 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:14:44.344750 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-cn7hz" event={"ID":"01e55058-0022-4c74-966c-50605ba63d4e","Type":"ContainerStarted","Data":"e5b976aac254194ab4bcce8a7774ce552264d176e345c89a6e15658b6c4e1ef8"} Jul 16 17:14:44.345215 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:14:44.345014 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-h7t4d" Jul 16 17:14:44.358945 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:14:44.358923 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-h7t4d" Jul 16 17:14:44.367667 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:14:44.367610 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-cn7hz" podStartSLOduration=3.248958229 podStartE2EDuration="28.36759786s" totalImagesPullingTime="25.118639631s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-16 17:14:16 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 17:14:44.36726962 +0000 UTC m=+28.853113509" watchObservedRunningTime="2026-07-16 17:14:44.36759786 +0000 UTC m=+28.853441748" Jul 16 17:14:45.009287 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:14:45.009218 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-rg5vf"] Jul 16 17:14:45.009441 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:14:45.009302 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-rg5vf" Jul 16 17:14:45.009441 ip-10-0-132-130 kubenswrapper[2578]: E0716 17:14:45.009403 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-rg5vf" podUID="4e00566f-55f0-43ec-9d1a-a883e839823e" Jul 16 17:14:45.012549 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:14:45.012525 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-kvr2r"] Jul 16 17:14:45.012663 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:14:45.012647 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-kvr2r" Jul 16 17:14:45.012802 ip-10-0-132-130 kubenswrapper[2578]: E0716 17:14:45.012782 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-kvr2r" podUID="f90822ad-ab14-4108-8294-2e76c35a8aa5" Jul 16 17:14:45.013094 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:14:45.013073 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-g4r9z"] Jul 16 17:14:45.013182 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:14:45.013168 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-g4r9z" Jul 16 17:14:45.013261 ip-10-0-132-130 kubenswrapper[2578]: E0716 17:14:45.013241 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-g4r9z" podUID="8232d058-70ea-4e6d-ae51-e5ba4303dd93" Jul 16 17:14:45.348785 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:14:45.348756 2578 generic.go:356] "Generic (PLEG): container finished" podID="fa2959ee-fcc6-49eb-9205-65cf1a08779f" containerID="39bf3362b8c100f6902b61de8c9a9c170652ee8f1997c9bcedd68255a13398d1" exitCode=0 Jul 16 17:14:45.349143 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:14:45.348839 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-6gzcp" event={"ID":"fa2959ee-fcc6-49eb-9205-65cf1a08779f","Type":"ContainerDied","Data":"39bf3362b8c100f6902b61de8c9a9c170652ee8f1997c9bcedd68255a13398d1"} Jul 16 17:14:47.209999 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:14:47.209909 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-g4r9z" Jul 16 17:14:47.209999 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:14:47.209938 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-kvr2r" Jul 16 17:14:47.209999 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:14:47.209953 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-rg5vf" Jul 16 17:14:47.210418 ip-10-0-132-130 kubenswrapper[2578]: E0716 17:14:47.210030 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-g4r9z" podUID="8232d058-70ea-4e6d-ae51-e5ba4303dd93" Jul 16 17:14:47.210418 ip-10-0-132-130 kubenswrapper[2578]: E0716 17:14:47.210095 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-rg5vf" podUID="4e00566f-55f0-43ec-9d1a-a883e839823e" Jul 16 17:14:47.210418 ip-10-0-132-130 kubenswrapper[2578]: E0716 17:14:47.210177 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-kvr2r" podUID="f90822ad-ab14-4108-8294-2e76c35a8aa5" Jul 16 17:14:47.355325 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:14:47.355291 2578 generic.go:356] "Generic (PLEG): container finished" podID="fa2959ee-fcc6-49eb-9205-65cf1a08779f" containerID="3e78d9adb52936f07fb66c4faffe39db7a820b7e425aa4c4b44a2e2c7bb2b80c" exitCode=0 Jul 16 17:14:47.355498 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:14:47.355349 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-6gzcp" event={"ID":"fa2959ee-fcc6-49eb-9205-65cf1a08779f","Type":"ContainerDied","Data":"3e78d9adb52936f07fb66c4faffe39db7a820b7e425aa4c4b44a2e2c7bb2b80c"} Jul 16 17:14:49.169799 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:14:49.169758 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-kq8j9_a8425d74-5613-43e9-b6ff-3884b419dbf6/konnectivity-agent/0.log" limit=1024 Jul 16 17:14:49.209753 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:14:49.209726 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-g4r9z" Jul 16 17:14:49.209753 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:14:49.209741 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-kvr2r" Jul 16 17:14:49.209939 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:14:49.209844 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-rg5vf" Jul 16 17:14:49.209939 ip-10-0-132-130 kubenswrapper[2578]: E0716 17:14:49.209849 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-kvr2r" podUID="f90822ad-ab14-4108-8294-2e76c35a8aa5" Jul 16 17:14:49.209939 ip-10-0-132-130 kubenswrapper[2578]: E0716 17:14:49.209910 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-g4r9z" podUID="8232d058-70ea-4e6d-ae51-e5ba4303dd93" Jul 16 17:14:49.210059 ip-10-0-132-130 kubenswrapper[2578]: E0716 17:14:49.209981 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-rg5vf" podUID="4e00566f-55f0-43ec-9d1a-a883e839823e" Jul 16 17:14:49.271973 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:14:49.271936 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-kq8j9_a8425d74-5613-43e9-b6ff-3884b419dbf6/konnectivity-agent/0.log" limit=1024 Jul 16 17:14:49.697843 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:14:49.697812 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/4e00566f-55f0-43ec-9d1a-a883e839823e-original-pull-secret\") pod \"global-pull-secret-syncer-rg5vf\" (UID: \"4e00566f-55f0-43ec-9d1a-a883e839823e\") " pod="kube-system/global-pull-secret-syncer-rg5vf" Jul 16 17:14:49.698017 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:14:49.697873 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f90822ad-ab14-4108-8294-2e76c35a8aa5-metrics-certs\") pod \"network-metrics-daemon-kvr2r\" (UID: \"f90822ad-ab14-4108-8294-2e76c35a8aa5\") " pod="openshift-multus/network-metrics-daemon-kvr2r" Jul 16 17:14:49.698017 ip-10-0-132-130 kubenswrapper[2578]: E0716 17:14:49.697963 2578 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 16 17:14:49.698144 ip-10-0-132-130 kubenswrapper[2578]: E0716 17:14:49.698027 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4e00566f-55f0-43ec-9d1a-a883e839823e-original-pull-secret podName:4e00566f-55f0-43ec-9d1a-a883e839823e nodeName:}" failed. No retries permitted until 2026-07-16 17:15:21.698009557 +0000 UTC m=+66.183853437 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/4e00566f-55f0-43ec-9d1a-a883e839823e-original-pull-secret") pod "global-pull-secret-syncer-rg5vf" (UID: "4e00566f-55f0-43ec-9d1a-a883e839823e") : object "kube-system"/"original-pull-secret" not registered Jul 16 17:14:49.698144 ip-10-0-132-130 kubenswrapper[2578]: E0716 17:14:49.697963 2578 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 16 17:14:49.698144 ip-10-0-132-130 kubenswrapper[2578]: E0716 17:14:49.698121 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f90822ad-ab14-4108-8294-2e76c35a8aa5-metrics-certs podName:f90822ad-ab14-4108-8294-2e76c35a8aa5 nodeName:}" failed. No retries permitted until 2026-07-16 17:15:21.698103588 +0000 UTC m=+66.183947455 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/f90822ad-ab14-4108-8294-2e76c35a8aa5-metrics-certs") pod "network-metrics-daemon-kvr2r" (UID: "f90822ad-ab14-4108-8294-2e76c35a8aa5") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 16 17:14:49.899958 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:14:49.899916 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-l64xp\" (UniqueName: \"kubernetes.io/projected/8232d058-70ea-4e6d-ae51-e5ba4303dd93-kube-api-access-l64xp\") pod \"network-check-target-g4r9z\" (UID: \"8232d058-70ea-4e6d-ae51-e5ba4303dd93\") " pod="openshift-network-diagnostics/network-check-target-g4r9z" Jul 16 17:14:49.900146 ip-10-0-132-130 kubenswrapper[2578]: E0716 17:14:49.900056 2578 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 16 17:14:49.900146 ip-10-0-132-130 kubenswrapper[2578]: E0716 17:14:49.900078 2578 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 16 17:14:49.900146 ip-10-0-132-130 kubenswrapper[2578]: E0716 17:14:49.900091 2578 projected.go:196] Error preparing data for projected volume kube-api-access-l64xp for pod openshift-network-diagnostics/network-check-target-g4r9z: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 16 17:14:49.900146 ip-10-0-132-130 kubenswrapper[2578]: E0716 17:14:49.900149 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/8232d058-70ea-4e6d-ae51-e5ba4303dd93-kube-api-access-l64xp podName:8232d058-70ea-4e6d-ae51-e5ba4303dd93 nodeName:}" failed. No retries permitted until 2026-07-16 17:15:21.900134605 +0000 UTC m=+66.385978472 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-l64xp" (UniqueName: "kubernetes.io/projected/8232d058-70ea-4e6d-ae51-e5ba4303dd93-kube-api-access-l64xp") pod "network-check-target-g4r9z" (UID: "8232d058-70ea-4e6d-ae51-e5ba4303dd93") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 16 17:14:50.273932 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:14:50.273757 2578 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-130.ec2.internal" event="NodeReady" Jul 16 17:14:50.274314 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:14:50.274054 2578 kubelet_node_status.go:489] "Fast updating node status as it just became ready" Jul 16 17:14:50.330203 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:14:50.330179 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-vtkgg"] Jul 16 17:14:50.336056 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:14:50.336028 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-p2d6d"] Jul 16 17:14:50.336178 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:14:50.336163 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-vtkgg" Jul 16 17:14:50.338212 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:14:50.338192 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\"" Jul 16 17:14:50.338212 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:14:50.338209 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\"" Jul 16 17:14:50.338485 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:14:50.338470 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-bfmt5\"" Jul 16 17:14:50.339931 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:14:50.339909 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-p2d6d" Jul 16 17:14:50.343323 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:14:50.343301 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\"" Jul 16 17:14:50.343513 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:14:50.343425 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\"" Jul 16 17:14:50.343513 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:14:50.343450 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\"" Jul 16 17:14:50.343513 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:14:50.343466 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-gn8hz\"" Jul 16 17:14:50.347607 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:14:50.347577 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-vtkgg"] Jul 16 17:14:50.361847 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:14:50.361827 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-p2d6d"] Jul 16 17:14:50.402897 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:14:50.402874 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/f99dd101-e4fa-4c68-a502-c3a14be5c7fb-metrics-tls\") pod \"dns-default-vtkgg\" (UID: \"f99dd101-e4fa-4c68-a502-c3a14be5c7fb\") " pod="openshift-dns/dns-default-vtkgg" Jul 16 17:14:50.403036 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:14:50.402902 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/f7ea4735-cb68-4db2-a4ed-b4395fe7e9f6-cert\") pod \"ingress-canary-p2d6d\" (UID: \"f7ea4735-cb68-4db2-a4ed-b4395fe7e9f6\") " pod="openshift-ingress-canary/ingress-canary-p2d6d" Jul 16 17:14:50.403036 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:14:50.402925 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/f99dd101-e4fa-4c68-a502-c3a14be5c7fb-config-volume\") pod \"dns-default-vtkgg\" (UID: \"f99dd101-e4fa-4c68-a502-c3a14be5c7fb\") " pod="openshift-dns/dns-default-vtkgg" Jul 16 17:14:50.403036 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:14:50.402946 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4l7ff\" (UniqueName: \"kubernetes.io/projected/f7ea4735-cb68-4db2-a4ed-b4395fe7e9f6-kube-api-access-4l7ff\") pod \"ingress-canary-p2d6d\" (UID: \"f7ea4735-cb68-4db2-a4ed-b4395fe7e9f6\") " pod="openshift-ingress-canary/ingress-canary-p2d6d" Jul 16 17:14:50.403142 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:14:50.403043 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f99dd101-e4fa-4c68-a502-c3a14be5c7fb-tmp-dir\") pod \"dns-default-vtkgg\" (UID: \"f99dd101-e4fa-4c68-a502-c3a14be5c7fb\") " pod="openshift-dns/dns-default-vtkgg" Jul 16 17:14:50.403142 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:14:50.403093 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-667mr\" (UniqueName: \"kubernetes.io/projected/f99dd101-e4fa-4c68-a502-c3a14be5c7fb-kube-api-access-667mr\") pod \"dns-default-vtkgg\" (UID: \"f99dd101-e4fa-4c68-a502-c3a14be5c7fb\") " pod="openshift-dns/dns-default-vtkgg" Jul 16 17:14:50.470745 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:14:50.470704 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-kq8j9_a8425d74-5613-43e9-b6ff-3884b419dbf6/konnectivity-agent/0.log" limit=1024 Jul 16 17:14:50.503629 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:14:50.503602 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/f99dd101-e4fa-4c68-a502-c3a14be5c7fb-metrics-tls\") pod \"dns-default-vtkgg\" (UID: \"f99dd101-e4fa-4c68-a502-c3a14be5c7fb\") " pod="openshift-dns/dns-default-vtkgg" Jul 16 17:14:50.503629 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:14:50.503628 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/f7ea4735-cb68-4db2-a4ed-b4395fe7e9f6-cert\") pod \"ingress-canary-p2d6d\" (UID: \"f7ea4735-cb68-4db2-a4ed-b4395fe7e9f6\") " pod="openshift-ingress-canary/ingress-canary-p2d6d" Jul 16 17:14:50.503835 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:14:50.503655 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/f99dd101-e4fa-4c68-a502-c3a14be5c7fb-config-volume\") pod \"dns-default-vtkgg\" (UID: \"f99dd101-e4fa-4c68-a502-c3a14be5c7fb\") " pod="openshift-dns/dns-default-vtkgg" Jul 16 17:14:50.503835 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:14:50.503682 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4l7ff\" (UniqueName: \"kubernetes.io/projected/f7ea4735-cb68-4db2-a4ed-b4395fe7e9f6-kube-api-access-4l7ff\") pod \"ingress-canary-p2d6d\" (UID: \"f7ea4735-cb68-4db2-a4ed-b4395fe7e9f6\") " pod="openshift-ingress-canary/ingress-canary-p2d6d" Jul 16 17:14:50.503835 ip-10-0-132-130 kubenswrapper[2578]: E0716 17:14:50.503781 2578 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 16 17:14:50.503968 ip-10-0-132-130 kubenswrapper[2578]: E0716 17:14:50.503855 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f99dd101-e4fa-4c68-a502-c3a14be5c7fb-metrics-tls podName:f99dd101-e4fa-4c68-a502-c3a14be5c7fb nodeName:}" failed. No retries permitted until 2026-07-16 17:14:51.003837223 +0000 UTC m=+35.489681104 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/f99dd101-e4fa-4c68-a502-c3a14be5c7fb-metrics-tls") pod "dns-default-vtkgg" (UID: "f99dd101-e4fa-4c68-a502-c3a14be5c7fb") : secret "dns-default-metrics-tls" not found Jul 16 17:14:50.503968 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:14:50.503937 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f99dd101-e4fa-4c68-a502-c3a14be5c7fb-tmp-dir\") pod \"dns-default-vtkgg\" (UID: \"f99dd101-e4fa-4c68-a502-c3a14be5c7fb\") " pod="openshift-dns/dns-default-vtkgg" Jul 16 17:14:50.504075 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:14:50.503984 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-667mr\" (UniqueName: \"kubernetes.io/projected/f99dd101-e4fa-4c68-a502-c3a14be5c7fb-kube-api-access-667mr\") pod \"dns-default-vtkgg\" (UID: \"f99dd101-e4fa-4c68-a502-c3a14be5c7fb\") " pod="openshift-dns/dns-default-vtkgg" Jul 16 17:14:50.504272 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:14:50.504246 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f99dd101-e4fa-4c68-a502-c3a14be5c7fb-tmp-dir\") pod \"dns-default-vtkgg\" (UID: \"f99dd101-e4fa-4c68-a502-c3a14be5c7fb\") " pod="openshift-dns/dns-default-vtkgg" Jul 16 17:14:50.505207 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:14:50.504636 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/f99dd101-e4fa-4c68-a502-c3a14be5c7fb-config-volume\") pod \"dns-default-vtkgg\" (UID: \"f99dd101-e4fa-4c68-a502-c3a14be5c7fb\") " pod="openshift-dns/dns-default-vtkgg" Jul 16 17:14:50.509157 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:14:50.509137 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/f7ea4735-cb68-4db2-a4ed-b4395fe7e9f6-cert\") pod \"ingress-canary-p2d6d\" (UID: \"f7ea4735-cb68-4db2-a4ed-b4395fe7e9f6\") " pod="openshift-ingress-canary/ingress-canary-p2d6d" Jul 16 17:14:50.515488 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:14:50.515468 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-667mr\" (UniqueName: \"kubernetes.io/projected/f99dd101-e4fa-4c68-a502-c3a14be5c7fb-kube-api-access-667mr\") pod \"dns-default-vtkgg\" (UID: \"f99dd101-e4fa-4c68-a502-c3a14be5c7fb\") " pod="openshift-dns/dns-default-vtkgg" Jul 16 17:14:50.516957 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:14:50.516935 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4l7ff\" (UniqueName: \"kubernetes.io/projected/f7ea4735-cb68-4db2-a4ed-b4395fe7e9f6-kube-api-access-4l7ff\") pod \"ingress-canary-p2d6d\" (UID: \"f7ea4735-cb68-4db2-a4ed-b4395fe7e9f6\") " pod="openshift-ingress-canary/ingress-canary-p2d6d" Jul 16 17:14:50.571136 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:14:50.571092 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-kq8j9_a8425d74-5613-43e9-b6ff-3884b419dbf6/konnectivity-agent/0.log" limit=1024 Jul 16 17:14:50.654485 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:14:50.654399 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-p2d6d" Jul 16 17:14:51.007191 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:14:51.007115 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/f99dd101-e4fa-4c68-a502-c3a14be5c7fb-metrics-tls\") pod \"dns-default-vtkgg\" (UID: \"f99dd101-e4fa-4c68-a502-c3a14be5c7fb\") " pod="openshift-dns/dns-default-vtkgg" Jul 16 17:14:51.007352 ip-10-0-132-130 kubenswrapper[2578]: E0716 17:14:51.007284 2578 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 16 17:14:51.007413 ip-10-0-132-130 kubenswrapper[2578]: E0716 17:14:51.007356 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f99dd101-e4fa-4c68-a502-c3a14be5c7fb-metrics-tls podName:f99dd101-e4fa-4c68-a502-c3a14be5c7fb nodeName:}" failed. No retries permitted until 2026-07-16 17:14:52.007335182 +0000 UTC m=+36.493179049 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/f99dd101-e4fa-4c68-a502-c3a14be5c7fb-metrics-tls") pod "dns-default-vtkgg" (UID: "f99dd101-e4fa-4c68-a502-c3a14be5c7fb") : secret "dns-default-metrics-tls" not found Jul 16 17:14:51.209431 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:14:51.209397 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-kvr2r" Jul 16 17:14:51.209431 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:14:51.209421 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-rg5vf" Jul 16 17:14:51.209614 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:14:51.209407 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-g4r9z" Jul 16 17:14:51.212126 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:14:51.212107 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Jul 16 17:14:51.212765 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:14:51.212746 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" Jul 16 17:14:51.212878 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:14:51.212761 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Jul 16 17:14:51.212878 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:14:51.212746 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-9l5d5\"" Jul 16 17:14:51.212878 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:14:51.212874 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" Jul 16 17:14:51.213164 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:14:51.213149 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-65bh8\"" Jul 16 17:14:51.359920 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:14:51.359796 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-p2d6d"] Jul 16 17:14:51.363186 ip-10-0-132-130 kubenswrapper[2578]: W0716 17:14:51.363165 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf7ea4735_cb68_4db2_a4ed_b4395fe7e9f6.slice/crio-f69cfc3582cfe1ab7776573dc3a59b15cfe7b7265e9f51c52bc754cd0a823125 WatchSource:0}: Error finding container f69cfc3582cfe1ab7776573dc3a59b15cfe7b7265e9f51c52bc754cd0a823125: Status 404 returned error can't find the container with id f69cfc3582cfe1ab7776573dc3a59b15cfe7b7265e9f51c52bc754cd0a823125 Jul 16 17:14:51.368901 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:14:51.368874 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-6gzcp" event={"ID":"fa2959ee-fcc6-49eb-9205-65cf1a08779f","Type":"ContainerStarted","Data":"78dfe8e915ebe179c7718ec01bd2a2a20d34ddcb089f8d5a8712ebf84b8251ad"} Jul 16 17:14:51.370261 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:14:51.370225 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-p2d6d" event={"ID":"f7ea4735-cb68-4db2-a4ed-b4395fe7e9f6","Type":"ContainerStarted","Data":"f69cfc3582cfe1ab7776573dc3a59b15cfe7b7265e9f51c52bc754cd0a823125"} Jul 16 17:14:52.016881 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:14:52.016839 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/f99dd101-e4fa-4c68-a502-c3a14be5c7fb-metrics-tls\") pod \"dns-default-vtkgg\" (UID: \"f99dd101-e4fa-4c68-a502-c3a14be5c7fb\") " pod="openshift-dns/dns-default-vtkgg" Jul 16 17:14:52.017052 ip-10-0-132-130 kubenswrapper[2578]: E0716 17:14:52.017019 2578 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 16 17:14:52.017098 ip-10-0-132-130 kubenswrapper[2578]: E0716 17:14:52.017082 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f99dd101-e4fa-4c68-a502-c3a14be5c7fb-metrics-tls podName:f99dd101-e4fa-4c68-a502-c3a14be5c7fb nodeName:}" failed. No retries permitted until 2026-07-16 17:14:54.01706622 +0000 UTC m=+38.502910088 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/f99dd101-e4fa-4c68-a502-c3a14be5c7fb-metrics-tls") pod "dns-default-vtkgg" (UID: "f99dd101-e4fa-4c68-a502-c3a14be5c7fb") : secret "dns-default-metrics-tls" not found Jul 16 17:14:52.375627 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:14:52.375587 2578 generic.go:356] "Generic (PLEG): container finished" podID="fa2959ee-fcc6-49eb-9205-65cf1a08779f" containerID="78dfe8e915ebe179c7718ec01bd2a2a20d34ddcb089f8d5a8712ebf84b8251ad" exitCode=0 Jul 16 17:14:52.376077 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:14:52.375664 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-6gzcp" event={"ID":"fa2959ee-fcc6-49eb-9205-65cf1a08779f","Type":"ContainerDied","Data":"78dfe8e915ebe179c7718ec01bd2a2a20d34ddcb089f8d5a8712ebf84b8251ad"} Jul 16 17:14:53.385337 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:14:53.385301 2578 generic.go:356] "Generic (PLEG): container finished" podID="fa2959ee-fcc6-49eb-9205-65cf1a08779f" containerID="32fe316ec633a79f33ca0c927c3ff5211495a0bd7a6654d82f7d654dc6234f90" exitCode=0 Jul 16 17:14:53.385758 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:14:53.385377 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-6gzcp" event={"ID":"fa2959ee-fcc6-49eb-9205-65cf1a08779f","Type":"ContainerDied","Data":"32fe316ec633a79f33ca0c927c3ff5211495a0bd7a6654d82f7d654dc6234f90"} Jul 16 17:14:54.031101 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:14:54.031063 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/f99dd101-e4fa-4c68-a502-c3a14be5c7fb-metrics-tls\") pod \"dns-default-vtkgg\" (UID: \"f99dd101-e4fa-4c68-a502-c3a14be5c7fb\") " pod="openshift-dns/dns-default-vtkgg" Jul 16 17:14:54.031272 ip-10-0-132-130 kubenswrapper[2578]: E0716 17:14:54.031171 2578 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 16 17:14:54.031272 ip-10-0-132-130 kubenswrapper[2578]: E0716 17:14:54.031222 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f99dd101-e4fa-4c68-a502-c3a14be5c7fb-metrics-tls podName:f99dd101-e4fa-4c68-a502-c3a14be5c7fb nodeName:}" failed. No retries permitted until 2026-07-16 17:14:58.031208783 +0000 UTC m=+42.517052650 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/f99dd101-e4fa-4c68-a502-c3a14be5c7fb-metrics-tls") pod "dns-default-vtkgg" (UID: "f99dd101-e4fa-4c68-a502-c3a14be5c7fb") : secret "dns-default-metrics-tls" not found Jul 16 17:14:54.389143 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:14:54.389111 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-p2d6d" event={"ID":"f7ea4735-cb68-4db2-a4ed-b4395fe7e9f6","Type":"ContainerStarted","Data":"6e5bc6f0e24835d527838dfe6fc65abeacff65e0eea94a16e3a83aff4e863847"} Jul 16 17:14:54.392411 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:14:54.392389 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-6gzcp" event={"ID":"fa2959ee-fcc6-49eb-9205-65cf1a08779f","Type":"ContainerStarted","Data":"f0edef82fe1450d811ff1cbaa1dc1eae1e6d6c14b1ffa70fd9a1133f976c449a"} Jul 16 17:14:54.407988 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:14:54.407938 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-p2d6d" podStartSLOduration=2.2221831 podStartE2EDuration="4.407922882s" totalImagesPullingTime="2.185739782s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 17:14:50 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 17:14:54.40735287 +0000 UTC m=+38.893196758" watchObservedRunningTime="2026-07-16 17:14:54.407922882 +0000 UTC m=+38.893766784" Jul 16 17:14:54.438988 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:14:54.438949 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-6gzcp" podStartSLOduration=8.426844689 podStartE2EDuration="38.438934799s" totalImagesPullingTime="30.01209011s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-16 17:14:16 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 17:14:54.438707137 +0000 UTC m=+38.924551020" watchObservedRunningTime="2026-07-16 17:14:54.438934799 +0000 UTC m=+38.924778689" Jul 16 17:14:54.573574 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:14:54.573539 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-kq8j9_a8425d74-5613-43e9-b6ff-3884b419dbf6/konnectivity-agent/0.log" limit=1024 Jul 16 17:14:54.755195 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:14:54.755116 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-kq8j9_a8425d74-5613-43e9-b6ff-3884b419dbf6/konnectivity-agent/0.log" limit=1024 Jul 16 17:14:56.175680 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:14:56.175518 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-kq8j9_a8425d74-5613-43e9-b6ff-3884b419dbf6/konnectivity-agent/0.log" limit=1024 Jul 16 17:14:58.060837 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:14:58.060806 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/f99dd101-e4fa-4c68-a502-c3a14be5c7fb-metrics-tls\") pod \"dns-default-vtkgg\" (UID: \"f99dd101-e4fa-4c68-a502-c3a14be5c7fb\") " pod="openshift-dns/dns-default-vtkgg" Jul 16 17:14:58.061197 ip-10-0-132-130 kubenswrapper[2578]: E0716 17:14:58.060906 2578 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 16 17:14:58.061197 ip-10-0-132-130 kubenswrapper[2578]: E0716 17:14:58.060956 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f99dd101-e4fa-4c68-a502-c3a14be5c7fb-metrics-tls podName:f99dd101-e4fa-4c68-a502-c3a14be5c7fb nodeName:}" failed. No retries permitted until 2026-07-16 17:15:06.060941763 +0000 UTC m=+50.546785630 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/f99dd101-e4fa-4c68-a502-c3a14be5c7fb-metrics-tls") pod "dns-default-vtkgg" (UID: "f99dd101-e4fa-4c68-a502-c3a14be5c7fb") : secret "dns-default-metrics-tls" not found Jul 16 17:15:06.111980 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:15:06.111941 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/f99dd101-e4fa-4c68-a502-c3a14be5c7fb-metrics-tls\") pod \"dns-default-vtkgg\" (UID: \"f99dd101-e4fa-4c68-a502-c3a14be5c7fb\") " pod="openshift-dns/dns-default-vtkgg" Jul 16 17:15:06.112412 ip-10-0-132-130 kubenswrapper[2578]: E0716 17:15:06.112080 2578 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 16 17:15:06.112412 ip-10-0-132-130 kubenswrapper[2578]: E0716 17:15:06.112139 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f99dd101-e4fa-4c68-a502-c3a14be5c7fb-metrics-tls podName:f99dd101-e4fa-4c68-a502-c3a14be5c7fb nodeName:}" failed. No retries permitted until 2026-07-16 17:15:22.112124806 +0000 UTC m=+66.597968673 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/f99dd101-e4fa-4c68-a502-c3a14be5c7fb-metrics-tls") pod "dns-default-vtkgg" (UID: "f99dd101-e4fa-4c68-a502-c3a14be5c7fb") : secret "dns-default-metrics-tls" not found Jul 16 17:15:16.365702 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:15:16.365663 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-h7t4d" Jul 16 17:15:21.717094 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:15:21.717037 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/4e00566f-55f0-43ec-9d1a-a883e839823e-original-pull-secret\") pod \"global-pull-secret-syncer-rg5vf\" (UID: \"4e00566f-55f0-43ec-9d1a-a883e839823e\") " pod="kube-system/global-pull-secret-syncer-rg5vf" Jul 16 17:15:21.717582 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:15:21.717127 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f90822ad-ab14-4108-8294-2e76c35a8aa5-metrics-certs\") pod \"network-metrics-daemon-kvr2r\" (UID: \"f90822ad-ab14-4108-8294-2e76c35a8aa5\") " pod="openshift-multus/network-metrics-daemon-kvr2r" Jul 16 17:15:21.719406 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:15:21.719384 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Jul 16 17:15:21.719496 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:15:21.719477 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Jul 16 17:15:21.728295 ip-10-0-132-130 kubenswrapper[2578]: E0716 17:15:21.728275 2578 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found Jul 16 17:15:21.728399 ip-10-0-132-130 kubenswrapper[2578]: E0716 17:15:21.728344 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f90822ad-ab14-4108-8294-2e76c35a8aa5-metrics-certs podName:f90822ad-ab14-4108-8294-2e76c35a8aa5 nodeName:}" failed. No retries permitted until 2026-07-16 17:16:25.72832453 +0000 UTC m=+130.214168399 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/f90822ad-ab14-4108-8294-2e76c35a8aa5-metrics-certs") pod "network-metrics-daemon-kvr2r" (UID: "f90822ad-ab14-4108-8294-2e76c35a8aa5") : secret "metrics-daemon-secret" not found Jul 16 17:15:21.732063 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:15:21.732045 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/4e00566f-55f0-43ec-9d1a-a883e839823e-original-pull-secret\") pod \"global-pull-secret-syncer-rg5vf\" (UID: \"4e00566f-55f0-43ec-9d1a-a883e839823e\") " pod="kube-system/global-pull-secret-syncer-rg5vf" Jul 16 17:15:21.839623 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:15:21.839599 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-rg5vf" Jul 16 17:15:21.919097 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:15:21.919071 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-l64xp\" (UniqueName: \"kubernetes.io/projected/8232d058-70ea-4e6d-ae51-e5ba4303dd93-kube-api-access-l64xp\") pod \"network-check-target-g4r9z\" (UID: \"8232d058-70ea-4e6d-ae51-e5ba4303dd93\") " pod="openshift-network-diagnostics/network-check-target-g4r9z" Jul 16 17:15:21.927242 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:15:21.927220 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" Jul 16 17:15:21.931872 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:15:21.931854 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" Jul 16 17:15:21.943008 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:15:21.942989 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-l64xp\" (UniqueName: \"kubernetes.io/projected/8232d058-70ea-4e6d-ae51-e5ba4303dd93-kube-api-access-l64xp\") pod \"network-check-target-g4r9z\" (UID: \"8232d058-70ea-4e6d-ae51-e5ba4303dd93\") " pod="openshift-network-diagnostics/network-check-target-g4r9z" Jul 16 17:15:21.958075 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:15:21.958051 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-rg5vf"] Jul 16 17:15:21.961567 ip-10-0-132-130 kubenswrapper[2578]: W0716 17:15:21.961537 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4e00566f_55f0_43ec_9d1a_a883e839823e.slice/crio-0d33f12caeea0fcc1aca10302bf59ee4a5760622779caf5f91e4d6ae4e1ea0bf WatchSource:0}: Error finding container 0d33f12caeea0fcc1aca10302bf59ee4a5760622779caf5f91e4d6ae4e1ea0bf: Status 404 returned error can't find the container with id 0d33f12caeea0fcc1aca10302bf59ee4a5760622779caf5f91e4d6ae4e1ea0bf Jul 16 17:15:22.120228 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:15:22.120192 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/f99dd101-e4fa-4c68-a502-c3a14be5c7fb-metrics-tls\") pod \"dns-default-vtkgg\" (UID: \"f99dd101-e4fa-4c68-a502-c3a14be5c7fb\") " pod="openshift-dns/dns-default-vtkgg" Jul 16 17:15:22.120382 ip-10-0-132-130 kubenswrapper[2578]: E0716 17:15:22.120316 2578 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 16 17:15:22.120382 ip-10-0-132-130 kubenswrapper[2578]: E0716 17:15:22.120368 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f99dd101-e4fa-4c68-a502-c3a14be5c7fb-metrics-tls podName:f99dd101-e4fa-4c68-a502-c3a14be5c7fb nodeName:}" failed. No retries permitted until 2026-07-16 17:15:54.120353461 +0000 UTC m=+98.606197328 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/f99dd101-e4fa-4c68-a502-c3a14be5c7fb-metrics-tls") pod "dns-default-vtkgg" (UID: "f99dd101-e4fa-4c68-a502-c3a14be5c7fb") : secret "dns-default-metrics-tls" not found Jul 16 17:15:22.136277 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:15:22.136258 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-9l5d5\"" Jul 16 17:15:22.144830 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:15:22.144814 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-g4r9z" Jul 16 17:15:22.270485 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:15:22.270453 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-g4r9z"] Jul 16 17:15:22.273927 ip-10-0-132-130 kubenswrapper[2578]: W0716 17:15:22.273905 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8232d058_70ea_4e6d_ae51_e5ba4303dd93.slice/crio-c3e3c3357d9b021104cfc0f0fb5ddfd0a4a3665e774a6d37b9521bc3d3e31390 WatchSource:0}: Error finding container c3e3c3357d9b021104cfc0f0fb5ddfd0a4a3665e774a6d37b9521bc3d3e31390: Status 404 returned error can't find the container with id c3e3c3357d9b021104cfc0f0fb5ddfd0a4a3665e774a6d37b9521bc3d3e31390 Jul 16 17:15:22.455425 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:15:22.455348 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-g4r9z" event={"ID":"8232d058-70ea-4e6d-ae51-e5ba4303dd93","Type":"ContainerStarted","Data":"c3e3c3357d9b021104cfc0f0fb5ddfd0a4a3665e774a6d37b9521bc3d3e31390"} Jul 16 17:15:22.456309 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:15:22.456288 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-rg5vf" event={"ID":"4e00566f-55f0-43ec-9d1a-a883e839823e","Type":"ContainerStarted","Data":"0d33f12caeea0fcc1aca10302bf59ee4a5760622779caf5f91e4d6ae4e1ea0bf"} Jul 16 17:15:27.471680 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:15:27.471647 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-rg5vf" event={"ID":"4e00566f-55f0-43ec-9d1a-a883e839823e","Type":"ContainerStarted","Data":"ae2dfdbdb27f283bb9e272bac207c7992d56ac8447ad9e789d7d13eb301f8ceb"} Jul 16 17:15:27.473093 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:15:27.473071 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-g4r9z" event={"ID":"8232d058-70ea-4e6d-ae51-e5ba4303dd93","Type":"ContainerStarted","Data":"b04a80d73bf7da8438a85cc06a151eb8b8b0d33fbfb4d86ae2e165c2920f54e8"} Jul 16 17:15:27.473203 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:15:27.473188 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-g4r9z" Jul 16 17:15:27.489294 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:15:27.489221 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-rg5vf" podStartSLOduration=66.856919673 podStartE2EDuration="1m11.489204488s" totalImagesPullingTime="4.632284815s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-16 17:14:16 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 17:15:27.48917719 +0000 UTC m=+71.975021080" watchObservedRunningTime="2026-07-16 17:15:27.489204488 +0000 UTC m=+71.975048377" Jul 16 17:15:27.506566 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:15:27.506523 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-g4r9z" podStartSLOduration=67.191318011 podStartE2EDuration="1m11.506510501s" totalImagesPullingTime="4.31519249s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 17:14:16 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 17:15:27.506013336 +0000 UTC m=+71.991857225" watchObservedRunningTime="2026-07-16 17:15:27.506510501 +0000 UTC m=+71.992354420" Jul 16 17:15:54.130065 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:15:54.129926 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/f99dd101-e4fa-4c68-a502-c3a14be5c7fb-metrics-tls\") pod \"dns-default-vtkgg\" (UID: \"f99dd101-e4fa-4c68-a502-c3a14be5c7fb\") " pod="openshift-dns/dns-default-vtkgg" Jul 16 17:15:54.130479 ip-10-0-132-130 kubenswrapper[2578]: E0716 17:15:54.130075 2578 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 16 17:15:54.130479 ip-10-0-132-130 kubenswrapper[2578]: E0716 17:15:54.130150 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f99dd101-e4fa-4c68-a502-c3a14be5c7fb-metrics-tls podName:f99dd101-e4fa-4c68-a502-c3a14be5c7fb nodeName:}" failed. No retries permitted until 2026-07-16 17:16:58.130133517 +0000 UTC m=+162.615977384 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/f99dd101-e4fa-4c68-a502-c3a14be5c7fb-metrics-tls") pod "dns-default-vtkgg" (UID: "f99dd101-e4fa-4c68-a502-c3a14be5c7fb") : secret "dns-default-metrics-tls" not found Jul 16 17:15:56.185297 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:15:56.185268 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-p872b"] Jul 16 17:15:56.189201 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:15:56.189184 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-p872b" Jul 16 17:15:56.192491 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:15:56.192472 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-tls\"" Jul 16 17:15:56.192587 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:15:56.192475 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\"" Jul 16 17:15:56.193737 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:15:56.193724 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemetry-config\"" Jul 16 17:15:56.193786 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:15:56.193733 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-dockercfg-gzfzv\"" Jul 16 17:15:56.193786 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:15:56.193724 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\"" Jul 16 17:15:56.217415 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:15:56.217391 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-p872b"] Jul 16 17:15:56.344245 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:15:56.344215 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/5bf41cfb-4cb4-4953-a458-9211602c6b74-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-5f7f494dd4-p872b\" (UID: \"5bf41cfb-4cb4-4953-a458-9211602c6b74\") " pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-p872b" Jul 16 17:15:56.344430 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:15:56.344260 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-f6jmh\" (UniqueName: \"kubernetes.io/projected/5bf41cfb-4cb4-4953-a458-9211602c6b74-kube-api-access-f6jmh\") pod \"cluster-monitoring-operator-5f7f494dd4-p872b\" (UID: \"5bf41cfb-4cb4-4953-a458-9211602c6b74\") " pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-p872b" Jul 16 17:15:56.344430 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:15:56.344318 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/5bf41cfb-4cb4-4953-a458-9211602c6b74-telemetry-config\") pod \"cluster-monitoring-operator-5f7f494dd4-p872b\" (UID: \"5bf41cfb-4cb4-4953-a458-9211602c6b74\") " pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-p872b" Jul 16 17:15:56.445515 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:15:56.445425 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/5bf41cfb-4cb4-4953-a458-9211602c6b74-telemetry-config\") pod \"cluster-monitoring-operator-5f7f494dd4-p872b\" (UID: \"5bf41cfb-4cb4-4953-a458-9211602c6b74\") " pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-p872b" Jul 16 17:15:56.445515 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:15:56.445502 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/5bf41cfb-4cb4-4953-a458-9211602c6b74-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-5f7f494dd4-p872b\" (UID: \"5bf41cfb-4cb4-4953-a458-9211602c6b74\") " pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-p872b" Jul 16 17:15:56.445722 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:15:56.445542 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-f6jmh\" (UniqueName: \"kubernetes.io/projected/5bf41cfb-4cb4-4953-a458-9211602c6b74-kube-api-access-f6jmh\") pod \"cluster-monitoring-operator-5f7f494dd4-p872b\" (UID: \"5bf41cfb-4cb4-4953-a458-9211602c6b74\") " pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-p872b" Jul 16 17:15:56.445779 ip-10-0-132-130 kubenswrapper[2578]: E0716 17:15:56.445738 2578 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Jul 16 17:15:56.445830 ip-10-0-132-130 kubenswrapper[2578]: E0716 17:15:56.445807 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5bf41cfb-4cb4-4953-a458-9211602c6b74-cluster-monitoring-operator-tls podName:5bf41cfb-4cb4-4953-a458-9211602c6b74 nodeName:}" failed. No retries permitted until 2026-07-16 17:15:56.945792907 +0000 UTC m=+101.431636774 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/5bf41cfb-4cb4-4953-a458-9211602c6b74-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-5f7f494dd4-p872b" (UID: "5bf41cfb-4cb4-4953-a458-9211602c6b74") : secret "cluster-monitoring-operator-tls" not found Jul 16 17:15:56.446118 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:15:56.446097 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/5bf41cfb-4cb4-4953-a458-9211602c6b74-telemetry-config\") pod \"cluster-monitoring-operator-5f7f494dd4-p872b\" (UID: \"5bf41cfb-4cb4-4953-a458-9211602c6b74\") " pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-p872b" Jul 16 17:15:56.457039 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:15:56.457012 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-f6jmh\" (UniqueName: \"kubernetes.io/projected/5bf41cfb-4cb4-4953-a458-9211602c6b74-kube-api-access-f6jmh\") pod \"cluster-monitoring-operator-5f7f494dd4-p872b\" (UID: \"5bf41cfb-4cb4-4953-a458-9211602c6b74\") " pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-p872b" Jul 16 17:15:56.948640 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:15:56.948610 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/5bf41cfb-4cb4-4953-a458-9211602c6b74-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-5f7f494dd4-p872b\" (UID: \"5bf41cfb-4cb4-4953-a458-9211602c6b74\") " pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-p872b" Jul 16 17:15:56.948838 ip-10-0-132-130 kubenswrapper[2578]: E0716 17:15:56.948759 2578 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Jul 16 17:15:56.948838 ip-10-0-132-130 kubenswrapper[2578]: E0716 17:15:56.948834 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5bf41cfb-4cb4-4953-a458-9211602c6b74-cluster-monitoring-operator-tls podName:5bf41cfb-4cb4-4953-a458-9211602c6b74 nodeName:}" failed. No retries permitted until 2026-07-16 17:15:57.94882054 +0000 UTC m=+102.434664413 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/5bf41cfb-4cb4-4953-a458-9211602c6b74-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-5f7f494dd4-p872b" (UID: "5bf41cfb-4cb4-4953-a458-9211602c6b74") : secret "cluster-monitoring-operator-tls" not found Jul 16 17:15:57.956295 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:15:57.956260 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/5bf41cfb-4cb4-4953-a458-9211602c6b74-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-5f7f494dd4-p872b\" (UID: \"5bf41cfb-4cb4-4953-a458-9211602c6b74\") " pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-p872b" Jul 16 17:15:57.956664 ip-10-0-132-130 kubenswrapper[2578]: E0716 17:15:57.956401 2578 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Jul 16 17:15:57.956664 ip-10-0-132-130 kubenswrapper[2578]: E0716 17:15:57.956457 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5bf41cfb-4cb4-4953-a458-9211602c6b74-cluster-monitoring-operator-tls podName:5bf41cfb-4cb4-4953-a458-9211602c6b74 nodeName:}" failed. No retries permitted until 2026-07-16 17:15:59.956441712 +0000 UTC m=+104.442285598 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/5bf41cfb-4cb4-4953-a458-9211602c6b74-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-5f7f494dd4-p872b" (UID: "5bf41cfb-4cb4-4953-a458-9211602c6b74") : secret "cluster-monitoring-operator-tls" not found Jul 16 17:15:58.477208 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:15:58.477178 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-g4r9z" Jul 16 17:15:59.969200 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:15:59.969160 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/5bf41cfb-4cb4-4953-a458-9211602c6b74-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-5f7f494dd4-p872b\" (UID: \"5bf41cfb-4cb4-4953-a458-9211602c6b74\") " pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-p872b" Jul 16 17:15:59.969648 ip-10-0-132-130 kubenswrapper[2578]: E0716 17:15:59.969266 2578 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Jul 16 17:15:59.969648 ip-10-0-132-130 kubenswrapper[2578]: E0716 17:15:59.969318 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5bf41cfb-4cb4-4953-a458-9211602c6b74-cluster-monitoring-operator-tls podName:5bf41cfb-4cb4-4953-a458-9211602c6b74 nodeName:}" failed. No retries permitted until 2026-07-16 17:16:03.969305062 +0000 UTC m=+108.455148929 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/5bf41cfb-4cb4-4953-a458-9211602c6b74-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-5f7f494dd4-p872b" (UID: "5bf41cfb-4cb4-4953-a458-9211602c6b74") : secret "cluster-monitoring-operator-tls" not found Jul 16 17:16:03.995899 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:16:03.995855 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/5bf41cfb-4cb4-4953-a458-9211602c6b74-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-5f7f494dd4-p872b\" (UID: \"5bf41cfb-4cb4-4953-a458-9211602c6b74\") " pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-p872b" Jul 16 17:16:03.996273 ip-10-0-132-130 kubenswrapper[2578]: E0716 17:16:03.995965 2578 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Jul 16 17:16:03.996273 ip-10-0-132-130 kubenswrapper[2578]: E0716 17:16:03.996017 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5bf41cfb-4cb4-4953-a458-9211602c6b74-cluster-monitoring-operator-tls podName:5bf41cfb-4cb4-4953-a458-9211602c6b74 nodeName:}" failed. No retries permitted until 2026-07-16 17:16:11.996004752 +0000 UTC m=+116.481848619 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/5bf41cfb-4cb4-4953-a458-9211602c6b74-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-5f7f494dd4-p872b" (UID: "5bf41cfb-4cb4-4953-a458-9211602c6b74") : secret "cluster-monitoring-operator-tls" not found Jul 16 17:16:05.831494 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:16:05.831462 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-5czbw"] Jul 16 17:16:05.835154 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:16:05.835135 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-5czbw" Jul 16 17:16:05.838291 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:16:05.838273 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\"" Jul 16 17:16:05.838370 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:16:05.838280 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\"" Jul 16 17:16:05.838839 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:16:05.838824 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\"" Jul 16 17:16:05.838974 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:16:05.838957 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-t9pcw\"" Jul 16 17:16:05.839197 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:16:05.839180 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\"" Jul 16 17:16:05.857393 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:16:05.857373 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-5czbw"] Jul 16 17:16:05.909577 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:16:05.909553 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/dbd1c30a-e583-4ecd-b5c6-ec71f3579e50-data-volume\") pod \"insights-runtime-extractor-5czbw\" (UID: \"dbd1c30a-e583-4ecd-b5c6-ec71f3579e50\") " pod="openshift-insights/insights-runtime-extractor-5czbw" Jul 16 17:16:05.909676 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:16:05.909585 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fh9j8\" (UniqueName: \"kubernetes.io/projected/dbd1c30a-e583-4ecd-b5c6-ec71f3579e50-kube-api-access-fh9j8\") pod \"insights-runtime-extractor-5czbw\" (UID: \"dbd1c30a-e583-4ecd-b5c6-ec71f3579e50\") " pod="openshift-insights/insights-runtime-extractor-5czbw" Jul 16 17:16:05.909752 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:16:05.909708 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/dbd1c30a-e583-4ecd-b5c6-ec71f3579e50-crio-socket\") pod \"insights-runtime-extractor-5czbw\" (UID: \"dbd1c30a-e583-4ecd-b5c6-ec71f3579e50\") " pod="openshift-insights/insights-runtime-extractor-5czbw" Jul 16 17:16:05.909752 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:16:05.909738 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/dbd1c30a-e583-4ecd-b5c6-ec71f3579e50-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-5czbw\" (UID: \"dbd1c30a-e583-4ecd-b5c6-ec71f3579e50\") " pod="openshift-insights/insights-runtime-extractor-5czbw" Jul 16 17:16:05.909833 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:16:05.909766 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/dbd1c30a-e583-4ecd-b5c6-ec71f3579e50-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-5czbw\" (UID: \"dbd1c30a-e583-4ecd-b5c6-ec71f3579e50\") " pod="openshift-insights/insights-runtime-extractor-5czbw" Jul 16 17:16:06.010640 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:16:06.010617 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/dbd1c30a-e583-4ecd-b5c6-ec71f3579e50-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-5czbw\" (UID: \"dbd1c30a-e583-4ecd-b5c6-ec71f3579e50\") " pod="openshift-insights/insights-runtime-extractor-5czbw" Jul 16 17:16:06.010764 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:16:06.010653 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/dbd1c30a-e583-4ecd-b5c6-ec71f3579e50-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-5czbw\" (UID: \"dbd1c30a-e583-4ecd-b5c6-ec71f3579e50\") " pod="openshift-insights/insights-runtime-extractor-5czbw" Jul 16 17:16:06.010764 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:16:06.010676 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/dbd1c30a-e583-4ecd-b5c6-ec71f3579e50-data-volume\") pod \"insights-runtime-extractor-5czbw\" (UID: \"dbd1c30a-e583-4ecd-b5c6-ec71f3579e50\") " pod="openshift-insights/insights-runtime-extractor-5czbw" Jul 16 17:16:06.010764 ip-10-0-132-130 kubenswrapper[2578]: E0716 17:16:06.010749 2578 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 16 17:16:06.010883 ip-10-0-132-130 kubenswrapper[2578]: E0716 17:16:06.010779 2578 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 16 17:16:06.010883 ip-10-0-132-130 kubenswrapper[2578]: E0716 17:16:06.010821 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/dbd1c30a-e583-4ecd-b5c6-ec71f3579e50-kube-rbac-proxy-cm podName:dbd1c30a-e583-4ecd-b5c6-ec71f3579e50 nodeName:}" failed. No retries permitted until 2026-07-16 17:16:06.510796617 +0000 UTC m=+110.996640483 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/dbd1c30a-e583-4ecd-b5c6-ec71f3579e50-kube-rbac-proxy-cm") pod "insights-runtime-extractor-5czbw" (UID: "dbd1c30a-e583-4ecd-b5c6-ec71f3579e50") : configmap "kube-rbac-proxy" not found Jul 16 17:16:06.010883 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:16:06.010858 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fh9j8\" (UniqueName: \"kubernetes.io/projected/dbd1c30a-e583-4ecd-b5c6-ec71f3579e50-kube-api-access-fh9j8\") pod \"insights-runtime-extractor-5czbw\" (UID: \"dbd1c30a-e583-4ecd-b5c6-ec71f3579e50\") " pod="openshift-insights/insights-runtime-extractor-5czbw" Jul 16 17:16:06.011029 ip-10-0-132-130 kubenswrapper[2578]: E0716 17:16:06.010885 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/dbd1c30a-e583-4ecd-b5c6-ec71f3579e50-insights-runtime-extractor-tls podName:dbd1c30a-e583-4ecd-b5c6-ec71f3579e50 nodeName:}" failed. No retries permitted until 2026-07-16 17:16:06.510865451 +0000 UTC m=+110.996709334 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/dbd1c30a-e583-4ecd-b5c6-ec71f3579e50-insights-runtime-extractor-tls") pod "insights-runtime-extractor-5czbw" (UID: "dbd1c30a-e583-4ecd-b5c6-ec71f3579e50") : secret "insights-runtime-extractor-tls" not found Jul 16 17:16:06.011029 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:16:06.010997 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/dbd1c30a-e583-4ecd-b5c6-ec71f3579e50-crio-socket\") pod \"insights-runtime-extractor-5czbw\" (UID: \"dbd1c30a-e583-4ecd-b5c6-ec71f3579e50\") " pod="openshift-insights/insights-runtime-extractor-5czbw" Jul 16 17:16:06.011029 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:16:06.011006 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/dbd1c30a-e583-4ecd-b5c6-ec71f3579e50-data-volume\") pod \"insights-runtime-extractor-5czbw\" (UID: \"dbd1c30a-e583-4ecd-b5c6-ec71f3579e50\") " pod="openshift-insights/insights-runtime-extractor-5czbw" Jul 16 17:16:06.011122 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:16:06.011061 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/dbd1c30a-e583-4ecd-b5c6-ec71f3579e50-crio-socket\") pod \"insights-runtime-extractor-5czbw\" (UID: \"dbd1c30a-e583-4ecd-b5c6-ec71f3579e50\") " pod="openshift-insights/insights-runtime-extractor-5czbw" Jul 16 17:16:06.026909 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:16:06.026889 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fh9j8\" (UniqueName: \"kubernetes.io/projected/dbd1c30a-e583-4ecd-b5c6-ec71f3579e50-kube-api-access-fh9j8\") pod \"insights-runtime-extractor-5czbw\" (UID: \"dbd1c30a-e583-4ecd-b5c6-ec71f3579e50\") " pod="openshift-insights/insights-runtime-extractor-5czbw" Jul 16 17:16:06.222375 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:16:06.222298 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-65db858894-msl7k"] Jul 16 17:16:06.224599 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:16:06.224579 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-msl7k" Jul 16 17:16:06.227631 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:16:06.227608 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-samples-operator\"/\"kube-root-ca.crt\"" Jul 16 17:16:06.227746 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:16:06.227629 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-samples-operator\"/\"openshift-service-ca.crt\"" Jul 16 17:16:06.227746 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:16:06.227704 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-samples-operator\"/\"cluster-samples-operator-dockercfg-f4bbl\"" Jul 16 17:16:06.227863 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:16:06.227850 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-samples-operator\"/\"samples-operator-tls\"" Jul 16 17:16:06.238383 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:16:06.238361 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-65db858894-msl7k"] Jul 16 17:16:06.312494 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:16:06.312467 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/8575b8ff-3e17-4577-a026-adef0cc639e0-samples-operator-tls\") pod \"cluster-samples-operator-65db858894-msl7k\" (UID: \"8575b8ff-3e17-4577-a026-adef0cc639e0\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-msl7k" Jul 16 17:16:06.312636 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:16:06.312547 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mz49f\" (UniqueName: \"kubernetes.io/projected/8575b8ff-3e17-4577-a026-adef0cc639e0-kube-api-access-mz49f\") pod \"cluster-samples-operator-65db858894-msl7k\" (UID: \"8575b8ff-3e17-4577-a026-adef0cc639e0\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-msl7k" Jul 16 17:16:06.339352 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:16:06.339324 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-service-ca-operator/service-ca-operator-7695fc6c8c-hvxkc"] Jul 16 17:16:06.341618 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:16:06.341598 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca-operator/service-ca-operator-7695fc6c8c-hvxkc" Jul 16 17:16:06.341861 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:16:06.341839 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-78495597f6-nqvsk"] Jul 16 17:16:06.344178 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:16:06.344159 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-storage-operator/volume-data-source-validator-78495597f6-nqvsk" Jul 16 17:16:06.344516 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:16:06.344501 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-8q957"] Jul 16 17:16:06.344709 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:16:06.344666 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca-operator\"/\"serving-cert\"" Jul 16 17:16:06.344709 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:16:06.344686 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"openshift-service-ca.crt\"" Jul 16 17:16:06.344962 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:16:06.344944 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"service-ca-operator-config\"" Jul 16 17:16:06.345015 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:16:06.344983 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"kube-root-ca.crt\"" Jul 16 17:16:06.345807 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:16:06.345791 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca-operator\"/\"service-ca-operator-dockercfg-689gv\"" Jul 16 17:16:06.346788 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:16:06.346772 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console-operator/console-operator-7866497494-drs2p"] Jul 16 17:16:06.346903 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:16:06.346888 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-8q957" Jul 16 17:16:06.348901 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:16:06.348886 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-storage-operator\"/\"kube-root-ca.crt\"" Jul 16 17:16:06.349512 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:16:06.349496 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console-operator/console-operator-7866497494-drs2p" Jul 16 17:16:06.349942 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:16:06.349926 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-storage-operator\"/\"volume-data-source-validator-dockercfg-c55ls\"" Jul 16 17:16:06.350028 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:16:06.349980 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"serving-cert\"" Jul 16 17:16:06.350075 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:16:06.350035 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"openshift-service-ca.crt\"" Jul 16 17:16:06.350075 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:16:06.350041 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-storage-operator\"/\"openshift-service-ca.crt\"" Jul 16 17:16:06.350150 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:16:06.350115 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"kube-storage-version-migrator-operator-dockercfg-q82zr\"" Jul 16 17:16:06.350186 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:16:06.350155 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"config\"" Jul 16 17:16:06.352829 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:16:06.352812 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"openshift-service-ca.crt\"" Jul 16 17:16:06.353273 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:16:06.353251 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"kube-root-ca.crt\"" Jul 16 17:16:06.353380 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:16:06.353324 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"console-operator-config\"" Jul 16 17:16:06.353510 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:16:06.353491 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"kube-root-ca.crt\"" Jul 16 17:16:06.353584 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:16:06.353563 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console-operator\"/\"console-operator-dockercfg-vqcw6\"" Jul 16 17:16:06.358551 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:16:06.358531 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console-operator\"/\"serving-cert\"" Jul 16 17:16:06.361360 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:16:06.361322 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"trusted-ca\"" Jul 16 17:16:06.363333 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:16:06.363315 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-8q957"] Jul 16 17:16:06.370601 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:16:06.370583 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console-operator/console-operator-7866497494-drs2p"] Jul 16 17:16:06.397178 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:16:06.397154 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca-operator/service-ca-operator-7695fc6c8c-hvxkc"] Jul 16 17:16:06.412901 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:16:06.412879 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mz49f\" (UniqueName: \"kubernetes.io/projected/8575b8ff-3e17-4577-a026-adef0cc639e0-kube-api-access-mz49f\") pod \"cluster-samples-operator-65db858894-msl7k\" (UID: \"8575b8ff-3e17-4577-a026-adef0cc639e0\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-msl7k" Jul 16 17:16:06.413016 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:16:06.412921 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/14f6cc76-f1b6-4d19-8bc0-74aa1e1665c7-serving-cert\") pod \"service-ca-operator-7695fc6c8c-hvxkc\" (UID: \"14f6cc76-f1b6-4d19-8bc0-74aa1e1665c7\") " pod="openshift-service-ca-operator/service-ca-operator-7695fc6c8c-hvxkc" Jul 16 17:16:06.413016 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:16:06.412950 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/8575b8ff-3e17-4577-a026-adef0cc639e0-samples-operator-tls\") pod \"cluster-samples-operator-65db858894-msl7k\" (UID: \"8575b8ff-3e17-4577-a026-adef0cc639e0\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-msl7k" Jul 16 17:16:06.413016 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:16:06.413001 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/14f6cc76-f1b6-4d19-8bc0-74aa1e1665c7-config\") pod \"service-ca-operator-7695fc6c8c-hvxkc\" (UID: \"14f6cc76-f1b6-4d19-8bc0-74aa1e1665c7\") " pod="openshift-service-ca-operator/service-ca-operator-7695fc6c8c-hvxkc" Jul 16 17:16:06.413200 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:16:06.413031 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jmccl\" (UniqueName: \"kubernetes.io/projected/14f6cc76-f1b6-4d19-8bc0-74aa1e1665c7-kube-api-access-jmccl\") pod \"service-ca-operator-7695fc6c8c-hvxkc\" (UID: \"14f6cc76-f1b6-4d19-8bc0-74aa1e1665c7\") " pod="openshift-service-ca-operator/service-ca-operator-7695fc6c8c-hvxkc" Jul 16 17:16:06.413200 ip-10-0-132-130 kubenswrapper[2578]: E0716 17:16:06.413110 2578 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Jul 16 17:16:06.413200 ip-10-0-132-130 kubenswrapper[2578]: E0716 17:16:06.413179 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8575b8ff-3e17-4577-a026-adef0cc639e0-samples-operator-tls podName:8575b8ff-3e17-4577-a026-adef0cc639e0 nodeName:}" failed. No retries permitted until 2026-07-16 17:16:06.913157638 +0000 UTC m=+111.399001506 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/8575b8ff-3e17-4577-a026-adef0cc639e0-samples-operator-tls") pod "cluster-samples-operator-65db858894-msl7k" (UID: "8575b8ff-3e17-4577-a026-adef0cc639e0") : secret "samples-operator-tls" not found Jul 16 17:16:06.413904 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:16:06.413885 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-78495597f6-nqvsk"] Jul 16 17:16:06.429519 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:16:06.429495 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mz49f\" (UniqueName: \"kubernetes.io/projected/8575b8ff-3e17-4577-a026-adef0cc639e0-kube-api-access-mz49f\") pod \"cluster-samples-operator-65db858894-msl7k\" (UID: \"8575b8ff-3e17-4577-a026-adef0cc639e0\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-msl7k" Jul 16 17:16:06.513636 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:16:06.513558 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/91a757db-58f1-4483-baf4-9b936e4b8072-serving-cert\") pod \"kube-storage-version-migrator-operator-799cb7f958-8q957\" (UID: \"91a757db-58f1-4483-baf4-9b936e4b8072\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-8q957" Jul 16 17:16:06.513636 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:16:06.513599 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/e55f447c-ad64-482b-a86b-0588bddf7b4d-trusted-ca\") pod \"console-operator-7866497494-drs2p\" (UID: \"e55f447c-ad64-482b-a86b-0588bddf7b4d\") " pod="openshift-console-operator/console-operator-7866497494-drs2p" Jul 16 17:16:06.513636 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:16:06.513616 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9j7qr\" (UniqueName: \"kubernetes.io/projected/e55f447c-ad64-482b-a86b-0588bddf7b4d-kube-api-access-9j7qr\") pod \"console-operator-7866497494-drs2p\" (UID: \"e55f447c-ad64-482b-a86b-0588bddf7b4d\") " pod="openshift-console-operator/console-operator-7866497494-drs2p" Jul 16 17:16:06.513855 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:16:06.513670 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/e55f447c-ad64-482b-a86b-0588bddf7b4d-config\") pod \"console-operator-7866497494-drs2p\" (UID: \"e55f447c-ad64-482b-a86b-0588bddf7b4d\") " pod="openshift-console-operator/console-operator-7866497494-drs2p" Jul 16 17:16:06.513855 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:16:06.513773 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jmccl\" (UniqueName: \"kubernetes.io/projected/14f6cc76-f1b6-4d19-8bc0-74aa1e1665c7-kube-api-access-jmccl\") pod \"service-ca-operator-7695fc6c8c-hvxkc\" (UID: \"14f6cc76-f1b6-4d19-8bc0-74aa1e1665c7\") " pod="openshift-service-ca-operator/service-ca-operator-7695fc6c8c-hvxkc" Jul 16 17:16:06.513855 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:16:06.513805 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/dbd1c30a-e583-4ecd-b5c6-ec71f3579e50-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-5czbw\" (UID: \"dbd1c30a-e583-4ecd-b5c6-ec71f3579e50\") " pod="openshift-insights/insights-runtime-extractor-5czbw" Jul 16 17:16:06.513855 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:16:06.513831 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/91a757db-58f1-4483-baf4-9b936e4b8072-config\") pod \"kube-storage-version-migrator-operator-799cb7f958-8q957\" (UID: \"91a757db-58f1-4483-baf4-9b936e4b8072\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-8q957" Jul 16 17:16:06.513990 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:16:06.513880 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/14f6cc76-f1b6-4d19-8bc0-74aa1e1665c7-serving-cert\") pod \"service-ca-operator-7695fc6c8c-hvxkc\" (UID: \"14f6cc76-f1b6-4d19-8bc0-74aa1e1665c7\") " pod="openshift-service-ca-operator/service-ca-operator-7695fc6c8c-hvxkc" Jul 16 17:16:06.513990 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:16:06.513928 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/e55f447c-ad64-482b-a86b-0588bddf7b4d-serving-cert\") pod \"console-operator-7866497494-drs2p\" (UID: \"e55f447c-ad64-482b-a86b-0588bddf7b4d\") " pod="openshift-console-operator/console-operator-7866497494-drs2p" Jul 16 17:16:06.513990 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:16:06.513948 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/14f6cc76-f1b6-4d19-8bc0-74aa1e1665c7-config\") pod \"service-ca-operator-7695fc6c8c-hvxkc\" (UID: \"14f6cc76-f1b6-4d19-8bc0-74aa1e1665c7\") " pod="openshift-service-ca-operator/service-ca-operator-7695fc6c8c-hvxkc" Jul 16 17:16:06.513990 ip-10-0-132-130 kubenswrapper[2578]: E0716 17:16:06.513961 2578 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 16 17:16:06.513990 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:16:06.513973 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bcj7k\" (UniqueName: \"kubernetes.io/projected/91a757db-58f1-4483-baf4-9b936e4b8072-kube-api-access-bcj7k\") pod \"kube-storage-version-migrator-operator-799cb7f958-8q957\" (UID: \"91a757db-58f1-4483-baf4-9b936e4b8072\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-8q957" Jul 16 17:16:06.514217 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:16:06.514025 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-c8sn4\" (UniqueName: \"kubernetes.io/projected/609ca387-5fdd-4c2d-829e-db5431ddb87f-kube-api-access-c8sn4\") pod \"volume-data-source-validator-78495597f6-nqvsk\" (UID: \"609ca387-5fdd-4c2d-829e-db5431ddb87f\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-78495597f6-nqvsk" Jul 16 17:16:06.514217 ip-10-0-132-130 kubenswrapper[2578]: E0716 17:16:06.514037 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/dbd1c30a-e583-4ecd-b5c6-ec71f3579e50-insights-runtime-extractor-tls podName:dbd1c30a-e583-4ecd-b5c6-ec71f3579e50 nodeName:}" failed. No retries permitted until 2026-07-16 17:16:07.514015446 +0000 UTC m=+111.999859314 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/dbd1c30a-e583-4ecd-b5c6-ec71f3579e50-insights-runtime-extractor-tls") pod "insights-runtime-extractor-5czbw" (UID: "dbd1c30a-e583-4ecd-b5c6-ec71f3579e50") : secret "insights-runtime-extractor-tls" not found Jul 16 17:16:06.514217 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:16:06.514116 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/dbd1c30a-e583-4ecd-b5c6-ec71f3579e50-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-5czbw\" (UID: \"dbd1c30a-e583-4ecd-b5c6-ec71f3579e50\") " pod="openshift-insights/insights-runtime-extractor-5czbw" Jul 16 17:16:06.514217 ip-10-0-132-130 kubenswrapper[2578]: E0716 17:16:06.514168 2578 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 16 17:16:06.514383 ip-10-0-132-130 kubenswrapper[2578]: E0716 17:16:06.514253 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/dbd1c30a-e583-4ecd-b5c6-ec71f3579e50-kube-rbac-proxy-cm podName:dbd1c30a-e583-4ecd-b5c6-ec71f3579e50 nodeName:}" failed. No retries permitted until 2026-07-16 17:16:07.514235117 +0000 UTC m=+112.000078995 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/dbd1c30a-e583-4ecd-b5c6-ec71f3579e50-kube-rbac-proxy-cm") pod "insights-runtime-extractor-5czbw" (UID: "dbd1c30a-e583-4ecd-b5c6-ec71f3579e50") : configmap "kube-rbac-proxy" not found Jul 16 17:16:06.514540 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:16:06.514518 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/14f6cc76-f1b6-4d19-8bc0-74aa1e1665c7-config\") pod \"service-ca-operator-7695fc6c8c-hvxkc\" (UID: \"14f6cc76-f1b6-4d19-8bc0-74aa1e1665c7\") " pod="openshift-service-ca-operator/service-ca-operator-7695fc6c8c-hvxkc" Jul 16 17:16:06.516423 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:16:06.516402 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/14f6cc76-f1b6-4d19-8bc0-74aa1e1665c7-serving-cert\") pod \"service-ca-operator-7695fc6c8c-hvxkc\" (UID: \"14f6cc76-f1b6-4d19-8bc0-74aa1e1665c7\") " pod="openshift-service-ca-operator/service-ca-operator-7695fc6c8c-hvxkc" Jul 16 17:16:06.525138 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:16:06.525112 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jmccl\" (UniqueName: \"kubernetes.io/projected/14f6cc76-f1b6-4d19-8bc0-74aa1e1665c7-kube-api-access-jmccl\") pod \"service-ca-operator-7695fc6c8c-hvxkc\" (UID: \"14f6cc76-f1b6-4d19-8bc0-74aa1e1665c7\") " pod="openshift-service-ca-operator/service-ca-operator-7695fc6c8c-hvxkc" Jul 16 17:16:06.614861 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:16:06.614830 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/e55f447c-ad64-482b-a86b-0588bddf7b4d-serving-cert\") pod \"console-operator-7866497494-drs2p\" (UID: \"e55f447c-ad64-482b-a86b-0588bddf7b4d\") " pod="openshift-console-operator/console-operator-7866497494-drs2p" Jul 16 17:16:06.614965 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:16:06.614866 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bcj7k\" (UniqueName: \"kubernetes.io/projected/91a757db-58f1-4483-baf4-9b936e4b8072-kube-api-access-bcj7k\") pod \"kube-storage-version-migrator-operator-799cb7f958-8q957\" (UID: \"91a757db-58f1-4483-baf4-9b936e4b8072\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-8q957" Jul 16 17:16:06.614965 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:16:06.614887 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-c8sn4\" (UniqueName: \"kubernetes.io/projected/609ca387-5fdd-4c2d-829e-db5431ddb87f-kube-api-access-c8sn4\") pod \"volume-data-source-validator-78495597f6-nqvsk\" (UID: \"609ca387-5fdd-4c2d-829e-db5431ddb87f\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-78495597f6-nqvsk" Jul 16 17:16:06.615047 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:16:06.614971 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/91a757db-58f1-4483-baf4-9b936e4b8072-serving-cert\") pod \"kube-storage-version-migrator-operator-799cb7f958-8q957\" (UID: \"91a757db-58f1-4483-baf4-9b936e4b8072\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-8q957" Jul 16 17:16:06.615047 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:16:06.615003 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/e55f447c-ad64-482b-a86b-0588bddf7b4d-trusted-ca\") pod \"console-operator-7866497494-drs2p\" (UID: \"e55f447c-ad64-482b-a86b-0588bddf7b4d\") " pod="openshift-console-operator/console-operator-7866497494-drs2p" Jul 16 17:16:06.615047 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:16:06.615020 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9j7qr\" (UniqueName: \"kubernetes.io/projected/e55f447c-ad64-482b-a86b-0588bddf7b4d-kube-api-access-9j7qr\") pod \"console-operator-7866497494-drs2p\" (UID: \"e55f447c-ad64-482b-a86b-0588bddf7b4d\") " pod="openshift-console-operator/console-operator-7866497494-drs2p" Jul 16 17:16:06.615201 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:16:06.615047 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/e55f447c-ad64-482b-a86b-0588bddf7b4d-config\") pod \"console-operator-7866497494-drs2p\" (UID: \"e55f447c-ad64-482b-a86b-0588bddf7b4d\") " pod="openshift-console-operator/console-operator-7866497494-drs2p" Jul 16 17:16:06.615201 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:16:06.615109 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/91a757db-58f1-4483-baf4-9b936e4b8072-config\") pod \"kube-storage-version-migrator-operator-799cb7f958-8q957\" (UID: \"91a757db-58f1-4483-baf4-9b936e4b8072\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-8q957" Jul 16 17:16:06.615762 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:16:06.615738 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/91a757db-58f1-4483-baf4-9b936e4b8072-config\") pod \"kube-storage-version-migrator-operator-799cb7f958-8q957\" (UID: \"91a757db-58f1-4483-baf4-9b936e4b8072\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-8q957" Jul 16 17:16:06.615855 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:16:06.615802 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/e55f447c-ad64-482b-a86b-0588bddf7b4d-config\") pod \"console-operator-7866497494-drs2p\" (UID: \"e55f447c-ad64-482b-a86b-0588bddf7b4d\") " pod="openshift-console-operator/console-operator-7866497494-drs2p" Jul 16 17:16:06.616013 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:16:06.615972 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/e55f447c-ad64-482b-a86b-0588bddf7b4d-trusted-ca\") pod \"console-operator-7866497494-drs2p\" (UID: \"e55f447c-ad64-482b-a86b-0588bddf7b4d\") " pod="openshift-console-operator/console-operator-7866497494-drs2p" Jul 16 17:16:06.616834 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:16:06.616816 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/e55f447c-ad64-482b-a86b-0588bddf7b4d-serving-cert\") pod \"console-operator-7866497494-drs2p\" (UID: \"e55f447c-ad64-482b-a86b-0588bddf7b4d\") " pod="openshift-console-operator/console-operator-7866497494-drs2p" Jul 16 17:16:06.617022 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:16:06.617005 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/91a757db-58f1-4483-baf4-9b936e4b8072-serving-cert\") pod \"kube-storage-version-migrator-operator-799cb7f958-8q957\" (UID: \"91a757db-58f1-4483-baf4-9b936e4b8072\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-8q957" Jul 16 17:16:06.634950 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:16:06.634923 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bcj7k\" (UniqueName: \"kubernetes.io/projected/91a757db-58f1-4483-baf4-9b936e4b8072-kube-api-access-bcj7k\") pod \"kube-storage-version-migrator-operator-799cb7f958-8q957\" (UID: \"91a757db-58f1-4483-baf4-9b936e4b8072\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-8q957" Jul 16 17:16:06.637381 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:16:06.637357 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9j7qr\" (UniqueName: \"kubernetes.io/projected/e55f447c-ad64-482b-a86b-0588bddf7b4d-kube-api-access-9j7qr\") pod \"console-operator-7866497494-drs2p\" (UID: \"e55f447c-ad64-482b-a86b-0588bddf7b4d\") " pod="openshift-console-operator/console-operator-7866497494-drs2p" Jul 16 17:16:06.637817 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:16:06.637798 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-c8sn4\" (UniqueName: \"kubernetes.io/projected/609ca387-5fdd-4c2d-829e-db5431ddb87f-kube-api-access-c8sn4\") pod \"volume-data-source-validator-78495597f6-nqvsk\" (UID: \"609ca387-5fdd-4c2d-829e-db5431ddb87f\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-78495597f6-nqvsk" Jul 16 17:16:06.651717 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:16:06.651686 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca-operator/service-ca-operator-7695fc6c8c-hvxkc" Jul 16 17:16:06.658664 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:16:06.658643 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-storage-operator/volume-data-source-validator-78495597f6-nqvsk" Jul 16 17:16:06.665550 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:16:06.665530 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-8q957" Jul 16 17:16:06.670506 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:16:06.670474 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console-operator/console-operator-7866497494-drs2p" Jul 16 17:16:06.839315 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:16:06.839284 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-78495597f6-nqvsk"] Jul 16 17:16:06.840095 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:16:06.840064 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca-operator/service-ca-operator-7695fc6c8c-hvxkc"] Jul 16 17:16:06.840800 ip-10-0-132-130 kubenswrapper[2578]: W0716 17:16:06.840774 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod14f6cc76_f1b6_4d19_8bc0_74aa1e1665c7.slice/crio-8eb11036e486fb7594ea4d185c9020f4649979b42abc4f7007352c91e9ab6769 WatchSource:0}: Error finding container 8eb11036e486fb7594ea4d185c9020f4649979b42abc4f7007352c91e9ab6769: Status 404 returned error can't find the container with id 8eb11036e486fb7594ea4d185c9020f4649979b42abc4f7007352c91e9ab6769 Jul 16 17:16:06.841328 ip-10-0-132-130 kubenswrapper[2578]: W0716 17:16:06.841233 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod609ca387_5fdd_4c2d_829e_db5431ddb87f.slice/crio-9c30e1fbe8978e5cd66ee6a56a413ce04ddd3eb91f95c70ea7adacbde6f9a7de WatchSource:0}: Error finding container 9c30e1fbe8978e5cd66ee6a56a413ce04ddd3eb91f95c70ea7adacbde6f9a7de: Status 404 returned error can't find the container with id 9c30e1fbe8978e5cd66ee6a56a413ce04ddd3eb91f95c70ea7adacbde6f9a7de Jul 16 17:16:06.856025 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:16:06.856006 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-8q957"] Jul 16 17:16:06.858388 ip-10-0-132-130 kubenswrapper[2578]: W0716 17:16:06.858367 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod91a757db_58f1_4483_baf4_9b936e4b8072.slice/crio-02021bb6ac49819562e0bb9d7049d4d6db12feddf49016d350ad18a14c7e830b WatchSource:0}: Error finding container 02021bb6ac49819562e0bb9d7049d4d6db12feddf49016d350ad18a14c7e830b: Status 404 returned error can't find the container with id 02021bb6ac49819562e0bb9d7049d4d6db12feddf49016d350ad18a14c7e830b Jul 16 17:16:06.868039 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:16:06.867988 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console-operator/console-operator-7866497494-drs2p"] Jul 16 17:16:06.871861 ip-10-0-132-130 kubenswrapper[2578]: W0716 17:16:06.871842 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode55f447c_ad64_482b_a86b_0588bddf7b4d.slice/crio-4a20907841a0a1adb75fd5f60339d7f31cb709ca673b69dc32c91fece94ee7e8 WatchSource:0}: Error finding container 4a20907841a0a1adb75fd5f60339d7f31cb709ca673b69dc32c91fece94ee7e8: Status 404 returned error can't find the container with id 4a20907841a0a1adb75fd5f60339d7f31cb709ca673b69dc32c91fece94ee7e8 Jul 16 17:16:06.918481 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:16:06.918456 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/8575b8ff-3e17-4577-a026-adef0cc639e0-samples-operator-tls\") pod \"cluster-samples-operator-65db858894-msl7k\" (UID: \"8575b8ff-3e17-4577-a026-adef0cc639e0\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-msl7k" Jul 16 17:16:06.918586 ip-10-0-132-130 kubenswrapper[2578]: E0716 17:16:06.918571 2578 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Jul 16 17:16:06.918648 ip-10-0-132-130 kubenswrapper[2578]: E0716 17:16:06.918630 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8575b8ff-3e17-4577-a026-adef0cc639e0-samples-operator-tls podName:8575b8ff-3e17-4577-a026-adef0cc639e0 nodeName:}" failed. No retries permitted until 2026-07-16 17:16:07.918613228 +0000 UTC m=+112.404457110 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/8575b8ff-3e17-4577-a026-adef0cc639e0-samples-operator-tls") pod "cluster-samples-operator-65db858894-msl7k" (UID: "8575b8ff-3e17-4577-a026-adef0cc639e0") : secret "samples-operator-tls" not found Jul 16 17:16:07.524236 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:16:07.524077 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/dbd1c30a-e583-4ecd-b5c6-ec71f3579e50-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-5czbw\" (UID: \"dbd1c30a-e583-4ecd-b5c6-ec71f3579e50\") " pod="openshift-insights/insights-runtime-extractor-5czbw" Jul 16 17:16:07.524236 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:16:07.524168 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/dbd1c30a-e583-4ecd-b5c6-ec71f3579e50-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-5czbw\" (UID: \"dbd1c30a-e583-4ecd-b5c6-ec71f3579e50\") " pod="openshift-insights/insights-runtime-extractor-5czbw" Jul 16 17:16:07.524236 ip-10-0-132-130 kubenswrapper[2578]: E0716 17:16:07.524193 2578 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 16 17:16:07.524517 ip-10-0-132-130 kubenswrapper[2578]: E0716 17:16:07.524262 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/dbd1c30a-e583-4ecd-b5c6-ec71f3579e50-kube-rbac-proxy-cm podName:dbd1c30a-e583-4ecd-b5c6-ec71f3579e50 nodeName:}" failed. No retries permitted until 2026-07-16 17:16:09.524243014 +0000 UTC m=+114.010086885 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/dbd1c30a-e583-4ecd-b5c6-ec71f3579e50-kube-rbac-proxy-cm") pod "insights-runtime-extractor-5czbw" (UID: "dbd1c30a-e583-4ecd-b5c6-ec71f3579e50") : configmap "kube-rbac-proxy" not found Jul 16 17:16:07.524517 ip-10-0-132-130 kubenswrapper[2578]: E0716 17:16:07.524342 2578 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 16 17:16:07.524517 ip-10-0-132-130 kubenswrapper[2578]: E0716 17:16:07.524406 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/dbd1c30a-e583-4ecd-b5c6-ec71f3579e50-insights-runtime-extractor-tls podName:dbd1c30a-e583-4ecd-b5c6-ec71f3579e50 nodeName:}" failed. No retries permitted until 2026-07-16 17:16:09.524387052 +0000 UTC m=+114.010230922 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/dbd1c30a-e583-4ecd-b5c6-ec71f3579e50-insights-runtime-extractor-tls") pod "insights-runtime-extractor-5czbw" (UID: "dbd1c30a-e583-4ecd-b5c6-ec71f3579e50") : secret "insights-runtime-extractor-tls" not found Jul 16 17:16:07.567004 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:16:07.566970 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-7695fc6c8c-hvxkc" event={"ID":"14f6cc76-f1b6-4d19-8bc0-74aa1e1665c7","Type":"ContainerStarted","Data":"8eb11036e486fb7594ea4d185c9020f4649979b42abc4f7007352c91e9ab6769"} Jul 16 17:16:07.568418 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:16:07.568390 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-7866497494-drs2p" event={"ID":"e55f447c-ad64-482b-a86b-0588bddf7b4d","Type":"ContainerStarted","Data":"4a20907841a0a1adb75fd5f60339d7f31cb709ca673b69dc32c91fece94ee7e8"} Jul 16 17:16:07.571968 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:16:07.571938 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-storage-operator/volume-data-source-validator-78495597f6-nqvsk" event={"ID":"609ca387-5fdd-4c2d-829e-db5431ddb87f","Type":"ContainerStarted","Data":"9c30e1fbe8978e5cd66ee6a56a413ce04ddd3eb91f95c70ea7adacbde6f9a7de"} Jul 16 17:16:07.574484 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:16:07.574434 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-8q957" event={"ID":"91a757db-58f1-4483-baf4-9b936e4b8072","Type":"ContainerStarted","Data":"02021bb6ac49819562e0bb9d7049d4d6db12feddf49016d350ad18a14c7e830b"} Jul 16 17:16:07.927972 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:16:07.927936 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/8575b8ff-3e17-4577-a026-adef0cc639e0-samples-operator-tls\") pod \"cluster-samples-operator-65db858894-msl7k\" (UID: \"8575b8ff-3e17-4577-a026-adef0cc639e0\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-msl7k" Jul 16 17:16:07.928467 ip-10-0-132-130 kubenswrapper[2578]: E0716 17:16:07.928157 2578 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Jul 16 17:16:07.928467 ip-10-0-132-130 kubenswrapper[2578]: E0716 17:16:07.928239 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8575b8ff-3e17-4577-a026-adef0cc639e0-samples-operator-tls podName:8575b8ff-3e17-4577-a026-adef0cc639e0 nodeName:}" failed. No retries permitted until 2026-07-16 17:16:09.928218704 +0000 UTC m=+114.414062585 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/8575b8ff-3e17-4577-a026-adef0cc639e0-samples-operator-tls") pod "cluster-samples-operator-65db858894-msl7k" (UID: "8575b8ff-3e17-4577-a026-adef0cc639e0") : secret "samples-operator-tls" not found Jul 16 17:16:07.960648 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:16:07.960621 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-zw2xf_ff79255c-4f85-4600-a7d2-54534f299e29/dns-node-resolver/0.log" Jul 16 17:16:08.565999 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:16:08.565962 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-lfsvs_dfca47c1-7bfb-4111-a283-7b86affa45a7/node-ca/0.log" Jul 16 17:16:08.580257 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:16:08.580186 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-storage-operator/volume-data-source-validator-78495597f6-nqvsk" event={"ID":"609ca387-5fdd-4c2d-829e-db5431ddb87f","Type":"ContainerStarted","Data":"58aa53510cc13cd7ce7a81a90e8f5da504f2528866069a421a28e5349146e623"} Jul 16 17:16:08.601372 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:16:08.601301 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-storage-operator/volume-data-source-validator-78495597f6-nqvsk" podStartSLOduration=1.38404064 podStartE2EDuration="2.601280654s" totalImagesPullingTime="1.217240014s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 17:16:06 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 17:16:08.601095518 +0000 UTC m=+113.086939407" watchObservedRunningTime="2026-07-16 17:16:08.601280654 +0000 UTC m=+113.087124541" Jul 16 17:16:09.544650 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:16:09.544607 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/dbd1c30a-e583-4ecd-b5c6-ec71f3579e50-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-5czbw\" (UID: \"dbd1c30a-e583-4ecd-b5c6-ec71f3579e50\") " pod="openshift-insights/insights-runtime-extractor-5czbw" Jul 16 17:16:09.545099 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:16:09.544722 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/dbd1c30a-e583-4ecd-b5c6-ec71f3579e50-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-5czbw\" (UID: \"dbd1c30a-e583-4ecd-b5c6-ec71f3579e50\") " pod="openshift-insights/insights-runtime-extractor-5czbw" Jul 16 17:16:09.545099 ip-10-0-132-130 kubenswrapper[2578]: E0716 17:16:09.544796 2578 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 16 17:16:09.545099 ip-10-0-132-130 kubenswrapper[2578]: E0716 17:16:09.544811 2578 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 16 17:16:09.545099 ip-10-0-132-130 kubenswrapper[2578]: E0716 17:16:09.544884 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/dbd1c30a-e583-4ecd-b5c6-ec71f3579e50-kube-rbac-proxy-cm podName:dbd1c30a-e583-4ecd-b5c6-ec71f3579e50 nodeName:}" failed. No retries permitted until 2026-07-16 17:16:13.544867295 +0000 UTC m=+118.030711162 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/dbd1c30a-e583-4ecd-b5c6-ec71f3579e50-kube-rbac-proxy-cm") pod "insights-runtime-extractor-5czbw" (UID: "dbd1c30a-e583-4ecd-b5c6-ec71f3579e50") : configmap "kube-rbac-proxy" not found Jul 16 17:16:09.545099 ip-10-0-132-130 kubenswrapper[2578]: E0716 17:16:09.544899 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/dbd1c30a-e583-4ecd-b5c6-ec71f3579e50-insights-runtime-extractor-tls podName:dbd1c30a-e583-4ecd-b5c6-ec71f3579e50 nodeName:}" failed. No retries permitted until 2026-07-16 17:16:13.544893281 +0000 UTC m=+118.030737149 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/dbd1c30a-e583-4ecd-b5c6-ec71f3579e50-insights-runtime-extractor-tls") pod "insights-runtime-extractor-5czbw" (UID: "dbd1c30a-e583-4ecd-b5c6-ec71f3579e50") : secret "insights-runtime-extractor-tls" not found Jul 16 17:16:09.560112 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:16:09.560086 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-p2d6d_f7ea4735-cb68-4db2-a4ed-b4395fe7e9f6/serve-healthcheck-canary/0.log" Jul 16 17:16:09.948309 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:16:09.948216 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/8575b8ff-3e17-4577-a026-adef0cc639e0-samples-operator-tls\") pod \"cluster-samples-operator-65db858894-msl7k\" (UID: \"8575b8ff-3e17-4577-a026-adef0cc639e0\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-msl7k" Jul 16 17:16:09.948482 ip-10-0-132-130 kubenswrapper[2578]: E0716 17:16:09.948388 2578 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Jul 16 17:16:09.948482 ip-10-0-132-130 kubenswrapper[2578]: E0716 17:16:09.948481 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8575b8ff-3e17-4577-a026-adef0cc639e0-samples-operator-tls podName:8575b8ff-3e17-4577-a026-adef0cc639e0 nodeName:}" failed. No retries permitted until 2026-07-16 17:16:13.948460574 +0000 UTC m=+118.434304443 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/8575b8ff-3e17-4577-a026-adef0cc639e0-samples-operator-tls") pod "cluster-samples-operator-65db858894-msl7k" (UID: "8575b8ff-3e17-4577-a026-adef0cc639e0") : secret "samples-operator-tls" not found Jul 16 17:16:10.588843 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:16:10.588801 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-8q957" event={"ID":"91a757db-58f1-4483-baf4-9b936e4b8072","Type":"ContainerStarted","Data":"246793b0185a86e70e543ba0a03280bee39480ea0ba8da5ca7103a2ae4fd86dd"} Jul 16 17:16:10.590356 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:16:10.590323 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-7695fc6c8c-hvxkc" event={"ID":"14f6cc76-f1b6-4d19-8bc0-74aa1e1665c7","Type":"ContainerStarted","Data":"2071a100c20519130d7c9944ea4c4658a02a13d320413f747e8727bdf5043229"} Jul 16 17:16:10.591926 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:16:10.591905 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-drs2p_e55f447c-ad64-482b-a86b-0588bddf7b4d/console-operator/0.log" Jul 16 17:16:10.592008 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:16:10.591946 2578 generic.go:356] "Generic (PLEG): container finished" podID="e55f447c-ad64-482b-a86b-0588bddf7b4d" containerID="02be2dee3c763bc28adc111175d028a770da1b7fc55864f47ceee6979273e1a4" exitCode=255 Jul 16 17:16:10.592008 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:16:10.591989 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-7866497494-drs2p" event={"ID":"e55f447c-ad64-482b-a86b-0588bddf7b4d","Type":"ContainerDied","Data":"02be2dee3c763bc28adc111175d028a770da1b7fc55864f47ceee6979273e1a4"} Jul 16 17:16:10.592207 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:16:10.592195 2578 scope.go:117] "RemoveContainer" containerID="02be2dee3c763bc28adc111175d028a770da1b7fc55864f47ceee6979273e1a4" Jul 16 17:16:10.611401 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:16:10.611358 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-8q957" podStartSLOduration=1.420742592 podStartE2EDuration="4.611340594s" totalImagesPullingTime="3.190598002s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 17:16:06 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 17:16:10.610160846 +0000 UTC m=+115.096004737" watchObservedRunningTime="2026-07-16 17:16:10.611340594 +0000 UTC m=+115.097184486" Jul 16 17:16:10.650478 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:16:10.650439 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-service-ca-operator/service-ca-operator-7695fc6c8c-hvxkc" podStartSLOduration=1.444964581 podStartE2EDuration="4.650423478s" totalImagesPullingTime="3.205458897s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 17:16:06 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 17:16:10.649725469 +0000 UTC m=+115.135569349" watchObservedRunningTime="2026-07-16 17:16:10.650423478 +0000 UTC m=+115.136267371" Jul 16 17:16:11.598302 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:16:11.598276 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-drs2p_e55f447c-ad64-482b-a86b-0588bddf7b4d/console-operator/1.log" Jul 16 17:16:11.598685 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:16:11.598670 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-drs2p_e55f447c-ad64-482b-a86b-0588bddf7b4d/console-operator/0.log" Jul 16 17:16:11.598754 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:16:11.598723 2578 generic.go:356] "Generic (PLEG): container finished" podID="e55f447c-ad64-482b-a86b-0588bddf7b4d" containerID="bef7964849d953f9f4286dae42d0ac9c9e652edca1f01a0c9351e4cce7e1da5f" exitCode=255 Jul 16 17:16:11.598815 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:16:11.598792 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-7866497494-drs2p" event={"ID":"e55f447c-ad64-482b-a86b-0588bddf7b4d","Type":"ContainerDied","Data":"bef7964849d953f9f4286dae42d0ac9c9e652edca1f01a0c9351e4cce7e1da5f"} Jul 16 17:16:11.599590 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:16:11.599574 2578 scope.go:117] "RemoveContainer" containerID="02be2dee3c763bc28adc111175d028a770da1b7fc55864f47ceee6979273e1a4" Jul 16 17:16:11.599686 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:16:11.599672 2578 scope.go:117] "RemoveContainer" containerID="bef7964849d953f9f4286dae42d0ac9c9e652edca1f01a0c9351e4cce7e1da5f" Jul 16 17:16:11.599842 ip-10-0-132-130 kubenswrapper[2578]: E0716 17:16:11.599826 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-7866497494-drs2p_openshift-console-operator(e55f447c-ad64-482b-a86b-0588bddf7b4d)\"" pod="openshift-console-operator/console-operator-7866497494-drs2p" podUID="e55f447c-ad64-482b-a86b-0588bddf7b4d" Jul 16 17:16:12.063393 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:16:12.063357 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/5bf41cfb-4cb4-4953-a458-9211602c6b74-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-5f7f494dd4-p872b\" (UID: \"5bf41cfb-4cb4-4953-a458-9211602c6b74\") " pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-p872b" Jul 16 17:16:12.063567 ip-10-0-132-130 kubenswrapper[2578]: E0716 17:16:12.063514 2578 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Jul 16 17:16:12.063609 ip-10-0-132-130 kubenswrapper[2578]: E0716 17:16:12.063580 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5bf41cfb-4cb4-4953-a458-9211602c6b74-cluster-monitoring-operator-tls podName:5bf41cfb-4cb4-4953-a458-9211602c6b74 nodeName:}" failed. No retries permitted until 2026-07-16 17:16:28.063564412 +0000 UTC m=+132.549408281 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/5bf41cfb-4cb4-4953-a458-9211602c6b74-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-5f7f494dd4-p872b" (UID: "5bf41cfb-4cb4-4953-a458-9211602c6b74") : secret "cluster-monitoring-operator-tls" not found Jul 16 17:16:12.603295 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:16:12.603270 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-drs2p_e55f447c-ad64-482b-a86b-0588bddf7b4d/console-operator/1.log" Jul 16 17:16:12.603660 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:16:12.603596 2578 scope.go:117] "RemoveContainer" containerID="bef7964849d953f9f4286dae42d0ac9c9e652edca1f01a0c9351e4cce7e1da5f" Jul 16 17:16:12.603791 ip-10-0-132-130 kubenswrapper[2578]: E0716 17:16:12.603771 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-7866497494-drs2p_openshift-console-operator(e55f447c-ad64-482b-a86b-0588bddf7b4d)\"" pod="openshift-console-operator/console-operator-7866497494-drs2p" podUID="e55f447c-ad64-482b-a86b-0588bddf7b4d" Jul 16 17:16:13.574791 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:16:13.574757 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/dbd1c30a-e583-4ecd-b5c6-ec71f3579e50-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-5czbw\" (UID: \"dbd1c30a-e583-4ecd-b5c6-ec71f3579e50\") " pod="openshift-insights/insights-runtime-extractor-5czbw" Jul 16 17:16:13.574950 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:16:13.574851 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/dbd1c30a-e583-4ecd-b5c6-ec71f3579e50-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-5czbw\" (UID: \"dbd1c30a-e583-4ecd-b5c6-ec71f3579e50\") " pod="openshift-insights/insights-runtime-extractor-5czbw" Jul 16 17:16:13.574950 ip-10-0-132-130 kubenswrapper[2578]: E0716 17:16:13.574908 2578 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 16 17:16:13.575036 ip-10-0-132-130 kubenswrapper[2578]: E0716 17:16:13.574947 2578 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 16 17:16:13.575036 ip-10-0-132-130 kubenswrapper[2578]: E0716 17:16:13.574971 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/dbd1c30a-e583-4ecd-b5c6-ec71f3579e50-insights-runtime-extractor-tls podName:dbd1c30a-e583-4ecd-b5c6-ec71f3579e50 nodeName:}" failed. No retries permitted until 2026-07-16 17:16:21.574955929 +0000 UTC m=+126.060799800 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/dbd1c30a-e583-4ecd-b5c6-ec71f3579e50-insights-runtime-extractor-tls") pod "insights-runtime-extractor-5czbw" (UID: "dbd1c30a-e583-4ecd-b5c6-ec71f3579e50") : secret "insights-runtime-extractor-tls" not found Jul 16 17:16:13.575036 ip-10-0-132-130 kubenswrapper[2578]: E0716 17:16:13.574996 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/dbd1c30a-e583-4ecd-b5c6-ec71f3579e50-kube-rbac-proxy-cm podName:dbd1c30a-e583-4ecd-b5c6-ec71f3579e50 nodeName:}" failed. No retries permitted until 2026-07-16 17:16:21.574982448 +0000 UTC m=+126.060826314 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/dbd1c30a-e583-4ecd-b5c6-ec71f3579e50-kube-rbac-proxy-cm") pod "insights-runtime-extractor-5czbw" (UID: "dbd1c30a-e583-4ecd-b5c6-ec71f3579e50") : configmap "kube-rbac-proxy" not found Jul 16 17:16:13.977669 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:16:13.977585 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/8575b8ff-3e17-4577-a026-adef0cc639e0-samples-operator-tls\") pod \"cluster-samples-operator-65db858894-msl7k\" (UID: \"8575b8ff-3e17-4577-a026-adef0cc639e0\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-msl7k" Jul 16 17:16:13.978029 ip-10-0-132-130 kubenswrapper[2578]: E0716 17:16:13.977733 2578 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Jul 16 17:16:13.978029 ip-10-0-132-130 kubenswrapper[2578]: E0716 17:16:13.977796 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8575b8ff-3e17-4577-a026-adef0cc639e0-samples-operator-tls podName:8575b8ff-3e17-4577-a026-adef0cc639e0 nodeName:}" failed. No retries permitted until 2026-07-16 17:16:21.977778413 +0000 UTC m=+126.463622301 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/8575b8ff-3e17-4577-a026-adef0cc639e0-samples-operator-tls") pod "cluster-samples-operator-65db858894-msl7k" (UID: "8575b8ff-3e17-4577-a026-adef0cc639e0") : secret "samples-operator-tls" not found Jul 16 17:16:16.671591 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:16:16.671556 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console-operator/console-operator-7866497494-drs2p" Jul 16 17:16:16.671996 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:16:16.671608 2578 kubelet.go:2654] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="openshift-console-operator/console-operator-7866497494-drs2p" Jul 16 17:16:16.672038 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:16:16.671997 2578 scope.go:117] "RemoveContainer" containerID="bef7964849d953f9f4286dae42d0ac9c9e652edca1f01a0c9351e4cce7e1da5f" Jul 16 17:16:16.672189 ip-10-0-132-130 kubenswrapper[2578]: E0716 17:16:16.672169 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-7866497494-drs2p_openshift-console-operator(e55f447c-ad64-482b-a86b-0588bddf7b4d)\"" pod="openshift-console-operator/console-operator-7866497494-drs2p" podUID="e55f447c-ad64-482b-a86b-0588bddf7b4d" Jul 16 17:16:21.634891 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:16:21.634859 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/dbd1c30a-e583-4ecd-b5c6-ec71f3579e50-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-5czbw\" (UID: \"dbd1c30a-e583-4ecd-b5c6-ec71f3579e50\") " pod="openshift-insights/insights-runtime-extractor-5czbw" Jul 16 17:16:21.635277 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:16:21.634992 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/dbd1c30a-e583-4ecd-b5c6-ec71f3579e50-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-5czbw\" (UID: \"dbd1c30a-e583-4ecd-b5c6-ec71f3579e50\") " pod="openshift-insights/insights-runtime-extractor-5czbw" Jul 16 17:16:21.635277 ip-10-0-132-130 kubenswrapper[2578]: E0716 17:16:21.635002 2578 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 16 17:16:21.635277 ip-10-0-132-130 kubenswrapper[2578]: E0716 17:16:21.635057 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/dbd1c30a-e583-4ecd-b5c6-ec71f3579e50-insights-runtime-extractor-tls podName:dbd1c30a-e583-4ecd-b5c6-ec71f3579e50 nodeName:}" failed. No retries permitted until 2026-07-16 17:16:37.635041648 +0000 UTC m=+142.120885514 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/dbd1c30a-e583-4ecd-b5c6-ec71f3579e50-insights-runtime-extractor-tls") pod "insights-runtime-extractor-5czbw" (UID: "dbd1c30a-e583-4ecd-b5c6-ec71f3579e50") : secret "insights-runtime-extractor-tls" not found Jul 16 17:16:21.635424 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:16:21.635409 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/dbd1c30a-e583-4ecd-b5c6-ec71f3579e50-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-5czbw\" (UID: \"dbd1c30a-e583-4ecd-b5c6-ec71f3579e50\") " pod="openshift-insights/insights-runtime-extractor-5czbw" Jul 16 17:16:22.038216 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:16:22.038134 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/8575b8ff-3e17-4577-a026-adef0cc639e0-samples-operator-tls\") pod \"cluster-samples-operator-65db858894-msl7k\" (UID: \"8575b8ff-3e17-4577-a026-adef0cc639e0\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-msl7k" Jul 16 17:16:22.040464 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:16:22.040429 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/8575b8ff-3e17-4577-a026-adef0cc639e0-samples-operator-tls\") pod \"cluster-samples-operator-65db858894-msl7k\" (UID: \"8575b8ff-3e17-4577-a026-adef0cc639e0\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-msl7k" Jul 16 17:16:22.136164 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:16:22.136138 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-samples-operator\"/\"cluster-samples-operator-dockercfg-f4bbl\"" Jul 16 17:16:22.144270 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:16:22.144254 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-msl7k" Jul 16 17:16:22.271773 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:16:22.271746 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-65db858894-msl7k"] Jul 16 17:16:22.633834 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:16:22.633741 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-msl7k" event={"ID":"8575b8ff-3e17-4577-a026-adef0cc639e0","Type":"ContainerStarted","Data":"b3314933a53bb3da459038ddb23ed463559a9dfa597813e8c87f833615c692ad"} Jul 16 17:16:25.761116 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:16:25.761089 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f90822ad-ab14-4108-8294-2e76c35a8aa5-metrics-certs\") pod \"network-metrics-daemon-kvr2r\" (UID: \"f90822ad-ab14-4108-8294-2e76c35a8aa5\") " pod="openshift-multus/network-metrics-daemon-kvr2r" Jul 16 17:16:25.761535 ip-10-0-132-130 kubenswrapper[2578]: E0716 17:16:25.761259 2578 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found Jul 16 17:16:25.761535 ip-10-0-132-130 kubenswrapper[2578]: E0716 17:16:25.761334 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f90822ad-ab14-4108-8294-2e76c35a8aa5-metrics-certs podName:f90822ad-ab14-4108-8294-2e76c35a8aa5 nodeName:}" failed. No retries permitted until 2026-07-16 17:18:27.76131276 +0000 UTC m=+252.247156626 (durationBeforeRetry 2m2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/f90822ad-ab14-4108-8294-2e76c35a8aa5-metrics-certs") pod "network-metrics-daemon-kvr2r" (UID: "f90822ad-ab14-4108-8294-2e76c35a8aa5") : secret "metrics-daemon-secret" not found Jul 16 17:16:26.645888 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:16:26.645858 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-msl7k" event={"ID":"8575b8ff-3e17-4577-a026-adef0cc639e0","Type":"ContainerStarted","Data":"d60abfc66cbdef3744d146b8ecaa68c2629d6970fd0c26f1dbb4d450b83051c8"} Jul 16 17:16:26.646036 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:16:26.645907 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-msl7k" event={"ID":"8575b8ff-3e17-4577-a026-adef0cc639e0","Type":"ContainerStarted","Data":"63646af00c1d4b285e19eadb8f7e14a421f018dd2f1019d1282c34a4b8efd077"} Jul 16 17:16:26.675369 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:16:26.675324 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-msl7k" podStartSLOduration=16.518754997 podStartE2EDuration="20.675309492s" totalImagesPullingTime="4.156554495s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 17:16:06 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 17:16:26.674856063 +0000 UTC m=+131.160699950" watchObservedRunningTime="2026-07-16 17:16:26.675309492 +0000 UTC m=+131.161153380" Jul 16 17:16:28.081432 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:16:28.081395 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/5bf41cfb-4cb4-4953-a458-9211602c6b74-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-5f7f494dd4-p872b\" (UID: \"5bf41cfb-4cb4-4953-a458-9211602c6b74\") " pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-p872b" Jul 16 17:16:28.083705 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:16:28.083669 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/5bf41cfb-4cb4-4953-a458-9211602c6b74-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-5f7f494dd4-p872b\" (UID: \"5bf41cfb-4cb4-4953-a458-9211602c6b74\") " pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-p872b" Jul 16 17:16:28.209967 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:16:28.209935 2578 scope.go:117] "RemoveContainer" containerID="bef7964849d953f9f4286dae42d0ac9c9e652edca1f01a0c9351e4cce7e1da5f" Jul 16 17:16:28.300306 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:16:28.300279 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-dockercfg-gzfzv\"" Jul 16 17:16:28.308721 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:16:28.308671 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-p872b" Jul 16 17:16:28.444363 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:16:28.444330 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-p872b"] Jul 16 17:16:28.447583 ip-10-0-132-130 kubenswrapper[2578]: W0716 17:16:28.447557 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5bf41cfb_4cb4_4953_a458_9211602c6b74.slice/crio-b2fa7dc4e24b3147fb591fa4498fd6463601b6e87c521b10be3572c86ce49323 WatchSource:0}: Error finding container b2fa7dc4e24b3147fb591fa4498fd6463601b6e87c521b10be3572c86ce49323: Status 404 returned error can't find the container with id b2fa7dc4e24b3147fb591fa4498fd6463601b6e87c521b10be3572c86ce49323 Jul 16 17:16:28.653145 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:16:28.653065 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-p872b" event={"ID":"5bf41cfb-4cb4-4953-a458-9211602c6b74","Type":"ContainerStarted","Data":"b2fa7dc4e24b3147fb591fa4498fd6463601b6e87c521b10be3572c86ce49323"} Jul 16 17:16:28.654796 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:16:28.654777 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-drs2p_e55f447c-ad64-482b-a86b-0588bddf7b4d/console-operator/1.log" Jul 16 17:16:28.654894 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:16:28.654839 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-7866497494-drs2p" event={"ID":"e55f447c-ad64-482b-a86b-0588bddf7b4d","Type":"ContainerStarted","Data":"e72e1abd28aa28cdc1a40ff3b36a8673f5d4cc30c78f9816e7ca5184f7f6f6e2"} Jul 16 17:16:28.655124 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:16:28.655098 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console-operator/console-operator-7866497494-drs2p" Jul 16 17:16:28.690018 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:16:28.689972 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console-operator/console-operator-7866497494-drs2p" podStartSLOduration=19.518998414 podStartE2EDuration="22.68995945s" totalImagesPullingTime="3.170961036s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 17:16:06 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 17:16:28.6896866 +0000 UTC m=+133.175530491" watchObservedRunningTime="2026-07-16 17:16:28.68995945 +0000 UTC m=+133.175803339" Jul 16 17:16:28.805907 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:16:28.805879 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console-operator/console-operator-7866497494-drs2p" Jul 16 17:16:30.664016 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:16:30.663975 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-p872b" event={"ID":"5bf41cfb-4cb4-4953-a458-9211602c6b74","Type":"ContainerStarted","Data":"992dd2d95c5a74df5c02adda16c86c5c43c56e9239b1fe1309515557c2e40221"} Jul 16 17:16:30.694370 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:16:30.694324 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-p872b" podStartSLOduration=32.629627277 podStartE2EDuration="34.694309684s" totalImagesPullingTime="2.064682407s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 17:15:56 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 17:16:30.693415094 +0000 UTC m=+135.179258984" watchObservedRunningTime="2026-07-16 17:16:30.694309684 +0000 UTC m=+135.180153573" Jul 16 17:16:37.655738 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:16:37.655701 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/dbd1c30a-e583-4ecd-b5c6-ec71f3579e50-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-5czbw\" (UID: \"dbd1c30a-e583-4ecd-b5c6-ec71f3579e50\") " pod="openshift-insights/insights-runtime-extractor-5czbw" Jul 16 17:16:37.658020 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:16:37.657989 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/dbd1c30a-e583-4ecd-b5c6-ec71f3579e50-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-5czbw\" (UID: \"dbd1c30a-e583-4ecd-b5c6-ec71f3579e50\") " pod="openshift-insights/insights-runtime-extractor-5czbw" Jul 16 17:16:37.949543 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:16:37.949466 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-t9pcw\"" Jul 16 17:16:37.955239 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:16:37.955221 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-5czbw" Jul 16 17:16:38.083578 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:16:38.083548 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-5czbw"] Jul 16 17:16:38.086435 ip-10-0-132-130 kubenswrapper[2578]: W0716 17:16:38.086412 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddbd1c30a_e583_4ecd_b5c6_ec71f3579e50.slice/crio-ee4245e2fc704a548529ce61d48b8fa36ff8225b2e474ef704c87108574371a0 WatchSource:0}: Error finding container ee4245e2fc704a548529ce61d48b8fa36ff8225b2e474ef704c87108574371a0: Status 404 returned error can't find the container with id ee4245e2fc704a548529ce61d48b8fa36ff8225b2e474ef704c87108574371a0 Jul 16 17:16:38.688860 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:16:38.688821 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-5czbw" event={"ID":"dbd1c30a-e583-4ecd-b5c6-ec71f3579e50","Type":"ContainerStarted","Data":"ee4245e2fc704a548529ce61d48b8fa36ff8225b2e474ef704c87108574371a0"} Jul 16 17:16:40.004792 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:16:40.004763 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-network-diagnostics/network-check-source-869dcfcdcd-bp4xd"] Jul 16 17:16:40.007997 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:16:40.007977 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-source-869dcfcdcd-bp4xd" Jul 16 17:16:40.010803 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:16:40.010775 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"network-diagnostics-dockercfg-jx6s9\"" Jul 16 17:16:40.019132 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:16:40.019109 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-source-869dcfcdcd-bp4xd"] Jul 16 17:16:40.074214 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:16:40.074181 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zx2lf\" (UniqueName: \"kubernetes.io/projected/7d388c86-ce98-4659-a1df-ddd1bf711eaa-kube-api-access-zx2lf\") pod \"network-check-source-869dcfcdcd-bp4xd\" (UID: \"7d388c86-ce98-4659-a1df-ddd1bf711eaa\") " pod="openshift-network-diagnostics/network-check-source-869dcfcdcd-bp4xd" Jul 16 17:16:40.124038 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:16:40.124009 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-77d98bc786-87596"] Jul 16 17:16:40.127213 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:16:40.127191 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-admission-webhook-77d98bc786-87596" Jul 16 17:16:40.130720 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:16:40.130628 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/downloads-68c9cbb497-lrlw4"] Jul 16 17:16:40.132825 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:16:40.132802 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-admission-webhook-tls\"" Jul 16 17:16:40.133658 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:16:40.133638 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-54565489c9-dcjs7"] Jul 16 17:16:40.133940 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:16:40.133794 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/downloads-68c9cbb497-lrlw4" Jul 16 17:16:40.134456 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:16:40.134435 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-admission-webhook-dockercfg-7b9k9\"" Jul 16 17:16:40.136670 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:16:40.136653 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-54565489c9-dcjs7" Jul 16 17:16:40.139284 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:16:40.139266 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"kube-root-ca.crt\"" Jul 16 17:16:40.140392 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:16:40.140370 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"openshift-service-ca.crt\"" Jul 16 17:16:40.167660 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:16:40.167602 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"oauth-serving-cert\"" Jul 16 17:16:40.167660 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:16:40.167633 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-oauth-config\"" Jul 16 17:16:40.167859 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:16:40.167674 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"service-ca\"" Jul 16 17:16:40.168221 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:16:40.168204 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"console-config\"" Jul 16 17:16:40.168579 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:16:40.168466 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"default-dockercfg-6zb75\"" Jul 16 17:16:40.168742 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:16:40.168682 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-dockercfg-z2nks\"" Jul 16 17:16:40.168932 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:16:40.168908 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-serving-cert\"" Jul 16 17:16:40.170271 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:16:40.170241 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-77d98bc786-87596"] Jul 16 17:16:40.171337 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:16:40.171317 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-ff66d4bb5-czxjw"] Jul 16 17:16:40.174519 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:16:40.174498 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-54565489c9-dcjs7"] Jul 16 17:16:40.174647 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:16:40.174604 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-ff66d4bb5-czxjw" Jul 16 17:16:40.174744 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:16:40.174711 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/d4cbd1db-79c1-4af2-9866-53b8aeee1bec-console-serving-cert\") pod \"console-54565489c9-dcjs7\" (UID: \"d4cbd1db-79c1-4af2-9866-53b8aeee1bec\") " pod="openshift-console/console-54565489c9-dcjs7" Jul 16 17:16:40.174831 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:16:40.174797 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/d4cbd1db-79c1-4af2-9866-53b8aeee1bec-console-config\") pod \"console-54565489c9-dcjs7\" (UID: \"d4cbd1db-79c1-4af2-9866-53b8aeee1bec\") " pod="openshift-console/console-54565489c9-dcjs7" Jul 16 17:16:40.174914 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:16:40.174834 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/d4cbd1db-79c1-4af2-9866-53b8aeee1bec-service-ca\") pod \"console-54565489c9-dcjs7\" (UID: \"d4cbd1db-79c1-4af2-9866-53b8aeee1bec\") " pod="openshift-console/console-54565489c9-dcjs7" Jul 16 17:16:40.174914 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:16:40.174874 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zx2lf\" (UniqueName: \"kubernetes.io/projected/7d388c86-ce98-4659-a1df-ddd1bf711eaa-kube-api-access-zx2lf\") pod \"network-check-source-869dcfcdcd-bp4xd\" (UID: \"7d388c86-ce98-4659-a1df-ddd1bf711eaa\") " pod="openshift-network-diagnostics/network-check-source-869dcfcdcd-bp4xd" Jul 16 17:16:40.175054 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:16:40.174919 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/d4cbd1db-79c1-4af2-9866-53b8aeee1bec-oauth-serving-cert\") pod \"console-54565489c9-dcjs7\" (UID: \"d4cbd1db-79c1-4af2-9866-53b8aeee1bec\") " pod="openshift-console/console-54565489c9-dcjs7" Jul 16 17:16:40.175054 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:16:40.174945 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/bfc93304-acba-424e-b6db-0844667a0803-tls-certificates\") pod \"prometheus-operator-admission-webhook-77d98bc786-87596\" (UID: \"bfc93304-acba-424e-b6db-0844667a0803\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-77d98bc786-87596" Jul 16 17:16:40.175054 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:16:40.174976 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/d4cbd1db-79c1-4af2-9866-53b8aeee1bec-console-oauth-config\") pod \"console-54565489c9-dcjs7\" (UID: \"d4cbd1db-79c1-4af2-9866-53b8aeee1bec\") " pod="openshift-console/console-54565489c9-dcjs7" Jul 16 17:16:40.175054 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:16:40.175003 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tbrzc\" (UniqueName: \"kubernetes.io/projected/fd67d296-3c7d-47e1-89c2-d4e3f8edb1cc-kube-api-access-tbrzc\") pod \"downloads-68c9cbb497-lrlw4\" (UID: \"fd67d296-3c7d-47e1-89c2-d4e3f8edb1cc\") " pod="openshift-console/downloads-68c9cbb497-lrlw4" Jul 16 17:16:40.175263 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:16:40.175094 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vnjf9\" (UniqueName: \"kubernetes.io/projected/d4cbd1db-79c1-4af2-9866-53b8aeee1bec-kube-api-access-vnjf9\") pod \"console-54565489c9-dcjs7\" (UID: \"d4cbd1db-79c1-4af2-9866-53b8aeee1bec\") " pod="openshift-console/console-54565489c9-dcjs7" Jul 16 17:16:40.181890 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:16:40.181839 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/downloads-68c9cbb497-lrlw4"] Jul 16 17:16:40.186359 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:16:40.186336 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-private-configuration\"" Jul 16 17:16:40.186462 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:16:40.186448 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"installation-pull-secrets\"" Jul 16 17:16:40.186658 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:16:40.186640 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-bqh7f\"" Jul 16 17:16:40.186992 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:16:40.186977 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-tls\"" Jul 16 17:16:40.200150 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:16:40.199601 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"trusted-ca\"" Jul 16 17:16:40.202775 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:16:40.202751 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zx2lf\" (UniqueName: \"kubernetes.io/projected/7d388c86-ce98-4659-a1df-ddd1bf711eaa-kube-api-access-zx2lf\") pod \"network-check-source-869dcfcdcd-bp4xd\" (UID: \"7d388c86-ce98-4659-a1df-ddd1bf711eaa\") " pod="openshift-network-diagnostics/network-check-source-869dcfcdcd-bp4xd" Jul 16 17:16:40.207144 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:16:40.207124 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-ff66d4bb5-czxjw"] Jul 16 17:16:40.275940 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:16:40.275873 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/4e1467f4-90b0-436c-a61a-394761bca783-trusted-ca\") pod \"image-registry-ff66d4bb5-czxjw\" (UID: \"4e1467f4-90b0-436c-a61a-394761bca783\") " pod="openshift-image-registry/image-registry-ff66d4bb5-czxjw" Jul 16 17:16:40.275940 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:16:40.275929 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/d4cbd1db-79c1-4af2-9866-53b8aeee1bec-oauth-serving-cert\") pod \"console-54565489c9-dcjs7\" (UID: \"d4cbd1db-79c1-4af2-9866-53b8aeee1bec\") " pod="openshift-console/console-54565489c9-dcjs7" Jul 16 17:16:40.276142 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:16:40.275961 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/bfc93304-acba-424e-b6db-0844667a0803-tls-certificates\") pod \"prometheus-operator-admission-webhook-77d98bc786-87596\" (UID: \"bfc93304-acba-424e-b6db-0844667a0803\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-77d98bc786-87596" Jul 16 17:16:40.276142 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:16:40.275987 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/4e1467f4-90b0-436c-a61a-394761bca783-registry-certificates\") pod \"image-registry-ff66d4bb5-czxjw\" (UID: \"4e1467f4-90b0-436c-a61a-394761bca783\") " pod="openshift-image-registry/image-registry-ff66d4bb5-czxjw" Jul 16 17:16:40.276142 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:16:40.276016 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/d4cbd1db-79c1-4af2-9866-53b8aeee1bec-console-oauth-config\") pod \"console-54565489c9-dcjs7\" (UID: \"d4cbd1db-79c1-4af2-9866-53b8aeee1bec\") " pod="openshift-console/console-54565489c9-dcjs7" Jul 16 17:16:40.276142 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:16:40.276040 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/4e1467f4-90b0-436c-a61a-394761bca783-bound-sa-token\") pod \"image-registry-ff66d4bb5-czxjw\" (UID: \"4e1467f4-90b0-436c-a61a-394761bca783\") " pod="openshift-image-registry/image-registry-ff66d4bb5-czxjw" Jul 16 17:16:40.276142 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:16:40.276065 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tbrzc\" (UniqueName: \"kubernetes.io/projected/fd67d296-3c7d-47e1-89c2-d4e3f8edb1cc-kube-api-access-tbrzc\") pod \"downloads-68c9cbb497-lrlw4\" (UID: \"fd67d296-3c7d-47e1-89c2-d4e3f8edb1cc\") " pod="openshift-console/downloads-68c9cbb497-lrlw4" Jul 16 17:16:40.276142 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:16:40.276091 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vnjf9\" (UniqueName: \"kubernetes.io/projected/d4cbd1db-79c1-4af2-9866-53b8aeee1bec-kube-api-access-vnjf9\") pod \"console-54565489c9-dcjs7\" (UID: \"d4cbd1db-79c1-4af2-9866-53b8aeee1bec\") " pod="openshift-console/console-54565489c9-dcjs7" Jul 16 17:16:40.276142 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:16:40.276129 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/d4cbd1db-79c1-4af2-9866-53b8aeee1bec-console-serving-cert\") pod \"console-54565489c9-dcjs7\" (UID: \"d4cbd1db-79c1-4af2-9866-53b8aeee1bec\") " pod="openshift-console/console-54565489c9-dcjs7" Jul 16 17:16:40.276489 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:16:40.276167 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/4e1467f4-90b0-436c-a61a-394761bca783-installation-pull-secrets\") pod \"image-registry-ff66d4bb5-czxjw\" (UID: \"4e1467f4-90b0-436c-a61a-394761bca783\") " pod="openshift-image-registry/image-registry-ff66d4bb5-czxjw" Jul 16 17:16:40.276489 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:16:40.276233 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/4e1467f4-90b0-436c-a61a-394761bca783-image-registry-private-configuration\") pod \"image-registry-ff66d4bb5-czxjw\" (UID: \"4e1467f4-90b0-436c-a61a-394761bca783\") " pod="openshift-image-registry/image-registry-ff66d4bb5-czxjw" Jul 16 17:16:40.276743 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:16:40.276607 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/d4cbd1db-79c1-4af2-9866-53b8aeee1bec-console-config\") pod \"console-54565489c9-dcjs7\" (UID: \"d4cbd1db-79c1-4af2-9866-53b8aeee1bec\") " pod="openshift-console/console-54565489c9-dcjs7" Jul 16 17:16:40.276743 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:16:40.276647 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/d4cbd1db-79c1-4af2-9866-53b8aeee1bec-service-ca\") pod \"console-54565489c9-dcjs7\" (UID: \"d4cbd1db-79c1-4af2-9866-53b8aeee1bec\") " pod="openshift-console/console-54565489c9-dcjs7" Jul 16 17:16:40.276743 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:16:40.276663 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/d4cbd1db-79c1-4af2-9866-53b8aeee1bec-oauth-serving-cert\") pod \"console-54565489c9-dcjs7\" (UID: \"d4cbd1db-79c1-4af2-9866-53b8aeee1bec\") " pod="openshift-console/console-54565489c9-dcjs7" Jul 16 17:16:40.277019 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:16:40.276755 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/4e1467f4-90b0-436c-a61a-394761bca783-registry-tls\") pod \"image-registry-ff66d4bb5-czxjw\" (UID: \"4e1467f4-90b0-436c-a61a-394761bca783\") " pod="openshift-image-registry/image-registry-ff66d4bb5-czxjw" Jul 16 17:16:40.277019 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:16:40.276796 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/4e1467f4-90b0-436c-a61a-394761bca783-ca-trust-extracted\") pod \"image-registry-ff66d4bb5-czxjw\" (UID: \"4e1467f4-90b0-436c-a61a-394761bca783\") " pod="openshift-image-registry/image-registry-ff66d4bb5-czxjw" Jul 16 17:16:40.277019 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:16:40.276822 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-msfgx\" (UniqueName: \"kubernetes.io/projected/4e1467f4-90b0-436c-a61a-394761bca783-kube-api-access-msfgx\") pod \"image-registry-ff66d4bb5-czxjw\" (UID: \"4e1467f4-90b0-436c-a61a-394761bca783\") " pod="openshift-image-registry/image-registry-ff66d4bb5-czxjw" Jul 16 17:16:40.277350 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:16:40.277329 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/d4cbd1db-79c1-4af2-9866-53b8aeee1bec-service-ca\") pod \"console-54565489c9-dcjs7\" (UID: \"d4cbd1db-79c1-4af2-9866-53b8aeee1bec\") " pod="openshift-console/console-54565489c9-dcjs7" Jul 16 17:16:40.277413 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:16:40.277329 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/d4cbd1db-79c1-4af2-9866-53b8aeee1bec-console-config\") pod \"console-54565489c9-dcjs7\" (UID: \"d4cbd1db-79c1-4af2-9866-53b8aeee1bec\") " pod="openshift-console/console-54565489c9-dcjs7" Jul 16 17:16:40.279231 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:16:40.279160 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/d4cbd1db-79c1-4af2-9866-53b8aeee1bec-console-oauth-config\") pod \"console-54565489c9-dcjs7\" (UID: \"d4cbd1db-79c1-4af2-9866-53b8aeee1bec\") " pod="openshift-console/console-54565489c9-dcjs7" Jul 16 17:16:40.279231 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:16:40.279216 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/d4cbd1db-79c1-4af2-9866-53b8aeee1bec-console-serving-cert\") pod \"console-54565489c9-dcjs7\" (UID: \"d4cbd1db-79c1-4af2-9866-53b8aeee1bec\") " pod="openshift-console/console-54565489c9-dcjs7" Jul 16 17:16:40.279435 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:16:40.279408 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/bfc93304-acba-424e-b6db-0844667a0803-tls-certificates\") pod \"prometheus-operator-admission-webhook-77d98bc786-87596\" (UID: \"bfc93304-acba-424e-b6db-0844667a0803\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-77d98bc786-87596" Jul 16 17:16:40.289866 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:16:40.289814 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tbrzc\" (UniqueName: \"kubernetes.io/projected/fd67d296-3c7d-47e1-89c2-d4e3f8edb1cc-kube-api-access-tbrzc\") pod \"downloads-68c9cbb497-lrlw4\" (UID: \"fd67d296-3c7d-47e1-89c2-d4e3f8edb1cc\") " pod="openshift-console/downloads-68c9cbb497-lrlw4" Jul 16 17:16:40.290224 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:16:40.290178 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vnjf9\" (UniqueName: \"kubernetes.io/projected/d4cbd1db-79c1-4af2-9866-53b8aeee1bec-kube-api-access-vnjf9\") pod \"console-54565489c9-dcjs7\" (UID: \"d4cbd1db-79c1-4af2-9866-53b8aeee1bec\") " pod="openshift-console/console-54565489c9-dcjs7" Jul 16 17:16:40.320065 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:16:40.320036 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-source-869dcfcdcd-bp4xd" Jul 16 17:16:40.377926 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:16:40.377892 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/4e1467f4-90b0-436c-a61a-394761bca783-image-registry-private-configuration\") pod \"image-registry-ff66d4bb5-czxjw\" (UID: \"4e1467f4-90b0-436c-a61a-394761bca783\") " pod="openshift-image-registry/image-registry-ff66d4bb5-czxjw" Jul 16 17:16:40.378071 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:16:40.377960 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/4e1467f4-90b0-436c-a61a-394761bca783-registry-tls\") pod \"image-registry-ff66d4bb5-czxjw\" (UID: \"4e1467f4-90b0-436c-a61a-394761bca783\") " pod="openshift-image-registry/image-registry-ff66d4bb5-czxjw" Jul 16 17:16:40.378071 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:16:40.377989 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/4e1467f4-90b0-436c-a61a-394761bca783-ca-trust-extracted\") pod \"image-registry-ff66d4bb5-czxjw\" (UID: \"4e1467f4-90b0-436c-a61a-394761bca783\") " pod="openshift-image-registry/image-registry-ff66d4bb5-czxjw" Jul 16 17:16:40.378071 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:16:40.378015 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-msfgx\" (UniqueName: \"kubernetes.io/projected/4e1467f4-90b0-436c-a61a-394761bca783-kube-api-access-msfgx\") pod \"image-registry-ff66d4bb5-czxjw\" (UID: \"4e1467f4-90b0-436c-a61a-394761bca783\") " pod="openshift-image-registry/image-registry-ff66d4bb5-czxjw" Jul 16 17:16:40.378071 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:16:40.378050 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/4e1467f4-90b0-436c-a61a-394761bca783-trusted-ca\") pod \"image-registry-ff66d4bb5-czxjw\" (UID: \"4e1467f4-90b0-436c-a61a-394761bca783\") " pod="openshift-image-registry/image-registry-ff66d4bb5-czxjw" Jul 16 17:16:40.378266 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:16:40.378110 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/4e1467f4-90b0-436c-a61a-394761bca783-registry-certificates\") pod \"image-registry-ff66d4bb5-czxjw\" (UID: \"4e1467f4-90b0-436c-a61a-394761bca783\") " pod="openshift-image-registry/image-registry-ff66d4bb5-czxjw" Jul 16 17:16:40.378266 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:16:40.378140 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/4e1467f4-90b0-436c-a61a-394761bca783-bound-sa-token\") pod \"image-registry-ff66d4bb5-czxjw\" (UID: \"4e1467f4-90b0-436c-a61a-394761bca783\") " pod="openshift-image-registry/image-registry-ff66d4bb5-czxjw" Jul 16 17:16:40.378266 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:16:40.378201 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/4e1467f4-90b0-436c-a61a-394761bca783-installation-pull-secrets\") pod \"image-registry-ff66d4bb5-czxjw\" (UID: \"4e1467f4-90b0-436c-a61a-394761bca783\") " pod="openshift-image-registry/image-registry-ff66d4bb5-czxjw" Jul 16 17:16:40.379216 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:16:40.379153 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/4e1467f4-90b0-436c-a61a-394761bca783-ca-trust-extracted\") pod \"image-registry-ff66d4bb5-czxjw\" (UID: \"4e1467f4-90b0-436c-a61a-394761bca783\") " pod="openshift-image-registry/image-registry-ff66d4bb5-czxjw" Jul 16 17:16:40.380195 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:16:40.380168 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/4e1467f4-90b0-436c-a61a-394761bca783-registry-certificates\") pod \"image-registry-ff66d4bb5-czxjw\" (UID: \"4e1467f4-90b0-436c-a61a-394761bca783\") " pod="openshift-image-registry/image-registry-ff66d4bb5-czxjw" Jul 16 17:16:40.380471 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:16:40.380449 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/4e1467f4-90b0-436c-a61a-394761bca783-trusted-ca\") pod \"image-registry-ff66d4bb5-czxjw\" (UID: \"4e1467f4-90b0-436c-a61a-394761bca783\") " pod="openshift-image-registry/image-registry-ff66d4bb5-czxjw" Jul 16 17:16:40.381572 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:16:40.381525 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/4e1467f4-90b0-436c-a61a-394761bca783-image-registry-private-configuration\") pod \"image-registry-ff66d4bb5-czxjw\" (UID: \"4e1467f4-90b0-436c-a61a-394761bca783\") " pod="openshift-image-registry/image-registry-ff66d4bb5-czxjw" Jul 16 17:16:40.383932 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:16:40.383891 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/4e1467f4-90b0-436c-a61a-394761bca783-installation-pull-secrets\") pod \"image-registry-ff66d4bb5-czxjw\" (UID: \"4e1467f4-90b0-436c-a61a-394761bca783\") " pod="openshift-image-registry/image-registry-ff66d4bb5-czxjw" Jul 16 17:16:40.384068 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:16:40.384021 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/4e1467f4-90b0-436c-a61a-394761bca783-registry-tls\") pod \"image-registry-ff66d4bb5-czxjw\" (UID: \"4e1467f4-90b0-436c-a61a-394761bca783\") " pod="openshift-image-registry/image-registry-ff66d4bb5-czxjw" Jul 16 17:16:40.388244 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:16:40.388209 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/4e1467f4-90b0-436c-a61a-394761bca783-bound-sa-token\") pod \"image-registry-ff66d4bb5-czxjw\" (UID: \"4e1467f4-90b0-436c-a61a-394761bca783\") " pod="openshift-image-registry/image-registry-ff66d4bb5-czxjw" Jul 16 17:16:40.392550 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:16:40.392530 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-msfgx\" (UniqueName: \"kubernetes.io/projected/4e1467f4-90b0-436c-a61a-394761bca783-kube-api-access-msfgx\") pod \"image-registry-ff66d4bb5-czxjw\" (UID: \"4e1467f4-90b0-436c-a61a-394761bca783\") " pod="openshift-image-registry/image-registry-ff66d4bb5-czxjw" Jul 16 17:16:40.439346 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:16:40.439322 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-admission-webhook-77d98bc786-87596" Jul 16 17:16:40.448213 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:16:40.448189 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/downloads-68c9cbb497-lrlw4" Jul 16 17:16:40.454372 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:16:40.454351 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-54565489c9-dcjs7" Jul 16 17:16:40.462542 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:16:40.462519 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-source-869dcfcdcd-bp4xd"] Jul 16 17:16:40.486499 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:16:40.486264 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-ff66d4bb5-czxjw" Jul 16 17:16:40.608915 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:16:40.608882 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-77d98bc786-87596"] Jul 16 17:16:40.634716 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:16:40.634674 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/downloads-68c9cbb497-lrlw4"] Jul 16 17:16:40.650643 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:16:40.650616 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-54565489c9-dcjs7"] Jul 16 17:16:42.460811 ip-10-0-132-130 kubenswrapper[2578]: W0716 17:16:42.460777 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbfc93304_acba_424e_b6db_0844667a0803.slice/crio-5d4dd5f6b86e4c702e13737a7f3f3f75fcf6c240260a20d47fe8a68d276739fd WatchSource:0}: Error finding container 5d4dd5f6b86e4c702e13737a7f3f3f75fcf6c240260a20d47fe8a68d276739fd: Status 404 returned error can't find the container with id 5d4dd5f6b86e4c702e13737a7f3f3f75fcf6c240260a20d47fe8a68d276739fd Jul 16 17:16:42.461897 ip-10-0-132-130 kubenswrapper[2578]: W0716 17:16:42.461668 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7d388c86_ce98_4659_a1df_ddd1bf711eaa.slice/crio-5b1e4807fd82c7c3860935a4c3047d58d6ee5cf27c4983ce732b15e63e9f0153 WatchSource:0}: Error finding container 5b1e4807fd82c7c3860935a4c3047d58d6ee5cf27c4983ce732b15e63e9f0153: Status 404 returned error can't find the container with id 5b1e4807fd82c7c3860935a4c3047d58d6ee5cf27c4983ce732b15e63e9f0153 Jul 16 17:16:42.501865 ip-10-0-132-130 kubenswrapper[2578]: W0716 17:16:42.501824 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfd67d296_3c7d_47e1_89c2_d4e3f8edb1cc.slice/crio-54bf2cf6595e4736821a6a5d3abd32f4ba376c17dbfdf3dfbf08056d4a8ba44c WatchSource:0}: Error finding container 54bf2cf6595e4736821a6a5d3abd32f4ba376c17dbfdf3dfbf08056d4a8ba44c: Status 404 returned error can't find the container with id 54bf2cf6595e4736821a6a5d3abd32f4ba376c17dbfdf3dfbf08056d4a8ba44c Jul 16 17:16:42.502271 ip-10-0-132-130 kubenswrapper[2578]: W0716 17:16:42.502247 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd4cbd1db_79c1_4af2_9866_53b8aeee1bec.slice/crio-e9d05a006c424680b445dbe99164b24648eae39abd00e9629c3f3d3bea7c3303 WatchSource:0}: Error finding container e9d05a006c424680b445dbe99164b24648eae39abd00e9629c3f3d3bea7c3303: Status 404 returned error can't find the container with id e9d05a006c424680b445dbe99164b24648eae39abd00e9629c3f3d3bea7c3303 Jul 16 17:16:42.704903 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:16:42.704867 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-54565489c9-dcjs7" event={"ID":"d4cbd1db-79c1-4af2-9866-53b8aeee1bec","Type":"ContainerStarted","Data":"e9d05a006c424680b445dbe99164b24648eae39abd00e9629c3f3d3bea7c3303"} Jul 16 17:16:42.706134 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:16:42.706108 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-admission-webhook-77d98bc786-87596" event={"ID":"bfc93304-acba-424e-b6db-0844667a0803","Type":"ContainerStarted","Data":"5d4dd5f6b86e4c702e13737a7f3f3f75fcf6c240260a20d47fe8a68d276739fd"} Jul 16 17:16:42.707289 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:16:42.707230 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-source-869dcfcdcd-bp4xd" event={"ID":"7d388c86-ce98-4659-a1df-ddd1bf711eaa","Type":"ContainerStarted","Data":"5b1e4807fd82c7c3860935a4c3047d58d6ee5cf27c4983ce732b15e63e9f0153"} Jul 16 17:16:42.708485 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:16:42.708465 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/downloads-68c9cbb497-lrlw4" event={"ID":"fd67d296-3c7d-47e1-89c2-d4e3f8edb1cc","Type":"ContainerStarted","Data":"54bf2cf6595e4736821a6a5d3abd32f4ba376c17dbfdf3dfbf08056d4a8ba44c"} Jul 16 17:16:43.287857 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:16:43.287599 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-ff66d4bb5-czxjw"] Jul 16 17:16:43.290188 ip-10-0-132-130 kubenswrapper[2578]: W0716 17:16:43.290159 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4e1467f4_90b0_436c_a61a_394761bca783.slice/crio-dca35d3977eeac93b7d2da4b1cbd91ff90f9ad9538adfa42173267c18b317a8f WatchSource:0}: Error finding container dca35d3977eeac93b7d2da4b1cbd91ff90f9ad9538adfa42173267c18b317a8f: Status 404 returned error can't find the container with id dca35d3977eeac93b7d2da4b1cbd91ff90f9ad9538adfa42173267c18b317a8f Jul 16 17:16:43.728933 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:16:43.723707 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-source-869dcfcdcd-bp4xd" event={"ID":"7d388c86-ce98-4659-a1df-ddd1bf711eaa","Type":"ContainerStarted","Data":"48a4abaf70d69b7304e30fb4c4a0fb01dcc26dac74204c7a3d1ef00b47d8c82f"} Jul 16 17:16:43.733586 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:16:43.733461 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-ff66d4bb5-czxjw" event={"ID":"4e1467f4-90b0-436c-a61a-394761bca783","Type":"ContainerStarted","Data":"4d6f32d2d994258aba5a92897f1a9b5dde31e656e58f3c5a1d9a7dc5e1e00a0c"} Jul 16 17:16:43.733586 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:16:43.733514 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-ff66d4bb5-czxjw" event={"ID":"4e1467f4-90b0-436c-a61a-394761bca783","Type":"ContainerStarted","Data":"dca35d3977eeac93b7d2da4b1cbd91ff90f9ad9538adfa42173267c18b317a8f"} Jul 16 17:16:43.734430 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:16:43.734227 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-ff66d4bb5-czxjw" Jul 16 17:16:43.738923 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:16:43.738863 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-5czbw" event={"ID":"dbd1c30a-e583-4ecd-b5c6-ec71f3579e50","Type":"ContainerStarted","Data":"483b3129eb05051394d36e016da08d6470e089d58776137b9d57aea974a7152b"} Jul 16 17:16:43.748716 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:16:43.748597 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-source-869dcfcdcd-bp4xd" podStartSLOduration=4.748578997 podStartE2EDuration="4.748578997s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 17:16:39 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 17:16:43.74805764 +0000 UTC m=+148.233901531" watchObservedRunningTime="2026-07-16 17:16:43.748578997 +0000 UTC m=+148.234422887" Jul 16 17:16:43.791319 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:16:43.790638 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/image-registry-ff66d4bb5-czxjw" podStartSLOduration=3.790617521 podStartE2EDuration="3.790617521s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 17:16:40 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 17:16:43.789451311 +0000 UTC m=+148.275295193" watchObservedRunningTime="2026-07-16 17:16:43.790617521 +0000 UTC m=+148.276461392" Jul 16 17:16:44.745430 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:16:44.745351 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-admission-webhook-77d98bc786-87596" event={"ID":"bfc93304-acba-424e-b6db-0844667a0803","Type":"ContainerStarted","Data":"4d1e03bed0c04f89e2db64289d78e89b3e2d7675b6d45310653f8ba8b221ad63"} Jul 16 17:16:44.772118 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:16:44.771994 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-operator-admission-webhook-77d98bc786-87596" podStartSLOduration=2.730979106 podStartE2EDuration="4.771974631s" totalImagesPullingTime="2.040995525s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 17:16:40 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 17:16:44.770639182 +0000 UTC m=+149.256483071" watchObservedRunningTime="2026-07-16 17:16:44.771974631 +0000 UTC m=+149.257818523" Jul 16 17:16:45.750125 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:16:45.750001 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-operator-admission-webhook-77d98bc786-87596" Jul 16 17:16:45.755193 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:16:45.755149 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-operator-admission-webhook-77d98bc786-87596" Jul 16 17:16:48.763142 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:16:48.763103 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-5czbw" event={"ID":"dbd1c30a-e583-4ecd-b5c6-ec71f3579e50","Type":"ContainerStarted","Data":"aa8ac10ecd62e06382d97c48b2a7aef0c7f004c3f1611cf2c4483cd77be13867"} Jul 16 17:16:48.764739 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:16:48.764711 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-54565489c9-dcjs7" event={"ID":"d4cbd1db-79c1-4af2-9866-53b8aeee1bec","Type":"ContainerStarted","Data":"e6d5a6a935f1725e01a9236966d24765ba029200bd81fcf4013a65492a65dda2"} Jul 16 17:16:48.791640 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:16:48.791586 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-54565489c9-dcjs7" podStartSLOduration=3.373103271 podStartE2EDuration="8.791570375s" totalImagesPullingTime="5.418467104s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 17:16:40 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 17:16:48.791534944 +0000 UTC m=+153.277378834" watchObservedRunningTime="2026-07-16 17:16:48.791570375 +0000 UTC m=+153.277414271" Jul 16 17:16:50.455207 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:16:50.455169 2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-54565489c9-dcjs7" Jul 16 17:16:50.455637 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:16:50.455239 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-54565489c9-dcjs7" Jul 16 17:16:50.461655 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:16:50.461630 2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-54565489c9-dcjs7" Jul 16 17:16:50.778291 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:16:50.778191 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-54565489c9-dcjs7" Jul 16 17:16:52.925333 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:16:52.925299 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/openshift-state-metrics-84bd68b896-kbbbg"] Jul 16 17:16:52.939831 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:16:52.939800 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/openshift-state-metrics-84bd68b896-kbbbg" Jul 16 17:16:52.953710 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:16:52.953666 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-kube-rbac-proxy-config\"" Jul 16 17:16:52.954966 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:16:52.954944 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\"" Jul 16 17:16:52.955098 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:16:52.954966 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-dockercfg-94jrd\"" Jul 16 17:16:52.955419 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:16:52.955404 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-tls\"" Jul 16 17:16:52.963390 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:16:52.963366 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/openshift-state-metrics-84bd68b896-kbbbg"] Jul 16 17:16:53.050672 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:16:53.050634 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-8gvqf"] Jul 16 17:16:53.056483 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:16:53.056454 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-8gvqf" Jul 16 17:16:53.060792 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:16:53.060767 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\"" Jul 16 17:16:53.061213 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:16:53.061183 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-b285m\"" Jul 16 17:16:53.068125 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:16:53.068022 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\"" Jul 16 17:16:53.076666 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:16:53.076475 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\"" Jul 16 17:16:53.094325 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:16:53.094287 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/486ff997-a129-4760-8f65-6fa79ba9eed3-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-84bd68b896-kbbbg\" (UID: \"486ff997-a129-4760-8f65-6fa79ba9eed3\") " pod="openshift-monitoring/openshift-state-metrics-84bd68b896-kbbbg" Jul 16 17:16:53.094523 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:16:53.094333 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/486ff997-a129-4760-8f65-6fa79ba9eed3-metrics-client-ca\") pod \"openshift-state-metrics-84bd68b896-kbbbg\" (UID: \"486ff997-a129-4760-8f65-6fa79ba9eed3\") " pod="openshift-monitoring/openshift-state-metrics-84bd68b896-kbbbg" Jul 16 17:16:53.094523 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:16:53.094449 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/486ff997-a129-4760-8f65-6fa79ba9eed3-openshift-state-metrics-tls\") pod \"openshift-state-metrics-84bd68b896-kbbbg\" (UID: \"486ff997-a129-4760-8f65-6fa79ba9eed3\") " pod="openshift-monitoring/openshift-state-metrics-84bd68b896-kbbbg" Jul 16 17:16:53.094668 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:16:53.094640 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pjbqw\" (UniqueName: \"kubernetes.io/projected/486ff997-a129-4760-8f65-6fa79ba9eed3-kube-api-access-pjbqw\") pod \"openshift-state-metrics-84bd68b896-kbbbg\" (UID: \"486ff997-a129-4760-8f65-6fa79ba9eed3\") " pod="openshift-monitoring/openshift-state-metrics-84bd68b896-kbbbg" Jul 16 17:16:53.195800 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:16:53.195723 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/12344972-7243-4730-9352-19dacb37fbcb-sys\") pod \"node-exporter-8gvqf\" (UID: \"12344972-7243-4730-9352-19dacb37fbcb\") " pod="openshift-monitoring/node-exporter-8gvqf" Jul 16 17:16:53.195800 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:16:53.195770 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/12344972-7243-4730-9352-19dacb37fbcb-node-exporter-wtmp\") pod \"node-exporter-8gvqf\" (UID: \"12344972-7243-4730-9352-19dacb37fbcb\") " pod="openshift-monitoring/node-exporter-8gvqf" Jul 16 17:16:53.195800 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:16:53.195794 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/12344972-7243-4730-9352-19dacb37fbcb-root\") pod \"node-exporter-8gvqf\" (UID: \"12344972-7243-4730-9352-19dacb37fbcb\") " pod="openshift-monitoring/node-exporter-8gvqf" Jul 16 17:16:53.196077 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:16:53.195817 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/12344972-7243-4730-9352-19dacb37fbcb-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-8gvqf\" (UID: \"12344972-7243-4730-9352-19dacb37fbcb\") " pod="openshift-monitoring/node-exporter-8gvqf" Jul 16 17:16:53.196077 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:16:53.195898 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/12344972-7243-4730-9352-19dacb37fbcb-metrics-client-ca\") pod \"node-exporter-8gvqf\" (UID: \"12344972-7243-4730-9352-19dacb37fbcb\") " pod="openshift-monitoring/node-exporter-8gvqf" Jul 16 17:16:53.196077 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:16:53.195939 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/12344972-7243-4730-9352-19dacb37fbcb-node-exporter-tls\") pod \"node-exporter-8gvqf\" (UID: \"12344972-7243-4730-9352-19dacb37fbcb\") " pod="openshift-monitoring/node-exporter-8gvqf" Jul 16 17:16:53.196077 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:16:53.195973 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/12344972-7243-4730-9352-19dacb37fbcb-node-exporter-accelerators-collector-config\") pod \"node-exporter-8gvqf\" (UID: \"12344972-7243-4730-9352-19dacb37fbcb\") " pod="openshift-monitoring/node-exporter-8gvqf" Jul 16 17:16:53.196077 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:16:53.196046 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pjbqw\" (UniqueName: \"kubernetes.io/projected/486ff997-a129-4760-8f65-6fa79ba9eed3-kube-api-access-pjbqw\") pod \"openshift-state-metrics-84bd68b896-kbbbg\" (UID: \"486ff997-a129-4760-8f65-6fa79ba9eed3\") " pod="openshift-monitoring/openshift-state-metrics-84bd68b896-kbbbg" Jul 16 17:16:53.196077 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:16:53.196067 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sqst9\" (UniqueName: \"kubernetes.io/projected/12344972-7243-4730-9352-19dacb37fbcb-kube-api-access-sqst9\") pod \"node-exporter-8gvqf\" (UID: \"12344972-7243-4730-9352-19dacb37fbcb\") " pod="openshift-monitoring/node-exporter-8gvqf" Jul 16 17:16:53.196373 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:16:53.196088 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/486ff997-a129-4760-8f65-6fa79ba9eed3-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-84bd68b896-kbbbg\" (UID: \"486ff997-a129-4760-8f65-6fa79ba9eed3\") " pod="openshift-monitoring/openshift-state-metrics-84bd68b896-kbbbg" Jul 16 17:16:53.196373 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:16:53.196113 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/486ff997-a129-4760-8f65-6fa79ba9eed3-metrics-client-ca\") pod \"openshift-state-metrics-84bd68b896-kbbbg\" (UID: \"486ff997-a129-4760-8f65-6fa79ba9eed3\") " pod="openshift-monitoring/openshift-state-metrics-84bd68b896-kbbbg" Jul 16 17:16:53.196373 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:16:53.196166 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/486ff997-a129-4760-8f65-6fa79ba9eed3-openshift-state-metrics-tls\") pod \"openshift-state-metrics-84bd68b896-kbbbg\" (UID: \"486ff997-a129-4760-8f65-6fa79ba9eed3\") " pod="openshift-monitoring/openshift-state-metrics-84bd68b896-kbbbg" Jul 16 17:16:53.196373 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:16:53.196190 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/12344972-7243-4730-9352-19dacb37fbcb-node-exporter-textfile\") pod \"node-exporter-8gvqf\" (UID: \"12344972-7243-4730-9352-19dacb37fbcb\") " pod="openshift-monitoring/node-exporter-8gvqf" Jul 16 17:16:53.197819 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:16:53.196958 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/486ff997-a129-4760-8f65-6fa79ba9eed3-metrics-client-ca\") pod \"openshift-state-metrics-84bd68b896-kbbbg\" (UID: \"486ff997-a129-4760-8f65-6fa79ba9eed3\") " pod="openshift-monitoring/openshift-state-metrics-84bd68b896-kbbbg" Jul 16 17:16:53.199113 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:16:53.199088 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/486ff997-a129-4760-8f65-6fa79ba9eed3-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-84bd68b896-kbbbg\" (UID: \"486ff997-a129-4760-8f65-6fa79ba9eed3\") " pod="openshift-monitoring/openshift-state-metrics-84bd68b896-kbbbg" Jul 16 17:16:53.200109 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:16:53.200087 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/486ff997-a129-4760-8f65-6fa79ba9eed3-openshift-state-metrics-tls\") pod \"openshift-state-metrics-84bd68b896-kbbbg\" (UID: \"486ff997-a129-4760-8f65-6fa79ba9eed3\") " pod="openshift-monitoring/openshift-state-metrics-84bd68b896-kbbbg" Jul 16 17:16:53.211159 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:16:53.211127 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-pjbqw\" (UniqueName: \"kubernetes.io/projected/486ff997-a129-4760-8f65-6fa79ba9eed3-kube-api-access-pjbqw\") pod \"openshift-state-metrics-84bd68b896-kbbbg\" (UID: \"486ff997-a129-4760-8f65-6fa79ba9eed3\") " pod="openshift-monitoring/openshift-state-metrics-84bd68b896-kbbbg" Jul 16 17:16:53.250989 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:16:53.250825 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/openshift-state-metrics-84bd68b896-kbbbg" Jul 16 17:16:53.296912 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:16:53.296873 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sqst9\" (UniqueName: \"kubernetes.io/projected/12344972-7243-4730-9352-19dacb37fbcb-kube-api-access-sqst9\") pod \"node-exporter-8gvqf\" (UID: \"12344972-7243-4730-9352-19dacb37fbcb\") " pod="openshift-monitoring/node-exporter-8gvqf" Jul 16 17:16:53.297096 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:16:53.296950 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/12344972-7243-4730-9352-19dacb37fbcb-node-exporter-textfile\") pod \"node-exporter-8gvqf\" (UID: \"12344972-7243-4730-9352-19dacb37fbcb\") " pod="openshift-monitoring/node-exporter-8gvqf" Jul 16 17:16:53.297096 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:16:53.296987 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/12344972-7243-4730-9352-19dacb37fbcb-sys\") pod \"node-exporter-8gvqf\" (UID: \"12344972-7243-4730-9352-19dacb37fbcb\") " pod="openshift-monitoring/node-exporter-8gvqf" Jul 16 17:16:53.297096 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:16:53.297019 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/12344972-7243-4730-9352-19dacb37fbcb-node-exporter-wtmp\") pod \"node-exporter-8gvqf\" (UID: \"12344972-7243-4730-9352-19dacb37fbcb\") " pod="openshift-monitoring/node-exporter-8gvqf" Jul 16 17:16:53.297096 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:16:53.297043 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/12344972-7243-4730-9352-19dacb37fbcb-root\") pod \"node-exporter-8gvqf\" (UID: \"12344972-7243-4730-9352-19dacb37fbcb\") " pod="openshift-monitoring/node-exporter-8gvqf" Jul 16 17:16:53.297096 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:16:53.297071 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/12344972-7243-4730-9352-19dacb37fbcb-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-8gvqf\" (UID: \"12344972-7243-4730-9352-19dacb37fbcb\") " pod="openshift-monitoring/node-exporter-8gvqf" Jul 16 17:16:53.297096 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:16:53.297134 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/12344972-7243-4730-9352-19dacb37fbcb-metrics-client-ca\") pod \"node-exporter-8gvqf\" (UID: \"12344972-7243-4730-9352-19dacb37fbcb\") " pod="openshift-monitoring/node-exporter-8gvqf" Jul 16 17:16:53.297957 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:16:53.297170 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/12344972-7243-4730-9352-19dacb37fbcb-node-exporter-tls\") pod \"node-exporter-8gvqf\" (UID: \"12344972-7243-4730-9352-19dacb37fbcb\") " pod="openshift-monitoring/node-exporter-8gvqf" Jul 16 17:16:53.297957 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:16:53.297172 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/12344972-7243-4730-9352-19dacb37fbcb-sys\") pod \"node-exporter-8gvqf\" (UID: \"12344972-7243-4730-9352-19dacb37fbcb\") " pod="openshift-monitoring/node-exporter-8gvqf" Jul 16 17:16:53.297957 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:16:53.297195 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/12344972-7243-4730-9352-19dacb37fbcb-node-exporter-accelerators-collector-config\") pod \"node-exporter-8gvqf\" (UID: \"12344972-7243-4730-9352-19dacb37fbcb\") " pod="openshift-monitoring/node-exporter-8gvqf" Jul 16 17:16:53.297957 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:16:53.297495 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/12344972-7243-4730-9352-19dacb37fbcb-node-exporter-wtmp\") pod \"node-exporter-8gvqf\" (UID: \"12344972-7243-4730-9352-19dacb37fbcb\") " pod="openshift-monitoring/node-exporter-8gvqf" Jul 16 17:16:53.297957 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:16:53.297561 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/12344972-7243-4730-9352-19dacb37fbcb-node-exporter-textfile\") pod \"node-exporter-8gvqf\" (UID: \"12344972-7243-4730-9352-19dacb37fbcb\") " pod="openshift-monitoring/node-exporter-8gvqf" Jul 16 17:16:53.297957 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:16:53.297584 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/12344972-7243-4730-9352-19dacb37fbcb-root\") pod \"node-exporter-8gvqf\" (UID: \"12344972-7243-4730-9352-19dacb37fbcb\") " pod="openshift-monitoring/node-exporter-8gvqf" Jul 16 17:16:53.298679 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:16:53.298024 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/12344972-7243-4730-9352-19dacb37fbcb-node-exporter-accelerators-collector-config\") pod \"node-exporter-8gvqf\" (UID: \"12344972-7243-4730-9352-19dacb37fbcb\") " pod="openshift-monitoring/node-exporter-8gvqf" Jul 16 17:16:53.298679 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:16:53.298393 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/12344972-7243-4730-9352-19dacb37fbcb-metrics-client-ca\") pod \"node-exporter-8gvqf\" (UID: \"12344972-7243-4730-9352-19dacb37fbcb\") " pod="openshift-monitoring/node-exporter-8gvqf" Jul 16 17:16:53.300918 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:16:53.300874 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/12344972-7243-4730-9352-19dacb37fbcb-node-exporter-tls\") pod \"node-exporter-8gvqf\" (UID: \"12344972-7243-4730-9352-19dacb37fbcb\") " pod="openshift-monitoring/node-exporter-8gvqf" Jul 16 17:16:53.301066 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:16:53.301045 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/12344972-7243-4730-9352-19dacb37fbcb-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-8gvqf\" (UID: \"12344972-7243-4730-9352-19dacb37fbcb\") " pod="openshift-monitoring/node-exporter-8gvqf" Jul 16 17:16:53.313250 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:16:53.313224 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-sqst9\" (UniqueName: \"kubernetes.io/projected/12344972-7243-4730-9352-19dacb37fbcb-kube-api-access-sqst9\") pod \"node-exporter-8gvqf\" (UID: \"12344972-7243-4730-9352-19dacb37fbcb\") " pod="openshift-monitoring/node-exporter-8gvqf" Jul 16 17:16:53.348034 ip-10-0-132-130 kubenswrapper[2578]: E0716 17:16:53.347994 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[metrics-tls], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-dns/dns-default-vtkgg" podUID="f99dd101-e4fa-4c68-a502-c3a14be5c7fb" Jul 16 17:16:53.369089 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:16:53.369055 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-8gvqf" Jul 16 17:16:53.786663 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:16:53.786321 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-vtkgg" Jul 16 17:16:54.006493 ip-10-0-132-130 kubenswrapper[2578]: W0716 17:16:54.006413 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod12344972_7243_4730_9352_19dacb37fbcb.slice/crio-5117904814d537b521ac57b021d44e4787b974fda50b4a06746bedddf106bdc0 WatchSource:0}: Error finding container 5117904814d537b521ac57b021d44e4787b974fda50b4a06746bedddf106bdc0: Status 404 returned error can't find the container with id 5117904814d537b521ac57b021d44e4787b974fda50b4a06746bedddf106bdc0 Jul 16 17:16:54.161057 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:16:54.161023 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/openshift-state-metrics-84bd68b896-kbbbg"] Jul 16 17:16:54.164823 ip-10-0-132-130 kubenswrapper[2578]: W0716 17:16:54.164793 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod486ff997_a129_4760_8f65_6fa79ba9eed3.slice/crio-a860e3bd64feb3a691071fd54818e838efcc26625f1d6c7479e7faf70310af1e WatchSource:0}: Error finding container a860e3bd64feb3a691071fd54818e838efcc26625f1d6c7479e7faf70310af1e: Status 404 returned error can't find the container with id a860e3bd64feb3a691071fd54818e838efcc26625f1d6c7479e7faf70310af1e Jul 16 17:16:54.220403 ip-10-0-132-130 kubenswrapper[2578]: E0716 17:16:54.220375 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[metrics-certs], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-multus/network-metrics-daemon-kvr2r" podUID="f90822ad-ab14-4108-8294-2e76c35a8aa5" Jul 16 17:16:54.792371 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:16:54.792323 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-8gvqf" event={"ID":"12344972-7243-4730-9352-19dacb37fbcb","Type":"ContainerStarted","Data":"5117904814d537b521ac57b021d44e4787b974fda50b4a06746bedddf106bdc0"} Jul 16 17:16:54.794764 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:16:54.794727 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-84bd68b896-kbbbg" event={"ID":"486ff997-a129-4760-8f65-6fa79ba9eed3","Type":"ContainerStarted","Data":"8177c2234f245fd5a36c30b68e5a0953523e836d1bc1c55e8f4666f6434a6c11"} Jul 16 17:16:54.794900 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:16:54.794798 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-84bd68b896-kbbbg" event={"ID":"486ff997-a129-4760-8f65-6fa79ba9eed3","Type":"ContainerStarted","Data":"77733883eeccdcfd345d8e3eeca0cd70dc82b6374e4e5d66d21177042313b3a5"} Jul 16 17:16:54.794900 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:16:54.794827 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-84bd68b896-kbbbg" event={"ID":"486ff997-a129-4760-8f65-6fa79ba9eed3","Type":"ContainerStarted","Data":"a860e3bd64feb3a691071fd54818e838efcc26625f1d6c7479e7faf70310af1e"} Jul 16 17:16:54.797515 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:16:54.797250 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-5czbw" event={"ID":"dbd1c30a-e583-4ecd-b5c6-ec71f3579e50","Type":"ContainerStarted","Data":"fe159464367102fd742379122ce25fdf33bb1c3b08ac0a3854b2c2295d7f371c"} Jul 16 17:16:55.803352 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:16:55.803321 2578 generic.go:356] "Generic (PLEG): container finished" podID="12344972-7243-4730-9352-19dacb37fbcb" containerID="2c7dd2bc360b0eb116c4e2a4f1b30b1c72caf640f7a98ed5cb0a10c2d320fb08" exitCode=0 Jul 16 17:16:55.803836 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:16:55.803410 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-8gvqf" event={"ID":"12344972-7243-4730-9352-19dacb37fbcb","Type":"ContainerDied","Data":"2c7dd2bc360b0eb116c4e2a4f1b30b1c72caf640f7a98ed5cb0a10c2d320fb08"} Jul 16 17:16:55.805941 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:16:55.805909 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-84bd68b896-kbbbg" event={"ID":"486ff997-a129-4760-8f65-6fa79ba9eed3","Type":"ContainerStarted","Data":"cce4a271da9b99a1e949d30aaf31500f0639b8c8b568255b4bdb8eade4bef82a"} Jul 16 17:16:55.835826 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:16:55.835771 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-5czbw" podStartSLOduration=34.983493522 podStartE2EDuration="50.835752557s" totalImagesPullingTime="15.852259035s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-16 17:16:05 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 17:16:54.839392907 +0000 UTC m=+159.325236802" watchObservedRunningTime="2026-07-16 17:16:55.835752557 +0000 UTC m=+160.321596445" Jul 16 17:16:55.863432 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:16:55.863266 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/openshift-state-metrics-84bd68b896-kbbbg" podStartSLOduration=2.818529872 podStartE2EDuration="3.863245278s" totalImagesPullingTime="1.044715406s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 17:16:52 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 17:16:55.861878885 +0000 UTC m=+160.347722775" watchObservedRunningTime="2026-07-16 17:16:55.863245278 +0000 UTC m=+160.349089168" Jul 16 17:16:56.058708 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:16:56.058598 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/thanos-querier-6769966f79-57q7b"] Jul 16 17:16:56.062482 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:16:56.062460 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/thanos-querier-6769966f79-57q7b" Jul 16 17:16:56.067340 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:16:56.067172 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy\"" Jul 16 17:16:56.067340 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:16:56.067178 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-web\"" Jul 16 17:16:56.067340 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:16:56.067218 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-tls\"" Jul 16 17:16:56.067340 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:16:56.067247 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-dockercfg-8p8gv\"" Jul 16 17:16:56.067941 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:16:56.067921 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-metrics\"" Jul 16 17:16:56.068048 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:16:56.067926 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-rules\"" Jul 16 17:16:56.068405 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:16:56.068383 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-grpc-tls-75pc4kkn4o9tg\"" Jul 16 17:16:56.081784 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:16:56.081747 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/thanos-querier-6769966f79-57q7b"] Jul 16 17:16:56.223074 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:16:56.223045 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/8cb88b9c-ade2-499e-b959-3e158a068497-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-6769966f79-57q7b\" (UID: \"8cb88b9c-ade2-499e-b959-3e158a068497\") " pod="openshift-monitoring/thanos-querier-6769966f79-57q7b" Jul 16 17:16:56.223247 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:16:56.223112 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/8cb88b9c-ade2-499e-b959-3e158a068497-metrics-client-ca\") pod \"thanos-querier-6769966f79-57q7b\" (UID: \"8cb88b9c-ade2-499e-b959-3e158a068497\") " pod="openshift-monitoring/thanos-querier-6769966f79-57q7b" Jul 16 17:16:56.223247 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:16:56.223154 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/8cb88b9c-ade2-499e-b959-3e158a068497-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-6769966f79-57q7b\" (UID: \"8cb88b9c-ade2-499e-b959-3e158a068497\") " pod="openshift-monitoring/thanos-querier-6769966f79-57q7b" Jul 16 17:16:56.223247 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:16:56.223185 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/8cb88b9c-ade2-499e-b959-3e158a068497-secret-grpc-tls\") pod \"thanos-querier-6769966f79-57q7b\" (UID: \"8cb88b9c-ade2-499e-b959-3e158a068497\") " pod="openshift-monitoring/thanos-querier-6769966f79-57q7b" Jul 16 17:16:56.223247 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:16:56.223206 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/8cb88b9c-ade2-499e-b959-3e158a068497-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-6769966f79-57q7b\" (UID: \"8cb88b9c-ade2-499e-b959-3e158a068497\") " pod="openshift-monitoring/thanos-querier-6769966f79-57q7b" Jul 16 17:16:56.223247 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:16:56.223242 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-t6jck\" (UniqueName: \"kubernetes.io/projected/8cb88b9c-ade2-499e-b959-3e158a068497-kube-api-access-t6jck\") pod \"thanos-querier-6769966f79-57q7b\" (UID: \"8cb88b9c-ade2-499e-b959-3e158a068497\") " pod="openshift-monitoring/thanos-querier-6769966f79-57q7b" Jul 16 17:16:56.223444 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:16:56.223374 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/8cb88b9c-ade2-499e-b959-3e158a068497-secret-thanos-querier-tls\") pod \"thanos-querier-6769966f79-57q7b\" (UID: \"8cb88b9c-ade2-499e-b959-3e158a068497\") " pod="openshift-monitoring/thanos-querier-6769966f79-57q7b" Jul 16 17:16:56.223444 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:16:56.223423 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/8cb88b9c-ade2-499e-b959-3e158a068497-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-6769966f79-57q7b\" (UID: \"8cb88b9c-ade2-499e-b959-3e158a068497\") " pod="openshift-monitoring/thanos-querier-6769966f79-57q7b" Jul 16 17:16:56.323944 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:16:56.323852 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/8cb88b9c-ade2-499e-b959-3e158a068497-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-6769966f79-57q7b\" (UID: \"8cb88b9c-ade2-499e-b959-3e158a068497\") " pod="openshift-monitoring/thanos-querier-6769966f79-57q7b" Jul 16 17:16:56.323944 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:16:56.323898 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/8cb88b9c-ade2-499e-b959-3e158a068497-secret-grpc-tls\") pod \"thanos-querier-6769966f79-57q7b\" (UID: \"8cb88b9c-ade2-499e-b959-3e158a068497\") " pod="openshift-monitoring/thanos-querier-6769966f79-57q7b" Jul 16 17:16:56.323944 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:16:56.323934 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/8cb88b9c-ade2-499e-b959-3e158a068497-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-6769966f79-57q7b\" (UID: \"8cb88b9c-ade2-499e-b959-3e158a068497\") " pod="openshift-monitoring/thanos-querier-6769966f79-57q7b" Jul 16 17:16:56.324203 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:16:56.323997 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-t6jck\" (UniqueName: \"kubernetes.io/projected/8cb88b9c-ade2-499e-b959-3e158a068497-kube-api-access-t6jck\") pod \"thanos-querier-6769966f79-57q7b\" (UID: \"8cb88b9c-ade2-499e-b959-3e158a068497\") " pod="openshift-monitoring/thanos-querier-6769966f79-57q7b" Jul 16 17:16:56.324421 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:16:56.324396 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/8cb88b9c-ade2-499e-b959-3e158a068497-secret-thanos-querier-tls\") pod \"thanos-querier-6769966f79-57q7b\" (UID: \"8cb88b9c-ade2-499e-b959-3e158a068497\") " pod="openshift-monitoring/thanos-querier-6769966f79-57q7b" Jul 16 17:16:56.325126 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:16:56.325105 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/8cb88b9c-ade2-499e-b959-3e158a068497-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-6769966f79-57q7b\" (UID: \"8cb88b9c-ade2-499e-b959-3e158a068497\") " pod="openshift-monitoring/thanos-querier-6769966f79-57q7b" Jul 16 17:16:56.325296 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:16:56.325277 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/8cb88b9c-ade2-499e-b959-3e158a068497-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-6769966f79-57q7b\" (UID: \"8cb88b9c-ade2-499e-b959-3e158a068497\") " pod="openshift-monitoring/thanos-querier-6769966f79-57q7b" Jul 16 17:16:56.325812 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:16:56.325793 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/8cb88b9c-ade2-499e-b959-3e158a068497-metrics-client-ca\") pod \"thanos-querier-6769966f79-57q7b\" (UID: \"8cb88b9c-ade2-499e-b959-3e158a068497\") " pod="openshift-monitoring/thanos-querier-6769966f79-57q7b" Jul 16 17:16:56.326644 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:16:56.326616 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/8cb88b9c-ade2-499e-b959-3e158a068497-metrics-client-ca\") pod \"thanos-querier-6769966f79-57q7b\" (UID: \"8cb88b9c-ade2-499e-b959-3e158a068497\") " pod="openshift-monitoring/thanos-querier-6769966f79-57q7b" Jul 16 17:16:56.327012 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:16:56.326987 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/8cb88b9c-ade2-499e-b959-3e158a068497-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-6769966f79-57q7b\" (UID: \"8cb88b9c-ade2-499e-b959-3e158a068497\") " pod="openshift-monitoring/thanos-querier-6769966f79-57q7b" Jul 16 17:16:56.327155 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:16:56.327130 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/8cb88b9c-ade2-499e-b959-3e158a068497-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-6769966f79-57q7b\" (UID: \"8cb88b9c-ade2-499e-b959-3e158a068497\") " pod="openshift-monitoring/thanos-querier-6769966f79-57q7b" Jul 16 17:16:56.327559 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:16:56.327531 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/8cb88b9c-ade2-499e-b959-3e158a068497-secret-thanos-querier-tls\") pod \"thanos-querier-6769966f79-57q7b\" (UID: \"8cb88b9c-ade2-499e-b959-3e158a068497\") " pod="openshift-monitoring/thanos-querier-6769966f79-57q7b" Jul 16 17:16:56.327637 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:16:56.327612 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/8cb88b9c-ade2-499e-b959-3e158a068497-secret-grpc-tls\") pod \"thanos-querier-6769966f79-57q7b\" (UID: \"8cb88b9c-ade2-499e-b959-3e158a068497\") " pod="openshift-monitoring/thanos-querier-6769966f79-57q7b" Jul 16 17:16:56.329315 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:16:56.329237 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/8cb88b9c-ade2-499e-b959-3e158a068497-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-6769966f79-57q7b\" (UID: \"8cb88b9c-ade2-499e-b959-3e158a068497\") " pod="openshift-monitoring/thanos-querier-6769966f79-57q7b" Jul 16 17:16:56.329476 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:16:56.329454 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/8cb88b9c-ade2-499e-b959-3e158a068497-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-6769966f79-57q7b\" (UID: \"8cb88b9c-ade2-499e-b959-3e158a068497\") " pod="openshift-monitoring/thanos-querier-6769966f79-57q7b" Jul 16 17:16:56.336046 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:16:56.336024 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-t6jck\" (UniqueName: \"kubernetes.io/projected/8cb88b9c-ade2-499e-b959-3e158a068497-kube-api-access-t6jck\") pod \"thanos-querier-6769966f79-57q7b\" (UID: \"8cb88b9c-ade2-499e-b959-3e158a068497\") " pod="openshift-monitoring/thanos-querier-6769966f79-57q7b" Jul 16 17:16:56.373085 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:16:56.373055 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/thanos-querier-6769966f79-57q7b" Jul 16 17:16:56.532836 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:16:56.532782 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/thanos-querier-6769966f79-57q7b"] Jul 16 17:16:56.537369 ip-10-0-132-130 kubenswrapper[2578]: W0716 17:16:56.537340 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8cb88b9c_ade2_499e_b959_3e158a068497.slice/crio-543bfe734f6b96bd3b090ce9ea896820444a3f913a5671e57b711cab336720cb WatchSource:0}: Error finding container 543bfe734f6b96bd3b090ce9ea896820444a3f913a5671e57b711cab336720cb: Status 404 returned error can't find the container with id 543bfe734f6b96bd3b090ce9ea896820444a3f913a5671e57b711cab336720cb Jul 16 17:16:56.811438 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:16:56.811406 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-6769966f79-57q7b" event={"ID":"8cb88b9c-ade2-499e-b959-3e158a068497","Type":"ContainerStarted","Data":"543bfe734f6b96bd3b090ce9ea896820444a3f913a5671e57b711cab336720cb"} Jul 16 17:16:56.813553 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:16:56.813522 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-8gvqf" event={"ID":"12344972-7243-4730-9352-19dacb37fbcb","Type":"ContainerStarted","Data":"ef57d4109a1b6a0ba14693206e3ab8b9fb8e9a64a7b7f40d37bced31484af064"} Jul 16 17:16:56.813663 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:16:56.813576 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-8gvqf" event={"ID":"12344972-7243-4730-9352-19dacb37fbcb","Type":"ContainerStarted","Data":"398e7f5ccb9567764b597e6a1966d597978a9d1f3625c7e12ce54cc0c76acdd7"} Jul 16 17:16:56.837182 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:16:56.837133 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-8gvqf" podStartSLOduration=2.775737969 podStartE2EDuration="3.837120995s" totalImagesPullingTime="983.08094ms" totalInitContainerRuntime="78.302086ms" isStatefulPod=true podCreationTimestamp="2026-07-16 17:16:53 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 17:16:56.835332286 +0000 UTC m=+161.321176170" watchObservedRunningTime="2026-07-16 17:16:56.837120995 +0000 UTC m=+161.322964883" Jul 16 17:16:57.708134 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:16:57.708094 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/metrics-server-74dc84475b-hznm2"] Jul 16 17:16:57.712528 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:16:57.712504 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/metrics-server-74dc84475b-hznm2" Jul 16 17:16:57.717603 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:16:57.717567 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kubelet-serving-ca-bundle\"" Jul 16 17:16:57.717603 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:16:57.717580 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-server-audit-profiles\"" Jul 16 17:16:57.717826 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:16:57.717647 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-client-certs\"" Jul 16 17:16:57.718447 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:16:57.718427 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-tls\"" Jul 16 17:16:57.718567 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:16:57.718520 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-ca4gmnf4njfo2\"" Jul 16 17:16:57.718567 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:16:57.718532 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-dockercfg-tmd26\"" Jul 16 17:16:57.733589 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:16:57.733569 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-7d9cb857b5-4c5kx"] Jul 16 17:16:57.737193 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:16:57.737172 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/metrics-server-74dc84475b-hznm2"] Jul 16 17:16:57.737299 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:16:57.737279 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7d9cb857b5-4c5kx" Jul 16 17:16:57.745367 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:16:57.745102 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"trusted-ca-bundle\"" Jul 16 17:16:57.758750 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:16:57.758730 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-7d9cb857b5-4c5kx"] Jul 16 17:16:57.839009 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:16:57.838968 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/1a3f3622-8378-41e6-87f7-74be93905ac4-console-oauth-config\") pod \"console-7d9cb857b5-4c5kx\" (UID: \"1a3f3622-8378-41e6-87f7-74be93905ac4\") " pod="openshift-console/console-7d9cb857b5-4c5kx" Jul 16 17:16:57.839410 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:16:57.839013 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/1a3f3622-8378-41e6-87f7-74be93905ac4-console-config\") pod \"console-7d9cb857b5-4c5kx\" (UID: \"1a3f3622-8378-41e6-87f7-74be93905ac4\") " pod="openshift-console/console-7d9cb857b5-4c5kx" Jul 16 17:16:57.839410 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:16:57.839042 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/bbd0744b-a415-4029-933b-c318369ac9ee-secret-metrics-server-client-certs\") pod \"metrics-server-74dc84475b-hznm2\" (UID: \"bbd0744b-a415-4029-933b-c318369ac9ee\") " pod="openshift-monitoring/metrics-server-74dc84475b-hznm2" Jul 16 17:16:57.839410 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:16:57.839088 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/bbd0744b-a415-4029-933b-c318369ac9ee-audit-log\") pod \"metrics-server-74dc84475b-hznm2\" (UID: \"bbd0744b-a415-4029-933b-c318369ac9ee\") " pod="openshift-monitoring/metrics-server-74dc84475b-hznm2" Jul 16 17:16:57.839410 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:16:57.839127 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/1a3f3622-8378-41e6-87f7-74be93905ac4-oauth-serving-cert\") pod \"console-7d9cb857b5-4c5kx\" (UID: \"1a3f3622-8378-41e6-87f7-74be93905ac4\") " pod="openshift-console/console-7d9cb857b5-4c5kx" Jul 16 17:16:57.839410 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:16:57.839166 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/1a3f3622-8378-41e6-87f7-74be93905ac4-console-serving-cert\") pod \"console-7d9cb857b5-4c5kx\" (UID: \"1a3f3622-8378-41e6-87f7-74be93905ac4\") " pod="openshift-console/console-7d9cb857b5-4c5kx" Jul 16 17:16:57.839410 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:16:57.839216 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/bbd0744b-a415-4029-933b-c318369ac9ee-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-74dc84475b-hznm2\" (UID: \"bbd0744b-a415-4029-933b-c318369ac9ee\") " pod="openshift-monitoring/metrics-server-74dc84475b-hznm2" Jul 16 17:16:57.839410 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:16:57.839254 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/bbd0744b-a415-4029-933b-c318369ac9ee-metrics-server-audit-profiles\") pod \"metrics-server-74dc84475b-hznm2\" (UID: \"bbd0744b-a415-4029-933b-c318369ac9ee\") " pod="openshift-monitoring/metrics-server-74dc84475b-hznm2" Jul 16 17:16:57.839410 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:16:57.839284 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/bbd0744b-a415-4029-933b-c318369ac9ee-secret-metrics-server-tls\") pod \"metrics-server-74dc84475b-hznm2\" (UID: \"bbd0744b-a415-4029-933b-c318369ac9ee\") " pod="openshift-monitoring/metrics-server-74dc84475b-hznm2" Jul 16 17:16:57.839410 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:16:57.839323 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/1a3f3622-8378-41e6-87f7-74be93905ac4-service-ca\") pod \"console-7d9cb857b5-4c5kx\" (UID: \"1a3f3622-8378-41e6-87f7-74be93905ac4\") " pod="openshift-console/console-7d9cb857b5-4c5kx" Jul 16 17:16:57.839410 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:16:57.839369 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/bbd0744b-a415-4029-933b-c318369ac9ee-client-ca-bundle\") pod \"metrics-server-74dc84475b-hznm2\" (UID: \"bbd0744b-a415-4029-933b-c318369ac9ee\") " pod="openshift-monitoring/metrics-server-74dc84475b-hznm2" Jul 16 17:16:57.839830 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:16:57.839451 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/1a3f3622-8378-41e6-87f7-74be93905ac4-trusted-ca-bundle\") pod \"console-7d9cb857b5-4c5kx\" (UID: \"1a3f3622-8378-41e6-87f7-74be93905ac4\") " pod="openshift-console/console-7d9cb857b5-4c5kx" Jul 16 17:16:57.839830 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:16:57.839482 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jbh5l\" (UniqueName: \"kubernetes.io/projected/1a3f3622-8378-41e6-87f7-74be93905ac4-kube-api-access-jbh5l\") pod \"console-7d9cb857b5-4c5kx\" (UID: \"1a3f3622-8378-41e6-87f7-74be93905ac4\") " pod="openshift-console/console-7d9cb857b5-4c5kx" Jul 16 17:16:57.839830 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:16:57.839503 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2crlx\" (UniqueName: \"kubernetes.io/projected/bbd0744b-a415-4029-933b-c318369ac9ee-kube-api-access-2crlx\") pod \"metrics-server-74dc84475b-hznm2\" (UID: \"bbd0744b-a415-4029-933b-c318369ac9ee\") " pod="openshift-monitoring/metrics-server-74dc84475b-hznm2" Jul 16 17:16:57.940588 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:16:57.940552 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/1a3f3622-8378-41e6-87f7-74be93905ac4-console-serving-cert\") pod \"console-7d9cb857b5-4c5kx\" (UID: \"1a3f3622-8378-41e6-87f7-74be93905ac4\") " pod="openshift-console/console-7d9cb857b5-4c5kx" Jul 16 17:16:57.940776 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:16:57.940618 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/bbd0744b-a415-4029-933b-c318369ac9ee-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-74dc84475b-hznm2\" (UID: \"bbd0744b-a415-4029-933b-c318369ac9ee\") " pod="openshift-monitoring/metrics-server-74dc84475b-hznm2" Jul 16 17:16:57.940776 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:16:57.940656 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/bbd0744b-a415-4029-933b-c318369ac9ee-metrics-server-audit-profiles\") pod \"metrics-server-74dc84475b-hznm2\" (UID: \"bbd0744b-a415-4029-933b-c318369ac9ee\") " pod="openshift-monitoring/metrics-server-74dc84475b-hznm2" Jul 16 17:16:57.940776 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:16:57.940681 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/bbd0744b-a415-4029-933b-c318369ac9ee-secret-metrics-server-tls\") pod \"metrics-server-74dc84475b-hznm2\" (UID: \"bbd0744b-a415-4029-933b-c318369ac9ee\") " pod="openshift-monitoring/metrics-server-74dc84475b-hznm2" Jul 16 17:16:57.940940 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:16:57.940810 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/1a3f3622-8378-41e6-87f7-74be93905ac4-service-ca\") pod \"console-7d9cb857b5-4c5kx\" (UID: \"1a3f3622-8378-41e6-87f7-74be93905ac4\") " pod="openshift-console/console-7d9cb857b5-4c5kx" Jul 16 17:16:57.940940 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:16:57.940850 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/bbd0744b-a415-4029-933b-c318369ac9ee-client-ca-bundle\") pod \"metrics-server-74dc84475b-hznm2\" (UID: \"bbd0744b-a415-4029-933b-c318369ac9ee\") " pod="openshift-monitoring/metrics-server-74dc84475b-hznm2" Jul 16 17:16:57.940940 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:16:57.940904 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/1a3f3622-8378-41e6-87f7-74be93905ac4-trusted-ca-bundle\") pod \"console-7d9cb857b5-4c5kx\" (UID: \"1a3f3622-8378-41e6-87f7-74be93905ac4\") " pod="openshift-console/console-7d9cb857b5-4c5kx" Jul 16 17:16:57.940940 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:16:57.940940 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jbh5l\" (UniqueName: \"kubernetes.io/projected/1a3f3622-8378-41e6-87f7-74be93905ac4-kube-api-access-jbh5l\") pod \"console-7d9cb857b5-4c5kx\" (UID: \"1a3f3622-8378-41e6-87f7-74be93905ac4\") " pod="openshift-console/console-7d9cb857b5-4c5kx" Jul 16 17:16:57.941136 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:16:57.940958 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2crlx\" (UniqueName: \"kubernetes.io/projected/bbd0744b-a415-4029-933b-c318369ac9ee-kube-api-access-2crlx\") pod \"metrics-server-74dc84475b-hznm2\" (UID: \"bbd0744b-a415-4029-933b-c318369ac9ee\") " pod="openshift-monitoring/metrics-server-74dc84475b-hznm2" Jul 16 17:16:57.941136 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:16:57.941040 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/1a3f3622-8378-41e6-87f7-74be93905ac4-console-oauth-config\") pod \"console-7d9cb857b5-4c5kx\" (UID: \"1a3f3622-8378-41e6-87f7-74be93905ac4\") " pod="openshift-console/console-7d9cb857b5-4c5kx" Jul 16 17:16:57.941136 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:16:57.941082 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/1a3f3622-8378-41e6-87f7-74be93905ac4-console-config\") pod \"console-7d9cb857b5-4c5kx\" (UID: \"1a3f3622-8378-41e6-87f7-74be93905ac4\") " pod="openshift-console/console-7d9cb857b5-4c5kx" Jul 16 17:16:57.941136 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:16:57.941113 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/bbd0744b-a415-4029-933b-c318369ac9ee-secret-metrics-server-client-certs\") pod \"metrics-server-74dc84475b-hznm2\" (UID: \"bbd0744b-a415-4029-933b-c318369ac9ee\") " pod="openshift-monitoring/metrics-server-74dc84475b-hznm2" Jul 16 17:16:57.941343 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:16:57.941142 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/bbd0744b-a415-4029-933b-c318369ac9ee-audit-log\") pod \"metrics-server-74dc84475b-hznm2\" (UID: \"bbd0744b-a415-4029-933b-c318369ac9ee\") " pod="openshift-monitoring/metrics-server-74dc84475b-hznm2" Jul 16 17:16:57.941343 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:16:57.941170 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/1a3f3622-8378-41e6-87f7-74be93905ac4-oauth-serving-cert\") pod \"console-7d9cb857b5-4c5kx\" (UID: \"1a3f3622-8378-41e6-87f7-74be93905ac4\") " pod="openshift-console/console-7d9cb857b5-4c5kx" Jul 16 17:16:57.942118 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:16:57.941794 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/1a3f3622-8378-41e6-87f7-74be93905ac4-service-ca\") pod \"console-7d9cb857b5-4c5kx\" (UID: \"1a3f3622-8378-41e6-87f7-74be93905ac4\") " pod="openshift-console/console-7d9cb857b5-4c5kx" Jul 16 17:16:57.942118 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:16:57.941812 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/bbd0744b-a415-4029-933b-c318369ac9ee-metrics-server-audit-profiles\") pod \"metrics-server-74dc84475b-hznm2\" (UID: \"bbd0744b-a415-4029-933b-c318369ac9ee\") " pod="openshift-monitoring/metrics-server-74dc84475b-hznm2" Jul 16 17:16:57.942118 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:16:57.942087 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/1a3f3622-8378-41e6-87f7-74be93905ac4-trusted-ca-bundle\") pod \"console-7d9cb857b5-4c5kx\" (UID: \"1a3f3622-8378-41e6-87f7-74be93905ac4\") " pod="openshift-console/console-7d9cb857b5-4c5kx" Jul 16 17:16:57.942320 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:16:57.942140 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/bbd0744b-a415-4029-933b-c318369ac9ee-audit-log\") pod \"metrics-server-74dc84475b-hznm2\" (UID: \"bbd0744b-a415-4029-933b-c318369ac9ee\") " pod="openshift-monitoring/metrics-server-74dc84475b-hznm2" Jul 16 17:16:57.942320 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:16:57.942239 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/1a3f3622-8378-41e6-87f7-74be93905ac4-oauth-serving-cert\") pod \"console-7d9cb857b5-4c5kx\" (UID: \"1a3f3622-8378-41e6-87f7-74be93905ac4\") " pod="openshift-console/console-7d9cb857b5-4c5kx" Jul 16 17:16:57.942726 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:16:57.942660 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/1a3f3622-8378-41e6-87f7-74be93905ac4-console-config\") pod \"console-7d9cb857b5-4c5kx\" (UID: \"1a3f3622-8378-41e6-87f7-74be93905ac4\") " pod="openshift-console/console-7d9cb857b5-4c5kx" Jul 16 17:16:57.943039 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:16:57.942956 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/bbd0744b-a415-4029-933b-c318369ac9ee-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-74dc84475b-hznm2\" (UID: \"bbd0744b-a415-4029-933b-c318369ac9ee\") " pod="openshift-monitoring/metrics-server-74dc84475b-hznm2" Jul 16 17:16:57.943883 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:16:57.943840 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/bbd0744b-a415-4029-933b-c318369ac9ee-secret-metrics-server-tls\") pod \"metrics-server-74dc84475b-hznm2\" (UID: \"bbd0744b-a415-4029-933b-c318369ac9ee\") " pod="openshift-monitoring/metrics-server-74dc84475b-hznm2" Jul 16 17:16:57.943982 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:16:57.943928 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/1a3f3622-8378-41e6-87f7-74be93905ac4-console-serving-cert\") pod \"console-7d9cb857b5-4c5kx\" (UID: \"1a3f3622-8378-41e6-87f7-74be93905ac4\") " pod="openshift-console/console-7d9cb857b5-4c5kx" Jul 16 17:16:57.944535 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:16:57.944515 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/1a3f3622-8378-41e6-87f7-74be93905ac4-console-oauth-config\") pod \"console-7d9cb857b5-4c5kx\" (UID: \"1a3f3622-8378-41e6-87f7-74be93905ac4\") " pod="openshift-console/console-7d9cb857b5-4c5kx" Jul 16 17:16:57.944885 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:16:57.944865 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/bbd0744b-a415-4029-933b-c318369ac9ee-secret-metrics-server-client-certs\") pod \"metrics-server-74dc84475b-hznm2\" (UID: \"bbd0744b-a415-4029-933b-c318369ac9ee\") " pod="openshift-monitoring/metrics-server-74dc84475b-hznm2" Jul 16 17:16:57.945036 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:16:57.945014 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/bbd0744b-a415-4029-933b-c318369ac9ee-client-ca-bundle\") pod \"metrics-server-74dc84475b-hznm2\" (UID: \"bbd0744b-a415-4029-933b-c318369ac9ee\") " pod="openshift-monitoring/metrics-server-74dc84475b-hznm2" Jul 16 17:16:57.955355 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:16:57.955330 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2crlx\" (UniqueName: \"kubernetes.io/projected/bbd0744b-a415-4029-933b-c318369ac9ee-kube-api-access-2crlx\") pod \"metrics-server-74dc84475b-hznm2\" (UID: \"bbd0744b-a415-4029-933b-c318369ac9ee\") " pod="openshift-monitoring/metrics-server-74dc84475b-hznm2" Jul 16 17:16:57.957501 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:16:57.957481 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jbh5l\" (UniqueName: \"kubernetes.io/projected/1a3f3622-8378-41e6-87f7-74be93905ac4-kube-api-access-jbh5l\") pod \"console-7d9cb857b5-4c5kx\" (UID: \"1a3f3622-8378-41e6-87f7-74be93905ac4\") " pod="openshift-console/console-7d9cb857b5-4c5kx" Jul 16 17:16:58.024758 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:16:58.024670 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/metrics-server-74dc84475b-hznm2" Jul 16 17:16:58.048406 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:16:58.048358 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7d9cb857b5-4c5kx" Jul 16 17:16:58.143529 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:16:58.143472 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/f99dd101-e4fa-4c68-a502-c3a14be5c7fb-metrics-tls\") pod \"dns-default-vtkgg\" (UID: \"f99dd101-e4fa-4c68-a502-c3a14be5c7fb\") " pod="openshift-dns/dns-default-vtkgg" Jul 16 17:16:58.146409 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:16:58.146369 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/f99dd101-e4fa-4c68-a502-c3a14be5c7fb-metrics-tls\") pod \"dns-default-vtkgg\" (UID: \"f99dd101-e4fa-4c68-a502-c3a14be5c7fb\") " pod="openshift-dns/dns-default-vtkgg" Jul 16 17:16:58.216233 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:16:58.216207 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/metrics-server-74dc84475b-hznm2"] Jul 16 17:16:58.219722 ip-10-0-132-130 kubenswrapper[2578]: W0716 17:16:58.219677 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbbd0744b_a415_4029_933b_c318369ac9ee.slice/crio-5122c2c15628cee03f7364dbbf86c72f0ffb97dfe86f7f20513522ee2fc16d1c WatchSource:0}: Error finding container 5122c2c15628cee03f7364dbbf86c72f0ffb97dfe86f7f20513522ee2fc16d1c: Status 404 returned error can't find the container with id 5122c2c15628cee03f7364dbbf86c72f0ffb97dfe86f7f20513522ee2fc16d1c Jul 16 17:16:58.240065 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:16:58.240039 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-7d9cb857b5-4c5kx"] Jul 16 17:16:58.243096 ip-10-0-132-130 kubenswrapper[2578]: W0716 17:16:58.243068 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1a3f3622_8378_41e6_87f7_74be93905ac4.slice/crio-65dc627053de11a73d97d139b89a85ef8b6919317cda37314a0a101235e3c003 WatchSource:0}: Error finding container 65dc627053de11a73d97d139b89a85ef8b6919317cda37314a0a101235e3c003: Status 404 returned error can't find the container with id 65dc627053de11a73d97d139b89a85ef8b6919317cda37314a0a101235e3c003 Jul 16 17:16:58.289198 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:16:58.289177 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-bfmt5\"" Jul 16 17:16:58.297344 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:16:58.297321 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-vtkgg" Jul 16 17:16:58.455620 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:16:58.455584 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-vtkgg"] Jul 16 17:16:58.459563 ip-10-0-132-130 kubenswrapper[2578]: W0716 17:16:58.459539 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf99dd101_e4fa_4c68_a502_c3a14be5c7fb.slice/crio-1a4eac04b3ad77205bd512ad9af7229a53b49f8aa4b168b4176bee8d4d76de3b WatchSource:0}: Error finding container 1a4eac04b3ad77205bd512ad9af7229a53b49f8aa4b168b4176bee8d4d76de3b: Status 404 returned error can't find the container with id 1a4eac04b3ad77205bd512ad9af7229a53b49f8aa4b168b4176bee8d4d76de3b Jul 16 17:16:58.824760 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:16:58.824719 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/metrics-server-74dc84475b-hznm2" event={"ID":"bbd0744b-a415-4029-933b-c318369ac9ee","Type":"ContainerStarted","Data":"5122c2c15628cee03f7364dbbf86c72f0ffb97dfe86f7f20513522ee2fc16d1c"} Jul 16 17:16:58.826488 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:16:58.826449 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7d9cb857b5-4c5kx" event={"ID":"1a3f3622-8378-41e6-87f7-74be93905ac4","Type":"ContainerStarted","Data":"c6339ab1f10122ff0f55c70741dd4411c1dab6e393dec249fb82cfde095bef7d"} Jul 16 17:16:58.826606 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:16:58.826502 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7d9cb857b5-4c5kx" event={"ID":"1a3f3622-8378-41e6-87f7-74be93905ac4","Type":"ContainerStarted","Data":"65dc627053de11a73d97d139b89a85ef8b6919317cda37314a0a101235e3c003"} Jul 16 17:16:58.827654 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:16:58.827630 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-vtkgg" event={"ID":"f99dd101-e4fa-4c68-a502-c3a14be5c7fb","Type":"ContainerStarted","Data":"1a4eac04b3ad77205bd512ad9af7229a53b49f8aa4b168b4176bee8d4d76de3b"} Jul 16 17:16:58.850180 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:16:58.850120 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-7d9cb857b5-4c5kx" podStartSLOduration=1.850100326 podStartE2EDuration="1.850100326s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 17:16:57 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 17:16:58.849187454 +0000 UTC m=+163.335031344" watchObservedRunningTime="2026-07-16 17:16:58.850100326 +0000 UTC m=+163.335944219" Jul 16 17:17:05.755471 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:17:05.755436 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-image-registry/image-registry-ff66d4bb5-czxjw" Jul 16 17:17:07.872764 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:17:07.872673 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/downloads-68c9cbb497-lrlw4" event={"ID":"fd67d296-3c7d-47e1-89c2-d4e3f8edb1cc","Type":"ContainerStarted","Data":"8f9d8e2b728c52219154403036a7f893b84fa2fc4d8a3f5c21c87d926f6b1df8"} Jul 16 17:17:07.873549 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:17:07.873500 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/downloads-68c9cbb497-lrlw4" Jul 16 17:17:07.875604 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:17:07.875579 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/metrics-server-74dc84475b-hznm2" event={"ID":"bbd0744b-a415-4029-933b-c318369ac9ee","Type":"ContainerStarted","Data":"9fe39c7b553b06cc109edb691871a0346f135d9a1311e14c9039114923366b2c"} Jul 16 17:17:07.886719 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:17:07.886677 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/downloads-68c9cbb497-lrlw4" Jul 16 17:17:07.900938 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:17:07.900840 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/downloads-68c9cbb497-lrlw4" podStartSLOduration=3.51291066 podStartE2EDuration="27.900823393s" totalImagesPullingTime="24.387912733s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 17:16:40 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 17:17:07.900794618 +0000 UTC m=+172.386638507" watchObservedRunningTime="2026-07-16 17:17:07.900823393 +0000 UTC m=+172.386667282" Jul 16 17:17:07.965438 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:17:07.965350 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/metrics-server-74dc84475b-hznm2" podStartSLOduration=2.325289169 podStartE2EDuration="10.965327904s" totalImagesPullingTime="8.640038735s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 17:16:57 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 17:17:07.963153658 +0000 UTC m=+172.448997546" watchObservedRunningTime="2026-07-16 17:17:07.965327904 +0000 UTC m=+172.451171797" Jul 16 17:17:08.048737 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:17:08.048682 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-7d9cb857b5-4c5kx" Jul 16 17:17:08.048916 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:17:08.048801 2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-7d9cb857b5-4c5kx" Jul 16 17:17:08.054487 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:17:08.054465 2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-7d9cb857b5-4c5kx" Jul 16 17:17:08.885432 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:17:08.885404 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-7d9cb857b5-4c5kx" Jul 16 17:17:08.968076 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:17:08.968053 2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-54565489c9-dcjs7"] Jul 16 17:17:09.209013 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:17:09.208939 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-kvr2r" Jul 16 17:17:09.891228 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:17:09.891189 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-6769966f79-57q7b" event={"ID":"8cb88b9c-ade2-499e-b959-3e158a068497","Type":"ContainerStarted","Data":"1abc6c3d8f6be6e4efd70332f2743ee55db9714303f94d390b6798e64f21ee71"} Jul 16 17:17:09.891717 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:17:09.891270 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-6769966f79-57q7b" event={"ID":"8cb88b9c-ade2-499e-b959-3e158a068497","Type":"ContainerStarted","Data":"7375aafc5ad5c6b904978449ede5e7a10262233dfc438a3f1f52793eb123d8c8"} Jul 16 17:17:09.891717 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:17:09.891314 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-6769966f79-57q7b" event={"ID":"8cb88b9c-ade2-499e-b959-3e158a068497","Type":"ContainerStarted","Data":"53b56f17e1816e14fc176669c4e2182592ffcea616b1c9bf65ed62ba195b2d23"} Jul 16 17:17:09.893285 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:17:09.893261 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-vtkgg" event={"ID":"f99dd101-e4fa-4c68-a502-c3a14be5c7fb","Type":"ContainerStarted","Data":"a73b277e33169a257c515e77e00098002b9e88370449f49ff1f69230abcf3aaf"} Jul 16 17:17:09.893446 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:17:09.893417 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-vtkgg" Jul 16 17:17:09.893548 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:17:09.893469 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-vtkgg" event={"ID":"f99dd101-e4fa-4c68-a502-c3a14be5c7fb","Type":"ContainerStarted","Data":"1ed5854cbd9b70fd31474dc733164a5ab9801f555c827cf97424daed98a869a8"} Jul 16 17:17:09.932905 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:17:09.932864 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/dns-default-vtkgg" podStartSLOduration=129.168524282 podStartE2EDuration="2m19.932847933s" totalImagesPullingTime="10.764323651s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 17:14:50 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 17:17:09.930350472 +0000 UTC m=+174.416194363" watchObservedRunningTime="2026-07-16 17:17:09.932847933 +0000 UTC m=+174.418691822" Jul 16 17:17:11.909649 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:17:11.909609 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-6769966f79-57q7b" event={"ID":"8cb88b9c-ade2-499e-b959-3e158a068497","Type":"ContainerStarted","Data":"659eabb6bc183aab2232f8c8cff236fce2d1a8fa1bf3a4c83dccb732e41e86df"} Jul 16 17:17:11.910134 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:17:11.909714 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-6769966f79-57q7b" event={"ID":"8cb88b9c-ade2-499e-b959-3e158a068497","Type":"ContainerStarted","Data":"0c7ef2ca02c179c671dc27a670a0c9792cd4bcab847bc8571a21e1dd2b816c91"} Jul 16 17:17:11.910134 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:17:11.909748 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-6769966f79-57q7b" event={"ID":"8cb88b9c-ade2-499e-b959-3e158a068497","Type":"ContainerStarted","Data":"64da1f87f1ae6cec6b694f0452eff3f114fc13cf7412f2e7a459cd486af1e692"} Jul 16 17:17:11.910134 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:17:11.909954 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/thanos-querier-6769966f79-57q7b" Jul 16 17:17:11.936406 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:17:11.936354 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/thanos-querier-6769966f79-57q7b" podStartSLOduration=1.9884643560000002 podStartE2EDuration="15.936333821s" totalImagesPullingTime="13.947869465s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 17:16:56 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 17:17:11.934674807 +0000 UTC m=+176.420518696" watchObservedRunningTime="2026-07-16 17:17:11.936333821 +0000 UTC m=+176.422177711" Jul 16 17:17:17.921837 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:17:17.921808 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/thanos-querier-6769966f79-57q7b" Jul 16 17:17:18.024821 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:17:18.024786 2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/metrics-server-74dc84475b-hznm2" Jul 16 17:17:18.024996 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:17:18.024870 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/metrics-server-74dc84475b-hznm2" Jul 16 17:17:19.901236 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:17:19.901201 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-vtkgg" Jul 16 17:17:25.978319 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:17:25.978287 2578 generic.go:356] "Generic (PLEG): container finished" podID="91a757db-58f1-4483-baf4-9b936e4b8072" containerID="246793b0185a86e70e543ba0a03280bee39480ea0ba8da5ca7103a2ae4fd86dd" exitCode=0 Jul 16 17:17:25.978878 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:17:25.978332 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-8q957" event={"ID":"91a757db-58f1-4483-baf4-9b936e4b8072","Type":"ContainerDied","Data":"246793b0185a86e70e543ba0a03280bee39480ea0ba8da5ca7103a2ae4fd86dd"} Jul 16 17:17:25.978878 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:17:25.978758 2578 scope.go:117] "RemoveContainer" containerID="246793b0185a86e70e543ba0a03280bee39480ea0ba8da5ca7103a2ae4fd86dd" Jul 16 17:17:26.984372 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:17:26.984337 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-8q957" event={"ID":"91a757db-58f1-4483-baf4-9b936e4b8072","Type":"ContainerStarted","Data":"92a09f35c70f21d26a730184be3bdff4f9441cac81b6a8e23213fec22b0d57fa"} Jul 16 17:17:32.005388 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:17:32.005352 2578 generic.go:356] "Generic (PLEG): container finished" podID="14f6cc76-f1b6-4d19-8bc0-74aa1e1665c7" containerID="2071a100c20519130d7c9944ea4c4658a02a13d320413f747e8727bdf5043229" exitCode=0 Jul 16 17:17:32.005869 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:17:32.005427 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-7695fc6c8c-hvxkc" event={"ID":"14f6cc76-f1b6-4d19-8bc0-74aa1e1665c7","Type":"ContainerDied","Data":"2071a100c20519130d7c9944ea4c4658a02a13d320413f747e8727bdf5043229"} Jul 16 17:17:32.005869 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:17:32.005771 2578 scope.go:117] "RemoveContainer" containerID="2071a100c20519130d7c9944ea4c4658a02a13d320413f747e8727bdf5043229" Jul 16 17:17:33.011744 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:17:33.011710 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-7695fc6c8c-hvxkc" event={"ID":"14f6cc76-f1b6-4d19-8bc0-74aa1e1665c7","Type":"ContainerStarted","Data":"10449772d0671f043c8cb969d4ee9ae85ae0228e68f6926272662a485de97246"} Jul 16 17:17:34.021928 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:17:34.021889 2578 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-54565489c9-dcjs7" podUID="d4cbd1db-79c1-4af2-9866-53b8aeee1bec" containerName="console" containerID="cri-o://e6d5a6a935f1725e01a9236966d24765ba029200bd81fcf4013a65492a65dda2" gracePeriod=15 Jul 16 17:17:34.301160 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:17:34.301131 2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-54565489c9-dcjs7" Jul 16 17:17:34.376273 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:17:34.376246 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/d4cbd1db-79c1-4af2-9866-53b8aeee1bec-console-serving-cert\") pod \"d4cbd1db-79c1-4af2-9866-53b8aeee1bec\" (UID: \"d4cbd1db-79c1-4af2-9866-53b8aeee1bec\") " Jul 16 17:17:34.376462 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:17:34.376291 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/d4cbd1db-79c1-4af2-9866-53b8aeee1bec-service-ca\") pod \"d4cbd1db-79c1-4af2-9866-53b8aeee1bec\" (UID: \"d4cbd1db-79c1-4af2-9866-53b8aeee1bec\") " Jul 16 17:17:34.376462 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:17:34.376323 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-vnjf9\" (UniqueName: \"kubernetes.io/projected/d4cbd1db-79c1-4af2-9866-53b8aeee1bec-kube-api-access-vnjf9\") pod \"d4cbd1db-79c1-4af2-9866-53b8aeee1bec\" (UID: \"d4cbd1db-79c1-4af2-9866-53b8aeee1bec\") " Jul 16 17:17:34.376462 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:17:34.376359 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/d4cbd1db-79c1-4af2-9866-53b8aeee1bec-console-config\") pod \"d4cbd1db-79c1-4af2-9866-53b8aeee1bec\" (UID: \"d4cbd1db-79c1-4af2-9866-53b8aeee1bec\") " Jul 16 17:17:34.376462 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:17:34.376375 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/d4cbd1db-79c1-4af2-9866-53b8aeee1bec-oauth-serving-cert\") pod \"d4cbd1db-79c1-4af2-9866-53b8aeee1bec\" (UID: \"d4cbd1db-79c1-4af2-9866-53b8aeee1bec\") " Jul 16 17:17:34.376462 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:17:34.376432 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/d4cbd1db-79c1-4af2-9866-53b8aeee1bec-console-oauth-config\") pod \"d4cbd1db-79c1-4af2-9866-53b8aeee1bec\" (UID: \"d4cbd1db-79c1-4af2-9866-53b8aeee1bec\") " Jul 16 17:17:34.376820 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:17:34.376792 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d4cbd1db-79c1-4af2-9866-53b8aeee1bec-service-ca" (OuterVolumeSpecName: "service-ca") pod "d4cbd1db-79c1-4af2-9866-53b8aeee1bec" (UID: "d4cbd1db-79c1-4af2-9866-53b8aeee1bec"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 16 17:17:34.376820 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:17:34.376806 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d4cbd1db-79c1-4af2-9866-53b8aeee1bec-console-config" (OuterVolumeSpecName: "console-config") pod "d4cbd1db-79c1-4af2-9866-53b8aeee1bec" (UID: "d4cbd1db-79c1-4af2-9866-53b8aeee1bec"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 16 17:17:34.376927 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:17:34.376810 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d4cbd1db-79c1-4af2-9866-53b8aeee1bec-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "d4cbd1db-79c1-4af2-9866-53b8aeee1bec" (UID: "d4cbd1db-79c1-4af2-9866-53b8aeee1bec"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 16 17:17:34.378504 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:17:34.378480 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d4cbd1db-79c1-4af2-9866-53b8aeee1bec-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "d4cbd1db-79c1-4af2-9866-53b8aeee1bec" (UID: "d4cbd1db-79c1-4af2-9866-53b8aeee1bec"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 16 17:17:34.378504 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:17:34.378490 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d4cbd1db-79c1-4af2-9866-53b8aeee1bec-kube-api-access-vnjf9" (OuterVolumeSpecName: "kube-api-access-vnjf9") pod "d4cbd1db-79c1-4af2-9866-53b8aeee1bec" (UID: "d4cbd1db-79c1-4af2-9866-53b8aeee1bec"). InnerVolumeSpecName "kube-api-access-vnjf9". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 16 17:17:34.378653 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:17:34.378501 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d4cbd1db-79c1-4af2-9866-53b8aeee1bec-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "d4cbd1db-79c1-4af2-9866-53b8aeee1bec" (UID: "d4cbd1db-79c1-4af2-9866-53b8aeee1bec"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 16 17:17:34.477326 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:17:34.477294 2578 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/d4cbd1db-79c1-4af2-9866-53b8aeee1bec-console-serving-cert\") on node \"ip-10-0-132-130.ec2.internal\" DevicePath \"\"" Jul 16 17:17:34.477326 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:17:34.477319 2578 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/d4cbd1db-79c1-4af2-9866-53b8aeee1bec-service-ca\") on node \"ip-10-0-132-130.ec2.internal\" DevicePath \"\"" Jul 16 17:17:34.477326 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:17:34.477327 2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-vnjf9\" (UniqueName: \"kubernetes.io/projected/d4cbd1db-79c1-4af2-9866-53b8aeee1bec-kube-api-access-vnjf9\") on node \"ip-10-0-132-130.ec2.internal\" DevicePath \"\"" Jul 16 17:17:34.477525 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:17:34.477338 2578 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/d4cbd1db-79c1-4af2-9866-53b8aeee1bec-console-config\") on node \"ip-10-0-132-130.ec2.internal\" DevicePath \"\"" Jul 16 17:17:34.477525 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:17:34.477346 2578 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/d4cbd1db-79c1-4af2-9866-53b8aeee1bec-oauth-serving-cert\") on node \"ip-10-0-132-130.ec2.internal\" DevicePath \"\"" Jul 16 17:17:34.477525 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:17:34.477358 2578 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/d4cbd1db-79c1-4af2-9866-53b8aeee1bec-console-oauth-config\") on node \"ip-10-0-132-130.ec2.internal\" DevicePath \"\"" Jul 16 17:17:35.020492 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:17:35.020456 2578 generic.go:356] "Generic (PLEG): container finished" podID="d4cbd1db-79c1-4af2-9866-53b8aeee1bec" containerID="e6d5a6a935f1725e01a9236966d24765ba029200bd81fcf4013a65492a65dda2" exitCode=0 Jul 16 17:17:35.020492 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:17:35.020496 2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-54565489c9-dcjs7" Jul 16 17:17:35.020718 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:17:35.020517 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-54565489c9-dcjs7" event={"ID":"d4cbd1db-79c1-4af2-9866-53b8aeee1bec","Type":"ContainerDied","Data":"e6d5a6a935f1725e01a9236966d24765ba029200bd81fcf4013a65492a65dda2"} Jul 16 17:17:35.020718 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:17:35.020563 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-54565489c9-dcjs7" event={"ID":"d4cbd1db-79c1-4af2-9866-53b8aeee1bec","Type":"ContainerDied","Data":"e9d05a006c424680b445dbe99164b24648eae39abd00e9629c3f3d3bea7c3303"} Jul 16 17:17:35.020718 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:17:35.020647 2578 scope.go:117] "RemoveContainer" containerID="e6d5a6a935f1725e01a9236966d24765ba029200bd81fcf4013a65492a65dda2" Jul 16 17:17:35.029039 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:17:35.028851 2578 scope.go:117] "RemoveContainer" containerID="e6d5a6a935f1725e01a9236966d24765ba029200bd81fcf4013a65492a65dda2" Jul 16 17:17:35.029274 ip-10-0-132-130 kubenswrapper[2578]: E0716 17:17:35.029126 2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e6d5a6a935f1725e01a9236966d24765ba029200bd81fcf4013a65492a65dda2\": container with ID starting with e6d5a6a935f1725e01a9236966d24765ba029200bd81fcf4013a65492a65dda2 not found: ID does not exist" containerID="e6d5a6a935f1725e01a9236966d24765ba029200bd81fcf4013a65492a65dda2" Jul 16 17:17:35.029274 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:17:35.029155 2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e6d5a6a935f1725e01a9236966d24765ba029200bd81fcf4013a65492a65dda2"} err="failed to get container status \"e6d5a6a935f1725e01a9236966d24765ba029200bd81fcf4013a65492a65dda2\": rpc error: code = NotFound desc = could not find container \"e6d5a6a935f1725e01a9236966d24765ba029200bd81fcf4013a65492a65dda2\": container with ID starting with e6d5a6a935f1725e01a9236966d24765ba029200bd81fcf4013a65492a65dda2 not found: ID does not exist" Jul 16 17:17:35.046432 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:17:35.046407 2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-54565489c9-dcjs7"] Jul 16 17:17:35.051892 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:17:35.051871 2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-54565489c9-dcjs7"] Jul 16 17:17:36.214571 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:17:36.214542 2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d4cbd1db-79c1-4af2-9866-53b8aeee1bec" path="/var/lib/kubelet/pods/d4cbd1db-79c1-4af2-9866-53b8aeee1bec/volumes" Jul 16 17:17:38.031161 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:17:38.031135 2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/metrics-server-74dc84475b-hznm2" Jul 16 17:17:38.038080 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:17:38.036916 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/metrics-server-74dc84475b-hznm2" Jul 16 17:18:15.676066 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:18:15.676029 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 16 17:18:15.676612 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:18:15.676334 2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d4cbd1db-79c1-4af2-9866-53b8aeee1bec" containerName="console" Jul 16 17:18:15.676612 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:18:15.676346 2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="d4cbd1db-79c1-4af2-9866-53b8aeee1bec" containerName="console" Jul 16 17:18:15.676612 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:18:15.676405 2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="d4cbd1db-79c1-4af2-9866-53b8aeee1bec" containerName="console" Jul 16 17:18:15.680002 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:18:15.679986 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Jul 16 17:18:15.690460 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:18:15.690441 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy\"" Jul 16 17:18:15.690568 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:18:15.690441 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-web-config\"" Jul 16 17:18:15.690884 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:18:15.690872 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-cluster-tls-config\"" Jul 16 17:18:15.690982 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:18:15.690873 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls-assets-0\"" Jul 16 17:18:15.691270 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:18:15.691253 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-web\"" Jul 16 17:18:15.691511 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:18:15.691496 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls\"" Jul 16 17:18:15.691771 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:18:15.691759 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-dockercfg-pb42v\"" Jul 16 17:18:15.694062 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:18:15.694048 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-generated\"" Jul 16 17:18:15.694545 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:18:15.694527 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-metric\"" Jul 16 17:18:15.703937 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:18:15.703915 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"alertmanager-trusted-ca-bundle\"" Jul 16 17:18:15.711890 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:18:15.711871 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 16 17:18:15.798091 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:18:15.798069 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/4d4f4ff2-a8be-4e99-8198-7c78dca29787-web-config\") pod \"alertmanager-main-0\" (UID: \"4d4f4ff2-a8be-4e99-8198-7c78dca29787\") " pod="openshift-monitoring/alertmanager-main-0" Jul 16 17:18:15.798206 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:18:15.798108 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/4d4f4ff2-a8be-4e99-8198-7c78dca29787-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"4d4f4ff2-a8be-4e99-8198-7c78dca29787\") " pod="openshift-monitoring/alertmanager-main-0" Jul 16 17:18:15.798206 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:18:15.798140 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/4d4f4ff2-a8be-4e99-8198-7c78dca29787-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"4d4f4ff2-a8be-4e99-8198-7c78dca29787\") " pod="openshift-monitoring/alertmanager-main-0" Jul 16 17:18:15.798206 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:18:15.798173 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/4d4f4ff2-a8be-4e99-8198-7c78dca29787-config-out\") pod \"alertmanager-main-0\" (UID: \"4d4f4ff2-a8be-4e99-8198-7c78dca29787\") " pod="openshift-monitoring/alertmanager-main-0" Jul 16 17:18:15.798388 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:18:15.798206 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/4d4f4ff2-a8be-4e99-8198-7c78dca29787-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"4d4f4ff2-a8be-4e99-8198-7c78dca29787\") " pod="openshift-monitoring/alertmanager-main-0" Jul 16 17:18:15.798388 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:18:15.798229 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mw8l5\" (UniqueName: \"kubernetes.io/projected/4d4f4ff2-a8be-4e99-8198-7c78dca29787-kube-api-access-mw8l5\") pod \"alertmanager-main-0\" (UID: \"4d4f4ff2-a8be-4e99-8198-7c78dca29787\") " pod="openshift-monitoring/alertmanager-main-0" Jul 16 17:18:15.798388 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:18:15.798272 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/4d4f4ff2-a8be-4e99-8198-7c78dca29787-config-volume\") pod \"alertmanager-main-0\" (UID: \"4d4f4ff2-a8be-4e99-8198-7c78dca29787\") " pod="openshift-monitoring/alertmanager-main-0" Jul 16 17:18:15.798388 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:18:15.798321 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/4d4f4ff2-a8be-4e99-8198-7c78dca29787-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"4d4f4ff2-a8be-4e99-8198-7c78dca29787\") " pod="openshift-monitoring/alertmanager-main-0" Jul 16 17:18:15.798388 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:18:15.798371 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/4d4f4ff2-a8be-4e99-8198-7c78dca29787-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"4d4f4ff2-a8be-4e99-8198-7c78dca29787\") " pod="openshift-monitoring/alertmanager-main-0" Jul 16 17:18:15.798607 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:18:15.798407 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/4d4f4ff2-a8be-4e99-8198-7c78dca29787-tls-assets\") pod \"alertmanager-main-0\" (UID: \"4d4f4ff2-a8be-4e99-8198-7c78dca29787\") " pod="openshift-monitoring/alertmanager-main-0" Jul 16 17:18:15.798607 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:18:15.798436 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/4d4f4ff2-a8be-4e99-8198-7c78dca29787-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"4d4f4ff2-a8be-4e99-8198-7c78dca29787\") " pod="openshift-monitoring/alertmanager-main-0" Jul 16 17:18:15.798607 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:18:15.798461 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/4d4f4ff2-a8be-4e99-8198-7c78dca29787-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"4d4f4ff2-a8be-4e99-8198-7c78dca29787\") " pod="openshift-monitoring/alertmanager-main-0" Jul 16 17:18:15.798607 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:18:15.798486 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/4d4f4ff2-a8be-4e99-8198-7c78dca29787-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"4d4f4ff2-a8be-4e99-8198-7c78dca29787\") " pod="openshift-monitoring/alertmanager-main-0" Jul 16 17:18:15.899364 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:18:15.899331 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/4d4f4ff2-a8be-4e99-8198-7c78dca29787-config-volume\") pod \"alertmanager-main-0\" (UID: \"4d4f4ff2-a8be-4e99-8198-7c78dca29787\") " pod="openshift-monitoring/alertmanager-main-0" Jul 16 17:18:15.899364 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:18:15.899370 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/4d4f4ff2-a8be-4e99-8198-7c78dca29787-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"4d4f4ff2-a8be-4e99-8198-7c78dca29787\") " pod="openshift-monitoring/alertmanager-main-0" Jul 16 17:18:15.899540 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:18:15.899394 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/4d4f4ff2-a8be-4e99-8198-7c78dca29787-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"4d4f4ff2-a8be-4e99-8198-7c78dca29787\") " pod="openshift-monitoring/alertmanager-main-0" Jul 16 17:18:15.899540 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:18:15.899419 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/4d4f4ff2-a8be-4e99-8198-7c78dca29787-tls-assets\") pod \"alertmanager-main-0\" (UID: \"4d4f4ff2-a8be-4e99-8198-7c78dca29787\") " pod="openshift-monitoring/alertmanager-main-0" Jul 16 17:18:15.899540 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:18:15.899438 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/4d4f4ff2-a8be-4e99-8198-7c78dca29787-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"4d4f4ff2-a8be-4e99-8198-7c78dca29787\") " pod="openshift-monitoring/alertmanager-main-0" Jul 16 17:18:15.899540 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:18:15.899454 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/4d4f4ff2-a8be-4e99-8198-7c78dca29787-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"4d4f4ff2-a8be-4e99-8198-7c78dca29787\") " pod="openshift-monitoring/alertmanager-main-0" Jul 16 17:18:15.899540 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:18:15.899469 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/4d4f4ff2-a8be-4e99-8198-7c78dca29787-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"4d4f4ff2-a8be-4e99-8198-7c78dca29787\") " pod="openshift-monitoring/alertmanager-main-0" Jul 16 17:18:15.899540 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:18:15.899495 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/4d4f4ff2-a8be-4e99-8198-7c78dca29787-web-config\") pod \"alertmanager-main-0\" (UID: \"4d4f4ff2-a8be-4e99-8198-7c78dca29787\") " pod="openshift-monitoring/alertmanager-main-0" Jul 16 17:18:15.899540 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:18:15.899527 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/4d4f4ff2-a8be-4e99-8198-7c78dca29787-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"4d4f4ff2-a8be-4e99-8198-7c78dca29787\") " pod="openshift-monitoring/alertmanager-main-0" Jul 16 17:18:15.899879 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:18:15.899557 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/4d4f4ff2-a8be-4e99-8198-7c78dca29787-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"4d4f4ff2-a8be-4e99-8198-7c78dca29787\") " pod="openshift-monitoring/alertmanager-main-0" Jul 16 17:18:15.899879 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:18:15.899585 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/4d4f4ff2-a8be-4e99-8198-7c78dca29787-config-out\") pod \"alertmanager-main-0\" (UID: \"4d4f4ff2-a8be-4e99-8198-7c78dca29787\") " pod="openshift-monitoring/alertmanager-main-0" Jul 16 17:18:15.899879 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:18:15.899622 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/4d4f4ff2-a8be-4e99-8198-7c78dca29787-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"4d4f4ff2-a8be-4e99-8198-7c78dca29787\") " pod="openshift-monitoring/alertmanager-main-0" Jul 16 17:18:15.899879 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:18:15.899643 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mw8l5\" (UniqueName: \"kubernetes.io/projected/4d4f4ff2-a8be-4e99-8198-7c78dca29787-kube-api-access-mw8l5\") pod \"alertmanager-main-0\" (UID: \"4d4f4ff2-a8be-4e99-8198-7c78dca29787\") " pod="openshift-monitoring/alertmanager-main-0" Jul 16 17:18:15.900084 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:18:15.899908 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/4d4f4ff2-a8be-4e99-8198-7c78dca29787-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"4d4f4ff2-a8be-4e99-8198-7c78dca29787\") " pod="openshift-monitoring/alertmanager-main-0" Jul 16 17:18:15.900638 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:18:15.900327 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/4d4f4ff2-a8be-4e99-8198-7c78dca29787-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"4d4f4ff2-a8be-4e99-8198-7c78dca29787\") " pod="openshift-monitoring/alertmanager-main-0" Jul 16 17:18:15.901162 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:18:15.901135 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/4d4f4ff2-a8be-4e99-8198-7c78dca29787-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"4d4f4ff2-a8be-4e99-8198-7c78dca29787\") " pod="openshift-monitoring/alertmanager-main-0" Jul 16 17:18:15.902642 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:18:15.902612 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/4d4f4ff2-a8be-4e99-8198-7c78dca29787-config-volume\") pod \"alertmanager-main-0\" (UID: \"4d4f4ff2-a8be-4e99-8198-7c78dca29787\") " pod="openshift-monitoring/alertmanager-main-0" Jul 16 17:18:15.902771 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:18:15.902720 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/4d4f4ff2-a8be-4e99-8198-7c78dca29787-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"4d4f4ff2-a8be-4e99-8198-7c78dca29787\") " pod="openshift-monitoring/alertmanager-main-0" Jul 16 17:18:15.902771 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:18:15.902728 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/4d4f4ff2-a8be-4e99-8198-7c78dca29787-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"4d4f4ff2-a8be-4e99-8198-7c78dca29787\") " pod="openshift-monitoring/alertmanager-main-0" Jul 16 17:18:15.902888 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:18:15.902801 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/4d4f4ff2-a8be-4e99-8198-7c78dca29787-web-config\") pod \"alertmanager-main-0\" (UID: \"4d4f4ff2-a8be-4e99-8198-7c78dca29787\") " pod="openshift-monitoring/alertmanager-main-0" Jul 16 17:18:15.903029 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:18:15.903007 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/4d4f4ff2-a8be-4e99-8198-7c78dca29787-config-out\") pod \"alertmanager-main-0\" (UID: \"4d4f4ff2-a8be-4e99-8198-7c78dca29787\") " pod="openshift-monitoring/alertmanager-main-0" Jul 16 17:18:15.903401 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:18:15.903384 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/4d4f4ff2-a8be-4e99-8198-7c78dca29787-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"4d4f4ff2-a8be-4e99-8198-7c78dca29787\") " pod="openshift-monitoring/alertmanager-main-0" Jul 16 17:18:15.903485 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:18:15.903465 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/4d4f4ff2-a8be-4e99-8198-7c78dca29787-tls-assets\") pod \"alertmanager-main-0\" (UID: \"4d4f4ff2-a8be-4e99-8198-7c78dca29787\") " pod="openshift-monitoring/alertmanager-main-0" Jul 16 17:18:15.903647 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:18:15.903630 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/4d4f4ff2-a8be-4e99-8198-7c78dca29787-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"4d4f4ff2-a8be-4e99-8198-7c78dca29787\") " pod="openshift-monitoring/alertmanager-main-0" Jul 16 17:18:15.904627 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:18:15.904606 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/4d4f4ff2-a8be-4e99-8198-7c78dca29787-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"4d4f4ff2-a8be-4e99-8198-7c78dca29787\") " pod="openshift-monitoring/alertmanager-main-0" Jul 16 17:18:15.909570 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:18:15.909552 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mw8l5\" (UniqueName: \"kubernetes.io/projected/4d4f4ff2-a8be-4e99-8198-7c78dca29787-kube-api-access-mw8l5\") pod \"alertmanager-main-0\" (UID: \"4d4f4ff2-a8be-4e99-8198-7c78dca29787\") " pod="openshift-monitoring/alertmanager-main-0" Jul 16 17:18:15.992935 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:18:15.992915 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-dockercfg-pb42v\"" Jul 16 17:18:16.000250 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:18:16.000228 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Jul 16 17:18:16.147841 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:18:16.147808 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 16 17:18:16.152513 ip-10-0-132-130 kubenswrapper[2578]: W0716 17:18:16.152487 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4d4f4ff2_a8be_4e99_8198_7c78dca29787.slice/crio-9fce25cc0be5643bbbf7bdd41902d69fbade2bbacaad3db4d945aca12e89df0c WatchSource:0}: Error finding container 9fce25cc0be5643bbbf7bdd41902d69fbade2bbacaad3db4d945aca12e89df0c: Status 404 returned error can't find the container with id 9fce25cc0be5643bbbf7bdd41902d69fbade2bbacaad3db4d945aca12e89df0c Jul 16 17:18:16.181574 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:18:16.181543 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"4d4f4ff2-a8be-4e99-8198-7c78dca29787","Type":"ContainerStarted","Data":"9fce25cc0be5643bbbf7bdd41902d69fbade2bbacaad3db4d945aca12e89df0c"} Jul 16 17:18:18.190555 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:18:18.190522 2578 generic.go:356] "Generic (PLEG): container finished" podID="4d4f4ff2-a8be-4e99-8198-7c78dca29787" containerID="0c0d7b0394b12e3022db35b06f16d0168ceb4191a5d9fe99eaf402fa333055bd" exitCode=0 Jul 16 17:18:18.190966 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:18:18.190612 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"4d4f4ff2-a8be-4e99-8198-7c78dca29787","Type":"ContainerDied","Data":"0c0d7b0394b12e3022db35b06f16d0168ceb4191a5d9fe99eaf402fa333055bd"} Jul 16 17:18:20.200590 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:18:20.200565 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"4d4f4ff2-a8be-4e99-8198-7c78dca29787","Type":"ContainerStarted","Data":"75d6a23a596cdcc61aa7665f5ad9d1d8347c3a0fb88db9ecd9e9c598cc4741f8"} Jul 16 17:18:20.200892 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:18:20.200631 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"4d4f4ff2-a8be-4e99-8198-7c78dca29787","Type":"ContainerStarted","Data":"0bf8ab199217426dfc3979033aa12691d5069d23582cd5fd66cd25f5c24b673b"} Jul 16 17:18:20.200892 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:18:20.200659 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"4d4f4ff2-a8be-4e99-8198-7c78dca29787","Type":"ContainerStarted","Data":"c9ee8a3196d0d6df879e1380490173ba182185d8b34594d12300322fd081d43f"} Jul 16 17:18:21.207638 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:18:21.207600 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"4d4f4ff2-a8be-4e99-8198-7c78dca29787","Type":"ContainerStarted","Data":"ffe01feda46e354c378e46acc6b47709273caea229b46df29b63e1805b760be6"} Jul 16 17:18:21.208031 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:18:21.207666 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"4d4f4ff2-a8be-4e99-8198-7c78dca29787","Type":"ContainerStarted","Data":"4a5f18d1b4aa218f67b0b92a0ff81d001a39e3e8a4e117e0a0242acbf7584e6e"} Jul 16 17:18:21.208031 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:18:21.207713 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"4d4f4ff2-a8be-4e99-8198-7c78dca29787","Type":"ContainerStarted","Data":"81cfc98f9d6b047157d3d0dc10aed86d7e01b842a1923b32ea7c155120acb5d6"} Jul 16 17:18:21.245528 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:18:21.245466 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/alertmanager-main-0" podStartSLOduration=3.161771726 podStartE2EDuration="6.245448323s" totalImagesPullingTime="3.069368873s" totalInitContainerRuntime="14.307724ms" isStatefulPod=false podCreationTimestamp="2026-07-16 17:18:15 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 17:18:21.243512944 +0000 UTC m=+245.729356834" watchObservedRunningTime="2026-07-16 17:18:21.245448323 +0000 UTC m=+245.731292212" Jul 16 17:18:22.691053 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:18:22.691019 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-648c849fdc-bgjwc"] Jul 16 17:18:22.694522 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:18:22.694496 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-648c849fdc-bgjwc" Jul 16 17:18:22.717919 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:18:22.717894 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-648c849fdc-bgjwc"] Jul 16 17:18:22.762910 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:18:22.762875 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/d84daf4f-d234-4222-8576-ad7dbdcff2d3-console-oauth-config\") pod \"console-648c849fdc-bgjwc\" (UID: \"d84daf4f-d234-4222-8576-ad7dbdcff2d3\") " pod="openshift-console/console-648c849fdc-bgjwc" Jul 16 17:18:22.762910 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:18:22.762913 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d84daf4f-d234-4222-8576-ad7dbdcff2d3-trusted-ca-bundle\") pod \"console-648c849fdc-bgjwc\" (UID: \"d84daf4f-d234-4222-8576-ad7dbdcff2d3\") " pod="openshift-console/console-648c849fdc-bgjwc" Jul 16 17:18:22.763116 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:18:22.763007 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/d84daf4f-d234-4222-8576-ad7dbdcff2d3-oauth-serving-cert\") pod \"console-648c849fdc-bgjwc\" (UID: \"d84daf4f-d234-4222-8576-ad7dbdcff2d3\") " pod="openshift-console/console-648c849fdc-bgjwc" Jul 16 17:18:22.763116 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:18:22.763044 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dtm5h\" (UniqueName: \"kubernetes.io/projected/d84daf4f-d234-4222-8576-ad7dbdcff2d3-kube-api-access-dtm5h\") pod \"console-648c849fdc-bgjwc\" (UID: \"d84daf4f-d234-4222-8576-ad7dbdcff2d3\") " pod="openshift-console/console-648c849fdc-bgjwc" Jul 16 17:18:22.763116 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:18:22.763080 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/d84daf4f-d234-4222-8576-ad7dbdcff2d3-console-config\") pod \"console-648c849fdc-bgjwc\" (UID: \"d84daf4f-d234-4222-8576-ad7dbdcff2d3\") " pod="openshift-console/console-648c849fdc-bgjwc" Jul 16 17:18:22.763223 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:18:22.763128 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/d84daf4f-d234-4222-8576-ad7dbdcff2d3-service-ca\") pod \"console-648c849fdc-bgjwc\" (UID: \"d84daf4f-d234-4222-8576-ad7dbdcff2d3\") " pod="openshift-console/console-648c849fdc-bgjwc" Jul 16 17:18:22.763223 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:18:22.763145 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/d84daf4f-d234-4222-8576-ad7dbdcff2d3-console-serving-cert\") pod \"console-648c849fdc-bgjwc\" (UID: \"d84daf4f-d234-4222-8576-ad7dbdcff2d3\") " pod="openshift-console/console-648c849fdc-bgjwc" Jul 16 17:18:22.863941 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:18:22.863894 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/d84daf4f-d234-4222-8576-ad7dbdcff2d3-service-ca\") pod \"console-648c849fdc-bgjwc\" (UID: \"d84daf4f-d234-4222-8576-ad7dbdcff2d3\") " pod="openshift-console/console-648c849fdc-bgjwc" Jul 16 17:18:22.863941 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:18:22.863942 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/d84daf4f-d234-4222-8576-ad7dbdcff2d3-console-serving-cert\") pod \"console-648c849fdc-bgjwc\" (UID: \"d84daf4f-d234-4222-8576-ad7dbdcff2d3\") " pod="openshift-console/console-648c849fdc-bgjwc" Jul 16 17:18:22.864150 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:18:22.863977 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/d84daf4f-d234-4222-8576-ad7dbdcff2d3-console-oauth-config\") pod \"console-648c849fdc-bgjwc\" (UID: \"d84daf4f-d234-4222-8576-ad7dbdcff2d3\") " pod="openshift-console/console-648c849fdc-bgjwc" Jul 16 17:18:22.864150 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:18:22.863994 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d84daf4f-d234-4222-8576-ad7dbdcff2d3-trusted-ca-bundle\") pod \"console-648c849fdc-bgjwc\" (UID: \"d84daf4f-d234-4222-8576-ad7dbdcff2d3\") " pod="openshift-console/console-648c849fdc-bgjwc" Jul 16 17:18:22.864150 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:18:22.864046 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/d84daf4f-d234-4222-8576-ad7dbdcff2d3-oauth-serving-cert\") pod \"console-648c849fdc-bgjwc\" (UID: \"d84daf4f-d234-4222-8576-ad7dbdcff2d3\") " pod="openshift-console/console-648c849fdc-bgjwc" Jul 16 17:18:22.864150 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:18:22.864071 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dtm5h\" (UniqueName: \"kubernetes.io/projected/d84daf4f-d234-4222-8576-ad7dbdcff2d3-kube-api-access-dtm5h\") pod \"console-648c849fdc-bgjwc\" (UID: \"d84daf4f-d234-4222-8576-ad7dbdcff2d3\") " pod="openshift-console/console-648c849fdc-bgjwc" Jul 16 17:18:22.864325 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:18:22.864284 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/d84daf4f-d234-4222-8576-ad7dbdcff2d3-console-config\") pod \"console-648c849fdc-bgjwc\" (UID: \"d84daf4f-d234-4222-8576-ad7dbdcff2d3\") " pod="openshift-console/console-648c849fdc-bgjwc" Jul 16 17:18:22.864747 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:18:22.864721 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/d84daf4f-d234-4222-8576-ad7dbdcff2d3-service-ca\") pod \"console-648c849fdc-bgjwc\" (UID: \"d84daf4f-d234-4222-8576-ad7dbdcff2d3\") " pod="openshift-console/console-648c849fdc-bgjwc" Jul 16 17:18:22.864867 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:18:22.864748 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/d84daf4f-d234-4222-8576-ad7dbdcff2d3-oauth-serving-cert\") pod \"console-648c849fdc-bgjwc\" (UID: \"d84daf4f-d234-4222-8576-ad7dbdcff2d3\") " pod="openshift-console/console-648c849fdc-bgjwc" Jul 16 17:18:22.864927 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:18:22.864914 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/d84daf4f-d234-4222-8576-ad7dbdcff2d3-console-config\") pod \"console-648c849fdc-bgjwc\" (UID: \"d84daf4f-d234-4222-8576-ad7dbdcff2d3\") " pod="openshift-console/console-648c849fdc-bgjwc" Jul 16 17:18:22.864976 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:18:22.864926 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d84daf4f-d234-4222-8576-ad7dbdcff2d3-trusted-ca-bundle\") pod \"console-648c849fdc-bgjwc\" (UID: \"d84daf4f-d234-4222-8576-ad7dbdcff2d3\") " pod="openshift-console/console-648c849fdc-bgjwc" Jul 16 17:18:22.866566 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:18:22.866549 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/d84daf4f-d234-4222-8576-ad7dbdcff2d3-console-serving-cert\") pod \"console-648c849fdc-bgjwc\" (UID: \"d84daf4f-d234-4222-8576-ad7dbdcff2d3\") " pod="openshift-console/console-648c849fdc-bgjwc" Jul 16 17:18:22.866679 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:18:22.866654 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/d84daf4f-d234-4222-8576-ad7dbdcff2d3-console-oauth-config\") pod \"console-648c849fdc-bgjwc\" (UID: \"d84daf4f-d234-4222-8576-ad7dbdcff2d3\") " pod="openshift-console/console-648c849fdc-bgjwc" Jul 16 17:18:22.875271 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:18:22.875254 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dtm5h\" (UniqueName: \"kubernetes.io/projected/d84daf4f-d234-4222-8576-ad7dbdcff2d3-kube-api-access-dtm5h\") pod \"console-648c849fdc-bgjwc\" (UID: \"d84daf4f-d234-4222-8576-ad7dbdcff2d3\") " pod="openshift-console/console-648c849fdc-bgjwc" Jul 16 17:18:23.004532 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:18:23.004446 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-648c849fdc-bgjwc" Jul 16 17:18:23.140987 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:18:23.140957 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-648c849fdc-bgjwc"] Jul 16 17:18:23.143839 ip-10-0-132-130 kubenswrapper[2578]: W0716 17:18:23.143813 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd84daf4f_d234_4222_8576_ad7dbdcff2d3.slice/crio-3d77fdfacd6b8410238dba690cdc03cb6a97077644009bf0746b50a1d5b4453c WatchSource:0}: Error finding container 3d77fdfacd6b8410238dba690cdc03cb6a97077644009bf0746b50a1d5b4453c: Status 404 returned error can't find the container with id 3d77fdfacd6b8410238dba690cdc03cb6a97077644009bf0746b50a1d5b4453c Jul 16 17:18:23.219335 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:18:23.219306 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-648c849fdc-bgjwc" event={"ID":"d84daf4f-d234-4222-8576-ad7dbdcff2d3","Type":"ContainerStarted","Data":"8c293cac624a991a0ebf2c814fcfc2c4f0a4d5e9a37922ae35f3558d628ed8fb"} Jul 16 17:18:23.219479 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:18:23.219354 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-648c849fdc-bgjwc" event={"ID":"d84daf4f-d234-4222-8576-ad7dbdcff2d3","Type":"ContainerStarted","Data":"3d77fdfacd6b8410238dba690cdc03cb6a97077644009bf0746b50a1d5b4453c"} Jul 16 17:18:23.241748 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:18:23.241649 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-648c849fdc-bgjwc" podStartSLOduration=1.241627823 podStartE2EDuration="1.241627823s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 17:18:22 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 17:18:23.241622459 +0000 UTC m=+247.727466349" watchObservedRunningTime="2026-07-16 17:18:23.241627823 +0000 UTC m=+247.727471712" Jul 16 17:18:27.810175 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:18:27.810138 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f90822ad-ab14-4108-8294-2e76c35a8aa5-metrics-certs\") pod \"network-metrics-daemon-kvr2r\" (UID: \"f90822ad-ab14-4108-8294-2e76c35a8aa5\") " pod="openshift-multus/network-metrics-daemon-kvr2r" Jul 16 17:18:27.812432 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:18:27.812411 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f90822ad-ab14-4108-8294-2e76c35a8aa5-metrics-certs\") pod \"network-metrics-daemon-kvr2r\" (UID: \"f90822ad-ab14-4108-8294-2e76c35a8aa5\") " pod="openshift-multus/network-metrics-daemon-kvr2r" Jul 16 17:18:28.112450 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:18:28.112377 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-65bh8\"" Jul 16 17:18:28.119896 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:18:28.119879 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-kvr2r" Jul 16 17:18:28.252626 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:18:28.252590 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-kvr2r"] Jul 16 17:18:28.256304 ip-10-0-132-130 kubenswrapper[2578]: W0716 17:18:28.256276 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf90822ad_ab14_4108_8294_2e76c35a8aa5.slice/crio-c9cb91f2c78dc75d66141000610e50dd48e61ce3933e757bee5ac055627641c2 WatchSource:0}: Error finding container c9cb91f2c78dc75d66141000610e50dd48e61ce3933e757bee5ac055627641c2: Status 404 returned error can't find the container with id c9cb91f2c78dc75d66141000610e50dd48e61ce3933e757bee5ac055627641c2 Jul 16 17:18:29.243759 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:18:29.243264 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-kvr2r" event={"ID":"f90822ad-ab14-4108-8294-2e76c35a8aa5","Type":"ContainerStarted","Data":"c9cb91f2c78dc75d66141000610e50dd48e61ce3933e757bee5ac055627641c2"} Jul 16 17:18:30.248511 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:18:30.248469 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-kvr2r" event={"ID":"f90822ad-ab14-4108-8294-2e76c35a8aa5","Type":"ContainerStarted","Data":"b1d5428328d8b4780a1ee996b1d273bf25f4437ed1d7ef3b024cb6413285af77"} Jul 16 17:18:30.248511 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:18:30.248513 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-kvr2r" event={"ID":"f90822ad-ab14-4108-8294-2e76c35a8aa5","Type":"ContainerStarted","Data":"bec6273a942058e0e1009264909628b99cb5ff23e9c826808b5294f49d728799"} Jul 16 17:18:30.269386 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:18:30.269338 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-kvr2r" podStartSLOduration=253.282462364 podStartE2EDuration="4m14.269322532s" totalImagesPullingTime="986.860168ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 17:14:16 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 17:18:30.268648392 +0000 UTC m=+254.754492283" watchObservedRunningTime="2026-07-16 17:18:30.269322532 +0000 UTC m=+254.755166421" Jul 16 17:18:33.005287 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:18:33.005246 2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-648c849fdc-bgjwc" Jul 16 17:18:33.005649 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:18:33.005310 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-648c849fdc-bgjwc" Jul 16 17:18:33.010033 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:18:33.010013 2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-648c849fdc-bgjwc" Jul 16 17:18:33.263063 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:18:33.262986 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-648c849fdc-bgjwc" Jul 16 17:18:33.351052 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:18:33.351006 2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-7d9cb857b5-4c5kx"] Jul 16 17:18:58.371595 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:18:58.371542 2578 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-7d9cb857b5-4c5kx" podUID="1a3f3622-8378-41e6-87f7-74be93905ac4" containerName="console" containerID="cri-o://c6339ab1f10122ff0f55c70741dd4411c1dab6e393dec249fb82cfde095bef7d" gracePeriod=15 Jul 16 17:18:58.617388 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:18:58.617364 2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7d9cb857b5-4c5kx" Jul 16 17:18:58.658635 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:18:58.658577 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/1a3f3622-8378-41e6-87f7-74be93905ac4-oauth-serving-cert\") pod \"1a3f3622-8378-41e6-87f7-74be93905ac4\" (UID: \"1a3f3622-8378-41e6-87f7-74be93905ac4\") " Jul 16 17:18:58.658635 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:18:58.658610 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/1a3f3622-8378-41e6-87f7-74be93905ac4-trusted-ca-bundle\") pod \"1a3f3622-8378-41e6-87f7-74be93905ac4\" (UID: \"1a3f3622-8378-41e6-87f7-74be93905ac4\") " Jul 16 17:18:58.658875 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:18:58.658639 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/1a3f3622-8378-41e6-87f7-74be93905ac4-console-oauth-config\") pod \"1a3f3622-8378-41e6-87f7-74be93905ac4\" (UID: \"1a3f3622-8378-41e6-87f7-74be93905ac4\") " Jul 16 17:18:58.658875 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:18:58.658735 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/1a3f3622-8378-41e6-87f7-74be93905ac4-console-config\") pod \"1a3f3622-8378-41e6-87f7-74be93905ac4\" (UID: \"1a3f3622-8378-41e6-87f7-74be93905ac4\") " Jul 16 17:18:58.658875 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:18:58.658776 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-jbh5l\" (UniqueName: \"kubernetes.io/projected/1a3f3622-8378-41e6-87f7-74be93905ac4-kube-api-access-jbh5l\") pod \"1a3f3622-8378-41e6-87f7-74be93905ac4\" (UID: \"1a3f3622-8378-41e6-87f7-74be93905ac4\") " Jul 16 17:18:58.658875 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:18:58.658802 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/1a3f3622-8378-41e6-87f7-74be93905ac4-service-ca\") pod \"1a3f3622-8378-41e6-87f7-74be93905ac4\" (UID: \"1a3f3622-8378-41e6-87f7-74be93905ac4\") " Jul 16 17:18:58.658875 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:18:58.658832 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/1a3f3622-8378-41e6-87f7-74be93905ac4-console-serving-cert\") pod \"1a3f3622-8378-41e6-87f7-74be93905ac4\" (UID: \"1a3f3622-8378-41e6-87f7-74be93905ac4\") " Jul 16 17:18:58.659125 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:18:58.659038 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1a3f3622-8378-41e6-87f7-74be93905ac4-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "1a3f3622-8378-41e6-87f7-74be93905ac4" (UID: "1a3f3622-8378-41e6-87f7-74be93905ac4"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 16 17:18:58.659125 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:18:58.659106 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1a3f3622-8378-41e6-87f7-74be93905ac4-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "1a3f3622-8378-41e6-87f7-74be93905ac4" (UID: "1a3f3622-8378-41e6-87f7-74be93905ac4"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 16 17:18:58.659125 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:18:58.659112 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1a3f3622-8378-41e6-87f7-74be93905ac4-console-config" (OuterVolumeSpecName: "console-config") pod "1a3f3622-8378-41e6-87f7-74be93905ac4" (UID: "1a3f3622-8378-41e6-87f7-74be93905ac4"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 16 17:18:58.659291 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:18:58.659132 2578 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/1a3f3622-8378-41e6-87f7-74be93905ac4-trusted-ca-bundle\") on node \"ip-10-0-132-130.ec2.internal\" DevicePath \"\"" Jul 16 17:18:58.659520 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:18:58.659488 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1a3f3622-8378-41e6-87f7-74be93905ac4-service-ca" (OuterVolumeSpecName: "service-ca") pod "1a3f3622-8378-41e6-87f7-74be93905ac4" (UID: "1a3f3622-8378-41e6-87f7-74be93905ac4"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 16 17:18:58.660909 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:18:58.660886 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1a3f3622-8378-41e6-87f7-74be93905ac4-kube-api-access-jbh5l" (OuterVolumeSpecName: "kube-api-access-jbh5l") pod "1a3f3622-8378-41e6-87f7-74be93905ac4" (UID: "1a3f3622-8378-41e6-87f7-74be93905ac4"). InnerVolumeSpecName "kube-api-access-jbh5l". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 16 17:18:58.661027 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:18:58.661005 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/1a3f3622-8378-41e6-87f7-74be93905ac4-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "1a3f3622-8378-41e6-87f7-74be93905ac4" (UID: "1a3f3622-8378-41e6-87f7-74be93905ac4"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 16 17:18:58.661071 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:18:58.661027 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/1a3f3622-8378-41e6-87f7-74be93905ac4-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "1a3f3622-8378-41e6-87f7-74be93905ac4" (UID: "1a3f3622-8378-41e6-87f7-74be93905ac4"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 16 17:18:58.759453 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:18:58.759428 2578 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/1a3f3622-8378-41e6-87f7-74be93905ac4-oauth-serving-cert\") on node \"ip-10-0-132-130.ec2.internal\" DevicePath \"\"" Jul 16 17:18:58.759453 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:18:58.759451 2578 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/1a3f3622-8378-41e6-87f7-74be93905ac4-console-oauth-config\") on node \"ip-10-0-132-130.ec2.internal\" DevicePath \"\"" Jul 16 17:18:58.759582 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:18:58.759463 2578 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/1a3f3622-8378-41e6-87f7-74be93905ac4-console-config\") on node \"ip-10-0-132-130.ec2.internal\" DevicePath \"\"" Jul 16 17:18:58.759582 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:18:58.759471 2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-jbh5l\" (UniqueName: \"kubernetes.io/projected/1a3f3622-8378-41e6-87f7-74be93905ac4-kube-api-access-jbh5l\") on node \"ip-10-0-132-130.ec2.internal\" DevicePath \"\"" Jul 16 17:18:58.759582 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:18:58.759483 2578 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/1a3f3622-8378-41e6-87f7-74be93905ac4-service-ca\") on node \"ip-10-0-132-130.ec2.internal\" DevicePath \"\"" Jul 16 17:18:58.759582 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:18:58.759491 2578 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/1a3f3622-8378-41e6-87f7-74be93905ac4-console-serving-cert\") on node \"ip-10-0-132-130.ec2.internal\" DevicePath \"\"" Jul 16 17:18:59.367237 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:18:59.367203 2578 generic.go:356] "Generic (PLEG): container finished" podID="1a3f3622-8378-41e6-87f7-74be93905ac4" containerID="c6339ab1f10122ff0f55c70741dd4411c1dab6e393dec249fb82cfde095bef7d" exitCode=0 Jul 16 17:18:59.367428 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:18:59.367251 2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7d9cb857b5-4c5kx" Jul 16 17:18:59.367428 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:18:59.367285 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7d9cb857b5-4c5kx" event={"ID":"1a3f3622-8378-41e6-87f7-74be93905ac4","Type":"ContainerDied","Data":"c6339ab1f10122ff0f55c70741dd4411c1dab6e393dec249fb82cfde095bef7d"} Jul 16 17:18:59.367428 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:18:59.367346 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7d9cb857b5-4c5kx" event={"ID":"1a3f3622-8378-41e6-87f7-74be93905ac4","Type":"ContainerDied","Data":"65dc627053de11a73d97d139b89a85ef8b6919317cda37314a0a101235e3c003"} Jul 16 17:18:59.367428 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:18:59.367367 2578 scope.go:117] "RemoveContainer" containerID="c6339ab1f10122ff0f55c70741dd4411c1dab6e393dec249fb82cfde095bef7d" Jul 16 17:18:59.376702 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:18:59.376508 2578 scope.go:117] "RemoveContainer" containerID="c6339ab1f10122ff0f55c70741dd4411c1dab6e393dec249fb82cfde095bef7d" Jul 16 17:18:59.376966 ip-10-0-132-130 kubenswrapper[2578]: E0716 17:18:59.376878 2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c6339ab1f10122ff0f55c70741dd4411c1dab6e393dec249fb82cfde095bef7d\": container with ID starting with c6339ab1f10122ff0f55c70741dd4411c1dab6e393dec249fb82cfde095bef7d not found: ID does not exist" containerID="c6339ab1f10122ff0f55c70741dd4411c1dab6e393dec249fb82cfde095bef7d" Jul 16 17:18:59.376966 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:18:59.376903 2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c6339ab1f10122ff0f55c70741dd4411c1dab6e393dec249fb82cfde095bef7d"} err="failed to get container status \"c6339ab1f10122ff0f55c70741dd4411c1dab6e393dec249fb82cfde095bef7d\": rpc error: code = NotFound desc = could not find container \"c6339ab1f10122ff0f55c70741dd4411c1dab6e393dec249fb82cfde095bef7d\": container with ID starting with c6339ab1f10122ff0f55c70741dd4411c1dab6e393dec249fb82cfde095bef7d not found: ID does not exist" Jul 16 17:18:59.396841 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:18:59.396819 2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-7d9cb857b5-4c5kx"] Jul 16 17:18:59.403429 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:18:59.403411 2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-7d9cb857b5-4c5kx"] Jul 16 17:19:00.214716 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:19:00.214655 2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="1a3f3622-8378-41e6-87f7-74be93905ac4" path="/var/lib/kubelet/pods/1a3f3622-8378-41e6-87f7-74be93905ac4/volumes" Jul 16 17:19:15.937797 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:19:15.937771 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-drs2p_e55f447c-ad64-482b-a86b-0588bddf7b4d/console-operator/1.log" Jul 16 17:19:15.944720 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:19:15.944681 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-drs2p_e55f447c-ad64-482b-a86b-0588bddf7b4d/console-operator/1.log" Jul 16 17:19:15.950128 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:19:15.950109 2578 kubelet.go:1648] "Image garbage collection succeeded" Jul 16 17:19:29.243974 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:19:29.243937 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-8594c5d9c7-b58gt"] Jul 16 17:19:29.246616 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:19:29.244418 2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1a3f3622-8378-41e6-87f7-74be93905ac4" containerName="console" Jul 16 17:19:29.246616 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:19:29.244431 2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="1a3f3622-8378-41e6-87f7-74be93905ac4" containerName="console" Jul 16 17:19:29.246616 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:19:29.244502 2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="1a3f3622-8378-41e6-87f7-74be93905ac4" containerName="console" Jul 16 17:19:29.247481 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:19:29.247463 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-8594c5d9c7-b58gt" Jul 16 17:19:29.260420 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:19:29.260394 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-8594c5d9c7-b58gt"] Jul 16 17:19:29.385098 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:19:29.385069 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/139cf555-37a5-442e-955f-f5ca53dad8b2-trusted-ca-bundle\") pod \"console-8594c5d9c7-b58gt\" (UID: \"139cf555-37a5-442e-955f-f5ca53dad8b2\") " pod="openshift-console/console-8594c5d9c7-b58gt" Jul 16 17:19:29.385239 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:19:29.385105 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/139cf555-37a5-442e-955f-f5ca53dad8b2-console-serving-cert\") pod \"console-8594c5d9c7-b58gt\" (UID: \"139cf555-37a5-442e-955f-f5ca53dad8b2\") " pod="openshift-console/console-8594c5d9c7-b58gt" Jul 16 17:19:29.385239 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:19:29.385135 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/139cf555-37a5-442e-955f-f5ca53dad8b2-oauth-serving-cert\") pod \"console-8594c5d9c7-b58gt\" (UID: \"139cf555-37a5-442e-955f-f5ca53dad8b2\") " pod="openshift-console/console-8594c5d9c7-b58gt" Jul 16 17:19:29.385239 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:19:29.385191 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/139cf555-37a5-442e-955f-f5ca53dad8b2-console-oauth-config\") pod \"console-8594c5d9c7-b58gt\" (UID: \"139cf555-37a5-442e-955f-f5ca53dad8b2\") " pod="openshift-console/console-8594c5d9c7-b58gt" Jul 16 17:19:29.385239 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:19:29.385229 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/139cf555-37a5-442e-955f-f5ca53dad8b2-console-config\") pod \"console-8594c5d9c7-b58gt\" (UID: \"139cf555-37a5-442e-955f-f5ca53dad8b2\") " pod="openshift-console/console-8594c5d9c7-b58gt" Jul 16 17:19:29.385365 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:19:29.385270 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fplp5\" (UniqueName: \"kubernetes.io/projected/139cf555-37a5-442e-955f-f5ca53dad8b2-kube-api-access-fplp5\") pod \"console-8594c5d9c7-b58gt\" (UID: \"139cf555-37a5-442e-955f-f5ca53dad8b2\") " pod="openshift-console/console-8594c5d9c7-b58gt" Jul 16 17:19:29.385365 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:19:29.385289 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/139cf555-37a5-442e-955f-f5ca53dad8b2-service-ca\") pod \"console-8594c5d9c7-b58gt\" (UID: \"139cf555-37a5-442e-955f-f5ca53dad8b2\") " pod="openshift-console/console-8594c5d9c7-b58gt" Jul 16 17:19:29.485953 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:19:29.485927 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/139cf555-37a5-442e-955f-f5ca53dad8b2-oauth-serving-cert\") pod \"console-8594c5d9c7-b58gt\" (UID: \"139cf555-37a5-442e-955f-f5ca53dad8b2\") " pod="openshift-console/console-8594c5d9c7-b58gt" Jul 16 17:19:29.486093 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:19:29.485960 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/139cf555-37a5-442e-955f-f5ca53dad8b2-console-oauth-config\") pod \"console-8594c5d9c7-b58gt\" (UID: \"139cf555-37a5-442e-955f-f5ca53dad8b2\") " pod="openshift-console/console-8594c5d9c7-b58gt" Jul 16 17:19:29.486093 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:19:29.485980 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/139cf555-37a5-442e-955f-f5ca53dad8b2-console-config\") pod \"console-8594c5d9c7-b58gt\" (UID: \"139cf555-37a5-442e-955f-f5ca53dad8b2\") " pod="openshift-console/console-8594c5d9c7-b58gt" Jul 16 17:19:29.486164 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:19:29.486102 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fplp5\" (UniqueName: \"kubernetes.io/projected/139cf555-37a5-442e-955f-f5ca53dad8b2-kube-api-access-fplp5\") pod \"console-8594c5d9c7-b58gt\" (UID: \"139cf555-37a5-442e-955f-f5ca53dad8b2\") " pod="openshift-console/console-8594c5d9c7-b58gt" Jul 16 17:19:29.486164 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:19:29.486142 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/139cf555-37a5-442e-955f-f5ca53dad8b2-service-ca\") pod \"console-8594c5d9c7-b58gt\" (UID: \"139cf555-37a5-442e-955f-f5ca53dad8b2\") " pod="openshift-console/console-8594c5d9c7-b58gt" Jul 16 17:19:29.486258 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:19:29.486201 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/139cf555-37a5-442e-955f-f5ca53dad8b2-trusted-ca-bundle\") pod \"console-8594c5d9c7-b58gt\" (UID: \"139cf555-37a5-442e-955f-f5ca53dad8b2\") " pod="openshift-console/console-8594c5d9c7-b58gt" Jul 16 17:19:29.486258 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:19:29.486240 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/139cf555-37a5-442e-955f-f5ca53dad8b2-console-serving-cert\") pod \"console-8594c5d9c7-b58gt\" (UID: \"139cf555-37a5-442e-955f-f5ca53dad8b2\") " pod="openshift-console/console-8594c5d9c7-b58gt" Jul 16 17:19:29.486726 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:19:29.486672 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/139cf555-37a5-442e-955f-f5ca53dad8b2-console-config\") pod \"console-8594c5d9c7-b58gt\" (UID: \"139cf555-37a5-442e-955f-f5ca53dad8b2\") " pod="openshift-console/console-8594c5d9c7-b58gt" Jul 16 17:19:29.486851 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:19:29.486795 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/139cf555-37a5-442e-955f-f5ca53dad8b2-oauth-serving-cert\") pod \"console-8594c5d9c7-b58gt\" (UID: \"139cf555-37a5-442e-955f-f5ca53dad8b2\") " pod="openshift-console/console-8594c5d9c7-b58gt" Jul 16 17:19:29.486923 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:19:29.486906 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/139cf555-37a5-442e-955f-f5ca53dad8b2-service-ca\") pod \"console-8594c5d9c7-b58gt\" (UID: \"139cf555-37a5-442e-955f-f5ca53dad8b2\") " pod="openshift-console/console-8594c5d9c7-b58gt" Jul 16 17:19:29.493484 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:19:29.487418 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/139cf555-37a5-442e-955f-f5ca53dad8b2-trusted-ca-bundle\") pod \"console-8594c5d9c7-b58gt\" (UID: \"139cf555-37a5-442e-955f-f5ca53dad8b2\") " pod="openshift-console/console-8594c5d9c7-b58gt" Jul 16 17:19:29.493484 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:19:29.488972 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/139cf555-37a5-442e-955f-f5ca53dad8b2-console-serving-cert\") pod \"console-8594c5d9c7-b58gt\" (UID: \"139cf555-37a5-442e-955f-f5ca53dad8b2\") " pod="openshift-console/console-8594c5d9c7-b58gt" Jul 16 17:19:29.493484 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:19:29.489229 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/139cf555-37a5-442e-955f-f5ca53dad8b2-console-oauth-config\") pod \"console-8594c5d9c7-b58gt\" (UID: \"139cf555-37a5-442e-955f-f5ca53dad8b2\") " pod="openshift-console/console-8594c5d9c7-b58gt" Jul 16 17:19:29.496368 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:19:29.496324 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fplp5\" (UniqueName: \"kubernetes.io/projected/139cf555-37a5-442e-955f-f5ca53dad8b2-kube-api-access-fplp5\") pod \"console-8594c5d9c7-b58gt\" (UID: \"139cf555-37a5-442e-955f-f5ca53dad8b2\") " pod="openshift-console/console-8594c5d9c7-b58gt" Jul 16 17:19:29.557489 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:19:29.557464 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-8594c5d9c7-b58gt" Jul 16 17:19:29.689936 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:19:29.689904 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-8594c5d9c7-b58gt"] Jul 16 17:19:29.693404 ip-10-0-132-130 kubenswrapper[2578]: W0716 17:19:29.693377 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod139cf555_37a5_442e_955f_f5ca53dad8b2.slice/crio-9a954fd98aa23268b0b16e2532e2d40be2db1881d1037370184474941d1d56f5 WatchSource:0}: Error finding container 9a954fd98aa23268b0b16e2532e2d40be2db1881d1037370184474941d1d56f5: Status 404 returned error can't find the container with id 9a954fd98aa23268b0b16e2532e2d40be2db1881d1037370184474941d1d56f5 Jul 16 17:19:30.495061 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:19:30.495029 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-8594c5d9c7-b58gt" event={"ID":"139cf555-37a5-442e-955f-f5ca53dad8b2","Type":"ContainerStarted","Data":"ef7e12829a27a505baa5939ec91d121b9332dceefc5b2857467cc0de685b208b"} Jul 16 17:19:30.495417 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:19:30.495080 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-8594c5d9c7-b58gt" event={"ID":"139cf555-37a5-442e-955f-f5ca53dad8b2","Type":"ContainerStarted","Data":"9a954fd98aa23268b0b16e2532e2d40be2db1881d1037370184474941d1d56f5"} Jul 16 17:19:30.517937 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:19:30.517894 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-8594c5d9c7-b58gt" podStartSLOduration=1.517878402 podStartE2EDuration="1.517878402s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 17:19:29 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 17:19:30.516433513 +0000 UTC m=+315.002277402" watchObservedRunningTime="2026-07-16 17:19:30.517878402 +0000 UTC m=+315.003722291" Jul 16 17:19:39.557891 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:19:39.557856 2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-8594c5d9c7-b58gt" Jul 16 17:19:39.558329 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:19:39.557924 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-8594c5d9c7-b58gt" Jul 16 17:19:39.562645 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:19:39.562623 2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-8594c5d9c7-b58gt" Jul 16 17:19:40.540612 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:19:40.540588 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-8594c5d9c7-b58gt" Jul 16 17:19:40.632100 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:19:40.632064 2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-648c849fdc-bgjwc"] Jul 16 17:20:05.652875 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:20:05.652833 2578 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-648c849fdc-bgjwc" podUID="d84daf4f-d234-4222-8576-ad7dbdcff2d3" containerName="console" containerID="cri-o://8c293cac624a991a0ebf2c814fcfc2c4f0a4d5e9a37922ae35f3558d628ed8fb" gracePeriod=15 Jul 16 17:20:05.890768 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:20:05.890744 2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-648c849fdc-bgjwc" Jul 16 17:20:06.080022 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:20:06.079994 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d84daf4f-d234-4222-8576-ad7dbdcff2d3-trusted-ca-bundle\") pod \"d84daf4f-d234-4222-8576-ad7dbdcff2d3\" (UID: \"d84daf4f-d234-4222-8576-ad7dbdcff2d3\") " Jul 16 17:20:06.080179 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:20:06.080067 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/d84daf4f-d234-4222-8576-ad7dbdcff2d3-console-oauth-config\") pod \"d84daf4f-d234-4222-8576-ad7dbdcff2d3\" (UID: \"d84daf4f-d234-4222-8576-ad7dbdcff2d3\") " Jul 16 17:20:06.080179 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:20:06.080084 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/d84daf4f-d234-4222-8576-ad7dbdcff2d3-console-serving-cert\") pod \"d84daf4f-d234-4222-8576-ad7dbdcff2d3\" (UID: \"d84daf4f-d234-4222-8576-ad7dbdcff2d3\") " Jul 16 17:20:06.080179 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:20:06.080102 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-dtm5h\" (UniqueName: \"kubernetes.io/projected/d84daf4f-d234-4222-8576-ad7dbdcff2d3-kube-api-access-dtm5h\") pod \"d84daf4f-d234-4222-8576-ad7dbdcff2d3\" (UID: \"d84daf4f-d234-4222-8576-ad7dbdcff2d3\") " Jul 16 17:20:06.080179 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:20:06.080128 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/d84daf4f-d234-4222-8576-ad7dbdcff2d3-oauth-serving-cert\") pod \"d84daf4f-d234-4222-8576-ad7dbdcff2d3\" (UID: \"d84daf4f-d234-4222-8576-ad7dbdcff2d3\") " Jul 16 17:20:06.080179 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:20:06.080173 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/d84daf4f-d234-4222-8576-ad7dbdcff2d3-service-ca\") pod \"d84daf4f-d234-4222-8576-ad7dbdcff2d3\" (UID: \"d84daf4f-d234-4222-8576-ad7dbdcff2d3\") " Jul 16 17:20:06.080427 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:20:06.080207 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/d84daf4f-d234-4222-8576-ad7dbdcff2d3-console-config\") pod \"d84daf4f-d234-4222-8576-ad7dbdcff2d3\" (UID: \"d84daf4f-d234-4222-8576-ad7dbdcff2d3\") " Jul 16 17:20:06.080585 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:20:06.080452 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d84daf4f-d234-4222-8576-ad7dbdcff2d3-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "d84daf4f-d234-4222-8576-ad7dbdcff2d3" (UID: "d84daf4f-d234-4222-8576-ad7dbdcff2d3"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 16 17:20:06.080774 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:20:06.080655 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d84daf4f-d234-4222-8576-ad7dbdcff2d3-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "d84daf4f-d234-4222-8576-ad7dbdcff2d3" (UID: "d84daf4f-d234-4222-8576-ad7dbdcff2d3"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 16 17:20:06.080774 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:20:06.080672 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d84daf4f-d234-4222-8576-ad7dbdcff2d3-service-ca" (OuterVolumeSpecName: "service-ca") pod "d84daf4f-d234-4222-8576-ad7dbdcff2d3" (UID: "d84daf4f-d234-4222-8576-ad7dbdcff2d3"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 16 17:20:06.080957 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:20:06.080805 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d84daf4f-d234-4222-8576-ad7dbdcff2d3-console-config" (OuterVolumeSpecName: "console-config") pod "d84daf4f-d234-4222-8576-ad7dbdcff2d3" (UID: "d84daf4f-d234-4222-8576-ad7dbdcff2d3"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 16 17:20:06.082265 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:20:06.082235 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d84daf4f-d234-4222-8576-ad7dbdcff2d3-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "d84daf4f-d234-4222-8576-ad7dbdcff2d3" (UID: "d84daf4f-d234-4222-8576-ad7dbdcff2d3"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 16 17:20:06.082348 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:20:06.082268 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d84daf4f-d234-4222-8576-ad7dbdcff2d3-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "d84daf4f-d234-4222-8576-ad7dbdcff2d3" (UID: "d84daf4f-d234-4222-8576-ad7dbdcff2d3"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 16 17:20:06.082348 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:20:06.082301 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d84daf4f-d234-4222-8576-ad7dbdcff2d3-kube-api-access-dtm5h" (OuterVolumeSpecName: "kube-api-access-dtm5h") pod "d84daf4f-d234-4222-8576-ad7dbdcff2d3" (UID: "d84daf4f-d234-4222-8576-ad7dbdcff2d3"). InnerVolumeSpecName "kube-api-access-dtm5h". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 16 17:20:06.180954 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:20:06.180929 2578 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/d84daf4f-d234-4222-8576-ad7dbdcff2d3-console-oauth-config\") on node \"ip-10-0-132-130.ec2.internal\" DevicePath \"\"" Jul 16 17:20:06.180954 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:20:06.180950 2578 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/d84daf4f-d234-4222-8576-ad7dbdcff2d3-console-serving-cert\") on node \"ip-10-0-132-130.ec2.internal\" DevicePath \"\"" Jul 16 17:20:06.180954 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:20:06.180959 2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-dtm5h\" (UniqueName: \"kubernetes.io/projected/d84daf4f-d234-4222-8576-ad7dbdcff2d3-kube-api-access-dtm5h\") on node \"ip-10-0-132-130.ec2.internal\" DevicePath \"\"" Jul 16 17:20:06.181119 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:20:06.180968 2578 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/d84daf4f-d234-4222-8576-ad7dbdcff2d3-oauth-serving-cert\") on node \"ip-10-0-132-130.ec2.internal\" DevicePath \"\"" Jul 16 17:20:06.181119 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:20:06.180977 2578 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/d84daf4f-d234-4222-8576-ad7dbdcff2d3-service-ca\") on node \"ip-10-0-132-130.ec2.internal\" DevicePath \"\"" Jul 16 17:20:06.181119 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:20:06.180986 2578 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/d84daf4f-d234-4222-8576-ad7dbdcff2d3-console-config\") on node \"ip-10-0-132-130.ec2.internal\" DevicePath \"\"" Jul 16 17:20:06.181119 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:20:06.180993 2578 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d84daf4f-d234-4222-8576-ad7dbdcff2d3-trusted-ca-bundle\") on node \"ip-10-0-132-130.ec2.internal\" DevicePath \"\"" Jul 16 17:20:06.639596 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:20:06.639563 2578 generic.go:356] "Generic (PLEG): container finished" podID="d84daf4f-d234-4222-8576-ad7dbdcff2d3" containerID="8c293cac624a991a0ebf2c814fcfc2c4f0a4d5e9a37922ae35f3558d628ed8fb" exitCode=0 Jul 16 17:20:06.639775 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:20:06.639615 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-648c849fdc-bgjwc" event={"ID":"d84daf4f-d234-4222-8576-ad7dbdcff2d3","Type":"ContainerDied","Data":"8c293cac624a991a0ebf2c814fcfc2c4f0a4d5e9a37922ae35f3558d628ed8fb"} Jul 16 17:20:06.639775 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:20:06.639625 2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-648c849fdc-bgjwc" Jul 16 17:20:06.639775 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:20:06.639656 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-648c849fdc-bgjwc" event={"ID":"d84daf4f-d234-4222-8576-ad7dbdcff2d3","Type":"ContainerDied","Data":"3d77fdfacd6b8410238dba690cdc03cb6a97077644009bf0746b50a1d5b4453c"} Jul 16 17:20:06.639775 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:20:06.639676 2578 scope.go:117] "RemoveContainer" containerID="8c293cac624a991a0ebf2c814fcfc2c4f0a4d5e9a37922ae35f3558d628ed8fb" Jul 16 17:20:06.647809 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:20:06.647791 2578 scope.go:117] "RemoveContainer" containerID="8c293cac624a991a0ebf2c814fcfc2c4f0a4d5e9a37922ae35f3558d628ed8fb" Jul 16 17:20:06.648069 ip-10-0-132-130 kubenswrapper[2578]: E0716 17:20:06.648049 2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8c293cac624a991a0ebf2c814fcfc2c4f0a4d5e9a37922ae35f3558d628ed8fb\": container with ID starting with 8c293cac624a991a0ebf2c814fcfc2c4f0a4d5e9a37922ae35f3558d628ed8fb not found: ID does not exist" containerID="8c293cac624a991a0ebf2c814fcfc2c4f0a4d5e9a37922ae35f3558d628ed8fb" Jul 16 17:20:06.648106 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:20:06.648078 2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8c293cac624a991a0ebf2c814fcfc2c4f0a4d5e9a37922ae35f3558d628ed8fb"} err="failed to get container status \"8c293cac624a991a0ebf2c814fcfc2c4f0a4d5e9a37922ae35f3558d628ed8fb\": rpc error: code = NotFound desc = could not find container \"8c293cac624a991a0ebf2c814fcfc2c4f0a4d5e9a37922ae35f3558d628ed8fb\": container with ID starting with 8c293cac624a991a0ebf2c814fcfc2c4f0a4d5e9a37922ae35f3558d628ed8fb not found: ID does not exist" Jul 16 17:20:06.674169 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:20:06.674146 2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-648c849fdc-bgjwc"] Jul 16 17:20:06.682222 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:20:06.682205 2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-648c849fdc-bgjwc"] Jul 16 17:20:08.214904 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:20:08.214834 2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d84daf4f-d234-4222-8576-ad7dbdcff2d3" path="/var/lib/kubelet/pods/d84daf4f-d234-4222-8576-ad7dbdcff2d3/volumes" Jul 16 17:20:59.120148 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:20:59.120111 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/320da698f62953fa8a11a45982d6c7ee4dcf44b832e3de43010dcb42b5hztjl"] Jul 16 17:20:59.120572 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:20:59.120447 2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d84daf4f-d234-4222-8576-ad7dbdcff2d3" containerName="console" Jul 16 17:20:59.120572 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:20:59.120459 2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="d84daf4f-d234-4222-8576-ad7dbdcff2d3" containerName="console" Jul 16 17:20:59.120572 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:20:59.120527 2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="d84daf4f-d234-4222-8576-ad7dbdcff2d3" containerName="console" Jul 16 17:20:59.123681 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:20:59.123665 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/320da698f62953fa8a11a45982d6c7ee4dcf44b832e3de43010dcb42b5hztjl" Jul 16 17:20:59.126159 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:20:59.126138 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\"" Jul 16 17:20:59.126436 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:20:59.126418 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\"" Jul 16 17:20:59.126818 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:20:59.126803 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-xgzqt\"" Jul 16 17:20:59.142992 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:20:59.142969 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/320da698f62953fa8a11a45982d6c7ee4dcf44b832e3de43010dcb42b5hztjl"] Jul 16 17:20:59.275813 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:20:59.275780 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/b52a6241-c8f4-4ddf-a7b3-dda0b3dca85c-bundle\") pod \"320da698f62953fa8a11a45982d6c7ee4dcf44b832e3de43010dcb42b5hztjl\" (UID: \"b52a6241-c8f4-4ddf-a7b3-dda0b3dca85c\") " pod="openshift-marketplace/320da698f62953fa8a11a45982d6c7ee4dcf44b832e3de43010dcb42b5hztjl" Jul 16 17:20:59.275813 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:20:59.275818 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/b52a6241-c8f4-4ddf-a7b3-dda0b3dca85c-util\") pod \"320da698f62953fa8a11a45982d6c7ee4dcf44b832e3de43010dcb42b5hztjl\" (UID: \"b52a6241-c8f4-4ddf-a7b3-dda0b3dca85c\") " pod="openshift-marketplace/320da698f62953fa8a11a45982d6c7ee4dcf44b832e3de43010dcb42b5hztjl" Jul 16 17:20:59.275994 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:20:59.275849 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qp7sp\" (UniqueName: \"kubernetes.io/projected/b52a6241-c8f4-4ddf-a7b3-dda0b3dca85c-kube-api-access-qp7sp\") pod \"320da698f62953fa8a11a45982d6c7ee4dcf44b832e3de43010dcb42b5hztjl\" (UID: \"b52a6241-c8f4-4ddf-a7b3-dda0b3dca85c\") " pod="openshift-marketplace/320da698f62953fa8a11a45982d6c7ee4dcf44b832e3de43010dcb42b5hztjl" Jul 16 17:20:59.376971 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:20:59.376903 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/b52a6241-c8f4-4ddf-a7b3-dda0b3dca85c-bundle\") pod \"320da698f62953fa8a11a45982d6c7ee4dcf44b832e3de43010dcb42b5hztjl\" (UID: \"b52a6241-c8f4-4ddf-a7b3-dda0b3dca85c\") " pod="openshift-marketplace/320da698f62953fa8a11a45982d6c7ee4dcf44b832e3de43010dcb42b5hztjl" Jul 16 17:20:59.376971 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:20:59.376937 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/b52a6241-c8f4-4ddf-a7b3-dda0b3dca85c-util\") pod \"320da698f62953fa8a11a45982d6c7ee4dcf44b832e3de43010dcb42b5hztjl\" (UID: \"b52a6241-c8f4-4ddf-a7b3-dda0b3dca85c\") " pod="openshift-marketplace/320da698f62953fa8a11a45982d6c7ee4dcf44b832e3de43010dcb42b5hztjl" Jul 16 17:20:59.377108 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:20:59.376976 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qp7sp\" (UniqueName: \"kubernetes.io/projected/b52a6241-c8f4-4ddf-a7b3-dda0b3dca85c-kube-api-access-qp7sp\") pod \"320da698f62953fa8a11a45982d6c7ee4dcf44b832e3de43010dcb42b5hztjl\" (UID: \"b52a6241-c8f4-4ddf-a7b3-dda0b3dca85c\") " pod="openshift-marketplace/320da698f62953fa8a11a45982d6c7ee4dcf44b832e3de43010dcb42b5hztjl" Jul 16 17:20:59.377304 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:20:59.377287 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/b52a6241-c8f4-4ddf-a7b3-dda0b3dca85c-util\") pod \"320da698f62953fa8a11a45982d6c7ee4dcf44b832e3de43010dcb42b5hztjl\" (UID: \"b52a6241-c8f4-4ddf-a7b3-dda0b3dca85c\") " pod="openshift-marketplace/320da698f62953fa8a11a45982d6c7ee4dcf44b832e3de43010dcb42b5hztjl" Jul 16 17:20:59.377343 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:20:59.377323 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/b52a6241-c8f4-4ddf-a7b3-dda0b3dca85c-bundle\") pod \"320da698f62953fa8a11a45982d6c7ee4dcf44b832e3de43010dcb42b5hztjl\" (UID: \"b52a6241-c8f4-4ddf-a7b3-dda0b3dca85c\") " pod="openshift-marketplace/320da698f62953fa8a11a45982d6c7ee4dcf44b832e3de43010dcb42b5hztjl" Jul 16 17:20:59.390012 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:20:59.389987 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qp7sp\" (UniqueName: \"kubernetes.io/projected/b52a6241-c8f4-4ddf-a7b3-dda0b3dca85c-kube-api-access-qp7sp\") pod \"320da698f62953fa8a11a45982d6c7ee4dcf44b832e3de43010dcb42b5hztjl\" (UID: \"b52a6241-c8f4-4ddf-a7b3-dda0b3dca85c\") " pod="openshift-marketplace/320da698f62953fa8a11a45982d6c7ee4dcf44b832e3de43010dcb42b5hztjl" Jul 16 17:20:59.433014 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:20:59.432992 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/320da698f62953fa8a11a45982d6c7ee4dcf44b832e3de43010dcb42b5hztjl" Jul 16 17:20:59.567218 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:20:59.567190 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/320da698f62953fa8a11a45982d6c7ee4dcf44b832e3de43010dcb42b5hztjl"] Jul 16 17:20:59.569729 ip-10-0-132-130 kubenswrapper[2578]: W0716 17:20:59.569703 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb52a6241_c8f4_4ddf_a7b3_dda0b3dca85c.slice/crio-375174de93f7b7022a8b943d4e0830c7efe9d844ff9404d05ed07a671af5ecd5 WatchSource:0}: Error finding container 375174de93f7b7022a8b943d4e0830c7efe9d844ff9404d05ed07a671af5ecd5: Status 404 returned error can't find the container with id 375174de93f7b7022a8b943d4e0830c7efe9d844ff9404d05ed07a671af5ecd5 Jul 16 17:20:59.571770 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:20:59.571749 2578 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 16 17:20:59.852026 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:20:59.851990 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/320da698f62953fa8a11a45982d6c7ee4dcf44b832e3de43010dcb42b5hztjl" event={"ID":"b52a6241-c8f4-4ddf-a7b3-dda0b3dca85c","Type":"ContainerStarted","Data":"375174de93f7b7022a8b943d4e0830c7efe9d844ff9404d05ed07a671af5ecd5"} Jul 16 17:21:05.880687 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:21:05.880608 2578 generic.go:356] "Generic (PLEG): container finished" podID="b52a6241-c8f4-4ddf-a7b3-dda0b3dca85c" containerID="221afa2e365c868bf1dcce62f044ab44b4be188afb205ea9a82610358affa5b2" exitCode=0 Jul 16 17:21:05.881075 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:21:05.880710 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/320da698f62953fa8a11a45982d6c7ee4dcf44b832e3de43010dcb42b5hztjl" event={"ID":"b52a6241-c8f4-4ddf-a7b3-dda0b3dca85c","Type":"ContainerDied","Data":"221afa2e365c868bf1dcce62f044ab44b4be188afb205ea9a82610358affa5b2"} Jul 16 17:21:08.895177 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:21:08.895139 2578 generic.go:356] "Generic (PLEG): container finished" podID="b52a6241-c8f4-4ddf-a7b3-dda0b3dca85c" containerID="a098fd0c9c52e67848e99d071cc446ba3201a5a0d6fec9fb095eee8a7623a718" exitCode=0 Jul 16 17:21:08.895645 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:21:08.895193 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/320da698f62953fa8a11a45982d6c7ee4dcf44b832e3de43010dcb42b5hztjl" event={"ID":"b52a6241-c8f4-4ddf-a7b3-dda0b3dca85c","Type":"ContainerDied","Data":"a098fd0c9c52e67848e99d071cc446ba3201a5a0d6fec9fb095eee8a7623a718"} Jul 16 17:21:16.936862 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:21:16.936823 2578 generic.go:356] "Generic (PLEG): container finished" podID="b52a6241-c8f4-4ddf-a7b3-dda0b3dca85c" containerID="b5601333e4a78ac49903ca6605d95314a3efce954eca1464d12ec4f8447ebed7" exitCode=0 Jul 16 17:21:16.937242 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:21:16.936897 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/320da698f62953fa8a11a45982d6c7ee4dcf44b832e3de43010dcb42b5hztjl" event={"ID":"b52a6241-c8f4-4ddf-a7b3-dda0b3dca85c","Type":"ContainerDied","Data":"b5601333e4a78ac49903ca6605d95314a3efce954eca1464d12ec4f8447ebed7"} Jul 16 17:21:18.081406 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:21:18.081381 2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/320da698f62953fa8a11a45982d6c7ee4dcf44b832e3de43010dcb42b5hztjl" Jul 16 17:21:18.243222 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:21:18.243151 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/b52a6241-c8f4-4ddf-a7b3-dda0b3dca85c-util\") pod \"b52a6241-c8f4-4ddf-a7b3-dda0b3dca85c\" (UID: \"b52a6241-c8f4-4ddf-a7b3-dda0b3dca85c\") " Jul 16 17:21:18.243222 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:21:18.243183 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-qp7sp\" (UniqueName: \"kubernetes.io/projected/b52a6241-c8f4-4ddf-a7b3-dda0b3dca85c-kube-api-access-qp7sp\") pod \"b52a6241-c8f4-4ddf-a7b3-dda0b3dca85c\" (UID: \"b52a6241-c8f4-4ddf-a7b3-dda0b3dca85c\") " Jul 16 17:21:18.243429 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:21:18.243267 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/b52a6241-c8f4-4ddf-a7b3-dda0b3dca85c-bundle\") pod \"b52a6241-c8f4-4ddf-a7b3-dda0b3dca85c\" (UID: \"b52a6241-c8f4-4ddf-a7b3-dda0b3dca85c\") " Jul 16 17:21:18.243934 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:21:18.243907 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b52a6241-c8f4-4ddf-a7b3-dda0b3dca85c-bundle" (OuterVolumeSpecName: "bundle") pod "b52a6241-c8f4-4ddf-a7b3-dda0b3dca85c" (UID: "b52a6241-c8f4-4ddf-a7b3-dda0b3dca85c"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 16 17:21:18.245395 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:21:18.245368 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b52a6241-c8f4-4ddf-a7b3-dda0b3dca85c-kube-api-access-qp7sp" (OuterVolumeSpecName: "kube-api-access-qp7sp") pod "b52a6241-c8f4-4ddf-a7b3-dda0b3dca85c" (UID: "b52a6241-c8f4-4ddf-a7b3-dda0b3dca85c"). InnerVolumeSpecName "kube-api-access-qp7sp". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 16 17:21:18.247027 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:21:18.247005 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b52a6241-c8f4-4ddf-a7b3-dda0b3dca85c-util" (OuterVolumeSpecName: "util") pod "b52a6241-c8f4-4ddf-a7b3-dda0b3dca85c" (UID: "b52a6241-c8f4-4ddf-a7b3-dda0b3dca85c"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 16 17:21:18.344249 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:21:18.344220 2578 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/b52a6241-c8f4-4ddf-a7b3-dda0b3dca85c-bundle\") on node \"ip-10-0-132-130.ec2.internal\" DevicePath \"\"" Jul 16 17:21:18.344249 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:21:18.344244 2578 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/b52a6241-c8f4-4ddf-a7b3-dda0b3dca85c-util\") on node \"ip-10-0-132-130.ec2.internal\" DevicePath \"\"" Jul 16 17:21:18.344407 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:21:18.344254 2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-qp7sp\" (UniqueName: \"kubernetes.io/projected/b52a6241-c8f4-4ddf-a7b3-dda0b3dca85c-kube-api-access-qp7sp\") on node \"ip-10-0-132-130.ec2.internal\" DevicePath \"\"" Jul 16 17:21:18.947373 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:21:18.947335 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/320da698f62953fa8a11a45982d6c7ee4dcf44b832e3de43010dcb42b5hztjl" event={"ID":"b52a6241-c8f4-4ddf-a7b3-dda0b3dca85c","Type":"ContainerDied","Data":"375174de93f7b7022a8b943d4e0830c7efe9d844ff9404d05ed07a671af5ecd5"} Jul 16 17:21:18.947373 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:21:18.947365 2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/320da698f62953fa8a11a45982d6c7ee4dcf44b832e3de43010dcb42b5hztjl" Jul 16 17:21:18.947568 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:21:18.947383 2578 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="375174de93f7b7022a8b943d4e0830c7efe9d844ff9404d05ed07a671af5ecd5" Jul 16 17:21:22.079506 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:21:22.079477 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-keda/custom-metrics-autoscaler-operator-6969cb7dfb-2z7fl"] Jul 16 17:21:22.079949 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:21:22.079802 2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b52a6241-c8f4-4ddf-a7b3-dda0b3dca85c" containerName="util" Jul 16 17:21:22.079949 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:21:22.079814 2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="b52a6241-c8f4-4ddf-a7b3-dda0b3dca85c" containerName="util" Jul 16 17:21:22.079949 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:21:22.079832 2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b52a6241-c8f4-4ddf-a7b3-dda0b3dca85c" containerName="extract" Jul 16 17:21:22.079949 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:21:22.079838 2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="b52a6241-c8f4-4ddf-a7b3-dda0b3dca85c" containerName="extract" Jul 16 17:21:22.079949 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:21:22.079847 2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b52a6241-c8f4-4ddf-a7b3-dda0b3dca85c" containerName="pull" Jul 16 17:21:22.079949 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:21:22.079854 2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="b52a6241-c8f4-4ddf-a7b3-dda0b3dca85c" containerName="pull" Jul 16 17:21:22.079949 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:21:22.079906 2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="b52a6241-c8f4-4ddf-a7b3-dda0b3dca85c" containerName="extract" Jul 16 17:21:22.111884 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:21:22.111858 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-keda/custom-metrics-autoscaler-operator-6969cb7dfb-2z7fl"] Jul 16 17:21:22.112005 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:21:22.111941 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-keda/custom-metrics-autoscaler-operator-6969cb7dfb-2z7fl" Jul 16 17:21:22.114407 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:21:22.114388 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-keda\"/\"openshift-service-ca.crt\"" Jul 16 17:21:22.114650 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:21:22.114634 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-keda\"/\"kube-root-ca.crt\"" Jul 16 17:21:22.114736 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:21:22.114636 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-keda\"/\"kedaorg-certs\"" Jul 16 17:21:22.114736 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:21:22.114664 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-keda\"/\"custom-metrics-autoscaler-operator-dockercfg-grh9l\"" Jul 16 17:21:22.170762 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:21:22.170738 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"certificates\" (UniqueName: \"kubernetes.io/secret/702b34ec-aa5d-421c-b843-9c1044de224b-certificates\") pod \"custom-metrics-autoscaler-operator-6969cb7dfb-2z7fl\" (UID: \"702b34ec-aa5d-421c-b843-9c1044de224b\") " pod="openshift-keda/custom-metrics-autoscaler-operator-6969cb7dfb-2z7fl" Jul 16 17:21:22.170859 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:21:22.170787 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jt542\" (UniqueName: \"kubernetes.io/projected/702b34ec-aa5d-421c-b843-9c1044de224b-kube-api-access-jt542\") pod \"custom-metrics-autoscaler-operator-6969cb7dfb-2z7fl\" (UID: \"702b34ec-aa5d-421c-b843-9c1044de224b\") " pod="openshift-keda/custom-metrics-autoscaler-operator-6969cb7dfb-2z7fl" Jul 16 17:21:22.271427 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:21:22.271401 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"certificates\" (UniqueName: \"kubernetes.io/secret/702b34ec-aa5d-421c-b843-9c1044de224b-certificates\") pod \"custom-metrics-autoscaler-operator-6969cb7dfb-2z7fl\" (UID: \"702b34ec-aa5d-421c-b843-9c1044de224b\") " pod="openshift-keda/custom-metrics-autoscaler-operator-6969cb7dfb-2z7fl" Jul 16 17:21:22.271523 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:21:22.271455 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jt542\" (UniqueName: \"kubernetes.io/projected/702b34ec-aa5d-421c-b843-9c1044de224b-kube-api-access-jt542\") pod \"custom-metrics-autoscaler-operator-6969cb7dfb-2z7fl\" (UID: \"702b34ec-aa5d-421c-b843-9c1044de224b\") " pod="openshift-keda/custom-metrics-autoscaler-operator-6969cb7dfb-2z7fl" Jul 16 17:21:22.273671 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:21:22.273654 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"certificates\" (UniqueName: \"kubernetes.io/secret/702b34ec-aa5d-421c-b843-9c1044de224b-certificates\") pod \"custom-metrics-autoscaler-operator-6969cb7dfb-2z7fl\" (UID: \"702b34ec-aa5d-421c-b843-9c1044de224b\") " pod="openshift-keda/custom-metrics-autoscaler-operator-6969cb7dfb-2z7fl" Jul 16 17:21:22.281597 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:21:22.281579 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jt542\" (UniqueName: \"kubernetes.io/projected/702b34ec-aa5d-421c-b843-9c1044de224b-kube-api-access-jt542\") pod \"custom-metrics-autoscaler-operator-6969cb7dfb-2z7fl\" (UID: \"702b34ec-aa5d-421c-b843-9c1044de224b\") " pod="openshift-keda/custom-metrics-autoscaler-operator-6969cb7dfb-2z7fl" Jul 16 17:21:22.422596 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:21:22.422533 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-keda/custom-metrics-autoscaler-operator-6969cb7dfb-2z7fl" Jul 16 17:21:22.580950 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:21:22.580894 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-keda/custom-metrics-autoscaler-operator-6969cb7dfb-2z7fl"] Jul 16 17:21:22.584359 ip-10-0-132-130 kubenswrapper[2578]: W0716 17:21:22.584332 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod702b34ec_aa5d_421c_b843_9c1044de224b.slice/crio-84b991837af3eecf6698acf4d7bc6a0bf0fec4d3ef97e50b8266b934788c93d9 WatchSource:0}: Error finding container 84b991837af3eecf6698acf4d7bc6a0bf0fec4d3ef97e50b8266b934788c93d9: Status 404 returned error can't find the container with id 84b991837af3eecf6698acf4d7bc6a0bf0fec4d3ef97e50b8266b934788c93d9 Jul 16 17:21:22.964751 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:21:22.964719 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-keda/custom-metrics-autoscaler-operator-6969cb7dfb-2z7fl" event={"ID":"702b34ec-aa5d-421c-b843-9c1044de224b","Type":"ContainerStarted","Data":"84b991837af3eecf6698acf4d7bc6a0bf0fec4d3ef97e50b8266b934788c93d9"} Jul 16 17:21:26.986995 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:21:26.986956 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-keda/custom-metrics-autoscaler-operator-6969cb7dfb-2z7fl" event={"ID":"702b34ec-aa5d-421c-b843-9c1044de224b","Type":"ContainerStarted","Data":"1a2f7dcf506f4f45f0555f099f305985e3b6caa055901ff45a923cc5050a6d9b"} Jul 16 17:21:26.987387 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:21:26.987092 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-keda/custom-metrics-autoscaler-operator-6969cb7dfb-2z7fl" Jul 16 17:21:27.011274 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:21:27.011229 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-keda/custom-metrics-autoscaler-operator-6969cb7dfb-2z7fl" podStartSLOduration=1.662653618 podStartE2EDuration="5.011214928s" totalImagesPullingTime="3.34856131s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 17:21:22 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 17:21:27.009008451 +0000 UTC m=+431.494852342" watchObservedRunningTime="2026-07-16 17:21:27.011214928 +0000 UTC m=+431.497058820" Jul 16 17:21:27.515772 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:21:27.515741 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-keda/keda-admission-56f9798bb4-5c95p"] Jul 16 17:21:27.518527 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:21:27.518505 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-keda/keda-admission-56f9798bb4-5c95p" Jul 16 17:21:27.521275 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:21:27.521255 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-keda\"/\"keda-operator-dockercfg-78645\"" Jul 16 17:21:27.521423 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:21:27.521401 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-keda\"/\"keda-admission-webhooks-certs\"" Jul 16 17:21:27.521523 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:21:27.521488 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-keda\"/\"keda-ocp-cabundle\"" Jul 16 17:21:27.534707 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:21:27.534676 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-keda/keda-admission-56f9798bb4-5c95p"] Jul 16 17:21:27.615923 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:21:27.615902 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"certificates\" (UniqueName: \"kubernetes.io/projected/ec87a1b4-bbf5-4ba6-918e-3ea77ad7700e-certificates\") pod \"keda-admission-56f9798bb4-5c95p\" (UID: \"ec87a1b4-bbf5-4ba6-918e-3ea77ad7700e\") " pod="openshift-keda/keda-admission-56f9798bb4-5c95p" Jul 16 17:21:27.616034 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:21:27.615947 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gn98t\" (UniqueName: \"kubernetes.io/projected/ec87a1b4-bbf5-4ba6-918e-3ea77ad7700e-kube-api-access-gn98t\") pod \"keda-admission-56f9798bb4-5c95p\" (UID: \"ec87a1b4-bbf5-4ba6-918e-3ea77ad7700e\") " pod="openshift-keda/keda-admission-56f9798bb4-5c95p" Jul 16 17:21:27.717306 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:21:27.717267 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"certificates\" (UniqueName: \"kubernetes.io/projected/ec87a1b4-bbf5-4ba6-918e-3ea77ad7700e-certificates\") pod \"keda-admission-56f9798bb4-5c95p\" (UID: \"ec87a1b4-bbf5-4ba6-918e-3ea77ad7700e\") " pod="openshift-keda/keda-admission-56f9798bb4-5c95p" Jul 16 17:21:27.717459 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:21:27.717326 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gn98t\" (UniqueName: \"kubernetes.io/projected/ec87a1b4-bbf5-4ba6-918e-3ea77ad7700e-kube-api-access-gn98t\") pod \"keda-admission-56f9798bb4-5c95p\" (UID: \"ec87a1b4-bbf5-4ba6-918e-3ea77ad7700e\") " pod="openshift-keda/keda-admission-56f9798bb4-5c95p" Jul 16 17:21:27.717459 ip-10-0-132-130 kubenswrapper[2578]: E0716 17:21:27.717419 2578 projected.go:266] Couldn't get secret openshift-keda/keda-admission-webhooks-certs: secret "keda-admission-webhooks-certs" not found Jul 16 17:21:27.717459 ip-10-0-132-130 kubenswrapper[2578]: E0716 17:21:27.717444 2578 projected.go:196] Error preparing data for projected volume certificates for pod openshift-keda/keda-admission-56f9798bb4-5c95p: secret "keda-admission-webhooks-certs" not found Jul 16 17:21:27.717619 ip-10-0-132-130 kubenswrapper[2578]: E0716 17:21:27.717495 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/ec87a1b4-bbf5-4ba6-918e-3ea77ad7700e-certificates podName:ec87a1b4-bbf5-4ba6-918e-3ea77ad7700e nodeName:}" failed. No retries permitted until 2026-07-16 17:21:28.217479496 +0000 UTC m=+432.703323363 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "certificates" (UniqueName: "kubernetes.io/projected/ec87a1b4-bbf5-4ba6-918e-3ea77ad7700e-certificates") pod "keda-admission-56f9798bb4-5c95p" (UID: "ec87a1b4-bbf5-4ba6-918e-3ea77ad7700e") : secret "keda-admission-webhooks-certs" not found Jul 16 17:21:27.736621 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:21:27.736593 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gn98t\" (UniqueName: \"kubernetes.io/projected/ec87a1b4-bbf5-4ba6-918e-3ea77ad7700e-kube-api-access-gn98t\") pod \"keda-admission-56f9798bb4-5c95p\" (UID: \"ec87a1b4-bbf5-4ba6-918e-3ea77ad7700e\") " pod="openshift-keda/keda-admission-56f9798bb4-5c95p" Jul 16 17:21:28.223072 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:21:28.223043 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"certificates\" (UniqueName: \"kubernetes.io/projected/ec87a1b4-bbf5-4ba6-918e-3ea77ad7700e-certificates\") pod \"keda-admission-56f9798bb4-5c95p\" (UID: \"ec87a1b4-bbf5-4ba6-918e-3ea77ad7700e\") " pod="openshift-keda/keda-admission-56f9798bb4-5c95p" Jul 16 17:21:28.225365 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:21:28.225342 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"certificates\" (UniqueName: \"kubernetes.io/projected/ec87a1b4-bbf5-4ba6-918e-3ea77ad7700e-certificates\") pod \"keda-admission-56f9798bb4-5c95p\" (UID: \"ec87a1b4-bbf5-4ba6-918e-3ea77ad7700e\") " pod="openshift-keda/keda-admission-56f9798bb4-5c95p" Jul 16 17:21:28.428392 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:21:28.428362 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-keda/keda-admission-56f9798bb4-5c95p" Jul 16 17:21:28.562950 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:21:28.562920 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-keda/keda-admission-56f9798bb4-5c95p"] Jul 16 17:21:28.565317 ip-10-0-132-130 kubenswrapper[2578]: W0716 17:21:28.565283 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podec87a1b4_bbf5_4ba6_918e_3ea77ad7700e.slice/crio-7e7f231b488eceb181e46601127010d901e528866b3bd1518c013d0710bc1c77 WatchSource:0}: Error finding container 7e7f231b488eceb181e46601127010d901e528866b3bd1518c013d0710bc1c77: Status 404 returned error can't find the container with id 7e7f231b488eceb181e46601127010d901e528866b3bd1518c013d0710bc1c77 Jul 16 17:21:28.997084 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:21:28.997047 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-keda/keda-admission-56f9798bb4-5c95p" event={"ID":"ec87a1b4-bbf5-4ba6-918e-3ea77ad7700e","Type":"ContainerStarted","Data":"7e7f231b488eceb181e46601127010d901e528866b3bd1518c013d0710bc1c77"} Jul 16 17:21:30.009118 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:21:30.009079 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-keda/keda-admission-56f9798bb4-5c95p" event={"ID":"ec87a1b4-bbf5-4ba6-918e-3ea77ad7700e","Type":"ContainerStarted","Data":"869cc67a3d47f29c67fad55d44127a8b83b3a618511e3e9a83a8965194a33f01"} Jul 16 17:21:30.009488 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:21:30.009216 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-keda/keda-admission-56f9798bb4-5c95p" Jul 16 17:21:30.035198 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:21:30.035147 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-keda/keda-admission-56f9798bb4-5c95p" podStartSLOduration=1.692238363 podStartE2EDuration="3.035128381s" totalImagesPullingTime="1.342890018s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 17:21:27 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 17:21:30.032498206 +0000 UTC m=+434.518342111" watchObservedRunningTime="2026-07-16 17:21:30.035128381 +0000 UTC m=+434.520972275" Jul 16 17:21:47.994399 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:21:47.994324 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-keda/custom-metrics-autoscaler-operator-6969cb7dfb-2z7fl" Jul 16 17:21:51.015964 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:21:51.015935 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-keda/keda-admission-56f9798bb4-5c95p" Jul 16 17:22:36.574942 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:22:36.574905 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve/kserve-controller-manager-5c5755fdcb-8g9zs"] Jul 16 17:22:36.578457 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:22:36.578437 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve/kserve-controller-manager-5c5755fdcb-8g9zs" Jul 16 17:22:36.580623 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:22:36.580600 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve\"/\"kserve-controller-manager-dockercfg-wlnn6\"" Jul 16 17:22:36.580751 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:22:36.580608 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve\"/\"openshift-service-ca.crt\"" Jul 16 17:22:36.580751 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:22:36.580608 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve\"/\"kserve-webhook-server-cert\"" Jul 16 17:22:36.580840 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:22:36.580811 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve\"/\"kube-root-ca.crt\"" Jul 16 17:22:36.591877 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:22:36.591858 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve/kserve-controller-manager-5c5755fdcb-8g9zs"] Jul 16 17:22:36.608889 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:22:36.608867 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve/llmisvc-controller-manager-565cf7448c-6d7tb"] Jul 16 17:22:36.611918 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:22:36.611905 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve/llmisvc-controller-manager-565cf7448c-6d7tb" Jul 16 17:22:36.614095 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:22:36.614078 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve\"/\"llmisvc-controller-manager-dockercfg-c5kk4\"" Jul 16 17:22:36.614201 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:22:36.614100 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve\"/\"llmisvc-webhook-server-cert\"" Jul 16 17:22:36.632051 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:22:36.632034 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve/llmisvc-controller-manager-565cf7448c-6d7tb"] Jul 16 17:22:36.662343 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:22:36.662314 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-g7t59\" (UniqueName: \"kubernetes.io/projected/0d017cd5-6194-4750-81b9-ec218e481efa-kube-api-access-g7t59\") pod \"kserve-controller-manager-5c5755fdcb-8g9zs\" (UID: \"0d017cd5-6194-4750-81b9-ec218e481efa\") " pod="kserve/kserve-controller-manager-5c5755fdcb-8g9zs" Jul 16 17:22:36.662460 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:22:36.662368 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/369760c7-a0fd-4fc0-9f59-befc0e43bc65-cert\") pod \"llmisvc-controller-manager-565cf7448c-6d7tb\" (UID: \"369760c7-a0fd-4fc0-9f59-befc0e43bc65\") " pod="kserve/llmisvc-controller-manager-565cf7448c-6d7tb" Jul 16 17:22:36.662460 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:22:36.662408 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/0d017cd5-6194-4750-81b9-ec218e481efa-cert\") pod \"kserve-controller-manager-5c5755fdcb-8g9zs\" (UID: \"0d017cd5-6194-4750-81b9-ec218e481efa\") " pod="kserve/kserve-controller-manager-5c5755fdcb-8g9zs" Jul 16 17:22:36.662460 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:22:36.662452 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-59hql\" (UniqueName: \"kubernetes.io/projected/369760c7-a0fd-4fc0-9f59-befc0e43bc65-kube-api-access-59hql\") pod \"llmisvc-controller-manager-565cf7448c-6d7tb\" (UID: \"369760c7-a0fd-4fc0-9f59-befc0e43bc65\") " pod="kserve/llmisvc-controller-manager-565cf7448c-6d7tb" Jul 16 17:22:36.763371 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:22:36.763345 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-g7t59\" (UniqueName: \"kubernetes.io/projected/0d017cd5-6194-4750-81b9-ec218e481efa-kube-api-access-g7t59\") pod \"kserve-controller-manager-5c5755fdcb-8g9zs\" (UID: \"0d017cd5-6194-4750-81b9-ec218e481efa\") " pod="kserve/kserve-controller-manager-5c5755fdcb-8g9zs" Jul 16 17:22:36.763476 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:22:36.763399 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/369760c7-a0fd-4fc0-9f59-befc0e43bc65-cert\") pod \"llmisvc-controller-manager-565cf7448c-6d7tb\" (UID: \"369760c7-a0fd-4fc0-9f59-befc0e43bc65\") " pod="kserve/llmisvc-controller-manager-565cf7448c-6d7tb" Jul 16 17:22:36.763476 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:22:36.763439 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/0d017cd5-6194-4750-81b9-ec218e481efa-cert\") pod \"kserve-controller-manager-5c5755fdcb-8g9zs\" (UID: \"0d017cd5-6194-4750-81b9-ec218e481efa\") " pod="kserve/kserve-controller-manager-5c5755fdcb-8g9zs" Jul 16 17:22:36.763558 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:22:36.763482 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-59hql\" (UniqueName: \"kubernetes.io/projected/369760c7-a0fd-4fc0-9f59-befc0e43bc65-kube-api-access-59hql\") pod \"llmisvc-controller-manager-565cf7448c-6d7tb\" (UID: \"369760c7-a0fd-4fc0-9f59-befc0e43bc65\") " pod="kserve/llmisvc-controller-manager-565cf7448c-6d7tb" Jul 16 17:22:36.763558 ip-10-0-132-130 kubenswrapper[2578]: E0716 17:22:36.763550 2578 secret.go:189] Couldn't get secret kserve/llmisvc-webhook-server-cert: secret "llmisvc-webhook-server-cert" not found Jul 16 17:22:36.763638 ip-10-0-132-130 kubenswrapper[2578]: E0716 17:22:36.763614 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/369760c7-a0fd-4fc0-9f59-befc0e43bc65-cert podName:369760c7-a0fd-4fc0-9f59-befc0e43bc65 nodeName:}" failed. No retries permitted until 2026-07-16 17:22:37.263598355 +0000 UTC m=+501.749442223 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/369760c7-a0fd-4fc0-9f59-befc0e43bc65-cert") pod "llmisvc-controller-manager-565cf7448c-6d7tb" (UID: "369760c7-a0fd-4fc0-9f59-befc0e43bc65") : secret "llmisvc-webhook-server-cert" not found Jul 16 17:22:36.765849 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:22:36.765833 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/0d017cd5-6194-4750-81b9-ec218e481efa-cert\") pod \"kserve-controller-manager-5c5755fdcb-8g9zs\" (UID: \"0d017cd5-6194-4750-81b9-ec218e481efa\") " pod="kserve/kserve-controller-manager-5c5755fdcb-8g9zs" Jul 16 17:22:36.776112 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:22:36.776087 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-g7t59\" (UniqueName: \"kubernetes.io/projected/0d017cd5-6194-4750-81b9-ec218e481efa-kube-api-access-g7t59\") pod \"kserve-controller-manager-5c5755fdcb-8g9zs\" (UID: \"0d017cd5-6194-4750-81b9-ec218e481efa\") " pod="kserve/kserve-controller-manager-5c5755fdcb-8g9zs" Jul 16 17:22:36.776195 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:22:36.776184 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-59hql\" (UniqueName: \"kubernetes.io/projected/369760c7-a0fd-4fc0-9f59-befc0e43bc65-kube-api-access-59hql\") pod \"llmisvc-controller-manager-565cf7448c-6d7tb\" (UID: \"369760c7-a0fd-4fc0-9f59-befc0e43bc65\") " pod="kserve/llmisvc-controller-manager-565cf7448c-6d7tb" Jul 16 17:22:36.888630 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:22:36.888570 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve/kserve-controller-manager-5c5755fdcb-8g9zs" Jul 16 17:22:37.023551 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:22:37.023519 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve/kserve-controller-manager-5c5755fdcb-8g9zs"] Jul 16 17:22:37.026519 ip-10-0-132-130 kubenswrapper[2578]: W0716 17:22:37.026489 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0d017cd5_6194_4750_81b9_ec218e481efa.slice/crio-68c57608866f6bacfc1b1fa43e71bc164134842ac2e5c19a398f5a74e15e1edb WatchSource:0}: Error finding container 68c57608866f6bacfc1b1fa43e71bc164134842ac2e5c19a398f5a74e15e1edb: Status 404 returned error can't find the container with id 68c57608866f6bacfc1b1fa43e71bc164134842ac2e5c19a398f5a74e15e1edb Jul 16 17:22:37.268223 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:22:37.268146 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/369760c7-a0fd-4fc0-9f59-befc0e43bc65-cert\") pod \"llmisvc-controller-manager-565cf7448c-6d7tb\" (UID: \"369760c7-a0fd-4fc0-9f59-befc0e43bc65\") " pod="kserve/llmisvc-controller-manager-565cf7448c-6d7tb" Jul 16 17:22:37.270382 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:22:37.270355 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/369760c7-a0fd-4fc0-9f59-befc0e43bc65-cert\") pod \"llmisvc-controller-manager-565cf7448c-6d7tb\" (UID: \"369760c7-a0fd-4fc0-9f59-befc0e43bc65\") " pod="kserve/llmisvc-controller-manager-565cf7448c-6d7tb" Jul 16 17:22:37.304988 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:22:37.304964 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve/kserve-controller-manager-5c5755fdcb-8g9zs" event={"ID":"0d017cd5-6194-4750-81b9-ec218e481efa","Type":"ContainerStarted","Data":"68c57608866f6bacfc1b1fa43e71bc164134842ac2e5c19a398f5a74e15e1edb"} Jul 16 17:22:37.520794 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:22:37.520708 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve/llmisvc-controller-manager-565cf7448c-6d7tb" Jul 16 17:22:37.674626 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:22:37.674595 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve/llmisvc-controller-manager-565cf7448c-6d7tb"] Jul 16 17:22:37.676965 ip-10-0-132-130 kubenswrapper[2578]: W0716 17:22:37.676933 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-pod369760c7_a0fd_4fc0_9f59_befc0e43bc65.slice/crio-2cbce43d974ec87a9de170ca601c5c4cdb4e679f3a8fde36c3b61206e4f8b2da WatchSource:0}: Error finding container 2cbce43d974ec87a9de170ca601c5c4cdb4e679f3a8fde36c3b61206e4f8b2da: Status 404 returned error can't find the container with id 2cbce43d974ec87a9de170ca601c5c4cdb4e679f3a8fde36c3b61206e4f8b2da Jul 16 17:22:38.311657 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:22:38.311622 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve/llmisvc-controller-manager-565cf7448c-6d7tb" event={"ID":"369760c7-a0fd-4fc0-9f59-befc0e43bc65","Type":"ContainerStarted","Data":"2cbce43d974ec87a9de170ca601c5c4cdb4e679f3a8fde36c3b61206e4f8b2da"} Jul 16 17:22:40.325332 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:22:40.325263 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve/kserve-controller-manager-5c5755fdcb-8g9zs" event={"ID":"0d017cd5-6194-4750-81b9-ec218e481efa","Type":"ContainerStarted","Data":"26308741dab10555578d84c849c99f62956c4240be7dd0e6c1789cefd67af819"} Jul 16 17:22:40.325775 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:22:40.325665 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve/kserve-controller-manager-5c5755fdcb-8g9zs" Jul 16 17:22:40.347139 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:22:40.347089 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve/kserve-controller-manager-5c5755fdcb-8g9zs" podStartSLOduration=1.887101106 podStartE2EDuration="4.347072725s" totalImagesPullingTime="2.459971619s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 17:22:36 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 17:22:40.345012612 +0000 UTC m=+504.830856503" watchObservedRunningTime="2026-07-16 17:22:40.347072725 +0000 UTC m=+504.832916614" Jul 16 17:22:43.340442 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:22:43.340408 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve/llmisvc-controller-manager-565cf7448c-6d7tb" event={"ID":"369760c7-a0fd-4fc0-9f59-befc0e43bc65","Type":"ContainerStarted","Data":"36524b353bac1799bb6ee6e82cadb75c948b8d746c45b71e1a5b95a4cfbe552c"} Jul 16 17:22:43.340893 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:22:43.340537 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve/llmisvc-controller-manager-565cf7448c-6d7tb" Jul 16 17:22:43.366726 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:22:43.366648 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve/llmisvc-controller-manager-565cf7448c-6d7tb" podStartSLOduration=2.789144553 podStartE2EDuration="7.366634176s" totalImagesPullingTime="4.577489623s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 17:22:36 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 17:22:43.364903714 +0000 UTC m=+507.850747604" watchObservedRunningTime="2026-07-16 17:22:43.366634176 +0000 UTC m=+507.852478084" Jul 16 17:23:11.335633 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:23:11.335603 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve/kserve-controller-manager-5c5755fdcb-8g9zs" Jul 16 17:23:14.347155 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:23:14.347124 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve/llmisvc-controller-manager-565cf7448c-6d7tb" Jul 16 17:23:22.316162 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:23:22.316127 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540mp7qj"] Jul 16 17:23:22.324768 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:23:22.324744 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540mp7qj" Jul 16 17:23:22.327025 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:23:22.327004 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\"" Jul 16 17:23:22.327145 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:23:22.327077 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-xgzqt\"" Jul 16 17:23:22.327493 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:23:22.327478 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\"" Jul 16 17:23:22.334580 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:23:22.334555 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540mp7qj"] Jul 16 17:23:22.348019 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:23:22.347999 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-x6frv\" (UniqueName: \"kubernetes.io/projected/3454e245-80b2-4f3a-ab54-5d4e1e9a13f3-kube-api-access-x6frv\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540mp7qj\" (UID: \"3454e245-80b2-4f3a-ab54-5d4e1e9a13f3\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540mp7qj" Jul 16 17:23:22.348104 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:23:22.348042 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/3454e245-80b2-4f3a-ab54-5d4e1e9a13f3-bundle\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540mp7qj\" (UID: \"3454e245-80b2-4f3a-ab54-5d4e1e9a13f3\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540mp7qj" Jul 16 17:23:22.348104 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:23:22.348059 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/3454e245-80b2-4f3a-ab54-5d4e1e9a13f3-util\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540mp7qj\" (UID: \"3454e245-80b2-4f3a-ab54-5d4e1e9a13f3\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540mp7qj" Jul 16 17:23:22.448550 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:23:22.448521 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-x6frv\" (UniqueName: \"kubernetes.io/projected/3454e245-80b2-4f3a-ab54-5d4e1e9a13f3-kube-api-access-x6frv\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540mp7qj\" (UID: \"3454e245-80b2-4f3a-ab54-5d4e1e9a13f3\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540mp7qj" Jul 16 17:23:22.448668 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:23:22.448574 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/3454e245-80b2-4f3a-ab54-5d4e1e9a13f3-bundle\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540mp7qj\" (UID: \"3454e245-80b2-4f3a-ab54-5d4e1e9a13f3\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540mp7qj" Jul 16 17:23:22.448668 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:23:22.448599 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/3454e245-80b2-4f3a-ab54-5d4e1e9a13f3-util\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540mp7qj\" (UID: \"3454e245-80b2-4f3a-ab54-5d4e1e9a13f3\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540mp7qj" Jul 16 17:23:22.448950 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:23:22.448935 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/3454e245-80b2-4f3a-ab54-5d4e1e9a13f3-util\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540mp7qj\" (UID: \"3454e245-80b2-4f3a-ab54-5d4e1e9a13f3\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540mp7qj" Jul 16 17:23:22.448988 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:23:22.448970 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/3454e245-80b2-4f3a-ab54-5d4e1e9a13f3-bundle\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540mp7qj\" (UID: \"3454e245-80b2-4f3a-ab54-5d4e1e9a13f3\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540mp7qj" Jul 16 17:23:22.460567 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:23:22.460546 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-x6frv\" (UniqueName: \"kubernetes.io/projected/3454e245-80b2-4f3a-ab54-5d4e1e9a13f3-kube-api-access-x6frv\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540mp7qj\" (UID: \"3454e245-80b2-4f3a-ab54-5d4e1e9a13f3\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540mp7qj" Jul 16 17:23:22.633865 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:23:22.633787 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540mp7qj" Jul 16 17:23:22.765307 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:23:22.765274 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540mp7qj"] Jul 16 17:23:22.768105 ip-10-0-132-130 kubenswrapper[2578]: W0716 17:23:22.768078 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3454e245_80b2_4f3a_ab54_5d4e1e9a13f3.slice/crio-357fe47ad2845d79b9c9b1c6645b716f46b558cd0aa48e01aa2efb6fc1b8bc5d WatchSource:0}: Error finding container 357fe47ad2845d79b9c9b1c6645b716f46b558cd0aa48e01aa2efb6fc1b8bc5d: Status 404 returned error can't find the container with id 357fe47ad2845d79b9c9b1c6645b716f46b558cd0aa48e01aa2efb6fc1b8bc5d Jul 16 17:23:23.523032 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:23:23.523000 2578 generic.go:356] "Generic (PLEG): container finished" podID="3454e245-80b2-4f3a-ab54-5d4e1e9a13f3" containerID="528b6dcb877700b469533142ea5cb8a27d9b9bc5c7b1407da05ca1e387b83a0f" exitCode=0 Jul 16 17:23:23.523612 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:23:23.523075 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540mp7qj" event={"ID":"3454e245-80b2-4f3a-ab54-5d4e1e9a13f3","Type":"ContainerDied","Data":"528b6dcb877700b469533142ea5cb8a27d9b9bc5c7b1407da05ca1e387b83a0f"} Jul 16 17:23:23.523612 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:23:23.523138 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540mp7qj" event={"ID":"3454e245-80b2-4f3a-ab54-5d4e1e9a13f3","Type":"ContainerStarted","Data":"357fe47ad2845d79b9c9b1c6645b716f46b558cd0aa48e01aa2efb6fc1b8bc5d"} Jul 16 17:23:25.533516 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:23:25.533443 2578 generic.go:356] "Generic (PLEG): container finished" podID="3454e245-80b2-4f3a-ab54-5d4e1e9a13f3" containerID="124ad19d9f44b23f7a819fa8197871e9b848b230f7855aea4a64c0d16ca88a9e" exitCode=0 Jul 16 17:23:25.533516 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:23:25.533507 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540mp7qj" event={"ID":"3454e245-80b2-4f3a-ab54-5d4e1e9a13f3","Type":"ContainerDied","Data":"124ad19d9f44b23f7a819fa8197871e9b848b230f7855aea4a64c0d16ca88a9e"} Jul 16 17:23:26.541748 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:23:26.541717 2578 generic.go:356] "Generic (PLEG): container finished" podID="3454e245-80b2-4f3a-ab54-5d4e1e9a13f3" containerID="77f468afff6d2a47901b82dcf01303fe083db6092d9be873d7143156f3583870" exitCode=0 Jul 16 17:23:26.542094 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:23:26.541788 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540mp7qj" event={"ID":"3454e245-80b2-4f3a-ab54-5d4e1e9a13f3","Type":"ContainerDied","Data":"77f468afff6d2a47901b82dcf01303fe083db6092d9be873d7143156f3583870"} Jul 16 17:23:27.669089 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:23:27.669062 2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540mp7qj" Jul 16 17:23:27.795383 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:23:27.795346 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-x6frv\" (UniqueName: \"kubernetes.io/projected/3454e245-80b2-4f3a-ab54-5d4e1e9a13f3-kube-api-access-x6frv\") pod \"3454e245-80b2-4f3a-ab54-5d4e1e9a13f3\" (UID: \"3454e245-80b2-4f3a-ab54-5d4e1e9a13f3\") " Jul 16 17:23:27.795535 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:23:27.795431 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/3454e245-80b2-4f3a-ab54-5d4e1e9a13f3-util\") pod \"3454e245-80b2-4f3a-ab54-5d4e1e9a13f3\" (UID: \"3454e245-80b2-4f3a-ab54-5d4e1e9a13f3\") " Jul 16 17:23:27.795535 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:23:27.795462 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/3454e245-80b2-4f3a-ab54-5d4e1e9a13f3-bundle\") pod \"3454e245-80b2-4f3a-ab54-5d4e1e9a13f3\" (UID: \"3454e245-80b2-4f3a-ab54-5d4e1e9a13f3\") " Jul 16 17:23:27.796163 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:23:27.796132 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3454e245-80b2-4f3a-ab54-5d4e1e9a13f3-bundle" (OuterVolumeSpecName: "bundle") pod "3454e245-80b2-4f3a-ab54-5d4e1e9a13f3" (UID: "3454e245-80b2-4f3a-ab54-5d4e1e9a13f3"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 16 17:23:27.797532 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:23:27.797463 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3454e245-80b2-4f3a-ab54-5d4e1e9a13f3-kube-api-access-x6frv" (OuterVolumeSpecName: "kube-api-access-x6frv") pod "3454e245-80b2-4f3a-ab54-5d4e1e9a13f3" (UID: "3454e245-80b2-4f3a-ab54-5d4e1e9a13f3"). InnerVolumeSpecName "kube-api-access-x6frv". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 16 17:23:27.800887 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:23:27.800852 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3454e245-80b2-4f3a-ab54-5d4e1e9a13f3-util" (OuterVolumeSpecName: "util") pod "3454e245-80b2-4f3a-ab54-5d4e1e9a13f3" (UID: "3454e245-80b2-4f3a-ab54-5d4e1e9a13f3"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 16 17:23:27.896232 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:23:27.896191 2578 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/3454e245-80b2-4f3a-ab54-5d4e1e9a13f3-util\") on node \"ip-10-0-132-130.ec2.internal\" DevicePath \"\"" Jul 16 17:23:27.896232 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:23:27.896227 2578 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/3454e245-80b2-4f3a-ab54-5d4e1e9a13f3-bundle\") on node \"ip-10-0-132-130.ec2.internal\" DevicePath \"\"" Jul 16 17:23:27.896232 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:23:27.896237 2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-x6frv\" (UniqueName: \"kubernetes.io/projected/3454e245-80b2-4f3a-ab54-5d4e1e9a13f3-kube-api-access-x6frv\") on node \"ip-10-0-132-130.ec2.internal\" DevicePath \"\"" Jul 16 17:23:28.553446 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:23:28.553418 2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540mp7qj" Jul 16 17:23:28.553446 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:23:28.553438 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540mp7qj" event={"ID":"3454e245-80b2-4f3a-ab54-5d4e1e9a13f3","Type":"ContainerDied","Data":"357fe47ad2845d79b9c9b1c6645b716f46b558cd0aa48e01aa2efb6fc1b8bc5d"} Jul 16 17:23:28.553642 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:23:28.553501 2578 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="357fe47ad2845d79b9c9b1c6645b716f46b558cd0aa48e01aa2efb6fc1b8bc5d" Jul 16 17:23:29.899836 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:23:29.899801 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-5c9855ffcc-s2jwl"] Jul 16 17:23:29.900281 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:23:29.900130 2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3454e245-80b2-4f3a-ab54-5d4e1e9a13f3" containerName="pull" Jul 16 17:23:29.900281 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:23:29.900141 2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="3454e245-80b2-4f3a-ab54-5d4e1e9a13f3" containerName="pull" Jul 16 17:23:29.900281 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:23:29.900156 2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3454e245-80b2-4f3a-ab54-5d4e1e9a13f3" containerName="extract" Jul 16 17:23:29.900281 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:23:29.900161 2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="3454e245-80b2-4f3a-ab54-5d4e1e9a13f3" containerName="extract" Jul 16 17:23:29.900281 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:23:29.900171 2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3454e245-80b2-4f3a-ab54-5d4e1e9a13f3" containerName="util" Jul 16 17:23:29.900281 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:23:29.900176 2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="3454e245-80b2-4f3a-ab54-5d4e1e9a13f3" containerName="util" Jul 16 17:23:29.900281 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:23:29.900235 2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="3454e245-80b2-4f3a-ab54-5d4e1e9a13f3" containerName="extract" Jul 16 17:23:29.903970 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:23:29.903950 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5c9855ffcc-s2jwl" Jul 16 17:23:29.917647 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:23:29.917622 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-5c9855ffcc-s2jwl"] Jul 16 17:23:30.013360 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:23:30.013325 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/286b759e-aeff-445b-b47b-247f4885b20b-console-oauth-config\") pod \"console-5c9855ffcc-s2jwl\" (UID: \"286b759e-aeff-445b-b47b-247f4885b20b\") " pod="openshift-console/console-5c9855ffcc-s2jwl" Jul 16 17:23:30.013360 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:23:30.013367 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/286b759e-aeff-445b-b47b-247f4885b20b-console-config\") pod \"console-5c9855ffcc-s2jwl\" (UID: \"286b759e-aeff-445b-b47b-247f4885b20b\") " pod="openshift-console/console-5c9855ffcc-s2jwl" Jul 16 17:23:30.013544 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:23:30.013382 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/286b759e-aeff-445b-b47b-247f4885b20b-trusted-ca-bundle\") pod \"console-5c9855ffcc-s2jwl\" (UID: \"286b759e-aeff-445b-b47b-247f4885b20b\") " pod="openshift-console/console-5c9855ffcc-s2jwl" Jul 16 17:23:30.013544 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:23:30.013400 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4vw9v\" (UniqueName: \"kubernetes.io/projected/286b759e-aeff-445b-b47b-247f4885b20b-kube-api-access-4vw9v\") pod \"console-5c9855ffcc-s2jwl\" (UID: \"286b759e-aeff-445b-b47b-247f4885b20b\") " pod="openshift-console/console-5c9855ffcc-s2jwl" Jul 16 17:23:30.013544 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:23:30.013481 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/286b759e-aeff-445b-b47b-247f4885b20b-console-serving-cert\") pod \"console-5c9855ffcc-s2jwl\" (UID: \"286b759e-aeff-445b-b47b-247f4885b20b\") " pod="openshift-console/console-5c9855ffcc-s2jwl" Jul 16 17:23:30.013544 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:23:30.013521 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/286b759e-aeff-445b-b47b-247f4885b20b-oauth-serving-cert\") pod \"console-5c9855ffcc-s2jwl\" (UID: \"286b759e-aeff-445b-b47b-247f4885b20b\") " pod="openshift-console/console-5c9855ffcc-s2jwl" Jul 16 17:23:30.013669 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:23:30.013558 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/286b759e-aeff-445b-b47b-247f4885b20b-service-ca\") pod \"console-5c9855ffcc-s2jwl\" (UID: \"286b759e-aeff-445b-b47b-247f4885b20b\") " pod="openshift-console/console-5c9855ffcc-s2jwl" Jul 16 17:23:30.114231 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:23:30.114203 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/286b759e-aeff-445b-b47b-247f4885b20b-console-oauth-config\") pod \"console-5c9855ffcc-s2jwl\" (UID: \"286b759e-aeff-445b-b47b-247f4885b20b\") " pod="openshift-console/console-5c9855ffcc-s2jwl" Jul 16 17:23:30.114408 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:23:30.114245 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/286b759e-aeff-445b-b47b-247f4885b20b-console-config\") pod \"console-5c9855ffcc-s2jwl\" (UID: \"286b759e-aeff-445b-b47b-247f4885b20b\") " pod="openshift-console/console-5c9855ffcc-s2jwl" Jul 16 17:23:30.114408 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:23:30.114268 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/286b759e-aeff-445b-b47b-247f4885b20b-trusted-ca-bundle\") pod \"console-5c9855ffcc-s2jwl\" (UID: \"286b759e-aeff-445b-b47b-247f4885b20b\") " pod="openshift-console/console-5c9855ffcc-s2jwl" Jul 16 17:23:30.114408 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:23:30.114385 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4vw9v\" (UniqueName: \"kubernetes.io/projected/286b759e-aeff-445b-b47b-247f4885b20b-kube-api-access-4vw9v\") pod \"console-5c9855ffcc-s2jwl\" (UID: \"286b759e-aeff-445b-b47b-247f4885b20b\") " pod="openshift-console/console-5c9855ffcc-s2jwl" Jul 16 17:23:30.114560 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:23:30.114422 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/286b759e-aeff-445b-b47b-247f4885b20b-console-serving-cert\") pod \"console-5c9855ffcc-s2jwl\" (UID: \"286b759e-aeff-445b-b47b-247f4885b20b\") " pod="openshift-console/console-5c9855ffcc-s2jwl" Jul 16 17:23:30.114560 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:23:30.114449 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/286b759e-aeff-445b-b47b-247f4885b20b-oauth-serving-cert\") pod \"console-5c9855ffcc-s2jwl\" (UID: \"286b759e-aeff-445b-b47b-247f4885b20b\") " pod="openshift-console/console-5c9855ffcc-s2jwl" Jul 16 17:23:30.114560 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:23:30.114507 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/286b759e-aeff-445b-b47b-247f4885b20b-service-ca\") pod \"console-5c9855ffcc-s2jwl\" (UID: \"286b759e-aeff-445b-b47b-247f4885b20b\") " pod="openshift-console/console-5c9855ffcc-s2jwl" Jul 16 17:23:30.115271 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:23:30.115243 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/286b759e-aeff-445b-b47b-247f4885b20b-trusted-ca-bundle\") pod \"console-5c9855ffcc-s2jwl\" (UID: \"286b759e-aeff-445b-b47b-247f4885b20b\") " pod="openshift-console/console-5c9855ffcc-s2jwl" Jul 16 17:23:30.115398 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:23:30.115234 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/286b759e-aeff-445b-b47b-247f4885b20b-console-config\") pod \"console-5c9855ffcc-s2jwl\" (UID: \"286b759e-aeff-445b-b47b-247f4885b20b\") " pod="openshift-console/console-5c9855ffcc-s2jwl" Jul 16 17:23:30.115398 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:23:30.115333 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/286b759e-aeff-445b-b47b-247f4885b20b-service-ca\") pod \"console-5c9855ffcc-s2jwl\" (UID: \"286b759e-aeff-445b-b47b-247f4885b20b\") " pod="openshift-console/console-5c9855ffcc-s2jwl" Jul 16 17:23:30.115398 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:23:30.115350 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/286b759e-aeff-445b-b47b-247f4885b20b-oauth-serving-cert\") pod \"console-5c9855ffcc-s2jwl\" (UID: \"286b759e-aeff-445b-b47b-247f4885b20b\") " pod="openshift-console/console-5c9855ffcc-s2jwl" Jul 16 17:23:30.116749 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:23:30.116731 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/286b759e-aeff-445b-b47b-247f4885b20b-console-oauth-config\") pod \"console-5c9855ffcc-s2jwl\" (UID: \"286b759e-aeff-445b-b47b-247f4885b20b\") " pod="openshift-console/console-5c9855ffcc-s2jwl" Jul 16 17:23:30.116884 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:23:30.116869 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/286b759e-aeff-445b-b47b-247f4885b20b-console-serving-cert\") pod \"console-5c9855ffcc-s2jwl\" (UID: \"286b759e-aeff-445b-b47b-247f4885b20b\") " pod="openshift-console/console-5c9855ffcc-s2jwl" Jul 16 17:23:30.124732 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:23:30.124685 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4vw9v\" (UniqueName: \"kubernetes.io/projected/286b759e-aeff-445b-b47b-247f4885b20b-kube-api-access-4vw9v\") pod \"console-5c9855ffcc-s2jwl\" (UID: \"286b759e-aeff-445b-b47b-247f4885b20b\") " pod="openshift-console/console-5c9855ffcc-s2jwl" Jul 16 17:23:30.213343 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:23:30.213284 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5c9855ffcc-s2jwl" Jul 16 17:23:30.352998 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:23:30.352972 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-5c9855ffcc-s2jwl"] Jul 16 17:23:30.355429 ip-10-0-132-130 kubenswrapper[2578]: W0716 17:23:30.355393 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod286b759e_aeff_445b_b47b_247f4885b20b.slice/crio-f0aa5d7ffa887e1725ef1e1e1d8495e03321533d5a44227f758729491c2dcd2f WatchSource:0}: Error finding container f0aa5d7ffa887e1725ef1e1e1d8495e03321533d5a44227f758729491c2dcd2f: Status 404 returned error can't find the container with id f0aa5d7ffa887e1725ef1e1e1d8495e03321533d5a44227f758729491c2dcd2f Jul 16 17:23:30.564489 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:23:30.564457 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5c9855ffcc-s2jwl" event={"ID":"286b759e-aeff-445b-b47b-247f4885b20b","Type":"ContainerStarted","Data":"3260a2808c2c9622feadace446ba293f2fb9963831fb9561590bdcc0942fd717"} Jul 16 17:23:30.564648 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:23:30.564505 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5c9855ffcc-s2jwl" event={"ID":"286b759e-aeff-445b-b47b-247f4885b20b","Type":"ContainerStarted","Data":"f0aa5d7ffa887e1725ef1e1e1d8495e03321533d5a44227f758729491c2dcd2f"} Jul 16 17:23:30.584567 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:23:30.584520 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-5c9855ffcc-s2jwl" podStartSLOduration=1.5845057809999998 podStartE2EDuration="1.584505781s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 17:23:29 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 17:23:30.582794091 +0000 UTC m=+555.068637980" watchObservedRunningTime="2026-07-16 17:23:30.584505781 +0000 UTC m=+555.070349670" Jul 16 17:23:35.267382 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:23:35.267347 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-684f79c9b9-sfktg"] Jul 16 17:23:35.270940 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:23:35.270924 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager-operator/cert-manager-operator-controller-manager-684f79c9b9-sfktg" Jul 16 17:23:35.273093 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:23:35.273074 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager-operator\"/\"kube-root-ca.crt\"" Jul 16 17:23:35.273192 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:23:35.273071 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager-operator\"/\"cert-manager-operator-controller-manager-dockercfg-ps5df\"" Jul 16 17:23:35.273192 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:23:35.273125 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager-operator\"/\"openshift-service-ca.crt\"" Jul 16 17:23:35.284810 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:23:35.284789 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-684f79c9b9-sfktg"] Jul 16 17:23:35.363348 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:23:35.363321 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/c0377a50-71b4-4323-93d1-9333e7b4c32b-tmp\") pod \"cert-manager-operator-controller-manager-684f79c9b9-sfktg\" (UID: \"c0377a50-71b4-4323-93d1-9333e7b4c32b\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-684f79c9b9-sfktg" Jul 16 17:23:35.363459 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:23:35.363365 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-htvmp\" (UniqueName: \"kubernetes.io/projected/c0377a50-71b4-4323-93d1-9333e7b4c32b-kube-api-access-htvmp\") pod \"cert-manager-operator-controller-manager-684f79c9b9-sfktg\" (UID: \"c0377a50-71b4-4323-93d1-9333e7b4c32b\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-684f79c9b9-sfktg" Jul 16 17:23:35.464327 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:23:35.464299 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/c0377a50-71b4-4323-93d1-9333e7b4c32b-tmp\") pod \"cert-manager-operator-controller-manager-684f79c9b9-sfktg\" (UID: \"c0377a50-71b4-4323-93d1-9333e7b4c32b\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-684f79c9b9-sfktg" Jul 16 17:23:35.464465 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:23:35.464343 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-htvmp\" (UniqueName: \"kubernetes.io/projected/c0377a50-71b4-4323-93d1-9333e7b4c32b-kube-api-access-htvmp\") pod \"cert-manager-operator-controller-manager-684f79c9b9-sfktg\" (UID: \"c0377a50-71b4-4323-93d1-9333e7b4c32b\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-684f79c9b9-sfktg" Jul 16 17:23:35.464679 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:23:35.464661 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/c0377a50-71b4-4323-93d1-9333e7b4c32b-tmp\") pod \"cert-manager-operator-controller-manager-684f79c9b9-sfktg\" (UID: \"c0377a50-71b4-4323-93d1-9333e7b4c32b\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-684f79c9b9-sfktg" Jul 16 17:23:35.474174 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:23:35.474148 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-htvmp\" (UniqueName: \"kubernetes.io/projected/c0377a50-71b4-4323-93d1-9333e7b4c32b-kube-api-access-htvmp\") pod \"cert-manager-operator-controller-manager-684f79c9b9-sfktg\" (UID: \"c0377a50-71b4-4323-93d1-9333e7b4c32b\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-684f79c9b9-sfktg" Jul 16 17:23:35.581033 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:23:35.581010 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager-operator/cert-manager-operator-controller-manager-684f79c9b9-sfktg" Jul 16 17:23:35.720439 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:23:35.720414 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-684f79c9b9-sfktg"] Jul 16 17:23:35.722708 ip-10-0-132-130 kubenswrapper[2578]: W0716 17:23:35.722662 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc0377a50_71b4_4323_93d1_9333e7b4c32b.slice/crio-a10bbe6256ac8c261f2fae35835a5848c9e33a7518154fc5806acc482e28b9d4 WatchSource:0}: Error finding container a10bbe6256ac8c261f2fae35835a5848c9e33a7518154fc5806acc482e28b9d4: Status 404 returned error can't find the container with id a10bbe6256ac8c261f2fae35835a5848c9e33a7518154fc5806acc482e28b9d4 Jul 16 17:23:36.593824 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:23:36.593789 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager-operator/cert-manager-operator-controller-manager-684f79c9b9-sfktg" event={"ID":"c0377a50-71b4-4323-93d1-9333e7b4c32b","Type":"ContainerStarted","Data":"a10bbe6256ac8c261f2fae35835a5848c9e33a7518154fc5806acc482e28b9d4"} Jul 16 17:23:40.216093 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:23:40.216002 2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-5c9855ffcc-s2jwl" Jul 16 17:23:40.216093 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:23:40.216048 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-5c9855ffcc-s2jwl" Jul 16 17:23:40.218564 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:23:40.218542 2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-5c9855ffcc-s2jwl" Jul 16 17:23:40.615223 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:23:40.615189 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager-operator/cert-manager-operator-controller-manager-684f79c9b9-sfktg" event={"ID":"c0377a50-71b4-4323-93d1-9333e7b4c32b","Type":"ContainerStarted","Data":"a4ec2a913d2554e8338e47377afecc1c31d2a86b6366f3c1c673816f3f1106f8"} Jul 16 17:23:40.619020 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:23:40.618997 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-5c9855ffcc-s2jwl" Jul 16 17:23:40.648617 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:23:40.648575 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager-operator/cert-manager-operator-controller-manager-684f79c9b9-sfktg" podStartSLOduration=1.519014548 podStartE2EDuration="5.648561172s" totalImagesPullingTime="4.129546624s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 17:23:35 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 17:23:40.648192085 +0000 UTC m=+565.134035987" watchObservedRunningTime="2026-07-16 17:23:40.648561172 +0000 UTC m=+565.134405060" Jul 16 17:23:40.710324 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:23:40.710289 2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-8594c5d9c7-b58gt"] Jul 16 17:23:44.484118 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:23:44.484090 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f54lb6"] Jul 16 17:23:44.488037 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:23:44.488015 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f54lb6" Jul 16 17:23:44.490004 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:23:44.489985 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\"" Jul 16 17:23:44.490471 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:23:44.490455 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-xgzqt\"" Jul 16 17:23:44.490521 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:23:44.490473 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\"" Jul 16 17:23:44.498251 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:23:44.498230 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f54lb6"] Jul 16 17:23:44.648493 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:23:44.648461 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/7c90c9f0-14a0-4f60-847a-3f827079cbc9-bundle\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f54lb6\" (UID: \"7c90c9f0-14a0-4f60-847a-3f827079cbc9\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f54lb6" Jul 16 17:23:44.648676 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:23:44.648525 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-v4rrz\" (UniqueName: \"kubernetes.io/projected/7c90c9f0-14a0-4f60-847a-3f827079cbc9-kube-api-access-v4rrz\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f54lb6\" (UID: \"7c90c9f0-14a0-4f60-847a-3f827079cbc9\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f54lb6" Jul 16 17:23:44.648676 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:23:44.648640 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/7c90c9f0-14a0-4f60-847a-3f827079cbc9-util\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f54lb6\" (UID: \"7c90c9f0-14a0-4f60-847a-3f827079cbc9\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f54lb6" Jul 16 17:23:44.750046 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:23:44.749985 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-v4rrz\" (UniqueName: \"kubernetes.io/projected/7c90c9f0-14a0-4f60-847a-3f827079cbc9-kube-api-access-v4rrz\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f54lb6\" (UID: \"7c90c9f0-14a0-4f60-847a-3f827079cbc9\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f54lb6" Jul 16 17:23:44.750194 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:23:44.750051 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/7c90c9f0-14a0-4f60-847a-3f827079cbc9-util\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f54lb6\" (UID: \"7c90c9f0-14a0-4f60-847a-3f827079cbc9\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f54lb6" Jul 16 17:23:44.750194 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:23:44.750092 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/7c90c9f0-14a0-4f60-847a-3f827079cbc9-bundle\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f54lb6\" (UID: \"7c90c9f0-14a0-4f60-847a-3f827079cbc9\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f54lb6" Jul 16 17:23:44.750512 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:23:44.750489 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/7c90c9f0-14a0-4f60-847a-3f827079cbc9-util\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f54lb6\" (UID: \"7c90c9f0-14a0-4f60-847a-3f827079cbc9\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f54lb6" Jul 16 17:23:44.750512 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:23:44.750502 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/7c90c9f0-14a0-4f60-847a-3f827079cbc9-bundle\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f54lb6\" (UID: \"7c90c9f0-14a0-4f60-847a-3f827079cbc9\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f54lb6" Jul 16 17:23:44.768301 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:23:44.768274 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-v4rrz\" (UniqueName: \"kubernetes.io/projected/7c90c9f0-14a0-4f60-847a-3f827079cbc9-kube-api-access-v4rrz\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f54lb6\" (UID: \"7c90c9f0-14a0-4f60-847a-3f827079cbc9\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f54lb6" Jul 16 17:23:44.797233 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:23:44.797210 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f54lb6" Jul 16 17:23:44.949728 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:23:44.949703 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f54lb6"] Jul 16 17:23:44.952464 ip-10-0-132-130 kubenswrapper[2578]: W0716 17:23:44.952432 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7c90c9f0_14a0_4f60_847a_3f827079cbc9.slice/crio-c32fe96f4defffddb7ea76c314932f69e404df8028cadabddbd63a899dbe8d10 WatchSource:0}: Error finding container c32fe96f4defffddb7ea76c314932f69e404df8028cadabddbd63a899dbe8d10: Status 404 returned error can't find the container with id c32fe96f4defffddb7ea76c314932f69e404df8028cadabddbd63a899dbe8d10 Jul 16 17:23:45.644749 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:23:45.644686 2578 generic.go:356] "Generic (PLEG): container finished" podID="7c90c9f0-14a0-4f60-847a-3f827079cbc9" containerID="44c0e6a2f6a792f22a1c7f590882b6c58c32dd9d9798aa0bc265b4ddb7f0e3dd" exitCode=0 Jul 16 17:23:45.645209 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:23:45.644770 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f54lb6" event={"ID":"7c90c9f0-14a0-4f60-847a-3f827079cbc9","Type":"ContainerDied","Data":"44c0e6a2f6a792f22a1c7f590882b6c58c32dd9d9798aa0bc265b4ddb7f0e3dd"} Jul 16 17:23:45.645209 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:23:45.644805 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f54lb6" event={"ID":"7c90c9f0-14a0-4f60-847a-3f827079cbc9","Type":"ContainerStarted","Data":"c32fe96f4defffddb7ea76c314932f69e404df8028cadabddbd63a899dbe8d10"} Jul 16 17:23:47.657132 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:23:47.657101 2578 generic.go:356] "Generic (PLEG): container finished" podID="7c90c9f0-14a0-4f60-847a-3f827079cbc9" containerID="c3dddf1b20dbca2d159e8eced7275fb7b28e816d2838a5a49ac02a2cf9d51679" exitCode=0 Jul 16 17:23:47.657430 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:23:47.657198 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f54lb6" event={"ID":"7c90c9f0-14a0-4f60-847a-3f827079cbc9","Type":"ContainerDied","Data":"c3dddf1b20dbca2d159e8eced7275fb7b28e816d2838a5a49ac02a2cf9d51679"} Jul 16 17:23:48.665252 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:23:48.665215 2578 generic.go:356] "Generic (PLEG): container finished" podID="7c90c9f0-14a0-4f60-847a-3f827079cbc9" containerID="b1d8feb3e07e2555051c688a82ac16c94090e3d50ffdaf20acfe2ba68585e813" exitCode=0 Jul 16 17:23:48.665654 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:23:48.665267 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f54lb6" event={"ID":"7c90c9f0-14a0-4f60-847a-3f827079cbc9","Type":"ContainerDied","Data":"b1d8feb3e07e2555051c688a82ac16c94090e3d50ffdaf20acfe2ba68585e813"} Jul 16 17:23:49.800053 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:23:49.800027 2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f54lb6" Jul 16 17:23:49.895520 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:23:49.895490 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/7c90c9f0-14a0-4f60-847a-3f827079cbc9-bundle\") pod \"7c90c9f0-14a0-4f60-847a-3f827079cbc9\" (UID: \"7c90c9f0-14a0-4f60-847a-3f827079cbc9\") " Jul 16 17:23:49.895660 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:23:49.895536 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-v4rrz\" (UniqueName: \"kubernetes.io/projected/7c90c9f0-14a0-4f60-847a-3f827079cbc9-kube-api-access-v4rrz\") pod \"7c90c9f0-14a0-4f60-847a-3f827079cbc9\" (UID: \"7c90c9f0-14a0-4f60-847a-3f827079cbc9\") " Jul 16 17:23:49.895660 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:23:49.895559 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/7c90c9f0-14a0-4f60-847a-3f827079cbc9-util\") pod \"7c90c9f0-14a0-4f60-847a-3f827079cbc9\" (UID: \"7c90c9f0-14a0-4f60-847a-3f827079cbc9\") " Jul 16 17:23:49.895958 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:23:49.895932 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7c90c9f0-14a0-4f60-847a-3f827079cbc9-bundle" (OuterVolumeSpecName: "bundle") pod "7c90c9f0-14a0-4f60-847a-3f827079cbc9" (UID: "7c90c9f0-14a0-4f60-847a-3f827079cbc9"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 16 17:23:49.897552 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:23:49.897532 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7c90c9f0-14a0-4f60-847a-3f827079cbc9-kube-api-access-v4rrz" (OuterVolumeSpecName: "kube-api-access-v4rrz") pod "7c90c9f0-14a0-4f60-847a-3f827079cbc9" (UID: "7c90c9f0-14a0-4f60-847a-3f827079cbc9"). InnerVolumeSpecName "kube-api-access-v4rrz". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 16 17:23:49.901235 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:23:49.901210 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7c90c9f0-14a0-4f60-847a-3f827079cbc9-util" (OuterVolumeSpecName: "util") pod "7c90c9f0-14a0-4f60-847a-3f827079cbc9" (UID: "7c90c9f0-14a0-4f60-847a-3f827079cbc9"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 16 17:23:49.996383 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:23:49.996327 2578 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/7c90c9f0-14a0-4f60-847a-3f827079cbc9-bundle\") on node \"ip-10-0-132-130.ec2.internal\" DevicePath \"\"" Jul 16 17:23:49.996383 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:23:49.996351 2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-v4rrz\" (UniqueName: \"kubernetes.io/projected/7c90c9f0-14a0-4f60-847a-3f827079cbc9-kube-api-access-v4rrz\") on node \"ip-10-0-132-130.ec2.internal\" DevicePath \"\"" Jul 16 17:23:49.996383 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:23:49.996361 2578 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/7c90c9f0-14a0-4f60-847a-3f827079cbc9-util\") on node \"ip-10-0-132-130.ec2.internal\" DevicePath \"\"" Jul 16 17:23:50.676579 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:23:50.676552 2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f54lb6" Jul 16 17:23:50.676579 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:23:50.676573 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f54lb6" event={"ID":"7c90c9f0-14a0-4f60-847a-3f827079cbc9","Type":"ContainerDied","Data":"c32fe96f4defffddb7ea76c314932f69e404df8028cadabddbd63a899dbe8d10"} Jul 16 17:23:50.676789 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:23:50.676614 2578 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="c32fe96f4defffddb7ea76c314932f69e404df8028cadabddbd63a899dbe8d10" Jul 16 17:24:01.187623 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:24:01.187590 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-55b4d954d5-q4h82"] Jul 16 17:24:01.188062 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:24:01.187966 2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7c90c9f0-14a0-4f60-847a-3f827079cbc9" containerName="pull" Jul 16 17:24:01.188062 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:24:01.187979 2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="7c90c9f0-14a0-4f60-847a-3f827079cbc9" containerName="pull" Jul 16 17:24:01.188062 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:24:01.187993 2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7c90c9f0-14a0-4f60-847a-3f827079cbc9" containerName="extract" Jul 16 17:24:01.188062 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:24:01.187999 2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="7c90c9f0-14a0-4f60-847a-3f827079cbc9" containerName="extract" Jul 16 17:24:01.188062 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:24:01.188007 2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7c90c9f0-14a0-4f60-847a-3f827079cbc9" containerName="util" Jul 16 17:24:01.188062 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:24:01.188013 2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="7c90c9f0-14a0-4f60-847a-3f827079cbc9" containerName="util" Jul 16 17:24:01.188243 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:24:01.188077 2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="7c90c9f0-14a0-4f60-847a-3f827079cbc9" containerName="extract" Jul 16 17:24:01.194509 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:24:01.194493 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-55b4d954d5-q4h82" Jul 16 17:24:01.197225 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:24:01.197188 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-dockercfg-z88d5\"" Jul 16 17:24:01.197355 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:24:01.197339 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"kube-root-ca.crt\"" Jul 16 17:24:01.197764 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:24:01.197749 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"openshift-service-ca.crt\"" Jul 16 17:24:01.209136 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:24:01.209112 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-55b4d954d5-q4h82"] Jul 16 17:24:01.394192 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:24:01.394166 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/216066f8-1e51-461d-8dc8-4415e1e5c560-bound-sa-token\") pod \"cert-manager-55b4d954d5-q4h82\" (UID: \"216066f8-1e51-461d-8dc8-4415e1e5c560\") " pod="cert-manager/cert-manager-55b4d954d5-q4h82" Jul 16 17:24:01.394348 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:24:01.394196 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tklk8\" (UniqueName: \"kubernetes.io/projected/216066f8-1e51-461d-8dc8-4415e1e5c560-kube-api-access-tklk8\") pod \"cert-manager-55b4d954d5-q4h82\" (UID: \"216066f8-1e51-461d-8dc8-4415e1e5c560\") " pod="cert-manager/cert-manager-55b4d954d5-q4h82" Jul 16 17:24:01.495167 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:24:01.495093 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/216066f8-1e51-461d-8dc8-4415e1e5c560-bound-sa-token\") pod \"cert-manager-55b4d954d5-q4h82\" (UID: \"216066f8-1e51-461d-8dc8-4415e1e5c560\") " pod="cert-manager/cert-manager-55b4d954d5-q4h82" Jul 16 17:24:01.495167 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:24:01.495133 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tklk8\" (UniqueName: \"kubernetes.io/projected/216066f8-1e51-461d-8dc8-4415e1e5c560-kube-api-access-tklk8\") pod \"cert-manager-55b4d954d5-q4h82\" (UID: \"216066f8-1e51-461d-8dc8-4415e1e5c560\") " pod="cert-manager/cert-manager-55b4d954d5-q4h82" Jul 16 17:24:01.510035 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:24:01.510007 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/216066f8-1e51-461d-8dc8-4415e1e5c560-bound-sa-token\") pod \"cert-manager-55b4d954d5-q4h82\" (UID: \"216066f8-1e51-461d-8dc8-4415e1e5c560\") " pod="cert-manager/cert-manager-55b4d954d5-q4h82" Jul 16 17:24:01.510163 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:24:01.510144 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tklk8\" (UniqueName: \"kubernetes.io/projected/216066f8-1e51-461d-8dc8-4415e1e5c560-kube-api-access-tklk8\") pod \"cert-manager-55b4d954d5-q4h82\" (UID: \"216066f8-1e51-461d-8dc8-4415e1e5c560\") " pod="cert-manager/cert-manager-55b4d954d5-q4h82" Jul 16 17:24:01.516981 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:24:01.516961 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-55b4d954d5-q4h82" Jul 16 17:24:01.652134 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:24:01.652112 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-55b4d954d5-q4h82"] Jul 16 17:24:01.654144 ip-10-0-132-130 kubenswrapper[2578]: W0716 17:24:01.654102 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod216066f8_1e51_461d_8dc8_4415e1e5c560.slice/crio-c11439c2d6652c5c232d89394e8598027f61ff83a1b52a6c699e3bc1d9ac1897 WatchSource:0}: Error finding container c11439c2d6652c5c232d89394e8598027f61ff83a1b52a6c699e3bc1d9ac1897: Status 404 returned error can't find the container with id c11439c2d6652c5c232d89394e8598027f61ff83a1b52a6c699e3bc1d9ac1897 Jul 16 17:24:01.729139 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:24:01.729108 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-55b4d954d5-q4h82" event={"ID":"216066f8-1e51-461d-8dc8-4415e1e5c560","Type":"ContainerStarted","Data":"c11439c2d6652c5c232d89394e8598027f61ff83a1b52a6c699e3bc1d9ac1897"} Jul 16 17:24:05.732425 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:24:05.732386 2578 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-8594c5d9c7-b58gt" podUID="139cf555-37a5-442e-955f-f5ca53dad8b2" containerName="console" containerID="cri-o://ef7e12829a27a505baa5939ec91d121b9332dceefc5b2857467cc0de685b208b" gracePeriod=15 Jul 16 17:24:05.751158 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:24:05.751130 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-55b4d954d5-q4h82" event={"ID":"216066f8-1e51-461d-8dc8-4415e1e5c560","Type":"ContainerStarted","Data":"297076fcba39dd494ca7a4705da59f99066bd2be0ed21ba9748ee2b79f4f35a0"} Jul 16 17:24:05.980843 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:24:05.980821 2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-8594c5d9c7-b58gt" Jul 16 17:24:06.005529 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:24:06.005454 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager/cert-manager-55b4d954d5-q4h82" podStartSLOduration=1.7107922370000002 podStartE2EDuration="5.005438594s" totalImagesPullingTime="3.294646357s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 17:24:01 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 17:24:05.785948398 +0000 UTC m=+590.271792286" watchObservedRunningTime="2026-07-16 17:24:06.005438594 +0000 UTC m=+590.491282484" Jul 16 17:24:06.029186 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:24:06.029165 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/139cf555-37a5-442e-955f-f5ca53dad8b2-oauth-serving-cert\") pod \"139cf555-37a5-442e-955f-f5ca53dad8b2\" (UID: \"139cf555-37a5-442e-955f-f5ca53dad8b2\") " Jul 16 17:24:06.029277 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:24:06.029194 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/139cf555-37a5-442e-955f-f5ca53dad8b2-service-ca\") pod \"139cf555-37a5-442e-955f-f5ca53dad8b2\" (UID: \"139cf555-37a5-442e-955f-f5ca53dad8b2\") " Jul 16 17:24:06.029329 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:24:06.029275 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/139cf555-37a5-442e-955f-f5ca53dad8b2-console-oauth-config\") pod \"139cf555-37a5-442e-955f-f5ca53dad8b2\" (UID: \"139cf555-37a5-442e-955f-f5ca53dad8b2\") " Jul 16 17:24:06.029366 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:24:06.029325 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/139cf555-37a5-442e-955f-f5ca53dad8b2-trusted-ca-bundle\") pod \"139cf555-37a5-442e-955f-f5ca53dad8b2\" (UID: \"139cf555-37a5-442e-955f-f5ca53dad8b2\") " Jul 16 17:24:06.029366 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:24:06.029358 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-fplp5\" (UniqueName: \"kubernetes.io/projected/139cf555-37a5-442e-955f-f5ca53dad8b2-kube-api-access-fplp5\") pod \"139cf555-37a5-442e-955f-f5ca53dad8b2\" (UID: \"139cf555-37a5-442e-955f-f5ca53dad8b2\") " Jul 16 17:24:06.029455 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:24:06.029396 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/139cf555-37a5-442e-955f-f5ca53dad8b2-console-serving-cert\") pod \"139cf555-37a5-442e-955f-f5ca53dad8b2\" (UID: \"139cf555-37a5-442e-955f-f5ca53dad8b2\") " Jul 16 17:24:06.029455 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:24:06.029419 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/139cf555-37a5-442e-955f-f5ca53dad8b2-console-config\") pod \"139cf555-37a5-442e-955f-f5ca53dad8b2\" (UID: \"139cf555-37a5-442e-955f-f5ca53dad8b2\") " Jul 16 17:24:06.029784 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:24:06.029570 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/139cf555-37a5-442e-955f-f5ca53dad8b2-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "139cf555-37a5-442e-955f-f5ca53dad8b2" (UID: "139cf555-37a5-442e-955f-f5ca53dad8b2"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 16 17:24:06.029784 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:24:06.029627 2578 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/139cf555-37a5-442e-955f-f5ca53dad8b2-oauth-serving-cert\") on node \"ip-10-0-132-130.ec2.internal\" DevicePath \"\"" Jul 16 17:24:06.029784 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:24:06.029641 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/139cf555-37a5-442e-955f-f5ca53dad8b2-service-ca" (OuterVolumeSpecName: "service-ca") pod "139cf555-37a5-442e-955f-f5ca53dad8b2" (UID: "139cf555-37a5-442e-955f-f5ca53dad8b2"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 16 17:24:06.029982 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:24:06.029913 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/139cf555-37a5-442e-955f-f5ca53dad8b2-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "139cf555-37a5-442e-955f-f5ca53dad8b2" (UID: "139cf555-37a5-442e-955f-f5ca53dad8b2"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 16 17:24:06.029982 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:24:06.029925 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/139cf555-37a5-442e-955f-f5ca53dad8b2-console-config" (OuterVolumeSpecName: "console-config") pod "139cf555-37a5-442e-955f-f5ca53dad8b2" (UID: "139cf555-37a5-442e-955f-f5ca53dad8b2"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 16 17:24:06.031613 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:24:06.031593 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/139cf555-37a5-442e-955f-f5ca53dad8b2-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "139cf555-37a5-442e-955f-f5ca53dad8b2" (UID: "139cf555-37a5-442e-955f-f5ca53dad8b2"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 16 17:24:06.032012 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:24:06.031993 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/139cf555-37a5-442e-955f-f5ca53dad8b2-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "139cf555-37a5-442e-955f-f5ca53dad8b2" (UID: "139cf555-37a5-442e-955f-f5ca53dad8b2"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 16 17:24:06.032074 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:24:06.032025 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/139cf555-37a5-442e-955f-f5ca53dad8b2-kube-api-access-fplp5" (OuterVolumeSpecName: "kube-api-access-fplp5") pod "139cf555-37a5-442e-955f-f5ca53dad8b2" (UID: "139cf555-37a5-442e-955f-f5ca53dad8b2"). InnerVolumeSpecName "kube-api-access-fplp5". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 16 17:24:06.130109 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:24:06.130089 2578 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/139cf555-37a5-442e-955f-f5ca53dad8b2-console-oauth-config\") on node \"ip-10-0-132-130.ec2.internal\" DevicePath \"\"" Jul 16 17:24:06.130109 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:24:06.130109 2578 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/139cf555-37a5-442e-955f-f5ca53dad8b2-trusted-ca-bundle\") on node \"ip-10-0-132-130.ec2.internal\" DevicePath \"\"" Jul 16 17:24:06.130284 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:24:06.130120 2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-fplp5\" (UniqueName: \"kubernetes.io/projected/139cf555-37a5-442e-955f-f5ca53dad8b2-kube-api-access-fplp5\") on node \"ip-10-0-132-130.ec2.internal\" DevicePath \"\"" Jul 16 17:24:06.130284 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:24:06.130128 2578 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/139cf555-37a5-442e-955f-f5ca53dad8b2-console-serving-cert\") on node \"ip-10-0-132-130.ec2.internal\" DevicePath \"\"" Jul 16 17:24:06.130284 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:24:06.130138 2578 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/139cf555-37a5-442e-955f-f5ca53dad8b2-console-config\") on node \"ip-10-0-132-130.ec2.internal\" DevicePath \"\"" Jul 16 17:24:06.130284 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:24:06.130146 2578 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/139cf555-37a5-442e-955f-f5ca53dad8b2-service-ca\") on node \"ip-10-0-132-130.ec2.internal\" DevicePath \"\"" Jul 16 17:24:06.757837 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:24:06.757805 2578 generic.go:356] "Generic (PLEG): container finished" podID="139cf555-37a5-442e-955f-f5ca53dad8b2" containerID="ef7e12829a27a505baa5939ec91d121b9332dceefc5b2857467cc0de685b208b" exitCode=0 Jul 16 17:24:06.758239 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:24:06.757843 2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-8594c5d9c7-b58gt" Jul 16 17:24:06.758239 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:24:06.757883 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-8594c5d9c7-b58gt" event={"ID":"139cf555-37a5-442e-955f-f5ca53dad8b2","Type":"ContainerDied","Data":"ef7e12829a27a505baa5939ec91d121b9332dceefc5b2857467cc0de685b208b"} Jul 16 17:24:06.758239 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:24:06.757942 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-8594c5d9c7-b58gt" event={"ID":"139cf555-37a5-442e-955f-f5ca53dad8b2","Type":"ContainerDied","Data":"9a954fd98aa23268b0b16e2532e2d40be2db1881d1037370184474941d1d56f5"} Jul 16 17:24:06.758239 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:24:06.757981 2578 scope.go:117] "RemoveContainer" containerID="ef7e12829a27a505baa5939ec91d121b9332dceefc5b2857467cc0de685b208b" Jul 16 17:24:06.767393 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:24:06.767378 2578 scope.go:117] "RemoveContainer" containerID="ef7e12829a27a505baa5939ec91d121b9332dceefc5b2857467cc0de685b208b" Jul 16 17:24:06.767611 ip-10-0-132-130 kubenswrapper[2578]: E0716 17:24:06.767593 2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ef7e12829a27a505baa5939ec91d121b9332dceefc5b2857467cc0de685b208b\": container with ID starting with ef7e12829a27a505baa5939ec91d121b9332dceefc5b2857467cc0de685b208b not found: ID does not exist" containerID="ef7e12829a27a505baa5939ec91d121b9332dceefc5b2857467cc0de685b208b" Jul 16 17:24:06.767656 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:24:06.767621 2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ef7e12829a27a505baa5939ec91d121b9332dceefc5b2857467cc0de685b208b"} err="failed to get container status \"ef7e12829a27a505baa5939ec91d121b9332dceefc5b2857467cc0de685b208b\": rpc error: code = NotFound desc = could not find container \"ef7e12829a27a505baa5939ec91d121b9332dceefc5b2857467cc0de685b208b\": container with ID starting with ef7e12829a27a505baa5939ec91d121b9332dceefc5b2857467cc0de685b208b not found: ID does not exist" Jul 16 17:24:06.777629 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:24:06.777607 2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-8594c5d9c7-b58gt"] Jul 16 17:24:06.788529 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:24:06.788509 2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-8594c5d9c7-b58gt"] Jul 16 17:24:08.217411 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:24:08.217346 2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="139cf555-37a5-442e-955f-f5ca53dad8b2" path="/var/lib/kubelet/pods/139cf555-37a5-442e-955f-f5ca53dad8b2/volumes" Jul 16 17:24:12.576819 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:24:12.576783 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/istiod-openshift-gateway-94bb8fbfd-qnh6j"] Jul 16 17:24:12.577154 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:24:12.577138 2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="139cf555-37a5-442e-955f-f5ca53dad8b2" containerName="console" Jul 16 17:24:12.577154 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:24:12.577148 2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="139cf555-37a5-442e-955f-f5ca53dad8b2" containerName="console" Jul 16 17:24:12.577229 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:24:12.577207 2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="139cf555-37a5-442e-955f-f5ca53dad8b2" containerName="console" Jul 16 17:24:12.582507 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:24:12.582479 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-qnh6j" Jul 16 17:24:12.585915 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:24:12.585884 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istio-kubeconfig\"" Jul 16 17:24:12.586205 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:24:12.586187 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-openshift-gateway-dockercfg-5w5s2\"" Jul 16 17:24:12.586303 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:24:12.586206 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-tls\"" Jul 16 17:24:12.586439 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:24:12.586418 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"cacerts\"" Jul 16 17:24:12.586522 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:24:12.586468 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\"" Jul 16 17:24:12.586522 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:24:12.586420 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\"" Jul 16 17:24:12.586626 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:24:12.586527 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\"" Jul 16 17:24:12.594220 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:24:12.594198 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-94bb8fbfd-qnh6j"] Jul 16 17:24:12.686139 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:24:12.686111 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/54c915b4-e326-4c4e-b79e-5d0c16cd2811-istio-token\") pod \"istiod-openshift-gateway-94bb8fbfd-qnh6j\" (UID: \"54c915b4-e326-4c4e-b79e-5d0c16cd2811\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-qnh6j" Jul 16 17:24:12.686283 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:24:12.686197 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/54c915b4-e326-4c4e-b79e-5d0c16cd2811-istio-kubeconfig\") pod \"istiod-openshift-gateway-94bb8fbfd-qnh6j\" (UID: \"54c915b4-e326-4c4e-b79e-5d0c16cd2811\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-qnh6j" Jul 16 17:24:12.686283 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:24:12.686234 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-d94qn\" (UniqueName: \"kubernetes.io/projected/54c915b4-e326-4c4e-b79e-5d0c16cd2811-kube-api-access-d94qn\") pod \"istiod-openshift-gateway-94bb8fbfd-qnh6j\" (UID: \"54c915b4-e326-4c4e-b79e-5d0c16cd2811\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-qnh6j" Jul 16 17:24:12.686283 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:24:12.686262 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/54c915b4-e326-4c4e-b79e-5d0c16cd2811-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-94bb8fbfd-qnh6j\" (UID: \"54c915b4-e326-4c4e-b79e-5d0c16cd2811\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-qnh6j" Jul 16 17:24:12.686406 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:24:12.686294 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/54c915b4-e326-4c4e-b79e-5d0c16cd2811-cacerts\") pod \"istiod-openshift-gateway-94bb8fbfd-qnh6j\" (UID: \"54c915b4-e326-4c4e-b79e-5d0c16cd2811\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-qnh6j" Jul 16 17:24:12.686406 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:24:12.686394 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/54c915b4-e326-4c4e-b79e-5d0c16cd2811-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-94bb8fbfd-qnh6j\" (UID: \"54c915b4-e326-4c4e-b79e-5d0c16cd2811\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-qnh6j" Jul 16 17:24:12.686478 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:24:12.686431 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/54c915b4-e326-4c4e-b79e-5d0c16cd2811-local-certs\") pod \"istiod-openshift-gateway-94bb8fbfd-qnh6j\" (UID: \"54c915b4-e326-4c4e-b79e-5d0c16cd2811\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-qnh6j" Jul 16 17:24:12.787271 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:24:12.787239 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/54c915b4-e326-4c4e-b79e-5d0c16cd2811-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-94bb8fbfd-qnh6j\" (UID: \"54c915b4-e326-4c4e-b79e-5d0c16cd2811\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-qnh6j" Jul 16 17:24:12.787395 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:24:12.787282 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/54c915b4-e326-4c4e-b79e-5d0c16cd2811-local-certs\") pod \"istiod-openshift-gateway-94bb8fbfd-qnh6j\" (UID: \"54c915b4-e326-4c4e-b79e-5d0c16cd2811\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-qnh6j" Jul 16 17:24:12.787493 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:24:12.787470 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/54c915b4-e326-4c4e-b79e-5d0c16cd2811-istio-token\") pod \"istiod-openshift-gateway-94bb8fbfd-qnh6j\" (UID: \"54c915b4-e326-4c4e-b79e-5d0c16cd2811\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-qnh6j" Jul 16 17:24:12.787543 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:24:12.787528 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/54c915b4-e326-4c4e-b79e-5d0c16cd2811-istio-kubeconfig\") pod \"istiod-openshift-gateway-94bb8fbfd-qnh6j\" (UID: \"54c915b4-e326-4c4e-b79e-5d0c16cd2811\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-qnh6j" Jul 16 17:24:12.787600 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:24:12.787565 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-d94qn\" (UniqueName: \"kubernetes.io/projected/54c915b4-e326-4c4e-b79e-5d0c16cd2811-kube-api-access-d94qn\") pod \"istiod-openshift-gateway-94bb8fbfd-qnh6j\" (UID: \"54c915b4-e326-4c4e-b79e-5d0c16cd2811\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-qnh6j" Jul 16 17:24:12.787654 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:24:12.787596 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/54c915b4-e326-4c4e-b79e-5d0c16cd2811-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-94bb8fbfd-qnh6j\" (UID: \"54c915b4-e326-4c4e-b79e-5d0c16cd2811\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-qnh6j" Jul 16 17:24:12.787654 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:24:12.787632 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/54c915b4-e326-4c4e-b79e-5d0c16cd2811-cacerts\") pod \"istiod-openshift-gateway-94bb8fbfd-qnh6j\" (UID: \"54c915b4-e326-4c4e-b79e-5d0c16cd2811\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-qnh6j" Jul 16 17:24:12.788024 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:24:12.787996 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/54c915b4-e326-4c4e-b79e-5d0c16cd2811-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-94bb8fbfd-qnh6j\" (UID: \"54c915b4-e326-4c4e-b79e-5d0c16cd2811\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-qnh6j" Jul 16 17:24:12.789974 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:24:12.789944 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/54c915b4-e326-4c4e-b79e-5d0c16cd2811-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-94bb8fbfd-qnh6j\" (UID: \"54c915b4-e326-4c4e-b79e-5d0c16cd2811\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-qnh6j" Jul 16 17:24:12.790090 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:24:12.789948 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/54c915b4-e326-4c4e-b79e-5d0c16cd2811-local-certs\") pod \"istiod-openshift-gateway-94bb8fbfd-qnh6j\" (UID: \"54c915b4-e326-4c4e-b79e-5d0c16cd2811\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-qnh6j" Jul 16 17:24:12.790231 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:24:12.790210 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/54c915b4-e326-4c4e-b79e-5d0c16cd2811-cacerts\") pod \"istiod-openshift-gateway-94bb8fbfd-qnh6j\" (UID: \"54c915b4-e326-4c4e-b79e-5d0c16cd2811\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-qnh6j" Jul 16 17:24:12.790286 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:24:12.790267 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/54c915b4-e326-4c4e-b79e-5d0c16cd2811-istio-kubeconfig\") pod \"istiod-openshift-gateway-94bb8fbfd-qnh6j\" (UID: \"54c915b4-e326-4c4e-b79e-5d0c16cd2811\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-qnh6j" Jul 16 17:24:12.796584 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:24:12.796557 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/54c915b4-e326-4c4e-b79e-5d0c16cd2811-istio-token\") pod \"istiod-openshift-gateway-94bb8fbfd-qnh6j\" (UID: \"54c915b4-e326-4c4e-b79e-5d0c16cd2811\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-qnh6j" Jul 16 17:24:12.796931 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:24:12.796913 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-d94qn\" (UniqueName: \"kubernetes.io/projected/54c915b4-e326-4c4e-b79e-5d0c16cd2811-kube-api-access-d94qn\") pod \"istiod-openshift-gateway-94bb8fbfd-qnh6j\" (UID: \"54c915b4-e326-4c4e-b79e-5d0c16cd2811\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-qnh6j" Jul 16 17:24:12.892499 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:24:12.892427 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-qnh6j" Jul 16 17:24:13.028559 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:24:13.028535 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-94bb8fbfd-qnh6j"] Jul 16 17:24:13.030830 ip-10-0-132-130 kubenswrapper[2578]: W0716 17:24:13.030806 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod54c915b4_e326_4c4e_b79e_5d0c16cd2811.slice/crio-3e7787dfed63b76fbac0cedaf2e091ef6fb0f1a4f0351945c5fffdf3ed27b181 WatchSource:0}: Error finding container 3e7787dfed63b76fbac0cedaf2e091ef6fb0f1a4f0351945c5fffdf3ed27b181: Status 404 returned error can't find the container with id 3e7787dfed63b76fbac0cedaf2e091ef6fb0f1a4f0351945c5fffdf3ed27b181 Jul 16 17:24:13.798207 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:24:13.798170 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-qnh6j" event={"ID":"54c915b4-e326-4c4e-b79e-5d0c16cd2811","Type":"ContainerStarted","Data":"3e7787dfed63b76fbac0cedaf2e091ef6fb0f1a4f0351945c5fffdf3ed27b181"} Jul 16 17:24:16.210170 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:24:16.210141 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-drs2p_e55f447c-ad64-482b-a86b-0588bddf7b4d/console-operator/1.log" Jul 16 17:24:16.210565 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:24:16.210304 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-drs2p_e55f447c-ad64-482b-a86b-0588bddf7b4d/console-operator/1.log" Jul 16 17:24:16.257241 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:24:16.257186 2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236216Ki","pods":"250"} Jul 16 17:24:16.257358 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:24:16.257269 2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236216Ki","pods":"250"} Jul 16 17:24:16.818020 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:24:16.817975 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-qnh6j" event={"ID":"54c915b4-e326-4c4e-b79e-5d0c16cd2811","Type":"ContainerStarted","Data":"09e52b7efb9b13e5ed427c718a170057501c80869f10c5828f63188b459e6d6d"} Jul 16 17:24:16.818195 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:24:16.818127 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-qnh6j" Jul 16 17:24:16.848206 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:24:16.848149 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-qnh6j" podStartSLOduration=1.623875602 podStartE2EDuration="4.848132441s" totalImagesPullingTime="3.224256839s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 17:24:12 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 17:24:16.846665425 +0000 UTC m=+601.332509424" watchObservedRunningTime="2026-07-16 17:24:16.848132441 +0000 UTC m=+601.333976330" Jul 16 17:24:17.824463 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:24:17.824426 2578 patch_prober.go:28] interesting pod/istiod-openshift-gateway-94bb8fbfd-qnh6j container/discovery namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body= Jul 16 17:24:17.824830 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:24:17.824488 2578 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-qnh6j" podUID="54c915b4-e326-4c4e-b79e-5d0c16cd2811" containerName="discovery" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 16 17:24:20.823825 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:24:20.823793 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-qnh6j" Jul 16 17:24:29.788807 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:24:29.788773 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/af2a5cb5935dca7d5808b44f590e0d3d1363f94abbf5c1d8a71e95fcabf8qpn"] Jul 16 17:24:29.792407 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:24:29.792391 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/af2a5cb5935dca7d5808b44f590e0d3d1363f94abbf5c1d8a71e95fcabf8qpn" Jul 16 17:24:29.794441 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:24:29.794418 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-xgzqt\"" Jul 16 17:24:29.794550 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:24:29.794509 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\"" Jul 16 17:24:29.795165 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:24:29.795148 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\"" Jul 16 17:24:29.801391 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:24:29.801369 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/af2a5cb5935dca7d5808b44f590e0d3d1363f94abbf5c1d8a71e95fcabf8qpn"] Jul 16 17:24:29.905763 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:24:29.905735 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/f13fb8b568e19b60b32189bed3ab51e2b7cfe62efa490862cd57eead76hbc6v"] Jul 16 17:24:29.909460 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:24:29.909438 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/f13fb8b568e19b60b32189bed3ab51e2b7cfe62efa490862cd57eead76hbc6v" Jul 16 17:24:29.919895 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:24:29.919871 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/f13fb8b568e19b60b32189bed3ab51e2b7cfe62efa490862cd57eead76hbc6v"] Jul 16 17:24:29.927056 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:24:29.927032 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/4c49c2c6-6348-49a1-a480-d63d7a562e69-util\") pod \"af2a5cb5935dca7d5808b44f590e0d3d1363f94abbf5c1d8a71e95fcabf8qpn\" (UID: \"4c49c2c6-6348-49a1-a480-d63d7a562e69\") " pod="openshift-marketplace/af2a5cb5935dca7d5808b44f590e0d3d1363f94abbf5c1d8a71e95fcabf8qpn" Jul 16 17:24:29.927144 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:24:29.927101 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/4c49c2c6-6348-49a1-a480-d63d7a562e69-bundle\") pod \"af2a5cb5935dca7d5808b44f590e0d3d1363f94abbf5c1d8a71e95fcabf8qpn\" (UID: \"4c49c2c6-6348-49a1-a480-d63d7a562e69\") " pod="openshift-marketplace/af2a5cb5935dca7d5808b44f590e0d3d1363f94abbf5c1d8a71e95fcabf8qpn" Jul 16 17:24:29.927144 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:24:29.927131 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wfwp7\" (UniqueName: \"kubernetes.io/projected/4c49c2c6-6348-49a1-a480-d63d7a562e69-kube-api-access-wfwp7\") pod \"af2a5cb5935dca7d5808b44f590e0d3d1363f94abbf5c1d8a71e95fcabf8qpn\" (UID: \"4c49c2c6-6348-49a1-a480-d63d7a562e69\") " pod="openshift-marketplace/af2a5cb5935dca7d5808b44f590e0d3d1363f94abbf5c1d8a71e95fcabf8qpn" Jul 16 17:24:30.027932 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:24:30.027902 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/d3884845-614f-482d-a893-e8bc1d72cda0-bundle\") pod \"f13fb8b568e19b60b32189bed3ab51e2b7cfe62efa490862cd57eead76hbc6v\" (UID: \"d3884845-614f-482d-a893-e8bc1d72cda0\") " pod="openshift-marketplace/f13fb8b568e19b60b32189bed3ab51e2b7cfe62efa490862cd57eead76hbc6v" Jul 16 17:24:30.028106 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:24:30.028077 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/4c49c2c6-6348-49a1-a480-d63d7a562e69-util\") pod \"af2a5cb5935dca7d5808b44f590e0d3d1363f94abbf5c1d8a71e95fcabf8qpn\" (UID: \"4c49c2c6-6348-49a1-a480-d63d7a562e69\") " pod="openshift-marketplace/af2a5cb5935dca7d5808b44f590e0d3d1363f94abbf5c1d8a71e95fcabf8qpn" Jul 16 17:24:30.028168 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:24:30.028122 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rvr8x\" (UniqueName: \"kubernetes.io/projected/d3884845-614f-482d-a893-e8bc1d72cda0-kube-api-access-rvr8x\") pod \"f13fb8b568e19b60b32189bed3ab51e2b7cfe62efa490862cd57eead76hbc6v\" (UID: \"d3884845-614f-482d-a893-e8bc1d72cda0\") " pod="openshift-marketplace/f13fb8b568e19b60b32189bed3ab51e2b7cfe62efa490862cd57eead76hbc6v" Jul 16 17:24:30.028168 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:24:30.028160 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/d3884845-614f-482d-a893-e8bc1d72cda0-util\") pod \"f13fb8b568e19b60b32189bed3ab51e2b7cfe62efa490862cd57eead76hbc6v\" (UID: \"d3884845-614f-482d-a893-e8bc1d72cda0\") " pod="openshift-marketplace/f13fb8b568e19b60b32189bed3ab51e2b7cfe62efa490862cd57eead76hbc6v" Jul 16 17:24:30.028248 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:24:30.028195 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/4c49c2c6-6348-49a1-a480-d63d7a562e69-bundle\") pod \"af2a5cb5935dca7d5808b44f590e0d3d1363f94abbf5c1d8a71e95fcabf8qpn\" (UID: \"4c49c2c6-6348-49a1-a480-d63d7a562e69\") " pod="openshift-marketplace/af2a5cb5935dca7d5808b44f590e0d3d1363f94abbf5c1d8a71e95fcabf8qpn" Jul 16 17:24:30.028291 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:24:30.028243 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wfwp7\" (UniqueName: \"kubernetes.io/projected/4c49c2c6-6348-49a1-a480-d63d7a562e69-kube-api-access-wfwp7\") pod \"af2a5cb5935dca7d5808b44f590e0d3d1363f94abbf5c1d8a71e95fcabf8qpn\" (UID: \"4c49c2c6-6348-49a1-a480-d63d7a562e69\") " pod="openshift-marketplace/af2a5cb5935dca7d5808b44f590e0d3d1363f94abbf5c1d8a71e95fcabf8qpn" Jul 16 17:24:30.028416 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:24:30.028400 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/4c49c2c6-6348-49a1-a480-d63d7a562e69-util\") pod \"af2a5cb5935dca7d5808b44f590e0d3d1363f94abbf5c1d8a71e95fcabf8qpn\" (UID: \"4c49c2c6-6348-49a1-a480-d63d7a562e69\") " pod="openshift-marketplace/af2a5cb5935dca7d5808b44f590e0d3d1363f94abbf5c1d8a71e95fcabf8qpn" Jul 16 17:24:30.028493 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:24:30.028476 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/4c49c2c6-6348-49a1-a480-d63d7a562e69-bundle\") pod \"af2a5cb5935dca7d5808b44f590e0d3d1363f94abbf5c1d8a71e95fcabf8qpn\" (UID: \"4c49c2c6-6348-49a1-a480-d63d7a562e69\") " pod="openshift-marketplace/af2a5cb5935dca7d5808b44f590e0d3d1363f94abbf5c1d8a71e95fcabf8qpn" Jul 16 17:24:30.030110 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:24:30.030093 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/0e9e3357d40614682d1ebdb402e8ba62e1e258fb473e6f54140aafff365pvsj"] Jul 16 17:24:30.033929 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:24:30.033907 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/0e9e3357d40614682d1ebdb402e8ba62e1e258fb473e6f54140aafff365pvsj" Jul 16 17:24:30.037300 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:24:30.037277 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wfwp7\" (UniqueName: \"kubernetes.io/projected/4c49c2c6-6348-49a1-a480-d63d7a562e69-kube-api-access-wfwp7\") pod \"af2a5cb5935dca7d5808b44f590e0d3d1363f94abbf5c1d8a71e95fcabf8qpn\" (UID: \"4c49c2c6-6348-49a1-a480-d63d7a562e69\") " pod="openshift-marketplace/af2a5cb5935dca7d5808b44f590e0d3d1363f94abbf5c1d8a71e95fcabf8qpn" Jul 16 17:24:30.047031 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:24:30.046960 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/0e9e3357d40614682d1ebdb402e8ba62e1e258fb473e6f54140aafff365pvsj"] Jul 16 17:24:30.102157 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:24:30.102130 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/af2a5cb5935dca7d5808b44f590e0d3d1363f94abbf5c1d8a71e95fcabf8qpn" Jul 16 17:24:30.111267 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:24:30.111240 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/299160ebe81f164e400242457c5a1580453e7d84fd8087b72a6eb317efjnrlp"] Jul 16 17:24:30.115327 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:24:30.115306 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/299160ebe81f164e400242457c5a1580453e7d84fd8087b72a6eb317efjnrlp" Jul 16 17:24:30.128969 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:24:30.128945 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rvr8x\" (UniqueName: \"kubernetes.io/projected/d3884845-614f-482d-a893-e8bc1d72cda0-kube-api-access-rvr8x\") pod \"f13fb8b568e19b60b32189bed3ab51e2b7cfe62efa490862cd57eead76hbc6v\" (UID: \"d3884845-614f-482d-a893-e8bc1d72cda0\") " pod="openshift-marketplace/f13fb8b568e19b60b32189bed3ab51e2b7cfe62efa490862cd57eead76hbc6v" Jul 16 17:24:30.129084 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:24:30.128991 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/d3884845-614f-482d-a893-e8bc1d72cda0-util\") pod \"f13fb8b568e19b60b32189bed3ab51e2b7cfe62efa490862cd57eead76hbc6v\" (UID: \"d3884845-614f-482d-a893-e8bc1d72cda0\") " pod="openshift-marketplace/f13fb8b568e19b60b32189bed3ab51e2b7cfe62efa490862cd57eead76hbc6v" Jul 16 17:24:30.129084 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:24:30.129069 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/d3884845-614f-482d-a893-e8bc1d72cda0-bundle\") pod \"f13fb8b568e19b60b32189bed3ab51e2b7cfe62efa490862cd57eead76hbc6v\" (UID: \"d3884845-614f-482d-a893-e8bc1d72cda0\") " pod="openshift-marketplace/f13fb8b568e19b60b32189bed3ab51e2b7cfe62efa490862cd57eead76hbc6v" Jul 16 17:24:30.129197 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:24:30.129103 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/9cafb707-c0e9-4bac-b722-c0912bc52fc5-bundle\") pod \"0e9e3357d40614682d1ebdb402e8ba62e1e258fb473e6f54140aafff365pvsj\" (UID: \"9cafb707-c0e9-4bac-b722-c0912bc52fc5\") " pod="openshift-marketplace/0e9e3357d40614682d1ebdb402e8ba62e1e258fb473e6f54140aafff365pvsj" Jul 16 17:24:30.129197 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:24:30.129145 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tmt5r\" (UniqueName: \"kubernetes.io/projected/9cafb707-c0e9-4bac-b722-c0912bc52fc5-kube-api-access-tmt5r\") pod \"0e9e3357d40614682d1ebdb402e8ba62e1e258fb473e6f54140aafff365pvsj\" (UID: \"9cafb707-c0e9-4bac-b722-c0912bc52fc5\") " pod="openshift-marketplace/0e9e3357d40614682d1ebdb402e8ba62e1e258fb473e6f54140aafff365pvsj" Jul 16 17:24:30.129334 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:24:30.129309 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/9cafb707-c0e9-4bac-b722-c0912bc52fc5-util\") pod \"0e9e3357d40614682d1ebdb402e8ba62e1e258fb473e6f54140aafff365pvsj\" (UID: \"9cafb707-c0e9-4bac-b722-c0912bc52fc5\") " pod="openshift-marketplace/0e9e3357d40614682d1ebdb402e8ba62e1e258fb473e6f54140aafff365pvsj" Jul 16 17:24:30.129538 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:24:30.129515 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/d3884845-614f-482d-a893-e8bc1d72cda0-bundle\") pod \"f13fb8b568e19b60b32189bed3ab51e2b7cfe62efa490862cd57eead76hbc6v\" (UID: \"d3884845-614f-482d-a893-e8bc1d72cda0\") " pod="openshift-marketplace/f13fb8b568e19b60b32189bed3ab51e2b7cfe62efa490862cd57eead76hbc6v" Jul 16 17:24:30.129601 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:24:30.129535 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/d3884845-614f-482d-a893-e8bc1d72cda0-util\") pod \"f13fb8b568e19b60b32189bed3ab51e2b7cfe62efa490862cd57eead76hbc6v\" (UID: \"d3884845-614f-482d-a893-e8bc1d72cda0\") " pod="openshift-marketplace/f13fb8b568e19b60b32189bed3ab51e2b7cfe62efa490862cd57eead76hbc6v" Jul 16 17:24:30.133898 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:24:30.133877 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/299160ebe81f164e400242457c5a1580453e7d84fd8087b72a6eb317efjnrlp"] Jul 16 17:24:30.139362 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:24:30.139326 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rvr8x\" (UniqueName: \"kubernetes.io/projected/d3884845-614f-482d-a893-e8bc1d72cda0-kube-api-access-rvr8x\") pod \"f13fb8b568e19b60b32189bed3ab51e2b7cfe62efa490862cd57eead76hbc6v\" (UID: \"d3884845-614f-482d-a893-e8bc1d72cda0\") " pod="openshift-marketplace/f13fb8b568e19b60b32189bed3ab51e2b7cfe62efa490862cd57eead76hbc6v" Jul 16 17:24:30.218075 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:24:30.218044 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/f13fb8b568e19b60b32189bed3ab51e2b7cfe62efa490862cd57eead76hbc6v" Jul 16 17:24:30.230837 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:24:30.230805 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-shdz7\" (UniqueName: \"kubernetes.io/projected/17246662-bdb1-4467-8515-88099547779c-kube-api-access-shdz7\") pod \"299160ebe81f164e400242457c5a1580453e7d84fd8087b72a6eb317efjnrlp\" (UID: \"17246662-bdb1-4467-8515-88099547779c\") " pod="openshift-marketplace/299160ebe81f164e400242457c5a1580453e7d84fd8087b72a6eb317efjnrlp" Jul 16 17:24:30.230952 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:24:30.230868 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/9cafb707-c0e9-4bac-b722-c0912bc52fc5-util\") pod \"0e9e3357d40614682d1ebdb402e8ba62e1e258fb473e6f54140aafff365pvsj\" (UID: \"9cafb707-c0e9-4bac-b722-c0912bc52fc5\") " pod="openshift-marketplace/0e9e3357d40614682d1ebdb402e8ba62e1e258fb473e6f54140aafff365pvsj" Jul 16 17:24:30.230952 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:24:30.230930 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/17246662-bdb1-4467-8515-88099547779c-util\") pod \"299160ebe81f164e400242457c5a1580453e7d84fd8087b72a6eb317efjnrlp\" (UID: \"17246662-bdb1-4467-8515-88099547779c\") " pod="openshift-marketplace/299160ebe81f164e400242457c5a1580453e7d84fd8087b72a6eb317efjnrlp" Jul 16 17:24:30.231086 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:24:30.231061 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/17246662-bdb1-4467-8515-88099547779c-bundle\") pod \"299160ebe81f164e400242457c5a1580453e7d84fd8087b72a6eb317efjnrlp\" (UID: \"17246662-bdb1-4467-8515-88099547779c\") " pod="openshift-marketplace/299160ebe81f164e400242457c5a1580453e7d84fd8087b72a6eb317efjnrlp" Jul 16 17:24:30.231161 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:24:30.231145 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/9cafb707-c0e9-4bac-b722-c0912bc52fc5-bundle\") pod \"0e9e3357d40614682d1ebdb402e8ba62e1e258fb473e6f54140aafff365pvsj\" (UID: \"9cafb707-c0e9-4bac-b722-c0912bc52fc5\") " pod="openshift-marketplace/0e9e3357d40614682d1ebdb402e8ba62e1e258fb473e6f54140aafff365pvsj" Jul 16 17:24:30.231215 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:24:30.231180 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/9cafb707-c0e9-4bac-b722-c0912bc52fc5-util\") pod \"0e9e3357d40614682d1ebdb402e8ba62e1e258fb473e6f54140aafff365pvsj\" (UID: \"9cafb707-c0e9-4bac-b722-c0912bc52fc5\") " pod="openshift-marketplace/0e9e3357d40614682d1ebdb402e8ba62e1e258fb473e6f54140aafff365pvsj" Jul 16 17:24:30.231215 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:24:30.231189 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tmt5r\" (UniqueName: \"kubernetes.io/projected/9cafb707-c0e9-4bac-b722-c0912bc52fc5-kube-api-access-tmt5r\") pod \"0e9e3357d40614682d1ebdb402e8ba62e1e258fb473e6f54140aafff365pvsj\" (UID: \"9cafb707-c0e9-4bac-b722-c0912bc52fc5\") " pod="openshift-marketplace/0e9e3357d40614682d1ebdb402e8ba62e1e258fb473e6f54140aafff365pvsj" Jul 16 17:24:30.231460 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:24:30.231420 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/9cafb707-c0e9-4bac-b722-c0912bc52fc5-bundle\") pod \"0e9e3357d40614682d1ebdb402e8ba62e1e258fb473e6f54140aafff365pvsj\" (UID: \"9cafb707-c0e9-4bac-b722-c0912bc52fc5\") " pod="openshift-marketplace/0e9e3357d40614682d1ebdb402e8ba62e1e258fb473e6f54140aafff365pvsj" Jul 16 17:24:30.238646 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:24:30.238609 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/af2a5cb5935dca7d5808b44f590e0d3d1363f94abbf5c1d8a71e95fcabf8qpn"] Jul 16 17:24:30.239889 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:24:30.239867 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tmt5r\" (UniqueName: \"kubernetes.io/projected/9cafb707-c0e9-4bac-b722-c0912bc52fc5-kube-api-access-tmt5r\") pod \"0e9e3357d40614682d1ebdb402e8ba62e1e258fb473e6f54140aafff365pvsj\" (UID: \"9cafb707-c0e9-4bac-b722-c0912bc52fc5\") " pod="openshift-marketplace/0e9e3357d40614682d1ebdb402e8ba62e1e258fb473e6f54140aafff365pvsj" Jul 16 17:24:30.241283 ip-10-0-132-130 kubenswrapper[2578]: W0716 17:24:30.241252 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4c49c2c6_6348_49a1_a480_d63d7a562e69.slice/crio-8405537f2cdb52d914c14fe814202d9f4ab71547d02add24d428bacc2ff936f6 WatchSource:0}: Error finding container 8405537f2cdb52d914c14fe814202d9f4ab71547d02add24d428bacc2ff936f6: Status 404 returned error can't find the container with id 8405537f2cdb52d914c14fe814202d9f4ab71547d02add24d428bacc2ff936f6 Jul 16 17:24:30.332636 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:24:30.332600 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/17246662-bdb1-4467-8515-88099547779c-bundle\") pod \"299160ebe81f164e400242457c5a1580453e7d84fd8087b72a6eb317efjnrlp\" (UID: \"17246662-bdb1-4467-8515-88099547779c\") " pod="openshift-marketplace/299160ebe81f164e400242457c5a1580453e7d84fd8087b72a6eb317efjnrlp" Jul 16 17:24:30.332820 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:24:30.332721 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-shdz7\" (UniqueName: \"kubernetes.io/projected/17246662-bdb1-4467-8515-88099547779c-kube-api-access-shdz7\") pod \"299160ebe81f164e400242457c5a1580453e7d84fd8087b72a6eb317efjnrlp\" (UID: \"17246662-bdb1-4467-8515-88099547779c\") " pod="openshift-marketplace/299160ebe81f164e400242457c5a1580453e7d84fd8087b72a6eb317efjnrlp" Jul 16 17:24:30.332820 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:24:30.332782 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/17246662-bdb1-4467-8515-88099547779c-util\") pod \"299160ebe81f164e400242457c5a1580453e7d84fd8087b72a6eb317efjnrlp\" (UID: \"17246662-bdb1-4467-8515-88099547779c\") " pod="openshift-marketplace/299160ebe81f164e400242457c5a1580453e7d84fd8087b72a6eb317efjnrlp" Jul 16 17:24:30.333129 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:24:30.333079 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/17246662-bdb1-4467-8515-88099547779c-bundle\") pod \"299160ebe81f164e400242457c5a1580453e7d84fd8087b72a6eb317efjnrlp\" (UID: \"17246662-bdb1-4467-8515-88099547779c\") " pod="openshift-marketplace/299160ebe81f164e400242457c5a1580453e7d84fd8087b72a6eb317efjnrlp" Jul 16 17:24:30.333246 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:24:30.333215 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/17246662-bdb1-4467-8515-88099547779c-util\") pod \"299160ebe81f164e400242457c5a1580453e7d84fd8087b72a6eb317efjnrlp\" (UID: \"17246662-bdb1-4467-8515-88099547779c\") " pod="openshift-marketplace/299160ebe81f164e400242457c5a1580453e7d84fd8087b72a6eb317efjnrlp" Jul 16 17:24:30.345864 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:24:30.345842 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-shdz7\" (UniqueName: \"kubernetes.io/projected/17246662-bdb1-4467-8515-88099547779c-kube-api-access-shdz7\") pod \"299160ebe81f164e400242457c5a1580453e7d84fd8087b72a6eb317efjnrlp\" (UID: \"17246662-bdb1-4467-8515-88099547779c\") " pod="openshift-marketplace/299160ebe81f164e400242457c5a1580453e7d84fd8087b72a6eb317efjnrlp" Jul 16 17:24:30.357376 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:24:30.357357 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/f13fb8b568e19b60b32189bed3ab51e2b7cfe62efa490862cd57eead76hbc6v"] Jul 16 17:24:30.358243 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:24:30.358223 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/0e9e3357d40614682d1ebdb402e8ba62e1e258fb473e6f54140aafff365pvsj" Jul 16 17:24:30.415796 ip-10-0-132-130 kubenswrapper[2578]: W0716 17:24:30.415756 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd3884845_614f_482d_a893_e8bc1d72cda0.slice/crio-7bf67c76c240b5d896e46a9eedfc68b2b08e83944b711ceff54df1dbe74cc659 WatchSource:0}: Error finding container 7bf67c76c240b5d896e46a9eedfc68b2b08e83944b711ceff54df1dbe74cc659: Status 404 returned error can't find the container with id 7bf67c76c240b5d896e46a9eedfc68b2b08e83944b711ceff54df1dbe74cc659 Jul 16 17:24:30.433352 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:24:30.433331 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/299160ebe81f164e400242457c5a1580453e7d84fd8087b72a6eb317efjnrlp" Jul 16 17:24:30.499912 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:24:30.499883 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/0e9e3357d40614682d1ebdb402e8ba62e1e258fb473e6f54140aafff365pvsj"] Jul 16 17:24:30.501118 ip-10-0-132-130 kubenswrapper[2578]: W0716 17:24:30.501092 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9cafb707_c0e9_4bac_b722_c0912bc52fc5.slice/crio-b832682d6f24b686aec4d63935a42c7457e99fd2e357f90921484a73132b048b WatchSource:0}: Error finding container b832682d6f24b686aec4d63935a42c7457e99fd2e357f90921484a73132b048b: Status 404 returned error can't find the container with id b832682d6f24b686aec4d63935a42c7457e99fd2e357f90921484a73132b048b Jul 16 17:24:30.581277 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:24:30.581077 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/299160ebe81f164e400242457c5a1580453e7d84fd8087b72a6eb317efjnrlp"] Jul 16 17:24:30.583822 ip-10-0-132-130 kubenswrapper[2578]: W0716 17:24:30.583798 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod17246662_bdb1_4467_8515_88099547779c.slice/crio-6833bdcc4b49b752d893559f7030c3005066fe384d0255ce6a0eb35ca5e2a7ca WatchSource:0}: Error finding container 6833bdcc4b49b752d893559f7030c3005066fe384d0255ce6a0eb35ca5e2a7ca: Status 404 returned error can't find the container with id 6833bdcc4b49b752d893559f7030c3005066fe384d0255ce6a0eb35ca5e2a7ca Jul 16 17:24:30.895520 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:24:30.895438 2578 generic.go:356] "Generic (PLEG): container finished" podID="17246662-bdb1-4467-8515-88099547779c" containerID="64d0cab3bdd43d605f858cb154c9f33a54f2db89a767671bd56532147dc4416f" exitCode=0 Jul 16 17:24:30.895520 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:24:30.895473 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/299160ebe81f164e400242457c5a1580453e7d84fd8087b72a6eb317efjnrlp" event={"ID":"17246662-bdb1-4467-8515-88099547779c","Type":"ContainerDied","Data":"64d0cab3bdd43d605f858cb154c9f33a54f2db89a767671bd56532147dc4416f"} Jul 16 17:24:30.895951 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:24:30.895536 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/299160ebe81f164e400242457c5a1580453e7d84fd8087b72a6eb317efjnrlp" event={"ID":"17246662-bdb1-4467-8515-88099547779c","Type":"ContainerStarted","Data":"6833bdcc4b49b752d893559f7030c3005066fe384d0255ce6a0eb35ca5e2a7ca"} Jul 16 17:24:30.897104 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:24:30.897027 2578 generic.go:356] "Generic (PLEG): container finished" podID="9cafb707-c0e9-4bac-b722-c0912bc52fc5" containerID="3bfe0b9bab7b42fe2fcb59337ed52d0540134a7b62f534255fd02174e44cb9b4" exitCode=0 Jul 16 17:24:30.897185 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:24:30.897102 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/0e9e3357d40614682d1ebdb402e8ba62e1e258fb473e6f54140aafff365pvsj" event={"ID":"9cafb707-c0e9-4bac-b722-c0912bc52fc5","Type":"ContainerDied","Data":"3bfe0b9bab7b42fe2fcb59337ed52d0540134a7b62f534255fd02174e44cb9b4"} Jul 16 17:24:30.897185 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:24:30.897141 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/0e9e3357d40614682d1ebdb402e8ba62e1e258fb473e6f54140aafff365pvsj" event={"ID":"9cafb707-c0e9-4bac-b722-c0912bc52fc5","Type":"ContainerStarted","Data":"b832682d6f24b686aec4d63935a42c7457e99fd2e357f90921484a73132b048b"} Jul 16 17:24:30.898669 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:24:30.898642 2578 generic.go:356] "Generic (PLEG): container finished" podID="d3884845-614f-482d-a893-e8bc1d72cda0" containerID="23176bb2e0dd31f0ff4aa0d3a0a3ed815741a6a4bea5c4648dcc91b821b0bad8" exitCode=0 Jul 16 17:24:30.898764 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:24:30.898724 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/f13fb8b568e19b60b32189bed3ab51e2b7cfe62efa490862cd57eead76hbc6v" event={"ID":"d3884845-614f-482d-a893-e8bc1d72cda0","Type":"ContainerDied","Data":"23176bb2e0dd31f0ff4aa0d3a0a3ed815741a6a4bea5c4648dcc91b821b0bad8"} Jul 16 17:24:30.898831 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:24:30.898764 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/f13fb8b568e19b60b32189bed3ab51e2b7cfe62efa490862cd57eead76hbc6v" event={"ID":"d3884845-614f-482d-a893-e8bc1d72cda0","Type":"ContainerStarted","Data":"7bf67c76c240b5d896e46a9eedfc68b2b08e83944b711ceff54df1dbe74cc659"} Jul 16 17:24:30.900530 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:24:30.900365 2578 generic.go:356] "Generic (PLEG): container finished" podID="4c49c2c6-6348-49a1-a480-d63d7a562e69" containerID="51ca9abd2e2ca6acdfa62a1f8e28563a650a2f60c13e693158dcf5eeb2a49b4a" exitCode=0 Jul 16 17:24:30.900530 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:24:30.900414 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/af2a5cb5935dca7d5808b44f590e0d3d1363f94abbf5c1d8a71e95fcabf8qpn" event={"ID":"4c49c2c6-6348-49a1-a480-d63d7a562e69","Type":"ContainerDied","Data":"51ca9abd2e2ca6acdfa62a1f8e28563a650a2f60c13e693158dcf5eeb2a49b4a"} Jul 16 17:24:30.900530 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:24:30.900455 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/af2a5cb5935dca7d5808b44f590e0d3d1363f94abbf5c1d8a71e95fcabf8qpn" event={"ID":"4c49c2c6-6348-49a1-a480-d63d7a562e69","Type":"ContainerStarted","Data":"8405537f2cdb52d914c14fe814202d9f4ab71547d02add24d428bacc2ff936f6"} Jul 16 17:24:31.908311 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:24:31.908247 2578 generic.go:356] "Generic (PLEG): container finished" podID="d3884845-614f-482d-a893-e8bc1d72cda0" containerID="81b75d61e03795194c46c4bff8e7069ae7d27b48266b6fab70b263698e89637f" exitCode=0 Jul 16 17:24:31.908702 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:24:31.908323 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/f13fb8b568e19b60b32189bed3ab51e2b7cfe62efa490862cd57eead76hbc6v" event={"ID":"d3884845-614f-482d-a893-e8bc1d72cda0","Type":"ContainerDied","Data":"81b75d61e03795194c46c4bff8e7069ae7d27b48266b6fab70b263698e89637f"} Jul 16 17:24:31.913019 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:24:31.912988 2578 generic.go:356] "Generic (PLEG): container finished" podID="17246662-bdb1-4467-8515-88099547779c" containerID="da0c5759685e66edcd7040a366fb4ec7c264afd4eeed21724d3e6a9038d69049" exitCode=0 Jul 16 17:24:31.913123 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:24:31.913048 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/299160ebe81f164e400242457c5a1580453e7d84fd8087b72a6eb317efjnrlp" event={"ID":"17246662-bdb1-4467-8515-88099547779c","Type":"ContainerDied","Data":"da0c5759685e66edcd7040a366fb4ec7c264afd4eeed21724d3e6a9038d69049"} Jul 16 17:24:32.920400 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:24:32.920365 2578 generic.go:356] "Generic (PLEG): container finished" podID="17246662-bdb1-4467-8515-88099547779c" containerID="0eea08b79824c3ef3cd622d1ab5a72c937a88a3957517a45a4e5312b69a1b935" exitCode=0 Jul 16 17:24:32.920814 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:24:32.920446 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/299160ebe81f164e400242457c5a1580453e7d84fd8087b72a6eb317efjnrlp" event={"ID":"17246662-bdb1-4467-8515-88099547779c","Type":"ContainerDied","Data":"0eea08b79824c3ef3cd622d1ab5a72c937a88a3957517a45a4e5312b69a1b935"} Jul 16 17:24:32.922085 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:24:32.922063 2578 generic.go:356] "Generic (PLEG): container finished" podID="9cafb707-c0e9-4bac-b722-c0912bc52fc5" containerID="f9868dfd446b8948529e8917a5c870a65e2746823f6e7b8c35f6b4710b5ab4bd" exitCode=0 Jul 16 17:24:32.922196 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:24:32.922127 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/0e9e3357d40614682d1ebdb402e8ba62e1e258fb473e6f54140aafff365pvsj" event={"ID":"9cafb707-c0e9-4bac-b722-c0912bc52fc5","Type":"ContainerDied","Data":"f9868dfd446b8948529e8917a5c870a65e2746823f6e7b8c35f6b4710b5ab4bd"} Jul 16 17:24:32.924437 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:24:32.924412 2578 generic.go:356] "Generic (PLEG): container finished" podID="d3884845-614f-482d-a893-e8bc1d72cda0" containerID="271d7cfa523a2a8f3ebd0da71196bc2dcf3edd893337c6c4ce00c148a644f27d" exitCode=0 Jul 16 17:24:32.924533 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:24:32.924493 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/f13fb8b568e19b60b32189bed3ab51e2b7cfe62efa490862cd57eead76hbc6v" event={"ID":"d3884845-614f-482d-a893-e8bc1d72cda0","Type":"ContainerDied","Data":"271d7cfa523a2a8f3ebd0da71196bc2dcf3edd893337c6c4ce00c148a644f27d"} Jul 16 17:24:33.931521 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:24:33.931485 2578 generic.go:356] "Generic (PLEG): container finished" podID="9cafb707-c0e9-4bac-b722-c0912bc52fc5" containerID="f82e27e9af26fef95799186fd249b415347ec1b5bbc43d2d4214e63dc9e572f5" exitCode=0 Jul 16 17:24:33.932016 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:24:33.931580 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/0e9e3357d40614682d1ebdb402e8ba62e1e258fb473e6f54140aafff365pvsj" event={"ID":"9cafb707-c0e9-4bac-b722-c0912bc52fc5","Type":"ContainerDied","Data":"f82e27e9af26fef95799186fd249b415347ec1b5bbc43d2d4214e63dc9e572f5"} Jul 16 17:24:34.089477 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:24:34.089454 2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/f13fb8b568e19b60b32189bed3ab51e2b7cfe62efa490862cd57eead76hbc6v" Jul 16 17:24:34.092471 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:24:34.092451 2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/299160ebe81f164e400242457c5a1580453e7d84fd8087b72a6eb317efjnrlp" Jul 16 17:24:34.166059 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:24:34.166034 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-shdz7\" (UniqueName: \"kubernetes.io/projected/17246662-bdb1-4467-8515-88099547779c-kube-api-access-shdz7\") pod \"17246662-bdb1-4467-8515-88099547779c\" (UID: \"17246662-bdb1-4467-8515-88099547779c\") " Jul 16 17:24:34.166207 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:24:34.166125 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/17246662-bdb1-4467-8515-88099547779c-util\") pod \"17246662-bdb1-4467-8515-88099547779c\" (UID: \"17246662-bdb1-4467-8515-88099547779c\") " Jul 16 17:24:34.166207 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:24:34.166179 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/d3884845-614f-482d-a893-e8bc1d72cda0-bundle\") pod \"d3884845-614f-482d-a893-e8bc1d72cda0\" (UID: \"d3884845-614f-482d-a893-e8bc1d72cda0\") " Jul 16 17:24:34.166299 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:24:34.166220 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/17246662-bdb1-4467-8515-88099547779c-bundle\") pod \"17246662-bdb1-4467-8515-88099547779c\" (UID: \"17246662-bdb1-4467-8515-88099547779c\") " Jul 16 17:24:34.166299 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:24:34.166254 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-rvr8x\" (UniqueName: \"kubernetes.io/projected/d3884845-614f-482d-a893-e8bc1d72cda0-kube-api-access-rvr8x\") pod \"d3884845-614f-482d-a893-e8bc1d72cda0\" (UID: \"d3884845-614f-482d-a893-e8bc1d72cda0\") " Jul 16 17:24:34.166299 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:24:34.166288 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/d3884845-614f-482d-a893-e8bc1d72cda0-util\") pod \"d3884845-614f-482d-a893-e8bc1d72cda0\" (UID: \"d3884845-614f-482d-a893-e8bc1d72cda0\") " Jul 16 17:24:34.166787 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:24:34.166748 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/17246662-bdb1-4467-8515-88099547779c-bundle" (OuterVolumeSpecName: "bundle") pod "17246662-bdb1-4467-8515-88099547779c" (UID: "17246662-bdb1-4467-8515-88099547779c"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 16 17:24:34.166946 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:24:34.166924 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d3884845-614f-482d-a893-e8bc1d72cda0-bundle" (OuterVolumeSpecName: "bundle") pod "d3884845-614f-482d-a893-e8bc1d72cda0" (UID: "d3884845-614f-482d-a893-e8bc1d72cda0"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 16 17:24:34.168314 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:24:34.168284 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/17246662-bdb1-4467-8515-88099547779c-kube-api-access-shdz7" (OuterVolumeSpecName: "kube-api-access-shdz7") pod "17246662-bdb1-4467-8515-88099547779c" (UID: "17246662-bdb1-4467-8515-88099547779c"). InnerVolumeSpecName "kube-api-access-shdz7". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 16 17:24:34.168922 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:24:34.168896 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d3884845-614f-482d-a893-e8bc1d72cda0-kube-api-access-rvr8x" (OuterVolumeSpecName: "kube-api-access-rvr8x") pod "d3884845-614f-482d-a893-e8bc1d72cda0" (UID: "d3884845-614f-482d-a893-e8bc1d72cda0"). InnerVolumeSpecName "kube-api-access-rvr8x". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 16 17:24:34.172100 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:24:34.172078 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/17246662-bdb1-4467-8515-88099547779c-util" (OuterVolumeSpecName: "util") pod "17246662-bdb1-4467-8515-88099547779c" (UID: "17246662-bdb1-4467-8515-88099547779c"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 16 17:24:34.172642 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:24:34.172622 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d3884845-614f-482d-a893-e8bc1d72cda0-util" (OuterVolumeSpecName: "util") pod "d3884845-614f-482d-a893-e8bc1d72cda0" (UID: "d3884845-614f-482d-a893-e8bc1d72cda0"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 16 17:24:34.267145 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:24:34.267112 2578 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/17246662-bdb1-4467-8515-88099547779c-bundle\") on node \"ip-10-0-132-130.ec2.internal\" DevicePath \"\"" Jul 16 17:24:34.267145 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:24:34.267139 2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-rvr8x\" (UniqueName: \"kubernetes.io/projected/d3884845-614f-482d-a893-e8bc1d72cda0-kube-api-access-rvr8x\") on node \"ip-10-0-132-130.ec2.internal\" DevicePath \"\"" Jul 16 17:24:34.267145 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:24:34.267148 2578 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/d3884845-614f-482d-a893-e8bc1d72cda0-util\") on node \"ip-10-0-132-130.ec2.internal\" DevicePath \"\"" Jul 16 17:24:34.267357 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:24:34.267158 2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-shdz7\" (UniqueName: \"kubernetes.io/projected/17246662-bdb1-4467-8515-88099547779c-kube-api-access-shdz7\") on node \"ip-10-0-132-130.ec2.internal\" DevicePath \"\"" Jul 16 17:24:34.267357 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:24:34.267167 2578 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/17246662-bdb1-4467-8515-88099547779c-util\") on node \"ip-10-0-132-130.ec2.internal\" DevicePath \"\"" Jul 16 17:24:34.267357 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:24:34.267175 2578 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/d3884845-614f-482d-a893-e8bc1d72cda0-bundle\") on node \"ip-10-0-132-130.ec2.internal\" DevicePath \"\"" Jul 16 17:24:34.938357 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:24:34.938329 2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/299160ebe81f164e400242457c5a1580453e7d84fd8087b72a6eb317efjnrlp" Jul 16 17:24:34.938813 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:24:34.938354 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/299160ebe81f164e400242457c5a1580453e7d84fd8087b72a6eb317efjnrlp" event={"ID":"17246662-bdb1-4467-8515-88099547779c","Type":"ContainerDied","Data":"6833bdcc4b49b752d893559f7030c3005066fe384d0255ce6a0eb35ca5e2a7ca"} Jul 16 17:24:34.938813 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:24:34.938403 2578 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="6833bdcc4b49b752d893559f7030c3005066fe384d0255ce6a0eb35ca5e2a7ca" Jul 16 17:24:34.940451 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:24:34.940414 2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/f13fb8b568e19b60b32189bed3ab51e2b7cfe62efa490862cd57eead76hbc6v" Jul 16 17:24:34.940597 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:24:34.940447 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/f13fb8b568e19b60b32189bed3ab51e2b7cfe62efa490862cd57eead76hbc6v" event={"ID":"d3884845-614f-482d-a893-e8bc1d72cda0","Type":"ContainerDied","Data":"7bf67c76c240b5d896e46a9eedfc68b2b08e83944b711ceff54df1dbe74cc659"} Jul 16 17:24:34.940597 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:24:34.940492 2578 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="7bf67c76c240b5d896e46a9eedfc68b2b08e83944b711ceff54df1dbe74cc659" Jul 16 17:24:35.062562 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:24:35.062538 2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/0e9e3357d40614682d1ebdb402e8ba62e1e258fb473e6f54140aafff365pvsj" Jul 16 17:24:35.173815 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:24:35.173785 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-tmt5r\" (UniqueName: \"kubernetes.io/projected/9cafb707-c0e9-4bac-b722-c0912bc52fc5-kube-api-access-tmt5r\") pod \"9cafb707-c0e9-4bac-b722-c0912bc52fc5\" (UID: \"9cafb707-c0e9-4bac-b722-c0912bc52fc5\") " Jul 16 17:24:35.173940 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:24:35.173846 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/9cafb707-c0e9-4bac-b722-c0912bc52fc5-util\") pod \"9cafb707-c0e9-4bac-b722-c0912bc52fc5\" (UID: \"9cafb707-c0e9-4bac-b722-c0912bc52fc5\") " Jul 16 17:24:35.174001 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:24:35.173945 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/9cafb707-c0e9-4bac-b722-c0912bc52fc5-bundle\") pod \"9cafb707-c0e9-4bac-b722-c0912bc52fc5\" (UID: \"9cafb707-c0e9-4bac-b722-c0912bc52fc5\") " Jul 16 17:24:35.179715 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:24:35.174745 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9cafb707-c0e9-4bac-b722-c0912bc52fc5-bundle" (OuterVolumeSpecName: "bundle") pod "9cafb707-c0e9-4bac-b722-c0912bc52fc5" (UID: "9cafb707-c0e9-4bac-b722-c0912bc52fc5"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 16 17:24:35.182304 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:24:35.182259 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9cafb707-c0e9-4bac-b722-c0912bc52fc5-util" (OuterVolumeSpecName: "util") pod "9cafb707-c0e9-4bac-b722-c0912bc52fc5" (UID: "9cafb707-c0e9-4bac-b722-c0912bc52fc5"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 16 17:24:35.182836 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:24:35.182811 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9cafb707-c0e9-4bac-b722-c0912bc52fc5-kube-api-access-tmt5r" (OuterVolumeSpecName: "kube-api-access-tmt5r") pod "9cafb707-c0e9-4bac-b722-c0912bc52fc5" (UID: "9cafb707-c0e9-4bac-b722-c0912bc52fc5"). InnerVolumeSpecName "kube-api-access-tmt5r". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 16 17:24:35.275184 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:24:35.275109 2578 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/9cafb707-c0e9-4bac-b722-c0912bc52fc5-bundle\") on node \"ip-10-0-132-130.ec2.internal\" DevicePath \"\"" Jul 16 17:24:35.275184 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:24:35.275136 2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-tmt5r\" (UniqueName: \"kubernetes.io/projected/9cafb707-c0e9-4bac-b722-c0912bc52fc5-kube-api-access-tmt5r\") on node \"ip-10-0-132-130.ec2.internal\" DevicePath \"\"" Jul 16 17:24:35.275184 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:24:35.275151 2578 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/9cafb707-c0e9-4bac-b722-c0912bc52fc5-util\") on node \"ip-10-0-132-130.ec2.internal\" DevicePath \"\"" Jul 16 17:24:35.948018 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:24:35.947990 2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/0e9e3357d40614682d1ebdb402e8ba62e1e258fb473e6f54140aafff365pvsj" Jul 16 17:24:35.948429 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:24:35.947988 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/0e9e3357d40614682d1ebdb402e8ba62e1e258fb473e6f54140aafff365pvsj" event={"ID":"9cafb707-c0e9-4bac-b722-c0912bc52fc5","Type":"ContainerDied","Data":"b832682d6f24b686aec4d63935a42c7457e99fd2e357f90921484a73132b048b"} Jul 16 17:24:35.948429 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:24:35.948116 2578 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="b832682d6f24b686aec4d63935a42c7457e99fd2e357f90921484a73132b048b" Jul 16 17:24:35.949925 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:24:35.949898 2578 generic.go:356] "Generic (PLEG): container finished" podID="4c49c2c6-6348-49a1-a480-d63d7a562e69" containerID="c922fa3c151c7190e898f97700d0748fd3abb6c4fe07a81221270213505cc36e" exitCode=0 Jul 16 17:24:35.950046 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:24:35.949966 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/af2a5cb5935dca7d5808b44f590e0d3d1363f94abbf5c1d8a71e95fcabf8qpn" event={"ID":"4c49c2c6-6348-49a1-a480-d63d7a562e69","Type":"ContainerDied","Data":"c922fa3c151c7190e898f97700d0748fd3abb6c4fe07a81221270213505cc36e"} Jul 16 17:24:36.956676 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:24:36.956638 2578 generic.go:356] "Generic (PLEG): container finished" podID="4c49c2c6-6348-49a1-a480-d63d7a562e69" containerID="3881166bc0b429a51deef7b2a752e9248d11a35c94134fbff6e88c59ee11c2a9" exitCode=0 Jul 16 17:24:36.957078 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:24:36.956725 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/af2a5cb5935dca7d5808b44f590e0d3d1363f94abbf5c1d8a71e95fcabf8qpn" event={"ID":"4c49c2c6-6348-49a1-a480-d63d7a562e69","Type":"ContainerDied","Data":"3881166bc0b429a51deef7b2a752e9248d11a35c94134fbff6e88c59ee11c2a9"} Jul 16 17:24:38.093794 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:24:38.093772 2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/af2a5cb5935dca7d5808b44f590e0d3d1363f94abbf5c1d8a71e95fcabf8qpn" Jul 16 17:24:38.196748 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:24:38.196665 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-wfwp7\" (UniqueName: \"kubernetes.io/projected/4c49c2c6-6348-49a1-a480-d63d7a562e69-kube-api-access-wfwp7\") pod \"4c49c2c6-6348-49a1-a480-d63d7a562e69\" (UID: \"4c49c2c6-6348-49a1-a480-d63d7a562e69\") " Jul 16 17:24:38.196748 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:24:38.196720 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/4c49c2c6-6348-49a1-a480-d63d7a562e69-bundle\") pod \"4c49c2c6-6348-49a1-a480-d63d7a562e69\" (UID: \"4c49c2c6-6348-49a1-a480-d63d7a562e69\") " Jul 16 17:24:38.196965 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:24:38.196794 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/4c49c2c6-6348-49a1-a480-d63d7a562e69-util\") pod \"4c49c2c6-6348-49a1-a480-d63d7a562e69\" (UID: \"4c49c2c6-6348-49a1-a480-d63d7a562e69\") " Jul 16 17:24:38.197342 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:24:38.197309 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4c49c2c6-6348-49a1-a480-d63d7a562e69-bundle" (OuterVolumeSpecName: "bundle") pod "4c49c2c6-6348-49a1-a480-d63d7a562e69" (UID: "4c49c2c6-6348-49a1-a480-d63d7a562e69"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 16 17:24:38.198809 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:24:38.198776 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4c49c2c6-6348-49a1-a480-d63d7a562e69-kube-api-access-wfwp7" (OuterVolumeSpecName: "kube-api-access-wfwp7") pod "4c49c2c6-6348-49a1-a480-d63d7a562e69" (UID: "4c49c2c6-6348-49a1-a480-d63d7a562e69"). InnerVolumeSpecName "kube-api-access-wfwp7". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 16 17:24:38.201842 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:24:38.201816 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4c49c2c6-6348-49a1-a480-d63d7a562e69-util" (OuterVolumeSpecName: "util") pod "4c49c2c6-6348-49a1-a480-d63d7a562e69" (UID: "4c49c2c6-6348-49a1-a480-d63d7a562e69"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 16 17:24:38.298564 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:24:38.298355 2578 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/4c49c2c6-6348-49a1-a480-d63d7a562e69-util\") on node \"ip-10-0-132-130.ec2.internal\" DevicePath \"\"" Jul 16 17:24:38.298564 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:24:38.298385 2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-wfwp7\" (UniqueName: \"kubernetes.io/projected/4c49c2c6-6348-49a1-a480-d63d7a562e69-kube-api-access-wfwp7\") on node \"ip-10-0-132-130.ec2.internal\" DevicePath \"\"" Jul 16 17:24:38.298564 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:24:38.298400 2578 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/4c49c2c6-6348-49a1-a480-d63d7a562e69-bundle\") on node \"ip-10-0-132-130.ec2.internal\" DevicePath \"\"" Jul 16 17:24:38.968411 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:24:38.968379 2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/af2a5cb5935dca7d5808b44f590e0d3d1363f94abbf5c1d8a71e95fcabf8qpn" Jul 16 17:24:38.968411 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:24:38.968405 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/af2a5cb5935dca7d5808b44f590e0d3d1363f94abbf5c1d8a71e95fcabf8qpn" event={"ID":"4c49c2c6-6348-49a1-a480-d63d7a562e69","Type":"ContainerDied","Data":"8405537f2cdb52d914c14fe814202d9f4ab71547d02add24d428bacc2ff936f6"} Jul 16 17:24:38.968611 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:24:38.968446 2578 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="8405537f2cdb52d914c14fe814202d9f4ab71547d02add24d428bacc2ff936f6" Jul 16 17:24:45.834760 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:24:45.834730 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/dns-operator-controller-manager-65b49595d7-m7vkn"] Jul 16 17:24:45.835121 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:24:45.835082 2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="17246662-bdb1-4467-8515-88099547779c" containerName="extract" Jul 16 17:24:45.835121 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:24:45.835093 2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="17246662-bdb1-4467-8515-88099547779c" containerName="extract" Jul 16 17:24:45.835121 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:24:45.835105 2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d3884845-614f-482d-a893-e8bc1d72cda0" containerName="extract" Jul 16 17:24:45.835121 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:24:45.835110 2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="d3884845-614f-482d-a893-e8bc1d72cda0" containerName="extract" Jul 16 17:24:45.835121 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:24:45.835117 2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="17246662-bdb1-4467-8515-88099547779c" containerName="util" Jul 16 17:24:45.835121 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:24:45.835124 2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="17246662-bdb1-4467-8515-88099547779c" containerName="util" Jul 16 17:24:45.835304 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:24:45.835131 2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9cafb707-c0e9-4bac-b722-c0912bc52fc5" containerName="extract" Jul 16 17:24:45.835304 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:24:45.835136 2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="9cafb707-c0e9-4bac-b722-c0912bc52fc5" containerName="extract" Jul 16 17:24:45.835304 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:24:45.835144 2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4c49c2c6-6348-49a1-a480-d63d7a562e69" containerName="pull" Jul 16 17:24:45.835304 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:24:45.835149 2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="4c49c2c6-6348-49a1-a480-d63d7a562e69" containerName="pull" Jul 16 17:24:45.835304 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:24:45.835161 2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4c49c2c6-6348-49a1-a480-d63d7a562e69" containerName="util" Jul 16 17:24:45.835304 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:24:45.835167 2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="4c49c2c6-6348-49a1-a480-d63d7a562e69" containerName="util" Jul 16 17:24:45.835304 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:24:45.835174 2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d3884845-614f-482d-a893-e8bc1d72cda0" containerName="pull" Jul 16 17:24:45.835304 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:24:45.835179 2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="d3884845-614f-482d-a893-e8bc1d72cda0" containerName="pull" Jul 16 17:24:45.835304 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:24:45.835185 2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9cafb707-c0e9-4bac-b722-c0912bc52fc5" containerName="pull" Jul 16 17:24:45.835304 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:24:45.835190 2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="9cafb707-c0e9-4bac-b722-c0912bc52fc5" containerName="pull" Jul 16 17:24:45.835304 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:24:45.835196 2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d3884845-614f-482d-a893-e8bc1d72cda0" containerName="util" Jul 16 17:24:45.835304 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:24:45.835200 2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="d3884845-614f-482d-a893-e8bc1d72cda0" containerName="util" Jul 16 17:24:45.835304 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:24:45.835205 2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4c49c2c6-6348-49a1-a480-d63d7a562e69" containerName="extract" Jul 16 17:24:45.835304 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:24:45.835210 2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="4c49c2c6-6348-49a1-a480-d63d7a562e69" containerName="extract" Jul 16 17:24:45.835304 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:24:45.835222 2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="17246662-bdb1-4467-8515-88099547779c" containerName="pull" Jul 16 17:24:45.835304 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:24:45.835226 2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="17246662-bdb1-4467-8515-88099547779c" containerName="pull" Jul 16 17:24:45.835304 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:24:45.835235 2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9cafb707-c0e9-4bac-b722-c0912bc52fc5" containerName="util" Jul 16 17:24:45.835304 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:24:45.835240 2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="9cafb707-c0e9-4bac-b722-c0912bc52fc5" containerName="util" Jul 16 17:24:45.835304 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:24:45.835297 2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="4c49c2c6-6348-49a1-a480-d63d7a562e69" containerName="extract" Jul 16 17:24:45.835304 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:24:45.835304 2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="d3884845-614f-482d-a893-e8bc1d72cda0" containerName="extract" Jul 16 17:24:45.835304 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:24:45.835311 2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="9cafb707-c0e9-4bac-b722-c0912bc52fc5" containerName="extract" Jul 16 17:24:45.835908 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:24:45.835318 2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="17246662-bdb1-4467-8515-88099547779c" containerName="extract" Jul 16 17:24:45.838317 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:24:45.838302 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/dns-operator-controller-manager-65b49595d7-m7vkn" Jul 16 17:24:45.840747 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:24:45.840721 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\"" Jul 16 17:24:45.840897 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:24:45.840781 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"dns-operator-controller-env\"" Jul 16 17:24:45.841315 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:24:45.841298 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\"" Jul 16 17:24:45.841361 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:24:45.841298 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"dns-operator-controller-manager-dockercfg-5ttpl\"" Jul 16 17:24:45.856952 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:24:45.856928 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/dns-operator-controller-manager-65b49595d7-m7vkn"] Jul 16 17:24:45.972413 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:24:45.972384 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-np55n\" (UniqueName: \"kubernetes.io/projected/366102cc-6258-4e5a-b8f6-4ce0659b3601-kube-api-access-np55n\") pod \"dns-operator-controller-manager-65b49595d7-m7vkn\" (UID: \"366102cc-6258-4e5a-b8f6-4ce0659b3601\") " pod="kuadrant-system/dns-operator-controller-manager-65b49595d7-m7vkn" Jul 16 17:24:46.073024 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:24:46.072993 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-np55n\" (UniqueName: \"kubernetes.io/projected/366102cc-6258-4e5a-b8f6-4ce0659b3601-kube-api-access-np55n\") pod \"dns-operator-controller-manager-65b49595d7-m7vkn\" (UID: \"366102cc-6258-4e5a-b8f6-4ce0659b3601\") " pod="kuadrant-system/dns-operator-controller-manager-65b49595d7-m7vkn" Jul 16 17:24:46.096847 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:24:46.096783 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-np55n\" (UniqueName: \"kubernetes.io/projected/366102cc-6258-4e5a-b8f6-4ce0659b3601-kube-api-access-np55n\") pod \"dns-operator-controller-manager-65b49595d7-m7vkn\" (UID: \"366102cc-6258-4e5a-b8f6-4ce0659b3601\") " pod="kuadrant-system/dns-operator-controller-manager-65b49595d7-m7vkn" Jul 16 17:24:46.148314 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:24:46.148288 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/dns-operator-controller-manager-65b49595d7-m7vkn" Jul 16 17:24:46.296436 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:24:46.296408 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/dns-operator-controller-manager-65b49595d7-m7vkn"] Jul 16 17:24:46.298911 ip-10-0-132-130 kubenswrapper[2578]: W0716 17:24:46.298881 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod366102cc_6258_4e5a_b8f6_4ce0659b3601.slice/crio-ddd76f10d6fec5fbf43e8c66cb25646c6d9c933cacd8bb88195c0caeb4e0ac52 WatchSource:0}: Error finding container ddd76f10d6fec5fbf43e8c66cb25646c6d9c933cacd8bb88195c0caeb4e0ac52: Status 404 returned error can't find the container with id ddd76f10d6fec5fbf43e8c66cb25646c6d9c933cacd8bb88195c0caeb4e0ac52 Jul 16 17:24:47.012489 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:24:47.012451 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/dns-operator-controller-manager-65b49595d7-m7vkn" event={"ID":"366102cc-6258-4e5a-b8f6-4ce0659b3601","Type":"ContainerStarted","Data":"ddd76f10d6fec5fbf43e8c66cb25646c6d9c933cacd8bb88195c0caeb4e0ac52"} Jul 16 17:24:50.031610 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:24:50.031573 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/dns-operator-controller-manager-65b49595d7-m7vkn" event={"ID":"366102cc-6258-4e5a-b8f6-4ce0659b3601","Type":"ContainerStarted","Data":"a0e30b57ce5d2836cd56365dd1087100fa25eed5754554b63db23c5715614148"} Jul 16 17:24:50.032064 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:24:50.031633 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/dns-operator-controller-manager-65b49595d7-m7vkn" Jul 16 17:24:50.083336 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:24:50.083288 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/dns-operator-controller-manager-65b49595d7-m7vkn" podStartSLOduration=2.264491273 podStartE2EDuration="5.083273637s" totalImagesPullingTime="2.818782364s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 17:24:45 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 17:24:50.079429402 +0000 UTC m=+634.565273290" watchObservedRunningTime="2026-07-16 17:24:50.083273637 +0000 UTC m=+634.569117526" Jul 16 17:24:51.642620 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:24:51.642589 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-bb7ddb7d7-fp8pd"] Jul 16 17:24:51.646346 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:24:51.646330 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-bb7ddb7d7-fp8pd" Jul 16 17:24:51.651922 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:24:51.651902 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-controller-manager-dockercfg-29wkg\"" Jul 16 17:24:51.674184 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:24:51.674153 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-bb7ddb7d7-fp8pd"] Jul 16 17:24:51.718712 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:24:51.718658 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/4c5a2957-e580-423a-97a9-5ffb61407ab4-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-bb7ddb7d7-fp8pd\" (UID: \"4c5a2957-e580-423a-97a9-5ffb61407ab4\") " pod="kuadrant-system/kuadrant-operator-controller-manager-bb7ddb7d7-fp8pd" Jul 16 17:24:51.718875 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:24:51.718763 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zj85z\" (UniqueName: \"kubernetes.io/projected/4c5a2957-e580-423a-97a9-5ffb61407ab4-kube-api-access-zj85z\") pod \"kuadrant-operator-controller-manager-bb7ddb7d7-fp8pd\" (UID: \"4c5a2957-e580-423a-97a9-5ffb61407ab4\") " pod="kuadrant-system/kuadrant-operator-controller-manager-bb7ddb7d7-fp8pd" Jul 16 17:24:51.820022 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:24:51.819985 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zj85z\" (UniqueName: \"kubernetes.io/projected/4c5a2957-e580-423a-97a9-5ffb61407ab4-kube-api-access-zj85z\") pod \"kuadrant-operator-controller-manager-bb7ddb7d7-fp8pd\" (UID: \"4c5a2957-e580-423a-97a9-5ffb61407ab4\") " pod="kuadrant-system/kuadrant-operator-controller-manager-bb7ddb7d7-fp8pd" Jul 16 17:24:51.820206 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:24:51.820135 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/4c5a2957-e580-423a-97a9-5ffb61407ab4-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-bb7ddb7d7-fp8pd\" (UID: \"4c5a2957-e580-423a-97a9-5ffb61407ab4\") " pod="kuadrant-system/kuadrant-operator-controller-manager-bb7ddb7d7-fp8pd" Jul 16 17:24:51.820507 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:24:51.820486 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/4c5a2957-e580-423a-97a9-5ffb61407ab4-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-bb7ddb7d7-fp8pd\" (UID: \"4c5a2957-e580-423a-97a9-5ffb61407ab4\") " pod="kuadrant-system/kuadrant-operator-controller-manager-bb7ddb7d7-fp8pd" Jul 16 17:24:51.846043 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:24:51.846009 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zj85z\" (UniqueName: \"kubernetes.io/projected/4c5a2957-e580-423a-97a9-5ffb61407ab4-kube-api-access-zj85z\") pod \"kuadrant-operator-controller-manager-bb7ddb7d7-fp8pd\" (UID: \"4c5a2957-e580-423a-97a9-5ffb61407ab4\") " pod="kuadrant-system/kuadrant-operator-controller-manager-bb7ddb7d7-fp8pd" Jul 16 17:24:51.957031 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:24:51.956949 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-bb7ddb7d7-fp8pd" Jul 16 17:24:52.159335 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:24:52.159311 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-bb7ddb7d7-fp8pd"] Jul 16 17:24:52.161123 ip-10-0-132-130 kubenswrapper[2578]: W0716 17:24:52.161090 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4c5a2957_e580_423a_97a9_5ffb61407ab4.slice/crio-4d4047ad6381fef62b90be41e338a388f42deb2eae01fd55b1963a6e2b43d3bc WatchSource:0}: Error finding container 4d4047ad6381fef62b90be41e338a388f42deb2eae01fd55b1963a6e2b43d3bc: Status 404 returned error can't find the container with id 4d4047ad6381fef62b90be41e338a388f42deb2eae01fd55b1963a6e2b43d3bc Jul 16 17:24:53.049999 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:24:53.049967 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-bb7ddb7d7-fp8pd" event={"ID":"4c5a2957-e580-423a-97a9-5ffb61407ab4","Type":"ContainerStarted","Data":"4d4047ad6381fef62b90be41e338a388f42deb2eae01fd55b1963a6e2b43d3bc"} Jul 16 17:24:55.501068 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:24:55.501030 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-operator-6d85f6564-c4d5z"] Jul 16 17:24:55.505197 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:24:55.505173 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-operator-6d85f6564-c4d5z" Jul 16 17:24:55.511003 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:24:55.510977 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-operator-dockercfg-8bxx7\"" Jul 16 17:24:55.549736 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:24:55.549686 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-operator-6d85f6564-c4d5z"] Jul 16 17:24:55.552073 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:24:55.552046 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6cdm4\" (UniqueName: \"kubernetes.io/projected/5798b263-2def-43aa-bc40-bd06b5948f2c-kube-api-access-6cdm4\") pod \"authorino-operator-6d85f6564-c4d5z\" (UID: \"5798b263-2def-43aa-bc40-bd06b5948f2c\") " pod="kuadrant-system/authorino-operator-6d85f6564-c4d5z" Jul 16 17:24:55.652889 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:24:55.652852 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6cdm4\" (UniqueName: \"kubernetes.io/projected/5798b263-2def-43aa-bc40-bd06b5948f2c-kube-api-access-6cdm4\") pod \"authorino-operator-6d85f6564-c4d5z\" (UID: \"5798b263-2def-43aa-bc40-bd06b5948f2c\") " pod="kuadrant-system/authorino-operator-6d85f6564-c4d5z" Jul 16 17:24:55.693913 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:24:55.693880 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6cdm4\" (UniqueName: \"kubernetes.io/projected/5798b263-2def-43aa-bc40-bd06b5948f2c-kube-api-access-6cdm4\") pod \"authorino-operator-6d85f6564-c4d5z\" (UID: \"5798b263-2def-43aa-bc40-bd06b5948f2c\") " pod="kuadrant-system/authorino-operator-6d85f6564-c4d5z" Jul 16 17:24:55.817288 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:24:55.817240 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-operator-6d85f6564-c4d5z" Jul 16 17:24:56.473816 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:24:56.473783 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-operator-6d85f6564-c4d5z"] Jul 16 17:24:56.475865 ip-10-0-132-130 kubenswrapper[2578]: W0716 17:24:56.475837 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5798b263_2def_43aa_bc40_bd06b5948f2c.slice/crio-574f90d33a289f32933e8e712542ab1f26c9c0b686151bf659381fa4ae270384 WatchSource:0}: Error finding container 574f90d33a289f32933e8e712542ab1f26c9c0b686151bf659381fa4ae270384: Status 404 returned error can't find the container with id 574f90d33a289f32933e8e712542ab1f26c9c0b686151bf659381fa4ae270384 Jul 16 17:24:57.080058 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:24:57.080018 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-bb7ddb7d7-fp8pd" event={"ID":"4c5a2957-e580-423a-97a9-5ffb61407ab4","Type":"ContainerStarted","Data":"a4792c2c063740309647f04bbe24b2e190ff9baf6c63521c824ea40cfb8b8f26"} Jul 16 17:24:57.080483 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:24:57.080149 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-bb7ddb7d7-fp8pd" Jul 16 17:24:57.081479 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:24:57.081461 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-operator-6d85f6564-c4d5z" event={"ID":"5798b263-2def-43aa-bc40-bd06b5948f2c","Type":"ContainerStarted","Data":"574f90d33a289f32933e8e712542ab1f26c9c0b686151bf659381fa4ae270384"} Jul 16 17:25:01.039366 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:25:01.039336 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/dns-operator-controller-manager-65b49595d7-m7vkn" Jul 16 17:25:01.063556 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:25:01.063510 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-bb7ddb7d7-fp8pd" podStartSLOduration=5.824469936 podStartE2EDuration="10.063492303s" totalImagesPullingTime="4.239022367s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 17:24:51 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 17:24:57.131952393 +0000 UTC m=+641.617796281" watchObservedRunningTime="2026-07-16 17:25:01.063492303 +0000 UTC m=+645.549336191" Jul 16 17:25:07.137487 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:25:07.137449 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-operator-6d85f6564-c4d5z" event={"ID":"5798b263-2def-43aa-bc40-bd06b5948f2c","Type":"ContainerStarted","Data":"1c816492dee203497c29e31d7280a457013c0c7e61b392040b81681d858620ce"} Jul 16 17:25:07.137901 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:25:07.137517 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/authorino-operator-6d85f6564-c4d5z" Jul 16 17:25:07.161864 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:25:07.161803 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-operator-6d85f6564-c4d5z" podStartSLOduration=2.380687173 podStartE2EDuration="12.161783883s" totalImagesPullingTime="9.78109671s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 17:24:55 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 17:25:07.161152605 +0000 UTC m=+651.646996517" watchObservedRunningTime="2026-07-16 17:25:07.161783883 +0000 UTC m=+651.647627775" Jul 16 17:25:08.088556 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:25:08.088523 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-bb7ddb7d7-fp8pd" Jul 16 17:25:18.145757 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:25:18.145727 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/authorino-operator-6d85f6564-c4d5z" Jul 16 17:25:40.035139 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:25:40.035104 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-limitador-ff68ff5cb-g8j52"] Jul 16 17:25:40.045897 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:25:40.045858 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-ff68ff5cb-g8j52" Jul 16 17:25:40.048465 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:25:40.048441 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"default-dockercfg-cppvr\"" Jul 16 17:25:40.048590 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:25:40.048440 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"limitador-limits-config-limitador\"" Jul 16 17:25:40.048958 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:25:40.048929 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-ff68ff5cb-g8j52"] Jul 16 17:25:40.095950 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:25:40.095920 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-ff68ff5cb-g8j52"] Jul 16 17:25:40.228033 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:25:40.227999 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qsv95\" (UniqueName: \"kubernetes.io/projected/0f0858ca-d897-4446-809d-0fc710f617b4-kube-api-access-qsv95\") pod \"limitador-limitador-ff68ff5cb-g8j52\" (UID: \"0f0858ca-d897-4446-809d-0fc710f617b4\") " pod="kuadrant-system/limitador-limitador-ff68ff5cb-g8j52" Jul 16 17:25:40.228197 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:25:40.228048 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/0f0858ca-d897-4446-809d-0fc710f617b4-config-file\") pod \"limitador-limitador-ff68ff5cb-g8j52\" (UID: \"0f0858ca-d897-4446-809d-0fc710f617b4\") " pod="kuadrant-system/limitador-limitador-ff68ff5cb-g8j52" Jul 16 17:25:40.329384 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:25:40.329353 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qsv95\" (UniqueName: \"kubernetes.io/projected/0f0858ca-d897-4446-809d-0fc710f617b4-kube-api-access-qsv95\") pod \"limitador-limitador-ff68ff5cb-g8j52\" (UID: \"0f0858ca-d897-4446-809d-0fc710f617b4\") " pod="kuadrant-system/limitador-limitador-ff68ff5cb-g8j52" Jul 16 17:25:40.329547 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:25:40.329411 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/0f0858ca-d897-4446-809d-0fc710f617b4-config-file\") pod \"limitador-limitador-ff68ff5cb-g8j52\" (UID: \"0f0858ca-d897-4446-809d-0fc710f617b4\") " pod="kuadrant-system/limitador-limitador-ff68ff5cb-g8j52" Jul 16 17:25:40.330224 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:25:40.330204 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/0f0858ca-d897-4446-809d-0fc710f617b4-config-file\") pod \"limitador-limitador-ff68ff5cb-g8j52\" (UID: \"0f0858ca-d897-4446-809d-0fc710f617b4\") " pod="kuadrant-system/limitador-limitador-ff68ff5cb-g8j52" Jul 16 17:25:40.342320 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:25:40.342295 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qsv95\" (UniqueName: \"kubernetes.io/projected/0f0858ca-d897-4446-809d-0fc710f617b4-kube-api-access-qsv95\") pod \"limitador-limitador-ff68ff5cb-g8j52\" (UID: \"0f0858ca-d897-4446-809d-0fc710f617b4\") " pod="kuadrant-system/limitador-limitador-ff68ff5cb-g8j52" Jul 16 17:25:40.358144 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:25:40.358106 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-ff68ff5cb-g8j52" Jul 16 17:25:40.512057 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:25:40.512033 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-ff68ff5cb-g8j52"] Jul 16 17:25:40.514239 ip-10-0-132-130 kubenswrapper[2578]: W0716 17:25:40.514216 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0f0858ca_d897_4446_809d_0fc710f617b4.slice/crio-54951485ea8f18415a257d2d7cf75bacd5bcefb64944de3ff53cc345c98d27dd WatchSource:0}: Error finding container 54951485ea8f18415a257d2d7cf75bacd5bcefb64944de3ff53cc345c98d27dd: Status 404 returned error can't find the container with id 54951485ea8f18415a257d2d7cf75bacd5bcefb64944de3ff53cc345c98d27dd Jul 16 17:25:41.324781 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:25:41.324745 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-ff68ff5cb-g8j52" event={"ID":"0f0858ca-d897-4446-809d-0fc710f617b4","Type":"ContainerStarted","Data":"54951485ea8f18415a257d2d7cf75bacd5bcefb64944de3ff53cc345c98d27dd"} Jul 16 17:25:45.348932 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:25:45.348896 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-ff68ff5cb-g8j52" event={"ID":"0f0858ca-d897-4446-809d-0fc710f617b4","Type":"ContainerStarted","Data":"305878ae6fb448b6a7614fbd3519987788555a99108a8c0c11c8072f2da4db30"} Jul 16 17:25:45.349599 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:25:45.349038 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-limitador-ff68ff5cb-g8j52" Jul 16 17:25:45.382660 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:25:45.382608 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-limitador-ff68ff5cb-g8j52" podStartSLOduration=1.597512882 podStartE2EDuration="5.382595641s" totalImagesPullingTime="3.785082759s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 17:25:40 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 17:25:45.382120446 +0000 UTC m=+689.867964335" watchObservedRunningTime="2026-07-16 17:25:45.382595641 +0000 UTC m=+689.868439529" Jul 16 17:25:56.355407 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:25:56.355377 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-limitador-ff68ff5cb-g8j52" Jul 16 17:25:56.359344 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:25:56.359321 2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-limitador-ff68ff5cb-g8j52"] Jul 16 17:25:56.414337 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:25:56.414303 2578 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/limitador-limitador-ff68ff5cb-g8j52" podUID="0f0858ca-d897-4446-809d-0fc710f617b4" containerName="limitador" containerID="cri-o://305878ae6fb448b6a7614fbd3519987788555a99108a8c0c11c8072f2da4db30" gracePeriod=30 Jul 16 17:25:56.971979 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:25:56.971954 2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-ff68ff5cb-g8j52" Jul 16 17:25:57.069225 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:25:57.069203 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/0f0858ca-d897-4446-809d-0fc710f617b4-config-file\") pod \"0f0858ca-d897-4446-809d-0fc710f617b4\" (UID: \"0f0858ca-d897-4446-809d-0fc710f617b4\") " Jul 16 17:25:57.069360 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:25:57.069236 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-qsv95\" (UniqueName: \"kubernetes.io/projected/0f0858ca-d897-4446-809d-0fc710f617b4-kube-api-access-qsv95\") pod \"0f0858ca-d897-4446-809d-0fc710f617b4\" (UID: \"0f0858ca-d897-4446-809d-0fc710f617b4\") " Jul 16 17:25:57.069534 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:25:57.069512 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0f0858ca-d897-4446-809d-0fc710f617b4-config-file" (OuterVolumeSpecName: "config-file") pod "0f0858ca-d897-4446-809d-0fc710f617b4" (UID: "0f0858ca-d897-4446-809d-0fc710f617b4"). InnerVolumeSpecName "config-file". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 16 17:25:57.071375 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:25:57.071345 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0f0858ca-d897-4446-809d-0fc710f617b4-kube-api-access-qsv95" (OuterVolumeSpecName: "kube-api-access-qsv95") pod "0f0858ca-d897-4446-809d-0fc710f617b4" (UID: "0f0858ca-d897-4446-809d-0fc710f617b4"). InnerVolumeSpecName "kube-api-access-qsv95". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 16 17:25:57.170468 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:25:57.170445 2578 reconciler_common.go:299] "Volume detached for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/0f0858ca-d897-4446-809d-0fc710f617b4-config-file\") on node \"ip-10-0-132-130.ec2.internal\" DevicePath \"\"" Jul 16 17:25:57.170468 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:25:57.170467 2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-qsv95\" (UniqueName: \"kubernetes.io/projected/0f0858ca-d897-4446-809d-0fc710f617b4-kube-api-access-qsv95\") on node \"ip-10-0-132-130.ec2.internal\" DevicePath \"\"" Jul 16 17:25:57.421099 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:25:57.421018 2578 generic.go:356] "Generic (PLEG): container finished" podID="0f0858ca-d897-4446-809d-0fc710f617b4" containerID="305878ae6fb448b6a7614fbd3519987788555a99108a8c0c11c8072f2da4db30" exitCode=0 Jul 16 17:25:57.421099 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:25:57.421066 2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-ff68ff5cb-g8j52" Jul 16 17:25:57.421099 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:25:57.421071 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-ff68ff5cb-g8j52" event={"ID":"0f0858ca-d897-4446-809d-0fc710f617b4","Type":"ContainerDied","Data":"305878ae6fb448b6a7614fbd3519987788555a99108a8c0c11c8072f2da4db30"} Jul 16 17:25:57.421528 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:25:57.421118 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-ff68ff5cb-g8j52" event={"ID":"0f0858ca-d897-4446-809d-0fc710f617b4","Type":"ContainerDied","Data":"54951485ea8f18415a257d2d7cf75bacd5bcefb64944de3ff53cc345c98d27dd"} Jul 16 17:25:57.421528 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:25:57.421142 2578 scope.go:117] "RemoveContainer" containerID="305878ae6fb448b6a7614fbd3519987788555a99108a8c0c11c8072f2da4db30" Jul 16 17:25:57.431717 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:25:57.431616 2578 scope.go:117] "RemoveContainer" containerID="305878ae6fb448b6a7614fbd3519987788555a99108a8c0c11c8072f2da4db30" Jul 16 17:25:57.433180 ip-10-0-132-130 kubenswrapper[2578]: E0716 17:25:57.433153 2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"305878ae6fb448b6a7614fbd3519987788555a99108a8c0c11c8072f2da4db30\": container with ID starting with 305878ae6fb448b6a7614fbd3519987788555a99108a8c0c11c8072f2da4db30 not found: ID does not exist" containerID="305878ae6fb448b6a7614fbd3519987788555a99108a8c0c11c8072f2da4db30" Jul 16 17:25:57.433293 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:25:57.433184 2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"305878ae6fb448b6a7614fbd3519987788555a99108a8c0c11c8072f2da4db30"} err="failed to get container status \"305878ae6fb448b6a7614fbd3519987788555a99108a8c0c11c8072f2da4db30\": rpc error: code = NotFound desc = could not find container \"305878ae6fb448b6a7614fbd3519987788555a99108a8c0c11c8072f2da4db30\": container with ID starting with 305878ae6fb448b6a7614fbd3519987788555a99108a8c0c11c8072f2da4db30 not found: ID does not exist" Jul 16 17:25:57.457397 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:25:57.457376 2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-limitador-ff68ff5cb-g8j52"] Jul 16 17:25:57.470371 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:25:57.470349 2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/limitador-limitador-ff68ff5cb-g8j52"] Jul 16 17:25:58.215856 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:25:58.215824 2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="0f0858ca-d897-4446-809d-0fc710f617b4" path="/var/lib/kubelet/pods/0f0858ca-d897-4446-809d-0fc710f617b4/volumes" Jul 16 17:26:13.255987 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:26:13.255943 2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve/kserve-controller-manager-5c5755fdcb-8g9zs"] Jul 16 17:26:13.256435 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:26:13.256195 2578 kuberuntime_container.go:873] "Killing container with a grace period" pod="kserve/kserve-controller-manager-5c5755fdcb-8g9zs" podUID="0d017cd5-6194-4750-81b9-ec218e481efa" containerName="manager" containerID="cri-o://26308741dab10555578d84c849c99f62956c4240be7dd0e6c1789cefd67af819" gracePeriod=10 Jul 16 17:26:13.287900 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:26:13.287876 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve/kserve-controller-manager-5c5755fdcb-g2x5r"] Jul 16 17:26:13.288243 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:26:13.288231 2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0f0858ca-d897-4446-809d-0fc710f617b4" containerName="limitador" Jul 16 17:26:13.288285 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:26:13.288245 2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="0f0858ca-d897-4446-809d-0fc710f617b4" containerName="limitador" Jul 16 17:26:13.288319 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:26:13.288312 2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="0f0858ca-d897-4446-809d-0fc710f617b4" containerName="limitador" Jul 16 17:26:13.291402 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:26:13.291386 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve/kserve-controller-manager-5c5755fdcb-g2x5r" Jul 16 17:26:13.301503 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:26:13.301481 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve/kserve-controller-manager-5c5755fdcb-g2x5r"] Jul 16 17:26:13.401208 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:26:13.401177 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cv9c2\" (UniqueName: \"kubernetes.io/projected/1e3dd3b6-f573-40d6-ae3f-6ce5ca836316-kube-api-access-cv9c2\") pod \"kserve-controller-manager-5c5755fdcb-g2x5r\" (UID: \"1e3dd3b6-f573-40d6-ae3f-6ce5ca836316\") " pod="kserve/kserve-controller-manager-5c5755fdcb-g2x5r" Jul 16 17:26:13.401352 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:26:13.401230 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/1e3dd3b6-f573-40d6-ae3f-6ce5ca836316-cert\") pod \"kserve-controller-manager-5c5755fdcb-g2x5r\" (UID: \"1e3dd3b6-f573-40d6-ae3f-6ce5ca836316\") " pod="kserve/kserve-controller-manager-5c5755fdcb-g2x5r" Jul 16 17:26:13.502492 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:26:13.502463 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cv9c2\" (UniqueName: \"kubernetes.io/projected/1e3dd3b6-f573-40d6-ae3f-6ce5ca836316-kube-api-access-cv9c2\") pod \"kserve-controller-manager-5c5755fdcb-g2x5r\" (UID: \"1e3dd3b6-f573-40d6-ae3f-6ce5ca836316\") " pod="kserve/kserve-controller-manager-5c5755fdcb-g2x5r" Jul 16 17:26:13.502640 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:26:13.502539 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/1e3dd3b6-f573-40d6-ae3f-6ce5ca836316-cert\") pod \"kserve-controller-manager-5c5755fdcb-g2x5r\" (UID: \"1e3dd3b6-f573-40d6-ae3f-6ce5ca836316\") " pod="kserve/kserve-controller-manager-5c5755fdcb-g2x5r" Jul 16 17:26:13.505082 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:26:13.505056 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/1e3dd3b6-f573-40d6-ae3f-6ce5ca836316-cert\") pod \"kserve-controller-manager-5c5755fdcb-g2x5r\" (UID: \"1e3dd3b6-f573-40d6-ae3f-6ce5ca836316\") " pod="kserve/kserve-controller-manager-5c5755fdcb-g2x5r" Jul 16 17:26:13.513020 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:26:13.512940 2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve/kserve-controller-manager-5c5755fdcb-8g9zs" Jul 16 17:26:13.515878 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:26:13.515849 2578 generic.go:356] "Generic (PLEG): container finished" podID="0d017cd5-6194-4750-81b9-ec218e481efa" containerID="26308741dab10555578d84c849c99f62956c4240be7dd0e6c1789cefd67af819" exitCode=0 Jul 16 17:26:13.515985 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:26:13.515932 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve/kserve-controller-manager-5c5755fdcb-8g9zs" event={"ID":"0d017cd5-6194-4750-81b9-ec218e481efa","Type":"ContainerDied","Data":"26308741dab10555578d84c849c99f62956c4240be7dd0e6c1789cefd67af819"} Jul 16 17:26:13.516041 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:26:13.516021 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve/kserve-controller-manager-5c5755fdcb-8g9zs" event={"ID":"0d017cd5-6194-4750-81b9-ec218e481efa","Type":"ContainerDied","Data":"68c57608866f6bacfc1b1fa43e71bc164134842ac2e5c19a398f5a74e15e1edb"} Jul 16 17:26:13.516089 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:26:13.516054 2578 scope.go:117] "RemoveContainer" containerID="26308741dab10555578d84c849c99f62956c4240be7dd0e6c1789cefd67af819" Jul 16 17:26:13.517176 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:26:13.517156 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-cv9c2\" (UniqueName: \"kubernetes.io/projected/1e3dd3b6-f573-40d6-ae3f-6ce5ca836316-kube-api-access-cv9c2\") pod \"kserve-controller-manager-5c5755fdcb-g2x5r\" (UID: \"1e3dd3b6-f573-40d6-ae3f-6ce5ca836316\") " pod="kserve/kserve-controller-manager-5c5755fdcb-g2x5r" Jul 16 17:26:13.528566 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:26:13.528549 2578 scope.go:117] "RemoveContainer" containerID="26308741dab10555578d84c849c99f62956c4240be7dd0e6c1789cefd67af819" Jul 16 17:26:13.528878 ip-10-0-132-130 kubenswrapper[2578]: E0716 17:26:13.528863 2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"26308741dab10555578d84c849c99f62956c4240be7dd0e6c1789cefd67af819\": container with ID starting with 26308741dab10555578d84c849c99f62956c4240be7dd0e6c1789cefd67af819 not found: ID does not exist" containerID="26308741dab10555578d84c849c99f62956c4240be7dd0e6c1789cefd67af819" Jul 16 17:26:13.528931 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:26:13.528893 2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"26308741dab10555578d84c849c99f62956c4240be7dd0e6c1789cefd67af819"} err="failed to get container status \"26308741dab10555578d84c849c99f62956c4240be7dd0e6c1789cefd67af819\": rpc error: code = NotFound desc = could not find container \"26308741dab10555578d84c849c99f62956c4240be7dd0e6c1789cefd67af819\": container with ID starting with 26308741dab10555578d84c849c99f62956c4240be7dd0e6c1789cefd67af819 not found: ID does not exist" Jul 16 17:26:13.603564 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:26:13.603530 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-g7t59\" (UniqueName: \"kubernetes.io/projected/0d017cd5-6194-4750-81b9-ec218e481efa-kube-api-access-g7t59\") pod \"0d017cd5-6194-4750-81b9-ec218e481efa\" (UID: \"0d017cd5-6194-4750-81b9-ec218e481efa\") " Jul 16 17:26:13.603774 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:26:13.603593 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/0d017cd5-6194-4750-81b9-ec218e481efa-cert\") pod \"0d017cd5-6194-4750-81b9-ec218e481efa\" (UID: \"0d017cd5-6194-4750-81b9-ec218e481efa\") " Jul 16 17:26:13.605516 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:26:13.605490 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/0d017cd5-6194-4750-81b9-ec218e481efa-cert" (OuterVolumeSpecName: "cert") pod "0d017cd5-6194-4750-81b9-ec218e481efa" (UID: "0d017cd5-6194-4750-81b9-ec218e481efa"). InnerVolumeSpecName "cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 16 17:26:13.605619 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:26:13.605558 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0d017cd5-6194-4750-81b9-ec218e481efa-kube-api-access-g7t59" (OuterVolumeSpecName: "kube-api-access-g7t59") pod "0d017cd5-6194-4750-81b9-ec218e481efa" (UID: "0d017cd5-6194-4750-81b9-ec218e481efa"). InnerVolumeSpecName "kube-api-access-g7t59". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 16 17:26:13.661320 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:26:13.661292 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve/kserve-controller-manager-5c5755fdcb-g2x5r" Jul 16 17:26:13.704896 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:26:13.704867 2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-g7t59\" (UniqueName: \"kubernetes.io/projected/0d017cd5-6194-4750-81b9-ec218e481efa-kube-api-access-g7t59\") on node \"ip-10-0-132-130.ec2.internal\" DevicePath \"\"" Jul 16 17:26:13.704896 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:26:13.704893 2578 reconciler_common.go:299] "Volume detached for volume \"cert\" (UniqueName: \"kubernetes.io/secret/0d017cd5-6194-4750-81b9-ec218e481efa-cert\") on node \"ip-10-0-132-130.ec2.internal\" DevicePath \"\"" Jul 16 17:26:13.796352 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:26:13.796326 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve/kserve-controller-manager-5c5755fdcb-g2x5r"] Jul 16 17:26:13.798509 ip-10-0-132-130 kubenswrapper[2578]: W0716 17:26:13.798478 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1e3dd3b6_f573_40d6_ae3f_6ce5ca836316.slice/crio-580cd1a8bfdde8081b5f50172f1931a880338728a8919e4313204b0d78f3e2d6 WatchSource:0}: Error finding container 580cd1a8bfdde8081b5f50172f1931a880338728a8919e4313204b0d78f3e2d6: Status 404 returned error can't find the container with id 580cd1a8bfdde8081b5f50172f1931a880338728a8919e4313204b0d78f3e2d6 Jul 16 17:26:13.799858 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:26:13.799839 2578 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 16 17:26:14.522525 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:26:14.522442 2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve/kserve-controller-manager-5c5755fdcb-8g9zs" Jul 16 17:26:14.523804 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:26:14.523776 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve/kserve-controller-manager-5c5755fdcb-g2x5r" event={"ID":"1e3dd3b6-f573-40d6-ae3f-6ce5ca836316","Type":"ContainerStarted","Data":"580cd1a8bfdde8081b5f50172f1931a880338728a8919e4313204b0d78f3e2d6"} Jul 16 17:26:14.551051 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:26:14.551029 2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve/kserve-controller-manager-5c5755fdcb-8g9zs"] Jul 16 17:26:14.563352 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:26:14.563324 2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kserve/kserve-controller-manager-5c5755fdcb-8g9zs"] Jul 16 17:26:15.530947 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:26:15.530915 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve/kserve-controller-manager-5c5755fdcb-g2x5r" event={"ID":"1e3dd3b6-f573-40d6-ae3f-6ce5ca836316","Type":"ContainerStarted","Data":"967ced60a253ca75b4141fb341edd6406df6ce1ee7a75589d42146c73562610d"} Jul 16 17:26:15.531405 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:26:15.531025 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve/kserve-controller-manager-5c5755fdcb-g2x5r" Jul 16 17:26:15.555112 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:26:15.555063 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve/kserve-controller-manager-5c5755fdcb-g2x5r" podStartSLOduration=1.775658202 podStartE2EDuration="2.55504637s" totalImagesPullingTime="779.388168ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 17:26:13 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 17:26:15.553062825 +0000 UTC m=+720.038906727" watchObservedRunningTime="2026-07-16 17:26:15.55504637 +0000 UTC m=+720.040890259" Jul 16 17:26:16.216672 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:26:16.216648 2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="0d017cd5-6194-4750-81b9-ec218e481efa" path="/var/lib/kubelet/pods/0d017cd5-6194-4750-81b9-ec218e481efa/volumes" Jul 16 17:26:46.542277 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:26:46.542245 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve/kserve-controller-manager-5c5755fdcb-g2x5r" Jul 16 17:26:47.942321 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:26:47.942291 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve/model-serving-api-fd65d7d6-h2w48"] Jul 16 17:26:47.942881 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:26:47.942863 2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0d017cd5-6194-4750-81b9-ec218e481efa" containerName="manager" Jul 16 17:26:47.942881 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:26:47.942882 2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="0d017cd5-6194-4750-81b9-ec218e481efa" containerName="manager" Jul 16 17:26:47.943080 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:26:47.942950 2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="0d017cd5-6194-4750-81b9-ec218e481efa" containerName="manager" Jul 16 17:26:47.945916 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:26:47.945899 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve/odh-model-controller-84b54b7f97-lf6xh"] Jul 16 17:26:47.946032 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:26:47.946013 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve/model-serving-api-fd65d7d6-h2w48" Jul 16 17:26:47.948156 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:26:47.948133 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve\"/\"model-serving-api-dockercfg-ztc85\"" Jul 16 17:26:47.948287 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:26:47.948171 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve\"/\"model-serving-api-tls\"" Jul 16 17:26:47.949404 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:26:47.949391 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve/odh-model-controller-84b54b7f97-lf6xh" Jul 16 17:26:47.951991 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:26:47.951974 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve\"/\"odh-model-controller-webhook-cert\"" Jul 16 17:26:47.952066 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:26:47.952052 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve\"/\"odh-model-controller-dockercfg-d6fqz\"" Jul 16 17:26:47.970972 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:26:47.970950 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve/odh-model-controller-84b54b7f97-lf6xh"] Jul 16 17:26:47.974365 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:26:47.974341 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve/model-serving-api-fd65d7d6-h2w48"] Jul 16 17:26:47.999475 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:26:47.999452 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/123068b3-bb2c-4819-a517-0362388d5fde-tls-certs\") pod \"model-serving-api-fd65d7d6-h2w48\" (UID: \"123068b3-bb2c-4819-a517-0362388d5fde\") " pod="kserve/model-serving-api-fd65d7d6-h2w48" Jul 16 17:26:47.999604 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:26:47.999487 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2jhfj\" (UniqueName: \"kubernetes.io/projected/123068b3-bb2c-4819-a517-0362388d5fde-kube-api-access-2jhfj\") pod \"model-serving-api-fd65d7d6-h2w48\" (UID: \"123068b3-bb2c-4819-a517-0362388d5fde\") " pod="kserve/model-serving-api-fd65d7d6-h2w48" Jul 16 17:26:47.999604 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:26:47.999510 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-79v7x\" (UniqueName: \"kubernetes.io/projected/bf4947fd-3845-4bb8-a7a0-f03de833d182-kube-api-access-79v7x\") pod \"odh-model-controller-84b54b7f97-lf6xh\" (UID: \"bf4947fd-3845-4bb8-a7a0-f03de833d182\") " pod="kserve/odh-model-controller-84b54b7f97-lf6xh" Jul 16 17:26:47.999681 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:26:47.999602 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/bf4947fd-3845-4bb8-a7a0-f03de833d182-cert\") pod \"odh-model-controller-84b54b7f97-lf6xh\" (UID: \"bf4947fd-3845-4bb8-a7a0-f03de833d182\") " pod="kserve/odh-model-controller-84b54b7f97-lf6xh" Jul 16 17:26:48.100028 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:26:48.099999 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/bf4947fd-3845-4bb8-a7a0-f03de833d182-cert\") pod \"odh-model-controller-84b54b7f97-lf6xh\" (UID: \"bf4947fd-3845-4bb8-a7a0-f03de833d182\") " pod="kserve/odh-model-controller-84b54b7f97-lf6xh" Jul 16 17:26:48.100178 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:26:48.100057 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/123068b3-bb2c-4819-a517-0362388d5fde-tls-certs\") pod \"model-serving-api-fd65d7d6-h2w48\" (UID: \"123068b3-bb2c-4819-a517-0362388d5fde\") " pod="kserve/model-serving-api-fd65d7d6-h2w48" Jul 16 17:26:48.100178 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:26:48.100094 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2jhfj\" (UniqueName: \"kubernetes.io/projected/123068b3-bb2c-4819-a517-0362388d5fde-kube-api-access-2jhfj\") pod \"model-serving-api-fd65d7d6-h2w48\" (UID: \"123068b3-bb2c-4819-a517-0362388d5fde\") " pod="kserve/model-serving-api-fd65d7d6-h2w48" Jul 16 17:26:48.100178 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:26:48.100127 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-79v7x\" (UniqueName: \"kubernetes.io/projected/bf4947fd-3845-4bb8-a7a0-f03de833d182-kube-api-access-79v7x\") pod \"odh-model-controller-84b54b7f97-lf6xh\" (UID: \"bf4947fd-3845-4bb8-a7a0-f03de833d182\") " pod="kserve/odh-model-controller-84b54b7f97-lf6xh" Jul 16 17:26:48.102474 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:26:48.102452 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/bf4947fd-3845-4bb8-a7a0-f03de833d182-cert\") pod \"odh-model-controller-84b54b7f97-lf6xh\" (UID: \"bf4947fd-3845-4bb8-a7a0-f03de833d182\") " pod="kserve/odh-model-controller-84b54b7f97-lf6xh" Jul 16 17:26:48.102579 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:26:48.102487 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/123068b3-bb2c-4819-a517-0362388d5fde-tls-certs\") pod \"model-serving-api-fd65d7d6-h2w48\" (UID: \"123068b3-bb2c-4819-a517-0362388d5fde\") " pod="kserve/model-serving-api-fd65d7d6-h2w48" Jul 16 17:26:48.127900 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:26:48.127876 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-79v7x\" (UniqueName: \"kubernetes.io/projected/bf4947fd-3845-4bb8-a7a0-f03de833d182-kube-api-access-79v7x\") pod \"odh-model-controller-84b54b7f97-lf6xh\" (UID: \"bf4947fd-3845-4bb8-a7a0-f03de833d182\") " pod="kserve/odh-model-controller-84b54b7f97-lf6xh" Jul 16 17:26:48.132089 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:26:48.132062 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2jhfj\" (UniqueName: \"kubernetes.io/projected/123068b3-bb2c-4819-a517-0362388d5fde-kube-api-access-2jhfj\") pod \"model-serving-api-fd65d7d6-h2w48\" (UID: \"123068b3-bb2c-4819-a517-0362388d5fde\") " pod="kserve/model-serving-api-fd65d7d6-h2w48" Jul 16 17:26:48.258314 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:26:48.258234 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve/model-serving-api-fd65d7d6-h2w48" Jul 16 17:26:48.263745 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:26:48.263724 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve/odh-model-controller-84b54b7f97-lf6xh" Jul 16 17:26:48.443409 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:26:48.443208 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve/model-serving-api-fd65d7d6-h2w48"] Jul 16 17:26:48.444610 ip-10-0-132-130 kubenswrapper[2578]: W0716 17:26:48.444587 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod123068b3_bb2c_4819_a517_0362388d5fde.slice/crio-1bea172049e9722c206f7c1f9c065edf5371ce7ae95494a0bf89bf6329342522 WatchSource:0}: Error finding container 1bea172049e9722c206f7c1f9c065edf5371ce7ae95494a0bf89bf6329342522: Status 404 returned error can't find the container with id 1bea172049e9722c206f7c1f9c065edf5371ce7ae95494a0bf89bf6329342522 Jul 16 17:26:48.446646 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:26:48.446618 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve/odh-model-controller-84b54b7f97-lf6xh"] Jul 16 17:26:48.447062 ip-10-0-132-130 kubenswrapper[2578]: W0716 17:26:48.447038 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbf4947fd_3845_4bb8_a7a0_f03de833d182.slice/crio-e1eacb3c617cfd8f8d6937daf12b3e66d5d2c2bfca9c04feae2cb603ad487fbe WatchSource:0}: Error finding container e1eacb3c617cfd8f8d6937daf12b3e66d5d2c2bfca9c04feae2cb603ad487fbe: Status 404 returned error can't find the container with id e1eacb3c617cfd8f8d6937daf12b3e66d5d2c2bfca9c04feae2cb603ad487fbe Jul 16 17:26:48.710953 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:26:48.710919 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve/odh-model-controller-84b54b7f97-lf6xh" event={"ID":"bf4947fd-3845-4bb8-a7a0-f03de833d182","Type":"ContainerStarted","Data":"e1eacb3c617cfd8f8d6937daf12b3e66d5d2c2bfca9c04feae2cb603ad487fbe"} Jul 16 17:26:48.712076 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:26:48.712053 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve/model-serving-api-fd65d7d6-h2w48" event={"ID":"123068b3-bb2c-4819-a517-0362388d5fde","Type":"ContainerStarted","Data":"1bea172049e9722c206f7c1f9c065edf5371ce7ae95494a0bf89bf6329342522"} Jul 16 17:26:51.734147 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:26:51.734106 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve/model-serving-api-fd65d7d6-h2w48" event={"ID":"123068b3-bb2c-4819-a517-0362388d5fde","Type":"ContainerStarted","Data":"dc131d4f85551b6637d9f23d27f751f46c06231361ed30bfadb77346eee730ed"} Jul 16 17:26:51.734551 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:26:51.734206 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve/model-serving-api-fd65d7d6-h2w48" Jul 16 17:26:51.735676 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:26:51.735649 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve/odh-model-controller-84b54b7f97-lf6xh" event={"ID":"bf4947fd-3845-4bb8-a7a0-f03de833d182","Type":"ContainerStarted","Data":"67fca7815fd6fd502bd0c106095099d8fd3d1125dc26855d97bae98a35d174b9"} Jul 16 17:26:51.735780 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:26:51.735732 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve/odh-model-controller-84b54b7f97-lf6xh" Jul 16 17:26:51.764362 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:26:51.764309 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve/model-serving-api-fd65d7d6-h2w48" podStartSLOduration=1.587459543 podStartE2EDuration="4.764295366s" totalImagesPullingTime="3.176835823s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 17:26:47 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 17:26:51.763473294 +0000 UTC m=+756.249317185" watchObservedRunningTime="2026-07-16 17:26:51.764295366 +0000 UTC m=+756.250139271" Jul 16 17:26:51.800932 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:26:51.800875 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve/odh-model-controller-84b54b7f97-lf6xh" podStartSLOduration=1.667389878 podStartE2EDuration="4.800860745s" totalImagesPullingTime="3.133470867s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 17:26:47 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 17:26:51.799870335 +0000 UTC m=+756.285714224" watchObservedRunningTime="2026-07-16 17:26:51.800860745 +0000 UTC m=+756.286704633" Jul 16 17:27:02.744617 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:27:02.744583 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve/odh-model-controller-84b54b7f97-lf6xh" Jul 16 17:27:02.746172 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:27:02.746151 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve/model-serving-api-fd65d7d6-h2w48" Jul 16 17:28:47.501654 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:28:47.501614 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve-ci-e2e-test/scheduler-v06-pd-migration-test-kserve-prefill-794df6ffb4-4cx7q"] Jul 16 17:28:47.505936 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:28:47.505912 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/scheduler-v06-pd-migration-test-kserve-prefill-794df6ffb4-4cx7q" Jul 16 17:28:47.509164 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:28:47.509139 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"kube-root-ca.crt\"" Jul 16 17:28:47.509335 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:28:47.509217 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"scheduler-v06-pd-migration-test-kserve-self-signed-certs\"" Jul 16 17:28:47.509335 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:28:47.509224 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"default-dockercfg-sdm8r\"" Jul 16 17:28:47.509436 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:28:47.509406 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"openshift-service-ca.crt\"" Jul 16 17:28:47.514447 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:28:47.514427 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/scheduler-v06-pd-migration-test-kserve-prefill-794df6ffb4-4cx7q"] Jul 16 17:28:47.594607 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:28:47.594571 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/e3264472-595d-4dcb-97f8-92a98a63c073-home\") pod \"scheduler-v06-pd-migration-test-kserve-prefill-794df6ffb4-4cx7q\" (UID: \"e3264472-595d-4dcb-97f8-92a98a63c073\") " pod="kserve-ci-e2e-test/scheduler-v06-pd-migration-test-kserve-prefill-794df6ffb4-4cx7q" Jul 16 17:28:47.594768 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:28:47.594613 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e3264472-595d-4dcb-97f8-92a98a63c073-tmp-dir\") pod \"scheduler-v06-pd-migration-test-kserve-prefill-794df6ffb4-4cx7q\" (UID: \"e3264472-595d-4dcb-97f8-92a98a63c073\") " pod="kserve-ci-e2e-test/scheduler-v06-pd-migration-test-kserve-prefill-794df6ffb4-4cx7q" Jul 16 17:28:47.594768 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:28:47.594724 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e3264472-595d-4dcb-97f8-92a98a63c073-model-cache\") pod \"scheduler-v06-pd-migration-test-kserve-prefill-794df6ffb4-4cx7q\" (UID: \"e3264472-595d-4dcb-97f8-92a98a63c073\") " pod="kserve-ci-e2e-test/scheduler-v06-pd-migration-test-kserve-prefill-794df6ffb4-4cx7q" Jul 16 17:28:47.594873 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:28:47.594769 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e3264472-595d-4dcb-97f8-92a98a63c073-tls-certs\") pod \"scheduler-v06-pd-migration-test-kserve-prefill-794df6ffb4-4cx7q\" (UID: \"e3264472-595d-4dcb-97f8-92a98a63c073\") " pod="kserve-ci-e2e-test/scheduler-v06-pd-migration-test-kserve-prefill-794df6ffb4-4cx7q" Jul 16 17:28:47.594873 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:28:47.594827 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kv26r\" (UniqueName: \"kubernetes.io/projected/e3264472-595d-4dcb-97f8-92a98a63c073-kube-api-access-kv26r\") pod \"scheduler-v06-pd-migration-test-kserve-prefill-794df6ffb4-4cx7q\" (UID: \"e3264472-595d-4dcb-97f8-92a98a63c073\") " pod="kserve-ci-e2e-test/scheduler-v06-pd-migration-test-kserve-prefill-794df6ffb4-4cx7q" Jul 16 17:28:47.594962 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:28:47.594872 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/e3264472-595d-4dcb-97f8-92a98a63c073-dshm\") pod \"scheduler-v06-pd-migration-test-kserve-prefill-794df6ffb4-4cx7q\" (UID: \"e3264472-595d-4dcb-97f8-92a98a63c073\") " pod="kserve-ci-e2e-test/scheduler-v06-pd-migration-test-kserve-prefill-794df6ffb4-4cx7q" Jul 16 17:28:47.695651 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:28:47.695604 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e3264472-595d-4dcb-97f8-92a98a63c073-tmp-dir\") pod \"scheduler-v06-pd-migration-test-kserve-prefill-794df6ffb4-4cx7q\" (UID: \"e3264472-595d-4dcb-97f8-92a98a63c073\") " pod="kserve-ci-e2e-test/scheduler-v06-pd-migration-test-kserve-prefill-794df6ffb4-4cx7q" Jul 16 17:28:47.695861 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:28:47.695680 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e3264472-595d-4dcb-97f8-92a98a63c073-model-cache\") pod \"scheduler-v06-pd-migration-test-kserve-prefill-794df6ffb4-4cx7q\" (UID: \"e3264472-595d-4dcb-97f8-92a98a63c073\") " pod="kserve-ci-e2e-test/scheduler-v06-pd-migration-test-kserve-prefill-794df6ffb4-4cx7q" Jul 16 17:28:47.695861 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:28:47.695734 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e3264472-595d-4dcb-97f8-92a98a63c073-tls-certs\") pod \"scheduler-v06-pd-migration-test-kserve-prefill-794df6ffb4-4cx7q\" (UID: \"e3264472-595d-4dcb-97f8-92a98a63c073\") " pod="kserve-ci-e2e-test/scheduler-v06-pd-migration-test-kserve-prefill-794df6ffb4-4cx7q" Jul 16 17:28:47.695861 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:28:47.695757 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kv26r\" (UniqueName: \"kubernetes.io/projected/e3264472-595d-4dcb-97f8-92a98a63c073-kube-api-access-kv26r\") pod \"scheduler-v06-pd-migration-test-kserve-prefill-794df6ffb4-4cx7q\" (UID: \"e3264472-595d-4dcb-97f8-92a98a63c073\") " pod="kserve-ci-e2e-test/scheduler-v06-pd-migration-test-kserve-prefill-794df6ffb4-4cx7q" Jul 16 17:28:47.695861 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:28:47.695804 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/e3264472-595d-4dcb-97f8-92a98a63c073-dshm\") pod \"scheduler-v06-pd-migration-test-kserve-prefill-794df6ffb4-4cx7q\" (UID: \"e3264472-595d-4dcb-97f8-92a98a63c073\") " pod="kserve-ci-e2e-test/scheduler-v06-pd-migration-test-kserve-prefill-794df6ffb4-4cx7q" Jul 16 17:28:47.695861 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:28:47.695828 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/e3264472-595d-4dcb-97f8-92a98a63c073-home\") pod \"scheduler-v06-pd-migration-test-kserve-prefill-794df6ffb4-4cx7q\" (UID: \"e3264472-595d-4dcb-97f8-92a98a63c073\") " pod="kserve-ci-e2e-test/scheduler-v06-pd-migration-test-kserve-prefill-794df6ffb4-4cx7q" Jul 16 17:28:47.696141 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:28:47.696104 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e3264472-595d-4dcb-97f8-92a98a63c073-tmp-dir\") pod \"scheduler-v06-pd-migration-test-kserve-prefill-794df6ffb4-4cx7q\" (UID: \"e3264472-595d-4dcb-97f8-92a98a63c073\") " pod="kserve-ci-e2e-test/scheduler-v06-pd-migration-test-kserve-prefill-794df6ffb4-4cx7q" Jul 16 17:28:47.696282 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:28:47.696262 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e3264472-595d-4dcb-97f8-92a98a63c073-model-cache\") pod \"scheduler-v06-pd-migration-test-kserve-prefill-794df6ffb4-4cx7q\" (UID: \"e3264472-595d-4dcb-97f8-92a98a63c073\") " pod="kserve-ci-e2e-test/scheduler-v06-pd-migration-test-kserve-prefill-794df6ffb4-4cx7q" Jul 16 17:28:47.696345 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:28:47.696274 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/e3264472-595d-4dcb-97f8-92a98a63c073-home\") pod \"scheduler-v06-pd-migration-test-kserve-prefill-794df6ffb4-4cx7q\" (UID: \"e3264472-595d-4dcb-97f8-92a98a63c073\") " pod="kserve-ci-e2e-test/scheduler-v06-pd-migration-test-kserve-prefill-794df6ffb4-4cx7q" Jul 16 17:28:47.698063 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:28:47.698038 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/e3264472-595d-4dcb-97f8-92a98a63c073-dshm\") pod \"scheduler-v06-pd-migration-test-kserve-prefill-794df6ffb4-4cx7q\" (UID: \"e3264472-595d-4dcb-97f8-92a98a63c073\") " pod="kserve-ci-e2e-test/scheduler-v06-pd-migration-test-kserve-prefill-794df6ffb4-4cx7q" Jul 16 17:28:47.698398 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:28:47.698378 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e3264472-595d-4dcb-97f8-92a98a63c073-tls-certs\") pod \"scheduler-v06-pd-migration-test-kserve-prefill-794df6ffb4-4cx7q\" (UID: \"e3264472-595d-4dcb-97f8-92a98a63c073\") " pod="kserve-ci-e2e-test/scheduler-v06-pd-migration-test-kserve-prefill-794df6ffb4-4cx7q" Jul 16 17:28:47.710020 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:28:47.709997 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kv26r\" (UniqueName: \"kubernetes.io/projected/e3264472-595d-4dcb-97f8-92a98a63c073-kube-api-access-kv26r\") pod \"scheduler-v06-pd-migration-test-kserve-prefill-794df6ffb4-4cx7q\" (UID: \"e3264472-595d-4dcb-97f8-92a98a63c073\") " pod="kserve-ci-e2e-test/scheduler-v06-pd-migration-test-kserve-prefill-794df6ffb4-4cx7q" Jul 16 17:28:47.817355 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:28:47.817321 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/scheduler-v06-pd-migration-test-kserve-prefill-794df6ffb4-4cx7q" Jul 16 17:28:47.873579 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:28:47.873546 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve-ci-e2e-test/scheduler-v06-pd-migration-test-kserve-router-scheduler-79ww96l"] Jul 16 17:28:47.881011 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:28:47.880982 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/scheduler-v06-pd-migration-test-kserve-router-scheduler-79ww96l" Jul 16 17:28:47.883489 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:28:47.883441 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"scheduler-v06-pd-migration-test-epp-sa-dockercfg-5xrmg\"" Jul 16 17:28:47.895391 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:28:47.895367 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/scheduler-v06-pd-migration-test-kserve-router-scheduler-79ww96l"] Jul 16 17:28:47.898628 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:28:47.898601 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-64jrn\" (UniqueName: \"kubernetes.io/projected/d606971e-db1c-4d32-8d4b-1475d09fd05b-kube-api-access-64jrn\") pod \"scheduler-v06-pd-migration-test-kserve-router-scheduler-79ww96l\" (UID: \"d606971e-db1c-4d32-8d4b-1475d09fd05b\") " pod="kserve-ci-e2e-test/scheduler-v06-pd-migration-test-kserve-router-scheduler-79ww96l" Jul 16 17:28:47.898752 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:28:47.898658 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d606971e-db1c-4d32-8d4b-1475d09fd05b-tls-certs\") pod \"scheduler-v06-pd-migration-test-kserve-router-scheduler-79ww96l\" (UID: \"d606971e-db1c-4d32-8d4b-1475d09fd05b\") " pod="kserve-ci-e2e-test/scheduler-v06-pd-migration-test-kserve-router-scheduler-79ww96l" Jul 16 17:28:47.898822 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:28:47.898793 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tokenizer-uds\" (UniqueName: \"kubernetes.io/empty-dir/d606971e-db1c-4d32-8d4b-1475d09fd05b-tokenizer-uds\") pod \"scheduler-v06-pd-migration-test-kserve-router-scheduler-79ww96l\" (UID: \"d606971e-db1c-4d32-8d4b-1475d09fd05b\") " pod="kserve-ci-e2e-test/scheduler-v06-pd-migration-test-kserve-router-scheduler-79ww96l" Jul 16 17:28:47.973035 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:28:47.972995 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/scheduler-v06-pd-migration-test-kserve-prefill-794df6ffb4-4cx7q"] Jul 16 17:28:47.974988 ip-10-0-132-130 kubenswrapper[2578]: W0716 17:28:47.974963 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode3264472_595d_4dcb_97f8_92a98a63c073.slice/crio-dfa0513de1d7bb00ae6c338985176c4759c20bd1f1b40355971339c17cf7a970 WatchSource:0}: Error finding container dfa0513de1d7bb00ae6c338985176c4759c20bd1f1b40355971339c17cf7a970: Status 404 returned error can't find the container with id dfa0513de1d7bb00ae6c338985176c4759c20bd1f1b40355971339c17cf7a970 Jul 16 17:28:47.999759 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:28:47.999737 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tokenizer-uds\" (UniqueName: \"kubernetes.io/empty-dir/d606971e-db1c-4d32-8d4b-1475d09fd05b-tokenizer-uds\") pod \"scheduler-v06-pd-migration-test-kserve-router-scheduler-79ww96l\" (UID: \"d606971e-db1c-4d32-8d4b-1475d09fd05b\") " pod="kserve-ci-e2e-test/scheduler-v06-pd-migration-test-kserve-router-scheduler-79ww96l" Jul 16 17:28:47.999848 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:28:47.999786 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-64jrn\" (UniqueName: \"kubernetes.io/projected/d606971e-db1c-4d32-8d4b-1475d09fd05b-kube-api-access-64jrn\") pod \"scheduler-v06-pd-migration-test-kserve-router-scheduler-79ww96l\" (UID: \"d606971e-db1c-4d32-8d4b-1475d09fd05b\") " pod="kserve-ci-e2e-test/scheduler-v06-pd-migration-test-kserve-router-scheduler-79ww96l" Jul 16 17:28:47.999848 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:28:47.999827 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d606971e-db1c-4d32-8d4b-1475d09fd05b-tls-certs\") pod \"scheduler-v06-pd-migration-test-kserve-router-scheduler-79ww96l\" (UID: \"d606971e-db1c-4d32-8d4b-1475d09fd05b\") " pod="kserve-ci-e2e-test/scheduler-v06-pd-migration-test-kserve-router-scheduler-79ww96l" Jul 16 17:28:48.000150 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:28:48.000126 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tokenizer-uds\" (UniqueName: \"kubernetes.io/empty-dir/d606971e-db1c-4d32-8d4b-1475d09fd05b-tokenizer-uds\") pod \"scheduler-v06-pd-migration-test-kserve-router-scheduler-79ww96l\" (UID: \"d606971e-db1c-4d32-8d4b-1475d09fd05b\") " pod="kserve-ci-e2e-test/scheduler-v06-pd-migration-test-kserve-router-scheduler-79ww96l" Jul 16 17:28:48.002018 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:28:48.001999 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d606971e-db1c-4d32-8d4b-1475d09fd05b-tls-certs\") pod \"scheduler-v06-pd-migration-test-kserve-router-scheduler-79ww96l\" (UID: \"d606971e-db1c-4d32-8d4b-1475d09fd05b\") " pod="kserve-ci-e2e-test/scheduler-v06-pd-migration-test-kserve-router-scheduler-79ww96l" Jul 16 17:28:48.013561 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:28:48.013543 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-64jrn\" (UniqueName: \"kubernetes.io/projected/d606971e-db1c-4d32-8d4b-1475d09fd05b-kube-api-access-64jrn\") pod \"scheduler-v06-pd-migration-test-kserve-router-scheduler-79ww96l\" (UID: \"d606971e-db1c-4d32-8d4b-1475d09fd05b\") " pod="kserve-ci-e2e-test/scheduler-v06-pd-migration-test-kserve-router-scheduler-79ww96l" Jul 16 17:28:48.199431 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:28:48.199350 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/scheduler-v06-pd-migration-test-kserve-router-scheduler-79ww96l" Jul 16 17:28:48.360069 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:28:48.360036 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/scheduler-v06-pd-migration-test-kserve-router-scheduler-79ww96l"] Jul 16 17:28:48.360617 ip-10-0-132-130 kubenswrapper[2578]: W0716 17:28:48.360592 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd606971e_db1c_4d32_8d4b_1475d09fd05b.slice/crio-95b0253c59def81b5a420a80a6819ce1fba2301725d13f90e3df742cebf49965 WatchSource:0}: Error finding container 95b0253c59def81b5a420a80a6819ce1fba2301725d13f90e3df742cebf49965: Status 404 returned error can't find the container with id 95b0253c59def81b5a420a80a6819ce1fba2301725d13f90e3df742cebf49965 Jul 16 17:28:48.385447 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:28:48.385419 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/scheduler-v06-pd-migration-test-kserve-prefill-794df6ffb4-4cx7q" event={"ID":"e3264472-595d-4dcb-97f8-92a98a63c073","Type":"ContainerStarted","Data":"dfa0513de1d7bb00ae6c338985176c4759c20bd1f1b40355971339c17cf7a970"} Jul 16 17:28:48.386864 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:28:48.386829 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/scheduler-v06-pd-migration-test-kserve-router-scheduler-79ww96l" event={"ID":"d606971e-db1c-4d32-8d4b-1475d09fd05b","Type":"ContainerStarted","Data":"95b0253c59def81b5a420a80a6819ce1fba2301725d13f90e3df742cebf49965"} Jul 16 17:28:50.405211 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:28:50.405174 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/scheduler-v06-pd-migration-test-kserve-prefill-794df6ffb4-4cx7q" event={"ID":"e3264472-595d-4dcb-97f8-92a98a63c073","Type":"ContainerStarted","Data":"63ce909d6a4b137e001ef1a5f83b6da311502c820dc91e5ecb781e642750330e"} Jul 16 17:28:50.428998 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:28:50.428940 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve-ci-e2e-test/scheduler-v06-pd-migration-test-kserve-prefill-794df6ffb4-4cx7q" podStartSLOduration=1.674472805 podStartE2EDuration="3.42892417s" totalImagesPullingTime="1.754451365s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 17:28:47 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 17:28:50.428822184 +0000 UTC m=+874.914666074" watchObservedRunningTime="2026-07-16 17:28:50.42892417 +0000 UTC m=+874.914768059" Jul 16 17:28:50.818314 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:28:50.818277 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/scheduler-v06-pd-migration-test-kserve-prefill-794df6ffb4-4cx7q" Jul 16 17:28:51.412482 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:28:51.412447 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/scheduler-v06-pd-migration-test-kserve-router-scheduler-79ww96l" event={"ID":"d606971e-db1c-4d32-8d4b-1475d09fd05b","Type":"ContainerStarted","Data":"0e3d11831fb40d33d627a07a8b69ed33bac0ed4e3fd57c98029f95be8474af39"} Jul 16 17:28:51.412898 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:28:51.412644 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/scheduler-v06-pd-migration-test-kserve-router-scheduler-79ww96l" Jul 16 17:28:51.437131 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:28:51.437082 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve-ci-e2e-test/scheduler-v06-pd-migration-test-kserve-router-scheduler-79ww96l" podStartSLOduration=1.8866102759999999 podStartE2EDuration="4.437067445s" totalImagesPullingTime="2.550457169s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 17:28:47 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 17:28:51.434594726 +0000 UTC m=+875.920438633" watchObservedRunningTime="2026-07-16 17:28:51.437067445 +0000 UTC m=+875.922911334" Jul 16 17:28:57.817607 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:28:57.817577 2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kserve-ci-e2e-test/scheduler-v06-pd-migration-test-kserve-prefill-794df6ffb4-4cx7q" Jul 16 17:28:57.829956 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:28:57.829933 2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kserve-ci-e2e-test/scheduler-v06-pd-migration-test-kserve-prefill-794df6ffb4-4cx7q" Jul 16 17:28:58.463202 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:28:58.463122 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/scheduler-v06-pd-migration-test-kserve-prefill-794df6ffb4-4cx7q" Jul 16 17:29:16.258873 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:29:16.258841 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-drs2p_e55f447c-ad64-482b-a86b-0588bddf7b4d/console-operator/1.log" Jul 16 17:29:16.259318 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:29:16.259284 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-drs2p_e55f447c-ad64-482b-a86b-0588bddf7b4d/console-operator/1.log" Jul 16 17:29:22.420128 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:29:22.420100 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/scheduler-v06-pd-migration-test-kserve-router-scheduler-79ww96l" Jul 16 17:30:29.426482 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:30:29.426437 2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/scheduler-v06-pd-migration-test-kserve-prefill-794df6ffb4-4cx7q"] Jul 16 17:30:29.428920 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:30:29.428891 2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/scheduler-v06-pd-migration-test-kserve-router-scheduler-79ww96l"] Jul 16 17:30:29.435926 ip-10-0-132-130 kubenswrapper[2578]: E0716 17:30:29.435891 2578 log.go:32] "ExecSync cmd from runtime service failed" err=< Jul 16 17:30:29.435926 ip-10-0-132-130 kubenswrapper[2578]: rpc error: code = Unknown desc = command error: executable file `/bin/sleep` not found: No such file or directory Jul 16 17:30:29.435926 ip-10-0-132-130 kubenswrapper[2578]: , stdout: , stderr: , exit code -1 Jul 16 17:30:29.435926 ip-10-0-132-130 kubenswrapper[2578]: > containerID="63ce909d6a4b137e001ef1a5f83b6da311502c820dc91e5ecb781e642750330e" cmd=["/bin/sleep","15"] Jul 16 17:30:29.436217 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:30:29.435946 2578 handlers.go:82] "Exec lifecycle hook for Container in Pod failed" execCommand=["/bin/sleep","15"] containerName="main" pod="kserve-ci-e2e-test/scheduler-v06-pd-migration-test-kserve-prefill-794df6ffb4-4cx7q" message="" err=< Jul 16 17:30:29.436217 ip-10-0-132-130 kubenswrapper[2578]: rpc error: code = Unknown desc = command error: executable file `/bin/sleep` not found: No such file or directory Jul 16 17:30:29.436217 ip-10-0-132-130 kubenswrapper[2578]: , stdout: , stderr: , exit code -1 Jul 16 17:30:29.436217 ip-10-0-132-130 kubenswrapper[2578]: > Jul 16 17:30:29.436217 ip-10-0-132-130 kubenswrapper[2578]: E0716 17:30:29.435962 2578 kuberuntime_container.go:755] "PreStop hook failed" err=< Jul 16 17:30:29.436217 ip-10-0-132-130 kubenswrapper[2578]: rpc error: code = Unknown desc = command error: executable file `/bin/sleep` not found: No such file or directory Jul 16 17:30:29.436217 ip-10-0-132-130 kubenswrapper[2578]: , stdout: , stderr: , exit code -1 Jul 16 17:30:29.436217 ip-10-0-132-130 kubenswrapper[2578]: > pod="kserve-ci-e2e-test/scheduler-v06-pd-migration-test-kserve-prefill-794df6ffb4-4cx7q" podUID="e3264472-595d-4dcb-97f8-92a98a63c073" containerName="main" containerID="cri-o://63ce909d6a4b137e001ef1a5f83b6da311502c820dc91e5ecb781e642750330e" Jul 16 17:30:29.436217 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:30:29.435995 2578 kuberuntime_container.go:873] "Killing container with a grace period" pod="kserve-ci-e2e-test/scheduler-v06-pd-migration-test-kserve-prefill-794df6ffb4-4cx7q" podUID="e3264472-595d-4dcb-97f8-92a98a63c073" containerName="main" containerID="cri-o://63ce909d6a4b137e001ef1a5f83b6da311502c820dc91e5ecb781e642750330e" gracePeriod=60 Jul 16 17:30:29.453329 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:30:29.453299 2578 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/scheduler-v06-pd-migration-test-kserve-prefill-794df6ffb4-4cx7q" podUID="e3264472-595d-4dcb-97f8-92a98a63c073" containerName="main" probeResult="failure" output="Get \"https://10.133.0.49:8000/health\": dial tcp 10.133.0.49:8000: connect: connection refused" Jul 16 17:30:29.802137 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:30:29.802114 2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/scheduler-v06-pd-migration-test-kserve-prefill-794df6ffb4-4cx7q" Jul 16 17:30:29.886274 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:30:29.886239 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/e3264472-595d-4dcb-97f8-92a98a63c073-home\") pod \"e3264472-595d-4dcb-97f8-92a98a63c073\" (UID: \"e3264472-595d-4dcb-97f8-92a98a63c073\") " Jul 16 17:30:29.886449 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:30:29.886343 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/e3264472-595d-4dcb-97f8-92a98a63c073-dshm\") pod \"e3264472-595d-4dcb-97f8-92a98a63c073\" (UID: \"e3264472-595d-4dcb-97f8-92a98a63c073\") " Jul 16 17:30:29.886449 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:30:29.886372 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e3264472-595d-4dcb-97f8-92a98a63c073-tmp-dir\") pod \"e3264472-595d-4dcb-97f8-92a98a63c073\" (UID: \"e3264472-595d-4dcb-97f8-92a98a63c073\") " Jul 16 17:30:29.886449 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:30:29.886389 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-kv26r\" (UniqueName: \"kubernetes.io/projected/e3264472-595d-4dcb-97f8-92a98a63c073-kube-api-access-kv26r\") pod \"e3264472-595d-4dcb-97f8-92a98a63c073\" (UID: \"e3264472-595d-4dcb-97f8-92a98a63c073\") " Jul 16 17:30:29.886449 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:30:29.886419 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e3264472-595d-4dcb-97f8-92a98a63c073-tls-certs\") pod \"e3264472-595d-4dcb-97f8-92a98a63c073\" (UID: \"e3264472-595d-4dcb-97f8-92a98a63c073\") " Jul 16 17:30:29.886616 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:30:29.886466 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e3264472-595d-4dcb-97f8-92a98a63c073-model-cache\") pod \"e3264472-595d-4dcb-97f8-92a98a63c073\" (UID: \"e3264472-595d-4dcb-97f8-92a98a63c073\") " Jul 16 17:30:29.886616 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:30:29.886508 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e3264472-595d-4dcb-97f8-92a98a63c073-home" (OuterVolumeSpecName: "home") pod "e3264472-595d-4dcb-97f8-92a98a63c073" (UID: "e3264472-595d-4dcb-97f8-92a98a63c073"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 16 17:30:29.886727 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:30:29.886659 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e3264472-595d-4dcb-97f8-92a98a63c073-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "e3264472-595d-4dcb-97f8-92a98a63c073" (UID: "e3264472-595d-4dcb-97f8-92a98a63c073"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 16 17:30:29.886826 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:30:29.886806 2578 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e3264472-595d-4dcb-97f8-92a98a63c073-tmp-dir\") on node \"ip-10-0-132-130.ec2.internal\" DevicePath \"\"" Jul 16 17:30:29.886888 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:30:29.886832 2578 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/e3264472-595d-4dcb-97f8-92a98a63c073-home\") on node \"ip-10-0-132-130.ec2.internal\" DevicePath \"\"" Jul 16 17:30:29.886888 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:30:29.886806 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e3264472-595d-4dcb-97f8-92a98a63c073-model-cache" (OuterVolumeSpecName: "model-cache") pod "e3264472-595d-4dcb-97f8-92a98a63c073" (UID: "e3264472-595d-4dcb-97f8-92a98a63c073"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 16 17:30:29.888753 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:30:29.888719 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e3264472-595d-4dcb-97f8-92a98a63c073-dshm" (OuterVolumeSpecName: "dshm") pod "e3264472-595d-4dcb-97f8-92a98a63c073" (UID: "e3264472-595d-4dcb-97f8-92a98a63c073"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 16 17:30:29.888753 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:30:29.888726 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e3264472-595d-4dcb-97f8-92a98a63c073-kube-api-access-kv26r" (OuterVolumeSpecName: "kube-api-access-kv26r") pod "e3264472-595d-4dcb-97f8-92a98a63c073" (UID: "e3264472-595d-4dcb-97f8-92a98a63c073"). InnerVolumeSpecName "kube-api-access-kv26r". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 16 17:30:29.888908 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:30:29.888762 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e3264472-595d-4dcb-97f8-92a98a63c073-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "e3264472-595d-4dcb-97f8-92a98a63c073" (UID: "e3264472-595d-4dcb-97f8-92a98a63c073"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 16 17:30:29.984819 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:30:29.984721 2578 generic.go:356] "Generic (PLEG): container finished" podID="e3264472-595d-4dcb-97f8-92a98a63c073" containerID="63ce909d6a4b137e001ef1a5f83b6da311502c820dc91e5ecb781e642750330e" exitCode=0 Jul 16 17:30:29.984819 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:30:29.984765 2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/scheduler-v06-pd-migration-test-kserve-prefill-794df6ffb4-4cx7q" Jul 16 17:30:29.984819 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:30:29.984798 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/scheduler-v06-pd-migration-test-kserve-prefill-794df6ffb4-4cx7q" event={"ID":"e3264472-595d-4dcb-97f8-92a98a63c073","Type":"ContainerDied","Data":"63ce909d6a4b137e001ef1a5f83b6da311502c820dc91e5ecb781e642750330e"} Jul 16 17:30:29.985092 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:30:29.984858 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/scheduler-v06-pd-migration-test-kserve-prefill-794df6ffb4-4cx7q" event={"ID":"e3264472-595d-4dcb-97f8-92a98a63c073","Type":"ContainerDied","Data":"dfa0513de1d7bb00ae6c338985176c4759c20bd1f1b40355971339c17cf7a970"} Jul 16 17:30:29.985092 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:30:29.984883 2578 scope.go:117] "RemoveContainer" containerID="63ce909d6a4b137e001ef1a5f83b6da311502c820dc91e5ecb781e642750330e" Jul 16 17:30:29.987505 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:30:29.987484 2578 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/e3264472-595d-4dcb-97f8-92a98a63c073-dshm\") on node \"ip-10-0-132-130.ec2.internal\" DevicePath \"\"" Jul 16 17:30:29.987505 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:30:29.987505 2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-kv26r\" (UniqueName: \"kubernetes.io/projected/e3264472-595d-4dcb-97f8-92a98a63c073-kube-api-access-kv26r\") on node \"ip-10-0-132-130.ec2.internal\" DevicePath \"\"" Jul 16 17:30:29.987711 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:30:29.987515 2578 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e3264472-595d-4dcb-97f8-92a98a63c073-tls-certs\") on node \"ip-10-0-132-130.ec2.internal\" DevicePath \"\"" Jul 16 17:30:29.987711 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:30:29.987524 2578 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e3264472-595d-4dcb-97f8-92a98a63c073-model-cache\") on node \"ip-10-0-132-130.ec2.internal\" DevicePath \"\"" Jul 16 17:30:29.994848 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:30:29.994816 2578 scope.go:117] "RemoveContainer" containerID="63ce909d6a4b137e001ef1a5f83b6da311502c820dc91e5ecb781e642750330e" Jul 16 17:30:29.995067 ip-10-0-132-130 kubenswrapper[2578]: E0716 17:30:29.995037 2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"63ce909d6a4b137e001ef1a5f83b6da311502c820dc91e5ecb781e642750330e\": container with ID starting with 63ce909d6a4b137e001ef1a5f83b6da311502c820dc91e5ecb781e642750330e not found: ID does not exist" containerID="63ce909d6a4b137e001ef1a5f83b6da311502c820dc91e5ecb781e642750330e" Jul 16 17:30:29.995154 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:30:29.995072 2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"63ce909d6a4b137e001ef1a5f83b6da311502c820dc91e5ecb781e642750330e"} err="failed to get container status \"63ce909d6a4b137e001ef1a5f83b6da311502c820dc91e5ecb781e642750330e\": rpc error: code = NotFound desc = could not find container \"63ce909d6a4b137e001ef1a5f83b6da311502c820dc91e5ecb781e642750330e\": container with ID starting with 63ce909d6a4b137e001ef1a5f83b6da311502c820dc91e5ecb781e642750330e not found: ID does not exist" Jul 16 17:30:30.012619 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:30:30.012592 2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/scheduler-v06-pd-migration-test-kserve-prefill-794df6ffb4-4cx7q"] Jul 16 17:30:30.016274 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:30:30.016248 2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kserve-ci-e2e-test/scheduler-v06-pd-migration-test-kserve-prefill-794df6ffb4-4cx7q"] Jul 16 17:30:30.215902 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:30:30.215872 2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e3264472-595d-4dcb-97f8-92a98a63c073" path="/var/lib/kubelet/pods/e3264472-595d-4dcb-97f8-92a98a63c073/volumes" Jul 16 17:30:32.420224 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:30:32.420190 2578 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/scheduler-v06-pd-migration-test-kserve-router-scheduler-79ww96l" podUID="d606971e-db1c-4d32-8d4b-1475d09fd05b" containerName="main" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")" Jul 16 17:30:42.419737 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:30:42.419637 2578 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/scheduler-v06-pd-migration-test-kserve-router-scheduler-79ww96l" podUID="d606971e-db1c-4d32-8d4b-1475d09fd05b" containerName="main" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")" Jul 16 17:30:44.448821 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:30:44.448782 2578 kuberuntime_container.go:873] "Killing container with a grace period" pod="kserve-ci-e2e-test/scheduler-v06-pd-migration-test-kserve-router-scheduler-79ww96l" podUID="d606971e-db1c-4d32-8d4b-1475d09fd05b" containerName="main" containerID="cri-o://0e3d11831fb40d33d627a07a8b69ed33bac0ed4e3fd57c98029f95be8474af39" gracePeriod=45 Jul 16 17:30:44.705240 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:30:44.705182 2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/scheduler-v06-pd-migration-test-kserve-router-scheduler-79ww96l" Jul 16 17:30:44.811682 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:30:44.811655 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d606971e-db1c-4d32-8d4b-1475d09fd05b-tls-certs\") pod \"d606971e-db1c-4d32-8d4b-1475d09fd05b\" (UID: \"d606971e-db1c-4d32-8d4b-1475d09fd05b\") " Jul 16 17:30:44.811844 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:30:44.811743 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tokenizer-uds\" (UniqueName: \"kubernetes.io/empty-dir/d606971e-db1c-4d32-8d4b-1475d09fd05b-tokenizer-uds\") pod \"d606971e-db1c-4d32-8d4b-1475d09fd05b\" (UID: \"d606971e-db1c-4d32-8d4b-1475d09fd05b\") " Jul 16 17:30:44.811844 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:30:44.811795 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-64jrn\" (UniqueName: \"kubernetes.io/projected/d606971e-db1c-4d32-8d4b-1475d09fd05b-kube-api-access-64jrn\") pod \"d606971e-db1c-4d32-8d4b-1475d09fd05b\" (UID: \"d606971e-db1c-4d32-8d4b-1475d09fd05b\") " Jul 16 17:30:44.812013 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:30:44.811988 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d606971e-db1c-4d32-8d4b-1475d09fd05b-tokenizer-uds" (OuterVolumeSpecName: "tokenizer-uds") pod "d606971e-db1c-4d32-8d4b-1475d09fd05b" (UID: "d606971e-db1c-4d32-8d4b-1475d09fd05b"). InnerVolumeSpecName "tokenizer-uds". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 16 17:30:44.813621 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:30:44.813598 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d606971e-db1c-4d32-8d4b-1475d09fd05b-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "d606971e-db1c-4d32-8d4b-1475d09fd05b" (UID: "d606971e-db1c-4d32-8d4b-1475d09fd05b"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 16 17:30:44.813747 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:30:44.813717 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d606971e-db1c-4d32-8d4b-1475d09fd05b-kube-api-access-64jrn" (OuterVolumeSpecName: "kube-api-access-64jrn") pod "d606971e-db1c-4d32-8d4b-1475d09fd05b" (UID: "d606971e-db1c-4d32-8d4b-1475d09fd05b"). InnerVolumeSpecName "kube-api-access-64jrn". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 16 17:30:44.912867 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:30:44.912841 2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-64jrn\" (UniqueName: \"kubernetes.io/projected/d606971e-db1c-4d32-8d4b-1475d09fd05b-kube-api-access-64jrn\") on node \"ip-10-0-132-130.ec2.internal\" DevicePath \"\"" Jul 16 17:30:44.912867 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:30:44.912863 2578 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d606971e-db1c-4d32-8d4b-1475d09fd05b-tls-certs\") on node \"ip-10-0-132-130.ec2.internal\" DevicePath \"\"" Jul 16 17:30:44.912994 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:30:44.912874 2578 reconciler_common.go:299] "Volume detached for volume \"tokenizer-uds\" (UniqueName: \"kubernetes.io/empty-dir/d606971e-db1c-4d32-8d4b-1475d09fd05b-tokenizer-uds\") on node \"ip-10-0-132-130.ec2.internal\" DevicePath \"\"" Jul 16 17:30:45.074396 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:30:45.074366 2578 generic.go:356] "Generic (PLEG): container finished" podID="d606971e-db1c-4d32-8d4b-1475d09fd05b" containerID="0e3d11831fb40d33d627a07a8b69ed33bac0ed4e3fd57c98029f95be8474af39" exitCode=0 Jul 16 17:30:45.074543 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:30:45.074410 2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/scheduler-v06-pd-migration-test-kserve-router-scheduler-79ww96l" Jul 16 17:30:45.074543 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:30:45.074427 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/scheduler-v06-pd-migration-test-kserve-router-scheduler-79ww96l" event={"ID":"d606971e-db1c-4d32-8d4b-1475d09fd05b","Type":"ContainerDied","Data":"0e3d11831fb40d33d627a07a8b69ed33bac0ed4e3fd57c98029f95be8474af39"} Jul 16 17:30:45.074543 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:30:45.074468 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/scheduler-v06-pd-migration-test-kserve-router-scheduler-79ww96l" event={"ID":"d606971e-db1c-4d32-8d4b-1475d09fd05b","Type":"ContainerDied","Data":"95b0253c59def81b5a420a80a6819ce1fba2301725d13f90e3df742cebf49965"} Jul 16 17:30:45.074543 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:30:45.074487 2578 scope.go:117] "RemoveContainer" containerID="0e3d11831fb40d33d627a07a8b69ed33bac0ed4e3fd57c98029f95be8474af39" Jul 16 17:30:45.088522 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:30:45.088506 2578 scope.go:117] "RemoveContainer" containerID="0e3d11831fb40d33d627a07a8b69ed33bac0ed4e3fd57c98029f95be8474af39" Jul 16 17:30:45.088799 ip-10-0-132-130 kubenswrapper[2578]: E0716 17:30:45.088774 2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0e3d11831fb40d33d627a07a8b69ed33bac0ed4e3fd57c98029f95be8474af39\": container with ID starting with 0e3d11831fb40d33d627a07a8b69ed33bac0ed4e3fd57c98029f95be8474af39 not found: ID does not exist" containerID="0e3d11831fb40d33d627a07a8b69ed33bac0ed4e3fd57c98029f95be8474af39" Jul 16 17:30:45.088895 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:30:45.088806 2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0e3d11831fb40d33d627a07a8b69ed33bac0ed4e3fd57c98029f95be8474af39"} err="failed to get container status \"0e3d11831fb40d33d627a07a8b69ed33bac0ed4e3fd57c98029f95be8474af39\": rpc error: code = NotFound desc = could not find container \"0e3d11831fb40d33d627a07a8b69ed33bac0ed4e3fd57c98029f95be8474af39\": container with ID starting with 0e3d11831fb40d33d627a07a8b69ed33bac0ed4e3fd57c98029f95be8474af39 not found: ID does not exist" Jul 16 17:30:45.109624 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:30:45.109587 2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/scheduler-v06-pd-migration-test-kserve-router-scheduler-79ww96l"] Jul 16 17:30:45.119018 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:30:45.118992 2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kserve-ci-e2e-test/scheduler-v06-pd-migration-test-kserve-router-scheduler-79ww96l"] Jul 16 17:30:46.216497 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:30:46.216467 2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d606971e-db1c-4d32-8d4b-1475d09fd05b" path="/var/lib/kubelet/pods/d606971e-db1c-4d32-8d4b-1475d09fd05b/volumes" Jul 16 17:30:53.569271 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:30:53.569239 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve-ci-e2e-test/scheduler-v06-threshold-migration-test-kserve-prefill-b58fmzjxs"] Jul 16 17:30:53.569686 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:30:53.569672 2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d606971e-db1c-4d32-8d4b-1475d09fd05b" containerName="main" Jul 16 17:30:53.569759 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:30:53.569702 2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="d606971e-db1c-4d32-8d4b-1475d09fd05b" containerName="main" Jul 16 17:30:53.569759 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:30:53.569721 2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e3264472-595d-4dcb-97f8-92a98a63c073" containerName="main" Jul 16 17:30:53.569759 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:30:53.569726 2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="e3264472-595d-4dcb-97f8-92a98a63c073" containerName="main" Jul 16 17:30:53.569850 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:30:53.569810 2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="e3264472-595d-4dcb-97f8-92a98a63c073" containerName="main" Jul 16 17:30:53.569850 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:30:53.569820 2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="d606971e-db1c-4d32-8d4b-1475d09fd05b" containerName="main" Jul 16 17:30:53.574242 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:30:53.574227 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/scheduler-v06-threshold-migration-test-kserve-prefill-b58fmzjxs" Jul 16 17:30:53.576677 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:30:53.576650 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"default-dockercfg-sdm8r\"" Jul 16 17:30:53.577099 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:30:53.577070 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"openshift-service-ca.crt\"" Jul 16 17:30:53.577540 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:30:53.577519 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"scheduler-v06-threshold-migration-test-kserve-self-signed-certs\"" Jul 16 17:30:53.577743 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:30:53.577726 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"kube-root-ca.crt\"" Jul 16 17:30:53.615165 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:30:53.615140 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/scheduler-v06-threshold-migration-test-kserve-prefill-b58fmzjxs"] Jul 16 17:30:53.685978 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:30:53.685945 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/2770dce5-64c7-411c-8959-a7a23d77d643-tmp-dir\") pod \"scheduler-v06-threshold-migration-test-kserve-prefill-b58fmzjxs\" (UID: \"2770dce5-64c7-411c-8959-a7a23d77d643\") " pod="kserve-ci-e2e-test/scheduler-v06-threshold-migration-test-kserve-prefill-b58fmzjxs" Jul 16 17:30:53.686119 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:30:53.685997 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/2770dce5-64c7-411c-8959-a7a23d77d643-dshm\") pod \"scheduler-v06-threshold-migration-test-kserve-prefill-b58fmzjxs\" (UID: \"2770dce5-64c7-411c-8959-a7a23d77d643\") " pod="kserve-ci-e2e-test/scheduler-v06-threshold-migration-test-kserve-prefill-b58fmzjxs" Jul 16 17:30:53.686119 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:30:53.686027 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/2770dce5-64c7-411c-8959-a7a23d77d643-tls-certs\") pod \"scheduler-v06-threshold-migration-test-kserve-prefill-b58fmzjxs\" (UID: \"2770dce5-64c7-411c-8959-a7a23d77d643\") " pod="kserve-ci-e2e-test/scheduler-v06-threshold-migration-test-kserve-prefill-b58fmzjxs" Jul 16 17:30:53.686119 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:30:53.686089 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/2770dce5-64c7-411c-8959-a7a23d77d643-home\") pod \"scheduler-v06-threshold-migration-test-kserve-prefill-b58fmzjxs\" (UID: \"2770dce5-64c7-411c-8959-a7a23d77d643\") " pod="kserve-ci-e2e-test/scheduler-v06-threshold-migration-test-kserve-prefill-b58fmzjxs" Jul 16 17:30:53.686273 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:30:53.686122 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/2770dce5-64c7-411c-8959-a7a23d77d643-model-cache\") pod \"scheduler-v06-threshold-migration-test-kserve-prefill-b58fmzjxs\" (UID: \"2770dce5-64c7-411c-8959-a7a23d77d643\") " pod="kserve-ci-e2e-test/scheduler-v06-threshold-migration-test-kserve-prefill-b58fmzjxs" Jul 16 17:30:53.686273 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:30:53.686154 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wbgtj\" (UniqueName: \"kubernetes.io/projected/2770dce5-64c7-411c-8959-a7a23d77d643-kube-api-access-wbgtj\") pod \"scheduler-v06-threshold-migration-test-kserve-prefill-b58fmzjxs\" (UID: \"2770dce5-64c7-411c-8959-a7a23d77d643\") " pod="kserve-ci-e2e-test/scheduler-v06-threshold-migration-test-kserve-prefill-b58fmzjxs" Jul 16 17:30:53.787037 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:30:53.787002 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/2770dce5-64c7-411c-8959-a7a23d77d643-home\") pod \"scheduler-v06-threshold-migration-test-kserve-prefill-b58fmzjxs\" (UID: \"2770dce5-64c7-411c-8959-a7a23d77d643\") " pod="kserve-ci-e2e-test/scheduler-v06-threshold-migration-test-kserve-prefill-b58fmzjxs" Jul 16 17:30:53.787179 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:30:53.787046 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/2770dce5-64c7-411c-8959-a7a23d77d643-model-cache\") pod \"scheduler-v06-threshold-migration-test-kserve-prefill-b58fmzjxs\" (UID: \"2770dce5-64c7-411c-8959-a7a23d77d643\") " pod="kserve-ci-e2e-test/scheduler-v06-threshold-migration-test-kserve-prefill-b58fmzjxs" Jul 16 17:30:53.787179 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:30:53.787076 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wbgtj\" (UniqueName: \"kubernetes.io/projected/2770dce5-64c7-411c-8959-a7a23d77d643-kube-api-access-wbgtj\") pod \"scheduler-v06-threshold-migration-test-kserve-prefill-b58fmzjxs\" (UID: \"2770dce5-64c7-411c-8959-a7a23d77d643\") " pod="kserve-ci-e2e-test/scheduler-v06-threshold-migration-test-kserve-prefill-b58fmzjxs" Jul 16 17:30:53.787179 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:30:53.787114 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/2770dce5-64c7-411c-8959-a7a23d77d643-tmp-dir\") pod \"scheduler-v06-threshold-migration-test-kserve-prefill-b58fmzjxs\" (UID: \"2770dce5-64c7-411c-8959-a7a23d77d643\") " pod="kserve-ci-e2e-test/scheduler-v06-threshold-migration-test-kserve-prefill-b58fmzjxs" Jul 16 17:30:53.787179 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:30:53.787146 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/2770dce5-64c7-411c-8959-a7a23d77d643-dshm\") pod \"scheduler-v06-threshold-migration-test-kserve-prefill-b58fmzjxs\" (UID: \"2770dce5-64c7-411c-8959-a7a23d77d643\") " pod="kserve-ci-e2e-test/scheduler-v06-threshold-migration-test-kserve-prefill-b58fmzjxs" Jul 16 17:30:53.787179 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:30:53.787163 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/2770dce5-64c7-411c-8959-a7a23d77d643-tls-certs\") pod \"scheduler-v06-threshold-migration-test-kserve-prefill-b58fmzjxs\" (UID: \"2770dce5-64c7-411c-8959-a7a23d77d643\") " pod="kserve-ci-e2e-test/scheduler-v06-threshold-migration-test-kserve-prefill-b58fmzjxs" Jul 16 17:30:53.787509 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:30:53.787484 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/2770dce5-64c7-411c-8959-a7a23d77d643-model-cache\") pod \"scheduler-v06-threshold-migration-test-kserve-prefill-b58fmzjxs\" (UID: \"2770dce5-64c7-411c-8959-a7a23d77d643\") " pod="kserve-ci-e2e-test/scheduler-v06-threshold-migration-test-kserve-prefill-b58fmzjxs" Jul 16 17:30:53.787551 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:30:53.787498 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/2770dce5-64c7-411c-8959-a7a23d77d643-tmp-dir\") pod \"scheduler-v06-threshold-migration-test-kserve-prefill-b58fmzjxs\" (UID: \"2770dce5-64c7-411c-8959-a7a23d77d643\") " pod="kserve-ci-e2e-test/scheduler-v06-threshold-migration-test-kserve-prefill-b58fmzjxs" Jul 16 17:30:53.787601 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:30:53.787569 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/2770dce5-64c7-411c-8959-a7a23d77d643-home\") pod \"scheduler-v06-threshold-migration-test-kserve-prefill-b58fmzjxs\" (UID: \"2770dce5-64c7-411c-8959-a7a23d77d643\") " pod="kserve-ci-e2e-test/scheduler-v06-threshold-migration-test-kserve-prefill-b58fmzjxs" Jul 16 17:30:53.789475 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:30:53.789450 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/2770dce5-64c7-411c-8959-a7a23d77d643-dshm\") pod \"scheduler-v06-threshold-migration-test-kserve-prefill-b58fmzjxs\" (UID: \"2770dce5-64c7-411c-8959-a7a23d77d643\") " pod="kserve-ci-e2e-test/scheduler-v06-threshold-migration-test-kserve-prefill-b58fmzjxs" Jul 16 17:30:53.789738 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:30:53.789723 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/2770dce5-64c7-411c-8959-a7a23d77d643-tls-certs\") pod \"scheduler-v06-threshold-migration-test-kserve-prefill-b58fmzjxs\" (UID: \"2770dce5-64c7-411c-8959-a7a23d77d643\") " pod="kserve-ci-e2e-test/scheduler-v06-threshold-migration-test-kserve-prefill-b58fmzjxs" Jul 16 17:30:53.799654 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:30:53.799630 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wbgtj\" (UniqueName: \"kubernetes.io/projected/2770dce5-64c7-411c-8959-a7a23d77d643-kube-api-access-wbgtj\") pod \"scheduler-v06-threshold-migration-test-kserve-prefill-b58fmzjxs\" (UID: \"2770dce5-64c7-411c-8959-a7a23d77d643\") " pod="kserve-ci-e2e-test/scheduler-v06-threshold-migration-test-kserve-prefill-b58fmzjxs" Jul 16 17:30:53.885533 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:30:53.885483 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/scheduler-v06-threshold-migration-test-kserve-prefill-b58fmzjxs" Jul 16 17:30:53.974911 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:30:53.974879 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve-ci-e2e-test/scheduler-v06-threshold-migration-test-kserve-router-sched6ws58"] Jul 16 17:30:53.979247 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:30:53.979215 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/scheduler-v06-threshold-migration-test-kserve-router-sched6ws58" Jul 16 17:30:53.981823 ip-10-0-132-130 kubenswrapper[2578]: E0716 17:30:53.981787 2578 status_manager.go:1018] "Failed to get status for pod" err="pods \"scheduler-v06-threshold-migration-test-kserve-router-sched6ws58\" is forbidden: User \"system:node:ip-10-0-132-130.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kserve-ci-e2e-test\": no relationship found between node 'ip-10-0-132-130.ec2.internal' and this object" podUID="5d24a580-9131-4cb9-8f77-98d17e27f506" pod="kserve-ci-e2e-test/scheduler-v06-threshold-migration-test-kserve-router-sched6ws58" Jul 16 17:30:53.981922 ip-10-0-132-130 kubenswrapper[2578]: E0716 17:30:53.981819 2578 reflector.go:205] "Failed to watch" err="failed to list *v1.Secret: secrets \"scheduler-v06-threshold-migration-test-epp-sa-dockercfg-klsww\" is forbidden: User \"system:node:ip-10-0-132-130.ec2.internal\" cannot list resource \"secrets\" in API group \"\" in the namespace \"kserve-ci-e2e-test\": no relationship found between node 'ip-10-0-132-130.ec2.internal' and this object" logger="UnhandledError" reflector="object-\"kserve-ci-e2e-test\"/\"scheduler-v06-threshold-migration-test-epp-sa-dockercfg-klsww\"" type="*v1.Secret" Jul 16 17:30:53.997819 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:30:53.997796 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/scheduler-v06-threshold-migration-test-kserve-router-sched6ws58"] Jul 16 17:30:54.046417 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:30:54.046395 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/scheduler-v06-threshold-migration-test-kserve-prefill-b58fmzjxs"] Jul 16 17:30:54.049058 ip-10-0-132-130 kubenswrapper[2578]: W0716 17:30:54.049031 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2770dce5_64c7_411c_8959_a7a23d77d643.slice/crio-8456430952b1b4c1a907e177befe0d6b8d2ec96c7618eadb08a20476a842b410 WatchSource:0}: Error finding container 8456430952b1b4c1a907e177befe0d6b8d2ec96c7618eadb08a20476a842b410: Status 404 returned error can't find the container with id 8456430952b1b4c1a907e177befe0d6b8d2ec96c7618eadb08a20476a842b410 Jul 16 17:30:54.089102 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:30:54.089073 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tcg5w\" (UniqueName: \"kubernetes.io/projected/5d24a580-9131-4cb9-8f77-98d17e27f506-kube-api-access-tcg5w\") pod \"scheduler-v06-threshold-migration-test-kserve-router-sched6ws58\" (UID: \"5d24a580-9131-4cb9-8f77-98d17e27f506\") " pod="kserve-ci-e2e-test/scheduler-v06-threshold-migration-test-kserve-router-sched6ws58" Jul 16 17:30:54.089194 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:30:54.089130 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tokenizer-uds\" (UniqueName: \"kubernetes.io/empty-dir/5d24a580-9131-4cb9-8f77-98d17e27f506-tokenizer-uds\") pod \"scheduler-v06-threshold-migration-test-kserve-router-sched6ws58\" (UID: \"5d24a580-9131-4cb9-8f77-98d17e27f506\") " pod="kserve-ci-e2e-test/scheduler-v06-threshold-migration-test-kserve-router-sched6ws58" Jul 16 17:30:54.089281 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:30:54.089265 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/5d24a580-9131-4cb9-8f77-98d17e27f506-tls-certs\") pod \"scheduler-v06-threshold-migration-test-kserve-router-sched6ws58\" (UID: \"5d24a580-9131-4cb9-8f77-98d17e27f506\") " pod="kserve-ci-e2e-test/scheduler-v06-threshold-migration-test-kserve-router-sched6ws58" Jul 16 17:30:54.128362 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:30:54.128325 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/scheduler-v06-threshold-migration-test-kserve-prefill-b58fmzjxs" event={"ID":"2770dce5-64c7-411c-8959-a7a23d77d643","Type":"ContainerStarted","Data":"c019304549bec3ed5731bfab37f67ea035cacf03b5d15a007ec1f621ea041739"} Jul 16 17:30:54.128495 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:30:54.128385 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/scheduler-v06-threshold-migration-test-kserve-prefill-b58fmzjxs" event={"ID":"2770dce5-64c7-411c-8959-a7a23d77d643","Type":"ContainerStarted","Data":"8456430952b1b4c1a907e177befe0d6b8d2ec96c7618eadb08a20476a842b410"} Jul 16 17:30:54.190622 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:30:54.190548 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tcg5w\" (UniqueName: \"kubernetes.io/projected/5d24a580-9131-4cb9-8f77-98d17e27f506-kube-api-access-tcg5w\") pod \"scheduler-v06-threshold-migration-test-kserve-router-sched6ws58\" (UID: \"5d24a580-9131-4cb9-8f77-98d17e27f506\") " pod="kserve-ci-e2e-test/scheduler-v06-threshold-migration-test-kserve-router-sched6ws58" Jul 16 17:30:54.190622 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:30:54.190588 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tokenizer-uds\" (UniqueName: \"kubernetes.io/empty-dir/5d24a580-9131-4cb9-8f77-98d17e27f506-tokenizer-uds\") pod \"scheduler-v06-threshold-migration-test-kserve-router-sched6ws58\" (UID: \"5d24a580-9131-4cb9-8f77-98d17e27f506\") " pod="kserve-ci-e2e-test/scheduler-v06-threshold-migration-test-kserve-router-sched6ws58" Jul 16 17:30:54.190837 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:30:54.190670 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/5d24a580-9131-4cb9-8f77-98d17e27f506-tls-certs\") pod \"scheduler-v06-threshold-migration-test-kserve-router-sched6ws58\" (UID: \"5d24a580-9131-4cb9-8f77-98d17e27f506\") " pod="kserve-ci-e2e-test/scheduler-v06-threshold-migration-test-kserve-router-sched6ws58" Jul 16 17:30:54.191043 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:30:54.191024 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tokenizer-uds\" (UniqueName: \"kubernetes.io/empty-dir/5d24a580-9131-4cb9-8f77-98d17e27f506-tokenizer-uds\") pod \"scheduler-v06-threshold-migration-test-kserve-router-sched6ws58\" (UID: \"5d24a580-9131-4cb9-8f77-98d17e27f506\") " pod="kserve-ci-e2e-test/scheduler-v06-threshold-migration-test-kserve-router-sched6ws58" Jul 16 17:30:54.193190 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:30:54.193162 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/5d24a580-9131-4cb9-8f77-98d17e27f506-tls-certs\") pod \"scheduler-v06-threshold-migration-test-kserve-router-sched6ws58\" (UID: \"5d24a580-9131-4cb9-8f77-98d17e27f506\") " pod="kserve-ci-e2e-test/scheduler-v06-threshold-migration-test-kserve-router-sched6ws58" Jul 16 17:30:54.210607 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:30:54.210580 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tcg5w\" (UniqueName: \"kubernetes.io/projected/5d24a580-9131-4cb9-8f77-98d17e27f506-kube-api-access-tcg5w\") pod \"scheduler-v06-threshold-migration-test-kserve-router-sched6ws58\" (UID: \"5d24a580-9131-4cb9-8f77-98d17e27f506\") " pod="kserve-ci-e2e-test/scheduler-v06-threshold-migration-test-kserve-router-sched6ws58" Jul 16 17:30:54.216542 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:30:54.216500 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve-ci-e2e-test/scheduler-v06-threshold-migration-test-kserve-prefill-b58fmzjxs" podStartSLOduration=1.216486241 podStartE2EDuration="1.216486241s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 17:30:53 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 17:30:54.214198551 +0000 UTC m=+998.700042463" watchObservedRunningTime="2026-07-16 17:30:54.216486241 +0000 UTC m=+998.702330130" Jul 16 17:30:54.886301 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:30:54.886266 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/scheduler-v06-threshold-migration-test-kserve-prefill-b58fmzjxs" Jul 16 17:30:55.291787 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:30:55.291754 2578 kubelet_pods.go:1085] "Unable to retrieve pull secret, the image pull may not succeed." pod="kserve-ci-e2e-test/scheduler-v06-threshold-migration-test-kserve-router-sched6ws58" secret="" err="failed to sync secret cache: timed out waiting for the condition" Jul 16 17:30:55.291942 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:30:55.291825 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/scheduler-v06-threshold-migration-test-kserve-router-sched6ws58" Jul 16 17:30:55.441963 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:30:55.441933 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/scheduler-v06-threshold-migration-test-kserve-router-sched6ws58"] Jul 16 17:30:55.445353 ip-10-0-132-130 kubenswrapper[2578]: W0716 17:30:55.445320 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5d24a580_9131_4cb9_8f77_98d17e27f506.slice/crio-1403c6839ed50d021b9270814d57d1fda26f554aca1ee83fc9b28799196d8ec4 WatchSource:0}: Error finding container 1403c6839ed50d021b9270814d57d1fda26f554aca1ee83fc9b28799196d8ec4: Status 404 returned error can't find the container with id 1403c6839ed50d021b9270814d57d1fda26f554aca1ee83fc9b28799196d8ec4 Jul 16 17:30:55.503234 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:30:55.503213 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"scheduler-v06-threshold-migration-test-epp-sa-dockercfg-klsww\"" Jul 16 17:30:56.148761 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:30:56.148721 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/scheduler-v06-threshold-migration-test-kserve-router-sched6ws58" event={"ID":"5d24a580-9131-4cb9-8f77-98d17e27f506","Type":"ContainerStarted","Data":"14da275592781d882619f7c557dbc961a256114696dee37eafee0bb58fbf706f"} Jul 16 17:30:56.149198 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:30:56.148800 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/scheduler-v06-threshold-migration-test-kserve-router-sched6ws58" event={"ID":"5d24a580-9131-4cb9-8f77-98d17e27f506","Type":"ContainerStarted","Data":"1403c6839ed50d021b9270814d57d1fda26f554aca1ee83fc9b28799196d8ec4"} Jul 16 17:30:56.176428 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:30:56.176378 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve-ci-e2e-test/scheduler-v06-threshold-migration-test-kserve-router-sched6ws58" podStartSLOduration=3.176364449 podStartE2EDuration="3.176364449s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 17:30:53 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 17:30:56.173253619 +0000 UTC m=+1000.659097533" watchObservedRunningTime="2026-07-16 17:30:56.176364449 +0000 UTC m=+1000.662208360" Jul 16 17:30:57.154758 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:30:57.154727 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/scheduler-v06-threshold-migration-test-kserve-router-sched6ws58" Jul 16 17:31:03.886663 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:31:03.886626 2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kserve-ci-e2e-test/scheduler-v06-threshold-migration-test-kserve-prefill-b58fmzjxs" Jul 16 17:31:03.900060 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:31:03.900030 2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kserve-ci-e2e-test/scheduler-v06-threshold-migration-test-kserve-prefill-b58fmzjxs" Jul 16 17:31:04.205180 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:31:04.205091 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/scheduler-v06-threshold-migration-test-kserve-prefill-b58fmzjxs" Jul 16 17:31:28.161979 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:31:28.161949 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/scheduler-v06-threshold-migration-test-kserve-router-sched6ws58" Jul 16 17:31:30.742720 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:31:30.742666 2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/scheduler-v06-threshold-migration-test-kserve-prefill-b58fmzjxs"] Jul 16 17:31:30.750330 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:31:30.750300 2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/scheduler-v06-threshold-migration-test-kserve-router-sched6ws58"] Jul 16 17:31:30.752728 ip-10-0-132-130 kubenswrapper[2578]: E0716 17:31:30.752663 2578 log.go:32] "ExecSync cmd from runtime service failed" err=< Jul 16 17:31:30.752728 ip-10-0-132-130 kubenswrapper[2578]: rpc error: code = Unknown desc = command error: executable file `/bin/sleep` not found: No such file or directory Jul 16 17:31:30.752728 ip-10-0-132-130 kubenswrapper[2578]: , stdout: , stderr: , exit code -1 Jul 16 17:31:30.752728 ip-10-0-132-130 kubenswrapper[2578]: > containerID="c019304549bec3ed5731bfab37f67ea035cacf03b5d15a007ec1f621ea041739" cmd=["/bin/sleep","15"] Jul 16 17:31:30.753016 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:31:30.752749 2578 handlers.go:82] "Exec lifecycle hook for Container in Pod failed" execCommand=["/bin/sleep","15"] containerName="main" pod="kserve-ci-e2e-test/scheduler-v06-threshold-migration-test-kserve-prefill-b58fmzjxs" message="" err=< Jul 16 17:31:30.753016 ip-10-0-132-130 kubenswrapper[2578]: rpc error: code = Unknown desc = command error: executable file `/bin/sleep` not found: No such file or directory Jul 16 17:31:30.753016 ip-10-0-132-130 kubenswrapper[2578]: , stdout: , stderr: , exit code -1 Jul 16 17:31:30.753016 ip-10-0-132-130 kubenswrapper[2578]: > Jul 16 17:31:30.753016 ip-10-0-132-130 kubenswrapper[2578]: E0716 17:31:30.752767 2578 kuberuntime_container.go:755] "PreStop hook failed" err=< Jul 16 17:31:30.753016 ip-10-0-132-130 kubenswrapper[2578]: rpc error: code = Unknown desc = command error: executable file `/bin/sleep` not found: No such file or directory Jul 16 17:31:30.753016 ip-10-0-132-130 kubenswrapper[2578]: , stdout: , stderr: , exit code -1 Jul 16 17:31:30.753016 ip-10-0-132-130 kubenswrapper[2578]: > pod="kserve-ci-e2e-test/scheduler-v06-threshold-migration-test-kserve-prefill-b58fmzjxs" podUID="2770dce5-64c7-411c-8959-a7a23d77d643" containerName="main" containerID="cri-o://c019304549bec3ed5731bfab37f67ea035cacf03b5d15a007ec1f621ea041739" Jul 16 17:31:30.753016 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:31:30.752810 2578 kuberuntime_container.go:873] "Killing container with a grace period" pod="kserve-ci-e2e-test/scheduler-v06-threshold-migration-test-kserve-prefill-b58fmzjxs" podUID="2770dce5-64c7-411c-8959-a7a23d77d643" containerName="main" containerID="cri-o://c019304549bec3ed5731bfab37f67ea035cacf03b5d15a007ec1f621ea041739" gracePeriod=60 Jul 16 17:31:31.113529 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:31:31.113504 2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/scheduler-v06-threshold-migration-test-kserve-prefill-b58fmzjxs" Jul 16 17:31:31.219363 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:31:31.219331 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/2770dce5-64c7-411c-8959-a7a23d77d643-model-cache\") pod \"2770dce5-64c7-411c-8959-a7a23d77d643\" (UID: \"2770dce5-64c7-411c-8959-a7a23d77d643\") " Jul 16 17:31:31.219363 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:31:31.219370 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/2770dce5-64c7-411c-8959-a7a23d77d643-dshm\") pod \"2770dce5-64c7-411c-8959-a7a23d77d643\" (UID: \"2770dce5-64c7-411c-8959-a7a23d77d643\") " Jul 16 17:31:31.219618 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:31:31.219390 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-wbgtj\" (UniqueName: \"kubernetes.io/projected/2770dce5-64c7-411c-8959-a7a23d77d643-kube-api-access-wbgtj\") pod \"2770dce5-64c7-411c-8959-a7a23d77d643\" (UID: \"2770dce5-64c7-411c-8959-a7a23d77d643\") " Jul 16 17:31:31.219618 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:31:31.219427 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/2770dce5-64c7-411c-8959-a7a23d77d643-home\") pod \"2770dce5-64c7-411c-8959-a7a23d77d643\" (UID: \"2770dce5-64c7-411c-8959-a7a23d77d643\") " Jul 16 17:31:31.219618 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:31:31.219470 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/2770dce5-64c7-411c-8959-a7a23d77d643-tls-certs\") pod \"2770dce5-64c7-411c-8959-a7a23d77d643\" (UID: \"2770dce5-64c7-411c-8959-a7a23d77d643\") " Jul 16 17:31:31.219618 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:31:31.219511 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/2770dce5-64c7-411c-8959-a7a23d77d643-tmp-dir\") pod \"2770dce5-64c7-411c-8959-a7a23d77d643\" (UID: \"2770dce5-64c7-411c-8959-a7a23d77d643\") " Jul 16 17:31:31.219839 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:31:31.219627 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2770dce5-64c7-411c-8959-a7a23d77d643-model-cache" (OuterVolumeSpecName: "model-cache") pod "2770dce5-64c7-411c-8959-a7a23d77d643" (UID: "2770dce5-64c7-411c-8959-a7a23d77d643"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 16 17:31:31.219839 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:31:31.219761 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2770dce5-64c7-411c-8959-a7a23d77d643-home" (OuterVolumeSpecName: "home") pod "2770dce5-64c7-411c-8959-a7a23d77d643" (UID: "2770dce5-64c7-411c-8959-a7a23d77d643"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 16 17:31:31.219839 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:31:31.219797 2578 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/2770dce5-64c7-411c-8959-a7a23d77d643-model-cache\") on node \"ip-10-0-132-130.ec2.internal\" DevicePath \"\"" Jul 16 17:31:31.219940 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:31:31.219830 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2770dce5-64c7-411c-8959-a7a23d77d643-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "2770dce5-64c7-411c-8959-a7a23d77d643" (UID: "2770dce5-64c7-411c-8959-a7a23d77d643"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 16 17:31:31.221506 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:31:31.221475 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2770dce5-64c7-411c-8959-a7a23d77d643-dshm" (OuterVolumeSpecName: "dshm") pod "2770dce5-64c7-411c-8959-a7a23d77d643" (UID: "2770dce5-64c7-411c-8959-a7a23d77d643"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 16 17:31:31.221506 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:31:31.221497 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/2770dce5-64c7-411c-8959-a7a23d77d643-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "2770dce5-64c7-411c-8959-a7a23d77d643" (UID: "2770dce5-64c7-411c-8959-a7a23d77d643"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 16 17:31:31.221651 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:31:31.221500 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2770dce5-64c7-411c-8959-a7a23d77d643-kube-api-access-wbgtj" (OuterVolumeSpecName: "kube-api-access-wbgtj") pod "2770dce5-64c7-411c-8959-a7a23d77d643" (UID: "2770dce5-64c7-411c-8959-a7a23d77d643"). InnerVolumeSpecName "kube-api-access-wbgtj". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 16 17:31:31.320913 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:31:31.320885 2578 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/2770dce5-64c7-411c-8959-a7a23d77d643-tls-certs\") on node \"ip-10-0-132-130.ec2.internal\" DevicePath \"\"" Jul 16 17:31:31.320913 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:31:31.320911 2578 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/2770dce5-64c7-411c-8959-a7a23d77d643-tmp-dir\") on node \"ip-10-0-132-130.ec2.internal\" DevicePath \"\"" Jul 16 17:31:31.320913 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:31:31.320919 2578 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/2770dce5-64c7-411c-8959-a7a23d77d643-dshm\") on node \"ip-10-0-132-130.ec2.internal\" DevicePath \"\"" Jul 16 17:31:31.321137 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:31:31.320927 2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-wbgtj\" (UniqueName: \"kubernetes.io/projected/2770dce5-64c7-411c-8959-a7a23d77d643-kube-api-access-wbgtj\") on node \"ip-10-0-132-130.ec2.internal\" DevicePath \"\"" Jul 16 17:31:31.321137 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:31:31.320937 2578 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/2770dce5-64c7-411c-8959-a7a23d77d643-home\") on node \"ip-10-0-132-130.ec2.internal\" DevicePath \"\"" Jul 16 17:31:31.344933 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:31:31.344903 2578 generic.go:356] "Generic (PLEG): container finished" podID="2770dce5-64c7-411c-8959-a7a23d77d643" containerID="c019304549bec3ed5731bfab37f67ea035cacf03b5d15a007ec1f621ea041739" exitCode=0 Jul 16 17:31:31.345073 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:31:31.344952 2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/scheduler-v06-threshold-migration-test-kserve-prefill-b58fmzjxs" Jul 16 17:31:31.345073 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:31:31.344982 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/scheduler-v06-threshold-migration-test-kserve-prefill-b58fmzjxs" event={"ID":"2770dce5-64c7-411c-8959-a7a23d77d643","Type":"ContainerDied","Data":"c019304549bec3ed5731bfab37f67ea035cacf03b5d15a007ec1f621ea041739"} Jul 16 17:31:31.345073 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:31:31.345034 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/scheduler-v06-threshold-migration-test-kserve-prefill-b58fmzjxs" event={"ID":"2770dce5-64c7-411c-8959-a7a23d77d643","Type":"ContainerDied","Data":"8456430952b1b4c1a907e177befe0d6b8d2ec96c7618eadb08a20476a842b410"} Jul 16 17:31:31.345073 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:31:31.345058 2578 scope.go:117] "RemoveContainer" containerID="c019304549bec3ed5731bfab37f67ea035cacf03b5d15a007ec1f621ea041739" Jul 16 17:31:31.355510 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:31:31.355492 2578 scope.go:117] "RemoveContainer" containerID="c019304549bec3ed5731bfab37f67ea035cacf03b5d15a007ec1f621ea041739" Jul 16 17:31:31.355826 ip-10-0-132-130 kubenswrapper[2578]: E0716 17:31:31.355805 2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c019304549bec3ed5731bfab37f67ea035cacf03b5d15a007ec1f621ea041739\": container with ID starting with c019304549bec3ed5731bfab37f67ea035cacf03b5d15a007ec1f621ea041739 not found: ID does not exist" containerID="c019304549bec3ed5731bfab37f67ea035cacf03b5d15a007ec1f621ea041739" Jul 16 17:31:31.355904 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:31:31.355832 2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c019304549bec3ed5731bfab37f67ea035cacf03b5d15a007ec1f621ea041739"} err="failed to get container status \"c019304549bec3ed5731bfab37f67ea035cacf03b5d15a007ec1f621ea041739\": rpc error: code = NotFound desc = could not find container \"c019304549bec3ed5731bfab37f67ea035cacf03b5d15a007ec1f621ea041739\": container with ID starting with c019304549bec3ed5731bfab37f67ea035cacf03b5d15a007ec1f621ea041739 not found: ID does not exist" Jul 16 17:31:31.394053 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:31:31.394030 2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/scheduler-v06-threshold-migration-test-kserve-prefill-b58fmzjxs"] Jul 16 17:31:31.406804 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:31:31.406779 2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kserve-ci-e2e-test/scheduler-v06-threshold-migration-test-kserve-prefill-b58fmzjxs"] Jul 16 17:31:32.215521 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:31:32.215489 2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="2770dce5-64c7-411c-8959-a7a23d77d643" path="/var/lib/kubelet/pods/2770dce5-64c7-411c-8959-a7a23d77d643/volumes" Jul 16 17:31:38.161655 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:31:38.161618 2578 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/scheduler-v06-threshold-migration-test-kserve-router-sched6ws58" podUID="5d24a580-9131-4cb9-8f77-98d17e27f506" containerName="main" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")" Jul 16 17:31:45.773633 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:31:45.773574 2578 kuberuntime_container.go:873] "Killing container with a grace period" pod="kserve-ci-e2e-test/scheduler-v06-threshold-migration-test-kserve-router-sched6ws58" podUID="5d24a580-9131-4cb9-8f77-98d17e27f506" containerName="main" containerID="cri-o://14da275592781d882619f7c557dbc961a256114696dee37eafee0bb58fbf706f" gracePeriod=45 Jul 16 17:31:46.034198 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:31:46.034143 2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/scheduler-v06-threshold-migration-test-kserve-router-sched6ws58" Jul 16 17:31:46.150776 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:31:46.150743 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-tcg5w\" (UniqueName: \"kubernetes.io/projected/5d24a580-9131-4cb9-8f77-98d17e27f506-kube-api-access-tcg5w\") pod \"5d24a580-9131-4cb9-8f77-98d17e27f506\" (UID: \"5d24a580-9131-4cb9-8f77-98d17e27f506\") " Jul 16 17:31:46.150929 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:31:46.150782 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/5d24a580-9131-4cb9-8f77-98d17e27f506-tls-certs\") pod \"5d24a580-9131-4cb9-8f77-98d17e27f506\" (UID: \"5d24a580-9131-4cb9-8f77-98d17e27f506\") " Jul 16 17:31:46.150929 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:31:46.150839 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tokenizer-uds\" (UniqueName: \"kubernetes.io/empty-dir/5d24a580-9131-4cb9-8f77-98d17e27f506-tokenizer-uds\") pod \"5d24a580-9131-4cb9-8f77-98d17e27f506\" (UID: \"5d24a580-9131-4cb9-8f77-98d17e27f506\") " Jul 16 17:31:46.151192 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:31:46.151161 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5d24a580-9131-4cb9-8f77-98d17e27f506-tokenizer-uds" (OuterVolumeSpecName: "tokenizer-uds") pod "5d24a580-9131-4cb9-8f77-98d17e27f506" (UID: "5d24a580-9131-4cb9-8f77-98d17e27f506"). InnerVolumeSpecName "tokenizer-uds". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 16 17:31:46.152765 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:31:46.152743 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5d24a580-9131-4cb9-8f77-98d17e27f506-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "5d24a580-9131-4cb9-8f77-98d17e27f506" (UID: "5d24a580-9131-4cb9-8f77-98d17e27f506"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 16 17:31:46.152848 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:31:46.152742 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5d24a580-9131-4cb9-8f77-98d17e27f506-kube-api-access-tcg5w" (OuterVolumeSpecName: "kube-api-access-tcg5w") pod "5d24a580-9131-4cb9-8f77-98d17e27f506" (UID: "5d24a580-9131-4cb9-8f77-98d17e27f506"). InnerVolumeSpecName "kube-api-access-tcg5w". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 16 17:31:46.251591 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:31:46.251564 2578 reconciler_common.go:299] "Volume detached for volume \"tokenizer-uds\" (UniqueName: \"kubernetes.io/empty-dir/5d24a580-9131-4cb9-8f77-98d17e27f506-tokenizer-uds\") on node \"ip-10-0-132-130.ec2.internal\" DevicePath \"\"" Jul 16 17:31:46.251591 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:31:46.251587 2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-tcg5w\" (UniqueName: \"kubernetes.io/projected/5d24a580-9131-4cb9-8f77-98d17e27f506-kube-api-access-tcg5w\") on node \"ip-10-0-132-130.ec2.internal\" DevicePath \"\"" Jul 16 17:31:46.251591 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:31:46.251597 2578 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/5d24a580-9131-4cb9-8f77-98d17e27f506-tls-certs\") on node \"ip-10-0-132-130.ec2.internal\" DevicePath \"\"" Jul 16 17:31:46.429279 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:31:46.429244 2578 generic.go:356] "Generic (PLEG): container finished" podID="5d24a580-9131-4cb9-8f77-98d17e27f506" containerID="14da275592781d882619f7c557dbc961a256114696dee37eafee0bb58fbf706f" exitCode=0 Jul 16 17:31:46.429446 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:31:46.429291 2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/scheduler-v06-threshold-migration-test-kserve-router-sched6ws58" Jul 16 17:31:46.429446 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:31:46.429323 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/scheduler-v06-threshold-migration-test-kserve-router-sched6ws58" event={"ID":"5d24a580-9131-4cb9-8f77-98d17e27f506","Type":"ContainerDied","Data":"14da275592781d882619f7c557dbc961a256114696dee37eafee0bb58fbf706f"} Jul 16 17:31:46.429446 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:31:46.429369 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/scheduler-v06-threshold-migration-test-kserve-router-sched6ws58" event={"ID":"5d24a580-9131-4cb9-8f77-98d17e27f506","Type":"ContainerDied","Data":"1403c6839ed50d021b9270814d57d1fda26f554aca1ee83fc9b28799196d8ec4"} Jul 16 17:31:46.429446 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:31:46.429389 2578 scope.go:117] "RemoveContainer" containerID="14da275592781d882619f7c557dbc961a256114696dee37eafee0bb58fbf706f" Jul 16 17:31:46.438294 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:31:46.438278 2578 scope.go:117] "RemoveContainer" containerID="14da275592781d882619f7c557dbc961a256114696dee37eafee0bb58fbf706f" Jul 16 17:31:46.438541 ip-10-0-132-130 kubenswrapper[2578]: E0716 17:31:46.438522 2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"14da275592781d882619f7c557dbc961a256114696dee37eafee0bb58fbf706f\": container with ID starting with 14da275592781d882619f7c557dbc961a256114696dee37eafee0bb58fbf706f not found: ID does not exist" containerID="14da275592781d882619f7c557dbc961a256114696dee37eafee0bb58fbf706f" Jul 16 17:31:46.438594 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:31:46.438548 2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"14da275592781d882619f7c557dbc961a256114696dee37eafee0bb58fbf706f"} err="failed to get container status \"14da275592781d882619f7c557dbc961a256114696dee37eafee0bb58fbf706f\": rpc error: code = NotFound desc = could not find container \"14da275592781d882619f7c557dbc961a256114696dee37eafee0bb58fbf706f\": container with ID starting with 14da275592781d882619f7c557dbc961a256114696dee37eafee0bb58fbf706f not found: ID does not exist" Jul 16 17:31:46.452265 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:31:46.452243 2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/scheduler-v06-threshold-migration-test-kserve-router-sched6ws58"] Jul 16 17:31:46.454636 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:31:46.454613 2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kserve-ci-e2e-test/scheduler-v06-threshold-migration-test-kserve-router-sched6ws58"] Jul 16 17:31:48.215268 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:31:48.215237 2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="5d24a580-9131-4cb9-8f77-98d17e27f506" path="/var/lib/kubelet/pods/5d24a580-9131-4cb9-8f77-98d17e27f506/volumes" Jul 16 17:31:53.389841 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:31:53.389807 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve-ci-e2e-test/precise-prefix-cache-test-kserve-6d6c8b5f7c-kgjrz"] Jul 16 17:31:53.390273 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:31:53.390212 2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2770dce5-64c7-411c-8959-a7a23d77d643" containerName="main" Jul 16 17:31:53.390273 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:31:53.390222 2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="2770dce5-64c7-411c-8959-a7a23d77d643" containerName="main" Jul 16 17:31:53.390273 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:31:53.390231 2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5d24a580-9131-4cb9-8f77-98d17e27f506" containerName="main" Jul 16 17:31:53.390273 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:31:53.390236 2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="5d24a580-9131-4cb9-8f77-98d17e27f506" containerName="main" Jul 16 17:31:53.390471 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:31:53.390327 2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="2770dce5-64c7-411c-8959-a7a23d77d643" containerName="main" Jul 16 17:31:53.390471 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:31:53.390338 2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="5d24a580-9131-4cb9-8f77-98d17e27f506" containerName="main" Jul 16 17:31:53.395271 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:31:53.395249 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/precise-prefix-cache-test-kserve-6d6c8b5f7c-kgjrz" Jul 16 17:31:53.399739 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:31:53.399718 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"seaweedfs-s3-creds\"" Jul 16 17:31:53.399828 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:31:53.399804 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"openshift-service-ca.crt\"" Jul 16 17:31:53.399894 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:31:53.399876 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"precise-prefix-cache-test-kserve-self-signed-certs\"" Jul 16 17:31:53.400309 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:31:53.400293 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"default-dockercfg-sdm8r\"" Jul 16 17:31:53.400361 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:31:53.400345 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"kube-root-ca.crt\"" Jul 16 17:31:53.407006 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:31:53.406984 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/precise-prefix-cache-test-kserve-6d6c8b5f7c-kgjrz"] Jul 16 17:31:53.512770 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:31:53.512744 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/dd594272-3fa8-4587-9386-600307eb243d-tmp-dir\") pod \"precise-prefix-cache-test-kserve-6d6c8b5f7c-kgjrz\" (UID: \"dd594272-3fa8-4587-9386-600307eb243d\") " pod="kserve-ci-e2e-test/precise-prefix-cache-test-kserve-6d6c8b5f7c-kgjrz" Jul 16 17:31:53.513140 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:31:53.512784 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/dd594272-3fa8-4587-9386-600307eb243d-model-cache\") pod \"precise-prefix-cache-test-kserve-6d6c8b5f7c-kgjrz\" (UID: \"dd594272-3fa8-4587-9386-600307eb243d\") " pod="kserve-ci-e2e-test/precise-prefix-cache-test-kserve-6d6c8b5f7c-kgjrz" Jul 16 17:31:53.513140 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:31:53.512806 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qdhzw\" (UniqueName: \"kubernetes.io/projected/dd594272-3fa8-4587-9386-600307eb243d-kube-api-access-qdhzw\") pod \"precise-prefix-cache-test-kserve-6d6c8b5f7c-kgjrz\" (UID: \"dd594272-3fa8-4587-9386-600307eb243d\") " pod="kserve-ci-e2e-test/precise-prefix-cache-test-kserve-6d6c8b5f7c-kgjrz" Jul 16 17:31:53.513140 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:31:53.512924 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/dd594272-3fa8-4587-9386-600307eb243d-tls-certs\") pod \"precise-prefix-cache-test-kserve-6d6c8b5f7c-kgjrz\" (UID: \"dd594272-3fa8-4587-9386-600307eb243d\") " pod="kserve-ci-e2e-test/precise-prefix-cache-test-kserve-6d6c8b5f7c-kgjrz" Jul 16 17:31:53.513140 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:31:53.512961 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/dd594272-3fa8-4587-9386-600307eb243d-home\") pod \"precise-prefix-cache-test-kserve-6d6c8b5f7c-kgjrz\" (UID: \"dd594272-3fa8-4587-9386-600307eb243d\") " pod="kserve-ci-e2e-test/precise-prefix-cache-test-kserve-6d6c8b5f7c-kgjrz" Jul 16 17:31:53.513140 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:31:53.512979 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/dd594272-3fa8-4587-9386-600307eb243d-dshm\") pod \"precise-prefix-cache-test-kserve-6d6c8b5f7c-kgjrz\" (UID: \"dd594272-3fa8-4587-9386-600307eb243d\") " pod="kserve-ci-e2e-test/precise-prefix-cache-test-kserve-6d6c8b5f7c-kgjrz" Jul 16 17:31:53.513140 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:31:53.513026 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/dd594272-3fa8-4587-9386-600307eb243d-kserve-provision-location\") pod \"precise-prefix-cache-test-kserve-6d6c8b5f7c-kgjrz\" (UID: \"dd594272-3fa8-4587-9386-600307eb243d\") " pod="kserve-ci-e2e-test/precise-prefix-cache-test-kserve-6d6c8b5f7c-kgjrz" Jul 16 17:31:53.614109 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:31:53.614073 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/dd594272-3fa8-4587-9386-600307eb243d-model-cache\") pod \"precise-prefix-cache-test-kserve-6d6c8b5f7c-kgjrz\" (UID: \"dd594272-3fa8-4587-9386-600307eb243d\") " pod="kserve-ci-e2e-test/precise-prefix-cache-test-kserve-6d6c8b5f7c-kgjrz" Jul 16 17:31:53.614109 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:31:53.614111 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qdhzw\" (UniqueName: \"kubernetes.io/projected/dd594272-3fa8-4587-9386-600307eb243d-kube-api-access-qdhzw\") pod \"precise-prefix-cache-test-kserve-6d6c8b5f7c-kgjrz\" (UID: \"dd594272-3fa8-4587-9386-600307eb243d\") " pod="kserve-ci-e2e-test/precise-prefix-cache-test-kserve-6d6c8b5f7c-kgjrz" Jul 16 17:31:53.614336 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:31:53.614216 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/dd594272-3fa8-4587-9386-600307eb243d-tls-certs\") pod \"precise-prefix-cache-test-kserve-6d6c8b5f7c-kgjrz\" (UID: \"dd594272-3fa8-4587-9386-600307eb243d\") " pod="kserve-ci-e2e-test/precise-prefix-cache-test-kserve-6d6c8b5f7c-kgjrz" Jul 16 17:31:53.614336 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:31:53.614243 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/dd594272-3fa8-4587-9386-600307eb243d-home\") pod \"precise-prefix-cache-test-kserve-6d6c8b5f7c-kgjrz\" (UID: \"dd594272-3fa8-4587-9386-600307eb243d\") " pod="kserve-ci-e2e-test/precise-prefix-cache-test-kserve-6d6c8b5f7c-kgjrz" Jul 16 17:31:53.614336 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:31:53.614262 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/dd594272-3fa8-4587-9386-600307eb243d-dshm\") pod \"precise-prefix-cache-test-kserve-6d6c8b5f7c-kgjrz\" (UID: \"dd594272-3fa8-4587-9386-600307eb243d\") " pod="kserve-ci-e2e-test/precise-prefix-cache-test-kserve-6d6c8b5f7c-kgjrz" Jul 16 17:31:53.614336 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:31:53.614277 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/dd594272-3fa8-4587-9386-600307eb243d-kserve-provision-location\") pod \"precise-prefix-cache-test-kserve-6d6c8b5f7c-kgjrz\" (UID: \"dd594272-3fa8-4587-9386-600307eb243d\") " pod="kserve-ci-e2e-test/precise-prefix-cache-test-kserve-6d6c8b5f7c-kgjrz" Jul 16 17:31:53.614555 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:31:53.614373 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/dd594272-3fa8-4587-9386-600307eb243d-tmp-dir\") pod \"precise-prefix-cache-test-kserve-6d6c8b5f7c-kgjrz\" (UID: \"dd594272-3fa8-4587-9386-600307eb243d\") " pod="kserve-ci-e2e-test/precise-prefix-cache-test-kserve-6d6c8b5f7c-kgjrz" Jul 16 17:31:53.614555 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:31:53.614490 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/dd594272-3fa8-4587-9386-600307eb243d-model-cache\") pod \"precise-prefix-cache-test-kserve-6d6c8b5f7c-kgjrz\" (UID: \"dd594272-3fa8-4587-9386-600307eb243d\") " pod="kserve-ci-e2e-test/precise-prefix-cache-test-kserve-6d6c8b5f7c-kgjrz" Jul 16 17:31:53.614664 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:31:53.614610 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/dd594272-3fa8-4587-9386-600307eb243d-home\") pod \"precise-prefix-cache-test-kserve-6d6c8b5f7c-kgjrz\" (UID: \"dd594272-3fa8-4587-9386-600307eb243d\") " pod="kserve-ci-e2e-test/precise-prefix-cache-test-kserve-6d6c8b5f7c-kgjrz" Jul 16 17:31:53.614751 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:31:53.614721 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/dd594272-3fa8-4587-9386-600307eb243d-tmp-dir\") pod \"precise-prefix-cache-test-kserve-6d6c8b5f7c-kgjrz\" (UID: \"dd594272-3fa8-4587-9386-600307eb243d\") " pod="kserve-ci-e2e-test/precise-prefix-cache-test-kserve-6d6c8b5f7c-kgjrz" Jul 16 17:31:53.614825 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:31:53.614747 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/dd594272-3fa8-4587-9386-600307eb243d-kserve-provision-location\") pod \"precise-prefix-cache-test-kserve-6d6c8b5f7c-kgjrz\" (UID: \"dd594272-3fa8-4587-9386-600307eb243d\") " pod="kserve-ci-e2e-test/precise-prefix-cache-test-kserve-6d6c8b5f7c-kgjrz" Jul 16 17:31:53.616587 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:31:53.616561 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/dd594272-3fa8-4587-9386-600307eb243d-dshm\") pod \"precise-prefix-cache-test-kserve-6d6c8b5f7c-kgjrz\" (UID: \"dd594272-3fa8-4587-9386-600307eb243d\") " pod="kserve-ci-e2e-test/precise-prefix-cache-test-kserve-6d6c8b5f7c-kgjrz" Jul 16 17:31:53.616754 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:31:53.616738 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/dd594272-3fa8-4587-9386-600307eb243d-tls-certs\") pod \"precise-prefix-cache-test-kserve-6d6c8b5f7c-kgjrz\" (UID: \"dd594272-3fa8-4587-9386-600307eb243d\") " pod="kserve-ci-e2e-test/precise-prefix-cache-test-kserve-6d6c8b5f7c-kgjrz" Jul 16 17:31:53.634388 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:31:53.634366 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qdhzw\" (UniqueName: \"kubernetes.io/projected/dd594272-3fa8-4587-9386-600307eb243d-kube-api-access-qdhzw\") pod \"precise-prefix-cache-test-kserve-6d6c8b5f7c-kgjrz\" (UID: \"dd594272-3fa8-4587-9386-600307eb243d\") " pod="kserve-ci-e2e-test/precise-prefix-cache-test-kserve-6d6c8b5f7c-kgjrz" Jul 16 17:31:53.698320 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:31:53.698241 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve-ci-e2e-test/precise-prefix-cache-test-kserve-router-scheduler-75779d4bffhzf"] Jul 16 17:31:53.702502 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:31:53.702480 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/precise-prefix-cache-test-kserve-router-scheduler-75779d4bffhzf" Jul 16 17:31:53.705795 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:31:53.705776 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/precise-prefix-cache-test-kserve-6d6c8b5f7c-kgjrz" Jul 16 17:31:53.706305 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:31:53.706285 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"precise-prefix-cache-test-epp-sa-dockercfg-nf5cs\"" Jul 16 17:31:53.721699 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:31:53.721661 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/precise-prefix-cache-test-kserve-router-scheduler-75779d4bffhzf"] Jul 16 17:31:53.819754 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:31:53.819724 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/9ff96acc-75a8-48e2-9709-8abb012acf3f-kserve-provision-location\") pod \"precise-prefix-cache-test-kserve-router-scheduler-75779d4bffhzf\" (UID: \"9ff96acc-75a8-48e2-9709-8abb012acf3f\") " pod="kserve-ci-e2e-test/precise-prefix-cache-test-kserve-router-scheduler-75779d4bffhzf" Jul 16 17:31:53.819921 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:31:53.819782 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-x4b9p\" (UniqueName: \"kubernetes.io/projected/9ff96acc-75a8-48e2-9709-8abb012acf3f-kube-api-access-x4b9p\") pod \"precise-prefix-cache-test-kserve-router-scheduler-75779d4bffhzf\" (UID: \"9ff96acc-75a8-48e2-9709-8abb012acf3f\") " pod="kserve-ci-e2e-test/precise-prefix-cache-test-kserve-router-scheduler-75779d4bffhzf" Jul 16 17:31:53.819921 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:31:53.819818 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tokenizer-tmp\" (UniqueName: \"kubernetes.io/empty-dir/9ff96acc-75a8-48e2-9709-8abb012acf3f-tokenizer-tmp\") pod \"precise-prefix-cache-test-kserve-router-scheduler-75779d4bffhzf\" (UID: \"9ff96acc-75a8-48e2-9709-8abb012acf3f\") " pod="kserve-ci-e2e-test/precise-prefix-cache-test-kserve-router-scheduler-75779d4bffhzf" Jul 16 17:31:53.820007 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:31:53.819941 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tokenizer-cache\" (UniqueName: \"kubernetes.io/empty-dir/9ff96acc-75a8-48e2-9709-8abb012acf3f-tokenizer-cache\") pod \"precise-prefix-cache-test-kserve-router-scheduler-75779d4bffhzf\" (UID: \"9ff96acc-75a8-48e2-9709-8abb012acf3f\") " pod="kserve-ci-e2e-test/precise-prefix-cache-test-kserve-router-scheduler-75779d4bffhzf" Jul 16 17:31:53.820049 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:31:53.820006 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/9ff96acc-75a8-48e2-9709-8abb012acf3f-tls-certs\") pod \"precise-prefix-cache-test-kserve-router-scheduler-75779d4bffhzf\" (UID: \"9ff96acc-75a8-48e2-9709-8abb012acf3f\") " pod="kserve-ci-e2e-test/precise-prefix-cache-test-kserve-router-scheduler-75779d4bffhzf" Jul 16 17:31:53.820101 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:31:53.820054 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tokenizer-uds\" (UniqueName: \"kubernetes.io/empty-dir/9ff96acc-75a8-48e2-9709-8abb012acf3f-tokenizer-uds\") pod \"precise-prefix-cache-test-kserve-router-scheduler-75779d4bffhzf\" (UID: \"9ff96acc-75a8-48e2-9709-8abb012acf3f\") " pod="kserve-ci-e2e-test/precise-prefix-cache-test-kserve-router-scheduler-75779d4bffhzf" Jul 16 17:31:53.862730 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:31:53.862702 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/precise-prefix-cache-test-kserve-6d6c8b5f7c-kgjrz"] Jul 16 17:31:53.863432 ip-10-0-132-130 kubenswrapper[2578]: W0716 17:31:53.863413 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddd594272_3fa8_4587_9386_600307eb243d.slice/crio-2709d115cd8403f949df64aa6156d8a00c636273b6305eb1798dfdd4d97bc7b9 WatchSource:0}: Error finding container 2709d115cd8403f949df64aa6156d8a00c636273b6305eb1798dfdd4d97bc7b9: Status 404 returned error can't find the container with id 2709d115cd8403f949df64aa6156d8a00c636273b6305eb1798dfdd4d97bc7b9 Jul 16 17:31:53.865246 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:31:53.865232 2578 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 16 17:31:53.921122 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:31:53.921099 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-x4b9p\" (UniqueName: \"kubernetes.io/projected/9ff96acc-75a8-48e2-9709-8abb012acf3f-kube-api-access-x4b9p\") pod \"precise-prefix-cache-test-kserve-router-scheduler-75779d4bffhzf\" (UID: \"9ff96acc-75a8-48e2-9709-8abb012acf3f\") " pod="kserve-ci-e2e-test/precise-prefix-cache-test-kserve-router-scheduler-75779d4bffhzf" Jul 16 17:31:53.921246 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:31:53.921147 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tokenizer-tmp\" (UniqueName: \"kubernetes.io/empty-dir/9ff96acc-75a8-48e2-9709-8abb012acf3f-tokenizer-tmp\") pod \"precise-prefix-cache-test-kserve-router-scheduler-75779d4bffhzf\" (UID: \"9ff96acc-75a8-48e2-9709-8abb012acf3f\") " pod="kserve-ci-e2e-test/precise-prefix-cache-test-kserve-router-scheduler-75779d4bffhzf" Jul 16 17:31:53.921246 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:31:53.921220 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tokenizer-cache\" (UniqueName: \"kubernetes.io/empty-dir/9ff96acc-75a8-48e2-9709-8abb012acf3f-tokenizer-cache\") pod \"precise-prefix-cache-test-kserve-router-scheduler-75779d4bffhzf\" (UID: \"9ff96acc-75a8-48e2-9709-8abb012acf3f\") " pod="kserve-ci-e2e-test/precise-prefix-cache-test-kserve-router-scheduler-75779d4bffhzf" Jul 16 17:31:53.921362 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:31:53.921254 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/9ff96acc-75a8-48e2-9709-8abb012acf3f-tls-certs\") pod \"precise-prefix-cache-test-kserve-router-scheduler-75779d4bffhzf\" (UID: \"9ff96acc-75a8-48e2-9709-8abb012acf3f\") " pod="kserve-ci-e2e-test/precise-prefix-cache-test-kserve-router-scheduler-75779d4bffhzf" Jul 16 17:31:53.921362 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:31:53.921305 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tokenizer-uds\" (UniqueName: \"kubernetes.io/empty-dir/9ff96acc-75a8-48e2-9709-8abb012acf3f-tokenizer-uds\") pod \"precise-prefix-cache-test-kserve-router-scheduler-75779d4bffhzf\" (UID: \"9ff96acc-75a8-48e2-9709-8abb012acf3f\") " pod="kserve-ci-e2e-test/precise-prefix-cache-test-kserve-router-scheduler-75779d4bffhzf" Jul 16 17:31:53.921467 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:31:53.921439 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/9ff96acc-75a8-48e2-9709-8abb012acf3f-kserve-provision-location\") pod \"precise-prefix-cache-test-kserve-router-scheduler-75779d4bffhzf\" (UID: \"9ff96acc-75a8-48e2-9709-8abb012acf3f\") " pod="kserve-ci-e2e-test/precise-prefix-cache-test-kserve-router-scheduler-75779d4bffhzf" Jul 16 17:31:53.921613 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:31:53.921589 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tokenizer-tmp\" (UniqueName: \"kubernetes.io/empty-dir/9ff96acc-75a8-48e2-9709-8abb012acf3f-tokenizer-tmp\") pod \"precise-prefix-cache-test-kserve-router-scheduler-75779d4bffhzf\" (UID: \"9ff96acc-75a8-48e2-9709-8abb012acf3f\") " pod="kserve-ci-e2e-test/precise-prefix-cache-test-kserve-router-scheduler-75779d4bffhzf" Jul 16 17:31:53.921678 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:31:53.921624 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tokenizer-uds\" (UniqueName: \"kubernetes.io/empty-dir/9ff96acc-75a8-48e2-9709-8abb012acf3f-tokenizer-uds\") pod \"precise-prefix-cache-test-kserve-router-scheduler-75779d4bffhzf\" (UID: \"9ff96acc-75a8-48e2-9709-8abb012acf3f\") " pod="kserve-ci-e2e-test/precise-prefix-cache-test-kserve-router-scheduler-75779d4bffhzf" Jul 16 17:31:53.921752 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:31:53.921678 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tokenizer-cache\" (UniqueName: \"kubernetes.io/empty-dir/9ff96acc-75a8-48e2-9709-8abb012acf3f-tokenizer-cache\") pod \"precise-prefix-cache-test-kserve-router-scheduler-75779d4bffhzf\" (UID: \"9ff96acc-75a8-48e2-9709-8abb012acf3f\") " pod="kserve-ci-e2e-test/precise-prefix-cache-test-kserve-router-scheduler-75779d4bffhzf" Jul 16 17:31:53.921752 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:31:53.921743 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/9ff96acc-75a8-48e2-9709-8abb012acf3f-kserve-provision-location\") pod \"precise-prefix-cache-test-kserve-router-scheduler-75779d4bffhzf\" (UID: \"9ff96acc-75a8-48e2-9709-8abb012acf3f\") " pod="kserve-ci-e2e-test/precise-prefix-cache-test-kserve-router-scheduler-75779d4bffhzf" Jul 16 17:31:53.923580 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:31:53.923559 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/9ff96acc-75a8-48e2-9709-8abb012acf3f-tls-certs\") pod \"precise-prefix-cache-test-kserve-router-scheduler-75779d4bffhzf\" (UID: \"9ff96acc-75a8-48e2-9709-8abb012acf3f\") " pod="kserve-ci-e2e-test/precise-prefix-cache-test-kserve-router-scheduler-75779d4bffhzf" Jul 16 17:31:53.932950 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:31:53.932927 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-x4b9p\" (UniqueName: \"kubernetes.io/projected/9ff96acc-75a8-48e2-9709-8abb012acf3f-kube-api-access-x4b9p\") pod \"precise-prefix-cache-test-kserve-router-scheduler-75779d4bffhzf\" (UID: \"9ff96acc-75a8-48e2-9709-8abb012acf3f\") " pod="kserve-ci-e2e-test/precise-prefix-cache-test-kserve-router-scheduler-75779d4bffhzf" Jul 16 17:31:54.014048 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:31:54.013985 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/precise-prefix-cache-test-kserve-router-scheduler-75779d4bffhzf" Jul 16 17:31:54.178169 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:31:54.178041 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/precise-prefix-cache-test-kserve-router-scheduler-75779d4bffhzf"] Jul 16 17:31:54.180261 ip-10-0-132-130 kubenswrapper[2578]: W0716 17:31:54.180236 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9ff96acc_75a8_48e2_9709_8abb012acf3f.slice/crio-938e18fbd74c9d7b17014bb5958537207c9a14283264507693c71e7dfb33721a WatchSource:0}: Error finding container 938e18fbd74c9d7b17014bb5958537207c9a14283264507693c71e7dfb33721a: Status 404 returned error can't find the container with id 938e18fbd74c9d7b17014bb5958537207c9a14283264507693c71e7dfb33721a Jul 16 17:31:54.475750 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:31:54.475715 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/precise-prefix-cache-test-kserve-6d6c8b5f7c-kgjrz" event={"ID":"dd594272-3fa8-4587-9386-600307eb243d","Type":"ContainerStarted","Data":"2709d115cd8403f949df64aa6156d8a00c636273b6305eb1798dfdd4d97bc7b9"} Jul 16 17:31:54.477027 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:31:54.477002 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/precise-prefix-cache-test-kserve-router-scheduler-75779d4bffhzf" event={"ID":"9ff96acc-75a8-48e2-9709-8abb012acf3f","Type":"ContainerStarted","Data":"938e18fbd74c9d7b17014bb5958537207c9a14283264507693c71e7dfb33721a"} Jul 16 17:31:58.508181 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:31:58.508090 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/precise-prefix-cache-test-kserve-router-scheduler-75779d4bffhzf" event={"ID":"9ff96acc-75a8-48e2-9709-8abb012acf3f","Type":"ContainerStarted","Data":"0860226f79ac6405624e6c49ec2bdf9519ae60a90d6b57cae80458342f2c3dce"} Jul 16 17:31:58.509768 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:31:58.509726 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/precise-prefix-cache-test-kserve-6d6c8b5f7c-kgjrz" event={"ID":"dd594272-3fa8-4587-9386-600307eb243d","Type":"ContainerStarted","Data":"91d44097143032ce28d2fdfefdf516019a74805f09465424b5d55d3365c2a9ca"} Jul 16 17:31:59.525108 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:31:59.525070 2578 generic.go:356] "Generic (PLEG): container finished" podID="9ff96acc-75a8-48e2-9709-8abb012acf3f" containerID="0860226f79ac6405624e6c49ec2bdf9519ae60a90d6b57cae80458342f2c3dce" exitCode=0 Jul 16 17:31:59.525534 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:31:59.525174 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/precise-prefix-cache-test-kserve-router-scheduler-75779d4bffhzf" event={"ID":"9ff96acc-75a8-48e2-9709-8abb012acf3f","Type":"ContainerDied","Data":"0860226f79ac6405624e6c49ec2bdf9519ae60a90d6b57cae80458342f2c3dce"} Jul 16 17:32:00.538900 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:32:00.538819 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/precise-prefix-cache-test-kserve-router-scheduler-75779d4bffhzf" event={"ID":"9ff96acc-75a8-48e2-9709-8abb012acf3f","Type":"ContainerStarted","Data":"bdedf150c2c95407b7ee62068e339956efac5eec023852cbfc19b3c68c44db36"} Jul 16 17:32:02.554531 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:32:02.554496 2578 generic.go:356] "Generic (PLEG): container finished" podID="dd594272-3fa8-4587-9386-600307eb243d" containerID="91d44097143032ce28d2fdfefdf516019a74805f09465424b5d55d3365c2a9ca" exitCode=0 Jul 16 17:32:02.554937 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:32:02.554570 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/precise-prefix-cache-test-kserve-6d6c8b5f7c-kgjrz" event={"ID":"dd594272-3fa8-4587-9386-600307eb243d","Type":"ContainerDied","Data":"91d44097143032ce28d2fdfefdf516019a74805f09465424b5d55d3365c2a9ca"} Jul 16 17:32:03.563324 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:32:03.563283 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/precise-prefix-cache-test-kserve-6d6c8b5f7c-kgjrz" event={"ID":"dd594272-3fa8-4587-9386-600307eb243d","Type":"ContainerStarted","Data":"c05f3ddc276e7c8fc8d621870e70039772b54fffd74051c2556d053538356b88"} Jul 16 17:32:03.594715 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:32:03.594618 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve-ci-e2e-test/precise-prefix-cache-test-kserve-6d6c8b5f7c-kgjrz" podStartSLOduration=3.090340865 podStartE2EDuration="10.594602059s" totalImagesPullingTime="4.327038157s" totalInitContainerRuntime="3.177223037s" isStatefulPod=false podCreationTimestamp="2026-07-16 17:31:53 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 17:32:03.594328785 +0000 UTC m=+1068.080172673" watchObservedRunningTime="2026-07-16 17:32:03.594602059 +0000 UTC m=+1068.080445968" Jul 16 17:32:03.706604 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:32:03.706567 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/precise-prefix-cache-test-kserve-6d6c8b5f7c-kgjrz" Jul 16 17:32:03.706781 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:32:03.706632 2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kserve-ci-e2e-test/precise-prefix-cache-test-kserve-6d6c8b5f7c-kgjrz" Jul 16 17:32:03.719115 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:32:03.719088 2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kserve-ci-e2e-test/precise-prefix-cache-test-kserve-6d6c8b5f7c-kgjrz" Jul 16 17:32:04.584967 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:32:04.584936 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/precise-prefix-cache-test-kserve-6d6c8b5f7c-kgjrz" Jul 16 17:32:10.613101 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:32:10.613066 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_precise-prefix-cache-test-kserve-router-scheduler-75779d4bffhzf_9ff96acc-75a8-48e2-9709-8abb012acf3f/main/0.log" Jul 16 17:32:10.613527 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:32:10.613407 2578 generic.go:356] "Generic (PLEG): container finished" podID="9ff96acc-75a8-48e2-9709-8abb012acf3f" containerID="bdedf150c2c95407b7ee62068e339956efac5eec023852cbfc19b3c68c44db36" exitCode=1 Jul 16 17:32:10.613527 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:32:10.613486 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/precise-prefix-cache-test-kserve-router-scheduler-75779d4bffhzf" event={"ID":"9ff96acc-75a8-48e2-9709-8abb012acf3f","Type":"ContainerDied","Data":"bdedf150c2c95407b7ee62068e339956efac5eec023852cbfc19b3c68c44db36"} Jul 16 17:32:43.836574 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:32:43.836535 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_precise-prefix-cache-test-kserve-router-scheduler-75779d4bffhzf_9ff96acc-75a8-48e2-9709-8abb012acf3f/main/0.log" Jul 16 17:32:43.837119 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:32:43.837079 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/precise-prefix-cache-test-kserve-router-scheduler-75779d4bffhzf" event={"ID":"9ff96acc-75a8-48e2-9709-8abb012acf3f","Type":"ContainerStarted","Data":"5711e1b5957c97d92061e7352b7360d9892ef7aa2e800c5932905f5008ae6de5"} Jul 16 17:32:43.837503 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:32:43.837484 2578 scope.go:117] "RemoveContainer" containerID="bdedf150c2c95407b7ee62068e339956efac5eec023852cbfc19b3c68c44db36" Jul 16 17:32:44.014839 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:32:44.014801 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/precise-prefix-cache-test-kserve-router-scheduler-75779d4bffhzf" Jul 16 17:32:44.015074 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:32:44.014890 2578 kubelet.go:2654] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="kserve-ci-e2e-test/precise-prefix-cache-test-kserve-router-scheduler-75779d4bffhzf" Jul 16 17:32:44.015074 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:32:44.014909 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/precise-prefix-cache-test-kserve-router-scheduler-75779d4bffhzf" Jul 16 17:32:44.015074 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:32:44.014929 2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kserve-ci-e2e-test/precise-prefix-cache-test-kserve-router-scheduler-75779d4bffhzf" Jul 16 17:32:44.846122 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:32:44.846094 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_precise-prefix-cache-test-kserve-router-scheduler-75779d4bffhzf_9ff96acc-75a8-48e2-9709-8abb012acf3f/main/0.log" Jul 16 17:32:44.846598 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:32:44.846499 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/precise-prefix-cache-test-kserve-router-scheduler-75779d4bffhzf" event={"ID":"9ff96acc-75a8-48e2-9709-8abb012acf3f","Type":"ContainerStarted","Data":"b6cdbfe0b29de6d86102f5df9e53a9d95645def85bda53bcf6295bbd57183932"} Jul 16 17:32:44.846672 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:32:44.846616 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/precise-prefix-cache-test-kserve-router-scheduler-75779d4bffhzf" Jul 16 17:32:44.882202 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:32:44.882154 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve-ci-e2e-test/precise-prefix-cache-test-kserve-router-scheduler-75779d4bffhzf" podStartSLOduration=3.330934323 podStartE2EDuration="51.882139546s" totalImagesPullingTime="47.884737824s" totalInitContainerRuntime="666.467399ms" isStatefulPod=false podCreationTimestamp="2026-07-16 17:31:53 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 17:32:44.88135534 +0000 UTC m=+1109.367199245" watchObservedRunningTime="2026-07-16 17:32:44.882139546 +0000 UTC m=+1109.367983435" Jul 16 17:32:54.015081 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:32:54.015046 2578 prober.go:120] "Probe failed" probeType="Startup" pod="kserve-ci-e2e-test/precise-prefix-cache-test-kserve-router-scheduler-75779d4bffhzf" podUID="9ff96acc-75a8-48e2-9709-8abb012acf3f" containerName="tokenizer" probeResult="failure" output="Get \"http://10.133.0.54:8082/healthz\": dial tcp 10.133.0.54:8082: connect: connection refused" Jul 16 17:32:54.015497 ip-10-0-132-130 kubenswrapper[2578]: W0716 17:32:54.015257 2578 logging.go:55] [core] [Channel #89 SubChannel #90]grpc: addrConn.createTransport failed to connect to {Addr: "10.133.0.54:9003", ServerName: "10.133.0.54:9003", BalancerAttributes: {"<%!p(pickfirstleaf.managedByPickfirstKeyType={})>": "<%!p(bool=true)>" }}. Err: connection error: desc = "transport: Error while dialing: dial tcp 10.133.0.54:9003: connect: connection refused" Jul 16 17:32:54.906014 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:32:54.905984 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_precise-prefix-cache-test-kserve-router-scheduler-75779d4bffhzf_9ff96acc-75a8-48e2-9709-8abb012acf3f/main/1.log" Jul 16 17:32:54.906711 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:32:54.906677 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_precise-prefix-cache-test-kserve-router-scheduler-75779d4bffhzf_9ff96acc-75a8-48e2-9709-8abb012acf3f/main/0.log" Jul 16 17:32:54.907043 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:32:54.907021 2578 generic.go:356] "Generic (PLEG): container finished" podID="9ff96acc-75a8-48e2-9709-8abb012acf3f" containerID="b6cdbfe0b29de6d86102f5df9e53a9d95645def85bda53bcf6295bbd57183932" exitCode=1 Jul 16 17:32:54.907128 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:32:54.907092 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/precise-prefix-cache-test-kserve-router-scheduler-75779d4bffhzf" event={"ID":"9ff96acc-75a8-48e2-9709-8abb012acf3f","Type":"ContainerDied","Data":"b6cdbfe0b29de6d86102f5df9e53a9d95645def85bda53bcf6295bbd57183932"} Jul 16 17:32:54.907180 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:32:54.907152 2578 scope.go:117] "RemoveContainer" containerID="bdedf150c2c95407b7ee62068e339956efac5eec023852cbfc19b3c68c44db36" Jul 16 17:32:54.907559 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:32:54.907542 2578 scope.go:117] "RemoveContainer" containerID="b6cdbfe0b29de6d86102f5df9e53a9d95645def85bda53bcf6295bbd57183932" Jul 16 17:32:54.907827 ip-10-0-132-130 kubenswrapper[2578]: E0716 17:32:54.907803 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"main\" with CrashLoopBackOff: \"back-off 10s restarting failed container=main pod=precise-prefix-cache-test-kserve-router-scheduler-75779d4bffhzf_kserve-ci-e2e-test(9ff96acc-75a8-48e2-9709-8abb012acf3f)\"" pod="kserve-ci-e2e-test/precise-prefix-cache-test-kserve-router-scheduler-75779d4bffhzf" podUID="9ff96acc-75a8-48e2-9709-8abb012acf3f" Jul 16 17:32:55.015016 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:32:55.014987 2578 prober.go:120] "Probe failed" probeType="Liveness" pod="kserve-ci-e2e-test/precise-prefix-cache-test-kserve-router-scheduler-75779d4bffhzf" podUID="9ff96acc-75a8-48e2-9709-8abb012acf3f" containerName="main" probeResult="failure" output="timeout: failed to connect service \"10.133.0.54:9003\" within 1s: context deadline exceeded" Jul 16 17:32:55.921377 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:32:55.921346 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_precise-prefix-cache-test-kserve-router-scheduler-75779d4bffhzf_9ff96acc-75a8-48e2-9709-8abb012acf3f/main/1.log" Jul 16 17:33:04.014883 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:33:04.014851 2578 kubelet.go:2654] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="kserve-ci-e2e-test/precise-prefix-cache-test-kserve-router-scheduler-75779d4bffhzf" Jul 16 17:33:04.015402 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:33:04.015321 2578 scope.go:117] "RemoveContainer" containerID="b6cdbfe0b29de6d86102f5df9e53a9d95645def85bda53bcf6295bbd57183932" Jul 16 17:33:04.015571 ip-10-0-132-130 kubenswrapper[2578]: E0716 17:33:04.015551 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"main\" with CrashLoopBackOff: \"back-off 10s restarting failed container=main pod=precise-prefix-cache-test-kserve-router-scheduler-75779d4bffhzf_kserve-ci-e2e-test(9ff96acc-75a8-48e2-9709-8abb012acf3f)\"" pod="kserve-ci-e2e-test/precise-prefix-cache-test-kserve-router-scheduler-75779d4bffhzf" podUID="9ff96acc-75a8-48e2-9709-8abb012acf3f" Jul 16 17:33:04.016550 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:33:04.016527 2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kserve-ci-e2e-test/precise-prefix-cache-test-kserve-router-scheduler-75779d4bffhzf" Jul 16 17:33:04.977100 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:33:04.977067 2578 scope.go:117] "RemoveContainer" containerID="b6cdbfe0b29de6d86102f5df9e53a9d95645def85bda53bcf6295bbd57183932" Jul 16 17:33:04.978135 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:33:04.978109 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/precise-prefix-cache-test-kserve-router-scheduler-75779d4bffhzf" Jul 16 17:33:05.989181 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:33:05.989147 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_precise-prefix-cache-test-kserve-router-scheduler-75779d4bffhzf_9ff96acc-75a8-48e2-9709-8abb012acf3f/main/1.log" Jul 16 17:33:05.989981 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:33:05.989951 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/precise-prefix-cache-test-kserve-router-scheduler-75779d4bffhzf" event={"ID":"9ff96acc-75a8-48e2-9709-8abb012acf3f","Type":"ContainerStarted","Data":"c8bb73ce588ce15d9ee7af633bb7bb973817b630370bb6d52abbe2de3b97081d"} Jul 16 17:33:05.990863 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:33:05.990842 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/precise-prefix-cache-test-kserve-router-scheduler-75779d4bffhzf" Jul 16 17:33:14.015881 ip-10-0-132-130 kubenswrapper[2578]: W0716 17:33:14.015845 2578 logging.go:55] [core] [Channel #91 SubChannel #92]grpc: addrConn.createTransport failed to connect to {Addr: "10.133.0.54:9003", ServerName: "10.133.0.54:9003", BalancerAttributes: {"<%!p(pickfirstleaf.managedByPickfirstKeyType={})>": "<%!p(bool=true)>" }}. Err: connection error: desc = "transport: Error while dialing: dial tcp 10.133.0.54:9003: connect: connection refused" Jul 16 17:33:15.015717 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:33:15.015649 2578 prober.go:120] "Probe failed" probeType="Liveness" pod="kserve-ci-e2e-test/precise-prefix-cache-test-kserve-router-scheduler-75779d4bffhzf" podUID="9ff96acc-75a8-48e2-9709-8abb012acf3f" containerName="main" probeResult="failure" output="timeout: failed to connect service \"10.133.0.54:9003\" within 1s: context deadline exceeded" Jul 16 17:33:38.004003 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:33:38.003978 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/precise-prefix-cache-test-kserve-router-scheduler-75779d4bffhzf" Jul 16 17:33:40.006264 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:33:40.006233 2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/precise-prefix-cache-test-kserve-6d6c8b5f7c-kgjrz"] Jul 16 17:33:40.014444 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:33:40.014418 2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/precise-prefix-cache-test-kserve-router-scheduler-75779d4bffhzf"] Jul 16 17:33:40.014681 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:33:40.014656 2578 kuberuntime_container.go:873] "Killing container with a grace period" pod="kserve-ci-e2e-test/precise-prefix-cache-test-kserve-router-scheduler-75779d4bffhzf" podUID="9ff96acc-75a8-48e2-9709-8abb012acf3f" containerName="tokenizer" containerID="cri-o://5711e1b5957c97d92061e7352b7360d9892ef7aa2e800c5932905f5008ae6de5" gracePeriod=60 Jul 16 17:33:40.016322 ip-10-0-132-130 kubenswrapper[2578]: E0716 17:33:40.016287 2578 log.go:32] "ExecSync cmd from runtime service failed" err=< Jul 16 17:33:40.016322 ip-10-0-132-130 kubenswrapper[2578]: rpc error: code = Unknown desc = command error: executable file `/bin/sleep` not found: No such file or directory Jul 16 17:33:40.016322 ip-10-0-132-130 kubenswrapper[2578]: , stdout: , stderr: , exit code -1 Jul 16 17:33:40.016322 ip-10-0-132-130 kubenswrapper[2578]: > containerID="c05f3ddc276e7c8fc8d621870e70039772b54fffd74051c2556d053538356b88" cmd=["/bin/sleep","15"] Jul 16 17:33:40.016601 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:33:40.016370 2578 handlers.go:82] "Exec lifecycle hook for Container in Pod failed" execCommand=["/bin/sleep","15"] containerName="main" pod="kserve-ci-e2e-test/precise-prefix-cache-test-kserve-6d6c8b5f7c-kgjrz" message="" err=< Jul 16 17:33:40.016601 ip-10-0-132-130 kubenswrapper[2578]: rpc error: code = Unknown desc = command error: executable file `/bin/sleep` not found: No such file or directory Jul 16 17:33:40.016601 ip-10-0-132-130 kubenswrapper[2578]: , stdout: , stderr: , exit code -1 Jul 16 17:33:40.016601 ip-10-0-132-130 kubenswrapper[2578]: > Jul 16 17:33:40.016601 ip-10-0-132-130 kubenswrapper[2578]: E0716 17:33:40.016388 2578 kuberuntime_container.go:755] "PreStop hook failed" err=< Jul 16 17:33:40.016601 ip-10-0-132-130 kubenswrapper[2578]: rpc error: code = Unknown desc = command error: executable file `/bin/sleep` not found: No such file or directory Jul 16 17:33:40.016601 ip-10-0-132-130 kubenswrapper[2578]: , stdout: , stderr: , exit code -1 Jul 16 17:33:40.016601 ip-10-0-132-130 kubenswrapper[2578]: > pod="kserve-ci-e2e-test/precise-prefix-cache-test-kserve-6d6c8b5f7c-kgjrz" podUID="dd594272-3fa8-4587-9386-600307eb243d" containerName="main" containerID="cri-o://c05f3ddc276e7c8fc8d621870e70039772b54fffd74051c2556d053538356b88" Jul 16 17:33:40.016601 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:33:40.016427 2578 kuberuntime_container.go:873] "Killing container with a grace period" pod="kserve-ci-e2e-test/precise-prefix-cache-test-kserve-6d6c8b5f7c-kgjrz" podUID="dd594272-3fa8-4587-9386-600307eb243d" containerName="main" containerID="cri-o://c05f3ddc276e7c8fc8d621870e70039772b54fffd74051c2556d053538356b88" gracePeriod=60 Jul 16 17:33:40.381819 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:33:40.381795 2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/precise-prefix-cache-test-kserve-6d6c8b5f7c-kgjrz" Jul 16 17:33:40.516463 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:33:40.516428 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/dd594272-3fa8-4587-9386-600307eb243d-tmp-dir\") pod \"dd594272-3fa8-4587-9386-600307eb243d\" (UID: \"dd594272-3fa8-4587-9386-600307eb243d\") " Jul 16 17:33:40.516666 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:33:40.516483 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/dd594272-3fa8-4587-9386-600307eb243d-tls-certs\") pod \"dd594272-3fa8-4587-9386-600307eb243d\" (UID: \"dd594272-3fa8-4587-9386-600307eb243d\") " Jul 16 17:33:40.516666 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:33:40.516510 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/dd594272-3fa8-4587-9386-600307eb243d-home\") pod \"dd594272-3fa8-4587-9386-600307eb243d\" (UID: \"dd594272-3fa8-4587-9386-600307eb243d\") " Jul 16 17:33:40.516666 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:33:40.516587 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/dd594272-3fa8-4587-9386-600307eb243d-model-cache\") pod \"dd594272-3fa8-4587-9386-600307eb243d\" (UID: \"dd594272-3fa8-4587-9386-600307eb243d\") " Jul 16 17:33:40.516666 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:33:40.516617 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/dd594272-3fa8-4587-9386-600307eb243d-dshm\") pod \"dd594272-3fa8-4587-9386-600307eb243d\" (UID: \"dd594272-3fa8-4587-9386-600307eb243d\") " Jul 16 17:33:40.516666 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:33:40.516642 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/dd594272-3fa8-4587-9386-600307eb243d-kserve-provision-location\") pod \"dd594272-3fa8-4587-9386-600307eb243d\" (UID: \"dd594272-3fa8-4587-9386-600307eb243d\") " Jul 16 17:33:40.516953 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:33:40.516731 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-qdhzw\" (UniqueName: \"kubernetes.io/projected/dd594272-3fa8-4587-9386-600307eb243d-kube-api-access-qdhzw\") pod \"dd594272-3fa8-4587-9386-600307eb243d\" (UID: \"dd594272-3fa8-4587-9386-600307eb243d\") " Jul 16 17:33:40.516953 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:33:40.516727 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/dd594272-3fa8-4587-9386-600307eb243d-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "dd594272-3fa8-4587-9386-600307eb243d" (UID: "dd594272-3fa8-4587-9386-600307eb243d"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 16 17:33:40.516953 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:33:40.516909 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/dd594272-3fa8-4587-9386-600307eb243d-model-cache" (OuterVolumeSpecName: "model-cache") pod "dd594272-3fa8-4587-9386-600307eb243d" (UID: "dd594272-3fa8-4587-9386-600307eb243d"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 16 17:33:40.517104 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:33:40.517062 2578 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/dd594272-3fa8-4587-9386-600307eb243d-tmp-dir\") on node \"ip-10-0-132-130.ec2.internal\" DevicePath \"\"" Jul 16 17:33:40.517104 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:33:40.517079 2578 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/dd594272-3fa8-4587-9386-600307eb243d-model-cache\") on node \"ip-10-0-132-130.ec2.internal\" DevicePath \"\"" Jul 16 17:33:40.517658 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:33:40.517618 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/dd594272-3fa8-4587-9386-600307eb243d-home" (OuterVolumeSpecName: "home") pod "dd594272-3fa8-4587-9386-600307eb243d" (UID: "dd594272-3fa8-4587-9386-600307eb243d"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 16 17:33:40.519360 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:33:40.519328 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/dd594272-3fa8-4587-9386-600307eb243d-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "dd594272-3fa8-4587-9386-600307eb243d" (UID: "dd594272-3fa8-4587-9386-600307eb243d"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 16 17:33:40.519570 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:33:40.519549 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/dd594272-3fa8-4587-9386-600307eb243d-dshm" (OuterVolumeSpecName: "dshm") pod "dd594272-3fa8-4587-9386-600307eb243d" (UID: "dd594272-3fa8-4587-9386-600307eb243d"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 16 17:33:40.519755 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:33:40.519681 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/dd594272-3fa8-4587-9386-600307eb243d-kube-api-access-qdhzw" (OuterVolumeSpecName: "kube-api-access-qdhzw") pod "dd594272-3fa8-4587-9386-600307eb243d" (UID: "dd594272-3fa8-4587-9386-600307eb243d"). InnerVolumeSpecName "kube-api-access-qdhzw". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 16 17:33:40.597019 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:33:40.596978 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/dd594272-3fa8-4587-9386-600307eb243d-kserve-provision-location" (OuterVolumeSpecName: "kserve-provision-location") pod "dd594272-3fa8-4587-9386-600307eb243d" (UID: "dd594272-3fa8-4587-9386-600307eb243d"). InnerVolumeSpecName "kserve-provision-location". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 16 17:33:40.617638 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:33:40.617609 2578 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/dd594272-3fa8-4587-9386-600307eb243d-dshm\") on node \"ip-10-0-132-130.ec2.internal\" DevicePath \"\"" Jul 16 17:33:40.617798 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:33:40.617643 2578 reconciler_common.go:299] "Volume detached for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/dd594272-3fa8-4587-9386-600307eb243d-kserve-provision-location\") on node \"ip-10-0-132-130.ec2.internal\" DevicePath \"\"" Jul 16 17:33:40.617798 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:33:40.617660 2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-qdhzw\" (UniqueName: \"kubernetes.io/projected/dd594272-3fa8-4587-9386-600307eb243d-kube-api-access-qdhzw\") on node \"ip-10-0-132-130.ec2.internal\" DevicePath \"\"" Jul 16 17:33:40.617798 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:33:40.617674 2578 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/dd594272-3fa8-4587-9386-600307eb243d-tls-certs\") on node \"ip-10-0-132-130.ec2.internal\" DevicePath \"\"" Jul 16 17:33:40.617798 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:33:40.617687 2578 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/dd594272-3fa8-4587-9386-600307eb243d-home\") on node \"ip-10-0-132-130.ec2.internal\" DevicePath \"\"" Jul 16 17:33:41.210624 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:33:41.210584 2578 generic.go:356] "Generic (PLEG): container finished" podID="dd594272-3fa8-4587-9386-600307eb243d" containerID="c05f3ddc276e7c8fc8d621870e70039772b54fffd74051c2556d053538356b88" exitCode=0 Jul 16 17:33:41.211074 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:33:41.210651 2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/precise-prefix-cache-test-kserve-6d6c8b5f7c-kgjrz" Jul 16 17:33:41.211074 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:33:41.210670 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/precise-prefix-cache-test-kserve-6d6c8b5f7c-kgjrz" event={"ID":"dd594272-3fa8-4587-9386-600307eb243d","Type":"ContainerDied","Data":"c05f3ddc276e7c8fc8d621870e70039772b54fffd74051c2556d053538356b88"} Jul 16 17:33:41.211074 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:33:41.210754 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/precise-prefix-cache-test-kserve-6d6c8b5f7c-kgjrz" event={"ID":"dd594272-3fa8-4587-9386-600307eb243d","Type":"ContainerDied","Data":"2709d115cd8403f949df64aa6156d8a00c636273b6305eb1798dfdd4d97bc7b9"} Jul 16 17:33:41.211074 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:33:41.210787 2578 scope.go:117] "RemoveContainer" containerID="c05f3ddc276e7c8fc8d621870e70039772b54fffd74051c2556d053538356b88" Jul 16 17:33:41.222504 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:33:41.222489 2578 scope.go:117] "RemoveContainer" containerID="91d44097143032ce28d2fdfefdf516019a74805f09465424b5d55d3365c2a9ca" Jul 16 17:33:41.233233 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:33:41.233207 2578 scope.go:117] "RemoveContainer" containerID="c05f3ddc276e7c8fc8d621870e70039772b54fffd74051c2556d053538356b88" Jul 16 17:33:41.233473 ip-10-0-132-130 kubenswrapper[2578]: E0716 17:33:41.233454 2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c05f3ddc276e7c8fc8d621870e70039772b54fffd74051c2556d053538356b88\": container with ID starting with c05f3ddc276e7c8fc8d621870e70039772b54fffd74051c2556d053538356b88 not found: ID does not exist" containerID="c05f3ddc276e7c8fc8d621870e70039772b54fffd74051c2556d053538356b88" Jul 16 17:33:41.233535 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:33:41.233483 2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c05f3ddc276e7c8fc8d621870e70039772b54fffd74051c2556d053538356b88"} err="failed to get container status \"c05f3ddc276e7c8fc8d621870e70039772b54fffd74051c2556d053538356b88\": rpc error: code = NotFound desc = could not find container \"c05f3ddc276e7c8fc8d621870e70039772b54fffd74051c2556d053538356b88\": container with ID starting with c05f3ddc276e7c8fc8d621870e70039772b54fffd74051c2556d053538356b88 not found: ID does not exist" Jul 16 17:33:41.233535 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:33:41.233507 2578 scope.go:117] "RemoveContainer" containerID="91d44097143032ce28d2fdfefdf516019a74805f09465424b5d55d3365c2a9ca" Jul 16 17:33:41.233752 ip-10-0-132-130 kubenswrapper[2578]: E0716 17:33:41.233735 2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"91d44097143032ce28d2fdfefdf516019a74805f09465424b5d55d3365c2a9ca\": container with ID starting with 91d44097143032ce28d2fdfefdf516019a74805f09465424b5d55d3365c2a9ca not found: ID does not exist" containerID="91d44097143032ce28d2fdfefdf516019a74805f09465424b5d55d3365c2a9ca" Jul 16 17:33:41.233801 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:33:41.233757 2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"91d44097143032ce28d2fdfefdf516019a74805f09465424b5d55d3365c2a9ca"} err="failed to get container status \"91d44097143032ce28d2fdfefdf516019a74805f09465424b5d55d3365c2a9ca\": rpc error: code = NotFound desc = could not find container \"91d44097143032ce28d2fdfefdf516019a74805f09465424b5d55d3365c2a9ca\": container with ID starting with 91d44097143032ce28d2fdfefdf516019a74805f09465424b5d55d3365c2a9ca not found: ID does not exist" Jul 16 17:33:41.241355 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:33:41.241330 2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/precise-prefix-cache-test-kserve-6d6c8b5f7c-kgjrz"] Jul 16 17:33:41.247849 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:33:41.247828 2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kserve-ci-e2e-test/precise-prefix-cache-test-kserve-6d6c8b5f7c-kgjrz"] Jul 16 17:33:42.215704 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:33:42.215669 2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="dd594272-3fa8-4587-9386-600307eb243d" path="/var/lib/kubelet/pods/dd594272-3fa8-4587-9386-600307eb243d/volumes" Jul 16 17:33:42.217961 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:33:42.217941 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_precise-prefix-cache-test-kserve-router-scheduler-75779d4bffhzf_9ff96acc-75a8-48e2-9709-8abb012acf3f/main/1.log" Jul 16 17:33:42.218654 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:33:42.218631 2578 generic.go:356] "Generic (PLEG): container finished" podID="9ff96acc-75a8-48e2-9709-8abb012acf3f" containerID="5711e1b5957c97d92061e7352b7360d9892ef7aa2e800c5932905f5008ae6de5" exitCode=0 Jul 16 17:33:42.218766 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:33:42.218721 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/precise-prefix-cache-test-kserve-router-scheduler-75779d4bffhzf" event={"ID":"9ff96acc-75a8-48e2-9709-8abb012acf3f","Type":"ContainerDied","Data":"5711e1b5957c97d92061e7352b7360d9892ef7aa2e800c5932905f5008ae6de5"} Jul 16 17:33:44.977782 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:33:44.977740 2578 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/precise-prefix-cache-test-kserve-router-scheduler-75779d4bffhzf" podUID="9ff96acc-75a8-48e2-9709-8abb012acf3f" containerName="tokenizer" probeResult="failure" output="Get \"http://10.133.0.54:8082/healthz\": dial tcp 10.133.0.54:8082: connect: connection refused" Jul 16 17:33:48.004279 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:33:48.004246 2578 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/precise-prefix-cache-test-kserve-router-scheduler-75779d4bffhzf" podUID="9ff96acc-75a8-48e2-9709-8abb012acf3f" containerName="main" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")" Jul 16 17:33:54.977374 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:33:54.977330 2578 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/precise-prefix-cache-test-kserve-router-scheduler-75779d4bffhzf" podUID="9ff96acc-75a8-48e2-9709-8abb012acf3f" containerName="tokenizer" probeResult="failure" output="Get \"http://10.133.0.54:8082/healthz\": dial tcp 10.133.0.54:8082: connect: connection refused" Jul 16 17:33:55.035990 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:33:55.035935 2578 kuberuntime_container.go:873] "Killing container with a grace period" pod="kserve-ci-e2e-test/precise-prefix-cache-test-kserve-router-scheduler-75779d4bffhzf" podUID="9ff96acc-75a8-48e2-9709-8abb012acf3f" containerName="main" containerID="cri-o://c8bb73ce588ce15d9ee7af633bb7bb973817b630370bb6d52abbe2de3b97081d" gracePeriod=45 Jul 16 17:33:55.290930 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:33:55.290903 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_precise-prefix-cache-test-kserve-router-scheduler-75779d4bffhzf_9ff96acc-75a8-48e2-9709-8abb012acf3f/main/1.log" Jul 16 17:33:55.291063 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:33:55.290903 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_precise-prefix-cache-test-kserve-router-scheduler-75779d4bffhzf_9ff96acc-75a8-48e2-9709-8abb012acf3f/main/1.log" Jul 16 17:33:55.291657 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:33:55.291634 2578 generic.go:356] "Generic (PLEG): container finished" podID="9ff96acc-75a8-48e2-9709-8abb012acf3f" containerID="c8bb73ce588ce15d9ee7af633bb7bb973817b630370bb6d52abbe2de3b97081d" exitCode=0 Jul 16 17:33:55.291772 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:33:55.291723 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/precise-prefix-cache-test-kserve-router-scheduler-75779d4bffhzf" event={"ID":"9ff96acc-75a8-48e2-9709-8abb012acf3f","Type":"ContainerDied","Data":"c8bb73ce588ce15d9ee7af633bb7bb973817b630370bb6d52abbe2de3b97081d"} Jul 16 17:33:55.291829 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:33:55.291816 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/precise-prefix-cache-test-kserve-router-scheduler-75779d4bffhzf" event={"ID":"9ff96acc-75a8-48e2-9709-8abb012acf3f","Type":"ContainerDied","Data":"938e18fbd74c9d7b17014bb5958537207c9a14283264507693c71e7dfb33721a"} Jul 16 17:33:55.291862 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:33:55.291841 2578 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="938e18fbd74c9d7b17014bb5958537207c9a14283264507693c71e7dfb33721a" Jul 16 17:33:55.291862 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:33:55.291855 2578 scope.go:117] "RemoveContainer" containerID="b6cdbfe0b29de6d86102f5df9e53a9d95645def85bda53bcf6295bbd57183932" Jul 16 17:33:55.294050 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:33:55.294032 2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/precise-prefix-cache-test-kserve-router-scheduler-75779d4bffhzf" Jul 16 17:33:55.327158 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:33:55.327136 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tokenizer-uds\" (UniqueName: \"kubernetes.io/empty-dir/9ff96acc-75a8-48e2-9709-8abb012acf3f-tokenizer-uds\") pod \"9ff96acc-75a8-48e2-9709-8abb012acf3f\" (UID: \"9ff96acc-75a8-48e2-9709-8abb012acf3f\") " Jul 16 17:33:55.327255 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:33:55.327186 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/9ff96acc-75a8-48e2-9709-8abb012acf3f-tls-certs\") pod \"9ff96acc-75a8-48e2-9709-8abb012acf3f\" (UID: \"9ff96acc-75a8-48e2-9709-8abb012acf3f\") " Jul 16 17:33:55.327255 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:33:55.327225 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-x4b9p\" (UniqueName: \"kubernetes.io/projected/9ff96acc-75a8-48e2-9709-8abb012acf3f-kube-api-access-x4b9p\") pod \"9ff96acc-75a8-48e2-9709-8abb012acf3f\" (UID: \"9ff96acc-75a8-48e2-9709-8abb012acf3f\") " Jul 16 17:33:55.327255 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:33:55.327241 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tokenizer-cache\" (UniqueName: \"kubernetes.io/empty-dir/9ff96acc-75a8-48e2-9709-8abb012acf3f-tokenizer-cache\") pod \"9ff96acc-75a8-48e2-9709-8abb012acf3f\" (UID: \"9ff96acc-75a8-48e2-9709-8abb012acf3f\") " Jul 16 17:33:55.327415 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:33:55.327266 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tokenizer-tmp\" (UniqueName: \"kubernetes.io/empty-dir/9ff96acc-75a8-48e2-9709-8abb012acf3f-tokenizer-tmp\") pod \"9ff96acc-75a8-48e2-9709-8abb012acf3f\" (UID: \"9ff96acc-75a8-48e2-9709-8abb012acf3f\") " Jul 16 17:33:55.327415 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:33:55.327297 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/9ff96acc-75a8-48e2-9709-8abb012acf3f-kserve-provision-location\") pod \"9ff96acc-75a8-48e2-9709-8abb012acf3f\" (UID: \"9ff96acc-75a8-48e2-9709-8abb012acf3f\") " Jul 16 17:33:55.327525 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:33:55.327439 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9ff96acc-75a8-48e2-9709-8abb012acf3f-tokenizer-uds" (OuterVolumeSpecName: "tokenizer-uds") pod "9ff96acc-75a8-48e2-9709-8abb012acf3f" (UID: "9ff96acc-75a8-48e2-9709-8abb012acf3f"). InnerVolumeSpecName "tokenizer-uds". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 16 17:33:55.327578 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:33:55.327555 2578 reconciler_common.go:299] "Volume detached for volume \"tokenizer-uds\" (UniqueName: \"kubernetes.io/empty-dir/9ff96acc-75a8-48e2-9709-8abb012acf3f-tokenizer-uds\") on node \"ip-10-0-132-130.ec2.internal\" DevicePath \"\"" Jul 16 17:33:55.327636 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:33:55.327610 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9ff96acc-75a8-48e2-9709-8abb012acf3f-tokenizer-tmp" (OuterVolumeSpecName: "tokenizer-tmp") pod "9ff96acc-75a8-48e2-9709-8abb012acf3f" (UID: "9ff96acc-75a8-48e2-9709-8abb012acf3f"). InnerVolumeSpecName "tokenizer-tmp". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 16 17:33:55.327670 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:33:55.327650 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9ff96acc-75a8-48e2-9709-8abb012acf3f-tokenizer-cache" (OuterVolumeSpecName: "tokenizer-cache") pod "9ff96acc-75a8-48e2-9709-8abb012acf3f" (UID: "9ff96acc-75a8-48e2-9709-8abb012acf3f"). InnerVolumeSpecName "tokenizer-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 16 17:33:55.328063 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:33:55.328041 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9ff96acc-75a8-48e2-9709-8abb012acf3f-kserve-provision-location" (OuterVolumeSpecName: "kserve-provision-location") pod "9ff96acc-75a8-48e2-9709-8abb012acf3f" (UID: "9ff96acc-75a8-48e2-9709-8abb012acf3f"). InnerVolumeSpecName "kserve-provision-location". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 16 17:33:55.329198 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:33:55.329174 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9ff96acc-75a8-48e2-9709-8abb012acf3f-kube-api-access-x4b9p" (OuterVolumeSpecName: "kube-api-access-x4b9p") pod "9ff96acc-75a8-48e2-9709-8abb012acf3f" (UID: "9ff96acc-75a8-48e2-9709-8abb012acf3f"). InnerVolumeSpecName "kube-api-access-x4b9p". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 16 17:33:55.329354 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:33:55.329329 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/9ff96acc-75a8-48e2-9709-8abb012acf3f-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "9ff96acc-75a8-48e2-9709-8abb012acf3f" (UID: "9ff96acc-75a8-48e2-9709-8abb012acf3f"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 16 17:33:55.428785 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:33:55.428746 2578 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/9ff96acc-75a8-48e2-9709-8abb012acf3f-tls-certs\") on node \"ip-10-0-132-130.ec2.internal\" DevicePath \"\"" Jul 16 17:33:55.428785 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:33:55.428783 2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-x4b9p\" (UniqueName: \"kubernetes.io/projected/9ff96acc-75a8-48e2-9709-8abb012acf3f-kube-api-access-x4b9p\") on node \"ip-10-0-132-130.ec2.internal\" DevicePath \"\"" Jul 16 17:33:55.428785 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:33:55.428794 2578 reconciler_common.go:299] "Volume detached for volume \"tokenizer-cache\" (UniqueName: \"kubernetes.io/empty-dir/9ff96acc-75a8-48e2-9709-8abb012acf3f-tokenizer-cache\") on node \"ip-10-0-132-130.ec2.internal\" DevicePath \"\"" Jul 16 17:33:55.428973 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:33:55.428803 2578 reconciler_common.go:299] "Volume detached for volume \"tokenizer-tmp\" (UniqueName: \"kubernetes.io/empty-dir/9ff96acc-75a8-48e2-9709-8abb012acf3f-tokenizer-tmp\") on node \"ip-10-0-132-130.ec2.internal\" DevicePath \"\"" Jul 16 17:33:55.428973 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:33:55.428811 2578 reconciler_common.go:299] "Volume detached for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/9ff96acc-75a8-48e2-9709-8abb012acf3f-kserve-provision-location\") on node \"ip-10-0-132-130.ec2.internal\" DevicePath \"\"" Jul 16 17:33:56.299734 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:33:56.299684 2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/precise-prefix-cache-test-kserve-router-scheduler-75779d4bffhzf" Jul 16 17:33:56.361858 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:33:56.361827 2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/precise-prefix-cache-test-kserve-router-scheduler-75779d4bffhzf"] Jul 16 17:33:56.363907 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:33:56.363873 2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kserve-ci-e2e-test/precise-prefix-cache-test-kserve-router-scheduler-75779d4bffhzf"] Jul 16 17:33:58.215948 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:33:58.215918 2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="9ff96acc-75a8-48e2-9709-8abb012acf3f" path="/var/lib/kubelet/pods/9ff96acc-75a8-48e2-9709-8abb012acf3f/volumes" Jul 16 17:34:09.483933 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:34:09.483897 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve-ci-e2e-test/llmisvc-router-managed-test-llm-e45d1f79-kserve-df8675f66-sskh8"] Jul 16 17:34:09.484374 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:34:09.484288 2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="dd594272-3fa8-4587-9386-600307eb243d" containerName="storage-initializer" Jul 16 17:34:09.484374 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:34:09.484299 2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="dd594272-3fa8-4587-9386-600307eb243d" containerName="storage-initializer" Jul 16 17:34:09.484374 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:34:09.484309 2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9ff96acc-75a8-48e2-9709-8abb012acf3f" containerName="storage-initializer" Jul 16 17:34:09.484374 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:34:09.484315 2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="9ff96acc-75a8-48e2-9709-8abb012acf3f" containerName="storage-initializer" Jul 16 17:34:09.484374 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:34:09.484323 2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="dd594272-3fa8-4587-9386-600307eb243d" containerName="main" Jul 16 17:34:09.484374 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:34:09.484329 2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="dd594272-3fa8-4587-9386-600307eb243d" containerName="main" Jul 16 17:34:09.484374 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:34:09.484339 2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9ff96acc-75a8-48e2-9709-8abb012acf3f" containerName="main" Jul 16 17:34:09.484374 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:34:09.484344 2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="9ff96acc-75a8-48e2-9709-8abb012acf3f" containerName="main" Jul 16 17:34:09.484374 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:34:09.484354 2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9ff96acc-75a8-48e2-9709-8abb012acf3f" containerName="main" Jul 16 17:34:09.484374 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:34:09.484359 2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="9ff96acc-75a8-48e2-9709-8abb012acf3f" containerName="main" Jul 16 17:34:09.484374 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:34:09.484373 2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9ff96acc-75a8-48e2-9709-8abb012acf3f" containerName="main" Jul 16 17:34:09.484374 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:34:09.484378 2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="9ff96acc-75a8-48e2-9709-8abb012acf3f" containerName="main" Jul 16 17:34:09.484765 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:34:09.484387 2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9ff96acc-75a8-48e2-9709-8abb012acf3f" containerName="tokenizer" Jul 16 17:34:09.484765 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:34:09.484392 2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="9ff96acc-75a8-48e2-9709-8abb012acf3f" containerName="tokenizer" Jul 16 17:34:09.484765 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:34:09.484444 2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="9ff96acc-75a8-48e2-9709-8abb012acf3f" containerName="tokenizer" Jul 16 17:34:09.484765 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:34:09.484453 2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="dd594272-3fa8-4587-9386-600307eb243d" containerName="main" Jul 16 17:34:09.484765 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:34:09.484461 2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="9ff96acc-75a8-48e2-9709-8abb012acf3f" containerName="main" Jul 16 17:34:09.484765 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:34:09.484467 2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="9ff96acc-75a8-48e2-9709-8abb012acf3f" containerName="main" Jul 16 17:34:09.488641 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:34:09.488613 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/llmisvc-router-managed-test-llm-e45d1f79-kserve-df8675f66-sskh8" Jul 16 17:34:09.492327 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:34:09.492305 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"openshift-service-ca.crt\"" Jul 16 17:34:09.492561 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:34:09.492361 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"default-dockercfg-sdm8r\"" Jul 16 17:34:09.492648 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:34:09.492562 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"kube-root-ca.crt\"" Jul 16 17:34:09.492930 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:34:09.492912 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"llmisv5c7e67b6c51568d1d6d13829a9337f2a-kserve-self-signed-certs\"" Jul 16 17:34:09.500349 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:34:09.500327 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/llmisvc-router-managed-test-llm-e45d1f79-kserve-df8675f66-sskh8"] Jul 16 17:34:09.545611 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:34:09.545584 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/4e9df972-7b0b-41b4-bd4b-92387d070889-dshm\") pod \"llmisvc-router-managed-test-llm-e45d1f79-kserve-df8675f66-sskh8\" (UID: \"4e9df972-7b0b-41b4-bd4b-92387d070889\") " pod="kserve-ci-e2e-test/llmisvc-router-managed-test-llm-e45d1f79-kserve-df8675f66-sskh8" Jul 16 17:34:09.545768 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:34:09.545629 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-g6hrx\" (UniqueName: \"kubernetes.io/projected/4e9df972-7b0b-41b4-bd4b-92387d070889-kube-api-access-g6hrx\") pod \"llmisvc-router-managed-test-llm-e45d1f79-kserve-df8675f66-sskh8\" (UID: \"4e9df972-7b0b-41b4-bd4b-92387d070889\") " pod="kserve-ci-e2e-test/llmisvc-router-managed-test-llm-e45d1f79-kserve-df8675f66-sskh8" Jul 16 17:34:09.545768 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:34:09.545658 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/4e9df972-7b0b-41b4-bd4b-92387d070889-home\") pod \"llmisvc-router-managed-test-llm-e45d1f79-kserve-df8675f66-sskh8\" (UID: \"4e9df972-7b0b-41b4-bd4b-92387d070889\") " pod="kserve-ci-e2e-test/llmisvc-router-managed-test-llm-e45d1f79-kserve-df8675f66-sskh8" Jul 16 17:34:09.545768 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:34:09.545710 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4e9df972-7b0b-41b4-bd4b-92387d070889-tmp-dir\") pod \"llmisvc-router-managed-test-llm-e45d1f79-kserve-df8675f66-sskh8\" (UID: \"4e9df972-7b0b-41b4-bd4b-92387d070889\") " pod="kserve-ci-e2e-test/llmisvc-router-managed-test-llm-e45d1f79-kserve-df8675f66-sskh8" Jul 16 17:34:09.545768 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:34:09.545757 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4e9df972-7b0b-41b4-bd4b-92387d070889-tls-certs\") pod \"llmisvc-router-managed-test-llm-e45d1f79-kserve-df8675f66-sskh8\" (UID: \"4e9df972-7b0b-41b4-bd4b-92387d070889\") " pod="kserve-ci-e2e-test/llmisvc-router-managed-test-llm-e45d1f79-kserve-df8675f66-sskh8" Jul 16 17:34:09.545917 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:34:09.545790 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/4e9df972-7b0b-41b4-bd4b-92387d070889-model-cache\") pod \"llmisvc-router-managed-test-llm-e45d1f79-kserve-df8675f66-sskh8\" (UID: \"4e9df972-7b0b-41b4-bd4b-92387d070889\") " pod="kserve-ci-e2e-test/llmisvc-router-managed-test-llm-e45d1f79-kserve-df8675f66-sskh8" Jul 16 17:34:09.647034 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:34:09.646997 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/4e9df972-7b0b-41b4-bd4b-92387d070889-dshm\") pod \"llmisvc-router-managed-test-llm-e45d1f79-kserve-df8675f66-sskh8\" (UID: \"4e9df972-7b0b-41b4-bd4b-92387d070889\") " pod="kserve-ci-e2e-test/llmisvc-router-managed-test-llm-e45d1f79-kserve-df8675f66-sskh8" Jul 16 17:34:09.647191 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:34:09.647069 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-g6hrx\" (UniqueName: \"kubernetes.io/projected/4e9df972-7b0b-41b4-bd4b-92387d070889-kube-api-access-g6hrx\") pod \"llmisvc-router-managed-test-llm-e45d1f79-kserve-df8675f66-sskh8\" (UID: \"4e9df972-7b0b-41b4-bd4b-92387d070889\") " pod="kserve-ci-e2e-test/llmisvc-router-managed-test-llm-e45d1f79-kserve-df8675f66-sskh8" Jul 16 17:34:09.647191 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:34:09.647107 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/4e9df972-7b0b-41b4-bd4b-92387d070889-home\") pod \"llmisvc-router-managed-test-llm-e45d1f79-kserve-df8675f66-sskh8\" (UID: \"4e9df972-7b0b-41b4-bd4b-92387d070889\") " pod="kserve-ci-e2e-test/llmisvc-router-managed-test-llm-e45d1f79-kserve-df8675f66-sskh8" Jul 16 17:34:09.647301 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:34:09.647223 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4e9df972-7b0b-41b4-bd4b-92387d070889-tmp-dir\") pod \"llmisvc-router-managed-test-llm-e45d1f79-kserve-df8675f66-sskh8\" (UID: \"4e9df972-7b0b-41b4-bd4b-92387d070889\") " pod="kserve-ci-e2e-test/llmisvc-router-managed-test-llm-e45d1f79-kserve-df8675f66-sskh8" Jul 16 17:34:09.647301 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:34:09.647255 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4e9df972-7b0b-41b4-bd4b-92387d070889-tls-certs\") pod \"llmisvc-router-managed-test-llm-e45d1f79-kserve-df8675f66-sskh8\" (UID: \"4e9df972-7b0b-41b4-bd4b-92387d070889\") " pod="kserve-ci-e2e-test/llmisvc-router-managed-test-llm-e45d1f79-kserve-df8675f66-sskh8" Jul 16 17:34:09.647404 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:34:09.647298 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/4e9df972-7b0b-41b4-bd4b-92387d070889-model-cache\") pod \"llmisvc-router-managed-test-llm-e45d1f79-kserve-df8675f66-sskh8\" (UID: \"4e9df972-7b0b-41b4-bd4b-92387d070889\") " pod="kserve-ci-e2e-test/llmisvc-router-managed-test-llm-e45d1f79-kserve-df8675f66-sskh8" Jul 16 17:34:09.647541 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:34:09.647518 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/4e9df972-7b0b-41b4-bd4b-92387d070889-home\") pod \"llmisvc-router-managed-test-llm-e45d1f79-kserve-df8675f66-sskh8\" (UID: \"4e9df972-7b0b-41b4-bd4b-92387d070889\") " pod="kserve-ci-e2e-test/llmisvc-router-managed-test-llm-e45d1f79-kserve-df8675f66-sskh8" Jul 16 17:34:09.647717 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:34:09.647577 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4e9df972-7b0b-41b4-bd4b-92387d070889-tmp-dir\") pod \"llmisvc-router-managed-test-llm-e45d1f79-kserve-df8675f66-sskh8\" (UID: \"4e9df972-7b0b-41b4-bd4b-92387d070889\") " pod="kserve-ci-e2e-test/llmisvc-router-managed-test-llm-e45d1f79-kserve-df8675f66-sskh8" Jul 16 17:34:09.647717 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:34:09.647617 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/4e9df972-7b0b-41b4-bd4b-92387d070889-model-cache\") pod \"llmisvc-router-managed-test-llm-e45d1f79-kserve-df8675f66-sskh8\" (UID: \"4e9df972-7b0b-41b4-bd4b-92387d070889\") " pod="kserve-ci-e2e-test/llmisvc-router-managed-test-llm-e45d1f79-kserve-df8675f66-sskh8" Jul 16 17:34:09.649270 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:34:09.649245 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/4e9df972-7b0b-41b4-bd4b-92387d070889-dshm\") pod \"llmisvc-router-managed-test-llm-e45d1f79-kserve-df8675f66-sskh8\" (UID: \"4e9df972-7b0b-41b4-bd4b-92387d070889\") " pod="kserve-ci-e2e-test/llmisvc-router-managed-test-llm-e45d1f79-kserve-df8675f66-sskh8" Jul 16 17:34:09.649659 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:34:09.649638 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4e9df972-7b0b-41b4-bd4b-92387d070889-tls-certs\") pod \"llmisvc-router-managed-test-llm-e45d1f79-kserve-df8675f66-sskh8\" (UID: \"4e9df972-7b0b-41b4-bd4b-92387d070889\") " pod="kserve-ci-e2e-test/llmisvc-router-managed-test-llm-e45d1f79-kserve-df8675f66-sskh8" Jul 16 17:34:09.674884 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:34:09.674852 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-g6hrx\" (UniqueName: \"kubernetes.io/projected/4e9df972-7b0b-41b4-bd4b-92387d070889-kube-api-access-g6hrx\") pod \"llmisvc-router-managed-test-llm-e45d1f79-kserve-df8675f66-sskh8\" (UID: \"4e9df972-7b0b-41b4-bd4b-92387d070889\") " pod="kserve-ci-e2e-test/llmisvc-router-managed-test-llm-e45d1f79-kserve-df8675f66-sskh8" Jul 16 17:34:09.802930 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:34:09.802904 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/llmisvc-router-managed-test-llm-e45d1f79-kserve-df8675f66-sskh8" Jul 16 17:34:09.961768 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:34:09.961685 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/llmisvc-router-managed-test-llm-e45d1f79-kserve-df8675f66-sskh8"] Jul 16 17:34:09.964151 ip-10-0-132-130 kubenswrapper[2578]: W0716 17:34:09.964123 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4e9df972_7b0b_41b4_bd4b_92387d070889.slice/crio-d8d2519b0df1566f0e32d2b2fe0fbaea07b06076e51de6c372e595f18c2de7ad WatchSource:0}: Error finding container d8d2519b0df1566f0e32d2b2fe0fbaea07b06076e51de6c372e595f18c2de7ad: Status 404 returned error can't find the container with id d8d2519b0df1566f0e32d2b2fe0fbaea07b06076e51de6c372e595f18c2de7ad Jul 16 17:34:10.378247 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:34:10.378213 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/llmisvc-router-managed-test-llm-e45d1f79-kserve-df8675f66-sskh8" event={"ID":"4e9df972-7b0b-41b4-bd4b-92387d070889","Type":"ContainerStarted","Data":"48d21aee6a2197a19b5f25defa20e709c606ddfb9d5381d2dbd788b78646bebe"} Jul 16 17:34:10.378405 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:34:10.378271 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/llmisvc-router-managed-test-llm-e45d1f79-kserve-df8675f66-sskh8" event={"ID":"4e9df972-7b0b-41b4-bd4b-92387d070889","Type":"ContainerStarted","Data":"d8d2519b0df1566f0e32d2b2fe0fbaea07b06076e51de6c372e595f18c2de7ad"} Jul 16 17:34:10.414577 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:34:10.414529 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve-ci-e2e-test/llmisvc-router-managed-test-llm-e45d1f79-kserve-df8675f66-sskh8" podStartSLOduration=1.414515677 podStartE2EDuration="1.414515677s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 17:34:09 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 17:34:10.413986605 +0000 UTC m=+1194.899830498" watchObservedRunningTime="2026-07-16 17:34:10.414515677 +0000 UTC m=+1194.900359568" Jul 16 17:34:10.803655 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:34:10.803620 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/llmisvc-router-managed-test-llm-e45d1f79-kserve-df8675f66-sskh8" Jul 16 17:34:16.298427 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:34:16.298395 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-drs2p_e55f447c-ad64-482b-a86b-0588bddf7b4d/console-operator/1.log" Jul 16 17:34:16.299003 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:34:16.298973 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-drs2p_e55f447c-ad64-482b-a86b-0588bddf7b4d/console-operator/1.log" Jul 16 17:34:19.803420 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:34:19.803385 2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kserve-ci-e2e-test/llmisvc-router-managed-test-llm-e45d1f79-kserve-df8675f66-sskh8" Jul 16 17:34:19.816193 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:34:19.816171 2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kserve-ci-e2e-test/llmisvc-router-managed-test-llm-e45d1f79-kserve-df8675f66-sskh8" Jul 16 17:34:20.455535 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:34:20.455509 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/llmisvc-router-managed-test-llm-e45d1f79-kserve-df8675f66-sskh8" Jul 16 17:34:44.000306 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:34:44.000265 2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/llmisvc-router-managed-test-llm-e45d1f79-kserve-df8675f66-sskh8"] Jul 16 17:34:44.009324 ip-10-0-132-130 kubenswrapper[2578]: E0716 17:34:44.009293 2578 log.go:32] "ExecSync cmd from runtime service failed" err=< Jul 16 17:34:44.009324 ip-10-0-132-130 kubenswrapper[2578]: rpc error: code = Unknown desc = command error: executable file `/bin/sleep` not found: No such file or directory Jul 16 17:34:44.009324 ip-10-0-132-130 kubenswrapper[2578]: , stdout: , stderr: , exit code -1 Jul 16 17:34:44.009324 ip-10-0-132-130 kubenswrapper[2578]: > containerID="48d21aee6a2197a19b5f25defa20e709c606ddfb9d5381d2dbd788b78646bebe" cmd=["/bin/sleep","15"] Jul 16 17:34:44.009554 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:34:44.009340 2578 handlers.go:82] "Exec lifecycle hook for Container in Pod failed" execCommand=["/bin/sleep","15"] containerName="main" pod="kserve-ci-e2e-test/llmisvc-router-managed-test-llm-e45d1f79-kserve-df8675f66-sskh8" message="" err=< Jul 16 17:34:44.009554 ip-10-0-132-130 kubenswrapper[2578]: rpc error: code = Unknown desc = command error: executable file `/bin/sleep` not found: No such file or directory Jul 16 17:34:44.009554 ip-10-0-132-130 kubenswrapper[2578]: , stdout: , stderr: , exit code -1 Jul 16 17:34:44.009554 ip-10-0-132-130 kubenswrapper[2578]: > Jul 16 17:34:44.009554 ip-10-0-132-130 kubenswrapper[2578]: E0716 17:34:44.009352 2578 kuberuntime_container.go:755] "PreStop hook failed" err=< Jul 16 17:34:44.009554 ip-10-0-132-130 kubenswrapper[2578]: rpc error: code = Unknown desc = command error: executable file `/bin/sleep` not found: No such file or directory Jul 16 17:34:44.009554 ip-10-0-132-130 kubenswrapper[2578]: , stdout: , stderr: , exit code -1 Jul 16 17:34:44.009554 ip-10-0-132-130 kubenswrapper[2578]: > pod="kserve-ci-e2e-test/llmisvc-router-managed-test-llm-e45d1f79-kserve-df8675f66-sskh8" podUID="4e9df972-7b0b-41b4-bd4b-92387d070889" containerName="main" containerID="cri-o://48d21aee6a2197a19b5f25defa20e709c606ddfb9d5381d2dbd788b78646bebe" Jul 16 17:34:44.009554 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:34:44.009385 2578 kuberuntime_container.go:873] "Killing container with a grace period" pod="kserve-ci-e2e-test/llmisvc-router-managed-test-llm-e45d1f79-kserve-df8675f66-sskh8" podUID="4e9df972-7b0b-41b4-bd4b-92387d070889" containerName="main" containerID="cri-o://48d21aee6a2197a19b5f25defa20e709c606ddfb9d5381d2dbd788b78646bebe" gracePeriod=60 Jul 16 17:34:44.372964 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:34:44.372939 2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/llmisvc-router-managed-test-llm-e45d1f79-kserve-df8675f66-sskh8" Jul 16 17:34:44.445231 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:34:44.445207 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/4e9df972-7b0b-41b4-bd4b-92387d070889-dshm\") pod \"4e9df972-7b0b-41b4-bd4b-92387d070889\" (UID: \"4e9df972-7b0b-41b4-bd4b-92387d070889\") " Jul 16 17:34:44.445371 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:34:44.445241 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4e9df972-7b0b-41b4-bd4b-92387d070889-tmp-dir\") pod \"4e9df972-7b0b-41b4-bd4b-92387d070889\" (UID: \"4e9df972-7b0b-41b4-bd4b-92387d070889\") " Jul 16 17:34:44.445371 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:34:44.445281 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-g6hrx\" (UniqueName: \"kubernetes.io/projected/4e9df972-7b0b-41b4-bd4b-92387d070889-kube-api-access-g6hrx\") pod \"4e9df972-7b0b-41b4-bd4b-92387d070889\" (UID: \"4e9df972-7b0b-41b4-bd4b-92387d070889\") " Jul 16 17:34:44.445371 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:34:44.445295 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/4e9df972-7b0b-41b4-bd4b-92387d070889-home\") pod \"4e9df972-7b0b-41b4-bd4b-92387d070889\" (UID: \"4e9df972-7b0b-41b4-bd4b-92387d070889\") " Jul 16 17:34:44.445371 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:34:44.445344 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4e9df972-7b0b-41b4-bd4b-92387d070889-tls-certs\") pod \"4e9df972-7b0b-41b4-bd4b-92387d070889\" (UID: \"4e9df972-7b0b-41b4-bd4b-92387d070889\") " Jul 16 17:34:44.445371 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:34:44.445369 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/4e9df972-7b0b-41b4-bd4b-92387d070889-model-cache\") pod \"4e9df972-7b0b-41b4-bd4b-92387d070889\" (UID: \"4e9df972-7b0b-41b4-bd4b-92387d070889\") " Jul 16 17:34:44.445624 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:34:44.445521 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4e9df972-7b0b-41b4-bd4b-92387d070889-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "4e9df972-7b0b-41b4-bd4b-92387d070889" (UID: "4e9df972-7b0b-41b4-bd4b-92387d070889"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 16 17:34:44.445624 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:34:44.445583 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4e9df972-7b0b-41b4-bd4b-92387d070889-home" (OuterVolumeSpecName: "home") pod "4e9df972-7b0b-41b4-bd4b-92387d070889" (UID: "4e9df972-7b0b-41b4-bd4b-92387d070889"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 16 17:34:44.445768 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:34:44.445719 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4e9df972-7b0b-41b4-bd4b-92387d070889-model-cache" (OuterVolumeSpecName: "model-cache") pod "4e9df972-7b0b-41b4-bd4b-92387d070889" (UID: "4e9df972-7b0b-41b4-bd4b-92387d070889"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 16 17:34:44.445819 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:34:44.445775 2578 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4e9df972-7b0b-41b4-bd4b-92387d070889-tmp-dir\") on node \"ip-10-0-132-130.ec2.internal\" DevicePath \"\"" Jul 16 17:34:44.445819 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:34:44.445790 2578 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/4e9df972-7b0b-41b4-bd4b-92387d070889-home\") on node \"ip-10-0-132-130.ec2.internal\" DevicePath \"\"" Jul 16 17:34:44.447211 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:34:44.447181 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4e9df972-7b0b-41b4-bd4b-92387d070889-dshm" (OuterVolumeSpecName: "dshm") pod "4e9df972-7b0b-41b4-bd4b-92387d070889" (UID: "4e9df972-7b0b-41b4-bd4b-92387d070889"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 16 17:34:44.447317 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:34:44.447297 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4e9df972-7b0b-41b4-bd4b-92387d070889-kube-api-access-g6hrx" (OuterVolumeSpecName: "kube-api-access-g6hrx") pod "4e9df972-7b0b-41b4-bd4b-92387d070889" (UID: "4e9df972-7b0b-41b4-bd4b-92387d070889"). InnerVolumeSpecName "kube-api-access-g6hrx". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 16 17:34:44.447359 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:34:44.447309 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/4e9df972-7b0b-41b4-bd4b-92387d070889-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "4e9df972-7b0b-41b4-bd4b-92387d070889" (UID: "4e9df972-7b0b-41b4-bd4b-92387d070889"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 16 17:34:44.546140 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:34:44.546116 2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-g6hrx\" (UniqueName: \"kubernetes.io/projected/4e9df972-7b0b-41b4-bd4b-92387d070889-kube-api-access-g6hrx\") on node \"ip-10-0-132-130.ec2.internal\" DevicePath \"\"" Jul 16 17:34:44.546140 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:34:44.546143 2578 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4e9df972-7b0b-41b4-bd4b-92387d070889-tls-certs\") on node \"ip-10-0-132-130.ec2.internal\" DevicePath \"\"" Jul 16 17:34:44.546330 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:34:44.546152 2578 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/4e9df972-7b0b-41b4-bd4b-92387d070889-model-cache\") on node \"ip-10-0-132-130.ec2.internal\" DevicePath \"\"" Jul 16 17:34:44.546330 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:34:44.546160 2578 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/4e9df972-7b0b-41b4-bd4b-92387d070889-dshm\") on node \"ip-10-0-132-130.ec2.internal\" DevicePath \"\"" Jul 16 17:34:44.583281 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:34:44.583246 2578 generic.go:356] "Generic (PLEG): container finished" podID="4e9df972-7b0b-41b4-bd4b-92387d070889" containerID="48d21aee6a2197a19b5f25defa20e709c606ddfb9d5381d2dbd788b78646bebe" exitCode=0 Jul 16 17:34:44.583426 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:34:44.583298 2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/llmisvc-router-managed-test-llm-e45d1f79-kserve-df8675f66-sskh8" Jul 16 17:34:44.583426 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:34:44.583329 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/llmisvc-router-managed-test-llm-e45d1f79-kserve-df8675f66-sskh8" event={"ID":"4e9df972-7b0b-41b4-bd4b-92387d070889","Type":"ContainerDied","Data":"48d21aee6a2197a19b5f25defa20e709c606ddfb9d5381d2dbd788b78646bebe"} Jul 16 17:34:44.583426 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:34:44.583380 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/llmisvc-router-managed-test-llm-e45d1f79-kserve-df8675f66-sskh8" event={"ID":"4e9df972-7b0b-41b4-bd4b-92387d070889","Type":"ContainerDied","Data":"d8d2519b0df1566f0e32d2b2fe0fbaea07b06076e51de6c372e595f18c2de7ad"} Jul 16 17:34:44.583426 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:34:44.583402 2578 scope.go:117] "RemoveContainer" containerID="48d21aee6a2197a19b5f25defa20e709c606ddfb9d5381d2dbd788b78646bebe" Jul 16 17:34:44.596219 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:34:44.596203 2578 scope.go:117] "RemoveContainer" containerID="48d21aee6a2197a19b5f25defa20e709c606ddfb9d5381d2dbd788b78646bebe" Jul 16 17:34:44.596495 ip-10-0-132-130 kubenswrapper[2578]: E0716 17:34:44.596475 2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"48d21aee6a2197a19b5f25defa20e709c606ddfb9d5381d2dbd788b78646bebe\": container with ID starting with 48d21aee6a2197a19b5f25defa20e709c606ddfb9d5381d2dbd788b78646bebe not found: ID does not exist" containerID="48d21aee6a2197a19b5f25defa20e709c606ddfb9d5381d2dbd788b78646bebe" Jul 16 17:34:44.596553 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:34:44.596504 2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"48d21aee6a2197a19b5f25defa20e709c606ddfb9d5381d2dbd788b78646bebe"} err="failed to get container status \"48d21aee6a2197a19b5f25defa20e709c606ddfb9d5381d2dbd788b78646bebe\": rpc error: code = NotFound desc = could not find container \"48d21aee6a2197a19b5f25defa20e709c606ddfb9d5381d2dbd788b78646bebe\": container with ID starting with 48d21aee6a2197a19b5f25defa20e709c606ddfb9d5381d2dbd788b78646bebe not found: ID does not exist" Jul 16 17:34:44.616681 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:34:44.616654 2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/llmisvc-router-managed-test-llm-e45d1f79-kserve-df8675f66-sskh8"] Jul 16 17:34:44.622242 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:34:44.622219 2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kserve-ci-e2e-test/llmisvc-router-managed-test-llm-e45d1f79-kserve-df8675f66-sskh8"] Jul 16 17:34:46.216683 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:34:46.216647 2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="4e9df972-7b0b-41b4-bd4b-92387d070889" path="/var/lib/kubelet/pods/4e9df972-7b0b-41b4-bd4b-92387d070889/volumes" Jul 16 17:35:11.854417 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:35:11.854385 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve-ci-e2e-test/llmisvc-router-managed-test-llm-5b1e8f15-kserve-df5b5c977-xdvj2"] Jul 16 17:35:11.856866 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:35:11.854855 2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4e9df972-7b0b-41b4-bd4b-92387d070889" containerName="main" Jul 16 17:35:11.856866 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:35:11.854869 2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="4e9df972-7b0b-41b4-bd4b-92387d070889" containerName="main" Jul 16 17:35:11.856866 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:35:11.854937 2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="9ff96acc-75a8-48e2-9709-8abb012acf3f" containerName="main" Jul 16 17:35:11.856866 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:35:11.854944 2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="4e9df972-7b0b-41b4-bd4b-92387d070889" containerName="main" Jul 16 17:35:11.858072 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:35:11.858054 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/llmisvc-router-managed-test-llm-5b1e8f15-kserve-df5b5c977-xdvj2" Jul 16 17:35:11.860151 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:35:11.860135 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"default-dockercfg-sdm8r\"" Jul 16 17:35:11.860290 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:35:11.860270 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"kube-root-ca.crt\"" Jul 16 17:35:11.860960 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:35:11.860944 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"llmisve55ae740357a3a31a27cdb8b66ffe20f-kserve-self-signed-certs\"" Jul 16 17:35:11.861494 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:35:11.861477 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"openshift-service-ca.crt\"" Jul 16 17:35:11.871118 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:35:11.871094 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/llmisvc-router-managed-test-llm-5b1e8f15-kserve-df5b5c977-xdvj2"] Jul 16 17:35:11.977557 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:35:11.977518 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ae64d07c-dbee-4be2-936d-7efc5dfdbd89-model-cache\") pod \"llmisvc-router-managed-test-llm-5b1e8f15-kserve-df5b5c977-xdvj2\" (UID: \"ae64d07c-dbee-4be2-936d-7efc5dfdbd89\") " pod="kserve-ci-e2e-test/llmisvc-router-managed-test-llm-5b1e8f15-kserve-df5b5c977-xdvj2" Jul 16 17:35:11.977557 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:35:11.977566 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ae64d07c-dbee-4be2-936d-7efc5dfdbd89-tmp-dir\") pod \"llmisvc-router-managed-test-llm-5b1e8f15-kserve-df5b5c977-xdvj2\" (UID: \"ae64d07c-dbee-4be2-936d-7efc5dfdbd89\") " pod="kserve-ci-e2e-test/llmisvc-router-managed-test-llm-5b1e8f15-kserve-df5b5c977-xdvj2" Jul 16 17:35:11.977799 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:35:11.977595 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ae64d07c-dbee-4be2-936d-7efc5dfdbd89-tls-certs\") pod \"llmisvc-router-managed-test-llm-5b1e8f15-kserve-df5b5c977-xdvj2\" (UID: \"ae64d07c-dbee-4be2-936d-7efc5dfdbd89\") " pod="kserve-ci-e2e-test/llmisvc-router-managed-test-llm-5b1e8f15-kserve-df5b5c977-xdvj2" Jul 16 17:35:11.977799 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:35:11.977666 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/ae64d07c-dbee-4be2-936d-7efc5dfdbd89-home\") pod \"llmisvc-router-managed-test-llm-5b1e8f15-kserve-df5b5c977-xdvj2\" (UID: \"ae64d07c-dbee-4be2-936d-7efc5dfdbd89\") " pod="kserve-ci-e2e-test/llmisvc-router-managed-test-llm-5b1e8f15-kserve-df5b5c977-xdvj2" Jul 16 17:35:11.977799 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:35:11.977746 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/ae64d07c-dbee-4be2-936d-7efc5dfdbd89-dshm\") pod \"llmisvc-router-managed-test-llm-5b1e8f15-kserve-df5b5c977-xdvj2\" (UID: \"ae64d07c-dbee-4be2-936d-7efc5dfdbd89\") " pod="kserve-ci-e2e-test/llmisvc-router-managed-test-llm-5b1e8f15-kserve-df5b5c977-xdvj2" Jul 16 17:35:11.977942 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:35:11.977867 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-v9xst\" (UniqueName: \"kubernetes.io/projected/ae64d07c-dbee-4be2-936d-7efc5dfdbd89-kube-api-access-v9xst\") pod \"llmisvc-router-managed-test-llm-5b1e8f15-kserve-df5b5c977-xdvj2\" (UID: \"ae64d07c-dbee-4be2-936d-7efc5dfdbd89\") " pod="kserve-ci-e2e-test/llmisvc-router-managed-test-llm-5b1e8f15-kserve-df5b5c977-xdvj2" Jul 16 17:35:12.078980 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:35:12.078948 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-v9xst\" (UniqueName: \"kubernetes.io/projected/ae64d07c-dbee-4be2-936d-7efc5dfdbd89-kube-api-access-v9xst\") pod \"llmisvc-router-managed-test-llm-5b1e8f15-kserve-df5b5c977-xdvj2\" (UID: \"ae64d07c-dbee-4be2-936d-7efc5dfdbd89\") " pod="kserve-ci-e2e-test/llmisvc-router-managed-test-llm-5b1e8f15-kserve-df5b5c977-xdvj2" Jul 16 17:35:12.079149 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:35:12.078989 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ae64d07c-dbee-4be2-936d-7efc5dfdbd89-model-cache\") pod \"llmisvc-router-managed-test-llm-5b1e8f15-kserve-df5b5c977-xdvj2\" (UID: \"ae64d07c-dbee-4be2-936d-7efc5dfdbd89\") " pod="kserve-ci-e2e-test/llmisvc-router-managed-test-llm-5b1e8f15-kserve-df5b5c977-xdvj2" Jul 16 17:35:12.079149 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:35:12.079014 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ae64d07c-dbee-4be2-936d-7efc5dfdbd89-tmp-dir\") pod \"llmisvc-router-managed-test-llm-5b1e8f15-kserve-df5b5c977-xdvj2\" (UID: \"ae64d07c-dbee-4be2-936d-7efc5dfdbd89\") " pod="kserve-ci-e2e-test/llmisvc-router-managed-test-llm-5b1e8f15-kserve-df5b5c977-xdvj2" Jul 16 17:35:12.079149 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:35:12.079140 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ae64d07c-dbee-4be2-936d-7efc5dfdbd89-tls-certs\") pod \"llmisvc-router-managed-test-llm-5b1e8f15-kserve-df5b5c977-xdvj2\" (UID: \"ae64d07c-dbee-4be2-936d-7efc5dfdbd89\") " pod="kserve-ci-e2e-test/llmisvc-router-managed-test-llm-5b1e8f15-kserve-df5b5c977-xdvj2" Jul 16 17:35:12.079324 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:35:12.079183 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/ae64d07c-dbee-4be2-936d-7efc5dfdbd89-home\") pod \"llmisvc-router-managed-test-llm-5b1e8f15-kserve-df5b5c977-xdvj2\" (UID: \"ae64d07c-dbee-4be2-936d-7efc5dfdbd89\") " pod="kserve-ci-e2e-test/llmisvc-router-managed-test-llm-5b1e8f15-kserve-df5b5c977-xdvj2" Jul 16 17:35:12.079324 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:35:12.079232 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/ae64d07c-dbee-4be2-936d-7efc5dfdbd89-dshm\") pod \"llmisvc-router-managed-test-llm-5b1e8f15-kserve-df5b5c977-xdvj2\" (UID: \"ae64d07c-dbee-4be2-936d-7efc5dfdbd89\") " pod="kserve-ci-e2e-test/llmisvc-router-managed-test-llm-5b1e8f15-kserve-df5b5c977-xdvj2" Jul 16 17:35:12.079430 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:35:12.079385 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ae64d07c-dbee-4be2-936d-7efc5dfdbd89-tmp-dir\") pod \"llmisvc-router-managed-test-llm-5b1e8f15-kserve-df5b5c977-xdvj2\" (UID: \"ae64d07c-dbee-4be2-936d-7efc5dfdbd89\") " pod="kserve-ci-e2e-test/llmisvc-router-managed-test-llm-5b1e8f15-kserve-df5b5c977-xdvj2" Jul 16 17:35:12.079490 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:35:12.079455 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ae64d07c-dbee-4be2-936d-7efc5dfdbd89-model-cache\") pod \"llmisvc-router-managed-test-llm-5b1e8f15-kserve-df5b5c977-xdvj2\" (UID: \"ae64d07c-dbee-4be2-936d-7efc5dfdbd89\") " pod="kserve-ci-e2e-test/llmisvc-router-managed-test-llm-5b1e8f15-kserve-df5b5c977-xdvj2" Jul 16 17:35:12.079563 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:35:12.079543 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/ae64d07c-dbee-4be2-936d-7efc5dfdbd89-home\") pod \"llmisvc-router-managed-test-llm-5b1e8f15-kserve-df5b5c977-xdvj2\" (UID: \"ae64d07c-dbee-4be2-936d-7efc5dfdbd89\") " pod="kserve-ci-e2e-test/llmisvc-router-managed-test-llm-5b1e8f15-kserve-df5b5c977-xdvj2" Jul 16 17:35:12.081351 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:35:12.081331 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/ae64d07c-dbee-4be2-936d-7efc5dfdbd89-dshm\") pod \"llmisvc-router-managed-test-llm-5b1e8f15-kserve-df5b5c977-xdvj2\" (UID: \"ae64d07c-dbee-4be2-936d-7efc5dfdbd89\") " pod="kserve-ci-e2e-test/llmisvc-router-managed-test-llm-5b1e8f15-kserve-df5b5c977-xdvj2" Jul 16 17:35:12.081681 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:35:12.081661 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ae64d07c-dbee-4be2-936d-7efc5dfdbd89-tls-certs\") pod \"llmisvc-router-managed-test-llm-5b1e8f15-kserve-df5b5c977-xdvj2\" (UID: \"ae64d07c-dbee-4be2-936d-7efc5dfdbd89\") " pod="kserve-ci-e2e-test/llmisvc-router-managed-test-llm-5b1e8f15-kserve-df5b5c977-xdvj2" Jul 16 17:35:12.090785 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:35:12.090759 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-v9xst\" (UniqueName: \"kubernetes.io/projected/ae64d07c-dbee-4be2-936d-7efc5dfdbd89-kube-api-access-v9xst\") pod \"llmisvc-router-managed-test-llm-5b1e8f15-kserve-df5b5c977-xdvj2\" (UID: \"ae64d07c-dbee-4be2-936d-7efc5dfdbd89\") " pod="kserve-ci-e2e-test/llmisvc-router-managed-test-llm-5b1e8f15-kserve-df5b5c977-xdvj2" Jul 16 17:35:12.169767 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:35:12.169673 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/llmisvc-router-managed-test-llm-5b1e8f15-kserve-df5b5c977-xdvj2" Jul 16 17:35:12.316366 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:35:12.316342 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/llmisvc-router-managed-test-llm-5b1e8f15-kserve-df5b5c977-xdvj2"] Jul 16 17:35:12.318514 ip-10-0-132-130 kubenswrapper[2578]: W0716 17:35:12.318486 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podae64d07c_dbee_4be2_936d_7efc5dfdbd89.slice/crio-7845767462b5758e4b5bf2be4558c7b847bec33dc9f7ebd9aed262bbe0a6c242 WatchSource:0}: Error finding container 7845767462b5758e4b5bf2be4558c7b847bec33dc9f7ebd9aed262bbe0a6c242: Status 404 returned error can't find the container with id 7845767462b5758e4b5bf2be4558c7b847bec33dc9f7ebd9aed262bbe0a6c242 Jul 16 17:35:12.746656 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:35:12.746626 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/llmisvc-router-managed-test-llm-5b1e8f15-kserve-df5b5c977-xdvj2" event={"ID":"ae64d07c-dbee-4be2-936d-7efc5dfdbd89","Type":"ContainerStarted","Data":"b7c49ccdd97ff433045f7776c4a93751b8c6cdb19523a54b63d731b09f3f21d9"} Jul 16 17:35:12.746853 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:35:12.746674 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/llmisvc-router-managed-test-llm-5b1e8f15-kserve-df5b5c977-xdvj2" event={"ID":"ae64d07c-dbee-4be2-936d-7efc5dfdbd89","Type":"ContainerStarted","Data":"7845767462b5758e4b5bf2be4558c7b847bec33dc9f7ebd9aed262bbe0a6c242"} Jul 16 17:35:12.767584 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:35:12.767535 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve-ci-e2e-test/llmisvc-router-managed-test-llm-5b1e8f15-kserve-df5b5c977-xdvj2" podStartSLOduration=1.767518519 podStartE2EDuration="1.767518519s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 17:35:11 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 17:35:12.764624352 +0000 UTC m=+1257.250468241" watchObservedRunningTime="2026-07-16 17:35:12.767518519 +0000 UTC m=+1257.253362435" Jul 16 17:35:13.170747 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:35:13.170678 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/llmisvc-router-managed-test-llm-5b1e8f15-kserve-df5b5c977-xdvj2" Jul 16 17:35:22.170512 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:35:22.170474 2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kserve-ci-e2e-test/llmisvc-router-managed-test-llm-5b1e8f15-kserve-df5b5c977-xdvj2" Jul 16 17:35:22.182453 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:35:22.182431 2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kserve-ci-e2e-test/llmisvc-router-managed-test-llm-5b1e8f15-kserve-df5b5c977-xdvj2" Jul 16 17:35:22.821544 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:35:22.821514 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/llmisvc-router-managed-test-llm-5b1e8f15-kserve-df5b5c977-xdvj2" Jul 16 17:38:16.352024 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:38:16.351992 2578 scope.go:117] "RemoveContainer" containerID="0860226f79ac6405624e6c49ec2bdf9519ae60a90d6b57cae80458342f2c3dce" Jul 16 17:39:16.343429 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:39:16.343400 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-drs2p_e55f447c-ad64-482b-a86b-0588bddf7b4d/console-operator/1.log" Jul 16 17:39:16.346667 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:39:16.346646 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-drs2p_e55f447c-ad64-482b-a86b-0588bddf7b4d/console-operator/1.log" Jul 16 17:39:16.370079 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:39:16.370055 2578 scope.go:117] "RemoveContainer" containerID="c8bb73ce588ce15d9ee7af633bb7bb973817b630370bb6d52abbe2de3b97081d" Jul 16 17:39:16.381363 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:39:16.381344 2578 scope.go:117] "RemoveContainer" containerID="5711e1b5957c97d92061e7352b7360d9892ef7aa2e800c5932905f5008ae6de5" Jul 16 17:44:16.379733 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:44:16.379684 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-drs2p_e55f447c-ad64-482b-a86b-0588bddf7b4d/console-operator/1.log" Jul 16 17:44:16.385099 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:44:16.385081 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-drs2p_e55f447c-ad64-482b-a86b-0588bddf7b4d/console-operator/1.log" Jul 16 17:44:45.800973 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:44:45.800934 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve-ci-e2e-test/fc-concurrency-test-kserve-router-scheduler-78c47c8645-m62qx"] Jul 16 17:44:45.804858 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:44:45.804835 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/fc-concurrency-test-kserve-router-scheduler-78c47c8645-m62qx" Jul 16 17:44:45.807407 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:44:45.807385 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"fc-concurrency-test-kserve-self-signed-certs\"" Jul 16 17:44:45.807515 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:44:45.807460 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"fc-concurrency-test-epp-sa-dockercfg-n5vt2\"" Jul 16 17:44:45.825663 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:44:45.825635 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/fc-concurrency-test-kserve-router-scheduler-78c47c8645-m62qx"] Jul 16 17:44:45.862158 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:44:45.862129 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d0224cda-56fd-4b88-b980-34220b0dd2dd-tls-certs\") pod \"fc-concurrency-test-kserve-router-scheduler-78c47c8645-m62qx\" (UID: \"d0224cda-56fd-4b88-b980-34220b0dd2dd\") " pod="kserve-ci-e2e-test/fc-concurrency-test-kserve-router-scheduler-78c47c8645-m62qx" Jul 16 17:44:45.862318 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:44:45.862181 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tokenizer-uds\" (UniqueName: \"kubernetes.io/empty-dir/d0224cda-56fd-4b88-b980-34220b0dd2dd-tokenizer-uds\") pod \"fc-concurrency-test-kserve-router-scheduler-78c47c8645-m62qx\" (UID: \"d0224cda-56fd-4b88-b980-34220b0dd2dd\") " pod="kserve-ci-e2e-test/fc-concurrency-test-kserve-router-scheduler-78c47c8645-m62qx" Jul 16 17:44:45.862318 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:44:45.862232 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2wpsm\" (UniqueName: \"kubernetes.io/projected/d0224cda-56fd-4b88-b980-34220b0dd2dd-kube-api-access-2wpsm\") pod \"fc-concurrency-test-kserve-router-scheduler-78c47c8645-m62qx\" (UID: \"d0224cda-56fd-4b88-b980-34220b0dd2dd\") " pod="kserve-ci-e2e-test/fc-concurrency-test-kserve-router-scheduler-78c47c8645-m62qx" Jul 16 17:44:45.963559 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:44:45.963529 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d0224cda-56fd-4b88-b980-34220b0dd2dd-tls-certs\") pod \"fc-concurrency-test-kserve-router-scheduler-78c47c8645-m62qx\" (UID: \"d0224cda-56fd-4b88-b980-34220b0dd2dd\") " pod="kserve-ci-e2e-test/fc-concurrency-test-kserve-router-scheduler-78c47c8645-m62qx" Jul 16 17:44:45.963727 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:44:45.963576 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tokenizer-uds\" (UniqueName: \"kubernetes.io/empty-dir/d0224cda-56fd-4b88-b980-34220b0dd2dd-tokenizer-uds\") pod \"fc-concurrency-test-kserve-router-scheduler-78c47c8645-m62qx\" (UID: \"d0224cda-56fd-4b88-b980-34220b0dd2dd\") " pod="kserve-ci-e2e-test/fc-concurrency-test-kserve-router-scheduler-78c47c8645-m62qx" Jul 16 17:44:45.963727 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:44:45.963593 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2wpsm\" (UniqueName: \"kubernetes.io/projected/d0224cda-56fd-4b88-b980-34220b0dd2dd-kube-api-access-2wpsm\") pod \"fc-concurrency-test-kserve-router-scheduler-78c47c8645-m62qx\" (UID: \"d0224cda-56fd-4b88-b980-34220b0dd2dd\") " pod="kserve-ci-e2e-test/fc-concurrency-test-kserve-router-scheduler-78c47c8645-m62qx" Jul 16 17:44:45.963964 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:44:45.963944 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tokenizer-uds\" (UniqueName: \"kubernetes.io/empty-dir/d0224cda-56fd-4b88-b980-34220b0dd2dd-tokenizer-uds\") pod \"fc-concurrency-test-kserve-router-scheduler-78c47c8645-m62qx\" (UID: \"d0224cda-56fd-4b88-b980-34220b0dd2dd\") " pod="kserve-ci-e2e-test/fc-concurrency-test-kserve-router-scheduler-78c47c8645-m62qx" Jul 16 17:44:45.966037 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:44:45.966015 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d0224cda-56fd-4b88-b980-34220b0dd2dd-tls-certs\") pod \"fc-concurrency-test-kserve-router-scheduler-78c47c8645-m62qx\" (UID: \"d0224cda-56fd-4b88-b980-34220b0dd2dd\") " pod="kserve-ci-e2e-test/fc-concurrency-test-kserve-router-scheduler-78c47c8645-m62qx" Jul 16 17:44:45.973081 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:44:45.973056 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2wpsm\" (UniqueName: \"kubernetes.io/projected/d0224cda-56fd-4b88-b980-34220b0dd2dd-kube-api-access-2wpsm\") pod \"fc-concurrency-test-kserve-router-scheduler-78c47c8645-m62qx\" (UID: \"d0224cda-56fd-4b88-b980-34220b0dd2dd\") " pod="kserve-ci-e2e-test/fc-concurrency-test-kserve-router-scheduler-78c47c8645-m62qx" Jul 16 17:44:46.119214 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:44:46.119148 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/fc-concurrency-test-kserve-router-scheduler-78c47c8645-m62qx" Jul 16 17:44:46.260856 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:44:46.260834 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/fc-concurrency-test-kserve-router-scheduler-78c47c8645-m62qx"] Jul 16 17:44:46.262654 ip-10-0-132-130 kubenswrapper[2578]: W0716 17:44:46.262618 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd0224cda_56fd_4b88_b980_34220b0dd2dd.slice/crio-4bfe8d7c6f64db1d6c085e6313ec969dbd4c355ea26df0469ebcf71315230608 WatchSource:0}: Error finding container 4bfe8d7c6f64db1d6c085e6313ec969dbd4c355ea26df0469ebcf71315230608: Status 404 returned error can't find the container with id 4bfe8d7c6f64db1d6c085e6313ec969dbd4c355ea26df0469ebcf71315230608 Jul 16 17:44:46.967552 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:44:46.967509 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/fc-concurrency-test-kserve-router-scheduler-78c47c8645-m62qx" event={"ID":"d0224cda-56fd-4b88-b980-34220b0dd2dd","Type":"ContainerStarted","Data":"182f33f96bc5d5c6c7758c4217f1611995957c8c4c8f8c7e178d3eeaacfaf65d"} Jul 16 17:44:46.968035 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:44:46.967579 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/fc-concurrency-test-kserve-router-scheduler-78c47c8645-m62qx" event={"ID":"d0224cda-56fd-4b88-b980-34220b0dd2dd","Type":"ContainerStarted","Data":"4bfe8d7c6f64db1d6c085e6313ec969dbd4c355ea26df0469ebcf71315230608"} Jul 16 17:44:46.968035 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:44:46.967609 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/fc-concurrency-test-kserve-router-scheduler-78c47c8645-m62qx" Jul 16 17:44:46.985864 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:44:46.985818 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve-ci-e2e-test/fc-concurrency-test-kserve-router-scheduler-78c47c8645-m62qx" podStartSLOduration=1.98580485 podStartE2EDuration="1.98580485s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 17:44:45 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 17:44:46.983636314 +0000 UTC m=+1831.469480205" watchObservedRunningTime="2026-07-16 17:44:46.98580485 +0000 UTC m=+1831.471648736" Jul 16 17:45:17.975505 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:45:17.975446 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/fc-concurrency-test-kserve-router-scheduler-78c47c8645-m62qx" Jul 16 17:49:16.417034 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:49:16.417008 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-drs2p_e55f447c-ad64-482b-a86b-0588bddf7b4d/console-operator/1.log" Jul 16 17:49:16.424641 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:49:16.424610 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-drs2p_e55f447c-ad64-482b-a86b-0588bddf7b4d/console-operator/1.log" Jul 16 17:50:47.706594 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:50:47.706560 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_llmisvc-router-managed-test-llm-5b1e8f15-kserve-df5b5c977-xdvj2_ae64d07c-dbee-4be2-936d-7efc5dfdbd89/main/0.log" Jul 16 17:51:13.712892 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:51:13.712855 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve-ci-e2e-test/llmisvcca2d2d7d499abb359505529ebe02c136-kserve-router-schep28gf"] Jul 16 17:51:13.716542 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:51:13.716524 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/llmisvcca2d2d7d499abb359505529ebe02c136-kserve-router-schep28gf" Jul 16 17:51:13.719241 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:51:13.719221 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"llmisvc-router-managed-test-llm-4b931143-epp-sa-dockercfg-2f9wr\"" Jul 16 17:51:13.719367 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:51:13.719228 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"llmisvca2d2d7d499abb359505529ebe02c136-kserve-self-signed-certs\"" Jul 16 17:51:13.732333 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:51:13.732300 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/llmisvcca2d2d7d499abb359505529ebe02c136-kserve-router-schep28gf"] Jul 16 17:51:13.743365 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:51:13.743337 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tokenizer-uds\" (UniqueName: \"kubernetes.io/empty-dir/1500b232-42ed-4480-a44b-41139e735a71-tokenizer-uds\") pod \"llmisvcca2d2d7d499abb359505529ebe02c136-kserve-router-schep28gf\" (UID: \"1500b232-42ed-4480-a44b-41139e735a71\") " pod="kserve-ci-e2e-test/llmisvcca2d2d7d499abb359505529ebe02c136-kserve-router-schep28gf" Jul 16 17:51:13.743464 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:51:13.743390 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/1500b232-42ed-4480-a44b-41139e735a71-tls-certs\") pod \"llmisvcca2d2d7d499abb359505529ebe02c136-kserve-router-schep28gf\" (UID: \"1500b232-42ed-4480-a44b-41139e735a71\") " pod="kserve-ci-e2e-test/llmisvcca2d2d7d499abb359505529ebe02c136-kserve-router-schep28gf" Jul 16 17:51:13.743464 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:51:13.743425 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lkhgp\" (UniqueName: \"kubernetes.io/projected/1500b232-42ed-4480-a44b-41139e735a71-kube-api-access-lkhgp\") pod \"llmisvcca2d2d7d499abb359505529ebe02c136-kserve-router-schep28gf\" (UID: \"1500b232-42ed-4480-a44b-41139e735a71\") " pod="kserve-ci-e2e-test/llmisvcca2d2d7d499abb359505529ebe02c136-kserve-router-schep28gf" Jul 16 17:51:13.844441 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:51:13.844416 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tokenizer-uds\" (UniqueName: \"kubernetes.io/empty-dir/1500b232-42ed-4480-a44b-41139e735a71-tokenizer-uds\") pod \"llmisvcca2d2d7d499abb359505529ebe02c136-kserve-router-schep28gf\" (UID: \"1500b232-42ed-4480-a44b-41139e735a71\") " pod="kserve-ci-e2e-test/llmisvcca2d2d7d499abb359505529ebe02c136-kserve-router-schep28gf" Jul 16 17:51:13.844558 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:51:13.844462 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/1500b232-42ed-4480-a44b-41139e735a71-tls-certs\") pod \"llmisvcca2d2d7d499abb359505529ebe02c136-kserve-router-schep28gf\" (UID: \"1500b232-42ed-4480-a44b-41139e735a71\") " pod="kserve-ci-e2e-test/llmisvcca2d2d7d499abb359505529ebe02c136-kserve-router-schep28gf" Jul 16 17:51:13.844558 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:51:13.844492 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lkhgp\" (UniqueName: \"kubernetes.io/projected/1500b232-42ed-4480-a44b-41139e735a71-kube-api-access-lkhgp\") pod \"llmisvcca2d2d7d499abb359505529ebe02c136-kserve-router-schep28gf\" (UID: \"1500b232-42ed-4480-a44b-41139e735a71\") " pod="kserve-ci-e2e-test/llmisvcca2d2d7d499abb359505529ebe02c136-kserve-router-schep28gf" Jul 16 17:51:13.844838 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:51:13.844819 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tokenizer-uds\" (UniqueName: \"kubernetes.io/empty-dir/1500b232-42ed-4480-a44b-41139e735a71-tokenizer-uds\") pod \"llmisvcca2d2d7d499abb359505529ebe02c136-kserve-router-schep28gf\" (UID: \"1500b232-42ed-4480-a44b-41139e735a71\") " pod="kserve-ci-e2e-test/llmisvcca2d2d7d499abb359505529ebe02c136-kserve-router-schep28gf" Jul 16 17:51:13.846915 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:51:13.846897 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/1500b232-42ed-4480-a44b-41139e735a71-tls-certs\") pod \"llmisvcca2d2d7d499abb359505529ebe02c136-kserve-router-schep28gf\" (UID: \"1500b232-42ed-4480-a44b-41139e735a71\") " pod="kserve-ci-e2e-test/llmisvcca2d2d7d499abb359505529ebe02c136-kserve-router-schep28gf" Jul 16 17:51:13.856120 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:51:13.856099 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lkhgp\" (UniqueName: \"kubernetes.io/projected/1500b232-42ed-4480-a44b-41139e735a71-kube-api-access-lkhgp\") pod \"llmisvcca2d2d7d499abb359505529ebe02c136-kserve-router-schep28gf\" (UID: \"1500b232-42ed-4480-a44b-41139e735a71\") " pod="kserve-ci-e2e-test/llmisvcca2d2d7d499abb359505529ebe02c136-kserve-router-schep28gf" Jul 16 17:51:14.027924 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:51:14.027853 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/llmisvcca2d2d7d499abb359505529ebe02c136-kserve-router-schep28gf" Jul 16 17:51:14.168753 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:51:14.168724 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/llmisvcca2d2d7d499abb359505529ebe02c136-kserve-router-schep28gf"] Jul 16 17:51:14.170904 ip-10-0-132-130 kubenswrapper[2578]: W0716 17:51:14.170875 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1500b232_42ed_4480_a44b_41139e735a71.slice/crio-c14933cbff901b6a27386a56c52ebb616333afd08c4f678130a885bdf0ced4a1 WatchSource:0}: Error finding container c14933cbff901b6a27386a56c52ebb616333afd08c4f678130a885bdf0ced4a1: Status 404 returned error can't find the container with id c14933cbff901b6a27386a56c52ebb616333afd08c4f678130a885bdf0ced4a1 Jul 16 17:51:14.186456 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:51:14.186426 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/llmisvcca2d2d7d499abb359505529ebe02c136-kserve-router-schep28gf" event={"ID":"1500b232-42ed-4480-a44b-41139e735a71","Type":"ContainerStarted","Data":"c14933cbff901b6a27386a56c52ebb616333afd08c4f678130a885bdf0ced4a1"} Jul 16 17:51:15.194217 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:51:15.194168 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/llmisvcca2d2d7d499abb359505529ebe02c136-kserve-router-schep28gf" event={"ID":"1500b232-42ed-4480-a44b-41139e735a71","Type":"ContainerStarted","Data":"e4ba736399112105c63519fcd5f00c37ce8b319b6f8255cc4f33f62a2a92ab86"} Jul 16 17:51:15.194675 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:51:15.194373 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/llmisvcca2d2d7d499abb359505529ebe02c136-kserve-router-schep28gf" Jul 16 17:51:15.216815 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:51:15.216764 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve-ci-e2e-test/llmisvcca2d2d7d499abb359505529ebe02c136-kserve-router-schep28gf" podStartSLOduration=2.216750526 podStartE2EDuration="2.216750526s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 17:51:13 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 17:51:15.21442094 +0000 UTC m=+2219.700264831" watchObservedRunningTime="2026-07-16 17:51:15.216750526 +0000 UTC m=+2219.702594415" Jul 16 17:51:46.201956 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:51:46.201927 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/llmisvcca2d2d7d499abb359505529ebe02c136-kserve-router-schep28gf" Jul 16 17:54:16.456674 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:54:16.456563 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-drs2p_e55f447c-ad64-482b-a86b-0588bddf7b4d/console-operator/1.log" Jul 16 17:54:16.464977 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:54:16.464957 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-drs2p_e55f447c-ad64-482b-a86b-0588bddf7b4d/console-operator/1.log" Jul 16 17:59:16.494025 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:59:16.493916 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-drs2p_e55f447c-ad64-482b-a86b-0588bddf7b4d/console-operator/1.log" Jul 16 17:59:16.520028 ip-10-0-132-130 kubenswrapper[2578]: I0716 17:59:16.520002 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-drs2p_e55f447c-ad64-482b-a86b-0588bddf7b4d/console-operator/1.log" Jul 16 18:04:16.534558 ip-10-0-132-130 kubenswrapper[2578]: I0716 18:04:16.534448 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-drs2p_e55f447c-ad64-482b-a86b-0588bddf7b4d/console-operator/1.log" Jul 16 18:04:16.560068 ip-10-0-132-130 kubenswrapper[2578]: I0716 18:04:16.560044 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-drs2p_e55f447c-ad64-482b-a86b-0588bddf7b4d/console-operator/1.log" Jul 16 18:06:52.089489 ip-10-0-132-130 kubenswrapper[2578]: I0716 18:06:52.089396 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_llmisvcca2d2d7d499abb359505529ebe02c136-kserve-router-schep28gf_1500b232-42ed-4480-a44b-41139e735a71/main/0.log" Jul 16 18:07:16.074494 ip-10-0-132-130 kubenswrapper[2578]: I0716 18:07:16.074455 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-with-7ca60146-kserve-74d68bf984-2dn6z"] Jul 16 18:07:16.078522 ip-10-0-132-130 kubenswrapper[2578]: I0716 18:07:16.078502 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-with-7ca60146-kserve-74d68bf984-2dn6z" Jul 16 18:07:16.080755 ip-10-0-132-130 kubenswrapper[2578]: I0716 18:07:16.080725 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"odh-kserve-custom-ca-bundle\"" Jul 16 18:07:16.080870 ip-10-0-132-130 kubenswrapper[2578]: I0716 18:07:16.080754 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"llmisv3e414c2ba058a022dfd694dbcbac5b51-kserve-self-signed-certs\"" Jul 16 18:07:16.080870 ip-10-0-132-130 kubenswrapper[2578]: I0716 18:07:16.080764 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"seaweedfs-s3-creds\"" Jul 16 18:07:16.103098 ip-10-0-132-130 kubenswrapper[2578]: I0716 18:07:16.103058 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-with-7ca60146-kserve-74d68bf984-2dn6z"] Jul 16 18:07:16.207259 ip-10-0-132-130 kubenswrapper[2578]: I0716 18:07:16.207212 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/258f94f2-e4b2-49e3-b38c-48d83c67a352-kserve-provision-location\") pod \"llmisvc-model-fb-opt-125m-with-7ca60146-kserve-74d68bf984-2dn6z\" (UID: \"258f94f2-e4b2-49e3-b38c-48d83c67a352\") " pod="kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-with-7ca60146-kserve-74d68bf984-2dn6z" Jul 16 18:07:16.207456 ip-10-0-132-130 kubenswrapper[2578]: I0716 18:07:16.207294 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/258f94f2-e4b2-49e3-b38c-48d83c67a352-tmp-dir\") pod \"llmisvc-model-fb-opt-125m-with-7ca60146-kserve-74d68bf984-2dn6z\" (UID: \"258f94f2-e4b2-49e3-b38c-48d83c67a352\") " pod="kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-with-7ca60146-kserve-74d68bf984-2dn6z" Jul 16 18:07:16.207456 ip-10-0-132-130 kubenswrapper[2578]: I0716 18:07:16.207345 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/258f94f2-e4b2-49e3-b38c-48d83c67a352-cabundle-cert\") pod \"llmisvc-model-fb-opt-125m-with-7ca60146-kserve-74d68bf984-2dn6z\" (UID: \"258f94f2-e4b2-49e3-b38c-48d83c67a352\") " pod="kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-with-7ca60146-kserve-74d68bf984-2dn6z" Jul 16 18:07:16.207456 ip-10-0-132-130 kubenswrapper[2578]: I0716 18:07:16.207364 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/258f94f2-e4b2-49e3-b38c-48d83c67a352-dshm\") pod \"llmisvc-model-fb-opt-125m-with-7ca60146-kserve-74d68bf984-2dn6z\" (UID: \"258f94f2-e4b2-49e3-b38c-48d83c67a352\") " pod="kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-with-7ca60146-kserve-74d68bf984-2dn6z" Jul 16 18:07:16.207456 ip-10-0-132-130 kubenswrapper[2578]: I0716 18:07:16.207403 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/258f94f2-e4b2-49e3-b38c-48d83c67a352-tls-certs\") pod \"llmisvc-model-fb-opt-125m-with-7ca60146-kserve-74d68bf984-2dn6z\" (UID: \"258f94f2-e4b2-49e3-b38c-48d83c67a352\") " pod="kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-with-7ca60146-kserve-74d68bf984-2dn6z" Jul 16 18:07:16.207456 ip-10-0-132-130 kubenswrapper[2578]: I0716 18:07:16.207420 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/258f94f2-e4b2-49e3-b38c-48d83c67a352-home\") pod \"llmisvc-model-fb-opt-125m-with-7ca60146-kserve-74d68bf984-2dn6z\" (UID: \"258f94f2-e4b2-49e3-b38c-48d83c67a352\") " pod="kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-with-7ca60146-kserve-74d68bf984-2dn6z" Jul 16 18:07:16.207773 ip-10-0-132-130 kubenswrapper[2578]: I0716 18:07:16.207503 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/258f94f2-e4b2-49e3-b38c-48d83c67a352-model-cache\") pod \"llmisvc-model-fb-opt-125m-with-7ca60146-kserve-74d68bf984-2dn6z\" (UID: \"258f94f2-e4b2-49e3-b38c-48d83c67a352\") " pod="kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-with-7ca60146-kserve-74d68bf984-2dn6z" Jul 16 18:07:16.207773 ip-10-0-132-130 kubenswrapper[2578]: I0716 18:07:16.207531 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-69w8j\" (UniqueName: \"kubernetes.io/projected/258f94f2-e4b2-49e3-b38c-48d83c67a352-kube-api-access-69w8j\") pod \"llmisvc-model-fb-opt-125m-with-7ca60146-kserve-74d68bf984-2dn6z\" (UID: \"258f94f2-e4b2-49e3-b38c-48d83c67a352\") " pod="kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-with-7ca60146-kserve-74d68bf984-2dn6z" Jul 16 18:07:16.309126 ip-10-0-132-130 kubenswrapper[2578]: I0716 18:07:16.309075 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/258f94f2-e4b2-49e3-b38c-48d83c67a352-home\") pod \"llmisvc-model-fb-opt-125m-with-7ca60146-kserve-74d68bf984-2dn6z\" (UID: \"258f94f2-e4b2-49e3-b38c-48d83c67a352\") " pod="kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-with-7ca60146-kserve-74d68bf984-2dn6z" Jul 16 18:07:16.309295 ip-10-0-132-130 kubenswrapper[2578]: I0716 18:07:16.309216 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/258f94f2-e4b2-49e3-b38c-48d83c67a352-model-cache\") pod \"llmisvc-model-fb-opt-125m-with-7ca60146-kserve-74d68bf984-2dn6z\" (UID: \"258f94f2-e4b2-49e3-b38c-48d83c67a352\") " pod="kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-with-7ca60146-kserve-74d68bf984-2dn6z" Jul 16 18:07:16.309295 ip-10-0-132-130 kubenswrapper[2578]: I0716 18:07:16.309258 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-69w8j\" (UniqueName: \"kubernetes.io/projected/258f94f2-e4b2-49e3-b38c-48d83c67a352-kube-api-access-69w8j\") pod \"llmisvc-model-fb-opt-125m-with-7ca60146-kserve-74d68bf984-2dn6z\" (UID: \"258f94f2-e4b2-49e3-b38c-48d83c67a352\") " pod="kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-with-7ca60146-kserve-74d68bf984-2dn6z" Jul 16 18:07:16.309423 ip-10-0-132-130 kubenswrapper[2578]: I0716 18:07:16.309292 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/258f94f2-e4b2-49e3-b38c-48d83c67a352-kserve-provision-location\") pod \"llmisvc-model-fb-opt-125m-with-7ca60146-kserve-74d68bf984-2dn6z\" (UID: \"258f94f2-e4b2-49e3-b38c-48d83c67a352\") " pod="kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-with-7ca60146-kserve-74d68bf984-2dn6z" Jul 16 18:07:16.309423 ip-10-0-132-130 kubenswrapper[2578]: I0716 18:07:16.309365 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/258f94f2-e4b2-49e3-b38c-48d83c67a352-tmp-dir\") pod \"llmisvc-model-fb-opt-125m-with-7ca60146-kserve-74d68bf984-2dn6z\" (UID: \"258f94f2-e4b2-49e3-b38c-48d83c67a352\") " pod="kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-with-7ca60146-kserve-74d68bf984-2dn6z" Jul 16 18:07:16.309423 ip-10-0-132-130 kubenswrapper[2578]: I0716 18:07:16.309412 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/258f94f2-e4b2-49e3-b38c-48d83c67a352-cabundle-cert\") pod \"llmisvc-model-fb-opt-125m-with-7ca60146-kserve-74d68bf984-2dn6z\" (UID: \"258f94f2-e4b2-49e3-b38c-48d83c67a352\") " pod="kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-with-7ca60146-kserve-74d68bf984-2dn6z" Jul 16 18:07:16.309577 ip-10-0-132-130 kubenswrapper[2578]: I0716 18:07:16.309437 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/258f94f2-e4b2-49e3-b38c-48d83c67a352-dshm\") pod \"llmisvc-model-fb-opt-125m-with-7ca60146-kserve-74d68bf984-2dn6z\" (UID: \"258f94f2-e4b2-49e3-b38c-48d83c67a352\") " pod="kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-with-7ca60146-kserve-74d68bf984-2dn6z" Jul 16 18:07:16.309577 ip-10-0-132-130 kubenswrapper[2578]: I0716 18:07:16.309493 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/258f94f2-e4b2-49e3-b38c-48d83c67a352-tls-certs\") pod \"llmisvc-model-fb-opt-125m-with-7ca60146-kserve-74d68bf984-2dn6z\" (UID: \"258f94f2-e4b2-49e3-b38c-48d83c67a352\") " pod="kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-with-7ca60146-kserve-74d68bf984-2dn6z" Jul 16 18:07:16.309681 ip-10-0-132-130 kubenswrapper[2578]: I0716 18:07:16.309599 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/258f94f2-e4b2-49e3-b38c-48d83c67a352-model-cache\") pod \"llmisvc-model-fb-opt-125m-with-7ca60146-kserve-74d68bf984-2dn6z\" (UID: \"258f94f2-e4b2-49e3-b38c-48d83c67a352\") " pod="kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-with-7ca60146-kserve-74d68bf984-2dn6z" Jul 16 18:07:16.309681 ip-10-0-132-130 kubenswrapper[2578]: I0716 18:07:16.309604 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/258f94f2-e4b2-49e3-b38c-48d83c67a352-home\") pod \"llmisvc-model-fb-opt-125m-with-7ca60146-kserve-74d68bf984-2dn6z\" (UID: \"258f94f2-e4b2-49e3-b38c-48d83c67a352\") " pod="kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-with-7ca60146-kserve-74d68bf984-2dn6z" Jul 16 18:07:16.309987 ip-10-0-132-130 kubenswrapper[2578]: I0716 18:07:16.309962 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/258f94f2-e4b2-49e3-b38c-48d83c67a352-tmp-dir\") pod \"llmisvc-model-fb-opt-125m-with-7ca60146-kserve-74d68bf984-2dn6z\" (UID: \"258f94f2-e4b2-49e3-b38c-48d83c67a352\") " pod="kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-with-7ca60146-kserve-74d68bf984-2dn6z" Jul 16 18:07:16.310185 ip-10-0-132-130 kubenswrapper[2578]: I0716 18:07:16.310153 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/258f94f2-e4b2-49e3-b38c-48d83c67a352-kserve-provision-location\") pod \"llmisvc-model-fb-opt-125m-with-7ca60146-kserve-74d68bf984-2dn6z\" (UID: \"258f94f2-e4b2-49e3-b38c-48d83c67a352\") " pod="kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-with-7ca60146-kserve-74d68bf984-2dn6z" Jul 16 18:07:16.310521 ip-10-0-132-130 kubenswrapper[2578]: I0716 18:07:16.310499 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/258f94f2-e4b2-49e3-b38c-48d83c67a352-cabundle-cert\") pod \"llmisvc-model-fb-opt-125m-with-7ca60146-kserve-74d68bf984-2dn6z\" (UID: \"258f94f2-e4b2-49e3-b38c-48d83c67a352\") " pod="kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-with-7ca60146-kserve-74d68bf984-2dn6z" Jul 16 18:07:16.312171 ip-10-0-132-130 kubenswrapper[2578]: I0716 18:07:16.312151 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/258f94f2-e4b2-49e3-b38c-48d83c67a352-dshm\") pod \"llmisvc-model-fb-opt-125m-with-7ca60146-kserve-74d68bf984-2dn6z\" (UID: \"258f94f2-e4b2-49e3-b38c-48d83c67a352\") " pod="kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-with-7ca60146-kserve-74d68bf984-2dn6z" Jul 16 18:07:16.312302 ip-10-0-132-130 kubenswrapper[2578]: I0716 18:07:16.312285 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/258f94f2-e4b2-49e3-b38c-48d83c67a352-tls-certs\") pod \"llmisvc-model-fb-opt-125m-with-7ca60146-kserve-74d68bf984-2dn6z\" (UID: \"258f94f2-e4b2-49e3-b38c-48d83c67a352\") " pod="kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-with-7ca60146-kserve-74d68bf984-2dn6z" Jul 16 18:07:16.326934 ip-10-0-132-130 kubenswrapper[2578]: I0716 18:07:16.326869 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-69w8j\" (UniqueName: \"kubernetes.io/projected/258f94f2-e4b2-49e3-b38c-48d83c67a352-kube-api-access-69w8j\") pod \"llmisvc-model-fb-opt-125m-with-7ca60146-kserve-74d68bf984-2dn6z\" (UID: \"258f94f2-e4b2-49e3-b38c-48d83c67a352\") " pod="kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-with-7ca60146-kserve-74d68bf984-2dn6z" Jul 16 18:07:16.389223 ip-10-0-132-130 kubenswrapper[2578]: I0716 18:07:16.389192 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-with-7ca60146-kserve-74d68bf984-2dn6z" Jul 16 18:07:16.641763 ip-10-0-132-130 kubenswrapper[2578]: I0716 18:07:16.641735 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-with-7ca60146-kserve-74d68bf984-2dn6z"] Jul 16 18:07:16.643373 ip-10-0-132-130 kubenswrapper[2578]: W0716 18:07:16.643348 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod258f94f2_e4b2_49e3_b38c_48d83c67a352.slice/crio-c93a4a08fa232c04944da1fd55e30640aed032ccdfa6251fe0c32ba04d0761bf WatchSource:0}: Error finding container c93a4a08fa232c04944da1fd55e30640aed032ccdfa6251fe0c32ba04d0761bf: Status 404 returned error can't find the container with id c93a4a08fa232c04944da1fd55e30640aed032ccdfa6251fe0c32ba04d0761bf Jul 16 18:07:16.826911 ip-10-0-132-130 kubenswrapper[2578]: I0716 18:07:16.826869 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-with-7ca60146-kserve-74d68bf984-2dn6z" event={"ID":"258f94f2-e4b2-49e3-b38c-48d83c67a352","Type":"ContainerStarted","Data":"5bf4ec205623c6b3a278633137a7ebcbe6f326140784f54413e5e678381f68d1"} Jul 16 18:07:16.827089 ip-10-0-132-130 kubenswrapper[2578]: I0716 18:07:16.826939 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-with-7ca60146-kserve-74d68bf984-2dn6z" event={"ID":"258f94f2-e4b2-49e3-b38c-48d83c67a352","Type":"ContainerStarted","Data":"c93a4a08fa232c04944da1fd55e30640aed032ccdfa6251fe0c32ba04d0761bf"} Jul 16 18:07:20.854572 ip-10-0-132-130 kubenswrapper[2578]: I0716 18:07:20.854530 2578 generic.go:356] "Generic (PLEG): container finished" podID="258f94f2-e4b2-49e3-b38c-48d83c67a352" containerID="5bf4ec205623c6b3a278633137a7ebcbe6f326140784f54413e5e678381f68d1" exitCode=0 Jul 16 18:07:20.855026 ip-10-0-132-130 kubenswrapper[2578]: I0716 18:07:20.854601 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-with-7ca60146-kserve-74d68bf984-2dn6z" event={"ID":"258f94f2-e4b2-49e3-b38c-48d83c67a352","Type":"ContainerDied","Data":"5bf4ec205623c6b3a278633137a7ebcbe6f326140784f54413e5e678381f68d1"} Jul 16 18:07:20.856184 ip-10-0-132-130 kubenswrapper[2578]: I0716 18:07:20.856166 2578 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 16 18:07:49.090014 ip-10-0-132-130 kubenswrapper[2578]: I0716 18:07:49.089934 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-with-7ca60146-kserve-74d68bf984-2dn6z" event={"ID":"258f94f2-e4b2-49e3-b38c-48d83c67a352","Type":"ContainerStarted","Data":"05a70d3185be453f75c8b8e6e77316d348e5977a79799a994d642dc573cfe160"} Jul 16 18:07:49.118448 ip-10-0-132-130 kubenswrapper[2578]: I0716 18:07:49.118400 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-with-7ca60146-kserve-74d68bf984-2dn6z" podStartSLOduration=1.359722846 podStartE2EDuration="33.118364162s" totalImagesPullingTime="27.978663085s" totalInitContainerRuntime="3.779978231s" isStatefulPod=false podCreationTimestamp="2026-07-16 18:07:16 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 18:07:49.115351066 +0000 UTC m=+3213.601194955" watchObservedRunningTime="2026-07-16 18:07:49.118364162 +0000 UTC m=+3213.604208050" Jul 16 18:07:49.390034 ip-10-0-132-130 kubenswrapper[2578]: I0716 18:07:49.389949 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-with-7ca60146-kserve-74d68bf984-2dn6z" Jul 16 18:07:56.390151 ip-10-0-132-130 kubenswrapper[2578]: I0716 18:07:56.390112 2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-with-7ca60146-kserve-74d68bf984-2dn6z" Jul 16 18:07:56.391997 ip-10-0-132-130 kubenswrapper[2578]: I0716 18:07:56.391967 2578 prober.go:120] "Probe failed" probeType="Startup" pod="kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-with-7ca60146-kserve-74d68bf984-2dn6z" podUID="258f94f2-e4b2-49e3-b38c-48d83c67a352" containerName="main" probeResult="failure" output="Get \"https://10.133.0.59:8000/health\": dial tcp 10.133.0.59:8000: connect: connection refused" Jul 16 18:08:06.390244 ip-10-0-132-130 kubenswrapper[2578]: I0716 18:08:06.390198 2578 prober.go:120] "Probe failed" probeType="Startup" pod="kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-with-7ca60146-kserve-74d68bf984-2dn6z" podUID="258f94f2-e4b2-49e3-b38c-48d83c67a352" containerName="main" probeResult="failure" output="Get \"https://10.133.0.59:8000/health\": dial tcp 10.133.0.59:8000: connect: connection refused" Jul 16 18:08:16.390245 ip-10-0-132-130 kubenswrapper[2578]: I0716 18:08:16.390191 2578 prober.go:120] "Probe failed" probeType="Startup" pod="kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-with-7ca60146-kserve-74d68bf984-2dn6z" podUID="258f94f2-e4b2-49e3-b38c-48d83c67a352" containerName="main" probeResult="failure" output="Get \"https://10.133.0.59:8000/health\": dial tcp 10.133.0.59:8000: connect: connection refused" Jul 16 18:08:26.390383 ip-10-0-132-130 kubenswrapper[2578]: I0716 18:08:26.390339 2578 prober.go:120] "Probe failed" probeType="Startup" pod="kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-with-7ca60146-kserve-74d68bf984-2dn6z" podUID="258f94f2-e4b2-49e3-b38c-48d83c67a352" containerName="main" probeResult="failure" output="Get \"https://10.133.0.59:8000/health\": dial tcp 10.133.0.59:8000: connect: connection refused" Jul 16 18:08:36.389791 ip-10-0-132-130 kubenswrapper[2578]: I0716 18:08:36.389747 2578 prober.go:120] "Probe failed" probeType="Startup" pod="kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-with-7ca60146-kserve-74d68bf984-2dn6z" podUID="258f94f2-e4b2-49e3-b38c-48d83c67a352" containerName="main" probeResult="failure" output="Get \"https://10.133.0.59:8000/health\": dial tcp 10.133.0.59:8000: connect: connection refused" Jul 16 18:08:46.389959 ip-10-0-132-130 kubenswrapper[2578]: I0716 18:08:46.389913 2578 prober.go:120] "Probe failed" probeType="Startup" pod="kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-with-7ca60146-kserve-74d68bf984-2dn6z" podUID="258f94f2-e4b2-49e3-b38c-48d83c67a352" containerName="main" probeResult="failure" output="Get \"https://10.133.0.59:8000/health\": dial tcp 10.133.0.59:8000: connect: connection refused" Jul 16 18:08:56.389826 ip-10-0-132-130 kubenswrapper[2578]: I0716 18:08:56.389782 2578 prober.go:120] "Probe failed" probeType="Startup" pod="kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-with-7ca60146-kserve-74d68bf984-2dn6z" podUID="258f94f2-e4b2-49e3-b38c-48d83c67a352" containerName="main" probeResult="failure" output="Get \"https://10.133.0.59:8000/health\": dial tcp 10.133.0.59:8000: connect: connection refused" Jul 16 18:09:06.400602 ip-10-0-132-130 kubenswrapper[2578]: I0716 18:09:06.400568 2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-with-7ca60146-kserve-74d68bf984-2dn6z" Jul 16 18:09:06.409028 ip-10-0-132-130 kubenswrapper[2578]: I0716 18:09:06.409002 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-with-7ca60146-kserve-74d68bf984-2dn6z" Jul 16 18:09:16.591392 ip-10-0-132-130 kubenswrapper[2578]: I0716 18:09:16.591363 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-drs2p_e55f447c-ad64-482b-a86b-0588bddf7b4d/console-operator/1.log" Jul 16 18:09:16.612088 ip-10-0-132-130 kubenswrapper[2578]: I0716 18:09:16.612064 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-drs2p_e55f447c-ad64-482b-a86b-0588bddf7b4d/console-operator/1.log" Jul 16 18:09:32.007918 ip-10-0-132-130 kubenswrapper[2578]: I0716 18:09:32.007885 2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-with-7ca60146-kserve-74d68bf984-2dn6z"] Jul 16 18:09:47.031319 ip-10-0-132-130 kubenswrapper[2578]: I0716 18:09:47.031235 2578 kuberuntime_container.go:873] "Killing container with a grace period" pod="kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-with-7ca60146-kserve-74d68bf984-2dn6z" podUID="258f94f2-e4b2-49e3-b38c-48d83c67a352" containerName="main" containerID="cri-o://05a70d3185be453f75c8b8e6e77316d348e5977a79799a994d642dc573cfe160" gracePeriod=45 Jul 16 18:09:47.401927 ip-10-0-132-130 kubenswrapper[2578]: I0716 18:09:47.401886 2578 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-with-7ca60146-kserve-74d68bf984-2dn6z" podUID="258f94f2-e4b2-49e3-b38c-48d83c67a352" containerName="main" probeResult="failure" output="Get \"https://10.133.0.59:8000/health\": dial tcp 10.133.0.59:8000: connect: connection refused" Jul 16 18:09:47.698400 ip-10-0-132-130 kubenswrapper[2578]: I0716 18:09:47.698373 2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-with-7ca60146-kserve-74d68bf984-2dn6z" Jul 16 18:09:47.815357 ip-10-0-132-130 kubenswrapper[2578]: I0716 18:09:47.815328 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-69w8j\" (UniqueName: \"kubernetes.io/projected/258f94f2-e4b2-49e3-b38c-48d83c67a352-kube-api-access-69w8j\") pod \"258f94f2-e4b2-49e3-b38c-48d83c67a352\" (UID: \"258f94f2-e4b2-49e3-b38c-48d83c67a352\") " Jul 16 18:09:47.815522 ip-10-0-132-130 kubenswrapper[2578]: I0716 18:09:47.815374 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/258f94f2-e4b2-49e3-b38c-48d83c67a352-home\") pod \"258f94f2-e4b2-49e3-b38c-48d83c67a352\" (UID: \"258f94f2-e4b2-49e3-b38c-48d83c67a352\") " Jul 16 18:09:47.815522 ip-10-0-132-130 kubenswrapper[2578]: I0716 18:09:47.815406 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/258f94f2-e4b2-49e3-b38c-48d83c67a352-dshm\") pod \"258f94f2-e4b2-49e3-b38c-48d83c67a352\" (UID: \"258f94f2-e4b2-49e3-b38c-48d83c67a352\") " Jul 16 18:09:47.815654 ip-10-0-132-130 kubenswrapper[2578]: I0716 18:09:47.815539 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/258f94f2-e4b2-49e3-b38c-48d83c67a352-kserve-provision-location\") pod \"258f94f2-e4b2-49e3-b38c-48d83c67a352\" (UID: \"258f94f2-e4b2-49e3-b38c-48d83c67a352\") " Jul 16 18:09:47.815654 ip-10-0-132-130 kubenswrapper[2578]: I0716 18:09:47.815645 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/258f94f2-e4b2-49e3-b38c-48d83c67a352-tls-certs\") pod \"258f94f2-e4b2-49e3-b38c-48d83c67a352\" (UID: \"258f94f2-e4b2-49e3-b38c-48d83c67a352\") " Jul 16 18:09:47.815882 ip-10-0-132-130 kubenswrapper[2578]: I0716 18:09:47.815677 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/258f94f2-e4b2-49e3-b38c-48d83c67a352-cabundle-cert\") pod \"258f94f2-e4b2-49e3-b38c-48d83c67a352\" (UID: \"258f94f2-e4b2-49e3-b38c-48d83c67a352\") " Jul 16 18:09:47.815882 ip-10-0-132-130 kubenswrapper[2578]: I0716 18:09:47.815747 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/258f94f2-e4b2-49e3-b38c-48d83c67a352-tmp-dir\") pod \"258f94f2-e4b2-49e3-b38c-48d83c67a352\" (UID: \"258f94f2-e4b2-49e3-b38c-48d83c67a352\") " Jul 16 18:09:47.815882 ip-10-0-132-130 kubenswrapper[2578]: I0716 18:09:47.815794 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/258f94f2-e4b2-49e3-b38c-48d83c67a352-model-cache\") pod \"258f94f2-e4b2-49e3-b38c-48d83c67a352\" (UID: \"258f94f2-e4b2-49e3-b38c-48d83c67a352\") " Jul 16 18:09:47.815882 ip-10-0-132-130 kubenswrapper[2578]: I0716 18:09:47.815861 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/258f94f2-e4b2-49e3-b38c-48d83c67a352-home" (OuterVolumeSpecName: "home") pod "258f94f2-e4b2-49e3-b38c-48d83c67a352" (UID: "258f94f2-e4b2-49e3-b38c-48d83c67a352"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 16 18:09:47.816199 ip-10-0-132-130 kubenswrapper[2578]: I0716 18:09:47.816119 2578 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/258f94f2-e4b2-49e3-b38c-48d83c67a352-home\") on node \"ip-10-0-132-130.ec2.internal\" DevicePath \"\"" Jul 16 18:09:47.816199 ip-10-0-132-130 kubenswrapper[2578]: I0716 18:09:47.816140 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/258f94f2-e4b2-49e3-b38c-48d83c67a352-cabundle-cert" (OuterVolumeSpecName: "cabundle-cert") pod "258f94f2-e4b2-49e3-b38c-48d83c67a352" (UID: "258f94f2-e4b2-49e3-b38c-48d83c67a352"). InnerVolumeSpecName "cabundle-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 16 18:09:47.816349 ip-10-0-132-130 kubenswrapper[2578]: I0716 18:09:47.816319 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/258f94f2-e4b2-49e3-b38c-48d83c67a352-model-cache" (OuterVolumeSpecName: "model-cache") pod "258f94f2-e4b2-49e3-b38c-48d83c67a352" (UID: "258f94f2-e4b2-49e3-b38c-48d83c67a352"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 16 18:09:47.817788 ip-10-0-132-130 kubenswrapper[2578]: I0716 18:09:47.817761 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/258f94f2-e4b2-49e3-b38c-48d83c67a352-dshm" (OuterVolumeSpecName: "dshm") pod "258f94f2-e4b2-49e3-b38c-48d83c67a352" (UID: "258f94f2-e4b2-49e3-b38c-48d83c67a352"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 16 18:09:47.818068 ip-10-0-132-130 kubenswrapper[2578]: I0716 18:09:47.818043 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/258f94f2-e4b2-49e3-b38c-48d83c67a352-kube-api-access-69w8j" (OuterVolumeSpecName: "kube-api-access-69w8j") pod "258f94f2-e4b2-49e3-b38c-48d83c67a352" (UID: "258f94f2-e4b2-49e3-b38c-48d83c67a352"). InnerVolumeSpecName "kube-api-access-69w8j". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 16 18:09:47.818296 ip-10-0-132-130 kubenswrapper[2578]: I0716 18:09:47.818263 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/258f94f2-e4b2-49e3-b38c-48d83c67a352-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "258f94f2-e4b2-49e3-b38c-48d83c67a352" (UID: "258f94f2-e4b2-49e3-b38c-48d83c67a352"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 16 18:09:47.827008 ip-10-0-132-130 kubenswrapper[2578]: I0716 18:09:47.826973 2578 generic.go:356] "Generic (PLEG): container finished" podID="258f94f2-e4b2-49e3-b38c-48d83c67a352" containerID="05a70d3185be453f75c8b8e6e77316d348e5977a79799a994d642dc573cfe160" exitCode=0 Jul 16 18:09:47.827138 ip-10-0-132-130 kubenswrapper[2578]: I0716 18:09:47.827034 2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-with-7ca60146-kserve-74d68bf984-2dn6z" Jul 16 18:09:47.827138 ip-10-0-132-130 kubenswrapper[2578]: I0716 18:09:47.827097 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-with-7ca60146-kserve-74d68bf984-2dn6z" event={"ID":"258f94f2-e4b2-49e3-b38c-48d83c67a352","Type":"ContainerDied","Data":"05a70d3185be453f75c8b8e6e77316d348e5977a79799a994d642dc573cfe160"} Jul 16 18:09:47.827265 ip-10-0-132-130 kubenswrapper[2578]: I0716 18:09:47.827182 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-with-7ca60146-kserve-74d68bf984-2dn6z" event={"ID":"258f94f2-e4b2-49e3-b38c-48d83c67a352","Type":"ContainerDied","Data":"c93a4a08fa232c04944da1fd55e30640aed032ccdfa6251fe0c32ba04d0761bf"} Jul 16 18:09:47.827265 ip-10-0-132-130 kubenswrapper[2578]: I0716 18:09:47.827212 2578 scope.go:117] "RemoveContainer" containerID="05a70d3185be453f75c8b8e6e77316d348e5977a79799a994d642dc573cfe160" Jul 16 18:09:47.832354 ip-10-0-132-130 kubenswrapper[2578]: I0716 18:09:47.832326 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/258f94f2-e4b2-49e3-b38c-48d83c67a352-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "258f94f2-e4b2-49e3-b38c-48d83c67a352" (UID: "258f94f2-e4b2-49e3-b38c-48d83c67a352"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 16 18:09:47.840560 ip-10-0-132-130 kubenswrapper[2578]: I0716 18:09:47.840542 2578 scope.go:117] "RemoveContainer" containerID="5bf4ec205623c6b3a278633137a7ebcbe6f326140784f54413e5e678381f68d1" Jul 16 18:09:47.849566 ip-10-0-132-130 kubenswrapper[2578]: I0716 18:09:47.849550 2578 scope.go:117] "RemoveContainer" containerID="05a70d3185be453f75c8b8e6e77316d348e5977a79799a994d642dc573cfe160" Jul 16 18:09:47.849850 ip-10-0-132-130 kubenswrapper[2578]: E0716 18:09:47.849831 2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"05a70d3185be453f75c8b8e6e77316d348e5977a79799a994d642dc573cfe160\": container with ID starting with 05a70d3185be453f75c8b8e6e77316d348e5977a79799a994d642dc573cfe160 not found: ID does not exist" containerID="05a70d3185be453f75c8b8e6e77316d348e5977a79799a994d642dc573cfe160" Jul 16 18:09:47.849905 ip-10-0-132-130 kubenswrapper[2578]: I0716 18:09:47.849858 2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"05a70d3185be453f75c8b8e6e77316d348e5977a79799a994d642dc573cfe160"} err="failed to get container status \"05a70d3185be453f75c8b8e6e77316d348e5977a79799a994d642dc573cfe160\": rpc error: code = NotFound desc = could not find container \"05a70d3185be453f75c8b8e6e77316d348e5977a79799a994d642dc573cfe160\": container with ID starting with 05a70d3185be453f75c8b8e6e77316d348e5977a79799a994d642dc573cfe160 not found: ID does not exist" Jul 16 18:09:47.849905 ip-10-0-132-130 kubenswrapper[2578]: I0716 18:09:47.849875 2578 scope.go:117] "RemoveContainer" containerID="5bf4ec205623c6b3a278633137a7ebcbe6f326140784f54413e5e678381f68d1" Jul 16 18:09:47.850125 ip-10-0-132-130 kubenswrapper[2578]: E0716 18:09:47.850109 2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5bf4ec205623c6b3a278633137a7ebcbe6f326140784f54413e5e678381f68d1\": container with ID starting with 5bf4ec205623c6b3a278633137a7ebcbe6f326140784f54413e5e678381f68d1 not found: ID does not exist" containerID="5bf4ec205623c6b3a278633137a7ebcbe6f326140784f54413e5e678381f68d1" Jul 16 18:09:47.850175 ip-10-0-132-130 kubenswrapper[2578]: I0716 18:09:47.850127 2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5bf4ec205623c6b3a278633137a7ebcbe6f326140784f54413e5e678381f68d1"} err="failed to get container status \"5bf4ec205623c6b3a278633137a7ebcbe6f326140784f54413e5e678381f68d1\": rpc error: code = NotFound desc = could not find container \"5bf4ec205623c6b3a278633137a7ebcbe6f326140784f54413e5e678381f68d1\": container with ID starting with 5bf4ec205623c6b3a278633137a7ebcbe6f326140784f54413e5e678381f68d1 not found: ID does not exist" Jul 16 18:09:47.878353 ip-10-0-132-130 kubenswrapper[2578]: I0716 18:09:47.878328 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/258f94f2-e4b2-49e3-b38c-48d83c67a352-kserve-provision-location" (OuterVolumeSpecName: "kserve-provision-location") pod "258f94f2-e4b2-49e3-b38c-48d83c67a352" (UID: "258f94f2-e4b2-49e3-b38c-48d83c67a352"). InnerVolumeSpecName "kserve-provision-location". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 16 18:09:47.917653 ip-10-0-132-130 kubenswrapper[2578]: I0716 18:09:47.917626 2578 reconciler_common.go:299] "Volume detached for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/258f94f2-e4b2-49e3-b38c-48d83c67a352-kserve-provision-location\") on node \"ip-10-0-132-130.ec2.internal\" DevicePath \"\"" Jul 16 18:09:47.917778 ip-10-0-132-130 kubenswrapper[2578]: I0716 18:09:47.917655 2578 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/258f94f2-e4b2-49e3-b38c-48d83c67a352-tls-certs\") on node \"ip-10-0-132-130.ec2.internal\" DevicePath \"\"" Jul 16 18:09:47.917778 ip-10-0-132-130 kubenswrapper[2578]: I0716 18:09:47.917670 2578 reconciler_common.go:299] "Volume detached for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/258f94f2-e4b2-49e3-b38c-48d83c67a352-cabundle-cert\") on node \"ip-10-0-132-130.ec2.internal\" DevicePath \"\"" Jul 16 18:09:47.917778 ip-10-0-132-130 kubenswrapper[2578]: I0716 18:09:47.917681 2578 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/258f94f2-e4b2-49e3-b38c-48d83c67a352-tmp-dir\") on node \"ip-10-0-132-130.ec2.internal\" DevicePath \"\"" Jul 16 18:09:47.917778 ip-10-0-132-130 kubenswrapper[2578]: I0716 18:09:47.917709 2578 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/258f94f2-e4b2-49e3-b38c-48d83c67a352-model-cache\") on node \"ip-10-0-132-130.ec2.internal\" DevicePath \"\"" Jul 16 18:09:47.917778 ip-10-0-132-130 kubenswrapper[2578]: I0716 18:09:47.917721 2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-69w8j\" (UniqueName: \"kubernetes.io/projected/258f94f2-e4b2-49e3-b38c-48d83c67a352-kube-api-access-69w8j\") on node \"ip-10-0-132-130.ec2.internal\" DevicePath \"\"" Jul 16 18:09:47.917778 ip-10-0-132-130 kubenswrapper[2578]: I0716 18:09:47.917732 2578 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/258f94f2-e4b2-49e3-b38c-48d83c67a352-dshm\") on node \"ip-10-0-132-130.ec2.internal\" DevicePath \"\"" Jul 16 18:09:48.152441 ip-10-0-132-130 kubenswrapper[2578]: I0716 18:09:48.152412 2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-with-7ca60146-kserve-74d68bf984-2dn6z"] Jul 16 18:09:48.157876 ip-10-0-132-130 kubenswrapper[2578]: I0716 18:09:48.157851 2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-with-7ca60146-kserve-74d68bf984-2dn6z"] Jul 16 18:09:48.215898 ip-10-0-132-130 kubenswrapper[2578]: I0716 18:09:48.215871 2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="258f94f2-e4b2-49e3-b38c-48d83c67a352" path="/var/lib/kubelet/pods/258f94f2-e4b2-49e3-b38c-48d83c67a352/volumes" Jul 16 18:09:55.848989 ip-10-0-132-130 kubenswrapper[2578]: I0716 18:09:55.848954 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-with-ba4d693a-kserve-9bddccd4c-z5w8r"] Jul 16 18:09:55.849387 ip-10-0-132-130 kubenswrapper[2578]: I0716 18:09:55.849374 2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="258f94f2-e4b2-49e3-b38c-48d83c67a352" containerName="main" Jul 16 18:09:55.849387 ip-10-0-132-130 kubenswrapper[2578]: I0716 18:09:55.849388 2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="258f94f2-e4b2-49e3-b38c-48d83c67a352" containerName="main" Jul 16 18:09:55.849489 ip-10-0-132-130 kubenswrapper[2578]: I0716 18:09:55.849405 2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="258f94f2-e4b2-49e3-b38c-48d83c67a352" containerName="storage-initializer" Jul 16 18:09:55.849489 ip-10-0-132-130 kubenswrapper[2578]: I0716 18:09:55.849412 2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="258f94f2-e4b2-49e3-b38c-48d83c67a352" containerName="storage-initializer" Jul 16 18:09:55.849489 ip-10-0-132-130 kubenswrapper[2578]: I0716 18:09:55.849485 2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="258f94f2-e4b2-49e3-b38c-48d83c67a352" containerName="main" Jul 16 18:09:55.854499 ip-10-0-132-130 kubenswrapper[2578]: I0716 18:09:55.854479 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-with-ba4d693a-kserve-9bddccd4c-z5w8r" Jul 16 18:09:55.856729 ip-10-0-132-130 kubenswrapper[2578]: I0716 18:09:55.856708 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"odh-kserve-custom-ca-bundle\"" Jul 16 18:09:55.856729 ip-10-0-132-130 kubenswrapper[2578]: I0716 18:09:55.856719 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"seaweedfs-s3-creds\"" Jul 16 18:09:55.856879 ip-10-0-132-130 kubenswrapper[2578]: I0716 18:09:55.856709 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"llmisv77ff2528d3e9b4972cd9335229fce9f0-kserve-self-signed-certs\"" Jul 16 18:09:55.864456 ip-10-0-132-130 kubenswrapper[2578]: I0716 18:09:55.864436 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-with-ba4d693a-kserve-9bddccd4c-z5w8r"] Jul 16 18:09:55.886412 ip-10-0-132-130 kubenswrapper[2578]: I0716 18:09:55.886372 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d8e3525f-b61f-4ed2-ad06-a136527b145c-tls-certs\") pod \"llmisvc-model-fb-opt-125m-with-ba4d693a-kserve-9bddccd4c-z5w8r\" (UID: \"d8e3525f-b61f-4ed2-ad06-a136527b145c\") " pod="kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-with-ba4d693a-kserve-9bddccd4c-z5w8r" Jul 16 18:09:55.886563 ip-10-0-132-130 kubenswrapper[2578]: I0716 18:09:55.886420 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tsqt9\" (UniqueName: \"kubernetes.io/projected/d8e3525f-b61f-4ed2-ad06-a136527b145c-kube-api-access-tsqt9\") pod \"llmisvc-model-fb-opt-125m-with-ba4d693a-kserve-9bddccd4c-z5w8r\" (UID: \"d8e3525f-b61f-4ed2-ad06-a136527b145c\") " pod="kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-with-ba4d693a-kserve-9bddccd4c-z5w8r" Jul 16 18:09:55.886563 ip-10-0-132-130 kubenswrapper[2578]: I0716 18:09:55.886503 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d8e3525f-b61f-4ed2-ad06-a136527b145c-home\") pod \"llmisvc-model-fb-opt-125m-with-ba4d693a-kserve-9bddccd4c-z5w8r\" (UID: \"d8e3525f-b61f-4ed2-ad06-a136527b145c\") " pod="kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-with-ba4d693a-kserve-9bddccd4c-z5w8r" Jul 16 18:09:55.886563 ip-10-0-132-130 kubenswrapper[2578]: I0716 18:09:55.886524 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d8e3525f-b61f-4ed2-ad06-a136527b145c-model-cache\") pod \"llmisvc-model-fb-opt-125m-with-ba4d693a-kserve-9bddccd4c-z5w8r\" (UID: \"d8e3525f-b61f-4ed2-ad06-a136527b145c\") " pod="kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-with-ba4d693a-kserve-9bddccd4c-z5w8r" Jul 16 18:09:55.886563 ip-10-0-132-130 kubenswrapper[2578]: I0716 18:09:55.886543 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d8e3525f-b61f-4ed2-ad06-a136527b145c-dshm\") pod \"llmisvc-model-fb-opt-125m-with-ba4d693a-kserve-9bddccd4c-z5w8r\" (UID: \"d8e3525f-b61f-4ed2-ad06-a136527b145c\") " pod="kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-with-ba4d693a-kserve-9bddccd4c-z5w8r" Jul 16 18:09:55.886858 ip-10-0-132-130 kubenswrapper[2578]: I0716 18:09:55.886571 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/d8e3525f-b61f-4ed2-ad06-a136527b145c-kserve-provision-location\") pod \"llmisvc-model-fb-opt-125m-with-ba4d693a-kserve-9bddccd4c-z5w8r\" (UID: \"d8e3525f-b61f-4ed2-ad06-a136527b145c\") " pod="kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-with-ba4d693a-kserve-9bddccd4c-z5w8r" Jul 16 18:09:55.886858 ip-10-0-132-130 kubenswrapper[2578]: I0716 18:09:55.886640 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/d8e3525f-b61f-4ed2-ad06-a136527b145c-cabundle-cert\") pod \"llmisvc-model-fb-opt-125m-with-ba4d693a-kserve-9bddccd4c-z5w8r\" (UID: \"d8e3525f-b61f-4ed2-ad06-a136527b145c\") " pod="kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-with-ba4d693a-kserve-9bddccd4c-z5w8r" Jul 16 18:09:55.886858 ip-10-0-132-130 kubenswrapper[2578]: I0716 18:09:55.886680 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d8e3525f-b61f-4ed2-ad06-a136527b145c-tmp-dir\") pod \"llmisvc-model-fb-opt-125m-with-ba4d693a-kserve-9bddccd4c-z5w8r\" (UID: \"d8e3525f-b61f-4ed2-ad06-a136527b145c\") " pod="kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-with-ba4d693a-kserve-9bddccd4c-z5w8r" Jul 16 18:09:55.987100 ip-10-0-132-130 kubenswrapper[2578]: I0716 18:09:55.987071 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tsqt9\" (UniqueName: \"kubernetes.io/projected/d8e3525f-b61f-4ed2-ad06-a136527b145c-kube-api-access-tsqt9\") pod \"llmisvc-model-fb-opt-125m-with-ba4d693a-kserve-9bddccd4c-z5w8r\" (UID: \"d8e3525f-b61f-4ed2-ad06-a136527b145c\") " pod="kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-with-ba4d693a-kserve-9bddccd4c-z5w8r" Jul 16 18:09:55.987252 ip-10-0-132-130 kubenswrapper[2578]: I0716 18:09:55.987122 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d8e3525f-b61f-4ed2-ad06-a136527b145c-home\") pod \"llmisvc-model-fb-opt-125m-with-ba4d693a-kserve-9bddccd4c-z5w8r\" (UID: \"d8e3525f-b61f-4ed2-ad06-a136527b145c\") " pod="kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-with-ba4d693a-kserve-9bddccd4c-z5w8r" Jul 16 18:09:55.987252 ip-10-0-132-130 kubenswrapper[2578]: I0716 18:09:55.987146 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d8e3525f-b61f-4ed2-ad06-a136527b145c-model-cache\") pod \"llmisvc-model-fb-opt-125m-with-ba4d693a-kserve-9bddccd4c-z5w8r\" (UID: \"d8e3525f-b61f-4ed2-ad06-a136527b145c\") " pod="kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-with-ba4d693a-kserve-9bddccd4c-z5w8r" Jul 16 18:09:55.987252 ip-10-0-132-130 kubenswrapper[2578]: I0716 18:09:55.987165 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d8e3525f-b61f-4ed2-ad06-a136527b145c-dshm\") pod \"llmisvc-model-fb-opt-125m-with-ba4d693a-kserve-9bddccd4c-z5w8r\" (UID: \"d8e3525f-b61f-4ed2-ad06-a136527b145c\") " pod="kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-with-ba4d693a-kserve-9bddccd4c-z5w8r" Jul 16 18:09:55.987252 ip-10-0-132-130 kubenswrapper[2578]: I0716 18:09:55.987188 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/d8e3525f-b61f-4ed2-ad06-a136527b145c-kserve-provision-location\") pod \"llmisvc-model-fb-opt-125m-with-ba4d693a-kserve-9bddccd4c-z5w8r\" (UID: \"d8e3525f-b61f-4ed2-ad06-a136527b145c\") " pod="kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-with-ba4d693a-kserve-9bddccd4c-z5w8r" Jul 16 18:09:55.987252 ip-10-0-132-130 kubenswrapper[2578]: I0716 18:09:55.987233 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/d8e3525f-b61f-4ed2-ad06-a136527b145c-cabundle-cert\") pod \"llmisvc-model-fb-opt-125m-with-ba4d693a-kserve-9bddccd4c-z5w8r\" (UID: \"d8e3525f-b61f-4ed2-ad06-a136527b145c\") " pod="kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-with-ba4d693a-kserve-9bddccd4c-z5w8r" Jul 16 18:09:55.987498 ip-10-0-132-130 kubenswrapper[2578]: I0716 18:09:55.987268 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d8e3525f-b61f-4ed2-ad06-a136527b145c-tmp-dir\") pod \"llmisvc-model-fb-opt-125m-with-ba4d693a-kserve-9bddccd4c-z5w8r\" (UID: \"d8e3525f-b61f-4ed2-ad06-a136527b145c\") " pod="kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-with-ba4d693a-kserve-9bddccd4c-z5w8r" Jul 16 18:09:55.987498 ip-10-0-132-130 kubenswrapper[2578]: I0716 18:09:55.987325 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d8e3525f-b61f-4ed2-ad06-a136527b145c-tls-certs\") pod \"llmisvc-model-fb-opt-125m-with-ba4d693a-kserve-9bddccd4c-z5w8r\" (UID: \"d8e3525f-b61f-4ed2-ad06-a136527b145c\") " pod="kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-with-ba4d693a-kserve-9bddccd4c-z5w8r" Jul 16 18:09:55.987612 ip-10-0-132-130 kubenswrapper[2578]: I0716 18:09:55.987584 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d8e3525f-b61f-4ed2-ad06-a136527b145c-model-cache\") pod \"llmisvc-model-fb-opt-125m-with-ba4d693a-kserve-9bddccd4c-z5w8r\" (UID: \"d8e3525f-b61f-4ed2-ad06-a136527b145c\") " pod="kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-with-ba4d693a-kserve-9bddccd4c-z5w8r" Jul 16 18:09:55.987668 ip-10-0-132-130 kubenswrapper[2578]: I0716 18:09:55.987634 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d8e3525f-b61f-4ed2-ad06-a136527b145c-home\") pod \"llmisvc-model-fb-opt-125m-with-ba4d693a-kserve-9bddccd4c-z5w8r\" (UID: \"d8e3525f-b61f-4ed2-ad06-a136527b145c\") " pod="kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-with-ba4d693a-kserve-9bddccd4c-z5w8r" Jul 16 18:09:55.987803 ip-10-0-132-130 kubenswrapper[2578]: I0716 18:09:55.987774 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d8e3525f-b61f-4ed2-ad06-a136527b145c-tmp-dir\") pod \"llmisvc-model-fb-opt-125m-with-ba4d693a-kserve-9bddccd4c-z5w8r\" (UID: \"d8e3525f-b61f-4ed2-ad06-a136527b145c\") " pod="kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-with-ba4d693a-kserve-9bddccd4c-z5w8r" Jul 16 18:09:55.987922 ip-10-0-132-130 kubenswrapper[2578]: I0716 18:09:55.987841 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/d8e3525f-b61f-4ed2-ad06-a136527b145c-kserve-provision-location\") pod \"llmisvc-model-fb-opt-125m-with-ba4d693a-kserve-9bddccd4c-z5w8r\" (UID: \"d8e3525f-b61f-4ed2-ad06-a136527b145c\") " pod="kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-with-ba4d693a-kserve-9bddccd4c-z5w8r" Jul 16 18:09:55.988057 ip-10-0-132-130 kubenswrapper[2578]: I0716 18:09:55.988039 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/d8e3525f-b61f-4ed2-ad06-a136527b145c-cabundle-cert\") pod \"llmisvc-model-fb-opt-125m-with-ba4d693a-kserve-9bddccd4c-z5w8r\" (UID: \"d8e3525f-b61f-4ed2-ad06-a136527b145c\") " pod="kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-with-ba4d693a-kserve-9bddccd4c-z5w8r" Jul 16 18:09:55.989437 ip-10-0-132-130 kubenswrapper[2578]: I0716 18:09:55.989417 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d8e3525f-b61f-4ed2-ad06-a136527b145c-dshm\") pod \"llmisvc-model-fb-opt-125m-with-ba4d693a-kserve-9bddccd4c-z5w8r\" (UID: \"d8e3525f-b61f-4ed2-ad06-a136527b145c\") " pod="kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-with-ba4d693a-kserve-9bddccd4c-z5w8r" Jul 16 18:09:55.989760 ip-10-0-132-130 kubenswrapper[2578]: I0716 18:09:55.989744 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d8e3525f-b61f-4ed2-ad06-a136527b145c-tls-certs\") pod \"llmisvc-model-fb-opt-125m-with-ba4d693a-kserve-9bddccd4c-z5w8r\" (UID: \"d8e3525f-b61f-4ed2-ad06-a136527b145c\") " pod="kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-with-ba4d693a-kserve-9bddccd4c-z5w8r" Jul 16 18:09:55.996808 ip-10-0-132-130 kubenswrapper[2578]: I0716 18:09:55.996787 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tsqt9\" (UniqueName: \"kubernetes.io/projected/d8e3525f-b61f-4ed2-ad06-a136527b145c-kube-api-access-tsqt9\") pod \"llmisvc-model-fb-opt-125m-with-ba4d693a-kserve-9bddccd4c-z5w8r\" (UID: \"d8e3525f-b61f-4ed2-ad06-a136527b145c\") " pod="kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-with-ba4d693a-kserve-9bddccd4c-z5w8r" Jul 16 18:09:56.167037 ip-10-0-132-130 kubenswrapper[2578]: I0716 18:09:56.166847 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-with-ba4d693a-kserve-9bddccd4c-z5w8r" Jul 16 18:09:56.327961 ip-10-0-132-130 kubenswrapper[2578]: I0716 18:09:56.327928 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-with-ba4d693a-kserve-9bddccd4c-z5w8r"] Jul 16 18:09:56.329151 ip-10-0-132-130 kubenswrapper[2578]: W0716 18:09:56.329125 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd8e3525f_b61f_4ed2_ad06_a136527b145c.slice/crio-3f719f96a870a4bfcfb06d31276c066a08c99cb172ec4f56ba56b6e62edde0ee WatchSource:0}: Error finding container 3f719f96a870a4bfcfb06d31276c066a08c99cb172ec4f56ba56b6e62edde0ee: Status 404 returned error can't find the container with id 3f719f96a870a4bfcfb06d31276c066a08c99cb172ec4f56ba56b6e62edde0ee Jul 16 18:09:56.882837 ip-10-0-132-130 kubenswrapper[2578]: I0716 18:09:56.882800 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-with-ba4d693a-kserve-9bddccd4c-z5w8r" event={"ID":"d8e3525f-b61f-4ed2-ad06-a136527b145c","Type":"ContainerStarted","Data":"dcf5cf29e5693d6c37126b5ca3260a1f89d6d10cc42e33d4e047409c58500be2"} Jul 16 18:09:56.883217 ip-10-0-132-130 kubenswrapper[2578]: I0716 18:09:56.882858 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-with-ba4d693a-kserve-9bddccd4c-z5w8r" event={"ID":"d8e3525f-b61f-4ed2-ad06-a136527b145c","Type":"ContainerStarted","Data":"3f719f96a870a4bfcfb06d31276c066a08c99cb172ec4f56ba56b6e62edde0ee"} Jul 16 18:09:59.911005 ip-10-0-132-130 kubenswrapper[2578]: I0716 18:09:59.910970 2578 generic.go:356] "Generic (PLEG): container finished" podID="d8e3525f-b61f-4ed2-ad06-a136527b145c" containerID="dcf5cf29e5693d6c37126b5ca3260a1f89d6d10cc42e33d4e047409c58500be2" exitCode=0 Jul 16 18:09:59.911355 ip-10-0-132-130 kubenswrapper[2578]: I0716 18:09:59.911038 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-with-ba4d693a-kserve-9bddccd4c-z5w8r" event={"ID":"d8e3525f-b61f-4ed2-ad06-a136527b145c","Type":"ContainerDied","Data":"dcf5cf29e5693d6c37126b5ca3260a1f89d6d10cc42e33d4e047409c58500be2"} Jul 16 18:10:00.918947 ip-10-0-132-130 kubenswrapper[2578]: I0716 18:10:00.918912 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-with-ba4d693a-kserve-9bddccd4c-z5w8r" event={"ID":"d8e3525f-b61f-4ed2-ad06-a136527b145c","Type":"ContainerStarted","Data":"f888d24b6db9e475c2466382df41b81cf2ea94da280b979d8b57f147b0605e4b"} Jul 16 18:10:00.943443 ip-10-0-132-130 kubenswrapper[2578]: I0716 18:10:00.943393 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-with-ba4d693a-kserve-9bddccd4c-z5w8r" podStartSLOduration=2.571057094 podStartE2EDuration="5.943377353s" totalImagesPullingTime="0s" totalInitContainerRuntime="3.372320259s" isStatefulPod=false podCreationTimestamp="2026-07-16 18:09:55 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 18:10:00.941521945 +0000 UTC m=+3345.427365836" watchObservedRunningTime="2026-07-16 18:10:00.943377353 +0000 UTC m=+3345.429221242" Jul 16 18:10:01.168147 ip-10-0-132-130 kubenswrapper[2578]: I0716 18:10:01.168113 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-with-ba4d693a-kserve-9bddccd4c-z5w8r" Jul 16 18:10:06.168116 ip-10-0-132-130 kubenswrapper[2578]: I0716 18:10:06.168079 2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-with-ba4d693a-kserve-9bddccd4c-z5w8r" Jul 16 18:10:06.169800 ip-10-0-132-130 kubenswrapper[2578]: I0716 18:10:06.169761 2578 prober.go:120] "Probe failed" probeType="Startup" pod="kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-with-ba4d693a-kserve-9bddccd4c-z5w8r" podUID="d8e3525f-b61f-4ed2-ad06-a136527b145c" containerName="main" probeResult="failure" output="Get \"https://10.133.0.60:8000/health\": dial tcp 10.133.0.60:8000: connect: connection refused" Jul 16 18:10:16.167500 ip-10-0-132-130 kubenswrapper[2578]: I0716 18:10:16.167454 2578 prober.go:120] "Probe failed" probeType="Startup" pod="kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-with-ba4d693a-kserve-9bddccd4c-z5w8r" podUID="d8e3525f-b61f-4ed2-ad06-a136527b145c" containerName="main" probeResult="failure" output="Get \"https://10.133.0.60:8000/health\": dial tcp 10.133.0.60:8000: connect: connection refused" Jul 16 18:10:26.167831 ip-10-0-132-130 kubenswrapper[2578]: I0716 18:10:26.167774 2578 prober.go:120] "Probe failed" probeType="Startup" pod="kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-with-ba4d693a-kserve-9bddccd4c-z5w8r" podUID="d8e3525f-b61f-4ed2-ad06-a136527b145c" containerName="main" probeResult="failure" output="Get \"https://10.133.0.60:8000/health\": dial tcp 10.133.0.60:8000: connect: connection refused" Jul 16 18:10:36.168749 ip-10-0-132-130 kubenswrapper[2578]: I0716 18:10:36.168703 2578 prober.go:120] "Probe failed" probeType="Startup" pod="kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-with-ba4d693a-kserve-9bddccd4c-z5w8r" podUID="d8e3525f-b61f-4ed2-ad06-a136527b145c" containerName="main" probeResult="failure" output="Get \"https://10.133.0.60:8000/health\": dial tcp 10.133.0.60:8000: connect: connection refused" Jul 16 18:10:46.168088 ip-10-0-132-130 kubenswrapper[2578]: I0716 18:10:46.168045 2578 prober.go:120] "Probe failed" probeType="Startup" pod="kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-with-ba4d693a-kserve-9bddccd4c-z5w8r" podUID="d8e3525f-b61f-4ed2-ad06-a136527b145c" containerName="main" probeResult="failure" output="Get \"https://10.133.0.60:8000/health\": dial tcp 10.133.0.60:8000: connect: connection refused" Jul 16 18:10:56.168230 ip-10-0-132-130 kubenswrapper[2578]: I0716 18:10:56.168180 2578 prober.go:120] "Probe failed" probeType="Startup" pod="kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-with-ba4d693a-kserve-9bddccd4c-z5w8r" podUID="d8e3525f-b61f-4ed2-ad06-a136527b145c" containerName="main" probeResult="failure" output="Get \"https://10.133.0.60:8000/health\": dial tcp 10.133.0.60:8000: connect: connection refused" Jul 16 18:11:06.168048 ip-10-0-132-130 kubenswrapper[2578]: I0716 18:11:06.168009 2578 prober.go:120] "Probe failed" probeType="Startup" pod="kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-with-ba4d693a-kserve-9bddccd4c-z5w8r" podUID="d8e3525f-b61f-4ed2-ad06-a136527b145c" containerName="main" probeResult="failure" output="Get \"https://10.133.0.60:8000/health\": dial tcp 10.133.0.60:8000: connect: connection refused" Jul 16 18:11:16.177705 ip-10-0-132-130 kubenswrapper[2578]: I0716 18:11:16.177657 2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-with-ba4d693a-kserve-9bddccd4c-z5w8r" Jul 16 18:11:16.185552 ip-10-0-132-130 kubenswrapper[2578]: I0716 18:11:16.185528 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-with-ba4d693a-kserve-9bddccd4c-z5w8r" Jul 16 18:14:16.631032 ip-10-0-132-130 kubenswrapper[2578]: I0716 18:14:16.630929 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-drs2p_e55f447c-ad64-482b-a86b-0588bddf7b4d/console-operator/1.log" Jul 16 18:14:16.662128 ip-10-0-132-130 kubenswrapper[2578]: I0716 18:14:16.662103 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-drs2p_e55f447c-ad64-482b-a86b-0588bddf7b4d/console-operator/1.log" Jul 16 18:19:16.680397 ip-10-0-132-130 kubenswrapper[2578]: I0716 18:19:16.680285 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-drs2p_e55f447c-ad64-482b-a86b-0588bddf7b4d/console-operator/1.log" Jul 16 18:19:16.705332 ip-10-0-132-130 kubenswrapper[2578]: I0716 18:19:16.705306 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-drs2p_e55f447c-ad64-482b-a86b-0588bddf7b4d/console-operator/1.log" Jul 16 18:24:16.719378 ip-10-0-132-130 kubenswrapper[2578]: I0716 18:24:16.719270 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-drs2p_e55f447c-ad64-482b-a86b-0588bddf7b4d/console-operator/1.log" Jul 16 18:24:16.748851 ip-10-0-132-130 kubenswrapper[2578]: I0716 18:24:16.748827 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-drs2p_e55f447c-ad64-482b-a86b-0588bddf7b4d/console-operator/1.log" Jul 16 18:26:18.068959 ip-10-0-132-130 kubenswrapper[2578]: I0716 18:26:18.068921 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_llmisvc-model-fb-opt-125m-with-ba4d693a-kserve-9bddccd4c-z5w8r_d8e3525f-b61f-4ed2-ad06-a136527b145c/storage-initializer/0.log" Jul 16 18:26:18.082027 ip-10-0-132-130 kubenswrapper[2578]: I0716 18:26:18.081993 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_llmisvc-model-fb-opt-125m-with-ba4d693a-kserve-9bddccd4c-z5w8r_d8e3525f-b61f-4ed2-ad06-a136527b145c/main/0.log" Jul 16 18:26:41.388507 ip-10-0-132-130 kubenswrapper[2578]: I0716 18:26:41.388479 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_fc-concurrency-test-kserve-router-scheduler-78c47c8645-m62qx_d0224cda-56fd-4b88-b980-34220b0dd2dd/main/0.log" Jul 16 18:26:41.629728 ip-10-0-132-130 kubenswrapper[2578]: I0716 18:26:41.629685 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_llmisvc-model-fb-opt-125m-with-ba4d693a-kserve-9bddccd4c-z5w8r_d8e3525f-b61f-4ed2-ad06-a136527b145c/main/0.log" Jul 16 18:26:41.640806 ip-10-0-132-130 kubenswrapper[2578]: I0716 18:26:41.640725 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_llmisvc-model-fb-opt-125m-with-ba4d693a-kserve-9bddccd4c-z5w8r_d8e3525f-b61f-4ed2-ad06-a136527b145c/storage-initializer/0.log" Jul 16 18:26:41.740485 ip-10-0-132-130 kubenswrapper[2578]: I0716 18:26:41.740456 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_llmisvc-router-managed-test-llm-5b1e8f15-kserve-df5b5c977-xdvj2_ae64d07c-dbee-4be2-936d-7efc5dfdbd89/main/0.log" Jul 16 18:26:41.761354 ip-10-0-132-130 kubenswrapper[2578]: I0716 18:26:41.761331 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_llmisvcca2d2d7d499abb359505529ebe02c136-kserve-router-schep28gf_1500b232-42ed-4480-a44b-41139e735a71/main/0.log" Jul 16 18:26:42.819736 ip-10-0-132-130 kubenswrapper[2578]: I0716 18:26:42.819684 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_fc-concurrency-test-kserve-router-scheduler-78c47c8645-m62qx_d0224cda-56fd-4b88-b980-34220b0dd2dd/main/0.log" Jul 16 18:26:43.017743 ip-10-0-132-130 kubenswrapper[2578]: I0716 18:26:43.017709 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_llmisvc-model-fb-opt-125m-with-ba4d693a-kserve-9bddccd4c-z5w8r_d8e3525f-b61f-4ed2-ad06-a136527b145c/main/0.log" Jul 16 18:26:43.032932 ip-10-0-132-130 kubenswrapper[2578]: I0716 18:26:43.032899 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_llmisvc-model-fb-opt-125m-with-ba4d693a-kserve-9bddccd4c-z5w8r_d8e3525f-b61f-4ed2-ad06-a136527b145c/storage-initializer/0.log" Jul 16 18:26:43.135551 ip-10-0-132-130 kubenswrapper[2578]: I0716 18:26:43.135444 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_llmisvc-router-managed-test-llm-5b1e8f15-kserve-df5b5c977-xdvj2_ae64d07c-dbee-4be2-936d-7efc5dfdbd89/main/0.log" Jul 16 18:26:43.163256 ip-10-0-132-130 kubenswrapper[2578]: I0716 18:26:43.163231 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_llmisvcca2d2d7d499abb359505529ebe02c136-kserve-router-schep28gf_1500b232-42ed-4480-a44b-41139e735a71/main/0.log" Jul 16 18:26:44.266641 ip-10-0-132-130 kubenswrapper[2578]: I0716 18:26:44.266607 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_fc-concurrency-test-kserve-router-scheduler-78c47c8645-m62qx_d0224cda-56fd-4b88-b980-34220b0dd2dd/main/0.log" Jul 16 18:26:44.430243 ip-10-0-132-130 kubenswrapper[2578]: I0716 18:26:44.430215 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_llmisvc-model-fb-opt-125m-with-ba4d693a-kserve-9bddccd4c-z5w8r_d8e3525f-b61f-4ed2-ad06-a136527b145c/main/0.log" Jul 16 18:26:44.437408 ip-10-0-132-130 kubenswrapper[2578]: I0716 18:26:44.437383 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_llmisvc-model-fb-opt-125m-with-ba4d693a-kserve-9bddccd4c-z5w8r_d8e3525f-b61f-4ed2-ad06-a136527b145c/storage-initializer/0.log" Jul 16 18:26:44.503135 ip-10-0-132-130 kubenswrapper[2578]: I0716 18:26:44.503114 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_llmisvc-router-managed-test-llm-5b1e8f15-kserve-df5b5c977-xdvj2_ae64d07c-dbee-4be2-936d-7efc5dfdbd89/main/0.log" Jul 16 18:26:44.518388 ip-10-0-132-130 kubenswrapper[2578]: I0716 18:26:44.518322 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_llmisvcca2d2d7d499abb359505529ebe02c136-kserve-router-schep28gf_1500b232-42ed-4480-a44b-41139e735a71/main/0.log" Jul 16 18:26:45.581300 ip-10-0-132-130 kubenswrapper[2578]: I0716 18:26:45.581269 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_fc-concurrency-test-kserve-router-scheduler-78c47c8645-m62qx_d0224cda-56fd-4b88-b980-34220b0dd2dd/main/0.log" Jul 16 18:26:45.748893 ip-10-0-132-130 kubenswrapper[2578]: I0716 18:26:45.748859 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_llmisvc-model-fb-opt-125m-with-ba4d693a-kserve-9bddccd4c-z5w8r_d8e3525f-b61f-4ed2-ad06-a136527b145c/main/0.log" Jul 16 18:26:45.756912 ip-10-0-132-130 kubenswrapper[2578]: I0716 18:26:45.756885 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_llmisvc-model-fb-opt-125m-with-ba4d693a-kserve-9bddccd4c-z5w8r_d8e3525f-b61f-4ed2-ad06-a136527b145c/storage-initializer/0.log" Jul 16 18:26:45.820950 ip-10-0-132-130 kubenswrapper[2578]: I0716 18:26:45.820925 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_llmisvc-router-managed-test-llm-5b1e8f15-kserve-df5b5c977-xdvj2_ae64d07c-dbee-4be2-936d-7efc5dfdbd89/main/0.log" Jul 16 18:26:45.835303 ip-10-0-132-130 kubenswrapper[2578]: I0716 18:26:45.835224 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_llmisvcca2d2d7d499abb359505529ebe02c136-kserve-router-schep28gf_1500b232-42ed-4480-a44b-41139e735a71/main/0.log" Jul 16 18:26:46.886675 ip-10-0-132-130 kubenswrapper[2578]: I0716 18:26:46.886643 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_fc-concurrency-test-kserve-router-scheduler-78c47c8645-m62qx_d0224cda-56fd-4b88-b980-34220b0dd2dd/main/0.log" Jul 16 18:26:47.054653 ip-10-0-132-130 kubenswrapper[2578]: I0716 18:26:47.054624 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_llmisvc-model-fb-opt-125m-with-ba4d693a-kserve-9bddccd4c-z5w8r_d8e3525f-b61f-4ed2-ad06-a136527b145c/main/0.log" Jul 16 18:26:47.063388 ip-10-0-132-130 kubenswrapper[2578]: I0716 18:26:47.063363 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_llmisvc-model-fb-opt-125m-with-ba4d693a-kserve-9bddccd4c-z5w8r_d8e3525f-b61f-4ed2-ad06-a136527b145c/storage-initializer/0.log" Jul 16 18:26:47.136389 ip-10-0-132-130 kubenswrapper[2578]: I0716 18:26:47.136369 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_llmisvc-router-managed-test-llm-5b1e8f15-kserve-df5b5c977-xdvj2_ae64d07c-dbee-4be2-936d-7efc5dfdbd89/main/0.log" Jul 16 18:26:47.152943 ip-10-0-132-130 kubenswrapper[2578]: I0716 18:26:47.152861 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_llmisvcca2d2d7d499abb359505529ebe02c136-kserve-router-schep28gf_1500b232-42ed-4480-a44b-41139e735a71/main/0.log" Jul 16 18:26:48.297076 ip-10-0-132-130 kubenswrapper[2578]: I0716 18:26:48.297051 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_fc-concurrency-test-kserve-router-scheduler-78c47c8645-m62qx_d0224cda-56fd-4b88-b980-34220b0dd2dd/main/0.log" Jul 16 18:26:48.466608 ip-10-0-132-130 kubenswrapper[2578]: I0716 18:26:48.466578 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_llmisvc-model-fb-opt-125m-with-ba4d693a-kserve-9bddccd4c-z5w8r_d8e3525f-b61f-4ed2-ad06-a136527b145c/main/0.log" Jul 16 18:26:48.475245 ip-10-0-132-130 kubenswrapper[2578]: I0716 18:26:48.475223 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_llmisvc-model-fb-opt-125m-with-ba4d693a-kserve-9bddccd4c-z5w8r_d8e3525f-b61f-4ed2-ad06-a136527b145c/storage-initializer/0.log" Jul 16 18:26:48.543061 ip-10-0-132-130 kubenswrapper[2578]: I0716 18:26:48.543036 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_llmisvc-router-managed-test-llm-5b1e8f15-kserve-df5b5c977-xdvj2_ae64d07c-dbee-4be2-936d-7efc5dfdbd89/main/0.log" Jul 16 18:26:48.567921 ip-10-0-132-130 kubenswrapper[2578]: I0716 18:26:48.567853 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_llmisvcca2d2d7d499abb359505529ebe02c136-kserve-router-schep28gf_1500b232-42ed-4480-a44b-41139e735a71/main/0.log" Jul 16 18:26:49.678849 ip-10-0-132-130 kubenswrapper[2578]: I0716 18:26:49.678822 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_fc-concurrency-test-kserve-router-scheduler-78c47c8645-m62qx_d0224cda-56fd-4b88-b980-34220b0dd2dd/main/0.log" Jul 16 18:26:49.851053 ip-10-0-132-130 kubenswrapper[2578]: I0716 18:26:49.851029 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_llmisvc-model-fb-opt-125m-with-ba4d693a-kserve-9bddccd4c-z5w8r_d8e3525f-b61f-4ed2-ad06-a136527b145c/main/0.log" Jul 16 18:26:49.859635 ip-10-0-132-130 kubenswrapper[2578]: I0716 18:26:49.859615 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_llmisvc-model-fb-opt-125m-with-ba4d693a-kserve-9bddccd4c-z5w8r_d8e3525f-b61f-4ed2-ad06-a136527b145c/storage-initializer/0.log" Jul 16 18:26:49.932529 ip-10-0-132-130 kubenswrapper[2578]: I0716 18:26:49.932456 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_llmisvc-router-managed-test-llm-5b1e8f15-kserve-df5b5c977-xdvj2_ae64d07c-dbee-4be2-936d-7efc5dfdbd89/main/0.log" Jul 16 18:26:49.952543 ip-10-0-132-130 kubenswrapper[2578]: I0716 18:26:49.952520 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_llmisvcca2d2d7d499abb359505529ebe02c136-kserve-router-schep28gf_1500b232-42ed-4480-a44b-41139e735a71/main/0.log" Jul 16 18:26:50.968991 ip-10-0-132-130 kubenswrapper[2578]: I0716 18:26:50.968957 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_fc-concurrency-test-kserve-router-scheduler-78c47c8645-m62qx_d0224cda-56fd-4b88-b980-34220b0dd2dd/main/0.log" Jul 16 18:26:51.134898 ip-10-0-132-130 kubenswrapper[2578]: I0716 18:26:51.134868 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_llmisvc-model-fb-opt-125m-with-ba4d693a-kserve-9bddccd4c-z5w8r_d8e3525f-b61f-4ed2-ad06-a136527b145c/main/0.log" Jul 16 18:26:51.141937 ip-10-0-132-130 kubenswrapper[2578]: I0716 18:26:51.141916 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_llmisvc-model-fb-opt-125m-with-ba4d693a-kserve-9bddccd4c-z5w8r_d8e3525f-b61f-4ed2-ad06-a136527b145c/storage-initializer/0.log" Jul 16 18:26:51.204999 ip-10-0-132-130 kubenswrapper[2578]: I0716 18:26:51.204975 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_llmisvc-router-managed-test-llm-5b1e8f15-kserve-df5b5c977-xdvj2_ae64d07c-dbee-4be2-936d-7efc5dfdbd89/main/0.log" Jul 16 18:26:51.218991 ip-10-0-132-130 kubenswrapper[2578]: I0716 18:26:51.218969 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_llmisvcca2d2d7d499abb359505529ebe02c136-kserve-router-schep28gf_1500b232-42ed-4480-a44b-41139e735a71/main/0.log" Jul 16 18:26:52.316897 ip-10-0-132-130 kubenswrapper[2578]: I0716 18:26:52.316860 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_fc-concurrency-test-kserve-router-scheduler-78c47c8645-m62qx_d0224cda-56fd-4b88-b980-34220b0dd2dd/main/0.log" Jul 16 18:26:52.479752 ip-10-0-132-130 kubenswrapper[2578]: I0716 18:26:52.479720 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_llmisvc-model-fb-opt-125m-with-ba4d693a-kserve-9bddccd4c-z5w8r_d8e3525f-b61f-4ed2-ad06-a136527b145c/main/0.log" Jul 16 18:26:52.487362 ip-10-0-132-130 kubenswrapper[2578]: I0716 18:26:52.487342 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_llmisvc-model-fb-opt-125m-with-ba4d693a-kserve-9bddccd4c-z5w8r_d8e3525f-b61f-4ed2-ad06-a136527b145c/storage-initializer/0.log" Jul 16 18:26:52.549775 ip-10-0-132-130 kubenswrapper[2578]: I0716 18:26:52.549754 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_llmisvc-router-managed-test-llm-5b1e8f15-kserve-df5b5c977-xdvj2_ae64d07c-dbee-4be2-936d-7efc5dfdbd89/main/0.log" Jul 16 18:26:52.564663 ip-10-0-132-130 kubenswrapper[2578]: I0716 18:26:52.564637 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_llmisvcca2d2d7d499abb359505529ebe02c136-kserve-router-schep28gf_1500b232-42ed-4480-a44b-41139e735a71/main/0.log" Jul 16 18:26:53.622956 ip-10-0-132-130 kubenswrapper[2578]: I0716 18:26:53.622926 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_fc-concurrency-test-kserve-router-scheduler-78c47c8645-m62qx_d0224cda-56fd-4b88-b980-34220b0dd2dd/main/0.log" Jul 16 18:26:53.785924 ip-10-0-132-130 kubenswrapper[2578]: I0716 18:26:53.785889 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_llmisvc-model-fb-opt-125m-with-ba4d693a-kserve-9bddccd4c-z5w8r_d8e3525f-b61f-4ed2-ad06-a136527b145c/main/0.log" Jul 16 18:26:53.793626 ip-10-0-132-130 kubenswrapper[2578]: I0716 18:26:53.793605 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_llmisvc-model-fb-opt-125m-with-ba4d693a-kserve-9bddccd4c-z5w8r_d8e3525f-b61f-4ed2-ad06-a136527b145c/storage-initializer/0.log" Jul 16 18:26:53.856640 ip-10-0-132-130 kubenswrapper[2578]: I0716 18:26:53.856600 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_llmisvc-router-managed-test-llm-5b1e8f15-kserve-df5b5c977-xdvj2_ae64d07c-dbee-4be2-936d-7efc5dfdbd89/main/0.log" Jul 16 18:26:53.870141 ip-10-0-132-130 kubenswrapper[2578]: I0716 18:26:53.870113 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_llmisvcca2d2d7d499abb359505529ebe02c136-kserve-router-schep28gf_1500b232-42ed-4480-a44b-41139e735a71/main/0.log" Jul 16 18:26:54.895250 ip-10-0-132-130 kubenswrapper[2578]: I0716 18:26:54.895217 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_fc-concurrency-test-kserve-router-scheduler-78c47c8645-m62qx_d0224cda-56fd-4b88-b980-34220b0dd2dd/main/0.log" Jul 16 18:26:55.069855 ip-10-0-132-130 kubenswrapper[2578]: I0716 18:26:55.069822 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_llmisvc-model-fb-opt-125m-with-ba4d693a-kserve-9bddccd4c-z5w8r_d8e3525f-b61f-4ed2-ad06-a136527b145c/main/0.log" Jul 16 18:26:55.076839 ip-10-0-132-130 kubenswrapper[2578]: I0716 18:26:55.076812 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_llmisvc-model-fb-opt-125m-with-ba4d693a-kserve-9bddccd4c-z5w8r_d8e3525f-b61f-4ed2-ad06-a136527b145c/storage-initializer/0.log" Jul 16 18:26:55.141123 ip-10-0-132-130 kubenswrapper[2578]: I0716 18:26:55.141095 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_llmisvc-router-managed-test-llm-5b1e8f15-kserve-df5b5c977-xdvj2_ae64d07c-dbee-4be2-936d-7efc5dfdbd89/main/0.log" Jul 16 18:26:55.155248 ip-10-0-132-130 kubenswrapper[2578]: I0716 18:26:55.155192 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_llmisvcca2d2d7d499abb359505529ebe02c136-kserve-router-schep28gf_1500b232-42ed-4480-a44b-41139e735a71/main/0.log" Jul 16 18:26:56.190323 ip-10-0-132-130 kubenswrapper[2578]: I0716 18:26:56.190292 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_fc-concurrency-test-kserve-router-scheduler-78c47c8645-m62qx_d0224cda-56fd-4b88-b980-34220b0dd2dd/main/0.log" Jul 16 18:26:56.351907 ip-10-0-132-130 kubenswrapper[2578]: I0716 18:26:56.351878 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_llmisvc-model-fb-opt-125m-with-ba4d693a-kserve-9bddccd4c-z5w8r_d8e3525f-b61f-4ed2-ad06-a136527b145c/main/0.log" Jul 16 18:26:56.359429 ip-10-0-132-130 kubenswrapper[2578]: I0716 18:26:56.359400 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_llmisvc-model-fb-opt-125m-with-ba4d693a-kserve-9bddccd4c-z5w8r_d8e3525f-b61f-4ed2-ad06-a136527b145c/storage-initializer/0.log" Jul 16 18:26:56.424466 ip-10-0-132-130 kubenswrapper[2578]: I0716 18:26:56.424428 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_llmisvc-router-managed-test-llm-5b1e8f15-kserve-df5b5c977-xdvj2_ae64d07c-dbee-4be2-936d-7efc5dfdbd89/main/0.log" Jul 16 18:26:56.440128 ip-10-0-132-130 kubenswrapper[2578]: I0716 18:26:56.440097 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_llmisvcca2d2d7d499abb359505529ebe02c136-kserve-router-schep28gf_1500b232-42ed-4480-a44b-41139e735a71/main/0.log" Jul 16 18:26:57.465620 ip-10-0-132-130 kubenswrapper[2578]: I0716 18:26:57.465586 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_fc-concurrency-test-kserve-router-scheduler-78c47c8645-m62qx_d0224cda-56fd-4b88-b980-34220b0dd2dd/main/0.log" Jul 16 18:26:57.628243 ip-10-0-132-130 kubenswrapper[2578]: I0716 18:26:57.628211 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_llmisvc-model-fb-opt-125m-with-ba4d693a-kserve-9bddccd4c-z5w8r_d8e3525f-b61f-4ed2-ad06-a136527b145c/main/0.log" Jul 16 18:26:57.635755 ip-10-0-132-130 kubenswrapper[2578]: I0716 18:26:57.635724 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_llmisvc-model-fb-opt-125m-with-ba4d693a-kserve-9bddccd4c-z5w8r_d8e3525f-b61f-4ed2-ad06-a136527b145c/storage-initializer/0.log" Jul 16 18:26:57.704679 ip-10-0-132-130 kubenswrapper[2578]: I0716 18:26:57.704624 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_llmisvc-router-managed-test-llm-5b1e8f15-kserve-df5b5c977-xdvj2_ae64d07c-dbee-4be2-936d-7efc5dfdbd89/main/0.log" Jul 16 18:26:57.719559 ip-10-0-132-130 kubenswrapper[2578]: I0716 18:26:57.719488 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_llmisvcca2d2d7d499abb359505529ebe02c136-kserve-router-schep28gf_1500b232-42ed-4480-a44b-41139e735a71/main/0.log" Jul 16 18:26:58.777633 ip-10-0-132-130 kubenswrapper[2578]: I0716 18:26:58.777599 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_fc-concurrency-test-kserve-router-scheduler-78c47c8645-m62qx_d0224cda-56fd-4b88-b980-34220b0dd2dd/main/0.log" Jul 16 18:26:58.945071 ip-10-0-132-130 kubenswrapper[2578]: I0716 18:26:58.945043 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_llmisvc-model-fb-opt-125m-with-ba4d693a-kserve-9bddccd4c-z5w8r_d8e3525f-b61f-4ed2-ad06-a136527b145c/main/0.log" Jul 16 18:26:58.952088 ip-10-0-132-130 kubenswrapper[2578]: I0716 18:26:58.952064 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_llmisvc-model-fb-opt-125m-with-ba4d693a-kserve-9bddccd4c-z5w8r_d8e3525f-b61f-4ed2-ad06-a136527b145c/storage-initializer/0.log" Jul 16 18:26:59.016090 ip-10-0-132-130 kubenswrapper[2578]: I0716 18:26:59.016059 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_llmisvc-router-managed-test-llm-5b1e8f15-kserve-df5b5c977-xdvj2_ae64d07c-dbee-4be2-936d-7efc5dfdbd89/main/0.log" Jul 16 18:26:59.032351 ip-10-0-132-130 kubenswrapper[2578]: I0716 18:26:59.032278 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_llmisvcca2d2d7d499abb359505529ebe02c136-kserve-router-schep28gf_1500b232-42ed-4480-a44b-41139e735a71/main/0.log" Jul 16 18:27:00.032144 ip-10-0-132-130 kubenswrapper[2578]: I0716 18:27:00.032114 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_fc-concurrency-test-kserve-router-scheduler-78c47c8645-m62qx_d0224cda-56fd-4b88-b980-34220b0dd2dd/main/0.log" Jul 16 18:27:00.202187 ip-10-0-132-130 kubenswrapper[2578]: I0716 18:27:00.202155 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_llmisvc-model-fb-opt-125m-with-ba4d693a-kserve-9bddccd4c-z5w8r_d8e3525f-b61f-4ed2-ad06-a136527b145c/main/0.log" Jul 16 18:27:00.209794 ip-10-0-132-130 kubenswrapper[2578]: I0716 18:27:00.209767 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_llmisvc-model-fb-opt-125m-with-ba4d693a-kserve-9bddccd4c-z5w8r_d8e3525f-b61f-4ed2-ad06-a136527b145c/storage-initializer/0.log" Jul 16 18:27:00.273603 ip-10-0-132-130 kubenswrapper[2578]: I0716 18:27:00.273566 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_llmisvc-router-managed-test-llm-5b1e8f15-kserve-df5b5c977-xdvj2_ae64d07c-dbee-4be2-936d-7efc5dfdbd89/main/0.log" Jul 16 18:27:00.287468 ip-10-0-132-130 kubenswrapper[2578]: I0716 18:27:00.287387 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_llmisvcca2d2d7d499abb359505529ebe02c136-kserve-router-schep28gf_1500b232-42ed-4480-a44b-41139e735a71/main/0.log" Jul 16 18:27:01.720729 ip-10-0-132-130 kubenswrapper[2578]: I0716 18:27:01.720666 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-94bb8fbfd-qnh6j_54c915b4-e326-4c4e-b79e-5d0c16cd2811/discovery/0.log" Jul 16 18:27:02.902573 ip-10-0-132-130 kubenswrapper[2578]: I0716 18:27:02.902541 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-94bb8fbfd-qnh6j_54c915b4-e326-4c4e-b79e-5d0c16cd2811/discovery/0.log" Jul 16 18:27:04.129121 ip-10-0-132-130 kubenswrapper[2578]: I0716 18:27:04.129087 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-94bb8fbfd-qnh6j_54c915b4-e326-4c4e-b79e-5d0c16cd2811/discovery/0.log" Jul 16 18:27:05.321936 ip-10-0-132-130 kubenswrapper[2578]: I0716 18:27:05.321905 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-94bb8fbfd-qnh6j_54c915b4-e326-4c4e-b79e-5d0c16cd2811/discovery/0.log" Jul 16 18:27:06.138245 ip-10-0-132-130 kubenswrapper[2578]: I0716 18:27:06.138213 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-operator-6d85f6564-c4d5z_5798b263-2def-43aa-bc40-bd06b5948f2c/manager/0.log" Jul 16 18:27:06.150618 ip-10-0-132-130 kubenswrapper[2578]: I0716 18:27:06.150576 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_dns-operator-controller-manager-65b49595d7-m7vkn_366102cc-6258-4e5a-b8f6-4ce0659b3601/manager/0.log" Jul 16 18:27:06.307936 ip-10-0-132-130 kubenswrapper[2578]: I0716 18:27:06.307905 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-controller-manager-bb7ddb7d7-fp8pd_4c5a2957-e580-423a-97a9-5ffb61407ab4/manager/0.log" Jul 16 18:27:07.194673 ip-10-0-132-130 kubenswrapper[2578]: I0716 18:27:07.194622 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-operator-6d85f6564-c4d5z_5798b263-2def-43aa-bc40-bd06b5948f2c/manager/0.log" Jul 16 18:27:07.211654 ip-10-0-132-130 kubenswrapper[2578]: I0716 18:27:07.211630 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_dns-operator-controller-manager-65b49595d7-m7vkn_366102cc-6258-4e5a-b8f6-4ce0659b3601/manager/0.log" Jul 16 18:27:07.378622 ip-10-0-132-130 kubenswrapper[2578]: I0716 18:27:07.378593 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-controller-manager-bb7ddb7d7-fp8pd_4c5a2957-e580-423a-97a9-5ffb61407ab4/manager/0.log" Jul 16 18:27:08.276642 ip-10-0-132-130 kubenswrapper[2578]: I0716 18:27:08.276611 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-operator-6d85f6564-c4d5z_5798b263-2def-43aa-bc40-bd06b5948f2c/manager/0.log" Jul 16 18:27:08.290000 ip-10-0-132-130 kubenswrapper[2578]: I0716 18:27:08.289976 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_dns-operator-controller-manager-65b49595d7-m7vkn_366102cc-6258-4e5a-b8f6-4ce0659b3601/manager/0.log" Jul 16 18:27:08.445615 ip-10-0-132-130 kubenswrapper[2578]: I0716 18:27:08.445584 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-controller-manager-bb7ddb7d7-fp8pd_4c5a2957-e580-423a-97a9-5ffb61407ab4/manager/0.log" Jul 16 18:27:09.306905 ip-10-0-132-130 kubenswrapper[2578]: I0716 18:27:09.306875 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-operator-6d85f6564-c4d5z_5798b263-2def-43aa-bc40-bd06b5948f2c/manager/0.log" Jul 16 18:27:09.320336 ip-10-0-132-130 kubenswrapper[2578]: I0716 18:27:09.320304 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_dns-operator-controller-manager-65b49595d7-m7vkn_366102cc-6258-4e5a-b8f6-4ce0659b3601/manager/0.log" Jul 16 18:27:09.475973 ip-10-0-132-130 kubenswrapper[2578]: I0716 18:27:09.475940 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-controller-manager-bb7ddb7d7-fp8pd_4c5a2957-e580-423a-97a9-5ffb61407ab4/manager/0.log" Jul 16 18:27:10.301326 ip-10-0-132-130 kubenswrapper[2578]: I0716 18:27:10.301296 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-operator-6d85f6564-c4d5z_5798b263-2def-43aa-bc40-bd06b5948f2c/manager/0.log" Jul 16 18:27:10.312092 ip-10-0-132-130 kubenswrapper[2578]: I0716 18:27:10.312066 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_dns-operator-controller-manager-65b49595d7-m7vkn_366102cc-6258-4e5a-b8f6-4ce0659b3601/manager/0.log" Jul 16 18:27:10.467251 ip-10-0-132-130 kubenswrapper[2578]: I0716 18:27:10.467220 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-controller-manager-bb7ddb7d7-fp8pd_4c5a2957-e580-423a-97a9-5ffb61407ab4/manager/0.log" Jul 16 18:27:11.279894 ip-10-0-132-130 kubenswrapper[2578]: I0716 18:27:11.279868 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-operator-6d85f6564-c4d5z_5798b263-2def-43aa-bc40-bd06b5948f2c/manager/0.log" Jul 16 18:27:11.291884 ip-10-0-132-130 kubenswrapper[2578]: I0716 18:27:11.291852 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_dns-operator-controller-manager-65b49595d7-m7vkn_366102cc-6258-4e5a-b8f6-4ce0659b3601/manager/0.log" Jul 16 18:27:11.445991 ip-10-0-132-130 kubenswrapper[2578]: I0716 18:27:11.445960 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-controller-manager-bb7ddb7d7-fp8pd_4c5a2957-e580-423a-97a9-5ffb61407ab4/manager/0.log" Jul 16 18:27:12.271820 ip-10-0-132-130 kubenswrapper[2578]: I0716 18:27:12.271778 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-operator-6d85f6564-c4d5z_5798b263-2def-43aa-bc40-bd06b5948f2c/manager/0.log" Jul 16 18:27:12.288580 ip-10-0-132-130 kubenswrapper[2578]: I0716 18:27:12.288556 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_dns-operator-controller-manager-65b49595d7-m7vkn_366102cc-6258-4e5a-b8f6-4ce0659b3601/manager/0.log" Jul 16 18:27:12.441479 ip-10-0-132-130 kubenswrapper[2578]: I0716 18:27:12.441441 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-controller-manager-bb7ddb7d7-fp8pd_4c5a2957-e580-423a-97a9-5ffb61407ab4/manager/0.log" Jul 16 18:27:13.296323 ip-10-0-132-130 kubenswrapper[2578]: I0716 18:27:13.296293 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-operator-6d85f6564-c4d5z_5798b263-2def-43aa-bc40-bd06b5948f2c/manager/0.log" Jul 16 18:27:13.309077 ip-10-0-132-130 kubenswrapper[2578]: I0716 18:27:13.309057 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_dns-operator-controller-manager-65b49595d7-m7vkn_366102cc-6258-4e5a-b8f6-4ce0659b3601/manager/0.log" Jul 16 18:27:13.464802 ip-10-0-132-130 kubenswrapper[2578]: I0716 18:27:13.464767 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-controller-manager-bb7ddb7d7-fp8pd_4c5a2957-e580-423a-97a9-5ffb61407ab4/manager/0.log" Jul 16 18:27:14.357855 ip-10-0-132-130 kubenswrapper[2578]: I0716 18:27:14.357819 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-operator-6d85f6564-c4d5z_5798b263-2def-43aa-bc40-bd06b5948f2c/manager/0.log" Jul 16 18:27:14.369522 ip-10-0-132-130 kubenswrapper[2578]: I0716 18:27:14.369498 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_dns-operator-controller-manager-65b49595d7-m7vkn_366102cc-6258-4e5a-b8f6-4ce0659b3601/manager/0.log" Jul 16 18:27:14.525285 ip-10-0-132-130 kubenswrapper[2578]: I0716 18:27:14.525257 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-controller-manager-bb7ddb7d7-fp8pd_4c5a2957-e580-423a-97a9-5ffb61407ab4/manager/0.log" Jul 16 18:27:15.359798 ip-10-0-132-130 kubenswrapper[2578]: I0716 18:27:15.359769 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-operator-6d85f6564-c4d5z_5798b263-2def-43aa-bc40-bd06b5948f2c/manager/0.log" Jul 16 18:27:15.372295 ip-10-0-132-130 kubenswrapper[2578]: I0716 18:27:15.372275 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_dns-operator-controller-manager-65b49595d7-m7vkn_366102cc-6258-4e5a-b8f6-4ce0659b3601/manager/0.log" Jul 16 18:27:15.526291 ip-10-0-132-130 kubenswrapper[2578]: I0716 18:27:15.526234 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-controller-manager-bb7ddb7d7-fp8pd_4c5a2957-e580-423a-97a9-5ffb61407ab4/manager/0.log" Jul 16 18:27:16.401156 ip-10-0-132-130 kubenswrapper[2578]: I0716 18:27:16.401117 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-operator-6d85f6564-c4d5z_5798b263-2def-43aa-bc40-bd06b5948f2c/manager/0.log" Jul 16 18:27:16.414198 ip-10-0-132-130 kubenswrapper[2578]: I0716 18:27:16.414173 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_dns-operator-controller-manager-65b49595d7-m7vkn_366102cc-6258-4e5a-b8f6-4ce0659b3601/manager/0.log" Jul 16 18:27:16.572870 ip-10-0-132-130 kubenswrapper[2578]: I0716 18:27:16.572828 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-controller-manager-bb7ddb7d7-fp8pd_4c5a2957-e580-423a-97a9-5ffb61407ab4/manager/0.log" Jul 16 18:27:17.423483 ip-10-0-132-130 kubenswrapper[2578]: I0716 18:27:17.423453 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-operator-6d85f6564-c4d5z_5798b263-2def-43aa-bc40-bd06b5948f2c/manager/0.log" Jul 16 18:27:17.442672 ip-10-0-132-130 kubenswrapper[2578]: I0716 18:27:17.442647 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_dns-operator-controller-manager-65b49595d7-m7vkn_366102cc-6258-4e5a-b8f6-4ce0659b3601/manager/0.log" Jul 16 18:27:17.606002 ip-10-0-132-130 kubenswrapper[2578]: I0716 18:27:17.605968 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-controller-manager-bb7ddb7d7-fp8pd_4c5a2957-e580-423a-97a9-5ffb61407ab4/manager/0.log" Jul 16 18:27:18.436267 ip-10-0-132-130 kubenswrapper[2578]: I0716 18:27:18.436238 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-operator-6d85f6564-c4d5z_5798b263-2def-43aa-bc40-bd06b5948f2c/manager/0.log" Jul 16 18:27:18.451583 ip-10-0-132-130 kubenswrapper[2578]: I0716 18:27:18.451555 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_dns-operator-controller-manager-65b49595d7-m7vkn_366102cc-6258-4e5a-b8f6-4ce0659b3601/manager/0.log" Jul 16 18:27:18.610197 ip-10-0-132-130 kubenswrapper[2578]: I0716 18:27:18.610161 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-controller-manager-bb7ddb7d7-fp8pd_4c5a2957-e580-423a-97a9-5ffb61407ab4/manager/0.log" Jul 16 18:27:19.439025 ip-10-0-132-130 kubenswrapper[2578]: I0716 18:27:19.438991 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-operator-6d85f6564-c4d5z_5798b263-2def-43aa-bc40-bd06b5948f2c/manager/0.log" Jul 16 18:27:19.451731 ip-10-0-132-130 kubenswrapper[2578]: I0716 18:27:19.451679 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_dns-operator-controller-manager-65b49595d7-m7vkn_366102cc-6258-4e5a-b8f6-4ce0659b3601/manager/0.log" Jul 16 18:27:19.604976 ip-10-0-132-130 kubenswrapper[2578]: I0716 18:27:19.604945 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-controller-manager-bb7ddb7d7-fp8pd_4c5a2957-e580-423a-97a9-5ffb61407ab4/manager/0.log" Jul 16 18:27:20.451866 ip-10-0-132-130 kubenswrapper[2578]: I0716 18:27:20.451835 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-operator-6d85f6564-c4d5z_5798b263-2def-43aa-bc40-bd06b5948f2c/manager/0.log" Jul 16 18:27:20.469826 ip-10-0-132-130 kubenswrapper[2578]: I0716 18:27:20.469795 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_dns-operator-controller-manager-65b49595d7-m7vkn_366102cc-6258-4e5a-b8f6-4ce0659b3601/manager/0.log" Jul 16 18:27:20.618267 ip-10-0-132-130 kubenswrapper[2578]: I0716 18:27:20.618240 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-controller-manager-bb7ddb7d7-fp8pd_4c5a2957-e580-423a-97a9-5ffb61407ab4/manager/0.log" Jul 16 18:27:21.507314 ip-10-0-132-130 kubenswrapper[2578]: I0716 18:27:21.507286 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-operator-6d85f6564-c4d5z_5798b263-2def-43aa-bc40-bd06b5948f2c/manager/0.log" Jul 16 18:27:21.520330 ip-10-0-132-130 kubenswrapper[2578]: I0716 18:27:21.520304 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_dns-operator-controller-manager-65b49595d7-m7vkn_366102cc-6258-4e5a-b8f6-4ce0659b3601/manager/0.log" Jul 16 18:27:21.678945 ip-10-0-132-130 kubenswrapper[2578]: I0716 18:27:21.678911 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-controller-manager-bb7ddb7d7-fp8pd_4c5a2957-e580-423a-97a9-5ffb61407ab4/manager/0.log" Jul 16 18:27:22.504118 ip-10-0-132-130 kubenswrapper[2578]: I0716 18:27:22.504088 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-operator-6d85f6564-c4d5z_5798b263-2def-43aa-bc40-bd06b5948f2c/manager/0.log" Jul 16 18:27:22.517037 ip-10-0-132-130 kubenswrapper[2578]: I0716 18:27:22.517013 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_dns-operator-controller-manager-65b49595d7-m7vkn_366102cc-6258-4e5a-b8f6-4ce0659b3601/manager/0.log" Jul 16 18:27:22.676069 ip-10-0-132-130 kubenswrapper[2578]: I0716 18:27:22.676041 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-controller-manager-bb7ddb7d7-fp8pd_4c5a2957-e580-423a-97a9-5ffb61407ab4/manager/0.log" Jul 16 18:27:23.532718 ip-10-0-132-130 kubenswrapper[2578]: I0716 18:27:23.532668 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-operator-6d85f6564-c4d5z_5798b263-2def-43aa-bc40-bd06b5948f2c/manager/0.log" Jul 16 18:27:23.546568 ip-10-0-132-130 kubenswrapper[2578]: I0716 18:27:23.546540 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_dns-operator-controller-manager-65b49595d7-m7vkn_366102cc-6258-4e5a-b8f6-4ce0659b3601/manager/0.log" Jul 16 18:27:23.706566 ip-10-0-132-130 kubenswrapper[2578]: I0716 18:27:23.706533 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-controller-manager-bb7ddb7d7-fp8pd_4c5a2957-e580-423a-97a9-5ffb61407ab4/manager/0.log" Jul 16 18:27:24.557003 ip-10-0-132-130 kubenswrapper[2578]: I0716 18:27:24.556966 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-operator-6d85f6564-c4d5z_5798b263-2def-43aa-bc40-bd06b5948f2c/manager/0.log" Jul 16 18:27:24.576288 ip-10-0-132-130 kubenswrapper[2578]: I0716 18:27:24.576264 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_dns-operator-controller-manager-65b49595d7-m7vkn_366102cc-6258-4e5a-b8f6-4ce0659b3601/manager/0.log" Jul 16 18:27:24.740536 ip-10-0-132-130 kubenswrapper[2578]: I0716 18:27:24.740510 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-controller-manager-bb7ddb7d7-fp8pd_4c5a2957-e580-423a-97a9-5ffb61407ab4/manager/0.log" Jul 16 18:27:25.548347 ip-10-0-132-130 kubenswrapper[2578]: I0716 18:27:25.548314 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-operator-6d85f6564-c4d5z_5798b263-2def-43aa-bc40-bd06b5948f2c/manager/0.log" Jul 16 18:27:25.562946 ip-10-0-132-130 kubenswrapper[2578]: I0716 18:27:25.562912 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_dns-operator-controller-manager-65b49595d7-m7vkn_366102cc-6258-4e5a-b8f6-4ce0659b3601/manager/0.log" Jul 16 18:27:25.717342 ip-10-0-132-130 kubenswrapper[2578]: I0716 18:27:25.717313 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-controller-manager-bb7ddb7d7-fp8pd_4c5a2957-e580-423a-97a9-5ffb61407ab4/manager/0.log" Jul 16 18:27:26.568984 ip-10-0-132-130 kubenswrapper[2578]: I0716 18:27:26.568953 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-operator-6d85f6564-c4d5z_5798b263-2def-43aa-bc40-bd06b5948f2c/manager/0.log" Jul 16 18:27:26.581448 ip-10-0-132-130 kubenswrapper[2578]: I0716 18:27:26.581422 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_dns-operator-controller-manager-65b49595d7-m7vkn_366102cc-6258-4e5a-b8f6-4ce0659b3601/manager/0.log" Jul 16 18:27:26.736384 ip-10-0-132-130 kubenswrapper[2578]: I0716 18:27:26.736356 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-controller-manager-bb7ddb7d7-fp8pd_4c5a2957-e580-423a-97a9-5ffb61407ab4/manager/0.log" Jul 16 18:27:27.584353 ip-10-0-132-130 kubenswrapper[2578]: I0716 18:27:27.584315 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-operator-6d85f6564-c4d5z_5798b263-2def-43aa-bc40-bd06b5948f2c/manager/0.log" Jul 16 18:27:27.597484 ip-10-0-132-130 kubenswrapper[2578]: I0716 18:27:27.597460 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_dns-operator-controller-manager-65b49595d7-m7vkn_366102cc-6258-4e5a-b8f6-4ce0659b3601/manager/0.log" Jul 16 18:27:27.752198 ip-10-0-132-130 kubenswrapper[2578]: I0716 18:27:27.752166 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-controller-manager-bb7ddb7d7-fp8pd_4c5a2957-e580-423a-97a9-5ffb61407ab4/manager/0.log" Jul 16 18:27:28.615497 ip-10-0-132-130 kubenswrapper[2578]: I0716 18:27:28.615465 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-operator-6d85f6564-c4d5z_5798b263-2def-43aa-bc40-bd06b5948f2c/manager/0.log" Jul 16 18:27:28.629097 ip-10-0-132-130 kubenswrapper[2578]: I0716 18:27:28.629070 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_dns-operator-controller-manager-65b49595d7-m7vkn_366102cc-6258-4e5a-b8f6-4ce0659b3601/manager/0.log" Jul 16 18:27:28.784890 ip-10-0-132-130 kubenswrapper[2578]: I0716 18:27:28.784856 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-controller-manager-bb7ddb7d7-fp8pd_4c5a2957-e580-423a-97a9-5ffb61407ab4/manager/0.log" Jul 16 18:27:29.647039 ip-10-0-132-130 kubenswrapper[2578]: I0716 18:27:29.647004 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-operator-6d85f6564-c4d5z_5798b263-2def-43aa-bc40-bd06b5948f2c/manager/0.log" Jul 16 18:27:29.658242 ip-10-0-132-130 kubenswrapper[2578]: I0716 18:27:29.658219 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_dns-operator-controller-manager-65b49595d7-m7vkn_366102cc-6258-4e5a-b8f6-4ce0659b3601/manager/0.log" Jul 16 18:27:29.813921 ip-10-0-132-130 kubenswrapper[2578]: I0716 18:27:29.813889 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-controller-manager-bb7ddb7d7-fp8pd_4c5a2957-e580-423a-97a9-5ffb61407ab4/manager/0.log" Jul 16 18:27:30.689903 ip-10-0-132-130 kubenswrapper[2578]: I0716 18:27:30.689872 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-operator-6d85f6564-c4d5z_5798b263-2def-43aa-bc40-bd06b5948f2c/manager/0.log" Jul 16 18:27:30.703825 ip-10-0-132-130 kubenswrapper[2578]: I0716 18:27:30.703796 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_dns-operator-controller-manager-65b49595d7-m7vkn_366102cc-6258-4e5a-b8f6-4ce0659b3601/manager/0.log" Jul 16 18:27:30.871501 ip-10-0-132-130 kubenswrapper[2578]: I0716 18:27:30.871464 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-controller-manager-bb7ddb7d7-fp8pd_4c5a2957-e580-423a-97a9-5ffb61407ab4/manager/0.log" Jul 16 18:27:31.782748 ip-10-0-132-130 kubenswrapper[2578]: I0716 18:27:31.782704 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-operator-6d85f6564-c4d5z_5798b263-2def-43aa-bc40-bd06b5948f2c/manager/0.log" Jul 16 18:27:31.794986 ip-10-0-132-130 kubenswrapper[2578]: I0716 18:27:31.794961 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_dns-operator-controller-manager-65b49595d7-m7vkn_366102cc-6258-4e5a-b8f6-4ce0659b3601/manager/0.log" Jul 16 18:27:31.951757 ip-10-0-132-130 kubenswrapper[2578]: I0716 18:27:31.951723 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-controller-manager-bb7ddb7d7-fp8pd_4c5a2957-e580-423a-97a9-5ffb61407ab4/manager/0.log" Jul 16 18:27:32.771067 ip-10-0-132-130 kubenswrapper[2578]: I0716 18:27:32.771033 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-operator-6d85f6564-c4d5z_5798b263-2def-43aa-bc40-bd06b5948f2c/manager/0.log" Jul 16 18:27:32.785957 ip-10-0-132-130 kubenswrapper[2578]: I0716 18:27:32.785933 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_dns-operator-controller-manager-65b49595d7-m7vkn_366102cc-6258-4e5a-b8f6-4ce0659b3601/manager/0.log" Jul 16 18:27:32.949131 ip-10-0-132-130 kubenswrapper[2578]: I0716 18:27:32.949099 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-controller-manager-bb7ddb7d7-fp8pd_4c5a2957-e580-423a-97a9-5ffb61407ab4/manager/0.log" Jul 16 18:27:33.780948 ip-10-0-132-130 kubenswrapper[2578]: I0716 18:27:33.780913 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-operator-6d85f6564-c4d5z_5798b263-2def-43aa-bc40-bd06b5948f2c/manager/0.log" Jul 16 18:27:33.801301 ip-10-0-132-130 kubenswrapper[2578]: I0716 18:27:33.801272 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_dns-operator-controller-manager-65b49595d7-m7vkn_366102cc-6258-4e5a-b8f6-4ce0659b3601/manager/0.log" Jul 16 18:27:33.962747 ip-10-0-132-130 kubenswrapper[2578]: I0716 18:27:33.962712 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-controller-manager-bb7ddb7d7-fp8pd_4c5a2957-e580-423a-97a9-5ffb61407ab4/manager/0.log" Jul 16 18:27:34.839953 ip-10-0-132-130 kubenswrapper[2578]: I0716 18:27:34.839917 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-operator-6d85f6564-c4d5z_5798b263-2def-43aa-bc40-bd06b5948f2c/manager/0.log" Jul 16 18:27:34.859521 ip-10-0-132-130 kubenswrapper[2578]: I0716 18:27:34.859496 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_dns-operator-controller-manager-65b49595d7-m7vkn_366102cc-6258-4e5a-b8f6-4ce0659b3601/manager/0.log" Jul 16 18:27:35.021093 ip-10-0-132-130 kubenswrapper[2578]: I0716 18:27:35.021063 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-controller-manager-bb7ddb7d7-fp8pd_4c5a2957-e580-423a-97a9-5ffb61407ab4/manager/0.log" Jul 16 18:27:35.867290 ip-10-0-132-130 kubenswrapper[2578]: I0716 18:27:35.867263 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-operator-6d85f6564-c4d5z_5798b263-2def-43aa-bc40-bd06b5948f2c/manager/0.log" Jul 16 18:27:35.882842 ip-10-0-132-130 kubenswrapper[2578]: I0716 18:27:35.882818 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_dns-operator-controller-manager-65b49595d7-m7vkn_366102cc-6258-4e5a-b8f6-4ce0659b3601/manager/0.log" Jul 16 18:27:36.039492 ip-10-0-132-130 kubenswrapper[2578]: I0716 18:27:36.039464 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-controller-manager-bb7ddb7d7-fp8pd_4c5a2957-e580-423a-97a9-5ffb61407ab4/manager/0.log" Jul 16 18:27:36.849052 ip-10-0-132-130 kubenswrapper[2578]: I0716 18:27:36.849026 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-operator-6d85f6564-c4d5z_5798b263-2def-43aa-bc40-bd06b5948f2c/manager/0.log" Jul 16 18:27:36.869340 ip-10-0-132-130 kubenswrapper[2578]: I0716 18:27:36.869317 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_dns-operator-controller-manager-65b49595d7-m7vkn_366102cc-6258-4e5a-b8f6-4ce0659b3601/manager/0.log" Jul 16 18:27:37.030116 ip-10-0-132-130 kubenswrapper[2578]: I0716 18:27:37.030088 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-controller-manager-bb7ddb7d7-fp8pd_4c5a2957-e580-423a-97a9-5ffb61407ab4/manager/0.log" Jul 16 18:27:37.957206 ip-10-0-132-130 kubenswrapper[2578]: I0716 18:27:37.957179 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-operator-6d85f6564-c4d5z_5798b263-2def-43aa-bc40-bd06b5948f2c/manager/0.log" Jul 16 18:27:37.978647 ip-10-0-132-130 kubenswrapper[2578]: I0716 18:27:37.978618 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_dns-operator-controller-manager-65b49595d7-m7vkn_366102cc-6258-4e5a-b8f6-4ce0659b3601/manager/0.log" Jul 16 18:27:38.153109 ip-10-0-132-130 kubenswrapper[2578]: I0716 18:27:38.153076 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-controller-manager-bb7ddb7d7-fp8pd_4c5a2957-e580-423a-97a9-5ffb61407ab4/manager/0.log" Jul 16 18:27:39.047815 ip-10-0-132-130 kubenswrapper[2578]: I0716 18:27:39.047786 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-operator-6d85f6564-c4d5z_5798b263-2def-43aa-bc40-bd06b5948f2c/manager/0.log" Jul 16 18:27:39.067287 ip-10-0-132-130 kubenswrapper[2578]: I0716 18:27:39.067251 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_dns-operator-controller-manager-65b49595d7-m7vkn_366102cc-6258-4e5a-b8f6-4ce0659b3601/manager/0.log" Jul 16 18:27:39.240585 ip-10-0-132-130 kubenswrapper[2578]: I0716 18:27:39.240553 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-controller-manager-bb7ddb7d7-fp8pd_4c5a2957-e580-423a-97a9-5ffb61407ab4/manager/0.log" Jul 16 18:27:40.147557 ip-10-0-132-130 kubenswrapper[2578]: I0716 18:27:40.147526 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-operator-6d85f6564-c4d5z_5798b263-2def-43aa-bc40-bd06b5948f2c/manager/0.log" Jul 16 18:27:40.160432 ip-10-0-132-130 kubenswrapper[2578]: I0716 18:27:40.160407 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_dns-operator-controller-manager-65b49595d7-m7vkn_366102cc-6258-4e5a-b8f6-4ce0659b3601/manager/0.log" Jul 16 18:27:40.323631 ip-10-0-132-130 kubenswrapper[2578]: I0716 18:27:40.323602 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-controller-manager-bb7ddb7d7-fp8pd_4c5a2957-e580-423a-97a9-5ffb61407ab4/manager/0.log" Jul 16 18:27:41.181303 ip-10-0-132-130 kubenswrapper[2578]: I0716 18:27:41.181272 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-operator-6d85f6564-c4d5z_5798b263-2def-43aa-bc40-bd06b5948f2c/manager/0.log" Jul 16 18:27:41.195686 ip-10-0-132-130 kubenswrapper[2578]: I0716 18:27:41.195662 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_dns-operator-controller-manager-65b49595d7-m7vkn_366102cc-6258-4e5a-b8f6-4ce0659b3601/manager/0.log" Jul 16 18:27:41.355601 ip-10-0-132-130 kubenswrapper[2578]: I0716 18:27:41.355557 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-controller-manager-bb7ddb7d7-fp8pd_4c5a2957-e580-423a-97a9-5ffb61407ab4/manager/0.log" Jul 16 18:27:42.222161 ip-10-0-132-130 kubenswrapper[2578]: I0716 18:27:42.222130 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-operator-6d85f6564-c4d5z_5798b263-2def-43aa-bc40-bd06b5948f2c/manager/0.log" Jul 16 18:27:42.237578 ip-10-0-132-130 kubenswrapper[2578]: I0716 18:27:42.237555 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_dns-operator-controller-manager-65b49595d7-m7vkn_366102cc-6258-4e5a-b8f6-4ce0659b3601/manager/0.log" Jul 16 18:27:42.396328 ip-10-0-132-130 kubenswrapper[2578]: I0716 18:27:42.396299 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-controller-manager-bb7ddb7d7-fp8pd_4c5a2957-e580-423a-97a9-5ffb61407ab4/manager/0.log" Jul 16 18:27:43.247379 ip-10-0-132-130 kubenswrapper[2578]: I0716 18:27:43.247339 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-operator-6d85f6564-c4d5z_5798b263-2def-43aa-bc40-bd06b5948f2c/manager/0.log" Jul 16 18:27:43.263599 ip-10-0-132-130 kubenswrapper[2578]: I0716 18:27:43.263570 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_dns-operator-controller-manager-65b49595d7-m7vkn_366102cc-6258-4e5a-b8f6-4ce0659b3601/manager/0.log" Jul 16 18:27:43.412707 ip-10-0-132-130 kubenswrapper[2578]: I0716 18:27:43.412661 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-controller-manager-bb7ddb7d7-fp8pd_4c5a2957-e580-423a-97a9-5ffb61407ab4/manager/0.log" Jul 16 18:27:44.256978 ip-10-0-132-130 kubenswrapper[2578]: I0716 18:27:44.256948 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-operator-6d85f6564-c4d5z_5798b263-2def-43aa-bc40-bd06b5948f2c/manager/0.log" Jul 16 18:27:44.272042 ip-10-0-132-130 kubenswrapper[2578]: I0716 18:27:44.272015 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_dns-operator-controller-manager-65b49595d7-m7vkn_366102cc-6258-4e5a-b8f6-4ce0659b3601/manager/0.log" Jul 16 18:27:44.432985 ip-10-0-132-130 kubenswrapper[2578]: I0716 18:27:44.432951 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-controller-manager-bb7ddb7d7-fp8pd_4c5a2957-e580-423a-97a9-5ffb61407ab4/manager/0.log" Jul 16 18:27:45.260861 ip-10-0-132-130 kubenswrapper[2578]: I0716 18:27:45.260830 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-operator-6d85f6564-c4d5z_5798b263-2def-43aa-bc40-bd06b5948f2c/manager/0.log" Jul 16 18:27:45.273543 ip-10-0-132-130 kubenswrapper[2578]: I0716 18:27:45.273516 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_dns-operator-controller-manager-65b49595d7-m7vkn_366102cc-6258-4e5a-b8f6-4ce0659b3601/manager/0.log" Jul 16 18:27:45.435312 ip-10-0-132-130 kubenswrapper[2578]: I0716 18:27:45.435279 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-controller-manager-bb7ddb7d7-fp8pd_4c5a2957-e580-423a-97a9-5ffb61407ab4/manager/0.log" Jul 16 18:27:46.268326 ip-10-0-132-130 kubenswrapper[2578]: I0716 18:27:46.268292 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-operator-6d85f6564-c4d5z_5798b263-2def-43aa-bc40-bd06b5948f2c/manager/0.log" Jul 16 18:27:46.281357 ip-10-0-132-130 kubenswrapper[2578]: I0716 18:27:46.281331 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_dns-operator-controller-manager-65b49595d7-m7vkn_366102cc-6258-4e5a-b8f6-4ce0659b3601/manager/0.log" Jul 16 18:27:46.447480 ip-10-0-132-130 kubenswrapper[2578]: I0716 18:27:46.447434 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-controller-manager-bb7ddb7d7-fp8pd_4c5a2957-e580-423a-97a9-5ffb61407ab4/manager/0.log" Jul 16 18:27:47.314019 ip-10-0-132-130 kubenswrapper[2578]: I0716 18:27:47.313982 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-operator-6d85f6564-c4d5z_5798b263-2def-43aa-bc40-bd06b5948f2c/manager/0.log" Jul 16 18:27:47.326838 ip-10-0-132-130 kubenswrapper[2578]: I0716 18:27:47.326810 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_dns-operator-controller-manager-65b49595d7-m7vkn_366102cc-6258-4e5a-b8f6-4ce0659b3601/manager/0.log" Jul 16 18:27:47.489268 ip-10-0-132-130 kubenswrapper[2578]: I0716 18:27:47.489228 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-controller-manager-bb7ddb7d7-fp8pd_4c5a2957-e580-423a-97a9-5ffb61407ab4/manager/0.log" Jul 16 18:27:48.378356 ip-10-0-132-130 kubenswrapper[2578]: I0716 18:27:48.378315 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-operator-6d85f6564-c4d5z_5798b263-2def-43aa-bc40-bd06b5948f2c/manager/0.log" Jul 16 18:27:48.390371 ip-10-0-132-130 kubenswrapper[2578]: I0716 18:27:48.390350 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_dns-operator-controller-manager-65b49595d7-m7vkn_366102cc-6258-4e5a-b8f6-4ce0659b3601/manager/0.log" Jul 16 18:27:48.548975 ip-10-0-132-130 kubenswrapper[2578]: I0716 18:27:48.548925 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-controller-manager-bb7ddb7d7-fp8pd_4c5a2957-e580-423a-97a9-5ffb61407ab4/manager/0.log" Jul 16 18:27:49.388089 ip-10-0-132-130 kubenswrapper[2578]: I0716 18:27:49.388060 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-operator-6d85f6564-c4d5z_5798b263-2def-43aa-bc40-bd06b5948f2c/manager/0.log" Jul 16 18:27:49.401203 ip-10-0-132-130 kubenswrapper[2578]: I0716 18:27:49.401179 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_dns-operator-controller-manager-65b49595d7-m7vkn_366102cc-6258-4e5a-b8f6-4ce0659b3601/manager/0.log" Jul 16 18:27:49.558972 ip-10-0-132-130 kubenswrapper[2578]: I0716 18:27:49.558940 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-controller-manager-bb7ddb7d7-fp8pd_4c5a2957-e580-423a-97a9-5ffb61407ab4/manager/0.log" Jul 16 18:27:50.405298 ip-10-0-132-130 kubenswrapper[2578]: I0716 18:27:50.405266 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-operator-6d85f6564-c4d5z_5798b263-2def-43aa-bc40-bd06b5948f2c/manager/0.log" Jul 16 18:27:50.418122 ip-10-0-132-130 kubenswrapper[2578]: I0716 18:27:50.418099 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_dns-operator-controller-manager-65b49595d7-m7vkn_366102cc-6258-4e5a-b8f6-4ce0659b3601/manager/0.log" Jul 16 18:27:50.577728 ip-10-0-132-130 kubenswrapper[2578]: I0716 18:27:50.577681 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-controller-manager-bb7ddb7d7-fp8pd_4c5a2957-e580-423a-97a9-5ffb61407ab4/manager/0.log" Jul 16 18:27:51.382661 ip-10-0-132-130 kubenswrapper[2578]: I0716 18:27:51.382625 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-operator-6d85f6564-c4d5z_5798b263-2def-43aa-bc40-bd06b5948f2c/manager/0.log" Jul 16 18:27:51.394884 ip-10-0-132-130 kubenswrapper[2578]: I0716 18:27:51.394852 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_dns-operator-controller-manager-65b49595d7-m7vkn_366102cc-6258-4e5a-b8f6-4ce0659b3601/manager/0.log" Jul 16 18:27:51.552360 ip-10-0-132-130 kubenswrapper[2578]: I0716 18:27:51.552330 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-controller-manager-bb7ddb7d7-fp8pd_4c5a2957-e580-423a-97a9-5ffb61407ab4/manager/0.log" Jul 16 18:27:52.370365 ip-10-0-132-130 kubenswrapper[2578]: I0716 18:27:52.370332 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-operator-6d85f6564-c4d5z_5798b263-2def-43aa-bc40-bd06b5948f2c/manager/0.log" Jul 16 18:27:52.384144 ip-10-0-132-130 kubenswrapper[2578]: I0716 18:27:52.384117 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_dns-operator-controller-manager-65b49595d7-m7vkn_366102cc-6258-4e5a-b8f6-4ce0659b3601/manager/0.log" Jul 16 18:27:52.545341 ip-10-0-132-130 kubenswrapper[2578]: I0716 18:27:52.545313 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-controller-manager-bb7ddb7d7-fp8pd_4c5a2957-e580-423a-97a9-5ffb61407ab4/manager/0.log" Jul 16 18:27:53.448311 ip-10-0-132-130 kubenswrapper[2578]: I0716 18:27:53.448279 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-operator-6d85f6564-c4d5z_5798b263-2def-43aa-bc40-bd06b5948f2c/manager/0.log" Jul 16 18:27:53.461578 ip-10-0-132-130 kubenswrapper[2578]: I0716 18:27:53.461557 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_dns-operator-controller-manager-65b49595d7-m7vkn_366102cc-6258-4e5a-b8f6-4ce0659b3601/manager/0.log" Jul 16 18:27:53.622685 ip-10-0-132-130 kubenswrapper[2578]: I0716 18:27:53.622640 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-controller-manager-bb7ddb7d7-fp8pd_4c5a2957-e580-423a-97a9-5ffb61407ab4/manager/0.log" Jul 16 18:27:54.504944 ip-10-0-132-130 kubenswrapper[2578]: I0716 18:27:54.504905 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-operator-6d85f6564-c4d5z_5798b263-2def-43aa-bc40-bd06b5948f2c/manager/0.log" Jul 16 18:27:54.516655 ip-10-0-132-130 kubenswrapper[2578]: I0716 18:27:54.516631 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_dns-operator-controller-manager-65b49595d7-m7vkn_366102cc-6258-4e5a-b8f6-4ce0659b3601/manager/0.log" Jul 16 18:27:54.673897 ip-10-0-132-130 kubenswrapper[2578]: I0716 18:27:54.673863 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-controller-manager-bb7ddb7d7-fp8pd_4c5a2957-e580-423a-97a9-5ffb61407ab4/manager/0.log" Jul 16 18:27:55.544702 ip-10-0-132-130 kubenswrapper[2578]: I0716 18:27:55.544668 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-operator-6d85f6564-c4d5z_5798b263-2def-43aa-bc40-bd06b5948f2c/manager/0.log" Jul 16 18:27:55.556963 ip-10-0-132-130 kubenswrapper[2578]: I0716 18:27:55.556936 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_dns-operator-controller-manager-65b49595d7-m7vkn_366102cc-6258-4e5a-b8f6-4ce0659b3601/manager/0.log" Jul 16 18:27:55.713279 ip-10-0-132-130 kubenswrapper[2578]: I0716 18:27:55.713244 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-controller-manager-bb7ddb7d7-fp8pd_4c5a2957-e580-423a-97a9-5ffb61407ab4/manager/0.log" Jul 16 18:27:56.617522 ip-10-0-132-130 kubenswrapper[2578]: I0716 18:27:56.617493 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-operator-6d85f6564-c4d5z_5798b263-2def-43aa-bc40-bd06b5948f2c/manager/0.log" Jul 16 18:27:56.637883 ip-10-0-132-130 kubenswrapper[2578]: I0716 18:27:56.637857 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_dns-operator-controller-manager-65b49595d7-m7vkn_366102cc-6258-4e5a-b8f6-4ce0659b3601/manager/0.log" Jul 16 18:27:56.793512 ip-10-0-132-130 kubenswrapper[2578]: I0716 18:27:56.793480 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-controller-manager-bb7ddb7d7-fp8pd_4c5a2957-e580-423a-97a9-5ffb61407ab4/manager/0.log" Jul 16 18:27:57.670402 ip-10-0-132-130 kubenswrapper[2578]: I0716 18:27:57.670361 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-operator-6d85f6564-c4d5z_5798b263-2def-43aa-bc40-bd06b5948f2c/manager/0.log" Jul 16 18:27:57.682658 ip-10-0-132-130 kubenswrapper[2578]: I0716 18:27:57.682629 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_dns-operator-controller-manager-65b49595d7-m7vkn_366102cc-6258-4e5a-b8f6-4ce0659b3601/manager/0.log" Jul 16 18:27:57.842181 ip-10-0-132-130 kubenswrapper[2578]: I0716 18:27:57.842144 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-controller-manager-bb7ddb7d7-fp8pd_4c5a2957-e580-423a-97a9-5ffb61407ab4/manager/0.log" Jul 16 18:27:58.718496 ip-10-0-132-130 kubenswrapper[2578]: I0716 18:27:58.718466 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-operator-6d85f6564-c4d5z_5798b263-2def-43aa-bc40-bd06b5948f2c/manager/0.log" Jul 16 18:27:58.732629 ip-10-0-132-130 kubenswrapper[2578]: I0716 18:27:58.732602 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_dns-operator-controller-manager-65b49595d7-m7vkn_366102cc-6258-4e5a-b8f6-4ce0659b3601/manager/0.log" Jul 16 18:27:58.899306 ip-10-0-132-130 kubenswrapper[2578]: I0716 18:27:58.899274 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-controller-manager-bb7ddb7d7-fp8pd_4c5a2957-e580-423a-97a9-5ffb61407ab4/manager/0.log" Jul 16 18:27:59.742700 ip-10-0-132-130 kubenswrapper[2578]: I0716 18:27:59.742667 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-operator-6d85f6564-c4d5z_5798b263-2def-43aa-bc40-bd06b5948f2c/manager/0.log" Jul 16 18:27:59.757800 ip-10-0-132-130 kubenswrapper[2578]: I0716 18:27:59.757770 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_dns-operator-controller-manager-65b49595d7-m7vkn_366102cc-6258-4e5a-b8f6-4ce0659b3601/manager/0.log" Jul 16 18:27:59.919929 ip-10-0-132-130 kubenswrapper[2578]: I0716 18:27:59.919896 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-controller-manager-bb7ddb7d7-fp8pd_4c5a2957-e580-423a-97a9-5ffb61407ab4/manager/0.log" Jul 16 18:28:00.730841 ip-10-0-132-130 kubenswrapper[2578]: I0716 18:28:00.730805 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-operator-6d85f6564-c4d5z_5798b263-2def-43aa-bc40-bd06b5948f2c/manager/0.log" Jul 16 18:28:00.746467 ip-10-0-132-130 kubenswrapper[2578]: I0716 18:28:00.746440 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_dns-operator-controller-manager-65b49595d7-m7vkn_366102cc-6258-4e5a-b8f6-4ce0659b3601/manager/0.log" Jul 16 18:28:00.907958 ip-10-0-132-130 kubenswrapper[2578]: I0716 18:28:00.907920 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-controller-manager-bb7ddb7d7-fp8pd_4c5a2957-e580-423a-97a9-5ffb61407ab4/manager/0.log" Jul 16 18:28:01.791010 ip-10-0-132-130 kubenswrapper[2578]: I0716 18:28:01.790980 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-operator-6d85f6564-c4d5z_5798b263-2def-43aa-bc40-bd06b5948f2c/manager/0.log" Jul 16 18:28:01.808281 ip-10-0-132-130 kubenswrapper[2578]: I0716 18:28:01.808235 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_dns-operator-controller-manager-65b49595d7-m7vkn_366102cc-6258-4e5a-b8f6-4ce0659b3601/manager/0.log" Jul 16 18:28:01.968665 ip-10-0-132-130 kubenswrapper[2578]: I0716 18:28:01.968632 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-controller-manager-bb7ddb7d7-fp8pd_4c5a2957-e580-423a-97a9-5ffb61407ab4/manager/0.log" Jul 16 18:28:02.832629 ip-10-0-132-130 kubenswrapper[2578]: I0716 18:28:02.832601 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-operator-6d85f6564-c4d5z_5798b263-2def-43aa-bc40-bd06b5948f2c/manager/0.log" Jul 16 18:28:02.844880 ip-10-0-132-130 kubenswrapper[2578]: I0716 18:28:02.844838 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_dns-operator-controller-manager-65b49595d7-m7vkn_366102cc-6258-4e5a-b8f6-4ce0659b3601/manager/0.log" Jul 16 18:28:03.007671 ip-10-0-132-130 kubenswrapper[2578]: I0716 18:28:03.007634 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-controller-manager-bb7ddb7d7-fp8pd_4c5a2957-e580-423a-97a9-5ffb61407ab4/manager/0.log" Jul 16 18:28:03.877280 ip-10-0-132-130 kubenswrapper[2578]: I0716 18:28:03.877246 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-operator-6d85f6564-c4d5z_5798b263-2def-43aa-bc40-bd06b5948f2c/manager/0.log" Jul 16 18:28:03.889928 ip-10-0-132-130 kubenswrapper[2578]: I0716 18:28:03.889903 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_dns-operator-controller-manager-65b49595d7-m7vkn_366102cc-6258-4e5a-b8f6-4ce0659b3601/manager/0.log" Jul 16 18:28:04.048573 ip-10-0-132-130 kubenswrapper[2578]: I0716 18:28:04.048540 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-controller-manager-bb7ddb7d7-fp8pd_4c5a2957-e580-423a-97a9-5ffb61407ab4/manager/0.log" Jul 16 18:28:04.927803 ip-10-0-132-130 kubenswrapper[2578]: I0716 18:28:04.927771 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-operator-6d85f6564-c4d5z_5798b263-2def-43aa-bc40-bd06b5948f2c/manager/0.log" Jul 16 18:28:04.940499 ip-10-0-132-130 kubenswrapper[2578]: I0716 18:28:04.940478 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_dns-operator-controller-manager-65b49595d7-m7vkn_366102cc-6258-4e5a-b8f6-4ce0659b3601/manager/0.log" Jul 16 18:28:05.098792 ip-10-0-132-130 kubenswrapper[2578]: I0716 18:28:05.098751 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-controller-manager-bb7ddb7d7-fp8pd_4c5a2957-e580-423a-97a9-5ffb61407ab4/manager/0.log" Jul 16 18:28:05.990279 ip-10-0-132-130 kubenswrapper[2578]: I0716 18:28:05.990244 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-operator-6d85f6564-c4d5z_5798b263-2def-43aa-bc40-bd06b5948f2c/manager/0.log" Jul 16 18:28:06.003175 ip-10-0-132-130 kubenswrapper[2578]: I0716 18:28:06.003154 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_dns-operator-controller-manager-65b49595d7-m7vkn_366102cc-6258-4e5a-b8f6-4ce0659b3601/manager/0.log" Jul 16 18:28:06.165144 ip-10-0-132-130 kubenswrapper[2578]: I0716 18:28:06.165114 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-controller-manager-bb7ddb7d7-fp8pd_4c5a2957-e580-423a-97a9-5ffb61407ab4/manager/0.log" Jul 16 18:28:07.086108 ip-10-0-132-130 kubenswrapper[2578]: I0716 18:28:07.086073 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-operator-6d85f6564-c4d5z_5798b263-2def-43aa-bc40-bd06b5948f2c/manager/0.log" Jul 16 18:28:07.099225 ip-10-0-132-130 kubenswrapper[2578]: I0716 18:28:07.099199 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_dns-operator-controller-manager-65b49595d7-m7vkn_366102cc-6258-4e5a-b8f6-4ce0659b3601/manager/0.log" Jul 16 18:28:07.257921 ip-10-0-132-130 kubenswrapper[2578]: I0716 18:28:07.257883 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-controller-manager-bb7ddb7d7-fp8pd_4c5a2957-e580-423a-97a9-5ffb61407ab4/manager/0.log" Jul 16 18:28:08.136634 ip-10-0-132-130 kubenswrapper[2578]: I0716 18:28:08.136603 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-operator-6d85f6564-c4d5z_5798b263-2def-43aa-bc40-bd06b5948f2c/manager/0.log" Jul 16 18:28:08.150313 ip-10-0-132-130 kubenswrapper[2578]: I0716 18:28:08.150282 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_dns-operator-controller-manager-65b49595d7-m7vkn_366102cc-6258-4e5a-b8f6-4ce0659b3601/manager/0.log" Jul 16 18:28:08.315109 ip-10-0-132-130 kubenswrapper[2578]: I0716 18:28:08.315075 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-controller-manager-bb7ddb7d7-fp8pd_4c5a2957-e580-423a-97a9-5ffb61407ab4/manager/0.log" Jul 16 18:28:14.682576 ip-10-0-132-130 kubenswrapper[2578]: I0716 18:28:14.682543 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-rg5vf_4e00566f-55f0-43ec-9d1a-a883e839823e/global-pull-secret-syncer/0.log" Jul 16 18:28:14.816225 ip-10-0-132-130 kubenswrapper[2578]: I0716 18:28:14.816193 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-kq8j9_a8425d74-5613-43e9-b6ff-3884b419dbf6/konnectivity-agent/0.log" Jul 16 18:28:14.918287 ip-10-0-132-130 kubenswrapper[2578]: I0716 18:28:14.918254 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-132-130.ec2.internal_7876ab904043bc7f87e7c28015b01ebc/haproxy/0.log" Jul 16 18:28:20.297505 ip-10-0-132-130 kubenswrapper[2578]: I0716 18:28:20.297466 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-operator-6d85f6564-c4d5z_5798b263-2def-43aa-bc40-bd06b5948f2c/manager/0.log" Jul 16 18:28:20.335140 ip-10-0-132-130 kubenswrapper[2578]: I0716 18:28:20.335105 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_dns-operator-controller-manager-65b49595d7-m7vkn_366102cc-6258-4e5a-b8f6-4ce0659b3601/manager/0.log" Jul 16 18:28:20.532656 ip-10-0-132-130 kubenswrapper[2578]: I0716 18:28:20.532619 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-controller-manager-bb7ddb7d7-fp8pd_4c5a2957-e580-423a-97a9-5ffb61407ab4/manager/0.log" Jul 16 18:28:21.724972 ip-10-0-132-130 kubenswrapper[2578]: I0716 18:28:21.724940 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_4d4f4ff2-a8be-4e99-8198-7c78dca29787/alertmanager/0.log" Jul 16 18:28:21.764790 ip-10-0-132-130 kubenswrapper[2578]: I0716 18:28:21.764764 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_4d4f4ff2-a8be-4e99-8198-7c78dca29787/config-reloader/0.log" Jul 16 18:28:21.804930 ip-10-0-132-130 kubenswrapper[2578]: I0716 18:28:21.804901 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_4d4f4ff2-a8be-4e99-8198-7c78dca29787/kube-rbac-proxy-web/0.log" Jul 16 18:28:21.841291 ip-10-0-132-130 kubenswrapper[2578]: I0716 18:28:21.841270 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_4d4f4ff2-a8be-4e99-8198-7c78dca29787/kube-rbac-proxy/0.log" Jul 16 18:28:21.878284 ip-10-0-132-130 kubenswrapper[2578]: I0716 18:28:21.878249 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_4d4f4ff2-a8be-4e99-8198-7c78dca29787/kube-rbac-proxy-metric/0.log" Jul 16 18:28:21.907423 ip-10-0-132-130 kubenswrapper[2578]: I0716 18:28:21.907399 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_4d4f4ff2-a8be-4e99-8198-7c78dca29787/prom-label-proxy/0.log" Jul 16 18:28:21.942560 ip-10-0-132-130 kubenswrapper[2578]: I0716 18:28:21.942528 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_4d4f4ff2-a8be-4e99-8198-7c78dca29787/init-config-reloader/0.log" Jul 16 18:28:22.007918 ip-10-0-132-130 kubenswrapper[2578]: I0716 18:28:22.007848 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_cluster-monitoring-operator-5f7f494dd4-p872b_5bf41cfb-4cb4-4953-a458-9211602c6b74/cluster-monitoring-operator/0.log" Jul 16 18:28:22.133887 ip-10-0-132-130 kubenswrapper[2578]: I0716 18:28:22.133864 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_metrics-server-74dc84475b-hznm2_bbd0744b-a415-4029-933b-c318369ac9ee/metrics-server/0.log" Jul 16 18:28:22.205711 ip-10-0-132-130 kubenswrapper[2578]: I0716 18:28:22.205669 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-8gvqf_12344972-7243-4730-9352-19dacb37fbcb/node-exporter/0.log" Jul 16 18:28:22.232885 ip-10-0-132-130 kubenswrapper[2578]: I0716 18:28:22.232843 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-8gvqf_12344972-7243-4730-9352-19dacb37fbcb/kube-rbac-proxy/0.log" Jul 16 18:28:22.262419 ip-10-0-132-130 kubenswrapper[2578]: I0716 18:28:22.262354 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-8gvqf_12344972-7243-4730-9352-19dacb37fbcb/init-textfile/0.log" Jul 16 18:28:22.510773 ip-10-0-132-130 kubenswrapper[2578]: I0716 18:28:22.510742 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-84bd68b896-kbbbg_486ff997-a129-4760-8f65-6fa79ba9eed3/kube-rbac-proxy-main/0.log" Jul 16 18:28:22.541368 ip-10-0-132-130 kubenswrapper[2578]: I0716 18:28:22.541347 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-84bd68b896-kbbbg_486ff997-a129-4760-8f65-6fa79ba9eed3/kube-rbac-proxy-self/0.log" Jul 16 18:28:22.574588 ip-10-0-132-130 kubenswrapper[2578]: I0716 18:28:22.574560 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-84bd68b896-kbbbg_486ff997-a129-4760-8f65-6fa79ba9eed3/openshift-state-metrics/0.log" Jul 16 18:28:22.889879 ip-10-0-132-130 kubenswrapper[2578]: I0716 18:28:22.889850 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-admission-webhook-77d98bc786-87596_bfc93304-acba-424e-b6db-0844667a0803/prometheus-operator-admission-webhook/0.log" Jul 16 18:28:23.040880 ip-10-0-132-130 kubenswrapper[2578]: I0716 18:28:23.040853 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-6769966f79-57q7b_8cb88b9c-ade2-499e-b959-3e158a068497/thanos-query/0.log" Jul 16 18:28:23.071082 ip-10-0-132-130 kubenswrapper[2578]: I0716 18:28:23.071054 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-6769966f79-57q7b_8cb88b9c-ade2-499e-b959-3e158a068497/kube-rbac-proxy-web/0.log" Jul 16 18:28:23.099838 ip-10-0-132-130 kubenswrapper[2578]: I0716 18:28:23.099818 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-6769966f79-57q7b_8cb88b9c-ade2-499e-b959-3e158a068497/kube-rbac-proxy/0.log" Jul 16 18:28:23.129737 ip-10-0-132-130 kubenswrapper[2578]: I0716 18:28:23.129720 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-6769966f79-57q7b_8cb88b9c-ade2-499e-b959-3e158a068497/prom-label-proxy/0.log" Jul 16 18:28:23.159320 ip-10-0-132-130 kubenswrapper[2578]: I0716 18:28:23.159247 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-6769966f79-57q7b_8cb88b9c-ade2-499e-b959-3e158a068497/kube-rbac-proxy-rules/0.log" Jul 16 18:28:23.185644 ip-10-0-132-130 kubenswrapper[2578]: I0716 18:28:23.185620 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-6769966f79-57q7b_8cb88b9c-ade2-499e-b959-3e158a068497/kube-rbac-proxy-metrics/0.log" Jul 16 18:28:24.565704 ip-10-0-132-130 kubenswrapper[2578]: I0716 18:28:24.565656 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-qh77t/perf-node-gather-daemonset-4sr5v"] Jul 16 18:28:24.569639 ip-10-0-132-130 kubenswrapper[2578]: I0716 18:28:24.569619 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-qh77t/perf-node-gather-daemonset-4sr5v" Jul 16 18:28:24.571492 ip-10-0-132-130 kubenswrapper[2578]: I0716 18:28:24.571469 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-qh77t\"/\"default-dockercfg-m6ppb\"" Jul 16 18:28:24.571603 ip-10-0-132-130 kubenswrapper[2578]: I0716 18:28:24.571514 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-qh77t\"/\"kube-root-ca.crt\"" Jul 16 18:28:24.571963 ip-10-0-132-130 kubenswrapper[2578]: I0716 18:28:24.571943 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-qh77t\"/\"openshift-service-ca.crt\"" Jul 16 18:28:24.579350 ip-10-0-132-130 kubenswrapper[2578]: I0716 18:28:24.579330 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-qh77t/perf-node-gather-daemonset-4sr5v"] Jul 16 18:28:24.678935 ip-10-0-132-130 kubenswrapper[2578]: I0716 18:28:24.678902 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/5d9ff131-a01d-433e-a1dd-9675c399b582-proc\") pod \"perf-node-gather-daemonset-4sr5v\" (UID: \"5d9ff131-a01d-433e-a1dd-9675c399b582\") " pod="openshift-must-gather-qh77t/perf-node-gather-daemonset-4sr5v" Jul 16 18:28:24.679128 ip-10-0-132-130 kubenswrapper[2578]: I0716 18:28:24.679018 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/5d9ff131-a01d-433e-a1dd-9675c399b582-sys\") pod \"perf-node-gather-daemonset-4sr5v\" (UID: \"5d9ff131-a01d-433e-a1dd-9675c399b582\") " pod="openshift-must-gather-qh77t/perf-node-gather-daemonset-4sr5v" Jul 16 18:28:24.679128 ip-10-0-132-130 kubenswrapper[2578]: I0716 18:28:24.679040 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/5d9ff131-a01d-433e-a1dd-9675c399b582-podres\") pod \"perf-node-gather-daemonset-4sr5v\" (UID: \"5d9ff131-a01d-433e-a1dd-9675c399b582\") " pod="openshift-must-gather-qh77t/perf-node-gather-daemonset-4sr5v" Jul 16 18:28:24.679226 ip-10-0-132-130 kubenswrapper[2578]: I0716 18:28:24.679173 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4bfl9\" (UniqueName: \"kubernetes.io/projected/5d9ff131-a01d-433e-a1dd-9675c399b582-kube-api-access-4bfl9\") pod \"perf-node-gather-daemonset-4sr5v\" (UID: \"5d9ff131-a01d-433e-a1dd-9675c399b582\") " pod="openshift-must-gather-qh77t/perf-node-gather-daemonset-4sr5v" Jul 16 18:28:24.679277 ip-10-0-132-130 kubenswrapper[2578]: I0716 18:28:24.679260 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/5d9ff131-a01d-433e-a1dd-9675c399b582-lib-modules\") pod \"perf-node-gather-daemonset-4sr5v\" (UID: \"5d9ff131-a01d-433e-a1dd-9675c399b582\") " pod="openshift-must-gather-qh77t/perf-node-gather-daemonset-4sr5v" Jul 16 18:28:24.780770 ip-10-0-132-130 kubenswrapper[2578]: I0716 18:28:24.780736 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/5d9ff131-a01d-433e-a1dd-9675c399b582-sys\") pod \"perf-node-gather-daemonset-4sr5v\" (UID: \"5d9ff131-a01d-433e-a1dd-9675c399b582\") " pod="openshift-must-gather-qh77t/perf-node-gather-daemonset-4sr5v" Jul 16 18:28:24.780770 ip-10-0-132-130 kubenswrapper[2578]: I0716 18:28:24.780769 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/5d9ff131-a01d-433e-a1dd-9675c399b582-podres\") pod \"perf-node-gather-daemonset-4sr5v\" (UID: \"5d9ff131-a01d-433e-a1dd-9675c399b582\") " pod="openshift-must-gather-qh77t/perf-node-gather-daemonset-4sr5v" Jul 16 18:28:24.780995 ip-10-0-132-130 kubenswrapper[2578]: I0716 18:28:24.780807 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4bfl9\" (UniqueName: \"kubernetes.io/projected/5d9ff131-a01d-433e-a1dd-9675c399b582-kube-api-access-4bfl9\") pod \"perf-node-gather-daemonset-4sr5v\" (UID: \"5d9ff131-a01d-433e-a1dd-9675c399b582\") " pod="openshift-must-gather-qh77t/perf-node-gather-daemonset-4sr5v" Jul 16 18:28:24.780995 ip-10-0-132-130 kubenswrapper[2578]: I0716 18:28:24.780856 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/5d9ff131-a01d-433e-a1dd-9675c399b582-lib-modules\") pod \"perf-node-gather-daemonset-4sr5v\" (UID: \"5d9ff131-a01d-433e-a1dd-9675c399b582\") " pod="openshift-must-gather-qh77t/perf-node-gather-daemonset-4sr5v" Jul 16 18:28:24.780995 ip-10-0-132-130 kubenswrapper[2578]: I0716 18:28:24.780873 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/5d9ff131-a01d-433e-a1dd-9675c399b582-proc\") pod \"perf-node-gather-daemonset-4sr5v\" (UID: \"5d9ff131-a01d-433e-a1dd-9675c399b582\") " pod="openshift-must-gather-qh77t/perf-node-gather-daemonset-4sr5v" Jul 16 18:28:24.780995 ip-10-0-132-130 kubenswrapper[2578]: I0716 18:28:24.780876 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/5d9ff131-a01d-433e-a1dd-9675c399b582-sys\") pod \"perf-node-gather-daemonset-4sr5v\" (UID: \"5d9ff131-a01d-433e-a1dd-9675c399b582\") " pod="openshift-must-gather-qh77t/perf-node-gather-daemonset-4sr5v" Jul 16 18:28:24.780995 ip-10-0-132-130 kubenswrapper[2578]: I0716 18:28:24.780919 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/5d9ff131-a01d-433e-a1dd-9675c399b582-podres\") pod \"perf-node-gather-daemonset-4sr5v\" (UID: \"5d9ff131-a01d-433e-a1dd-9675c399b582\") " pod="openshift-must-gather-qh77t/perf-node-gather-daemonset-4sr5v" Jul 16 18:28:24.780995 ip-10-0-132-130 kubenswrapper[2578]: I0716 18:28:24.780941 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/5d9ff131-a01d-433e-a1dd-9675c399b582-proc\") pod \"perf-node-gather-daemonset-4sr5v\" (UID: \"5d9ff131-a01d-433e-a1dd-9675c399b582\") " pod="openshift-must-gather-qh77t/perf-node-gather-daemonset-4sr5v" Jul 16 18:28:24.780995 ip-10-0-132-130 kubenswrapper[2578]: I0716 18:28:24.780992 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/5d9ff131-a01d-433e-a1dd-9675c399b582-lib-modules\") pod \"perf-node-gather-daemonset-4sr5v\" (UID: \"5d9ff131-a01d-433e-a1dd-9675c399b582\") " pod="openshift-must-gather-qh77t/perf-node-gather-daemonset-4sr5v" Jul 16 18:28:24.790520 ip-10-0-132-130 kubenswrapper[2578]: I0716 18:28:24.790495 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4bfl9\" (UniqueName: \"kubernetes.io/projected/5d9ff131-a01d-433e-a1dd-9675c399b582-kube-api-access-4bfl9\") pod \"perf-node-gather-daemonset-4sr5v\" (UID: \"5d9ff131-a01d-433e-a1dd-9675c399b582\") " pod="openshift-must-gather-qh77t/perf-node-gather-daemonset-4sr5v" Jul 16 18:28:24.869118 ip-10-0-132-130 kubenswrapper[2578]: I0716 18:28:24.869031 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-drs2p_e55f447c-ad64-482b-a86b-0588bddf7b4d/console-operator/1.log" Jul 16 18:28:24.877177 ip-10-0-132-130 kubenswrapper[2578]: I0716 18:28:24.877152 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-drs2p_e55f447c-ad64-482b-a86b-0588bddf7b4d/console-operator/2.log" Jul 16 18:28:24.880403 ip-10-0-132-130 kubenswrapper[2578]: I0716 18:28:24.880383 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-qh77t/perf-node-gather-daemonset-4sr5v" Jul 16 18:28:25.028630 ip-10-0-132-130 kubenswrapper[2578]: I0716 18:28:25.028605 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-qh77t/perf-node-gather-daemonset-4sr5v"] Jul 16 18:28:25.030844 ip-10-0-132-130 kubenswrapper[2578]: W0716 18:28:25.030806 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-pod5d9ff131_a01d_433e_a1dd_9675c399b582.slice/crio-bfc074631f83de75a050dff2636bd4ee804c24f9aef23d9a69869f135c97ead3 WatchSource:0}: Error finding container bfc074631f83de75a050dff2636bd4ee804c24f9aef23d9a69869f135c97ead3: Status 404 returned error can't find the container with id bfc074631f83de75a050dff2636bd4ee804c24f9aef23d9a69869f135c97ead3 Jul 16 18:28:25.385283 ip-10-0-132-130 kubenswrapper[2578]: I0716 18:28:25.385254 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_console-5c9855ffcc-s2jwl_286b759e-aeff-445b-b47b-247f4885b20b/console/0.log" Jul 16 18:28:25.436257 ip-10-0-132-130 kubenswrapper[2578]: I0716 18:28:25.436168 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_downloads-68c9cbb497-lrlw4_fd67d296-3c7d-47e1-89c2-d4e3f8edb1cc/download-server/0.log" Jul 16 18:28:25.598340 ip-10-0-132-130 kubenswrapper[2578]: I0716 18:28:25.598304 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-qh77t/perf-node-gather-daemonset-4sr5v" event={"ID":"5d9ff131-a01d-433e-a1dd-9675c399b582","Type":"ContainerStarted","Data":"a55660ce2413f0e0dce4958946e65af7c99267bbe1f19c5d43f737bb3cb93ea8"} Jul 16 18:28:25.598837 ip-10-0-132-130 kubenswrapper[2578]: I0716 18:28:25.598496 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-qh77t/perf-node-gather-daemonset-4sr5v" Jul 16 18:28:25.598837 ip-10-0-132-130 kubenswrapper[2578]: I0716 18:28:25.598535 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-qh77t/perf-node-gather-daemonset-4sr5v" event={"ID":"5d9ff131-a01d-433e-a1dd-9675c399b582","Type":"ContainerStarted","Data":"bfc074631f83de75a050dff2636bd4ee804c24f9aef23d9a69869f135c97ead3"} Jul 16 18:28:25.614953 ip-10-0-132-130 kubenswrapper[2578]: I0716 18:28:25.614910 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-qh77t/perf-node-gather-daemonset-4sr5v" podStartSLOduration=1.6148944200000002 podStartE2EDuration="1.61489442s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-16 18:28:24 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 18:28:25.613929171 +0000 UTC m=+4450.099773057" watchObservedRunningTime="2026-07-16 18:28:25.61489442 +0000 UTC m=+4450.100738313" Jul 16 18:28:26.126978 ip-10-0-132-130 kubenswrapper[2578]: I0716 18:28:26.126941 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-cluster-storage-operator_volume-data-source-validator-78495597f6-nqvsk_609ca387-5fdd-4c2d-829e-db5431ddb87f/volume-data-source-validator/0.log" Jul 16 18:28:27.336137 ip-10-0-132-130 kubenswrapper[2578]: I0716 18:28:27.336100 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-vtkgg_f99dd101-e4fa-4c68-a502-c3a14be5c7fb/dns/0.log" Jul 16 18:28:27.360915 ip-10-0-132-130 kubenswrapper[2578]: I0716 18:28:27.360889 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-vtkgg_f99dd101-e4fa-4c68-a502-c3a14be5c7fb/kube-rbac-proxy/0.log" Jul 16 18:28:27.448869 ip-10-0-132-130 kubenswrapper[2578]: I0716 18:28:27.448841 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-zw2xf_ff79255c-4f85-4600-a7d2-54534f299e29/dns-node-resolver/0.log" Jul 16 18:28:28.025934 ip-10-0-132-130 kubenswrapper[2578]: I0716 18:28:28.025905 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_image-registry-ff66d4bb5-czxjw_4e1467f4-90b0-436c-a61a-394761bca783/registry/0.log" Jul 16 18:28:28.087682 ip-10-0-132-130 kubenswrapper[2578]: I0716 18:28:28.087656 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-lfsvs_dfca47c1-7bfb-4111-a283-7b86affa45a7/node-ca/0.log" Jul 16 18:28:29.633600 ip-10-0-132-130 kubenswrapper[2578]: I0716 18:28:29.633566 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-94bb8fbfd-qnh6j_54c915b4-e326-4c4e-b79e-5d0c16cd2811/discovery/0.log" Jul 16 18:28:30.356991 ip-10-0-132-130 kubenswrapper[2578]: I0716 18:28:30.356961 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-p2d6d_f7ea4735-cb68-4db2-a4ed-b4395fe7e9f6/serve-healthcheck-canary/0.log" Jul 16 18:28:30.919007 ip-10-0-132-130 kubenswrapper[2578]: I0716 18:28:30.918977 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-5czbw_dbd1c30a-e583-4ecd-b5c6-ec71f3579e50/kube-rbac-proxy/0.log" Jul 16 18:28:30.944545 ip-10-0-132-130 kubenswrapper[2578]: I0716 18:28:30.944491 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-5czbw_dbd1c30a-e583-4ecd-b5c6-ec71f3579e50/exporter/0.log" Jul 16 18:28:30.973078 ip-10-0-132-130 kubenswrapper[2578]: I0716 18:28:30.973050 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-5czbw_dbd1c30a-e583-4ecd-b5c6-ec71f3579e50/extractor/0.log" Jul 16 18:28:31.615053 ip-10-0-132-130 kubenswrapper[2578]: I0716 18:28:31.615027 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-qh77t/perf-node-gather-daemonset-4sr5v" Jul 16 18:28:34.627181 ip-10-0-132-130 kubenswrapper[2578]: I0716 18:28:34.627147 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-94bb8fbfd-qnh6j_54c915b4-e326-4c4e-b79e-5d0c16cd2811/discovery/0.log" Jul 16 18:28:35.568094 ip-10-0-132-130 kubenswrapper[2578]: I0716 18:28:35.568066 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve_kserve-controller-manager-5c5755fdcb-g2x5r_1e3dd3b6-f573-40d6-ae3f-6ce5ca836316/manager/0.log" Jul 16 18:28:35.809685 ip-10-0-132-130 kubenswrapper[2578]: I0716 18:28:35.809643 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve_llmisvc-controller-manager-565cf7448c-6d7tb_369760c7-a0fd-4fc0-9f59-befc0e43bc65/manager/0.log" Jul 16 18:28:35.841829 ip-10-0-132-130 kubenswrapper[2578]: I0716 18:28:35.841722 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve_model-serving-api-fd65d7d6-h2w48_123068b3-bb2c-4819-a517-0362388d5fde/server/0.log" Jul 16 18:28:36.161852 ip-10-0-132-130 kubenswrapper[2578]: I0716 18:28:36.161827 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve_odh-model-controller-84b54b7f97-lf6xh_bf4947fd-3845-4bb8-a7a0-f03de833d182/manager/0.log" Jul 16 18:28:36.278626 ip-10-0-132-130 kubenswrapper[2578]: I0716 18:28:36.278586 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-94bb8fbfd-qnh6j_54c915b4-e326-4c4e-b79e-5d0c16cd2811/discovery/0.log" Jul 16 18:28:41.722835 ip-10-0-132-130 kubenswrapper[2578]: I0716 18:28:41.722734 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator-operator_kube-storage-version-migrator-operator-799cb7f958-8q957_91a757db-58f1-4483-baf4-9b936e4b8072/kube-storage-version-migrator-operator/1.log" Jul 16 18:28:41.724315 ip-10-0-132-130 kubenswrapper[2578]: I0716 18:28:41.724230 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator-operator_kube-storage-version-migrator-operator-799cb7f958-8q957_91a757db-58f1-4483-baf4-9b936e4b8072/kube-storage-version-migrator-operator/0.log"